Auto Research Agent

Overview

This agent is designed to automate the iterative process of optimizing machine learning model performance. It operates based on a structured research template, allowing users to define specific goals (e.g., maximizing accuracy) and systematically test targeted modifications across various components like network layers, training rates, and regularization techniques.

It enforces strict boundaries by defining what can and cannot be changed in the codebase, ensuring that experiments remain controlled, reproducible, and focused on measurable improvements.

Capabilities

• Goal Definition: Establishes a clear objective function (metric) and baseline performance for optimization.
• Targeted Modification: Identifies specific, editable areas within model definition (model.py) and training scripts (train.py).
• Structured Experimentation: Implements a prioritized testing sequence, moving from high-impact single changes to complex combinations.
• Constraint Adherence: Strictly avoids modifying fixed datasets, evaluation logic, or the random seed for guaranteed reproducibility.

Example Use Cases

1. Model Architecture Tuning: If your model's performance plateaus, use this agent to systematically test increasing hidden layer sizes or adjusting activation functions (e.g., switching from ReLU to GELU).
2. Hyperparameter Sweeping: When optimizing training stability, define experiments to test different learning rate decay schedules (like cosine warmup) or adjust batch sizes.
3. Systematic Improvement: Use it as a guided research partner when you suspect the optimal configuration lies in combining two seemingly unrelated changes (e.g., changing dropout and adjusting weight decay).

By following its defined priority list, the agent ensures that improvements are tested methodically, maximizing the chance of finding the best performing model with minimal manual guesswork.