SkillOpt Project

Train agent skills like neural networks — with epochs, batch size, learning rates, and validation gates — without touching model weights.

Based on Microsoft SkillOpt (paper).

What is SkillOpt?

SkillOpt is a text-space optimizer that improves a frozen language agent by iteratively editing a natural-language skill document — never the model weights. The skill document is a Markdown file that conditions a target model as it executes tasks.

Deep-Learning ↔ SkillOpt Analogy

Deep Learning	SkillOpt
Model weights	Skill document (Markdown)
Forward pass	Rollout — target runs tasks
Loss / score	Task evaluator
Backprop / gradients	Reflect → edit patches
Gradient clipping	Rank & select top-k edits
Learning rate	optimizer.learning_rate (edits/step)
Optimizer step	Apply patches to the document
Validation set	Selection split (valid_seen)
Early stopping	Validation gate

Quick Start

# Install
pip install -e .

# Configure credentials
cp .env.example .env
# edit .env

# Train a skill
python scripts/train.py \
    --config configs/searchqa/default.yaml \
    --split_dir data/searchqa_split \
    --optimizer_model gpt-5.5 \
    --target_model gpt-5.5

# Evaluate a skill
python scripts/eval_only.py \
    --config configs/searchqa/default.yaml \
    --skill ckpt/searchqa/gpt5.5_skill.md \
    --split valid_unseen

Project Structure

skillopt/
├── config.py              # YAML config loading & inheritance
├── engine/trainer.py      # Training loop (ReflACTTrainer)
├── gradient/
│   ├── reflect.py         # Error/success analysts
│   └── aggregate.py       # Hierarchical patch merging
├── optimizer/
│   ├── skill.py           # Edit application to skill document
│   ├── scheduler.py       # Learning rate scheduling
│   ├── clip.py            # Edit selection & ranking
│   ├── slow_update.py     # Epoch-boundary momentum
│   └── meta_skill.py      # Cross-epoch optimizer memory
├── evaluation/gate.py     # Accept/reject validation gate
├── model/                 # Backend clients (OpenAI, Claude, Qwen)
└── envs/<benchmark>/      # Per-benchmark adapters
    ├── adapter.py
    ├── dataloader.py
    ├── rollout.py
    └── evaluator.py

Configuration

Configs use structured YAML with _base_ inheritance:

# configs/searchqa/default.yaml
_base_: ../_base_/default.yaml
train:
  train_size: 400
  batch_size: 40
optimizer:
  learning_rate: 4
env:
  name: searchqa
  split_dir: data/searchqa_split

Override any key at CLI:

python scripts/train.py --config configs/searchqa/default.yaml \
    --cfg-options optimizer.learning_rate=16 optimizer.lr_scheduler=linear

Benchmarks

Benchmark	Type	Config
SearchQA	Question answering	configs/searchqa/default.yaml
DocVQA	Document QA	configs/docvqa/default.yaml
Custom	Your own task	See §7.2 in docs

License

MIT