Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
143 commits
Select commit Hold shift + click to select a range
4ae9ab8
skills: add webwright.skills memory/skill-library module + skill_use …
DEM1TASSE Jun 30, 2026
91a2090
skills: prompt hint helper + skill_mode overlay; env-based model for …
DEM1TASSE Jun 30, 2026
a96336b
skills: README for the module + env-based model for bare CLI
DEM1TASSE Jun 30, 2026
82fe0ba
docs+tests: README skill-library section + skills unit tests
DEM1TASSE Jun 30, 2026
05aab3a
skills: translate all comments/docstrings to English
DEM1TASSE Jun 30, 2026
bb0d0cd
skills: drop dead code in update (grow path superseded by evolve)
DEM1TASSE Jun 30, 2026
ba4fa2a
skills: fix refine truncation + timeout + fence stripping (found in e…
DEM1TASSE Jul 2, 2026
2784fc6
skills README: add end-to-end How-to-use (manifest schema, params, so…
DEM1TASSE Jul 3, 2026
c293a48
skills README: batch pipeline example (tasks.json -> solve loop -> ga…
DEM1TASSE Jul 3, 2026
7deab76
skills README: manifest field-by-field table; Results summary with We…
DEM1TASSE Jul 3, 2026
f7b0779
skills: harden pipeline entry points (review findings)
DEM1TASSE Jul 3, 2026
d0c6d84
skills: manifest 'admit' must be a JSON boolean (hand-written "false"…
DEM1TASSE Jul 3, 2026
6060bd5
skills: fix cleanroom-repro findings (silent empty-library skip, rela…
DEM1TASSE Jul 8, 2026
4682a47
skills: add examples/ — a real evolve-produced skill, measured saving…
DEM1TASSE Jul 8, 2026
1234788
skills README: narrative intro (programs not notes, gate, aggregation…
DEM1TASSE Jul 8, 2026
3612e89
skills: friendly entry — 'learn' turns a folder of runs into skills (…
DEM1TASSE Jul 8, 2026
b656e6c
skills: fix learn-interface test findings (silent-reuse-off, tracebac…
DEM1TASSE Jul 8, 2026
5cac233
skills README: env exports at the top of Quickstart (step-1-only read…
DEM1TASSE Jul 8, 2026
64a4feb
skills: release-gap fixes — gate honesty, aggregation showcase, regre…
DEM1TASSE Jul 8, 2026
ddca588
ci: run skills unit tests + wrapper usage check on module changes
DEM1TASSE Jul 8, 2026
06d72b3
evals: make the WebArena numbers reproducible (evals/webarena/)
DEM1TASSE Jul 9, 2026
6513d87
skills: address Copilot review findings
DEM1TASSE Jul 9, 2026
8ea59be
skills: address review — complete Quickstart, drop skill_mode.yaml, c…
DEM1TASSE Jul 10, 2026
7acbdd4
skills: Quickstart on Google Flights — Webwright's own example task
DEM1TASSE Jul 11, 2026
223597a
skills: record the three-way consistency check on the held-out route
DEM1TASSE Jul 11, 2026
9286493
skills: measured cost table in Quickstart; single flights example
DEM1TASSE Jul 11, 2026
bb67e3e
skills: trim Quickstart cost table to steps + wall time
DEM1TASSE Jul 11, 2026
b9b95e2
skills: step savings scale with unfamiliarity/variance, not zero
DEM1TASSE Jul 11, 2026
b32cc76
skills: self_verify also honors the agent's own final report (free si…
DEM1TASSE Jul 11, 2026
3dfdfd0
skills: end-to-end regression for the status-aware self_verify gate
DEM1TASSE Jul 11, 2026
4a6036b
skills: one-command quickstart.sh — every parameter pre-filled
DEM1TASSE Jul 11, 2026
c8b27c5
skills: quickstart.sh usage path robust to caller cwd
DEM1TASSE Jul 11, 2026
dc8c870
evals: collapse 100 per-task result files into one results.json
DEM1TASSE Jul 11, 2026
f729099
evals: drop the WebArena repro package from the PR (kept locally)
DEM1TASSE Jul 11, 2026
7afb6c0
skills: skills never write artifacts next to __file__; gateway solve-…
DEM1TASSE Jul 11, 2026
c34eb9b
skills: docs run standalone skills from scratch dirs, not repo dirs
DEM1TASSE Jul 11, 2026
f8d047d
skills: pipeline diagram shows evolve's code input and template bucke…
DEM1TASSE Jul 11, 2026
9786424
skills: replay-verify distilled skills before they enter the library
DEM1TASSE Jul 11, 2026
0c29b9f
skills: replay comparator normalizes scalars; manifest carries creden…
DEM1TASSE Jul 11, 2026
693d816
skills: rejected skills leave their runs un-learned (retry possible)
DEM1TASSE Jul 11, 2026
0b74074
skills: standalone-reproduces-its-answers is now the default bar
DEM1TASSE Jul 11, 2026
bcd8210
skills: two skill grades + regression-checked incremental refine
DEM1TASSE Jul 11, 2026
c3331c4
rename: webwright.skills -> webwright.skill_lab (Webwright Skill Lab)
DEM1TASSE Jul 11, 2026
9859176
rename: final name Web Skill Factory (module webwright.skill_factory)
DEM1TASSE Jul 11, 2026
aaf9f99
examples: single checked-in library, diagram moved to assets/
DEM1TASSE Jul 11, 2026
aefbd53
README: tight overview, details pushed down, demo shipped in assets
DEM1TASSE Jul 11, 2026
fb9aa04
Revise README for skill factory module
DEM1TASSE Jul 11, 2026
325994a
README: landing page in the parent repo's register; reference into docs/
DEM1TASSE Jul 12, 2026
cbd7db4
docs: three references; full eval table and design move into the README
DEM1TASSE Jul 12, 2026
d12cee0
README: cost table to quickstart doc, richer Design; diagram gains re…
DEM1TASSE Jul 12, 2026
36c5780
quickstart: close the gateway traps from the external repro
DEM1TASSE Jul 14, 2026
90c90c2
skill_factory: surface replay-crash cause and keep rejected candidates
DEM1TASSE Jul 16, 2026
6fc404f
examples: switch the checked-in skill to earliest-nonstop flight
DEM1TASSE Jul 16, 2026
26a8470
docs(README): restructure landing page around code-native skills
DEM1TASSE Jul 16, 2026
c445fc3
skill_factory: add init and build — a spec-file path from need to skill
DEM1TASSE Jul 16, 2026
e45e66b
examples: let the demo take your own route, and show what the skill did
DEM1TASSE Jul 16, 2026
da372b3
examples: fix a misleading demo hint, and reject bad dates at the door
DEM1TASSE Jul 16, 2026
3f691a9
skill_factory: build --jobs N to solve instances in parallel
DEM1TASSE Jul 16, 2026
4dde15a
skill_factory: init should generalize the need, and say the right thi…
DEM1TASSE Jul 16, 2026
5719989
docs(README): compare against the actual projects, from their source
DEM1TASSE Jul 16, 2026
e986cb6
docs(README): tighten the comparison to the rows we actually own
DEM1TASSE Jul 16, 2026
d6fd77b
docs(README): keep published SKILL.md as a column
DEM1TASSE Jul 16, 2026
23cd7d5
build: report parallel solves as they finish, not in submission order
DEM1TASSE Jul 16, 2026
04521fc
build: judge a solve by its answer file, not its exit code
DEM1TASSE Jul 16, 2026
37bceb6
init: pick the verify mode from whether the answer drifts
DEM1TASSE Jul 16, 2026
0850179
build: show what each solve is doing while it runs
DEM1TASSE Jul 16, 2026
4563680
skill_factory: say what learn is doing during distill and verify
DEM1TASSE Jul 16, 2026
c53db68
Revise README for clarity and detail on Skill Factory
DEM1TASSE Jul 16, 2026
5eccd9c
tests: cover build and init
DEM1TASSE Jul 16, 2026
82f927b
docs(README): show the three ways in, including the one that is the u…
DEM1TASSE Jul 16, 2026
3fdec9d
docs(quickstart): three ways in, and the one that is the user's
DEM1TASSE Jul 16, 2026
7515056
assets: bring the pipeline diagram back in line with the code
DEM1TASSE Jul 16, 2026
b31b2ff
docs(README): fix a false claim about OpenCLI's adapt path
DEM1TASSE Jul 16, 2026
08be44f
assets: make replay-verify legible — verify mode and grade are two th…
DEM1TASSE Jul 16, 2026
f559667
docs(README): stop claiming what the audit says isn't true, and say t…
DEM1TASSE Jul 16, 2026
47a445f
assets: slim the replay-verify box back down
DEM1TASSE Jul 16, 2026
c030ab9
assets: put replay-verify on the demo's pipeline
DEM1TASSE Jul 16, 2026
7124f6d
assets: demo — T3 lands after repair, and the two gates read apart
DEM1TASSE Jul 16, 2026
7f2506d
Revise limitations and roadmap in README
DEM1TASSE Jul 16, 2026
3170561
assets: cut replay-verify to two lines — the update lane was overhanging
DEM1TASSE Jul 16, 2026
5bd5746
docs(README): say what the task is, split the two ways in, drop the d…
DEM1TASSE Jul 16, 2026
7685061
docs: move the module's docs to docs/skill_factory/, drop an orphan
DEM1TASSE Jul 16, 2026
ff76c62
docs: say what --jobs N is, since 3 read like a required value
DEM1TASSE Jul 16, 2026
f481957
Revise quick start and task execution sections
DEM1TASSE Jul 16, 2026
0f0a0f8
docs(quickstart): fix four things the reread turned up
DEM1TASSE Jul 16, 2026
28f2ac7
docs(README): keep the demo's timing honest, and say which task ask/s…
DEM1TASSE Jul 16, 2026
117dfdb
Revise README with new demo link and task instructions
DEM1TASSE Jul 16, 2026
14fb75c
skill_factory: retry a rejected distillation, and stop blaming the us…
DEM1TASSE Jul 16, 2026
e323139
examples: retire `full` in favour of a spec you can read
DEM1TASSE Jul 16, 2026
c601e06
examples: ship three solved runs so `learn` can be tried without solv…
DEM1TASSE Jul 16, 2026
cc10875
examples: make the trajectory demo not expire — --verify off
DEM1TASSE Jul 16, 2026
b08a1ec
docs(README): three steps, each command exactly once
DEM1TASSE Jul 16, 2026
abfce14
skill_factory: give the untested skill a grade of its own — unverified
DEM1TASSE Jul 16, 2026
bf86065
docs(README): put the cost comparison where the reader just felt it
DEM1TASSE Jul 16, 2026
8fb198a
examples: one held-out route, not three
DEM1TASSE Jul 16, 2026
435b5b9
docs(quickstart): stop the tutorial arguing with itself
DEM1TASSE Jul 16, 2026
08b6732
docs: manual mode isn't legacy — but it needed a sign on the door; re…
DEM1TASSE Jul 16, 2026
dc4adf2
docs(manual): say when you MUST use it, not just that you probably sh…
DEM1TASSE Jul 16, 2026
a014a65
docs(manual): the gitlab example is the point, not a leftover
DEM1TASSE Jul 16, 2026
b46a974
docs: measure the agent-with-library path, and say the WebArena examp…
DEM1TASSE Jul 16, 2026
06a5d12
docs(reference): --jobs belongs in the table, not in 22 lines of its own
DEM1TASSE Jul 16, 2026
42b6e02
docs+init: audit every flag against the code — two real gaps
DEM1TASSE Jul 16, 2026
1213f89
docs(reference): the env table never said the thing that makes it con…
DEM1TASSE Jul 16, 2026
87f6be9
docs(reference): say that the env vars and the yaml set the same two …
DEM1TASSE Jul 16, 2026
102a3a3
docs(reference): restructure — two gates split, params before internals
DEM1TASSE Jul 16, 2026
09dbdcf
docs(reference): restore the skill_use table, trim the rest; fix --ou…
DEM1TASSE Jul 16, 2026
90beac0
Revise manual mode documentation in skill factory
DEM1TASSE Jul 17, 2026
0e48f16
Clarify input/output gates and environment variables
DEM1TASSE Jul 17, 2026
cca39f6
Update documentation for skill factory commands
DEM1TASSE Jul 17, 2026
1ab6ede
docs(reference): fix two wrong claims, restore the env section lost t…
DEM1TASSE Jul 17, 2026
91048fe
docs(reference): drift can be a reason for --verify off after all — I…
DEM1TASSE Jul 17, 2026
ee7aa6d
docs(reference): stop presenting an inherited gpt-4o as our default; …
DEM1TASSE Jul 17, 2026
4bcdea0
docs(reference): reference isn't what a failed replay gives you by de…
DEM1TASSE Jul 17, 2026
26fabe3
llm: say out loud when nobody named a model, instead of quietly using…
DEM1TASSE Jul 17, 2026
2f5cb29
docs(reference): "neither of those" — point the sentence back at the …
DEM1TASSE Jul 17, 2026
4805451
docs(quickstart): give "stable vs drifting" one home instead of four
DEM1TASSE Jul 17, 2026
fdac1f2
Revise quickstart.md for skill execution clarity
DEM1TASSE Jul 17, 2026
6d9f2ca
docs(quickstart): land the notes in section 3-4; the spec's "verbatim…
DEM1TASSE Jul 17, 2026
df0df6c
docs(quickstart): three sections, the same three as the README
DEM1TASSE Jul 17, 2026
57c67ea
docs: fold the quickstart into the README and delete it
DEM1TASSE Jul 17, 2026
4e0fa7b
Revise README for improved clarity on usage
DEM1TASSE Jul 17, 2026
c9b4e8d
docs+test: the README's spec was hiding two keys; pin it to init's ou…
DEM1TASSE Jul 17, 2026
1e4566f
Revise README for clarity on init and build commands
DEM1TASSE Jul 17, 2026
7fe7632
docs(README): the ledger holds runs, not skills — and reference gets …
DEM1TASSE Jul 17, 2026
964eaba
Clarify rejection costs and documentation details
DEM1TASSE Jul 17, 2026
20cc682
docs(README): say how to get a working environment before saying "run…
DEM1TASSE Jul 17, 2026
bf7a1d4
docs(README): a reference skill is ledgered without passing verification
DEM1TASSE Jul 17, 2026
480a7dc
docs(README): all the model setup in one place, before the first command
DEM1TASSE Jul 17, 2026
25a8dc3
docs(README): make the gateway setup something you can follow, not infer
DEM1TASSE Jul 17, 2026
9184647
docs(README): the shipped-spec command didn't run from anywhere
DEM1TASSE Jul 17, 2026
3e5c233
docs(README): the shipped spec is next to the reader, not at the repo…
DEM1TASSE Jul 17, 2026
b00b529
build: warn about a half-configured gateway while it can still stop you
DEM1TASSE Jul 17, 2026
1d63204
docs(README): put the gateway -c on the build command itself
DEM1TASSE Jul 17, 2026
be8acf3
build: point the agent at the gateway you already named, instead of w…
DEM1TASSE Jul 17, 2026
a505b05
replay: fold how an answer is written, not what it says
DEM1TASSE Jul 17, 2026
72a85be
evolve: a solve that recognises its answer is not material for a skill
DEM1TASSE Jul 17, 2026
b2b0809
docs(reference): the gates judge answers; one check judges the method
DEM1TASSE Jul 17, 2026
1039ebb
docs(reference): cut the lookup check down to what it is and why
DEM1TASSE Jul 17, 2026
32644f1
Revise README for Web Skill Factory
DEM1TASSE Jul 17, 2026
7063ce8
quickstart.sh: honour the gateway you exported, like build does
DEM1TASSE Jul 17, 2026
eeab275
Revise README setup instructions for clarity
DEM1TASSE Jul 17, 2026
19402d7
docs(README): the checked-in skill is Linux-shaped; say so, and give …
DEM1TASSE Jul 17, 2026
b174e1a
build/quickstart: the env path dropped the agent's output budget from…
DEM1TASSE Jul 17, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 28 additions & 0 deletions .github/workflows/skills-tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
name: skills-tests
on:
push:
paths: ["src/webwright/skill_factory/**", "src/webwright/tools/skill_use.py", "tests/skill_factory/**"]
pull_request:
paths: ["src/webwright/skill_factory/**", "src/webwright/tools/skill_use.py", "tests/skill_factory/**"]
jobs:
unit:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with: { python-version: "3.12" }
- run: pip install httpx pyyaml jinja2 pydantic
- name: skills unit tests (LLM-free)
run: |
set -e
for t in tests/skill_factory/test_*.py; do
echo "== $t"; PYTHONPATH=src python "$t"
done
- name: wrapper usage check (F6)
run: |
set +e
bash src/webwright/skill_factory/examples/solve_with_library.sh > /dev/null 2>&1
[ $? -eq 1 ] && echo "usage-exit OK" || { echo "wrapper must exit 1 on missing args"; exit 1; }
bash -n src/webwright/skill_factory/examples/quickstart.sh || exit 1
bash src/webwright/skill_factory/examples/quickstart.sh badmode > /dev/null 2>&1
[ $? -eq 1 ] && echo "quickstart usage-exit OK" || { echo "quickstart must exit 1 on bad mode"; exit 1; }
19 changes: 19 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -165,6 +165,25 @@ python assets/task_showcase/app.py \

---

## 🧠 Skill Library (reuse solved tasks across tasks)

[`webwright.skills`](src/webwright/skills/) turns solved tasks into **reusable, executable code
skills**, retrieves and judges them at solve time, gates what enters the library, and grows the
library incrementally — a self-evolving *store → retrieve → use/adapt → gate → evolve* loop on top
of Webwright's code-as-action solves. Plugs in with **no change to the agent loop**:

- **Reuse** — the agent calls `python -m webwright.tools.skill_use --task "..." --library ...`
(like `self_reflection`/`image_qa`); it returns `{verdict: use|adapt|skip, source_path}`.
- **Grow** — `python -m webwright.skills.update --manifest batch.json --library ./library`
distills a batch of gate-passed solves into a parameterized, primitive-decomposed skill.

Validated end-to-end on a real public website (read-only GitHub): solve two repos from scratch →
`update` builds a parameterized skill → a held-out repo is solved by reusing it (agent calls
`skill_use`, verdict `use`, answer correct); a wrong solve is kept out by the gate; a second batch
improves the existing skill in place. See [`src/webwright/skills/README.md`](src/webwright/skills/README.md).

---

## 🚀 Quick Start

### Prerequisites
Expand Down
Binary file added assets/skill_factory_demo.mp4
Binary file not shown.
Binary file added assets/skill_factory_pipeline.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
173 changes: 173 additions & 0 deletions assets/skill_factory_pipeline.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading