Add docfs benchmark pilot runner#5

Draft

rgodha24 wants to merge 1 commit into

mainfrom

docfs-benchmark

rgodha24 commented Jul 10, 2026 •

edited

Loading

Summary

add a local-filesystem docfs runner for the existing task dataset
capture shell command and cache-hydration metrics instead of HTTP fetch/404 metrics
document the 20-site pilot, token audit, and study limitations

Validation

python3 -m py_compile url_discovery_bench/runners.py url_discovery_bench/docfs_run.py
python3 -m url_discovery_bench.docfs_run --help
live cold-mount Claude/Codex pilot across 20 sites

The documented token means include two runaway navigation attempts; medians and outlier-free means are included in DOCFS_PILOT.md.


          Add docfs benchmark pilot runner

c30083e

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet