feat(hooks): intake-sweep feeds deferred-link sources into the staging funnel#279
Merged
Conversation
…g funnel backlog-stage harvests session transcripts into _meta/backlog-staging.md, but persisted "review later" stores (a digest keeper ledger, saved browser collections) had no path into the board funnel at all. intake-sweep.py closes that: generic jsonl + command adapters, declared per-consumer in _meta/intake-sources.json (the kit carries no personal paths; no config file means silent no-op), rendering through the ONE staging-block grammar (lib/learn/staging-format.py) so `board promote` keeps the human gate. Dedup is three-layered and durable: normalized title vs board + staging (all block states), URL membership in either file (catches renamed rows), and a swept-keys state file so a once-staged key is never re-proposed after promote/ reject. Self-throttles to daily; adapter failures are per-source fail-safe; always exits 0. Wiring: backlog-stage.sh runs the sweep on its --surface pass, so the existing SessionStart hook line is the single install point and the surfaced candidate count already includes swept items. Tests: tests/test-intake-sweep.sh (19 assertions, hermetic) covering both adapter kinds, all three dedup layers with negative controls, the config gate, the throttle, and the surface wiring end to end.
The meta-integrity suite pins README layout/summary/table counts and the architecture fallback-layer table to the live hooks/*.sh census; the new hook made them 24.
This was referenced Jul 20, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
hooks/intake-sweep.py+ shim: sweep a consumer's persisted "review later" sources (digest keeper ledger, saved browser collections, any future store) into the_meta/backlog-staging.mdfunnel, through the one staging-block grammar, keepingboard promoteas the human gate. Wired intobacklog-stage.sh --surfaceso the existing SessionStart hook line is the single install point.Why
Session transcripts already flow to the board via backlog-stage, but deferred-link stores had no promote path (ops-toolkit ID-379). The kit stays personal-data-free: sources are declared per-consumer in
_meta/intake-sources.json; no config = silent no-op for every other consumer.Design points
jsonl(field filter + field map) andcommand(JSON array on stdout). Per-source fail-safe; 15s command timeout; always exits 0.INTAKE_SWEEP_MIN_INTERVAL),--forcefor manual/test runs.Test
bash tests/test-intake-sweep.sh, 19 assertions, hermetic (temp repos, no model call): both adapter kinds, all three dedup layers with negative controls (skip-verdict, board dup, staging-url dup, promoted-away key), config gate, throttle, and the--surfacewiring end to end. Regression: test-learn-drain, test-learn-propose, test-board all PASS.