fix(skills): align search guidance with hybrid retrieval and teach answer-card phrasing - #29
Open
vraspar wants to merge 1 commit into
Open
fix(skills): align search guidance with hybrid retrieval and teach answer-card phrasing#29vraspar wants to merge 1 commit into
vraspar wants to merge 1 commit into
Conversation
… phrasing Retrieval is hybrid (lexical + dense, RRF-fused), so the tenjin-search skill's "lexical, not semantic" line was wrong guidance: it pushed agents to compress a question into keywords, which is exactly what discards the meaning signal. It now says lookup matches wording and meaning and to send the whole question as one natural-language sentence. Privacy hygiene and the MISS-is-fine rule are unchanged. Closes the demand-to-supply loop: the candidate-parking example passes --question, which publish prefills into the answer card's questionsAnswered, so a searcher's phrasing becomes a match target on the published card. tenjin-publish gets the card-phrasing rules the server docs carry: 5 to 10 questionsAnswered entries at 200 characters, varied register, tasks kept out of questionsAnswered, and a dense factual scope because scope is embedded too. The README lookup paragraph matches. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Retrieval went hybrid in prod (lexical + dense, RRF-fused, calibration
hybrid-v1), but the vendored skills still taught the old model.tenjin-searchtold agents "search is lexical, not semantic: it matches words, not meaning", which pushes exactly the wrong behavior: compressing a question into keywords throws away the signal the dense leg runs on. This is the CLI-side half of guidance parity with the server docs in BackTrackCo/tenjin#464.What changed:
skills/tenjin-search/SKILL.md: lookup matches wording and meaning; send the complete question as one natural-language sentence, do not compress to keywords. Privacy hygiene (strip private identifiers, internal service names, secrets, do not search if it cannot be generalized) and the MISS-is-fine rule are untouched.--question "<the question you looked up>". Verified insrc/commands/candidate.ts(stores it in the candidate meta, capped at the server's 200-char bound) andsrc/commands/publish.ts(prefills it into the card'squestionsAnsweredas a fallback, still losing to an explicit--questionor frontmatter). So the exact phrasing a searcher used becomes a match target on the published card.skills/tenjin-publish/SKILL.md: the answer-card draft rule was one vague sentence. It now carries the same phrasing guidance as the server docs: 5 to 10questionsAnsweredentries at 200 characters max, varied in register (natural symptom sentence, terse keyword line or verbatim error string, why/how question), same-register rephrasings add nothing,tasksSupportedis for tasks and not questions, andscopeis embedded too so write it dense and factual.README.md: the lookup paragraph said "derive the smallest public phrasing of your task", which is the same compress-to-keywords advice. Now: strip the private parts, then send what is left as one complete sentence.On the vendored zero-install skill (
skills/tenjin/SKILL.md): it is byte-identical to livetenjin.blog/skills.md, sopnpm sync:skillproduces zero diff, and it already carries the Resource card section, thelookupandsubmit_feedbackMCP tools, andpreviewin the discovery search description. Commit b3e91d1 (#25) resynced it on 2026-07-24. One stale claim rides along: line 103 still describes lookup matching as "honest lexical, not a semantic score", so this package ships one skill saying lookup matches wording and meaning and another saying it does not. That line cannot be corrected in this repo without breaking skill-drift CI, since the file is a mirror. It gets fixed upstream under BackTrackCo/tenjin#464, and this repo picks it up on the nextpnpm sync:skillafter that ships.src/fixtures/openapi.fixture.jsonhas the same shape of residual: it is a vendored server contract snapshot consumed bysrc/contract.test.tsand still pinscalibrationto a constlexical-v1, and it refreshes on the next contract sync, not here.On why the earlier drift sat unnoticed:
skill-drift.ymlis configured correctly (daily schedule, workflow_dispatch, and a PR trigger onskills/**+scripts/sync-skill.mjs) and it did catch the drift. Scheduled runs on main failed on 2026-07-20 and 2026-07-21. The gap is alerting, not detection: a scheduled failure on main gates no PR, opens no issue, and only emails the workflow file's last committer, so main stayed red for days. Adding a failure-path step that opens a tracking issue needsissues: writeon the job, so it is left for a follow-up rather than escalating workflow permissions in a docs PR.No
lookuptosearchrenaming here, to stay out of BackTrackCo/tenjin#463's way.Gates: typecheck, lint, prettier check, and the full vitest suite (709 passed, 9 skipped) all green.
🤖 Generated with Claude Code