Skip to content

Future work: prefilter recalls Hindsight memory and/or chat history for 'done-before' check #101

@right-bot

Description

@right-bot

Future work: prefilter recalls Hindsight memory and/or chat history for "done-before" check

Context

Today the skill-learning prefilter (crates/bot/src/learning_prefilter.rs::build_prompt) decides skip / patch_existing / create_new from a snapshot of the just-finished turn:

  • user_msg_text (≤2000 chars)
  • assistant_reply_text (≤4000 chars)
  • used_skill_receipts
  • existing rightx-* skill index (name + ≤120-char description)
  • per-turn cost / num_turns / elapsed vs agent's P50/P90/P99 baseline

That's enough to spot trivial chats and existing-skill reuse, but it has a blind spot: the agent may have solved the same problem days ago without producing a skill receipt, and the prefilter has no signal for that. Result: a create_new decision for a procedure that's already in Hindsight memory or recent chat history, and a redundant probe-writer run downstream.

Proposal

Add a lightweight "have we done this before?" lookup to the prefilter input:

  1. Hindsight recall. Issue a short memory_recall query keyed on the topic of the turn (extracted from the user msg or assistant reply) and pass the top N (≤3) hits into the prompt as PRIOR PROCEDURE EVIDENCE.
  2. Chat history search. Optionally run thread_search / chat_search against the current chat for similar prior turns. Pass back compact snippets (caller, ts, ≤200 chars) so Haiku can see "we already walked the user through this on date X".
  3. Decision impact. Reword the framing so that prior evidence biases toward patch_existing (if a matching skill exists) or skip (if the procedure was handled inline and is too narrow to deserve a skill yet). Keep create_new for genuinely novel work.

Considerations

  • Latency / cost. Recall + search adds round trips and tokens. Cap to single-shot queries with hard byte budgets so the prefilter stays inside its 30s timeout and the Haiku prompt stays small.
  • Daily learning budget. Whatever recall/search costs should be counted against LEARNING_SOURCES so today_spend_usd still gates correctly.
  • False suppression. A naive recall match can suppress legitimately new work that just happens to share keywords. Make the prompt explicit: prior evidence is a hint, not a verdict — Haiku still decides.
  • Schema. No new decision values needed. Optionally extend reason so we can audit which prior-evidence link triggered a patch_existing or skip.

Out of scope

  • Changing the curator / probe-writer side. This is strictly a prefilter input enrichment.
  • Cross-agent recall. Stay scoped to the current agent's own memory and chat for now.

Files

  • crates/bot/src/learning_prefilter.rsbuild_prompt, schema extension if needed
  • crates/bot/src/learning_pipeline.rs — wiring of the new input
  • crates/right-agent/src/usage.rs — accounting for the recall/search cost

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions