Skip to content

Treat RouterOS version strings as first-class query tokens across routeros_search #116

Description

@mobileskyfi

Follow-up carved out of #89 (see scope-split comment). #89's video-title slice shipped in #115 — matching the video title via videos_fts surfaces title-only words like "changelog". This issue tracks the other, higher-blast-radius half: making a RouterOS version string a first-class query token everywhere, not just in video search.

Problem

extractTerms() (src/query.ts) does .replace(/[^\w\s-]/g, " ") before splitting, so a dotted version is shredded: 7.237 (dropped by MIN_TERM_LENGTH) + 23 (a near-meaningless 2-digit token that matches unrelated numbers). Separately, "routeros" is a hardcoded stop word (STOP_WORDS) — sensible for page search where nearly every doc says RouterOS, but it guts version/changelog queries.

Both live on the shared query path (page search, property search, changelog, video), so a change here moves the golden/eval baseline — which is exactly why it was kept out of the #115 low-risk slice and wants a deliberate design pass.

Scope / directions (not yet settled — needs a design decision before agent-ready)

  • Preserve dotted version numbers (\d+\.\d+, and \d+\.\d+(beta|rc)\d*) as a single token before the punctuation split in extractTerms(), instead of destructively splitting on ..
  • Decide whether "routeros" should stay a stop word globally, or be context-dependent (e.g. kept for page search, relaxed for changelog/video), and whether a detected version token should suppress the stop-word entirely.
  • Consider whether this belongs in the classifier (src/classify.ts already detects versions) feeding a structured term rather than in raw extractTerms().

Acceptance (draft — refine before labeling agent-ready)

  • A dotted version in a natural-language query survives as one token across routeros_search (page + changelog + video surfaces), verified by a golden query.
  • No regression to the Phase 0 retrieval eval baseline; if the change shifts it, the baseline is re-grounded deliberately (not loosened to green).
  • Decision on "routeros" stop-word handling recorded (DESIGN.md or the issue), with the chosen behavior pinned by a test.

Related

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions