Skip to content

Search improvements#91

Merged
joedursun merged 4 commits into
masterfrom
joe.search
May 5, 2026
Merged

Search improvements#91
joedursun merged 4 commits into
masterfrom
joe.search

Conversation

@joedursun

Copy link
Copy Markdown
Collaborator

Over time we are experimenting with better search results rankings and this will continue to be an iterative process.

Overview of the changes

Search ranking (constants.py, index.py)

  • New aliases for common natural-language terms: yesterday→previous, today→snapshot, company/companies→ticker+details, biggest→gainers, info/detail(s)→details+overview.
  • rate/rates and open made polysemous (Treasury+aggregate, market-status — disambiguated by other tokens in the query).
  • Market detection: added gainers/losers/movers as Stocks defaults; an explicit Crypto/Forex/Options/Futures/Indices keyword still wins (specific market beats Stocks default).
  • New uppercase-ticker fallback: a 2-5 letter uppercase token that isn't a known acronym (RSI, USD, BTC, VIX, …) infers Stocks. Catches RSI for AAPL, trades for GOOG, etc.
  • Indices vocabulary expanded (spx, ndx, rut, vix).
  • Response-attribute index now skips envelope metadata (status, request_id, results, …) so common tokens keep useful IDF.

Token economy in formatted output (index.py)

  • Strip the trailing Use Cases: … marketing sentence from formatted endpoint descriptions (FTS index keeps it).
  • In more mode, collapse field.gt/field.lte/field.any_of operator variants into one annotated row (ticker (string, optional) [filters: any_of, gt, gte, lt, lte]). verbose still renders every line.

Results

On the 52-query benchmark vs master:

  • top-1: 55.8% → 88.5% (+17 queries promoted)
  • top-3: 67.3% → 96.2%
  • top-7: 73.1% → 100%
  • Zero regressions on the benchmark set.

Stocks and options queries (daily candles for AAPL, RSI for AAPL, options chain for AAPL, last trade for MSFT, current price of TSLA, today's biggest gainers, …) all end up at #1.

@Smartoneinok Smartoneinok left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@joedursun joedursun merged commit c58ec7e into master May 5, 2026
5 checks passed
@joedursun joedursun deleted the joe.search branch May 5, 2026 20:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants