Skip to content

ENG-1405: Optimize recall with LRU blob cache and batched SEAL decrypt#120

Merged
ducnmm merged 1 commit intodevfrom
enhance/ENG-1405-recall-optimization
May 7, 2026
Merged

ENG-1405: Optimize recall with LRU blob cache and batched SEAL decrypt#120
ducnmm merged 1 commit intodevfrom
enhance/ENG-1405-recall-optimization

Conversation

@ducnmm
Copy link
Copy Markdown
Collaborator

@ducnmm ducnmm commented Apr 23, 2026

Summary

  • Add in-process LRU cache for Walrus blob ciphertext — warm recall skips aggregator round-trip (~500ms–2s saved)
  • Add batched SEAL decrypt (/seal/decrypt-batch) — one sidecar call for all blobs vs N×200–500ms
  • Add per-phase timing logs (embed, vector search, walrus fetch, seal decrypt)

Files changed

  • services/server/src/seal.rsseal_decrypt_batch(), DecryptOutcome enum
  • services/server/src/routes.rs — recall rewrite with cache-first fetch + batch decrypt
  • services/server/src/types.rsblob_cache field, DEFAULT_BLOB_CACHE_SIZE
  • services/server/src/main.rs — blob cache initialization
  • services/server/Cargo.tomllru = "0.12"

Test plan

  • cargo check passes
  • Server starts without errors
  • Recall returns correct results (first call cold, second call warm from cache)
  • Logs show per-phase timing and cache hit/miss counts

@ducnmm ducnmm force-pushed the enhance/ENG-1405-recall-optimization branch 4 times, most recently from e72ff93 to 39c08c7 Compare May 7, 2026 02:06
@railway-app railway-app Bot temporarily deployed to MemWal / dev May 7, 2026 02:07 Inactive
@ducnmm ducnmm force-pushed the enhance/ENG-1405-recall-optimization branch from 39c08c7 to 4f7ca7e Compare May 7, 2026 02:27
@railway-app railway-app Bot temporarily deployed to MemWal / dev May 7, 2026 02:27 Inactive
@ducnmm ducnmm force-pushed the enhance/ENG-1405-recall-optimization branch from 4f7ca7e to 728153b Compare May 7, 2026 02:34
@railway-app railway-app Bot temporarily deployed to MemWal / dev May 7, 2026 02:34 Inactive
@hungtranphamminh hungtranphamminh self-requested a review May 7, 2026 04:15
@ducnmm ducnmm merged commit 901acbd into dev May 7, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants