Skip to content

perf: cache smart routing decisions#45

Open
saurabhhhcodes wants to merge 1 commit into
Ruthwik000:mainfrom
saurabhhhcodes:perf/smart-routing-cache-37
Open

perf: cache smart routing decisions#45
saurabhhhcodes wants to merge 1 commit into
Ruthwik000:mainfrom
saurabhhhcodes:perf/smart-routing-cache-37

Conversation

@saurabhhhcodes
Copy link
Copy Markdown

Summary

  • add bounded, TTL-based routing decision caching for repeated failure/prompt contexts
  • reduce repeated lookup work in fallback/context/cost routing strategies and error classification
  • document the new performance options and add a benchmark-style regression test for the <10ms overhead target

Validation

  • npm run build
  • node tests/performance-optimization.test.js
  • node tests/cross-provider.test.js
  • git diff --check

Closes #37

@saurabhhhcodes
Copy link
Copy Markdown
Author

Quick scoring-label request: this PR closes GSSoC issue #37, which is labeled gssoc'26, level2, and feature: smart-routing. Could you please mirror the applicable GSSoC/difficulty/type labels onto the PR before merge?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[GSSoC'26] Performance Optimization

1 participant