Run saveUsageCost in background to fix slow proxy responses#4244
Run saveUsageCost in background to fix slow proxy responses#4244jurgenwerk wants to merge 1 commit intomainfrom
Conversation
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: bc12a4d213
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| // 6. Calculate and deduct credits in the background | ||
| // so we don't block the response on OpenRouter's generation cost API | ||
| destinationConfig.creditStrategy | ||
| .saveUsageCost(dbAdapter, matrixUserId, responseData) |
There was a problem hiding this comment.
Making saveUsageCost fire-and-forget here returns the client response before billing is recorded, but credit enforcement still happens earlier via validateCredits, so subsequent requests are authorized against stale balances. In this codebase, saveUsageCost can back off/retry for up to minutes when generation cost is not yet available, which creates a large window where a user can issue multiple requests and bypass the minimum-credit guard before deductions land; this is a production under-billing regression (and the same pattern is now used in the streaming [DONE] branch too).
Useful? React with 👍 / 👎.
jurgenwerk
force-pushed
the
cs-10501-ai-proxy-request-is-very-slow
branch
4 times, most recently
from
ff5274e to
dbe6040
Compare
The request forward handler was awaiting saveUsageCost before returning the response. This function polls OpenRouter's generation cost API with exponential backoff (1s, 2s, 4s, 8s...) because cost data is often not immediately available, adding 10-15+ seconds of latency. Run it in the background instead so responses return immediately. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
jurgenwerk
force-pushed
the
cs-10501-ai-proxy-request-is-very-slow
branch
from
dbe6040 to
26ab140
Compare
1 participant
Summary
_request-forwardhandler wasawaitingsaveUsageCostbefore returning the response to the clientsaveUsageCostpolls OpenRouter's generation cost API (/api/v1/generation?id=...) with exponential backoff (1s, 2s, 4s, 8s...) because cost data is often not immediately available (404), adding 10-15+ seconds of latencysaveUsageCostin the background so responses return immediatelyTest plan
_request-forwardto OpenRouter and verify the response returns promptly (~5s instead of ~20s)[DONE]event is not delayed🤖 Generated with Claude Code