Provider leaderboard over-attributes request cost to billed hedge losers

[Brisbanehuang]

Summary

Provider cost leaderboard can over-attribute a full request cost to a hedge loser provider when hedge loser billing appends hedge_loser_billed as the last provider_chain entry.

This is a provider-level reporting issue. The request total itself is not necessarily overbilled, but the provider leaderboard / usage_ledger.final_provider_id attribution becomes wrong.

What happens

For a hedged request:

1. Provider A is selected first and later becomes a billed hedge loser.
2. Provider B wins and returns the response to the client.
3. The request row stores total cost as:

message_request.cost_usd = winnerCost + SUM(hedge_losers[].costUsd)

4. The provider chain ends with an entry like:

{ "id": <provider A id>, "reason": "hedge_loser_billed" }

5. fn_upsert_usage_ledger() derives usage_ledger.final_provider_id from the last provider_chain entry:

v_final_provider_id := (NEW.provider_chain -> -1 ->> 'id')::integer;

6. Provider leaderboard groups by usage_ledger.final_provider_id and sums usage_ledger.cost_usd.

Result: Provider A, the loser, receives the entire request cost in the provider leaderboard, even though only hedge_losers[].costUsd belongs to Provider A.

Why this looks wrong

hedge_loser_billed is a terminal accounting/audit entry, but not the provider that served the successful response. Treating the last chain entry as final_provider_id is no longer reliable once hedge loser billing appends loser billing records after the winner.

The affected code paths appear to be:

• src/lib/ledger-backfill/trigger.sql: fn_upsert_usage_ledger() derives final_provider_id from provider_chain -> -1.
• src/lib/ledger-backfill/service.ts: backfill uses the same last-chain-entry rule.
• src/repository/leaderboard.ts: provider leaderboard groups by usage_ledger.final_provider_id and sums usage_ledger.cost_usd.

Minimal reproduction shape

A single request row with:

provider_id = Provider B  -- winner / request row provider
cost_usd = 0.54           -- winner + loser total
hedge_losers = [
  { providerId: Provider A, costUsd: "0.03", attemptNumber: 1 }
]
provider_chain last entry =
  { id: Provider A, reason: "hedge_loser_billed" }

Current ledger result:

usage_ledger.final_provider_id = Provider A
usage_ledger.cost_usd = 0.54

Expected attribution for provider-level cost reporting:

Provider B: winner portion, roughly 0.51
Provider A: hedge loser portion, 0.03

or, at minimum, final_provider_id should continue to point to the actual winning/serving provider rather than the trailing hedge_loser_billed audit entry.

Suggested directions

Possible fixes:

1. When deriving final_provider_id, ignore trailing hedge_loser_billed / hedge_loser_cancelled entries and use the winning provider entry, for example hedge_winner, request_success, or retry_success.
2. For provider cost leaderboard, split hedge loser costs explicitly by expanding message_request.hedge_losers, so provider cost becomes:
   • winner provider: message_request.cost_usd - SUM(hedge_losers[].costUsd)
   • each loser provider: its own hedge_losers[].costUsd
3. Longer term, consider a per-provider ledger table/rows for hedge billing rather than one request-level row with a single final_provider_id.

Option 1 fixes winner attribution, but option 2 is needed if the provider leaderboard should include billed loser costs under the loser provider as well.

Impact

Provider cost leaderboard and provider-level cost summaries can be significantly inflated for providers that frequently appear as billed hedge losers. The total user/request cost can still be correct; the bug is in provider attribution.

[coderabbitai]

🔗 Related PRs

#581 - Fix/logs provider badge overflow [merged]
#753 - feat(leaderboard): provider avg-cost metrics and cache-hit model drilldown [merged]
#992 - fix: preserve hedge model redirect audit per attempt [merged]
#1247 - feat(billing): 供应商竞速输家计费(保活拿回响应 + 幂等累加回写) [merged]
#1255 - fix(proxy): preserve hedge loser Codex priority billing [merged]

---

📝 Issue Planner

_{Check the box below or use the @coderabbitai plan command to generate an implementation plan and prompts that you can use with your favorite coding assistant.}

• Create Plan

---

🧪 Issue enrichment is currently in open beta.

You can configure auto-planning by selecting labels in the issue_enrichment configuration.

To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

issue_enrichment:
  auto_enrich:
    enabled: false

💬 Have feedback or questions? Drop into our discord!

[github-actions]

Thank you for this detailed bug report.

Analysis:

I've confirmed the issue by examining the relevant code paths. The problem occurs when hedge loser billing appends a hedge_loser_billed entry to the end of provider_chain, which then becomes the final_provider_id used for provider leaderboard attribution.

Affected code:

The core issue is in how final_provider_id is derived from the last chain entry:

• src/lib/ledger-backfill/trigger.sql (lines 182-191): fn_upsert_usage_ledger() extracts final_provider_id from the last provider_chain entry:

v_final_provider_id := (NEW.provider_chain -> -1 ->> 'id')::integer;

• src/lib/ledger-backfill/service.ts (lines 49-60): The backfill service uses identical logic, reading provider_chain -> -1 to determine final_provider_id.

• src/repository/leaderboard.ts (lines 679-699): Provider leaderboard queries group by usage_ledger.final_provider_id and sum usage_ledger.cost_usd, so the entire request cost (winner + loser) is attributed to whichever provider ID appears last in the chain.

How hedge loser billing creates the issue:

When a hedged request completes:

1. Winner records first: src/app/v1/_lib/proxy/forwarder.ts (lines 4404-4410) appends the winning provider with reason: "hedge_winner" or "request_success" to the chain.

2. Loser records after: If the loser is billed, src/app/v1/_lib/proxy/forwarder.ts (lines 3912-3921) appends the loser provider with reason: "hedge_loser_billed" as the last chain entry.

3. Cost accumulation: src/app/v1/_lib/proxy/response-handler.ts (lines 3858-4007) calculates the loser cost and calls addMessageRequestHedgeLoserCost, which adds the loser cost to the request total via:

cost_usd = COALESCE(cost_usd, 0) + delta

(see src/repository/message.ts lines 439)

4. Wrong attribution: The trigger/backfill logic sees hedge_loser_billed as the last chain entry and assigns the entire cost_usd (winner + loser total) to the loser provider's final_provider_id.

Result:

Provider leaderboard inflates costs for providers that frequently appear as billed hedge losers. The total user/request cost remains correct; only the provider-level attribution is wrong.

Verification:

All file paths, line numbers, and code references have been verified against the current codebase. The issue reproduction matches the code behavior exactly as described.

Your suggested fixes are sound:

Option 1 (ignore trailing audit entries) would fix winner attribution but wouldn't show loser costs under the loser provider. Option 2 (split hedge loser costs) would provide full visibility by expanding hedge_losers and attributing each portion correctly. Option 3 (per-provider ledger rows) is the cleanest long-term solution for accurate per-provider accounting.

---

Automated response from Claude AI