Skip to content

fix: extend TTFT histogram upper-tail buckets#1272

Open
pjb157 wants to merge 1 commit into
mainfrom
peter/extend-ttft-histogram-buckets
Open

fix: extend TTFT histogram upper-tail buckets#1272
pjb157 wants to merge 1 commit into
mainfrom
peter/extend-ttft-histogram-buckets

Conversation

@pjb157

@pjb157 pjb157 commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

Summary\n- preserve the existing time-to-first-token histogram boundaries through 10 seconds\n- add a sparse upper tail through 120 seconds\n- verify the exported boundary set with a focused regression test\n\n## Cardinality\nThis adds eight classic histogram bucket series per populated label set. The spacing widens as latency increases to improve upper-tail resolution without introducing a dense upper range.\n\n## Rollout\nBecause the classic histogram keeps its existing metric name, lookback queries that span deployment temporarily mix the old and new bucket schemas. The 24h, 7d, and 30d views may show distorted quantiles until each window contains only post-deploy samples.\n\n## Verification\n- cargo fmt --all --check\n- cargo test -p dwctl metrics::gen_ai::tests\n- SQLX_OFFLINE=true cargo clippy -p dwctl --lib -- -D warnings

Copilot AI review requested due to automatic review settings July 14, 2026 13:47
@cloudflare-workers-and-pages

Copy link
Copy Markdown

Deploying control-layer with  Cloudflare Pages  Cloudflare Pages

Latest commit: 9440c06
Status: ✅  Deploy successful!
Preview URL: https://bb5f563d.control-layer.pages.dev
Branch Preview URL: https://peter-extend-ttft-histogram.control-layer.pages.dev

View logs

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Extends the Prometheus classic histogram bucket boundaries for GenAI “time to first token” (TTFT) to better capture slow streaming responses while keeping the existing OTel-derived buckets through 10 seconds, and adds a focused regression test to lock in the exported boundary set.

Changes:

  • Extend gen_ai_server_time_to_first_token_seconds histogram buckets with a sparse upper-tail out to 120 seconds (preserving existing buckets through 10s).
  • Add a unit test that asserts the gathered TTFT histogram exports the expected upper-bound list.

Comment on lines +284 to +286
#[test]
fn test_time_to_first_token_histogram_buckets_cover_slow_requests() {
let registry = Registry::new();
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants