fix: extend TTFT histogram upper-tail buckets#1272
Open
pjb157 wants to merge 1 commit into
Open
Conversation
Deploying control-layer with
|
| Latest commit: |
9440c06
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://bb5f563d.control-layer.pages.dev |
| Branch Preview URL: | https://peter-extend-ttft-histogram.control-layer.pages.dev |
Contributor
There was a problem hiding this comment.
Pull request overview
Extends the Prometheus classic histogram bucket boundaries for GenAI “time to first token” (TTFT) to better capture slow streaming responses while keeping the existing OTel-derived buckets through 10 seconds, and adds a focused regression test to lock in the exported boundary set.
Changes:
- Extend
gen_ai_server_time_to_first_token_secondshistogram buckets with a sparse upper-tail out to 120 seconds (preserving existing buckets through 10s). - Add a unit test that asserts the gathered TTFT histogram exports the expected upper-bound list.
Comment on lines
+284
to
+286
| #[test] | ||
| fn test_time_to_first_token_histogram_buckets_cover_slow_requests() { | ||
| let registry = Registry::new(); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary\n- preserve the existing time-to-first-token histogram boundaries through 10 seconds\n- add a sparse upper tail through 120 seconds\n- verify the exported boundary set with a focused regression test\n\n## Cardinality\nThis adds eight classic histogram bucket series per populated label set. The spacing widens as latency increases to improve upper-tail resolution without introducing a dense upper range.\n\n## Rollout\nBecause the classic histogram keeps its existing metric name, lookback queries that span deployment temporarily mix the old and new bucket schemas. The 24h, 7d, and 30d views may show distorted quantiles until each window contains only post-deploy samples.\n\n## Verification\n- cargo fmt --all --check\n- cargo test -p dwctl metrics::gen_ai::tests\n- SQLX_OFFLINE=true cargo clippy -p dwctl --lib -- -D warnings