Skip to content

Fix worker metrics panic caused by inconsistent Prometheus label cardinality#185

Open
pablo1664 wants to merge 3 commits intoclyso:mainfrom
pablo1664:fpi-wild-chicken-amber
Open

Fix worker metrics panic caused by inconsistent Prometheus label cardinality#185
pablo1664 wants to merge 3 commits intoclyso:mainfrom
pablo1664:fpi-wild-chicken-amber

Conversation

@pablo1664
Copy link
Contributor

Pull Request

Description

This PR fixes a worker panic when metrics are enabled.

Problem
copy_in_progress_obj_bytes is a GaugeVec defined with 3 labels (flow, user, bucket), but the worker was conditionally building labels and sometimes sending only 2 (for example missing user).
This caused:
panic: inconsistent label cardinality: expected 3 label values but got 2

Fix

  • Refactored worker copy metric labeling to always provide all expected label keys.
  • Added a shared helper used by both:
    • WorkerInProgressBytesInc()
    • WorkerInProgressBytesDec()
  • Empty values are now passed as empty strings instead of omitting labels, preventing cardinality mismatch panics.

Changed file:

Validation

  • Targeted tests executed successfully:
    • go test ./pkg/metrics ./service/worker/copy ./service/worker/handler (exit code 0)

Signed-off-by: François PIRES <francois@pires.eus>
@pablo1664 pablo1664 force-pushed the fpi-wild-chicken-amber branch from 30642f3 to 4abc168 Compare February 26, 2026 14:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant