Skip to content

fix(openfeature): use delta temporality for OTLP metrics export#11426

Merged
gh-worker-dd-mergequeue-cf854d[bot] merged 3 commits into
masterfrom
sameerank/fix-flag-eval-metrics-delta-temporality
May 22, 2026
Merged

fix(openfeature): use delta temporality for OTLP metrics export#11426
gh-worker-dd-mergequeue-cf854d[bot] merged 3 commits into
masterfrom
sameerank/fix-flag-eval-metrics-delta-temporality

Conversation

@sameerank
Copy link
Copy Markdown
Contributor

@sameerank sameerank commented May 20, 2026

What Does This Do

Changes the OTLP metrics exporter from alwaysCumulative() to deltaPreferred() for feature flag evaluation metrics.

Motivation

OTLP receivers (like the Datadog Agent) expect delta temporality for counter metrics. With cumulative temporality, each export sends the total count since start (5, 10, 15...), and receivers that sum these values produce overcounted results (5, 15, 30...).

With delta temporality, each export sends only the increment since the last export (5, 5, 5...), which receivers correctly sum to the actual total.

Testing in https://github.com/DataDog/ffe-dogfooding I saw Java was overcounting with OTLP, compared to the other SDKs.

This aligns with Datadog's OTLP documentation which recommends delta temporality for metrics.

Datadog works best with delta aggregation temporality for monotonic sums, histograms, and exponential histograms.

Additional Notes

  • deltaPreferred() uses DELTA for Counter/Histogram and CUMULATIVE for UpDownCounter (per OTel spec)
  • Updated test to use delta temporality to match production configuration
  • FlagEvalMetrics is the only direct OTLP exporter in dd-trace-java; other OTel metrics go through the DogStatsD shim

Contributor Checklist

@sameerank sameerank added the type: bug Bug report and fix label May 20, 2026
@sameerank sameerank marked this pull request as ready for review May 20, 2026 05:25
@sameerank sameerank requested a review from a team as a code owner May 20, 2026 05:26
@sameerank sameerank requested review from leoromanovsky and typotter and removed request for a team May 20, 2026 05:26
@dd-octo-sts
Copy link
Copy Markdown
Contributor

dd-octo-sts Bot commented May 20, 2026

Hi! 👋 Thanks for your pull request! 🎉

To help us review it, please make sure to:

  • Add at least one type, and one component or instrumentation label to the pull request

If you need help, please check our contributing guidelines.

@datadog-datadog-prod-us1

This comment has been minimized.

@sameerank sameerank added the comp: metrics Metrics label May 20, 2026
@sameerank sameerank force-pushed the sameerank/fix-flag-eval-metrics-delta-temporality branch from 4f318d6 to 4880dbe Compare May 20, 2026 06:07
@sameerank sameerank requested a review from dd-oleksii May 20, 2026 19:21
@dd-oleksii dd-oleksii added the comp: openfeature OpenFeature label May 22, 2026
@sameerank sameerank added this pull request to the merge queue May 22, 2026
@dd-octo-sts
Copy link
Copy Markdown
Contributor

dd-octo-sts Bot commented May 22, 2026

/merge

@gh-worker-devflow-routing-ef8351
Copy link
Copy Markdown

gh-worker-devflow-routing-ef8351 Bot commented May 22, 2026

View all feedbacks in Devflow UI.

2026-05-22 17:11:37 UTC ℹ️ Start processing command /merge


2026-05-22 17:11:43 UTC ℹ️ MergeQueue: pull request added to the queue

The expected merge time in master is approximately 1h (p90).


2026-05-22 17:19:47 UTCMergeQueue: This merge request was updated

This PR is rejected because it was updated

@github-merge-queue github-merge-queue Bot removed this pull request from the merge queue due to failed status checks May 22, 2026
@sameerank sameerank added this pull request to the merge queue May 22, 2026
@dd-octo-sts
Copy link
Copy Markdown
Contributor

dd-octo-sts Bot commented May 22, 2026

/merge

@gh-worker-devflow-routing-ef8351
Copy link
Copy Markdown

gh-worker-devflow-routing-ef8351 Bot commented May 22, 2026

View all feedbacks in Devflow UI.

2026-05-22 19:31:57 UTC ℹ️ Start processing command /merge


2026-05-22 19:32:01 UTC ℹ️ MergeQueue: pull request added to the queue

The expected merge time in master is approximately 1h (p90).


2026-05-22 19:37:30 UTCMergeQueue: This merge request was updated

This PR is rejected because it was updated

@github-merge-queue github-merge-queue Bot removed this pull request from the merge queue due to failed status checks May 22, 2026
@dd-octo-sts
Copy link
Copy Markdown
Contributor

dd-octo-sts Bot commented May 22, 2026

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite Status
Startup 🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results

Startup Time

Scenario This PR master Change
insecure-bank / iast 13,967 ms 13,864 ms +0.7%
insecure-bank / tracing 12,973 ms 12,980 ms -0.1%
petclinic / appsec 16,527 ms 16,344 ms +1.1%
petclinic / iast 15,533 ms 16,514 ms -5.9%
petclinic / profiling 16,374 ms 16,529 ms -0.9%
petclinic / tracing 15,857 ms 15,795 ms +0.4%

Commit: 98af643e · CI Pipeline · Benchmarking Platform UI


Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

@sameerank
Copy link
Copy Markdown
Contributor Author

/merge

@gh-worker-devflow-routing-ef8351
Copy link
Copy Markdown

gh-worker-devflow-routing-ef8351 Bot commented May 22, 2026

View all feedbacks in Devflow UI.

2026-05-22 20:15:55 UTC ℹ️ Start processing command /merge


2026-05-22 20:16:00 UTC ℹ️ MergeQueue: pull request added to the queue

The expected merge time in master is approximately 1h (p90).


2026-05-22 21:28:30 UTC ℹ️ MergeQueue: This merge request was merged

@gh-worker-dd-mergequeue-cf854d gh-worker-dd-mergequeue-cf854d Bot merged commit 845da1b into master May 22, 2026
569 checks passed
@gh-worker-dd-mergequeue-cf854d gh-worker-dd-mergequeue-cf854d Bot deleted the sameerank/fix-flag-eval-metrics-delta-temporality branch May 22, 2026 21:28
@github-actions github-actions Bot added this to the 1.63.0 milestone May 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp: metrics Metrics comp: openfeature OpenFeature type: bug Bug report and fix

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants