Add per-operation self-tuning dense-store sizing (SizingHint)#12056
Draft
dougqh wants to merge 4 commits into
Draft
Add per-operation self-tuning dense-store sizing (SizingHint)#12056dougqh wants to merge 4 commits into
dougqh wants to merge 4 commits into
Conversation
Sizes each span's dense TagMap store from a per-operation hint instead of a generic default, then feeds the actual known-tag count back on finish so the hint converges to the operation's real high-water mark. - FlatHashtable: open-addressed find-or-create over self-contained entries; static-polymorphism via a concrete-typed Helper singleton so the JIT devirtualizes/inlines hash/matches/create per call site. - SizingHint / SizingHelper: opaque per-operation hint + its String-key helper. - SizingHintTable: process-wide, pure-static two-lane (entry vs child) registry keyed by operation name; fixed-capacity + deliberately lock-free/racy (a lost update or double-mint only mis-sizes an array, never corrupts tag data). - CoreTracer.buildSpanContext resolves the hint by operationName + entry-ness and hands it to DDSpanContext, which sizes its TagMap and records the size back via DDSpan.finishAndAddToTrace. - Tests: FlatHashtableTest, SizingHintTableTest (incl. self-tuning + content-keying). - Benchmarks: FlatHashtableBenchmark; DenseStoreAllocBenchmark buildMapSized arm. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
SpanPrototype and SizingHint converge in a later PR; on this branch the hint is a SizingHint keyed by operation name (SizingHintTable), so the illustrative SpanPrototype mentions were misleading. Reword comments and rename the benchmark field to sizingHint. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signals intent: this delegating overload has no production caller (CoreTracer uses the full ctor); its ~20 callers are tests/jmh. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The three delegating overloads exist only to spare tests the param churn from signature changes; none has a production caller. Mark them all @deprecated (steer new code to the full ctor) and @VisibleForTesting (signal intent). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This comment has been minimized.
This comment has been minimized.
Contributor
🟢 Java Benchmark SLOs — All performance SLOs passed
PR vs. master results
Commit: Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion. |
Contributor
Bits has a CI fix ready🟢 Investigated · 🟢 Fix prepared · ⚪ Validation skipped · 🟠 Ready Registered View in Datadog | Reviewed commit 956f2c9 · Any feedback? Reach out in #deveng-pr-agent |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Sizes each span's dense
TagMapstore from a per-operation sizing hint instead of a generic default, then feeds the actual known-tag count back on span finish so the hint converges to that operation's real high-water mark. Over-provisioning shrinks toward the operation's true shape without any hand-tuned per-type tables.This is the SizingHint layer of the dense-store stack, sitting between the generator (
dougqh/generator-v2) and the id-keyedTagMap.set(long)work (dougqh/tag-id-api, #11901).How
FlatHashtable— open-addressed find-or-create over self-contained entries; static-polymorphism via a concrete-typedHelpersingleton so the JIT devirtualizes/inlineshash/matches/createper call site. Single-reference publish → torn-free under racy access.SizingHint/SizingHelper— opaque per-operation hint (label + cached hash + tuned size) and itsString-key helper.SizingHintTable— process-wide, pure-static two-lane (entry vs child span) registry keyed by operation name. Fixed-capacity and deliberately lock-free/racy: a lost update or double-mint only mis-sizes an array (over/under-provision) for a span or two — it never corrupts tag data.CoreTracer.buildSpanContextresolves the hint by operation name + entry-ness and hands it toDDSpanContext, which sizes itsTagMapfrom it and records the final size back viaDDSpan.finishAndAddToTrace.Testing
FlatHashtableTest,SizingHintTableTest(incl. the self-tuningrecordSizeloop and content-keying so aUTF8BytesStringoperation name resolves to the same hint as itsStringform).FlatHashtableBenchmark; abuildMapSizedarm inDenseStoreAllocBenchmark.spotlessJavaCheck,:internal-api:spotbugsMain,:dd-trace-core:spotbugsMainall green.Notes
DDSpanContextctors are marked@Deprecated @VisibleForTestingto steer new code to the full ctor.🤖 Generated with Claude Code