Skip to content

Generate KnownTags from tag-conventions via the tag-registry code generator (phase 2)#12047

Draft
dougqh wants to merge 1 commit into
dougqh/bloom-v2from
dougqh/generator-v2
Draft

Generate KnownTags from tag-conventions via the tag-registry code generator (phase 2)#12047
dougqh wants to merge 1 commit into
dougqh/bloom-v2from
dougqh/generator-v2

Conversation

@dougqh

@dougqh dougqh commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

What Does This Do

Replaces the hand-maintained known-tag table with one generated from a single source of truth (tag-conventions.yaml + a Java overlay), so tag ids / the id↔name resolver / (later) per-type layouts stop drifting across core, instrumentation, and language tracers.

Generator (buildSrc)

Modeled on SupportedConfigPlugin (data file → .java via PrintWriter):

  • TagRegistryGeneratorPlugin + GenerateKnownTagsTask — read tag-conventions.yaml (domain) + tag-conventions.java.yaml (Java overlay: intercepted hints + virtual/special registry), resolve per-type tag sets (extends/include/applies, de-duped), assign ids in stable serial order, and greedily graph-color the colorable set into slots.
  • KnownTagsEmitter — emits google-java-format-clean Java to committed internal-api/src/generated with a literal id + // tagId(serial=…, group=…, field=…) audit comment (literal ⟹ javac constant-folds; comment ⟹ auditable).
  • verifyKnownTags — regenerates + git-diffs to catch stale commits. Generated classes are jacoco-excluded.

Committing the generated sources also lets us audit what ids/layout any release branch shipped via git checkout.

internal-api

  • Deletes the interim hand-maintained src/main/.../KnownTags.java in favor of the generated src/generated/.../KnownTags.java (byte-equal ids — the equivalence gate).
  • Read-through stays chain-aware: keeps this stack's multi-level parentDenseVisible over the pre-reconcile single-level form.
  • Generated keyOf table uses StringIndex.EmbeddingSupport (matching the tagset base rename) in both the emitter and its committed output, so regeneration is drift-free.

Verification

:internal-api:generateKnownTags (no drift), verifyKnownTags, KnownTagsTest, TagMapDenseForkedTest, TagMapDenseFuzzForkedTest, spotlessJavaCheck, spotbugsMain — all green.

Stack

tagset → dense-store (#12045) → two-tier bloom (#12046) → **generator**. Base dougqh/bloom-v2. Supersedes old #11961.

Non-goals (follow-on)

Wiring SpanLayouts into the dense store; full virtual-dispatch table → TagInterceptor retirement; bandwidth-minimizing packer (greedy is v1).

🤖 Generated with Claude Code

@dougqh dougqh added comp: core Tracer core tag: ai generated Largely based on code generated by an AI or LLM tag: no release notes Changes to exclude from release notes type: refactoring labels Jul 23, 2026
@datadog-prod-us1-4

datadog-prod-us1-4 Bot commented Jul 23, 2026

Copy link
Copy Markdown

Bits has a CI fix ready

🟢 Investigated · 🟢 Fix prepared · ⚪ Validation skipped · 🟠 Ready

CoreTracer.java had a line exceeding the required formatter’s wrapping rules. Wrapped the dense-tag initialization comment to comply with the repository style.

Commit fix to this PR


View in Datadog | Reviewed commit e6800b8 · Any feedback? Reach out in #deveng-pr-agent

@datadog-prod-us1-4

datadog-prod-us1-4 Bot commented Jul 23, 2026

Copy link
Copy Markdown

🎯 Code Coverage (details)
Patch Coverage: 43.53%
Overall Coverage: 57.38% (-0.02%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: e6800b8 | Docs | Datadog PR Page | Give us feedback!

@dd-octo-sts

dd-octo-sts Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite Status
Startup 🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results
Scenario Candidate master Δ (95% CI of mean)
startup:insecure-bank:iast:Agent 14.82 s 14.74 s [-0.4%; +1.4%] (no difference)
startup:insecure-bank:tracing:Agent 13.60 s 13.76 s [-2.1%; -0.2%] (maybe better)
startup:petclinic:appsec:Agent 17.04 s 17.00 s [-0.7%; +1.2%] (no difference)
startup:petclinic:iast:Agent 17.08 s 17.07 s [-0.8%; +1.0%] (no difference)
startup:petclinic:profiling:Agent 16.76 s 16.88 s [-1.8%; +0.4%] (no difference)
startup:petclinic:sca:Agent 16.98 s 16.89 s [-0.3%; +1.4%] (no difference)
startup:petclinic:tracing:Agent 16.20 s 16.20 s [-1.0%; +1.0%] (no difference)

Commit: e6800b8c · CI Pipeline · Benchmarking Platform UI


Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

Comment thread internal-api/src/generated/java/datadog/trace/api/KnownTags.java Outdated
Comment thread internal-api/src/generated/java/datadog/trace/api/KnownTags.java Outdated
Comment thread internal-api/src/generated/java/datadog/trace/api/KnownTags.java Outdated
@dougqh
dougqh force-pushed the dougqh/generator-v2 branch from 79bc101 to 7f3d0b1 Compare July 23, 2026 02:34
…erator

Replaces the hand-maintained known-tag table with one emitted from a single
source of truth, so tag ids / resolver / (later) per-type layouts stop drifting
across core, instrumentation, and language tracers.

Generator (buildSrc, à la SupportedConfigPlugin):
- TagRegistryGeneratorPlugin + GenerateKnownTagsTask read tag-conventions.yaml
  (domain) + tag-conventions.java.yaml (Java overlay), resolve per-type tag sets
  (extends/include/applies, de-duped), assign ids, and greedily graph-color the
  colorable set into slots.
- KnownTagsEmitter writes google-java-format-clean Java to committed
  internal-api/src/generated (literal id + `// tagId(...)` audit comment).
- VerifyKnownTagsTask (verifyKnownTags) regenerates + git-diffs to catch stale
  commits; generated classes are jacoco-excluded.

internal-api:
- Deletes the interim hand-maintained src/main KnownTags in favor of the
  generated src/generated/.../KnownTags.java (byte-equal ids).
- Read-through stays chain-aware: keeps parentDenseVisible (multi-level) over
  the pre-reconcile single-level form.

The generated keyOf table uses StringIndex.EmbeddingSupport (matching the tagset
base rename) in both the emitter and its committed output, so regeneration is
drift-free.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@dougqh
dougqh force-pushed the dougqh/generator-v2 branch from 7f3d0b1 to e6800b8 Compare July 23, 2026 02:42
@dougqh dougqh changed the title Generate KnownTags from tag-conventions via the tag-registry code generator Generate KnownTags from tag-conventions via the tag-registry code generator (phase 2) Jul 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp: core Tracer core tag: ai generated Largely based on code generated by an AI or LLM tag: no release notes Changes to exclude from release notes type: refactoring

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant