DI: backfill CodeTracker registry with iseqs for pre-loaded files

[p-datadog]

What does this PR do?

When CodeTracker starts, backfills the iseq registry with instruction sequences for files that were loaded before tracking began. Also adds a DI.iseq_type C extension (Ruby 3.1+) for precise whole-file iseq detection.

Motivation:

CodeTracker only captures files loaded after DI starts via :script_compiled TracePoint events. Most application code and all gems are loaded at boot time before DI activates, making them invisible to line probes. This change walks the Ruby object space via the all_iseqs C extension (#5111) and recovers whole-file iseqs for already-loaded code.

Implementation:

• CodeTracker#backfill_registry walks the object space via DI.file_iseqs, filters to whole-file iseqs, and stores them in the registry without overwriting entries from :script_compiled.
• Whole-file iseq detection uses DI.iseq_type on Ruby 3.1+ (returns :top for require/load, :main for the entry script). On Ruby < 3.1, falls back to first_lineno == 0 — which produces the same result because Ruby's rb_iseq_new_top and rb_iseq_new_main pass INT2FIX(0) while method/class/block definitions get their source line (>= 1).
• DI.iseq_type wraps the internal rb_iseq_type() function added in Ruby 3.1 (commit 89a02d89 by Koichi Sasada). Guarded behind have_func('rb_iseq_type') in extconf.rb — only compiled when the symbol exists.
• Backfill runs inside start after the trace point is enabled, so files loaded concurrently are captured by the trace point (backfill won't overwrite them).
• compile_file iseqs are excluded via first_lineno == 0 check on all Ruby versions. Targeted TracePoints are bound to the specific iseq object — a compile_file iseq is distinct from the require-produced iseq, so a probe installed against it silently never fires. Verified by reading vm_trace.c and iseq.c.

Change log entry

Yes. DI/LD: more third-party libraries are now instrumentable in DI/LD.

Additional Notes:

• Depends on DI: add a C extension #5111 (the all_iseqs and exception_message C extensions)
• Only whole-file iseqs are backfilled — per-method iseq support is in DI: support per-method iseqs for line probes on pre-loaded files #5501
• Some stdlib and gem iseqs are garbage collected and unavailable via object space walk — per-method iseq fallback in DI: support per-method iseqs for line probes on pre-loaded files #5501 addresses this (86% of GC'd files have surviving per-method iseqs)
• Adds di:bootsnap CI task with bootsnap gem in rails8 appraisals for Bootsnap integration testing
• E2e tests cover both code paths: :script_compiled (file loaded after tracking) and backfill (file loaded before tracking), including snapshot capture with local variables

Documentation: Updated in #5501. While this PR qualitatively improves which third-party code is instrumentable, in our testing the gains from whole-file backfill alone are ~10% of pre-loaded files (the rest have GC'd top iseqs and require per-method backfill from #5501). To avoid churn, #5501 removes the entire "Code Tracking Requirement" section from docs/DynamicInstrumentation.md; this PR intentionally keeps the existing docs as-is.

How to test the change?

Unit tests for backfill_registry (filtering, overwrite protection, error boundary, fallback path, compile_file exclusion). Integration tests: backfill path (load before tracking → probe installs and fires → snapshot captured) and :script_compiled path (load after tracking → probe installs and fires → snapshot captured). Bootsnap integration test (requires bootsnap gem, runs in di:bootsnap CI job). TracePoint binding tests verifying compile_file iseqs don't fire for require-produced code.

Manually tested in gobo by inspecting the DI code tracer registry. Before:

After:

Manual testing via gobo targeting stdlib:

[github-actions]

Typing analysis

Note: Ignored files are excluded from the next sections.

steep:ignore comments

This PR introduces 1 steep:ignore comment.

steep:ignore comments (+1-0)

❌ Introduced:

lib/datadog/di/base.rb:109

Untyped methods

This PR introduces 1 partially typed method, and clears 1 partially typed method. It increases the percentage of typed methods from 61.85% to 61.89% (+0.04%).

Partially typed methods (+1-1)

❌ Introduced:

sig/datadog/di/code_tracker.rbs:15
└── def iseqs_for_path_suffix: (String suffix) -> untyped

✅ Cleared:

sig/datadog/di/code_tracker.rbs:14
└── def iseqs_for_path_suffix: (String suffix) -> untyped

If you believe a method or an attribute is rightfully untyped or partially typed, you can add # untyped:accept on the line before the definition to remove it from the stats.

[datadog-prod-us1-6]

⚠️ Tests

✨ Fix all issues with BitsAI or with Cursor

⚠️ Other Violations

🧪 1 Test failed

DI CodeTracker with Bootsnap file loaded via Bootsnap iseq cache Bootsnap cache was actually used (not just normal compilation) from rspec     (Fix with Cursor)

No Bootsnap cache files found in /tmp/bootsnap_di_test20260409-11556-21368z. Bootsnap may not have been properly initialized.

Failure/Error:
  expect(cache_files).not_to be_empty,
    "No Bootsnap cache files found in #{cache_dir}. " \
    "Bootsnap may not have been properly initialized."

  No Bootsnap cache files found in /tmp/bootsnap_di_test20260409-11556-21368z. Bootsnap may not have been properly initialized.
./spec/datadog/di/contrib/bootsnap/code_tracker_bootsnap_spec.rb:241:in \`block (3 levels) in <top (required)>'
./spec/datadog/di/spec_helper.rb:128:in \`block in di_test'
...

ℹ️ Info

No other issues found (see more)

❄️ No new flaky tests detected

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 62f71b1 | Docs | Datadog PR Page | Was this helpful? React with 👍/👎 or give us feedback!}

[pr-commenter]

Benchmarks

Benchmark execution time: 2026-04-09 17:38:34

Comparing candidate commit 52f34fb in PR branch di-iseq-backfill with baseline commit 2365975 in branch master.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 45 metrics, 1 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

• 🟩 = significantly better candidate vs. baseline
• 🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

[p-datadog]

@codex review

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: cd1e2d3b72

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[p-datadog]

@codex review

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a104ffd302

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Install pending line probes for backfilled files

When backfill_registry adds a pre-loaded file to registry, it never triggers probe_manager.install_pending_line_probes(path), unlike the :script_compiled path in start. This leaves line probes stuck in pending_probes if they were queued before tracking started (e.g., delayed/programmatic activate_tracking! or tracker restart), so the newly backfilled file remains uninstrumented until it is reloaded and emits another compile event. Calling pending-probe installation when a new backfilled path is inserted would make backfill behave consistently with runtime load tracking.

Useful? React with 👍 / 👎.