Use otel injector by swiatekm · Pull Request #5347 · open-telemetry/opentelemetry-operator

swiatekm · 2026-07-21T18:08:50Z

Description:

Add a feature flag to switch to the otel injector for injecting instrumentation. The injector is included in instrumentation images unconditionally.

Link to tracking Issue(s):

Resolves: #issue-number

Testing:

Documentation:

…entation When the operator.autoinstrumentation.injector feature gate is enabled, the operator activates the Java agent by LD_PRELOAD-ing the opentelemetry-injector shared object instead of setting JAVA_TOOL_OPTIONS on the container. The injector appends the -javaagent flag in-process, only for processes that are actually JVMs. The injector binary is added to the Java auto-instrumentation image for amd64 and arm64; other architectures ship without it. Claude-Session: https://claude.ai/code/session_01WdcmBbsVDxxQSPvpPvR2LQ Signed-off-by: Mikołaj Świątek <mail@mikolajswiatek.com>

…entation When the operator.autoinstrumentation.injector feature gate is enabled, the operator activates the .NET instrumentation by LD_PRELOAD-ing the opentelemetry-injector shared object instead of setting the CORECLR_*/DOTNET_* environment variables on the container. The injector sets those variables in-process, only for .NET applications, and detects the libc flavor (glibc/musl) and CPU architecture of the application on its own, making the otel-dotnet-auto-runtime annotation unnecessary. The injector expects separate glibc and musl installation roots, so the .NET auto-instrumentation image maps that layout onto its merged installation with symlinked directories. Claude-Session: https://claude.ai/code/session_01WdcmBbsVDxxQSPvpPvR2LQ Signed-off-by: Mikołaj Świątek <mail@mikolajswiatek.com>

…n auto-instrumentation When the operator.autoinstrumentation.injector feature gate is enabled, the operator activates the Node.js and Python instrumentation by LD_PRELOAD-ing the opentelemetry-injector shared object instead of setting NODE_OPTIONS or PYTHONPATH on the container. The injector sets those variables in-process, only for processes that actually run the matching runtime. For Python, the injector detects the libc flavor (glibc/musl) of the application on its own, making the otel-python-platform annotation unnecessary. To support this, the init container copies both the glibc and musl installations into the shared volume, and the image gains a root-level sitecustomize.py symlink in each installation since the injector activates the instrumentation by prepending a single directory to PYTHONPATH. Claude-Session: https://claude.ai/code/session_01WdcmBbsVDxxQSPvpPvR2LQ Signed-off-by: Mikołaj Świątek <mail@mikolajswiatek.com> # Conflicts: # autoinstrumentation/nodejs/Dockerfile # autoinstrumentation/python/Dockerfile

…mentation Dockerfiles The injector version is pinned as the injector_version build ARG in the Java, .NET, Node.js and Python auto-instrumentation Dockerfiles. A regex custom manager tracks it against opentelemetry-injector GitHub releases, and all four Dockerfiles are grouped into a single update PR. Claude-Session: https://claude.ai/code/session_01WdcmBbsVDxxQSPvpPvR2LQ Signed-off-by: Mikołaj Świątek <mail@mikolajswiatek.com>

…ry-injector The new e2e-instrumentation-injector CI group runs the operator with the operator.autoinstrumentation.injector feature gate and the auto-instrumentation images built from the repository state, mirroring the e2e-instrumentation group. The tests are copies of the base Java, .NET, Node.js and Python suites with the pod-shape assertions adjusted for injector mode: the containers carry LD_PRELOAD and the injector agent path variables instead of JAVA_TOOL_OPTIONS/CORECLR_*/NODE_OPTIONS/PYTHONPATH. The telemetry validation steps are unchanged, which matters more in injector mode since activation happens in-process at runtime. The Python suite runs an Alpine-based app without the otel-python-platform annotation, so musl detection is exercised end-to-end. The instrumentation-python-oldest variant is not carried over because it pins an older instrumentation image that does not ship the injector. Claude-Session: https://claude.ai/code/session_01WdcmBbsVDxxQSPvpPvR2LQ Signed-off-by: Mikołaj Świątek <mail@mikolajswiatek.com>

The injector only publishes binaries for amd64 and arm64, while the Java, Node.js and Python auto-instrumentation images are also built for s390x and ppc64le. The operator cannot reliably determine the target architecture of a pod at admission time, so with the feature gate enabled, pods on unsupported architectures are not instrumented, and for Java and Python fail to start since their instrumentation init containers reference the missing injector library. Spell this out in the feature gate documentation and the changelog: the gate must not graduate before the injector covers all architectures the images are built for. Claude-Session: https://claude.ai/code/session_01WdcmBbsVDxxQSPvpPvR2LQ Signed-off-by: Mikołaj Świątek <mail@mikolajswiatek.com>

Move the injector version out of the per-language Dockerfile ARG defaults into a single autoinstrumentation/injector_version.txt, mirroring the version.txt convention used for the instrumentation agents. The Makefile and the publish workflows read the file and pass the value to the image builds as the injector_version build-arg; the Dockerfiles no longer carry a default. Renovate now updates just this one file, and the publish workflows trigger on it, so an injector bump republishes all four images. Claude-Session: https://claude.ai/code/session_01WdcmBbsVDxxQSPvpPvR2LQ Signed-off-by: Mikołaj Świątek <mail@mikolajswiatek.com>

Interpolate TARGETARCH into the release asset URL instead of duplicating the download instructions in per-architecture build stages, matching how the operator binary Dockerfiles select their artifacts. The .NET Dockerfile only builds for amd64 and arm64, so its per-architecture stages disappear entirely; the other Dockerfiles keep empty s390x and ppc64le fallback stages, with the per-arch stage names reduced to aliases of a single injector stage. Claude-Session: https://claude.ai/code/session_01WdcmBbsVDxxQSPvpPvR2LQ Signed-off-by: Mikołaj Świątek <mail@mikolajswiatek.com>

codecov · 2026-07-21T18:44:04Z

Codecov Report

❌ Patch coverage is 98.07692% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 63.28%. Comparing base (c0b22a2) to head (11a5d7f).
⚠️ Report is 4 commits behind head on main.
✅ All tests successful. No failed tests found.

Files with missing lines	Patch %	Lines
internal/instrumentation/javaagent.go	93.33%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #5347      +/-   ##
==========================================
+ Coverage   63.09%   63.28%   +0.18%     
==========================================
  Files         215      215              
  Lines       15304    15394      +90     
==========================================
+ Hits         9656     9742      +86     
- Misses       4968     4971       +3     
- Partials      680      681       +1

Flag	Coverage Δ
integration	`68.10% <ø> (ø)`
unittests	`63.28% <98.07%> (+0.18%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

mmanciop · 2026-07-21T20:29:01Z

+      value: always_on
+    - name: LD_PRELOAD
+      value: /otel-auto-instrumentation-dotnet/libotelinject.so
+    - name: DOTNET_AUTO_INSTRUMENTATION_AGENT_PATH_PREFIX


The injector supports setting this via files too, and I think it’d make the change less cluttered

But then I'd need to set an env variable for the config file location (since it has to be on a shared volume), so it's the same number of variables, but the image can more easily control it. I like that because it would make it easier for users to build their own images.

mmanciop

I have not tried the PR, just read the diff. Looks legit, but I think the biggest gap is documentation, especially in explaining why users should opt in to the Injector (in this PR is mostly resilience against modifications to JAVA_TOOL_OPTIONS and similar env vars in the entrypoint of the container).

swiatekm · 2026-07-22T11:19:02Z

I have not tried the PR, just read the diff. Looks legit, but I think the biggest gap is documentation, especially in explaining why users should opt in to the Injector (in this PR is mostly resilience against modifications to JAVA_TOOL_OPTIONS and similar env vars in the entrypoint of the container).

It's intended as a PoC, so it's deliberately not included. In all honesty though, this particular change is an implementation detail in my view. Users don't have to know, or care, and in an ideal world they won't even notice it happened, aside from some old bugs being resolved.

mmanciop · 2026-07-22T11:36:21Z

I have not tried the PR, just read the diff. Looks legit, but I think the biggest gap is documentation, especially in explaining why users should opt in to the Injector (in this PR is mostly resilience against modifications to JAVA_TOOL_OPTIONS and similar env vars in the entrypoint of the container).

It's intended as a PoC, so it's deliberately not included. In all honesty though, this particular change is an implementation detail in my view. Users don't have to know, or care, and in an ideal world they won't even notice it happened, aside from some old bugs being resolved.

I agree, it feels like the kind of feature gate that gets flipped on by default a few months down the line.

pavolloffay · 2026-07-23T10:19:32Z

By how much does it increase the image sizes?

swiatekm · 2026-07-23T17:24:09Z

By how much does it increase the image sizes?

~ 3MB uncompressed. To me, the main argument in favor of keeping the injector in a separate image is that it will make building custom images easier. If the injector comes separately, the only condition a custom image would need fulfill would be putting an injector-compatible artifact in the right path. If we bake the injector into our images, users building custom images would need to do it too.

swiatekm added 7 commits July 21, 2026 19:12

swiatekm changed the title ~~Use otel injector when a feature flag is set~~ Use otel injector Jul 21, 2026

mmanciop reviewed Jul 21, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Use otel injector - #5347

Use otel injector#5347
swiatekm wants to merge 8 commits into
open-telemetry:mainfrom
swiatekm:feat/injector

swiatekm commented Jul 21, 2026

Uh oh!

codecov Bot commented Jul 21, 2026 •

edited

Loading

Uh oh!

mmanciop Jul 21, 2026

Uh oh!

swiatekm Jul 22, 2026

Uh oh!

mmanciop left a comment •

edited

Loading

Uh oh!

swiatekm commented Jul 22, 2026

Uh oh!

mmanciop commented Jul 22, 2026

Uh oh!

pavolloffay commented Jul 23, 2026

Uh oh!

swiatekm commented Jul 23, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

swiatekm commented Jul 21, 2026

Uh oh!

codecov Bot commented Jul 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

mmanciop Jul 21, 2026

Choose a reason for hiding this comment

Uh oh!

swiatekm Jul 22, 2026

Choose a reason for hiding this comment

Uh oh!

mmanciop left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

swiatekm commented Jul 22, 2026

Uh oh!

mmanciop commented Jul 22, 2026

Uh oh!

pavolloffay commented Jul 23, 2026

Uh oh!

swiatekm commented Jul 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

codecov Bot commented Jul 21, 2026 •

edited

Loading

mmanciop left a comment •

edited

Loading

swiatekm commented Jul 23, 2026 •

edited

Loading