Skip to content

Add Test_AnomalyDetectionTags for AI Guard span enrichment#6784

Open
smola wants to merge 1 commit intomainfrom
smola/anomaly-detection-tags
Open

Add Test_AnomalyDetectionTags for AI Guard span enrichment#6784
smola wants to merge 1 commit intomainfrom
smola/anomaly-detection-tags

Conversation

@smola
Copy link
Copy Markdown
Member

@smola smola commented Apr 22, 2026

Motivation

APPSEC-62448

Changes

Add a new test class that verifies anomaly detection attributes are propagated from the local root span into every AI Guard span with the ai_guard. prefix:

  • ai_guard.http.client_ip
  • ai_guard.network.client.ip
  • ai_guard.http.useragent
  • ai_guard.usr.id
  • ai_guard.session.id

Update the /ai_guard/evaluate endpoint in all weblogs to accept two new optional HTTP headers:

  • X-User-Id: sets usr.id on the local root span
  • X-Session-Id: sets session.id on the local root span

Workflow

  1. ⚠️ Create your PR as draft ⚠️
  2. Work on you PR until the CI passes
  3. Mark it as ready for review
    • Test logic is modified? -> Get a review from RFC owner.
    • Framework is modified, or non obvious usage of it -> get a review from R&P team

🚀 Once your PR is reviewed and the CI green, you can merge it!

🛟 #apm-shared-testing 🛟

Reviewer checklist

  • Anything but tests/ or manifests/ is modified ? I have the approval from R&P team
  • A docker base image is modified?
    • the relevant build-XXX-image label is present
  • A scenario is added, removed or renamed?

@smola
Copy link
Copy Markdown
Member Author

smola commented Apr 22, 2026

@codex review

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 22, 2026

CODEOWNERS have been resolved as:

docs/understand/weblogs/end-to-end_weblog.md                            @DataDog/system-tests-core
manifests/cpp_httpd.yml                                                 @DataDog/dd-trace-cpp
manifests/cpp_nginx.yml                                                 @DataDog/dd-trace-cpp
manifests/dotnet.yml                                                    @DataDog/apm-dotnet @DataDog/asm-dotnet
manifests/golang.yml                                                    @DataDog/dd-trace-go-guild
manifests/java.yml                                                      @DataDog/asm-java @DataDog/apm-java
manifests/nodejs.yml                                                    @DataDog/dd-trace-js
manifests/php.yml                                                       @DataDog/apm-php @DataDog/asm-php
manifests/python.yml                                                    @DataDog/apm-python @DataDog/asm-python
manifests/ruby.yml                                                      @DataDog/ruby-guild @DataDog/asm-ruby
tests/ai_guard/test_ai_guard_sdk.py                                     @DataDog/k9-ai-guard @DataDog/system-tests-core
utils/build/docker/java/spring-boot/src/main/java/com/datadoghq/system_tests/springboot/ai_guard/AIGuardController.java  @DataDog/apm-java @DataDog/asm-java @DataDog/system-tests-core
utils/build/docker/nodejs/express/app.js                                @DataDog/dd-trace-js @DataDog/system-tests-core
utils/build/docker/python/flask/app.py                                  @DataDog/apm-python @DataDog/asm-python @DataDog/system-tests-core
utils/build/docker/ruby/rails52/app/controllers/ai_guard_controller.rb  @DataDog/ruby-guild @DataDog/asm-ruby @DataDog/system-tests-core
utils/build/docker/ruby/rails61/app/controllers/ai_guard_controller.rb  @DataDog/ruby-guild @DataDog/asm-ruby @DataDog/system-tests-core
utils/build/docker/ruby/rails72/app/controllers/ai_guard_controller.rb  @DataDog/ruby-guild @DataDog/asm-ruby @DataDog/system-tests-core
utils/build/docker/ruby/rails80/app/controllers/ai_guard_controller.rb  @DataDog/ruby-guild @DataDog/asm-ruby @DataDog/system-tests-core

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5309eb6ed5

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread tests/ai_guard/test_ai_guard_sdk.py
@datadog-prod-us1-4
Copy link
Copy Markdown

datadog-prod-us1-4 Bot commented Apr 22, 2026

Tests

🎉 All green!

❄️ No new flaky tests detected
🧪 All tests passed

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: e81dfc6 | Docs | Datadog PR Page | Give us feedback!

Add a new test class that verifies anomaly detection attributes are propagated
from the local root span into every AI Guard span with the ai_guard. prefix:
  - ai_guard.http.client_ip
  - ai_guard.network.client.ip
  - ai_guard.http.useragent
  - ai_guard.usr.id
  - ai_guard.session.id

Update the /ai_guard/evaluate endpoint in all weblogs to accept two new
optional HTTP headers:
  - X-User-Id: sets usr.id on the local root span
  - X-Session-Id: sets session.id on the local root span

The two headers are handled independently. All languages use the local root
span API (not the active/current span) to ensure tags land on the right span:
  - Python: tracer.current_root_span()
  - Node.js: tracer.scope().active()?.context()._trace.started[0]
  - Java: ((MutableSpan) activeSpan).getLocalRootSpan()
  - Ruby: Datadog::Tracing.active_trace

The test is marked missing_feature (APPSEC-62448) across all tracers.
Manifests and endpoint docs updated accordingly.
@smola smola force-pushed the smola/anomaly-detection-tags branch from 5309eb6 to e81dfc6 Compare April 22, 2026 11:11
@smola smola marked this pull request as ready for review April 22, 2026 12:28
@smola smola requested review from a team as code owners April 22, 2026 12:28
@smola smola requested review from PerfectSlayer, brettlangdon, daniel-romano-DD, dubloom and jandro996 and removed request for a team April 22, 2026 12:28
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e81dfc6514

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment on lines +565 to +569
interfaces.library.validate_one_span(
self.r,
validator=self._assert_span(root_span=root_span),
full_trace=True,
)
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Validate anomaly tags on every AI Guard span

The new test claims to verify propagation to every ai_guard span, but it calls validate_one_span, which stops after the first matching span and leaves any additional ai_guard spans unchecked. If a trace contains multiple AI Guard spans (for example, after SDK/tracer changes), a regression on later spans would pass unnoticed, so this test does not enforce the behavior described by the class and method docs.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Contributor

@PerfectSlayer PerfectSlayer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved for the java part (java manifest + SB controller)

Copy link
Copy Markdown
Contributor

@obordeau obordeau left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Test looks good to me!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants