Skip to content

Comments

Add tests for new feature API Endpoints: Business logic events for LLM-based SDKs#6339

Draft
estringana wants to merge 10 commits intomainfrom
estringana/add-llm-tests
Draft

Add tests for new feature API Endpoints: Business logic events for LLM-based SDKs#6339
estringana wants to merge 10 commits intomainfrom
estringana/add-llm-tests

Conversation

@estringana
Copy link
Contributor

Motivation

This PR creates all required tests in order to fully test the new feature API Endpoints: Business logic events for LLM-based SDKs

You can find the RFC here RFC API Endpoints: Business logic events for LLM-based SDKs

Changes

Workflow

  1. ⚠️ Create your PR as draft ⚠️
  2. Work on you PR until the CI passes
  3. Mark it as ready for review
    • Test logic is modified? -> Get a review from RFC owner.
    • Framework is modified, or non obvious usage of it -> get a review from R&P team

🚀 Once your PR is reviewed and the CI green, you can merge it!

🛟 #apm-shared-testing 🛟

Reviewer checklist

  • Anything but tests/ or manifests/ is modified ? I have the approval from R&P team
  • A docker base image is modified?
    • the relevant build-XXX-image label is present
  • A scenario is added, removed or renamed?

@github-actions
Copy link
Contributor

github-actions bot commented Feb 18, 2026

CODEOWNERS have been resolved as:

tests/appsec/api_security/test_endpoints.py                             @DataDog/asm-libraries @DataDog/system-tests-core
utils/build/docker/php/common/llm.php                                   @DataDog/apm-php @DataDog/system-tests-core
docs/weblog/end-to-end_weblog.md                                        @DataDog/system-tests-core
manifests/cpp.yml                                                       @DataDog/dd-trace-cpp
manifests/cpp_httpd.yml                                                 @DataDog/dd-trace-cpp
manifests/cpp_nginx.yml                                                 @DataDog/dd-trace-cpp
manifests/dotnet.yml                                                    @DataDog/apm-dotnet @DataDog/asm-dotnet
manifests/envoy.yml                                                     @DataDog/system-tests-core
manifests/golang.yml                                                    @DataDog/dd-trace-go-guild
manifests/haproxy.yml                                                   @DataDog/system-tests-core
manifests/java.yml                                                      @DataDog/asm-java @DataDog/apm-java
manifests/java_otel.yml                                                 @DataDog/system-tests-core
manifests/nodejs.yml                                                    @DataDog/dd-trace-js
manifests/php.yml                                                       @DataDog/apm-php @DataDog/asm-php
manifests/python.yml                                                    @DataDog/apm-python @DataDog/asm-python
manifests/ruby.yml                                                      @DataDog/ruby-guild @DataDog/asm-ruby
manifests/rust.yml                                                      @DataDog/apm-rust
tests/appsec/rasp/rasp_ruleset.json                                     @DataDog/asm-libraries @DataDog/system-tests-core
utils/_context/_scenarios/appsec_rasp.py                                @DataDog/system-tests-core
utils/_features.py                                                      @DataDog/system-tests-core
utils/build/docker/internal_server/app.py                               @DataDog/system-tests-core
utils/build/docker/php/apache-mod/php.conf                              @DataDog/apm-php @DataDog/system-tests-core
utils/build/docker/php/common/composer.json                             @DataDog/apm-php @DataDog/system-tests-core
utils/build/docker/php/php-fpm/php-fpm.conf                             @DataDog/apm-php @DataDog/system-tests-core

@estringana estringana force-pushed the estringana/add-llm-tests branch from 0b79828 to 6772469 Compare February 18, 2026 16:21
}
}
},
{
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For now, LLM rules are not merged yet on appsec-event-rules repository. For that reason they are harcoded here but in the future they will be removed from here. You can see the PR where this rules have been copy from here https://github.com/DataDog/appsec-event-rules/pull/265

@rfc(
"https://docs.google.com/document/d/1TIFxbtbkldjOA6S5JFlCTMfqniXfJXZmDKptI5w2pnk/edit?tab=t.0#heading=h.xtljwwxyhqk7"
)
@scenarios.appsec_rasp
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It does not belong to RASP but rasp scenarios for now are the ones which have a way of injecting harcoded rules. Once llm rules are merged we can revisit the right scenario for these tests. More info on this comment https://github.com/DataDog/system-tests/pull/6339/changes#r2827405239

def api_llm_endpoint(test_object):
"""API Security - Business logic events for LLM-based SDKs

https://feature-parity.us1.prod.dog/#/?feature=543
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is 543 the right number? No sure where I need to get it from

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Close, but probably no : https://feature-parity.us1.prod.dog/#/?feature=543

I'm in Paris, we can meet to see how to play with the Feature Parity Dashboard

image

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll be in Paris tomorrow. Let's meet

@datadog-datadog-prod-us1
Copy link

datadog-datadog-prod-us1 bot commented Feb 19, 2026

✅ Tests

🎉 All green!

❄️ No new flaky tests detected
🧪 All tests passed

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 4a7f6a5 | Docs | Datadog PR Page | Was this helpful? Give us feedback!

@estringana estringana force-pushed the estringana/add-llm-tests branch from 8601851 to 7cb22da Compare February 19, 2026 15:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants