Push llm event address by estringana · Pull Request #3664 · DataDog/dd-trace-php

estringana · 2026-02-19T15:18:18Z

Description

This PR enables appsec capabilities when using openai-php client. The implementation push appsec addresses to the waf and then eventually they are reported to the backend.

More info on RFC: API Endpoints: AI usage

Reviewer checklist

Test coverage seems ok.
Appropriate labels assigned.

datadog-official · 2026-02-19T15:27:23Z

⚠️ Tests

✨ Fix all issues with BitsAI or with Cursor

⚠️ Warnings

🧪 1028 Tests failed

testSearchPhpBinaries from integration.DDTrace\Tests\Integration\PHPInstallerTest (Datadog) (Fix with Cursor)

Risky Test
phpvfscomposer://tests/vendor/phpunit/phpunit/phpunit:52

testSimplePushAndProcess from laravel-58-test.DDTrace\Tests\Integrations\Laravel\V5_8\QueueTest (Datadog) (Fix with Cursor)

DDTrace\Tests\Integrations\Laravel\V5_8\QueueTest::testSimplePushAndProcess
Test code or tested code printed unexpected output: spanLinksTraceId: 69a1ce380000000085e10a8419472e5f
tid: 69a1ce3800000000
hexProcessTraceId: 85e10a8419472e5f
hexProcessSpanId: a4090723264fe196
processTraceId: 9647003439280369247
processSpanId: 11819986544558596502

phpvfscomposer://tests/vendor/phpunit/phpunit/phpunit:106

testSimplePushAndProcess from laravel-8x-test.DDTrace\Tests\Integrations\Laravel\V8_x\QueueTest (Datadog) (Fix with Cursor)

DDTrace\Tests\Integrations\Laravel\V8_x\QueueTest::testSimplePushAndProcess
Test code or tested code printed unexpected output: spanLinksTraceId: 69a1ce9f0000000099f6eefc357654bf
tid: 69a1ce9f00000000
hexProcessTraceId: 99f6eefc357654bf
hexProcessSpanId: 6121f9218a7fdcbb
processTraceId: 11094317499069912255
processSpanId: 6999149218363333819

phpvfscomposer://tests/vendor/phpunit/phpunit/phpunit:106

View all

ℹ️ Info

❄️ No new flaky tests detected

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: e618728 | Docs | Datadog PR Page | Was this helpful? React with 👍/👎 or give us feedback!}

codecov-commenter · 2026-02-19T15:33:49Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 62.10%. Comparing base (a3409cd) to head (e618728).
⚠️ Report is 5 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #3664      +/-   ##
==========================================
- Coverage   62.22%   62.10%   -0.12%     
==========================================
  Files         141      141              
  Lines       13352    13352              
  Branches     1746     1746              
==========================================
- Hits         8308     8292      -16     
- Misses       4253     4266      +13     
- Partials      791      794       +3

see 3 files with indirect coverage changes

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a3409cd...e618728. Read the comment docs.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

pr-commenter · 2026-02-19T16:40:33Z

Benchmarks [ tracer ]

Benchmark execution time: 2026-02-27 18:12:24

Comparing candidate commit e618728 in PR branch estringana/add-openai-integration with baseline commit a3409cd in branch master.

Found 4 performance improvements and 27 performance regressions! Performance is the same for 162 metrics, 1 unstable metrics.

scenario:BM_TeaSapiSpindown

🟩 execution_time [-26.057µs; -13.177µs] or [-4.754%; -2.404%]

scenario:ComposerTelemetryBench/benchTelemetryParsing

🟥 mem_peak [+86.936KB; +86.936KB] or [+2.176%; +2.176%]
🟩 execution_time [-1.653µs; -0.747µs] or [-13.116%; -5.931%]

scenario:ContextPropagationBench/benchExtractHeaders128Bit

🟥 mem_peak [+98.648KB; +98.648KB] or [+2.469%; +2.469%]

scenario:ContextPropagationBench/benchExtractHeaders64Bit

🟥 mem_peak [+98.648KB; +98.648KB] or [+2.469%; +2.469%]

scenario:ContextPropagationBench/benchExtractTraceContext128Bit

🟥 mem_peak [+98.648KB; +98.648KB] or [+2.469%; +2.469%]

scenario:ContextPropagationBench/benchExtractTraceContext64Bit

🟥 mem_peak [+98.648KB; +98.648KB] or [+2.469%; +2.469%]

scenario:ContextPropagationBench/benchInject128Bit

🟥 mem_peak [+98.656KB; +98.656KB] or [+2.469%; +2.469%]

scenario:ContextPropagationBench/benchInject64Bit

🟥 mem_peak [+98.656KB; +98.656KB] or [+2.469%; +2.469%]

scenario:HookBench/benchHookOverheadInstallHookOnFunction

🟥 mem_peak [+98.520KB; +98.520KB] or [+2.466%; +2.466%]

scenario:HookBench/benchHookOverheadInstallHookOnMethod

🟥 mem_peak [+98.520KB; +98.520KB] or [+2.466%; +2.466%]

scenario:HookBench/benchHookOverheadTraceFunction

🟥 mem_peak [+107.360KB; +107.360KB] or [+2.417%; +2.417%]

scenario:HookBench/benchHookOverheadTraceMethod

🟥 mem_peak [+107.358KB; +107.361KB] or [+2.383%; +2.383%]

scenario:HookBench/benchWithoutHook

🟥 mem_peak [+98.536KB; +98.536KB] or [+2.467%; +2.467%]

scenario:MessagePackSerializationBench/benchMessagePackSerialization

🟥 mem_peak [+107.360KB; +107.360KB] or [+2.524%; +2.524%]
🟩 execution_time [-10.527µs; -7.813µs] or [-9.355%; -6.944%]

scenario:MessagePackSerializationBench/benchMessagePackSerialization-opcache

🟩 execution_time [-5.963µs; -4.417µs] or [-5.377%; -3.983%]

scenario:PDOBench/benchPDOBaseline

🟥 mem_peak [+107.360KB; +107.360KB] or [+2.662%; +2.662%]

scenario:PHPRedisBench/benchRedisBaseline

🟥 mem_peak [+97.776KB; +97.776KB] or [+2.447%; +2.447%]

scenario:SamplingRuleMatchingBench/benchGlobMatching1

🟥 mem_peak [+101.776KB; +101.776KB] or [+2.548%; +2.548%]

scenario:SamplingRuleMatchingBench/benchGlobMatching2

🟥 mem_peak [+101.776KB; +101.776KB] or [+2.548%; +2.548%]

scenario:SamplingRuleMatchingBench/benchGlobMatching3

🟥 mem_peak [+101.776KB; +101.776KB] or [+2.548%; +2.548%]

scenario:SamplingRuleMatchingBench/benchGlobMatching4

🟥 mem_peak [+101.776KB; +101.776KB] or [+2.548%; +2.548%]

scenario:SamplingRuleMatchingBench/benchRegexMatching1

🟥 mem_peak [+101.776KB; +101.776KB] or [+2.547%; +2.547%]

scenario:SamplingRuleMatchingBench/benchRegexMatching2

🟥 mem_peak [+101.776KB; +101.776KB] or [+2.547%; +2.547%]

scenario:SamplingRuleMatchingBench/benchRegexMatching3

🟥 mem_peak [+101.776KB; +101.776KB] or [+2.547%; +2.547%]

scenario:SamplingRuleMatchingBench/benchRegexMatching4

🟥 mem_peak [+101.776KB; +101.776KB] or [+2.547%; +2.547%]

scenario:SpanBench/benchDatadogAPI

🟥 mem_peak [+100.008KB; +100.008KB] or [+2.503%; +2.503%]

scenario:TraceAnnotationsBench/benchTraceAnnotationOverhead

🟥 mem_peak [+107.354KB; +107.361KB] or [+2.378%; +2.379%]

scenario:TraceFlushBench/benchFlushTrace

🟥 mem_peak [+107.360KB; +107.360KB] or [+2.633%; +2.633%]

scenario:TraceSerializationBench/benchSerializeTrace

🟥 mem_peak [+107.360KB; +107.360KB] or [+2.573%; +2.573%]

cataphract

I'd wait for a production rule to be ready (if it ain't already), update the recommended.json files and then also write an integration test. This would validate the correctness of the address and its parameters.

src/DDTrace/Integrations/OpenAI/OpenAIIntegration.php

estringana · 2026-02-24T12:28:29Z

I'd wait for a production rule to be ready (if it ain't already), update the recommended.json files and then also write an integration test. This would validate the correctness of the address and its parameters.

I have the rule https://github.com/DataDog/appsec-event-rules/pull/265 but it's not merged yet. Also we would need to mock the openai library http call. Do we have a system for that already on integration?
@cataphract

.../tests/integration/src/main/groovy/com/datadog/appsec/php/mock_agent/MockOpenAIServer.groovy

cataphract · 2026-02-25T22:27:54Z

...tests/integration/src/main/groovy/com/datadog/appsec/php/mock_openai/MockOpenAIServer.groovy

+    int getPort() {
+        PORT
+    }


same effect could be gotten in groovy by not making PORT private (with no access modifiers, it generates setters/getters)

appsec/tests/integration/src/main/groovy/com/datadog/appsec/php/docker/AppSecContainer.groovy

...c/tests/integration/src/test/groovy/com/datadog/appsec/php/integration/LlmEventsTests.groovy

appsec/tests/integration/src/test/www/llm/public/llm.php

cataphract · 2026-02-25T22:36:40Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3c92b6746e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

src/DDTrace/Integrations/OpenAI/OpenAIIntegration.php

cataphract

Seems better now, pending test success, but see some new comments

cataphract · 2026-02-27T16:52:16Z

src/DDTrace/Integrations/OpenAI/OpenAIIntegration.php

+            };
+        };
+
+        $handleRequestPrehook = fn ($streamed, $operationID, $reportApm, $reportAppsec) => function (\DDTrace\SpanData $span, $args) use ($operationID, $streamed, $reportApm, $reportAppsec, $pushAppsecEvent) {


this is php 7.4 syntax

cataphract · 2026-02-27T16:52:34Z

src/DDTrace/Integrations/OpenAI/OpenAIIntegration.php

        );

-        $handleRequestPrehook = fn ($streamed, $operationID, $reportApm, $reportAppsec) => function (\DDTrace\SpanData $span, $args) use ($operationID, $streamed, $reportApm, $reportAppsec) {
+        $pushAppsecEvent = static function (bool $reportAppsec, array $args): void {


could just be separate function since it doesn't capture any variable

cataphract · 2026-02-27T16:55:48Z

...c/tests/integration/src/test/groovy/com/datadog/appsec/php/integration/LlmEventsTests.groovy

+    }
+
+    @Container
+    public static final MockOpenAIServer mockOpenAIServer = new MockOpenAIServer()


don't think this should be annotated with @Container

estringana force-pushed the estringana/add-openai-integration branch 2 times, most recently from cff1eed to 1ae39aa Compare February 20, 2026 12:06

estringana marked this pull request as ready for review February 22, 2026 14:57

estringana requested review from a team as code owners February 22, 2026 14:57

estringana force-pushed the estringana/add-openai-integration branch from f363215 to 3fadba0 Compare February 23, 2026 09:55

estringana added 8 commits February 23, 2026 10:55

Push llm event address

0fe616d

Add missing operations

ddb5115

Add tests

c826cf7

Make appsec mock to work in memory

70dac6e

Move push address to prehook

0b41651

Create fake response

123295c

Update snapshots

b5e2c9a

Fix Mysql implementation of mock

6028af2

estringana force-pushed the estringana/add-openai-integration branch from 3fadba0 to 6028af2 Compare February 23, 2026 09:55

Fix addEvent signature

b64379c

estringana requested review from bwoebi and cataphract February 23, 2026 14:16

cataphract reviewed Feb 24, 2026

View reviewed changes

src/DDTrace/Integrations/OpenAI/OpenAIIntegration.php Show resolved Hide resolved

Explain the values of the array

86bde1a

Add integration tests for llm events

905ba6a

estringana requested a review from a team as a code owner February 25, 2026 16:57

estringana added 2 commits February 25, 2026 21:28

Fix port

0eb5a02

Make it compatible with php 7.0

3c92b67

cataphract requested changes Feb 25, 2026

View reviewed changes

chatgpt-codex-connector bot reviewed Feb 25, 2026

View reviewed changes

src/DDTrace/Integrations/OpenAI/OpenAIIntegration.php Outdated Show resolved Hide resolved

estringana added 2 commits February 27, 2026 12:33

Amend comments from PR

193aafa

Avoid creating not required span

277957f

estringana force-pushed the estringana/add-openai-integration branch from e713572 to 277957f Compare February 27, 2026 16:21

Fix tests

e618728

cataphract approved these changes Feb 27, 2026

View reviewed changes

Conversation

estringana commented Feb 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Reviewer checklist

Uh oh!

datadog-official bot commented Feb 19, 2026 • edited by datadog-datadog-prod-us1-2 bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ Warnings

ℹ️ Info

Uh oh!

codecov-commenter commented Feb 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

pr-commenter bot commented Feb 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks [ tracer ]

scenario:BM_TeaSapiSpindown

scenario:ComposerTelemetryBench/benchTelemetryParsing

scenario:ContextPropagationBench/benchExtractHeaders128Bit

scenario:ContextPropagationBench/benchExtractHeaders64Bit

scenario:ContextPropagationBench/benchExtractTraceContext128Bit

scenario:ContextPropagationBench/benchExtractTraceContext64Bit

scenario:ContextPropagationBench/benchInject128Bit

scenario:ContextPropagationBench/benchInject64Bit

scenario:HookBench/benchHookOverheadInstallHookOnFunction

scenario:HookBench/benchHookOverheadInstallHookOnMethod

scenario:HookBench/benchHookOverheadTraceFunction

scenario:HookBench/benchHookOverheadTraceMethod

scenario:HookBench/benchWithoutHook

scenario:MessagePackSerializationBench/benchMessagePackSerialization

scenario:MessagePackSerializationBench/benchMessagePackSerialization-opcache

scenario:PDOBench/benchPDOBaseline

scenario:PHPRedisBench/benchRedisBaseline

scenario:SamplingRuleMatchingBench/benchGlobMatching1

scenario:SamplingRuleMatchingBench/benchGlobMatching2

scenario:SamplingRuleMatchingBench/benchGlobMatching3

scenario:SamplingRuleMatchingBench/benchGlobMatching4

scenario:SamplingRuleMatchingBench/benchRegexMatching1

scenario:SamplingRuleMatchingBench/benchRegexMatching2

scenario:SamplingRuleMatchingBench/benchRegexMatching3

scenario:SamplingRuleMatchingBench/benchRegexMatching4

scenario:SpanBench/benchDatadogAPI

scenario:TraceAnnotationsBench/benchTraceAnnotationOverhead

scenario:TraceFlushBench/benchFlushTrace

scenario:TraceSerializationBench/benchSerializeTrace

Uh oh!

cataphract left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

estringana commented Feb 24, 2026

Uh oh!

Uh oh!

cataphract Feb 25, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cataphract commented Feb 25, 2026

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

cataphract left a comment

Choose a reason for hiding this comment

Uh oh!

cataphract Feb 27, 2026

Choose a reason for hiding this comment

Uh oh!

cataphract Feb 27, 2026

Choose a reason for hiding this comment

Uh oh!

cataphract Feb 27, 2026

estringana commented Feb 19, 2026 •

edited

Loading

datadog-official bot commented Feb 19, 2026 •

edited by datadog-datadog-prod-us1-2 bot

Loading

codecov-commenter commented Feb 19, 2026 •

edited

Loading

pr-commenter bot commented Feb 19, 2026 •

edited

Loading