Add helper rewrite in rust by cataphract · Pull Request #3581 · DataDog/dd-trace-php

cataphract · 2026-01-16T16:47:40Z

Description

Passing integration and system-tests.

Further integration into sidecar and protocol changes pending.

Reviewer checklist

Test coverage seems ok.
Appropriate labels assigned.

datadog-datadog-prod-us1 · 2026-01-16T17:12:41Z

⚠️ Tests

✨ Fix all issues with BitsAI or with Cursor

⚠️ Warnings

🧪 1028 Tests failed

testSearchPhpBinaries from integration.DDTrace\Tests\Integration\PHPInstallerTest (Datadog) (Fix with Cursor)
Risky Test
phpvfscomposer://tests/vendor/phpunit/phpunit/phpunit:97
testSimplePushAndProcess from laravel-58-test.DDTrace\Tests\Integrations\Laravel\V5_8\QueueTest (Datadog) (Fix with Cursor)
DDTrace\Tests\Integrations\Laravel\V5_8\QueueTest::testSimplePushAndProcess
Test code or tested code printed unexpected output: spanLinksTraceId: 69a1f7ed00000000618577efe3dbe452
tid: 69a1f7ed00000000
hexProcessTraceId: 618577efe3dbe452
hexProcessSpanId: 516738592b57b2c3
processTraceId: 7027154665785254994
processSpanId: 5865718995303772867

phpvfscomposer://tests/vendor/phpunit/phpunit/phpunit:106
testSimplePushAndProcess from laravel-8x-test.DDTrace\Tests\Integrations\Laravel\V8_x\QueueTest (Datadog) (Fix with Cursor)
DDTrace\Tests\Integrations\Laravel\V8_x\QueueTest::testSimplePushAndProcess
Test code or tested code printed unexpected output: spanLinksTraceId: 69a1f80c00000000863338c77b573cad
tid: 69a1f80c00000000
hexProcessTraceId: 863338c77b573cad
hexProcessSpanId: aa85239d2d50c65a
processTraceId: 9670135254313548973
processSpanId: 12287266316327372378
View all

ℹ️ Info

❄️ No new flaky tests detected

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: edf9007 | Docs | Datadog PR Page | Was this helpful? React with 👍/👎 or give us feedback!}

codecov-commenter · 2026-01-16T17:16:27Z

Codecov Report

❌ Patch coverage is 84.83167% with 820 lines in your changes missing coverage. Please review.
✅ Project coverage is 68.62%. Comparing base (f785afe) to head (edf9007).
⚠️ Report is 7 commits behind head on master.

Files with missing lines	Patch %	Lines
appsec/helper-rust/src/client.rs	89.58%	97 Missing ⚠️
appsec/helper-rust/src/lib.rs	53.47%	87 Missing ⚠️
appsec/helper-rust/src/rc.rs	85.71%	68 Missing ⚠️
appsec/src/extension/helper_process.c	6.55%	57 Missing ⚠️
appsec/helper-rust/src/telemetry/sidecar.rs	79.10%	56 Missing ⚠️
appsec/helper-rust/src/service.rs	91.01%	54 Missing ⚠️
appsec/src/extension/ddappsec.c	16.12%	48 Missing and 4 partials ⚠️
appsec/helper-rust/src/lock.rs	68.37%	37 Missing ⚠️
appsec/helper-rust/src/service/sampler.rs	90.76%	34 Missing ⚠️
appsec/helper-rust/src/service/waf_diag.rs	87.96%	29 Missing ⚠️
... and 19 more

❌ Your patch status has failed because the patch coverage (84.83%) is below the target coverage (90.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #3581      +/-   ##
==========================================
+ Coverage   62.21%   68.62%   +6.40%     
==========================================
  Files         141      165      +24     
  Lines       13388    18766    +5378     
  Branches     1753     1772      +19     
==========================================
+ Hits         8330    12879    +4549     
- Misses       4260     5082     +822     
- Partials      798      805       +7

Flag	Coverage Δ
helper-rust-integration	`78.71% <78.71%> (?)`
helper-rust-unit	`49.67% <49.67%> (?)`

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
appsec/helper-rust/src/service/metrics.rs	`100.00% <100.00%> (ø)`
appsec/src/extension/configuration.h	`100.00% <ø> (ø)`
appsec/src/extension/user_tracking.c	`75.00% <100.00%> (+0.08%)`	⬆️
appsec/src/helper/client.cpp	`75.46% <100.00%> (+0.05%)`	⬆️
appsec/src/helper/client.hpp	`89.58% <ø> (ø)`
appsec/src/helper/engine.cpp	`91.73% <100.00%> (ø)`
appsec/src/helper/engine.hpp	`100.00% <ø> (ø)`
appsec/src/helper/network/proto.hpp	`93.22% <ø> (ø)`
appsec/src/helper/subscriber/base.hpp	`100.00% <ø> (ø)`
appsec/src/helper/subscriber/waf.cpp	`71.57% <100.00%> (+0.04%)`	⬆️
... and 30 more

... and 2 files with indirect coverage changes

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f785afe...edf9007. Read the comment docs.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

pr-commenter · 2026-01-16T18:10:02Z

Benchmarks [ appsec ]

Benchmark execution time: 2026-02-27 20:30:46

Comparing candidate commit edf9007 in PR branch glopes/helper-rust with baseline commit f785afe in branch master.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

pr-commenter · 2026-01-16T18:40:27Z

Benchmarks [ tracer ]

Benchmark execution time: 2026-02-27 21:09:04

Comparing candidate commit edf9007 in PR branch glopes/helper-rust with baseline commit f785afe in branch master.

Found 2 performance improvements and 4 performance regressions! Performance is the same for 184 metrics, 4 unstable metrics.

scenario:MessagePackSerializationBench/benchMessagePackSerialization

🟩 execution_time [-7.487µs; -5.533µs] or [-6.804%; -5.028%]

scenario:MessagePackSerializationBench/benchMessagePackSerialization-opcache

🟩 execution_time [-7.220µs; -6.320µs] or [-6.468%; -5.662%]

scenario:SamplingRuleMatchingBench/benchRegexMatching1

🟥 execution_time [+81.110ns; +117.090ns] or [+7.027%; +10.144%]

scenario:SamplingRuleMatchingBench/benchRegexMatching2

🟥 execution_time [+98.091ns; +167.109ns] or [+8.448%; +14.392%]

scenario:SamplingRuleMatchingBench/benchRegexMatching3

🟥 execution_time [+99.071ns; +131.129ns] or [+8.569%; +11.341%]

scenario:SamplingRuleMatchingBench/benchRegexMatching4

🟥 execution_time [+91.376ns; +132.824ns] or [+7.938%; +11.539%]

morrisonlevi · 2026-01-18T05:39:56Z

This PR is so large that github will only permit me to review it one file at a time. I didn't even know that was a thing! You're going to need to break it down into a series of smaller PRs, probably.

bwoebi · 2026-01-20T12:47:21Z

@morrisonlevi I've had success for very big PRs with the PHPStorm/CLion github integrations in the past. Doesn't matter for small PRs, but can definitely recommend it for extra-large PRs :-)

Changes the request_exec message format from [rasp_rule, data] to [data, options_map]. The options map supports: - rasp_rule: string (same as before) - subctx_id: optional string (accepted but ignored) - subctx_last_call: optional bool (accepted but ignored) This prepares the protocol for curl/subcontext support while maintaining backwards compatibility. The subcontext fields are accepted by the protocol but not implemented in the business logic. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

- Upgrade testcontainers - Add some assertions - Better debug output for metrics

…elper-rust

Adds helper_runtime identification to distinguish Rust helper from C++ helper in telemetry and tracing: Protocol changes: - Both helpers send runtime type ("rust" or "cpp") in client_init response as 6th field - Extension extracts and stores helper runtime in thread-local storage Span metadata: - Tag `_dd.appsec.helper_runtime:rust` added to ALL spans when using Rust helper - Tag added only for Rust helper; C++ helper does not add this span tag - Tags added via dd_tags_add_tags() in tags.c Telemetry: - Extension adds `helper_runtime:rust` tag to appsec telemetry metrics (only for Rust) - Rust helper adds `helper_runtime:rust` tag to all telemetry (metrics and logs) downstream in submitters Tests: - Updated protocol tests to expect 6th field in client_init response - Updated integration tests to conditionally check for helper_runtime tag - Updated mock helper to send correct protocol format - Removed test cases from request_exec.phpt that pass non-array values to request_exec() 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

Update test expectations for the new client_init response format: - The helper_runtime field is now a dedicated optional field in the response structure, not part of the meta map - BrokerTest.SendClientInit: pack 6 elements including helper_runtime - ClientTest.ClientInit/ClientInitInvalidRules: check helper_runtime field instead of meta["helper_runtime"], adjust meta.size() to 2 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

Add CI job to build the Rust appsec helper and include it alongside the C++ helper in the final packages. This enables distribution of both helper implementations. Changes: - Add build-appsec-helper-rust.sh script for building the Rust helper - Add "compile appsec helper rust" CI job in generate-package.php - Update generate-final-artifact.sh to include libddappsec-helper-rust.so - Update generate-ssi-package.sh to include libddappsec-helper-rust.so

Add a new configuration setting DD_APPSEC_HELPER_RUST_REDIRECTION that enables automatic redirection to the Rust helper if available. When enabled (and DD_APPSEC_HELPER_RUST_REDIRECTION=true), the extension will look for libddappsec-helper-rust.so in the same directory as the configured DD_APPSEC_HELPER_PATH. If found, the Rust helper is used instead of the C++ helper. The setting defaults to: - true on PHP 8.5+ - false on earlier PHP versions This allows gradual migration to the Rust helper while maintaining backward compatibility.

…helper-rust-redirection

Compile the test fixture library via build.rs using the target's C compiler (cc crate), rather than invoking rustc at test runtime. The runtime rustc approach produced a binary with dependencies incompatible with musl's dynamic linker, causing dlopen to fail with "No such file or directory" even though the .so was present. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

The function searched /proc/self/maps for "/libddappsec-helper.so" but the Rust helper loads as "libddappsec-helper-rust.so". The suffix "-rust" before ".so" means the old substring never matched, so the fallback path resolution failed causing client_init to return an error. Match "/libddappsec-helper" as a prefix so both the C++ and Rust variants are found. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

cataphract requested review from a team as code owners January 16, 2026 16:47

cataphract marked this pull request as draft January 16, 2026 16:47

cataphract force-pushed the glopes/helper-rust branch from f4914c3 to 2fc2654 Compare January 16, 2026 17:33

cataphract force-pushed the glopes/appsec-curl branch from bbc593e to 192a4a1 Compare January 16, 2026 17:33

cataphract force-pushed the glopes/helper-rust branch 9 times, most recently from fb4432d to 8d1029e Compare January 18, 2026 03:49

cataphract force-pushed the glopes/helper-rust branch 2 times, most recently from a218cd6 to 332fd93 Compare January 20, 2026 12:39

cataphract force-pushed the glopes/helper-rust branch from fc9100c to 718f800 Compare January 29, 2026 12:58

cataphract changed the base branch from glopes/appsec-curl to master January 29, 2026 12:58

cataphract force-pushed the glopes/helper-rust branch from d68a588 to 5d247e2 Compare February 10, 2026 17:42

cataphract and others added 3 commits February 10, 2026 17:47

Improve appsec integration tests

fc86ff8

- Upgrade testcontainers - Add some assertions - Better debug output for metrics

Move helper-rust to dd-trace-php

be7efa4

Merge commit '21d2f00064f089eb451df371b58fb056fd348c4c' into glopes/h…

57bcc57

…elper-rust

cataphract force-pushed the glopes/helper-rust branch from 5d247e2 to 57bcc57 Compare February 10, 2026 17:49

cataphract and others added 4 commits February 11, 2026 14:27

helper-rust: wait for sidecar to be read b4 submitting telemetry

a645d2e

Statically build libddwaf into helper-rust

a13bfb9

Universal rust helper build. Int test in alpine too

0d242c6

cataphract force-pushed the glopes/helper-rust branch from 62aaf55 to 0d242c6 Compare February 20, 2026 14:05

Merge remote-tracking branch 'origin/master' into glopes/helper-rust

dd440cb

cataphract force-pushed the glopes/helper-rust branch from d4571c9 to dd440cb Compare February 20, 2026 16:03

cataphract and others added 7 commits February 20, 2026 16:29

Fix build

e3e03ed

Merge remote-tracking branch 'origin/glopes/helper-rust' into glopes/…

a739f58

…helper-rust-redirection

appsec: format_fix

cfa0cbb

helper-rust: fix log message origin

30bab4c

cataphract force-pushed the glopes/helper-rust branch 2 times, most recently from 719e83e to c100434 Compare February 24, 2026 10:19

Have helper-rust report telemetry log for failed product rc update

67c7672

cataphract force-pushed the glopes/helper-rust branch from 46b5f2f to ec93c63 Compare February 24, 2026 16:17

Misc fixes

bb60003

cataphract force-pushed the glopes/helper-rust branch from ec93c63 to bb60003 Compare February 24, 2026 16:41

Fix build (GIT_SUBMODULE_PATHS)

d06640c

cataphract force-pushed the glopes/helper-rust branch from 0385bb7 to d06640c Compare February 27, 2026 17:34

cataphract and others added 3 commits February 27, 2026 17:49

Try to fix test in helper-rust/src/ffi.rs

493aead

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add helper rewrite in rust#3581

Add helper rewrite in rust#3581
cataphract wants to merge 22 commits intomasterfrom
glopes/helper-rust

cataphract commented Jan 16, 2026 •

edited

Loading

Uh oh!

datadog-datadog-prod-us1 bot commented Jan 16, 2026 •

edited by datadog-datadog-prod-us1-2 bot

Loading

Uh oh!

codecov-commenter commented Jan 16, 2026 •

edited

Loading

Uh oh!

pr-commenter bot commented Jan 16, 2026 •

edited

Loading

Uh oh!

pr-commenter bot commented Jan 16, 2026 •

edited

Loading

Uh oh!

morrisonlevi commented Jan 18, 2026 •

edited

Loading

Uh oh!

bwoebi commented Jan 20, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

cataphract commented Jan 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Reviewer checklist

Uh oh!

datadog-datadog-prod-us1 bot commented Jan 16, 2026 • edited by datadog-datadog-prod-us1-2 bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ Warnings

ℹ️ Info

Uh oh!

codecov-commenter commented Jan 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

pr-commenter bot commented Jan 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks [ appsec ]

Uh oh!

pr-commenter bot commented Jan 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks [ tracer ]

scenario:MessagePackSerializationBench/benchMessagePackSerialization

scenario:MessagePackSerializationBench/benchMessagePackSerialization-opcache

scenario:SamplingRuleMatchingBench/benchRegexMatching1

scenario:SamplingRuleMatchingBench/benchRegexMatching2

scenario:SamplingRuleMatchingBench/benchRegexMatching3

scenario:SamplingRuleMatchingBench/benchRegexMatching4

Uh oh!

morrisonlevi commented Jan 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bwoebi commented Jan 20, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

cataphract commented Jan 16, 2026 •

edited

Loading

datadog-datadog-prod-us1 bot commented Jan 16, 2026 •

edited by datadog-datadog-prod-us1-2 bot

Loading

codecov-commenter commented Jan 16, 2026 •

edited

Loading

pr-commenter bot commented Jan 16, 2026 •

edited

Loading

pr-commenter bot commented Jan 16, 2026 •

edited

Loading

morrisonlevi commented Jan 18, 2026 •

edited

Loading