Fix schema validation failure in new CSS obfuscation scenarios

[robertomonteromiguel]

Motivation

The dd-trace-js (and python, ruby, etc.) scheduled System Tests started failing on 2026-06-13 with Test_DdtraceSchemas::test_library schema errors on /v0.6/stats payloads:

• .../v0.6_stats.json: 'Service' is a required property on instance $
• .../v0.6_stats.json: 'IsTraceRoot' is a required property on instance $.Stats[0].Stats[0]
  This is the already-known, accepted gap APMSP-2158: several tracers don't emit the top-level Service field nor the per-stat IsTraceRoot field in their /v0.6/stats payloads. That gap is whitelisted in tests/schemas/test_schemas.py via a SchemaBug exclusion — but only for the appsec_blocking, trace_stats_computation, and tracing_config_nondefault_3 scenarios.

PR #6648 ("Add css obfuscation test APMSP-2764") introduced four new end-to-end scenarios that also generate /v0.6/stats payloads but were not added to the APMSP-2158 whitelist, so the previously-suppressed schema errors now fail across all affected tracers and weblogs.

Changes

Workflow

1. ⚠️ Create your PR as draft ⚠️
2. Work on you PR until the CI passes
3. Mark it as ready for review
   • Test logic is modified? -> Get a review from RFC owner.
   • Framework is modified, or non obvious usage of it -> get a review from R&P team

🚀 Once your PR is reviewed and the CI green, you can merge it!

🛟 #apm-shared-testing 🛟

Reviewer checklist

• Anything but tests/ or manifests/ is modified ? I have the approval from R&P team
• A docker base image is modified?
  • the relevant build-XXX-image label is present
• A scenario is added, removed or renamed?
  • Get a review from R&P team

[github-actions]

CODEOWNERS have been resolved as:

tests/schemas/test_schemas.py                                           @DataDog/system-tests-core

[datadog-prod-us1-5]

 

✨ Fix all issues with BitsAI

⚠️ Warnings

🚦 81 Pipeline jobs failed

Testing the test | System Tests (php, dev) / End-to-end #1 / apache-mod-7.0 1     

🧪 1 Test failed

tests.ffe.test_flag_eval_metrics.Test_FFE_Eval_Config_Exists_Flag_Missing.test_ffe_eval_config_exists_flag_missing[apache-mod-7.0] from system_tests_suite   (Fix with Cursor)

AssertionError: Expected metric for non-existent flag 'non-existent-eval-metric-flag', found none. All: []
assert 0 > 0
 +  where 0 = len([])

self = <tests.ffe.test_flag_eval_metrics.Test_FFE_Eval_Config_Exists_Flag_Missing object at 0x7f68cc5f8710>

    def test_ffe_eval_config_exists_flag_missing(self):
        """Test that missing flag (with config loaded) produces error.type=flag_not_found."""
        assert self.r.status_code == 200, f"Flag evaluation request failed: {self.r.text}"
    
...

Testing the test | System Tests (php, dev) / End-to-end #1 / apache-mod-7.0-zts 1     

🧪 2 Tests failed

tests.ffe.test_exposures.Test_FFE_Exposure_Events.test_ffe_multiple_remote_config_files[apache-mod-7.0-zts] from system_tests_suite   (Fix with Cursor)

AssertionError: Expected to find flags 'test-flag-1' or 'test-flag-2' in exposure events, found: set()
assert ('test-flag-1' in set() or 'test-flag-2' in set())
 +  where 'test-flag-1' = <tests.ffe.test_exposures.Test_FFE_Exposure_Events object at 0x7f3c38c075f0>.flag_1
 +  and   'test-flag-2' = <tests.ffe.test_exposures.Test_FFE_Exposure_Events object at 0x7f3c38c075f0>.flag_2

self = <tests.ffe.test_exposures.Test_FFE_Exposure_Events object at 0x7f3c38c075f0>

    def test_ffe_multiple_remote_config_files(self):
        """Test that FFE correctly handles multiple remote config files with different flags."""
        assert self.r1.status_code == 200, f"First flag evaluation failed: {self.r1.text}"
...

tests.ffe.test_flag_eval_metrics.Test_FFE_Eval_Config_Exists_Flag_Missing.test_ffe_eval_config_exists_flag_missing[apache-mod-7.0-zts] from system_tests_suite   (Fix with Cursor)

AssertionError: Expected metric for non-existent flag 'non-existent-eval-metric-flag', found none. All: []
assert 0 > 0
 +  where 0 = len([])

self = <tests.ffe.test_flag_eval_metrics.Test_FFE_Eval_Config_Exists_Flag_Missing object at 0x7f3c38c076e0>

    def test_ffe_eval_config_exists_flag_missing(self):
        """Test that missing flag (with config loaded) produces error.type=flag_not_found."""
        assert self.r.status_code == 200, f"Flag evaluation request failed: {self.r.text}"
    
...

Testing the test | System Tests (php, dev) / End-to-end #1 / apache-mod-7.1-zts 1     

🧪 1 Test failed

tests.ffe.test_flag_eval_metrics.Test_FFE_Eval_Config_Exists_Flag_Missing.test_ffe_eval_config_exists_flag_missing[apache-mod-7.1-zts] from system_tests_suite   (Fix with Cursor)

AssertionError: Expected metric for non-existent flag 'non-existent-eval-metric-flag', found none. All: []
assert 0 > 0
 +  where 0 = len([])

self = <tests.ffe.test_flag_eval_metrics.Test_FFE_Eval_Config_Exists_Flag_Missing object at 0x7fd3b8af2420>

    def test_ffe_eval_config_exists_flag_missing(self):
        """Test that missing flag (with config loaded) produces error.type=flag_not_found."""
        assert self.r.status_code == 200, f"Flag evaluation request failed: {self.r.text}"
    
...

View all 81 failed jobs.

ℹ️ Info

No other issues found (see more)

❄️ No new flaky tests detected

Useful? React with 👍 / 👎

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 68d6e34 | Docs | Datadog PR Page | Give us feedback!}

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 68d6e34ef1

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[chatgpt-codex-connector]

Narrow the stats schema exemption to APMSP-2158 paths

For these newly added scenarios, this SchemaBug still has data_path=None, and tests/schemas/utils/schemas_validators.py:27 documents that None treats every data path as a known bug; assert_no_schema_error therefore drops every /v0.6/stats schema violation for the listed libraries in these CSS obfuscation scenarios. Since APMSP-2158 is only the known missing stats fields, unrelated regressions in the stats payload shape for these scenarios will now pass silently; please add path-specific exemptions instead of putting the scenarios under the endpoint-wide exemption.

Useful? React with 👍 / 👎.