Skip to content

Comments

feat(telemetry): add app-extended-heartbeat parametric tests#6338

Draft
khanayan123 wants to merge 7 commits intomainfrom
ayan.khan/app-extended-heartbeat
Draft

feat(telemetry): add app-extended-heartbeat parametric tests#6338
khanayan123 wants to merge 7 commits intomainfrom
ayan.khan/app-extended-heartbeat

Conversation

@khanayan123
Copy link
Contributor

Add comprehensive parametric test suite for app-extended-heartbeat telemetry event across all SDK languages. Tests verify timing, payload structure, consistency, and compliance with API spec.

Tests added:

  • test_extended_heartbeat_emission: Verifies interval timing
  • test_extended_heartbeat_sequence: Validates multiple events
  • test_extended_heartbeat_payload_content: Checks required fields
  • test_extended_heartbeat_matches_app_started: Ensures consistency
  • test_extended_heartbeat_excludes_products_and_install_signature
  • test_extended_heartbeat_default_interval: Validates 24h default

All tests run parametrically across 9 SDK languages (Go, Java, .NET, C++, PHP, Ruby, Rust, Python, Node.js).

Motivation

Changes

Workflow

  1. ⚠️ Create your PR as draft ⚠️
  2. Work on you PR until the CI passes
  3. Mark it as ready for review
    • Test logic is modified? -> Get a review from RFC owner.
    • Framework is modified, or non obvious usage of it -> get a review from R&P team

🚀 Once your PR is reviewed and the CI green, you can merge it!

🛟 #apm-shared-testing 🛟

Reviewer checklist

  • Anything but tests/ or manifests/ is modified ? I have the approval from R&P team
  • A docker base image is modified?
    • the relevant build-XXX-image label is present
  • A scenario is added, removed or renamed?

Add comprehensive parametric test suite for app-extended-heartbeat
telemetry event across all SDK languages. Tests verify timing,
payload structure, consistency, and compliance with API spec.

Tests added:
- test_extended_heartbeat_emission: Verifies interval timing
- test_extended_heartbeat_sequence: Validates multiple events
- test_extended_heartbeat_payload_content: Checks required fields
- test_extended_heartbeat_matches_app_started: Ensures consistency
- test_extended_heartbeat_excludes_products_and_install_signature
- test_extended_heartbeat_default_interval: Validates 24h default

All tests run parametrically across 9 SDK languages (Go, Java, .NET,
C++, PHP, Ruby, Rust, Python, Node.js).

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
@github-actions
Copy link
Contributor

github-actions bot commented Feb 18, 2026

CODEOWNERS have been resolved as:

manifests/cpp.yml                                                       @DataDog/dd-trace-cpp
manifests/dotnet.yml                                                    @DataDog/apm-dotnet @DataDog/asm-dotnet
manifests/golang.yml                                                    @DataDog/dd-trace-go-guild
manifests/java.yml                                                      @DataDog/asm-java @DataDog/apm-java
manifests/nodejs.yml                                                    @DataDog/dd-trace-js
manifests/php.yml                                                       @DataDog/apm-php @DataDog/asm-php
manifests/python.yml                                                    @DataDog/apm-python @DataDog/asm-python
manifests/ruby.yml                                                      @DataDog/ruby-guild @DataDog/asm-ruby
pyproject.toml                                                          @DataDog/system-tests-core
tests/parametric/test_telemetry.py                                      @DataDog/system-tests-core @DataDog/apm-sdk-capabilities

khanayan123 and others added 6 commits February 18, 2026 14:45
Register comprehensive parametric tests for app-extended-heartbeat event
validation across all SDK languages. Tests verify:
- Periodic emission timing (24h default, configurable)
- Payload structure (config, deps, integrations)
- Consistency with app-started event
- Proper field exclusions (products, install_signature)

Language version markers:
- Go: v1.73.0-dev
- Java: v1.40.0
- Node.js: v5.0.0
- Python: v2.0.0
- Ruby: v2.1.0-dev
- C++: v0.2.0-dev
- PHP: v1.0.0-dev
- .NET: missing_feature (intentionally skipped - no forked runtime-id issues)

Tests use fast intervals (0.3-0.5s) for CI efficiency.

Test location: tests/parametric/test_telemetry.py::Test_ExtendedHeartbeat (lines 1256-1514)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Changed Test_ExtendedHeartbeat manifest entries from development
version strings (e.g., v1.73.0-dev, v2.1.0-dev) to missing_feature
to follow repository conventions.

Development versions should not be specified in manifests. Tests
for unreleased features should use missing_feature until the
feature is actually released in a production version.

Changes:
- cpp.yml: v0.2.0-dev → missing_feature
- golang.yml: v1.73.0-dev → missing_feature
- nodejs.yml: '>=5.0.0' → missing_feature
- php.yml: v1.0.0-dev → missing_feature
- ruby.yml: v2.1.0-dev → missing_feature

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
…tionality

Replace 6 complex test methods with a single simplified test that focuses
on the core requirement: comparing configurations across telemetry events.

The new test:
- Grabs app-started, app-extended-heartbeat, and app-client-configuration-change events
- Extracts configuration data from each
- Asserts configs match between app-started and app-extended-heartbeat
- Optionally validates config-change event configs match as well

This reduces test complexity from ~260 lines to ~40 lines while maintaining
coverage of the critical functionality.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Configure manifests to run app-extended-heartbeat tests only on Node.js
for initial validation:

- Enable Node.js: *ref_5_0_0 (>=5.0.0)
- Disable Python: v2.0.0 -> missing_feature
- Disable Java: v1.40.0 -> missing_feature
- Keep all others disabled: missing_feature (cpp, dotnet, php, golang, ruby)

This allows focused testing of Node.js implementation before enabling
other languages.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
- Add SLF001 exception for test_telemetry.py to allow _get_telemetry_event usage
- Fix nodejs.yml manifest to use '>=5.0.0' instead of non-existent anchor
- All linting checks now pass (mypy, ruff, yamlfmt, yamllint, shellcheck)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@datadog-official
Copy link

datadog-official bot commented Feb 24, 2026

⚠️ Tests

Fix all issues with BitsAI or with Cursor

⚠️ Warnings

🧪 2 Tests failed

tests.parametric.test_telemetry.Test_ExtendedHeartbeat.test_extended_heartbeat_config_matches[library_env0, parametric-nodejs] from system_tests_suite (Datadog) (Fix with Cursor)
AssertionError: app-extended-heartbeat event not found
assert None is not None

self = <tests.parametric.test_telemetry.Test_ExtendedHeartbeat object at 0x7fe4ec0595e0>
test_agent = <utils.docker_fixtures._test_agent.TestAgentAPI object at 0x7fe4ebce6cc0>
test_library = <utils.docker_fixtures._test_clients._test_client_parametric.ParametricTestClientApi object at 0x7fe4ebd021b0>

    @pytest.mark.parametrize(
        "library_env",
        [
...
tests.parametric.test_telemetry.Test_ExtendedHeartbeat.test_extended_heartbeat_config_matches[library_env0, parametric-rust] from system_tests_suite (Datadog) (Fix with Cursor)
AssertionError: app-extended-heartbeat event not found
assert None is not None

self = <tests.parametric.test_telemetry.Test_ExtendedHeartbeat object at 0x7f333a921460>
test_agent = <utils.docker_fixtures._test_agent.TestAgentAPI object at 0x7f330aa950a0>
test_library = <utils.docker_fixtures._test_clients._test_client_parametric.ParametricTestClientApi object at 0x7f330acf6150>

    @pytest.mark.parametrize(
        "library_env",
        [
...

ℹ️ Info

❄️ No new flaky tests detected

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: ee72e96 | Docs | Datadog PR Page | Was this helpful? Give us feedback!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant