Skip to content

Automated code coverage reporting for system tests.#860

Open
ximon18 wants to merge 2 commits into
mainfrom
code-coverage-support
Open

Automated code coverage reporting for system tests.#860
ximon18 wants to merge 2 commits into
mainfrom
code-coverage-support

Conversation

@ximon18

@ximon18 ximon18 commented Jul 6, 2026

Copy link
Copy Markdown
Member

Adds +coverage to act-wrapper which:

  • Creates a run specific tmp dir in /tmp/cascade-coverage-data/xxx.
  • Instruments cascade and cascaded using RUSTFLAGS="-C instrument-coverage". (see
    https://doc.rust-lang.org/beta/rustc/instrument-coverage.html)
  • Sets LLVM_PROFILE_FILE to control where instrumentation counter data is written to. Uses uuid and GITHUB_JOB to write data to unique paths per matrix and job run. Exports LLVM_PROFILE_FILE to the env of the job so that all invocations of cascade and cascaded pick it up.
  • Adds pkill cascaded to all system test jobs to ensure that coverage data gets written out before the test container is destroyed.
  • Generates a HTML report in /tmp/cascade-coverage-data/xxx/report.

Screenshots

image image

Live sample

A live example is temporarily available at http://cascade.nlnetlabs.nl/coverage/.

Things NOT currently covered by the system tests

NOTE: This list may be incomplete.

In addition to various unhappy flows, based on the sample report the following are not currently exercised by the system tests.

  • cascade zone reset
  • cascade zone override
  • cascade zone approve|reject of an unsigned zone
  • cascade zone history
  • cascade zone maintenance
  • cascade zone status when a zone is in any state other than idle
  • cascade zone add with import of a key file ending in .private
  • cascade zone add with import of a KMIP key
  • cascade zone add with an IP source with port
  • cascade debug change-logging
  • cascade hsm list
  • cascade hsm get
  • cascade hsm add with client certificates
  • cascade keyset ksk|zsk|csk|algorithm|removekey|get
  • cascade policy list|show
  • cascade status
  • cascade tsig add using the x:y:z form
  • Config file settings such as log level, daemon identity, pid file, complex name sever specifications
  • Systemd socket activation supplied sockets
  • Reading of zone contents via Persister?
  • SignedZonePatcher::unsigned_diff()
  • ZonePatcher::next_patchset() accumulation of remaining changes
  • Unwind after failure in Center::add_zone()
  • SOA timer based zone refresh
  • Querying Prometheus metrics
  • Receiving malformed XFR
  • Key generation with algorithms other than ECDSAP256SHA256
  • Access control of downstream XFR
  • IXFR from review servers
  • Aspects of incremental signing
  • Aspects of signing queue behaviour
  • TSIG state restore
  • TSIG algorithms other than HmacSha256
  • Not exposed functionality to generate a TSIG key
  • HTTP APIs relating to the functionality mentioned above
  • Serial policy modes

Not included

Code coverage metrics for domain, dnst or cascade-hsm-bridge.


  • If you are changing Rust code or integration tests (Cargo.*, crates/, etc/, integration-tests/, src/):
    • Did you run the integration tests with act through the act-wrapper (as described in TESTING.md)?

Adds +coverage which:
- Creates a run specific tmp dir in /tmp/cascade-coverage-data/xxx.
- Instruments cascade and cascaded using RUSTFLAGS="-C
instrument-coverage". (see
https://doc.rust-lang.org/beta/rustc/instrument-coverage.html)
- Sets LLVM_PROFILE_FILE to control where instrumentation counter data
is written to. Uses uuid and GITHUB_JOB to write data to unique paths
per matrix and job run. Exports LLVM_PROFILE_FILE to the env of the job
so that all invocations of cascade and cascaded pick it up.
- Adds pkill cascaded to all system test jobs to ensure that coverage
data gets written out before the test container is destroyed.
- Generates a HTML report in /tmp/cascade-coverage-data/xxx/report.
@ximon18 ximon18 added the testing Automatic verification of Cascade's correctness and reliability. label Jul 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

testing Automatic verification of Cascade's correctness and reliability.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant