Automated code coverage reporting for system tests.#860
Open
ximon18 wants to merge 2 commits into
Open
Conversation
Adds +coverage which: - Creates a run specific tmp dir in /tmp/cascade-coverage-data/xxx. - Instruments cascade and cascaded using RUSTFLAGS="-C instrument-coverage". (see https://doc.rust-lang.org/beta/rustc/instrument-coverage.html) - Sets LLVM_PROFILE_FILE to control where instrumentation counter data is written to. Uses uuid and GITHUB_JOB to write data to unique paths per matrix and job run. Exports LLVM_PROFILE_FILE to the env of the job so that all invocations of cascade and cascaded pick it up. - Adds pkill cascaded to all system test jobs to ensure that coverage data gets written out before the test container is destroyed. - Generates a HTML report in /tmp/cascade-coverage-data/xxx/report.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds
+coveragetoact-wrapperwhich:https://doc.rust-lang.org/beta/rustc/instrument-coverage.html)
Screenshots
Live sample
A live example is temporarily available at http://cascade.nlnetlabs.nl/coverage/.
Things NOT currently covered by the system tests
NOTE: This list may be incomplete.
In addition to various unhappy flows, based on the sample report the following are not currently exercised by the system tests.
cascade zone resetcascade zone overridecascade zone approve|rejectof an unsigned zonecascade zone historycascade zone maintenancecascade zone statuswhen a zone is in any state other than idlecascade zone addwith import of a key file ending in.privatecascade zone addwith import of a KMIP keycascade zone addwith an IP source with portcascade debug change-loggingcascade hsm listcascade hsm getcascade hsm addwith client certificatescascade keyset ksk|zsk|csk|algorithm|removekey|getcascade policy list|showcascade statuscascade tsig addusing the x:y:z formPersister?SignedZonePatcher::unsigned_diff()ZonePatcher::next_patchset()accumulation of remaining changesCenter::add_zone()Not included
Code coverage metrics for
domain,dnstorcascade-hsm-bridge.Cargo.*,crates/,etc/,integration-tests/,src/):actthrough theact-wrapper(as described inTESTING.md)?