Add Garak (NVIDIA LLM vulnerability scanner) parser#15013
Conversation
Adds a "Garak Scan" parser that ingests garak's JSON Lines hit log (garak.<run_id>.hitlog.jsonl). Each detector hit becomes a Finding; hits for the same probe/target/detector are aggregated (nb_occurences, keeping the most severe rung). - Severity derived from the detector score and adjusted by probe family - Probe family mapped to CWE (1427 prompt-injection, 79 xss, 200 leak, 1426 default = Improper Validation of Generative AI Output) - Prompt/output text extracted from garak's nested Conversation/Message - hash_code deduplication on title/severity/component_name (settings.dist.py) Includes unit tests (0/1/many, severity matrix, aggregation escalation, nested extraction, bytes/BOM/unicode, invalid input) and documentation. Signed-off-by: David Dashti <dashti.dat@gmail.com>
|
This implements the garak parser requested in #14878. @mcdonaid1379 — you'd mentioned wanting to take it on; I had a working implementation ready, so opened this. Happy to coordinate if you'd already started, and of course defer to the maintainers on direction. A note for reviewers: I couldn't run the full DojoTestCase suite locally (a network issue pulling the test-DB image), so the unit tests here are pending the first-time-contributor CI approval — keen to see them run and will fix anything that surfaces. |
| if SEVERITY_LADDER.index(severity) > SEVERITY_LADDER.index(finding.severity): | ||
| finding.severity = severity |
There was a problem hiding this comment.
Does this mean the severity could change over time when it finds more or less occurences in certain "probe". If yes, this could affect deduplication in its current config. Do we really need severity in the hash code config? It rarely is useful and rarely stable enough to be reliable.
Description
Adds a parser for garak, NVIDIA's LLM vulnerability scanner, implementing the request in #14878 (and aligned with the AI-testing direction in discussion #13242).
The parser ingests garak's JSON Lines hit log (
garak.<run_id>.hitlog.jsonl). Every line in a hit log is a detector hit, so each record becomes a Finding; hits for the same probe/target/detector are aggregated into one Finding (nb_occurences, keeping the most severe rung seen).score(0–1) and adjusted by probe family — active-attack/jailbreak families (e.g.promptinject,dan,malwaregen,xss) nudge up; content/quality families (e.g.continuation,misleading) nudge down. Note: many garak detectors emit a binary1.0, so in practice the probe-family adjustment does most of the differentiating. Happy to tune this to maintainer preference.Conversation/Messagestructures (pergarak/evaluators/base.py), with fallbacks for plain-string payloads.hash_codededuplication ontitle+severity+component_name.descriptionis intentionally excluded because garak samples the prompt/output non-deterministically per run, so including it would prevent the same weakness from deduplicating across repeated scans.Test results
Adds
unittests/tools/test_garak_parser.pycovering: zero/one/many findings, the full score→severity matrix, the CWE mapping, aggregation severity-escalation (and that a later lower-scored hit does not downgrade), nested prompt/output extraction, bytes + UTF-8 BOM + non-ASCII input, and rejection of non-JSONL input. Sample hit logs are underunittests/scans/garak/.Marked as a draft so CI can validate the full
DojoTestCasesuite before review.Documentation
Adds
docs/content/supported_tools/parsers/file/garak.md.Checklist
devruff.toml)Import Scansandsettings_changes(touchessettings.dist.pyfor deduplication)Closes #14878