[REVIEW] dast-config: add API route coverage evidence gates

[yanziwei]

Skill Being Reviewed

Skill name: dast-config
Skill path: skills/devsecops/dast-config/

False Positive Analysis

Benign DAST result that can be over-credited as full API coverage:

zap_plan:
  jobs:
    - type: openapi
      parameters:
        apiFile: openapi.yaml
    - type: activeScan
      parameters:
        scanOnlyInScope: true

scan_summary:
  imported_operations: 42
  high_findings: 0
  status: pass

runtime_routes:
  total_routes: 64
  admin_routes_missing_from_spec: 6
  routes_blocked_by_auth_or_waf: 8
  route_request_log_available: false

Why this is a false positive:

The current skill checks that OpenAPI import exists and that the spec is current, but it does not require proof that the scanner imported and exercised each deployed route. A green ZAP openapi job can still miss runtime-only routes, admin routes hidden from the spec, operations rejected during import, routes skipped by auth/session state, or endpoints blocked by WAF/rate limits. Treating import success as route coverage can understate API attack surface risk.

Coverage Gaps

Missed variant 1: runtime routes missing from OpenAPI

Runtime route dump:
POST /api/admin/reindex
DELETE /api/users/{id}

OpenAPI spec:
GET /api/users/{id}
GET /api/orders/{id}

Why it should be caught: The DAST scan cannot generate tests for undocumented routes unless they are discovered by crawling or another route inventory. Sensitive/admin routes missing from the spec should be a coverage finding, not a passing API scan.

Missed variant 2: OpenAPI import succeeds partially

ZAP openapi job:
- imported: 37 operations
- failed: 5 operations
- parse warning: unsupported auth scheme for /api/billing/*

Why it should be caught: Partial import failures leave endpoints untested. The skill should require operation-level import logs and owners for failed/skipped operations.

Missed variant 3: route imported but never exercised

OpenAPI operation: POST /api/v1/orders/{id}/refund
Imported: yes
Request log / HAR evidence: none
Reason: test user lacks finance role

Why it should be caught: Import alone does not prove the scanner sent requests with the intended auth context. The review should compare spec, runtime inventory, import logs, and request logs.

Edge Cases

• Dangerous routes should remain excluded from active scanning, but they still need manual-test, passive-only, or intentionally-untested evidence.
• Multi-role APIs may require separate user contexts before route coverage is meaningful.
• WAF or rate-limit blocks can make a scan look clean while suppressing payload delivery.
• Generated OpenAPI files can be stale if not tied to the deployed build or commit.
• GraphQL has separate concerns, but REST route coverage still needs method/path evidence.

Remediation Quality

• Fix resolves the vulnerability
• Fix doesn't introduce new security issues
• Fix doesn't break functionality
• Issues found: Add API route coverage evidence gates for OpenAPI freshness, runtime route inventory, ZAP import logs, request-log/HAR evidence, auth role coverage, skipped-route reasons, and owner remediation.

Comparison to Other Tools

Tool	Catches this?	Notes
ZAP OpenAPI import	Partial	Imports operations, but reviewers must inspect failures and request logs.
Framework route dumps	Partial	Shows deployed routes, but not scanner coverage.
API gateways/service mesh exports	Partial	Useful runtime evidence, but must be reconciled to OpenAPI and scan logs.
Current dast-config skill	Partial	Mentions OpenAPI currency and endpoint inclusion, but lacks a route-level coverage matrix and request-log proof.

Overall Assessment

Strengths: Strong ZAP Automation Framework guidance, scan policy review, auth setup, CI/CD integration, scope controls, and DAST triage workflow.

Needs improvement: API scan coverage should be measured per route/operation, not inferred from the presence of an OpenAPI job. The skill should require a matrix that reconciles OpenAPI, runtime routes, import results, request logs, auth role, and gap owner.

Priority recommendations:

1. Add API route coverage evidence after OpenAPI import guidance.
2. Add a route coverage table to the output format.
3. Classify low route coverage and sensitive runtime-only routes as findings.
4. Add a pitfall warning against counting OpenAPI import as endpoint coverage.

Bounty Info

• I have read and agree to the CONTRIBUTING.md bounty terms
• Preferred payment method: Can provide privately after maintainer acceptance.

[rock2089]

I'd like to work on this bounty.

[yanziwei]

Submitted implementation PR: #1709

[heroking777]

Strategic Proposal for Enhancing API Route Coverage Evidence Gates

File: skills/devsecops/dast-config/

Root Cause Analysis:
The current implementation of the dast-config skill lacks comprehensive checks to ensure that all deployed routes are covered by the DAST scan. Specifically, it does not verify if each runtime route is imported and tested by the scanner, leading to false positives where partial or no coverage is reported as a pass.

Proposed Solution:
To address this issue, we propose adding evidence gates that require proof of API route coverage at the operation level. This involves:

1. Runtime Route Inventory: Ensure that the skill captures all deployed routes from the runtime environment.
2. ZAP Import Logs: Verify that each operation in the OpenAPI spec is successfully imported by ZAP.
3. Request Log Evidence: Confirm that requests were sent to each route with the appropriate authentication context.

Diff Summary:

• Add a step to generate and compare runtime routes against the OpenAPI spec.
• Integrate ZAP import logs to check for successful operation imports.
• Require HAR evidence for each tested route to prove request execution.

By implementing these changes, the skill will provide a more accurate assessment of API coverage, ensuring that all critical routes are properly tested. This approach maintains simplicity while significantly improving the reliability of the DAST scan results.

Next Steps:

1. Review and validate the proposed solution with stakeholders.
2. If approved, proceed to implement the evidence gates in a future PR.

This strategic enhancement will align the dast-config skill more closely with best practices for API security testing, ensuring that no critical routes are overlooked during the DAST process.