Skip to content

Add CIS AWS benchmark version preflight to aws-review#1710

Open
yui-stingray wants to merge 1 commit into
UnitOneAI:mainfrom
yui-stingray:improve/aws-review-cis-v5-preflight
Open

Add CIS AWS benchmark version preflight to aws-review#1710
yui-stingray wants to merge 1 commit into
UnitOneAI:mainfrom
yui-stingray:improve/aws-review-cis-v5-preflight

Conversation

@yui-stingray
Copy link
Copy Markdown

@yui-stingray yui-stingray commented Jun 8, 2026

Skill Improvement ($50-150 Bounty)

Skill Modified

Skill name: aws-review
Skill path: skills/cloud/aws-review/

What Was Wrong

The aws-review skill was fixed to CIS AWS Foundations Benchmark v3.0.0 in several places. That made current posture reports easy to overstate because the output format used v3.0.0 assumptions such as fixed section denominators and did not require benchmark version, Security Hub standard, or support-status evidence before scoring.

Addresses #213 as a scoped preflight/output improvement. This PR does not attempt a full CIS AWS v5.0.0 control-map rewrite.

What This PR Fixes

  • Adds a benchmark version preflight before control evaluation.
  • Records benchmark version, source, Security Hub standard ARN/version, legacy-baseline status, and denominator source.
  • Adds control support statuses for current, legacy, removed, unsupported, manual, and not evaluable controls.
  • Changes section scoring to use selected benchmark/control-family denominator sources instead of hard-coded v3 section counts.
  • Marks the existing v3.0.0 section map as a legacy baseline.
  • Updates README and index metadata so aws-review is no longer described as v3-only while avoiding a top-level claim of complete v5 control-map coverage.
  • Adds a focused ASFF-like test fixture for v5.0.0 Security Hub evidence, v3.0.0 legacy evidence, and missing-version evidence.

Evidence

Before (skill could overstate current coverage):

Framework: CIS Amazon Web Services Foundations Benchmark v3.0.0
Total CIS recommendations evaluated: <N>/62
Section Scores: X/22, X/10, X/11, X/16, X/6

After (now version-aware before scoring):

Framework: CIS Amazon Web Services Foundations Benchmark <selected version>
Benchmark source: <CIS / AWS Security Hub CSPM / supplied evidence>
Security Hub standard: <standard ARN/version or "not provided">
Legacy baseline: yes/no
Total CIS recommendations evaluated: <N>/<selected benchmark denominator or "source-specific">
Control support status: Current / Legacy / Removed / Unsupported / Manual / Not Evaluable

Reference sources used:

Test Cases Added/Updated

  • Added test cases: skills/cloud/aws-review/tests/cis-benchmark-version-preflight.md
  • Added vulnerable test cases (tests/vulnerable/) - N/A for this reporting/preflight guardrail
  • Added benign test cases (tests/benign/) - N/A for this reporting/preflight guardrail
  • Relevant static checks run locally

Validation run:

git diff --check
# required frontmatter check for changed aws-review SKILL.md
# prompt-injection scan logic from injection-scan.yml
# indexed-file existence check equivalent to validate-index.yml

Note: upstream GitHub Actions runs are currently waiting for maintainer approval for this fork PR (action_required) before jobs are created.

Bounty Tier

  • Minor ($50) — Doc update, small logic tweak, typo fix
  • Moderate ($100) — New edge case coverage, FP reduction with evidence
  • Substantial ($150) — Rewritten detection logic, major coverage expansion

Bounty Info

  • I have read and agree to the CONTRIBUTING.md bounty terms
  • Preferred payment method: To be provided privately after acceptance

@yui-stingray yui-stingray force-pushed the improve/aws-review-cis-v5-preflight branch from 5191cdf to 1fd4369 Compare June 8, 2026 01:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant