Skip to content

Add OWASP Top 10 for LLM assessment coverage#53

Open
agasthik wants to merge 3 commits into
aws-samples:mainfrom
agasthik:feat-owasp-top-llm
Open

Add OWASP Top 10 for LLM assessment coverage#53
agasthik wants to merge 3 commits into
aws-samples:mainfrom
agasthik:feat-owasp-top-llm

Conversation

@agasthik

Copy link
Copy Markdown
Contributor

Closes #52

Summary

  • Adds an optional OWASP Top 10 for LLM (2025) assessment module that emits findings OW-01OW-12 and renders them in the HTML report under a new "By Compliance Standard" section.
  • Reuses the existing Bedrock / SageMaker / AgentCore / FinServ CSVs: OW-01…OW-10 are re-mapped from those source findings via a centralised OWASP_CHECK_MAPPINGS in the OWASP Lambda; OW-11 and OW-12 are native LLM07 checks (Lambda env-var inspection for embedded system prompts, Bedrock Guardrail DENY-topic inspection for system-prompt disclosure).
  • Enabling OWASP force-runs FinServ so the FS-* rows feed the OWASP mappings even when EnableFinServAssessment=false; in that case FS-* rows are consumed silently and hidden from the report UI.
  • New EnableOWASPAssessment CFN parameter on template.yaml, template-multi-account.yaml, and deployment/aiml-security-single-account.yaml / deployment/2-aiml-security-codebuild.yaml, threaded through buildspec.yml → Step Functions execution input → OWASP Choice branch in statemachine/assessments.asl.json.
  • Introduces a COMPLIANCE_STANDARDS registry in report_template.py so future compliance standards (NIST AI RMF, EU AI Act) can be added as data-only changes.

Design references

Full OW-## → LLM-## mapping and per-check derivation are documented in docs/SECURITY_CHECKS_OWASP.md.

Test plan

  • .venv/bin/python -m pytest tests/ -v --tb=short — core + report tests (includes new test_owasp_checks.py, test_report_template_owasp.py, and test_consolidate_finserv.py::test_fs_rows_hidden_when_enable_finserv_false).
  • .venv/bin/python -m pytest aiml-security-assessment/functions/security/finserv_tests/ -v --tb=short — FinServ suite unaffected.
  • (cd aiml-security-assessment/functions/security/generate_consolidated_report && ../../../../.venv/bin/python -m pytest test_generate_report.py -v --tb=short) — report pipeline.
  • .venv/bin/ruff check aiml-security-assessment/functions/security/ and .venv/bin/ruff format --check aiml-security-assessment/functions/security/ clean.
  • .venv/bin/cfn-lint deployment/*.yaml aiml-security-assessment/template.yaml aiml-security-assessment/template-multi-account.yaml clean.
  • sam validate --template aiml-security-assessment/template.yaml --lint clean.
  • End-to-end single-account deploy with EnableOWASPAssessment=true, EnableFinServAssessment=true: verify HTML report shows the "By Compliance Standard → OWASP Top 10 LLM" section with OW-01…OW-12 rows.
  • End-to-end single-account deploy with EnableOWASPAssessment=true, EnableFinServAssessment=false: verify OW-* rows render, no finserv section in the UI, and FinServ CSV still lands in S3.
  • End-to-end deploy with both flags false: verify no OW-* rows and no OWASP section.

@agasthik agasthik added enhancement New feature or request and removed bedrock sagemaker agentcore labels Jul 19, 2026
agasthik added a commit to agasthik/sample-aiml-security-assessment that referenced this pull request Jul 19, 2026
CI's ruff (0.15.20) collapses a few multi-line literals in
tests/test_sagemaker_checks.py that this branch left unformatted, so
python-lint failed on PR aws-samples#53. Reformat to match ruff's default style.
@agasthik
agasthik force-pushed the feat-owasp-top-llm branch from a8f9b81 to 17c0cb0 Compare July 19, 2026 01:47
@agasthik
agasthik requested a review from vivekmittal514 July 19, 2026 01:57
agasthik added 2 commits July 18, 2026 22:35
Regenerate the sample HTML and dashboard screenshots so the checked-in
sample reflects the OWASP Top 10 for LLM section added on this branch.
@agasthik
agasthik force-pushed the feat-owasp-top-llm branch from fa64711 to 24160ce Compare July 19, 2026 02:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature request: OWASP Top 10 for LLM assessment coverage

1 participant