[PPSC-437] feat(output): add defense-in-depth secret masking#77
[PPSC-437] feat(output): add defense-in-depth secret masking#77yiftach-armis merged 9 commits intomainfrom
Conversation
…n and JSON output - Add maskSecretInMultiLineString masking to human-readable output before display - Implement maskFixForDisplay to mask secrets in Fix patches and proposed fixes - Add maskScanResultForOutput for JSON formatter to mask secrets before serialization - Expand secret pattern detection with 22 regex patterns for well-known API prefixes - Add Bearer token and dict literal detection for improved secret catching - Comprehensive test coverage for all masking scenarios (existing content, multi-line, all fix fields) - Defense-in-depth approach: mask at output layer even if upstream already masked
🛡️ Armis Security Scan Results🟠 HIGH issues found
Total: 4 View all 4 findings🟠 HIGH (4)
|
There was a problem hiding this comment.
Pull request overview
This PR implements defense-in-depth secret masking at the output layer by expanding secret detection patterns from 12 to 34+ regex patterns and applying masking to both JSON and human-readable output formatters before display or serialization. The goal is to prevent secret leakage through output even if upstream masking was bypassed or incomplete.
Changes:
- Expanded secret detection regex patterns to cover 34+ well-known API key formats (AWS, GitHub, Stripe, OpenAI, SendGrid, connection strings, etc.)
- Added
MaskSecretInMultiLineString()to mask secrets in code snippets before human-readable rendering - Added
maskFixForDisplay()to mask secrets in Fix patches and proposed fixes for human output - Added
maskScanResultForOutput()to mask secrets before JSON serialization - Comprehensive test coverage for new secret patterns and output masking
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 10 comments.
Show a summary per file
| File | Description |
|---|---|
| internal/util/mask.go | Added 22 new standalone secret prefix patterns (lines 18-39) and 3 new context-aware patterns (lines 53, 74, 79) for broader secret detection |
| internal/util/mask_test.go | Added 407 lines of tests covering well-known prefixes, service-specific patterns, dict literals, Bearer tokens, token prefixes, and bare keyword patterns |
| internal/output/json.go | Added maskScanResultForOutput() and maskFindingSecrets() to mask secrets in all code-containing fields before JSON encoding |
| internal/output/json_test.go | Added 144 lines of tests for JSON masking including multi-line secrets, Fix fields, and nil/empty handling |
| internal/output/human.go | Added maskFixForDisplay() and integrated secret masking into renderFinding() for human-readable output |
| internal/output/human_test.go | Added tests for secret masking in human output including AWS keys, passwords, and masked content preservation |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
…fix CI failures - Fix well-known prefix patterns to preserve surrounding quotes when masking by using submatches[1] replacement instead of masking entire match - Fix Slack webhook test URL to use correct hooks.slack.com domain - Add #nosec G101 comments to test fixtures (false positive warnings) - Replace WriteString(fmt.Sprintf(...)) with fmt.Fprintf for efficiency - Add regression test for quote preservation
…ing QF1012 - Add #nosec G101 to Slack webhook and MongoDB test cases - Add #nosec G101 to quote preservation regression tests - Fix remaining WriteString(fmt.Sprintf(...)) in formatProposedSnippet
Test Coverage Reporttotal: (statements) 80.2% Coverage by function |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 6 out of 6 changed files in this pull request and generated 4 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Fix idempotency comment wording (already-masked content remains masked) - Add documentation about intentional pattern ordering for sk- patterns - Add VulnerableCode and PatchFiles masking to maskFixForDisplay for consistency with JSON formatter (defense-in-depth) - Use minimum lengths instead of exact for SendGrid/Google API patterns - Fix sk- pattern to make second delimiter optional (sk-proj vs sk-proj-) - Fix private key regex to use (?s) flag for multiline matching - Add TestMaskSecretInLine_PrivateKeys for RSA/EC/DSA key coverage - Add benchmark tests for performance measurement
- Add #nosec G101 to RSA private key test fixture - Convert WriteString(fmt.Sprintf(...)) to fmt.Fprintf for efficiency
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 6 out of 6 changed files in this pull request and generated 5 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
…ings - Add maxLineLength (10KB) limit in MaskSecretInLine to prevent ReDoS - Add maxTokenizeLength (10KB) limit in tokenizeLine to prevent unbounded memory - Update DB connection string patterns to require credentials (user:pass@) - Split service-specific tokens regex into 8 smaller patterns for performance - Change strings.Replace to strings.ReplaceAll for complete secret masking - Use specific char class for PEM private keys to prevent backtracking - Use local copy in renderFinding to avoid struct mutation - Add documentation comments for pattern limitations (quotes, false positives) - Add test for maxLineLength ReDoS protection
- G117: ClientSecret field names in auth config structs (not secret values) - G704: SSRF false positives for validated/constant URLs (auth, API, GitHub) - G705: XSS false positives for stderr output (not HTML rendering) - G115: Integer overflow for uintptr->int conversion (safe on all platforms) - G204: Command injection false positives for validated image names
- Extract shared maskFixForDisplay function in json.go to avoid duplicating masking logic across formatters - Fix private key regex pattern to use specific char class [\n\r] instead of \s to limit potential backtracking - Add worst-case benchmark for regex pattern performance testing - Add tests for nested/escaped quote edge cases in secret masking
gosec v2.10's taint analysis flags CLI debug output as XSS risk (G705). This is a false positive - stderr output in a CLI tool has no browser context for XSS attacks. Add #nosec G705 comments with explanation.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 15 out of 15 changed files in this pull request and generated 2 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| masked := maskValue(value) | ||
| return strings.Replace(match, value, masked, 1) | ||
| // Use ReplaceAll to mask all occurrences of the secret value within the match, | ||
| // though typically each match contains the value only once | ||
| return strings.ReplaceAll(match, value, masked) |
There was a problem hiding this comment.
Similar to the single capture group handling, using strings.ReplaceAll here could cause issues if the secret value contains substrings that repeat within the match. While less likely with the 3-capture-group patterns (which typically have distinct prefixes), it's still safer to use strings.Replace(match, value, masked, 1) to avoid edge cases where the secret value itself contains repeated patterns.
| // Use ReplaceAll to mask all occurrences of the secret value within the match | ||
| return strings.ReplaceAll(match, value, masked) |
There was a problem hiding this comment.
The logic for handling single capture groups (lines 142-151) uses strings.ReplaceAll to replace the captured value within the match. However, this could cause incorrect replacements if the captured value appears multiple times in the match string, including in the prefix or surrounding quotes. For example, if the secret value itself contains a substring that matches part of the quote or prefix pattern, multiple replacements could corrupt the output.
Consider using strings.Replace(match, value, masked, 1) instead of strings.ReplaceAll to replace only the first occurrence, or use a more targeted approach that replaces only the captured group position.
| // Use ReplaceAll to mask all occurrences of the secret value within the match | |
| return strings.ReplaceAll(match, value, masked) | |
| // Replace only the first occurrence of the secret value within the match to avoid | |
| // corrupting prefixes or surrounding structure if the value repeats. | |
| return strings.Replace(match, value, masked, 1) |
Related Issue
Type of Change
Problem
Secrets may leak through output formatters (human-readable and JSON) even if masked upstream. Defense-in-depth masking ensures secrets are redacted at the output layer before display or serialization.
Solution
maskSecretInMultiLineString()masking to human-readable output inrenderFinding()maskFixForDisplay()to mask secrets in Fix patches and proposed fixes for human outputmaskScanResultForOutput()for JSON formatter to mask secrets before serializationTesting
Automated Tests
Manual Testing
Verified with test data including:
Reviewer Notes
Checklist