[REVIEW] secrets-management: separate exposed-secret findings from control gaps

[sato820]

Skill Being Reviewed

Skill name: secrets-management
Skill path: skills/devsecops/secrets-management/

False Positive Analysis

Benign code/config that triggers a false positive if line 195 is followed:

# .github/workflows/ci.yml
name: ci
on: [push, pull_request]
jobs:
  test:
    runs-on: ubuntu-latest
    permissions:
      contents: read
    steps:
      - uses: actions/checkout@v4
      - run: npm ci
      - run: npm test

node_modules/
dist/
.env
.env.*

There are no committed credentials, no .env files, and no secret-like values. The only weakness is that the repo has no local .gitleaks.toml, .secrets.baseline, TruffleHog job, or pre-commit hook.

Why this is a false positive:

The skill contradicts itself:

• skills/devsecops/secrets-management/SKILL.md:169-173 says absence of secret detection tooling is not a secrets finding and should be noted in the Detection Tooling Status table instead of the findings count.
• skills/devsecops/secrets-management/SKILL.md:195 classifies "No secret detection tooling deployed" as Critical.

If an agent follows line 195, a benign repo with no exposed credential receives a Critical secrets finding. That mixes actual leaked secrets with program-maturity gaps and conflicts with the skill's own scope rule that numbered findings should only report cases where a credential/key/token/cert is actually present.

Coverage Gaps

Missed variant 1: platform-native scanning without local config

GitHub Secret Protection / GitLab Secret Detection is enabled at org or repo
settings level, but no .gitleaks.toml, .trufflehog.yml, or .secrets.baseline is
committed.

Why it should be caught: The current tooling table only lists repo-visible tools. Real programs often configure scanning at the platform level, so the skill should ask for or record platform-native scanner status before declaring "no tooling."

Missed variant 2: secrets outside the git tree

An active token appears in a pull request body, issue comment, Discussion, wiki,
or public secret gist while the repository files and git history are clean.

Why it should be caught: The discovery process only uses Glob/Grep against files/config/history. GitHub secret scanning can cover pull requests, issues, discussions, wikis, and secret gists, so the skill should add an optional collaboration-surface check when exports/API results are available, or clearly mark that surface out of scope.

Edge Cases

1. A repo with GitHub Secret Protection enabled and no local scanner config can be incorrectly classified as Critical "no tooling."
2. A repo with no actual secrets but no pre-commit hook can receive higher severity than a repo with an inactive, already-revoked historical token. The skill needs validity/remediation state.
3. A detect-secrets baseline can prevent new leaks while old findings are tracked separately. Treating the baseline as proof of safety or proof of active leakage without checking audit/rotation status misclassifies risk.

Remediation Quality

• Fix resolves the vulnerability
• Fix doesn't introduce new security issues
• Fix doesn't break functionality
• Issues found: The current output model blurs "exposed secret found" with "secrets program maturity gap."

Recommended fix:

1. Secret Exposure Findings: only actual leaked, embedded, or unrotated credentials. Critical should require an exposed active credential, valid private key, or known unrotated historical secret.
2. Secrets Control Gaps: no scanner, no pre-commit hook, no history scan, no centralized manager, no automated rotation, excessive allowlists. These may be High/Medium, but should not count as leaked-secret findings unless an actual credential is present.
3. Add platform-native scanner checks: GitHub Secret Protection, GitLab Secret Detection, cloud repo scanners, and organization-level push protection.
4. Add validity/remediation fields: active, inactive/revoked, unknown, rotated, purged from history, provider notified.

Comparison to Other Tools

Tool	Catches this?	Notes
Gitleaks	Partial	Finds secrets in files/history; does not decide whether missing tooling is exposure.
TruffleHog	Partial	Can scan history and verify some secrets; tool absence is still a maturity signal.
GitHub Secret Protection	Partial	Can detect repository and collaboration-surface secrets and expose validity metadata, often outside the repo tree.
Semgrep	Partial	Rules can catch hardcoded secret patterns, but not org-level scanner coverage or validity.

Overall Assessment

Strengths: Good discovery patterns, strong placeholder filtering, and useful guidance for .env, history, vault integration, rotation, and agent credential TTLs.

Needs improvement: Resolve the classification contradiction so agents do not emit Critical findings for repos with no actual leaked credential.

Priority recommendations:

1. Move "no tooling" into a control-gap table, not the numbered exposure findings table.
2. Add platform-native scanner coverage before declaring scanner absence.
3. Add secret validity/remediation-state fields.

Bounty Info

• I have read and agree to the CONTRIBUTING.md bounty terms
• Preferred payment method: Crypto

[JamesJi79]

/attempt

[NiXouuuu]

/attempt #1611

I will make a focused improvement pass on skills/devsecops/secrets-management/SKILL.md to separate actual Secret Exposure Findings from Secrets Control Gaps. Planned scope: remove the Critical classification contradiction for missing tooling, add platform-native scanner checks, add collaboration-surface coverage where evidence is available, add validity/remediation-state fields, and update output/common-pitfall guidance so control maturity gaps do not count as leaked-secret findings. Payment preference can be provided privately after maintainer acceptance.

[modelsbridgeaicom-ship-it]

Implemented in PR #1612: #1612

Scope:

• Separates confirmed Secret Exposure findings from Secrets Control Gaps.
• Removes missing scanner evidence from Critical exposure classification unless an actual active/unrotated secret is present.
• Adds GitHub/GitLab platform-native scanner evidence, validity/remediation states, and dedicated control-gap output.
• Adds benign/vulnerable calibration fixtures for the exact false-positive case in this review.

Validation is listed in the PR body. Payment preference can be provided privately after maintainer acceptance.

[JamesJi79]

Implemented in PR #1613. Gate file: skills/devsecops/secrets-management/gates/secret-exposure-vs-control-gap-gate.md

[JamesJi79]

Implemented in PR #1613. See the gate file for details.

[JamesJi79]

Implemented. Gate file submitted in the related PR.