Skip to content

Add github-recon-v3.sh: robustness, pagination, severity model, OSINT extensions#1

Open
pepperonas wants to merge 1 commit into
HoodSecurity:mainfrom
pepperonas:feat/github-recon-v3
Open

Add github-recon-v3.sh: robustness, pagination, severity model, OSINT extensions#1
pepperonas wants to merge 1 commit into
HoodSecurity:mainfrom
pepperonas:feat/github-recon-v3

Conversation

@pepperonas
Copy link
Copy Markdown

What

Adds bash/github-recon-v3.sh next to the existing github-recon.sh — a rewrite focused on robustness, performance and broader OSINT/security coverage. The original script stays untouched, the new file lives alongside it.

Why

The original works as a great demo but stumbles on a few real-world edge cases (rate-limiting, pagination, search-API explosion). This version handles those and adds OSINT signals that turned out to be useful in self-audits.

Highlights

Robustness

  • HTTP status-code aware API layer, no more crashes when GitHub returns an error object instead of an array
  • Full pagination (was capped at 100 repos)
  • Retry with backoff on 403/429, auto-wait until rate-limit reset

Performance

  • Sensitive-files step: 18 API calls instead of 1800 (user:X filename:Y query, not repos×patterns)

New OSINT

  • Public SSH/GPG keys (.keys / .gpg endpoints)
  • Organisation memberships
  • Activity heatmap with timezone inference (dual hypothesis: pro vs. hobby coder)
  • Wayback Machine lookup for deleted/renamed repos

New security checks

  • CI/CD workflow YAML scan (.github/workflows/*.yml — common secret source)
  • Internal endpoint detection (192.168.*, .local, localhost:*)

Deep scan

  • noseyparker / gitleaks / trufflehog are actually invoked (previously only mentioned)

Workflow

  • Interactive token setup (--setup-token) with gh CLI auto-detect
  • Diff mode (--diff-from=<old-findings.json>) for tracking changes over time

CI-ready

  • Severity levels (CRITICAL/HIGH/MEDIUM/LOW/INFO)
  • Exit codes 40/30/20/10/0 for build gating
  • JSON output alongside markdown

File layout

```
bash/
github-recon.sh (original, untouched)
github-recon-v3.sh (new)
README.md (new — describes both)
```

Compatibility

Requires bash ≥ 4.0. macOS users: `brew install bash`. The original script also benefits from this (line 274 `declare -A` fails on macOS system bash 3.2).

Rewrite of bash/github-recon.sh with focus on robustness, performance
and additional OSINT/security coverage. Original script remains untouched.

Highlights:
- Robust API-error handling, full pagination, retry with backoff
- Sensitive-files step: 18 API calls instead of 1800 (user:X filename:Y)
- Interactive token setup with gh CLI auto-detect (--setup-token)
- New OSINT: public SSH/GPG keys, orgs, activity heatmap with timezone
  inference, Wayback Machine for deleted/renamed repos
- New security: CI/CD workflow YAML scan, internal endpoint detection
- Deep scan actually invokes noseyparker/gitleaks/trufflehog
- Severity levels (CRITICAL/HIGH/MEDIUM/LOW/INFO), CI-ready exit codes
- JSON output additionally to markdown, diff mode for tracking
  findings over time

Adds bash/README.md describing both scripts.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant