Python | MIT License | Cross-Platform
π Next-Generation Subdomain Discovery with AI-Powered Intelligence
SubGrab is a high-performance, multi-threaded subdomain enumeration tool designed for security researchers, penetration testers, and bug bounty hunters. Enhanced with OpenRouter AI and Grok AI integration, it combines traditional reconnaissance techniques with advanced artificial intelligence to uncover subdomains that other tools might miss.
SubGrab stands out with its robust feature set and AI-driven capabilities, making it an essential tool for comprehensive subdomain discovery:
- π€ AI-Powered Discovery: Leverages OpenRouter API with multiple AI models for intelligent subdomain generation.
- π Multi-Source Enumeration: Combines 25+ discovery techniques for maximum coverage.
- β‘ High Performance: Multi-threaded architecture with intelligent rate limiting.
- π― Comprehensive Coverage: Combines passive and active reconnaissance methods.
- π Rich Output: Supports multiple export formats with detailed reporting.
- π₯οΈ User-Friendly: Offers both CLI and GUI interfaces.
- π Security-Focused: Includes built-in subdomain takeover detection.
- π± Cross-Platform: Compatible with Windows, Linux, and macOS.
- Certificate Transparency Logs: Enhanced queries to crt.sh and CertSpotter with comprehensive parsing (4000+ certificates processed).
- DNS Enumeration: Supports brute force, SRV records, and zone transfers.
- Web Archives: Extracts subdomains from Wayback Machine, CommonCrawl, and other archives.
- Search Engine Reconnaissance: Uses Google dorks and other search engines for indexed subdomains.
- C99 Subdomain Finder: Automated scan retrieval from subdomainfinder.c99.nl with IP and Cloudflare status data.
- Threat Intelligence Sources: AlienVault OTX, Anubis, ThreatCrowd, HackerTarget, Robtex, Sitedossier.
- Premium APIs: BeVigil, BufferOver, C99.nl, Chaos, FullHunt, IntelX, Netlas, LeakIX, ZoomEye.
- Additional Sources: FOFA, Hunter, Quake, WhoisXML, BuiltWith, Facebook Graph API.
- Security APIs: Integrates with VirusTotal, SecurityTrails, Censys, and Shodan for enriched data.
- GitHub Code Search: Analyzes code repositories for subdomain leaks.
- Reverse DNS Lookups: Maps IPs to domains for additional insights.
- π€ AI-Powered Generation: Uses OpenRouter to intelligently generate subdomain candidates.
- Subdomain Takeover Detection: Identifies vulnerabilities across 50+ services (e.g., AWS S3, GitHub Pages, Heroku).
- SSH Service Detection: Scans for open SSH services on port 22.
- HTTP/HTTPS Status Verification: Checks for live subdomains and their status codes.
- Wildcard DNS Detection: Identifies wildcard DNS configurations to avoid false positives.
- Port Scanning Integration: Detects common services (HTTP, HTTPS, FTP, SMTP) and their versions.
- Multiple Formats: Exports results in TXT, CSV, JSON, and HTML.
- Interactive HTML Reports: Includes charts and statistics for easy analysis.
- Real-time Progress Tracking: Displays live scan updates.
- Detailed Vulnerability Reports: Highlights security findings like takeover risks.
- Tool Compatibility: Exports results compatible with Nmap and Masscan.
- Multi-threading: Configurable threads (1-200) for performance tuning.
- Proxy Support: Supports HTTP, HTTPS, and SOCKS proxies for stealth and bypassing rate limits.
- Custom Wordlists: Allows user-defined wordlists for DNS brute forcing.
- Rate Limiting & Stealth Mode: Minimizes detection with random delays and throttling.
- Custom DNS Servers: Supports user-specified DNS resolvers.
- Timeout Configuration: Adjustable request timeouts for flexibility.
SubGrab's OpenRouter API integration enhances subdomain discovery with cutting-edge AI capabilities.
| Model | Provider | Best For | Performance | Cost |
|---|---|---|---|---|
| Claude 3.5 Sonnet | Anthropic | Best overall | βββββ | Medium |
| Claude 3 Haiku | Anthropic | Fast & cost-effective | ββββ | Low |
| Grok 3 | xAI | General use (recommended) | βββββ | Low |
| Grok 4 | xAI | Complex pattern analysis | βββββ | Medium |
| GPT-4o | OpenAI | High-quality analysis | βββββ | High |
| GPT-4o Mini | OpenAI | Balanced performance | ββββ | Medium |
| Gemini Pro 1.5 | Good alternative | ββββ | Medium | |
| Llama 3.1 8B | Meta | Open source option | βββ | Low |
- Traditional Discovery: Gathers initial subdomains using passive and active methods.
- Pattern Analysis: AI analyzes discovered subdomains to identify naming patterns, numbering schemes, and environments.
- Intelligent Generation: Generates new subdomain candidates based on observed patterns (e.g., if
api1exists, suggestsapi2,api3). - Quality Validation: Filters AI-generated candidates for relevance and likelihood before DNS testing.
- π Pattern Recognition: Identifies naming conventions and organizational patterns.
- π§ Intelligent Variations: Suggests logical extensions based on existing subdomains.
- π’ Context Awareness: Considers organization type and industry-specific patterns.
- π Quality Validation: Ensures high-probability subdomain candidates.
- β‘ Efficiency: Activates AI analysis only when sufficient data (3+ subdomains) is available.
# Clone the repository
git clone https://github.com/bidhata/SubGrab.git
cd SubGrab
# Install dependencies
pip install -r requirements.txt
# Run SubGrab
python subgrab.py example.com- Download the latest release from Releases.
- Extract the ZIP file.
- Double-click
QuickStart.batfor an interactive menu, or runsubgrab.exe example.comdirectly.
# Build Docker image
docker build -t subgrab .
# Run with Docker
docker run -v $(pwd)/results:/app/results subgrab example.com# Simple subdomain enumeration
python subgrab.py example.com
# With AI enhancement (OpenRouter)
python subgrab.py example.com --openrouter-key sk-or-xxxxx
# Fast scan with multiple threads
python subgrab.py example.com --fast --threads 100
# Stealth mode with delays
python subgrab.py example.com --stealth# FREE: Using Grok AI (xAI) - Recommended for budget-conscious users
python subgrab.py example.com \
--grok-key xai-xxxxx
# Grok with specific model
python subgrab.py example.com \
--grok-key xai-xxxxx \
--grok-model grok-3
# AI with Claude 3.5 Sonnet (OpenRouter)
python subgrab.py example.com \
--openrouter-key sk-or-xxxxx \
--openrouter-model anthropic/claude-3.5-sonnet
# Cost-effective with Claude Haiku
python subgrab.py example.com \
--openrouter-key sk-or-xxxxx \
--openrouter-model anthropic/claude-3-haiku
# Use BOTH Grok AND OpenRouter for maximum AI coverage
python subgrab.py example.com \
--grok-key xai-xxxxx \
--openrouter-key sk-or-xxxxx \
--openrouter-model anthropic/claude-3.5-sonnet
# Maximum coverage with multiple APIs
python subgrab.py example.com \
--grok-key xai-xxxxx \
--openrouter-key sk-or-xxxxx \
--shodan-key YOUR_SHODAN_KEY \
--virustotal-key YOUR_VT_KEY# Launch GUI
python subgrab_gui.py
# Or on Windows
subgrab_gui.exeWith the new sources, SubGrab can now discover significantly more subdomains:
- Before: ~200-500 subdomains for typical domains
- After: ~2000-22000+ subdomains for the same domains
- Example: example.com went from 5 to 22,705 subdomains discovered!
usage: subgrab.py [-h] [--threads THREADS] [--timeout TIMEOUT] [--fast] [--stealth]
[--proxy-file PROXY_FILE] [--wordlist WORDLIST] [--nameservers NAMESERVERS]
[--shodan-key SHODAN_KEY] [--securitytrails-key SECURITYTRAILS_KEY]
[--virustotal-key VIRUSTOTAL_KEY] [--censys-id CENSYS_ID]
[--censys-secret CENSYS_SECRET] [--github-token GITHUB_TOKEN]
[--whoisxml-key WHOISXML_KEY]
[--openrouter-key OPENROUTER_KEY] [--openrouter-model OPENROUTER_MODEL]
[--grok-key GROK_KEY] [--grok-model GROK_MODEL]
domain
positional arguments:
domain Target domain to enumerate
optional arguments:
-h, --help Show this help message and exit
-t, --threads THREADS Number of threads (default: 50)
--timeout TIMEOUT Request timeout in seconds (default: 30)
--fast Fast mode - skip intensive tasks
--stealth Enable stealth mode with random delays
--proxy-file PROXY_FILE File containing proxy list
--wordlist WORDLIST Custom wordlist file
--nameservers NAMESERVERS DNS nameservers to use
AI Integration:
--openrouter-key OPENROUTER_KEY OpenRouter API key
--openrouter-model OPENROUTER_MODEL OpenRouter model (default: anthropic/claude-3.5-sonnet)
--grok-key GROK_KEY xAI Grok API key (FREE tier available)
--grok-model GROK_MODEL Grok model (default: grok-3, also: grok-3-mini, grok-4, grok-4.1-fast)
API Keys:
--shodan-key SHODAN_KEY Shodan API key
--securitytrails-key SECURITYTRAILS_KEY SecurityTrails API key
--virustotal-key VIRUSTOTAL_KEY VirusTotal API key
--censys-id CENSYS_ID Censys API ID
--censys-secret CENSYS_SECRET Censys API secret
--github-token GITHUB_TOKEN GitHub API token
--whoisxml-key WHOISXML_KEY WhoisXML API key (500 free credits)
Note: 25+ API sources supported! Use GUI for easy configuration of all API keys.# Simple scan
python subgrab.py example.com
# With custom wordlist
python subgrab.py example.com --wordlist /path/to/wordlist.txt
# Using custom DNS servers
python subgrab.py example.com --nameservers 1.1.1.1 8.8.8.8# AI pattern analysis
python subgrab.py example.com --openrouter-key sk-or-xxxxx
# Specific AI model
python subgrab.py example.com \
--openrouter-key sk-or-xxxxx \
--openrouter-model openai/gpt-4o# High-speed scanning
python subgrab.py example.com --threads 200 --timeout 10
# Stealth mode
python subgrab.py example.com --stealth --threads 10python subgrab.py example.com \
--openrouter-key sk-or-xxxxx \
--shodan-key YOUR_SHODAN_KEY \
--virustotal-key YOUR_VT_KEY \
--securitytrails-key YOUR_ST_KEY \
--censys-id YOUR_CENSYS_ID \
--censys-secret YOUR_CENSYS_SECRET \
--github-token YOUR_GITHUB_TOKEN- Visit console.x.ai or x.ai/api.
- Sign up for a free account.
- Free credits available for new accounts.
- Generate an API key from the console.
- Use the key with
--grok-key xai-xxxxx.
Why Grok?
- π° Free credits for new accounts
- π High-quality AI from xAI
- β‘ Fast and efficient
- π OpenAI-compatible API
- π Very affordable token pricing
- Visit openrouter.ai.
- Sign up and add $5-10 credits.
- Generate an API key.
- Use the key with
--openrouter-key sk-or-xxxxx.
Create api_keys.json (or use the GUI for easy management):
{
"grok": "xai-xxxxx",
"openrouter": "sk-or-xxxxx",
"shodan": "your_shodan_key",
"virustotal": "your_vt_key",
"securitytrails": "your_st_key",
"censys": {
"id": "your_censys_id",
"secret": "your_censys_secret"
},
"github": "your_github_token",
"bevigil": "your_bevigil_key",
"bufferover": "your_bufferover_key",
"c99": "your_c99_key",
"chaos": "your_chaos_key",
"fullhunt": "your_fullhunt_key",
"intelx": "your_intelx_key",
"netlas": "your_netlas_key",
"leakix": "your_leakix_key",
"zoomeye": "your_zoomeye_key",
"fofa": "your_fofa_key",
"hunter": "your_hunter_key",
"quake": "your_quake_key",
"whoisxml": "your_whoisxml_key",
"builtwith": "your_builtwith_key",
"facebook": "your_facebook_token"
}π‘ Tip: Use the GUI's "API Keys" tab for easy configuration with organized categories and direct links to get API keys!
# Example wordlist (custom_subdomains.txt)
www
api
dev
staging
admin
mail
ftp# Proxy file format (proxies.txt)
http://proxy1:8080
https://user:pass@proxy2:3128
socks5://proxy3:1080example.com_results/
βββ all_subdomains.txt # Complete subdomain list
βββ active_subdomains.txt # HTTP/HTTPS responsive
βββ inactive_subdomains.txt # Non-responsive
βββ ssh_enabled.txt # SSH service detected
βββ takeover_candidates.txt # Potential takeovers
βββ scan_results.json # Detailed JSON report
βββ scan_results.csv # CSV format
βββ report.html # Interactive HTML report
- Complete subdomain lists with status.
- Categorized by response type.
- Security findings (e.g., takeovers, SSH).
- Statistics and summaries.
{
"scan_info": {
"target": "example.com",
"start_time": "2024-01-01T12:00:00Z",
"duration": 120.5,
"total_subdomains": 156,
"active_subdomains": 89,
"ai_generated": 23
},
"subdomains": [
{
"subdomain": "api.example.com",
"status": "active",
"ip": "192.168.1.1",
"http_status": 200,
"technologies": ["nginx", "cloudflare"],
"discovery_method": "certificate_transparency"
}
],
"vulnerabilities": [
{
"subdomain": "old.example.com",
"type": "subdomain_takeover",
"service": "github.io",
"confidence": "high"
}
]
}Interactive dashboard with:
- π Statistics charts.
- πΊοΈ Discovery method breakdown.
- π Searchable subdomain table.
β οΈ Security findings.- π Response time analysis.
| Service | Purpose | Free Tier | Rate Limit |
|---|---|---|---|
| OpenRouter | AI subdomain generation | β | Model dependent |
| Grok (xAI) | AI subdomain generation | β Free credits | Model dependent |
| Shodan | Infrastructure discovery | β Limited | 1 req/sec |
| VirusTotal | Domain intelligence | β 4 req/min | 4 req/min |
| SecurityTrails | DNS history | β 50 req/month | Varies |
| Censys | Certificate data | β 250 req/month | 0.2 req/sec |
| GitHub | Code search | β 5000 req/hour | 30 req/min |
| Service | Purpose | Coverage | Performance |
|---|---|---|---|
| AlienVault OTX | Passive DNS data | β Free | High |
| Anubis | Subdomain database | β Free | Very High |
| ThreatCrowd | Community intelligence | β Free | Medium |
| HackerTarget | DNS reconnaissance | β Free | Medium |
| Robtex | DNS/IP intelligence | β Free | Medium |
| Sitedossier | Domain analysis | β Free | Low |
| Service | Purpose | Pricing | Coverage |
|---|---|---|---|
| BeVigil | Mobile app security | π° Paid | Mobile-focused |
| BufferOver | DNS data provider | π° Paid | High |
| C99.nl | Multi-tool platform | π° Paid | Very High |
| Chaos | ProjectDiscovery data | π° Paid | High |
| FullHunt | Attack surface mgmt | π° Paid | High |
| IntelX | Intelligence platform | π° Paid | Very High |
| Netlas | Internet assets search | π° Paid | High |
| LeakIX | Leak detection | π° Paid | Medium |
| ZoomEye | Cyberspace search | π° Paid | High |
| FOFA | Cyberspace assets | π° Paid | High |
| Hunter | Threat intelligence | π° Paid | High |
| Quake | Cyberspace mapping | π° Paid | High |
| WhoisXML | Subdomain lookup | β 500 credits | High |
| BuiltWith | Technology profiler | π° Paid | Medium |
| Social platform API | π° Paid | Low |
- π Easy Configuration: Point-and-click setup with organized tabs.
- π Real-time Progress: Live scan updates with detailed output.
- π Enhanced API Key Management: 25+ API sources organized in categories with direct "Get Key" links.
- π Result Management: Direct access to reports and results folder.
- π¨ Modern Interface: Clean, scrollable design with improved layout.
- πΎ Save/Load Configurations: Export and import API key configurations.
- π±οΈ Mouse Wheel Scrolling: Smooth navigation through extensive API key lists.
python subgrab_gui.py
# Or on Windows
subgrab_gui.exesubgrab.exe: Command-line version (213 MB).subgrab_gui.exe: GUI version (29 MB).QuickStart.bat,run_cli.bat,run_gui.bat: Easy launchers.- Complete documentation and examples.
- Download the latest release ZIP.
- Extract to a folder.
- Run
QuickStart.batand choose GUI or CLI mode.
- Enhanced Certificate Transparency: Comprehensive crt.sh parsing (4000+ certificates), CertSpotter integration.
- C99 Subdomain Finder: Automated scan retrieval from subdomainfinder.c99.nl with IP and Cloudflare data.
- Threat Intelligence: AlienVault OTX (500+ subdomains), Anubis (20000+ subdomains), ThreatCrowd, HackerTarget.
- DNS Intelligence: Brute forcing, SRV records, zone transfers, reverse DNS, Robtex, Sitedossier.
- Web Archives: Wayback Machine, CommonCrawl, and Archive.today for historical data.
- Search Engines: Google, Bing, DuckDuckGo with advanced operators.
- Premium APIs: BeVigil, BufferOver, C99.nl, Chaos, FullHunt, IntelX, Netlas, LeakIX, ZoomEye.
- Additional Sources: FOFA, Hunter, Quake, WhoisXML, BuiltWith, Facebook Graph API.
- Security APIs: VirusTotal, SecurityTrails, Censys, Shodan.
- Code Repositories: GitHub and GitLab code search.
- HTTP/HTTPS Probing: Status code verification, technology fingerprinting.
- Port Scanning: SSH, HTTP, HTTPS, FTP, SMTP detection.
- Intelligent Generation: Context-aware subdomain suggestions.
- Pattern Analysis: Identifies naming conventions and environments.
- Supported Services: AWS S3, Azure, GitHub Pages, Heroku, Netlify, and 50+ others.
- Detection Methods: DNS resolution, HTTP response analysis, SSL certificate validation.
- Authorized testing only.
- Rate limiting and stealth mode for responsible scanning.
- Proxy support for anonymity.
- Comprehensive logging for audit trails.
| Metric | Traditional Tools | SubGrab v1 | SubGrab v2 Enhanced |
|---|---|---|---|
| Subdomains Found | 200-300 | 400-600 | 2000-22000+ |
| Discovery Sources | 5-10 | 15+ | 25+ |
| Execution Time | 5-10 min | 3-7 min | 5-15 min |
| False Positives | 10-15% | 5-8% | 2-5% |
| Unique Discoveries | Standard | Enhanced | Comprehensive |
| API Integrations | 2-3 | 6 | 25+ |
Real Example: example.com discovery improved from 5 subdomains to 22,705 subdomains with enhanced sources!
- WhoisXML Subdomain API: Comprehensive subdomain lookup with 500 free credits
- AI Retry Logic: Automatic retry with exponential backoff for transient API errors (502/503/504)
- Grok AI Integration: xAI Grok support for affordable AI-powered subdomain discovery
- 25+ Discovery Sources: Expanded from 15 to 25+ enumeration techniques
- C99 Subdomain Finder: Automated scan retrieval with IP and Cloudflare status data
- Comprehensive Certificate Transparency: Processes 4000+ certificates with improved parsing
- Free Threat Intelligence: AlienVault OTX, Anubis (20K+ subdomains), ThreatCrowd, HackerTarget
- Premium API Support: 15+ premium sources including BeVigil, C99.nl, FullHunt, IntelX, Netlas
- Enhanced GUI: Organized API key management with 25+ sources, scrollable interface, save/load configs
- 22,000+ Subdomains: Real example of example.com going from 5 to 22,705 subdomains
- Intelligent Rate Limiting: Respectful crawling with automatic backoff
- Comprehensive Coverage: Extensive subdomain extraction from multiple scan sources
- Multiple Extraction Methods: Table parsing, regex extraction, JSON API calls
- Organized Categories: Core APIs, Premium Intelligence, Additional Sources
- Direct Links: "Get Key" buttons for each API provider
- Save/Load Configurations: Easy API key management
- Scrollable Interface: Handles 25+ API sources efficiently
- Professional Layout: Clean design with proper spacing and organization
We welcome contributions from the security community! To contribute:
- Fork the repository.
- Create a feature branch.
- Make changes and test thoroughly.
- Submit a pull request with clear documentation.
See the Contributing section in the original documentation for detailed guidelines.
This project is licensed under the MIT License. See the LICENSE file for details.
- β Use only for authorized testing.
- β Follow responsible disclosure practices.
- β Do not use for malicious purposes.
- β No warranty; use at your own risk.
Author: Krishnendu Paul
GitHub: bidhata/SubGrab
Support: Open an issue on GitHub or contact the author for assistance.
β If SubGrab helped you, please give it a star! β
Made with β€οΈ for the Security Community.