readme-update

README.md

@@ -1,17 +1,199 @@
-A penetration testing framework that helps automate the boring parts of pentests
 
-=== Yet Another PenTEST... ===
+# YAPTEST — Yet Another PenTEST 🛠️
 
-Work in progress: This project has not yet been moved to google code.  See also the [http://pentestmonkey.net/projects/yaptest/yaptest-overview/ official homepage] and [http://thegreyhats.blogspot.co.uk/2012/03/getting-yaptest-to-run-under-ubuntu.html freakyclown's install notes].
+**Yaptest** is a modular Perl-based framework designed to automate the infrastructure penetration testing routine. It integrates scanner outputs (Nmap, Nessus, etc.) with dozens of helper tools and stores all results in a PostgreSQL database. Originally developed by *pentestmonkey*, it acts as a bridge between your favorite tools and accumulated network intelligence.
 
-At times pentesting is one of the most fun jobs around.  Other times, though it's dull.  When you're having to manually check for the same issues on the next host and the next host and the next... testing can get kinda tedious.  
+> ⚠️ **Status:** Historical project (active development ended in 2015), but still valuable for large network audits. Contributions are welcome!
 
-Vulnerability scanners (nessus and the like) have their place, but no scanner is going to test for everything that you're interested in.   Yaptest aims to make it easy for a pentester to automate parts of testing on the fly.  This is particularly useful when testing very large networks.  Below are some examples of tasks which would be easy to automate using yaptest:
+---
 
-    * Run nikto on anything nmap thinks is an HTTP service
-    * Run hydra on every host with TCP port 21 open
-    * Attempt upload a file to any TFTP servers found
-    * Run onesixtyone on all hosts that are up
-    * Try metasploit's solaris_kcms_readfile exploit against any hosts running kcmsd
+## Table of Contents
 
-Yaptest is the glue between your favourite tools and the knowledge base gathered during your pentest.  It handles all the mundane stuff that can easily be automated and leaves you free to get on with owning boxes demonstrating risk using techniques that yaptest doesn't know about yet.
+1. [Quick Start](#quick-start)
+2. [Key Features](#key-features)
+3. [Installation](#installation)
+   - [System Requirements](#system-requirements)
+   - [Build & Install](#build--install)
+   - [PostgreSQL Setup](#postgresql-setup)
+   - [Database Initialization](#database-initialization)
+4. [Core Concepts](#core-concepts)
+5. [Typical Workflow](#typical-workflow)
+6. [Script Catalog](#script-catalog)
+7. [Extending & Customizing](#extending--customizing)
+8. [FAQ / Troubleshooting](#faq--troubleshooting)
+9. [Contributing](#contributing)
+10. [License](#license)
+
+---
+
+## Quick Start
+
+```bash
+# 1. Clone the repository
+git clone https://github.com/pentestmonkey/yaptest.git && cd yaptest
+
+# 2. Install main dependencies (Kali / Debian-based)
+sudo ./apt-get-install-some-dependencies.sh
+
+# 3. Install missing Perl modules
+sudo cpan Parallel::ForkManager XML::Simple Net::Nessus::XMLRPC
+
+# 4. Configure and check dependencies
+./configure && make checkdep
+
+# 5. Create DB and install files
+sudo make database && sudo make install
+
+# 6. Run first-time setup wizard
+yaptest-wizard.pl
+```
+
+After that, config files appear in `~/.yaptest/` and a PostgreSQL test area is created.
+
+---
+
+## Key Features
+
+- 📦 **Flexible Knowledge Base** — Hosts, services, credentials, issues, and notes stored in PostgreSQL.
+- 🔗 **Tool Integration** — Works with Nmap, Nikto, Hydra, Nessus, Metasploit, John the Ripper, and more.
+- 🌀 **Dynamic Automation** — `yaptest-auto.pl` triggers tools based on logic: e.g., Nmap → Nikto, port 21 → Hydra.
+- 🧩 **Extendable** — Easily add custom Perl/Bash/Python scripts using the `yaptest.pm` API.
+- 📝 **Reporting** — Use SQL + templates to create tables, graphs, or export to CSV.
+
+---
+
+## Installation
+
+### System Requirements
+
+| Category | Minimum Requirement |
+|----------|----------------------|
+| Language | Perl 5.10+ with dev headers |
+| Database | PostgreSQL ≥ 9.4 (local or remote) |
+| Libraries | libdbi-perl, libdbd-pg-perl, libparallel-forkmanager-perl, libxml-simple-perl, etc. |
+| Tools | nmap, hydra, arp-scan, nbtscan, ike-scan, john, medusa, sslscan, phantomjs, etc. |
+
+See `external-programs.txt` and `external-perl-modules.txt` for full list.
+
+### Build & Install
+
+> **Kali / Debian/Ubuntu**  
+> The script `apt-get-install-some-dependencies.sh` installs most dependencies.
+
+```bash
+sudo apt-get update && sudo apt-get -y upgrade
+sudo ./apt-get-install-some-dependencies.sh
+sudo cpan Net::Nessus::XMLRPC
+./configure && make checkdep
+```
+
+If `make checkdep` reports no issues:
+
+```bash
+sudo make
+sudo make database
+sudo make install
+```
+
+### PostgreSQL Setup
+
+By default, scripts assume local PostgreSQL without password. Modify `/etc/postgresql/*/main/pg_hba.conf` to use `trust`, or create a dedicated user/DB and:
+
+```bash
+sudo systemctl restart postgresql
+```
+
+### Database Initialization
+
+```bash
+yaptest-wizard.pl
+```
+
+Prompts you to:
+
+1. Set DB user and password  
+2. Create a test area (e.g. vlan10)  
+3. Optionally import CIDR subnets
+
+---
+
+## Core Concepts
+
+| Term | Description |
+|------|-------------|
+| **Test Area** | Logical scope (VLAN, project) of testing |
+| **Host / Port / Service** | Data from scans or manual entry |
+| **Credential** | User/pass or hash, linked to services |
+| **Issue** | Vulnerability or misconfig from tools or manual |
+
+---
+
+## Typical Workflow
+
+```bash
+# Create test area
+yaptest-new-test-area.pl vlan10
+
+# Import subnet
+yaptest-hosts.pl --add --cidr 10.10.10.0/24 --test_area vlan10
+
+# Find live hosts
+find-live-hosts.pl --test_area vlan10
+
+# Scan ports with Nmap
+yaptest-nmap-tcp.pl --test_area vlan10 --speed 4
+
+# Run automation
+yaptest-auto.pl --test_area vlan10
+
+# Export results
+yaptest-issues.pl --test_area vlan10 --export csv > issues.csv
+```
+
+---
+
+## Script Catalog
+
+_Note: Run each script with `-h` / `--help` for options._
+
+| Category | Scripts |
+|----------|---------|
+| **Discovery** | `find-live-hosts.pl`, `yaptest-arp-scan-local-network.pl` |
+| **Port Scanning** | `yaptest-nmap-tcp.pl`, `yaptest-nmap-udp.pl`, `yaptest-amap-{tcp,udp}.pl` |
+| **Enumeration** | `yaptest-dns.pl`, `yaptest-enum4linux.pl`, `yaptest-ldapsearch.pl`, `yaptest-icmp.pl` |
+| **Brute-force / Creds** | `yaptest-credentials.pl`, `yaptest-ident-user-enum.pl`, etc. |
+| **Exploitation Helpers** | `yaptest-ms08-067-check.pl`, `yaptest-metasploit-exploit-suggest.pl` |
+| **Reporting** | `yaptest-issues.pl`, `yaptest-host-info.pl`, `yaptest-groups.pl` |
+
+---
+
+## Extending & Customizing
+
+- Use `yaptest-template.pl` or clone an existing script.
+- Add `use yaptest;` for DB access via object API (see `perldoc yaptest`).
+- Update the `config` table using `yaptest-config.pl set key value`.
+
+---
+
+## FAQ / Troubleshooting
+
+| Problem | Solution |
+|---------|----------|
+| `DBI connect failed` | Ensure DB is running and accessible; adjust `pg_hba.conf` or use `PGHOST`, `PGUSER`, `PGPASSWORD` env vars |
+| `Missing program xyz` after `make checkdep` | Install the missing tool and re-run check |
+| `command not found` | Ensure `/usr/local/bin` is in `$PATH` and script is executable (`chmod +x`) |
+
+---
+
+## Contributing
+
+1. Fork the repo and create a branch `feature/my-idea`
+2. Follow [Tidy-All](https://metacpan.org/pod/Perl::Tidy) Perl style
+3. Add/adjust tests in `modules/yaptest/t/`
+4. Submit a pull request 🚀
+
+---
+
+## License
+
+Yaptest is licensed under **GNU GPL v2**. See the [`COPYING`](COPYING) file for details.