diff --git a/README.md b/README.md index a538401..6e1403f 100644 --- a/README.md +++ b/README.md @@ -49,14 +49,14 @@ unzip windows-essential-0.2.0.zip ```toml [scanner] -sigma_rules_path = "windows-essential/rules/sigma" -yara_rules_path = "windows-essential/rules/yara" +sigma_rules_path = "windows-essential/sigma" +yara_rules_path = "windows-essential/yara" [ioc] -hashes_path = "windows-essential/rules/ioc/hashes.txt" -ips_path = "windows-essential/rules/ioc/ips.txt" -domains_path = "windows-essential/rules/ioc/domains.txt" -paths_regex_path = "windows-essential/rules/ioc/paths_regex.txt" +hashes_path = "windows-essential/ioc/hashes.txt" +ips_path = "windows-essential/ioc/ips.txt" +domains_path = "windows-essential/ioc/domains.txt" +paths_regex_path = "windows-essential/ioc/paths_regex.txt" ``` **3. Confirm it works.** The Essential packs ship the **EICAR** test IOC set — drop a standard EICAR test file on disk and Rustinel raises an IOC alert in `logs/alerts.json.`. @@ -94,7 +94,7 @@ Full catalog and per-pack rule inventory: **[docs/packs.md](docs/packs.md)**. `rustinel-rules` is versioned **independently** from the engine — detection content evolves faster. Each pack manifest declares the engine version it needs: ```yaml -pack_schema_version: 1 +pack_schema_version: 2 requires_rustinel: ">=1.0.2" ``` diff --git a/docs/authoring.md b/docs/authoring.md index b3ffb17..139042d 100644 --- a/docs/authoring.md +++ b/docs/authoring.md @@ -93,7 +93,9 @@ Then add the `id` to a pack, e.g. `packs/windows/essential/pack.yml`: ```yaml rules: - - 7f3a1c2e-4b5d-4e6f-8a90-1b2c3d4e5f60 # Suspicious Encoded PowerShell Command Line + has: + sigma: + - 7f3a1c2e-4b5d-4e6f-8a90-1b2c3d4e5f60 # Suspicious Encoded PowerShell Command Line ``` --- diff --git a/docs/packs.md b/docs/packs.md index ca4256c..0fa0e5e 100644 --- a/docs/packs.md +++ b/docs/packs.md @@ -18,7 +18,7 @@ Essential ⊂ Advanced ⊂ Hunting | [macOS Essential](#macos-essential) | essential | ❌ | low | experimental | | [macOS Advanced](#macos-advanced) | advanced | ❌ | medium | experimental | -> All packs declare `requires_rustinel: ">=1.0.2"`, `pack_schema_version: 1`, and license +> All packs declare `pack_schema_version: 2`, `requires_rustinel: ">=1.0.2"`, and license > `DRL-1.1`. `status: experimental` reflects the early state of v1 content — expect curation to > tighten as coverage grows. diff --git a/docs/repository.md b/docs/repository.md index d01d01c..294e03c 100644 --- a/docs/repository.md +++ b/docs/repository.md @@ -53,9 +53,11 @@ Key manifest fields: | `id` / `name` | Stable id (`^[a-z0-9]+(-[a-z0-9]+)*$`) and human name. | | `os` | `windows` \| `linux` \| `macos`. | | `level` | `essential` \| `advanced` \| `hunting`. | +| `pack_schema_version` | Required pack manifest schema version (must be `2`). | | `default` | Whether this pack is enabled by default. | | `extends` | Pack ids cumulatively included (rules merged, never duplicated). | -| `rules` | Artifact ids added by *this* pack. | +| `rules` | Optional dictionary specifying rules directly in this pack (`has`), or rules to include (`includes`) / exclude (`excludes`) from extended packs or automatic sources. | +| `sources` | Optional dictionary of upstream sources categorized by type (`manual`, `sigma`, `yara`). | | `requires_rustinel` | Engine version constraint, e.g. `">=1.0.2"`. | | `attack_coverage` | ATT&CK technique ids covered (drift-checked against members). | | `telemetry_requirements` | Rustinel telemetry channels the pack needs. | @@ -82,18 +84,17 @@ dist/ ├── catalog.json # Website catalog: rules, ATT&CK techniques, packs ├── windows-essential/ # Materialized flat pack folder (engine drop-in) │ ├── pack.yml # Cleaned manifest + build metadata (version, counts) -│ └── rules/ -│ ├── sigma/ # .yml -> scanner.sigma_rules_path -│ ├── yara/ # .yar -> scanner.yara_rules_path -│ └── ioc/ # hashes.txt / ips.txt / domains.txt / paths_regex.txt +│ ├── sigma/ # .yml -> scanner.sigma_rules_path +│ ├── yara/ # .yar -> scanner.yara_rules_path +│ └── ioc/ # hashes.txt / ips.txt / domains.txt / paths_regex.txt ├── windows-essential-.zip # Rustinel-compatible artifact └── ... ``` What the build does for each pack: -1. **Resolves** the cumulative rule list (`extends` + `rules`, de-duplicated). -2. **Copies** each Sigma/YARA file into the flat `rules/sigma` and `rules/yara` folders. +1. **Resolves** the cumulative rule list (`extends` + `rules` with `has`/`includes`/`excludes` or pack subfolders, de-duplicated). +2. **Copies** each Sigma/YARA file into the flat `sigma` and `yara` folders. 3. **Flattens** every referenced IOC set into the four per-type files in `VALUE;COMMENT` format, prefixing each line with its source set id (`[ioc-…]`) so provenance survives into alerts. 4. **Writes** a cleaned `pack.yml` with build metadata and zips the folder. @@ -131,12 +132,12 @@ automatically: "sha256": "…", "artifact": "windows-essential-0.2.0.zip", "engine": { - "sigma_rules_path": "windows-essential/rules/sigma", - "yara_rules_path": "windows-essential/rules/yara", - "hashes_path": "windows-essential/rules/ioc/hashes.txt", - "ips_path": "windows-essential/rules/ioc/ips.txt", - "domains_path": "windows-essential/rules/ioc/domains.txt", - "paths_regex_path": "windows-essential/rules/ioc/paths_regex.txt" + "sigma_rules_path": "windows-essential/sigma", + "yara_rules_path": "windows-essential/yara", + "hashes_path": "windows-essential/ioc/hashes.txt", + "ips_path": "windows-essential/ioc/ips.txt", + "domains_path": "windows-essential/ioc/domains.txt", + "paths_regex_path": "windows-essential/ioc/paths_regex.txt" } } ``` diff --git a/docs/usage.md b/docs/usage.md index efabd3e..c490fb6 100644 --- a/docs/usage.md +++ b/docs/usage.md @@ -18,11 +18,13 @@ dist/ ├── index.json ├── windows-essential/ # <- drop-in folder │ ├── pack.yml -│ └── rules/{sigma,yara,ioc}/ -└── windows-essential-0.2.0.zip # <- distributable artifact +│ ├── sigma/ +│ ├── yara/ +│ └── ioc/ +└── windows-essential-0.1.0.zip # <- distributable artifact ``` -Pass `--version X.Y.Z` to `build_packs.py` to stamp a release version (default `0.2.0`). +Pass `--version X.Y.Z` to `build_packs.py` to stamp a release version (default `0.1.0`). ## 2. Point `config.toml` at the pack @@ -32,16 +34,16 @@ Use the paths from the pack's `engine` block in `dist/index.json` (shown here fo ```toml [scanner] sigma_enabled = true -sigma_rules_path = "windows-essential/rules/sigma" +sigma_rules_path = "windows-essential/sigma" yara_enabled = true -yara_rules_path = "windows-essential/rules/yara" +yara_rules_path = "windows-essential/yara" [ioc] enabled = true -hashes_path = "windows-essential/rules/ioc/hashes.txt" -ips_path = "windows-essential/rules/ioc/ips.txt" -domains_path = "windows-essential/rules/ioc/domains.txt" -paths_regex_path = "windows-essential/rules/ioc/paths_regex.txt" +hashes_path = "windows-essential/ioc/hashes.txt" +ips_path = "windows-essential/ioc/ips.txt" +domains_path = "windows-essential/ioc/domains.txt" +paths_regex_path = "windows-essential/ioc/paths_regex.txt" ``` > Swap `windows-essential` for any other pack id (`windows-advanced`, `linux-essential`, …). Because @@ -67,11 +69,8 @@ Rebuild a pack into the same location and the engine picks it up without a resta ## `config.toml` reference (rule-relevant sections) -This is the pack-installer's subset of the engine config — only the keys that affect loading and -matching detection content. The **full, canonical reference** (logging, alerts, network -aggregation, every default) lives in the engine docs: -[Configuration](https://docs.rustinel.io/configuration/). Defaults shown here are the engine -defaults; only the paths above are required to load a pack, the rest are tuning knobs. +Defaults shown are the engine defaults. Only the paths above are required to load a pack; the rest +are tuning knobs. ### `[scanner]` — Sigma & YARA @@ -115,7 +114,6 @@ defaults; only the paths above are required to load a pack, the rest are tuning | --- | ------- | ------- | | `paths` | OS-shipped dirs (e.g. `C:\Windows\`, `/usr/bin/`, `/System/`) | Trusted prefixes applied to YARA, hash IOC, and response. Per-module allowlists fall back to this. | - ### `[response]` — optional active response | Key | Default | Meaning | @@ -145,3 +143,4 @@ defaults; only the paths above are required to load a pack, the rest are tuning For the engine itself (install, run as a service/daemon, telemetry setup), see the [Rustinel documentation](https://docs.rustinel.io/). + diff --git a/packs/linux/advanced/pack.yml b/packs/linux/advanced/pack.yml index b68f58d..c5b9457 100644 --- a/packs/linux/advanced/pack.yml +++ b/packs/linux/advanced/pack.yml @@ -1,12 +1,12 @@ name: Linux Advanced id: linux-advanced description: > - Linux Essential plus broader production detections. More false positives may - occur than in Essential (notably from package installs); tune per environment - before relying on by default. + Linux Essential plus broader production detections. More false positives may + occur than in Essential (notably from package installs); tune per environment + before relying on by default. os: linux level: advanced -pack_schema_version: 1 +pack_schema_version: 2 requires_rustinel: ">=1.0.2" default: false expected_false_positive_level: medium @@ -27,10 +27,13 @@ telemetry_requirements: - file_scan test_status: none sources: - - https://attack.mitre.org/ + manual: + - https://attack.mitre.org/ rules: - - 3d4e5f60-7182-4394-9da5-2e3f4a5b6c73 # Systemd Unit Persistence (sigma) - - 4e5f6071-8293-44a5-8eb6-3f4a5b6c7d84 # Cron Job Persistence (sigma) - - 5f607182-93a4-45b6-9fc7-4a5b6c7d8e95 # Shell Profile / RC File Persistence (sigma) - - 60718293-a4b5-46c7-8fd8-5b6c7d8e9fa6 # Execution from World-Writable / Temporary Directory (sigma) - - 94fb53c7-debd-4287-99e4-6eb0ba923731 # Linux Download and Execute Piped to Shell (sigma) + has: + sigma: + - 3d4e5f60-7182-4394-9da5-2e3f4a5b6c73 # Systemd Unit Persistence (sigma) + - 4e5f6071-8293-44a5-8eb6-3f4a5b6c7d84 # Cron Job Persistence (sigma) + - 5f607182-93a4-45b6-9fc7-4a5b6c7d8e95 # Shell Profile / RC File Persistence (sigma) + - 60718293-a4b5-46c7-8fd8-5b6c7d8e9fa6 # Execution from World-Writable / Temporary Directory (sigma) + - 94fb53c7-debd-4287-99e4-6eb0ba923731 # Linux Download and Execute Piped to Shell (sigma) diff --git a/packs/linux/essential/pack.yml b/packs/linux/essential/pack.yml index 1aae8df..0cbc94a 100644 --- a/packs/linux/essential/pack.yml +++ b/packs/linux/essential/pack.yml @@ -3,7 +3,7 @@ id: linux-essential description: Low-noise, high-confidence Linux detections for Rustinel. Safe default pack. os: linux level: essential -pack_schema_version: 1 +pack_schema_version: 2 requires_rustinel: ">=1.0.2" default: true expected_false_positive_level: low @@ -25,13 +25,18 @@ telemetry_requirements: - file_scan test_status: none sources: - - https://attack.mitre.org/ + manual: + - https://attack.mitre.org/ rules: - - 0a1b2c3d-4e5f-4061-8a72-9b0c1d2e3f40 # Dynamic Linker Hijacking via ld.so.preload (sigma) - - 1b2c3d4e-5f60-4172-9b83-0c1d2e3f4a51 # SSH authorized_keys Created or Replaced (sigma) - - 2c3d4e5f-6071-4283-8c94-1d2e3f4a5b62 # Sudoers Configuration Tampering (sigma) - - a3c5c821-004a-4e52-8684-0f7f9ea0404c # Linux Reverse Shell via /dev/tcp (sigma) - - 8fd4d1d9-38cf-4704-8009-00e41a009c98 # Linux Web Server Spawning Interactive Shell (sigma) - - 9ac8732e-1818-4cda-89ac-01ca08a7b836 # SSH Daemon Configuration Tampering (sigma) - - yara-lnx-xmrig-coinminer # XMRig / coinminer strings in Linux ELF binaries (yara) - - ioc-eicar-test # EICAR safe end-to-end IOC test set (ioc) + has: + sigma: + - 0a1b2c3d-4e5f-4061-8a72-9b0c1d2e3f40 # Dynamic Linker Hijacking via ld.so.preload (sigma) + - 1b2c3d4e-5f60-4172-9b83-0c1d2e3f4a51 # SSH authorized_keys Created or Replaced (sigma) + - 2c3d4e5f-6071-4283-8c94-1d2e3f4a5b62 # Sudoers Configuration Tampering (sigma) + - a3c5c821-004a-4e52-8684-0f7f9ea0404c # Linux Reverse Shell via /dev/tcp (sigma) + - 8fd4d1d9-38cf-4704-8009-00e41a009c98 # Linux Web Server Spawning Interactive Shell (sigma) + - 9ac8732e-1818-4cda-89ac-01ca08a7b836 # SSH Daemon Configuration Tampering (sigma) + yara: + - yara-lnx-xmrig-coinminer # XMRig / coinminer strings in Linux ELF binaries (yara) + ioc: + - ioc-eicar-test # EICAR safe end-to-end IOC test set (ioc) # EICAR safe end-to-end IOC test set (ioc) # EICAR safe end-to-end IOC test set (ioc) diff --git a/packs/macos/advanced/pack.yml b/packs/macos/advanced/pack.yml index 7250ef0..35d6214 100644 --- a/packs/macos/advanced/pack.yml +++ b/packs/macos/advanced/pack.yml @@ -1,14 +1,14 @@ name: macOS Advanced id: macos-advanced description: > - macOS Essential plus broader detections: launch-item persistence, shell - download cradles, privileged account creation and execution from temporary - paths. More false positives may occur — notably from application installers — - so tune per environment before relying on by default. Experimental and - post-v1; disabled by default. + macOS Essential plus broader detections: launch-item persistence, shell + download cradles, privileged account creation and execution from temporary + paths. More false positives may occur — notably from application installers — + so tune per environment before relying on by default. Experimental and + post-v1; disabled by default. os: macos level: advanced -pack_schema_version: 1 +pack_schema_version: 2 requires_rustinel: ">=1.0.2" default: false expected_false_positive_level: medium @@ -29,9 +29,12 @@ telemetry_requirements: - file_event test_status: none sources: - - https://attack.mitre.org/ + manual: + - https://attack.mitre.org/ rules: - - 4d4f5162-7e80-4193-9da3-4e5f60712304 # Launch Agent or Daemon Persistence Plist Created (sigma) - - 3e506273-8f91-42a4-8eb4-5f6071823405 # Shell Download-and-Execute Pipe Cradle (sigma) - - 2f617384-9012-43b5-9fc5-60718293a416 # Local Admin Account Created via Directory Services (sigma) - - 1a728495-a123-44c6-8ad6-718293a4b527 # Execution from World-Writable or Temporary Directory (sigma) + has: + sigma: + - 4d4f5162-7e80-4193-9da3-4e5f60712304 # Launch Agent or Daemon Persistence Plist Created (sigma) + - 3e506273-8f91-42a4-8eb4-5f6071823405 # Shell Download-and-Execute Pipe Cradle (sigma) + - 2f617384-9012-43b5-9fc5-60718293a416 # Local Admin Account Created via Directory Services (sigma) + - 1a728495-a123-44c6-8ad6-718293a4b527 # Execution from World-Writable or Temporary Directory (sigma) diff --git a/packs/macos/essential/pack.yml b/packs/macos/essential/pack.yml index 0c94b52..31f9214 100644 --- a/packs/macos/essential/pack.yml +++ b/packs/macos/essential/pack.yml @@ -1,13 +1,13 @@ name: macOS Essential id: macos-essential description: > - Low-noise, high-confidence macOS detections for Rustinel, focused on the - dominant macOS threats: infostealers, keychain theft, Gatekeeper bypass and - cryptominers. Experimental and post-v1 — macOS packs are not yet - production-ready, so this pack is disabled by default. + Low-noise, high-confidence macOS detections for Rustinel, focused on the + dominant macOS threats: infostealers, keychain theft, Gatekeeper bypass and + cryptominers. Experimental and post-v1 — macOS packs are not yet + production-ready, so this pack is disabled by default. os: macos level: essential -pack_schema_version: 1 +pack_schema_version: 2 requires_rustinel: ">=1.0.2" default: false expected_false_positive_level: low @@ -28,11 +28,16 @@ telemetry_requirements: - file_scan test_status: none sources: - - https://attack.mitre.org/ + manual: + - https://attack.mitre.org/ rules: - - 7a1c2e3f-4b5d-4e6f-8a90-1b2c3d4e5f01 # Keychain Credential Dump via security (sigma) - - 6b2d3f40-5c6e-4f71-9b81-2c3d4e5f6012 # osascript Credential Prompt or Suspicious Admin Shell (sigma) - - 5c3e4051-6d7f-4082-8c92-3d4e5f601203 # Gatekeeper or Quarantine Protection Disabled (sigma) - - be06a6b0-fa9f-4df4-9d4a-9f8794037508 # macOS Reverse Shell via /dev/tcp (sigma) - - yara-macos-coinminer-strings # Cryptominer Mach-O strings (yara) - - ioc-eicar-test # EICAR safe end-to-end IOC test set (ioc) + has: + sigma: + - 7a1c2e3f-4b5d-4e6f-8a90-1b2c3d4e5f01 # Keychain Credential Dump via security (sigma) + - 6b2d3f40-5c6e-4f71-9b81-2c3d4e5f6012 # osascript Credential Prompt or Suspicious Admin Shell (sigma) + - 5c3e4051-6d7f-4082-8c92-3d4e5f601203 # Gatekeeper or Quarantine Protection Disabled (sigma) + - be06a6b0-fa9f-4df4-9d4a-9f8794037508 # macOS Reverse Shell via /dev/tcp (sigma) + yara: + - yara-macos-coinminer-strings # Cryptominer Mach-O strings (yara) + ioc: + - ioc-eicar-test # EICAR safe end-to-end IOC test set (ioc) # EICAR safe end-to-end IOC test set (ioc) # EICAR safe end-to-end IOC test set (ioc) diff --git a/packs/windows/advanced/pack.yml b/packs/windows/advanced/pack.yml index 0ec4825..de16f60 100644 --- a/packs/windows/advanced/pack.yml +++ b/packs/windows/advanced/pack.yml @@ -1,11 +1,11 @@ name: Windows Advanced id: windows-advanced description: > - Windows Essential plus broader production detections. More false positives may - occur than in Essential; tune per environment before relying on by default. + Windows Essential plus broader production detections. More false positives may + occur than in Essential; tune per environment before relying on by default. os: windows level: advanced -pack_schema_version: 1 +pack_schema_version: 2 requires_rustinel: ">=1.0.2" default: false expected_false_positive_level: medium @@ -30,13 +30,16 @@ telemetry_requirements: - service_creation test_status: manual sources: - - https://attack.mitre.org/ - - https://lolbas-project.github.io/ + manual: + - https://attack.mitre.org/ + - https://lolbas-project.github.io/ rules: - - 9c8b7a6e-1d2f-4a3b-9c0d-2e3f4a5b6c71 # Rundll32 Execution Without Standard Arguments - - 6f0d2a91-3b7c-4e58-9a16-8c4e1d7b2a09 # Local Account Created or Added to Administrators - - a1b2c3d4-7e8f-4012-9a3b-4c5d6e7f0a10 # Registry Run Key Persistence - - c3d4e5f6-9012-4234-9c5d-6e7f8a901a12 # PowerShell Download-and-Execute Cradle - - d4e5f6a7-0123-4345-9d6e-7f8a9b012a13 # Suspicious Service Binary Path - - 4a596825-4674-4db3-b360-842435edf11a # Scheduled Task Creation via Schtasks - - 2a3c3182-80b8-42d4-bd39-83c55582ef70 # WMI Process Execution via WMIC + has: + sigma: + - 9c8b7a6e-1d2f-4a3b-9c0d-2e3f4a5b6c71 # Rundll32 Execution Without Standard Arguments + - 6f0d2a91-3b7c-4e58-9a16-8c4e1d7b2a09 # Local Account Created or Added to Administrators + - a1b2c3d4-7e8f-4012-9a3b-4c5d6e7f0a10 # Registry Run Key Persistence + - c3d4e5f6-9012-4234-9c5d-6e7f8a901a12 # PowerShell Download-and-Execute Cradle + - d4e5f6a7-0123-4345-9d6e-7f8a9b012a13 # Suspicious Service Binary Path + - 4a596825-4674-4db3-b360-842435edf11a # Scheduled Task Creation via Schtasks + - 2a3c3182-80b8-42d4-bd39-83c55582ef70 # WMI Process Execution via WMIC diff --git a/packs/windows/essential/pack.yml b/packs/windows/essential/pack.yml index 9ec1398..d67481c 100644 --- a/packs/windows/essential/pack.yml +++ b/packs/windows/essential/pack.yml @@ -3,7 +3,7 @@ id: windows-essential description: Low-noise, high-confidence Windows detections for Rustinel. Safe default pack. os: windows level: essential -pack_schema_version: 1 +pack_schema_version: 2 requires_rustinel: ">=1.0.2" default: true expected_false_positive_level: low @@ -33,21 +33,26 @@ telemetry_requirements: - file_scan test_status: mixed sources: - - https://attack.mitre.org/ - - https://lolbas-project.github.io/ + manual: + - https://attack.mitre.org/ + - https://lolbas-project.github.io/ rules: - - 7f3a1c2e-4b5d-4e6f-8a90-1b2c3d4e5f60 # Suspicious Encoded PowerShell Command Line (sigma) - - 7d2a8f31-4b6c-49e0-a1f8-3c5d9e0b2a05 # LSASS Memory Dump via comsvcs.dll MiniDump (sigma) - - 1e9b4d68-2c7a-4f93-8b15-6d0a2e4c1b04 # Volume Shadow Copy Deletion (sigma) - - 9a0c3e75-6d8b-4f12-bc34-1e7a5d9c0b06 # Regsvr32 Remote Scriptlet Execution / Squiblydoo (sigma) - - 3c6f9b22-7e1a-4d85-9c0b-2a8d4e6f1b07 # Mshta Remote or Inline Script Execution (sigma) - - c4a7f2e9-5d3b-4810-a6c1-9e0f3b7d2a03 # Office Application Spawning a Command Shell (sigma) - - 5e8b1d44-9c2a-4f67-8d03-7b1a6e3c9d08 # Windows Event Log Cleared via Command Line (sigma) - - b2c3d4e5-8f90-4123-8b4c-5d6e7f801a11 # Microsoft Defender Tampering via Registry (sigma) - - e7a1b9c2-4d6f-4a83-b2e5-1c0d9f3a4b20 # Registry Hive Dump via reg.exe save (sigma) - - f2b8c4d1-6e90-4a27-9d3b-5a1c2e7f8b21 # UAC Bypass via Auto-Elevating LOLBin (sigma) - - a3c7e9b4-2f81-4d56-8c0a-6b2d1f4e9c22 # Active Directory Database (NTDS.dit) Extraction (sigma) - - b4d8f0c5-3a92-4e67-9d1b-7c3e2f5a0d23 # WDigest Cleartext Credential Caching Enabled (sigma) - - 3a1cdc64-a2a1-444b-a04b-32a3f1d09a99 # Scheduled Task Created With Suspicious Action (sigma) - - yara-win-mimikatz-strings # Mimikatz credential-dumping strings (yara) - - ioc-eicar-test # EICAR safe end-to-end IOC test set (ioc) + has: + sigma: + - 7f3a1c2e-4b5d-4e6f-8a90-1b2c3d4e5f60 # Suspicious Encoded PowerShell Command Line (sigma) # Suspicious Encoded PowerShell Command Line + - 7d2a8f31-4b6c-49e0-a1f8-3c5d9e0b2a05 # LSASS Memory Dump via comsvcs.dll MiniDump (sigma) + - 1e9b4d68-2c7a-4f93-8b15-6d0a2e4c1b04 # Volume Shadow Copy Deletion (sigma) + - 9a0c3e75-6d8b-4f12-bc34-1e7a5d9c0b06 # Regsvr32 Remote Scriptlet Execution / Squiblydoo (sigma) + - 3c6f9b22-7e1a-4d85-9c0b-2a8d4e6f1b07 # Mshta Remote or Inline Script Execution (sigma) + - c4a7f2e9-5d3b-4810-a6c1-9e0f3b7d2a03 # Office Application Spawning a Command Shell (sigma) + - 5e8b1d44-9c2a-4f67-8d03-7b1a6e3c9d08 # Windows Event Log Cleared via Command Line (sigma) + - b2c3d4e5-8f90-4123-8b4c-5d6e7f801a11 # Microsoft Defender Tampering via Registry (sigma) + - e7a1b9c2-4d6f-4a83-b2e5-1c0d9f3a4b20 # Registry Hive Dump via reg.exe save (sigma) + - f2b8c4d1-6e90-4a27-9d3b-5a1c2e7f8b21 # UAC Bypass via Auto-Elevating LOLBin (sigma) + - a3c7e9b4-2f81-4d56-8c0a-6b2d1f4e9c22 # Active Directory Database (NTDS.dit) Extraction (sigma) + - b4d8f0c5-3a92-4e67-9d1b-7c3e2f5a0d23 # WDigest Cleartext Credential Caching Enabled (sigma) + - 3a1cdc64-a2a1-444b-a04b-32a3f1d09a99 # Scheduled Task Created With Suspicious Action (sigma) + yara: + - yara-win-mimikatz-strings # Mimikatz credential-dumping strings (yara) + ioc: + - ioc-eicar-test # EICAR safe end-to-end IOC test set (ioc) # EICAR safe end-to-end IOC test set (ioc) # EICAR safe end-to-end IOC test set (ioc) diff --git a/packs/windows/hunting/pack.yml b/packs/windows/hunting/pack.yml index c0530c9..330e443 100644 --- a/packs/windows/hunting/pack.yml +++ b/packs/windows/hunting/pack.yml @@ -1,12 +1,12 @@ name: Windows Hunting id: windows-hunting description: > - Windows Advanced plus broad, noisier hunting content intended for analyst-driven - investigation. Not enabled by default and not suitable as a standing alert source - without tuning. + Windows Advanced plus broad, noisier hunting content intended for analyst-driven + investigation. Not enabled by default and not suitable as a standing alert source + without tuning. os: windows level: hunting -pack_schema_version: 1 +pack_schema_version: 2 requires_rustinel: ">=1.0.2" default: false expected_false_positive_level: high @@ -20,7 +20,10 @@ telemetry_requirements: - process_creation test_status: none sources: - - https://attack.mitre.org/ - - https://lolbas-project.github.io/ + manual: + - https://attack.mitre.org/ + - https://lolbas-project.github.io/ rules: - - 3e2d1c0b-5a4f-4938-8271-6a5b4c3d2e10 # Certutil Used to Download Remote Content (Hunting) + has: + sigma: + - 3e2d1c0b-5a4f-4938-8271-6a5b4c3d2e10 # Certutil Used to Download Remote Content (Hunting) diff --git a/pyproject.toml b/pyproject.toml index 7b28506..cc07181 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -8,6 +8,7 @@ dependencies = [ "pyyaml>=6.0", "jsonschema>=4.0", "yara-x>=1.0", + "ruamel.yaml>=0.18.0", ] [dependency-groups] @@ -21,6 +22,7 @@ dev = [ # backend, no install of this project itself). [tool.uv] package = false +exclude-newer = "3 days" # Prevent supply-chain attacks by excluding recently published versions of dependencies. [tool.ruff] line-length = 100 diff --git a/schemas/pack.schema.json b/schemas/pack.schema.json index 6b1bf5a..3cf6f1c 100644 --- a/schemas/pack.schema.json +++ b/schemas/pack.schema.json @@ -1,7 +1,7 @@ { "$schema": "http://json-schema.org/draft-07/schema#", "$id": "https://github.com/rustinel-rules/schemas/pack.schema.json", - "title": "Rustinel pack.yml (v1)", + "title": "Rustinel pack.yml (v2)", "description": "Schema for a rustinel-rules pack manifest. Packs reference canonical rules by id and may extend lower-level packs cumulatively (Essential ⊂ Advanced ⊂ Hunting).", "type": "object", "additionalProperties": false, @@ -15,8 +15,7 @@ "requires_rustinel", "default", "status", - "extends", - "rules" + "extends" ], "properties": { "name": { @@ -43,7 +42,7 @@ }, "pack_schema_version": { "type": "integer", - "const": 1 + "const": 2 }, "requires_rustinel": { "type": "string", @@ -69,15 +68,102 @@ "uniqueItems": true }, "rules": { - "type": "array", - "description": "Rule ids (Sigma UUIDs or YARA rule names) added by THIS pack on top of any it extends.", - "items": { "type": "string" }, - "uniqueItems": true + "type": "object", + "description": "Rules included, inherited, or excluded by this pack.", + "additionalProperties": false, + "properties": { + "has": { + "type": "object", + "description": "Rules directly included by this pack.", + "additionalProperties": false, + "properties": { + "sigma": { + "type": "array", + "items": { "type": "string" }, + "uniqueItems": true + }, + "yara": { + "type": "array", + "items": { "type": "string" }, + "uniqueItems": true + }, + "ioc": { + "type": "array", + "items": { "type": "string" }, + "uniqueItems": true + } + } + }, + "includes": { + "type": "object", + "description": "Rules that should be included inside this pack from the packs that it is extending.", + "additionalProperties": false, + "properties": { + "sigma": { + "type": "array", + "items": { "type": "string" }, + "uniqueItems": true + }, + "yara": { + "type": "array", + "items": { "type": "string" }, + "uniqueItems": true + }, + "ioc": { + "type": "array", + "items": { "type": "string" }, + "uniqueItems": true + } + } + }, + "excludes": { + "type": "object", + "description": "Rules that should not be taken from an extended pack, or ignored if loading from automatic sources.", + "additionalProperties": false, + "properties": { + "sigma": { + "type": "array", + "items": { "type": "string" }, + "uniqueItems": true + }, + "yara": { + "type": "array", + "items": { "type": "string" }, + "uniqueItems": true + }, + "ioc": { + "type": "array", + "items": { "type": "string" }, + "uniqueItems": true + } + } + } + } }, "sources": { - "type": "array", + "type": "object", "description": "Upstream / CTI references that informed this pack.", - "items": { "type": "string" } + "additionalProperties": false, + "properties": { + "manual": { + "type": "array", + "description": "Rules/sources that were extracted manually.", + "items": { "type": "string" }, + "uniqueItems": true + }, + "sigma": { + "type": "array", + "description": "Sigma sources that can be cloned with git and copied automatically.", + "items": { "type": "string" }, + "uniqueItems": true + }, + "yara": { + "type": "array", + "description": "YARA sources that can be cloned with git and copied automatically.", + "items": { "type": "string" }, + "uniqueItems": true + } + } }, "license": { "type": "string", diff --git a/tests/atomic/run_atomics.py b/tests/atomic/run_atomics.py index 75dc9a7..967cda6 100644 --- a/tests/atomic/run_atomics.py +++ b/tests/atomic/run_atomics.py @@ -222,16 +222,16 @@ def setup_engine(engine_dir: Path, dist_dir: Path, pack: dict) -> Path: config = f"""# Generated by run_atomics.py - points the engine at pack '{pack_id}'. [scanner] sigma_enabled = true -sigma_rules_path = "{pack_id}/rules/sigma" +sigma_rules_path = "{pack_id}/sigma" yara_enabled = true -yara_rules_path = "{pack_id}/rules/yara" +yara_rules_path = "{pack_id}/yara" [ioc] enabled = true -hashes_path = "{pack_id}/rules/ioc/hashes.txt" -ips_path = "{pack_id}/rules/ioc/ips.txt" -domains_path = "{pack_id}/rules/ioc/domains.txt" -paths_regex_path = "{pack_id}/rules/ioc/paths_regex.txt" +hashes_path = "{pack_id}/ioc/hashes.txt" +ips_path = "{pack_id}/ioc/ips.txt" +domains_path = "{pack_id}/ioc/domains.txt" +paths_regex_path = "{pack_id}/ioc/paths_regex.txt" [logging] level = "info" diff --git a/tools/build_packs.py b/tools/build_packs.py index 82245c3..5e4e0a0 100644 --- a/tools/build_packs.py +++ b/tools/build_packs.py @@ -4,12 +4,12 @@ For each pack manifest this produces, under dist/: - dist// flat materialized pack folder == what the pack.yml Rustinel engine loads directly: - rules/sigma/.yml -> scanner.sigma_rules_path - rules/yara/.yar -> scanner.yara_rules_path - rules/ioc/hashes.txt -> [ioc].hashes_path - rules/ioc/ips.txt -> [ioc].ips_path - rules/ioc/domains.txt -> [ioc].domains_path - rules/ioc/paths_regex.txt -> [ioc].paths_regex_path + sigma/.yml -> scanner.sigma_rules_path + yara/.yar -> scanner.yara_rules_path + ioc/hashes.txt -> [ioc].hashes_path + ioc/ips.txt -> [ioc].ips_path + ioc/domains.txt -> [ioc].domains_path + ioc/paths_regex.txt -> [ioc].paths_regex_path - dist/-.zip zipped pack (one folder = one pack = one zip) - dist/index.json catalog of all packs + checksums + engine paths @@ -77,8 +77,9 @@ def materialize_pack(pack: dict, by_id, artifact_index, version: str) -> dict: out_dir = lib.DIST_DIR / pack_id if out_dir.exists(): shutil.rmtree(out_dir) - (out_dir / "rules" / "sigma").mkdir(parents=True, exist_ok=True) - (out_dir / "rules" / "yara").mkdir(parents=True, exist_ok=True) + (out_dir / "sigma").mkdir(parents=True, exist_ok=True) + (out_dir / "yara").mkdir(parents=True, exist_ok=True) + (out_dir / "ioc").mkdir(parents=True, exist_ok=True) resolved_ids = lib.resolve_pack_rules(pack, by_id) copied = [] @@ -91,11 +92,10 @@ def materialize_pack(pack: dict, by_id, artifact_index, version: str) -> dict: f"[build] pack '{pack_id}' references unknown artifact '{artifact_id}'" ) - if art.kind in ("sigma", "yara"): - dest = out_dir / "rules" / art.kind / art.path.name - shutil.copy2(art.path, dest) - copied.append({"id": artifact_id, "kind": art.kind, "file": art.path.name}) - elif art.kind == "ioc": + dest = out_dir / art.kind / art.path.name + dest.parent.mkdir(parents=True, exist_ok=True) + shutil.copy2(art.path, dest) + if art.kind == "ioc": count = 0 for ioc_type, entries in art.indicators.items(): for value, comment in entries: @@ -103,9 +103,13 @@ def materialize_pack(pack: dict, by_id, artifact_index, version: str) -> dict: full = f"[{art.id}] {comment}" if comment else f"[{art.id}]" ioc_acc[ioc_type].append((value, full)) count += 1 - copied.append({"id": artifact_id, "kind": "ioc", "indicators": count}) + copied.append( + {"id": artifact_id, "kind": "ioc", "file": art.path.name, "indicators": count} + ) + else: + copied.append({"id": artifact_id, "kind": art.kind, "file": art.path.name}) - ioc_counts = write_ioc_files(out_dir / "rules" / "ioc", ioc_acc) + ioc_counts = write_ioc_files(out_dir / "ioc", ioc_acc) # Write a clean manifest (drop internal __path__) plus build metadata. manifest = {k: v for k, v in pack.items() if not k.startswith("__")} @@ -134,6 +138,7 @@ def materialize_pack(pack: dict, by_id, artifact_index, version: str) -> dict: "version": version, "default": manifest.get("default", False), "requires_rustinel": manifest.get("requires_rustinel"), + "schema": manifest.get("schema"), "pack_schema_version": manifest.get("pack_schema_version"), "status": manifest.get("status"), "expected_false_positive_level": manifest.get("expected_false_positive_level"), @@ -146,12 +151,12 @@ def materialize_pack(pack: dict, by_id, artifact_index, version: str) -> dict: "sha256": sha256_file(zip_path), # Drop-in paths for Rustinel config.toml (relative to dist/). "engine": { - "sigma_rules_path": f"{pack_id}/rules/sigma", - "yara_rules_path": f"{pack_id}/rules/yara", - "hashes_path": f"{pack_id}/rules/ioc/hashes.txt", - "ips_path": f"{pack_id}/rules/ioc/ips.txt", - "domains_path": f"{pack_id}/rules/ioc/domains.txt", - "paths_regex_path": f"{pack_id}/rules/ioc/paths_regex.txt", + "sigma_rules_path": f"{pack_id}/sigma", + "yara_rules_path": f"{pack_id}/yara", + "hashes_path": f"{pack_id}/ioc/hashes.txt", + "ips_path": f"{pack_id}/ioc/ips.txt", + "domains_path": f"{pack_id}/ioc/domains.txt", + "paths_regex_path": f"{pack_id}/ioc/paths_regex.txt", }, } diff --git a/tools/lib.py b/tools/lib.py index 0d5a105..afc8d9d 100644 --- a/tools/lib.py +++ b/tools/lib.py @@ -9,9 +9,9 @@ content. A pack materializes (see ``build_packs.py``) into exactly the layout the Rustinel engine loads: - rules/sigma/ recursive dir of Sigma .yml -> scanner.sigma_rules_path - rules/yara/ recursive dir of YARA .yar -> scanner.yara_rules_path - rules/ioc/hashes.txt ips.txt domains.txt paths_regex.txt + sigma/ recursive dir of Sigma .yml -> scanner.sigma_rules_path + yara/ recursive dir of YARA .yar -> scanner.yara_rules_path + ioc/hashes.txt ips.txt domains.txt paths_regex.txt -> [ioc].*_path Requires PyYAML (see tools/requirements.txt). @@ -188,27 +188,160 @@ def packs_by_id() -> dict[str, dict]: return {p["id"]: p for p in load_packs() if "id" in p} +def _get_rule_ids_from_dict(sub_dict: dict | list | None) -> list[str]: + if not sub_dict: + return [] + if isinstance(sub_dict, list): + return [str(item) for item in sub_dict] + ids = [] + for category in ("sigma", "yara", "ioc"): + for item in sub_dict.get(category) or []: + ids.append(str(item)) + return ids + + +def get_pack_subfolder_rules(pack: dict) -> dict[str, list[str]]: + """Scan the pack's directories and return a dict of {category: [rule_ids]}.""" + result = {"sigma": [], "yara": [], "ioc": []} + pack_path_str = pack.get("__path__") + if not pack_path_str: + return result + pack_dir = Path(pack_path_str).parent + + # Build a map of filename to artifact ID from canonical rules + filename_to_id = {} + for art in load_all_artifacts(): + if art.id: + filename_to_id[(art.kind, art.path.name)] = art.id + + # 1. Sigma + sigma_dir = pack_dir / "sigma" + if sigma_dir.is_dir(): + for file in sorted(sigma_dir.glob("*")): + if file.is_file() and file.suffix in (".yml", ".yaml"): + if ("sigma", file.name) in filename_to_id: + result["sigma"].append(filename_to_id[("sigma", file.name)]) + else: + try: + doc = yaml.safe_load(file.read_text(encoding="utf-8")) or {} + rule_id = str(doc.get("id", "")).strip() + if rule_id: + result["sigma"].append(rule_id) + except Exception: + pass + + # 2. Yara + yara_dir = pack_dir / "yara" + if yara_dir.is_dir(): + for file in sorted(yara_dir.glob("*")): + if file.is_file() and file.suffix in (".yar", ".yara"): + if ("yara", file.name) in filename_to_id: + result["yara"].append(filename_to_id[("yara", file.name)]) + else: + try: + raw = file.read_text(encoding="utf-8") + meta_id = _YARA_META_ID_RE.search(raw) + name = _YARA_RULE_RE.search(raw) + rule_id = ( + meta_id.group(1) if meta_id else name.group(1) if name else "" + ).strip() + if rule_id: + result["yara"].append(rule_id) + except Exception: + pass + + # 3. IOC + ioc_dir = pack_dir / "ioc" + if ioc_dir.is_dir(): + ioc_rule_ids = set() + for file in ioc_dir.glob("*.txt"): + try: + content = file.read_text(encoding="utf-8") + for match in re.finditer(r"\brule=([a-zA-Z0-9_-]+)", content): + ioc_rule_ids.add(match.group(1)) + except Exception: + pass + result["ioc"] = sorted(list(ioc_rule_ids)) + + return result + + def resolve_pack_rules(pack: dict, by_id: dict[str, dict]) -> list[str]: """Return the ordered, de-duplicated artifact ids for a pack, including all transitively extended packs. Lower-level packs come first. + Applies the dictionary-based 'includes' and 'excludes' filtering. + Authoritative rules are loaded from the pack's rules subfolder if present, + otherwise falling back to 'has' under rules in the pack manifest. + Raises ValueError on a missing extends target or an extends cycle. """ - ordered: list[str] = [] - seen: set = set() - def visit(pack_id: str, stack: list[str]): + def visit(pack_id: str, stack: list[str]) -> list[str]: if pack_id in stack: raise ValueError(f"extends cycle: {' -> '.join(stack + [pack_id])}") if pack_id not in by_id: raise ValueError(f"unknown pack in extends: {pack_id}") node = by_id[pack_id] + + # 1. Resolve parent rules recursively + extended_rules = [] + extended_seen = set() for parent in node.get("extends", []) or []: - visit(parent, stack + [pack_id]) - for rule_id in node.get("rules", []) or []: - if rule_id not in seen: - seen.add(rule_id) - ordered.append(rule_id) - - visit(pack["id"], []) - return ordered + parent_resolved = visit(parent, stack + [pack_id]) + for r in parent_resolved: + if r not in extended_seen: + extended_seen.add(r) + extended_rules.append(r) + + # 2. Extract rules dictionary from manifest + rules_dict = node.get("rules") or {} + + # If rules is a list (old format), treat it as 'has' list of rules + if isinstance(rules_dict, list): + has_dict_list = rules_dict + includes_list = [] + excludes_list = [] + else: + has_dict_list = _get_rule_ids_from_dict(rules_dict.get("has")) + includes_list = _get_rule_ids_from_dict(rules_dict.get("includes")) + excludes_list = _get_rule_ids_from_dict(rules_dict.get("excludes")) + + # 3. Filter extended rules using 'includes' and 'excludes' + # If 'includes' key is specified (or rules is list, where we don't have includes), + # filter extended rules to only those in the include list. + if not isinstance(rules_dict, list) and "includes" in rules_dict: + include_ids = set(includes_list) + filtered_extended = [r for r in extended_rules if r in include_ids] + else: + filtered_extended = list(extended_rules) + + # If 'excludes' key is specified (or in rules), filter them out. + exclude_ids = set(excludes_list) + filtered_extended = [r for r in filtered_extended if r not in exclude_ids] + + # 4. Get 'has' rules. + # Check subfolders (authoritative) first if any rules are there, + # otherwise use manifest's 'has'. + subfolder_rules = get_pack_subfolder_rules(node) + sub_rule_ids = _get_rule_ids_from_dict(subfolder_rules) + if sub_rule_ids: + has_rules = sub_rule_ids + else: + has_rules = has_dict_list + + has_rules = [r for r in has_rules if r not in exclude_ids] + + # Combine everything + combined = filtered_extended + has_rules + + # De-duplicate preserving order + resolved = [] + seen = set() + for r in combined: + if r not in seen: + seen.add(r) + resolved.append(r) + return resolved + + return visit(pack["id"], []) diff --git a/tools/validate.py b/tools/validate.py index 25a631d..5c4cff1 100644 --- a/tools/validate.py +++ b/tools/validate.py @@ -236,7 +236,6 @@ def check_ioc_set(art, rep: Report, schema_validate): "default", "status", "extends", - "rules", ] @@ -255,8 +254,8 @@ def check_packs(packs, artifacts, rep: Report): for field in REQUIRED_PACK_FIELDS: if field not in pack: rep.error(where, f"missing required field '{field}'") - if pack.get("pack_schema_version") != 1: - rep.error(where, "pack_schema_version must be 1 for v1") + if pack.get("pack_schema_version") != 2: + rep.error(where, "pack_schema_version must be 2 for v2") if not pack.get("license"): rep.warn(where, "no 'license' field on pack") @@ -264,7 +263,20 @@ def check_packs(packs, artifacts, rep: Report): schema_validate(pack, where, rep) # Referential integrity: rules and extends must resolve. - for rule_id in pack.get("rules", []) or []: + rules_dict = pack.get("rules") or {} + rule_ids_to_check = [] + if isinstance(rules_dict, list): + rule_ids_to_check = rules_dict + elif isinstance(rules_dict, dict): + for key in ("has", "includes", "excludes"): + sub = rules_dict.get(key) or {} + if isinstance(sub, dict): + for cat in ("sigma", "yara", "ioc"): + rule_ids_to_check.extend(sub.get(cat) or []) + elif isinstance(sub, list): + rule_ids_to_check.extend(sub) + + for rule_id in rule_ids_to_check: if rule_id not in artifact_index: rep.error(where, f"references unknown artifact id '{rule_id}'") try: diff --git a/uv.lock b/uv.lock index ec82543..14fa884 100644 --- a/uv.lock +++ b/uv.lock @@ -2,6 +2,10 @@ version = 1 revision = 3 requires-python = ">=3.12" +[options] +exclude-newer = "0001-01-01T00:00:00Z" # This has no effect and is included for backwards compatibility when using relative exclude-newer values. +exclude-newer-span = "P3D" + [[package]] name = "attrs" version = "26.1.0" @@ -208,6 +212,15 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/79/cb/966040123eb102371559746908ef2c9471f4d43e17ec9a645a2258dab64b/rpds_py-2026.5.1-cp315-cp315t-win_amd64.whl", hash = "sha256:90bd6630002a1c7f09e7843dd79f0d24f3d2897cc25a753480917865d14f15b3", size = 225441, upload-time = "2026-05-28T12:01:51.408Z" }, ] +[[package]] +name = "ruamel-yaml" +version = "0.19.1" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/c7/3b/ebda527b56beb90cb7652cb1c7e4f91f48649fbcd8d2eb2fb6e77cd3329b/ruamel_yaml-0.19.1.tar.gz", hash = "sha256:53eb66cd27849eff968ebf8f0bf61f46cdac2da1d1f3576dd4ccee9b25c31993", size = 142709, upload-time = "2026-01-02T16:50:31.84Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/b8/0c/51f6841f1d84f404f92463fc2b1ba0da357ca1e3db6b7fbda26956c3b82a/ruamel_yaml-0.19.1-py3-none-any.whl", hash = "sha256:27592957fedf6e0b62f281e96effd28043345e0e66001f97683aa9a40c667c93", size = 118102, upload-time = "2026-01-02T16:50:29.201Z" }, +] + [[package]] name = "ruff" version = "0.15.16" @@ -240,6 +253,7 @@ source = { virtual = "." } dependencies = [ { name = "jsonschema" }, { name = "pyyaml" }, + { name = "ruamel-yaml" }, { name = "yara-x" }, ] @@ -253,6 +267,7 @@ dev = [ requires-dist = [ { name = "jsonschema", specifier = ">=4.0" }, { name = "pyyaml", specifier = ">=6.0" }, + { name = "ruamel-yaml", specifier = ">=0.18.0" }, { name = "yara-x", specifier = ">=1.0" }, ]