Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 1 addition & 5 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,14 +8,12 @@ updates:
runtime-minors:
patterns: ["*"]
update-types: ["minor", "patch"]
open-pull-requests-limit: 10
open-pull-requests-limit: 5
rebase-strategy: "auto"
commit-message:
prefix: "😨"
labels:
- "python-dependencies"
cooldown:
default-days: 7

- package-ecosystem: "github-actions"
directory: "/"
Expand All @@ -31,6 +29,4 @@ updates:
prefix: "😨"
labels:
- "github-actions-dependencies"
cooldown:
default-days: 7

7 changes: 4 additions & 3 deletions .github/workflows/build_docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ jobs:
if: ${{ github.event_name == 'workflow_dispatch' || github.event.workflow_run.conclusion == 'success' }}
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v5
with:
fetch-depth: 0
persist-credentials: true
Expand All @@ -32,14 +32,15 @@ jobs:
VERSION=$(grep -m1 '^version' pyproject.toml | cut -d '"' -f2 | cut -d '.' -f 1,2)
echo "VERSION=$VERSION" >> "$GITHUB_ENV"

- uses: astral-sh/setup-uv@v7
- uses: astral-sh/setup-uv@v6

- name: Set up Python
run: uv python install ${{ vars.PYTHON_VERSION }}

- name: Set up venv
run: |
uv sync --group docs --no-default-groups
uv venv -p ${{ vars.PYTHON_VERSION }}
uv pip install '.[docs]'

- name: Run Mike build
run: |
Expand Down
7 changes: 4 additions & 3 deletions .github/workflows/doc_examples.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,8 @@ jobs:
RF_ASI_TOKEN: ${{ secrets.RF_ASI_TOKEN }}

steps:
- uses: actions/checkout@v6
- uses: astral-sh/setup-uv@v7
- uses: actions/checkout@v5
- uses: astral-sh/setup-uv@v6

- name: Install GNU parallel
run: sudo apt-get update && sudo apt-get install -y parallel
Expand All @@ -25,7 +25,8 @@ jobs:

- name: Set up venv
run: |
uv sync --group docs --no-default-groups
uv venv -p ${{ vars.PYTHON_VERSION }}
uv pip install '.[docs]'

- name: Run Examples
shell: bash
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/format.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ jobs:
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v5

- name: Run ruff check
uses: astral-sh/ruff-action@v3
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/import_checks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ jobs:
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v5
- name: Check imports
run: |
if matches=$(grep -RIn --line-number -E '^from[[:space:]]+psengine' psengine) ; then
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/lint.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ jobs:
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v5

- name: Run ruff check
uses: astral-sh/ruff-action@v3
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/typos.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,10 +16,10 @@ jobs:
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v5

- name: Spell Check Repo
uses: crate-ci/typos@v1.45.2
uses: crate-ci/typos@v1.35.5
with:
files: |
./psengine/
Expand Down
16 changes: 9 additions & 7 deletions .github/workflows/unittests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,15 +27,17 @@ jobs:
MIN_COVERAGE: 94

steps:
- uses: actions/checkout@v6
- uses: astral-sh/setup-uv@v7
- uses: actions/checkout@v5
- uses: astral-sh/setup-uv@v6

- name: Set up Python
run: uv python install ${{ matrix.python_version }}

- name: Set up venv
run: uv sync

run: |
uv venv -p ${{ matrix.python_version }}
uv pip install '.[dev]'

- name: Run tests without config
run: |
uv run -p ${{ matrix.python_version}} pytest tests_without_config
Expand All @@ -55,7 +57,7 @@ jobs:
- name: Upload coverage HTML
if: always()
id: cov_html
uses: actions/upload-artifact@v7
uses: actions/upload-artifact@v4
with:
name: coverage-html-${{ matrix.python_version }}
path: htmlcov
Expand All @@ -64,7 +66,7 @@ jobs:

- name: Comment link to coverage HTML
if: ${{ always() && github.event_name == 'pull_request' }}
uses: mshick/add-pr-comment@v3
uses: mshick/add-pr-comment@v2
with:
message: |
Coverage HTML (Python ${{ matrix.python_version }}): ${{ steps.cov_html.outputs.artifact-url }}
Expand All @@ -81,7 +83,7 @@ jobs:

- name: Post test results to PR
if: ${{ always() && github.event_name == 'pull_request'}}
uses: mshick/add-pr-comment@v3
uses: mshick/add-pr-comment@v2
with:
message-path: FINAL_REPORT.md
message-id: TestResults-${{ matrix.python_version }}
15 changes: 8 additions & 7 deletions .github/workflows/workflow.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ jobs:
version: ${{ steps.get_version.outputs.version }}
tag_exists: ${{ steps.tagcheck.outputs.exists }}
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v5
with: { fetch-depth: 0 }

- name: Extract version
Expand Down Expand Up @@ -81,14 +81,15 @@ jobs:
with:
ref: ${{ needs.release.outputs.version }}
fetch-depth: 0

- uses: astral-sh/setup-uv@v7


- uses: actions/setup-python@v5
with:
python-version: "3.9"

- name: Build release distributions
run: |
uv python install ${{ vars.PYTHON_VERSION }}
uv build

python -m pip install --upgrade pip build
python -m build

- name: Publish release distributions to PyPI
uses: pypa/gh-action-pypi-publish@release/v1
Expand Down
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -39,3 +39,5 @@ bundles/


site/

.idea/
1 change: 0 additions & 1 deletion .python-version

This file was deleted.

2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
FOLDERS=psengine tests docs

help:
@echo "Available targets"
@echo "Available targets:"
@echo " test - run pytest"
@echo " format - run ruff format"
@echo " lint - run ruff check --fix"
Expand Down
12 changes: 2 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,7 @@ PSEngine is a simple, yet elegant, library for rapid development of integrations

PSEngine allows you to interact with the Recorded Future API extremely easily. There’s no need to manually build the URLs and query parameters, just use the modules dedicated to individual API endpoints.

PSEngine is a Python package solely built and maintained by the Recorded Future Cyber Security Engineering team powering a number of high profile integrations, such as: [Banshee](https://recordedfuture-professionalservices.github.io/banshee); [Recorded Future Alerts for QRadar](https://apps.xforce.ibmcloud.com/extension/b36efdf42b7bf5e3759d036dbcdbf606); Anomali ThreatStream: [Alerts](https://support.recordedfuture.com/hc/en-us/articles/29255683708691-Recorded-Future-Alerts-for-Anomali-ThreatStream) and [Analyst Notes](https://support.recordedfuture.com/hc/en-us/articles/12928414947475-Recorded-Future-Analyst-Notes-for-Anomali-ThreatStream) integrations; [Google SecOps](https://app.recordedfuture.com/portal/integration-center/detail/google-chronicle-nbfi?organization=uhash%3A5cJsHMHeSM&filter_tab=all) and many more.

PSEngine is a Python package solely built and maintained by the Cyber Security Engineering team powering a number of high profile integrations, such as: Elasticsearch, QRadar, Anomali, Jira, TheHive, etc.

## Installation

Expand All @@ -21,7 +20,7 @@ PSEngine is a Python package that can be installed using `pip`. To install PSeng
pip install psengine
```

PSEngine officially supports Python >= 3.10, < 3.15.
PSEngine officially supports Python >= 3.9, < 3.14.


## Supported Features & Best Practices
Expand All @@ -38,7 +37,6 @@ It can easily interact with the following Recorded Future datasets:
- Fusion File management
- Identity Exposures management
- List management
- Malware Intelligence (including Auto Yara and Auto Sigma)
- Malware Sandbox reports download
- On demand IOC enrichment
- Risklists
Expand All @@ -53,9 +51,3 @@ And facilitate the development with features like:
- Markdown creation from certain data types
- Proxy support

## Previous versions and version documentation

PSEngine has been made public from our internal version 2.0.4. Previous versions, including version 1, are not publicly available.

The documentation arrived at version 2.1.1. Older versions are not explicitly documented and changes can be found in the [Release History](./CHANGELOG.md) section.

35 changes: 4 additions & 31 deletions docs/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,39 +1,12 @@
# Changelog

## v2.7.0 - 2026-05-01
## v2.6.1 - 2026-05-08

- Added support for Threat Map and Malware Map via the `ThreatMapMgr`.

## v2.6.0 - 2026-04-29
- `psengine.links` now supports the Links API via the `LinksMgr` class.

### Added

- Added support for Python 3.14
- `PlaybookAlertMgr.search` now supports filter for a single or a list of organisations.
- `TimeHelpers.rel_time_to_date` now supports a starting time from where the math begins. If not specified it will be the UTC execution time.
- `TimeHelpers.rel_time_to_date` now supports increment calculation, with `+1h`.
- `TimeHelpers.rel_time_to_date` now supports increment or decrement of minutes with `+10m` or `-10m`.
- Added `malware_intel.AutoYaraMgr` and `malware_intel.AutoSigmaMgr` managers to interact with auto-yara and auto-sigma APIs.
- `ClassicAlertMgr.fetch` and `ClassicAlertMgr.fetch_bulk` now allow to fetch images directly via the `fetch_images` argument. Defaults to `False`.
- `AnalystNote` model now supports `is_threat_actor` field.

### Fixed

- `playbook_alerts.helpers.save_pba_images` now allow for all alert types that support images.
- `IdentityMgr.fetch_incident_report` now support a single string `organization_id` field.

### Changed

- `IdentityMgr` methods now support 1000 maximum identities returned instead of 500.

### Removed

- Removed support for Python 3.9

## v2.5.2 - 2026-04-27

### Fixed
- When parsing any Playbook Alert object the `panel_log_v2.[].added.[].url` field no longer fails validation when the URL format does not conform to strict HTTP URL requirements.
- `psengine.links` implements Links search and metadata listing via `LinksMgr`.

## v2.5.1 - 2026-03-26

Expand Down Expand Up @@ -61,7 +34,7 @@

### Fixed

- `PlaybookAlertMgr.fetch_bulk()` now correctly respects the `alerts_per_page` parameter for bulk lookup batching instead of using a hardcoded internal constant.
-`PlaybookAlertMgr.fetch_bulk()` now correctly respects the `alerts_per_page` parameter for bulk lookup batching instead of using a hardcoded internal constant.
- `PBA_GeopoliticsFacility` event `url` field no longer fails validation when the URL format does not conform to strict HTTP URL requirements.


Expand Down
1 change: 1 addition & 0 deletions docs/api/links/errors.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
::: psengine.links.errors
1 change: 1 addition & 0 deletions docs/api/links/links_adt.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
::: psengine.links.links
8 changes: 8 additions & 0 deletions docs/api/links/links_mgr.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
::: psengine.links.links_mgr.LinksMgr
options:
members:
- __init__
- list_sections
- list_events
- list_entity_types
- search
1 change: 1 addition & 0 deletions docs/api/links/links_models.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
::: psengine.links.models
11 changes: 0 additions & 11 deletions docs/api/malware_intel/auto_sigma_mgr.md

This file was deleted.

10 changes: 0 additions & 10 deletions docs/api/malware_intel/auto_yara_mgr.md

This file was deleted.

1 change: 0 additions & 1 deletion docs/api/malware_intel/helpers.md

This file was deleted.

1 change: 0 additions & 1 deletion docs/api/threat_maps/errors.md

This file was deleted.

1 change: 0 additions & 1 deletion docs/api/threat_maps/models.md

This file was deleted.

1 change: 0 additions & 1 deletion docs/api/threat_maps/threat_map.md

This file was deleted.

7 changes: 0 additions & 7 deletions docs/api/threat_maps/threat_map_mgr.md

This file was deleted.

7 changes: 2 additions & 5 deletions docs/examples/classic_alerts/example_2.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,8 @@
ALERT_IDS = ['9Z0ts8', '9Z0ttT']

mgr = ClassicAlertMgr()
alerts = mgr.fetch_bulk(
ALERT_IDS,
fetch_images=True,
max_workers=2,
)
alerts = mgr.fetch_bulk(ALERT_IDS, max_workers=2)

for alert in alerts:
mgr.fetch_all_images(alert)
save_images(alert, OUTPUT_DIR)
4 changes: 2 additions & 2 deletions docs/examples/collective_insights/example_1.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
from datetime import UTC, datetime
import datetime

from psengine.collective_insights import (
DETECTION_SUB_TYPE_SIGMA,
Expand All @@ -9,7 +9,7 @@

ci = CollectiveInsights()

now = datetime.now(UTC).isoformat()
now = datetime.datetime.utcnow().isoformat()[:-3] + 'Z'

insight1 = ci.create(
ioc_value='fbee00cb1d1ea4d7e0604436d9a36def71a9f3be804f1e2b8d117fd5d35aeabc',
Expand Down
4 changes: 1 addition & 3 deletions docs/examples/enrich/example_3.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,5 @@
with enriched_file.open('w', newline='') as f:
writer = csv.writer(f)
writer.writerow(['ip', 'score'])
for ip, enriched in zip(
ips_to_enrich, enriched_ips, strict=False
):
for ip, enriched in zip(ips_to_enrich, enriched_ips):
writer.writerow([ip, enriched.content.risk.score])
File renamed without changes.
Loading
Loading