Skip to content

feat(dark-web-informer): add passthrough external-import connector (#6595)#6596

Open
gamezilla19 wants to merge 2 commits into
OpenCTI-Platform:masterfrom
gamezilla19:feature/dark-web-informer
Open

feat(dark-web-informer): add passthrough external-import connector (#6595)#6596
gamezilla19 wants to merge 2 commits into
OpenCTI-Platform:masterfrom
gamezilla19:feature/dark-web-informer

Conversation

@gamezilla19

@gamezilla19 gamezilla19 commented Jun 3, 2026

Copy link
Copy Markdown

Proposed changes

  • Add a new external-import/dark-web-informer connector for importing Dark Web Informer intelligence into OpenCTI.
  • Fetch the prebuilt STIX 2.1 bundles published by Dark Web Informer (/api/stix/export_feed.json, /api/stix/export_ransomware.json, /api/stix/export_iocs.json, or the combined /api/stix/export.json) and forward them to OpenCTI unchanged via send_stix2_bundle (passthrough). No STIX is built or rewritten on the connector side, mirroring the official TAXII 2.1 connector.
  • Authenticate with X-API-Key plus a single-use X-Nonce (<10-digit epoch>:<≥6 chars>, 120s window); a generic TAXII2/file-import connector cannot generate this per-request nonce.
  • Support source selection (feed, ransomware, iocs, or all) and an optional preview mode (use_preview_endpoint) using the smaller on-demand /api/stix.json endpoint for testing.
  • Add connector configuration, Docker assets, README documentation, metadata manifest, and unit tests.

Related issues

Checklist

  • I consider the submitted work as finished
  • I have signed my commits using GPG key.
  • I tested the code for its functionality using different use cases
  • I added/update the relevant documentation (either on github or on notion)
  • Where necessary I refactored code to improve the overall quality

Further comments

This connector intentionally has no conversion layer. Dark Web Informer publishes ready-made STIX 2.1 bundles, so the connector forwards them as-is, mirroring the official TAXII 2.1 connector. An earlier JSON-to-STIX variant was discarded because the conversion layer was the source of ingestion errors; passthrough removes that surface entirely while preserving the provider's identities, markings, relationships, and deterministic IDs. A custom connector (rather than the generic TAXII2/file-import one) is still required because the API uses a single-use per-request X-Nonce that those connectors cannot generate.

Tested locally:

  • PYTHONPATH=src python -m pytest tests/3 passed
  • STIX ID plugin (no_generated_id_stix) → 10.00/10
  • black, isort, flake8 --ignore=E,W → clean

Validated end-to-end by the data provider against a live OpenCTI platform (>= 6.8.12) and the production API: ~42,000 objects ingested with no errors, objects landing correctly, and re-ingestion deduplicating on deterministic STIX IDs. STIX exports regenerate every 30 minutes, so CONNECTOR_DURATION_PERIOD defaults to PT6H and hourly-or-slower is recommended.

@filigran-cla-bot filigran-cla-bot Bot added the cla:pending CLA signature required. label Jun 3, 2026
@filigran-cla-bot

filigran-cla-bot Bot commented Jun 3, 2026

Copy link
Copy Markdown

Contributor License Agreement

CLA signed 💚

Thank you @gamezilla19 for signing the Contributor License Agreement! Your pull request can now be reviewed and merged.

We appreciate your contribution to Filigran's open source projects! ❤️

This is an automated message from the Filigran CLA Bot.

@filigran-cla-bot filigran-cla-bot Bot removed the cla:pending CLA signature required. label Jun 3, 2026
@SamuelHassine SamuelHassine changed the title [Dark Web Informer] Add passthrough external-import connector feat(dark-web-informer): add passthrough external-import connector (#6595) Jun 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(dark-web-informer): create the external import connector

2 participants