feat(dark-web-informer): add passthrough external-import connector (#6595)#6596
Open
gamezilla19 wants to merge 2 commits into
Open
feat(dark-web-informer): add passthrough external-import connector (#6595)#6596gamezilla19 wants to merge 2 commits into
gamezilla19 wants to merge 2 commits into
Conversation
Contributor License Agreement✅ CLA signed 💚 Thank you @gamezilla19 for signing the Contributor License Agreement! Your pull request can now be reviewed and merged. We appreciate your contribution to Filigran's open source projects! ❤️ This is an automated message from the Filigran CLA Bot. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Proposed changes
external-import/dark-web-informerconnector for importing Dark Web Informer intelligence into OpenCTI./api/stix/export_feed.json,/api/stix/export_ransomware.json,/api/stix/export_iocs.json, or the combined/api/stix/export.json) and forward them to OpenCTI unchanged viasend_stix2_bundle(passthrough). No STIX is built or rewritten on the connector side, mirroring the official TAXII 2.1 connector.X-API-Keyplus a single-useX-Nonce(<10-digit epoch>:<≥6 chars>, 120s window); a generic TAXII2/file-import connector cannot generate this per-request nonce.feed,ransomware,iocs, orall) and an optional preview mode (use_preview_endpoint) using the smaller on-demand/api/stix.jsonendpoint for testing.Related issues
Checklist
Further comments
This connector intentionally has no conversion layer. Dark Web Informer publishes ready-made STIX 2.1 bundles, so the connector forwards them as-is, mirroring the official TAXII 2.1 connector. An earlier JSON-to-STIX variant was discarded because the conversion layer was the source of ingestion errors; passthrough removes that surface entirely while preserving the provider's identities, markings, relationships, and deterministic IDs. A custom connector (rather than the generic TAXII2/file-import one) is still required because the API uses a single-use per-request
X-Noncethat those connectors cannot generate.Tested locally:
PYTHONPATH=src python -m pytest tests/→3 passedno_generated_id_stix) →10.00/10black,isort,flake8 --ignore=E,W→ cleanValidated end-to-end by the data provider against a live OpenCTI platform (>= 6.8.12) and the production API: ~42,000 objects ingested with no errors, objects landing correctly, and re-ingestion deduplicating on deterministic STIX IDs. STIX exports regenerate every 30 minutes, so
CONNECTOR_DURATION_PERIODdefaults toPT6Hand hourly-or-slower is recommended.