feat(lab539-aitm-feed): add external import connector (#6703)#6704
Open
Lab539 wants to merge 1 commit into
Open
feat(lab539-aitm-feed): add external import connector (#6703)#6704Lab539 wants to merge 1 commit into
Lab539 wants to merge 1 commit into
Conversation
Contributor License Agreement✅ CLA signed 💚 Thank you @Lab539 for signing the Contributor License Agreement! Your pull request can now be reviewed and merged. We appreciate your contribution to Filigran's open source projects! ❤️ This is an automated message from the Filigran CLA Bot. |
Contributor
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
Adds a new external-import OpenCTI connector for Lab539’s AiTM Feed, including configuration, API client, STIX conversion logic, containerization assets, and unit tests.
Changes:
- Introduce connector runtime (settings, API client, scheduler, STIX bundle generation).
- Add Docker/Docker Compose deployment artifacts and manifest/README documentation.
- Add pytest-based unit tests and shared fixtures.
Reviewed changes
Copilot reviewed 18 out of 20 changed files in this pull request and generated 10 comments.
Show a summary per file
| File | Description |
|---|---|
| external-import/lab539-aitm-feed/src/main.py | Adds connector entry point wiring settings → helper → connector run loop |
| external-import/lab539-aitm-feed/src/lab539_aitm_connector/settings.py | Defines connector and feed configuration models and defaults |
| external-import/lab539-aitm-feed/src/lab539_aitm_connector/api_client.py | Implements AiTM Feed API client for last-event and list endpoints |
| external-import/lab539-aitm-feed/src/lab539_aitm_connector/converter_to_stix.py | Converts feed records into STIX Indicator + author identity + bundle dedupe |
| external-import/lab539-aitm-feed/src/lab539_aitm_connector/connector.py | Implements OpenCTI connector run behavior, state tracking, and ingestion |
| external-import/lab539-aitm-feed/src/lab539_aitm_connector/init.py | Exposes connector class as package public API |
| external-import/lab539-aitm-feed/src/requirements.txt | Adds Python dependencies for the connector runtime |
| external-import/lab539-aitm-feed/Dockerfile | Container build for connector execution |
| external-import/lab539-aitm-feed/docker-compose.yml | Example deployment configuration |
| external-import/lab539-aitm-feed/src/.env.sample | Sample env vars for local/docker usage |
| external-import/lab539-aitm-feed/.dockerignore | Excludes dev/test artifacts from Docker build context |
| external-import/lab539-aitm-feed/metadata/connector_manifest.json | Registers connector metadata for catalog/registry |
| external-import/lab539-aitm-feed/README.md | Documents features, configuration, and deployment |
| external-import/lab539-aitm-feed/tests/conftest.py | Adds shared pytest fixtures |
| external-import/lab539-aitm-feed/tests/test_api_client.py | Unit tests for API client behavior and error handling |
| external-import/lab539-aitm-feed/tests/test_converter_to_stix.py | Unit tests for STIX conversion, deterministic IDs, dedupe, TLP marking |
| external-import/lab539-aitm-feed/tests/test_settings.py | Unit tests for settings validation and defaults |
| external-import/lab539-aitm-feed/tests/test-requirements.txt | Test dependencies for running pytest |
Comment on lines
+81
to
+86
| if last_run is None: | ||
| self.helper.connector_logger.info( | ||
| f"{self.config.connector.name}: First run, pulling " | ||
| f"{self.config.aitm_feed.first_run_lookback_days} days of data" | ||
| ) | ||
| records = self.client.get_records() |
Comment on lines
+45
to
+51
| def _get_ip_type(self, ip: str) -> str: | ||
| """Return the STIX type for a given IP address.""" | ||
| try: | ||
| ipaddress.IPv4Address(ip) | ||
| return "ipv4-addr" | ||
| except ValueError: | ||
| return "ipv6-addr" |
Comment on lines
+72
to
+77
| def _build_pattern(self, ip: str, ip_type: str, domain_value: str) -> str: | ||
| """Build a STIX pattern from IP and domain values.""" | ||
| pattern_parts = [f"[{ip_type}:value = '{ip}']"] | ||
| if domain_value: | ||
| pattern_parts.append(f"[domain-name:value = '{domain_value}']") | ||
| return " OR ".join(pattern_parts) |
Comment on lines
+41
to
+44
| tlp_level: Literal["white", "green", "amber", "amber+strict", "red"] = Field( | ||
| description="TLP marking level applied to all imported objects.", | ||
| default="amber", | ||
| ) |
Comment on lines
+44
to
+48
| def _is_new_data_available(self) -> bool: | ||
| """Lightweight check via last-event endpoint before pulling full dataset.""" | ||
| state = self.helper.get_state() | ||
| last_event_id = state.get("last_event_id") if state else None | ||
| current_event_id = self.client.get_last_event() |
Comment on lines
+56
to
+62
| def _update_last_event_id(self) -> None: | ||
| """Store the current latest eventid in connector state.""" | ||
| current_event_id = self.client.get_last_event() | ||
| if current_event_id: | ||
| state = self.helper.get_state() or {} | ||
| state["last_event_id"] = current_event_id | ||
| self.helper.set_state(state) |
Comment on lines
+17
to
+24
| def get_last_event(self) -> str | None: | ||
| """Lightweight check for new data. No auth required.""" | ||
| try: | ||
| response = requests.get(f"{self.base_url}/last-event", timeout=10) | ||
| response.raise_for_status() | ||
| return response.json().get("eventid") | ||
| except Exception: # pylint: disable=broad-exception-caught | ||
| return None |
| requests==2.33.1 | ||
| stix2==3.0.2 | ||
| isodate==0.7.2 | ||
| connectors-sdk @ git+https://github.com/OpenCTI-Platform/connectors.git@master#subdirectory=connectors-sdk |
Comment on lines
+18
to
+22
| | Object | Type | Description | | ||
| |--------|------|-------------| | ||
| | Indicator | `indicator` | STIX pattern combining IP and domain, with confidence score and detection metadata | | ||
| | IP Address | `ipv4-addr` / `ipv6-addr` | The infrastructure IP address | | ||
| | Domain Name | `domain-name` | The hostname or domain associated with the infrastructure | |
Comment on lines
+44
to
+45
| | Parameter | Docker environment variable | Default | Description | | ||
| |-----------|-----------------------------|---------|-------------| |
Lab539
force-pushed
the
feature/lab539-aitm-feed
branch
from
7fca080 to
df0852d
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Proposed changes
Related issues
Checklist
Further comments
API credentials are required to test this connector. Happy to provide these privately - please reach out via Filigran Slack or directly. Docker image is published at lab539/opencti-aitm-feed-connector:latest.