refactor(greynoise-vuln): migrate connector to manager-supported mode (#5265)#6665
refactor(greynoise-vuln): migrate connector to manager-supported mode (#5265)#6665jabesq wants to merge 16 commits into
Conversation
|
🤖 [AI-generated] Hey @jabesq! 👋 I'm Sam, your convention guardian. I reviewed PR #6665 and made one small adjustment to keep it aligned with our conventions: 🏷️ Labels
✅ Your title ✅ Your description is thorough and well-structured — proposed changes, linked issue (
Thanks for contributing! 🚀 |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #6665 +/- ##
===========================================
- Coverage 31.77% 0.35% -31.43%
===========================================
Files 1977 1888 -89
Lines 121783 119187 -2596
===========================================
- Hits 38700 426 -38274
- Misses 83083 118761 +35678
📢 Thoughts on this report? Let us know! 🚀 New features to boost your workflow:
|
- Switch Dockerfile to direct python3 entrypoint (drop entrypoint.sh) - Move config.yml.sample to connector root - Update connector_manifest.json with last_verified_date - Refactor connector.py: simplify stix note building, drop redundant confidence fields, use cleanup_inconsistent_bundle on send, use bool() for kev flag
…, and type annotations - Make tlp local in _extract_and_check_markings, raise ValueError directly instead of returning bool - Remove stop_time param from _generate_stix_relationship to avoid Redis time-bucket explosion (VC324) - Replace manual connect_scope string parsing with self.config.connector.scope - Fix return type annotations: _process_labels tuple -> list, _generate_stix_relationship dict -> stix2.Relationship
…(VC319) - Send original stix_objects bundle back when event_type is absent so playbook chain is not broken - Add event_type to log context for easier debugging
…hing Copy requirements.txt before source code so pip install layer is only invalidated when dependencies change, not on every source edit.
29 new tests in tests/tests_connector/test_connector.py covering: - _extract_and_check_markings (valid/invalid TLP, no TLP) - _generate_stix_relationship (with and without start_time) - _create_custom_label (success, API returns None) - _get_match (found, not found) - _process_labels (all active, none active, missing keys) - _generate_stix_external_reference, _generate_stix_note - _generate_stix_software (normal, empty product/vendor) - _generate_greynoise_stix_identity - _generate_stix_vulnerability (kev true/false) - _generate_stix_bundle - _process_message (CVE found/not found, invalid TLP, API error, out-of-scope with/without event_type) - process_message error handling, run connector.py reaches 100%, total src coverage 92%
b70acdf to
60aa923
Compare
There was a problem hiding this comment.
Pull request overview
This PR migrates the internal-enrichment/greynoise-vuln connector to the connectors-sdk “manager-supported” structure by introducing typed Pydantic settings, a new package/module layout, and an updated runtime entrypoint—while also adding unit tests and updating deployment/config artifacts to match the new configuration model.
Changes:
- Introduces
ConnectorSettings/GreyNoiseVulnConfigbased onconnectors_sdkand updates the runtime entrypoint to instantiate the helper + connector via constructor injection. - Refactors connector implementation into
src/connector/connector.pyand adds test coverage for settings, entrypoint, and core enrichment behaviors. - Updates Docker/build and configuration artifacts (manifest, schema, samples) for manager-supported mode and new env var namespaces.
Reviewed changes
Copilot reviewed 19 out of 19 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| internal-enrichment/greynoise-vuln/tests/tests_connector/test_settings.py | Adds settings validation tests for the new Pydantic configuration models. |
| internal-enrichment/greynoise-vuln/tests/tests_connector/test_connector.py | Adds unit tests covering enrichment logic helpers and message processing behavior. |
| internal-enrichment/greynoise-vuln/tests/test-requirements.txt | Introduces connector-specific pytest dependency set for the new test suite. |
| internal-enrichment/greynoise-vuln/tests/test_main.py | Adds tests for entrypoint wiring (settings → helper → connector construction). |
| internal-enrichment/greynoise-vuln/tests/conftest.py | Ensures src/ is importable during tests. |
| internal-enrichment/greynoise-vuln/src/requirements.txt | Adds connectors-sdk + pydantic dependencies required for manager-supported settings. |
| internal-enrichment/greynoise-vuln/src/main.py | Replaces legacy config loading with manager-supported settings + dependency injection. |
| internal-enrichment/greynoise-vuln/src/connector/settings.py | Defines typed connector settings/config models (connectors-sdk + Pydantic). |
| internal-enrichment/greynoise-vuln/src/connector/connector.py | Moves/refactors connector logic into manager-supported module layout. |
| internal-enrichment/greynoise-vuln/src/connector/init.py | Exposes connector public API (GreyNoiseVulnConnector, ConnectorSettings). |
| internal-enrichment/greynoise-vuln/src/config.yml.sample | Removes old sample config from src/ (legacy layout). |
| internal-enrichment/greynoise-vuln/entrypoint.sh | Removes shell entrypoint (Docker now directly runs Python entrypoint). |
| internal-enrichment/greynoise-vuln/Dockerfile | Improves layer caching and updates entrypoint to run main.py directly. |
| internal-enrichment/greynoise-vuln/docker-compose.yml | Updates environment variable names to match new Pydantic namespaces. |
| internal-enrichment/greynoise-vuln/config.yml.sample | Adds new root-level sample config matching manager-supported namespaces. |
| internal-enrichment/greynoise-vuln/.env.sample | Adds new env sample using manager-supported variable names. |
| internal-enrichment/greynoise-vuln/metadata/connector_manifest.json | Sets manager_supported: true and updates verification metadata. |
| internal-enrichment/greynoise-vuln/metadata/connector_config_schema.json | Adds generated JSON schema for manager-supported configuration. |
| internal-enrichment/greynoise-vuln/metadata/CONNECTOR_CONFIG_DOC.md | Adds generated human-readable configuration documentation. |
Proposed changes
get_config_variablecalls with typedConnectorSettings/GreyNoiseVulnConfigmodels built onconnectors_sdkbase classes, enabling manager-supported mode and validating all config at startup.GreyNoiseVulnConnectornow receives settings and helper via constructor injection, decoupling config loading from business logic and making the class independently testable.src/connector/__init__.pyandsrc/main.pyentry point to satisfy the manager-supported module layout.stop_timehandling, tightened scope lookup, corrected return type annotation on_generate_stix_bundle.COPYinto separate layers so source changes don't invalidate the dependency install layer.test_connector.py,test_settings.py,test_main.pyandconftest.py, reaching 92% coverage.manager_supported: true, regeneratedconnector_config_schema.jsonand globalmanifest.json..env.sample,config.yml.sample, anddocker-compose.ymlto reflect new Pydantic settings key names.Related issues
Checklist
Further comments
This PR follows the standard manager-supported migration pattern: a
settings.pyPydantic model tree,main.pyentry point with constructor injection, andmanager_supported: truein the manifest. No functional changes to the enrichment logic itself — GreyNoise CVE API calls, STIX bundle generation, and label/note/software creation are preserved as-is.