Skip to content

refactor(greynoise-vuln): migrate connector to manager-supported mode (#5265)#6665

Open
jabesq wants to merge 16 commits into
masterfrom
feat/5265-greynoise-vuln-manager-supported
Open

refactor(greynoise-vuln): migrate connector to manager-supported mode (#5265)#6665
jabesq wants to merge 16 commits into
masterfrom
feat/5265-greynoise-vuln-manager-supported

Conversation

@jabesq

@jabesq jabesq commented Jun 8, 2026

Copy link
Copy Markdown
Member

Proposed changes

  • Pydantic settings — replaced legacy get_config_variable calls with typed ConnectorSettings / GreyNoiseVulnConfig models built on connectors_sdk base classes, enabling manager-supported mode and validating all config at startup.
  • Connector class refactorGreyNoiseVulnConnector now receives settings and helper via constructor injection, decoupling config loading from business logic and making the class independently testable.
  • Package structure — added src/connector/__init__.py and src/main.py entry point to satisfy the manager-supported module layout.
  • VC319 compliance — when the entity type is not in scope, the original STIX bundle is returned unchanged instead of being silently dropped.
  • Code quality clean-up — simplified TLP check, fixed stop_time handling, tightened scope lookup, corrected return type annotation on _generate_stix_bundle.
  • Dockerfile layer caching — split COPY into separate layers so source changes don't invalidate the dependency install layer.
  • Unit tests — added test_connector.py, test_settings.py, test_main.py and conftest.py, reaching 92% coverage.
  • Manifest & schema — set manager_supported: true, regenerated connector_config_schema.json and global manifest.json.
  • Config files — updated .env.sample, config.yml.sample, and docker-compose.yml to reflect new Pydantic settings key names.

Related issues

Checklist

  • I consider the submitted work as finished
  • I have signed my commits using GPG key.
  • I tested the code for its functionality using different use cases
  • I added/update the relevant documentation (either on github or on notion)
  • Where necessary I refactored code to improve the overall quality

Further comments

This PR follows the standard manager-supported migration pattern: a settings.py Pydantic model tree, main.py entry point with constructor injection, and manager_supported: true in the manifest. No functional changes to the enrichment logic itself — GreyNoise CVE API calls, STIX bundle generation, and label/note/software creation are preserved as-is.

@github-actions github-actions Bot added the filigran team Item from the Filigran team. label Jun 8, 2026
@Filigran-Automation

Copy link
Copy Markdown
Member

🤖 [AI-generated]

Hey @jabesq! 👋 I'm Sam, your convention guardian. I reviewed PR #6665 and made one small adjustment to keep it aligned with our conventions:

🏷️ Labels

Action Label Reason
✅ Added filigran team Per .github/LABELS.md, every pull request must carry exactly one ownership label (filigran team or community). As a Filigran team member's PR, this is the correct ownership label — it was missing.

✅ Your title refactor(greynoise-vuln): migrate connector to manager-supported mode (#5265) already follows the Conventional Commits format perfectly (valid type, lowercase scope, lowercase description, no trailing period, required (#issue) reference).

✅ Your description is thorough and well-structured — proposed changes, linked issue (Closes #5265), and a complete checklist. Great work! 🙌

💡 For future PRs, see the contribution conventions and the label taxonomy.

Thanks for contributing! 🚀

@codecov

codecov Bot commented Jun 8, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 95.30201% with 7 lines in your changes missing coverage. Please review.
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
internal-enrichment/greynoise-vuln/src/main.py 0.00% 7 Missing ⚠️

❗ There is a different number of reports uploaded between BASE (c6db90a) and HEAD (9f94c68). Click for more details.

HEAD has 111 uploads less than BASE
Flag BASE (c6db90a) HEAD (9f94c68)
connectors 115 4
Additional details and impacted files
@@             Coverage Diff             @@
##           master    #6665       +/-   ##
===========================================
- Coverage   31.77%    0.35%   -31.43%     
===========================================
  Files        1977     1888       -89     
  Lines      121783   119187     -2596     
===========================================
- Hits        38700      426    -38274     
- Misses      83083   118761    +35678     
Files with missing lines Coverage Δ
...nrichment/greynoise-vuln/src/connector/__init__.py 100.00% <100.00%> (ø)
...richment/greynoise-vuln/src/connector/connector.py 100.00% <100.00%> (ø)
...nrichment/greynoise-vuln/src/connector/settings.py 100.00% <100.00%> (ø)
internal-enrichment/greynoise-vuln/src/main.py 0.00% <0.00%> (ø)

... and 1094 files with indirect coverage changes

📢 Thoughts on this report? Let us know!

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@ncarenton ncarenton requested a review from throuxel June 9, 2026 07:41
Comment thread internal-enrichment/greynoise-vuln/src/connector/settings.py Outdated
Comment thread manifest.json
jabesq added 15 commits June 11, 2026 10:33
- Switch Dockerfile to direct python3 entrypoint (drop entrypoint.sh)
- Move config.yml.sample to connector root
- Update connector_manifest.json with last_verified_date
- Refactor connector.py: simplify stix note building, drop redundant
  confidence fields, use cleanup_inconsistent_bundle on send, use
  bool() for kev flag
…, and type annotations

- Make tlp local in _extract_and_check_markings, raise ValueError directly
  instead of returning bool
- Remove stop_time param from _generate_stix_relationship to avoid
  Redis time-bucket explosion (VC324)
- Replace manual connect_scope string parsing with self.config.connector.scope
- Fix return type annotations: _process_labels tuple -> list,
  _generate_stix_relationship dict -> stix2.Relationship
…(VC319)

- Send original stix_objects bundle back when event_type is absent so
  playbook chain is not broken
- Add event_type to log context for easier debugging
…hing

Copy requirements.txt before source code so pip install layer is only
invalidated when dependencies change, not on every source edit.
29 new tests in tests/tests_connector/test_connector.py covering:
- _extract_and_check_markings (valid/invalid TLP, no TLP)
- _generate_stix_relationship (with and without start_time)
- _create_custom_label (success, API returns None)
- _get_match (found, not found)
- _process_labels (all active, none active, missing keys)
- _generate_stix_external_reference, _generate_stix_note
- _generate_stix_software (normal, empty product/vendor)
- _generate_greynoise_stix_identity
- _generate_stix_vulnerability (kev true/false)
- _generate_stix_bundle
- _process_message (CVE found/not found, invalid TLP, API error,
  out-of-scope with/without event_type)
- process_message error handling, run

connector.py reaches 100%, total src coverage 92%
Copilot AI review requested due to automatic review settings June 11, 2026 08:33
@jabesq jabesq force-pushed the feat/5265-greynoise-vuln-manager-supported branch from b70acdf to 60aa923 Compare June 11, 2026 08:33
@jabesq jabesq requested a review from throuxel June 11, 2026 08:34

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR migrates the internal-enrichment/greynoise-vuln connector to the connectors-sdk “manager-supported” structure by introducing typed Pydantic settings, a new package/module layout, and an updated runtime entrypoint—while also adding unit tests and updating deployment/config artifacts to match the new configuration model.

Changes:

  • Introduces ConnectorSettings / GreyNoiseVulnConfig based on connectors_sdk and updates the runtime entrypoint to instantiate the helper + connector via constructor injection.
  • Refactors connector implementation into src/connector/connector.py and adds test coverage for settings, entrypoint, and core enrichment behaviors.
  • Updates Docker/build and configuration artifacts (manifest, schema, samples) for manager-supported mode and new env var namespaces.

Reviewed changes

Copilot reviewed 19 out of 19 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
internal-enrichment/greynoise-vuln/tests/tests_connector/test_settings.py Adds settings validation tests for the new Pydantic configuration models.
internal-enrichment/greynoise-vuln/tests/tests_connector/test_connector.py Adds unit tests covering enrichment logic helpers and message processing behavior.
internal-enrichment/greynoise-vuln/tests/test-requirements.txt Introduces connector-specific pytest dependency set for the new test suite.
internal-enrichment/greynoise-vuln/tests/test_main.py Adds tests for entrypoint wiring (settings → helper → connector construction).
internal-enrichment/greynoise-vuln/tests/conftest.py Ensures src/ is importable during tests.
internal-enrichment/greynoise-vuln/src/requirements.txt Adds connectors-sdk + pydantic dependencies required for manager-supported settings.
internal-enrichment/greynoise-vuln/src/main.py Replaces legacy config loading with manager-supported settings + dependency injection.
internal-enrichment/greynoise-vuln/src/connector/settings.py Defines typed connector settings/config models (connectors-sdk + Pydantic).
internal-enrichment/greynoise-vuln/src/connector/connector.py Moves/refactors connector logic into manager-supported module layout.
internal-enrichment/greynoise-vuln/src/connector/init.py Exposes connector public API (GreyNoiseVulnConnector, ConnectorSettings).
internal-enrichment/greynoise-vuln/src/config.yml.sample Removes old sample config from src/ (legacy layout).
internal-enrichment/greynoise-vuln/entrypoint.sh Removes shell entrypoint (Docker now directly runs Python entrypoint).
internal-enrichment/greynoise-vuln/Dockerfile Improves layer caching and updates entrypoint to run main.py directly.
internal-enrichment/greynoise-vuln/docker-compose.yml Updates environment variable names to match new Pydantic namespaces.
internal-enrichment/greynoise-vuln/config.yml.sample Adds new root-level sample config matching manager-supported namespaces.
internal-enrichment/greynoise-vuln/.env.sample Adds new env sample using manager-supported variable names.
internal-enrichment/greynoise-vuln/metadata/connector_manifest.json Sets manager_supported: true and updates verification metadata.
internal-enrichment/greynoise-vuln/metadata/connector_config_schema.json Adds generated JSON schema for manager-supported configuration.
internal-enrichment/greynoise-vuln/metadata/CONNECTOR_CONFIG_DOC.md Adds generated human-readable configuration documentation.

Comment thread internal-enrichment/greynoise-vuln/src/connector/connector.py
Comment thread internal-enrichment/greynoise-vuln/src/connector/connector.py
Comment thread internal-enrichment/greynoise-vuln/src/connector/connector.py
Comment thread internal-enrichment/greynoise-vuln/src/connector/connector.py Outdated
Comment thread internal-enrichment/greynoise-vuln/src/connector/connector.py Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

filigran team Item from the Filigran team.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

refactor(greynoise-vulnerability): migrate connector to be connector manager supported

5 participants