Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 24 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,13 @@
[![CI](https://github.com/hashgraph-online/codex-plugin-scanner/actions/workflows/ci.yml/badge.svg)](https://github.com/hashgraph-online/codex-plugin-scanner/actions/workflows/ci.yml)
[![OpenSSF Scorecard](https://api.scorecard.dev/projects/github.com/hashgraph-online/codex-plugin-scanner/badge)](https://scorecard.dev/viewer/?uri=github.com/hashgraph-online/codex-plugin-scanner)

A security and security-ops scanner for [Codex plugins](https://developers.openai.com/codex/plugins). It scores the applicable plugin surface from 0-100, emits structured findings, and can run Cisco's `skill-scanner` against plugin skills for deeper analysis.
A security, publishability, and security-ops scanner for [Codex plugins](https://developers.openai.com/codex/plugins). It scores the applicable plugin surface from 0-100, emits structured findings, validates install-surface metadata, hardens MCP transport expectations, and can run Cisco's `skill-scanner` against plugin skills for deeper analysis.

## What's New in v1.2.0

- Publishability checks for Codex `interface` metadata, assets, and HTTPS links.
- MCP transport hardening for remote `.mcp.json` endpoints.
- A new `Operational Security` category for GitHub Actions pinning, privileged workflow patterns, Dependabot, and lockfile hygiene.

## Installation

Expand Down Expand Up @@ -48,25 +54,35 @@ codex-plugin-scanner ./my-plugin --cisco-skill-scan on --cisco-policy strict
### Example Output

```
🔗 Codex Plugin Scanner v1.1.0
🔗 Codex Plugin Scanner v1.2.0
Scanning: ./my-plugin

── Manifest Validation (25/25) ──
── Manifest Validation (31/31) ──
✅ plugin.json exists +4
✅ Valid JSON +4
✅ Required fields present +5
✅ Version follows semver +3
✅ Name is kebab-case +2
✅ Recommended metadata present +4
✅ Interface metadata complete if declared +3
✅ Interface links and assets valid if declared +3
✅ Declared paths are safe +3

── Security (16/16) ──
✅ SECURITY.md found +3
✅ LICENSE found +3
✅ No hardcoded secrets +7
✅ No dangerous MCP commands +0
✅ MCP remote transports are hardened +0
✅ No approval bypass defaults +3

── Operational Security (0/0) ──
✅ Third-party GitHub Actions pinned to SHAs +0
✅ No write-all GitHub Actions permissions +0
✅ No privileged untrusted checkout patterns +0
✅ Dependabot configured for automation surfaces +0
✅ Dependency manifests have lockfiles +0

── Skill Security (15/15) ──
✅ Cisco skill scan completed +3
✅ No elevated Cisco skill findings +8
Expand All @@ -85,8 +101,9 @@ Optional surfaces such as `marketplace.json`, `.mcp.json`, and plugin skills are

| Category | Max Points | Checks |
|----------|-----------|--------|
| Manifest Validation | 25 | plugin.json, required fields, semver, kebab-case, recommended metadata, safe declared paths |
| Security | 20 | SECURITY.md, LICENSE, no hardcoded secrets, no dangerous MCP commands, no approval bypass defaults |
| Manifest Validation | 31 | plugin.json, required fields, semver, kebab-case, recommended metadata, interface metadata, interface assets, safe declared paths |
| Security | 24 | SECURITY.md, LICENSE, no hardcoded secrets, no dangerous MCP commands, MCP remote transport hardening, no approval bypass defaults |
| Operational Security | 20 | GitHub Actions SHA pinning, no `write-all`, no privileged untrusted checkout, Dependabot, dependency lockfiles |
| Best Practices | 15 | README.md, skills directory, SKILL.md frontmatter, no committed `.env`, `.codexignore` |
| Marketplace | 15 | marketplace.json validity, policy fields, safe source paths |
| Skill Security | 15 | Cisco scan availability, elevated skill findings, analyzability |
Expand All @@ -108,10 +125,12 @@ The scanner detects:

- **Hardcoded secrets**: AWS keys, GitHub tokens, OpenAI keys, Slack tokens, GitLab tokens, generic password/secret/token patterns
- **Dangerous MCP commands**: `rm -rf`, `sudo`, `curl|sh`, `wget|sh`, `eval`, `exec`, `powershell -c`
- **Insecure MCP remotes**: non-HTTPS remote endpoints and non-loopback HTTP MCP transports
- **Risky Codex defaults**: approval bypass and unrestricted sandbox defaults in plugin-shipped config/docs
- **Shell injection**: template literals with unsanitized interpolation in exec/spawn calls
- **Unsafe code**: `eval()` and `new Function()` usage
- **Cisco skill threats**: policy violations and risky behaviors detected by Cisco `skill-scanner`
- **Workflow hardening gaps**: unpinned third-party actions, `write-all`, privileged untrusted checkouts, missing Dependabot, missing lockfiles

## Report Formats

Expand Down
6 changes: 3 additions & 3 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,10 @@ build-backend = "hatchling.build"

[project]
name = "codex-plugin-scanner"
version = "1.1.0"
description = "Security and security-ops scanner for Codex plugins with Cisco-backed skill analysis."
version = "1.2.0"
description = "Security, operational-security, and publishability scanner for Codex plugins with Cisco-backed skill analysis."
readme = "README.md"
license = {text = "Apache-2.0"}
license = "Apache-2.0"
requires-python = ">=3.10"
authors = [
{ name = "Hashgraph Online", email = "dev@hol.org" },
Expand Down
2 changes: 1 addition & 1 deletion src/codex_plugin_scanner/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
from .models import GRADE_LABELS, CategoryResult, CheckResult, Finding, ScanOptions, ScanResult, Severity, get_grade
from .scanner import scan_plugin

__version__ = "1.1.0"
__version__ = "1.2.0"
__all__ = [
"GRADE_LABELS",
"CategoryResult",
Expand Down
201 changes: 200 additions & 1 deletion src/codex_plugin_scanner/checks/manifest.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
"""Manifest validation checks (25 points)."""
"""Manifest validation checks (31 points)."""

from __future__ import annotations

Expand All @@ -11,6 +11,10 @@
SEMVER_RE = re.compile(r"^\d+\.\d+\.\d+$")
KEBAB_RE = re.compile(r"^[a-z0-9]+(-[a-z0-9]+)*$")
RECOMMENDED_FIELDS = ("author", "homepage", "repository", "license", "keywords")
INTERFACE_METADATA_FIELDS = ("type", "displayName", "shortDescription", "longDescription", "developerName", "category")
INTERFACE_URL_FIELDS = ("websiteURL", "privacyPolicyURL", "termsOfServiceURL")
INTERFACE_ASSET_FIELDS = ("composerIcon", "logo")
HEX_COLOR_RE = re.compile(r"^#[0-9A-Fa-f]{6}$")


def load_manifest(plugin_dir: Path) -> dict | None:
Expand Down Expand Up @@ -54,6 +58,17 @@ def _is_safe_relative_path(plugin_dir: Path, value: str) -> bool:
return True


def _load_interface(manifest: dict | None) -> dict | None:
if manifest is None:
return None
interface = manifest.get("interface")
if interface is None:
return None
if isinstance(interface, dict):
return interface
return {}


def check_plugin_json_exists(plugin_dir: Path) -> CheckResult:
exists = (plugin_dir / ".codex-plugin" / "plugin.json").exists()
findings = ()
Expand Down Expand Up @@ -274,6 +289,188 @@ def check_recommended_metadata(plugin_dir: Path) -> CheckResult:
)


def check_interface_metadata(plugin_dir: Path) -> CheckResult:
manifest = load_manifest(plugin_dir)
interface = _load_interface(manifest)
if interface is None:
return CheckResult(
name="Interface metadata complete if declared",
passed=True,
points=0,
max_points=0,
message="No interface metadata declared, check not applicable.",
applicable=False,
)

if not interface:
return CheckResult(
name="Interface metadata complete if declared",
passed=False,
points=0,
max_points=3,
message="Manifest interface must be a JSON object.",
findings=(
_manifest_finding(
"PLUGIN_JSON_INTERFACE_INVALID",
"Manifest interface is not a JSON object",
'The "interface" field must be an object when it is declared.',
'Replace "interface" with a JSON object that contains the documented publishable metadata.',
),
),
)

missing = [
field
for field in INTERFACE_METADATA_FIELDS
if not isinstance(interface.get(field), str) or not interface.get(field, "").strip()
]

capabilities = interface.get("capabilities")
if (
not isinstance(capabilities, list)
or not capabilities
or not all(isinstance(item, str) and item.strip() for item in capabilities)
):
missing.append("capabilities")

default_prompt = interface.get("defaultPrompt")
if default_prompt is not None:
valid_string_prompt = isinstance(default_prompt, str) and default_prompt.strip()
valid_list_prompt = (
isinstance(default_prompt, list)
and default_prompt
and all(isinstance(item, str) and item.strip() for item in default_prompt)
)
if not valid_string_prompt and not valid_list_prompt:
missing.append("defaultPrompt")

brand_color = interface.get("brandColor")
if brand_color is not None and (not isinstance(brand_color, str) or not HEX_COLOR_RE.match(brand_color)):
missing.append("brandColor")

if not missing:
return CheckResult(
name="Interface metadata complete if declared",
passed=True,
points=3,
max_points=3,
message="Interface metadata contains the expected publishable fields.",
)

findings = tuple(
_manifest_finding(
f"PLUGIN_JSON_INTERFACE_{field.upper()}",
f'Interface field "{field}" is missing or invalid',
f'The interface object is missing a valid "{field}" value.',
f'Add a valid "{field}" field to the interface metadata.',
severity=Severity.INFO,
)
for field in missing
)
return CheckResult(
name="Interface metadata complete if declared",
passed=False,
points=0,
max_points=3,
message=f"Interface metadata is missing or invalid: {', '.join(missing)}",
findings=findings,
)


def _is_https_url(value: object) -> bool:
return isinstance(value, str) and value.startswith("https://")


def _interface_asset_paths(value: object) -> list[str]:
if isinstance(value, str):
return [value]
if isinstance(value, list):
return [item for item in value if isinstance(item, str)]
return []


def check_interface_assets(plugin_dir: Path) -> CheckResult:
manifest = load_manifest(plugin_dir)
interface = _load_interface(manifest)
if interface is None:
return CheckResult(
name="Interface links and assets valid if declared",
passed=True,
points=0,
max_points=0,
message="No interface metadata declared, check not applicable.",
applicable=False,
)

if not interface:
return CheckResult(
name="Interface links and assets valid if declared",
passed=False,
points=0,
max_points=3,
message="Manifest interface must be a JSON object.",
findings=(
_manifest_finding(
"PLUGIN_JSON_INTERFACE_INVALID",
"Manifest interface is not a JSON object",
'The "interface" field must be an object when it is declared.',
'Replace "interface" with a JSON object that contains valid URLs and asset paths.',
),
),
)

issues: list[str] = []
for field in INTERFACE_URL_FIELDS:
if not _is_https_url(interface.get(field)):
issues.append(field)

for field in INTERFACE_ASSET_FIELDS:
value = interface.get(field)
if (
not isinstance(value, str)
or not _is_safe_relative_path(plugin_dir, value)
or not (plugin_dir / value).exists()
):
issues.append(field)

screenshots = _interface_asset_paths(interface.get("screenshots"))
if not screenshots:
issues.append("screenshots")
else:
for screenshot in screenshots:
if not _is_safe_relative_path(plugin_dir, screenshot) or not (plugin_dir / screenshot).exists():
issues.append("screenshots")
break

if not issues:
return CheckResult(
name="Interface links and assets valid if declared",
passed=True,
points=3,
max_points=3,
message="Interface URLs use HTTPS and referenced assets exist within the plugin directory.",
)

findings = tuple(
_manifest_finding(
f"PLUGIN_JSON_INTERFACE_ASSET_{field.upper()}",
f'Interface asset or URL "{field}" is invalid',
f'The interface field "{field}" must use HTTPS or point to a safe in-repo asset.',
f'Update "{field}" to use HTTPS or an existing relative asset path.',
severity=Severity.INFO,
)
for field in issues
)
return CheckResult(
name="Interface links and assets valid if declared",
passed=False,
points=0,
max_points=3,
message=f"Interface links or assets are invalid: {', '.join(issues)}",
findings=findings,
)


def check_declared_paths_safe(plugin_dir: Path) -> CheckResult:
manifest = load_manifest(plugin_dir)
if manifest is None:
Expand Down Expand Up @@ -332,5 +529,7 @@ def run_manifest_checks(plugin_dir: Path) -> tuple[CheckResult, ...]:
check_semver(plugin_dir),
check_kebab_case(plugin_dir),
check_recommended_metadata(plugin_dir),
check_interface_metadata(plugin_dir),
check_interface_assets(plugin_dir),
check_declared_paths_safe(plugin_dir),
)
Loading
Loading