Skip to content

[codex] Fix audit findings for defense readiness#16

Merged
olyannaa merged 8 commits into
mainfrom
feature/live-mode-mcp-readme
Jul 6, 2026
Merged

[codex] Fix audit findings for defense readiness#16
olyannaa merged 8 commits into
mainfrom
feature/live-mode-mcp-readme

Conversation

@olyannaa

@olyannaa olyannaa commented Jul 6, 2026

Copy link
Copy Markdown
Owner

What changed

This PR applies code-level fixes from the pre-defense audit, split into focused commits:

  • fixes package author metadata;
  • adds pytest-cov and an 80% coverage gate to CI;
  • adds live ClickHouse integration tests for real SQL -> expected rule_id behavior;
  • declares optional ML experiment dependencies and records SHA-256 hashes for experiment artifacts;
  • hardens the Streamable HTTP MCP demo endpoint with bearer-token enforcement, rate limiting, and local-only detect_ch_version by default;
  • adds a leakage-aware risk baseline without explicit rule-derived features and rule-shaped parser flags;
  • syncs MCP rule catalog wording and tier filters with the 119-rule registry.

Why

The audit found several real engineering and DS risks: unverifiable coverage, weak live integration coverage, ML dependency reproducibility gaps, public MCP security concerns, and a risk of overstating rule-derived ML metrics. These changes make the repository easier to defend because the claims are now backed by code, CI gates, artifacts, and explicit security boundaries.

Validation

Ran locally:

poetry run python scripts/rules/validate_catalog.py
poetry run python scripts/benchmark/validate_cases.py
poetry run pytest --ignore=tests/integration -q
poetry run ruff check clickadvisor tests scripts
poetry run mypy clickadvisor
poetry run python scripts/eval/run_benchmark.py --cases-dir benchmark/cases/synthetic_expanded --mode strict
poetry run pytest tests/integration/ -q

Results:

  • 119 rule cards validated;
  • 327 benchmark cases validated;
  • unit tests passed;
  • Ruff passed;
  • mypy passed;
  • strict benchmark: 222 cases, precision/recall/F1 = 1.000;
  • integration tests: 6 passed.

@olyannaa
olyannaa marked this pull request as ready for review July 6, 2026 08:32
@olyannaa
olyannaa merged commit 947b592 into main Jul 6, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant