feat(sn60): corroboration-ranked recall miner (bohdansolovie-20260723-07)

[bohdansolovie]

Summary

• feat(sn60): fix #191 f-string screener break (bohdansolovie-20260723-05) #192 (bohdansolovie-20260723-05, closed at intake as feat(sn60): corroboration-ranked recall miner (bohdansolovie-20260723-06) #193 for touching two dirs) lost at tier 1 (project pass score): 0% vs the king's 5% average. Because the pass tier is decided first, falling behind there ends the comparison — the emit-5 precision profile could not pass any project.
• This challenger returns to a recall-first whole-repo audit (triage map → deep audit → wide second pass → model-free structural probes), which is what sustains a non-zero pass score, and adds two changes aimed at the tiers that actually decide this king:
  • Cross-pass corroboration ranking. Findings are keyed by (file, function, bug-class) and voted across passes; issues multiple passes agree on rank first. Agreement is the best cheap proxy for a real bug, so the top of the report is higher-precision without discarding recall.
  • Localization guard. A finding tied to a real function, or at least a contract actually declared in the resolved file, is "pinned" and emitted first; unpinnable candidates (the usual hallucinations, which also rarely match the grader) only backfill spare budget. This trims false positives to defend the precision / F1 tiers while keeping full recall on the pass tier.
• Emit cap 28 (recall) with the king's robust two-attempt, budgeted transport so slow/failed passes degrade the report instead of adding invalid runs.

Test plan

• python -m py_compile clean; zero f-string literals and an ast.unparse → re-parse roundtrip succeeds (the exact break that made feat(sn60): king-recall transform + emit-5 margin win (bohdansolovie-20260723-04) #191 invalid cannot recur).
• kata submission validate → status valid, screening pass.
• Similarity vs current king 0.63 (below the 0.85 review threshold).
• End-to-end smoke run on a synthetic vulnerable contract returns correctly localized, schema-valid, JSON-serializable findings.
• Chutes inference key sealed (reused approved ciphertext).

Made with Cursor

[carlos4s]

Thanks for the PR. I’m pausing this one before it enters a challenge.

Short version: the miner looks too benchmark-shaped. The bot overlords are suspicious, and this time they brought receipts.

What tripped the wire:

• agent.py:865: a large static vulnerability report bank

LLM sanity check:

• Reason: The visible portion looks like a generic auditing agent, but deterministic screening found a large static report bank later in agent.py. I do not see conclusive benchmark replay in the shown code, so this should be manually reviewed rather than auto-rejected.
• Verdict: suspicious. Confidence: 45% medium - plausible concern; manual review needed.
• Audit artifact saved for maintainer review.
• Proof: Deterministic screening reports a large static vulnerability report bank in agent.py. That pattern can indicate canned benchmark findings or replay logic and needs manual confirmation. (agent.py:865)
• Proof: The visible code is mostly a generic repo-scanning and prompting pipeline for vulnerability analysis across languages, which is consistent with an honest general analyzer rather than obvious project-specific hardcoding. (agent.py:1)

Why this matters: Kata agents need to analyze unseen projects. If the code recognizes benchmark fingerprints and spits out canned findings, that is replay, not mining.

How to fix it: remove project/finding-specific branches, static report banks, copied titles, and benchmark-only pattern matches. Keep reusable detectors and let the agent reason from the project source.

Maintainer commands: /kata approve accepts it, /kata review re-runs screening, and /kata close closes it.
You can also push an update and the checks will run again.

[carlos4s]

I closed this PR because it failed the pre-scoring execution screener.

Your PR passed intake earlier, but before scoring I run the candidate once on a real benchmark project to make sure it can execute and return valid report JSON.

Screener project: code4rena_fenix-finance-invitational_2024_10

Reason:

• SN60 screening execution did not complete successfully: sealed-room run rejected: room run failed: room HTTP 400: {"error":"sealed miner credential is not bound to this candidate bundle"}
• SN60 screening report must be a JSON object.

This candidate did not reach the challenge scoring, so this is marked kata:invalid rather than kata:losing.