Skip to content

feat(sn60): add Dexterity104-20260723-01#198

Closed
Dexterity104 wants to merge 1 commit into
Autovara:mainfrom
Dexterity104:submission/sn60-dexterity104-20260721-02
Closed

feat(sn60): add Dexterity104-20260723-01#198
Dexterity104 wants to merge 1 commit into
Autovara:mainfrom
Dexterity104:submission/sn60-dexterity104-20260721-02

Conversation

@Dexterity104

Copy link
Copy Markdown
Contributor

Adds one miner bundle for the sn60__bitsec miner lane.

@carlos4s carlos4s added the kata:review Kata held this PR for manual screening review before a round. label Jul 23, 2026
@carlos4s

Copy link
Copy Markdown
Member

Thanks for the PR. I’m pausing this one before it enters a challenge.

Short version: the miner looks too benchmark-shaped. The bot overlords are suspicious, and this time they brought receipts.

What tripped the wire:

  • agent.py: Screening review required: submission agent is highly similar to the current lane king implementation (similarity 0.94).

LLM sanity check:

  • Reason: I do not see benchmark hardcoding or secret leakage in the submitted code, and it behaves like a generic repo auditor. However, the deterministic similarity score to the king agent is very high, so this should get a manual copycat review before acceptance.
  • Verdict: suspicious. Confidence: 62% medium - plausible concern; manual review needed.
  • Audit artifact saved for maintainer review.
  • Proof: File header describes a generic multi-language whole-repo auditor, which is allowed and does not itself show benchmark replay. (agent.py:1)
  • Proof: Uses validator-provided inference proxy and environment key x-inference-api-key, this is normal proxy usage and not an answer-key leak token. (agent.py:72)
  • Proof: Large system prompt and checklist are generic audit instructions covering common bug classes across languages, not benchmark-specific findings or project IDs. (agent.py:96)

Why this matters: Kata agents need to analyze unseen projects. If the code recognizes benchmark fingerprints and spits out canned findings, that is replay, not mining.

How to fix it: remove project/finding-specific branches, static report banks, copied titles, and benchmark-only pattern matches. Keep reusable detectors and let the agent reason from the project source.

Maintainer commands: /kata approve accepts it, /kata review re-runs screening, and /kata close closes it.
You can also push an update and the checks will run again.

@carlos4s carlos4s added the kata:lane:sn60__bitsec Kata competition lane this PR belongs to (membership only, not a routing input). label Jul 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

kata:lane:sn60__bitsec Kata competition lane this PR belongs to (membership only, not a routing input). kata:review Kata held this PR for manual screening review before a round.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants