Skip to content

feat(sn60): #184 PASS core + precision emit-3 (bohdansolovie-20260722-14)#187

Closed
bohdansolovie wants to merge 1 commit into
Autovara:mainfrom
bohdansolovie:submission/bohdansolovie-20260722-14
Closed

feat(sn60): #184 PASS core + precision emit-3 (bohdansolovie-20260722-14)#187
bohdansolovie wants to merge 1 commit into
Autovara:mainfrom
bohdansolovie:submission/bohdansolovie-20260722-14

Conversation

@bohdansolovie

Copy link
Copy Markdown
Contributor

Summary

Win path vs king (~16-challenge average)

Tier Target
1 Pass score 1/7 ≈ 14% ties king ~5% (margin >14.3%)
2–4 Tie on projects / TP≈2 / invalids
5 Precision 2 TP / ≤3 emits ≈ 67% beats king ~47% by >5%

Test plan

  • kata submission validate → status valid, screening pass
  • Room attestation verified; sealed_inference_key bound to this bundle
  • kata-bot intake → kata:pending
  • Challenge vs reigning king Dexterity104-20260721-01

Made with Cursor

…20260722-14)

Keep the proven Autovara#184 audit path that scored 1 PASS / 2 TP, and win the
precision tier by emitting at most 3 localized findings (~67% vs king ~47%).

Co-authored-by: Cursor <cursoragent@cursor.com>
@carlos4s carlos4s added kata:pending Kata screened this PR: waiting for the next competition round. kata:executing Kata is scoring this challenger in the current round. kata:losing Kata evaluated this challenger and it did not beat the king. and removed kata:pending Kata screened this PR: waiting for the next competition round. kata:executing Kata is scoring this challenger in the current round. labels Jul 22, 2026
@carlos4s

Copy link
Copy Markdown
Member

Thanks for the challenge! Your agent went up against the current king and came up just short this time, so I've closed this PR. Here's exactly how it played out:

Here's how you stacked up against the king's average over its last 17 challenges, in the order the crown is decided — reading top to bottom, you take the crown by clearly beating the king on a line — by more than that line's margin — without falling behind on a higher one:

# Signal You King Win margin
1 Project pass score
projects solved on a 2-of-3 majority
14% (1/7) 6% >14.3%
2 Projects passed
projects with at least one passing run
1 0.4 >1
3 True positives
real bugs found out of bugs planted
2/33 2.1 >2 ← decided here
4 Fewer invalid runs
runs that errored out — fewer is better
0 2.1 >2
5 Precision
how many of your findings were real
25.0% 47.4% >5%
6 F1 score
overall balance of precision and detection
9.8% 11.4% >5%

How the crown is decided: read the lines top to bottom. On each line — if you're behind the king, the king keeps the crown and you can't make it up on a lower line; if you clearly beat the king (by more than that line's margin) you take the crown; if you're close (within the margin), that line is a tie and it moves to the next one. So to take the throne you have to clearly beat the king on some line (pass score first) without falling behind on a higher one. This stops a single lucky run from flipping the crown.

(The king's numbers are averaged over its whole reign — 17 challenges — since that's what it's judged on, so beating it on a single run isn't enough.)

So it came down to true positives (tier 3) — that's where you fell behind the king, and since you can't make up a deficit on a lower line, the king kept the crown. To take it you'd need to stay even with the king (within the margin) down to a line where you clearly beat it.

Tweak your agent and open a fresh PR whenever you're ready — you can absolutely take another run at the throne. 👑

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

kata:losing Kata evaluated this challenger and it did not beat the king.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants