Skip to content

Cut publication commands over to arena#201

Open
vaibhav-i wants to merge 1 commit into
codex/arena-separation-remove-core-daeb-runtime-policyfrom
codex/arena-separation-publication-cli-cutover
Open

Cut publication commands over to arena#201
vaibhav-i wants to merge 1 commit into
codex/arena-separation-remove-core-daeb-runtime-policyfrom
codex/arena-separation-publication-cli-cutover

Conversation

@vaibhav-i

@vaibhav-i vaibhav-i commented Jul 22, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • delegate competitive, publication-bundle, and export-publication from ax-eval to the arena CLI through the shell-free allowlisted compatibility launcher
  • remove duplicate core publication/export and cross-target competitive-rendering implementations
  • keep competitive reporting restricted to --from <sealed-publication-bundle>; legacy raw --results input fails closed
  • accept legacy publication selectors only when suite, vendor, and effort choices exactly match the immutable batch
  • add import guards preventing arena publication policy or competitive rendering from returning to core

Trust and compatibility

Official bundle, export, and competitive outputs remain bound to complete pinned-oci + hosted-trusted production evidence with a verified detached GitHub OIDC attestation. Missing, incomplete, local, unsealed, or invalid inputs emit no bundle. The deprecated ax-eval names remain one-minor process-launcher aliases, but they cannot bypass the sealed arena path.

Verification

  • core tests: 549 passed
  • arena tests: 207 passed (--maxWorkers=1 locally to avoid host-load timeout contention); focused cutover tests: 21 passed at default concurrency
  • npm run typecheck
  • npm run build
  • npm run pack:check
  • core and arena npm --cache .npm-cache pack --dry-run
  • import-boundary checks and git diff --check

All tests are offline and keyless. No live or credentialed evaluation was run.

Independent audit

A separate review-only agent audited exact commit 1327479 against PR #200. It found two documentation inconsistencies; both were fixed, then the amended commit was re-audited. Final audit: clean with no actionable security, compatibility, ownership, or correctness findings.

Stack

Base: #200 core DAEB execution-policy removal.

Rollback: revert this PR to restore the duplicate core publication and competitive implementations while leaving arena implementations intact.

Next slice: move DAEB pack-composition policy into arena-owned authoring APIs.

@github-actions

Copy link
Copy Markdown

Normalized records diff

Overall is a macro-average of participating surface scores. Agents are never averaged together.

Overall by agent

Product Agent Base pass@1 Head pass@1 Δ Base pass³ Head pass³ Surfaces
fixture-db claude-code 92.9% 92.9% +0.0 pp 78.6% (11/14) 78.6% (11/14) 2
fixture-db codex 78.6% 78.6% +0.0 pp 64.3% (9/14) 64.3% (9/14) 2

Surface cells

Product Agent Surface Base pass@1 Head pass@1 Δ Base pass³ Head pass³ Latency ms Cost USD Tokens Harness version
fixture-db claude-code api 85.7% 85.7% +0.0 pp 71.4% (5/7) 71.4% (5/7) 1100 → 1100 0.4200 → 0.4200 3350 → 3350 2.1.5 → 2.1.5
fixture-db claude-code cli 100.0% 100.0% +0.0 pp 85.7% (6/7) 85.7% (6/7) 1300 → 1300 0.5100 → 0.5100 3720 → 3720 2.1.5 → 2.1.5
fixture-db codex api 85.7% 85.7% +0.0 pp 71.4% (5/7) 71.4% (5/7) 1200 → 1200 — → — 3600 → 3600 0.121.0 → 0.121.0
fixture-db codex cli 71.4% 71.4% +0.0 pp 57.1% (4/7) 57.1% (4/7) 1500 → 1500 — → — 4100 → 4100 0.121.0 → 0.121.0

Operational metrics are context only and never affect ranking.

@vaibhav-i
vaibhav-i force-pushed the codex/arena-separation-remove-core-daeb-runtime-policy branch from 8846cbe to b7c2d9d Compare July 22, 2026 15:00
@vaibhav-i
vaibhav-i force-pushed the codex/arena-separation-publication-cli-cutover branch from c33838e to 2c1a98e Compare July 22, 2026 15:00
@vaibhav-i
vaibhav-i force-pushed the codex/arena-separation-remove-core-daeb-runtime-policy branch from b7c2d9d to 379c07d Compare July 23, 2026 00:38
@vaibhav-i
vaibhav-i force-pushed the codex/arena-separation-publication-cli-cutover branch from 2c1a98e to 1327479 Compare July 23, 2026 01:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant