Benchmark Results

Private staging repository for Cline benchmark result bundles.

This repository is intended to become a public transparency archive for Terminal-Bench side-by-side runs. Large trace archives are stored with Git LFS; summaries, manifests, provenance, and scripts are regular Git files.

Current Dataset

terminal-bench/2026-06-open-weights/

This dataset contains sanitized Terminal-Bench full 89-task runs for:

• cline-non: Cline with reasoning/thinking off, excluding Kimi because that OpenRouter endpoint requires reasoning.
• cline-medium: Cline with medium reasoning/thinking.
• opencode-medium: OpenCode with medium reasoning/thinking.
• pi-medium: Pi-Code with medium reasoning/thinking.

Start with:

• terminal-bench/2026-06-open-weights/summary.md
• terminal-bench/2026-06-open-weights/summary.csv
• terminal-bench/2026-06-open-weights/provenance/archive-manifest.csv

Archive Layout

Each archive is one full Harbor run:

terminal-bench/2026-06-open-weights/archives/<group>/<model>.zip

The ZIP root preserves the original Harbor run directory name. Per trial, the package includes available result.json, config.json, trial.log, verifier outputs, agent traces, trajectories, and Cline session message JSON.

Excluded from the source package: Harbor DB files, provider settings, cache, setup logs, and local agent logs.

Verification

Check archive hashes:

python3 terminal-bench/2026-06-open-weights/scripts/verify-archives.py

List LFS-managed files:

git lfs ls-files

Publication Checklist

Before making the repository public:

• Run the archive verifier.
• Run a fresh secret scan on the unzipped archives.
• Review SECURITY.md.
• Confirm the README caveats match the numbers being cited externally.