feat(harness): sk harness init — scaffold universal harness for any project#734
Merged
Conversation
…roject (#731) - Add harness-init.py: auto-detects project type (Python/Node/Rust/Go/Java), creates harness.yaml + .harness/{tasks,reports}/ dirs, optional CI workflow - Register 'harness init' + full harness group in harness-manifest.json - Wire sk.py _run_harness() to dispatch harness init subcommand - Add docs/HARNESS-PHILOSOPHY.md: why/what/how/when/limitations (+36% perf gap, SWE-bench/AgentBench references, decision matrix, sk architecture, best practices) - Add .github/skills/harness/SKILL.md: guided harness engineering workflow skill - Update docs/HARNESS.md: cross-reference to HARNESS-PHILOSOPHY.md - Add 20 I731 tests in test_fixes.py covering: project detection, yaml generation, idempotency, --force, --skeleton-only, --json, manifest registration, sk.py routing Resolves #731, #732, #733 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Adds a new “universal harness scaffolding” workflow to the sk harness namespace by introducing sk harness init, wiring it into sk.py routing and the harness manifest, and documenting the harness philosophy/skill. The goal is to let users quickly scaffold a harness.yaml + .harness/ structure (and optional CI workflow) across multiple project types.
Changes:
- Introduces
harness-init.pyimplementingsk harness init(project detection + scaffold generation + optional GitHub Actions workflow). - Registers/routs the new subcommand via
harness-manifest.jsonandsk.pyhelp + dispatch logic. - Adds I731 tests in
test_fixes.pyplus harness docs/skill content.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 12 comments.
Show a summary per file
| File | Description |
|---|---|
harness-init.py |
New scaffolding script: project detection, harness.yaml template, .harness/ dirs, optional CI workflow, .gitignore updates. |
sk.py |
Routes sk harness init to the new script and updates help/usage text. |
harness-manifest.json |
Adds a new harness group and registers harness init/show/check/doctor/config entries. |
test_fixes.py |
Adds I731 tests covering detection, generation, idempotency, skeleton-only behavior, and routing/manifest checks. |
docs/HARNESS.md |
Adds a cross-reference to the new philosophy document. |
docs/HARNESS-PHILOSOPHY.md |
New long-form philosophy/motivation document for harness engineering. |
.github/skills/harness/SKILL.md |
New /harness skill content describing harness workflows and command usage. |
Comment on lines
+37
to
+41
| reporting: | ||
| format: "jsonl" | ||
| output_dir: ".copilot/harness-reports/" | ||
| telemetry: true | ||
|
|
Comment on lines
+90
to
+105
| markers = [ | ||
| ("pyproject.toml", "python-uv", | ||
| "python3 -m pytest", "python3 -m ruff check ."), | ||
| ("requirements.txt", "python-pip", | ||
| "python3 -m pytest", "python3 -m ruff check . 2>/dev/null || true"), | ||
| ("setup.py", "python-pip", | ||
| "python3 -m pytest", "python3 -m ruff check . 2>/dev/null || true"), | ||
| ("Cargo.toml", "rust", | ||
| "cargo test", "cargo clippy -- -D warnings"), | ||
| ("go.mod", "go", | ||
| "go test ./...", "golangci-lint run 2>/dev/null || go vet ./..."), | ||
| ("pom.xml", "java-maven", | ||
| "mvn test -q", "mvn checkstyle:check -q 2>/dev/null || true"), | ||
| ("build.gradle", "java-gradle", | ||
| "./gradlew test -q", "./gradlew check -q 2>/dev/null || true"), | ||
| ] |
Comment on lines
+120
to
+122
| proj_type = "node" | ||
| test_cmd = f"{pm} test" | ||
| lint_cmd = f"{pm} run lint 2>/dev/null || true" |
Comment on lines
+243
to
+245
| if args.skeleton_only: | ||
| print("✓ .harness/ directory initialized (skeleton-only)") | ||
| return 0 |
Comment on lines
+219
to
+223
| harness_yaml_path = os.path.join(target, "harness.yaml") | ||
| if os.path.exists(harness_yaml_path) and not args.force and not args.skeleton_only: | ||
| print(f"✗ harness.yaml already exists at {harness_yaml_path}") | ||
| print(" Use --force to overwrite.") | ||
| return 1 |
Comment on lines
+883
to
885
| print("sk harness subcommands: init, config, show, check, doctor") | ||
| print(" sk harness init [--target PATH] [--name NAME] [--skeleton-only] [--ci] [--yes]") | ||
| print(" sk harness config list|get|set") |
Comment on lines
+660
to
+663
| "harness init": { | ||
| "script": "harness-init.py", | ||
| "description": "Scaffold universal project harness (harness.yaml + .harness/ + CI workflow)", | ||
| "tags": [ |
Comment on lines
+701
to
+704
| "harness config": { | ||
| "script": "sk.py", | ||
| "description": "Manage harness env vars (SK_HARNESS, SK_DRY_RUN, SK_DEBUG_TIMING)", | ||
| "tags": [ |
Comment on lines
+660
to
+663
| "harness init": { | ||
| "script": "harness-init.py", | ||
| "description": "Scaffold universal project harness (harness.yaml + .harness/ + CI workflow)", | ||
| "tags": [ |
Comment on lines
+701
to
+704
| "harness config": { | ||
| "script": "sk.py", | ||
| "description": "Manage harness env vars (SK_HARNESS, SK_DRY_RUN, SK_DEBUG_TIMING)", | ||
| "tags": [ |
… (ruff F541) harness-init.py:294 had f"..." with no placeholders — fixed to plain string. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…mat check) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Fix output_dir template: .copilot/harness-reports/ → .harness/reports/ - Remove non-portable POSIX 2>/dev/null shell from lint_cmd defaults - Add JSON emit path for --skeleton-only + --json - Add JSON error path when harness.yaml already exists - Trim SKILL.md to 160 lines (was 237, AC required <200) - Expand sk.py help text with --no-ci, --force, --json flags - Add harness-init.py to install.py TOOL_FILES - Fix harness config manifest description to include SK_TOOLS_DIR Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This was referenced May 30, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements
sk harness init— a universal harness scaffolding command for any project type, plus philosophy documentation and a guided skill.Changes
New files
harness-init.py(306 lines): Universal harness init scriptharness.yamlwith detected test/lint commands.harness/{tasks,reports}/directories.github/workflows/harness-ci.yml.gitignoreto ignore report outputs--force--target,--name,--skeleton-only,--ci,--no-ci,--yes,--force,--jsondocs/HARNESS-PHILOSOPHY.md(552 lines): Why/What/How/When harness engineering.github/skills/harness/SKILL.md(237 lines): Guided/harnessskillModified files
harness-manifest.json: Addedharnessgroup with 5 entries (init, show, check, doctor, config)sk.py: Addedinitsubcommand routing + updated help textdocs/HARNESS.md: Added cross-reference to HARNESS-PHILOSOPHY.mdtest_fixes.py: 20 new I731 tests (detection, yaml generation, idempotency, skeleton-only, manifest)Test evidence
All 20 new tests pass + all prior tests green:
Security: 13/13 passed.
Resolves
Closes #731, #732, #733