Skip to content

chore: align optionals repository guardrails#64

Merged
martinfrancois merged 5 commits into
mainfrom
chore/align-optionals-repo-guardrails
Jun 8, 2026
Merged

chore: align optionals repository guardrails#64
martinfrancois merged 5 commits into
mainfrom
chore/align-optionals-repo-guardrails

Conversation

@martinfrancois

@martinfrancois martinfrancois commented Jun 8, 2026

Copy link
Copy Markdown
Owner

Summary

  • Aligns Optionals CI/release guardrails with the Streams repo: Tessl CLI 0.82.0, stronger publish ref validation, Release Please status recovery, pinned action digest updates, scoped Tessl review token use, and Renovate tracking of the npm tessl CLI package.
  • Extracts nontrivial GitHub Actions Bash/Python into scripts so workflow logic is easier to validate with bash -n, Python compilation, smoke tests, and future shellcheck.
  • Ports the main/reference/regression eval split tooling and docs: scripts/run_eval_suite.sh, scripts/classify_eval_result.py, regression evidence_type, validator checks for suite placement, stale scenario references, duplicate scenarios, and docs/agents self-containment.
  • Restores useful PR template guidance while keeping the newer eval-suite validation checklist.
  • Updates public metadata wording to explicitly cover new code, review, and cleanup, matching the Streams positioning.
  • Keeps runtime skill behavior unchanged: no SKILL.md or runtime reference edits, no active eval task/scoring/membership changes.

Commit/Risk Areas

  • ci: align optionals automation guardrails
    • CI/release hardening, Tessl CLI pin update, and extracted workflow helper scripts.
    • Not lift-sensitive for hosted eval scoring.
  • test(evals): add suite guardrail tooling
    • Adds eval runner/classifier tooling and strengthens local eval validation.
    • Adds machine-readable regression evidence metadata required by the validator.
    • Active eval tasks, scoring, and membership are unchanged.
  • docs(evals): document suite workflow policy
    • Maintainer/contributor docs for main/reference/regression suite policy and rate-limit-conscious eval order.
    • Docs only; not lift-sensitive.
  • docs: restore pull request template guidance
    • Restores useful template guidance from review comments: change-type instruction, bug-fix details, validation explanation, authentication note, and human verification.
    • Docs only; not lift-sensitive.
  • docs: clarify optionals public metadata
    • Public package metadata/docs wording only.
    • No runtime SKILL.md or referenced skill files changed.

Validation

Local and Tessl checks run after the final history rewrite:

scripts/validate_repo.sh
=> Skill is valid.
=> Validated 52 scenario(s): 4 natural, 48 explicit.
=> Warning-only runtime-reference overlap notices for existing focused Optionals coverage.
=> JSON ok
=> YAML metadata ok
=> Plugin martinfrancois/java-optionals@1.0.0 is valid

python3 - << JSON parse
=> JSON ok

python3 - << YAML parse
=> PyYAML unavailable; skipped locally

actionlint v1.7.12 against .github/workflows/*.yml
=> pass

tessl skill review --threshold 100 skills/java-optionals/SKILL.md
=> Review Score: 100%

bash scripts/check_publish_dry_run.sh .
=> next patch 1.0.1 dry-run passed

tessl plugin publish --dry-run --bump patch .
=> next patch 1.0.1 dry-run passed

Extracted script smoke tests:
=> `scripts/check_tessl_token_available.sh` available/unavailable paths
=> `scripts/validate_publish_ref.sh` release-tag and non-tag override paths
=> `scripts/read_release_pr_output.sh` Release Please JSON parsing path
=> `scripts/install_commitlint.sh` writes COMMITLINT env output
=> `scripts/lint_pr_commits.sh` non-PR branch path with stubbed commitlint binary
=> `scripts/post_commit_status_for_outcome.sh` invalid-outcome rejection
=> `scripts/trigger_tessl_publish.sh` missing-env rejection

Workflow render checks previously passed for this branch:

gh workflow view ci.yml --ref chore/align-optionals-repo-guardrails --yaml => ok
gh workflow view release-please.yml --ref chore/align-optionals-repo-guardrails --yaml => ok
gh workflow view publish-tessl.yml --ref chore/align-optionals-repo-guardrails --yaml => ok
gh workflow view commitlint.yml --ref chore/align-optionals-repo-guardrails --yaml => ok
gh workflow view skill-review.yml --ref chore/align-optionals-repo-guardrails --yaml => ok

Additional review:

codex --dangerously-bypass-approvals-and-sandbox review --base main
=> no actionable findings before the later PR-template/workflow extraction docs-only/CI-only updates; later changes are covered by local validation/actionlint/smoke tests.

composer-style hosted evals were not run because runtime skill text, runtime references, active eval tasks, active eval scoring, and active eval membership were not changed.

Hosted Evals

No hosted evals were run for this PR because runtime skill text, runtime references, active eval tasks, active eval scoring, and active eval membership were not changed. The docs now make the same suite policy as Streams explicit: main/reference run both variants, regression runs with-context only by default, and pure suite moves do not require reruns when task/scoring/capability text is unchanged.

Notes

  • Package-runner install examples were preserved.
  • Plugin package format was preserved; pinned Tessl CLI 0.82.0 validates the current .tessl-plugin/plugin.json format.
  • Benchmark claims were not strengthened.
  • Release Please always-update was not added. The pinned v5 action metadata does not expose an always-update input; the fallback lookup plus extracted scripts keeps the current behavior explicit and testable.

Comment thread .github/pull_request_template.md
Comment thread .github/pull_request_template.md
Comment thread .github/pull_request_template.md
Comment thread .github/pull_request_template.md
@martinfrancois martinfrancois force-pushed the chore/align-optionals-repo-guardrails branch 6 times, most recently from 0d27437 to ba5b86c Compare June 8, 2026 12:51
@martinfrancois martinfrancois force-pushed the chore/align-optionals-repo-guardrails branch from ba5b86c to b2cd1e1 Compare June 8, 2026 13:04
@martinfrancois martinfrancois merged commit d79fc5e into main Jun 8, 2026
8 checks passed
@martinfrancois martinfrancois deleted the chore/align-optionals-repo-guardrails branch June 8, 2026 13:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant