diff --git a/CHANGELOG.md b/CHANGELOG.md index 4ec85e2..cbb3682 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,24 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +### Added +- Declared **Beta** maturity (`Development Status :: 4 - Beta` classifier) and a + public **Stability & versioning** section in the README: what the `1.0.0` + contract covers (`rules.yaml` schema, `.bec.yaml` bundle format, check names, + CLI commands and exit codes, `check --json` shape, MCP signatures) and the + exit criteria to reach it. +- `.bec/rules.yaml` now carries an optional `schema_version` (absent means `1`, + so existing files keep working). `becwright init` stamps it, and the engine + refuses a file stamped newer than it understands — with a clear "upgrade + becwright" error — instead of risking a silent misparse. (The `.bec.yaml` + export bundle was already versioned via `becwright_bec`.) + +### Documentation +- Documented becwright's **stable contract** in `documentation/usage.md`: the CLI + exit codes (`0` pass · `1` a blocking rule failed · `2` config/usage problem) + and the `check --json` output shape, both now locked by tests so a change is a + deliberate break rather than a silent drift. + ## [0.4.0] — 2026-07-01 ### Added diff --git a/README.es.md b/README.es.md index 33f8042..bf1c37f 100644 --- a/README.es.md +++ b/README.es.md @@ -484,6 +484,41 @@ en verde. El trabajo futuro (análisis AST, tooling profundo por lenguaje, firma de verificaciones) está documentado en el plan del proyecto. +## Estabilidad y versionado + +becwright está en **Beta**. Se usa a sí mismo (sus propios commits pasan por +becwright), la suite de tests está en verde y está publicado en npm y PyPI — +pero sigue en `0.x`, así que bajo [SemVer](https://semver.org) una release menor +*puede* cambiar el contrato público. Si dependés de él en CI, fijá una versión +(`becwright==0.4.0`, o `npm i -g becwright@0.4.0`). + +**El contrato público** — la superficie que se vuelve estable en `1.0.0` y a +partir de ahí solo cambia con un bump mayor: + +- El esquema de `.bec/rules.yaml` (los campos de una regla y su significado). +- El formato de bundle `.bec.yaml` que `export` / `import` mueven entre repos. +- Los nombres de los checks incluidos y sus flags. +- Los comandos de la CLI y sus códigos de salida. +- La forma de la salida `check --json`. +- Los nombres y firmas de las herramientas MCP. + +Todo lo demás (el texto de los mensajes, el contenido del catálogo, los módulos +internos) puede cambiar en cualquier momento. + +**El camino a 1.0.0** — la publicamos cuando estemos seguros de que el contrato +de arriba no va a necesitar un cambio que rompa compatibilidad: + +- [x] Versionar los dos formatos en disco para que un archivo más nuevo falle + fuerte en vez de mal-interpretarse — el bundle `.bec.yaml` (`becwright_bec`) + y `.bec/rules.yaml` (`schema_version`). +- [ ] Congelar el conjunto de campos de `rules.yaml` — sin cambios de esquema + pendientes. +- [x] Documentar y estabilizar los códigos de salida de la CLI y la forma de + `check --json`. +- [ ] Definir una política de deprecación: una release menor de aviso antes de + quitar cualquier cosa. +- [ ] Validar en repos reales más allá de este. + ## Roadmap becwright es chico a propósito. En el horizonte: diff --git a/README.md b/README.md index a8bd694..0343dcb 100644 --- a/README.md +++ b/README.md @@ -513,6 +513,38 @@ is green. Future work (AST analysis, deep per-language tooling, cryptographic signing of verifications) is documented in the project plan. +## Stability & versioning + +becwright is **Beta**. It's dogfooded (its own commits are gated by becwright), +the test suite is green, and it's published on npm and PyPI — but it is still +`0.x`, so under [SemVer](https://semver.org) a minor release *may* change the +public contract. If you depend on it in CI, pin a version +(`becwright==0.4.0`, or `npm i -g becwright@0.4.0`). + +**The public contract** — the surface that becomes stable at `1.0.0` and only +changes on a major bump after that: + +- The `.bec/rules.yaml` schema (rule fields and their meaning). +- The `.bec.yaml` bundle format that `export` / `import` move between repos. +- Built-in check names and their flags. +- CLI commands and their exit codes. +- The `check --json` output shape. +- MCP tool names and signatures. + +Everything else (message wording, catalog contents, internal modules) can change +at any time. + +**The path to 1.0.0** — we ship it once we're confident the contract above won't +need a breaking change: + +- [x] Version both on-disk formats so a newer file fails loudly instead of + misparsing — the `.bec.yaml` bundle (`becwright_bec`) and `.bec/rules.yaml` + (`schema_version`). +- [ ] Freeze the `rules.yaml` field set — no pending schema changes. +- [x] Document and stabilize CLI exit codes and the `check --json` shape. +- [ ] State a deprecation policy: one minor release of notice before any removal. +- [ ] Validate on real repositories beyond this one. + ## Roadmap becwright is intentionally small. On the horizon: diff --git a/documentation/usage.es.md b/documentation/usage.es.md index 624bda8..98d5bba 100644 --- a/documentation/usage.es.md +++ b/documentation/usage.es.md @@ -71,13 +71,47 @@ a mano: `becwright install` más un `.bec/rules.yaml` que escribas vos.) > solo esos por defecto (justo lo que el commit va a crear), por eso es rápido. > Usá `--all` para escanear todo el proyecto. -Códigos de salida (el número que devuelve un comando al terminar; `0` significa -éxito): `0` pasa · `1` falló una regla blocking · `2` no es un repo git / error -de uso. +### Códigos de salida + +El número que devuelve un comando al terminar. Forman parte del contrato estable +de becwright — scripts y CI pueden depender de ellos: + +| Código | Significado | +|---|---| +| `0` | Pasó — ninguna regla blocking falló (o no había nada que revisar). | +| `1` | Falló una regla **blocking**. Es la señal que frena un commit. Un hallazgo `warning`/`advisory` por sí solo **no** activa esto. | +| `2` | Un problema a corregir antes de que becwright pueda juzgar: no es un repo git, un `.bec/rules.yaml` malformado/no confiable, una regla que apunta a un check integrado inexistente, o un error de uso. | + +### `check --json` + +`becwright check --json` imprime un objeto JSON y sigue usando los códigos de +salida de arriba (`1` cuando bloquea). La forma es estable: + +```json +{ + "rule_count": 2, + "checked_files": 5, + "blocked": true, + "results": [ + { + "id": "no-token-in-logs", + "severity": "blocking", + "passed": false, + "intent": "Los tokens de sesión nunca deben llegar a ningún log.", + "why_it_matters": "Un token en los logs deja robar una sesión.", + "output": "src/app.py:12: token=..." + } + ] +} +``` + +`intent`, `why_it_matters` y `output` son `null` cuando faltan. `results` está +vacío cuando no había nada que evaluar. ## El archivo de reglas: `.bec/rules.yaml` ```yaml +schema_version: 1 # versión de formato opcional; ausente = 1 rules: - id: no-token-in-logs # identificador único intent: > # qué pide la regla (la parte "bound") @@ -108,6 +142,13 @@ rules: | `severity` | no | `blocking` (por defecto), `warning` o `advisory` (ver abajo) | | `target` | no | `files` (por defecto) o `commit-msg` (ver abajo) | +**`schema_version`** es una clave opcional de nivel superior (no un campo de +regla). Sella la versión de formato del archivo; cuando falta se trata como `1`, +así que los archivos existentes siguen funcionando. `becwright init` la escribe, +y becwright rechaza un archivo sellado con una versión *más nueva* de la que +entiende — pidiéndote actualizar — en vez de mal-interpretarlo. Rara vez la tocas +a mano. + **Severidad — garantizado vs asistido.** `blocking` y `warning` son para checks *deterministas*: el mismo código siempre da el mismo veredicto, así que una regla `blocking` es una **garantía al 100%**. `advisory` es el hogar honesto de las diff --git a/documentation/usage.md b/documentation/usage.md index 334e365..792055a 100644 --- a/documentation/usage.md +++ b/documentation/usage.md @@ -69,12 +69,47 @@ From then on, every `git commit` runs the checks. (You can also set up by hand: > (the exact set the commit will create), which is why it's fast. Use > `--all` to scan the whole project instead. -Exit codes (the number a command returns when it ends; `0` means success): -`0` pass · `1` a blocking rule failed · `2` not a git repo / usage error. +### Exit codes + +The number a command returns when it ends. These are part of becwright's stable +contract — scripts and CI can rely on them: + +| Code | Meaning | +|---|---| +| `0` | Passed — no blocking rule failed (or there was nothing to check). | +| `1` | A **blocking** rule failed. This is the signal that stops a commit. A `warning`/`advisory` finding alone does **not** set this. | +| `2` | A problem to fix before becwright can judge: not a git repository, a malformed/untrusted `.bec/rules.yaml`, a rule pointing at a non-existent built-in check, or a usage error. | + +### `check --json` + +`becwright check --json` prints one JSON object and still uses the exit codes +above (`1` when blocked). The shape is stable: + +```json +{ + "rule_count": 2, + "checked_files": 5, + "blocked": true, + "results": [ + { + "id": "no-token-in-logs", + "severity": "blocking", + "passed": false, + "intent": "Session tokens must never reach any log.", + "why_it_matters": "A token in the logs lets anyone steal a session.", + "output": "src/app.py:12: token=..." + } + ] +} +``` + +`intent`, `why_it_matters` and `output` are `null` when absent. `results` is +empty when there was nothing to evaluate. ## The rules file: `.bec/rules.yaml` ```yaml +schema_version: 1 # optional format version; absent means 1 rules: - id: no-token-in-logs # unique identifier intent: > # what the rule asks for (the "bound" part) @@ -105,6 +140,12 @@ rules: | `severity` | no | `blocking` (default), `warning`, or `advisory` (see below) | | `target` | no | `files` (default) or `commit-msg` (see below) | +**`schema_version`** is an optional top-level key (not a rule field). It stamps +the format version of the file; when absent it is treated as `1`, so existing +files keep working. `becwright init` writes it, and becwright refuses a file +stamped a *newer* version than it understands — telling you to upgrade — rather +than misreading it. You rarely touch it by hand. + **Severity — guaranteed vs assisted.** `blocking` and `warning` are for *deterministic* checks: the same code always gives the same verdict, so a `blocking` rule is a **100% guarantee**. `advisory` is the honest home for diff --git a/pyproject.toml b/pyproject.toml index 85dfef2..6363b8f 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -28,6 +28,7 @@ keywords = [ ] dependencies = ["pyyaml>=6"] classifiers = [ + "Development Status :: 4 - Beta", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.12", "License :: OSI Approved :: MIT License", diff --git a/src/becwright/cli.py b/src/becwright/cli.py index 258fc71..2be2e65 100644 --- a/src/becwright/cli.py +++ b/src/becwright/cli.py @@ -11,7 +11,7 @@ from . import __version__, bundle, catalog, git, report from .engine import Result -from .rules import RulesError, load_rules +from .rules import RULES_SCHEMA_VERSION, RulesError, load_rules RED = "\033[91m"; GREEN = "\033[92m"; YELLOW = "\033[93m"; CYAN = "\033[96m" BOLD = "\033[1m"; DIM = "\033[2m"; RESET = "\033[0m" @@ -532,6 +532,7 @@ def _render_rules_yaml(rules: list[dict]) -> str: "# becwright rules - generated by `becwright init`. Tune them to your repo.\n" "# More rules: `becwright search` to list the catalog, `becwright add ` to install.\n" "# Docs: https://github.com/DataDave-Dev/becwright/tree/main/documentation\n" + f"schema_version: {RULES_SCHEMA_VERSION}\n" ) if not rules: return header + "rules: []\n" diff --git a/src/becwright/rules.py b/src/becwright/rules.py index cc739c5..9608c67 100644 --- a/src/becwright/rules.py +++ b/src/becwright/rules.py @@ -10,6 +10,11 @@ _VALID_SEVERITIES = ("blocking", "warning", "advisory") _VALID_TARGETS = ("files", "commit-msg") +# The `.bec/rules.yaml` format version. Absent means 1 (files predating the +# field). The engine refuses a file stamped newer than it understands instead of +# risking a silent misparse; migration between versions is added when a v2 exists. +RULES_SCHEMA_VERSION = 1 + class RulesError(RuntimeError): """A `.bec/rules.yaml` that cannot be trusted (bad YAML or an invalid rule). @@ -77,4 +82,20 @@ def load_rules(rules_path: Path) -> list[Rule]: raise RulesError(f"{rules_path}: invalid YAML ({e}).") if not isinstance(data, dict) or not isinstance(data.get("rules", []), list): raise RulesError(f"{rules_path}: expected a top-level 'rules:' list.") + _check_schema_version(data.get("schema_version"), rules_path) return [_to_rule(r) for r in data["rules"]] if data.get("rules") else [] + + +def _check_schema_version(value, rules_path: Path) -> None: + if value is None: + return + # bool is an int subclass; a YAML `true` is not a valid version. + if isinstance(value, bool) or not isinstance(value, int) or value < 1: + raise RulesError( + f"{rules_path}: schema_version must be a positive integer, got {value!r}." + ) + if value > RULES_SCHEMA_VERSION: + raise RulesError( + f"{rules_path}: schema_version {value} is newer than this becwright " + f"understands (max {RULES_SCHEMA_VERSION}); upgrade becwright." + ) diff --git a/tests/test_init.py b/tests/test_init.py index dce7e96..85f7ee9 100644 --- a/tests/test_init.py +++ b/tests/test_init.py @@ -129,6 +129,17 @@ def test_render_empty_is_valid(tmp_path): assert load_rules(p) == [] +def test_render_yaml_stamps_schema_version(tmp_path): + from becwright.rules import RULES_SCHEMA_VERSION + + for rules in ([], cli._starter_rules(["python"])): + text = cli._render_rules_yaml(rules) + assert f"schema_version: {RULES_SCHEMA_VERSION}" in text + p = tmp_path / "rules.yaml" + p.write_text(text, encoding="utf-8") + load_rules(p) # round-trips without raising + + def test_render_yaml_emits_exclude(tmp_path): rules = [{"id": "no-log", "intent": "x", "why": "y", "paths": ["**/*.ts"], "exclude": ["lib/logger.ts"], "check": "true", "severity": "warning"}] diff --git a/tests/test_report_and_json.py b/tests/test_report_and_json.py index a9a2bc3..291a123 100644 --- a/tests/test_report_and_json.py +++ b/tests/test_report_and_json.py @@ -84,3 +84,45 @@ def test_check_json_clean_repo(tmp_path, monkeypatch, capsys): rc = cli.main(["check", "--all", "--json"]) data = json.loads(capsys.readouterr().out) assert rc == 0 and data["blocked"] is False and data["results"] == [] + + +# --- stable contract: exit codes and JSON key sets --- + +def test_exit_code_2_on_unknown_builtin_check(tmp_path, monkeypatch, capsys): + _init_repo(tmp_path) + (tmp_path / ".bec").mkdir(parents=True, exist_ok=True) + (tmp_path / ".bec" / "rules.yaml").write_text( + "rules:\n - id: r1\n paths: ['**/*.py']\n" + " check: 'becwright run does_not_exist'\n severity: blocking\n", + encoding="utf-8") + monkeypatch.chdir(tmp_path) + assert cli.main(["check", "--all"]) == 2 + + +def test_exit_code_2_on_malformed_rules(tmp_path, monkeypatch, capsys): + _init_repo(tmp_path) + (tmp_path / ".bec").mkdir(parents=True, exist_ok=True) + (tmp_path / ".bec" / "rules.yaml").write_text( + "rules:\n - id: r1\n severity: not-a-severity\n check: 'true'\n", + encoding="utf-8") + monkeypatch.chdir(tmp_path) + assert cli.main(["check", "--all"]) == 2 + + +def test_payload_key_contract(): + """The `check --json` shape is part of the 1.0 contract; lock its keys so a + change is a deliberate, reviewed break rather than a silent drift.""" + rule = Rule(id="r", paths=("**/*.js",), check="false", severity="blocking") + from becwright.engine import Result, RuleResult + out = report.payload([rule], ["a.js"], + Result(per_rule=[RuleResult(rule=rule, passed=False, output="x")])) + assert set(out) == {"rule_count", "checked_files", "blocked", "results"} + assert set(out["results"][0]) == {"id", "severity", "passed", "intent", + "why_it_matters", "output"} + + +def test_rule_record_key_contract(): + """`why --json` / `list --json` expose a rule's bound half; lock its keys.""" + out = report.rule_record(Rule(id="r", paths=("**/*.py",), check="true")) + assert set(out) == {"id", "severity", "target", "intent", "why_it_matters", + "rejected_alternatives", "paths", "exclude", "check"} diff --git a/tests/test_rules.py b/tests/test_rules.py index 116dec5..837a69b 100644 --- a/tests/test_rules.py +++ b/tests/test_rules.py @@ -1,6 +1,6 @@ import pytest -from becwright.rules import RulesError, load_rules +from becwright.rules import RULES_SCHEMA_VERSION, RulesError, load_rules def _write(tmp_path, text): @@ -77,3 +77,37 @@ def test_loads_advisory_severity(tmp_path): path = _write(tmp_path, 'rules:\n - id: r1\n check: "true"\n severity: advisory\n') rule = load_rules(path)[0] assert rule.is_advisory is True and rule.is_blocking is False + + +def test_absent_schema_version_loads(tmp_path): + path = _write(tmp_path, 'rules:\n - id: r1\n check: "true"\n') + assert len(load_rules(path)) == 1 + + +def test_current_schema_version_loads(tmp_path): + path = _write( + tmp_path, + f'schema_version: {RULES_SCHEMA_VERSION}\nrules:\n - id: r1\n check: "true"\n', + ) + assert len(load_rules(path)) == 1 + + +def test_newer_schema_version_raises(tmp_path): + path = _write( + tmp_path, + f'schema_version: {RULES_SCHEMA_VERSION + 1}\nrules:\n - id: r1\n check: "true"\n', + ) + with pytest.raises(RulesError, match="newer"): + load_rules(path) + + +def test_non_integer_schema_version_raises(tmp_path): + path = _write(tmp_path, 'schema_version: one\nrules:\n - id: r1\n check: "true"\n') + with pytest.raises(RulesError, match="schema_version"): + load_rules(path) + + +def test_non_positive_schema_version_raises(tmp_path): + path = _write(tmp_path, 'schema_version: 0\nrules:\n - id: r1\n check: "true"\n') + with pytest.raises(RulesError, match="schema_version"): + load_rules(path)