From a10fd2f3554dd43442d27d743803709c454c2521 Mon Sep 17 00:00:00 2001 From: Quratulain-bilal Date: Mon, 13 Jul 2026 19:10:14 +0500 Subject: [PATCH 01/45] docs: document copilot skills mode (--skills) and markdown deprecation (#3313) * docs: document copilot skills mode (--skills) and markdown deprecation as of #3256 (v0.12.3) the copilot integration supports a skills mode via `--integration-options "--skills"`, and installing without it warns that the legacy markdown default is being phased out. this was undocumented: - the copilot row in the supported-agents table had an empty notes cell while other skills-capable agents describe their behavior there. - `--skills` was missing from the integration-specific options table (only generic and kimi were listed). fill both. wording matches the code: skills scaffold as speckit-/SKILL.md under .github/skills/ and are invoked as /speckit-; without the flag the install emits the deprecation warning from _warn_legacy_markdown_default(). fixes #3300 * docs: describe copilot default as legacy markdown mode (.agent.md + .prompt.md) the copilot rows said the default installs .agent.md files, but the default scaffold also writes companion .prompt.md files under .github/prompts/. also reworded to 'legacy markdown mode' to match the deprecation warning users actually see and to avoid ambiguity, since skills are markdown too. * docs: spell out copilot legacy markdown paths and use in copilot rows address the follow-up copilot review: name where the default scaffold writes files (.github/agents/*.agent.md plus .github/prompts/*.prompt.md and a .vscode/settings.json merge), and switch speckit- to speckit- to match the rest of the table. verified all three paths against the copilot integration source. * docs: use --integration-options="..." form in copilot notes cell match the equals form the rest of the doc uses (generic row, the options table, and the install example) so readers don't mistake the quotes for part of the value. addresses copilot review feedback. --- docs/reference/integrations.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/reference/integrations.md b/docs/reference/integrations.md index f538fac86f..df5e8801ed 100644 --- a/docs/reference/integrations.md +++ b/docs/reference/integrations.md @@ -18,7 +18,7 @@ The Specify CLI supports a wide range of AI coding agents. When you run `specify | [Firebender](https://firebender.com/) | `firebender` | IDE-based agent for Android Studio / IntelliJ | | [Forge](https://forgecode.dev/) | `forge` | | | [Gemini CLI](https://github.com/google-gemini/gemini-cli) | `gemini` | | -| [GitHub Copilot](https://code.visualstudio.com/) | `copilot` | | +| [GitHub Copilot](https://code.visualstudio.com/) | `copilot` | Defaults to legacy markdown mode: `.agent.md` command files under `.github/agents/`, companion `.prompt.md` files under `.github/prompts/`, and a `.vscode/settings.json` merge. Pass `--integration-options="--skills"` to scaffold skills as `speckit-/SKILL.md` under `.github/skills/` instead. Legacy markdown mode is deprecated and will stop being the default in a future release. | | [Goose](https://goose-docs.ai/) | `goose` | Uses YAML recipe format in `.goose/recipes/` | | [Hermes](https://github.com/NousResearch/hermes-agent) | `hermes` | Skills-based integration; installs skills globally into `~/.hermes/skills/` | | [IBM Bob](https://www.ibm.com/products/bob) | `bob` | IDE-based agent | @@ -219,6 +219,7 @@ Some integrations accept additional options via `--integration-options`: | ----------- | ------------------- | -------------------------------------------------------------- | | `generic` | `--commands-dir` | Required. Directory for command files | | `kimi` | `--migrate-legacy` | Migrate legacy `.kimi/skills/` installs to `.kimi-code/skills/` (including dotted→hyphenated skill naming, e.g. `speckit.xxx` → `speckit-xxx`) | +| `copilot` | `--skills` | Scaffold commands as agent skills (`speckit-/SKILL.md` under `.github/skills/`, invoked as `/speckit-`) instead of the default legacy markdown mode (`.github/agents/*.agent.md` plus `.github/prompts/*.prompt.md` and a `.vscode/settings.json` merge). Without this flag, install warns that legacy markdown mode is deprecated. | Example: From e59da78677ca8bd271ecbf7f55dea5f26b1375b4 Mon Sep 17 00:00:00 2001 From: Ali jawwad <33836051+jawwad-ali@users.noreply.github.com> Date: Mon, 13 Jul 2026 19:14:32 +0500 Subject: [PATCH 02/45] fix(extensions): handle prefix-colliding env vars in _get_env_config (#3350) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * fix(extensions): handle prefix-colliding env vars in _get_env_config _get_env_config built the nested dict with 'if part not in current: current[part] = {}' and an unconditional leaf assignment. Two env vars that collide on a prefix — e.g. SPECKIT_X_CONNECTION and SPECKIT_X_CONNECTION_URL — then either crash (scalar processed first: the walk indexes into a str -> TypeError 'str object does not support item assignment') or silently clobber the nested dict (scalar processed last). Via should_execute_hook's blanket except, the crash silently disables every config-based hook for the extension. Guard the walk and the leaf assignment with isinstance checks so a colliding scalar yields to the nested dict; result is order-independent ({'connection': {'url': ...}} either way), matching _merge_configs' dict-preserving semantics. Co-Authored-By: Claude Fable 5 * test(extensions): assert via public should_execute_hook, not private helper Per review: the colliding-env hook test described should_execute_hook swallowing the TypeError, but asserted on the private _evaluate_condition. Assert on the public should_execute_hook instead — it returns False (silently disabled) before the fix and True after, matching the real-world failure mode and not coupling to a private helper. Co-Authored-By: Claude Fable 5 * fix(extensions): ignore malformed env var names in _get_env_config Per review: a name like SPECKIT__ (no key) or with consecutive underscores produced empty path components, creating surprising entries under an empty key (env_config[''] = ...). Filter out empty parts and skip the variable entirely when nothing remains. Co-Authored-By: Claude Fable 5 --------- Co-authored-by: Claude Fable 5 --- src/specify_cli/extensions/__init__.py | 26 ++++++++++--- tests/test_extensions.py | 53 ++++++++++++++++++++++++++ 2 files changed, 73 insertions(+), 6 deletions(-) diff --git a/src/specify_cli/extensions/__init__.py b/src/specify_cli/extensions/__init__.py index b84b836545..1f898c5f78 100644 --- a/src/specify_cli/extensions/__init__.py +++ b/src/specify_cli/extensions/__init__.py @@ -2755,18 +2755,32 @@ def _get_env_config(self) -> Dict[str, Any]: if not key.startswith(prefix): continue - # Remove prefix and split into parts - config_path = key[len(prefix) :].lower().split("_") + # Remove prefix and split into parts. Drop empty components from a + # malformed name (e.g. ``SPECKIT__`` with no key, or + # consecutive underscores ``SPECKIT_X__Y``) so we never create an + # entry under an empty key. + config_path = [p for p in key[len(prefix) :].lower().split("_") if p] + if not config_path: + continue - # Build nested dict + # Build nested dict. Two env vars can collide on a prefix, e.g. + # SPECKIT_X_CONNECTION=a and SPECKIT_X_CONNECTION_URL=b. Guard the + # walk so a colliding scalar is replaced by a dict (deeper/more + # specific vars win) instead of being indexed into — which raised + # TypeError ('str' object does not support item assignment) — and + # guard the leaf so a scalar processed after the nested var does + # not clobber the nested dict. Order-independent: both insertion + # orders yield {'connection': {'url': ...}}. Nested-wins mirrors + # _merge_configs' dict-preserving semantics. current = env_config for part in config_path[:-1]: - if part not in current: + if not isinstance(current.get(part), dict): current[part] = {} current = current[part] - # Set the final value - current[config_path[-1]] = value + # Set the final value, unless a nested dict already occupies it. + if not isinstance(current.get(config_path[-1]), dict): + current[config_path[-1]] = value return env_config diff --git a/tests/test_extensions.py b/tests/test_extensions.py index 2885180360..cb3120f492 100644 --- a/tests/test_extensions.py +++ b/tests/test_extensions.py @@ -7628,3 +7628,56 @@ def test_hook_condition_returns_false_without_raising(self, tmp_path): (ext_dir / "jira-config.yml").write_text("just a string\n", encoding="utf-8") executor = HookExecutor(tmp_path) assert executor._evaluate_condition("config.x is set", "jira") is False + + +class TestConfigManagerEnvPrefixCollision: + """Prefix-colliding env vars must not crash or clobber nested config.""" + + def test_scalar_then_nested_yields_nested(self, tmp_path, monkeypatch): + """SPECKIT_X_CONNECTION=x then SPECKIT_X_CONNECTION_URL=y. + + The scalar-first order previously raised TypeError ('str' object + does not support item assignment) when the walk indexed into 'x'. + """ + monkeypatch.setenv("SPECKIT_TESTEXT_CONNECTION", "x") + monkeypatch.setenv("SPECKIT_TESTEXT_CONNECTION_URL", "y") + cm = ConfigManager(tmp_path, "testext") + assert cm._get_env_config() == {"connection": {"url": "y"}} + + def test_nested_then_scalar_does_not_clobber(self, tmp_path, monkeypatch): + """Reverse order previously returned {'connection': 'x'}, losing url.""" + monkeypatch.setenv("SPECKIT_TESTEXT_CONNECTION_URL", "y") + monkeypatch.setenv("SPECKIT_TESTEXT_CONNECTION", "x") + cm = ConfigManager(tmp_path, "testext") + assert cm._get_env_config() == {"connection": {"url": "y"}} + + def test_colliding_env_does_not_disable_hook_condition(self, tmp_path, monkeypatch): + """`config.connection.url is set` must stay True under colliding env. + + Before the fix the TypeError propagated into should_execute_hook's + blanket `except Exception: return False`, silently disabling the hook. + """ + ext_dir = tmp_path / ".specify" / "extensions" / "testext" + ext_dir.mkdir(parents=True) + (ext_dir / "testext-config.yml").write_text( + "connection:\n url: https://example.com\n", encoding="utf-8" + ) + monkeypatch.setenv("SPECKIT_TESTEXT_CONNECTION", "x") + monkeypatch.setenv("SPECKIT_TESTEXT_CONNECTION_URL", "y") + executor = HookExecutor(tmp_path) + # Exercise the public API: before the fix the TypeError was swallowed + # by should_execute_hook's `except Exception: return False`, so the + # hook was silently disabled (False); after the fix it returns True. + assert executor.should_execute_hook( + {"condition": "config.connection.url is set", "extension": "testext"} + ) is True + + def test_malformed_env_names_ignored(self, tmp_path, monkeypatch): + """A name with no key (SPECKIT_X_) or empty parts (consecutive + underscores) must not create an entry under an empty key.""" + monkeypatch.setenv("SPECKIT_TESTEXT_", "orphan") # no key at all + monkeypatch.setenv("SPECKIT_TESTEXT_A__B", "z") # empty middle part + cm = ConfigManager(tmp_path, "testext") + cfg = cm._get_env_config() + assert "" not in cfg + assert cfg == {"a": {"b": "z"}} From ba1f13a8b17a141afbdc1451f7652ef551e27de5 Mon Sep 17 00:00:00 2001 From: Ali jawwad <33836051+jawwad-ali@users.noreply.github.com> Date: Mon, 13 Jul 2026 19:15:55 +0500 Subject: [PATCH 03/45] =?UTF-8?q?fix(bundle):=20reject=20file://=20/=20loc?= =?UTF-8?q?al=20download=5Furl=20=E2=80=94=20catalog=20URLs=20are=20HTTPS-?= =?UTF-8?q?only=20(#3344)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * fix(bundle): resolve file:// download_url via the file-URL helper _download_manifest built the local path from raw parsed.path, which keeps the leading slash of file:///C:/x (yielding a \C:\x path that never exists on Windows) and skips percent-decoding (my%20bundles stays encoded on every OS) — so a catalog entry whose download_url is the canonical URI Python itself produces via Path.as_uri() always fails with 'Bundle manifest not found'. Route the file scheme through the existing bundler.services.adapters._file_url_to_path helper, which already handles drive letters, UNC hosts, and percent-decoding for catalog file:// URLs (make_catalog_fetcher). The bare-path branch is unchanged. Co-Authored-By: Claude Fable 5 * fix(bundle): reject file:// / local download_url; catalog URLs are HTTPS-only Per maintainer review (route B): file:// in a catalog download_url was never intended — catalog URLs are HTTPS-only (http for localhost) across the extensions/presets/workflows catalog systems, and disk installs go through the positional path (specify bundle install ), handled by _local_manifest_source before catalog resolution. Remove the file:///bare-path branch from _download_manifest and route everything through _download_remote_manifest (HTTPS-only via _require_https), with an actionable error pointing at the positional install. Invert the file:// tests to assert rejection (+ a positional-path resolution test), and migrate the three bundle-info contract tests off local download_urls onto an HTTPS-only entry with a mocked manifest fetch. Co-Authored-By: Claude Fable 5 * fix(bundle): validate HTTPS before the offline gate in _download_manifest Per review: for a non-local download_url the offline check ran before any URL validation, so an invalid/non-HTTPS scheme surfaced a misleading 'Network access disabled' error under --offline when the real problem is the URL would be rejected even online. Call _require_https before the offline gate so the correct error is reported in every mode. Co-Authored-By: Claude Fable 5 * fix(bundle): reword non-HTTPS download_url error to not mislabel scheme-less URLs A scheme-less download_url (urlparse scheme == "") can be a bare filesystem path OR a missing-scheme value like 'example.com/foo.zip', not necessarily file://. Reword the reject error to state the real HTTPS-only constraint and enumerate what is rejected (file://, local path, scheme-less), instead of labeling every case 'local/file://'. Behavior unchanged; the 'bundle install' actionable hint is preserved, so the existing reject-path tests still pass. Co-Authored-By: Claude Opus 4.8 (1M context) --------- Co-authored-by: Claude Fable 5 --- src/specify_cli/commands/bundle/__init__.py | 60 ++++++++++++------- tests/contract/test_bundle_cli.py | 36 ++++++++--- .../integration/test_bundler_local_install.py | 59 ++++++++++++++++++ 3 files changed, 125 insertions(+), 30 deletions(-) diff --git a/src/specify_cli/commands/bundle/__init__.py b/src/specify_cli/commands/bundle/__init__.py index b3c84c6ba7..224d63fc5f 100644 --- a/src/specify_cli/commands/bundle/__init__.py +++ b/src/specify_cli/commands/bundle/__init__.py @@ -746,11 +746,16 @@ def _resolve_manifest_path(path: Path | None) -> Path: def _download_manifest(resolved, *, offline: bool): """Resolve a bundle's manifest from its catalog ``download_url``. - Local/``file://`` URLs always work offline and may point at a ``.zip`` - artifact, a bundle directory, or a ``bundle.yml`` (handled by - :func:`_local_manifest_source`). Remote ``https://`` URLs are fetched with - the shared authenticated, redirect-validated HTTP client, and only when not - ``--offline``. + Catalog ``download_url``s are HTTPS-only (``http`` allowed for localhost), + matching the extensions/presets/workflows catalog systems. Remote URLs are + fetched with the shared authenticated, redirect-validated HTTP client, and + only when not ``--offline``. + + Local and ``file://`` sources are intentionally not resolved here: to + install a bundle from disk, pass the path positionally + (``specify bundle install ./path/to/bundle.yml`` — a bundle directory or a + ``.zip`` artifact also works), which :func:`_local_manifest_source` handles + before catalog resolution and which never touches ``download_url``. """ from urllib.parse import urlparse @@ -763,26 +768,35 @@ def _download_manifest(resolved, *, offline: bool): parsed = urlparse(url) scheme = parsed.scheme.lower() - # On Windows an absolute path like ``C:\bundle.yml`` parses with a - # single-letter ``scheme``; treat it as a local file, not a URL scheme. + # ``file://`` URLs and bare filesystem paths (including Windows drive paths + # like ``C:\bundle.yml``, which urlparse reads as a single-letter scheme) + # are not valid catalog download URLs. Catalog URLs are HTTPS-only across + # every catalog system; installing from disk is done by passing the path + # positionally, which never reaches URL resolution. Give an actionable + # error rather than accepting a scheme the rest of the codebase rejects. if scheme in ("", "file") or re.match(r"^[A-Za-z]:[\\/]", url): - local = Path(parsed.path if scheme == "file" else url) - manifest = _local_manifest_source(str(local)) - if manifest is None: - raise BundlerError(f"Bundle manifest not found: {local}") - return manifest - - if scheme in ("http", "https"): - if offline: - raise BundlerError( - f"Network access disabled; cannot download bundle '{resolved.entry.id}' " - f"from {url}." - ) - return _download_remote_manifest(resolved.entry.id, url) + raise BundlerError( + f"Catalog entry '{resolved.entry.id}' has a non-HTTP(S) download_url " + f"({url}); catalog download URLs must be HTTPS (http for localhost) — " + "a file:// URL, a local filesystem path, or a scheme-less value " + "(e.g. 'example.com/bundle.zip') is not accepted. " + "To install a bundle from disk, pass the path directly: " + "'specify bundle install '." + ) - raise BundlerError( - f"Unsupported download_url scheme for bundle '{resolved.entry.id}': {url}" - ) + # Validate the scheme/host *before* the offline gate so an invalid or + # non-HTTPS download_url reports the real problem in every mode, rather + # than a misleading "Network access disabled" under --offline. + # (_download_remote_manifest re-checks this, but only once network access + # is permitted.) HTTPS-only, http allowed for localhost. + _require_https(f"bundle '{resolved.entry.id}'", url) + + if offline: + raise BundlerError( + f"Network access disabled; cannot download bundle '{resolved.entry.id}' " + f"from {url}." + ) + return _download_remote_manifest(resolved.entry.id, url) def _require_https(label: str, url: str) -> None: diff --git a/tests/contract/test_bundle_cli.py b/tests/contract/test_bundle_cli.py index 1705c5945d..df9c8dae10 100644 --- a/tests/contract/test_bundle_cli.py +++ b/tests/contract/test_bundle_cli.py @@ -175,7 +175,23 @@ def test_build_produces_artifact(project: Path): assert len(artifacts) == 1 -def test_info_expands_full_component_set(project: Path): +def _mock_manifest_download(monkeypatch, source_path: Path) -> None: + """Mock the HTTPS manifest fetch to return a locally-authored manifest. + + Catalog ``download_url``s are HTTPS-only, so ``info`` tests can no longer + point one at a local file. Patch ``_download_manifest`` to return the + manifest parsed from *source_path* (a bundle.yml or a .zip artifact), + exercising ``info``'s expansion without a network call. + """ + from specify_cli.commands.bundle import _local_manifest_source + + monkeypatch.setattr( + "specify_cli.commands.bundle._download_manifest", + lambda resolved, *, offline: _local_manifest_source(str(source_path)), + ) + + +def test_info_expands_full_component_set(project: Path, monkeypatch): bundle_dir = project / "src-bundle" bundle_dir.mkdir() (bundle_dir / "bundle.yml").write_text( @@ -183,13 +199,14 @@ def test_info_expands_full_component_set(project: Path): ) catalog = project / "local-catalog.json" entry = catalog_entry_dict( - "demo-bundle", download_url=str(bundle_dir / "bundle.yml") + "demo-bundle", download_url="https://example.com/demo-bundle.zip" ) write_catalog_file(catalog, {"demo-bundle": entry}) added = runner.invoke( app, ["bundle", "catalog", "add", str(catalog), "--id", "local"] ) assert added.exit_code == 0, added.output + _mock_manifest_download(monkeypatch, bundle_dir / "bundle.yml") result = runner.invoke(app, ["bundle", "info", "demo-bundle", "--json", "--offline"]) assert result.exit_code == 0, result.output @@ -207,7 +224,7 @@ def test_info_expands_full_component_set(project: Path): assert "Trust" in text.output -def test_info_expands_discovery_only_bundle(project: Path): +def test_info_expands_discovery_only_bundle(project: Path, monkeypatch): # Discovery-only bundles must still be fully inspectable via `info`; # only `install` is refused for them. bundle_dir = project / "disc-bundle" @@ -217,7 +234,7 @@ def test_info_expands_discovery_only_bundle(project: Path): ) catalog = project / "disc-catalog.json" entry = catalog_entry_dict( - "demo-bundle", download_url=str(bundle_dir / "bundle.yml") + "demo-bundle", download_url="https://example.com/demo-bundle.zip" ) write_catalog_file(catalog, {"demo-bundle": entry}) config = { @@ -230,6 +247,7 @@ def test_info_expands_discovery_only_bundle(project: Path): (project / ".specify" / "bundle-catalogs.yml").write_text( yaml.safe_dump(config), encoding="utf-8" ) + _mock_manifest_download(monkeypatch, bundle_dir / "bundle.yml") result = runner.invoke(app, ["bundle", "info", "demo-bundle", "--json", "--offline"]) assert result.exit_code == 0, result.output payload = json.loads(result.output) @@ -237,8 +255,9 @@ def test_info_expands_discovery_only_bundle(project: Path): assert ("extensions", "ext-a") in components -def test_info_resolves_local_zip_download_url(project: Path): - # A local .zip artifact as download_url is extracted to read bundle.yml. +def test_info_expands_zip_sourced_bundle(project: Path, monkeypatch): + # A .zip artifact is extracted to read bundle.yml; info expands it. (The + # download itself is HTTPS-only now and mocked here — see contract note.) bundle_dir = project / "zip-src" bundle_dir.mkdir() (bundle_dir / "bundle.yml").write_text( @@ -249,12 +268,15 @@ def test_info_resolves_local_zip_download_url(project: Path): catalog = project / "zip-catalog.json" write_catalog_file( catalog, - {"demo-bundle": catalog_entry_dict("demo-bundle", download_url=str(artifact))}, + {"demo-bundle": catalog_entry_dict( + "demo-bundle", download_url="https://example.com/demo-bundle.zip" + )}, ) added = runner.invoke( app, ["bundle", "catalog", "add", str(catalog), "--id", "local"] ) assert added.exit_code == 0, added.output + _mock_manifest_download(monkeypatch, artifact) result = runner.invoke(app, ["bundle", "info", "demo-bundle", "--json", "--offline"]) assert result.exit_code == 0, result.output payload = json.loads(result.output) diff --git a/tests/integration/test_bundler_local_install.py b/tests/integration/test_bundler_local_install.py index a150ffce04..32972ac684 100644 --- a/tests/integration/test_bundler_local_install.py +++ b/tests/integration/test_bundler_local_install.py @@ -112,3 +112,62 @@ def test_install_bundled_extension_from_zip_offline(tmp_path: Path): assert not ExtensionManager(project).registry.is_installed("agent-context") finally: os.chdir(previous) + + +def test_download_manifest_rejects_file_url(tmp_path: Path): + """A catalog ``file://`` download_url is rejected — catalog URLs are + HTTPS-only, matching extensions/presets/workflows. Disk installs go through + the positional path (see the local-source tests above), not download_url. + """ + from types import SimpleNamespace + + from specify_cli.commands.bundle import _download_manifest + + manifest_path = write_manifest(tmp_path / "my bundles") + resolved = SimpleNamespace( + entry=SimpleNamespace(id="demo-bundle", download_url=manifest_path.as_uri()) + ) + + with pytest.raises(BundlerError, match="bundle install"): + _download_manifest(resolved, offline=True) + + +def test_download_manifest_rejects_bare_path(tmp_path: Path): + """A bare filesystem path download_url is likewise rejected.""" + from types import SimpleNamespace + + from specify_cli.commands.bundle import _download_manifest + + manifest_path = write_manifest(tmp_path / "plain") + resolved = SimpleNamespace( + entry=SimpleNamespace(id="demo-bundle", download_url=str(manifest_path)) + ) + + with pytest.raises(BundlerError, match="bundle install"): + _download_manifest(resolved, offline=True) + + +def test_local_install_still_resolves_via_positional_path(tmp_path: Path): + """The supported local route — a positional path, not a download_url — + still resolves the manifest via _local_manifest_source.""" + manifest_path = write_manifest(tmp_path / "my bundles") + manifest = _local_manifest_source(str(manifest_path)) + assert manifest is not None + assert manifest.bundle.id == "demo-bundle" + + +def test_download_manifest_rejects_non_https_url_even_offline(tmp_path: Path): + """A non-HTTPS download_url must report the HTTPS problem, not a misleading + 'Network access disabled', even under --offline (scheme is validated before + the offline gate).""" + from types import SimpleNamespace + + from specify_cli.commands.bundle import _download_manifest + + resolved = SimpleNamespace( + entry=SimpleNamespace( + id="demo-bundle", download_url="http://example.com/bundle.zip" + ) + ) + with pytest.raises(BundlerError, match="HTTPS"): + _download_manifest(resolved, offline=True) From c2af5c5a52230da62ff0644b802fe31b623c038f Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Mon, 13 Jul 2026 09:17:43 -0500 Subject: [PATCH 04/45] Add Verify Review Ship extension to community catalog (#3450) Add verify-review-ship extension submitted by @cadugevaerd to: - extensions/catalog.community.json (alphabetical order) - docs/community/extensions.md community extensions table Closes #3429 Assisted-by: GitHub Copilot (model: claude-sonnet-4.6, autonomous) Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> --- docs/community/extensions.md | 1 + extensions/catalog.community.json | 36 ++++++++++++++++++++++++++++++- 2 files changed, 36 insertions(+), 1 deletion(-) diff --git a/docs/community/extensions.md b/docs/community/extensions.md index 45fd2dc0f5..fd34673951 100644 --- a/docs/community/extensions.md +++ b/docs/community/extensions.md @@ -148,6 +148,7 @@ The following community-contributed extensions are available in [`catalog.commun | Token Economy | Token routing, measured savings, and context audit workflows | `process` | Read+Write | [spec-kit-token-economy](https://github.com/formin/spec-kit-token-economy) | | V-Model Extension Pack | Enforces V-Model paired generation of development specs and test specs with full traceability | `docs` | Read+Write | [spec-kit-v-model](https://github.com/leocamello/spec-kit-v-model) | | Verify Extension | Post-implementation quality gate that validates implemented code against specification artifacts | `code` | Read-only | [spec-kit-verify](https://github.com/ismaelJimenez/spec-kit-verify) | +| Verify Review Ship | Adds post-implementation verify, review, and ship readiness gates to Spec Kit workflows | `process` | Read-only | [spec-kit-verify-review-ship](https://github.com/cadugevaerd/spec-kit-verify-review-ship) | | Verify Tasks Extension | Detect phantom completions: tasks marked [X] in tasks.md with no real implementation | `code` | Read-only | [spec-kit-verify-tasks](https://github.com/datastone-inc/spec-kit-verify-tasks) | | Version Guard | Verify tech stack versions against live npm registries before planning and implementation | `process` | Read-only | [spec-kit-version-guard](https://github.com/KevinBrown5280/spec-kit-version-guard) | | What-if Analysis | Preview the downstream impact (complexity, effort, tasks, risks) of requirement changes before committing to them | `visibility` | Read-only | [spec-kit-whatif](https://github.com/DevAbdullah90/spec-kit-whatif) | diff --git a/extensions/catalog.community.json b/extensions/catalog.community.json index 6b074648fe..67e1ade49f 100644 --- a/extensions/catalog.community.json +++ b/extensions/catalog.community.json @@ -1,6 +1,6 @@ { "schema_version": "1.0", - "updated_at": "2026-07-08T00:00:00Z", + "updated_at": "2026-07-10T00:00:00Z", "catalog_url": "https://raw.githubusercontent.com/github/spec-kit/main/extensions/catalog.community.json", "extensions": { "aide": { @@ -4356,6 +4356,40 @@ "created_at": "2026-03-03T00:00:00Z", "updated_at": "2026-04-09T00:00:00Z" }, + "verify-review-ship": { + "name": "Verify Review Ship", + "id": "verify-review-ship", + "description": "Adds post-implementation verify, review, and ship readiness gates to Spec Kit workflows.", + "author": "Carlos Eduardo Gevaerd Araujo", + "version": "0.1.0", + "download_url": "https://github.com/cadugevaerd/spec-kit-verify-review-ship/archive/refs/tags/v0.1.0.zip", + "repository": "https://github.com/cadugevaerd/spec-kit-verify-review-ship", + "homepage": "https://github.com/cadugevaerd/spec-kit-verify-review-ship", + "documentation": "https://github.com/cadugevaerd/spec-kit-verify-review-ship/blob/main/README.md", + "changelog": "https://github.com/cadugevaerd/spec-kit-verify-review-ship/blob/main/CHANGELOG.md", + "license": "MIT", + "category": "process", + "effect": "read-only", + "requires": { + "speckit_version": ">=0.1.0" + }, + "provides": { + "commands": 3, + "hooks": 1 + }, + "tags": [ + "quality", + "review", + "shipping", + "workflow", + "testing" + ], + "verified": false, + "downloads": 0, + "stars": 0, + "created_at": "2026-07-10T00:00:00Z", + "updated_at": "2026-07-10T00:00:00Z" + }, "verify-tasks": { "name": "Verify Tasks Extension", "id": "verify-tasks", From 5c90a0547e649a1588240547b7397dea65c2d9a5 Mon Sep 17 00:00:00 2001 From: Ali jawwad <33836051+jawwad-ali@users.noreply.github.com> Date: Mon, 13 Jul 2026 19:32:53 +0500 Subject: [PATCH 05/45] fix(workflows): harden catalog.py against mis-shaped registry & non-string fields (#3375) * fix(workflows): harden catalog.py against mis-shaped registry & non-string fields Two robustness gaps where WorkflowRegistry/WorkflowCatalog diverged from their StepRegistry/StepCatalog siblings, which already guard both: - WorkflowRegistry._load returned json.load() verbatim, so a JSON-valid but mis-shaped registry (a list root, or a dict lacking a 'workflows' mapping) made is_installed/get/list/remove/add crash with TypeError/KeyError. Mirror StepRegistry._load: validate the shape and reset to default, and widen the except tuple to OSError/UnicodeError. - WorkflowCatalog.search joined name/description/id without coercion, so a null or non-string field raised TypeError. Coerce with str(... or '') exactly as StepCatalog.search does. Co-Authored-By: Claude Fable 5 * test(workflows): tighten mis-shaped-registry assertions Per review: WorkflowRegistry.list() always returns a dict, so assert '== {}' directly (the previous '== {} or == []' called list() twice and admitted a shape it never returns), and reference WorkflowRegistry.SCHEMA_VERSION instead of hard-coding '1.0'. Co-Authored-By: Claude Fable 5 --------- Co-authored-by: Claude Fable 5 --- src/specify_cli/workflows/catalog.py | 18 ++++++++--- tests/test_workflows.py | 47 ++++++++++++++++++++++++++++ 2 files changed, 60 insertions(+), 5 deletions(-) diff --git a/src/specify_cli/workflows/catalog.py b/src/specify_cli/workflows/catalog.py index 97bf58a04e..294dd8759a 100644 --- a/src/specify_cli/workflows/catalog.py +++ b/src/specify_cli/workflows/catalog.py @@ -76,8 +76,16 @@ def _load(self) -> dict[str, Any]: if self.registry_path.exists(): try: with open(self.registry_path, encoding="utf-8") as f: - return json.load(f) - except (json.JSONDecodeError, ValueError): + data = json.load(f) + # Validate shape: must be a dict with a dict "workflows" field, + # otherwise every method that indexes data["workflows"] crashes. + # Mirrors StepRegistry._load. + if not isinstance(data, dict): + return {"schema_version": self.SCHEMA_VERSION, "workflows": {}} + if not isinstance(data.get("workflows"), dict): + data["workflows"] = {} + return data + except (json.JSONDecodeError, ValueError, OSError, UnicodeError): # Corrupted registry file — reset to default return {"schema_version": self.SCHEMA_VERSION, "workflows": {}} return {"schema_version": self.SCHEMA_VERSION, "workflows": {}} @@ -438,9 +446,9 @@ def search( q = query.lower() searchable = " ".join( [ - wf_data.get("name", ""), - wf_data.get("description", ""), - wf_data.get("id", ""), + str(wf_data.get("name") or ""), + str(wf_data.get("description") or ""), + str(wf_data.get("id") or ""), ] ).lower() if q not in searchable: diff --git a/tests/test_workflows.py b/tests/test_workflows.py index d7cff20f6d..5984fe7c07 100644 --- a/tests/test_workflows.py +++ b/tests/test_workflows.py @@ -4746,12 +4746,59 @@ def test_persistence(self, project_dir): registry2 = WorkflowRegistry(project_dir) assert registry2.is_installed("test-wf") + @pytest.mark.parametrize("bad_content", ["[]", '{"schema_version": "1.0"}']) + def test_load_tolerates_misshaped_registry(self, project_dir, bad_content): + """A JSON-valid but mis-shaped registry file must not crash every method. + + A list root, or a dict lacking a 'workflows' mapping, previously made + is_installed/get/list/remove/add raise TypeError/KeyError. Mirrors the + shape guard StepRegistry._load already has. + """ + from specify_cli.workflows.catalog import WorkflowRegistry + + reg_path = project_dir / ".specify" / "workflows" / "workflow-registry.json" + reg_path.parent.mkdir(parents=True, exist_ok=True) + reg_path.write_text(bad_content, encoding="utf-8") + + registry = WorkflowRegistry(project_dir) + assert registry.data == { + "schema_version": WorkflowRegistry.SCHEMA_VERSION, + "workflows": {}, + } + # None of these should raise on the recovered-default shape. + assert registry.is_installed("x") is False + assert registry.get("x") is None + assert registry.list() == {} # list() always returns a dict + registry.remove("x") + registry.add("x", {"name": "X"}) + assert registry.is_installed("x") + # ===== Workflow Catalog Tests ===== class TestWorkflowCatalog: """Test WorkflowCatalog catalog resolution.""" + def test_search_with_non_string_fields(self, project_dir, monkeypatch): + """Non-string workflow fields (null/int name/description) must not + raise TypeError in search — StepCatalog.search already coerces these.""" + from specify_cli.workflows.catalog import WorkflowCatalog + + catalog = WorkflowCatalog(project_dir) + monkeypatch.setattr(catalog, "_get_merged_workflows", lambda **kw: { + "42": { + "id": 42, + "name": None, + "description": 99, + "_catalog_name": "test", + "_install_allowed": True, + }, + }) + + assert len(catalog.search()) == 1 + assert len(catalog.search(query="42")) == 1 + assert len(catalog.search(query="missing")) == 0 + def test_default_catalogs(self, project_dir, monkeypatch): from specify_cli.workflows.catalog import WorkflowCatalog From a3bcd679250da88ec3f80a13049973d2ae64c170 Mon Sep 17 00:00:00 2001 From: Ali jawwad <33836051+jawwad-ali@users.noreply.github.com> Date: Mon, 13 Jul 2026 19:33:39 +0500 Subject: [PATCH 06/45] docs(bundles): document --integration on 'bundle update' (#3271) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The 'bundle update' command accepts --integration (verified via 'specify bundle update --help' and the command signature), used as the integration override when the project's active integration can't be detected. The Update Bundles options table in reference/bundles.md omitted it, listing only --all and --offline — unlike the install/init tables which already document --integration. Add the missing row. Co-authored-by: Claude Opus 4.8 --- docs/reference/bundles.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/docs/reference/bundles.md b/docs/reference/bundles.md index 57f3c700b1..2bd33c960b 100644 --- a/docs/reference/bundles.md +++ b/docs/reference/bundles.md @@ -51,10 +51,11 @@ If the current directory is not yet a Spec Kit project, `install` initializes on specify bundle update [] ``` -| Option | Description | -| ------------ | ------------------------------------ | -| `--all` | Update every installed bundle | -| `--offline` | Do not access the network | +| Option | Description | +| ---------------- | --------------------------------------------------------------------------------------------------------------------- | +| `--all` | Update every installed bundle | +| `--integration` | Override the integration used when refreshing components; applied only when the project's active integration can't be determined | +| `--offline` | Do not access the network | Re-resolves a bundle and **refreshes** its components through each primitive's update path, bringing already-installed components up to the bundle's newly pinned versions while preserving primitive-level overrides (such as preset priority). Provide a bundle id, or use `--all` to update everything installed. From 9d96c629015188a4d720c04d16563be8bc1b3367 Mon Sep 17 00:00:00 2001 From: Ali jawwad <33836051+jawwad-ali@users.noreply.github.com> Date: Mon, 13 Jul 2026 19:34:29 +0500 Subject: [PATCH 07/45] fix(workflows): engine loop cap ignores bool max_iterations (#3270) The while/do-while loop cap guard 'not isinstance(max_iters, int) or max_iters < 1' does not fall back to the default for a boolean max_iterations: isinstance(True, int) is True and True < 1 is False. The loop then runs range(max_iters - 1) == range(True - 1) == range(0), capping at a single iteration instead of the default 10. Exclude bools, mirroring the merged while/do-while validators (#3237) and this function's own continue_on_error bool handling. execute() does not auto-validate, so this engine guard is the only defence. Co-authored-by: Claude Opus 4.8 --- src/specify_cli/workflows/engine.py | 11 ++++++- tests/test_workflows.py | 50 +++++++++++++++++++++++++++++ 2 files changed, 60 insertions(+), 1 deletion(-) diff --git a/src/specify_cli/workflows/engine.py b/src/specify_cli/workflows/engine.py index 6025c30c5c..fd207b9ec5 100644 --- a/src/specify_cli/workflows/engine.py +++ b/src/specify_cli/workflows/engine.py @@ -982,7 +982,16 @@ def _execute_steps( from .expressions import evaluate_condition max_iters = step_config.get("max_iterations") - if not isinstance(max_iters, int) or max_iters < 1: + # A bool is an int in Python (isinstance(True, int) is True + # and True == 1), so a bool max_iterations would slip past + # the int check and cap the loop at range(0)==1 iteration + # instead of the default. Exclude bools, mirroring the + # while/do-while validators and the continue_on_error guard. + if ( + isinstance(max_iters, bool) + or not isinstance(max_iters, int) + or max_iters < 1 + ): max_iters = 10 condition = step_config.get("condition", False) for _loop_iter in range(max_iters - 1): diff --git a/tests/test_workflows.py b/tests/test_workflows.py index 5984fe7c07..f2f41bce78 100644 --- a/tests/test_workflows.py +++ b/tests/test_workflows.py @@ -3872,6 +3872,56 @@ def test_while_loop_runs_to_max_when_condition_stays_true(self, project_dir): assert "retry-loop:tick:1" in state.step_results assert "retry-loop:tick:2" in state.step_results + def test_loop_with_bool_max_iterations_uses_default_cap(self, project_dir): + """A boolean max_iterations must fall back to the default cap of 10, + not be treated as the int 1 (bool-is-int trap). + + ``max_iterations: true`` would otherwise slip past the int check + (``isinstance(True, int)`` is True and ``True < 1`` is False) and + cap the loop at ``range(True - 1) == range(0)`` — a single + iteration. ``execute()`` does not auto-validate, so the engine's own + guard is the only line of defence here. + """ + from specify_cli.workflows.engine import WorkflowEngine, WorkflowDefinition + from specify_cli.workflows.base import RunStatus + + import sys + + counter_file = project_dir / ".counter" + counter_file.write_text("0", encoding="utf-8") + py = sys.executable + script_file = project_dir / "_tick.py" + script_file.write_text( + f"import pathlib; p = pathlib.Path(r'{counter_file}')\n" + "n = int(p.read_text()) + 1; p.write_text(str(n))\n" + "print('pending', end='')\n", + encoding="utf-8", + ) + + yaml_str = f""" +schema_version: "1.0" +workflow: + id: "while-bool-max-iterations" + name: "While Bool Max Iterations" + version: "1.0.0" +steps: + - id: retry-loop + type: while + condition: "{{{{ 'done' not in steps.tick.output.stdout }}}}" + max_iterations: true + steps: + - id: tick + type: shell + run: '"{py}" "{script_file}"' +""" + definition = WorkflowDefinition.from_string(yaml_str) + engine = WorkflowEngine(project_dir) + state = engine.execute(definition) + + assert state.status == RunStatus.COMPLETED + # Falls back to the default cap of 10, not range(True - 1) == 1 run. + assert counter_file.read_text(encoding="utf-8").strip() == "10" + def test_do_while_loop_runs_to_max_when_condition_stays_true(self, project_dir): """Do-while loop must still run to max_iterations when the condition never becomes false. From f6f3540409badcbe570a9d5cccca35beee932845 Mon Sep 17 00:00:00 2001 From: Ali jawwad <33836051+jawwad-ali@users.noreply.github.com> Date: Mon, 13 Jul 2026 19:35:51 +0500 Subject: [PATCH 08/45] fix(presets): set-priority repairs corrupted boolean priority (#3269) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Same bool-is-int trap as the extension set-priority command: the skip guard 'isinstance(raw_priority, int) and raw_priority == priority' treats a stored boolean as a match (isinstance(True, int) is True, True == 1), so a corrupted boolean priority reports 'already has priority N' and is never rewritten to a real int — contradicting the adjacent comment. Exclude bools explicitly, mirroring normalize_priority's bool guard. Co-authored-by: Claude Opus 4.8 --- src/specify_cli/presets/_commands.py | 9 +++++++- tests/test_presets.py | 34 ++++++++++++++++++++++++++++ 2 files changed, 42 insertions(+), 1 deletion(-) diff --git a/src/specify_cli/presets/_commands.py b/src/specify_cli/presets/_commands.py index 4e2cb1bbf5..0c5632358e 100644 --- a/src/specify_cli/presets/_commands.py +++ b/src/specify_cli/presets/_commands.py @@ -469,7 +469,14 @@ def preset_set_priority( raw_priority = metadata.get("priority") # Only skip if the stored value is already a valid int equal to requested priority # This ensures corrupted values (e.g., "high") get repaired even when setting to default (10) - if isinstance(raw_priority, int) and raw_priority == priority: + # A bool is an int in Python (isinstance(True, int) is True), so exclude it explicitly — + # mirroring normalize_priority's bool guard — otherwise a corrupted True/False priority + # equals 1/0 here and is never repaired. + if ( + isinstance(raw_priority, int) + and not isinstance(raw_priority, bool) + and raw_priority == priority + ): console.print(f"[yellow]Preset '{preset_id}' already has priority {priority}[/yellow]") raise typer.Exit(0) diff --git a/tests/test_presets.py b/tests/test_presets.py index 54cb9dd5b3..a4c78d9268 100644 --- a/tests/test_presets.py +++ b/tests/test_presets.py @@ -4060,6 +4060,40 @@ def test_set_priority_same_value_no_change(self, project_dir, pack_dir): plain = strip_ansi(result.output) assert "already has priority 5" in plain + def test_set_priority_repairs_corrupted_bool(self, project_dir, pack_dir): + """A corrupted boolean priority must be repaired, not skipped. + + ``isinstance(True, int)`` is True and ``True == 1`` in Python, so a + stored ``True`` priority would short-circuit the ``already has + priority 1`` skip path and never get rewritten to a real int — + contradicting the comment that promises corrupted values are + repaired. The guard must exclude bools (like normalize_priority). + """ + from typer.testing import CliRunner + from unittest.mock import patch + from specify_cli import app + + runner = CliRunner() + + manager = PresetManager(project_dir) + manager.install_from_directory(pack_dir, "0.1.5", priority=5) + # Inject a corrupted boolean priority (True == 1). + manager.registry.update("test-pack", {"priority": True}) + + with patch.object(Path, "cwd", return_value=project_dir): + result = runner.invoke(app, ["preset", "set-priority", "test-pack", "1"]) + + assert result.exit_code == 0, result.output + plain = strip_ansi(result.output) + # The corrupted bool must be repaired, not reported as already-set. + assert "already has priority" not in plain + assert "priority changed" in plain + + # The stored value is now a real int, not a bool. + reloaded = PresetManager(project_dir).registry.get("test-pack") + assert reloaded["priority"] == 1 + assert not isinstance(reloaded["priority"], bool) + def test_set_priority_invalid_value(self, project_dir, pack_dir): """Test set-priority rejects invalid priority values.""" from typer.testing import CliRunner From 903d707d21e95ad93a2481726d1233559bdc9532 Mon Sep 17 00:00:00 2001 From: Ali jawwad <33836051+jawwad-ali@users.noreply.github.com> Date: Mon, 13 Jul 2026 19:41:47 +0500 Subject: [PATCH 09/45] fix(extensions): set-priority repairs corrupted boolean priority (#3268) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The set-priority skip guard 'isinstance(raw_priority, int) and raw_priority == priority' treats a stored boolean as a match because isinstance(True, int) is True and True == 1 (False == 0). So a corrupted boolean priority short-circuits to 'already has priority N' and is never rewritten to a real int — contradicting the adjacent comment that promises corrupted values get repaired. Exclude bools explicitly, mirroring normalize_priority's own bool guard. Co-authored-by: Claude Opus 4.8 --- src/specify_cli/extensions/_commands.py | 9 ++++++- tests/test_extensions.py | 36 +++++++++++++++++++++++++ 2 files changed, 44 insertions(+), 1 deletion(-) diff --git a/src/specify_cli/extensions/_commands.py b/src/specify_cli/extensions/_commands.py index 7f874407ac..4494e15114 100644 --- a/src/specify_cli/extensions/_commands.py +++ b/src/specify_cli/extensions/_commands.py @@ -1566,7 +1566,14 @@ def extension_set_priority( raw_priority = metadata.get("priority") # Only skip if the stored value is already a valid int equal to requested priority # This ensures corrupted values (e.g., "high") get repaired even when setting to default (10) - if isinstance(raw_priority, int) and raw_priority == priority: + # A bool is an int in Python (isinstance(True, int) is True), so exclude it explicitly — + # mirroring normalize_priority's bool guard — otherwise a corrupted True/False priority + # equals 1/0 here and is never repaired. + if ( + isinstance(raw_priority, int) + and not isinstance(raw_priority, bool) + and raw_priority == priority + ): console.print(f"[yellow]Extension '{_escape_markup(str(display_name))}' already has priority {priority}[/yellow]") raise typer.Exit(0) diff --git a/tests/test_extensions.py b/tests/test_extensions.py index cb3120f492..0003699693 100644 --- a/tests/test_extensions.py +++ b/tests/test_extensions.py @@ -6424,6 +6424,42 @@ def test_set_priority_same_value_no_change(self, extension_dir, project_dir): plain = strip_ansi(result.output) assert "already has priority 5" in plain + def test_set_priority_repairs_corrupted_bool(self, extension_dir, project_dir): + """A corrupted boolean priority must be repaired, not skipped. + + ``isinstance(True, int)`` is True and ``True == 1`` in Python, so a + stored ``True`` priority would short-circuit the ``already has + priority 1`` skip path and never get rewritten to a real int — + contradicting the comment that promises corrupted values are + repaired. The guard must exclude bools (like normalize_priority). + """ + from typer.testing import CliRunner + from unittest.mock import patch + from specify_cli import app + + runner = CliRunner() + + manager = ExtensionManager(project_dir) + manager.install_from_directory( + extension_dir, "0.1.0", register_commands=False, priority=5 + ) + # Inject a corrupted boolean priority (True == 1). + manager.registry.update("test-ext", {"priority": True}) + + with patch.object(Path, "cwd", return_value=project_dir): + result = runner.invoke(app, ["extension", "set-priority", "test-ext", "1"]) + + assert result.exit_code == 0, result.output + plain = strip_ansi(result.output) + # The corrupted bool must be repaired, not reported as already-set. + assert "already has priority" not in plain + assert "priority changed" in plain + + # The stored value is now a real int, not a bool. + reloaded = ExtensionManager(project_dir).registry.get("test-ext") + assert reloaded["priority"] == 1 + assert not isinstance(reloaded["priority"], bool) + def test_set_priority_invalid_value(self, extension_dir, project_dir): """Test set-priority rejects invalid priority values.""" from typer.testing import CliRunner From 7ff4522cf3aa4cd5277cef56203088c82e4ac613 Mon Sep 17 00:00:00 2001 From: Manfred Riem <15701806+mnriem@users.noreply.github.com> Date: Mon, 13 Jul 2026 10:03:36 -0500 Subject: [PATCH 10/45] chore: release 0.12.12, begin 0.12.13.dev0 development (#3490) * chore: bump version to 0.12.12 * chore: begin 0.12.13.dev0 development --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> --- CHANGELOG.md | 15 +++++++++++++++ pyproject.toml | 2 +- 2 files changed, 16 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 5282e1f955..0485ee07ef 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,21 @@ +## [0.12.12] - 2026-07-13 + +### Changed + +- fix(extensions): set-priority repairs corrupted boolean priority (#3268) +- fix(presets): set-priority repairs corrupted boolean priority (#3269) +- fix(workflows): engine loop cap ignores bool max_iterations (#3270) +- docs(bundles): document --integration on 'bundle update' (#3271) +- fix(workflows): harden catalog.py against mis-shaped registry & non-string fields (#3375) +- Add Verify Review Ship extension to community catalog (#3450) +- fix(bundle): reject file:// / local download_url — catalog URLs are HTTPS-only (#3344) +- fix(extensions): handle prefix-colliding env vars in _get_env_config (#3350) +- docs: document copilot skills mode (--skills) and markdown deprecation (#3313) +- chore: release 0.12.11, begin 0.12.12.dev0 development (#3460) + ## [0.12.11] - 2026-07-10 ### Changed diff --git a/pyproject.toml b/pyproject.toml index ef803bb324..bade423da9 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,6 +1,6 @@ [project] name = "specify-cli" -version = "0.12.12.dev0" +version = "0.12.13.dev0" description = "Specify CLI, part of GitHub Spec Kit. A tool to bootstrap your projects for Spec-Driven Development (SDD)." readme = "README.md" requires-python = ">=3.11" From 55c66125f0d5813855246538c1d0010cc276bb64 Mon Sep 17 00:00:00 2001 From: Ali jawwad <33836051+jawwad-ali@users.noreply.github.com> Date: Mon, 13 Jul 2026 20:14:55 +0500 Subject: [PATCH 11/45] fix(workflows): if-step validate accepts falsy non-list else (#3264) * fix(workflows): if-step validate accepts falsy non-list else IfThenStep.validate() guarded the 'else' branch with 'if else_branch and not isinstance(else_branch, list)'. The leading truthiness check short-circuits for falsy non-list values (False, 0, '', {}), so a malformed else-branch passes validation and is then silently skipped at runtime. The sibling 'then' branch is validated strictly; 'else' now matches by switching to an 'is not None' guard. Co-Authored-By: Claude Opus 4.8 * test(workflows): cover explicit else:None and missing-else separately Per Copilot feedback: the parametrized valid-else test omitted the 'else' key when the value was None, so it covered only the missing-else case, not an explicit 'else: None'. Set 'else' explicitly (including None) in the parametrized test and add a dedicated missing-else test, so both accepted shapes are pinned. Co-Authored-By: Claude Opus 4.8 --------- Co-authored-by: Claude Opus 4.8 --- .../workflows/steps/if_then/__init__.py | 4 +- tests/test_workflows.py | 40 +++++++++++++++++++ 2 files changed, 42 insertions(+), 2 deletions(-) diff --git a/src/specify_cli/workflows/steps/if_then/__init__.py b/src/specify_cli/workflows/steps/if_then/__init__.py index 5b921a31a5..e7179a418a 100644 --- a/src/specify_cli/workflows/steps/if_then/__init__.py +++ b/src/specify_cli/workflows/steps/if_then/__init__.py @@ -47,8 +47,8 @@ def validate(self, config: dict[str, Any]) -> list[str]: errors.append( f"If step {config.get('id', '?')!r}: 'then' must be a list of steps." ) - else_branch = config.get("else", []) - if else_branch and not isinstance(else_branch, list): + else_branch = config.get("else") + if else_branch is not None and not isinstance(else_branch, list): errors.append( f"If step {config.get('id', '?')!r}: 'else' must be a list of steps." ) diff --git a/tests/test_workflows.py b/tests/test_workflows.py index f2f41bce78..2c94966f96 100644 --- a/tests/test_workflows.py +++ b/tests/test_workflows.py @@ -1988,6 +1988,46 @@ def test_validate_missing_condition(self): errors = step.validate({"id": "test", "then": []}) assert any("missing 'condition'" in e for e in errors) + @pytest.mark.parametrize("bad_else", [False, 0, "", {}, 42]) + def test_validate_rejects_non_list_else(self, bad_else): + """A non-list 'else' must be rejected even when it is falsy. + + The original guard used ``if else_branch and ...`` which + short-circuits for falsy non-list values (False/0/''/{}), letting a + malformed else-branch pass validation only to be silently skipped at + runtime. ``then`` is already strictly validated; ``else`` must match. + """ + from specify_cli.workflows.steps.if_then import IfThenStep + + step = IfThenStep() + errors = step.validate( + {"id": "i", "condition": "true", "then": [], "else": bad_else} + ) + assert any("'else' must be a list of steps" in e for e in errors) + + @pytest.mark.parametrize("ok_else", [None, [], [{"id": "x", "command": "/y"}]]) + def test_validate_accepts_valid_else(self, ok_else): + """An explicit 'else' of None or a list stays valid. + + ``else`` is set explicitly here (including ``else: None``) so the + explicit-None case is exercised, not just the missing-key case. + """ + from specify_cli.workflows.steps.if_then import IfThenStep + + step = IfThenStep() + errors = step.validate( + {"id": "i", "condition": "true", "then": [], "else": ok_else} + ) + assert not any("'else'" in e for e in errors) + + def test_validate_accepts_missing_else(self): + """A missing 'else' key stays valid (no else branch).""" + from specify_cli.workflows.steps.if_then import IfThenStep + + step = IfThenStep() + errors = step.validate({"id": "i", "condition": "true", "then": []}) + assert not any("'else'" in e for e in errors) + class TestSwitchStep: """Test the switch step type.""" From 86d769b47c8f394a34a25fc48babd65264030148 Mon Sep 17 00:00:00 2001 From: Quratulain-bilal Date: Mon, 13 Jul 2026 20:20:48 +0500 Subject: [PATCH 12/45] fix(workflows): don't crash on membership test against a non-iterable (#3448) * fix(workflows): don't crash on membership test against a non-iterable the `in` / `not in` operators in _evaluate_simple_expression only guarded `right is not None`, so `left in right` still raised a raw TypeError when the right operand was any other non-iterable (int, bool, float). a condition like `{{ inputs.tag in inputs.count }}` where count is a number crashed the whole workflow run instead of evaluating. nothing is contained in a non-iterable, so treat membership as False (`not in` as True) via a new _safe_membership helper that swallows TypeError. this generalizes the old None guard and mirrors _safe_compare, which already catches TypeError for the ordering operators. added a regression test; confirmed it fails on the pre-fix code (raw TypeError) and that genuine list/substring membership still works. * address review: float membership case + broaden _safe_membership docstring - add a float right-operand assertion so the test matches its comment (was claiming float coverage while only exercising int/bool/None). - reword the _safe_membership docstring to describe TypeError generally (non-iterable right is the common case, but also e.g. an unhashable left against a set) rather than implying only the right operand matters. --- src/specify_cli/workflows/expressions.py | 24 ++++++++++++++++++-- tests/test_workflows.py | 28 ++++++++++++++++++++++++ 2 files changed, 50 insertions(+), 2 deletions(-) diff --git a/src/specify_cli/workflows/expressions.py b/src/specify_cli/workflows/expressions.py index d6736bc321..9953f5ff61 100644 --- a/src/specify_cli/workflows/expressions.py +++ b/src/specify_cli/workflows/expressions.py @@ -464,9 +464,9 @@ def _evaluate_simple_expression(expr: str, namespace: dict[str, Any]) -> Any: if op == "<=": return _safe_compare(left, right, "<=") if op == " in ": - return left in right if right is not None else False + return _safe_membership(left, right, negate=False) if op == " not in ": - return left not in right if right is not None else True + return _safe_membership(left, right, negate=True) # Numeric literal try: @@ -511,6 +511,26 @@ def _coerce_number(value: Any) -> Any: return value +def _safe_membership(left: Any, right: Any, *, negate: bool) -> bool: + """Safely evaluate ``left in right`` (or ``not in``) without crashing. + + ``left in right`` raises ``TypeError`` whenever the operands don't support + membership testing — most commonly a non-iterable right operand (``None``, + an int, a bool), but also cases like an unhashable ``left`` against a set. + In every such case the membership relation is undefined, so treat it as + ``False`` (``not in`` as ``True``) rather than leaking the error out of the + evaluator and crashing the whole workflow. Mirrors the graceful + ``TypeError`` handling in ``_safe_compare`` for the ordering operators, and + generalizes the previous ``right is not None`` guard to any operand pair + that can't be membership-tested. + """ + try: + contained = left in right + except TypeError: + contained = False + return not contained if negate else contained + + def _safe_compare(left: Any, right: Any, op: str) -> bool: """Compare two values for ordering, coercing numeric strings when possible. diff --git a/tests/test_workflows.py b/tests/test_workflows.py index 2c94966f96..0026124f43 100644 --- a/tests/test_workflows.py +++ b/tests/test_workflows.py @@ -460,6 +460,34 @@ def test_pipe_detection_is_quote_aware(self): assert evaluate_expression("{{ inputs.s | contains('ab') }}", ctx2) is True assert evaluate_expression("{{ inputs.missing | default('a|b') }}", ctx2) == "a|b" + def test_membership_against_non_iterable_is_false_not_error(self): + from specify_cli.workflows.expressions import ( + evaluate_condition, + evaluate_expression, + ) + from specify_cli.workflows.base import StepContext + + # A non-iterable right operand (int, bool, None, float) makes a raw + # `x in y` raise TypeError in Python. The evaluator must treat it as + # "not contained" (False, and `not in` as True) instead of leaking the + # TypeError and crashing the whole workflow run. This generalizes the + # previous `right is not None` guard and mirrors _safe_compare, which + # already swallows TypeError for the ordering operators. + ctx = StepContext(inputs={"tag": "x", "count": 5, "ratio": 1.5, "flag": True}) + assert evaluate_expression("{{ inputs.tag in inputs.count }}", ctx) is False + assert evaluate_expression("{{ inputs.tag not in inputs.count }}", ctx) is True + assert evaluate_expression("{{ 'a' in inputs.ratio }}", ctx) is False + assert evaluate_expression("{{ 'a' in inputs.flag }}", ctx) is False + assert evaluate_expression("{{ inputs.tag in inputs.missing }}", ctx) is False + # A condition that would otherwise crash the run now evaluates cleanly. + assert evaluate_condition("{{ inputs.tag in inputs.count }}", ctx) is False + + # Regression: genuine membership over a real iterable still works. + ok = StepContext(inputs={"items": ["x", "y"], "s": "xyz"}) + assert evaluate_expression("{{ 'x' in inputs.items }}", ok) is True + assert evaluate_expression("{{ 'z' not in inputs.items }}", ok) is True + assert evaluate_expression("{{ 'y' in inputs.s }}", ok) is True + def test_filter_default(self): from specify_cli.workflows.expressions import evaluate_expression from specify_cli.workflows.base import StepContext From 82c078bb3ab7e6f66a334b27f0f4f596b31880a8 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Emre=20De=C4=9Firmenci?= Date: Mon, 13 Jul 2026 18:27:18 +0300 Subject: [PATCH 13/45] docs: clarify that release tags keep the leading v prefix (#3463) Readers were replacing vX.Y.Z with bare versions like 0.12.11, which fails because git tags are named v0.12.11. Assisted-by: Cursor Grok 4.5 (supervised) Co-authored-by: Cursor --- README.md | 2 +- docs/install/one-time.md | 3 ++- docs/install/pipx.md | 3 ++- docs/installation.md | 2 +- 4 files changed, 6 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index c759f6e05e..1386905297 100644 --- a/README.md +++ b/README.md @@ -45,7 +45,7 @@ Spec-Driven Development **flips the script** on traditional software development ### 1. Install Specify CLI -Requires **[uv](https://docs.astral.sh/uv/)** ([install uv](./docs/install/uv.md)). Replace `vX.Y.Z` with the latest tag from [Releases](https://github.com/github/spec-kit/releases): +Requires **[uv](https://docs.astral.sh/uv/)** ([install uv](./docs/install/uv.md)). Replace `vX.Y.Z` with the latest release tag from [Releases](https://github.com/github/spec-kit/releases) — keep the leading `v` (for example, `v0.12.11`, not `0.12.11`): ```bash uv tool install specify-cli --from git+https://github.com/github/spec-kit.git@vX.Y.Z diff --git a/docs/install/one-time.md b/docs/install/one-time.md index 134cb0b11d..ed8bdef4ee 100644 --- a/docs/install/one-time.md +++ b/docs/install/one-time.md @@ -11,7 +11,8 @@ If you want to try Spec Kit without installing it permanently, use `uvx` to run # Create a new project (latest from main) uvx --from git+https://github.com/github/spec-kit.git specify init -# Or target a specific release (replace vX.Y.Z with a tag from Releases) +# Or target a specific release (replace vX.Y.Z with a tag from Releases; +# keep the leading v, e.g. v0.12.11 not 0.12.11) uvx --from git+https://github.com/github/spec-kit.git@vX.Y.Z specify init # Initialize in the current directory diff --git a/docs/install/pipx.md b/docs/install/pipx.md index 3a25b16489..8e64ee083a 100644 --- a/docs/install/pipx.md +++ b/docs/install/pipx.md @@ -7,7 +7,8 @@ Pin a specific release tag for stability (check [Releases](https://github.com/github/spec-kit/releases) for the latest): ```bash -# Install a specific stable release (recommended — replace vX.Y.Z with the latest tag) +# Install a specific stable release (recommended — replace vX.Y.Z with the +# latest tag, keeping the leading v, e.g. v0.12.11 not 0.12.11) pipx install git+https://github.com/github/spec-kit.git@vX.Y.Z # Or install latest from main (may include unreleased changes) diff --git a/docs/installation.md b/docs/installation.md index 744423b29c..509c353829 100644 --- a/docs/installation.md +++ b/docs/installation.md @@ -15,7 +15,7 @@ ### Persistent Installation (Recommended) -Install once and use everywhere. Replace `vX.Y.Z` with a tag from [Releases](https://github.com/github/spec-kit/releases): +Install once and use everywhere. Replace `vX.Y.Z` with a release tag from [Releases](https://github.com/github/spec-kit/releases) — keep the leading `v` (for example, `v0.12.11`, not `0.12.11`): > [!NOTE] > The command below requires **[uv](https://docs.astral.sh/uv/)**. If you see `command not found: uv`, [install uv first](./install/uv.md). From 32952c94f4634095fc288469ddc04027c21e9e98 Mon Sep 17 00:00:00 2001 From: Noor ul ain Date: Mon, 13 Jul 2026 20:32:58 +0500 Subject: [PATCH 14/45] feat(workflows): make shell step timeout configurable (#3327) (#3328) * feat(workflows): make shell step timeout configurable (#3327) The `shell` step hardcoded a 300s subprocess timeout, so any command that legitimately runs longer than five minutes (a full build, a linter aggregator, an integration-test target) was killed with TimeoutExpired and failed the whole run, with no YAML knob to raise the limit. Add an optional `timeout` field (seconds) that defaults to 300 for backward compatibility and is threaded through to `subprocess.run`. The timeout failure message now reports the configured value instead of a hardcoded 300. `validate` rejects a `timeout` that is not a positive number (bool is rejected explicitly, since it is an int subclass but a config error rather than a duration). Co-Authored-By: Claude Opus 4.8 (1M context) * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * test(workflows): cover non-finite timeout rejection in shell step The isfinite guard added in 955d46a rejects YAML .inf/.nan timeouts, but no test asserted it. inf and nan are floats that pass a plain > 0 check (nan <= 0 is False), so without an explicit case a regression could silently reaccept them and crash subprocess.run(timeout=...) at runtime. Addresses the remaining Copilot review comment on PR #3328. Co-Authored-By: Claude Opus 4.8 (1M context) * docs(workflows): document configurable shell step timeout Address Copilot review feedback on #3328: the per-step `timeout` option was not reflected in the public workflow docs. The Shell Steps section only showed `run:`, so readers couldn't discover `timeout:`, its unit (seconds), or its default (300). Co-Authored-By: Claude Opus 4.8 (1M context) * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * refactor(workflows): consolidate shell-step timeout validation into one path Address Copilot review feedback on #3328: - Remove the dead "fall back to default" timeout block in execute(): it re-read `timeout` from config immediately after, so the fallback was discarded and its comment contradicted the new fail-on-invalid behavior. - Extract a single `_timeout_error()` helper shared by execute() and validate() so both reject the same values with the same message, instead of two drifting copies of the check. - Hoist the duplicated inline `import math` to module scope. - Add test_execute_fails_cleanly_on_invalid_timeout: asserts execute() fails the step (rather than raising) on an unvalidated string/bool/inf/0 timeout, covering the engine-skips-validate path Copilot flagged. Co-Authored-By: Claude Opus 4.8 (1M context) --------- Co-authored-by: Claude Opus 4.8 (1M context) Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> --- .../workflows/steps/shell/__init__.py | 63 +++-- tests/test_workflows.py | 225 ++++++++++-------- workflows/README.md | 6 + 3 files changed, 173 insertions(+), 121 deletions(-) diff --git a/src/specify_cli/workflows/steps/shell/__init__.py b/src/specify_cli/workflows/steps/shell/__init__.py index ed5df8fc36..9faac62e4d 100644 --- a/src/specify_cli/workflows/steps/shell/__init__.py +++ b/src/specify_cli/workflows/steps/shell/__init__.py @@ -3,6 +3,7 @@ from __future__ import annotations import json +import math import subprocess from typing import Any @@ -25,15 +26,20 @@ def execute(self, config: dict[str, Any], context: StepContext) -> StepResult: run_cmd = str(run_cmd) cwd = context.project_root or "." - # Defensive: the engine does not auto-validate step config, so an - # invalid ``timeout`` (string, None, ...) would otherwise raise a - # TypeError from subprocess.run() and crash the whole run. Mirror - # the engine's handling of unvalidated ``continue_on_error`` by - # only honoring well-formed values and falling back to the default. + # Per-step execution timeout in seconds; defaults to 300 for backward + # compatibility. The engine does not auto-validate step config, so + # validate here as well — a caller that skips WorkflowEngine.validate() + # must fail the step cleanly rather than crash subprocess.run() with a + # TypeError (or silently coerce ``timeout: true`` to a 1s duration, + # since bool is an int subclass). timeout = config.get("timeout", 300) - if isinstance(timeout, bool) or not isinstance(timeout, int) or timeout <= 0: - timeout = 300 - + timeout_error = self._timeout_error(config) + if timeout_error is not None: + return StepResult( + status=StepStatus.FAILED, + error=timeout_error, + output={"exit_code": -1, "stdout": "", "stderr": "invalid timeout"}, + ) # NOTE: shell=True is required to support pipes, redirects, and # multi-command expressions in workflow YAML. Workflow authors # control commands; catalog-installed workflows should be reviewed @@ -92,6 +98,32 @@ def execute(self, config: dict[str, Any], context: StepContext) -> StepResult: output={"exit_code": -1, "stdout": "", "stderr": str(exc)}, ) + @staticmethod + def _timeout_error(config: dict[str, Any]) -> str | None: + """Return an error message if ``config['timeout']`` is invalid, else None. + + Shared by execute() and validate() so both paths reject the same + values with the same message. An absent ``timeout`` is valid (the + default is used). bool is a subclass of int, but ``timeout: true`` is a + config error rather than a duration, so it is rejected explicitly. + Non-finite floats (YAML ``.inf``/``.nan``) pass a plain ``> 0`` check + but would raise in subprocess.run(), so they are rejected too. + """ + if "timeout" not in config: + return None + timeout = config["timeout"] + if ( + isinstance(timeout, bool) + or not isinstance(timeout, (int, float)) + or not math.isfinite(timeout) + or timeout <= 0 + ): + return ( + f"Shell step {config.get('id', '?')!r}: 'timeout' must be a " + f"positive number of seconds, got {timeout!r}." + ) + return None + def validate(self, config: dict[str, Any]) -> list[str]: errors = super().validate(config) if "run" not in config: @@ -114,16 +146,7 @@ def validate(self, config: dict[str, Any]) -> list[str]: f"Shell step {config.get('id', '?')!r}: 'output_format' must " f"be 'json' when present, got {output_format!r}." ) - if "timeout" in config: - timeout = config["timeout"] - # bool is an int subclass, so reject it explicitly. - if ( - isinstance(timeout, bool) - or not isinstance(timeout, int) - or timeout <= 0 - ): - errors.append( - f"Shell step {config.get('id', '?')!r}: 'timeout' must be a " - f"positive integer (seconds) when present, got {timeout!r}." - ) + timeout_error = self._timeout_error(config) + if timeout_error is not None: + errors.append(timeout_error) return errors diff --git a/tests/test_workflows.py b/tests/test_workflows.py index 0026124f43..11ff138074 100644 --- a/tests/test_workflows.py +++ b/tests/test_workflows.py @@ -1381,107 +1381,6 @@ def test_validate_accepts_string_and_expression_run(self): assert step.validate({"id": "s", "run": "echo hi"}) == [] assert step.validate({"id": "s", "run": "{{ steps.x.output }}"}) == [] - def test_timeout_is_configurable(self, monkeypatch): - """A 'timeout' field overrides the 300s default (#3327).""" - import subprocess as sp - - from specify_cli.workflows.steps.shell import ShellStep - from specify_cli.workflows.base import StepContext, StepStatus - - seen = {} - real_run = sp.run - - def spy_run(*args, **kwargs): - seen["timeout"] = kwargs.get("timeout") - return real_run(*args, **kwargs) - - monkeypatch.setattr( - "specify_cli.workflows.steps.shell.subprocess.run", spy_run - ) - step = ShellStep() - result = step.execute( - {"id": "t", "run": "echo hi", "timeout": 1800}, StepContext() - ) - assert result.status == StepStatus.COMPLETED - assert seen["timeout"] == 1800 - - def test_timeout_defaults_to_300(self, monkeypatch): - import subprocess as sp - - from specify_cli.workflows.steps.shell import ShellStep - from specify_cli.workflows.base import StepContext, StepStatus - - seen = {} - real_run = sp.run - - def spy_run(*args, **kwargs): - seen["timeout"] = kwargs.get("timeout") - return real_run(*args, **kwargs) - - monkeypatch.setattr( - "specify_cli.workflows.steps.shell.subprocess.run", spy_run - ) - result = ShellStep().execute({"id": "t", "run": "echo hi"}, StepContext()) - assert result.status == StepStatus.COMPLETED - assert seen["timeout"] == 300 - - def test_timeout_error_reports_configured_value(self, monkeypatch): - import subprocess as sp - - from specify_cli.workflows.steps.shell import ShellStep - from specify_cli.workflows.base import StepContext, StepStatus - - def raise_timeout(*args, **kwargs): - raise sp.TimeoutExpired(cmd="x", timeout=kwargs.get("timeout")) - - monkeypatch.setattr( - "specify_cli.workflows.steps.shell.subprocess.run", raise_timeout - ) - result = ShellStep().execute( - {"id": "t", "run": "sleep 999", "timeout": 7}, StepContext() - ) - assert result.status == StepStatus.FAILED - assert "7 seconds" in result.error - - @pytest.mark.parametrize("bad", [0, -5, "600", 1.5, None, True]) - def test_execute_ignores_unvalidated_bad_timeout(self, bad, monkeypatch): - """execute() falls back to 300 when config skipped validation (#3327).""" - import subprocess as sp - - from specify_cli.workflows.steps.shell import ShellStep - from specify_cli.workflows.base import StepContext, StepStatus - - seen = {} - real_run = sp.run - - def spy_run(*args, **kwargs): - seen["timeout"] = kwargs.get("timeout") - return real_run(*args, **kwargs) - - monkeypatch.setattr( - "specify_cli.workflows.steps.shell.subprocess.run", spy_run - ) - result = ShellStep().execute( - {"id": "t", "run": "echo hi", "timeout": bad}, StepContext() - ) - assert result.status == StepStatus.COMPLETED - assert seen["timeout"] == 300 - - @pytest.mark.parametrize("bad", [0, -5, "600", 1.5, None, True]) - def test_validate_rejects_bad_timeout(self, bad): - from specify_cli.workflows.steps.shell import ShellStep - - errors = ShellStep().validate({"id": "s", "run": "echo hi", "timeout": bad}) - assert any("'timeout'" in e for e in errors) - - def test_validate_accepts_positive_int_timeout(self): - from specify_cli.workflows.steps.shell import ShellStep - - assert ( - ShellStep().validate({"id": "s", "run": "echo hi", "timeout": 1800}) == [] - ) - - def test_output_format_json_exposes_data(self, tmp_path): from specify_cli.workflows.steps.shell import ShellStep from specify_cli.workflows.base import StepContext, StepStatus @@ -1538,6 +1437,130 @@ def test_validate_rejects_unknown_output_format(self): errors = step.validate({"id": "emit", "run": "exit 0", "output_format": "yaml"}) assert any("'output_format' must be 'json'" in e for e in errors) + def test_configured_timeout_is_passed_to_subprocess(self, monkeypatch): + """A ``timeout:`` value on the step overrides the 300s default and is + threaded through to ``subprocess.run`` (issue #3327).""" + import subprocess + + from specify_cli.workflows.steps.shell import ShellStep + from specify_cli.workflows.base import StepContext, StepStatus + + captured: dict[str, object] = {} + + def fake_run(*args, **kwargs): + captured["timeout"] = kwargs.get("timeout") + return subprocess.CompletedProcess( + args=args[0] if args else "", returncode=0, stdout="", stderr="" + ) + + monkeypatch.setattr(subprocess, "run", fake_run) + step = ShellStep() + result = step.execute( + {"id": "qa", "run": "echo hi", "timeout": 1800}, StepContext() + ) + assert result.status == StepStatus.COMPLETED + assert captured["timeout"] == 1800 + + def test_default_timeout_preserved_when_omitted(self, monkeypatch): + """Omitting ``timeout:`` preserves the historical 300s default.""" + import subprocess + + from specify_cli.workflows.steps.shell import ShellStep + from specify_cli.workflows.base import StepContext + + captured: dict[str, object] = {} + + def fake_run(*args, **kwargs): + captured["timeout"] = kwargs.get("timeout") + return subprocess.CompletedProcess( + args=args[0] if args else "", returncode=0, stdout="", stderr="" + ) + + monkeypatch.setattr(subprocess, "run", fake_run) + step = ShellStep() + step.execute({"id": "qa", "run": "echo hi"}, StepContext()) + assert captured["timeout"] == 300 + + def test_timeout_error_reports_configured_value(self, monkeypatch): + """The timeout failure message reflects the configured duration, not a + hardcoded 300.""" + import subprocess + + from specify_cli.workflows.steps.shell import ShellStep + from specify_cli.workflows.base import StepContext, StepStatus + + def fake_run(*args, **kwargs): + raise subprocess.TimeoutExpired(cmd="echo hi", timeout=5) + + monkeypatch.setattr(subprocess, "run", fake_run) + step = ShellStep() + result = step.execute( + {"id": "qa", "run": "echo hi", "timeout": 5}, StepContext() + ) + assert result.status == StepStatus.FAILED + assert "5 seconds" in (result.error or "") + + def test_execute_fails_cleanly_on_invalid_timeout(self, monkeypatch): + """execute() must fail the step (not raise) on an invalid timeout even + when validate() was skipped — the engine does not auto-validate step + config, so an unvalidated string/bool/non-finite timeout would + otherwise crash subprocess.run() and take down the whole run.""" + import subprocess + + from specify_cli.workflows.steps.shell import ShellStep + from specify_cli.workflows.base import StepContext, StepStatus + + def fail_if_called(*args, **kwargs): + raise AssertionError("subprocess.run should not run on invalid timeout") + + monkeypatch.setattr(subprocess, "run", fail_if_called) + step = ShellStep() + # A string would raise TypeError; ``True`` would silently become a 1s + # timeout (bool is an int subclass); ``.inf`` would raise at runtime. + for bad in ("30", True, float("inf"), 0): + result = step.execute( + {"id": "qa", "run": "echo hi", "timeout": bad}, StepContext() + ) + assert result.status == StepStatus.FAILED + assert "'timeout' must be a positive number" in (result.error or "") + + def test_validate_rejects_non_positive_timeout(self): + from specify_cli.workflows.steps.shell import ShellStep + + step = ShellStep() + for bad in (0, -30): + errors = step.validate({"id": "qa", "run": "echo hi", "timeout": bad}) + assert any("'timeout' must be a positive number" in e for e in errors) + + def test_validate_rejects_non_numeric_timeout(self): + from specify_cli.workflows.steps.shell import ShellStep + + step = ShellStep() + # A string and a bool are both invalid (bool is an int subclass but a + # config error, not a duration). + for bad in ("30", True): + errors = step.validate({"id": "qa", "run": "echo hi", "timeout": bad}) + assert any("'timeout' must be a positive number" in e for e in errors) + + def test_validate_rejects_non_finite_timeout(self): + from specify_cli.workflows.steps.shell import ShellStep + + step = ShellStep() + # inf/nan are floats and slip past a plain ``> 0`` check (``nan <= 0`` + # is False), but ``subprocess.run(timeout=...)`` would then fail at + # runtime. YAML ``.inf``/``.nan`` scalars parse to these via safe_load. + for bad in (float("inf"), float("-inf"), float("nan")): + errors = step.validate({"id": "qa", "run": "echo hi", "timeout": bad}) + assert any("'timeout' must be a positive number" in e for e in errors) + + def test_validate_accepts_positive_numeric_timeout(self): + from specify_cli.workflows.steps.shell import ShellStep + + step = ShellStep() + for good in (1, 300, 1800, 12.5): + errors = step.validate({"id": "qa", "run": "echo hi", "timeout": good}) + assert not any("'timeout'" in e for e in errors) + class _StubStdin: """Stdin stub exposing only a fixed ``isatty`` result. diff --git a/workflows/README.md b/workflows/README.md index 966e2d390f..93b006ec5d 100644 --- a/workflows/README.md +++ b/workflows/README.md @@ -112,8 +112,14 @@ Run a shell command and capture output: - id: run-tests type: shell run: "cd {{ inputs.project_dir }} && npm test" + timeout: 1800 # Optional: max seconds before the command is killed (default 300) ``` +`timeout` is the maximum time in seconds the command may run before it is +killed and the step fails; it must be a positive number and defaults to +`300` (five minutes) when omitted. Raise it for long-running gates such as +full builds, linter aggregators, or integration-test targets. + ### Init Steps Bootstrap a project the same way `specify init` does — scaffolding From 086929e5463c5d494e5c9c7b5f244c28b7c7b8db Mon Sep 17 00:00:00 2001 From: Marsel Safin <179933638+marcelsafin@users.noreply.github.com> Date: Mon, 13 Jul 2026 17:49:27 +0200 Subject: [PATCH 15/45] fix(templates): point constitution sync checklist at installed command files (#3418) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * fix(templates): point constitution sync checklist at installed command files The consistency-propagation checklist told the agent to read .specify/templates/commands/*.md, but specify init never creates that directory — command templates are rendered straight into the agent-specific directory (.github/prompts/, .claude/commands/, ...). The checklist step could therefore never run against real files. Point it at the installed speckit.* command files for the active agent instead. Fixes #660 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(templates): cover hyphenated and skills-mode command filenames Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(templates): use actual integration output directories in examples Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * docs(templates): cover skills-based command layouts in sync checklist Copilot skills mode installs speckit-/SKILL.md under .github/skills/, not .github/agents/. Mention both directories and the SKILL.md layout. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * docs(templates): restore hyphenated speckit-* naming in sync checklist The previous commit dropped the speckit-* flat-file variant used by Cline and others while adding the SKILL.md layout. Name all three: speckit.*, speckit-*, and speckit-/SKILL.md. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * docs: clarify agent-specific reference phrasing in constitution template Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- templates/commands/constitution.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/templates/commands/constitution.md b/templates/commands/constitution.md index 7ba1c7640f..9d2d4ccc1a 100644 --- a/templates/commands/constitution.md +++ b/templates/commands/constitution.md @@ -81,7 +81,7 @@ Follow this execution flow: - Read `.specify/templates/plan-template.md` and ensure any "Constitution Check" or rules align with updated principles. - Read `.specify/templates/spec-template.md` for scope/requirements alignment—update if constitution adds/removes mandatory sections or constraints. - Read `.specify/templates/tasks-template.md` and ensure task categorization reflects new or removed principle-driven task types (e.g., observability, versioning, testing discipline). - - Read each command file in `.specify/templates/commands/*.md` (including this one) to verify no outdated references (agent-specific names like CLAUDE only) remain when generic guidance is required. + - Read each installed Spec Kit command file for your agent (including this one) — named `speckit.*` or `speckit-*` (dot or hyphen depending on the agent), or laid out as `speckit-/SKILL.md` for skills-based integrations, e.g. in `.github/agents/`, `.github/skills/`, `.claude/skills/`, or your agent's equivalent commands directory — to verify no outdated references (CLAUDE-only or other agent-specific names) remain when generic guidance is required. - Read any runtime guidance docs (e.g., `README.md`, `docs/quickstart.md`, or agent-specific guidance files if present). Update references to principles changed. 5. Produce a Sync Impact Report (prepend as an HTML comment at top of the constitution file after update): From 3b7d95a408a98839dbdd6c14eb1cb4a805ee6731 Mon Sep 17 00:00:00 2001 From: Marsel Safin <179933638+marcelsafin@users.noreply.github.com> Date: Mon, 13 Jul 2026 17:50:59 +0200 Subject: [PATCH 16/45] fix: rewrite extension-relative subdir paths in generated command bodies (#3444) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * fix: rewrite extension-relative subdir paths in generated command bodies Extension command bodies reference bundled files relative to the extension root (agents/, knowledge-base/, templates/, ...). Generated SKILL.md and command files emitted those paths verbatim, so agents resolved them against the workspace root where they do not exist. Add CommandRegistrar.rewrite_extension_paths, which rewrites references to subdirectories that actually exist in the installed extension to .specify/extensions//..., and call it once in register_commands so every output format and alias gets the fix. commands/, specs/ and dot-directories are never rewritten. Fixes #2101 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: only rewrite relative extension path references Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: use callable re.sub replacement for extension subdir rewrite subdir and extension_id come from filesystem directory names and were interpolated into a re.sub string replacement template. A directory name containing a backslash (e.g. assets\q) would raise re.error: bad escape, aborting command registration even when the body didn't reference it. Use a callable replacement so these values are treated literally. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: make subdir rewrite regression test cross-platform Renamed the test's subdir fixture from "assets\\q" to "assets[q]": on Windows, backslash is a path separator, so mkdir would create nested "assets/q" dirs instead of one literally-named directory, and iterdir() would only discover "assets", never exercising the rewrite. extension_id keeps a real backslash/"\\1" since it isn't used to create a directory, still verifying the callable replacement handles it literally. Added a sanity assertion for this assumption. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: apply extension subdir path rewrite in skills-mode renderer register_commands() rewrote extension-relative subdir references (agents/, knowledge-base/, etc.) via rewrite_extension_paths(), but _register_extension_skills() - the separate renderer used for active non-native skills agents (e.g. Claude with ai_skills: true) - never called it. Generated SKILL.md files left agents/... and knowledge-base/... unresolved, and mapped the extension's own templates/ through the generic project-level rewrite instead of its installed .specify/extensions//templates/ location. Reuse the existing rewrite_extension_paths() helper in _register_extension_skills() at the same point register_commands() applies it (before resolve_skill_placeholders' generic rewrite), and add a skills-mode regression test. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: apply extension subdir path rewrite on preset restore/reconcile paths _unregister_skills() restored extension-backed SKILL.md content via resolve_skill_placeholders() without first calling rewrite_extension_paths(), so removing a preset override that shadowed an extension command restored the bare, unresolvable agents/... and knowledge-base/... references. Carried extension_id/extension_dir through _build_extension_skill_restore_index() and applied the same rewrite used at initial registration before restoring. Found the identical gap in _reconcile_composed_commands()'s non-skill agent path: when a removed preset's command reverts to an extension winner, register_commands_for_non_skill_agents() was called without extension_id, so the rewrite never ran for plain command-file agents either. Passed extension_id through there too. Added regression tests for both restore paths (skills-mode and non-skill-agent command files) in tests/test_presets.py. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: apply extension subdir path rewrite when composing over extension base PresetResolver.resolve_content() read the effective base layer's raw content directly via path.read_text() before composing append/prepend/ wrap overlays on top of it, and its outright-replace shortcut did the same. When that base layer was extension-provided, neither read path applied rewrite_extension_paths(), so composing a preset over an extension command (or an extension winning outright through resolve_content) left bare, unresolvable agents/... and knowledge-base/... references in the composed output. All three call sites (PresetManager._register_commands()'s composed path, _reconcile_composed_commands()'s composed path, and skills-mode reading the .composed file written by either) consume resolve_content's return value, so fixing the read at its source covers command output, skill output, and both initial-install and reconcile flows without threading extension identity through each caller. Tagged extension layers in collect_all_layers() with extension_id/ extension_dir, and added a _read_layer_content() helper in resolve_content() that applies rewrite_extension_paths() whenever a layer carries that extension identity — used at both raw-read sites (outright-replace shortcut and composition base). Composing (append/prepend/wrap) layers are never extension-provided (extensions are always inserted with strategy "replace"), so no other read site needs the rewrite. Added regression tests: a parametrized resolve_content() test covering append/prepend/wrap composing over an extension base, a skills-mode test asserting the composed SKILL.md resolves the extension's subdir references, and a non-skill-agent (Gemini) install-time test matching the reported live repro. Assisted-by: GitHub Copilot (model: Claude Sonnet 5, autonomous) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- src/specify_cli/agents.py | 49 ++++ src/specify_cli/extensions/__init__.py | 5 + src/specify_cli/presets/__init__.py | 42 +++- tests/test_extension_skills.py | 59 +++++ tests/test_extensions.py | 179 +++++++++++++ tests/test_presets.py | 333 +++++++++++++++++++++++++ 6 files changed, 665 insertions(+), 2 deletions(-) diff --git a/src/specify_cli/agents.py b/src/specify_cli/agents.py index c535869121..e4d09ffe99 100644 --- a/src/specify_cli/agents.py +++ b/src/specify_cli/agents.py @@ -213,6 +213,52 @@ def rewrite_project_relative_paths( ".specify.specify/", ".specify/" ) + @staticmethod + def rewrite_extension_paths( + text: str, extension_id: str, extension_dir: Path + ) -> str: + """Rewrite extension-relative paths to their installed locations. + + Extension command bodies reference bundled files relative to the + extension root (e.g. ``agents/control/commander.md``). After install + those files live under ``.specify/extensions//``, so bare + references would resolve against the workspace root and never be + found (#2101). + + Only directories that actually exist inside *extension_dir* are + rewritten, keeping the behaviour conservative and avoiding false + positives on prose. ``commands`` (slash-command sources), ``specs`` + (user project artifacts) and dot-directories are never rewritten. + """ + if not isinstance(text, str) or not text: + return text + + skip = {"commands", ".git", "specs"} + try: + subdirs = [ + entry.name + for entry in extension_dir.iterdir() + if entry.is_dir() + and entry.name not in skip + and not entry.name.startswith(".") + ] + except OSError: + return text + + for subdir in subdirs: + # Only rewrite relative references (subdir/... or ./subdir/...); + # absolute paths like /subdir/... keep their meaning. Use a + # callable replacement: subdir/extension_id come from the + # filesystem and could contain backslashes or "\1"-like + # sequences, which would corrupt a string replacement template. + replacement = f".specify/extensions/{extension_id}/{subdir}/" + text = re.sub( + r'(^|[\s`"\'(])(?:\./)?' + re.escape(subdir) + "/", + lambda m: m.group(1) + replacement, + text, + ) + return text + def render_markdown_command( self, frontmatter: dict, body: str, source_id: str, context_note: str = None ) -> str: @@ -639,6 +685,9 @@ def register_commands( frontmatter[key] = core_frontmatter[key] frontmatter.pop("strategy", None) + if extension_id: + body = self.rewrite_extension_paths(body, extension_id, source_root) + frontmatter = self._adjust_script_paths( frontmatter, extension_id=extension_id ) diff --git a/src/specify_cli/extensions/__init__.py b/src/specify_cli/extensions/__init__.py index 1f898c5f78..aac1ba6481 100644 --- a/src/specify_cli/extensions/__init__.py +++ b/src/specify_cli/extensions/__init__.py @@ -1078,6 +1078,11 @@ def _register_extension_skills( frontmatter = registrar._adjust_script_paths( frontmatter, extension_id=manifest.id ) + # Mirror the register_commands() rewrite (#2101): resolve + # extension-relative subdir references (agents/, knowledge-base/, + # etc.) to their installed .specify/extensions// location + # before the generic placeholder/path resolution below. + body = registrar.rewrite_extension_paths(body, manifest.id, extension_dir) body = registrar.resolve_skill_placeholders( selected_ai, frontmatter, body, self.project_root, extension_id=manifest.id ) diff --git a/src/specify_cli/presets/__init__.py b/src/specify_cli/presets/__init__.py index 863b6ef7dc..81e8eefc62 100644 --- a/src/specify_cli/presets/__init__.py +++ b/src/specify_cli/presets/__init__.py @@ -778,6 +778,7 @@ def _reconcile_composed_commands(self, command_names: List[str]) -> None: matching_cmds, ext_id, ext_dir, self.project_root, context_note=f"\n\n\n", + extension_id=ext_id, ) registered = True except Exception: @@ -1199,6 +1200,8 @@ def _build_extension_skill_restore_index(self) -> Dict[str, Dict[str, Any]]: "command_name": cmd_name, "source_file": source_file, "source": f"extension:{manifest.id}", + "extension_id": manifest.id, + "extension_dir": ext_root, } modern_skill_name, legacy_skill_name = self._skill_names_for_command(cmd_name) restore_index.setdefault(modern_skill_name, restore_info) @@ -1463,6 +1466,17 @@ def _unregister_skills(self, skill_names: List[str], preset_dir: Path) -> None: if extension_restore: content = extension_restore["source_file"].read_text(encoding="utf-8") frontmatter, body = registrar.parse_frontmatter(content) + # Mirror the register-time rewrite (#2101): resolve + # extension-relative subdir references (agents/, + # knowledge-base/, etc.) to their installed location before + # the generic placeholder resolution below, otherwise + # restoring after a preset override removal would leave + # bare, unresolvable paths in the skill body. + body = registrar.rewrite_extension_paths( + body, + extension_restore["extension_id"], + extension_restore["extension_dir"], + ) if isinstance(selected_ai, str): body = registrar.resolve_skill_placeholders( selected_ai, frontmatter, body, self.project_root @@ -3038,6 +3052,8 @@ def _find_in_subdirs(base_dir: Path) -> Optional[Path]: "path": candidate, "source": source, "strategy": "replace", + "extension_id": ext_id, + "extension_dir": ext_dir, }) # Priority 4: Core templates (always "replace") @@ -3157,10 +3173,32 @@ def resolve_content( if not layers: return None + def _read_layer_content(layer: Dict[str, Any]) -> str: + """Read a layer's raw text, rewriting extension-relative subdir + references (agents/, knowledge-base/, etc.) to their installed + location when the layer is extension-provided (#2101). + + Extension layers are always inserted with strategy "replace" + (see collect_all_layers), so a layer only ever needs this + rewrite when it wins outright above or serves as the + composition base below — never as a mid-stack composing + (append/prepend/wrap) layer. + """ + text = layer["path"].read_text(encoding="utf-8") + extension_id = layer.get("extension_id") + extension_dir = layer.get("extension_dir") + if extension_id and extension_dir: + from ..agents import CommandRegistrar + + text = CommandRegistrar.rewrite_extension_paths( + text, extension_id, extension_dir + ) + return text + # If the top (highest-priority) layer is replace, it wins entirely — # lower layers are irrelevant regardless of their strategies. if layers[0]["strategy"] == "replace": - return layers[0]["path"].read_text(encoding="utf-8") + return _read_layer_content(layers[0]) # Composition: build content bottom-up from the effective base. # The base is the nearest replace layer scanning from highest priority @@ -3183,7 +3221,7 @@ def resolve_content( # Convert to reversed_layers index base_reversed_idx = len(layers) - 1 - base_layer_idx - content = layers[base_layer_idx]["path"].read_text(encoding="utf-8") + content = _read_layer_content(layers[base_layer_idx]) # Compose only the layers above the base (higher priority = lower index in layers, # higher index in reversed_layers). Process bottom-up from base+1. start_idx = base_reversed_idx + 1 diff --git a/tests/test_extension_skills.py b/tests/test_extension_skills.py index c2d9bb439e..2ebd3f5e6b 100644 --- a/tests/test_extension_skills.py +++ b/tests/test_extension_skills.py @@ -911,6 +911,65 @@ def test_skill_registration_uses_extension_local_script_paths(self, project_dir, assert ".specify/scripts/bash/resolve-skill.sh" not in content assert ".specify/scripts/bash/ensure-skills.sh" not in content + def test_skill_registration_rewrites_extension_subdir_paths(self, project_dir, temp_dir): + """Auto-registered skills should resolve extension-relative subdir + references (agents/, knowledge-base/) to their installed location, + matching the rewrite already applied by register_commands() (#2101).""" + _create_init_options(project_dir, ai="claude", ai_skills=True) + skills_dir = _create_skills_dir(project_dir, ai="claude") + + ext_dir = temp_dir / "path-ext" + ext_dir.mkdir() + manifest_data = { + "schema_version": "1.0", + "extension": { + "id": "path-ext", + "name": "Path Extension", + "version": "1.0.0", + "description": "Test", + }, + "requires": {"speckit_version": ">=0.1.0"}, + "provides": { + "commands": [ + { + "name": "speckit.path-ext.run", + "file": "commands/run.md", + "description": "Run command", + } + ] + }, + } + with open(ext_dir / "extension.yml", "w") as f: + yaml.safe_dump(manifest_data, f) + + (ext_dir / "commands").mkdir() + (ext_dir / "agents" / "control").mkdir(parents=True) + (ext_dir / "agents" / "control" / "commander.md").write_text("# Commander\n") + (ext_dir / "knowledge-base").mkdir() + (ext_dir / "knowledge-base" / "agent-scores.yaml").write_text("scores: {}\n") + (ext_dir / "templates").mkdir() + (ext_dir / "templates" / "kill-report.md").write_text("# Kill Report\n") + + (ext_dir / "commands" / "run.md").write_text( + "---\n" + "description: Run command\n" + "---\n\n" + "Read agents/control/commander.md and knowledge-base/agent-scores.yaml.\n" + "Use templates/kill-report.md as the report template.\n" + ) + + manager = ExtensionManager(project_dir) + manager.install_from_directory(ext_dir, "0.1.0", register_commands=False) + + content = (skills_dir / "speckit-path-ext-run" / "SKILL.md").read_text() + assert ".specify/extensions/path-ext/agents/control/commander.md" in content + assert ".specify/extensions/path-ext/knowledge-base/agent-scores.yaml" in content + # extension's own templates/ dir must resolve under the extension, + # not the project-level .specify/templates/ + assert ".specify/extensions/path-ext/templates/kill-report.md" in content + assert "Read agents/control" not in content + assert "and knowledge-base/" not in content + def test_missing_command_file_skipped(self, skills_project, temp_dir): """Commands with missing source files should be skipped gracefully.""" project_dir, skills_dir = skills_project diff --git a/tests/test_extensions.py b/tests/test_extensions.py index 0003699693..d9ffa5ca6f 100644 --- a/tests/test_extensions.py +++ b/tests/test_extensions.py @@ -2376,6 +2376,185 @@ def test_codex_skill_registration_fallback_prefers_powershell_on_windows( assert ".specify/scripts/powershell/setup-plan.ps1 -Json" in content assert ".specify/scripts/bash/setup-plan.sh" not in content + @staticmethod + def _make_subdir_extension(temp_dir, ext_id="echelon", aliases=None): + """Create an extension whose command body references bundled subdirs.""" + import yaml + + ext_dir = temp_dir / ext_id + ext_dir.mkdir() + (ext_dir / "commands").mkdir() + (ext_dir / "agents" / "control").mkdir(parents=True) + (ext_dir / "knowledge-base").mkdir() + (ext_dir / "templates").mkdir() + (ext_dir / "specs" / "001-internal").mkdir(parents=True) + + command = { + "name": f"speckit.{ext_id}.run", + "file": "commands/run.md", + "description": "Run", + } + if aliases: + command["aliases"] = aliases + manifest_data = { + "schema_version": "1.0", + "extension": { + "id": ext_id, + "name": "Echelon", + "version": "1.0.0", + "description": "Test", + }, + "requires": {"speckit_version": ">=0.1.0"}, + "provides": {"commands": [command]}, + } + with open(ext_dir / "extension.yml", "w") as f: + yaml.dump(manifest_data, f) + + (ext_dir / "commands" / "run.md").write_text( + "---\ndescription: Run\n---\n\n" + "Read agents/control/commander.md for instructions.\n" + "Load knowledge-base/agent-scores.yaml for calibration.\n" + "Use templates/kill-report.md as output format.\n" + "Artifacts go to specs/001-internal/plan.md.\n" + "See commands/run.md for the source.\n" + ) + return ext_dir + + def test_codex_skill_registration_rewrites_extension_subdir_paths( + self, project_dir, temp_dir + ): + """Extension-relative subdir refs must point at the installed location.""" + ext_dir = self._make_subdir_extension(temp_dir) + + skills_dir = project_dir / ".agents" / "skills" + skills_dir.mkdir(parents=True) + + manifest = ExtensionManifest(ext_dir / "extension.yml") + registrar = CommandRegistrar() + registrar.register_commands_for_agent("codex", manifest, ext_dir, project_dir) + + content = (skills_dir / "speckit-echelon-run" / "SKILL.md").read_text() + assert ".specify/extensions/echelon/agents/control/commander.md" in content + assert ".specify/extensions/echelon/knowledge-base/agent-scores.yaml" in content + assert ".specify/extensions/echelon/templates/kill-report.md" in content + assert "Read agents/" not in content + # specs/ refs point at the user's project artifacts, never the extension + assert "to specs/001-internal/plan.md" in content + assert ".specify/extensions/echelon/specs/" not in content + # commands/ refs are slash-command sources, not runtime reads + assert "See commands/run.md" in content + + def test_skill_registration_rewrites_extension_subdir_paths_in_aliases( + self, project_dir, temp_dir + ): + """Alias skills reuse the rewritten body.""" + ext_dir = self._make_subdir_extension( + temp_dir, ext_id="ext-alias-paths", aliases=["speckit.ext-alias-paths.go"] + ) + + skills_dir = project_dir / ".agents" / "skills" + skills_dir.mkdir(parents=True) + + manifest = ExtensionManifest(ext_dir / "extension.yml") + registrar = CommandRegistrar() + registrar.register_commands_for_agent("codex", manifest, ext_dir, project_dir) + + alias_content = ( + skills_dir / "speckit-ext-alias-paths-go" / "SKILL.md" + ).read_text() + assert ( + ".specify/extensions/ext-alias-paths/agents/control/commander.md" + in alias_content + ) + assert "Read agents/" not in alias_content + + def test_markdown_registration_rewrites_extension_subdir_paths( + self, project_dir, temp_dir + ): + """Markdown-format agents get the same rewrite via the shared path.""" + ext_dir = self._make_subdir_extension(temp_dir, ext_id="ext-md-paths") + + amp_dir = project_dir / ".agents" / "commands" + amp_dir.mkdir(parents=True) + + manifest = ExtensionManifest(ext_dir / "extension.yml") + registrar = CommandRegistrar() + registrar.register_commands_for_agent("amp", manifest, ext_dir, project_dir) + + content = (amp_dir / "speckit.ext-md-paths.run.md").read_text() + assert ".specify/extensions/ext-md-paths/agents/control/commander.md" in content + assert "Read agents/" not in content + + def test_rewrite_extension_paths_only_rewrites_existing_subdirs(self, temp_dir): + """Only directories present in the extension are rewritten.""" + from specify_cli.agents import CommandRegistrar as AgentCommandRegistrar + + ext_dir = temp_dir / "ext-existing" + (ext_dir / "agents").mkdir(parents=True) + (ext_dir / ".hidden").mkdir() + + text = ( + "Read agents/one.md then knowledge-base/two.md.\n" + "Also ./agents/three.md but not /agents/abs.md.\n" + "Keep .hidden/secret.md alone.\n" + ) + rewritten = AgentCommandRegistrar.rewrite_extension_paths( + text, "ext-existing", ext_dir + ) + + assert ".specify/extensions/ext-existing/agents/one.md" in rewritten + assert "Also .specify/extensions/ext-existing/agents/three.md" in rewritten + # absolute paths keep their meaning + assert "not /agents/abs.md" in rewritten + # knowledge-base/ does not exist in this extension: left untouched + assert "then knowledge-base/two.md" in rewritten + assert ".hidden/secret.md" in rewritten + assert ".specify/extensions/ext-existing/.hidden/" not in rewritten + + def test_rewrite_extension_paths_handles_regex_special_replacement_text( + self, temp_dir + ): + """subdir/extension_id containing regex-replacement-special characters + (e.g. backslash / group references) must not raise or be misinterpreted + by re.sub's replacement template (#2101). + + The subdir name uses brackets rather than a backslash: on Windows, + "\\" is a path separator, so a subdir literally named "assets\\q" + would create nested directories "assets/q" instead of a single + directory, and iterdir() would then only discover "assets" - never + exercising the intended replacement text. extension_id isn't used to + create a directory, so it's free to contain a real backslash/"\\1" + to verify the callable replacement treats it literally. + """ + from specify_cli.agents import CommandRegistrar as AgentCommandRegistrar + + ext_dir = temp_dir / "ext-backslash" + weird_subdir = "assets[q]" + (ext_dir / weird_subdir).mkdir(parents=True) + # sanity-check the cross-platform assumption above + assert [p.name for p in ext_dir.iterdir()] == [weird_subdir] + + text = f"Read {weird_subdir}/file.md but not /{weird_subdir}/abs.md.\n" + rewritten = AgentCommandRegistrar.rewrite_extension_paths( + text, "ext\\1", ext_dir + ) + + assert f".specify/extensions/ext\\1/{weird_subdir}/file.md" in rewritten + # absolute paths are still left untouched + assert f"/{weird_subdir}/abs.md" in rewritten + + def test_rewrite_extension_paths_missing_dir_returns_text(self, temp_dir): + """A missing extension directory leaves the text unchanged.""" + from specify_cli.agents import CommandRegistrar as AgentCommandRegistrar + + text = "Read agents/one.md." + assert ( + AgentCommandRegistrar.rewrite_extension_paths( + text, "gone", temp_dir / "does-not-exist" + ) + == text + ) + def test_register_commands_for_copilot(self, extension_dir, project_dir): """Test registering commands for Copilot agent with .agent.md extension.""" # Create .github/agents directory (Copilot project) diff --git a/tests/test_presets.py b/tests/test_presets.py index a4c78d9268..20abab39cf 100644 --- a/tests/test_presets.py +++ b/tests/test_presets.py @@ -3663,6 +3663,8 @@ def test_extension_skill_restored_on_preset_remove(self, project_dir, temp_dir): extension_dir = project_dir / ".specify" / "extensions" / "fakeext" (extension_dir / "commands").mkdir(parents=True, exist_ok=True) + (extension_dir / "agents" / "control").mkdir(parents=True, exist_ok=True) + (extension_dir / "agents" / "control" / "commander.md").write_text("# Commander\n") (extension_dir / "commands" / "cmd.md").write_text( "---\n" "description: Extension fakeext cmd\n" @@ -3671,6 +3673,7 @@ def test_extension_skill_restored_on_preset_remove(self, project_dir, temp_dir): "---\n\n" "extension:fakeext\n" "Run {SCRIPT}\n" + "Read agents/control/commander.md for context.\n" ) extension_manifest = { "schema_version": "1.0", @@ -3736,8 +3739,92 @@ def test_extension_skill_restored_on_preset_remove(self, project_dir, temp_dir): assert "source: extension:fakeext" in content assert "extension:fakeext" in content assert '.specify/scripts/bash/setup-plan.sh --json "$ARGUMENTS"' in content + # Extension-relative subdir references must resolve to their + # installed location on restore too (#2101), not just on first + # registration. + assert ".specify/extensions/fakeext/agents/control/commander.md" in content + assert "Read agents/control" not in content assert "# Fakeext Cmd Skill" in content + def test_skill_composed_over_extension_base_rewrites_subdir_paths( + self, project_dir, temp_dir + ): + """When a preset composes (append) over an extension-provided base + command, the resulting skill (read from the .composed output) must + still resolve the extension's own subdir references (#2101), not + just when the extension wins outright (replace).""" + self._write_init_options(project_dir, ai="codex") + skills_dir = project_dir / ".agents" / "skills" + self._create_skill(skills_dir, "speckit-fakeext-cmd", body="original extension skill") + + extension_dir = project_dir / ".specify" / "extensions" / "fakeext" + (extension_dir / "commands").mkdir(parents=True, exist_ok=True) + (extension_dir / "agents" / "control").mkdir(parents=True, exist_ok=True) + (extension_dir / "agents" / "control" / "commander.md").write_text("# Commander\n") + (extension_dir / "commands" / "cmd.md").write_text( + "---\ndescription: Extension fakeext cmd\n---\n\n" + "Read agents/control/commander.md for context.\n" + ) + extension_manifest = { + "schema_version": "1.0", + "extension": { + "id": "fakeext", + "name": "Fake Extension", + "version": "1.0.0", + "description": "Test", + }, + "requires": {"speckit_version": ">=0.1.0"}, + "provides": { + "commands": [ + { + "name": "speckit.fakeext.cmd", + "file": "commands/cmd.md", + "description": "Fake extension command", + } + ] + }, + } + with open(extension_dir / "extension.yml", "w") as f: + yaml.dump(extension_manifest, f) + + preset_dir = temp_dir / "ext-base-append-skill" + preset_dir.mkdir() + (preset_dir / "commands").mkdir() + (preset_dir / "commands" / "speckit.fakeext.cmd.md").write_text( + "---\ndescription: Preset overlay\n---\n\n## Extra\n" + ) + preset_manifest = { + "schema_version": "1.0", + "preset": { + "id": "ext-base-append-skill", + "name": "Ext Base Append Skill", + "version": "1.0.0", + "description": "Test", + }, + "requires": {"speckit_version": ">=0.1.0"}, + "provides": { + "templates": [ + { + "type": "command", + "name": "speckit.fakeext.cmd", + "file": "commands/speckit.fakeext.cmd.md", + "strategy": "append", + } + ] + }, + } + with open(preset_dir / "preset.yml", "w") as f: + yaml.dump(preset_manifest, f) + + manager = PresetManager(project_dir) + manager.install_from_directory(preset_dir, "0.1.5") + + skill_file = skills_dir / "speckit-fakeext-cmd" / "SKILL.md" + content = skill_file.read_text() + assert ".specify/extensions/fakeext/agents/control/commander.md" in content + assert "Read agents/control" not in content + assert "## Extra" in content + def test_preset_remove_skips_skill_dir_without_skill_file(self, project_dir, temp_dir): """Preset removal should not delete arbitrary directories missing SKILL.md.""" self._write_init_options(project_dir, ai="codex") @@ -5972,6 +6059,86 @@ def test_resolve_content_replace_over_wrap(self, project_dir, temp_dir, valid_pa content = resolver.resolve_content("spec-template") assert content == "# Replaced content\n" + @pytest.mark.parametrize("strategy", ["append", "prepend", "wrap"]) + def test_resolve_content_rewrites_extension_base_subdir_paths( + self, project_dir, temp_dir, strategy + ): + """Composing over an extension-provided base command must resolve the + extension's own subdir references (agents/, knowledge-base/) to their + installed location (#2101), not just when the extension wins outright. + """ + extension_dir = project_dir / ".specify" / "extensions" / "fakeext" + (extension_dir / "commands").mkdir(parents=True, exist_ok=True) + (extension_dir / "agents" / "control").mkdir(parents=True, exist_ok=True) + (extension_dir / "agents" / "control" / "commander.md").write_text("# Commander\n") + (extension_dir / "commands" / "cmd.md").write_text( + "---\ndescription: Extension fakeext cmd\n---\n\n" + "Read agents/control/commander.md for context.\n" + ) + extension_manifest = { + "schema_version": "1.0", + "extension": { + "id": "fakeext", + "name": "Fake Extension", + "version": "1.0.0", + "description": "Test", + }, + "requires": {"speckit_version": ">=0.1.0"}, + "provides": { + "commands": [ + { + "name": "speckit.fakeext.cmd", + "file": "commands/cmd.md", + "description": "Fake extension command", + } + ] + }, + } + with open(extension_dir / "extension.yml", "w") as f: + yaml.dump(extension_manifest, f) + + preset_dir = temp_dir / f"ext-base-{strategy}" + preset_dir.mkdir() + (preset_dir / "commands").mkdir() + overlay_body = ( + "{CORE_TEMPLATE}\n## Extra\n" if strategy == "wrap" else "## Extra\n" + ) + (preset_dir / "commands" / "speckit.fakeext.cmd.md").write_text( + f"---\ndescription: Preset overlay\n---\n\n{overlay_body}" + ) + preset_manifest = { + "schema_version": "1.0", + "preset": { + "id": f"ext-base-{strategy}", + "name": "Ext Base", + "version": "1.0.0", + "description": "Test", + }, + "requires": {"speckit_version": ">=0.1.0"}, + "provides": { + "templates": [ + { + "type": "command", + "name": "speckit.fakeext.cmd", + "file": "commands/speckit.fakeext.cmd.md", + "strategy": strategy, + } + ] + }, + } + with open(preset_dir / "preset.yml", "w") as f: + yaml.dump(preset_manifest, f) + + manager = PresetManager(project_dir) + manager.install_from_directory(preset_dir, "0.1.5") + + resolver = PresetResolver(project_dir) + content = resolver.resolve_content("speckit.fakeext.cmd", "command") + assert content is not None + assert ".specify/extensions/fakeext/agents/control/commander.md" in content + assert "Read agents/control" not in content + assert "## Extra" in content + class TestCollectAllLayers: """Test PresetResolver.collect_all_layers() method.""" @@ -6052,6 +6219,172 @@ def test_layers_read_strategy_from_manifest(self, project_dir, temp_dir, valid_p class TestRemoveReconciliation: """Test that removing a preset re-registers the next layer's command.""" + def test_remove_restores_extension_command_subdir_paths_for_non_skill_agent( + self, project_dir, temp_dir + ): + """When a preset override of an extension command is removed, the + reconciled non-skill-agent command file should have the extension's + own subdir references rewritten to their installed location (#2101), + not left as bare, unresolvable paths.""" + gemini_dir = project_dir / ".gemini" / "commands" + gemini_dir.mkdir(parents=True) + + extension_dir = project_dir / ".specify" / "extensions" / "fakeext" + (extension_dir / "commands").mkdir(parents=True, exist_ok=True) + (extension_dir / "agents" / "control").mkdir(parents=True, exist_ok=True) + (extension_dir / "agents" / "control" / "commander.md").write_text("# Commander\n") + (extension_dir / "commands" / "cmd.md").write_text( + "---\ndescription: Extension fakeext cmd\n---\n\n" + "Read agents/control/commander.md for context.\n" + ) + extension_manifest = { + "schema_version": "1.0", + "extension": { + "id": "fakeext", + "name": "Fake Extension", + "version": "1.0.0", + "description": "Test", + }, + "requires": {"speckit_version": ">=0.1.0"}, + "provides": { + "commands": [ + { + "name": "speckit.fakeext.cmd", + "file": "commands/cmd.md", + "description": "Fake extension command", + } + ] + }, + } + with open(extension_dir / "extension.yml", "w") as f: + yaml.dump(extension_manifest, f) + + manager = PresetManager(project_dir) + + preset_dir = temp_dir / "ext-cmd-override" + preset_dir.mkdir() + (preset_dir / "commands").mkdir() + (preset_dir / "commands" / "speckit.fakeext.cmd.md").write_text( + "---\ndescription: Override fakeext cmd\n---\n\npreset override content\n" + ) + preset_manifest = { + "schema_version": "1.0", + "preset": { + "id": "ext-cmd-override", + "name": "Ext Cmd Override", + "version": "1.0.0", + "description": "Test", + }, + "requires": {"speckit_version": ">=0.1.0"}, + "provides": { + "templates": [ + { + "type": "command", + "name": "speckit.fakeext.cmd", + "file": "commands/speckit.fakeext.cmd.md", + } + ] + }, + } + with open(preset_dir / "preset.yml", "w") as f: + yaml.dump(preset_manifest, f) + + manager.install_from_directory(preset_dir, "0.1.5") + + cmd_files = list(gemini_dir.glob("*fakeext*")) + assert cmd_files, "Command file should exist in gemini dir" + assert "preset override content" in cmd_files[0].read_text() + + manager.remove("ext-cmd-override") + + cmd_files = list(gemini_dir.glob("*fakeext*")) + assert cmd_files, "Command file should still exist after removal" + content = cmd_files[0].read_text() + assert "preset override content" not in content + assert ".specify/extensions/fakeext/agents/control/commander.md" in content + assert "Read agents/control" not in content + + def test_install_composes_extension_command_and_rewrites_subdir_paths_for_non_skill_agent( + self, project_dir, temp_dir + ): + """When a preset overlays (append) an extension-provided base command, + the initial composed non-skill-agent command file must have the + extension's own subdir references rewritten to their installed + location (#2101), matching the live repro: extension body + 'Read agents/control/commander.md', preset appends to + speckit.fakeext.cmd, generated Gemini content retains the bare path.""" + gemini_dir = project_dir / ".gemini" / "commands" + gemini_dir.mkdir(parents=True) + + extension_dir = project_dir / ".specify" / "extensions" / "fakeext" + (extension_dir / "commands").mkdir(parents=True, exist_ok=True) + (extension_dir / "agents" / "control").mkdir(parents=True, exist_ok=True) + (extension_dir / "agents" / "control" / "commander.md").write_text("# Commander\n") + (extension_dir / "commands" / "cmd.md").write_text( + "---\ndescription: Extension fakeext cmd\n---\n\n" + "Read agents/control/commander.md for context.\n" + ) + extension_manifest = { + "schema_version": "1.0", + "extension": { + "id": "fakeext", + "name": "Fake Extension", + "version": "1.0.0", + "description": "Test", + }, + "requires": {"speckit_version": ">=0.1.0"}, + "provides": { + "commands": [ + { + "name": "speckit.fakeext.cmd", + "file": "commands/cmd.md", + "description": "Fake extension command", + } + ] + }, + } + with open(extension_dir / "extension.yml", "w") as f: + yaml.dump(extension_manifest, f) + + preset_dir = temp_dir / "ext-cmd-append" + preset_dir.mkdir() + (preset_dir / "commands").mkdir() + (preset_dir / "commands" / "speckit.fakeext.cmd.md").write_text( + "---\ndescription: Append fakeext cmd\n---\n\n## Extra\n" + ) + preset_manifest = { + "schema_version": "1.0", + "preset": { + "id": "ext-cmd-append", + "name": "Ext Cmd Append", + "version": "1.0.0", + "description": "Test", + }, + "requires": {"speckit_version": ">=0.1.0"}, + "provides": { + "templates": [ + { + "type": "command", + "name": "speckit.fakeext.cmd", + "file": "commands/speckit.fakeext.cmd.md", + "strategy": "append", + } + ] + }, + } + with open(preset_dir / "preset.yml", "w") as f: + yaml.dump(preset_manifest, f) + + manager = PresetManager(project_dir) + manager.install_from_directory(preset_dir, "0.1.5") + + cmd_files = list(gemini_dir.glob("*fakeext*")) + assert cmd_files, "Command file should exist in gemini dir" + content = cmd_files[0].read_text() + assert ".specify/extensions/fakeext/agents/control/commander.md" in content + assert "Read agents/control" not in content + assert "## Extra" in content + def test_remove_restores_lower_priority_command( self, project_dir, temp_dir, valid_pack_data ): From 6664cf813cb943fd9ac0ab2aab60c11798913c13 Mon Sep 17 00:00:00 2001 From: NgoQuocViet2001 <123613986+NgoQuocViet2001@users.noreply.github.com> Date: Mon, 13 Jul 2026 22:57:26 +0700 Subject: [PATCH 17/45] fix: mark Kiro integration as multi-install safe (#3472) Assisted-by: Codex (model: GPT-5, autonomous) --- src/specify_cli/integrations/kiro_cli/__init__.py | 1 + tests/integrations/test_integration_kiro_cli.py | 3 +++ 2 files changed, 4 insertions(+) diff --git a/src/specify_cli/integrations/kiro_cli/__init__.py b/src/specify_cli/integrations/kiro_cli/__init__.py index 4c176e5127..0b8348af7b 100644 --- a/src/specify_cli/integrations/kiro_cli/__init__.py +++ b/src/specify_cli/integrations/kiro_cli/__init__.py @@ -13,6 +13,7 @@ class KiroCliIntegration(MarkdownIntegration): key = "kiro-cli" + multi_install_safe = True config = { "name": "Kiro CLI", "folder": ".kiro/", diff --git a/tests/integrations/test_integration_kiro_cli.py b/tests/integrations/test_integration_kiro_cli.py index 29adb0a4a6..a871e79a46 100644 --- a/tests/integrations/test_integration_kiro_cli.py +++ b/tests/integrations/test_integration_kiro_cli.py @@ -42,6 +42,9 @@ class TestKiroCliIntegration(MarkdownIntegrationTests): COMMANDS_SUBDIR = "prompts" REGISTRAR_DIR = ".kiro/prompts" + def test_declares_multi_install_safe(self): + assert get_integration(self.KEY).multi_install_safe is True + def test_registrar_config(self): """Override base assertion: kiro-cli uses a prose fallback for args because Kiro CLI file-based prompts do not natively substitute From e649bbdc44c272aa08f6d5db9e2dbf30a035879c Mon Sep 17 00:00:00 2001 From: Yoshiyuki Kinjo Date: Tue, 14 Jul 2026 02:59:31 +0900 Subject: [PATCH 18/45] Cleanup agent-file-template.md (#2579) * follow fc3d1244c07c612e146cfad65d9e36542ee1417 agent-file-template.md is removed at fc3d1244c07c612e146cfad65d9e36542ee1417 * Fix ruled line for constitution-template.md * fix test --- presets/ARCHITECTURE.md | 3 +-- presets/self-test/preset.yml | 6 ------ presets/self-test/templates/agent-file-template.md | 9 --------- tests/test_presets.py | 2 +- 4 files changed, 2 insertions(+), 18 deletions(-) delete mode 100644 presets/self-test/templates/agent-file-template.md diff --git a/presets/ARCHITECTURE.md b/presets/ARCHITECTURE.md index 85e9dea3c7..c533976b8a 100644 --- a/presets/ARCHITECTURE.md +++ b/presets/ARCHITECTURE.md @@ -158,8 +158,7 @@ presets/ ├── plan-template.md ├── tasks-template.md ├── checklist-template.md - ├── constitution-template.md - └── agent-file-template.md + └── constitution-template.md ``` ## Module Structure diff --git a/presets/self-test/preset.yml b/presets/self-test/preset.yml index 8e718430aa..7a9db93b0a 100644 --- a/presets/self-test/preset.yml +++ b/presets/self-test/preset.yml @@ -44,12 +44,6 @@ provides: description: "Self-test constitution template" replaces: "constitution-template" - - type: "template" - name: "agent-file-template" - file: "templates/agent-file-template.md" - description: "Self-test agent file template" - replaces: "agent-file-template" - - type: "command" name: "speckit.specify" file: "commands/speckit.specify.md" diff --git a/presets/self-test/templates/agent-file-template.md b/presets/self-test/templates/agent-file-template.md deleted file mode 100644 index 7b9267bade..0000000000 --- a/presets/self-test/templates/agent-file-template.md +++ /dev/null @@ -1,9 +0,0 @@ -# Agent File (Self-Test Preset) - - - -> This template is provided by the self-test preset. - -## Agent Instructions - -Follow these guidelines when working on this project. diff --git a/tests/test_presets.py b/tests/test_presets.py index 20abab39cf..b762c62c57 100644 --- a/tests/test_presets.py +++ b/tests/test_presets.py @@ -2620,7 +2620,7 @@ def test_self_test_manifest_valid(self): assert manifest.id == "self-test" assert manifest.name == "Self-Test Preset" assert manifest.version == "1.0.0" - assert len(manifest.templates) == 8 # 6 templates + 2 commands + assert len(manifest.templates) == 7 # 5 templates + 2 commands def test_self_test_provides_all_core_templates(self): """Verify the self-test preset provides an override for every core template.""" From e590cd80072205160c224a5c9580065225878c68 Mon Sep 17 00:00:00 2001 From: Noor ul ain Date: Mon, 13 Jul 2026 23:02:04 +0500 Subject: [PATCH 19/45] fix(workflows): fail switch step on non-mapping cases instead of crashing (#3481) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `SwitchStep.validate()` already rejects a non-mapping `cases`, but the engine's `execute()` path does not auto-validate (see `WorkflowEngine.load_workflow`, whose docstring notes the definition is "not yet validated"). On an unvalidated run, `execute` called `cases.items()` on the raw value, so a list or scalar `cases` authoring mistake raised `AttributeError` and took down the whole run — the engine invokes `step_impl.execute()` with no surrounding try/except. Guard `execute` to return a FAILED StepResult naming the type error instead, mirroring the fan-out step's non-list `items` handling. The expression is still evaluated first, so its value is surfaced in the step output for downstream context. Co-authored-by: Claude Opus 4.8 (1M context) --- .../workflows/steps/switch/__init__.py | 14 +++++++++ tests/test_workflows.py | 29 +++++++++++++++++++ 2 files changed, 43 insertions(+) diff --git a/src/specify_cli/workflows/steps/switch/__init__.py b/src/specify_cli/workflows/steps/switch/__init__.py index e58d3c23c3..a63b283432 100644 --- a/src/specify_cli/workflows/steps/switch/__init__.py +++ b/src/specify_cli/workflows/steps/switch/__init__.py @@ -26,6 +26,20 @@ def execute(self, config: dict[str, Any], context: StepContext) -> StepResult: str_value = str(value) if value is not None else "" cases = config.get("cases", {}) + if not isinstance(cases, dict): + # The engine does not auto-validate step config, so an unvalidated + # run with a non-mapping ``cases`` (a list/scalar authoring mistake) + # would otherwise raise AttributeError from ``.items()`` below and + # crash the whole run. Fail this step loudly instead, mirroring the + # fan-out step's non-list ``items`` handling. + return StepResult( + status=StepStatus.FAILED, + error=( + f"Switch step {config.get('id', '?')!r}: 'cases' must be a " + f"mapping, got {type(cases).__name__}." + ), + output={"matched_case": None, "expression_value": value}, + ) for case_key, case_steps in cases.items(): if str(case_key) == str_value: return StepResult( diff --git a/tests/test_workflows.py b/tests/test_workflows.py index 11ff138074..4ec99f3994 100644 --- a/tests/test_workflows.py +++ b/tests/test_workflows.py @@ -2143,6 +2143,35 @@ def test_execute_no_default_no_match(self): assert result.output["matched_case"] == "__default__" assert result.next_steps == [] + def test_execute_non_dict_cases_fails_loudly(self): + """A non-mapping ``cases`` must fail the step, not crash the run. + + ``validate`` rejects a non-dict ``cases``, but the engine's + ``execute()`` does not auto-validate (see ``WorkflowEngine.load_workflow`` + docstring). Before the guard, ``execute`` called ``cases.items()`` on the + raw value, so an unvalidated run with a list/scalar ``cases`` raised + AttributeError and took down the whole run instead of failing this step. + Mirrors the fan-out step's non-list ``items`` handling. + """ + from specify_cli.workflows.steps.switch import SwitchStep + from specify_cli.workflows.base import StepContext, StepStatus + + step = SwitchStep() + ctx = StepContext(steps={"review": {"output": {"choice": "approve"}}}) + for bad_cases in (["approve"], "approve", 5): + result = step.execute( + { + "id": "route", + "expression": "{{ steps.review.output.choice }}", + "cases": bad_cases, + }, + ctx, + ) + assert result.status == StepStatus.FAILED + assert "'cases' must be a mapping" in (result.error or "") + # expression is still evaluated, so its value is surfaced for context. + assert result.output["expression_value"] == "approve" + def test_validate_missing_expression(self): from specify_cli.workflows.steps.switch import SwitchStep From 8cb0889f4a88dc1ad47a370f393b69ba49d7dc04 Mon Sep 17 00:00:00 2001 From: Manfred Riem <15701806+mnriem@users.noreply.github.com> Date: Mon, 13 Jul 2026 13:08:42 -0500 Subject: [PATCH 20/45] chore: release 0.12.13, begin 0.12.14.dev0 development (#3498) * chore: bump version to 0.12.13 * chore: begin 0.12.14.dev0 development --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> --- CHANGELOG.md | 15 +++++++++++++++ pyproject.toml | 2 +- 2 files changed, 16 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 0485ee07ef..62ad808d89 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,21 @@ +## [0.12.13] - 2026-07-13 + +### Changed + +- fix(workflows): fail switch step on non-mapping cases instead of crashing (#3481) +- Cleanup agent-file-template.md (#2579) +- fix: mark Kiro integration as multi-install safe (#3472) +- fix: rewrite extension-relative subdir paths in generated command bodies (#3444) +- fix(templates): point constitution sync checklist at installed command files (#3418) +- feat(workflows): make shell step timeout configurable (#3327) (#3328) +- docs: clarify that release tags keep the leading v prefix (#3463) +- fix(workflows): don't crash on membership test against a non-iterable (#3448) +- fix(workflows): if-step validate accepts falsy non-list else (#3264) +- chore: release 0.12.12, begin 0.12.13.dev0 development (#3490) + ## [0.12.12] - 2026-07-13 ### Changed diff --git a/pyproject.toml b/pyproject.toml index bade423da9..50082664c4 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,6 +1,6 @@ [project] name = "specify-cli" -version = "0.12.13.dev0" +version = "0.12.14.dev0" description = "Specify CLI, part of GitHub Spec Kit. A tool to bootstrap your projects for Spec-Driven Development (SDD)." readme = "README.md" requires-python = ">=3.11" From a965413a24f127ba0bde027008b1ac6606237a41 Mon Sep 17 00:00:00 2001 From: Noor ul ain Date: Mon, 13 Jul 2026 23:30:09 +0500 Subject: [PATCH 21/45] fix(workflows): fail fan-in step on non-list wait_for instead of crashing (#3482) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `FanInStep.validate()` and the engine's fan-in checks both reject a non-list `wait_for`, but the engine's `execute()` path does not auto-validate (see `WorkflowEngine.load_workflow`, whose docstring notes the definition is "not yet validated"). On an unvalidated run, `execute` iterated the raw value with `for step_id in wait_for`, with two bad outcomes: * a scalar (`wait_for: 5`, `wait_for: null`) raised `TypeError` and took down the whole run — the engine invokes `step_impl.execute()` with no surrounding try/except; and * a string (`wait_for: stepA`) silently iterated its characters and returned a join of empty results with a COMPLETED status — the exact "silent empty result + COMPLETED" wiring bug the engine's own fan-in validation comment warns against. Guard `execute` to return a FAILED StepResult naming the type error instead, mirroring the fan-out step's non-list `items` handling. A missing `wait_for` key still defaults to an empty list (COMPLETED), unchanged; the guard fires only on an explicit non-list value. Co-authored-by: Claude Opus 4.8 (1M context) --- .../workflows/steps/fan_in/__init__.py | 18 +++++++++++++++ tests/test_workflows.py | 23 +++++++++++++++++++ 2 files changed, 41 insertions(+) diff --git a/src/specify_cli/workflows/steps/fan_in/__init__.py b/src/specify_cli/workflows/steps/fan_in/__init__.py index 7b82df50b3..1e466e5fa8 100644 --- a/src/specify_cli/workflows/steps/fan_in/__init__.py +++ b/src/specify_cli/workflows/steps/fan_in/__init__.py @@ -24,6 +24,24 @@ def execute(self, config: dict[str, Any], context: StepContext) -> StepResult: if not isinstance(output_config, dict): output_config = {} + # The engine does not auto-validate step config, so an unvalidated run + # with a non-list ``wait_for`` reaches here raw. Iterating it then + # either crashes the whole run (a scalar like an int or None raises + # TypeError) or, worse, silently iterates a string's characters and + # yields a bogus join of empty results with a COMPLETED status — the + # exact "silent empty result + COMPLETED" wiring bug the engine's + # fan-in validation guards against. Fail this step loudly instead, + # mirroring the fan-out step's non-list ``items`` handling. + if not isinstance(wait_for, list): + return StepResult( + status=StepStatus.FAILED, + error=( + f"Fan-in step {config.get('id', '?')!r}: 'wait_for' must be " + f"a list of step IDs, got {type(wait_for).__name__}." + ), + output={"results": []}, + ) + # Collect results from referenced steps results = [] for step_id in wait_for: diff --git a/tests/test_workflows.py b/tests/test_workflows.py index 4ec99f3994..dcd7d19b84 100644 --- a/tests/test_workflows.py +++ b/tests/test_workflows.py @@ -2475,6 +2475,29 @@ def test_execute_missing_wait_for_step(self): result = step.execute(config, ctx) assert result.output["results"] == [{}] + @pytest.mark.parametrize("bad_wait_for", ["stepA", 5, None, {"a": 1}]) + def test_execute_non_list_wait_for_fails_loudly(self, bad_wait_for): + """A non-list ``wait_for`` must fail the step, not crash the run or + silently produce a bogus join. + + ``validate`` rejects a non-list ``wait_for``, but the engine's + ``execute()`` does not auto-validate. Before the guard, ``execute`` + iterated the raw value: a scalar (int/None) raised TypeError and took + down the whole run, while a string silently iterated its characters and + returned a join of empty results with a COMPLETED status — the exact + "silent empty result + COMPLETED" wiring bug the engine's fan-in + validation warns against. Mirrors the fan-out non-list ``items`` guard. + """ + from specify_cli.workflows.steps.fan_in import FanInStep + from specify_cli.workflows.base import StepContext, StepStatus + + step = FanInStep() + ctx = StepContext(steps={"a": {"output": {"x": 1}}}) + result = step.execute({"id": "collect", "wait_for": bad_wait_for}, ctx) + assert result.status == StepStatus.FAILED + assert "'wait_for' must be a list" in (result.error or "") + assert result.output["results"] == [] + def test_validate_empty_wait_for(self): from specify_cli.workflows.steps.fan_in import FanInStep From 0acb5c6461e45f5f7586eece76d0ba5dd5fdffc8 Mon Sep 17 00:00:00 2001 From: Noor ul ain Date: Tue, 14 Jul 2026 00:13:44 +0500 Subject: [PATCH 22/45] fix(integrations): declare kiro-cli multi-install safe (#3471) (#3485) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit kiro-cli confines all of its managed files to an isolated agent root (`.kiro/`, with commands in `.kiro/prompts`) that no other integration writes to, so it meets every documented criterion for multi-install safety — but `KiroCliIntegration` never set `multi_install_safe = True`. As a result, co-installing kiro-cli alongside any other integration left `specify integration status` permanently in ERROR: error unsafe-multi-install: Installed integrations are not all declared multi-install safe: kiro-cli `--force` bypasses the install-time gate but does not clear the status error, and there is no flag or config to acknowledge it, so the error is permanent while both integrations remain installed. Set `multi_install_safe = True`. The registry's parametrized multi-install-safe contract tests (static isolated root, distinct agent roots / command dirs, disjoint manifests) now cover kiro-cli automatically, and a focused regression test pins the declaration so a future edit cannot silently drop it and reintroduce the error. Co-authored-by: Claude Opus 4.8 (1M context) --- src/specify_cli/integrations/kiro_cli/__init__.py | 7 +++++++ tests/integrations/test_registry.py | 14 ++++++++++++++ 2 files changed, 21 insertions(+) diff --git a/src/specify_cli/integrations/kiro_cli/__init__.py b/src/specify_cli/integrations/kiro_cli/__init__.py index 0b8348af7b..37f743d5c2 100644 --- a/src/specify_cli/integrations/kiro_cli/__init__.py +++ b/src/specify_cli/integrations/kiro_cli/__init__.py @@ -27,3 +27,10 @@ class KiroCliIntegration(MarkdownIntegration): "args": _KIRO_ARG_FALLBACK, "extension": ".md", } + + # Kiro CLI keeps everything under a static, isolated agent root + # (``.kiro/`` with commands in ``.kiro/prompts``) that no other + # integration writes to, so it is safe to install alongside others + # (issue #3471). The registry's multi-install-safe contract tests + # enforce that isolation for every integration setting this flag. + multi_install_safe = True diff --git a/tests/integrations/test_registry.py b/tests/integrations/test_registry.py index d3049b4265..13d4968927 100644 --- a/tests/integrations/test_registry.py +++ b/tests/integrations/test_registry.py @@ -272,6 +272,20 @@ def test_safe_integrations_have_disjoint_manifests( f"these files: {sorted(overlap)}" ) + def test_kiro_cli_is_declared_multi_install_safe(self): + """kiro-cli confines itself to an isolated ``.kiro/`` root that no + other integration touches, so it must be declared multi-install safe + (issue #3471). + + Before the fix, co-installing kiro-cli alongside another integration + left ``specify integration status`` permanently in ERROR + (``unsafe-multi-install``) with no way to acknowledge it. The + parametrized isolation/manifest contracts above already exercise + kiro-cli once the flag is set; this pins the declaration itself so a + future edit cannot silently drop it and reintroduce the error. + """ + assert INTEGRATION_REGISTRY["kiro-cli"].multi_install_safe is True + class TestCatalogParity: """The discovery catalog must list every registered integration.""" From c05a626cbc485c315e0b395035e316d3bc8987f9 Mon Sep 17 00:00:00 2001 From: Noor ul ain Date: Tue, 14 Jul 2026 00:33:02 +0500 Subject: [PATCH 23/45] fix(integrations): exit cleanly on unbalanced quote in --integration-options (#3457) (#3466) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * fix(integrations): exit cleanly on unbalanced quote in --integration-options (#3457) `_parse_integration_options` called `shlex.split(raw_options)` unguarded, so an unbalanced quote in the flag value (e.g. `--integration-options='--commands-dir "foo'`) made shlex raise `ValueError: No closing quotation` and a raw traceback escaped — unlike every other bad-input path in this function (unknown option, missing value, unexpected value), which print a message and exit 1. Reachable from `specify init --integration-options=...` and every `specify integration install/switch/upgrade/migrate --integration-options=...`. Wrap the split in a try/except ValueError that prints a one-line error and raises `typer.Exit(1)`, matching the existing loud-fail UX. Add a test asserting the unbalanced-quote input raises `typer.Exit` with exit code 1. Co-Authored-By: Claude Opus 4.8 (1M context) * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> --- src/specify_cli/integrations/_helpers.py | 10 ++++++++- .../test_integration_subcommand.py | 21 +++++++++++++++++++ 2 files changed, 30 insertions(+), 1 deletion(-) diff --git a/src/specify_cli/integrations/_helpers.py b/src/specify_cli/integrations/_helpers.py index d1bf051f77..ce273f9d72 100644 --- a/src/specify_cli/integrations/_helpers.py +++ b/src/specify_cli/integrations/_helpers.py @@ -190,7 +190,15 @@ def _parse_integration_options(integration: Any, raw_options: str) -> dict[str, """ import shlex parsed: dict[str, Any] = {} - tokens = shlex.split(raw_options) + try: + tokens = shlex.split(raw_options) + except ValueError as exc: + # An unbalanced quote (e.g. --integration-options='--commands-dir "foo') + # makes shlex raise "No closing quotation". Translate it into the same + # clean exit-1 UX as every other bad-input path below rather than + # letting a raw traceback escape. + console.print(f"[red]Error:[/red] Could not parse integration options: {exc}.") + raise typer.Exit(1) declared_options = list(integration.options()) declared = {opt.name.lstrip("-"): opt for opt in declared_options} allowed = ", ".join(sorted(opt.name for opt in declared_options)) diff --git a/tests/integrations/test_integration_subcommand.py b/tests/integrations/test_integration_subcommand.py index 56f338cc82..7200572659 100644 --- a/tests/integrations/test_integration_subcommand.py +++ b/tests/integrations/test_integration_subcommand.py @@ -2675,6 +2675,27 @@ def test_equals_form_parsed(self): assert result_space["commands_dir"] == "./mydir" assert result_equals["commands_dir"] == "./mydir" + def test_unbalanced_quote_exits_cleanly(self, capsys): + """An unbalanced quote must exit(1) with a message, not a raw ValueError. + + shlex.split() raises ValueError("No closing quotation") on an unbalanced + quote; the parser must translate that into the same clean typer.Exit(1) + UX as unknown-option / missing-value, rather than letting the traceback + escape (issue #3457). + """ + import typer + + from specify_cli.integrations._commands import _parse_integration_options + from specify_cli.integrations import get_integration + + integration = get_integration("generic") + assert integration is not None + + with pytest.raises(typer.Exit) as excinfo: + _parse_integration_options(integration, '--commands-dir "foo') + assert excinfo.value.exit_code == 1 + assert "Error: Could not parse integration options: No closing quotation." in capsys.readouterr().out + class TestUninstallNoManifestClearsInitOptions: def test_init_options_cleared_on_no_manifest_uninstall(self, tmp_path): From 801ff888ff09590c9d4346b0cd12d6b37730679e Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Mon, 13 Jul 2026 15:16:45 -0500 Subject: [PATCH 24/45] [extension] Add Quality Gates (Enforcement Layer) extension to community catalog (#3431) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Add Quality Gates (Enforcement Layer) extension to community catalog Add gates extension submitted by @schwichtgit to: - extensions/catalog.community.json (alphabetical order, between fx-to-dotnet and github-issues) - docs/community/extensions.md community extensions table Closes #3414 Assisted-by: GitHub Copilot (model: claude-sonnet-4.6, autonomous) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: revert unrelated catalog reformatting and remove empty changelog field from gates entry - Restore original ordering/formatting of aide, checkpoint, critique, threatmodel entries and inline requires.tools objects that were inadvertently reordered in the previous commit - Remove `"changelog": ""` from the gates entry (empty URL is inconsistent with catalog conventions; field should be omitted when no changelog URL exists) Addresses review comments: - github/spec-kit#3431 (comment) — unrelated reformatting/reordering - github/spec-kit#3431 (comment) — empty changelog field Assisted-by: GitHub Copilot (model: claude-sonnet-4.5, autonomous) * Fix gates entry tool requirements: git required, add node and shellcheck optional - Mark git as required (per v0.1.0 README: \"jq and git — the hooks and verify.sh require them\" and release notes: \"Requires Spec Kit >=0.12.0, jq, and git\") - Add node as optional tool (per issue #3414 submission) - Add shellcheck as optional tool (per issue #3414 submission) - Update gates entry updated_at and top-level updated_at to 2026-07-13 Assisted-by: GitHub Copilot (model: claude-sonnet-4.5, autonomous)" --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> --- docs/community/extensions.md | 1 + extensions/catalog.community.json | 53 ++++++++++++++++++++++++++++++- 2 files changed, 53 insertions(+), 1 deletion(-) diff --git a/docs/community/extensions.md b/docs/community/extensions.md index fd34673951..db7dc6dc32 100644 --- a/docs/community/extensions.md +++ b/docs/community/extensions.md @@ -98,6 +98,7 @@ The following community-contributed extensions are available in [`catalog.commun | Project Health Check | Diagnose a Spec Kit project and report health issues across structure, agents, features, scripts, extensions, and git | `visibility` | Read-only | [spec-kit-doctor](https://github.com/KhawarHabibKhan/spec-kit-doctor) | | Project Status | Show current SDD workflow progress — active feature, artifact status, task completion, workflow phase, and extensions summary | `visibility` | Read-only | [spec-kit-status](https://github.com/KhawarHabibKhan/spec-kit-status) | | QA Testing Extension | Systematic QA testing with browser-driven or CLI-based validation of acceptance criteria from spec | `code` | Read-only | [spec-kit-qa](https://github.com/arunt14/spec-kit-qa) | +| Quality Gates (Enforcement Layer) | Deterministic quality enforcement for Spec Kit projects at three boundaries — agent hooks, git pre-commit, CI — with one policy file and provable enforcement (attestations, canaries, verified parity). | `process` | Read+Write | [spec-gates](https://github.com/schwichtgit/spec-gates) | | RAG Azure Builder | Spec Kit extension for onboarding and operating an Azure RAG stack with guided workflows. | `process` | Read+Write | [spec-kit-extension-rag-azure-builder](https://github.com/Sertxito/spec-kit-extension-rag-azure-builder) | | Ralph Loop | Autonomous implementation loop using AI agent CLI | `code` | Read+Write | [spec-kit-ralph](https://github.com/Rubiss-Projects/spec-kit-ralph) | | Reconcile Extension | Reconcile implementation drift by surgically updating feature artifacts. | `docs` | Read+Write | [spec-kit-reconcile](https://github.com/stn1slv/spec-kit-reconcile) | diff --git a/extensions/catalog.community.json b/extensions/catalog.community.json index 67e1ade49f..78df8c0979 100644 --- a/extensions/catalog.community.json +++ b/extensions/catalog.community.json @@ -1,6 +1,6 @@ { "schema_version": "1.0", - "updated_at": "2026-07-10T00:00:00Z", + "updated_at": "2026-07-13T00:00:00Z", "catalog_url": "https://raw.githubusercontent.com/github/spec-kit/main/extensions/catalog.community.json", "extensions": { "aide": { @@ -1427,6 +1427,57 @@ "created_at": "2026-05-06T00:00:00Z", "updated_at": "2026-05-06T00:00:00Z" }, + "gates": { + "name": "Quality Gates (Enforcement Layer)", + "id": "gates", + "description": "Deterministic quality enforcement for Spec Kit projects at three boundaries — agent hooks, git pre-commit, CI — with one policy file and provable enforcement (attestations, canaries, verified parity).", + "author": "schwichtgit", + "version": "0.1.0", + "download_url": "https://github.com/schwichtgit/spec-gates/releases/download/v0.1.0/gates-0.1.0.zip", + "repository": "https://github.com/schwichtgit/spec-gates", + "homepage": "https://github.com/schwichtgit/spec-gates", + "documentation": "https://github.com/schwichtgit/spec-gates/blob/main/docs/how-it-works.md", + "license": "MIT", + "category": "process", + "effect": "read-write", + "requires": { + "speckit_version": ">=0.12.0", + "tools": [ + { + "name": "jq", + "required": true + }, + { + "name": "git", + "required": true + }, + { + "name": "node", + "required": false + }, + { + "name": "shellcheck", + "required": false + } + ] + }, + "provides": { + "commands": 5, + "hooks": 1 + }, + "tags": [ + "quality", + "enforcement", + "hooks", + "ci", + "governance" + ], + "verified": false, + "downloads": 0, + "stars": 0, + "created_at": "2026-07-09T00:00:00Z", + "updated_at": "2026-07-13T00:00:00Z" + }, "github-issues": { "name": "GitHub Issues Integration 1", "id": "github-issues", From 993083405ee84ea4d424e428d364a77fde84d8f6 Mon Sep 17 00:00:00 2001 From: Ali jawwad <33836051+jawwad-ali@users.noreply.github.com> Date: Tue, 14 Jul 2026 01:29:15 +0500 Subject: [PATCH 25/45] fix(init): don't block on confirmation for 'init --here' without a TTY (#3236) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * fix(init): don't block on confirmation for 'init --here' without a TTY When 'specify init --here' targets a non-empty directory without --force, it called typer.confirm() unconditionally. In a non-interactive session (no TTY -- CI, piped, agent) there is no input, so the prompt reads EOF and aborts unhelpfully (or blocks), with no actionable message. The named-project path already fails fast and points to --force; --here was the inconsistent outlier. Guard the confirmation with the existing _stdin_is_interactive() helper: when non-interactive, print a clear 'directory not empty; re-run with --force' error and exit 1 instead of prompting. Co-Authored-By: Claude Opus 4.8 (1M context) * fix(init): honor piped confirmation for 'init --here'; only fail-fast on empty stdin The first version of this fix short-circuited on '_stdin_is_interactive()' (isatty) before typer.confirm, which broke 'init --here' when confirmation is piped (e.g. 'echo y | specify init --here' / CliRunner input='y\n') -- a non-TTY pipe with valid input was wrongly rejected, regressing test_init_here_without_force_preserves_shared_infra. Instead, call typer.confirm normally (piped 'y'/'n' is honored) and catch the Abort/EOFError it raises only when stdin is empty, converting that to the actionable '--force' guidance. This keeps the UX win for the no-input case without rejecting piped input. Co-Authored-By: Claude Opus 4.8 (1M context) * fix(init): distinguish interactive cancel from no-input; defer merge warning Address Copilot review on the --here non-empty path: (1) treat typer.Abort during an interactive confirm (e.g. Ctrl+C) as a normal cancellation (exit 0), and only emit the '--force' guidance + exit 1 when there is no TTY (empty stdin / EOF) -- no longer conflating the two; (2) move the 'will be merged / may overwrite' warning so it only shows when actually proceeding (force) or folded into the confirmation prompt, not on the fail-fast path where nothing is merged. Piped confirmation (e.g. 'echo y | specify init --here') is still honored, which is why the prompt is attempted rather than refused outright when non-interactive -- the existing test_init_here_without_force_preserves_shared_infra pipes 'y' and must succeed. Co-Authored-By: Claude Opus 4.8 (1M context) * fix(init): fail fast on non-interactive --here instead of prompting Per Copilot review: do not call typer.confirm when stdin is not a TTY -- an open-but-idle non-TTY stdin (CI/agent) could block on the prompt. When the directory is non-empty and --force is not given, fail fast with '--force' guidance unless an interactive terminal is present. Interactive confirm still offers the merge-but-preserve path (distinct from --force, which overwrites); a Ctrl+C there is treated as a normal cancellation (exit 0). The merge/overwrite warning is only printed when actually proceeding, not on the fail-fast path. Updated the preserve-merge E2E test to simulate an interactive terminal so it exercises the confirm path (non-interactive sessions now require --force). Co-Authored-By: Claude Opus 4.8 (1M context) * fix(init): honor piped y/n for 'init --here', error only on no-input Per maintainer review: restore the second-revision shape. Calling typer.confirm normally keeps 'echo y | specify init --here' reaching the non-destructive preserve-merge path (and piped 'n' cancels with exit 0). Only when no confirmation input is available at all (closed/empty stdin -> typer.Abort/EOFError) is it converted into the actionable error that points at --force. This drops the _stdin_is_interactive fail-fast that broke the common piped-confirm idiom and made preserve-merge interactive-only. The preserve test no longer needs to monkeypatch _stdin_is_interactive - it passes on the real contract. Co-Authored-By: Claude Opus 4.8 * fix(init): preserve interactive-cancel semantics; fold merge risk into the prompt Two review-driven refinements to the 'init --here' non-empty confirm, keeping the maintainer-endorsed control flow (piped y/n honored; non-interactive EOF → actionable --force error): 1. typer.confirm raises typer.Abort for BOTH an interactive Ctrl+C and an EOF on closed/empty stdin. Catching it unconditionally reported 'no confirmation input available, use --force' and exited 1 even when the user cancelled at a real TTY. Branch on _stdin_is_interactive(): a TTY cancel is a normal exit 0 ('Operation cancelled'); only non-interactive EOF becomes the --force error. 2. Fold the merge-risk warning into the confirmation question instead of printing it unconditionally beforehand, so the EOF/no-input path (which exits without changing anything) no longer prints a misleading 'will be merged' line first. Adds test_init_here_interactive_cancel_exits_zero (fails before: exit 1 with --force; passes after: exit 0, 'cancelled', pre-existing file untouched). The non-interactive EOF and piped-y preserve-merge tests are unchanged and still pass. Co-Authored-By: Claude Opus 4.8 (1M context) --------- Co-authored-by: Claude Opus 4.8 (1M context) --- src/specify_cli/commands/init.py | 39 ++++++++++++++++++--- tests/integrations/test_cli.py | 60 +++++++++++++++++++++++++++++++- 2 files changed, 93 insertions(+), 6 deletions(-) diff --git a/src/specify_cli/commands/init.py b/src/specify_cli/commands/init.py index dd815b8c5d..9eb8302888 100644 --- a/src/specify_cli/commands/init.py +++ b/src/specify_cli/commands/init.py @@ -220,16 +220,45 @@ def init( console.print( f"[yellow]Warning:[/yellow] Current directory is not empty ({len(existing_items)} items)" ) - console.print( - "[yellow]Template files will be merged with existing content and may overwrite existing files[/yellow]" - ) if force: + # Proceeding: the merge/overwrite warning is accurate here. + console.print( + "[yellow]Template files will be merged with existing content and may overwrite existing files[/yellow]" + ) console.print( "[cyan]--force supplied: skipping confirmation and proceeding with merge[/cyan]" ) else: - response = typer.confirm("Do you want to continue?") - if not response: + # Fold the merge risk into the confirmation prompt rather than + # printing it unconditionally first: on the EOF/no-input path + # below the command exits without changing anything, so a + # standalone "will be merged" line would mislead. Interactive + # users still see the risk as part of the question. + # + # Call typer.confirm normally so piped y/n is honored — e.g. + # `echo y | specify init --here` keeps reaching the + # non-destructive preserve-merge path. + try: + proceed = typer.confirm( + "Template files will be merged with existing content " + "and may overwrite existing files. Do you want to continue?" + ) + except (typer.Abort, EOFError): + # typer.confirm raises Abort for BOTH an interactive Ctrl+C + # and an EOF on closed/empty stdin. Distinguish them: a real + # TTY cancellation is a normal exit (0, "cancelled"), while a + # missing-input EOF (non-interactive) becomes an actionable + # error pointing at --force. + if _stdin_is_interactive(): + console.print("[yellow]Operation cancelled[/yellow]") + raise typer.Exit(0) from None + console.print( + "[red]Error:[/red] Current directory is not empty and no " + "confirmation input is available. Re-run with " + "[bold]--force[/bold] to merge into it." + ) + raise typer.Exit(1) from None + if not proceed: console.print("[yellow]Operation cancelled[/yellow]") raise typer.Exit(0) else: diff --git a/tests/integrations/test_cli.py b/tests/integrations/test_cli.py index cb2d136e6f..7b6461b4da 100644 --- a/tests/integrations/test_cli.py +++ b/tests/integrations/test_cli.py @@ -115,6 +115,63 @@ def fail_select(*_args, **_kwargs): data = json.loads((project / ".specify" / "integration.json").read_text(encoding="utf-8")) assert data["integration"] == specify_cli.DEFAULT_INIT_INTEGRATION + def test_init_here_nonempty_noninteractive_errors_with_force_guidance(self, tmp_path): + """`init --here` on a non-empty directory with no confirmation input (empty + stdin) must fail fast with guidance to use --force, instead of the bare + 'Aborted.' from an EOF on typer.confirm. CliRunner with no `input=` provides + empty stdin, so typer.confirm raises Abort, which the command converts to the + actionable error.""" + from typer.testing import CliRunner + from specify_cli import app + + project = tmp_path / "nonempty-here" + project.mkdir() + (project / "existing.txt").write_text("keep me", encoding="utf-8") + old_cwd = os.getcwd() + try: + os.chdir(project) + result = CliRunner().invoke(app, [ + "init", "--here", "--integration", "copilot", "--script", "sh", "--ignore-agent-tools", + ], catch_exceptions=False) + finally: + os.chdir(old_cwd) + + assert result.exit_code == 1, result.output + assert "--force" in result.output + # Aborted before scaffolding: the pre-existing file is untouched. + assert (project / "existing.txt").read_text(encoding="utf-8") == "keep me" + + def test_init_here_interactive_cancel_exits_zero(self, tmp_path, monkeypatch): + """An interactive Ctrl+C at the merge confirmation (typer.Abort on a TTY) + is a normal cancellation — exit 0, "cancelled" — NOT the missing-input + --force error, which is reserved for non-interactive EOF. Guards the + regression where Abort was caught unconditionally and every cancel became + an exit-1 --force error.""" + from typer.testing import CliRunner + from specify_cli import app + import specify_cli.commands.init as init_mod + + # Simulate an interactive terminal so the Abort is treated as a cancel. + monkeypatch.setattr(init_mod, "_stdin_is_interactive", lambda: True) + + project = tmp_path / "cancel-here" + project.mkdir() + (project / "existing.txt").write_text("keep me", encoding="utf-8") + old_cwd = os.getcwd() + try: + os.chdir(project) + # No input → typer.confirm raises Abort (stands in for Ctrl+C). + result = CliRunner().invoke(app, [ + "init", "--here", "--integration", "copilot", "--script", "sh", "--ignore-agent-tools", + ], catch_exceptions=False) + finally: + os.chdir(old_cwd) + + assert result.exit_code == 0, result.output + assert "cancelled" in result.output.lower() + assert "--force" not in result.output # not the missing-input error + assert (project / "existing.txt").read_text(encoding="utf-8") == "keep me" + def test_integration_copilot_auto_promotes(self, tmp_path): from typer.testing import CliRunner from specify_cli import app @@ -835,7 +892,8 @@ def test_init_here_force_overwrites_shared_infra(self, tmp_path): assert (scripts_dir / "common.sh").read_text(encoding="utf-8") != custom_content def test_init_here_without_force_preserves_shared_infra(self, tmp_path): - """E2E: specify init --here (no --force) preserves existing shared infra files.""" + """E2E: confirming the merge with piped "y" (no --force) preserves + existing shared infra files (unlike --force, which overwrites them).""" from typer.testing import CliRunner from specify_cli import app From fc1a3fd76c8b657c2991b269af1b32cd8e68876e Mon Sep 17 00:00:00 2001 From: Ali jawwad <33836051+jawwad-ali@users.noreply.github.com> Date: Tue, 14 Jul 2026 01:40:36 +0500 Subject: [PATCH 26/45] fix(presets): resolve() honors manifest-declared file: for installed presets (#3351) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * fix(presets): resolve() honors manifest-declared file: for installed presets PresetResolver.resolve()'s tier-2 (installed presets) loop was convention-only: it looked for templates/.md and .md, ignoring a preset manifest that declares the template with an explicit, non-convention file: path. So resolve() returned the core template (and resolve_with_source() misattributed source='core') while collect_all_layers()/resolve_content() correctly used the preset's declared file — a divergence inside the same class. It could also return a stray convention-path file the manifest deliberately points away from. Mirror collect_all_layers()'s manifest-first logic: use the declared file: when present (skip convention fallback if it's missing, to avoid masking typos), and fall back to the convention walk only when the manifest is absent or doesn't list the template. Co-Authored-By: Claude Fable 5 * docs(presets): clarify the empty/falsey manifest-file branch comment Per review: 'file' is a required key for every template entry (PresetManifest._validate()), so the manifest-found branch is reached for an empty/falsey/non-usable 'file' value, not a truly absent one. Reword the comment to say so. Comment-only. Co-Authored-By: Claude Fable 5 * fix(presets): resolve() returns only real files; test missing-file skip Per review: - Use is_file() (not exists()) when honoring a manifest-declared file: so a manifest pointing at a directory is treated as missing rather than returned to a caller that will read_text() it. Applied in both resolve() and collect_all_layers() so the two stay consistent. - Add a regression test for the skip-convention-fallback-when-declared-file- missing behavior: manifest declares a missing custom/spec.md while the pack has a convention templates/spec-template.md; resolve() must skip the pack and fall through to core, not pick up the stray convention file. Co-Authored-By: Claude Fable 5 * fix(presets): resolve()/collect_all_layers() require a regular file for manifest file: A manifest-declared file: path is honored via exists(), which also accepts a directory. If a preset points file: at a directory, resolve() returned it and downstream read_text() crashes. Use is_file() in both resolve() and collect_all_layers() so a non-file (directory) is treated as missing and the convention fallback is skipped (pack yields to core), matching the existing missing-file behavior. Adds a directory-at-file: test (fails on exists(), passes on is_file()) that also asserts collect_all_layers() never returns the directory as a layer. Co-Authored-By: Claude Opus 4.8 (1M context) * refactor(presets): extract shared _manifest_declared_template for resolve()/collect_all_layers() Both methods reimplemented the manifest-entry lookup + authoritative-fallback rules independently — the exact duplication that let them diverge and caused the bug this PR fixes. Extract a single _manifest_declared_template(pack_dir, name, type) -> (entry, candidate) helper (candidate is the declared file only when it is_file(); a declared-but-unusable file returns (entry, None) so callers skip the convention fallback). resolve() and collect_all_layers() now both call it, so their manifest-first resolution cannot silently diverge again. Pure refactor, behavior-preserving: full test_presets.py (331) still passes, including the directory-at-file:, missing-file, and manifest-file-wins cases. Co-Authored-By: Claude Opus 4.8 (1M context) --------- Co-authored-by: Claude Fable 5 --- src/specify_cli/presets/__init__.py | 91 ++++++++++---- tests/test_presets.py | 180 ++++++++++++++++++++++++++++ 2 files changed, 248 insertions(+), 23 deletions(-) diff --git a/src/specify_cli/presets/__init__.py b/src/specify_cli/presets/__init__.py index 81e8eefc62..0f662a72ae 100644 --- a/src/specify_cli/presets/__init__.py +++ b/src/specify_cli/presets/__init__.py @@ -2588,6 +2588,39 @@ def _get_manifest(self, pack_dir: Path) -> Optional["PresetManifest"]: self._manifest_cache[key] = None return self._manifest_cache[key] + def _manifest_declared_template( + self, pack_dir: Path, template_name: str, template_type: str + ) -> tuple[dict | None, Path | None]: + """Resolve a preset's manifest-declared template entry and usable file. + + Returns ``(entry, candidate)``: + - ``entry`` is the matching ``provides.templates`` mapping, or ``None`` if + the manifest is absent or does not list this ``(name, type)``. + - ``candidate`` is the declared ``file:`` resolved under ``pack_dir`` IFF + it is a regular file (``is_file()``); ``None`` otherwise — a missing, + empty, or non-file (e.g. directory) declaration yields ``(entry, None)``. + + The manifest is authoritative: when it declares a template (``entry`` is + not ``None``) but the file is unusable (``candidate`` is ``None``), + callers must NOT fall back to the convention lookup — that would mask a + typo or pick up an undeclared file. Shared by ``resolve()`` and + ``collect_all_layers()`` so their manifest-first resolution cannot + silently diverge again (the divergence this fix addressed). + """ + manifest = self._get_manifest(pack_dir) + if not manifest: + return None, None + for tmpl in manifest.templates: + if tmpl.get("name") == template_name and tmpl.get("type") == template_type: + file_path = tmpl.get("file") + if file_path: + manifest_candidate = pack_dir / file_path + return tmpl, ( + manifest_candidate if manifest_candidate.is_file() else None + ) + return tmpl, None + return None, None + def _get_all_extensions_by_priority(self) -> list[tuple[int, str, dict | None]]: """Build unified list of registered and unregistered extensions sorted by priority. @@ -2690,6 +2723,27 @@ def resolve( registry = PresetRegistry(self.presets_dir) for pack_id, _metadata in registry.list_by_priority(): pack_dir = self.presets_dir / pack_id + # The preset manifest is authoritative: if it declares this + # template with an explicit ``file:``, resolve to that path — + # and do NOT fall back to convention when it's missing, to + # avoid masking typos or picking up an undeclared file. Only + # when the manifest is absent or doesn't list this template do + # we use the convention-based subdir lookup. Mirrors + # collect_all_layers()/resolve_content() so resolve() and + # resolve_with_source() agree with them instead of returning + # the core template (or a stray convention file). + entry, manifest_candidate = self._manifest_declared_template( + pack_dir, template_name, template_type + ) + if manifest_candidate is not None: + return manifest_candidate + if entry is not None: + # Manifest declares this template but the file is missing, + # non-file (e.g. a directory), or an empty/falsey ``file`` + # value. The manifest is authoritative, so skip this pack's + # convention fallback rather than mask a typo — mirrors + # collect_all_layers(). + continue for subdir in subdirs: if subdir: candidate = pack_dir / subdir / f"{template_name}{ext}" @@ -2957,31 +3011,22 @@ def _find_in_subdirs(base_dir: Path) -> Optional[Path]: pack_dir = self.presets_dir / pack_id # Read strategy and manifest file path from preset manifest strategy = "replace" - manifest_file_path = None manifest_has_strategy = False - manifest_found_entry = False - manifest = self._get_manifest(pack_dir) - if manifest: - for tmpl in manifest.templates: - if (tmpl.get("name") == template_name - and tmpl.get("type") == template_type): - strategy = tmpl.get("strategy", "replace") - manifest_has_strategy = "strategy" in tmpl - manifest_file_path = tmpl.get("file") - manifest_found_entry = True - break - # Use manifest file path if specified, otherwise convention-based - # lookup — but only when the manifest doesn't exist or doesn't - # list this template, so preset.yml stays authoritative. + entry, manifest_candidate = self._manifest_declared_template( + pack_dir, template_name, template_type + ) + if entry is not None: + strategy = entry.get("strategy", "replace") + manifest_has_strategy = "strategy" in entry + # Use the manifest's declared file when it's a usable regular file; + # only fall back to convention-based lookup when the manifest + # doesn't list this template at all, so preset.yml stays + # authoritative (a declared-but-unusable file skips convention — + # parity with resolve()). candidate = None - if manifest_file_path: - manifest_candidate = pack_dir / manifest_file_path - if manifest_candidate.exists(): - candidate = manifest_candidate - # Explicit file path that doesn't exist: skip convention - # fallback to avoid masking typos or picking up unintended files. - elif not manifest_found_entry: - # Manifest doesn't list this template — check convention paths + if manifest_candidate is not None: + candidate = manifest_candidate + elif entry is None: candidate = _find_in_subdirs(pack_dir) if candidate: # Legacy fallback: if manifest doesn't explicitly declare a diff --git a/tests/test_presets.py b/tests/test_presets.py index b762c62c57..12dbf90afb 100644 --- a/tests/test_presets.py +++ b/tests/test_presets.py @@ -884,6 +884,186 @@ def test_resolve_pack_takes_priority_over_core(self, project_dir, pack_dir): assert result is not None assert "Custom Spec Template" in result.read_text() + def _install_pack_with_manifest_file(self, project_dir, *, extra_file=False): + """Create a pack whose manifest declares a NON-convention file: path. + + Returns the pack dir under the project. The declared file lives at + custom/spec.md (not the convention templates/spec-template.md). + """ + presets_dir = project_dir / ".specify" / "presets" + pack_dir = presets_dir / "mypack" + (pack_dir / "custom").mkdir(parents=True) + (pack_dir / "custom" / "spec.md").write_text( + "# Manifest-declared Spec\n", encoding="utf-8" + ) + if extra_file: + # An undeclared convention-path file the manifest points away from. + (pack_dir / "templates").mkdir() + (pack_dir / "templates" / "spec-template.md").write_text( + "# Stray Convention Spec\n", encoding="utf-8" + ) + manifest = { + "schema_version": "1.0", + "preset": { + "id": "mypack", + "name": "My Pack", + "version": "1.0.0", + "description": "declares a non-convention file path", + }, + "requires": {"speckit_version": ">=0.1.0"}, + "provides": { + "templates": [ + { + "type": "template", + "name": "spec-template", + "file": "custom/spec.md", + "strategy": "replace", + } + ] + }, + } + with open(pack_dir / "preset.yml", "w") as f: + yaml.dump(manifest, f) + PresetRegistry(presets_dir).add( + "mypack", {"version": "1.0.0", "priority": 10} + ) + return pack_dir + + def test_resolve_uses_manifest_declared_file_path(self, project_dir): + """resolve() must honor a manifest-declared non-convention file: path. + + Previously the tier-2 loop was convention-only, so it returned the + core template and resolve_with_source() misattributed source='core', + diverging from collect_all_layers()/resolve_content(). + """ + pack_dir = self._install_pack_with_manifest_file(project_dir) + resolver = PresetResolver(project_dir) + + result = resolver.resolve("spec-template") + assert result == pack_dir / "custom" / "spec.md" + assert "Manifest-declared Spec" in result.read_text() + + sourced = resolver.resolve_with_source("spec-template") + assert sourced is not None + assert "mypack" in sourced["source"] + # resolve() must agree with collect_all_layers()'s top layer. + layers = resolver.collect_all_layers("spec-template") + assert Path(layers[0]["path"]) == pack_dir / "custom" / "spec.md" + + def test_resolve_manifest_file_wins_over_undeclared_convention_file( + self, project_dir + ): + """A stray convention-path file must not shadow the manifest's file:.""" + pack_dir = self._install_pack_with_manifest_file( + project_dir, extra_file=True + ) + resolver = PresetResolver(project_dir) + result = resolver.resolve("spec-template") + assert result == pack_dir / "custom" / "spec.md" + assert "Manifest-declared Spec" in result.read_text() + + def test_resolve_skips_convention_when_manifest_file_missing(self, project_dir): + """When the manifest declares a file: that does not exist, resolve() + must NOT fall back to a convention file in the same pack (that would + mask a typo) — it skips the pack and resolves core instead.""" + presets_dir = project_dir / ".specify" / "presets" + pack_dir = presets_dir / "mypack" + # Manifest declares custom/spec.md (MISSING); a convention file exists + # in the pack and must NOT be used. + (pack_dir / "templates").mkdir(parents=True) + (pack_dir / "templates" / "spec-template.md").write_text( + "# Stray Convention Spec\n", encoding="utf-8" + ) + manifest = { + "schema_version": "1.0", + "preset": { + "id": "mypack", + "name": "My Pack", + "version": "1.0.0", + "description": "declares a missing file path", + }, + "requires": {"speckit_version": ">=0.1.0"}, + "provides": { + "templates": [ + { + "type": "template", + "name": "spec-template", + "file": "custom/spec.md", + "strategy": "replace", + } + ] + }, + } + with open(pack_dir / "preset.yml", "w") as f: + yaml.dump(manifest, f) + PresetRegistry(presets_dir).add( + "mypack", {"version": "1.0.0", "priority": 10} + ) + + resolver = PresetResolver(project_dir) + result = resolver.resolve("spec-template") + assert result is not None + content = result.read_text() + assert "Stray Convention Spec" not in content # pack convention skipped + assert "Core Spec Template" in content # fell through to core + + def test_resolve_skips_convention_when_manifest_file_is_directory( + self, project_dir + ): + """When the manifest's file: path resolves to a DIRECTORY (not a regular + file), resolve()/collect_all_layers() must treat it as missing — exists() + would accept it and downstream read_text() on a directory would crash. + The pack is skipped (no convention fallback), so core wins.""" + presets_dir = project_dir / ".specify" / "presets" + pack_dir = presets_dir / "mypack" + # Declared file: custom/spec.md is created as a DIRECTORY. + (pack_dir / "custom" / "spec.md").mkdir(parents=True) + # A convention file also exists and must NOT be used. + (pack_dir / "templates").mkdir(parents=True) + (pack_dir / "templates" / "spec-template.md").write_text( + "# Stray Convention Spec\n", encoding="utf-8" + ) + manifest = { + "schema_version": "1.0", + "preset": { + "id": "mypack", + "name": "My Pack", + "version": "1.0.0", + "description": "declares a file: that is actually a directory", + }, + "requires": {"speckit_version": ">=0.1.0"}, + "provides": { + "templates": [ + { + "type": "template", + "name": "spec-template", + "file": "custom/spec.md", + "strategy": "replace", + } + ] + }, + } + with open(pack_dir / "preset.yml", "w") as f: + yaml.dump(manifest, f) + PresetRegistry(presets_dir).add( + "mypack", {"version": "1.0.0", "priority": 10} + ) + + resolver = PresetResolver(project_dir) + result = resolver.resolve("spec-template") + assert result is not None + assert result.is_file() # never a directory + content = result.read_text() + assert "Stray Convention Spec" not in content # pack convention skipped + assert "Core Spec Template" in content # fell through to core + # collect_all_layers() must agree: the directory is not a layer. + layers = resolver.collect_all_layers("spec-template") + assert all(Path(layer["path"]).is_file() for layer in layers) + assert all( + Path(layer["path"]) != pack_dir / "custom" / "spec.md" + for layer in layers + ) + def test_resolve_override_takes_priority_over_pack(self, project_dir, pack_dir): """Test that overrides take priority over installed packs.""" # Install the pack From 52c1acf8ba86879d1a4418bc1b46573ca43d5873 Mon Sep 17 00:00:00 2001 From: Ali jawwad <33836051+jawwad-ali@users.noreply.github.com> Date: Tue, 14 Jul 2026 01:54:59 +0500 Subject: [PATCH 27/45] fix(workflows): validate command step input/options are mappings (#3262) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * fix(workflows): validate command step input/options are mappings CommandStep.validate() only checked for 'command'; execute() then does input.items() and options.update(step_options). A non-mapping input:/options: (e.g. a YAML list or scalar) raised AttributeError at run time, bypassing the per-step FAILED/continue-on-error contract -- unlike the sibling steps (switch 'cases', fan-out 'step') which type-check their config fields in validate(). Add the same checks, plus a defense-in-depth coercion in execute() since the engine does not auto-validate before running a step. Co-Authored-By: Claude Opus 4.8 (1M context) * docs: fix code-comment typo in CommandStep.validate The explanatory comment said options.update(options) but execute() does options.update(step_options). Comment-only change; no behavior change. Co-Authored-By: Claude Opus 4.8 * fix(workflows): command step FAILS on malformed input/options instead of coercing execute() previously coerced a non-mapping 'input' to {} and silently ignored a non-mapping 'options', then dispatched the command anyway. For a workflow that skipped validation (the engine does not auto-validate before execute()), that let an explicitly malformed step run with empty args and report COMPLETED — masking the config error and defeating the per-step FAILED / continue_on_error semantics this change is meant to provide. Both now return a FAILED StepResult with the same contract error validate() reports (never crashing on .items()/.update()). Valid mapping configs are unaffected. Strengthened the execute() test to assert FAILED + the exact 'must be a mapping' error for input and options (fails before: the result carried the downstream dispatch error, not the shape error). Co-Authored-By: Claude Opus 4.8 (1M context) --------- Co-authored-by: Claude Opus 4.8 (1M context) --- .../workflows/steps/command/__init__.py | 40 ++++++++++++++++++- tests/test_workflows.py | 27 +++++++++++++ 2 files changed, 65 insertions(+), 2 deletions(-) diff --git a/src/specify_cli/workflows/steps/command/__init__.py b/src/specify_cli/workflows/steps/command/__init__.py index 891b9da4e7..7a6d893ed0 100644 --- a/src/specify_cli/workflows/steps/command/__init__.py +++ b/src/specify_cli/workflows/steps/command/__init__.py @@ -31,6 +31,20 @@ class CommandStep(StepBase): def execute(self, config: dict[str, Any], context: StepContext) -> StepResult: command = config.get("command", "") input_data = config.get("input", {}) + # validate() rejects a non-mapping input, but the engine does not + # auto-validate before execute(); a workflow that skipped validation can + # still reach here. Fail the step with the same contract error rather + # than silently coercing to {} and dispatching with empty args — that + # would change the command's meaning, hide the config error, and report + # COMPLETED, defeating the per-step FAILED / continue_on_error behavior. + if not isinstance(input_data, dict): + return StepResult( + status=StepStatus.FAILED, + error=( + f"Command step {config.get('id', '?')!r}: 'input' must be a " + f"mapping, got {type(input_data).__name__}." + ), + ) # Resolve expressions in input resolved_input: dict[str, Any] = {} @@ -50,8 +64,18 @@ def execute(self, config: dict[str, Any], context: StepContext) -> StepResult: # Merge options (workflow defaults ← step overrides) options = dict(context.default_options) step_options = config.get("options", {}) - if step_options: - options.update(step_options) + # Same rationale as 'input': a malformed options fails the step rather + # than being silently ignored (which would let an invalid step run and + # apparently complete). + if not isinstance(step_options, dict): + return StepResult( + status=StepStatus.FAILED, + error=( + f"Command step {config.get('id', '?')!r}: 'options' must be a " + f"mapping, got {type(step_options).__name__}." + ), + ) + options.update(step_options) # Attempt CLI dispatch args_str = str(resolved_input.get("args", "")) @@ -155,4 +179,16 @@ def validate(self, config: dict[str, Any]) -> list[str]: errors.append( f"Command step {config.get('id', '?')!r} is missing 'command' field." ) + # execute() iterates input.items() and options.update(step_options); a + # non-mapping here would raise at run time. Validate the shape like the + # sibling steps (switch 'cases', fan-out 'step') so it is reported, not + # crashed on. + if "input" in config and not isinstance(config["input"], dict): + errors.append( + f"Command step {config.get('id', '?')!r}: 'input' must be a mapping." + ) + if "options" in config and not isinstance(config["options"], dict): + errors.append( + f"Command step {config.get('id', '?')!r}: 'options' must be a mapping." + ) return errors diff --git a/tests/test_workflows.py b/tests/test_workflows.py index dcd7d19b84..81aec96d3e 100644 --- a/tests/test_workflows.py +++ b/tests/test_workflows.py @@ -953,6 +953,33 @@ def test_validate_missing_command(self): errors = step.validate({"id": "test"}) assert any("missing 'command'" in e for e in errors) + def test_validate_rejects_non_mapping_input_and_options(self): + from specify_cli.workflows.steps.command import CommandStep + from specify_cli.workflows.base import StepContext, StepStatus + + step = CommandStep() + # execute() does input.items() / options.update(); a non-mapping must be + # reported by validate(), not crash at run time (like switch 'cases'). + for bad in (None, "args", ["a", "b"], 5): + errs = step.validate({"id": "c", "command": "/x", "input": bad}) + assert any("'input' must be a mapping" in e for e in errs), bad + errs = step.validate({"id": "c", "command": "/x", "options": 42}) + assert any("'options' must be a mapping" in e for e in errs) + # a valid mapping config is still accepted + assert step.validate({"id": "c", "command": "/x", "input": {"args": "y"}, "options": {"k": 1}}) == [] + # execute() has no auto-validation guarantee (the engine may skip + # validate), so a non-mapping input/options FAILS the step with the same + # contract error — it does not silently coerce to empty and report + # COMPLETED (which would defeat continue_on_error). + res_in = step.execute({"id": "c", "command": "echo", "input": None}, StepContext()) + assert res_in.status is StepStatus.FAILED + assert "'input' must be a mapping" in (res_in.error or "") + res_opt = step.execute( + {"id": "c", "command": "echo", "input": {}, "options": 42}, StepContext() + ) + assert res_opt.status is StepStatus.FAILED + assert "'options' must be a mapping" in (res_opt.error or "") + def test_step_override_integration(self): from unittest.mock import patch from specify_cli.workflows.steps.command import CommandStep From 3c9aa1f81b5e70a938cf3f3d967349e23c4ba87d Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Mon, 13 Jul 2026 17:13:17 -0500 Subject: [PATCH 28/45] Add Autonomous Run Governance preset to community catalog (#3501) Add autonomous-run-governance preset submitted by @hindermath to: - presets/catalog.community.json (alphabetical order) - docs/community/presets.md community presets table Closes #3499 Assisted-by: GitHub Copilot (model: claude-sonnet-4.6, autonomous) Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- docs/community/presets.md | 1 + presets/catalog.community.json | 30 +++++++++++++++++++++++++++++- 2 files changed, 30 insertions(+), 1 deletion(-) diff --git a/docs/community/presets.md b/docs/community/presets.md index 52f923a3ad..b44d9587d6 100644 --- a/docs/community/presets.md +++ b/docs/community/presets.md @@ -11,6 +11,7 @@ The following community-contributed presets customize how Spec Kit behaves — o | Agent Parity Governance | Adds shared-guidance parity, audit-ready Spec-Kit run evidence, and agent-neutral model-routing guidance across a project's declared AI-agent instruction surfaces so agent guidance does not drift. | 6 templates, 3 commands | — | [spec-kit-preset-agent-parity-governance](https://github.com/hindermath/spec-kit-preset-agent-parity-governance) | | AIDE In-Place Migration | Adapts the AIDE extension workflow for in-place technology migrations (X → Y pattern) — adds migration objectives, verification gates, knowledge documents, and behavioral equivalence criteria | 2 templates, 8 commands | AIDE extension | [spec-kit-presets](https://github.com/mnriem/spec-kit-presets) | | Architecture Governance | Adds secure software architecture, STRIDE+CAPEC threat modeling, arc42 security cross-cutting concepts, S-ADRs, Zero Trust applicability, OWASP SAMM governance, BSI C3A cloud autonomy, BSI C5 cloud compliance assurance, and audit-ready Spec Kit run evidence | 13 templates, 3 commands | — | [spec-kit-preset-architecture-governance](https://github.com/hindermath/spec-kit-preset-architecture-governance) | +| Autonomous Run Governance | Adds permission-bounded, evidence-first governance for autonomous Spec Kit delivery, convergence, resume, closeout, and retrospective learning. | 10 templates, 2 commands | — | [spec-kit-preset-autonomous-run-governance](https://github.com/hindermath/spec-kit-preset-autonomous-run-governance) | | Canon Core | Adapts original Spec Kit workflow to work together with Canon extension | 2 templates, 8 commands | — | [spec-kit-canon](https://github.com/maximiliamus/spec-kit-canon) | | Claude AskUserQuestion | Upgrades `/speckit.clarify` and `/speckit.checklist` on Claude Code from Markdown-table prompts to the native AskUserQuestion picker, with a recommended option and reasoning on every question | 2 commands | — | [spec-kit-preset-claude-ask-questions](https://github.com/0xrafasec/spec-kit-preset-claude-ask-questions) | | Command Density | Compacts the nine core Spec Kit command prompts while preserving scripts, handoffs, placeholders, hook output blocks, and rule structure | 9 commands | — | [spec-kit-preset-command-density](https://github.com/Xopoko/spec-kit-preset-command-density) | diff --git a/presets/catalog.community.json b/presets/catalog.community.json index 24c312195c..eda764a828 100644 --- a/presets/catalog.community.json +++ b/presets/catalog.community.json @@ -1,6 +1,6 @@ { "schema_version": "1.0", - "updated_at": "2026-06-30T00:00:00Z", + "updated_at": "2026-07-13T00:00:00Z", "catalog_url": "https://raw.githubusercontent.com/github/spec-kit/main/presets/catalog.community.json", "presets": { "a11y-governance": { @@ -131,6 +131,34 @@ "created_at": "2026-04-27T00:00:00Z", "updated_at": "2026-06-14T00:00:00Z" }, + "autonomous-run-governance": { + "name": "Autonomous Run Governance", + "id": "autonomous-run-governance", + "version": "0.1.1", + "description": "Adds permission-bounded, evidence-first governance for autonomous Spec Kit delivery, convergence, resume, closeout, and retrospective learning.", + "author": "Thorsten Hindermann", + "repository": "https://github.com/hindermath/spec-kit-preset-autonomous-run-governance", + "download_url": "https://github.com/hindermath/spec-kit-preset-autonomous-run-governance/archive/refs/tags/v0.1.1.zip", + "homepage": "https://github.com/hindermath/spec-kit-preset-autonomous-run-governance", + "documentation": "https://github.com/hindermath/spec-kit-preset-autonomous-run-governance/blob/main/README.md", + "license": "MIT", + "requires": { + "speckit_version": ">=0.8.3" + }, + "provides": { + "templates": 10, + "commands": 2 + }, + "tags": [ + "autonomous", + "governance", + "evidence", + "permissions", + "retrospective" + ], + "created_at": "2026-07-13T00:00:00Z", + "updated_at": "2026-07-13T00:00:00Z" + }, "canon-core": { "name": "Canon Core", "id": "canon-core", From 5f59a5b238a49dc28ba2b8fc2567255c0b8c647d Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Mon, 13 Jul 2026 17:43:04 -0500 Subject: [PATCH 29/45] Add Test-First Governance preset to community catalog (#3504) Add test-first-governance preset submitted by @mnriem to: - presets/catalog.community.json (alphabetical order) - docs/community/presets.md community presets table Closes #3502 Assisted-by: GitHub Copilot (model: claude-sonnet-4.6, autonomous) Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- docs/community/presets.md | 1 + presets/catalog.community.json | 28 ++++++++++++++++++++++++++++ 2 files changed, 29 insertions(+) diff --git a/docs/community/presets.md b/docs/community/presets.md index b44d9587d6..81c01933aa 100644 --- a/docs/community/presets.md +++ b/docs/community/presets.md @@ -29,6 +29,7 @@ The following community-contributed presets customize how Spec Kit behaves — o | SicarioSpec Core | Baseline secure-by-default Spec Kit governance profile. | 5 templates | — | [sicario-spec](https://github.com/dfirs1car1o/sicario-spec) | | Spec2Cloud | Spec-driven workflow tuned for shipping to Azure: spec → plan → tasks → implement → deploy | 5 templates, 8 commands | — | [spec2cloud](https://github.com/Azure-Samples/Spec2Cloud) | | Table of Contents Navigation | Adds a navigable Table of Contents to generated spec.md, plan.md, and tasks.md documents | 3 templates, 3 commands | — | [spec-kit-preset-toc-navigation](https://github.com/Quratulain-bilal/spec-kit-preset-toc-navigation) | +| Test-First Governance | Governs TDD with coverage-complete BDD/ATDD Gherkin scenarios, explicit suite ownership, professional test reports, traceability, and risk-based quality gates. | 10 templates, 8 commands | — | [spec-kit-preset-test-first-governance](https://github.com/ka-zo/spec-kit-preset-test-first-governance) | | VS Code Ask Questions | Enhances the clarify command to use `vscode/askQuestions` for batched interactive questioning. | 1 command | — | [spec-kit-presets](https://github.com/fdcastel/spec-kit-presets) | | Workflow Preset | Behavior-first specification, design artifacts, and agent-native handoff orchestration — adds requirement-phase behavior drafts, formal BDD/UIF/behavior contracts, optional design artifacts, and scoped implementation handoffs with Core Agent, Vertical Planner Agent, and Worker Agent modes | 22 templates, 8 commands | — | [spec-kit-workflow-preset](https://github.com/bigsmartben/spec-kit-workflow-preset) | diff --git a/presets/catalog.community.json b/presets/catalog.community.json index eda764a828..b6bed5dc78 100644 --- a/presets/catalog.community.json +++ b/presets/catalog.community.json @@ -646,6 +646,34 @@ "created_at": "2026-04-30T00:00:00Z", "updated_at": "2026-04-30T00:00:00Z" }, + "test-first-governance": { + "name": "Test-First Governance", + "id": "test-first-governance", + "version": "1.3.0", + "description": "Governs TDD with coverage-complete BDD/ATDD Gherkin scenarios, explicit suite ownership, professional test reports, traceability, and risk-based quality gates.", + "author": "Zoltán Katona, PhD", + "repository": "https://github.com/ka-zo/spec-kit-preset-test-first-governance", + "download_url": "https://github.com/ka-zo/spec-kit-preset-test-first-governance/archive/refs/tags/1.3.0.zip", + "homepage": "https://github.com/ka-zo/spec-kit-preset-test-first-governance", + "documentation": "https://github.com/ka-zo/spec-kit-preset-test-first-governance/blob/main/README.md", + "license": "MIT", + "requires": { + "speckit_version": ">=0.12.11" + }, + "provides": { + "templates": 10, + "commands": 8 + }, + "tags": [ + "tdd", + "bdd", + "atdd", + "quality-gates", + "traceability" + ], + "created_at": "2026-07-13T00:00:00Z", + "updated_at": "2026-07-13T00:00:00Z" + }, "toc-navigation": { "name": "Table of Contents Navigation", "id": "toc-navigation", From a8d3038ece94e7282209f439d0b835d39ebec311 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Mon, 13 Jul 2026 17:50:58 -0500 Subject: [PATCH 30/45] [extension] Add Spec Kit Memory extension to community catalog (#3455) * Add Spec Kit Memory extension to community catalog Add memory extension submitted by @zaytsevand to: - extensions/catalog.community.json (alphabetical order) - docs/community/extensions.md community extensions table Closes #3446 Assisted-by: GitHub Copilot (model: claude-sonnet-4.6, autonomous) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: resolve merge conflicts with main branch - extensions/catalog.community.json: keep updated_at 2026-07-10 (more recent) - docs/community/extensions.md: include both Spec Kit Figma (main) and Spec Kit Memory (this PR) in alphabetical order Assisted-by: GitHub Copilot (model: claude-sonnet-4.5, autonomous) * fix: add memsearch optional tool dependency to memory extension catalog entry Assisted-by: GitHub Copilot (model: claude-sonnet-4.5, autonomous) --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> --- docs/community/extensions.md | 1 + extensions/catalog.community.json | 36 +++++++++++++++++++++++++++++++ 2 files changed, 37 insertions(+) diff --git a/docs/community/extensions.md b/docs/community/extensions.md index db7dc6dc32..bf23d63621 100644 --- a/docs/community/extensions.md +++ b/docs/community/extensions.md @@ -120,6 +120,7 @@ The following community-contributed extensions are available in [`catalog.commun | Spec Diagram | Auto-generate Mermaid diagrams of SDD workflow state, feature progress, and task dependencies | `visibility` | Read-only | [spec-kit-diagram-](https://github.com/Quratulain-bilal/spec-kit-diagram-) | | Spec Kit Discovery Extension | Run technical discovery commands for feasibility, technology selection, scenario-specific technical decisions, legacy codebase assessment, implementation understanding, and proof-of-concept validation | `process` | Read+Write | [spec-kit-discovery](https://github.com/bigsmartben/spec-kit-discovery) | | Spec Kit Figma | Agent-agnostic SpecKit extension that grounds spec, plan & task generation in Figma design context — REST + optional MCP, single/mono/multi-repo, macOS/Linux/Windows. | `integration` | Read+Write | [spec-kit-figma](https://github.com/Fyloss/spec-kit-figma) | +| Spec Kit Memory | Recalls prior specs and decisions from configurable memory tools (e.g. memsearch) before SDLC stages, so planning and specification start from what the project already knows | `docs` | Read+Write | [spec-kit-memory](https://github.com/zaytsevand/spec-kit-memory) | | Spec Kit Preview | Generate evidence-backed low, mid, or high fidelity previews from Spec Kit artifacts as Markdown or self-contained HTML | `docs` | Read+Write | [spec-kit-preview](https://github.com/bigsmartben/spec-kit-preview) | | Spec Kit Schedule | Optimal multi-agent task scheduling via CP-SAT — DAG precedence, hallucination-aware caps, file-conflict avoidance, stochastic durations, replanning, and interactive HTML output | `process` | Read+Write | [spec-kit-schedule](https://github.com/jfranc38/spec-kit-schedule) | | Spec Kit TLDR | Render a feature's spec.md / plan.md into a review-oriented TLDR (self-contained HTML dashboard + PR-native Markdown) that surfaces risks for faster PR review. | `visibility` | Read+Write | [speckit-tldr](https://github.com/qurore/speckit-tldr) | diff --git a/extensions/catalog.community.json b/extensions/catalog.community.json index 78df8c0979..bcc23e3d73 100644 --- a/extensions/catalog.community.json +++ b/extensions/catalog.community.json @@ -2268,6 +2268,42 @@ "created_at": "2026-05-08T00:00:00Z", "updated_at": "2026-05-08T00:00:00Z" }, + "memory": { + "name": "Spec Kit Memory", + "id": "memory", + "description": "Recalls prior specs and decisions from configurable memory tools (e.g. memsearch) before SDLC stages, so planning and specification start from what the project already knows.", + "author": "Andrey Zaytsev", + "version": "0.3.0", + "download_url": "https://github.com/zaytsevand/spec-kit-memory/archive/refs/tags/v0.3.0.zip", + "repository": "https://github.com/zaytsevand/spec-kit-memory", + "homepage": "https://github.com/zaytsevand/spec-kit-memory", + "documentation": "https://github.com/zaytsevand/spec-kit-memory/blob/main/README.md", + "changelog": "", + "license": "MIT", + "category": "docs", + "effect": "read-write", + "requires": { + "speckit_version": ">=0.2.0", + "tools": [ + { "name": "memsearch", "required": false } + ] + }, + "provides": { + "commands": 2, + "hooks": 3 + }, + "tags": [ + "memory", + "recall", + "research", + "memsearch" + ], + "verified": false, + "downloads": 0, + "stars": 0, + "created_at": "2026-07-10T00:00:00Z", + "updated_at": "2026-07-10T00:00:00Z" + }, "memory-loader": { "name": "Memory Loader", "id": "memory-loader", From 654793b65906417c449d77269407638cdebb6f8a Mon Sep 17 00:00:00 2001 From: Manfred Riem <15701806+mnriem@users.noreply.github.com> Date: Mon, 13 Jul 2026 17:52:54 -0500 Subject: [PATCH 31/45] chore: release 0.12.14, begin 0.12.15.dev0 development (#3506) * chore: bump version to 0.12.14 * chore: begin 0.12.15.dev0 development --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> --- CHANGELOG.md | 16 ++++++++++++++++ pyproject.toml | 2 +- 2 files changed, 17 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 62ad808d89..61360b0f93 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,22 @@ +## [0.12.14] - 2026-07-13 + +### Changed + +- [extension] Add Spec Kit Memory extension to community catalog (#3455) +- Add Test-First Governance preset to community catalog (#3504) +- Add Autonomous Run Governance preset to community catalog (#3501) +- fix(workflows): validate command step input/options are mappings (#3262) +- fix(presets): resolve() honors manifest-declared file: for installed presets (#3351) +- fix(init): don't block on confirmation for 'init --here' without a TTY (#3236) +- [extension] Add Quality Gates (Enforcement Layer) extension to community catalog (#3431) +- fix(integrations): exit cleanly on unbalanced quote in --integration-options (#3457) (#3466) +- fix(integrations): declare kiro-cli multi-install safe (#3471) (#3485) +- fix(workflows): fail fan-in step on non-list wait_for instead of crashing (#3482) +- chore: release 0.12.13, begin 0.12.14.dev0 development (#3498) + ## [0.12.13] - 2026-07-13 ### Changed diff --git a/pyproject.toml b/pyproject.toml index 50082664c4..933c35eadf 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,6 +1,6 @@ [project] name = "specify-cli" -version = "0.12.14.dev0" +version = "0.12.15.dev0" description = "Specify CLI, part of GitHub Spec Kit. A tool to bootstrap your projects for Spec-Driven Development (SDD)." readme = "README.md" requires-python = ">=3.11" From e48f134c3b1340609414ca9614ce74f33edc2391 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Tue, 14 Jul 2026 06:50:20 -0500 Subject: [PATCH 32/45] [extension] Add Multi-Repo Branch Sync extension to community catalog (#3411) * Add Multi-Repo Branch Sync extension to community catalog Add multi-repo-sync extension submitted by @sebastienthibaud to: - extensions/catalog.community.json (alphabetical order) - docs/community/extensions.md community extensions table Closes #3406 Assisted-by: GitHub Copilot (model: claude-sonnet-4.6, autonomous) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: restore multi-repo-sync sha256 * fix: update multi-repo-sync link text and catalog updated_at - Change extensions.md link text from 'spec-kit-multi-repo-sync' to 'multi-repo-sync' to match extension ID convention - Refresh catalog.community.json top-level updated_at to 2026-07-13T00:00:00Z Assisted-by: GitHub Copilot (model: claude-sonnet-4.5, autonomous) * fix: update multi-repo-sync entry timestamps to 2026-07-13 Set created_at and updated_at to 2026-07-13T00:00:00Z to match the catalog publication date, per add-community-extension/SKILL.md:86-87. Assisted-by: GitHub Copilot (model: claude-sonnet-4.6, autonomous) --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> --- docs/community/extensions.md | 1 + extensions/catalog.community.json | 42 +++++++++++++++++++++++++++++++ 2 files changed, 43 insertions(+) diff --git a/docs/community/extensions.md b/docs/community/extensions.md index bf23d63621..22e69ae000 100644 --- a/docs/community/extensions.md +++ b/docs/community/extensions.md @@ -84,6 +84,7 @@ The following community-contributed extensions are available in [`catalog.commun | MemoryLint | Evidence-driven instruction drift checker: audits agent memory files for boundary, reality, conflict, and redundancy drift. | `process` | Read+Write | [memorylint](https://github.com/RbBtSn0w/spec-kit-extensions/tree/main/memorylint) | | Microsoft 365 Integration | Fetch Teams messages, meeting transcripts, and SharePoint/OneDrive files as local Markdown for spec generation | `integration` | Read+Write | [spec-kit-m365](https://github.com/BenBtg/spec-kit-m365) | | Multi-Model Review | Cross-model Spec Kit handoffs for spec authoring, implementation routing, and review. | `process` | Read+Write | [multi-model-review](https://github.com/formin/multi-model-review) | +| Multi-Repo Branch Sync | Creates the feature branch in affected sub-repositories and git submodules via plan/tasks hooks | `process` | Read+Write | [multi-repo-sync](https://github.com/fyloss/spec-kit-multi-repo-sync) | | Multi-Sites Spec Kit | Multi-site aware specify command with per-site spec folders, auto-increment, and Drupal support | `process` | Read+Write | [spec-kit-multi-sites](https://github.com/teeyo/spec-kit-multi-sites) | | .NET Framework to Modern .NET Migration | Orchestrate end-to-end .NET Framework to modern .NET migration across 7 phases, with SDD lifecycle integration | `process` | Read+Write | [spec-kit-fx-to-net](https://github.com/RogerBestMsft/spec-kit-FxToNet) | | Onboard | Contextual onboarding and progressive growth for developers new to spec-kit projects. Explains specs, maps dependencies, validates understanding, and guides the next step | `process` | Read+Write | [spec-kit-onboard](https://github.com/dmux/spec-kit-onboard) | diff --git a/extensions/catalog.community.json b/extensions/catalog.community.json index bcc23e3d73..db0ae54201 100644 --- a/extensions/catalog.community.json +++ b/extensions/catalog.community.json @@ -2458,6 +2458,48 @@ "created_at": "2026-05-04T02:51:52Z", "updated_at": "2026-06-18T00:00:00Z" }, + "multi-repo-sync": { + "name": "Multi-Repo Branch Sync", + "id": "multi-repo-sync", + "description": "Creates the feature branch in affected sub-repositories and git submodules via plan/tasks hooks", + "author": "Fyloss", + "version": "1.0.0", + "download_url": "https://github.com/fyloss/spec-kit-multi-repo-sync/releases/download/v1.0.0/spec-kit-multi-repo-sync.zip", + "sha256": "12a5c7392145b4424b20715aaa3d8b6a8218c143dea596873e344146c1a76ba0", + "repository": "https://github.com/fyloss/spec-kit-multi-repo-sync", + "homepage": "https://github.com/fyloss/spec-kit-multi-repo-sync", + "documentation": "https://github.com/fyloss/spec-kit-multi-repo-sync/blob/main/README.md", + "changelog": "https://github.com/fyloss/spec-kit-multi-repo-sync/blob/main/CHANGELOG.md", + "license": "MIT", + "category": "process", + "effect": "read-write", + "requires": { + "speckit_version": ">=0.2.0", + "tools": [ + { + "name": "git", + "version": ">=2.31", + "required": true + } + ] + }, + "provides": { + "commands": 3, + "hooks": 2 + }, + "tags": [ + "git", + "branching", + "multi-repo", + "submodules", + "workflow" + ], + "verified": false, + "downloads": 0, + "stars": 0, + "created_at": "2026-07-13T00:00:00Z", + "updated_at": "2026-07-13T00:00:00Z" + }, "multi-sites": { "name": "Multi-Sites Spec Kit", "id": "multi-sites", From d956ab722bd64aa6632b2de202cc2fbfe65552c2 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Tue, 14 Jul 2026 06:51:50 -0500 Subject: [PATCH 33/45] =?UTF-8?q?[extension]=20Update=20DocGuard=20?= =?UTF-8?q?=E2=80=94=20CDD=20Enforcement=20extension=20to=20v0.32.0=20(#34?= =?UTF-8?q?89)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Update DocGuard — CDD Enforcement extension to v0.32.0 Update docguard extension submitted by @raccioly: - extensions/catalog.community.json (version, download_url, description, updated_at) - docs/community/extensions.md community extensions table Closes #3483 Assisted-by: GitHub Copilot (model: claude-sonnet-4.6, autonomous) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: add npx and specify to docguard requires.tools in catalog Assisted-by: GitHub Copilot (model: claude-sonnet-4.5, autonomous) * fix: shorten docguard description to ≤200 chars and correct validator count to 24 - Description was 275 chars, now 196 (under the 200-char catalog limit) - Change "27 validators" → "24 validators" to match v0.32.0 README - Applied to both extensions/catalog.community.json and docs/community/extensions.md Assisted-by: GitHub Copilot (model: claude-sonnet-4.6, autonomous) --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> --- docs/community/extensions.md | 2 +- extensions/catalog.community.json | 16 ++++++++++++---- 2 files changed, 13 insertions(+), 5 deletions(-) diff --git a/docs/community/extensions.md b/docs/community/extensions.md index 22e69ae000..30f0e5cd37 100644 --- a/docs/community/extensions.md +++ b/docs/community/extensions.md @@ -51,7 +51,7 @@ The following community-contributed extensions are available in [`catalog.commun | Confluence Extension | Create a doc in Confluence summarizing the specifications and planning files | `integration` | Read+Write | [spec-kit-confluence](https://github.com/aaronrsun/spec-kit-confluence) | | Cost Tracker | Track real LLM dollar cost across SDD workflows — per-feature budgets, per-integration comparison, and finance-ready exports | `visibility` | Read+Write | [spec-kit-cost](https://github.com/Quratulain-bilal/spec-kit-cost) | | Data Model Diagram | Generates Mermaid ER diagrams from Spec Kit data models after planning | `docs` | Read+Write | [spec-kit-data-model-diagram](https://github.com/benizzio/spec-kit-data-model-diagram) | -| DocGuard — CDD Enforcement | The only doc-integrity engine with an MCP server, SARIF output, and a deterministic zero-LLM core. Validates, scores, and traces documentation against code — 24 validators, stable finding codes, GitHub Action with PR annotations, spec-kit hooks. Pure Node.js, one pinned dep. | `docs` | Read+Write | [spec-kit-docguard](https://github.com/raccioly/docguard) | +| DocGuard — CDD Enforcement | Doc-integrity engine with MCP server, SARIF output, and zero-LLM core. Validates, scores, and traces documentation against code — 24 validators, stable finding codes, spec-kit hooks. Pure Node.js. | `docs` | Read+Write | [spec-kit-docguard](https://github.com/raccioly/docguard) | | EARS Requirements Syntax | Author, lint, and convert requirements using EARS - the five industry-standard sentence patterns for unambiguous, testable requirements | `docs` | Read+Write | [spec-kit-ears](https://github.com/dhruv-15-03/spec-kit-ears) | | Extensify | Create and validate extensions and extension catalogs | `process` | Read+Write | [extensify](https://github.com/mnriem/spec-kit-extensions/tree/main/extensify) | | Fix Findings | Automated analyze-fix-reanalyze loop that resolves spec findings until clean | `code` | Read+Write | [spec-kit-fix-findings](https://github.com/Quratulain-bilal/spec-kit-fix-findings) | diff --git a/extensions/catalog.community.json b/extensions/catalog.community.json index db0ae54201..505c71c676 100644 --- a/extensions/catalog.community.json +++ b/extensions/catalog.community.json @@ -1106,10 +1106,10 @@ "docguard": { "name": "DocGuard — CDD Enforcement", "id": "docguard", - "description": "The only doc-integrity engine with an MCP server, SARIF output, and a deterministic zero-LLM core. Validates, scores, and traces documentation against code — 24 validators, stable finding codes, GitHub Action with PR annotations, spec-kit hooks. Pure Node.js, one pinned dep.", + "description": "Doc-integrity engine with MCP server, SARIF output, and zero-LLM core. Validates, scores, and traces documentation against code — 24 validators, stable finding codes, spec-kit hooks. Pure Node.js.", "author": "raccioly", - "version": "0.30.0", - "download_url": "https://github.com/raccioly/docguard/releases/download/v0.30.0/spec-kit-docguard-v0.30.0.zip", + "version": "0.32.0", + "download_url": "https://github.com/raccioly/docguard/releases/download/v0.32.0/spec-kit-docguard-v0.32.0.zip", "repository": "https://github.com/raccioly/docguard", "homepage": "https://www.npmjs.com/package/docguard-cli", "documentation": "https://github.com/raccioly/docguard/blob/main/extensions/spec-kit-docguard/README.md", @@ -1124,6 +1124,14 @@ "name": "node", "version": ">=18.0.0", "required": true + }, + { + "name": "npx", + "required": true + }, + { + "name": "specify", + "required": false } ] }, @@ -1145,7 +1153,7 @@ "downloads": 0, "stars": 0, "created_at": "2026-03-13T00:00:00Z", - "updated_at": "2026-07-06T00:00:00Z" + "updated_at": "2026-07-13T00:00:00Z" }, "doctor": { "name": "Project Health Check", From 6ab0c1dac1bbc4b3d4d738f30eb25d0d3f68d426 Mon Sep 17 00:00:00 2001 From: Marsel Safin <179933638+marcelsafin@users.noreply.github.com> Date: Tue, 14 Jul 2026 14:15:04 +0200 Subject: [PATCH 34/45] fix(integrations): escape control characters in goose recipe YAML renderer (#3384) * fix(integrations): escape control characters in goose recipe YAML renderer YAML forbids C0 control characters (except tab and newline) and DEL in every scalar form, and a bare CR acts as a line break inside a block scalar. _render_yaml wrote the body verbatim into a |2 literal block scalar, so such bodies produced recipes the YAML parser rejects. Detect block-scalar-unsafe characters and fall back to an escaped double-quoted scalar via yaml.safe_dump, mirroring the TOML renderer's fallback strategy from #3341. Fixes #3382 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(integrations): use sys.maxsize instead of float inf for yaml width Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(integrations): extend block-scalar guard to C1 controls and Unicode line breaks YAML's printable set excludes C1 controls (U+0080-U+009F except NEL), and YAML 1.1 treats NEL/LS/PS as line breaks inside a literal block scalar, so bodies carrying any of these still produced unparseable recipes. Widen the fallback guard to the full class and cover it in the regression loop. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(integrations): also treat surrogates and U+FFFE/U+FFFF as block-scalar unsafe YAML's printable set also excludes lone UTF-16 surrogates and the non-characters U+FFFE/U+FFFF; bodies carrying them still hit the literal block path and produced unparseable recipes. Extend the guard and the regression loop. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * docs(integrations): clarify YAML prompt serialization Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- src/specify_cli/integrations/base.py | 34 +++++++++++++++++-- .../test_integration_base_yaml.py | 30 ++++++++++++++++ 2 files changed, 61 insertions(+), 3 deletions(-) diff --git a/src/specify_cli/integrations/base.py b/src/specify_cli/integrations/base.py index bfbb81b85b..a425d4e012 100644 --- a/src/specify_cli/integrations/base.py +++ b/src/specify_cli/integrations/base.py @@ -1122,6 +1122,17 @@ def setup( # YamlIntegration — YAML-format agents (Goose) # --------------------------------------------------------------------------- +# Characters a YAML literal block scalar cannot carry: C0 controls other +# than tab/LF (a bare CR acts as a line break inside the scalar), DEL, the +# C1 range, lone UTF-16 surrogates, and the non-characters U+FFFE/U+FFFF. +# NEL (U+0085) is YAML-printable but, like LS/PS (U+2028/U+2029), YAML 1.1 +# treats it as a line break, which corrupts the block scalar's structure +# just the same, so all three are included. +_YAML_BLOCK_SCALAR_UNSAFE = re.compile( + r"[\x00-\x08\x0b-\x1f\x7f-\x9f\u2028\u2029\ud800-\udfff\ufffe\uffff]" +) + + class YamlIntegration(IntegrationBase): """Concrete base for integrations that use YAML recipe format. @@ -1227,9 +1238,9 @@ def _build_yaml_header(cls, title: str, description: str) -> dict[str, Any]: def _render_yaml(cls, title: str, description: str, body: str, source_id: str) -> str: """Render a YAML recipe file from title, description, and body. - Produces a Goose-compatible recipe with a literal block scalar - for the prompt content. Uses ``yaml.safe_dump()`` for the - header fields to ensure proper escaping. + Produces a Goose-compatible recipe with a literal block scalar for + normal prompt content, or an escaped quoted scalar when control + characters require it. Uses ``yaml.safe_dump()`` for the header fields. """ header = cls._build_yaml_header(title, description) @@ -1240,6 +1251,23 @@ def _render_yaml(cls, title: str, description: str, body: str, source_id: str) - default_flow_style=False, ).strip() + # YAML forbids C0 control characters (except tab and newline) and + # DEL in every scalar form, and a bare CR acts as a line break + # inside a block scalar. A literal block scalar emits such bytes + # verbatim, producing a recipe the YAML parser rejects, so fall + # back to an escaped double-quoted scalar for those bodies. + if _YAML_BLOCK_SCALAR_UNSAFE.search(body): + prompt_yaml = yaml.safe_dump( + {"prompt": body}, allow_unicode=True, default_style='"', width=sys.maxsize + ).strip() + lines = [ + header_yaml, + prompt_yaml, + "", + f"# Source: {source_id}", + ] + return "\n".join(lines) + "\n" + # Indent the body for YAML block scalar. Use an explicit indentation # indicator ("|2") rather than a bare "|": YAML infers a plain block # scalar's indentation from its first non-empty line, so a body whose diff --git a/tests/integrations/test_integration_base_yaml.py b/tests/integrations/test_integration_base_yaml.py index 56bed09eb2..a3968384f2 100644 --- a/tests/integrations/test_integration_base_yaml.py +++ b/tests/integrations/test_integration_base_yaml.py @@ -201,6 +201,36 @@ def test_yaml_prompt_with_indented_first_line_stays_valid(self): parsed = yaml.safe_load("\n".join(yaml_lines)) assert parsed["prompt"].rstrip("\n") == body + def test_yaml_prompt_with_control_characters_stays_valid(self): + """A body containing control characters must still produce parseable YAML. + + YAML forbids C0 control characters (except tab and newline), DEL, + C1 controls, lone surrogates and U+FFFE/U+FFFF in every scalar form, + and YAML 1.1 treats NEL (U+0085), LS (U+2028) and PS (U+2029) as + line breaks that corrupt a literal block scalar's structure. The + renderer falls back to an escaped double-quoted scalar for such + bodies.""" + for ch in ( + "\x08", "\x0c", "\x1b", "\x7f", + "\x80", "\x84", "\x85", "\x86", "\x9f", + "\u2028", "\u2029", + "\ud800", "\udfff", "\ufffe", "\uffff", + ): + body = f"before{ch}after\nsecond line" + rendered = YamlIntegration._render_yaml("Title", "Desc", body, "src") + parsed = yaml.safe_load(rendered) + assert parsed["prompt"].rstrip("\n") == body, f"char {ch!r} round-trip" + + def test_yaml_prompt_with_bare_carriage_return_stays_valid(self): + """A bare CR (not part of CRLF) must not break the generated YAML. + + Inside a block scalar a lone \r acts as a line break, corrupting + the document structure.""" + body = "line1\rstill line1\nline2" + rendered = YamlIntegration._render_yaml("Title", "Desc", body, "src") + parsed = yaml.safe_load(rendered) + assert parsed["prompt"].rstrip("\n") == body + def test_plan_command_has_no_context_placeholder(self, tmp_path): """The generated plan command must not carry a context-file placeholder. From 2537be814428e553cea61a51d9b745ade2bf8e4c Mon Sep 17 00:00:00 2001 From: thejesh23 <35212698+thejesh23@users.noreply.github.com> Date: Tue, 14 Jul 2026 05:17:41 -0700 Subject: [PATCH 35/45] fix(extensions): stop env-var config leaking across prefix-colliding extension IDs (#3497) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * fix(extensions): stop env-var config leaking across prefix-colliding IDs (#3494) Because ``_`` doubles as both the separator between an extension ID and its config path AND the substitute for ``-`` inside an extension ID, an env var like ``SPECKIT_GIT_HOOKS_URL`` starts with *both* the ``SPECKIT_GIT_`` prefix of the ``git`` extension and the ``SPECKIT_GIT_HOOKS_`` prefix of a co-installed ``git-hooks`` extension. ``ConfigManager._get_env_config`` matched only on the shorter prefix, so the same env var silently surfaced inside both extensions' configs (as ``{'hooks': {'url': ...}}`` for ``git`` and ``{'url': ...}`` for ``git-hooks``). Impact: config intended for one extension leaked into another and, worse, could flip ``config. is set`` hook conditions on the wrong extension. Route the env var to the extension whose normalized ID is the longest match — the more specific one. When another installed sibling's normalized ID + ``_`` claims the remainder, skip the var here. The sibling scan reads ``.specify/extensions/`` directly and degrades to a no-op if the dir is missing (fresh project / ad-hoc harness), so the pre-fix single-extension behaviour is unchanged when there is no collision. Distinct from #3350 (intra-extension prefix collision between two keys of the same extension) — this fixes the cross-extension case. Fixes #3494 * fix(extensions): source sibling scan from registry, not directory Address Copilot review on #3497: ``ExtensionManager.remove(..., keep_config=True)`` preserves the extension directory but drops the registry entry, so the previous directory-scan approach would treat a config-only leftover as an installed sibling and silently discard ``SPECKIT__*`` env vars into no owner. Sourced the sibling list from ``ExtensionRegistry.keys()`` — the registry is the source of truth for "installed" — and kept the same graceful ``[]`` fallback so the fresh-project / ad-hoc harness path is unaffected. Updated the ``TestConfigManagerCrossExtensionEnvLeak`` ``_install`` helper to register its fake installations and added ``test_config_only_leftover_not_treated_as_sibling`` to lock in the new behaviour for the ``keep_config=True`` scenario. Full suite: 3978 passed, 110 skipped. * fix(extensions): swallow non-UTF-8 registry in sibling scan Address Copilot follow-up on #3497: ``ExtensionRegistry._load()`` catches ``JSONDecodeError`` / ``FileNotFoundError`` but not decode failures — a registry file with invalid text encoding would surface a ``UnicodeDecodeError`` out of ``_sibling_extension_ids`` and break every config read instead of degrading to the documented pre-fix behaviour. Extend the fallback in ``_sibling_extension_ids`` to also catch ``UnicodeError`` and add ``test_non_utf8_registry_does_not_crash`` as a regression pin (kept ``_load()`` itself out of scope — that broader hardening belongs in a separate PR since it affects all readers). Full suite: 3979 passed, 110 skipped. --- src/specify_cli/extensions/__init__.py | 66 ++++++++++++- tests/test_extensions.py | 132 +++++++++++++++++++++++++ 2 files changed, 197 insertions(+), 1 deletion(-) diff --git a/src/specify_cli/extensions/__init__.py b/src/specify_cli/extensions/__init__.py index aac1ba6481..e00b9c247a 100644 --- a/src/specify_cli/extensions/__init__.py +++ b/src/specify_cli/extensions/__init__.py @@ -2737,6 +2737,36 @@ def _get_local_config(self) -> Dict[str, Any]: config_file = self.extension_dir / "local-config.yml" return self._load_yaml_config(config_file) + def _sibling_extension_ids(self) -> list[str]: + """Return IDs of other extensions installed alongside this one. + + Sourced from ``ExtensionRegistry`` (``.specify/extensions/.registry``) + rather than a directory scan: ``ExtensionManager.remove(..., + keep_config=True)`` deliberately preserves the extension directory + while dropping the registry entry, so a directory scan would treat + that config-only leftover as an installed sibling and keep silently + absorbing its ``SPECKIT__*`` env vars into no one. The + registry is the source of truth for "installed". + + Returns an empty list if the registry is missing or corrupted + (fresh project, ad-hoc test harness) so ``_get_env_config`` degrades + to its pre-fix behaviour rather than crashing. ``UnicodeError`` is + caught alongside ``OSError`` because ``ExtensionRegistry._load()`` + opens the file in text mode and only handles ``JSONDecodeError`` / + ``FileNotFoundError``, so a registry file with non-UTF-8 bytes would + otherwise surface a ``UnicodeDecodeError`` here and break *every* + config read instead of degrading gracefully. + + Used by ``_get_env_config`` to detect env vars whose remainder claims + a longer, sibling-owned prefix (e.g. ``SPECKIT_GIT_HOOKS_URL`` is + owned by ``git-hooks`` when it is co-installed with ``git``). + """ + extensions_dir = self.project_root / ".specify" / "extensions" + try: + return list(ExtensionRegistry(extensions_dir).keys()) + except (OSError, UnicodeError): + return [] + def _get_env_config(self) -> Dict[str, Any]: """Get configuration from environment variables. @@ -2756,15 +2786,49 @@ def _get_env_config(self) -> Dict[str, Any]: ext_id_upper = self.extension_id.replace("-", "_").upper() prefix = f"SPECKIT_{ext_id_upper}_" + # Cross-extension prefix collision: because ``_`` doubles as both the + # separator between the extension ID and the config path *and* the + # substitute for ``-`` inside an extension ID, an env var like + # ``SPECKIT_GIT_HOOKS_URL`` begins with *both* the ``SPECKIT_GIT_`` + # prefix of the ``git`` extension and the ``SPECKIT_GIT_HOOKS_`` prefix + # of a co-installed ``git-hooks`` extension. It logically belongs to + # the extension whose normalized ID is the longer, more specific match + # — otherwise config intended for one extension silently surfaces + # inside another and can drive hooks that only inspect + # ``config. is set``. Build the list of sibling-owned + # remainder-prefixes here so a later env var can be skipped if it + # matches one. + sibling_prefixes: list[str] = [] + for sibling_id in self._sibling_extension_ids(): + if sibling_id == self.extension_id: + continue + sib_upper = sibling_id.replace("-", "_").upper() + # A sibling collides only when its normalized ID *extends* our own + # (i.e. starts with ``_``). ``git`` vs ``not-git`` is not a + # collision; ``git`` vs ``git-hooks`` is. + if sib_upper.startswith(ext_id_upper + "_"): + # The portion of the env-var *remainder* the sibling claims, + # including the trailing ``_`` so a shorter ID that shares a + # non-boundary prefix cannot false-positive (e.g. sibling + # ``hook`` would not eat env vars under key ``hooks``). + sibling_prefixes.append(sib_upper[len(ext_id_upper) + 1 :] + "_") + for key, value in os.environ.items(): if not key.startswith(prefix): continue + remainder = key[len(prefix) :] + # Skip when a longer sibling ID claims this var — see the block + # above. Keeps ``SPECKIT_GIT_HOOKS_URL`` out of the ``git`` + # extension's config when ``git-hooks`` is co-installed. + if any(remainder.startswith(sp) for sp in sibling_prefixes): + continue + # Remove prefix and split into parts. Drop empty components from a # malformed name (e.g. ``SPECKIT__`` with no key, or # consecutive underscores ``SPECKIT_X__Y``) so we never create an # entry under an empty key. - config_path = [p for p in key[len(prefix) :].lower().split("_") if p] + config_path = [p for p in remainder.lower().split("_") if p] if not config_path: continue diff --git a/tests/test_extensions.py b/tests/test_extensions.py index d9ffa5ca6f..d2f4af8264 100644 --- a/tests/test_extensions.py +++ b/tests/test_extensions.py @@ -7896,3 +7896,135 @@ def test_malformed_env_names_ignored(self, tmp_path, monkeypatch): cfg = cm._get_env_config() assert "" not in cfg assert cfg == {"a": {"b": "z"}} + + +class TestConfigManagerCrossExtensionEnvLeak: + """Cross-extension env-var leak: a longer, co-installed sibling ID must + own its own env vars instead of leaking them into a shorter-prefix sibling. + + Before the fix, ``SPECKIT_GIT_HOOKS_URL`` (intended for a ``git-hooks`` + extension) also surfaced inside the ``git`` extension's config as + ``{'hooks': {'url': ...}}`` because ``SPECKIT_GIT_`` is a strict prefix of + ``SPECKIT_GIT_HOOKS_``. + """ + + def _install(self, project_root, ext_id): + extensions_dir = project_root / ".specify" / "extensions" + (extensions_dir / ext_id).mkdir(parents=True) + # Register in the extension registry — the registry is the source of + # truth for "installed" (a bare directory can be a config-only leftover + # from ``ExtensionManager.remove(..., keep_config=True)``). + ExtensionRegistry(extensions_dir).add(ext_id, {}) + + def test_sibling_owns_longer_prefix_env(self, tmp_path, monkeypatch): + """SPECKIT_GIT_HOOKS_URL belongs to git-hooks when co-installed with git.""" + self._install(tmp_path, "git") + self._install(tmp_path, "git-hooks") + monkeypatch.setenv("SPECKIT_GIT_URL", "for_git") + monkeypatch.setenv("SPECKIT_GIT_HOOKS_URL", "for_git_hooks") + + git_cfg = ConfigManager(tmp_path, "git")._get_env_config() + gh_cfg = ConfigManager(tmp_path, "git-hooks")._get_env_config() + + # 'git' must NOT see the git-hooks var — no cross-extension leak. + assert git_cfg == {"url": "for_git"} + # 'git-hooks' still receives its own var (unchanged behaviour). + assert gh_cfg == {"url": "for_git_hooks"} + + def test_no_sibling_installed_keeps_legacy_absorption(self, tmp_path, monkeypatch): + """Without a longer-prefix sibling installed, the legacy behaviour is + preserved: ``SPECKIT_GIT_HOOKS_URL`` is absorbed as a nested key of + the ``git`` extension. This keeps the fix strictly to the *collision* + case and avoids surprising users who deliberately set a nested key + via env with no sibling to disambiguate against. + """ + self._install(tmp_path, "git") + monkeypatch.setenv("SPECKIT_GIT_HOOKS_URL", "for_git_hooks") + + cfg = ConfigManager(tmp_path, "git")._get_env_config() + assert cfg == {"hooks": {"url": "for_git_hooks"}} + + def test_non_prefix_sibling_ignored(self, tmp_path, monkeypatch): + """A sibling whose ID does not extend our own is not a collision. + + e.g. current='git' and sibling='not-git' — 'not-git' normalized to + 'NOT_GIT' does not start with 'GIT_', so its presence must not + influence git's env-var interpretation. + """ + self._install(tmp_path, "git") + self._install(tmp_path, "not-git") + monkeypatch.setenv("SPECKIT_GIT_HOOKS_URL", "for_git_hooks") + + cfg = ConfigManager(tmp_path, "git")._get_env_config() + assert cfg == {"hooks": {"url": "for_git_hooks"}} + + def test_boundary_prevents_false_positive(self, tmp_path, monkeypatch): + """Sibling ID 'hook' (not 'hooks') must NOT eat env keys starting + with 'hooks'. The trailing-underscore boundary in the sibling prefix + prevents this false positive. + """ + self._install(tmp_path, "git") + self._install(tmp_path, "git-hook") + monkeypatch.setenv("SPECKIT_GIT_HOOKS_URL", "for_git_key_hooks") + + # git-hook's prefix is 'HOOK_', which does not match 'HOOKS_URL', + # so 'git' keeps the env var (single-installed semantics). + cfg = ConfigManager(tmp_path, "git")._get_env_config() + assert cfg == {"hooks": {"url": "for_git_key_hooks"}} + + def test_missing_extensions_dir_does_not_crash(self, tmp_path, monkeypatch): + """A ConfigManager built against a project without ``.specify/extensions`` + (fresh project, ad-hoc test harness) must still evaluate env config + rather than raising from the sibling scan. + """ + # Note: no _install call — extensions dir intentionally absent. + monkeypatch.setenv("SPECKIT_TESTEXT_URL", "v") + + cfg = ConfigManager(tmp_path, "testext")._get_env_config() + assert cfg == {"url": "v"} + + def test_config_only_leftover_not_treated_as_sibling(self, tmp_path, monkeypatch): + """A directory left behind by ``remove(..., keep_config=True)`` must + NOT be treated as an installed sibling. + + ``ExtensionManager.remove(keep_config=True)`` preserves the extension + directory (config files remain, dormant, for a possible reinstall) but + removes the registry entry. The sibling scan is sourced from the + registry, so a leftover ``git-hooks/`` directory without a registry + entry must not silently discard ``SPECKIT_GIT_HOOKS_*`` from ``git``. + """ + self._install(tmp_path, "git") + # Simulate ``remove('git-hooks', keep_config=True)``: dir present, + # config file preserved, but no registry entry. + gh_dir = tmp_path / ".specify" / "extensions" / "git-hooks" + gh_dir.mkdir(parents=True) + (gh_dir / "git-hooks-config.yml").write_text("url: leftover\n") + # Sanity: git-hooks is NOT registered. + registry = ExtensionRegistry(tmp_path / ".specify" / "extensions") + assert "git-hooks" not in registry.keys() + + monkeypatch.setenv("SPECKIT_GIT_HOOKS_URL", "for_git") + + cfg = ConfigManager(tmp_path, "git")._get_env_config() + # git absorbs the var (no registered sibling owns it). + assert cfg == {"hooks": {"url": "for_git"}} + + def test_non_utf8_registry_does_not_crash(self, tmp_path, monkeypatch): + """A registry file with invalid text encoding must NOT propagate + ``UnicodeDecodeError`` out of the sibling scan and abort every + config read. ``ExtensionRegistry._load()`` catches ``JSONDecodeError`` + / ``FileNotFoundError`` only, so ``_sibling_extension_ids`` must + additionally swallow ``UnicodeError`` and degrade to the documented + pre-fix behaviour. + """ + extensions_dir = tmp_path / ".specify" / "extensions" + extensions_dir.mkdir(parents=True) + # Write bytes that are not valid UTF-8 to the registry file. + (extensions_dir / ExtensionRegistry.REGISTRY_FILE).write_bytes( + b"\xff\xfe invalid utf-8 registry \xc3\x28" + ) + monkeypatch.setenv("SPECKIT_TESTEXT_URL", "v") + + # Must not raise; must fall back to the "no siblings" path. + cfg = ConfigManager(tmp_path, "testext")._get_env_config() + assert cfg == {"url": "v"} From d7b662621812db64de0ddafc0acf72a61aab5dd9 Mon Sep 17 00:00:00 2001 From: Marsel Safin <179933638+marcelsafin@users.noreply.github.com> Date: Tue, 14 Jul 2026 15:03:33 +0200 Subject: [PATCH 36/45] feat(workflows): align workflow CLI with extension command surface (#3419) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * feat(workflows): align workflow CLI with extension command surface Adds the missing workflow commands and flags so the workflow CLI matches the extension/preset pattern: add --dev and --from, search --author, update, enable and disable. Disabled workflows are blocked from running and marked in list output. Fixes #2342 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(workflows): preserve disabled state on update, guard corrupted registry entries Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(workflows): guard list against corrupted registry entries, re-raise typer.Exit in catalog install workflow list now skips non-dict registry entries with a warning instead of crashing, matching update/enable/disable. The broad except in _install_workflow_from_catalog no longer swallows typer.Exit, so precise errors like the non-HTTPS redirect message are not duplicated. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(workflows): escape rich markup in id-mismatch errors and validate --from source early The two id-mismatch error paths interpolated repr() into Rich markup, so a stray bracket in a user typo could be parsed as markup. Route both through rich.markup.escape. `workflow add --from ` also validated the source only after downloading. Validate it up front so a URL/path/typo fails without a network fetch. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(workflows): escape rich markup in list output and catalog install errors, isolate update failures workflow list now escapes id/name/version/description before printing, matching how extensions render user-editable fields. The catalog install helper computes safe_wf_id once and uses it for every early error path plus the final failure message. workflow update wraps _safe_workflow_id_dir and the backup read inside the try/except typer.Exit block so an unsafe id in a corrupted registry fails that one workflow and the rest continue. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(workflows): escape rich markup in --from download exception message Matches how the catalog install path escapes exception strings. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(workflows): catch OSError in per-workflow update loop and make restore best-effort Transient FS errors (perms, disk full) from backup read or write no longer abort the whole update run. The restore is wrapped in its own try/except so a failed write only warns, and the offending workflow is reported via 'Failed to update' like other per-workflow failures. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(workflows): escape rich markup in search output workflow search now escapes catalog-derived name/id/version/description/ tags before printing, matching extension search and workflow list. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: escape workflow validation errors before Rich output Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(workflows): escape remaining unescaped Rich markup paths Covers the last few review threads not yet addressed: - Escape yaml.YAMLError text in the local workflow add install path (matches the already-escaped download/catalog paths). - Escape the non---dev local directory fallback's "No workflow.yml found in " message (the --dev branch already escaped it). - Escape the redirected final_url in the --from non-HTTPS redirect error (IPv6 literals like http://[::1]/... are legal and contain brackets). - Escape the "Downloaded workflow is invalid" exception message in _install_workflow_from_catalog, matching the sibling catalog-install exception handler a few lines above it. Adds regression tests for each in TestWorkflowCliAlignment, following the existing escaping-test pattern in this class. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(workflows): escape workflow name/id in install success messages Workflow names and ids come from user-controlled YAML or external catalog data; printing them unescaped lets bracket characters be interpreted as Rich tags. Escape them in the add/catalog-install success messages and the remaining catalog error paths, matching the rest of the output hardening. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(workflows): fail cleanly on unparseable catalog install URLs urlparse raises ValueError on e.g. an unbalanced IPv6 literal before the invalid-URL branch is reached; on workflow update that also bypassed the per-workflow handler and aborted the whole command. Convert the parse failure into a clean error so add fails cleanly and update skips just the affected workflow. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(workflows): reject catalog updates whose downloaded version mismatches The update path never verified the downloaded workflow carries the catalog version that triggered the update, so a stale or misconfigured URL could report success while leaving the old version installed or downgrading it. Pass the expected version into the install helper and fail the update when the downloaded definition does not match. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(workflows): validate workflow ID in run command and document new CLI flags Path-equivalent spellings like "align-wf/" previously bypassed the registry disabled check because the engine normalizes the path while the registry matches the raw string. workflow run now validates non-file sources against the workflow ID pattern before lookup. Also updates docs/reference/workflows.md with --dev/--from install options, update/enable/disable commands, and the search --author flag. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(workflows): enforce disabled state for direct paths to installed workflows Running the installed copy's YAML directly (specify workflow run .specify/workflows/align-wf/workflow.yml) skipped the registry check. File sources resolving inside .specify/workflows// now map back to the workflow ID and refuse to run while disabled. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(workflows): reject explicit empty --from URL instead of catalog fallback 'workflow add foo --from ""' fell through 'from_url or ...' to a catalog install. Distinguish None from empty string so explicit values stay on the URL-validation path and fail closed. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(workflows): registry rollback on save failure, consistent disabled check, honest update summary - WorkflowRegistry.add now rolls back its in-memory mutation when save() raises, so a later successful save cannot persist metadata for a failed update alongside the restored YAML backup. - workflow run uses the same truthiness check for 'enabled' as list and disable, so malformed values like 0 or null refuse to run. - workflow update reports 'No workflows were eligible for update' when every target was skipped instead of claiming all are up to date. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(workflows): guard non-string catalog URL and keep enable/disable rollback intact - A truthy non-string catalog url (e.g. 123) reached urlparse and raised AttributeError, escaping the clean error path; validate it is a string. - enable/disable mutated the live registry entry before add(), so add's rollback snapshot captured the already-toggled object; pass a fresh mapping instead. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(workflows): tolerate non-dict registry entries in add and clarify test docstrings A corrupted-but-parseable registry entry (e.g. a string value) crashed WorkflowRegistry.add with AttributeError on existing.get. Guard the non-dict case while still restoring the original raw value on rollback. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(workflows): atomic registry save and accurate mixed-target update summary - save() wrote the registry with open('w'), so a failed dump truncated the file and the next load reset every entry. Write to a sibling temp file and os.replace into place. - workflow update no longer claims all workflows are up to date when some targets were skipped; it reports checked-only status with a skipped count. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(workflows): exclusive temp file for registry save and cwd-independent disabled guard - save() now uses tempfile.mkstemp in the workflows dir (matching the engine's atomic writer), so a pre-created symlink at a predictable .tmp path cannot redirect the write and concurrent processes cannot collide. - The direct-path disabled guard derives the owning project from the resolved file path instead of the caller's cwd, so running an installed workflow's YAML from outside the project still refuses when disabled. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(workflows): symlink guards and shape validation in workflow registry, dev-dir file check - WorkflowRegistry now mirrors StepRegistry: _load refuses symlinked parents/registry file and normalizes a non-dict workflows field; save() rejects symlinked paths before writing. - workflow add --dev requires workflow.yml to be a regular file so a directory named workflow.yml gets the documented CLI error instead of an uncaught IsADirectoryError. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(workflows): validate download redirects before following them All three workflow download sites (add --from, catalog install, step install) passed no redirect_validator to open_url, so an HTTPS URL redirecting to cleartext HTTP issued the insecure request before the post-hoc geturl() check reported it. Shared validator now rejects non-HTTPS redirects (loopback HTTP allowed) pre-follow, matching the preset download path. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * test(workflows): accept redirect_validator kwarg in step-add open_url fakes Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(workflows): guard directory-shaped workflow.yml and unreadable registry - workflow add's plain local-path fallback (no --dev) checked wf_file.exists() before installing, so a directory literally named workflow.yml passed the guard and _validate_and_install_local() leaked an uncaught IsADirectoryError instead of the documented CLI error. Use is_file(), matching the --dev branch's existing guard. - WorkflowRegistry._load() treated any OSError while reading an existing registry the same as corrupted JSON, resetting to an empty in-memory registry. A later save() would then silently persist that empty state via os.replace, discarding every previously installed workflow entry. Track a _load_error flag on OSError-during-read and have save() refuse to write when it is set, so a transient I/O failure can no longer overwrite intact data on disk. - docs/reference/workflows.md: document `--from ` with its value placeholder, matching extensions.md and presets.md. Assisted-by: GitHub Copilot (model: Claude Sonnet 5, autonomous) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(workflow): rollback registry.remove() and guard OSError at CLI boundaries Critical: WorkflowRegistry.remove() deleted the in-memory entry then called save() with no rollback, unlike add(). Combined with workflow_remove deleting the workflow directory before calling registry.remove(), a save failure permanently destroyed the workflow's files, left the on-disk registry still claiming it installed, and surfaced a raw unhandled OSError with no CLI message. - WorkflowRegistry.remove() now rolls back the in-memory entry on a save() OSError, mirroring add()'s existing rollback pattern. - workflow_remove persists the registry removal (registry.remove(), wrapped in try/except OSError -> clean escaped message) before deleting any files, so a save failure never touches the workflow directory. Important sibling paths: workflow add (local/--dev/--from and catalog), enable, and disable all called registry.add() without catching its deliberate OSError, so a save failure surfaced either an orphaned install directory (fresh local/catalog installs) or a raw/unhandled exception with no clean CLI output. - _validate_and_install_local (backs local/--dev/--from) now removes the freshly created directory on a fresh install, or restores the prior workflow.yml bytes on a reinstall-over-existing-local install, before raising a clean escaped error. - _install_workflow_from_catalog wraps the final registry.add() using the function's own established convention (rmtree the just-downloaded workflow_dir, then a clean escaped error) -- workflow_update's existing backup/restore around this function is unaffected. - workflow_enable/workflow_disable catch registry.add()'s OSError and print a clean escaped message instead of leaking the exception. Added failing-first tests proving each behavior (registry-unit rollback test, CLI-level remove/add/enable/disable save-failure tests parametrized where they share one root cause), all confirmed red before the fix and green after. Assisted-by: GitHub Copilot (model: Claude Sonnet 5, autonomous) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(workflow): preserve prior catalog install on reinstall registry-save failure _install_workflow_from_catalog's final registry.add() failure handler unconditionally rmtree'd workflow_dir. That's safe for a brand-new install, but plain `workflow add ` also allows re-adding an already-installed workflow, downloading the new version over the existing directory first. If registry.add() then failed to save, the unconditional rmtree deleted the prior working install while the registry (after its own rollback) still reported it installed -- data loss with no way back. workflow_update already avoids this via an outer backup/restore around this function, but plain add has no such caller. Fix mirrors _validate_and_install_local's existed-before/backup-aware handling: capture whether workflow_dir existed and back up its workflow.yml bytes before any download write, then on a registry.add() OSError, restore those bytes for a reinstall or rmtree only a brand-new directory. Only one file (workflow.yml) is ever written by this path, so no further per-file bookkeeping is needed. Added a failing-first regression: install a catalog workflow, re-add it with a simulated registry save OSError, and assert a clean error, the original workflow.yml restored byte-for-byte, and the registry still reporting the original version installed. Confirmed red (prior file deleted) before the fix, green after. Assisted-by: GitHub Copilot (model: Claude Sonnet 5, autonomous) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(workflow): centralize catalog-install cleanup across all failure branches _install_workflow_from_catalog is new in this PR and has seven failure branches after the mkdir/download step, each independently rmtree'ing workflow_dir: redirect-to-non-HTTPS rejection, a generic download exception, invalid downloaded YAML, a validate_workflow failure, a workflow-id/catalog-key mismatch, a version mismatch, and (fixed in the prior commit) a registry.add() OSError. Only the last one had been special-cased to spare a prior working install on reinstall; the other six still unconditionally deleted the whole directory, so re-adding an already-installed catalog workflow and hitting any of those six earlier failures destroyed the working install even though nothing about it had actually changed. Replaced all seven ad hoc rmtree call sites with a single local _cleanup_failed_install() helper that closes over the existed_before / prior_workflow_bytes captured once at the top of the function: restore the prior workflow.yml for a reinstall, or rmtree only a directory that this attempt itself created. Every failure branch now calls this one helper, so the fix is structural rather than duplicated, and every existing error message/exit code is unchanged -- only the cleanup performed before each message is different. Added a parametrized regression test covering the four early-failure trigger points reachable from plain workflow add (redirect rejection, download exception, invalid YAML, ID mismatch): each installs a catalog workflow, re-adds it while forcing that specific failure, and asserts a clean error plus the original workflow.yml surviving byte-for-byte. Confirmed red against the unfixed code (all four raised FileNotFoundError reading the deleted file) before applying the helper, green after. Assisted-by: GitHub Copilot (model: Claude Sonnet 5, autonomous) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(workflow): restore registry entry verbatim on post-removal rmtree failure workflow_remove now persists registry.remove() before deleting any files (fixed previously), but if the registry write succeeds and the subsequent shutil.rmtree(workflow_dir) then fails, the registry was left claiming the workflow uninstalled while its directory remained on disk -- an orphaned install with no path back to a clean state. workflow_step_remove already handles this exact sequencing by capturing the registry entry before removal and restoring it directly into registry.data plus save() (bypassing add(), which would stamp a new updated_at) if the directory removal fails afterwards. Applied the same pattern to workflow_remove: capture registry_metadata via registry.get() before registry.remove(), and on an rmtree OSError, write it straight back into registry.data["workflows"][workflow_id] and save(), matching workflow_step_remove's restore-failure handling (a yellow warning, not a hard failure, since the primary error is already about to be reported). Existing error message and exit behavior for the rmtree failure are unchanged. Added a failing-first regression: install a workflow, monkeypatch shutil.rmtree to raise OSError, and assert a clean existing error message, the directory remaining (rmtree never actually deleted anything), and the registry entry restored byte-for-byte identical (including installed_at/updated_at) -- proving the fix bypasses add() and doesn't re-stamp timestamps. Confirmed red (registry entry stayed None) before the fix, green after. Assisted-by: GitHub Copilot (model: Claude Sonnet 5, autonomous) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fix 4 current Copilot review findings on workflow run/registry/install 1. workflow run ownership check followed symlinks via Path.resolve() before mapping a direct YAML path back to its installed workflow ID. A symlinked .specify/workflows//workflow.yml resolved outside the tree, missed the ownership match entirely, and let the disabled-workflow guard be silently skipped while engine.load_workflow still followed the symlink. Now maps ownership from a lexically-normalized path (os.path. normpath, no symlink following) and explicitly refuses to run if the installed directory or workflow.yml leaf is itself a symlink. Direct external workflow paths that don't match .specify/workflows/... are unaffected. 2. WorkflowRegistry._load() caught a read OSError and silently fell back to an empty in-memory registry, only blocking a later save(). Callers that only query is_installed()/get()/list() before writing a file (e.g. commands/init.py's bundled speckit install, which overwrites workflow.yml once is_installed() reports false) could act on that false-empty state and destroy real data before ever reaching save(). _load() now raises OSError immediately so an unreadable registry fails closed at construction, before any query or side effect is possible. Added _open_workflow_registry() to give every CLI command a consistent clean-error boundary around registry construction. 3. _validate_and_install_local's mkdir/copy2 ran before the try/except that protected registry.add(); a copy2 failure (e.g. a truncating partial write on a reinstall) was not caught at all, so the existing backup-restore cleanup never ran and the prior working workflow.yml was corrupted with a raw traceback surfaced to the user. mkdir/copy2 now run inside the same rollback-protected section as registry.add(), sharing one _cleanup_failed_install() helper. 4. workflow update's skip message claimed any non-catalog source was installed "from a local path or URL", which is wrong for the bundled speckit workflow (source: "bundled"). Message is now source-neutral. Verified all 4 threads are current (not outdated) via GraphQL review thread query on PR #3419, HEAD 812050a. Tests: strict TDD per fix (red test proving each bug, minimal production change, green). tests/test_workflows.py: 474 passed. Full suite: 3976 passed, 110 skipped. ruff check: all checks passed on touched files and full src tree. Assisted-by: GitHub Copilot (model: Claude Sonnet 5, autonomous) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fix disabled-workflow bypass via symlinked .specify project root workflow run's ownership check derived registry_root/registered_id from the lexical path, then checked the id directory and workflow.yml leaf for symlinks -- but never checked .specify or .specify/workflows themselves for that derived root. _reject_unsafe_workflow_storage only guards the cwd's project_root, which can differ from the path-derived registry_root (a direct path into an unrelated project, or that project's own .specify being a symlink to an attacker-controlled tree). WorkflowRegistry's own symlinked-parent handling silently substitutes an empty registry instead of raising, so a query against it (is_installed/get returning "not found") is not a safety signal a caller can rely on: with a symlinked .specify, the disabled check saw no registry entry and let a disabled workflow run anyway. Fix: reject an unsafe .specify/.specify-workflows for the actual derived registry_root before ever consulting the registry, reusing the existing _reject_unsafe_dir helper already used by _reject_unsafe_workflow_storage. Red-first end-to-end repro: victim project's .specify symlinked to an attacker-controlled tree containing a disabled workflow entry, run invoked with a direct path from an unrelated cwd -- confirmed the disabled workflow executed (exit 0) before the fix, now refused cleanly. Tests: tests/test_workflows.py 475 passed. Full suite: 3977 passed, 110 skipped. ruff check: all checks passed. Assisted-by: GitHub Copilot (model: Claude Sonnet 5, autonomous) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fix raw exception leak in bundle remove primitive boundary remove_bundle() had no exception handling around its component removal loop, unlike install_bundle() which converts any raw exception into a clean BundlerError. Since WorkflowRegistry now fails closed (raises OSError) on an unreadable registry file, and _WorkflowKindManager.__init__ constructs WorkflowRegistry with no try/except, an unreadable workflow registry surfaced as a raw OSError through remove_bundle(). The bundle_remove CLI command only catches BundlerError, so the raw OSError propagated uncaught, producing exit_code=1 with empty output instead of a clean, actionable message. Wrap remove_bundle()'s component loop in the same try/except BundlerError: raise / except Exception: raise BundlerError(...) from exc pattern already used by install_bundle(), converting any raw exception at this shared boundary. save_records() remains outside the try block, so a failure still leaves the bundle's record untouched (no removal side effects recorded). Tests: - tests/integration/test_bundler_install_flow.py::test_remove_converts_raw_installer_exception_to_bundler_error (function-level regression: a raw OSError from installer.is_installed must become a clean BundlerError, and the bundle record must survive) - tests/contract/test_bundle_cli.py::test_remove_reports_clean_error_when_primitive_raises_raw_exception (CLI-level regression: `specify bundle remove` must print a clean actionable message and exit non-zero instead of raw/empty output) Both tests were confirmed red beforehand: the raw OSError propagated uncaught out of remove_bundle(), and the CLI-level CliRunner result showed exit_code=1 with empty output. Assisted-by: GitHub Copilot (model: Claude Sonnet 5, autonomous) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fix 8 current Copilot review findings on registry fail-closed, rollback orphans, backup-read boundaries, and Rich escaping 1. WorkflowRegistry._load(): a symlinked .specify/.specify/workflows parent (or a symlinked registry file) silently returned an empty registry instead of raising, unlike an unreadable-file read failure. A read-only caller (notably the bundler's remove path) querying is_installed() before ever writing could conclude an installed workflow is absent, skip removing it, then delete the bundle record -- leaving the workflow untracked but still on disk. Now raises OSError immediately, matching the existing unreadable-file fail-closed behavior. 2/8. _validate_and_install_local and _install_workflow_from_catalog: when the destination directory already existed but had no prior workflow.yml (e.g. a leftover empty dir), existed_before was True but there were no backup bytes to restore, so the rollback closure did nothing on a later failure -- leaving the newly copied/ downloaded file behind. Both now unlink the newly created file in this case, restoring the pre-existing directory to its prior (empty) state. 3/4. Both install paths read the prior workflow.yml bytes (to seed the reinstall rollback) *before* any try/except boundary: a read failure on the existing file (e.g. a transient permission/FS issue) leaked a raw, unescaped OSError instead of the same clean CLI error used by every other failure branch in these functions. Both reads are now guarded by their own try/except OSError, with no writes attempted before the read succeeds (so there is nothing to roll back on this specific failure). 5. remove_bundle's exception-conversion message unconditionally claimed "No changes were recorded," even though a failure can occur after earlier components in the same bundle have already been removed from disk (save_records never runs on this path, so the record is left claiming the bundle fully installed). The message now reports how many components were already removed when that happened, instead of asserting no changes occurred. 6/7. workflow_remove's new post-registry-removal directory-failure error and its restore-failure warning interpolated workflow_dir and the exception values into Rich markup unescaped. A project path or OS error message containing Rich-markup-like brackets could be parsed as markup and hide/corrupt the displayed text. Both now use the existing _escape_markup helper, consistent with every other error path in this file. Tests (tests/test_workflows.py unless noted): - TestWorkflowRegistry::test_load_symlinked_workflows_dir_fails_closed_not_silently_empty (1) - TestWorkflowCliAlignment::test_add_dev_fresh_install_into_preexisting_empty_dir_cleans_new_file (2) - TestWorkflowCliAlignment::test_add_catalog_fresh_install_into_preexisting_empty_dir_cleans_new_file (8) - TestWorkflowCliAlignment::test_add_dev_reinstall_backup_read_failure_gives_clean_error (3) - TestWorkflowCliAlignment::test_add_catalog_reinstall_backup_read_failure_gives_clean_error (4) - tests/integration/test_bundler_install_flow.py::test_remove_partial_failure_message_reflects_partial_state (5) - TestWorkflowRemoveGuard::test_remove_directory_and_restore_failure_escapes_rich_markup (6/7) All seven were confirmed red beforehand, matching each thread's described failure mode exactly (silent empty registry instead of a raise; orphaned new file left behind; raw unescaped OSError leaking; a misleading "no changes were recorded" claim; Rich markup consuming bracketed path/exception text). Also updated test_registry_save_refuses_symlinked_parent, a pre-existing test that asserted the symlinked-parent raise at add()/save() time -- it now raises at construction instead, per fix #1, so the test was adjusted to match without weakening its guarantee (still asserts no writes occur under the symlinked target). Assisted-by: GitHub Copilot (model: Claude Sonnet 5, autonomous) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fix 3 current Copilot review findings: bookkeeping-aware BundlerError removal, bounded workflow downloads 1. bundle remove: BundlerError raised by the primitive installer itself (e.g. from a kind manager) bypassed the partial-removal bookkeeping message added previously via a bare `except BundlerError: raise`. Now routes through the same detail-construction logic as generic exceptions, so a mid-loop BundlerError after an earlier successful removal still reports that the project may be partially uninstalled, while a zero-removal BundlerError still reports "No components were removed." Both preserve the original exception message and chain `from exc`. 2/3. workflow add --from and catalog install/update downloads used unbounded `response.read()`, buffering the entire server-controlled body into memory before any size check, and trusted Content-Length alone where checked at all. Added a single shared `_read_response_within_limit()` helper reused by both call sites: it fails fast on an oversized declared Content-Length, and separately enforces the same cap while streaming in 64KiB chunks so a chunked or Content-Length-less response cannot bypass the limit by lying about or omitting its size. Chose 5 MiB as the cap: workflow YAML definitions are small step/metadata text, not binaries, so this is generous headroom against a malicious/misbehaving server without affecting any legitimate workflow definition. Both call sites already route any raised exception through their existing clean-error and rollback (`_cleanup_failed_install`) paths, so no additional error-handling plumbing was needed. Tests: extended the shared `_FakeResponse` test helper (and 5 duplicate per-test FakeResponse classes) to support `.read(amt)` chunked reads with an internal cursor (backward compatible with existing bare `.read()` callers) plus header simulation. Added red-first tests for: BundlerError after partial removal reporting partial state, BundlerError with zero removals reporting no changes, --from oversized-Content-Length rejection, --from oversized-streamed-body-without-Content-Length rejection, and the same two cases for the catalog install path (asserting no orphan directory/registry mutation on rejection). tests/integration/test_bundler_install_flow.py: 17 passed tests/test_workflows.py: 485 passed tests -q: 3992 passed, 110 skipped ruff check: clean on all touched files Assisted-by: GitHub Copilot (model: Claude Sonnet 5, autonomous) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fix temp-file leak in workflow add --from and strengthen size-limit test assertions workflow_add's --from download path opened a NamedTemporaryFile(delete=False) -- which creates the file on disk immediately -- then wrote the size-limited response body before assigning `tmp_path`. If `_read_response_within_limit` raised (oversized declared Content-Length, or an over-cap streamed body with no/understated Content-Length), the exception propagated out of the `with` block before `tmp_path` was ever set, so the outer except handler had no path to clean up: a 0-byte `.yml` temp file was left behind permanently on every rejected/failed --from download. Fixed by assigning `tmp_path` immediately after the file is opened (before the size-limited read/write), and unlinking it in the except branch when set. Normal post-download cleanup in the existing `finally: tmp_path.unlink(missing_ok=True)` is unchanged. Verified (not assumed) the catalog install path has no equivalent leak: it writes the response bytes directly to `workflow_file` inside `workflow_dir` (no separate temp file), and any read/size-limit failure is already caught by the existing `except Exception: _cleanup_failed_install()` handler, which correctly restores a reinstalled file or removes a freshly-created directory. While investigating, found the previous round's 4 size-limit tests were false positives: `_read_response_within_limit`'s `max_bytes` parameter had its default bound to `_MAX_WORKFLOW_YAML_BYTES` at function-definition time, so monkeypatching the module attribute in tests had no effect on the function's actual behavior -- the tests were passing because the oversized mock bodies failed downstream YAML/id validation instead of the size check. Fixed by resolving `max_bytes` from the module attribute at call time (default `None`, resolved inside the function body) so tests can actually override the effective limit, and strengthened all 4 tests' assertions to match the specific size-limit error text (whitespace-collapsed to tolerate Rich's line-wrapping), so they now prove the real code path fires. Tests: added 2 red-first regression tests (oversized-streamed-body and oversized-Content-Length --from downloads leave no leftover temp file, verified against a scratch tempfile.tempdir), confirmed red (real 0-byte file found) before the fix and green after. Strengthened the pre-existing 4 --from/catalog size-limit tests to assert on the actual error message instead of generic exit-code/non-empty-output checks. tests/test_workflows.py: 487 passed tests -k bundler: 186 passed tests -q: 3994 passed, 110 skipped ruff check: clean on all touched files Assisted-by: GitHub Copilot (model: Claude Sonnet 5, autonomous) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Harden workflow install/remove transactions with atomic staging Addresses 5 Copilot review findings on HEAD b8269c8, all centered on transaction integrity around workflow install/remove/registry writes, following the atomic_write_json pattern already used in _utils.py: 1. WorkflowRegistry.save() now preserves the existing registry file's mode (e.g. 0640/0644) across a save instead of silently downgrading it to mkstemp's 0600 default; a brand-new registry still gets the secure 0600 default. 2. workflow_remove now stages the install directory out of the way via an atomic rename *before* the registry write, rather than deleting it directly with shutil.rmtree after the registry already claims it removed. This closes a real data-integrity gap: a partially-failed rmtree could no longer leave a damaged directory re-marked "installed" by the old manual restore-after-rmtree-failure code (now deleted -- it's structurally impossible to need it). A registry-write failure renames the staged directory back (guarded, with an explicit warning if the restore-back rename itself fails); a registry-write success is durable, so a later failure to delete the staged directory is now a warning (exit 0), not a contradictory "Error: Failed to remove" (exit 1) that used to claim failure while the registry already recorded success. 3. Local (--dev/--from/plain path) and catalog install/reinstall now write new content to a same-directory staging file and commit it onto the destination workflow.yml via a single atomic swap, instead of writing/downloading directly into the destination file. A prior file (reinstall) is renamed aside rather than overwritten in place, so it can be restored via rename -- never a content rewrite -- if registry.add() subsequently fails; a rollback failure is now explicitly reported as a warning instead of escaping unguarded and masking the original clean error. This also removes the need to read the prior file's bytes into memory before installing (that read-before-write step and its failure mode are now unreachable), and both local and catalog installs share the same four small helpers (_stage_workflow_file / _commit_workflow_file / _discard_staged_workflow_file / _rollback_committed_workflow_file, plus guarded wrappers) rather than duplicating the logic. 4. Updated a stale comment (workflow_run's ownership-guard rationale) that still described WorkflowRegistry._load() as silently substituting an empty registry; it now fails closed by raising OSError, which the comment now states plainly. Tests: rewrote the two workflow_remove tests whose assertions encoded the old (incoherent) rmtree-then-restore contract to instead prove the new stage-then-commit contract (post-registry-success cleanup failure is a warning+exit 0; pre-registry-success stage-restore failure is guarded and escapes markup correctly). Rewrote the local/catalog "backup read failure" tests, which tested a step the new design no longer performs, into "restore-rename failure" tests proving the new guarded rollback boundary. Added registry file-mode preservation tests. All other existing install/remove/reinstall tests (save-failure cleanup, pre-existing-empty-dir handling, early-failure-during- reinstall parametrized cases, Rich markup escaping) continue to pass unmodified against the new implementation. Verified via GraphQL that all 5 threads are current (not outdated/ resolved) before fixing. Full suite: 3996 passed, 110 skipped. Ruff clean on all touched files. Assisted-by: GitHub Copilot (model: Claude Sonnet 5, autonomous) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Discard reinstall backup file after registry.add() succeeds _commit_workflow_file() renames a prior workflow.yml aside to workflow.yml.bak so it can be restored if registry.add() subsequently fails. Neither the local install/reinstall path nor the catalog install/reinstall path ever cleaned up that backup after a successful registry.add() -- every successful reinstall permanently left a workflow.yml.bak sibling, which later reinstalls would silently overwrite/re-orphan. Add a shared _discard_committed_backup_file() helper, called from both success paths right after registry.add() durably succeeds (and before the final "installed" message, preserving output ordering). A fresh install (backup_file is None) is a no-op. A cleanup failure is reported as a warning (exit 0), not a failure, since the install itself already succeeded -- consistent with workflow_remove's post-commit cleanup warning semantics. Add red-first regression tests proving: (1) successful local reinstall leaves no workflow.yml.bak sibling, (2) successful catalog reinstall leaves no workflow.yml.bak sibling, (3) a cleanup failure on the backup file after a successful reinstall reports a warning and still exits 0 with the registry correctly reflecting the new install. Assisted-by: GitHub Copilot (model: Claude Sonnet 5, autonomous) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Clean up freshly-created dest_dir when staging mkstemp fails _stage_workflow_file() does dest_dir.mkdir(parents=True, exist_ok=True) then tempfile.mkstemp(dir=dest_dir, ...). For a fresh install (no prior directory), if mkdir succeeds but mkstemp then raises (disk full/EMFILE/quota), the exception previously propagated straight past both the local-install and catalog-install call sites without any cleanup, leaving the newly-created empty workflow directory orphaned on disk with no error indicating why. Fix at the shared _stage_workflow_file() boundary instead of duplicating cleanup at each call site: track whether this call created dest_dir: on a mkstemp failure, remove that directory via a guarded rmdir (never a broad rmtree, so any concurrently written content would be left untouched) before re-raising the original OSError unchanged. A pre-existing (reinstall) dest_dir is never touched by this cleanup, and a cleanup failure is reported as its own warning without masking the original error. Add red-first regression tests proving: a fresh local install (--dev, plain local path, --from) and a fresh catalog install both clean up the orphaned directory on a simulated mkstemp failure, and a reinstall over a pre-existing directory is left untouched by the same failure. Assisted-by: GitHub Copilot (model: Claude Sonnet 5, autonomous) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fix installed-workflow ownership/disabled bypass and resume enforcement Address 3 current Copilot review findings on the disabled-workflow guard in `workflow run`/`workflow resume`: - The lexical `.specify/workflows/` ownership scan stopped at the first match scanning from the start of the path. A nested project living beneath an outer installed workflow's own directory tree (reusing the same segment names) was attributed to the wrong (outer) workflow and ID, gating the run on an unrelated workflow's disabled state. `_scan_for_workflow_owner` now scans from the end so the nearest (innermost) owner always wins. - A path with no `.specify/workflows` segments of its own (e.g. `/tmp/alias.yml`) that is itself a symlink resolving *into* installed storage bypassed the disabled check entirely, since only the raw lexical path was inspected. `_resolve_installed_workflow_ownership` now additionally resolves the real path when the lexical scan finds no owner and re-runs the same scan against it, so an outward-pointing alias into a disabled workflow is caught too. Genuinely standalone external files (no symlink anywhere on the path) are unaffected. - `workflow resume` bypassed the disabled check altogether: engine.resume() replays a persisted run directly from disk with no registry awareness. RunState now optionally persists `installed_workflow_id` and `installed_registry_root` at run start (set by workflow_run when the source resolved to an installed ID); `workflow_resume` pre-loads the run state and re-checks the registry's *current* disabled state before calling engine.resume(), mirroring workflow_run's own guard. Both new fields default to None via RunState.load()'s `.get()`, so runs from a direct/non-installed source, and any run persisted before this schema addition, resume exactly as before. The ownership-mapping logic (previously inlined in workflow_run) is extracted into `_resolve_installed_workflow_ownership` / `_scan_for_workflow_owner` so both the lexical and resolved-path cases share the same scan and the existing inward-symlink-component refusal. Assisted-by: GitHub Copilot (model: Claude Sonnet 5, autonomous) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Guard --from temp cleanup; drop redundant update rollback; mark POSIX-only tests Two more current Copilot review findings, both in workflow_add/update: - `workflow add --from`'s `finally: tmp_path.unlink(missing_ok=True)` ran unguarded after `_validate_and_install_local` had already committed the file and registry entry (success) or already raised its own clean `typer.Exit` (failure). An OSError from that cleanup unlink would surface as an unhandled failure even though the install itself succeeded. It is now wrapped in try/except OSError, printing a neutral warning that doesn't claim success or failure (the finally runs on both outcomes) instead of propagating. - `workflow_update`'s per-item loop performed its own outer backup (`wf_file.read_bytes()`) and restore (`wf_file.write_bytes(backup)`) around `_install_workflow_from_catalog`, which is itself fully transactional (staged download, atomic rename-based commit, its own rollback on registry failure) and never leaves a raw OSError or a partially-written workflow.yml. The outer restore was therefore dead weight for its stated purpose, and — being an unguarded byte-level write — was itself an unnecessary place a second failure could truncate an already-safely-preserved file. Removed; the loop now only records success/failure. Also marks 3 registry-save file-mode tests (`test_registry_save_preserves_existing_file_mode`, `test_registry_save_on_new_registry_uses_secure_default_mode`, `test_registry_save_failure_preserves_file_on_disk`) as POSIX-only via the repo's existing `skipif(sys.platform == "win32", ...)` pattern, since they assert exact POSIX permission bits that don't hold on Windows. Assisted-by: GitHub Copilot (model: Claude Sonnet 5, autonomous) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Report possible partial changes on zero-removed bundle removal failure The final Copilot review finding: `remove_bundle`'s zero-removed-components error message claimed "No components were removed." even when the failing installer component may have deleted files before raising -- prior review rounds already established that DefaultPrimitiveInstaller's removal paths are not atomic and can leave partial filesystem changes despite raising before `result.uninstalled` is populated. The zero-count message is now a conservative caution ("...but the failing component may have made partial changes before raising, so the project may be partially uninstalled.") instead of an unconditional claim of no side effects. The >0-removed path (which already reports the confirmed partial list) is unchanged. Assisted-by: GitHub Copilot (model: Claude Sonnet 5, autonomous) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fix workflow resume disabled-check bypass after project move/rename RunState.installed_registry_root previously persisted the creation-time absolute project path unconditionally whenever a run belonged to an installed workflow. After the whole project directory was renamed or moved, workflow_resume would open a WorkflowRegistry at that now nonexistent path, get back an empty/default registry, and silently skip the disabled-workflow check -- a paused run for a disabled workflow could be resumed successfully from the new location. Fix persists installed_registry_root only when the owning root genuinely differs from the current project_root (true cross-project direct-file- source invocations). The common same-project case now persists None and is re-derived from the live project_root at resume time via a new _resolve_run_owner_root() helper, which also falls back to project_root if a stored root no longer exists on disk -- covering both the common case transparently surviving project moves and the cross-project case degrading safely if its owner project vanishes, rather than silently skipping the disabled check. Backward compatible: state files missing the new fields, and states with a still-existing distinct cross-project root, behave unchanged. Added regression tests: - resume blocked after project moved then disabled at new location - resume still works after project moved while workflow stays enabled - cross-project registry root is still correctly honored when it exists - resume falls back to current project's registry when a stored cross-project root no longer exists Assisted-by: GitHub Copilot (model: Claude Sonnet 5, autonomous) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fix silenced cleanup failures and malformed run-state type validation Four fixes from Copilot review on HEAD 4f24735: 1. _discard_staged_workflow_file's fresh-install directory removal used shutil.rmtree(dest_dir, ignore_errors=True), so a genuine cleanup failure there could never reach _safe_discard_staged_workflow_file's warning -- an orphaned directory was left behind with zero report. Now removes only if dest_dir still exists and lets a real OSError propagate to the existing safe wrapper, which warns while the already-printed original install error remains primary. 2. _rollback_committed_workflow_file's fresh-install directory removal (post registry.add() failure) had the same ignore_errors=True gap; fixed identically so _safe_rollback_committed_workflow_file's warning can actually fire. 3. In the --from download-failure branch, tmp_path.unlink(missing_ok= True) was unguarded: if it raised (e.g. read-only tempdir), it replaced the original "Failed to download workflow" error with a raw unhandled OSError instead of a clean typer.Exit. Now guarded exactly like the later post-install finally cleanup: a cleanup failure prints a warning and the original download error is still reported cleanly. 4. RunState.load() trusted installed_workflow_id/installed_registry_root straight out of state.json with no type validation. A malformed value (int/list/dict/bool instead of str-or-null) would crash deep inside _resolve_run_owner_root or the registry lookup (TypeError building a Path, unhashable dict/list as a mapping key) instead of failing cleanly. Both fields are now validated as str | None during load, raising a clear ValueError that workflow_resume's existing ValueError boundary already converts into a clean CLI error with no traceback. Valid values (including the empty-string fallback already handled by _resolve_run_owner_root) continue to load unchanged. Added red-first regression tests for each: staged-discard cleanup warning, rollback cleanup warning, download-failure cleanup-vs-original- error precedence, and parameterized malformed/valid run-state field coverage. Assisted-by: GitHub Copilot (model: Claude Sonnet 5, autonomous) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Add ValueError boundary to workflow status single-run lookup QUALITY re-review flagged that fa410d3's new RunState.load() type validation (malformed installed_workflow_id/installed_registry_root raising ValueError) leaked as a raw unhandled traceback through `workflow status `, which only caught FileNotFoundError. `workflow resume` already had the matching ValueError boundary. Adds an `except ValueError as exc: console.print(f"[red]Error:[/red] {exc}"); raise typer.Exit(1)` clause mirroring resume's exact pattern (unescaped interpolation, consistent with the existing convention at every other ValueError boundary in this file). FileNotFoundError behavior and the no-run-id list-all-runs path are unchanged. Added parametrized regression covering malformed installed_workflow_id/ installed_registry_root (int/list) via `workflow status`, plus regressions locking in the unaffected FileNotFoundError and no-run-id list-path behaviors. Assisted-by: GitHub Copilot (model: Claude Sonnet 5, autonomous) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: bound workflow step downloads Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: preserve workflow reinstall state Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: fail closed on workflow registry state Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: make workflow installs transactional Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: close workflow transaction races Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: clean up failed workflow transactions Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: clean up workflow removal state Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: guard workflow update transactions Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: harden workflow lifecycle edge cases Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: verify installed workflow ownership Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: isolate workflow rollback cleanup Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: preserve unique workflow backups Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: bind workflow staging to file descriptors Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: fail closed on corrupt workflow registry Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: bind workflow ownership and source identity Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(workflows): harden redirects and Windows tests Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(workflows): restore staged removals on serialization errors Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(workflows): preserve state across interrupted writes Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(workflows): harden resume ownership checks Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(workflows): validate persisted run state Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(workflows): validate origin and release metadata Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- docs/reference/workflows.md | 29 +- src/specify_cli/_github_http.py | 35 +- src/specify_cli/bundler/lib/yamlio.py | 52 +- src/specify_cli/bundler/services/installer.py | 42 +- src/specify_cli/workflows/_commands.py | 1658 +++++- src/specify_cli/workflows/catalog.py | 175 +- src/specify_cli/workflows/engine.py | 123 +- tests/contract/test_bundle_cli.py | 50 +- .../integration/test_bundler_install_flow.py | 210 +- tests/test_github_http.py | 38 + tests/test_presets.py | 49 +- tests/test_workflows.py | 4999 ++++++++++++++++- 12 files changed, 7193 insertions(+), 267 deletions(-) diff --git a/docs/reference/workflows.md b/docs/reference/workflows.md index 16bbe0893e..8bd4a8778d 100644 --- a/docs/reference/workflows.md +++ b/docs/reference/workflows.md @@ -86,8 +86,30 @@ Lists workflows installed in the current project. specify workflow add ``` +| Option | Description | +| --------------- | ------------------------------------------------------ | +| `--dev` | Install from a local workflow YAML file or directory | +| `--from ` | Install from a custom URL (`` names the expected workflow ID) | + Installs a workflow from the catalog, a URL (HTTPS required), or a local file path. +## Update Workflows + +```bash +specify workflow update [workflow_id] +``` + +Updates one installed catalog workflow — or all of them when no ID is given — to the latest catalog version. Prompts for confirmation and keeps the installed copy if a download or validation fails. + +## Enable or Disable a Workflow + +```bash +specify workflow enable +specify workflow disable +``` + +Disabled workflows stay installed and listed (marked `[disabled]`) but refuse to run until re-enabled. + ## Remove a Workflow ```bash @@ -102,9 +124,10 @@ Removes an installed workflow from the project. specify workflow search [query] ``` -| Option | Description | -| ------- | --------------- | -| `--tag` | Filter by tag | +| Option | Description | +| ---------- | ----------------- | +| `--tag` | Filter by tag | +| `--author` | Filter by author | Searches all active catalogs for workflows matching the query. diff --git a/src/specify_cli/_github_http.py b/src/specify_cli/_github_http.py index 1cfb18f03f..aa8223bcad 100644 --- a/src/specify_cli/_github_http.py +++ b/src/specify_cli/_github_http.py @@ -24,6 +24,7 @@ "api.github.com", "codeload.github.com", }) +_MAX_RELEASE_METADATA_BYTES = 5 * 1024 * 1024 def build_github_request(url: str) -> urllib.request.Request: @@ -68,6 +69,8 @@ def resolve_github_release_asset_api_url( open_url_fn: Callable, timeout: int = 60, github_hosts: tuple[str, ...] = (), + redirect_validator: Callable[[str, str], None] | None = None, + max_metadata_bytes: int = _MAX_RELEASE_METADATA_BYTES, ) -> Optional[str]: """Resolve a GitHub release browser-download URL to its REST API asset URL. @@ -91,6 +94,8 @@ def resolve_github_release_asset_api_url( authenticated release-metadata lookup. timeout: Per-request timeout in seconds. github_hosts: Host patterns to treat as GitHub Enterprise Server. + redirect_validator: Optional policy applied to metadata redirects. + max_metadata_bytes: Maximum release-metadata response size. """ import json import urllib.error @@ -149,13 +154,33 @@ def _is_asset_path(segments: list[str]) -> bool: release_url = f"{api_base}/repos/{owner}/{repo}/releases/tags/{encoded_tag}" try: - with open_url_fn(release_url, timeout=timeout) as response: - release_data = json.loads(response.read()) - except (urllib.error.URLError, json.JSONDecodeError): + open_kwargs = {"timeout": timeout} + if redirect_validator is not None: + open_kwargs["redirect_validator"] = redirect_validator + with open_url_fn(release_url, **open_kwargs) as response: + raw_release_data = response.read(max_metadata_bytes + 1) + if len(raw_release_data) > max_metadata_bytes: + raise ValueError("GitHub release metadata exceeds size limit") + release_data = json.loads(raw_release_data) + except ( + urllib.error.URLError, + json.JSONDecodeError, + TypeError, + ValueError, + ): return None - for asset in release_data.get("assets", []): - if asset.get("name") == asset_name and asset.get("url"): + if not isinstance(release_data, dict): + return None + assets = release_data.get("assets", []) + if not isinstance(assets, list): + return None + for asset in assets: + if ( + isinstance(asset, dict) + and asset.get("name") == asset_name + and asset.get("url") + ): return str(asset["url"]) return None diff --git a/src/specify_cli/bundler/lib/yamlio.py b/src/specify_cli/bundler/lib/yamlio.py index e4cd538a09..fc90580275 100644 --- a/src/specify_cli/bundler/lib/yamlio.py +++ b/src/specify_cli/bundler/lib/yamlio.py @@ -10,6 +10,8 @@ import json import os import re +import stat +import tempfile from pathlib import Path, PurePosixPath from typing import Any @@ -87,17 +89,63 @@ def loads_json(text: str, *, origin: str = "") -> Any: def dump_json(path: Path, data: Any, *, within: Path | None = None) -> Path: - """Write *data* as pretty JSON to *path* (optionally confined to *within*).""" + """Atomically write pretty JSON to *path* (optionally confined to *within*).""" path = Path(path) if within is not None: path = ensure_within(within, path) + fd = -1 + temp_path: Path | None = None try: path.parent.mkdir(parents=True, exist_ok=True) - with path.open("w", encoding="utf-8") as handle: + fd, temp_name = tempfile.mkstemp( + dir=path.parent, + prefix=f".{path.name}.", + suffix=".tmp", + ) + temp_path = Path(temp_name) + with os.fdopen(os.dup(fd), "w", encoding="utf-8") as handle: json.dump(data, handle, indent=2, sort_keys=False) handle.write("\n") + + try: + if path.exists(): + existing = path.stat(follow_symlinks=False) + if stat.S_ISREG(existing.st_mode) and hasattr(os, "fchmod"): + os.fchmod(fd, stat.S_IMODE(existing.st_mode)) + if stat.S_ISREG(existing.st_mode) and hasattr(os, "fchown"): + try: + os.fchown(fd, existing.st_uid, existing.st_gid) + except PermissionError: + pass + except OSError: + pass + + staged = os.stat(temp_path, follow_symlinks=False) + opened = os.fstat(fd) + if ( + not stat.S_ISREG(staged.st_mode) + or staged.st_dev != opened.st_dev + or staged.st_ino != opened.st_ino + ): + raise OSError("staged JSON file changed before commit") + + os.close(fd) + fd = -1 + os.replace(temp_path, path) + temp_path = None except OSError as exc: raise BundlerError(f"Could not write {path}: {exc}") from exc + finally: + if fd >= 0: + try: + os.close(fd) + except OSError: + pass + if temp_path is not None: + try: + temp_path.unlink(missing_ok=True) + except OSError: + pass return path diff --git a/src/specify_cli/bundler/services/installer.py b/src/specify_cli/bundler/services/installer.py index 3e61ded040..7c5abf84db 100644 --- a/src/specify_cli/bundler/services/installer.py +++ b/src/specify_cli/bundler/services/installer.py @@ -187,19 +187,41 @@ def remove_bundle( still_needed = components_still_needed(records, exclude_bundle_id=bundle_id) result = InstallResult(bundle_id=bundle_id) + remove_attempted = False - for component in target.contributed_components: - key = (component.kind, component.id) - if key in still_needed: - result.skipped.append(component) - continue - if installer.is_installed(project_root, component): - installer.remove(project_root, component) - result.uninstalled.append(component) + try: + for component in target.contributed_components: + key = (component.kind, component.id) + if key in still_needed: + result.skipped.append(component) + continue + if installer.is_installed(project_root, component): + remove_attempted = True + installer.remove(project_root, component) + result.uninstalled.append(component) + save_records(project_root, remove_record(records, bundle_id)) + except Exception as exc: # noqa: BLE001 + if result.uninstalled: + detail = ( + f"{len(result.uninstalled)} component(s) were already removed " + "before this failure; the bundle record was left unchanged, " + "so the project may be partially uninstalled." + ) + elif remove_attempted: + detail = ( + "No components were removed, but the failing component may " + "have made partial changes before raising, so the project " + "may be partially uninstalled." + ) else: - result.skipped.append(component) + detail = ( + "No components were removed and no removal was attempted; " + "the bundle record was left unchanged." + ) + raise BundlerError( + f"Failed to remove bundle '{bundle_id}': {exc}. {detail}" + ) from exc - save_records(project_root, remove_record(records, bundle_id)) return result diff --git a/src/specify_cli/workflows/_commands.py b/src/specify_cli/workflows/_commands.py index e1d29b47d5..9ab199c023 100644 --- a/src/specify_cli/workflows/_commands.py +++ b/src/specify_cli/workflows/_commands.py @@ -58,6 +58,96 @@ def _error_console(json_output: bool): return err_console if json_output else console +def _open_workflow_registry(project_root: Path, out=None): + """Construct a WorkflowRegistry, exiting cleanly on an unreadable file. + + WorkflowRegistry fails closed (raises OSError) at construction when its + file can't be read, rather than falling back to an empty registry a + caller could mistake for "nothing installed". Every CLI command that + opens a registry needs this same clean-error boundary. + """ + from .catalog import WorkflowRegistry + + try: + return WorkflowRegistry(project_root) + except OSError as exc: + (out or console).print( + f"[red]Error:[/red] Failed to read workflow registry: {_escape_markup(str(exc))}" + ) + raise typer.Exit(1) + + +def _require_enabled_workflow( + registry_root: Path, workflow_id: str, out: Any +) -> bool: + """Fail closed for corrupted or explicitly disabled registry entries.""" + metadata = _open_workflow_registry(registry_root, out).get(workflow_id) + if metadata is not None and not isinstance(metadata, dict): + out.print( + f"[red]Error:[/red] Registry entry for " + f"'{_escape_markup(workflow_id)}' is corrupted" + ) + raise typer.Exit(1) + if isinstance(metadata, dict) and not metadata.get("enabled", True): + out.print( + f"[red]Error:[/red] Workflow '{_escape_markup(workflow_id)}' is disabled. " + f"Enable with: specify workflow enable {_escape_markup(workflow_id)}" + ) + raise typer.Exit(1) + return metadata is not None + + +def _path_has_symlink_component(path: Path) -> bool: + """Return whether any component of an absolute path is a symlink.""" + absolute = Path(os.path.abspath(path)) + current = Path(absolute.anchor) + for part in absolute.parts[1:]: + current /= part + if current.is_symlink(): + return True + return False + + +def _same_existing_path(left: Path, right: Path) -> bool: + """Return whether two existing paths identify the same filesystem entry.""" + try: + return os.path.samefile(left, right) + except OSError: + return left == right + + +def _resolve_run_owner_root( + installed_registry_root: str | None, project_root: Path +) -> Path: + """Determine which project's registry gates resuming a run. + + ``installed_registry_root`` is only ever persisted when the run's + installed workflow genuinely belongs to a *different* project than the + one whose ``runs/`` directory holds this run's own state (a direct + external workflow-file invocation) -- see ``workflow_run``. The common + case (an installed workflow run from its own project) stores ``None``, + so a later project rename/move is transparently picked up here by + falling back to the *current* ``project_root`` instead of a stale + absolute path baked in at run start. + + A persisted cross-project root that no longer exists cannot be safely + rediscovered and must fail closed instead of consulting the unrelated + project that happens to store the run state. + """ + if installed_registry_root: + candidate = Path(installed_registry_root) + if ( + candidate.is_absolute() + and not _path_has_symlink_component(candidate) + and candidate.is_dir() + ): + return candidate + raise ValueError( + "Installed workflow owner is unavailable; cannot safely resume" + ) + return project_root + + def _parse_input_values( input_values: list[str] | None, *, json_output: bool = False ) -> dict[str, Any]: @@ -104,15 +194,266 @@ def _reject_unsafe_workflow_storage(project_root: Path) -> None: ) +def _scan_for_workflow_owner(parts: tuple[str, ...]) -> int | None: + """Find the *nearest* (innermost) ``.specify/workflows/`` owner in + *parts*, scanning from the end of the path. + + Scanning from the end (rather than stopping at the first match from the + start) matters for a project nested beneath an unrelated outer path that + happens to reuse the same ``.specify``/``workflows`` segment names: the + first-from-start match would pick the outer directory and the wrong + workflow ID, silently missing the real (inner) owner's disabled check. + + Returns the index of the owning ``.specify`` segment, or ``None`` if no + owner segment is present. + """ + for i in range(len(parts) - 3, -1, -1): + if ( + parts[i].casefold() == ".specify" + and parts[i + 1].casefold() == "workflows" + ): + return i + return None + + +def _expand_first_symlink_target(path: Path) -> Path | None: + """Expand one symlink component while preserving the remaining path.""" + parts = path.parts + current = Path(path.anchor) if path.is_absolute() else Path() + start = 1 if path.is_absolute() else 0 + for index in range(start, len(parts)): + current = current / parts[index] + if not current.is_symlink(): + continue + try: + target = Path(os.readlink(current)) + except OSError: + return None + if not target.is_absolute(): + target = current.parent / target + expanded = target.joinpath(*parts[index + 1 :]) + return Path(os.path.normpath(str(expanded.absolute()))) + return None + + +def _resolve_installed_workflow_ownership( + source_path: Path, err +) -> tuple[Path | None, str | None]: + """Map a direct ``workflow.yml`` *source_path* back to the installed + workflow (``registry_root``, ``registered_id``) it belongs to, if any. + + A registered path can point at installed storage three ways, all of + which must receive the same registry disabled-check: + + 1. Lexically: the path's own (symlink-preserving) segments identify + ``.specify/workflows/`` -- collapsing ``..``/``.`` but + never resolving symlinks, so a symlinked ``workflow.yml`` leaf (or + symlinked ```` directory) inside the owned tree is caught by the + inward-symlink refusal below rather than silently followed. + 2. Via an intermediate alias target whose lexical path identifies + ``.specify/workflows/`` before a symlinked storage ancestor is + resolved away. + 3. Via an outward-pointing alias whose fully resolved target lands + inside legitimate installed storage, even though the raw invocation + path has no ownership segments. + + Returns ``(None, None)`` when neither applies -- a genuinely standalone + external workflow file, which is allowed to run unchecked. + """ + def ownership_for(candidate: Path) -> tuple[Path, str] | None: + parts = candidate.parts + i = _scan_for_workflow_owner(parts) + if i is None: + return None + registry_root = ( + Path(*parts[:i]) if i else Path(candidate.anchor or ".") + ) + candidate_specify = Path(*parts[: i + 1]) + candidate_workflows = Path(*parts[: i + 2]) + candidate_id_dir = Path(*parts[: i + 3]) + canonical_specify = registry_root / ".specify" + canonical_workflows = canonical_specify / "workflows" + # The path-derived registry_root here may differ from the cwd's + # project_root already checked by _reject_unsafe_workflow_storage + # (e.g. this path points into another project entirely, or this + # project's own .specify is itself a symlink to an + # attacker-controlled tree) -- check it explicitly rather than + # trusting that cwd-scoped guard, and don't rely on + # WorkflowRegistry's own symlinked-parent handling as the safety + # signal here: it fails closed by raising OSError at construction + # time (see catalog.py's _load), but that surfaces as an opaque + # exception rather than this guard's clean, specific CLI error for + # the actual owning project root. + _reject_unsafe_dir(canonical_specify, ".specify") + _reject_unsafe_dir(canonical_workflows, ".specify/workflows") + _reject_unsafe_dir(candidate_specify, ".specify") + _reject_unsafe_dir(candidate_workflows, ".specify/workflows") + try: + if not os.path.samefile(candidate_specify, canonical_specify): + return None + if not os.path.samefile( + candidate_workflows, canonical_workflows + ): + return None + except OSError: + return None + registry = _open_workflow_registry(registry_root, err) + registered_id = None + for workflow_id in registry.list(): + if ( + not isinstance(workflow_id, str) + or workflow_id in _RESERVED_WORKFLOW_IDS + or not _WORKFLOW_ID_PATTERN.fullmatch(workflow_id) + ): + continue + try: + if os.path.samefile( + candidate_id_dir, + canonical_workflows / workflow_id, + ): + registered_id = workflow_id + break + except OSError: + continue + if registered_id is None: + return None + # A legitimately installed workflow's own directory tree never + # contains a symlink (workflow add/remove both refuse one at + # install time); one appearing here means the file actually loaded + # below would not be the file this ownership match is based on, so + # refuse rather than silently mismatch. + for k in range(i + 2, len(parts) + 1): + if Path(*parts[:k]).is_symlink(): + err.print( + "[red]Error:[/red] Refusing to run: " + f".specify/workflows/{_escape_markup(registered_id)} " + "contains a symlinked path component" + ) + raise typer.Exit(1) + return registry_root, registered_id + + lexical = Path(os.path.normpath(str(source_path.absolute()))) + ownership = ownership_for(lexical) + if ownership is not None: + return ownership + + # Inspect each intermediate symlink target before fully resolving it. + # Full resolution can erase .specify/workflows ownership segments when + # one of those storage directories is itself a symlink. + candidate = lexical + seen = {candidate} + for _ in range(40): + expanded = _expand_first_symlink_target(candidate) + if expanded is None or expanded in seen: + break + ownership = ownership_for(expanded) + if ownership is not None: + return ownership + seen.add(expanded) + candidate = expanded + + # A fully resolved target may still land in legitimate installed + # storage through an unrelated-looking alias. + try: + resolved = source_path.resolve(strict=False) + except (OSError, RuntimeError): + return None, None + if resolved == lexical: + # Nothing on this path is a symlink; already covered above. + return None, None + ownership = ownership_for(resolved) + return ownership if ownership is not None else (None, None) + + _WORKFLOW_ID_PATTERN = re.compile(r"^[a-z0-9](?:[a-z0-9-]*[a-z0-9])?$") _RESERVED_WORKFLOW_IDS: frozenset[str] = frozenset({"runs", "steps"}) +def _reject_insecure_download_redirect(old_url: str, new_url: str) -> None: + """Reject insecure redirects before they are followed.""" + import urllib.error + from ipaddress import ip_address + from urllib.parse import urlparse + + def _is_loopback_http(url: str) -> bool: + parsed = urlparse(url) + if parsed.scheme != "http": + return False + host = parsed.hostname or "" + if host == "localhost": + return True + try: + return ip_address(host).is_loopback + except ValueError: + return False + + if urlparse(new_url).scheme == "https": + return + if _is_loopback_http(old_url) and _is_loopback_http(new_url): + return + raise urllib.error.URLError( + "redirect target must use HTTPS; loopback HTTP may only redirect from loopback HTTP" + ) + + +# Workflow YAML definitions are small step/metadata text, not binaries, so +# this is generous headroom against a malicious or misbehaving server -- not +# a ceiling any legitimate workflow definition should ever approach. +_MAX_WORKFLOW_YAML_BYTES = 5 * 1024 * 1024 # 5 MiB +_DOWNLOAD_CHUNK_SIZE = 65536 + + +def _read_response_within_limit(response, max_bytes: int | None = None) -> bytes: + """Read *response* fully, enforcing *max_bytes* via bounded streaming. + + A ``Content-Length`` header is checked up front to fail fast, but it is + never trusted alone: the actual bytes read are also counted as they + stream in, so a chunked or ``Content-Length``-less response that lies + about (or omits) its size still cannot exceed the limit. + + ``max_bytes`` defaults to ``None`` (resolved to the module-level + ``_MAX_WORKFLOW_YAML_BYTES`` at call time, not at function-definition + time) so tests can override the effective limit via monkeypatching the + module attribute. + """ + if max_bytes is None: + max_bytes = _MAX_WORKFLOW_YAML_BYTES + content_length = None + getheader = getattr(response, "getheader", None) + if callable(getheader): + try: + raw_length = getheader("Content-Length") + except Exception: + raw_length = None + if raw_length is not None: + try: + content_length = int(raw_length) + except (TypeError, ValueError): + content_length = None + if content_length is not None and content_length > max_bytes: + raise ValueError( + f"response declared {content_length} bytes, exceeding the " + f"{max_bytes}-byte workflow size limit" + ) + + chunks: list[bytes] = [] + total = 0 + while True: + chunk = response.read(_DOWNLOAD_CHUNK_SIZE) + if not chunk: + break + total += len(chunk) + if total > max_bytes: + raise ValueError(f"response exceeds the {max_bytes}-byte workflow size limit") + chunks.append(chunk) + return b"".join(chunks) + + def _validate_workflow_id_or_exit(workflow_id: str) -> None: """Validate that ``workflow_id`` is a safe installed-workflow directory name.""" if ( workflow_id in _RESERVED_WORKFLOW_IDS - or not _WORKFLOW_ID_PATTERN.match(workflow_id) + or not _WORKFLOW_ID_PATTERN.fullmatch(workflow_id) ): console.print( f"[red]Error:[/red] Invalid workflow ID: {_escape_markup(repr(workflow_id))}" @@ -170,6 +511,368 @@ def _safe_workflow_id_dir(workflows_dir: Path, workflow_id: str) -> Path: return dest_dir +class _StagedWorkflowFile: + """Exclusive staging inode kept open until its atomic commit.""" + + def __init__(self, path: Path, fd: int) -> None: + self.path = path + self.fd = fd + + def _write(self, chunks) -> None: + os.lseek(self.fd, 0, os.SEEK_SET) + os.ftruncate(self.fd, 0) + for chunk in chunks: + view = memoryview(chunk) + while view: + written = os.write(self.fd, view) + if written <= 0: + raise OSError("Failed to write staged workflow file") + view = view[written:] + + def write_bytes(self, data: bytes) -> None: + self._write((data,)) + + def verify_path(self) -> None: + import stat + + try: + path_stat = self.path.stat(follow_symlinks=False) + open_stat = os.fstat(self.fd) + except OSError as exc: + raise OSError( + "Staged workflow file changed before commit" + ) from exc + if ( + not stat.S_ISREG(path_stat.st_mode) + or path_stat.st_dev != open_stat.st_dev + or path_stat.st_ino != open_stat.st_ino + ): + raise OSError("Staged workflow file changed before commit") + + def set_mode(self, mode: int) -> None: + if hasattr(os, "fchmod"): + os.fchmod(self.fd, mode) + + def close(self) -> None: + if self.fd < 0: + return + fd, self.fd = self.fd, -1 + try: + os.close(fd) + except OSError: + pass + + +def _stage_workflow_file( + dest_dir: Path, *, use_project_file_mode: bool = False +) -> _StagedWorkflowFile: + """Reserve a same-directory staging file so new/updated workflow.yml + content can be written and validated without ever touching (and risking + truncating) an existing destination file before the final atomic swap. + Shared by the local-install and catalog-install paths. + + If dest_dir did not already exist, this call creates it; if mkstemp then + fails (disk full/EMFILE/quota), the freshly-created directory is removed + again via a guarded rmdir (never a broad rmtree, so any concurrently + written content is left untouched) before the original OSError is + re-raised unchanged. A pre-existing dest_dir (reinstall) is never + touched by this cleanup. For catalog-created files, + ``use_project_file_mode`` recreates the reserved path exclusively with + mode 0666 so the process umask supplies the normal project-file mode. + The final descriptor remains open so callers write to and verify the + reserved inode rather than reopening a replaceable pathname.""" + import tempfile + + created_dir = not dest_dir.exists() + dest_dir.mkdir(parents=True, exist_ok=True) + fd = -1 + staged_file: Path | None = None + try: + fd, tmp_name = tempfile.mkstemp(dir=dest_dir, prefix=".workflow.yml.", suffix=".tmp") + staged_file = Path(tmp_name) + if use_project_file_mode: + os.close(fd) + fd = -1 + staged_file.unlink() + flags = os.O_RDWR | os.O_CREAT | os.O_EXCL + flags |= getattr(os, "O_NOFOLLOW", 0) + fd = os.open(staged_file, flags, 0o666) + except OSError: + if fd >= 0: + try: + os.close(fd) + except OSError: + pass + if staged_file is not None: + try: + staged_file.unlink(missing_ok=True) + except OSError: + pass + if created_dir: + try: + dest_dir.rmdir() + except OSError as cleanup_exc: + console.print( + "[yellow]Warning:[/yellow] Failed to remove incomplete " + f"workflow directory: {_escape_markup(str(cleanup_exc))}" + ) + raise + assert staged_file is not None + return _StagedWorkflowFile(staged_file, fd) + + +@contextlib.contextmanager +def _workflow_install_transaction(project_root: Path): + """Serialize workflow file swaps with their registry updates.""" + from ..shared_infra import _ensure_safe_shared_directory + + lock_dir = project_root / ".specify" + try: + _ensure_safe_shared_directory( + project_root, lock_dir, context="workflow install lock directory" + ) + except ValueError as exc: + raise OSError(str(exc)) from exc + lock_file = lock_dir / ".workflow-install.lock" + if lock_file.is_symlink(): + raise OSError(f"Refusing to use symlinked workflow install lock: {lock_file}") + + flags = os.O_RDWR | os.O_CREAT + flags |= getattr(os, "O_NOFOLLOW", 0) + flags |= getattr(os, "O_CLOEXEC", 0) + fd = os.open(lock_file, flags, 0o600) + try: + if lock_file.is_symlink(): + raise OSError( + f"Refusing to use symlinked workflow install lock: {lock_file}" + ) + if os.name == "nt": + import errno + import msvcrt + import time + + if os.fstat(fd).st_size == 0: + os.write(fd, b"\0") + while True: + os.lseek(fd, 0, os.SEEK_SET) + try: + msvcrt.locking(fd, msvcrt.LK_NBLCK, 1) + break + except OSError as exc: + if exc.errno not in (errno.EACCES, errno.EDEADLK): + raise + time.sleep(0.05) + else: + import fcntl + + fcntl.flock(fd, fcntl.LOCK_EX) + yield + finally: + os.close(fd) + + +def _commit_workflow_file( + staged_file: Path | _StagedWorkflowFile, + dest_file: Path, + existed_before: bool, +) -> Path | None: + """Atomically swap ``staged_file`` onto ``dest_file``. If a prior file + existed, it is first renamed to a unique sibling (path returned) so a + later failure (e.g. registry.add()) can restore it via rename instead + of a content rewrite -- the destination is never truncated/overwritten + in place. If the second rename fails after the first succeeded, the + prior file is put back immediately so dest_file is never left simply + missing.""" + staged_path = ( + staged_file.path + if isinstance(staged_file, _StagedWorkflowFile) + else staged_file + ) + if isinstance(staged_file, _StagedWorkflowFile): + staged_file.verify_path() + if existed_before and dest_file.exists(): + import tempfile + + dest_state = dest_file.stat(follow_symlinks=False) + mode = dest_state.st_mode & 0o7777 + if isinstance(staged_file, _StagedWorkflowFile): + staged_file.set_mode(mode) + else: + staged_path.chmod(mode) + fd, backup_name = tempfile.mkstemp( + dir=dest_file.parent, + prefix=f".{dest_file.name}.", + suffix=".bak", + ) + try: + placeholder_state = os.fstat(fd) + finally: + os.close(fd) + backup_file = Path(backup_name) + try: + os.replace(dest_file, backup_file) + except BaseException as move_exc: + backup_state = None + try: + backup_state = backup_file.stat(follow_symlinks=False) + except OSError: + pass + if ( + backup_state is not None + and os.path.samestat(dest_state, backup_state) + ): + try: + os.replace(backup_file, dest_file) + except OSError as restore_exc: + raise OSError( + f"Failed to stage prior workflow ({move_exc}); failed " + f"to restore it from {backup_file} ({restore_exc}). " + f"The prior workflow remains at {backup_file}." + ) from restore_exc + elif ( + backup_state is not None + and os.path.samestat(placeholder_state, backup_state) + ): + try: + backup_file.unlink(missing_ok=True) + except OSError: + pass + raise + try: + if isinstance(staged_file, _StagedWorkflowFile): + staged_file.verify_path() + # Windows cannot replace an open file. Verify through the + # exclusive descriptor, then close immediately before rename. + staged_file.close() + os.replace(staged_path, dest_file) + except BaseException as commit_exc: + try: + os.replace(backup_file, dest_file) + except OSError as restore_exc: + raise OSError( + f"Failed to commit workflow file ({commit_exc}); failed " + f"to restore the prior workflow from {backup_file} " + f"({restore_exc}). The prior workflow remains at " + f"{backup_file}." + ) from restore_exc + raise + return backup_file + if isinstance(staged_file, _StagedWorkflowFile): + staged_file.verify_path() + staged_file.close() + os.replace(staged_path, dest_file) + return None + + +def _discard_staged_workflow_file( + staged_file: Path | _StagedWorkflowFile, + dest_dir: Path, + existed_before: bool, +) -> None: + """Clean up after a pre-commit failure (staged_file was never swapped + onto dest_file): remove the staged file, and for a fresh install (no + prior directory) remove the now-orphaned dest_dir too. A genuine + removal failure must propagate (not be swallowed) so the safe wrapper + below can warn instead of silently leaving an orphan; a dest_dir + already absent is not itself an error.""" + staged_path = ( + staged_file.path + if isinstance(staged_file, _StagedWorkflowFile) + else staged_file + ) + if isinstance(staged_file, _StagedWorkflowFile): + staged_file.close() + staged_path.unlink(missing_ok=True) + if not existed_before and dest_dir.exists(): + import errno + + try: + dest_dir.rmdir() + except OSError as exc: + # Another concurrent install may already have committed content + # into this once-fresh directory. Never recursively delete it. + if exc.errno not in (errno.ENOTEMPTY, errno.EEXIST): + raise + + +def _rollback_committed_workflow_file( + dest_file: Path, dest_dir: Path, existed_before: bool, backup_file: Path | None +) -> None: + """Undo a successful _commit_workflow_file swap after a later failure + (registry.add()): restore the prior file via rename, remove the newly + committed file for a reinstall over a pre-existing empty directory + (no backup), or remove the new file and then its directory when empty + for a fresh install. A genuine removal failure must propagate (not be + swallowed) so the safe wrapper below can warn instead of silently + leaving an orphan; a dest_dir already absent is not itself an error.""" + if backup_file is not None: + os.replace(backup_file, dest_file) + else: + dest_file.unlink(missing_ok=True) + if not existed_before and dest_dir.exists(): + import errno + + try: + dest_dir.rmdir() + except OSError as exc: + # Another installer may have staged a sibling before taking + # the transaction lock. Preserve it rather than recursively + # deleting the shared directory during this rollback. + if exc.errno not in (errno.ENOTEMPTY, errno.EEXIST): + raise + + +def _safe_discard_staged_workflow_file( + staged_file: Path | _StagedWorkflowFile, + dest_dir: Path, + existed_before: bool, +) -> None: + """Guarded wrapper: a cleanup failure must be reported, never crash or + silently mask the original install error that triggered it.""" + try: + _discard_staged_workflow_file(staged_file, dest_dir, existed_before) + except OSError as exc: + console.print( + "[yellow]Warning:[/yellow] Failed to clean up incomplete workflow " + f"install: {_escape_markup(str(exc))}" + ) + + +def _safe_rollback_committed_workflow_file( + dest_file: Path, dest_dir: Path, existed_before: bool, backup_file: Path | None +) -> None: + """Guarded wrapper: a rollback failure must be reported, never crash or + silently claim the prior workflow file was restored when it wasn't.""" + try: + _rollback_committed_workflow_file(dest_file, dest_dir, existed_before, backup_file) + except OSError as exc: + console.print( + "[yellow]Warning:[/yellow] Failed to restore prior workflow file " + f"after registry update failure: {_escape_markup(str(exc))}" + ) + + +def _discard_committed_backup_file(backup_file: Path | None) -> None: + """Once registry.add()/registry.remove() has durably succeeded after a + _commit_workflow_file() swap, the renamed-aside prior file is no longer + needed for rollback -- it must be discarded, not left as a permanent + orphan sibling that every future reinstall would silently accumulate or + clobber. A cleanup failure here must not turn an already-successful + install into a reported failure; it's reported as a warning, consistent + with workflow_remove's post-commit cleanup semantics. A fresh install + (backup_file is None) is a no-op.""" + if backup_file is None: + return + try: + backup_file.unlink(missing_ok=True) + except OSError as exc: + console.print( + "[yellow]Warning:[/yellow] Workflow installed, but its backup file " + f"could not be cleaned up: {_escape_markup(str(exc))}. Remove it " + f"manually: {_escape_markup(str(backup_file))}" + ) + + # Root helper re-fetched at call time so test monkeypatching of # `specify_cli._require_specify_project` keeps working after the move. def _require_specify_project(*args, **kwargs): @@ -348,6 +1051,32 @@ def workflow_run( engine.on_step_start = lambda sid, label: console.print(f" \u25b8 [{sid}] {label} \u2026") err = _error_console(json_output) + + registered_id: str | None = None + registry_root = project_root + if not is_file_source: + # Reject path-equivalent spellings ("align-wf/", "align-wf/.") that + # would miss the registry lookup yet still load the installed file, + # bypassing the disabled check below. + if source in _RESERVED_WORKFLOW_IDS or not _WORKFLOW_ID_PATTERN.fullmatch(source): + err.print( + f"[red]Error:[/red] Invalid workflow ID: {_escape_markup(repr(source))}" + ) + raise typer.Exit(1) + registered_id = source + else: + # A direct YAML path may still point at an installed workflow's own + # file (lexically, or via a symlinked alias pointing into installed + # storage); map it back to its owning project and ID so the + # disabled check below can't be silently bypassed. + owner_root, owner_id = _resolve_installed_workflow_ownership(source_path, err) + if owner_id is not None: + registry_root = owner_root + registered_id = owner_id + + if registered_id is not None: + _require_enabled_workflow(registry_root, registered_id, err) + try: definition = engine.load_workflow(source_path if is_file_source else source) except FileNotFoundError: @@ -362,7 +1091,7 @@ def workflow_run( if errors: err.print("[red]Workflow validation failed:[/red]") for verr in errors: - err.print(f" • {verr}") + err.print(f" • {_escape_markup(str(verr))}") raise typer.Exit(1) # Parse inputs @@ -374,7 +1103,26 @@ def workflow_run( try: with _stdout_to_stderr_when(json_output): - state = engine.execute(definition, inputs) + state = engine.execute( + definition, + inputs, + installed_workflow_id=registered_id, + # Only persist an explicit root when the installed workflow + # genuinely belongs to a *different* project than the one + # whose runs/ directory holds this run's own state (a + # direct external workflow-file invocation) -- the common + # case (an installed workflow run from its own project) + # leaves this None so resume re-derives the owning root + # from wherever the project currently is, transparently + # surviving a project rename/move instead of baking in a + # stale absolute path at run start. + installed_registry_root=( + registry_root.resolve(strict=True) + if registered_id + and not _same_existing_path(registry_root, project_root) + else None + ), + ) except ValueError as exc: err.print(f"[red]Error:[/red] {exc}") raise typer.Exit(1) @@ -416,7 +1164,7 @@ def workflow_resume( ): """Resume a paused or failed workflow run.""" from . import load_custom_steps - from .engine import WorkflowEngine + from .engine import RunState, WorkflowEngine project_root = _require_specify_project() load_custom_steps(project_root) @@ -427,6 +1175,48 @@ def workflow_resume( inputs = _parse_input_values(input_values, json_output=json_output) err = _error_console(json_output) + # Pre-load the persisted run state so a run started from an installed + # workflow that has since been disabled cannot resume unchecked -- + # engine.resume() replays the run directly from disk with no registry + # awareness at all, which would otherwise bypass the same disabled + # guard `workflow run` enforces. Runs without installed_workflow_id + # (a direct/non-installed source, or a run persisted before this field + # existed) are unaffected and resume exactly as before. + try: + pre_state = RunState.load(run_id, project_root) + except FileNotFoundError: + err.print(f"[red]Error:[/red] Run not found: {run_id}") + raise typer.Exit(1) + except ValueError as exc: + err.print(f"[red]Error:[/red] {_escape_markup(str(exc))}") + raise typer.Exit(1) + except OSError as exc: + err.print(f"[red]Resume failed:[/red] {_escape_markup(str(exc))}") + raise typer.Exit(1) + + if pre_state.installed_workflow_id is not None: + try: + owner_root = _resolve_run_owner_root( + pre_state.installed_registry_root, project_root + ) + except ValueError as exc: + err.print(f"[red]Error:[/red] {_escape_markup(str(exc))}") + raise typer.Exit(1) + _require_enabled_workflow( + owner_root, pre_state.installed_workflow_id, err + ) + elif not pre_state.installed_origin_tracked: + if _require_enabled_workflow( + project_root, pre_state.workflow_id, err + ): + pre_state.installed_workflow_id = pre_state.workflow_id + pre_state.installed_origin_tracked = True + try: + pre_state.save() + except OSError as exc: + err.print(f"[red]Resume failed:[/red] {_escape_markup(str(exc))}") + raise typer.Exit(1) + try: with _stdout_to_stderr_when(json_output): state = engine.resume(run_id, inputs or None) @@ -434,10 +1224,10 @@ def workflow_resume( err.print(f"[red]Error:[/red] Run not found: {run_id}") raise typer.Exit(1) except ValueError as exc: - err.print(f"[red]Error:[/red] {exc}") + err.print(f"[red]Error:[/red] {_escape_markup(str(exc))}") raise typer.Exit(1) except Exception as exc: - err.print(f"[red]Resume failed:[/red] {exc}") + err.print(f"[red]Resume failed:[/red] {_escape_markup(str(exc))}") raise typer.Exit(1) if json_output: @@ -478,6 +1268,9 @@ def workflow_status( except FileNotFoundError: console.print(f"[red]Error:[/red] Run not found: {run_id}") raise typer.Exit(1) + except ValueError as exc: + console.print(f"[red]Error:[/red] {_escape_markup(str(exc))}") + raise typer.Exit(1) if json_output: # Build on the shared run/resume payload so the common fields @@ -556,10 +1349,8 @@ def workflow_status( @workflow_app.command("list") def workflow_list(): """List installed workflows.""" - from .catalog import WorkflowRegistry - project_root = _require_specify_project() - registry = WorkflowRegistry(project_root) + registry = _open_workflow_registry(project_root) installed = registry.list() if not installed: @@ -570,36 +1361,61 @@ def workflow_list(): console.print("\n[bold cyan]Installed Workflows:[/bold cyan]\n") for wf_id, wf_data in installed.items(): - console.print(f" [bold]{wf_data.get('name', wf_id)}[/bold] ({wf_id}) v{wf_data.get('version', '?')}") + safe_id = _escape_markup(wf_id) + if not isinstance(wf_data, dict): + console.print(f" [yellow]Warning:[/yellow] Skipping corrupted registry entry '{safe_id}'.\n") + continue + marker = "" if wf_data.get("enabled", True) else " [red]\\[disabled][/red]" + name = _escape_markup(str(wf_data.get("name", wf_id))) + version = _escape_markup(str(wf_data.get("version", "?"))) + console.print(f" [bold]{name}[/bold] ({safe_id}) v{version}{marker}") desc = wf_data.get("description", "") if desc: - console.print(f" {desc}") + console.print(f" {_escape_markup(str(desc))}") console.print() @workflow_app.command("add") def workflow_add( source: str = typer.Argument(..., help="Workflow ID, URL, or local path"), + dev: bool = typer.Option(False, "--dev", help="Install from a local workflow YAML file or directory"), + from_url: str | None = typer.Option(None, "--from", help="Install from a custom URL"), ): """Install a workflow from catalog, URL, or local path.""" - from .catalog import WorkflowCatalog, WorkflowRegistry, WorkflowCatalogError from .engine import WorkflowDefinition project_root = _require_specify_project() - registry = WorkflowRegistry(project_root) + _open_workflow_registry(project_root) workflows_dir = project_root / ".specify" / "workflows" + # With --from, source names the expected workflow ID: validate it up + # front so a URL/path/typo fails without a network fetch. + if from_url is not None and not dev: + _validate_workflow_id_or_exit(source) # Reject a symlinked .specify / .specify/workflows before any write so an # install can't escape the project root (covers the local, URL, and # catalog branches below — all write beneath workflows_dir). _reject_unsafe_dir(project_root / ".specify", ".specify") _reject_unsafe_dir(workflows_dir, ".specify/workflows") - def _validate_and_install_local(yaml_path: Path, source_label: str) -> None: + def _validate_and_install_local( + yaml_path: Path, source_label: str, expected_id: str | None = None + ) -> None: """Validate and install a workflow from a local YAML file.""" try: - definition = WorkflowDefinition.from_yaml(yaml_path) - except (ValueError, yaml.YAMLError) as exc: - console.print(f"[red]Error:[/red] Invalid workflow YAML: {exc}") + with yaml_path.open("rb") as source_file: + source_mode = os.fstat(source_file.fileno()).st_mode & 0o7777 + source_content = source_file.read() + definition = WorkflowDefinition.from_string( + source_content.decode("utf-8") + ) + except OSError as exc: + console.print( + f"[red]Error:[/red] Failed to read workflow YAML: " + f"{_escape_markup(str(exc))}" + ) + raise typer.Exit(1) + except (UnicodeDecodeError, ValueError, yaml.YAMLError) as exc: + console.print(f"[red]Error:[/red] Invalid workflow YAML: {_escape_markup(str(exc))}") raise typer.Exit(1) # Non-string ids (e.g. unquoted ``id: 123`` or ``id: 0``) fall through # to validate_workflow below, which reports a typed error instead of @@ -618,31 +1434,144 @@ def _validate_and_install_local(yaml_path: Path, source_label: str) -> None: if errors: console.print("[red]Error:[/red] Workflow validation failed:") for err in errors: - console.print(f" \u2022 {err}") + console.print(f" \u2022 {_escape_markup(str(err))}") + raise typer.Exit(1) + + if expected_id is not None and definition.id != expected_id: + console.print( + f"[red]Error:[/red] Workflow ID in YAML ({_escape_markup(repr(definition.id))}) " + f"does not match the requested workflow ID ({_escape_markup(repr(expected_id))})." + ) raise typer.Exit(1) dest_dir = _safe_workflow_id_dir(workflows_dir, definition.id) - dest_dir.mkdir(parents=True, exist_ok=True) - import shutil - shutil.copy2(yaml_path, dest_dir / "workflow.yml") - registry.add(definition.id, { - "name": definition.name, - "version": definition.version, - "description": definition.description, - "source": source_label, - }) - console.print(f"[green]✓[/green] Workflow '{definition.name}' ({definition.id}) installed") - - # Try as URL (http/https) - if source.startswith("http://") or source.startswith("https://"): + dest_file = dest_dir / "workflow.yml" + existed_before = dest_dir.is_dir() + + try: + staged_file = _stage_workflow_file(dest_dir) + except OSError as exc: + console.print( + f"[red]Error:[/red] Failed to install workflow " + f"'{_escape_markup(definition.id)}': {_escape_markup(str(exc))}" + ) + raise typer.Exit(1) + + try: + # Write the exact bytes parsed above so a concurrent source edit + # cannot desynchronize installed content from validated metadata. + staged_file.write_bytes(source_content) + staged_file.set_mode(source_mode) + except OSError as exc: + _safe_discard_staged_workflow_file(staged_file, dest_dir, existed_before) + console.print( + f"[red]Error:[/red] Failed to install workflow " + f"'{_escape_markup(definition.id)}': {_escape_markup(str(exc))}" + ) + raise typer.Exit(1) + + try: + transaction = _workflow_install_transaction(project_root) + with transaction: + transaction_existed_before = existed_before or dest_file.exists() + transaction_registry = _open_workflow_registry(project_root) + # Commit the staged copy onto dest_file via an atomic swap. A + # prior file is renamed aside so registry failure can restore it. + try: + backup_file = _commit_workflow_file( + staged_file, dest_file, transaction_existed_before + ) + except OSError as exc: + _safe_discard_staged_workflow_file( + staged_file, dest_dir, existed_before + ) + console.print( + f"[red]Error:[/red] Failed to install workflow " + f"'{_escape_markup(definition.id)}': " + f"{_escape_markup(str(exc))}" + ) + raise typer.Exit(1) + try: + entry = { + "name": definition.name, + "version": definition.version, + "description": definition.description, + "source": source_label, + } + existing = transaction_registry.get(definition.id) + if isinstance(existing, dict) and not existing.get( + "enabled", True + ): + entry["enabled"] = False + transaction_registry.add(definition.id, entry) + except (OSError, TypeError, ValueError) as exc: + _safe_rollback_committed_workflow_file( + dest_file, + dest_dir, + transaction_existed_before, + backup_file, + ) + console.print( + f"[red]Error:[/red] Failed to update workflow registry for " + f"'{_escape_markup(definition.id)}': " + f"{_escape_markup(str(exc))}" + ) + raise typer.Exit(1) + # Registry update succeeded while the transaction lock is held. + _discard_committed_backup_file(backup_file) + except typer.Exit: + _safe_discard_staged_workflow_file( + staged_file, dest_dir, existed_before + ) + raise + except OSError as exc: + _safe_discard_staged_workflow_file(staged_file, dest_dir, existed_before) + console.print( + f"[red]Error:[/red] Failed to lock workflow install " + f"'{_escape_markup(definition.id)}': {_escape_markup(str(exc))}" + ) + raise typer.Exit(1) + console.print( + f"[green]✓[/green] Workflow '{_escape_markup(definition.name)}' " + f"({_escape_markup(definition.id)}) installed" + ) + + # Explicit local install (mirrors `extension add --dev`). --dev takes + # precedence over --from so a URL that would be ignored is never fetched. + if dev: + dev_path = Path(source).expanduser() + if dev_path.is_file() and dev_path.suffix in (".yml", ".yaml"): + _validate_and_install_local(dev_path, str(dev_path)) + return + if dev_path.is_dir(): + dev_wf_file = dev_path / "workflow.yml" + if not dev_wf_file.is_file(): + console.print(f"[red]Error:[/red] No workflow.yml found in {_escape_markup(source)}") + raise typer.Exit(1) + _validate_and_install_local(dev_wf_file, str(dev_path)) + return + console.print( + "[red]Error:[/red] --dev source must be a workflow YAML file or a " + f"directory containing workflow.yml: {_escape_markup(source)}" + ) + raise typer.Exit(1) + + # Try as URL (http/https) — either the positional source is a URL, or an + # explicit --from URL names where to fetch it (mirrors `extension add --from`). + download_url = ( + from_url + if from_url is not None + else (source if source.startswith(("http://", "https://")) else None) + ) + if download_url is not None: from ipaddress import ip_address from urllib.parse import urlparse from specify_cli.authentication.http import open_url as _open_url try: - parsed_src = urlparse(source) + parsed_src = urlparse(download_url) except ValueError: - console.print(f"[red]Error:[/red] Invalid URL: {_escape_markup(source)}") + console.print(f"[red]Error:[/red] Invalid URL: {_escape_markup(download_url)}") raise typer.Exit(1) src_host = parsed_src.hostname or "" src_loopback = src_host == "localhost" @@ -656,20 +1585,52 @@ def _validate_and_install_local(yaml_path: Path, source_label: str) -> None: console.print("[red]Error:[/red] Only HTTPS URLs are allowed, except HTTP for localhost.") raise typer.Exit(1) + if from_url is not None: + from rich.panel import Panel + + safe_url = _escape_markup(from_url) + console.print() + console.print( + Panel( + "[bold]You are installing a workflow from an external URL " + "that is not\nlisted in any of your configured workflow " + "catalogs.[/bold]\n\n" + f"URL: {safe_url}\n\n" + "Only install workflows from sources you trust.", + title="[bold yellow]⚠ Untrusted Source[/bold yellow]", + border_style="yellow", + padding=(1, 2), + ) + ) + console.print() + if not typer.confirm("Continue with installation?", default=False): + console.print("Cancelled") + raise typer.Exit(0) + from specify_cli._github_http import resolve_github_release_asset_api_url as _resolve_gh_asset from specify_cli.authentication.http import github_provider_hosts as _github_provider_hosts _wf_url_extra_headers = None _resolved_wf_url = _resolve_gh_asset( - source, _open_url, timeout=30, github_hosts=_github_provider_hosts() + download_url, + _open_url, + timeout=30, + github_hosts=_github_provider_hosts(), + redirect_validator=_reject_insecure_download_redirect, ) if _resolved_wf_url: - source = _resolved_wf_url + download_url = _resolved_wf_url _wf_url_extra_headers = {"Accept": "application/octet-stream"} import tempfile + tmp_path: Path | None = None try: - with _open_url(source, timeout=30, extra_headers=_wf_url_extra_headers) as resp: + with _open_url( + download_url, + timeout=30, + extra_headers=_wf_url_extra_headers, + redirect_validator=_reject_insecure_download_redirect, + ) as resp: final_url = resp.geturl() final_parsed = urlparse(final_url) final_host = final_parsed.hostname or "" @@ -681,20 +1642,58 @@ def _validate_and_install_local(yaml_path: Path, source_label: str) -> None: # Redirect host is not an IP literal; keep loopback as determined above. pass if final_parsed.scheme != "https" and not (final_parsed.scheme == "http" and final_lb): - console.print(f"[red]Error:[/red] URL redirected to non-HTTPS: {final_url}") + console.print( + f"[red]Error:[/red] URL redirected to non-HTTPS: {_escape_markup(final_url)}" + ) raise typer.Exit(1) with tempfile.NamedTemporaryFile(suffix=".yml", delete=False) as tmp: - tmp.write(resp.read()) + # Assign tmp_path immediately: NamedTemporaryFile(delete=False) + # creates the file on disk right away, before any bytes are + # written, so a failure in the size-limited read below must + # still be able to find and remove it. tmp_path = Path(tmp.name) + tmp.write(_read_response_within_limit(resp)) except typer.Exit: raise except Exception as exc: - console.print(f"[red]Error:[/red] Failed to download workflow: {exc}") + if tmp_path is not None: + # A cleanup failure here must never replace/mask the + # original download error below with a raw, unhandled + # OSError -- warn about it and keep going, exactly like the + # later post-install finally cleanup does. + try: + tmp_path.unlink(missing_ok=True) + except OSError as cleanup_exc: + console.print( + "[yellow]Warning:[/yellow] Could not remove temporary " + f"download file {_escape_markup(str(tmp_path))}: " + f"{_escape_markup(str(cleanup_exc))}" + ) + console.print(f"[red]Error:[/red] Failed to download workflow: {_escape_markup(str(exc))}") raise typer.Exit(1) try: - _validate_and_install_local(tmp_path, source) + # When installed via --from, the positional argument names the + # workflow the user expects — enforce it like the catalog branch. + _validate_and_install_local( + tmp_path, + download_url, + expected_id=source if from_url else None, + ) finally: - tmp_path.unlink(missing_ok=True) + # Best-effort: _validate_and_install_local may already have + # committed the file + registry entry (success) or already + # raised its own clean typer.Exit (failure) by this point -- + # either way, a cleanup OSError here must never mask that + # outcome or surface as its own unhandled failure. Warn instead, + # same as the committed-backup cleanup above. + try: + tmp_path.unlink(missing_ok=True) + except OSError as exc: + console.print( + "[yellow]Warning:[/yellow] Could not remove temporary " + f"download file {_escape_markup(str(tmp_path))}: " + f"{_escape_markup(str(exc))}" + ) return # Try as a local file/directory @@ -705,40 +1704,86 @@ def _validate_and_install_local(yaml_path: Path, source_label: str) -> None: return elif source_path.is_dir(): wf_file = source_path / "workflow.yml" - if not wf_file.exists(): - console.print(f"[red]Error:[/red] No workflow.yml found in {source}") + if not wf_file.is_file(): + console.print(f"[red]Error:[/red] No workflow.yml found in {_escape_markup(source)}") raise typer.Exit(1) _validate_and_install_local(wf_file, str(source_path)) return # Try from catalog + _install_workflow_from_catalog(project_root, workflows_dir, source) + + +def _install_workflow_from_catalog( + project_root: Path, + workflows_dir: Path, + workflow_id: str, + expected_version: str | None = None, + expected_installed_version: str | None = None, +) -> None: + """Download, validate, and register a catalog workflow. + + Shared by ``workflow add`` and ``workflow update``. Raises ``typer.Exit`` + on any failure; the registry entry is only written on full success. + ``expected_version``, when given, rejects a downloaded workflow whose + version does not match the catalog version that triggered the install. + ``expected_installed_version``, when given by ``workflow update``, aborts + if another process changes the installed source or version before commit. + """ + from .catalog import WorkflowCatalog, WorkflowCatalogError + from .engine import WorkflowDefinition + + def versions_match(actual: object, expected: str) -> bool: + from packaging import version as pkg_version + + try: + return pkg_version.Version(str(actual)) == pkg_version.Version( + expected + ) + except pkg_version.InvalidVersion: + return str(actual) == expected + + safe_wf_id = _escape_markup(workflow_id) + catalog = WorkflowCatalog(project_root) try: - info = catalog.get_workflow_info(source) + info = catalog.get_workflow_info(workflow_id) except WorkflowCatalogError as exc: - console.print(f"[red]Error:[/red] {exc}") + console.print(f"[red]Error:[/red] {_escape_markup(str(exc))}") raise typer.Exit(1) if not info: - console.print(f"[red]Error:[/red] Workflow '{source}' not found in catalog") + console.print(f"[red]Error:[/red] Workflow '{safe_wf_id}' not found in catalog") raise typer.Exit(1) if not info.get("_install_allowed", True): - console.print(f"[yellow]Warning:[/yellow] Workflow '{source}' is from a discovery-only catalog") + console.print(f"[yellow]Warning:[/yellow] Workflow '{safe_wf_id}' is from a discovery-only catalog") console.print("Direct installation is not enabled for this catalog source.") raise typer.Exit(1) workflow_url = info.get("url") if not workflow_url: - console.print(f"[red]Error:[/red] Workflow '{source}' does not have an install URL in the catalog") + console.print(f"[red]Error:[/red] Workflow '{safe_wf_id}' does not have an install URL in the catalog") + raise typer.Exit(1) + if not isinstance(workflow_url, str): + # Untrusted catalog payload; a non-string would crash urlparse below. + console.print( + f"[red]Error:[/red] Workflow '{safe_wf_id}' has a malformed install URL." + ) raise typer.Exit(1) # Validate URL scheme (HTTPS required, HTTP allowed for localhost only) from ipaddress import ip_address from urllib.parse import urlparse - parsed_url = urlparse(workflow_url) - url_host = parsed_url.hostname or "" + try: + parsed_url = urlparse(workflow_url) + url_host = parsed_url.hostname or "" + except ValueError: + console.print( + f"[red]Error:[/red] Workflow '{safe_wf_id}' has a malformed install URL." + ) + raise typer.Exit(1) is_loopback = False if url_host == "localhost": is_loopback = True @@ -750,16 +1795,33 @@ def _validate_and_install_local(yaml_path: Path, source_label: str) -> None: pass if parsed_url.scheme != "https" and not (parsed_url.scheme == "http" and is_loopback): console.print( - f"[red]Error:[/red] Workflow '{source}' has an invalid install URL. " + f"[red]Error:[/red] Workflow '{safe_wf_id}' has an invalid install URL. " "Only HTTPS URLs are allowed, except HTTP for localhost/loopback." ) raise typer.Exit(1) # Reject path traversal, symlinked , and a symlinked workflow.yml leaf # before any mkdir/download writes beneath the install directory. - workflow_dir = _safe_workflow_id_dir(workflows_dir, source) + workflow_dir = _safe_workflow_id_dir(workflows_dir, workflow_id) workflow_file = workflow_dir / "workflow.yml" + # Captured before any mkdir/download writes so every failure branch below + # can tell a fresh install from a reinstall-over-an-existing-one, + # mirroring _validate_and_install_local's existed-before-aware cleanup. + existed_before = workflow_dir.is_dir() + + try: + staged_file = _stage_workflow_file( + workflow_dir, + use_project_file_mode=not workflow_file.exists(), + ) + except OSError as exc: + console.print( + f"[red]Error:[/red] Failed to install workflow '{safe_wf_id}' from catalog: " + f"{_escape_markup(str(exc))}" + ) + raise typer.Exit(1) + try: from specify_cli.authentication.http import open_url as _open_url from specify_cli.authentication.http import github_provider_hosts as _github_provider_hosts @@ -767,14 +1829,22 @@ def _validate_and_install_local(yaml_path: Path, source_label: str) -> None: _wf_cat_extra_headers = None _resolved_workflow_url = _resolve_gh_asset( - workflow_url, _open_url, timeout=30, github_hosts=_github_provider_hosts() + workflow_url, + _open_url, + timeout=30, + github_hosts=_github_provider_hosts(), + redirect_validator=_reject_insecure_download_redirect, ) if _resolved_workflow_url: workflow_url = _resolved_workflow_url _wf_cat_extra_headers = {"Accept": "application/octet-stream"} - workflow_dir.mkdir(parents=True, exist_ok=True) - with _open_url(workflow_url, timeout=30, extra_headers=_wf_cat_extra_headers) as response: + with _open_url( + workflow_url, + timeout=30, + extra_headers=_wf_cat_extra_headers, + redirect_validator=_reject_insecure_download_redirect, + ) as response: # Validate final URL after redirects final_url = response.geturl() final_parsed = urlparse(final_url) @@ -787,82 +1857,168 @@ def _validate_and_install_local(yaml_path: Path, source_label: str) -> None: # Host is not an IP literal (e.g., a regular hostname); treat as non-loopback. pass if final_parsed.scheme != "https" and not (final_parsed.scheme == "http" and final_loopback): - if workflow_dir.exists(): - import shutil - shutil.rmtree(workflow_dir, ignore_errors=True) + _safe_discard_staged_workflow_file(staged_file, workflow_dir, existed_before) console.print( - f"[red]Error:[/red] Workflow '{source}' redirected to non-HTTPS URL: {final_url}" + f"[red]Error:[/red] Workflow '{safe_wf_id}' redirected to non-HTTPS URL: {_escape_markup(final_url)}" ) raise typer.Exit(1) - workflow_file.write_bytes(response.read()) + # Written to the staging file, never workflow_file directly, so a + # reinstall's prior working copy is never touched until the + # atomic commit below runs. + downloaded_content = _read_response_within_limit(response) + staged_file.write_bytes(downloaded_content) + except typer.Exit: + raise except Exception as exc: - if workflow_dir.exists(): - import shutil - shutil.rmtree(workflow_dir, ignore_errors=True) - console.print(f"[red]Error:[/red] Failed to install workflow '{source}' from catalog: {exc}") + _safe_discard_staged_workflow_file(staged_file, workflow_dir, existed_before) + console.print(f"[red]Error:[/red] Failed to install workflow '{safe_wf_id}' from catalog: {_escape_markup(str(exc))}") raise typer.Exit(1) - # Validate the downloaded workflow before registering + # Validate the downloaded workflow (still staged, not yet committed) + # before registering. try: - definition = WorkflowDefinition.from_yaml(workflow_file) - except (ValueError, yaml.YAMLError) as exc: - import shutil - shutil.rmtree(workflow_dir, ignore_errors=True) - console.print(f"[red]Error:[/red] Downloaded workflow is invalid: {exc}") + definition = WorkflowDefinition.from_string( + downloaded_content.decode("utf-8") + ) + except (UnicodeDecodeError, ValueError, yaml.YAMLError) as exc: + _safe_discard_staged_workflow_file(staged_file, workflow_dir, existed_before) + console.print(f"[red]Error:[/red] Downloaded workflow is invalid: {_escape_markup(str(exc))}") raise typer.Exit(1) from .engine import validate_workflow errors = validate_workflow(definition) if errors: - import shutil - shutil.rmtree(workflow_dir, ignore_errors=True) + _safe_discard_staged_workflow_file(staged_file, workflow_dir, existed_before) console.print("[red]Error:[/red] Downloaded workflow validation failed:") for err in errors: - console.print(f" \u2022 {err}") + console.print(f" \u2022 {_escape_markup(str(err))}") raise typer.Exit(1) # Enforce that the workflow's internal ID matches the catalog key - if definition.id and definition.id != source: - import shutil - shutil.rmtree(workflow_dir, ignore_errors=True) + if definition.id and definition.id != workflow_id: + _safe_discard_staged_workflow_file(staged_file, workflow_dir, existed_before) console.print( - f"[red]Error:[/red] Workflow ID in YAML ({definition.id!r}) " - f"does not match catalog key ({source!r}). " + f"[red]Error:[/red] Workflow ID in YAML ({_escape_markup(repr(definition.id))}) " + f"does not match catalog key ({_escape_markup(repr(workflow_id))}). " f"The catalog entry may be misconfigured." ) raise typer.Exit(1) - registry.add(source, { - "name": definition.name or info.get("name", source), - "version": definition.version or info.get("version", "0.0.0"), - "description": definition.description or info.get("description", ""), - "source": "catalog", - "catalog_name": info.get("_catalog_name", ""), - "url": workflow_url, - }) - console.print(f"[green]✓[/green] Workflow '{info.get('name', source)}' installed from catalog") - + # A stale or misconfigured URL can serve a different version than the + # catalog advertised; without this check `update` would report success + # while leaving the old version installed (or even downgrading). + if expected_version is not None: + if not versions_match(definition.version, expected_version): + _safe_discard_staged_workflow_file(staged_file, workflow_dir, existed_before) + console.print( + f"[red]Error:[/red] Downloaded workflow version ({_escape_markup(str(definition.version))}) " + f"does not match the catalog version ({_escape_markup(expected_version)}). " + f"The catalog entry may be stale or misconfigured." + ) + raise typer.Exit(1) -@workflow_app.command("remove") -def workflow_remove( - workflow_id: str = typer.Argument(..., help="Workflow ID to uninstall"), -): - """Uninstall a workflow.""" - from .catalog import WorkflowRegistry + try: + transaction = _workflow_install_transaction(project_root) + with transaction: + transaction_existed_before = ( + existed_before or workflow_file.exists() + ) + transaction_registry = _open_workflow_registry(project_root) + if expected_installed_version is not None: + current = transaction_registry.get(workflow_id) + if ( + not isinstance(current, dict) + or current.get("source") != "catalog" + or not versions_match( + current.get("version"), expected_installed_version + ) + ): + console.print( + f"[yellow]Warning:[/yellow] Workflow '{safe_wf_id}' " + "changed during update; rerun the command to use its " + "current source and version." + ) + raise typer.Exit(1) + # Commit the staged download onto workflow_file via an atomic + # swap. A prior file is renamed aside for registry rollback. + try: + backup_file = _commit_workflow_file( + staged_file, workflow_file, transaction_existed_before + ) + except OSError as exc: + _safe_discard_staged_workflow_file( + staged_file, workflow_dir, existed_before + ) + console.print( + f"[red]Error:[/red] Failed to install workflow " + f"'{safe_wf_id}' from catalog: {_escape_markup(str(exc))}" + ) + raise typer.Exit(1) - project_root = _require_specify_project() - workflows_dir = project_root / ".specify" / "workflows" - _validate_workflow_id_or_exit(workflow_id) + entry = { + "name": definition.name or info.get("name", workflow_id), + "version": definition.version or info.get("version", "0.0.0"), + "description": definition.description + or info.get("description", ""), + "source": "catalog", + "catalog_name": info.get("_catalog_name", ""), + "url": workflow_url, + } + # Preserve a prior disabled state across updates/reinstalls. + existing = transaction_registry.get(workflow_id) + if isinstance(existing, dict) and not existing.get( + "enabled", True + ): + entry["enabled"] = False + try: + transaction_registry.add(workflow_id, entry) + except (OSError, TypeError, ValueError) as exc: + _safe_rollback_committed_workflow_file( + workflow_file, + workflow_dir, + transaction_existed_before, + backup_file, + ) + console.print( + f"[red]Error:[/red] Failed to update workflow registry for " + f"'{_escape_markup(workflow_id)}': " + f"{_escape_markup(str(exc))}" + ) + raise typer.Exit(1) + # Registry update succeeded while the transaction lock is held. + _discard_committed_backup_file(backup_file) + except typer.Exit: + _safe_discard_staged_workflow_file( + staged_file, workflow_dir, existed_before + ) + raise + except OSError as exc: + _safe_discard_staged_workflow_file(staged_file, workflow_dir, existed_before) + console.print( + f"[red]Error:[/red] Failed to lock workflow install " + f"'{safe_wf_id}': " + f"{_escape_markup(str(exc))}" + ) + raise typer.Exit(1) + console.print( + f"[green]✓[/green] Workflow '{_escape_markup(str(info.get('name', workflow_id)))}' " + "installed from catalog" + ) - registry = WorkflowRegistry(project_root) +def _remove_workflow_locked( + project_root: Path, workflows_dir: Path, workflow_id: str +) -> Path | None: + """Stage a workflow directory and persist removal while locked.""" + registry = _open_workflow_registry(project_root) + safe_id = _escape_markup(workflow_id) if not registry.is_installed(workflow_id): - console.print(f"[red]Error:[/red] Workflow '{workflow_id}' is not installed") + console.print( + f"[red]Error:[/red] Workflow '{safe_id}' is not installed" + ) raise typer.Exit(1) - # Remove workflow files workflow_dir_unresolved = workflows_dir / workflow_id - safe_id = _escape_markup(workflow_id) if workflow_dir_unresolved.is_symlink(): console.print( f"[red]Error:[/red] Refusing to remove symlinked " @@ -875,39 +2031,301 @@ def workflow_remove( rel_parts = workflow_dir.relative_to(workflows_dir.resolve()).parts except ValueError: console.print( - f"[red]Error:[/red] Invalid workflow ID: {_escape_markup(repr(workflow_id))}" + f"[red]Error:[/red] Invalid workflow ID: " + f"{_escape_markup(repr(workflow_id))}" ) raise typer.Exit(1) if rel_parts != (workflow_id,): console.print( - f"[red]Error:[/red] Invalid workflow ID: {_escape_markup(repr(workflow_id))}" + f"[red]Error:[/red] Invalid workflow ID: " + f"{_escape_markup(repr(workflow_id))}" ) raise typer.Exit(1) if workflow_dir.exists() and not workflow_dir.is_dir(): console.print( - f"[red]Error:[/red] .specify/workflows/{safe_id} exists but is not a directory" + f"[red]Error:[/red] .specify/workflows/{safe_id} exists " + "but is not a directory" ) raise typer.Exit(1) + import tempfile + + staged_dir: Path | None = None if workflow_dir.exists(): - import shutil try: - shutil.rmtree(workflow_dir) + reserved = Path( + tempfile.mkdtemp( + prefix=f".{workflow_id}.removing-", dir=workflows_dir + ) + ) + reserved.rmdir() + os.rename(workflow_dir, reserved) + staged_dir = reserved except OSError as exc: console.print( - f"[red]Error:[/red] Failed to remove workflow directory {workflow_dir}: {exc}" + f"[red]Error:[/red] Failed to stage workflow directory " + f"{_escape_markup(str(workflow_dir))} for removal: " + f"{_escape_markup(str(exc))}" ) raise typer.Exit(1) - registry.remove(workflow_id) + try: + registry.remove(workflow_id) + except (OSError, TypeError, ValueError) as exc: + if staged_dir is not None: + try: + os.rename(staged_dir, workflow_dir) + except OSError as restore_exc: + console.print( + f"[yellow]Warning:[/yellow] Failed to restore workflow " + "directory after registry update failure; it remains " + f"staged at {_escape_markup(str(staged_dir))}: " + f"{_escape_markup(str(restore_exc))}" + ) + console.print( + f"[red]Error:[/red] Failed to update workflow registry for " + f"'{safe_id}': {_escape_markup(str(exc))}" + ) + raise typer.Exit(1) + return staged_dir + + +@workflow_app.command("remove") +def workflow_remove( + workflow_id: str = typer.Argument(..., help="Workflow ID to uninstall"), +): + """Uninstall a workflow.""" + project_root = _require_specify_project() + workflows_dir = project_root / ".specify" / "workflows" + _validate_workflow_id_or_exit(workflow_id) + safe_id = _escape_markup(workflow_id) + import shutil + try: + with _workflow_install_transaction(project_root): + staged_dir = _remove_workflow_locked( + project_root, workflows_dir, workflow_id + ) + except OSError as exc: + console.print( + f"[red]Error:[/red] Failed to lock workflow removal " + f"'{safe_id}': {_escape_markup(str(exc))}" + ) + raise typer.Exit(1) + console.print(f"[green]✓[/green] Workflow '{workflow_id}' removed") + # The registry has already durably committed the removal at this point, + # so it must stand regardless of what happens below: deleting the staged + # directory is now just cleanup, not a data-integrity concern, and a + # failure here is reported as a warning (not an error) to avoid + # contradicting the registry state that already succeeded. + if staged_dir is not None: + try: + shutil.rmtree(staged_dir) + except OSError as exc: + console.print( + f"[yellow]Warning:[/yellow] Workflow '{safe_id}' was removed, but its " + f"staged directory could not be deleted: {_escape_markup(str(exc))}. " + f"Remove it manually: {_escape_markup(str(staged_dir))}" + ) + + +@workflow_app.command("update") +def workflow_update( + workflow_id: str | None = typer.Argument(None, help="Workflow ID to update (default: all)"), +): + """Update installed workflow(s) to the latest catalog version.""" + from packaging import version as pkg_version + + from .catalog import WorkflowCatalog, WorkflowCatalogError + + project_root = _require_specify_project() + registry = _open_workflow_registry(project_root) + workflows_dir = project_root / ".specify" / "workflows" + _reject_unsafe_dir(project_root / ".specify", ".specify") + _reject_unsafe_dir(workflows_dir, ".specify/workflows") + + installed = registry.list() + if workflow_id: + if not registry.is_installed(workflow_id): + console.print(f"[red]Error:[/red] Workflow '{_escape_markup(workflow_id)}' is not installed") + raise typer.Exit(1) + targets = [workflow_id] + else: + targets = list(installed) + + if not targets: + console.print("[yellow]No workflows installed[/yellow]") + raise typer.Exit(0) + + catalog = WorkflowCatalog(project_root) + console.print("🔄 Checking for updates...\n") + + updates_available: list[dict[str, str]] = [] + checked = 0 + for wf_id in targets: + safe_id = _escape_markup(str(wf_id)) + metadata = installed.get(wf_id) + if not isinstance(metadata, dict): + console.print(f"⚠ {safe_id}: Registry entry is corrupted (skipping)") + continue + if metadata.get("source") != "catalog": + console.print(f"⚠ {safe_id}: Not installed from a catalog — re-add to update (skipping)") + continue + try: + installed_version = pkg_version.Version(str(metadata.get("version"))) + except pkg_version.InvalidVersion: + console.print( + f"⚠ {safe_id}: Invalid installed version '{_escape_markup(str(metadata.get('version')))}' in registry (skipping)" + ) + continue + try: + info = catalog.get_workflow_info(wf_id) + except WorkflowCatalogError as exc: + console.print(f"[red]Error:[/red] {_escape_markup(str(exc))}") + raise typer.Exit(1) + if not info: + console.print(f"⚠ {safe_id}: Not found in catalog (skipping)") + continue + if not info.get("_install_allowed", True): + console.print( + f"⚠ {safe_id}: Updates not allowed from '{_escape_markup(str(info.get('_catalog_name', 'catalog')))}' (skipping)" + ) + continue + try: + catalog_version = pkg_version.Version(str(info.get("version"))) + except pkg_version.InvalidVersion: + console.print( + f"⚠ {safe_id}: Invalid catalog version '{_escape_markup(str(info.get('version')))}' (skipping)" + ) + continue + if catalog_version > installed_version: + checked += 1 + updates_available.append( + {"id": wf_id, "installed": str(installed_version), "available": str(catalog_version)} + ) + else: + checked += 1 + console.print(f"✓ {safe_id}: Up to date (v{installed_version})") + + if not updates_available: + if not checked: + console.print("\n[yellow]No workflows were eligible for update[/yellow]") + elif checked == len(targets): + console.print("\n[green]All workflows are up to date![/green]") + else: + console.print( + f"\n[green]All checked workflows are up to date[/green] " + f"[yellow]({len(targets) - checked} skipped)[/yellow]" + ) + raise typer.Exit(0) + + console.print("\n[bold]Updates available:[/bold]\n") + for update in updates_available: + console.print( + f" • {_escape_markup(update['id'])}: {update['installed']} → {update['available']}" + ) + console.print() + if not typer.confirm("Update these workflows?"): + console.print("Cancelled") + raise typer.Exit(0) + + console.print() + failed: list[str] = [] + for update in updates_available: + # _install_workflow_from_catalog is fully transactional (staged + # download, atomic commit, rename-based rollback on registry + # failure): it never leaves a partially-written workflow.yml, so + # this loop only needs to record success/failure, not perform its + # own backup/restore. + try: + _install_workflow_from_catalog( + project_root, workflows_dir, update["id"], + expected_version=update["available"], + expected_installed_version=update["installed"], + ) + except (typer.Exit, OSError) as exc: + if isinstance(exc, OSError): + console.print( + f"[red]Error:[/red] Filesystem error updating " + f"'{_escape_markup(update['id'])}': {_escape_markup(str(exc))}" + ) + failed.append(update["id"]) + + if failed: + console.print( + f"\n[red]Failed to update:[/red] {', '.join(_escape_markup(f) for f in failed)}" + ) + raise typer.Exit(1) + + +def _set_workflow_enabled(workflow_id: str, enabled: bool) -> None: + """Update enabled state from a fresh registry snapshot while locked.""" + project_root = _require_specify_project() + safe_id = _escape_markup(workflow_id) + try: + with _workflow_install_transaction(project_root): + registry = _open_workflow_registry(project_root) + metadata = registry.get(workflow_id) + if metadata is None: + console.print( + f"[red]Error:[/red] Workflow '{safe_id}' is not installed" + ) + raise typer.Exit(1) + if not isinstance(metadata, dict): + console.print( + f"[red]Error:[/red] Registry entry for '{safe_id}' " + "is corrupted" + ) + raise typer.Exit(1) + current = bool(metadata.get("enabled", True)) + state = "enabled" if enabled else "disabled" + if current is enabled: + console.print( + f"[yellow]Workflow '{safe_id}' is already {state}[/yellow]" + ) + raise typer.Exit(0) + try: + registry.add(workflow_id, {**metadata, "enabled": enabled}) + except OSError as exc: + console.print( + f"[red]Error:[/red] Failed to update workflow registry " + f"for '{safe_id}': {_escape_markup(str(exc))}" + ) + raise typer.Exit(1) + except OSError as exc: + console.print( + f"[red]Error:[/red] Failed to lock workflow registry for " + f"'{safe_id}': {_escape_markup(str(exc))}" + ) + raise typer.Exit(1) + state = "enabled" if enabled else "disabled" + console.print(f"[green]✓[/green] Workflow '{safe_id}' {state}") + + +@workflow_app.command("enable") +def workflow_enable( + workflow_id: str = typer.Argument(..., help="Workflow ID to enable"), +): + """Enable a disabled workflow.""" + _set_workflow_enabled(workflow_id, True) + + +@workflow_app.command("disable") +def workflow_disable( + workflow_id: str = typer.Argument(..., help="Workflow ID to disable"), +): + """Disable a workflow without removing it.""" + _set_workflow_enabled(workflow_id, False) + console.print(f"To re-enable: specify workflow enable {_escape_markup(workflow_id)}") + @workflow_app.command("search") def workflow_search( query: str | None = typer.Argument(None, help="Search query"), tag: str | None = typer.Option(None, "--tag", help="Filter by tag"), + author: str | None = typer.Option(None, "--author", help="Filter by author"), ): """Search workflow catalogs.""" from .catalog import WorkflowCatalog, WorkflowCatalogError @@ -916,9 +2334,9 @@ def workflow_search( catalog = WorkflowCatalog(project_root) try: - results = catalog.search(query=query, tag=tag) + results = catalog.search(query=query, tag=tag, author=author) except WorkflowCatalogError as exc: - console.print(f"[red]Error:[/red] {exc}") + console.print(f"[red]Error:[/red] {_escape_markup(str(exc))}") raise typer.Exit(1) if not results: @@ -927,13 +2345,17 @@ def workflow_search( console.print(f"\n[bold cyan]Workflows ({len(results)}):[/bold cyan]\n") for wf in results: - console.print(f" [bold]{wf.get('name', wf.get('id', '?'))}[/bold] ({wf.get('id', '?')}) v{wf.get('version', '?')}") + name = _escape_markup(str(wf.get("name", wf.get("id", "?")))) + wf_id = _escape_markup(str(wf.get("id", "?"))) + version = _escape_markup(str(wf.get("version", "?"))) + console.print(f" [bold]{name}[/bold] ({wf_id}) v{version}") desc = wf.get("description", "") if desc: - console.print(f" {desc}") + console.print(f" {_escape_markup(str(desc))}") tags = wf.get("tags", []) if tags: - console.print(f" [dim]Tags: {', '.join(tags)}[/dim]") + safe_tags = _escape_markup(", ".join(str(t) for t in tags)) + console.print(f" [dim]Tags: {safe_tags}[/dim]") console.print() @@ -942,13 +2364,13 @@ def workflow_info( workflow_id: str = typer.Argument(..., help="Workflow ID"), ): """Show workflow details and step graph.""" - from .catalog import WorkflowCatalog, WorkflowRegistry, WorkflowCatalogError + from .catalog import WorkflowCatalog, WorkflowCatalogError from .engine import WorkflowEngine project_root = _require_specify_project() # Check installed first - registry = WorkflowRegistry(project_root) + registry = _open_workflow_registry(project_root) installed = registry.get(workflow_id) engine = WorkflowEngine(project_root) @@ -1264,7 +2686,9 @@ def _safe_fetch(url: str) -> bytes: raise ValueError(f"Refusing to fetch from non-HTTPS URL: {url}") if not parsed.hostname: raise ValueError(f"Refusing to fetch from URL with no hostname: {url}") - with _open_url(url, timeout=30) as resp: + with _open_url( + url, timeout=30, redirect_validator=_reject_insecure_download_redirect + ) as resp: final_url = resp.geturl() final_parsed = urlparse(final_url) final_is_localhost = final_parsed.hostname in ("localhost", "127.0.0.1", "::1") @@ -1274,7 +2698,7 @@ def _safe_fetch(url: str) -> bytes: raise ValueError(f"Redirect to non-HTTPS URL: {final_url}") if not final_parsed.hostname: raise ValueError(f"Redirect to URL with no hostname: {final_url}") - return resp.read() + return _read_response_within_limit(resp) _validate_step_id_or_exit(step_id) diff --git a/src/specify_cli/workflows/catalog.py b/src/specify_cli/workflows/catalog.py index 294dd8759a..654fa76e8b 100644 --- a/src/specify_cli/workflows/catalog.py +++ b/src/specify_cli/workflows/catalog.py @@ -13,6 +13,8 @@ import hashlib import json import os +import stat +import tempfile import time from dataclasses import dataclass from pathlib import Path @@ -71,48 +73,180 @@ def __init__(self, project_root: Path) -> None: self.registry_path = self.workflows_dir / self.REGISTRY_FILE self.data = self._load() + def _has_symlinked_parent(self) -> bool: + """Return True if any directory under .specify/workflows is a symlink.""" + current = self.project_root + for part in (".specify", "workflows"): + current = current / part + if current.is_symlink(): + return True + return False + def _load(self) -> dict[str, Any]: """Load registry from disk or create default.""" + default_registry: dict[str, Any] = { + "schema_version": self.SCHEMA_VERSION, + "workflows": {}, + } + # Defense-in-depth: refuse to read through symlinked parents or a + # symlinked registry file. Unlike StepRegistry (read-only best-effort + # elsewhere), a fabricated empty registry here is not safe: read-only + # callers (notably the bundler's remove path) query is_installed() + # before ever writing, and would otherwise conclude an installed + # workflow is absent, skip removing it, then delete the bundle + # record -- leaving the workflow untracked but still on disk. Fail + # closed here just like the unreadable-file case below. + if self._has_symlinked_parent() or self.registry_path.is_symlink(): + raise OSError( + f"Refusing to read workflow registry at {self.registry_path}: " + "a parent directory or the registry file itself is a symlink" + ) if self.registry_path.exists(): try: with open(self.registry_path, encoding="utf-8") as f: data = json.load(f) - # Validate shape: must be a dict with a dict "workflows" field, - # otherwise every method that indexes data["workflows"] crashes. - # Mirrors StepRegistry._load. - if not isinstance(data, dict): - return {"schema_version": self.SCHEMA_VERSION, "workflows": {}} - if not isinstance(data.get("workflows"), dict): - data["workflows"] = {} - return data - except (json.JSONDecodeError, ValueError, OSError, UnicodeError): - # Corrupted registry file — reset to default - return {"schema_version": self.SCHEMA_VERSION, "workflows": {}} - return {"schema_version": self.SCHEMA_VERSION, "workflows": {}} + except OSError as exc: + # The real data may still be intact on disk. Fail closed at + # construction rather than fabricating an empty registry that + # a read-only caller could mistake for "nothing installed." + raise OSError( + f"Failed to read workflow registry at {self.registry_path}: {exc}" + ) from exc + except ( + json.JSONDecodeError, + ValueError, + UnicodeError, + ) as exc: + raise OSError( + f"Workflow registry at {self.registry_path} is corrupted: " + f"{exc}" + ) from exc + # Validate shape: must be a dict with a dict "workflows" field. + if not isinstance(data, dict): + raise OSError( + f"Workflow registry at {self.registry_path} is corrupted: " + "top-level value must be an object" + ) + if not isinstance(data.get("workflows"), dict): + raise OSError( + f"Workflow registry at {self.registry_path} is corrupted: " + "'workflows' must be an object" + ) + return data + return default_registry def save(self) -> None: - """Persist registry to disk.""" + """Persist registry to disk atomically.""" + # Refuse to write through symlinked parents (mirrors StepRegistry.save + # and the CLI-level _reject_unsafe_dir guard). + if self._has_symlinked_parent() or self.registry_path.is_symlink(): + raise OSError( + "Refusing to write workflow registry through a symlinked path." + ) self.workflows_dir.mkdir(parents=True, exist_ok=True) - with open(self.registry_path, "w", encoding="utf-8") as f: - json.dump(self.data, f, indent=2) + # Unique, exclusive temp then replace: a failed dump cannot truncate + # the registry, a pre-created symlink cannot redirect the write, and + # concurrent CLI processes cannot collide on the same temp path. + fd, tmp = tempfile.mkstemp( + dir=str(self.registry_path.parent), + prefix=f".{self.registry_path.name}.", + suffix=".tmp", + ) + try: + # Write through a duplicate so the exclusive mkstemp descriptor + # stays open for fd-based metadata updates and inode verification. + with os.fdopen(os.dup(fd), "w", encoding="utf-8") as f: + json.dump(self.data, f, indent=2) + # mkstemp creates the temp file at 0600. A pre-existing registry + # may be shared more permissively (e.g. 0640/0644); preserve its + # mode across the replace so a save doesn't silently lock other + # project users out. A brand-new registry has no prior mode to + # preserve, so mkstemp's secure 0600 default stands. Mirrors + # _utils.py's atomic_write_json (best-effort; data safety over + # metadata preservation). + try: + if self.registry_path.exists(): + existing_stat = self.registry_path.stat( + follow_symlinks=False + ) + if stat.S_ISREG(existing_stat.st_mode) and hasattr( + os, "fchmod" + ): + os.fchmod(fd, stat.S_IMODE(existing_stat.st_mode)) + if stat.S_ISREG(existing_stat.st_mode) and hasattr( + os, "fchown" + ): + try: + os.fchown( + fd, existing_stat.st_uid, existing_stat.st_gid + ) + except PermissionError: + pass + except OSError: + pass + staged_stat = os.stat(tmp, follow_symlinks=False) + open_stat = os.fstat(fd) + if ( + not stat.S_ISREG(staged_stat.st_mode) + or staged_stat.st_dev != open_stat.st_dev + or staged_stat.st_ino != open_stat.st_ino + ): + raise OSError( + "Refusing to replace workflow registry: " + "staged file changed before commit" + ) + os.close(fd) + fd = -1 + os.replace(tmp, self.registry_path) + except BaseException: + if fd >= 0: + try: + os.close(fd) + except OSError: + pass + try: + os.unlink(tmp) + except OSError: + pass + raise def add(self, workflow_id: str, metadata: dict[str, Any]) -> None: """Add or update an installed workflow entry.""" from datetime import datetime, timezone - existing = self.data["workflows"].get(workflow_id, {}) + raw_existing = self.data["workflows"].get(workflow_id) + had_entry = workflow_id in self.data["workflows"] + # Corrupted-but-parseable registries may hold non-dict entries. + existing = raw_existing if isinstance(raw_existing, dict) else {} metadata["installed_at"] = existing.get( "installed_at", datetime.now(timezone.utc).isoformat() ) metadata["updated_at"] = datetime.now(timezone.utc).isoformat() self.data["workflows"][workflow_id] = metadata - self.save() + try: + self.save() + except (OSError, TypeError, ValueError): + # Roll back the in-memory mutation so a later successful save + # cannot persist metadata for a write that failed. + if had_entry: + self.data["workflows"][workflow_id] = raw_existing + else: + del self.data["workflows"][workflow_id] + raise def remove(self, workflow_id: str) -> bool: """Remove an installed workflow entry. Returns True if found.""" if workflow_id in self.data["workflows"]: + removed_entry = self.data["workflows"][workflow_id] del self.data["workflows"][workflow_id] - self.save() + try: + self.save() + except (OSError, TypeError, ValueError): + # Roll back the in-memory deletion so a save failure can't + # desync this instance from the untouched file on disk, + # mirroring add()'s rollback-on-save-failure. + self.data["workflows"][workflow_id] = removed_entry + raise return True return False @@ -435,6 +569,7 @@ def search( self, query: str | None = None, tag: str | None = None, + author: str | None = None, ) -> list[dict[str, Any]]: """Search workflows across all configured catalogs.""" merged = self._get_merged_workflows() @@ -459,6 +594,10 @@ def search( normalized_tags = [t.lower() for t in tags if isinstance(t, str)] if tag.lower() not in normalized_tags: continue + if author: + wf_author = wf_data.get("author", "") + if not isinstance(wf_author, str) or wf_author.lower() != author.lower(): + continue results.append(wf_data) return results diff --git a/src/specify_cli/workflows/engine.py b/src/specify_cli/workflows/engine.py index fd207b9ec5..ed52710bed 100644 --- a/src/specify_cli/workflows/engine.py +++ b/src/specify_cli/workflows/engine.py @@ -150,7 +150,7 @@ def validate_workflow(definition: WorkflowDefinition) -> list[str]: f"'workflow.id' must be a string, got " f"{type(definition.id).__name__} ({definition.id!r})." ) - elif not _ID_PATTERN.match(definition.id): + elif not _ID_PATTERN.fullmatch(definition.id): errors.append( f"Workflow ID {definition.id!r} must be lowercase alphanumeric " f"with hyphens." @@ -172,7 +172,7 @@ def validate_workflow(definition: WorkflowDefinition) -> list[str]: f"{type(definition.version).__name__} ({definition.version!r}) — " f'quote it in YAML (version: "1.0.0").' ) - elif not re.match(r"^\d+\.\d+\.\d+$", definition.version): + elif not re.fullmatch(r"\d+\.\d+\.\d+", definition.version): errors.append( f"Workflow version {definition.version!r} is not valid " f"semantic versioning (expected X.Y.Z)." @@ -416,18 +416,57 @@ def _validate_run_id(cls, run_id: str) -> None: ID into a path so a malicious value cannot probe or read files outside ``.specify/workflows/runs//``. """ - if not isinstance(run_id, str) or not cls._RUN_ID_PATTERN.match(run_id): + if not isinstance(run_id, str) or not cls._RUN_ID_PATTERN.fullmatch(run_id): raise ValueError( f"Invalid run_id {run_id!r}: must be alphanumeric with " "hyphens/underscores only (and must start with an " "alphanumeric character)." ) + @staticmethod + def _validate_installed_origin( + installed_workflow_id: str | None, + installed_registry_root: str | None, + ) -> None: + """Validate persisted installed-workflow ownership metadata.""" + if installed_workflow_id is not None: + if not isinstance(installed_workflow_id, str): + raise ValueError( + "Invalid run state: 'installed_workflow_id' must be a " + f"string or null, got {type(installed_workflow_id).__name__}" + ) + if not _ID_PATTERN.fullmatch(installed_workflow_id): + raise ValueError( + "Invalid run state: 'installed_workflow_id' must be a " + "lowercase alphanumeric workflow ID with hyphens" + ) + if installed_registry_root is not None: + if not isinstance(installed_registry_root, str): + raise ValueError( + "Invalid run state: 'installed_registry_root' must be a " + f"string or null, got {type(installed_registry_root).__name__}" + ) + if not installed_registry_root or not Path( + installed_registry_root + ).is_absolute(): + raise ValueError( + "Invalid run state: 'installed_registry_root' must be " + "an absolute path or null" + ) + if installed_workflow_id is None: + raise ValueError( + "Invalid run state: 'installed_registry_root' requires " + "'installed_workflow_id'" + ) + def __init__( self, run_id: str | None = None, workflow_id: str = "", project_root: Path | None = None, + installed_workflow_id: str | None = None, + installed_registry_root: str | None = None, + installed_origin_tracked: bool = True, ) -> None: # ``run_id is None`` (omitted) → auto-generate. An explicit empty # string is *not* the same as "omitted" and must be validated like @@ -439,8 +478,22 @@ def __init__( else: self.run_id = run_id self._validate_run_id(self.run_id) + self._validate_installed_origin( + installed_workflow_id, installed_registry_root + ) self.workflow_id = workflow_id self.project_root = project_root or Path(".") + # Identifies the installed workflow (if any) this run was started + # from, and the project root that owns its registry — set by + # execute() when the source was resolved to an installed ID (see + # workflow_run's ownership mapping). None for a direct/non-installed + # YAML source. ``installed_origin_tracked`` distinguishes those + # explicit None values from legacy state files that predate both + # fields, allowing the CLI to conservatively infer same-project + # registry ownership before resuming. + self.installed_workflow_id = installed_workflow_id + self.installed_registry_root = installed_registry_root + self.installed_origin_tracked = installed_origin_tracked self.status = RunStatus.CREATED self.current_step_index = 0 self.current_step_id: str | None = None @@ -503,6 +556,8 @@ def save(self) -> None: state_data = { "run_id": self.run_id, "workflow_id": self.workflow_id, + "installed_workflow_id": self.installed_workflow_id, + "installed_registry_root": self.installed_registry_root, "status": self.status.value, "current_step_index": self.current_step_index, "current_step_id": self.current_step_id, @@ -554,11 +609,46 @@ def load(cls, run_id: str, project_root: Path) -> RunState: with open(state_path, encoding="utf-8") as f: state_data = json.load(f) + if not isinstance(state_data, dict): + raise ValueError("Invalid run state: expected a JSON object") + missing_fields = [ + field + for field in ("run_id", "workflow_id", "status") + if field not in state_data + ] + if missing_fields: + raise ValueError( + "Invalid run state: missing required field(s): " + + ", ".join(missing_fields) + ) + + workflow_id = state_data["workflow_id"] + if not isinstance(workflow_id, str) or not _ID_PATTERN.fullmatch( + workflow_id + ): + raise ValueError( + "Invalid run state: 'workflow_id' must be a lowercase " + "alphanumeric workflow ID with hyphens" + ) + + has_installed_workflow_id = "installed_workflow_id" in state_data + has_installed_registry_root = "installed_registry_root" in state_data + if has_installed_workflow_id != has_installed_registry_root: + raise ValueError( + "Invalid run state: installed workflow origin fields must " + "either both be present or both be absent" + ) + + installed_workflow_id = state_data.get("installed_workflow_id") + installed_registry_root = state_data.get("installed_registry_root") state = cls( run_id=state_data["run_id"], - workflow_id=state_data["workflow_id"], + workflow_id=workflow_id, project_root=project_root, + installed_workflow_id=installed_workflow_id, + installed_registry_root=installed_registry_root, + installed_origin_tracked=has_installed_workflow_id, ) state.status = RunStatus(state_data["status"]) state.current_step_index = state_data.get("current_step_index", 0) @@ -571,7 +661,16 @@ def load(cls, run_id: str, project_root: Path) -> RunState: if inputs_path.exists(): with open(inputs_path, encoding="utf-8") as f: inputs_data = json.load(f) - state.inputs = inputs_data.get("inputs", {}) + if not isinstance(inputs_data, dict): + raise ValueError( + "Invalid run inputs: expected a JSON object" + ) + inputs = inputs_data.get("inputs", {}) + if not isinstance(inputs, dict): + raise ValueError( + "Invalid run inputs: 'inputs' must be a JSON object" + ) + state.inputs = inputs return state @@ -654,6 +753,8 @@ def execute( definition: WorkflowDefinition, inputs: dict[str, Any] | None = None, run_id: str | None = None, + installed_workflow_id: str | None = None, + installed_registry_root: Path | None = None, ) -> RunState: """Execute a workflow definition. @@ -665,6 +766,12 @@ def execute( User-provided input values. run_id: Optional run ID (uses SPECKIT_WORKFLOW_RUN_ID when set, otherwise auto-generated). + installed_workflow_id, installed_registry_root: + When the run was started from an installed workflow (as opposed + to a direct/non-installed YAML source), identifies it and its + owning registry root so a later ``resume`` can re-check the + registry's current disabled state before continuing — see + ``workflow_resume``. Returns ------- @@ -682,6 +789,12 @@ def execute( run_id=effective_run_id, workflow_id=definition.id, project_root=self.project_root, + installed_workflow_id=installed_workflow_id, + installed_registry_root=( + str(installed_registry_root) + if installed_registry_root is not None + else None + ), ) # Persist a copy of the workflow definition so resume can diff --git a/tests/contract/test_bundle_cli.py b/tests/contract/test_bundle_cli.py index df9c8dae10..db2bb0c948 100644 --- a/tests/contract/test_bundle_cli.py +++ b/tests/contract/test_bundle_cli.py @@ -6,6 +6,7 @@ """ from __future__ import annotations +import io import json from pathlib import Path from unittest.mock import patch @@ -63,6 +64,42 @@ def test_commands_outside_project_fail_with_guidance(tmp_path: Path, monkeypatch assert "Spec Kit project" in result.output +def test_remove_reports_clean_error_when_primitive_raises_raw_exception( + project: Path, +): + """A raw exception from a primitive installer (e.g. an OSError from an + unreadable workflow registry surfacing through _WorkflowKindManager's + fail-closed construction) must not propagate uncaught through + `specify bundle remove` -- the command only catches BundlerError, so + without a conversion at the remove_bundle boundary this would exit + with an unhandled exception and empty/raw output instead of a clean, + actionable message, and no removal side effects should occur either.""" + from specify_cli.bundler.models.manifest import BundleManifest + from specify_cli.bundler.models.records import load_records + from specify_cli.bundler.services.adapters import DefaultPrimitiveInstaller + from specify_cli.bundler.services.installer import install_bundle + from specify_cli.bundler.services.resolver import resolve_install_plan + from tests.bundler_helpers import FakeInstaller + + manifest = BundleManifest.from_dict(valid_manifest_dict()) + plan = resolve_install_plan( + manifest, speckit_version="0.11.2", active_integration="copilot" + ) + install_bundle(project, plan, FakeInstaller(), manifest=manifest) + + def boom(self, project_root, component): + raise OSError("workflow registry unreadable") + + with pytest.MonkeyPatch.context() as mp: + mp.setattr(DefaultPrimitiveInstaller, "is_installed", boom) + result = runner.invoke(app, ["bundle", "remove", "demo-bundle"]) + + assert result.exit_code != 0 + assert result.output.strip() != "" + assert result.exception is None or isinstance(result.exception, SystemExit) + assert {r.bundle_id for r in load_records(project)} == {"demo-bundle"} + + def test_fail_writes_error_to_stderr_not_stdout(capsys): """_fail must write to stderr, not stdout: every bundle command routes errors through it, and under --json the error would otherwise corrupt the JSON payload @@ -432,25 +469,16 @@ def test_install_integration_override_cannot_bypass_clash_guard(project: Path): # ===== Private GitHub release asset URL resolution ===== -class FakeBundleResponse: +class FakeBundleResponse(io.BytesIO): """Minimal context-manager response stub for open_url fakes.""" def __init__(self, data: bytes, url: str = "https://api.github.com/repos/org/repo/releases/assets/99"): - self._data = data + super().__init__(data) self._url = url - def read(self) -> bytes: - return self._data - def geturl(self) -> str: return self._url - def __enter__(self): - return self - - def __exit__(self, *_): - return False - def _make_catalog_config(catalog_path: Path, project: Path) -> None: """Write a bundle-catalogs.yml pointing at *catalog_path* in *project*.""" diff --git a/tests/integration/test_bundler_install_flow.py b/tests/integration/test_bundler_install_flow.py index bf066b660e..655094168d 100644 --- a/tests/integration/test_bundler_install_flow.py +++ b/tests/integration/test_bundler_install_flow.py @@ -11,7 +11,7 @@ from specify_cli.bundler import BundlerError from specify_cli.bundler.models.manifest import BundleManifest -from specify_cli.bundler.models.records import load_records +from specify_cli.bundler.models.records import load_records, records_path from specify_cli.bundler.services.installer import install_bundle, remove_bundle from specify_cli.bundler.services.resolver import resolve_install_plan from tests.bundler_helpers import FakeInstaller, make_project, valid_manifest_dict @@ -97,6 +97,212 @@ def test_remove_unknown_bundle_errors(tmp_path: Path): remove_bundle(tmp_path, "ghost", FakeInstaller()) +def test_remove_converts_raw_installer_exception_to_bundler_error(tmp_path: Path): + """A raw exception from a primitive installer (e.g. an OSError from an + unreadable workflow registry surfacing through _WorkflowKindManager's + fail-closed construction) must not propagate uncaught out of + remove_bundle: install_bundle already converts any non-BundlerError + exception into a clean BundlerError, but remove_bundle had no such + conversion, so the CLI's `bundle remove` (which only catches + BundlerError) would let a raw exception through with no clean message + and no removal side effects should occur either.""" + make_project(tmp_path) + manifest = BundleManifest.from_dict(valid_manifest_dict()) + installer = FakeInstaller() + install_bundle(tmp_path, _plan(manifest), installer, manifest=manifest) + + def boom(project_root, component): + raise OSError("workflow registry unreadable") + + with pytest.MonkeyPatch.context() as mp: + mp.setattr(installer, "is_installed", boom) + with pytest.raises(BundlerError): + remove_bundle(tmp_path, "demo-bundle", installer) + + # No removal side effects: the bundle record must still be present. + assert {r.bundle_id for r in load_records(tmp_path)} == {"demo-bundle"} + + +def test_remove_partial_failure_message_reflects_partial_state(tmp_path: Path): + """A failure can occur after earlier components in the same bundle have + already been removed from disk. The bundle record is left unchanged + (save_records never runs on this path), so it still claims the bundle + fully installed -- but the message must not claim "No changes were + recorded" when components were, in fact, already removed.""" + make_project(tmp_path) + manifest = BundleManifest.from_dict(valid_manifest_dict()) + installer = FakeInstaller() + install_bundle(tmp_path, _plan(manifest), installer, manifest=manifest) + + real_remove = installer.remove + calls = {"n": 0} + + def remove_then_fail(project_root, component): + calls["n"] += 1 + if calls["n"] == 1: + return real_remove(project_root, component) + raise OSError("disk full") + + with pytest.MonkeyPatch.context() as mp: + mp.setattr(installer, "remove", remove_then_fail) + with pytest.raises(BundlerError) as exc_info: + remove_bundle(tmp_path, "demo-bundle", installer) + + message = str(exc_info.value) + assert "no changes were recorded" not in message.lower() + assert {r.bundle_id for r in load_records(tmp_path)} == {"demo-bundle"} + + +def test_remove_bundler_error_from_installer_after_partial_removal_reports_partial_state( + tmp_path: Path, +): + """If the primitive installer itself raises BundlerError (not a raw/ + unexpected exception) after an earlier component in the same bundle was + already removed, the surfaced message must still carry the same + partial-removal detail as the generic-exception path -- a bare + ``except BundlerError: raise`` would re-raise the installer's original + message verbatim with no mention that the project may now be partially + uninstalled.""" + make_project(tmp_path) + manifest = BundleManifest.from_dict(valid_manifest_dict()) + installer = FakeInstaller() + install_bundle(tmp_path, _plan(manifest), installer, manifest=manifest) + + real_remove = installer.remove + calls = {"n": 0} + + def remove_then_raise_bundler_error(project_root, component): + calls["n"] += 1 + if calls["n"] == 1: + return real_remove(project_root, component) + raise BundlerError("kind manager refused removal") + + with pytest.MonkeyPatch.context() as mp: + mp.setattr(installer, "remove", remove_then_raise_bundler_error) + with pytest.raises(BundlerError) as exc_info: + remove_bundle(tmp_path, "demo-bundle", installer) + + message = str(exc_info.value) + assert "no changes were recorded" not in message.lower() + assert "kind manager refused removal" in message + assert "partially uninstalled" in message.lower() + assert {r.bundle_id for r in load_records(tmp_path)} == {"demo-bundle"} + + +def test_remove_bundler_error_from_installer_with_zero_removed_reports_no_changes( + tmp_path: Path, +): + """When the installer raises BundlerError before anything was actually + removed, the message should not misleadingly claim partial state.""" + make_project(tmp_path) + manifest = BundleManifest.from_dict(valid_manifest_dict()) + installer = FakeInstaller() + install_bundle(tmp_path, _plan(manifest), installer, manifest=manifest) + + def boom(project_root, component): + raise BundlerError("kind manager unavailable") + + with pytest.MonkeyPatch.context() as mp: + mp.setattr(installer, "is_installed", boom) + with pytest.raises(BundlerError) as exc_info: + remove_bundle(tmp_path, "demo-bundle", installer) + + message = str(exc_info.value) + assert "no components were removed" in message.lower() + assert "no removal was attempted" in message.lower() + assert "partially uninstalled" not in message.lower() + assert "kind manager unavailable" in message + assert {r.bundle_id for r in load_records(tmp_path)} == {"demo-bundle"} + + +def test_remove_zero_completed_removals_still_cautions_about_partial_changes( + tmp_path: Path, +): + """`result.uninstalled` only records a component after its `remove()` + call returns successfully. If the very first `remove()` call itself + raises after already deleting some files, zero completed removals are + recorded even though the project may already be partially uninstalled -- + the zero-count message must not claim "No components were removed" as + an unqualified fact; it must caution that the failing component may + have made partial changes before raising.""" + make_project(tmp_path) + manifest = BundleManifest.from_dict(valid_manifest_dict()) + installer = FakeInstaller() + install_bundle(tmp_path, _plan(manifest), installer, manifest=manifest) + + def boom(project_root, component): + # Simulates a remove() that deletes some files before raising -- + # from the caller's perspective this component was never recorded + # as completed, but disk state may already be partially changed. + raise OSError("disk full partway through removal") + + with pytest.MonkeyPatch.context() as mp: + mp.setattr(installer, "remove", boom) + with pytest.raises(BundlerError) as exc_info: + remove_bundle(tmp_path, "demo-bundle", installer) + + message = str(exc_info.value) + assert "no components were removed" in message.lower() + assert "partial" in message.lower() + assert "partially uninstalled" in message.lower() + assert {r.bundle_id for r in load_records(tmp_path)} == {"demo-bundle"} + + +def test_remove_record_save_failure_reports_partial_state(tmp_path: Path): + make_project(tmp_path) + manifest = BundleManifest.from_dict(valid_manifest_dict()) + installer = FakeInstaller() + install_bundle(tmp_path, _plan(manifest), installer, manifest=manifest) + record_file = records_path(tmp_path) + original_record = record_file.read_bytes() + + def fail_dump(_data, handle, *_args, **_kwargs): + handle.write('{"partial":') + handle.flush() + raise OSError("disk full") + + with pytest.MonkeyPatch.context() as mp: + mp.setattr( + "specify_cli.bundler.lib.yamlio.json.dump", + fail_dump, + ) + with pytest.raises(BundlerError) as exc_info: + remove_bundle(tmp_path, "demo-bundle", installer) + + message = str(exc_info.value) + assert "disk full" in message + assert "partially uninstalled" in message.lower() + assert installer.installed == set() + assert record_file.read_bytes() == original_record + assert {r.bundle_id for r in load_records(tmp_path)} == {"demo-bundle"} + + +def test_remove_record_save_failure_without_remove_attempt_is_not_partial( + tmp_path: Path, +): + make_project(tmp_path) + manifest = BundleManifest.from_dict(valid_manifest_dict()) + installer = FakeInstaller() + install_bundle(tmp_path, _plan(manifest), installer, manifest=manifest) + installer.installed.clear() + + def fail_save(*_args, **_kwargs): + raise OSError("disk full") + + with pytest.MonkeyPatch.context() as mp: + mp.setattr( + "specify_cli.bundler.services.installer.save_records", + fail_save, + ) + with pytest.raises(BundlerError) as exc_info: + remove_bundle(tmp_path, "demo-bundle", installer) + + message = str(exc_info.value) + assert "no removal was attempted" in message.lower() + assert "partially uninstalled" not in message.lower() + assert {r.bundle_id for r in load_records(tmp_path)} == {"demo-bundle"} + + def test_remove_reports_uninstalled_not_installed(tmp_path: Path): make_project(tmp_path) manifest = BundleManifest.from_dict(valid_manifest_dict()) @@ -128,7 +334,7 @@ def test_remove_counts_only_components_actually_removed(tmp_path: Path): assert len(result.uninstalled) == 3 assert (gone.kind, gone.id) not in installer.remove_calls - assert gone in result.skipped + assert gone not in result.skipped make_project(tmp_path) manifest = BundleManifest.from_dict(valid_manifest_dict()) installer = FakeInstaller() diff --git a/tests/test_github_http.py b/tests/test_github_http.py index a03fe9186f..3f6ca8e53d 100644 --- a/tests/test_github_http.py +++ b/tests/test_github_http.py @@ -152,6 +152,44 @@ def failing_open(url, timeout=None, extra_headers=None): ) assert result is None + def test_metadata_lookup_is_bounded_and_redirect_validated(self): + """Release metadata reads stay bounded and use the caller's policy.""" + captured = {} + + class OversizedResponse: + def read(self, amount=None): + captured["read_amount"] = amount + return b"x" * amount + + def __enter__(self): + return self + + def __exit__(self, *args): + return False + + def redirect_validator(old_url, new_url): + return None + + def fake_open( + url, + timeout=None, + extra_headers=None, + redirect_validator=None, + ): + captured["redirect_validator"] = redirect_validator + return OversizedResponse() + + result = resolve_github_release_asset_api_url( + "https://github.com/org/repo/releases/download/v1/pack.zip", + fake_open, + redirect_validator=redirect_validator, + max_metadata_bytes=8, + ) + + assert result is None + assert captured["read_amount"] == 9 + assert captured["redirect_validator"] is redirect_validator + def test_tag_with_special_characters_is_url_encoded(self): """Tags with reserved characters (e.g. '/') are encoded in the API URL.""" captured_urls = [] diff --git a/tests/test_presets.py b/tests/test_presets.py index 12dbf90afb..8208a416e2 100644 --- a/tests/test_presets.py +++ b/tests/test_presets.py @@ -5016,26 +5016,13 @@ def test_preset_add_from_github_release_url_resolves_and_downloads(self, project captured_urls = [] - class FakeResponse: - def __init__(self, data): - self._data = data - - def read(self): - return self._data - - def __enter__(self): - return self - - def __exit__(self, *a): - return False - def fake_open_url(url, timeout=None, extra_headers=None, redirect_validator=None): captured_urls.append((url, extra_headers)) if "releases/tags/" in url: - return FakeResponse(json.dumps({ + return io.BytesIO(json.dumps({ "assets": [{"name": "preset.zip", "url": "https://api.github.com/repos/org/repo/releases/assets/42"}] }).encode()) - return FakeResponse(zip_bytes) + return io.BytesIO(zip_bytes) runner = CliRunner() with patch.object(Path, "cwd", return_value=project_dir), \ @@ -5074,22 +5061,9 @@ def test_preset_add_from_direct_api_asset_url_passes_through(self, project_dir): captured_urls = [] - class FakeResponse: - def __init__(self, data): - self._data = data - - def read(self): - return self._data - - def __enter__(self): - return self - - def __exit__(self, *a): - return False - def fake_open_url(url, timeout=None, extra_headers=None, redirect_validator=None): captured_urls.append((url, extra_headers)) - return FakeResponse(zip_bytes) + return io.BytesIO(zip_bytes) runner = CliRunner() with patch.object(Path, "cwd", return_value=project_dir), \ @@ -5131,26 +5105,13 @@ def test_preset_add_from_ghes_release_url_resolves_via_api_v3(self, project_dir, captured_urls = [] - class FakeResponse: - def __init__(self, data): - self._data = data - - def read(self): - return self._data - - def __enter__(self): - return self - - def __exit__(self, *a): - return False - def fake_open_url(url, timeout=None, extra_headers=None, redirect_validator=None): captured_urls.append((url, extra_headers)) if "releases/tags/" in url: - return FakeResponse(json.dumps({ + return io.BytesIO(json.dumps({ "assets": [{"name": "preset.zip", "url": "https://ghes.example/api/v3/repos/org/repo/releases/assets/42"}] }).encode()) - return FakeResponse(zip_bytes) + return io.BytesIO(zip_bytes) runner = CliRunner() with patch.object(Path, "cwd", return_value=project_dir), \ diff --git a/tests/test_workflows.py b/tests/test_workflows.py index 81aec96d3e..d5a6d26971 100644 --- a/tests/test_workflows.py +++ b/tests/test_workflows.py @@ -15,6 +15,7 @@ import json import os import shutil +import stat import sys import tempfile from pathlib import Path @@ -2964,6 +2965,21 @@ def test_invalid_id_format(self): errors = validate_workflow(definition) assert any("lowercase alphanumeric" in e for e in errors) + def test_workflow_id_with_trailing_newline_is_invalid(self): + from specify_cli.workflows.engine import WorkflowDefinition, validate_workflow + + definition = WorkflowDefinition.from_string(""" +workflow: + id: "valid\\n" + name: "Test" + version: "1.0.0" +steps: + - id: step-one + command: speckit.specify +""") + errors = validate_workflow(definition) + assert any("lowercase alphanumeric" in e for e in errors) + def test_non_string_workflow_id_reports_error(self): from specify_cli.workflows.engine import WorkflowDefinition, validate_workflow @@ -3009,6 +3025,21 @@ def test_unquoted_float_version_reports_error(self): errors = validate_workflow(definition) assert any("workflow.version" in e and "quote" in e for e in errors) + def test_version_with_trailing_newline_is_invalid(self): + from specify_cli.workflows.engine import WorkflowDefinition, validate_workflow + + definition = WorkflowDefinition.from_string(""" +workflow: + id: "test" + name: "Test" + version: "1.0.0\\n" +steps: + - id: step-one + command: speckit.specify +""") + errors = validate_workflow(definition) + assert any("semantic version" in e for e in errors) + def test_non_string_step_id_reports_error(self): from specify_cli.workflows.engine import WorkflowDefinition, validate_workflow @@ -3345,6 +3376,38 @@ def test_execute_simple_workflow(self, project_dir): assert state.step_results["step-one"]["output"]["command"] == "speckit.specify" assert state.step_results["step-one"]["output"]["input"]["args"] == "login" + def test_execute_rejects_invalid_origin_before_creating_run_state( + self, project_dir + ): + from specify_cli.workflows.engine import WorkflowDefinition, WorkflowEngine + + definition = WorkflowDefinition.from_string(""" +schema_version: "1.0" +workflow: + id: "simple" + name: "Simple" + version: "1.0.0" +steps: [] +""") + engine = WorkflowEngine(project_dir) + + with pytest.raises(ValueError, match="installed_registry_root"): + engine.execute( + definition, + run_id="invalid-origin", + installed_workflow_id="simple", + installed_registry_root=Path("relative-owner"), + ) + + run_dir = ( + project_dir + / ".specify" + / "workflows" + / "runs" + / "invalid-origin" + ) + assert not run_dir.exists() + def test_execute_with_gate_pauses(self, project_dir): from specify_cli.workflows.engine import WorkflowEngine, WorkflowDefinition from specify_cli.workflows.base import RunStatus @@ -4754,6 +4817,39 @@ def test_load_not_found(self, project_dir): with pytest.raises(FileNotFoundError): RunState.load("nonexistent", project_dir) + @pytest.mark.parametrize( + ("installed_workflow_id", "installed_registry_root"), + [ + ("", None), + ("gated-wf\n", None), + ("gated-wf", "relative-owner"), + ], + ) + def test_init_rejects_invalid_installed_origin( + self, installed_workflow_id, installed_registry_root + ): + from specify_cli.workflows.engine import RunState + + with pytest.raises(ValueError, match="Invalid run state"): + RunState( + run_id="test-run", + workflow_id="test-workflow", + installed_workflow_id=installed_workflow_id, + installed_registry_root=installed_registry_root, + ) + + def test_init_rejects_registry_root_without_workflow_id( + self, project_dir + ): + from specify_cli.workflows.engine import RunState + + with pytest.raises(ValueError, match="requires"): + RunState( + run_id="test-run", + workflow_id="test-workflow", + installed_registry_root=str(project_dir), + ) + @pytest.mark.parametrize( "malicious_run_id", [ @@ -4824,6 +4920,7 @@ def test_load_rejects_path_traversal(self, project_dir, malicious_run_id): "../escape", # parent-directory traversal "foo/bar", # embedded path separator ".hidden", # leading non-alphanumeric + "valid\n", # regex end-anchor bypass "", # empty / degenerate ], ) @@ -4936,6 +5033,32 @@ def test_remove(self, project_dir): registry.remove("test-wf") assert not registry.is_installed("test-wf") + @pytest.mark.parametrize("error_type", [OSError, TypeError, ValueError]) + def test_remove_rolls_back_in_memory_on_save_failure( + self, project_dir, monkeypatch, error_type + ): + """A save() failure during remove() must not leave the in-memory registry + out of sync with the (unchanged) file on disk, mirroring add()'s rollback.""" + from specify_cli.workflows.catalog import WorkflowRegistry + import specify_cli.workflows.catalog as catalog_mod + + registry = WorkflowRegistry(project_dir) + registry.add("test-wf", {"name": "Test", "version": "1.0.0"}) + + def boom(*args, **kwargs): + raise error_type("save failed") + + monkeypatch.setattr(catalog_mod.json, "dump", boom) + with pytest.raises(error_type): + registry.remove("test-wf") + monkeypatch.undo() + + # In-memory state must still show the entry (rolled back), matching + # the untouched file on disk. + assert registry.is_installed("test-wf") + fresh = WorkflowRegistry(project_dir) + assert fresh.is_installed("test-wf") + def test_list(self, project_dir): from specify_cli.workflows.catalog import WorkflowRegistry @@ -4966,32 +5089,83 @@ def test_persistence(self, project_dir): registry2 = WorkflowRegistry(project_dir) assert registry2.is_installed("test-wf") - @pytest.mark.parametrize("bad_content", ["[]", '{"schema_version": "1.0"}']) - def test_load_tolerates_misshaped_registry(self, project_dir, bad_content): - """A JSON-valid but mis-shaped registry file must not crash every method. + def test_load_read_oserror_refuses_to_save_over_existing_data(self, project_dir, monkeypatch): + """A transient read failure (e.g. temporarily unreadable file) must not be + treated the same as a corrupted/missing registry: constructing a registry + on top of it -- and thus any query a caller makes before ever calling + save() -- must fail closed instead of silently reporting an empty + registry that a caller could then act on and overwrite.""" + from specify_cli.workflows.catalog import WorkflowRegistry + import builtins + + registry1 = WorkflowRegistry(project_dir) + registry1.add("test-wf", {"name": "Test", "version": "1.0.0"}) + registry_path = registry1.registry_path + real_open = builtins.open - A list root, or a dict lacking a 'workflows' mapping, previously made - is_installed/get/list/remove/add raise TypeError/KeyError. Mirrors the - shape guard StepRegistry._load already has. - """ + def _raising_open(file, mode="r", *args, **kwargs): + if Path(file) == registry_path and "r" in mode: + raise OSError("simulated read failure") + return real_open(file, mode, *args, **kwargs) + + monkeypatch.setattr(builtins, "open", _raising_open) + with pytest.raises(OSError): + WorkflowRegistry(project_dir) + # The original entry must survive on disk untouched. + data = json.loads(registry_path.read_text(encoding="utf-8")) + assert "test-wf" in data["workflows"] + + def test_load_read_oserror_fails_closed_not_silently_empty(self, project_dir, monkeypatch): + """Root cause: a registry that failed to read must never let a query + method (is_installed/get/list) report as if nothing were installed -- + a caller (e.g. bundled workflow install) that only checks + is_installed() before writing a file would otherwise overwrite real + data on a transient read failure, long before any save() call could + catch it. The failure must surface at construction, before any query + or side effect is possible.""" from specify_cli.workflows.catalog import WorkflowRegistry + import builtins - reg_path = project_dir / ".specify" / "workflows" / "workflow-registry.json" - reg_path.parent.mkdir(parents=True, exist_ok=True) - reg_path.write_text(bad_content, encoding="utf-8") + registry1 = WorkflowRegistry(project_dir) + registry1.add("test-wf", {"name": "Test", "version": "1.0.0"}) + registry_path = registry1.registry_path + real_open = builtins.open - registry = WorkflowRegistry(project_dir) - assert registry.data == { - "schema_version": WorkflowRegistry.SCHEMA_VERSION, - "workflows": {}, - } - # None of these should raise on the recovered-default shape. - assert registry.is_installed("x") is False - assert registry.get("x") is None - assert registry.list() == {} # list() always returns a dict - registry.remove("x") - registry.add("x", {"name": "X"}) - assert registry.is_installed("x") + def _raising_open(file, mode="r", *args, **kwargs): + if Path(file) == registry_path and "r" in mode: + raise OSError("simulated read failure") + return real_open(file, mode, *args, **kwargs) + + monkeypatch.setattr(builtins, "open", _raising_open) + with pytest.raises(OSError): + WorkflowRegistry(project_dir) + + @pytest.mark.skipif(not hasattr(os, "symlink"), reason="symlinks are unavailable") + def test_load_symlinked_workflows_dir_fails_closed_not_silently_empty( + self, project_dir + ): + """A symlinked .specify/workflows is the same fail-open hazard as a + read OSError: silently reporting an empty registry lets a read-only + caller (e.g. the bundler's remove path) conclude a workflow isn't + installed, skip removing it, and then delete the bundle record -- + leaving the workflow untracked but still on disk. Raise here too, + exactly like the unreadable-file case, so callers cannot act on + fabricated empty state.""" + from specify_cli.workflows.catalog import WorkflowRegistry + import json as _json + + outside = project_dir.parent / "outside-workflows" + outside.mkdir(parents=True, exist_ok=True) + (outside / "workflow-registry.json").write_text( + _json.dumps({"schema_version": "1.0", "workflows": {"evil": {}}}), + encoding="utf-8", + ) + workflows_link = project_dir / ".specify" / "workflows" + workflows_link.rmdir() + workflows_link.symlink_to(outside, target_is_directory=True) + + with pytest.raises(OSError): + WorkflowRegistry(project_dir) # ===== Workflow Catalog Tests ===== @@ -6120,6 +6294,143 @@ def test_remove_refuses_non_directory_workflow_path(self, project_dir, monkeypat assert workflow_path.read_text(encoding="utf-8") == "not a directory" assert WorkflowRegistry(project_dir).is_installed("test-wf") + @pytest.mark.parametrize("error_type", [OSError, TypeError, ValueError]) + def test_remove_registry_save_failure_preserves_files_and_registry( + self, project_dir, monkeypatch, error_type + ): + """If persisting the registry removal fails, the workflow's files must + not have already been deleted: the CLI must not delete files before the + registry successfully records the removal, and it must fail cleanly.""" + from typer.testing import CliRunner + from specify_cli import app + from specify_cli.workflows.catalog import WorkflowRegistry + + registry = WorkflowRegistry(project_dir) + registry.add("test-wf", {"name": "Test", "version": "1.0.0"}) + workflow_dir = project_dir / ".specify" / "workflows" / "test-wf" + workflow_dir.mkdir(parents=True, exist_ok=True) + (workflow_dir / "workflow.yml").write_text("keep-me", encoding="utf-8") + + def boom(self): + raise error_type("save failed") + + monkeypatch.chdir(project_dir) + with pytest.MonkeyPatch.context() as mp: + mp.setattr(WorkflowRegistry, "save", boom) + result = CliRunner().invoke(app, ["workflow", "remove", "test-wf"]) + + assert result.exit_code != 0 + assert result.exception is None or isinstance(result.exception, SystemExit) + assert result.output.strip() != "" + # Files must survive a registry-save failure. + assert (workflow_dir / "workflow.yml").read_text(encoding="utf-8") == "keep-me" + # The on-disk registry must still claim the workflow installed. + assert WorkflowRegistry(project_dir).is_installed("test-wf") + # The directory must be restored to its exact original location, with + # no leftover staging directory from the stage/restore-on-failure + # sequence. + entries = [ + p.name + for p in (project_dir / ".specify" / "workflows").iterdir() + if p.name != "workflow-registry.json" + ] + assert entries == ["test-wf"] + + def test_remove_staged_cleanup_failure_reports_warning_not_error( + self, project_dir, monkeypatch + ): + """The directory is staged (atomically renamed out of + .specify/workflows/) *before* the registry write, and the actual + deletion of the staged directory only happens *after* the registry + has already durably recorded the removal. If that final deletion + fails, the registry write already succeeded and must stand -- an + "Error: Failed to remove..." message at that point would contradict + the registry, which is exactly the incoherent state this staging + order exists to prevent. It must be reported as a cleanup warning, + and the command must still succeed.""" + from typer.testing import CliRunner + from specify_cli import app + from specify_cli.workflows.catalog import WorkflowRegistry + + registry = WorkflowRegistry(project_dir) + registry.add("test-wf", {"name": "Test", "version": "1.0.0"}) + workflow_dir = project_dir / ".specify" / "workflows" / "test-wf" + workflow_dir.mkdir(parents=True, exist_ok=True) + (workflow_dir / "workflow.yml").write_text("keep-me", encoding="utf-8") + + def boom(*args, **kwargs): + raise OSError("permission denied") + + monkeypatch.chdir(project_dir) + with pytest.MonkeyPatch.context() as mp: + mp.setattr("shutil.rmtree", boom) + result = CliRunner().invoke(app, ["workflow", "remove", "test-wf"]) + + assert result.exit_code == 0 + assert "Warning" in result.output + # The registry write already committed -- it must stand. + assert not WorkflowRegistry(project_dir).is_installed("test-wf") + # The original install path is gone (staged away before the registry + # write ever ran); only a leftover staged directory remains, never + # at the original path the registry/CLI would treat as installed. + assert not workflow_dir.exists() + leftovers = [ + p + for p in (project_dir / ".specify" / "workflows").iterdir() + if p.name != "workflow-registry.json" + ] + assert len(leftovers) == 1 + assert (leftovers[0] / "workflow.yml").read_text(encoding="utf-8") == "keep-me" + + def test_remove_stage_restore_failure_escapes_rich_markup( + self, temp_dir, monkeypatch + ): + """When the registry write fails (already rolled back in-memory by + WorkflowRegistry.remove()) and the attempt to rename the staged + directory back to its original location also fails, both the + restore exception and the registry-update exception interpolated + into these warning/error messages must be escaped like every other + error path here.""" + from typer.testing import CliRunner + from specify_cli import app + from specify_cli.workflows.catalog import WorkflowRegistry + + project_dir = temp_dir / "weird[project]" + project_dir.mkdir() + (project_dir / ".specify").mkdir() + (project_dir / ".specify" / "workflows").mkdir() + + registry = WorkflowRegistry(project_dir) + registry.add("test-wf", {"name": "Test", "version": "1.0.0"}) + workflow_dir = project_dir / ".specify" / "workflows" / "test-wf" + workflow_dir.mkdir(parents=True, exist_ok=True) + (workflow_dir / "workflow.yml").write_text("keep-me", encoding="utf-8") + + def save_boom(self): + raise OSError("[reg] disk full") + + real_rename = os.rename + rename_calls = {"n": 0} + + def rename_boom(src, dst): + rename_calls["n"] += 1 + if rename_calls["n"] == 1: + # Allow the initial stage-out rename to succeed so the + # restore-back rename (the second call) is what fails. + return real_rename(src, dst) + raise OSError("[stage] permission denied") + + monkeypatch.chdir(project_dir) + with pytest.MonkeyPatch.context() as mp: + mp.setattr(WorkflowRegistry, "save", save_boom) + mp.setattr(os, "rename", rename_boom) + result = CliRunner().invoke(app, ["workflow", "remove", "test-wf"]) + + assert result.exit_code != 0 + assert result.exception is None or isinstance(result.exception, SystemExit) + output_compact = "".join(result.output.split()) + assert "[stage]permissiondenied" in output_compact + assert "[reg]diskfull" in output_compact class TestWorkflowAddSymlinkGuard: def test_add_malformed_ipv6_url_exits_cleanly(self, temp_dir, monkeypatch): @@ -6172,6 +6483,34 @@ def test_add_refuses_symlinked_workflows_dir(self, temp_dir, monkeypatch): assert result.exit_code != 0 assert "symlinked .specify/workflows" in result.output + def test_add_escapes_rich_markup_in_validation_errors(self, temp_dir, monkeypatch): + """User-controlled YAML values in validation errors must not be parsed as Rich markup.""" + from typer.testing import CliRunner + from specify_cli import app + + (temp_dir / ".specify" / "workflows").mkdir(parents=True) + src = temp_dir / "incoming.yml" + src.write_text( + """ +schema_version: "1.0" +workflow: + id: "markup-wf" + name: "Markup" + version: "[bold]bad[/bold]" + +steps: + - id: step-one + command: speckit.specify +""", + encoding="utf-8", + ) + + monkeypatch.chdir(temp_dir) + result = CliRunner().invoke(app, ["workflow", "add", str(src)]) + + assert result.exit_code != 0 + assert "[bold]bad[/bold]" in result.output + @pytest.mark.skipif(not hasattr(os, "symlink"), reason="symlinks are unavailable") def test_add_refuses_symlinked_id_dir(self, temp_dir, monkeypatch, sample_workflow_yaml): """A symlinked install dir must not let a copy escape the project root.""" @@ -6352,6 +6691,71 @@ def _fake_get_step_info(self, step_id): assert result.exit_code != 0 assert "Refusing to use symlinked step directory" in result.output + def test_add_rejects_oversized_step_response(self, project_dir, monkeypatch): + from typer.testing import CliRunner + from specify_cli import app + from specify_cli.workflows import _commands as wf_commands + from specify_cli.workflows.catalog import StepCatalog + from specify_cli.authentication import http as auth_http + + monkeypatch.chdir(project_dir) + monkeypatch.setattr(wf_commands, "_MAX_WORKFLOW_YAML_BYTES", 100) + monkeypatch.setattr( + StepCatalog, + "get_step_info", + lambda self, step_id: { + "id": step_id, + "name": "Test Step", + "url": "https://example.com/step.yml", + "init_url": "https://example.com/__init__.py", + "_install_allowed": True, + }, + ) + + class _FakeResponse: + def __init__(self, url): + self.url = url + self.body = b"x" * 500 + self.offset = 0 + + def __enter__(self): + return self + + def __exit__(self, exc_type, exc, tb): + return False + + def getheader(self, name): + return None + + def geturl(self): + return self.url + + def read(self, size=-1): + if size < 0: + size = len(self.body) - self.offset + chunk = self.body[self.offset : self.offset + size] + self.offset += len(chunk) + return chunk + + monkeypatch.setattr( + auth_http, + "open_url", + lambda url, timeout=30, redirect_validator=None: _FakeResponse(url), + ) + + result = CliRunner().invoke( + app, ["workflow", "step", "add", "my-step"] + ) + + assert result.exit_code != 0 + assert ( + "responseexceedsthe100-byteworkflowsizelimit" + in "".join(result.output.split()) + ) + assert not ( + project_dir / ".specify" / "workflows" / "steps" / "my-step" + ).exists() + def test_add_rejects_non_string_extra_files_key(self, project_dir, monkeypatch): from typer.testing import CliRunner from specify_cli import app @@ -6382,7 +6786,10 @@ def __enter__(self): def __exit__(self, exc_type, exc, tb): return False - def read(self): + def read(self, size=-1): + if getattr(self, "_read", False): + return b"" + self._read = True if self.url.endswith("/step.yml"): return b"step:\n type_key: my-step\n" return b"" @@ -6390,7 +6797,7 @@ def read(self): def geturl(self): return self.url - def _fake_open_url(url, timeout=30): + def _fake_open_url(url, timeout=30, redirect_validator=None): return _FakeResponse(url) monkeypatch.setattr(StepCatalog, "get_step_info", _fake_get_step_info) @@ -6441,7 +6848,10 @@ def __enter__(self): def __exit__(self, exc_type, exc, tb): return False - def read(self): + def read(self, size=-1): + if getattr(self, "_read", False): + return b"" + self._read = True if self.url.endswith("/step.yml"): return b"step:\n type_key: my-step\n" return b"" @@ -6449,7 +6859,7 @@ def read(self): def geturl(self): return self.url - def _fake_open_url(url, timeout=30): + def _fake_open_url(url, timeout=30, redirect_validator=None): return _FakeResponse(url) monkeypatch.setattr(StepCatalog, "get_step_info", _fake_get_step_info) @@ -6489,7 +6899,10 @@ def __enter__(self): def __exit__(self, exc_type, exc, tb): return False - def read(self): + def read(self, size=-1): + if getattr(self, "_read", False): + return b"" + self._read = True if self.url.endswith("/step.yml"): return b"step:\n type_key: my-step\n" return b"" @@ -6497,7 +6910,7 @@ def read(self): def geturl(self): return self.url - def _fake_open_url(url, timeout=30): + def _fake_open_url(url, timeout=30, redirect_validator=None): return _FakeResponse(url) monkeypatch.setattr(StepCatalog, "get_step_info", _fake_get_step_info) @@ -6806,8 +7219,16 @@ def __init__(self, data, url=None): self._data = data self._url = url or "https://api.github.com/repos/org/repo/releases/assets/42" - def read(self): - return self._data + def read(self, amt=None): + if not hasattr(self, "_pos"): + self._pos = 0 + if amt is None: + chunk = self._data[self._pos :] + self._pos = len(self._data) + return chunk + chunk = self._data[self._pos : self._pos + amt] + self._pos += len(chunk) + return chunk def geturl(self): return self._url @@ -6818,8 +7239,10 @@ def __enter__(self): def __exit__(self, *a): return False - def fake_open_url(url, timeout=None, extra_headers=None): - captured_urls.append((url, extra_headers, timeout)) + def fake_open_url(url, timeout=None, extra_headers=None, redirect_validator=None): + captured_urls.append( + (url, extra_headers, timeout, redirect_validator) + ) if "releases/tags/" in url: return FakeResponse(json.dumps({ "assets": [{"name": "workflow.yml", "url": "https://api.github.com/repos/org/repo/releases/assets/42"}] @@ -6837,11 +7260,20 @@ def fake_open_url(url, timeout=None, extra_headers=None): assert result.exit_code == 0, result.output assert "Test Workflow" in result.output # First call resolves the release tag with timeout=30 - tag_calls = [(url, h, t) for url, h, t in captured_urls if "releases/tags/" in url] + tag_calls = [ + (url, headers, timeout, validator) + for url, headers, timeout, validator in captured_urls + if "releases/tags/" in url + ] assert len(tag_calls) == 1 assert tag_calls[0][2] == 30 # timeout matches download timeout + assert tag_calls[0][3] is not None # Second call downloads from the resolved asset URL with octet-stream - asset_calls = [(url, h, t) for url, h, t in captured_urls if "releases/assets/" in url] + asset_calls = [ + (url, headers, timeout, validator) + for url, headers, timeout, validator in captured_urls + if "releases/assets/" in url + ] assert len(asset_calls) >= 1 assert asset_calls[0][1] == {"Accept": "application/octet-stream"} @@ -6858,8 +7290,16 @@ def __init__(self, data, url=None): self._data = data self._url = url or "https://api.github.com/repos/org/repo/releases/assets/42" - def read(self): - return self._data + def read(self, amt=None): + if not hasattr(self, "_pos"): + self._pos = 0 + if amt is None: + chunk = self._data[self._pos :] + self._pos = len(self._data) + return chunk + chunk = self._data[self._pos : self._pos + amt] + self._pos += len(chunk) + return chunk def geturl(self): return self._url @@ -6870,7 +7310,7 @@ def __enter__(self): def __exit__(self, *a): return False - def fake_open_url(url, timeout=None, extra_headers=None): + def fake_open_url(url, timeout=None, extra_headers=None, redirect_validator=None): captured_urls.append((url, extra_headers)) return FakeResponse(self.VALID_WORKFLOW_YAML.encode()) @@ -6901,8 +7341,16 @@ def __init__(self, data, url=None): self._data = data self._url = url or "https://api.github.com/repos/org/repo/releases/assets/55" - def read(self): - return self._data + def read(self, amt=None): + if not hasattr(self, "_pos"): + self._pos = 0 + if amt is None: + chunk = self._data[self._pos :] + self._pos = len(self._data) + return chunk + chunk = self._data[self._pos : self._pos + amt] + self._pos += len(chunk) + return chunk def geturl(self): return self._url @@ -6913,8 +7361,8 @@ def __enter__(self): def __exit__(self, *a): return False - def fake_open_url(url, timeout=None, extra_headers=None): - captured_urls.append((url, extra_headers)) + def fake_open_url(url, timeout=None, extra_headers=None, redirect_validator=None): + captured_urls.append((url, extra_headers, redirect_validator)) if "releases/tags/" in url: return FakeResponse(json.dumps({ "assets": [{"name": "workflow.yml", "url": "https://api.github.com/repos/org/repo/releases/assets/55"}] @@ -6950,11 +7398,20 @@ def fake_open_url(url, timeout=None, extra_headers=None): assert result.exit_code == 0, result.output # Should resolve via releases/tags API - tag_calls = [url for url, _ in captured_urls if "releases/tags/" in url] + tag_calls = [ + (url, validator) + for url, _, validator in captured_urls + if "releases/tags/" in url + ] assert len(tag_calls) == 1 - assert "releases/tags/v2.0" in tag_calls[0] + assert "releases/tags/v2.0" in tag_calls[0][0] + assert tag_calls[0][1] is not None # Should download from resolved asset URL with octet-stream - asset_calls = [(url, h) for url, h in captured_urls if "releases/assets/" in url] + asset_calls = [ + (url, headers) + for url, headers, _ in captured_urls + if "releases/assets/" in url + ] assert len(asset_calls) >= 1 assert asset_calls[0][1] == {"Accept": "application/octet-stream"} @@ -6977,8 +7434,16 @@ def __init__(self, data, url=None): self._data = data self._url = url or "https://ghes.example/api/v3/repos/org/repo/releases/assets/42" - def read(self): - return self._data + def read(self, amt=None): + if not hasattr(self, "_pos"): + self._pos = 0 + if amt is None: + chunk = self._data[self._pos :] + self._pos = len(self._data) + return chunk + chunk = self._data[self._pos : self._pos + amt] + self._pos += len(chunk) + return chunk def geturl(self): return self._url @@ -6989,7 +7454,7 @@ def __enter__(self): def __exit__(self, *a): return False - def fake_open_url(url, timeout=None, extra_headers=None): + def fake_open_url(url, timeout=None, extra_headers=None, redirect_validator=None): captured_urls.append((url, extra_headers)) if "releases/tags/" in url: return FakeResponse(json.dumps({ @@ -7032,8 +7497,16 @@ def __init__(self, data, url=None): self._data = data self._url = url or "https://ghes.example/api/v3/repos/org/repo/releases/assets/55" - def read(self): - return self._data + def read(self, amt=None): + if not hasattr(self, "_pos"): + self._pos = 0 + if amt is None: + chunk = self._data[self._pos :] + self._pos = len(self._data) + return chunk + chunk = self._data[self._pos : self._pos + amt] + self._pos += len(chunk) + return chunk def geturl(self): return self._url @@ -7057,7 +7530,7 @@ def __exit__(self, *a): run: "echo hello" """ - def fake_open_url(url, timeout=None, extra_headers=None): + def fake_open_url(url, timeout=None, extra_headers=None, redirect_validator=None): captured_urls.append((url, extra_headers)) if "releases/tags/" in url: return FakeResponse(json.dumps({ @@ -7461,3 +7934,4429 @@ def test_add_non_string_step_id_reports_validation_error( assert result.exit_code == 1 assert result.exception is None or isinstance(result.exception, SystemExit) assert "Step ID" in result.output + + +class TestWorkflowCliAlignment: + """CLI alignment with extension/preset commands (#2342).""" + + WORKFLOW_YAML = """ +schema_version: "1.0" +workflow: + id: "align-wf" + name: "Align Workflow" + version: "{version}" + description: "CLI alignment test workflow" +steps: + - id: step-one + type: shell + run: "echo hello" +""" + + def _write_workflow_dir(self, base, version="1.0.0"): + d = base / "wf-src" + d.mkdir(parents=True, exist_ok=True) + (d / "workflow.yml").write_text( + self.WORKFLOW_YAML.format(version=version), encoding="utf-8" + ) + return d + + def _install_dev(self, runner, app, project_dir): + src = self._write_workflow_dir(project_dir) + result = runner.invoke(app, ["workflow", "add", str(src), "--dev"]) + assert result.exit_code == 0, result.output + return src + + # -- add --dev ----------------------------------------------------- + + def test_add_dev_directory_installs(self, project_dir, monkeypatch): + from typer.testing import CliRunner + from specify_cli import app + from specify_cli.workflows.catalog import WorkflowRegistry + + monkeypatch.chdir(project_dir) + runner = CliRunner() + self._install_dev(runner, app, project_dir) + assert WorkflowRegistry(project_dir).is_installed("align-wf") + + def test_add_dev_yaml_file_installs(self, project_dir, monkeypatch): + from typer.testing import CliRunner + from specify_cli import app + from specify_cli.workflows.catalog import WorkflowRegistry + + monkeypatch.chdir(project_dir) + src = self._write_workflow_dir(project_dir) + runner = CliRunner() + result = runner.invoke(app, ["workflow", "add", str(src / "workflow.yml"), "--dev"]) + assert result.exit_code == 0, result.output + assert WorkflowRegistry(project_dir).is_installed("align-wf") + + def test_add_dev_missing_path_errors(self, project_dir, monkeypatch): + from typer.testing import CliRunner + from specify_cli import app + + monkeypatch.chdir(project_dir) + runner = CliRunner() + result = runner.invoke(app, ["workflow", "add", str(project_dir / "missing"), "--dev"]) + assert result.exit_code != 0 + assert "--dev" in result.output + + def test_add_dev_dir_without_workflow_yml_errors(self, project_dir, monkeypatch): + from typer.testing import CliRunner + from specify_cli import app + + monkeypatch.chdir(project_dir) + empty = project_dir / "empty-src" + empty.mkdir() + runner = CliRunner() + result = runner.invoke(app, ["workflow", "add", str(empty), "--dev"]) + assert result.exit_code != 0 + assert "No workflow.yml found" in result.output + + def test_add_local_dir_without_workflow_yml_errors(self, project_dir, monkeypatch): + """Same as the --dev case, but for the plain local-path fallback (no --dev).""" + from typer.testing import CliRunner + from specify_cli import app + + monkeypatch.chdir(project_dir) + empty = project_dir / "empty-src-[bracket]" + empty.mkdir() + runner = CliRunner() + result = runner.invoke(app, ["workflow", "add", str(empty)]) + assert result.exit_code != 0 + assert "No workflow.yml found" in result.output + assert "[bracket]" in result.output + + def test_add_local_dir_with_workflow_yml_directory_errors_cleanly(self, project_dir, monkeypatch): + """Same as the --dev case, but for the plain local-path fallback (no --dev): + a directory named workflow.yml must not reach open() and leak IsADirectoryError.""" + from typer.testing import CliRunner + from specify_cli import app + + monkeypatch.chdir(project_dir) + src_dir = project_dir / "local-wf" + (src_dir / "workflow.yml").mkdir(parents=True) + runner = CliRunner() + result = runner.invoke(app, ["workflow", "add", str(src_dir)]) + assert result.exit_code != 0 + assert result.exception is None or isinstance(result.exception, SystemExit) + assert "No workflow.yml found" in result.output + + def test_add_yaml_parse_error_escapes_rich_markup(self, project_dir, monkeypatch): + """A YAML syntax error can quote the offending line verbatim; brackets in it must not be Rich markup.""" + from unittest.mock import patch + from typer.testing import CliRunner + from specify_cli import app + from specify_cli.workflows.engine import WorkflowDefinition + + monkeypatch.chdir(project_dir) + bad = project_dir / "bad.yml" + bad.write_text("workflow:\n id: wf\n", encoding="utf-8") + runner = CliRunner() + with patch.object( + WorkflowDefinition, + "from_string", + side_effect=ValueError('bad snippet: "New [Feature]"'), + ): + result = runner.invoke(app, ["workflow", "add", str(bad)]) + assert result.exit_code != 0 + assert 'bad snippet: "New [Feature]"' in result.output + + # -- add --from ---------------------------------------------------- + + class _FakeResponse: + def __init__(self, data, url="https://example.com/workflow.yml", headers=None): + self._data = data + self._url = url + self._pos = 0 + self._headers = headers or {} + + def read(self, amt=None): + if amt is None: + chunk = self._data[self._pos :] + self._pos = len(self._data) + return chunk + chunk = self._data[self._pos : self._pos + amt] + self._pos += len(chunk) + return chunk + + def getheader(self, name, default=None): + return self._headers.get(name, default) + + def geturl(self): + return self._url + + def __enter__(self): + return self + + def __exit__(self, *a): + return False + + @pytest.mark.parametrize("mode", ["dev", "local", "from"]) + def test_reinstall_preserves_disabled_state( + self, project_dir, monkeypatch, mode + ): + from unittest.mock import patch + from typer.testing import CliRunner + from specify_cli import app + from specify_cli.workflows.catalog import WorkflowRegistry + + monkeypatch.chdir(project_dir) + runner = CliRunner() + src = self._install_dev(runner, app, project_dir) + result = runner.invoke(app, ["workflow", "disable", "align-wf"]) + assert result.exit_code == 0, result.output + + if mode == "dev": + result = runner.invoke( + app, ["workflow", "add", str(src), "--dev"] + ) + elif mode == "local": + result = runner.invoke(app, ["workflow", "add", str(src)]) + else: + data = self.WORKFLOW_YAML.format(version="2.0.0").encode() + with patch( + "specify_cli.authentication.http.open_url", + side_effect=lambda url, timeout=None, extra_headers=None, + redirect_validator=None: self._FakeResponse(data, url), + ): + result = runner.invoke( + app, + [ + "workflow", "add", "align-wf", + "--from", "https://example.com/workflow.yml", + ], + input="y\n", + ) + + assert result.exit_code == 0, result.output + assert WorkflowRegistry(project_dir).get("align-wf")["enabled"] is False + + def test_add_from_url_rejects_oversized_content_length(self, project_dir, monkeypatch): + """A --from download must not trust an advertised Content-Length + alone by reading the whole body first -- it must reject a response + that declares a size over the workflow YAML limit before reading + the (potentially huge) body into memory at all.""" + from unittest.mock import patch + from typer.testing import CliRunner + from specify_cli import app + from specify_cli.workflows import _commands as wf_commands + + monkeypatch.chdir(project_dir) + monkeypatch.setattr(wf_commands, "_MAX_WORKFLOW_YAML_BYTES", 100) + small_body = b"id: align-wf\n" # small actual body; Content-Length lies + runner = CliRunner() + with patch( + "specify_cli.authentication.http.open_url", + side_effect=lambda url, timeout=None, extra_headers=None, redirect_validator=None: self._FakeResponse( + small_body, url, headers={"Content-Length": "1000"} + ), + ): + result = runner.invoke( + app, + ["workflow", "add", "align-wf", "--from", "https://example.com/workflow.yml"], + input="y\n", + ) + assert result.exit_code != 0 + assert result.exception is None or isinstance(result.exception, SystemExit) + assert "exceedingthe100-byteworkflowsizelimit" in "".join(result.output.split()) + + def test_add_from_url_requires_default_deny_confirmation( + self, project_dir, monkeypatch + ): + from unittest.mock import patch + + from typer.testing import CliRunner + from specify_cli import app + + monkeypatch.chdir(project_dir) + with patch( + "specify_cli.authentication.http.open_url", + side_effect=AssertionError("download should not start"), + ): + result = CliRunner().invoke( + app, + [ + "workflow", + "add", + "align-wf", + "--from", + "https://example.com/workflow.yml", + ], + input="n\n", + ) + + assert result.exit_code == 0, result.output + assert "Untrusted Source" in result.output + assert "Cancelled" in result.output + + def test_add_from_url_rejects_oversized_streamed_body_without_content_length( + self, project_dir, monkeypatch + ): + """A chunked/no-Content-Length response must still be capped by + actually counting streamed bytes -- a malicious or misbehaving + server cannot bypass the limit merely by omitting or lying about + Content-Length.""" + from unittest.mock import patch + from typer.testing import CliRunner + from specify_cli import app + from specify_cli.workflows import _commands as wf_commands + + monkeypatch.chdir(project_dir) + monkeypatch.setattr(wf_commands, "_MAX_WORKFLOW_YAML_BYTES", 100) + oversized_body = b"x" * 500 # no Content-Length header at all + runner = CliRunner() + with patch( + "specify_cli.authentication.http.open_url", + side_effect=lambda url, timeout=None, extra_headers=None, redirect_validator=None: self._FakeResponse( + oversized_body, url + ), + ): + result = runner.invoke( + app, + ["workflow", "add", "align-wf", "--from", "https://example.com/workflow.yml"], + input="y\n", + ) + assert result.exit_code != 0 + assert result.exception is None or isinstance(result.exception, SystemExit) + assert "exceedsthe100-byteworkflowsizelimit" in "".join(result.output.split()) + + def test_add_from_url_oversized_streamed_body_leaves_no_temp_file( + self, project_dir, monkeypatch, tmp_path + ): + """A rejected --from download (oversized streamed body, no + Content-Length) must not leave the 0-byte NamedTemporaryFile behind: + the file is created on disk as soon as it is opened (delete=False), + before any bytes are written, so a failure inside the size-limit + check must still clean it up rather than merely erroring out.""" + import tempfile as tempfile_mod + from unittest.mock import patch + from typer.testing import CliRunner + from specify_cli import app + from specify_cli.workflows import _commands as wf_commands + + monkeypatch.chdir(project_dir) + monkeypatch.setattr(wf_commands, "_MAX_WORKFLOW_YAML_BYTES", 100) + scratch_tmp = tmp_path / "scratch-tmp" + scratch_tmp.mkdir() + monkeypatch.setattr(tempfile_mod, "tempdir", str(scratch_tmp)) + oversized_body = b"x" * 500 # no Content-Length header at all + runner = CliRunner() + with patch( + "specify_cli.authentication.http.open_url", + side_effect=lambda url, timeout=None, extra_headers=None, redirect_validator=None: self._FakeResponse( + oversized_body, url + ), + ): + result = runner.invoke( + app, + ["workflow", "add", "align-wf", "--from", "https://example.com/workflow.yml"], + input="y\n", + ) + assert result.exit_code != 0 + assert "exceedsthe100-byteworkflowsizelimit" in "".join(result.output.split()) + leaked = list(scratch_tmp.glob("*.yml")) + assert leaked == [], f"leaked temp files: {leaked}" + + def test_add_from_url_oversized_content_length_leaves_no_temp_file( + self, project_dir, monkeypatch, tmp_path + ): + """Same guarantee for the fail-fast Content-Length rejection path: + it must not even leave a 0-byte temp file behind.""" + import tempfile as tempfile_mod + from unittest.mock import patch + from typer.testing import CliRunner + from specify_cli import app + from specify_cli.workflows import _commands as wf_commands + + monkeypatch.chdir(project_dir) + monkeypatch.setattr(wf_commands, "_MAX_WORKFLOW_YAML_BYTES", 100) + scratch_tmp = tmp_path / "scratch-tmp" + scratch_tmp.mkdir() + monkeypatch.setattr(tempfile_mod, "tempdir", str(scratch_tmp)) + small_body = b"id: align-wf\n" # small actual body; Content-Length lies + runner = CliRunner() + with patch( + "specify_cli.authentication.http.open_url", + side_effect=lambda url, timeout=None, extra_headers=None, redirect_validator=None: self._FakeResponse( + small_body, url, headers={"Content-Length": "1000"} + ), + ): + result = runner.invoke( + app, + ["workflow", "add", "align-wf", "--from", "https://example.com/workflow.yml"], + input="y\n", + ) + assert result.exit_code != 0 + assert "exceedingthe100-byteworkflowsizelimit" in "".join(result.output.split()) + leaked = list(scratch_tmp.glob("*.yml")) + assert leaked == [], f"leaked temp files: {leaked}" + + def test_add_from_url_download_failure_cleanup_error_preserves_original_error( + self, project_dir, monkeypatch, tmp_path + ): + """The --from download-failure branch's `tmp_path.unlink(missing_ok= + True)` can itself raise (e.g. read-only tempdir) before the clean + "Failed to download workflow" message is ever printed, replacing it + with a raw unhandled OSError. A cleanup failure there must be + guarded exactly like the later post-install finally cleanup: warn + about the cleanup failure, then still preserve/report the original + download error via a clean typer.Exit, never a raw traceback.""" + import tempfile as tempfile_mod + from unittest.mock import patch + from typer.testing import CliRunner + from specify_cli import app + from specify_cli.workflows import _commands as wf_commands + from specify_cli.workflows.catalog import WorkflowRegistry + + monkeypatch.chdir(project_dir) + monkeypatch.setattr(wf_commands, "_MAX_WORKFLOW_YAML_BYTES", 100) + scratch_tmp = tmp_path / "scratch-tmp" + scratch_tmp.mkdir() + monkeypatch.setattr(tempfile_mod, "tempdir", str(scratch_tmp)) + oversized_body = b"x" * 500 # no Content-Length header at all + runner = CliRunner() + + real_unlink = Path.unlink + + def unlink_boom(self_path, *args, **kwargs): + if self_path.suffix == ".yml" and self_path.parent == scratch_tmp: + raise OSError("cleanup denied") + return real_unlink(self_path, *args, **kwargs) + + with patch( + "specify_cli.authentication.http.open_url", + side_effect=lambda url, timeout=None, extra_headers=None, redirect_validator=None: self._FakeResponse( + oversized_body, url + ), + ), pytest.MonkeyPatch.context() as mp: + mp.setattr(Path, "unlink", unlink_boom) + result = runner.invoke( + app, + ["workflow", "add", "align-wf", "--from", "https://example.com/workflow.yml"], + input="y\n", + ) + + assert result.exit_code != 0 + assert result.exception is None or isinstance(result.exception, SystemExit) + # Original download error remains present. + assert "exceedsthe100-byteworkflowsizelimit" in "".join(result.output.split()) + # Cleanup failure is reported too, not silently swallowed / crashing. + assert "cleanup denied" in result.output + assert "Warning" in result.output + assert not WorkflowRegistry(project_dir).is_installed("align-wf") + + def test_add_from_url_installs(self, project_dir, monkeypatch): + from unittest.mock import patch + from typer.testing import CliRunner + from specify_cli import app + from specify_cli.workflows.catalog import WorkflowRegistry + + monkeypatch.chdir(project_dir) + data = self.WORKFLOW_YAML.format(version="1.0.0").encode() + runner = CliRunner() + with patch( + "specify_cli.authentication.http.open_url", + side_effect=lambda url, timeout=None, extra_headers=None, redirect_validator=None: self._FakeResponse(data, url), + ): + result = runner.invoke( + app, + ["workflow", "add", "align-wf", "--from", "https://example.com/workflow.yml"], + input="y\n", + ) + assert result.exit_code == 0, result.output + assert WorkflowRegistry(project_dir).is_installed("align-wf") + + def test_add_from_url_temp_cleanup_failure_after_success_still_exits_zero( + self, project_dir, monkeypatch + ): + """An OSError while deleting the --from download's temp file after + _validate_and_install_local() has already committed the file and + registry entry must not surface as an unhandled failure for an + install that already succeeded -- it must be a warning, exit 0.""" + from unittest.mock import patch + from typer.testing import CliRunner + from specify_cli import app + from specify_cli.workflows.catalog import WorkflowRegistry + + monkeypatch.chdir(project_dir) + data = self.WORKFLOW_YAML.format(version="1.0.0").encode() + runner = CliRunner() + + import tempfile + + real_unlink = Path.unlink + + def unlink_boom(self_path, *args, **kwargs): + if self_path.suffix == ".yml" and self_path.parent == Path(tempfile.gettempdir()): + raise OSError("permission denied") + return real_unlink(self_path, *args, **kwargs) + + with patch( + "specify_cli.authentication.http.open_url", + side_effect=lambda url, timeout=None, extra_headers=None, redirect_validator=None: self._FakeResponse(data, url), + ), pytest.MonkeyPatch.context() as mp: + mp.setattr(Path, "unlink", unlink_boom) + result = runner.invoke( + app, + ["workflow", "add", "align-wf", "--from", "https://example.com/workflow.yml"], + input="y\n", + ) + + assert result.exit_code == 0, result.output + assert "Warning" in result.output + assert "permissiondenied" in "".join(result.output.split()) + assert WorkflowRegistry(project_dir).is_installed("align-wf") + + def test_add_from_url_id_mismatch_errors(self, project_dir, monkeypatch): + from unittest.mock import patch + from typer.testing import CliRunner + from specify_cli import app + from specify_cli.workflows.catalog import WorkflowRegistry + + monkeypatch.chdir(project_dir) + data = self.WORKFLOW_YAML.format(version="1.0.0").encode() + runner = CliRunner() + with patch( + "specify_cli.authentication.http.open_url", + side_effect=lambda url, timeout=None, extra_headers=None, redirect_validator=None: self._FakeResponse(data, url), + ): + result = runner.invoke( + app, + ["workflow", "add", "other-id", "--from", "https://example.com/workflow.yml"], + input="y\n", + ) + assert result.exit_code != 0 + assert "does not match" in result.output + assert not WorkflowRegistry(project_dir).is_installed("align-wf") + + def test_add_from_empty_url_rejected_not_catalog_fallback(self, project_dir, monkeypatch): + """--from "" must fail URL validation, not silently install from the catalog.""" + from typer.testing import CliRunner + from specify_cli import app + from specify_cli.workflows.catalog import WorkflowRegistry + + monkeypatch.chdir(project_dir) + runner = CliRunner() + result = runner.invoke(app, ["workflow", "add", "align-wf", "--from", ""]) + assert result.exit_code != 0 + assert "HTTPS" in result.output + assert not WorkflowRegistry(project_dir).is_installed("align-wf") + + def test_add_from_url_non_https_redirect_escapes_rich_markup(self, project_dir, monkeypatch): + """A redirect to a non-HTTPS IPv6 literal (legally bracketed) must not be parsed as Rich markup.""" + from unittest.mock import patch + from typer.testing import CliRunner + from specify_cli import app + + monkeypatch.chdir(project_dir) + redirected_url = "http://[2001:db8::1]/workflow.yml" + runner = CliRunner() + with patch( + "specify_cli.authentication.http.open_url", + side_effect=lambda url, timeout=None, extra_headers=None, redirect_validator=None: self._FakeResponse(b"", redirected_url), + ): + result = runner.invoke( + app, + ["workflow", "add", "align-wf", "--from", "https://example.com/workflow.yml"], + input="y\n", + ) + assert result.exit_code != 0 + assert redirected_url in result.output + + def test_add_from_rejects_invalid_source_id_without_fetch(self, project_dir, monkeypatch): + """--from with a non-workflow-id source (URL, path, uppercase) fails before any network fetch.""" + from unittest.mock import patch + from typer.testing import CliRunner + from specify_cli import app + + monkeypatch.chdir(project_dir) + calls: list[str] = [] + + def _fake_open(url, timeout=None, extra_headers=None, redirect_validator=None): + calls.append(url) + raise AssertionError(f"network fetch attempted: {url}") + + runner = CliRunner() + with patch("specify_cli.authentication.http.open_url", side_effect=_fake_open): + for bad_source in ("https://x/y.yml", "./local.yml", "BadCase"): + result = runner.invoke( + app, + ["workflow", "add", bad_source, "--from", "https://example.com/workflow.yml"], + ) + assert result.exit_code != 0 + assert "Invalid workflow ID" in result.output + assert calls == [] + + # -- search --author ----------------------------------------------- + + def test_search_author_filters(self, project_dir, monkeypatch): + from typer.testing import CliRunner + from specify_cli import app + from specify_cli.workflows.catalog import WorkflowCatalog + + monkeypatch.chdir(project_dir) + workflows = { + "wf-a": {"name": "Workflow A", "version": "1.0.0", "description": "", "author": "alice"}, + "wf-b": {"name": "Workflow B", "version": "1.0.0", "description": "", "author": "bob"}, + } + monkeypatch.setattr( + WorkflowCatalog, + "_get_merged_workflows", + lambda self, force_refresh=False: {k: dict(v) for k, v in workflows.items()}, + ) + runner = CliRunner() + result = runner.invoke(app, ["workflow", "search", "--author", "Alice"]) + assert result.exit_code == 0, result.output + assert "wf-a" in result.output + assert "wf-b" not in result.output + + def test_search_escapes_rich_markup_in_catalog_fields(self, project_dir, monkeypatch): + """Catalog-derived name/description/tags must not be parsed as Rich markup.""" + from typer.testing import CliRunner + from specify_cli import app + from specify_cli.workflows.catalog import WorkflowCatalog + + monkeypatch.chdir(project_dir) + workflows = { + "wf-a": { + "name": "Bracket [Search]", + "version": "1.0.0", + "description": "desc [with] brackets", + "tags": ["tag[1]", "tag2"], + }, + } + monkeypatch.setattr( + WorkflowCatalog, + "_get_merged_workflows", + lambda self, force_refresh=False: {k: dict(v) for k, v in workflows.items()}, + ) + runner = CliRunner() + result = runner.invoke(app, ["workflow", "search"]) + assert result.exit_code == 0, result.output + assert "Bracket [Search]" in result.output + assert "desc [with] brackets" in result.output + assert "tag[1]" in result.output + + # -- update ---------------------------------------------------------- + + def test_update_no_workflows_installed(self, project_dir, monkeypatch): + from typer.testing import CliRunner + from specify_cli import app + + monkeypatch.chdir(project_dir) + runner = CliRunner() + result = runner.invoke(app, ["workflow", "update"]) + assert result.exit_code == 0, result.output + assert "No workflows installed" in result.output + + def test_update_not_installed_errors(self, project_dir, monkeypatch): + from typer.testing import CliRunner + from specify_cli import app + + monkeypatch.chdir(project_dir) + runner = CliRunner() + result = runner.invoke(app, ["workflow", "update", "ghost"]) + assert result.exit_code != 0 + assert "not installed" in result.output + + def test_update_skips_non_catalog_sources(self, project_dir, monkeypatch): + from typer.testing import CliRunner + from specify_cli import app + + monkeypatch.chdir(project_dir) + runner = CliRunner() + self._install_dev(runner, app, project_dir) + result = runner.invoke(app, ["workflow", "update"]) + assert result.exit_code == 0, result.output + assert "re-add to update" in result.output + # Every target was skipped — must not claim everything is up to date. + assert "No workflows were eligible for update" in result.output + assert "up to date!" not in result.output + + def test_update_skip_message_accurate_for_bundled_source(self, project_dir, monkeypatch): + """A workflow registered with source "bundled" (e.g. the speckit + workflow installed by `specify init`) was never installed from a + local path or URL; the skip message must not claim otherwise.""" + from typer.testing import CliRunner + from specify_cli import app + from specify_cli.workflows.catalog import WorkflowRegistry + + monkeypatch.chdir(project_dir) + registry = WorkflowRegistry(project_dir) + registry.add( + "speckit", + {"name": "Speckit", "version": "1.0.0", "source": "bundled"}, + ) + runner = CliRunner() + result = runner.invoke(app, ["workflow", "update"]) + assert result.exit_code == 0, result.output + assert "local path or URL" not in result.output + assert "re-add to update" in result.output + + def test_registry_add_rolls_back_memory_on_save_failure(self, project_dir, monkeypatch): + from specify_cli.workflows.catalog import WorkflowRegistry + + registry = WorkflowRegistry(project_dir) + registry.add("align-wf", {"version": "1.0.0", "source": "catalog"}) + + def boom(): + raise OSError("disk full") + + monkeypatch.setattr(registry, "save", boom) + with pytest.raises(OSError): + registry.add("align-wf", {"version": "2.0.0", "source": "catalog"}) + assert registry.get("align-wf")["version"] == "1.0.0" + + with pytest.raises(OSError): + registry.add("other-wf", {"version": "1.0.0", "source": "catalog"}) + assert registry.get("other-wf") is None + + @pytest.mark.parametrize("error_type", [TypeError, ValueError]) + def test_registry_add_rolls_back_memory_on_serialization_failure( + self, project_dir, monkeypatch, error_type + ): + from specify_cli.workflows.catalog import WorkflowRegistry + + registry = WorkflowRegistry(project_dir) + registry.add("align-wf", {"version": "1.0.0", "source": "catalog"}) + + def boom(): + raise error_type("not JSON serializable") + + monkeypatch.setattr(registry, "save", boom) + with pytest.raises(error_type): + registry.add("align-wf", {"version": "2.0.0", "source": "catalog"}) + assert registry.get("align-wf")["version"] == "1.0.0" + + with pytest.raises(error_type): + registry.add("other-wf", {"version": "1.0.0", "source": "catalog"}) + assert registry.get("other-wf") is None + + def test_registry_add_survives_non_dict_existing_entry(self, project_dir): + from specify_cli.workflows.catalog import WorkflowRegistry + + registry = WorkflowRegistry(project_dir) + registry.data["workflows"]["align-wf"] = "corrupted" + registry.add("align-wf", {"version": "1.0.0", "source": "catalog"}) + assert registry.get("align-wf")["version"] == "1.0.0" + + @pytest.mark.parametrize( + "contents", + [ + "not json", + "[]", + '{"schema_version": "1.0"}', + '{"schema_version": "1.0", "workflows": "broken"}', + ], + ) + def test_registry_load_rejects_corrupt_contents( + self, project_dir, contents + ): + from specify_cli.workflows.catalog import WorkflowRegistry + + registry = WorkflowRegistry(project_dir) + registry.workflows_dir.mkdir(parents=True, exist_ok=True) + registry.registry_path.write_text(contents, encoding="utf-8") + + with pytest.raises(OSError, match="corrupt"): + WorkflowRegistry(project_dir) + + assert registry.registry_path.read_text(encoding="utf-8") == contents + + def test_registry_save_refuses_symlinked_parent(self, project_dir, tmp_path): + """Construction now fails closed on a symlinked .specify just like + an unreadable registry file: a symlinked parent must never be + silently tolerated up to save() -- it must raise immediately.""" + from specify_cli.workflows.catalog import WorkflowRegistry + + outside = tmp_path / "outside-specify" + outside.mkdir() + specify_dir = project_dir / ".specify" + if specify_dir.exists(): + shutil.rmtree(specify_dir) + specify_dir.symlink_to(outside) + with pytest.raises(OSError, match="symlink"): + WorkflowRegistry(project_dir) + assert not (outside / "workflows").exists() + + @pytest.mark.skipif(sys.platform == "win32", reason="chmod mode bits not reliable on Windows") + def test_registry_save_preserves_existing_file_mode(self, project_dir): + """A registry shared as 0640/0644 must keep that mode after a save, + not be silently replaced by mkstemp's 0600 default -- otherwise + every add/remove locks other project users out of a previously + shared registry file.""" + from specify_cli.workflows.catalog import WorkflowRegistry + + registry = WorkflowRegistry(project_dir) + registry.add("first-wf", {"name": "First"}) + registry.registry_path.chmod(0o644) + + registry.add("second-wf", {"name": "Second"}) + + mode = stat.S_IMODE(registry.registry_path.stat().st_mode) + assert mode == 0o644, f"expected 0644, got {oct(mode)}" + + @pytest.mark.skipif(not hasattr(os, "fchown"), reason="os.fchown is unavailable") + def test_registry_save_preserves_existing_owner_group( + self, project_dir, monkeypatch + ): + from specify_cli.workflows.catalog import WorkflowRegistry + import specify_cli.workflows.catalog as catalog_mod + + registry = WorkflowRegistry(project_dir) + registry.add("first-wf", {"name": "First"}) + existing = registry.registry_path.stat() + calls: list[tuple[os.stat_result, int, int]] = [] + + monkeypatch.setattr( + catalog_mod.os, + "fchown", + lambda fd, uid, gid: calls.append((os.fstat(fd), uid, gid)), + ) + registry.add("second-wf", {"name": "Second"}) + + assert len(calls) == 1 + temp_stat, uid, gid = calls[0] + assert stat.S_ISREG(temp_stat.st_mode) + assert uid == existing.st_uid + assert gid == existing.st_gid + + @pytest.mark.skipif(sys.platform == "win32", reason="chmod mode bits not reliable on Windows") + def test_registry_save_on_new_registry_uses_secure_default_mode(self, project_dir): + """A brand-new registry file (no prior mode to preserve) should keep + mkstemp's secure 0600 default rather than something more permissive.""" + from specify_cli.workflows.catalog import WorkflowRegistry + + registry = WorkflowRegistry(project_dir) + registry.add("first-wf", {"name": "First"}) + + mode = stat.S_IMODE(registry.registry_path.stat().st_mode) + assert mode == 0o600, f"expected 0600, got {oct(mode)}" + + @pytest.mark.skipif(not hasattr(os, "symlink"), reason="symlinks are unavailable") + def test_registry_save_rejects_swapped_temp_without_touching_target( + self, project_dir, monkeypatch + ): + from specify_cli.workflows.catalog import WorkflowRegistry + import specify_cli.workflows.catalog as catalog_mod + + registry = WorkflowRegistry(project_dir) + registry.add("first-wf", {"name": "First"}) + registry.registry_path.chmod(0o640) + + victim = project_dir / "victim.txt" + victim.write_text("untouched", encoding="utf-8") + victim.chmod(0o600) + + tmp_path = None + real_mkstemp = catalog_mod.tempfile.mkstemp + real_dump = catalog_mod.json.dump + + def tracking_mkstemp(*args, **kwargs): + nonlocal tmp_path + fd, name = real_mkstemp(*args, **kwargs) + tmp_path = Path(name) + return fd, name + + def swap_after_dump(*args, **kwargs): + result = real_dump(*args, **kwargs) + assert tmp_path is not None + tmp_path.unlink() + tmp_path.symlink_to(victim) + return result + + monkeypatch.setattr(catalog_mod.tempfile, "mkstemp", tracking_mkstemp) + monkeypatch.setattr(catalog_mod.json, "dump", swap_after_dump) + + with pytest.raises(OSError): + registry.add("second-wf", {"name": "Second"}) + + assert victim.read_text(encoding="utf-8") == "untouched" + if sys.platform != "win32": + assert stat.S_IMODE(victim.stat().st_mode) == 0o600 + assert not registry.registry_path.is_symlink() + assert WorkflowRegistry(project_dir).is_installed("first-wf") + + def test_add_dev_dir_with_workflow_yml_directory_errors_cleanly(self, project_dir, monkeypatch): + from typer.testing import CliRunner + from specify_cli import app + + monkeypatch.chdir(project_dir) + dev_dir = project_dir / "dev-wf" + (dev_dir / "workflow.yml").mkdir(parents=True) + runner = CliRunner() + result = runner.invoke(app, ["workflow", "add", "--dev", str(dev_dir)]) + assert result.exit_code != 0 + assert result.exception is None or isinstance(result.exception, SystemExit) + assert "No workflow.yml found" in result.output + + @pytest.mark.parametrize("mode", ["dev", "local", "from_url"]) + def test_add_save_failure_leaves_no_orphan_directory(self, project_dir, monkeypatch, mode): + """A registry.add() save failure during a fresh install must not leave + an orphaned workflow directory on disk, and must fail with a clean + escaped message instead of a raw OSError traceback. Shared by --dev, + the plain local-path fallback, and --from since all three funnel + through _validate_and_install_local's single install choke point.""" + import contextlib + from unittest.mock import patch + from typer.testing import CliRunner + from specify_cli import app + from specify_cli.workflows.catalog import WorkflowRegistry + + monkeypatch.chdir(project_dir) + runner = CliRunner() + + def boom(self): + raise OSError("disk full") + + if mode == "from_url": + data = self.WORKFLOW_YAML.format(version="1.0.0").encode() + args = ["workflow", "add", "align-wf", "--from", "https://example.com/workflow.yml"] + url_patch = patch( + "specify_cli.authentication.http.open_url", + side_effect=lambda url, timeout=None, extra_headers=None, redirect_validator=None: self._FakeResponse(data, url), + ) + else: + src = self._write_workflow_dir(project_dir) + args = ["workflow", "add", str(src)] + (["--dev"] if mode == "dev" else []) + url_patch = contextlib.nullcontext() + + with url_patch, pytest.MonkeyPatch.context() as mp: + mp.setattr(WorkflowRegistry, "save", boom) + result = runner.invoke( + app, args, input="y\n" if mode == "from_url" else None + ) + + assert result.exit_code != 0 + assert result.exception is None or isinstance(result.exception, SystemExit) + assert result.output.strip() != "" + dest_dir = project_dir / ".specify" / "workflows" / "align-wf" + assert not dest_dir.exists() + assert not WorkflowRegistry(project_dir).is_installed("align-wf") + + @pytest.mark.parametrize("mode", ["dev", "catalog"]) + def test_add_non_json_description_rolls_back_transaction( + self, project_dir, monkeypatch, mode + ): + import contextlib + from unittest.mock import patch + + from typer.testing import CliRunner + from specify_cli import app + from specify_cli.workflows.catalog import WorkflowCatalog, WorkflowRegistry + + monkeypatch.chdir(project_dir) + data = self.WORKFLOW_YAML.format(version="1.0.0").replace( + 'description: "CLI alignment test workflow"', + "description: 2026-01-02", + ).encode() + + if mode == "dev": + source = project_dir / "dated-description" + source.mkdir() + (source / "workflow.yml").write_bytes(data) + args = ["workflow", "add", str(source), "--dev"] + download = contextlib.nullcontext() + else: + monkeypatch.setattr( + WorkflowCatalog, + "get_workflow_info", + lambda self, wid: { + "id": wid, + "name": "Align Workflow", + "version": "1.0.0", + "url": "https://example.com/workflow.yml", + "_install_allowed": True, + "_catalog_name": "test-catalog", + }, + ) + args = ["workflow", "add", "align-wf"] + download = patch( + "specify_cli.authentication.http.open_url", + side_effect=lambda url, timeout=None, extra_headers=None, + redirect_validator=None: self._FakeResponse(data, url), + ) + + with download: + result = CliRunner().invoke(app, args) + + assert result.exit_code != 0 + assert result.exception is None or isinstance(result.exception, SystemExit) + assert "Failed to update workflow registry" in result.output + dest_dir = project_dir / ".specify" / "workflows" / "align-wf" + assert not dest_dir.exists() + assert not WorkflowRegistry(project_dir).is_installed("align-wf") + + @pytest.mark.parametrize("mode", ["dev", "local", "from_url"]) + def test_add_fresh_install_mkstemp_failure_leaves_no_orphan_directory( + self, project_dir, monkeypatch, mode + ): + """_stage_workflow_file() does mkdir(dest_dir) then mkstemp() inside + it. For a fresh install (no prior directory), if mkdir succeeds but + mkstemp then fails (disk full/EMFILE/quota), the freshly-created + empty dest_dir must not be left orphaned -- it must be removed, and + the original mkstemp error must still be reported cleanly.""" + import contextlib + from unittest.mock import patch + from typer.testing import CliRunner + from specify_cli import app + from specify_cli.workflows.catalog import WorkflowRegistry + + monkeypatch.chdir(project_dir) + runner = CliRunner() + + def boom(*args, **kwargs): + raise OSError("disk full") + + if mode == "from_url": + data = self.WORKFLOW_YAML.format(version="1.0.0").encode() + args = ["workflow", "add", "align-wf", "--from", "https://example.com/workflow.yml"] + url_patch = patch( + "specify_cli.authentication.http.open_url", + side_effect=lambda url, timeout=None, extra_headers=None, redirect_validator=None: self._FakeResponse(data, url), + ) + else: + src = self._write_workflow_dir(project_dir) + args = ["workflow", "add", str(src)] + (["--dev"] if mode == "dev" else []) + url_patch = contextlib.nullcontext() + + with url_patch, pytest.MonkeyPatch.context() as mp: + mp.setattr("tempfile.mkstemp", boom) + result = runner.invoke( + app, args, input="y\n" if mode == "from_url" else None + ) + + assert result.exit_code != 0 + assert result.exception is None or isinstance(result.exception, SystemExit) + assert result.output.strip() != "" + dest_dir = project_dir / ".specify" / "workflows" / "align-wf" + assert not dest_dir.exists(), "fresh-install dest_dir left orphaned after mkstemp failure" + assert not WorkflowRegistry(project_dir).is_installed("align-wf") + + def test_add_reinstall_mkstemp_failure_preserves_preexisting_directory( + self, project_dir, monkeypatch + ): + """A pre-existing (reinstall) dest_dir must never be removed by the + mkstemp-failure cleanup -- only a directory _stage_workflow_file + itself just created.""" + from typer.testing import CliRunner + from specify_cli import app + from specify_cli.workflows.catalog import WorkflowRegistry + + monkeypatch.chdir(project_dir) + runner = CliRunner() + src = self._install_dev(runner, app, project_dir) + installed_yaml = project_dir / ".specify" / "workflows" / "align-wf" / "workflow.yml" + original_bytes = installed_yaml.read_bytes() + original_registry_entry = WorkflowRegistry(project_dir).get("align-wf") + + (src / "workflow.yml").write_text( + self.WORKFLOW_YAML.format(version="2.0.0"), encoding="utf-8" + ) + + def boom(*args, **kwargs): + raise OSError("disk full") + + with pytest.MonkeyPatch.context() as mp: + mp.setattr("tempfile.mkstemp", boom) + result = runner.invoke(app, ["workflow", "add", str(src), "--dev"]) + + assert result.exit_code != 0 + assert result.exception is None or isinstance(result.exception, SystemExit) + assert result.output.strip() != "" + assert installed_yaml.parent.is_dir() + assert installed_yaml.read_bytes() == original_bytes + assert WorkflowRegistry(project_dir).get("align-wf") == original_registry_entry + + @pytest.mark.skipif(not hasattr(os, "symlink"), reason="symlinks are unavailable") + @pytest.mark.parametrize("mode", ["dev", "catalog"]) + def test_stage_write_rejects_swapped_symlink( + self, project_dir, monkeypatch, mode + ): + from typer.testing import CliRunner + from specify_cli import app + from specify_cli.workflows import _commands + from specify_cli.workflows.catalog import WorkflowCatalog + + monkeypatch.chdir(project_dir) + victim = project_dir / "victim.txt" + victim.write_text("untouched", encoding="utf-8") + + real_stage = _commands._stage_workflow_file + + def raced_stage(*args, **kwargs): + staged = real_stage(*args, **kwargs) + staged_path = getattr(staged, "path", staged) + staged_path.unlink() + staged_path.symlink_to(victim) + return staged + + monkeypatch.setattr(_commands, "_stage_workflow_file", raced_stage) + + if mode == "dev": + source = self._write_workflow_dir(project_dir) + args = ["workflow", "add", str(source), "--dev"] + else: + monkeypatch.setattr( + WorkflowCatalog, + "get_workflow_info", + lambda self, wid: { + "id": wid, + "name": "Align Workflow", + "version": "1.0.0", + "url": "https://example.com/workflow.yml", + "_install_allowed": True, + "_catalog_name": "test-catalog", + }, + ) + data = self.WORKFLOW_YAML.format(version="1.0.0").encode() + monkeypatch.setattr( + "specify_cli.authentication.http.open_url", + lambda url, timeout=None, extra_headers=None, + redirect_validator=None: self._FakeResponse(data, url), + ) + args = ["workflow", "add", "align-wf"] + + result = CliRunner().invoke(app, args) + + assert result.exit_code != 0 + assert victim.read_text(encoding="utf-8") == "untouched" + + def test_local_install_writes_the_same_bytes_it_validates( + self, project_dir, monkeypatch + ): + from typer.testing import CliRunner + from specify_cli import app + from specify_cli.workflows import _commands + from specify_cli.workflows.catalog import WorkflowRegistry + + monkeypatch.chdir(project_dir) + source = self._write_workflow_dir(project_dir) + source_file = source / "workflow.yml" + validated_content = source_file.read_text(encoding="utf-8") + replacement_content = self.WORKFLOW_YAML.format(version="9.9.9") + + real_stage = _commands._stage_workflow_file + + def replace_source_after_validation(*args, **kwargs): + staged = real_stage(*args, **kwargs) + source_file.write_text(replacement_content, encoding="utf-8") + return staged + + monkeypatch.setattr( + _commands, + "_stage_workflow_file", + replace_source_after_validation, + ) + + result = CliRunner().invoke( + app, ["workflow", "add", str(source), "--dev"] + ) + + assert result.exit_code == 0, result.output + installed_file = ( + project_dir + / ".specify" + / "workflows" + / "align-wf" + / "workflow.yml" + ) + assert installed_file.read_text(encoding="utf-8") == validated_content + assert WorkflowRegistry(project_dir).get("align-wf")["version"] == "1.0.0" + + def test_add_fresh_install_staged_discard_cleanup_failure_reports_warning( + self, project_dir, monkeypatch + ): + """A genuine fresh-directory rmdir failure must be reported while + the original copy failure remains the primary error.""" + from typer.testing import CliRunner + from specify_cli import app + from specify_cli.workflows import _commands + from specify_cli.workflows.catalog import WorkflowRegistry + + monkeypatch.chdir(project_dir) + runner = CliRunner() + src = self._write_workflow_dir(project_dir) + + def copy_boom(self, data): + raise OSError("disk full") + + real_rmdir = Path.rmdir + + def rmdir_boom(path): + if path.name == "align-wf": + raise OSError("cleanup denied") + return real_rmdir(path) + + with pytest.MonkeyPatch.context() as mp: + mp.setattr(_commands._StagedWorkflowFile, "write_bytes", copy_boom) + mp.setattr(Path, "rmdir", rmdir_boom) + result = runner.invoke(app, ["workflow", "add", str(src), "--dev"]) + + assert result.exit_code != 0 + assert result.exception is None or isinstance(result.exception, SystemExit) + # Original install error remains present and primary. + assert "disk full" in result.output + # Cleanup failure is now reported, not silently swallowed. + assert "cleanup denied" in result.output + assert "Warning" in result.output + assert not WorkflowRegistry(project_dir).is_installed("align-wf") + + def test_add_fresh_install_registry_rollback_cleanup_failure_reports_warning( + self, project_dir, monkeypatch + ): + """A fresh-install rollback directory-removal failure must be + reported while the registry-update error remains primary.""" + from typer.testing import CliRunner + from specify_cli import app + from specify_cli.workflows.catalog import WorkflowRegistry + + monkeypatch.chdir(project_dir) + runner = CliRunner() + src = self._write_workflow_dir(project_dir) + + def save_boom(self): + raise OSError("registry disk full") + + real_rmdir = Path.rmdir + + def rmdir_boom(path): + if path.name == "align-wf": + raise OSError("cleanup denied") + return real_rmdir(path) + + with pytest.MonkeyPatch.context() as mp: + mp.setattr(WorkflowRegistry, "save", save_boom) + mp.setattr(Path, "rmdir", rmdir_boom) + result = runner.invoke(app, ["workflow", "add", str(src), "--dev"]) + + assert result.exit_code != 0 + assert result.exception is None or isinstance(result.exception, SystemExit) + # Original registry-update error remains present and primary. + assert "registry disk full" in result.output + # Cleanup failure is now reported, not silently swallowed. + assert "cleanup denied" in result.output + assert "Warning" in result.output + + def test_add_dev_reinstall_copy_failure_leaves_prior_file_untouched( + self, project_dir, monkeypatch + ): + """A staged descriptor-copy failure cannot touch the prior installed + workflow or leave a staging file behind.""" + from typer.testing import CliRunner + from specify_cli import app + from specify_cli.workflows import _commands + from specify_cli.workflows.catalog import WorkflowRegistry + + monkeypatch.chdir(project_dir) + runner = CliRunner() + src = self._install_dev(runner, app, project_dir) + installed_yaml = project_dir / ".specify" / "workflows" / "align-wf" / "workflow.yml" + original_bytes = installed_yaml.read_bytes() + original_registry_entry = WorkflowRegistry(project_dir).get("align-wf") + + # Point --dev at a new version of the same workflow to trigger a + # reinstall (overwrite) rather than a fresh install. + (src / "workflow.yml").write_text( + self.WORKFLOW_YAML.format(version="2.0.0"), encoding="utf-8" + ) + + def boom(staged, data): + # Simulate a truncating partial write followed by an OSError on + # the reserved staging inode, mirroring disk exhaustion. + os.ftruncate(staged.fd, 0) + raise OSError("disk full") + + with pytest.MonkeyPatch.context() as mp: + mp.setattr(_commands._StagedWorkflowFile, "write_bytes", boom) + result = runner.invoke(app, ["workflow", "add", str(src), "--dev"]) + + assert result.exit_code != 0 + assert result.exception is None or isinstance(result.exception, SystemExit) + assert result.output.strip() != "" + assert installed_yaml.read_bytes() == original_bytes + assert WorkflowRegistry(project_dir).get("align-wf") == original_registry_entry + # No orphaned staging file left behind in the workflow directory. + leftovers = [p.name for p in installed_yaml.parent.iterdir() if p.name != "workflow.yml"] + assert leftovers == [] + + def test_add_dev_successful_reinstall_leaves_no_backup_file( + self, project_dir, monkeypatch + ): + """Once registry.add() succeeds, the unique rollback backup must be + discarded rather than left as a permanent orphan sibling.""" + from typer.testing import CliRunner + from specify_cli import app + from specify_cli.workflows.catalog import WorkflowRegistry + + monkeypatch.chdir(project_dir) + runner = CliRunner() + src = self._install_dev(runner, app, project_dir) + workflow_dir = project_dir / ".specify" / "workflows" / "align-wf" + + # Reinstall (overwrite) with a new version -- a successful reinstall, + # not a failure path. + (src / "workflow.yml").write_text( + self.WORKFLOW_YAML.format(version="2.0.0"), encoding="utf-8" + ) + result = runner.invoke(app, ["workflow", "add", str(src), "--dev"]) + + assert result.exit_code == 0, result.output + registry = WorkflowRegistry(project_dir) + assert registry.is_installed("align-wf") + assert registry.get("align-wf")["version"] == "2.0.0" + assert (workflow_dir / "workflow.yml").read_text(encoding="utf-8") == ( + self.WORKFLOW_YAML.format(version="2.0.0") + ) + leftovers = [p.name for p in workflow_dir.iterdir() if p.name != "workflow.yml"] + assert leftovers == [], f"orphan sibling(s) left behind: {leftovers}" + + def test_add_dev_successful_reinstall_backup_cleanup_failure_still_succeeds( + self, project_dir, monkeypatch + ): + """A failure to clean up the now-unneeded backup file after a + successful registry.add() must not turn the already-successful + install into a reported failure: it must be a warning (exit 0), + consistent with workflow_remove's post-commit cleanup semantics.""" + from typer.testing import CliRunner + from specify_cli import app + from specify_cli.workflows.catalog import WorkflowRegistry + + monkeypatch.chdir(project_dir) + runner = CliRunner() + src = self._install_dev(runner, app, project_dir) + + (src / "workflow.yml").write_text( + self.WORKFLOW_YAML.format(version="2.0.0"), encoding="utf-8" + ) + + real_unlink = Path.unlink + + def unlink_boom(self_path, *args, **kwargs): + if self_path.name.endswith(".bak"): + raise OSError("permission denied") + return real_unlink(self_path, *args, **kwargs) + + with pytest.MonkeyPatch.context() as mp: + mp.setattr(Path, "unlink", unlink_boom) + result = runner.invoke(app, ["workflow", "add", str(src), "--dev"]) + + assert result.exit_code == 0, result.output + assert "Warning" in result.output + assert "permissiondenied" in "".join(result.output.split()) + registry = WorkflowRegistry(project_dir) + assert registry.is_installed("align-wf") + assert registry.get("align-wf")["version"] == "2.0.0" + + def test_add_dev_reinstall_restore_failure_reports_warning_and_original_error( + self, project_dir, monkeypatch + ): + """The prior file is now restored via an atomic rename (not a + content rewrite) when registry.add() fails on a reinstall. If that + restore rename itself also fails (e.g. a transient FS issue), it + must not silently claim success or crash with a raw traceback: it + must report a clear warning about the restore failure in addition + to the original clean registry error.""" + from typer.testing import CliRunner + from specify_cli import app + from specify_cli.workflows.catalog import WorkflowRegistry + + monkeypatch.chdir(project_dir) + runner = CliRunner() + src = self._install_dev(runner, app, project_dir) + + (src / "workflow.yml").write_text( + self.WORKFLOW_YAML.format(version="2.0.0"), encoding="utf-8" + ) + + def save_boom(self): + raise OSError("disk full") + + real_replace = os.replace + calls = {"n": 0} + + def replace_boom(src_path, dst_path): + # The commit swap for a reinstall makes exactly two os.replace + # calls (backup-aside, then staged-into-dest); let both succeed + # and only fail the third call -- the post-registry-failure + # restore-back rename. + calls["n"] += 1 + if calls["n"] <= 2: + return real_replace(src_path, dst_path) + raise OSError("permission denied") + + with pytest.MonkeyPatch.context() as mp: + mp.setattr(WorkflowRegistry, "save", save_boom) + mp.setattr(os, "replace", replace_boom) + result = runner.invoke(app, ["workflow", "add", str(src), "--dev"]) + + + assert result.exit_code != 0 + assert result.exception is None or isinstance(result.exception, SystemExit) + output_compact = "".join(result.output.split()) + assert "Warning" in result.output + assert "diskfull" in output_compact + assert "permissiondenied" in output_compact + + def test_add_dev_fresh_install_into_preexisting_empty_dir_cleans_new_file( + self, project_dir, monkeypatch + ): + """When the destination directory already exists but has no + workflow.yml (e.g. an empty dir left over from elsewhere), a later + registry.add() failure must remove the newly copied file -- the + rollback previously did nothing in this case (existed_before=True + with no backup bytes), leaving the new file behind -- while leaving + the pre-existing directory itself intact.""" + from typer.testing import CliRunner + from specify_cli import app + from specify_cli.workflows.catalog import WorkflowRegistry + + monkeypatch.chdir(project_dir) + runner = CliRunner() + src = self._write_workflow_dir(project_dir) + dest_dir = project_dir / ".specify" / "workflows" / "align-wf" + dest_dir.mkdir(parents=True) # pre-existing, but empty: no workflow.yml + + def boom(self, *args, **kwargs): + raise OSError("disk full") + + with pytest.MonkeyPatch.context() as mp: + mp.setattr(WorkflowRegistry, "add", boom) + result = runner.invoke(app, ["workflow", "add", str(src), "--dev"]) + + assert result.exit_code != 0 + assert result.output.strip() != "" + assert dest_dir.is_dir() + assert not (dest_dir / "workflow.yml").exists() + + def test_add_catalog_save_failure_leaves_no_orphan_directory(self, project_dir, monkeypatch): + """Same guarantee as the local-install paths, but for a fresh catalog + install: a registry.add() failure must clean up the freshly-downloaded + directory and fail with a clean escaped message.""" + from typer.testing import CliRunner + from specify_cli import app + from specify_cli.workflows.catalog import WorkflowCatalog, WorkflowRegistry + + monkeypatch.chdir(project_dir) + monkeypatch.setattr( + WorkflowCatalog, + "get_workflow_info", + lambda self, wid: { + "id": wid, + "name": "Align Workflow", + "version": "1.0.0", + "url": "https://example.com/workflow.yml", + "_install_allowed": True, + "_catalog_name": "test-catalog", + }, + ) + data = self.WORKFLOW_YAML.format(version="1.0.0").encode() + + def boom(self): + raise OSError("disk full") + + runner = CliRunner() + with pytest.MonkeyPatch.context() as mp: + mp.setattr( + "specify_cli.authentication.http.open_url", + lambda url, timeout=None, extra_headers=None, redirect_validator=None: self._FakeResponse(data, url), + ) + mp.setattr(WorkflowRegistry, "save", boom) + result = runner.invoke(app, ["workflow", "add", "align-wf"]) + + assert result.exit_code != 0 + assert result.exception is None or isinstance(result.exception, SystemExit) + assert result.output.strip() != "" + dest_dir = project_dir / ".specify" / "workflows" / "align-wf" + assert not dest_dir.exists() + assert not WorkflowRegistry(project_dir).is_installed("align-wf") + + def test_add_catalog_fresh_install_mkstemp_failure_leaves_no_orphan_directory( + self, project_dir, monkeypatch + ): + """Same guarantee as the local-install fresh-install case, but for a + fresh catalog install: if _stage_workflow_file's mkdir succeeds but + its mkstemp then fails, the freshly-created empty directory must not + be left orphaned.""" + from typer.testing import CliRunner + from specify_cli import app + from specify_cli.workflows.catalog import WorkflowCatalog, WorkflowRegistry + + monkeypatch.chdir(project_dir) + monkeypatch.setattr( + WorkflowCatalog, + "get_workflow_info", + lambda self, wid: { + "id": wid, + "name": "Align Workflow", + "version": "1.0.0", + "url": "https://example.com/workflow.yml", + "_install_allowed": True, + "_catalog_name": "test-catalog", + }, + ) + data = self.WORKFLOW_YAML.format(version="1.0.0").encode() + + def boom(*args, **kwargs): + raise OSError("disk full") + + runner = CliRunner() + with pytest.MonkeyPatch.context() as mp: + mp.setattr( + "specify_cli.authentication.http.open_url", + lambda url, timeout=None, extra_headers=None, redirect_validator=None: self._FakeResponse(data, url), + ) + mp.setattr("tempfile.mkstemp", boom) + result = runner.invoke(app, ["workflow", "add", "align-wf"]) + + assert result.exit_code != 0 + assert result.exception is None or isinstance(result.exception, SystemExit) + assert result.output.strip() != "" + dest_dir = project_dir / ".specify" / "workflows" / "align-wf" + assert not dest_dir.exists(), "fresh-install dest_dir left orphaned after mkstemp failure" + assert not WorkflowRegistry(project_dir).is_installed("align-wf") + + def test_add_catalog_rejects_oversized_content_length(self, project_dir, monkeypatch): + """Catalog installs must share the same size cap as --from: a + response that declares an oversized Content-Length is rejected + before its body is read into memory, and no orphan directory or + registry mutation is left behind.""" + from typer.testing import CliRunner + from specify_cli import app + from specify_cli.workflows import _commands as wf_commands + from specify_cli.workflows.catalog import WorkflowCatalog, WorkflowRegistry + + monkeypatch.chdir(project_dir) + monkeypatch.setattr(wf_commands, "_MAX_WORKFLOW_YAML_BYTES", 100) + monkeypatch.setattr( + WorkflowCatalog, + "get_workflow_info", + lambda self, wid: { + "id": wid, + "name": "Align Workflow", + "version": "1.0.0", + "url": "https://example.com/workflow.yml", + "_install_allowed": True, + "_catalog_name": "test-catalog", + }, + ) + small_body = b"id: align-wf\n" # actual body is small; header lies + runner = CliRunner() + with pytest.MonkeyPatch.context() as mp: + mp.setattr( + "specify_cli.authentication.http.open_url", + lambda url, timeout=None, extra_headers=None, redirect_validator=None: self._FakeResponse( + small_body, url, headers={"Content-Length": "1000"} + ), + ) + result = runner.invoke(app, ["workflow", "add", "align-wf"]) + + assert result.exit_code != 0 + assert result.exception is None or isinstance(result.exception, SystemExit) + assert "exceedingthe100-byteworkflowsizelimit" in "".join(result.output.split()) + dest_dir = project_dir / ".specify" / "workflows" / "align-wf" + assert not dest_dir.exists() + assert not WorkflowRegistry(project_dir).is_installed("align-wf") + + def test_add_catalog_rejects_oversized_streamed_body_without_content_length( + self, project_dir, monkeypatch + ): + """Catalog installs must also cap actual streamed bytes when + Content-Length is absent or understated, leaving no orphan + directory or registry mutation behind.""" + from typer.testing import CliRunner + from specify_cli import app + from specify_cli.workflows import _commands as wf_commands + from specify_cli.workflows.catalog import WorkflowCatalog, WorkflowRegistry + + monkeypatch.chdir(project_dir) + monkeypatch.setattr(wf_commands, "_MAX_WORKFLOW_YAML_BYTES", 100) + monkeypatch.setattr( + WorkflowCatalog, + "get_workflow_info", + lambda self, wid: { + "id": wid, + "name": "Align Workflow", + "version": "1.0.0", + "url": "https://example.com/workflow.yml", + "_install_allowed": True, + "_catalog_name": "test-catalog", + }, + ) + oversized_body = b"x" * 500 # no Content-Length header at all + runner = CliRunner() + with pytest.MonkeyPatch.context() as mp: + mp.setattr( + "specify_cli.authentication.http.open_url", + lambda url, timeout=None, extra_headers=None, redirect_validator=None: self._FakeResponse( + oversized_body, url + ), + ) + result = runner.invoke(app, ["workflow", "add", "align-wf"]) + + assert result.exit_code != 0 + assert result.exception is None or isinstance(result.exception, SystemExit) + assert "exceedsthe100-byteworkflowsizelimit" in "".join(result.output.split()) + dest_dir = project_dir / ".specify" / "workflows" / "align-wf" + assert not dest_dir.exists() + assert not WorkflowRegistry(project_dir).is_installed("align-wf") + + def test_add_catalog_reinstall_save_failure_restores_prior_file(self, project_dir, monkeypatch): + """Re-adding an already-installed catalog workflow downloads the new + version over the existing install directory. If registry.add() then + fails to save, the prior working workflow.yml must be restored + byte-for-byte (not left overwritten with the new download, and not + deleted like a fresh install) and the registry must remain valid and + still point at the original version -- the update path's caller has + an outer backup/restore for this, but plain `workflow add` does not, + so _install_workflow_from_catalog must handle it itself.""" + from typer.testing import CliRunner + from specify_cli import app + from specify_cli.workflows.catalog import WorkflowCatalog, WorkflowRegistry + + monkeypatch.chdir(project_dir) + monkeypatch.setattr( + WorkflowCatalog, + "get_workflow_info", + lambda self, wid: { + "id": wid, + "name": "Align Workflow", + "version": "1.0.0", + "url": "https://example.com/workflow.yml", + "_install_allowed": True, + "_catalog_name": "test-catalog", + }, + ) + source_data = self.WORKFLOW_YAML.format(version="1.0.0").encode() + runner = CliRunner() + with pytest.MonkeyPatch.context() as mp: + mp.setattr( + "specify_cli.authentication.http.open_url", + lambda url, timeout=None, extra_headers=None, redirect_validator=None: self._FakeResponse( + source_data, url + ), + ) + result = runner.invoke(app, ["workflow", "add", "align-wf"]) + assert result.exit_code == 0, result.output + + dest_file = project_dir / ".specify" / "workflows" / "align-wf" / "workflow.yml" + original_data = dest_file.read_bytes() + + new_data = self.WORKFLOW_YAML.format(version="2.0.0").encode() + + def boom(self): + raise OSError("disk full") + + with pytest.MonkeyPatch.context() as mp: + mp.setattr( + "specify_cli.authentication.http.open_url", + lambda url, timeout=None, extra_headers=None, redirect_validator=None: self._FakeResponse( + new_data, url + ), + ) + mp.setattr(WorkflowRegistry, "save", boom) + result = runner.invoke(app, ["workflow", "add", "align-wf"]) + + assert result.exit_code != 0 + assert result.exception is None or isinstance(result.exception, SystemExit) + assert result.output.strip() != "" + # The prior working install must survive untouched, byte-for-byte. + assert dest_file.read_bytes() == original_data + registry = WorkflowRegistry(project_dir) + assert registry.is_installed("align-wf") + assert registry.get("align-wf")["version"] == "1.0.0" + + def test_add_catalog_successful_reinstall_leaves_no_backup_file( + self, project_dir, monkeypatch + ): + """Same orphan-backup gap as the local-install path: a successful + catalog reinstall must not leave its unique backup behind once + registry.add() durably succeeds.""" + from typer.testing import CliRunner + from specify_cli import app + from specify_cli.workflows.catalog import WorkflowCatalog, WorkflowRegistry + + monkeypatch.chdir(project_dir) + monkeypatch.setattr( + WorkflowCatalog, + "get_workflow_info", + lambda self, wid: { + "id": wid, + "name": "Align Workflow", + "version": "1.0.0", + "url": "https://example.com/workflow.yml", + "_install_allowed": True, + "_catalog_name": "test-catalog", + }, + ) + original_data = self.WORKFLOW_YAML.format(version="1.0.0").encode() + runner = CliRunner() + with pytest.MonkeyPatch.context() as mp: + mp.setattr( + "specify_cli.authentication.http.open_url", + lambda url, timeout=None, extra_headers=None, redirect_validator=None: self._FakeResponse( + original_data, url + ), + ) + result = runner.invoke(app, ["workflow", "add", "align-wf"]) + assert result.exit_code == 0, result.output + + new_data = self.WORKFLOW_YAML.format(version="2.0.0").encode() + with pytest.MonkeyPatch.context() as mp: + mp.setattr( + "specify_cli.authentication.http.open_url", + lambda url, timeout=None, extra_headers=None, redirect_validator=None: self._FakeResponse( + new_data, url + ), + ) + result = runner.invoke(app, ["workflow", "add", "align-wf"]) + + assert result.exit_code == 0, result.output + workflow_dir = project_dir / ".specify" / "workflows" / "align-wf" + registry = WorkflowRegistry(project_dir) + assert registry.is_installed("align-wf") + assert registry.get("align-wf")["version"] == "2.0.0" + assert (workflow_dir / "workflow.yml").read_bytes() == new_data + leftovers = [p.name for p in workflow_dir.iterdir() if p.name != "workflow.yml"] + assert leftovers == [], f"orphan sibling(s) left behind: {leftovers}" + + @pytest.mark.skipif(os.name == "nt", reason="POSIX permission bits") + def test_add_catalog_fresh_install_uses_project_file_mode( + self, project_dir, monkeypatch + ): + import stat + from typer.testing import CliRunner + from specify_cli import app + from specify_cli.workflows.catalog import WorkflowCatalog + + monkeypatch.chdir(project_dir) + monkeypatch.setattr( + WorkflowCatalog, + "get_workflow_info", + lambda self, wid: { + "id": wid, + "name": "Align Workflow", + "version": "1.0.0", + "url": "https://example.com/workflow.yml", + "_install_allowed": True, + "_catalog_name": "test-catalog", + }, + ) + data = self.WORKFLOW_YAML.format(version="1.0.0").encode() + + previous_umask = os.umask(0o022) + try: + with pytest.MonkeyPatch.context() as mp: + mp.setattr( + "specify_cli.authentication.http.open_url", + lambda url, timeout=None, extra_headers=None, + redirect_validator=None: self._FakeResponse(data, url), + ) + result = CliRunner().invoke( + app, ["workflow", "add", "align-wf"] + ) + finally: + os.umask(previous_umask) + + assert result.exit_code == 0, result.output + workflow_file = ( + project_dir + / ".specify" + / "workflows" + / "align-wf" + / "workflow.yml" + ) + assert stat.S_IMODE(workflow_file.stat().st_mode) == 0o644 + + def test_concurrent_catalog_reinstalls_keep_file_and_registry_aligned( + self, project_dir, monkeypatch + ): + import threading + from specify_cli.workflows import _commands + from specify_cli.workflows.catalog import WorkflowCatalog, WorkflowRegistry + from specify_cli.workflows.engine import WorkflowDefinition + + workflows_dir = project_dir / ".specify" / "workflows" + workflow_file = workflows_dir / "align-wf" / "workflow.yml" + workflow_file.parent.mkdir(parents=True) + workflow_file.write_text( + self.WORKFLOW_YAML.format(version="1.0.0"), encoding="utf-8" + ) + WorkflowRegistry(project_dir).add( + "align-wf", + { + "name": "Align Workflow", + "version": "1.0.0", + "source": "catalog", + }, + ) + + versions = {"install-a": "2.0.0", "install-b": "3.0.0"} + monkeypatch.setattr( + WorkflowCatalog, + "get_workflow_info", + lambda self, wid: { + "id": wid, + "name": "Align Workflow", + "version": versions[threading.current_thread().name], + "url": ( + "https://example.com/" + f"{versions[threading.current_thread().name]}.yml" + ), + "_install_allowed": True, + "_catalog_name": "test-catalog", + }, + ) + monkeypatch.setattr( + "specify_cli.authentication.http.open_url", + lambda url, timeout=None, extra_headers=None, + redirect_validator=None: self._FakeResponse( + self.WORKFLOW_YAML.format( + version=url.rsplit("/", 1)[-1].removesuffix(".yml") + ).encode(), + url, + ), + ) + + a_committed = threading.Event() + b_committed = threading.Event() + a_saving = threading.Event() + b_saved = threading.Event() + real_commit = _commands._commit_workflow_file + real_save = WorkflowRegistry.save + + def coordinated_commit(*args, **kwargs): + backup = real_commit(*args, **kwargs) + if threading.current_thread().name == "install-a": + a_committed.set() + b_committed.wait(0.5) + else: + b_committed.set() + return backup + + def coordinated_save(registry): + if threading.current_thread().name == "install-a": + a_saving.set() + b_saved.wait(0.5) + return real_save(registry) + assert a_saving.wait(2) + real_save(registry) + b_saved.set() + + monkeypatch.setattr( + _commands, "_commit_workflow_file", coordinated_commit + ) + monkeypatch.setattr(WorkflowRegistry, "save", coordinated_save) + + errors = [] + + def install(): + try: + _commands._install_workflow_from_catalog( + project_dir, + workflows_dir, + "align-wf", + ) + except BaseException as exc: + errors.append(exc) + + first = threading.Thread(target=install, name="install-a") + second = threading.Thread(target=install, name="install-b") + first.start() + assert a_committed.wait(2) + second.start() + first.join(5) + second.join(5) + + assert not first.is_alive() + assert not second.is_alive() + assert errors == [] + file_version = WorkflowDefinition.from_yaml(workflow_file).version + registry_version = WorkflowRegistry(project_dir).get("align-wf")[ + "version" + ] + assert file_version == registry_version + + def test_precommit_discard_preserves_concurrent_install(self, project_dir): + from specify_cli.workflows import _commands + + workflow_dir = ( + project_dir / ".specify" / "workflows" / "concurrent-wf" + ) + workflow_dir.mkdir(parents=True) + staged_file = workflow_dir / ".workflow.yml.staged.tmp" + staged_file.write_text("staged", encoding="utf-8") + committed_file = workflow_dir / "workflow.yml" + committed_file.write_text("committed", encoding="utf-8") + + _commands._discard_staged_workflow_file( + staged_file, workflow_dir, existed_before=False + ) + + assert committed_file.read_text(encoding="utf-8") == "committed" + assert not staged_file.exists() + + def test_fresh_install_rollback_preserves_concurrent_staged_file( + self, project_dir + ): + """A second installer stages before taking the transaction lock, so + the first installer's rollback must not recursively remove siblings.""" + from specify_cli.workflows import _commands + + workflow_dir = ( + project_dir / ".specify" / "workflows" / "concurrent-wf" + ) + workflow_dir.mkdir(parents=True) + committed_file = workflow_dir / "workflow.yml" + committed_file.write_text("failed install", encoding="utf-8") + concurrent_stage = workflow_dir / ".workflow.yml.concurrent.tmp" + concurrent_stage.write_text("next install", encoding="utf-8") + + _commands._rollback_committed_workflow_file( + committed_file, + workflow_dir, + existed_before=False, + backup_file=None, + ) + + assert not committed_file.exists() + assert concurrent_stage.read_text(encoding="utf-8") == "next install" + + def test_add_dev_registry_reopen_exit_discards_staged_file( + self, project_dir, monkeypatch + ): + import typer + from typer.testing import CliRunner + from specify_cli import app + from specify_cli.workflows import _commands + + monkeypatch.chdir(project_dir) + source_dir = self._write_workflow_dir(project_dir) + real_open_registry = _commands._open_workflow_registry + calls = 0 + + def fail_transaction_reopen(root): + nonlocal calls + calls += 1 + if calls == 2: + raise typer.Exit(1) + return real_open_registry(root) + + monkeypatch.setattr( + _commands, "_open_workflow_registry", fail_transaction_reopen + ) + result = CliRunner().invoke( + app, ["workflow", "add", str(source_dir), "--dev"] + ) + + assert result.exit_code != 0 + assert not ( + project_dir / ".specify" / "workflows" / "align-wf" + ).exists() + + def test_add_catalog_registry_reopen_exit_discards_staged_file( + self, project_dir, monkeypatch + ): + import typer + from specify_cli.workflows import _commands + from specify_cli.workflows.catalog import WorkflowCatalog + + workflows_dir = project_dir / ".specify" / "workflows" + monkeypatch.setattr( + WorkflowCatalog, + "get_workflow_info", + lambda self, wid: { + "id": wid, + "name": "Align Workflow", + "version": "1.0.0", + "url": "https://example.com/workflow.yml", + "_install_allowed": True, + "_catalog_name": "test-catalog", + }, + ) + data = self.WORKFLOW_YAML.format(version="1.0.0").encode() + monkeypatch.setattr( + "specify_cli.authentication.http.open_url", + lambda url, timeout=None, extra_headers=None, + redirect_validator=None: self._FakeResponse(data, url), + ) + monkeypatch.setattr( + _commands, + "_open_workflow_registry", + lambda _root: (_ for _ in ()).throw(typer.Exit(1)), + ) + + with pytest.raises(typer.Exit): + _commands._install_workflow_from_catalog( + project_dir, + workflows_dir, + "align-wf", + ) + + assert not (workflows_dir / "align-wf").exists() + + def test_remove_serializes_with_concurrent_catalog_install( + self, project_dir, monkeypatch + ): + import threading + from specify_cli.workflows import _commands + from specify_cli.workflows.catalog import WorkflowCatalog, WorkflowRegistry + from specify_cli.workflows.engine import WorkflowDefinition + + workflows_dir = project_dir / ".specify" / "workflows" + workflow_file = workflows_dir / "align-wf" / "workflow.yml" + workflow_file.parent.mkdir(parents=True) + workflow_file.write_text( + self.WORKFLOW_YAML.format(version="1.0.0"), encoding="utf-8" + ) + WorkflowRegistry(project_dir).add( + "align-wf", + { + "name": "Align Workflow", + "version": "1.0.0", + "source": "catalog", + }, + ) + monkeypatch.setattr( + _commands, "_require_specify_project", lambda: project_dir + ) + monkeypatch.setattr( + WorkflowCatalog, + "get_workflow_info", + lambda self, wid: { + "id": wid, + "name": "Align Workflow", + "version": "2.0.0", + "url": "https://example.com/workflow.yml", + "_install_allowed": True, + "_catalog_name": "test-catalog", + }, + ) + new_data = self.WORKFLOW_YAML.format(version="2.0.0").encode() + monkeypatch.setattr( + "specify_cli.authentication.http.open_url", + lambda url, timeout=None, extra_headers=None, + redirect_validator=None: self._FakeResponse(new_data, url), + ) + + removal_ready = threading.Event() + install_done = threading.Event() + real_remove = WorkflowRegistry.remove + + def coordinated_remove(registry, workflow_id): + if threading.current_thread().name == "remove": + removal_ready.set() + install_done.wait(0.5) + return real_remove(registry, workflow_id) + + monkeypatch.setattr(WorkflowRegistry, "remove", coordinated_remove) + errors = [] + + def remove(): + try: + _commands.workflow_remove("align-wf") + except BaseException as exc: + errors.append(exc) + + def install(): + try: + _commands._install_workflow_from_catalog( + project_dir, + workflows_dir, + "align-wf", + ) + except BaseException as exc: + errors.append(exc) + finally: + install_done.set() + + remove_thread = threading.Thread(target=remove, name="remove") + install_thread = threading.Thread(target=install, name="install") + remove_thread.start() + assert removal_ready.wait(2) + install_thread.start() + remove_thread.join(5) + install_thread.join(5) + + assert not remove_thread.is_alive() + assert not install_thread.is_alive() + assert errors == [] + assert WorkflowDefinition.from_yaml(workflow_file).version == "2.0.0" + metadata = WorkflowRegistry(project_dir).get("align-wf") + assert metadata["version"] == "2.0.0" + + @pytest.mark.parametrize( + ("command_name", "initial_enabled", "expected_enabled"), + [ + ("enable", False, True), + ("disable", True, False), + ], + ) + def test_toggle_serializes_with_concurrent_catalog_update( + self, + project_dir, + monkeypatch, + command_name, + initial_enabled, + expected_enabled, + ): + import threading + from specify_cli.workflows import _commands + from specify_cli.workflows.catalog import WorkflowCatalog, WorkflowRegistry + from specify_cli.workflows.engine import WorkflowDefinition + + workflows_dir = project_dir / ".specify" / "workflows" + workflow_file = workflows_dir / "align-wf" / "workflow.yml" + workflow_file.parent.mkdir(parents=True) + workflow_file.write_text( + self.WORKFLOW_YAML.format(version="1.0.0"), encoding="utf-8" + ) + WorkflowRegistry(project_dir).add( + "align-wf", + { + "name": "Align Workflow", + "version": "1.0.0", + "source": "catalog", + "enabled": initial_enabled, + }, + ) + monkeypatch.setattr( + _commands, "_require_specify_project", lambda: project_dir + ) + monkeypatch.setattr( + WorkflowCatalog, + "get_workflow_info", + lambda self, wid: { + "id": wid, + "name": "Align Workflow", + "version": "2.0.0", + "url": "https://example.com/workflow.yml", + "_install_allowed": True, + "_catalog_name": "test-catalog", + }, + ) + new_data = self.WORKFLOW_YAML.format(version="2.0.0").encode() + monkeypatch.setattr( + "specify_cli.authentication.http.open_url", + lambda url, timeout=None, extra_headers=None, + redirect_validator=None: self._FakeResponse(new_data, url), + ) + + toggle_ready = threading.Event() + update_done = threading.Event() + real_add = WorkflowRegistry.add + + def coordinated_add(registry, workflow_id, metadata): + if threading.current_thread().name == "toggle": + toggle_ready.set() + update_done.wait(0.5) + return real_add(registry, workflow_id, metadata) + + monkeypatch.setattr(WorkflowRegistry, "add", coordinated_add) + errors = [] + + def toggle(): + try: + getattr(_commands, f"workflow_{command_name}")("align-wf") + except BaseException as exc: + errors.append(exc) + + def update(): + try: + _commands._install_workflow_from_catalog( + project_dir, + workflows_dir, + "align-wf", + ) + except BaseException as exc: + errors.append(exc) + finally: + update_done.set() + + toggle_thread = threading.Thread(target=toggle, name="toggle") + update_thread = threading.Thread(target=update, name="update") + toggle_thread.start() + assert toggle_ready.wait(2) + update_thread.start() + toggle_thread.join(5) + update_thread.join(5) + + assert not toggle_thread.is_alive() + assert not update_thread.is_alive() + assert errors == [] + assert WorkflowDefinition.from_yaml(workflow_file).version == "2.0.0" + metadata = WorkflowRegistry(project_dir).get("align-wf") + assert metadata["version"] == "2.0.0" + assert metadata.get("enabled", True) is expected_enabled + + def test_add_catalog_reinstall_restore_failure_reports_warning_and_original_error( + self, project_dir, monkeypatch + ): + """Same restore-rename boundary as the local-install path: the + prior file is restored via an atomic rename (not a content rewrite) + when registry.add() fails on a reinstall. If that restore rename + itself also fails, it must report a clear warning in addition to + the original clean registry error, never crash or silently claim + success.""" + from typer.testing import CliRunner + from specify_cli import app + from specify_cli.workflows.catalog import WorkflowCatalog, WorkflowRegistry + + monkeypatch.chdir(project_dir) + monkeypatch.setattr( + WorkflowCatalog, + "get_workflow_info", + lambda self, wid: { + "id": wid, + "name": "Align Workflow", + "version": "1.0.0", + "url": "https://example.com/workflow.yml", + "_install_allowed": True, + "_catalog_name": "test-catalog", + }, + ) + original_data = self.WORKFLOW_YAML.format(version="1.0.0").encode() + runner = CliRunner() + with pytest.MonkeyPatch.context() as mp: + mp.setattr( + "specify_cli.authentication.http.open_url", + lambda url, timeout=None, extra_headers=None, redirect_validator=None: self._FakeResponse( + original_data, url + ), + ) + result = runner.invoke(app, ["workflow", "add", "align-wf"]) + assert result.exit_code == 0, result.output + + new_data = self.WORKFLOW_YAML.format(version="2.0.0").encode() + + def save_boom(self): + raise OSError("disk full") + + real_replace = os.replace + calls = {"n": 0} + + def replace_boom(src_path, dst_path): + # The commit swap for a reinstall makes exactly two os.replace + # calls (backup-aside, then staged-into-dest); let both succeed + # and only fail the third call -- the post-registry-failure + # restore-back rename. + calls["n"] += 1 + if calls["n"] <= 2: + return real_replace(src_path, dst_path) + raise OSError("permission denied") + + with pytest.MonkeyPatch.context() as mp: + mp.setattr( + "specify_cli.authentication.http.open_url", + lambda url, timeout=None, extra_headers=None, redirect_validator=None: self._FakeResponse( + new_data, url + ), + ) + mp.setattr(WorkflowRegistry, "save", save_boom) + mp.setattr(os, "replace", replace_boom) + result = runner.invoke(app, ["workflow", "add", "align-wf"]) + + assert result.exit_code != 0 + assert result.exception is None or isinstance(result.exception, SystemExit) + output_compact = "".join(result.output.split()) + assert "Warning" in result.output + assert "diskfull" in output_compact + assert "permissiondenied" in output_compact + + def test_add_catalog_fresh_install_into_preexisting_empty_dir_cleans_new_file( + self, project_dir, monkeypatch + ): + """Same rollback orphan gap as the local-install path, but for a + fresh catalog install: a pre-existing empty destination directory + (no workflow.yml) sets existed_before=True with no backup bytes, so + the rollback previously did nothing on a later failure -- leaving + the freshly downloaded workflow.yml behind. It must be removed, + leaving the pre-existing directory itself intact.""" + from typer.testing import CliRunner + from specify_cli import app + from specify_cli.workflows.catalog import WorkflowCatalog, WorkflowRegistry + + monkeypatch.chdir(project_dir) + monkeypatch.setattr( + WorkflowCatalog, + "get_workflow_info", + lambda self, wid: { + "id": wid, + "name": "Align Workflow", + "version": "1.0.0", + "url": "https://example.com/workflow.yml", + "_install_allowed": True, + "_catalog_name": "test-catalog", + }, + ) + dest_dir = project_dir / ".specify" / "workflows" / "align-wf" + dest_dir.mkdir(parents=True) # pre-existing, but empty: no workflow.yml + data = self.WORKFLOW_YAML.format(version="1.0.0").encode() + + def boom(self): + raise OSError("disk full") + + runner = CliRunner() + with pytest.MonkeyPatch.context() as mp: + mp.setattr( + "specify_cli.authentication.http.open_url", + lambda url, timeout=None, extra_headers=None, redirect_validator=None: self._FakeResponse(data, url), + ) + mp.setattr(WorkflowRegistry, "save", boom) + result = runner.invoke(app, ["workflow", "add", "align-wf"]) + + assert result.exit_code != 0 + assert result.output.strip() != "" + assert dest_dir.is_dir() + assert not (dest_dir / "workflow.yml").exists() + + @pytest.mark.parametrize( + "mode", ["redirect_rejected", "download_exception", "invalid_yaml", "id_mismatch"] + ) + def test_add_catalog_reinstall_early_failure_restores_prior_file( + self, project_dir, monkeypatch, mode + ): + """Every _install_workflow_from_catalog failure branch that runs after + the mkdir/download step -- not just the registry.add() OSError case + -- must route through the same existed-before/backup-aware cleanup: + on a reinstall, a redirect rejection, a download exception, invalid + YAML, or a workflow-id mismatch must restore the prior working + workflow.yml rather than deleting the whole directory. One shared + root cause (the cleanup helper), so parametrized over trigger point.""" + from typer.testing import CliRunner + from specify_cli import app + from specify_cli.workflows.catalog import WorkflowCatalog, WorkflowRegistry + + monkeypatch.chdir(project_dir) + monkeypatch.setattr( + WorkflowCatalog, + "get_workflow_info", + lambda self, wid: { + "id": wid, + "name": "Align Workflow", + "version": "1.0.0", + "url": "https://example.com/workflow.yml", + "_install_allowed": True, + "_catalog_name": "test-catalog", + }, + ) + source_data = self.WORKFLOW_YAML.format(version="1.0.0").encode() + runner = CliRunner() + with pytest.MonkeyPatch.context() as mp: + mp.setattr( + "specify_cli.authentication.http.open_url", + lambda url, timeout=None, extra_headers=None, redirect_validator=None: self._FakeResponse( + source_data, url + ), + ) + result = runner.invoke(app, ["workflow", "add", "align-wf"]) + assert result.exit_code == 0, result.output + + dest_file = project_dir / ".specify" / "workflows" / "align-wf" / "workflow.yml" + original_data = dest_file.read_bytes() + + if mode == "redirect_rejected": + def fake_open_url(url, timeout=None, extra_headers=None, redirect_validator=None): + return self._FakeResponse(b"irrelevant", "http://evil.example.com/workflow.yml") + elif mode == "download_exception": + def fake_open_url(url, timeout=None, extra_headers=None, redirect_validator=None): + raise OSError("network down") + elif mode == "invalid_yaml": + def fake_open_url(url, timeout=None, extra_headers=None, redirect_validator=None): + return self._FakeResponse(b": : not valid yaml: [", url) + else: # id_mismatch + mismatched_yaml = self.WORKFLOW_YAML.format(version="2.0.0").replace( + 'id: "align-wf"', 'id: "different-workflow"' + ) + + def fake_open_url(url, timeout=None, extra_headers=None, redirect_validator=None): + return self._FakeResponse(mismatched_yaml.encode(), url) + + with pytest.MonkeyPatch.context() as mp: + mp.setattr("specify_cli.authentication.http.open_url", fake_open_url) + result = runner.invoke(app, ["workflow", "add", "align-wf"]) + + assert result.exit_code != 0 + assert result.exception is None or isinstance(result.exception, SystemExit) + assert result.output.strip() != "" + assert dest_file.read_bytes() == original_data + registry = WorkflowRegistry(project_dir) + assert registry.is_installed("align-wf") + assert registry.get("align-wf")["version"] == "1.0.0" + + def test_download_redirect_validator_rejects_http_before_follow(self): + import urllib.error + + from specify_cli.workflows._commands import _reject_insecure_download_redirect + + with pytest.raises(urllib.error.URLError): + _reject_insecure_download_redirect( + "https://example.com/wf.yml", "http://evil.example.com/wf.yml" + ) + with pytest.raises(urllib.error.URLError): + _reject_insecure_download_redirect( + "https://example.com/wf.yml", "http://localhost:8000/wf.yml" + ) + # Allowed: HTTPS anywhere, or loopback HTTP that stays on loopback HTTP. + _reject_insecure_download_redirect( + "https://example.com/wf.yml", "https://cdn.example.com/wf.yml" + ) + _reject_insecure_download_redirect( + "http://localhost:7000/wf.yml", "http://localhost:8000/wf.yml" + ) + _reject_insecure_download_redirect( + "http://127.0.0.1/source.yml", "http://127.0.0.1/wf.yml" + ) + + def test_add_from_url_passes_redirect_validator(self, project_dir, monkeypatch): + from unittest.mock import patch + + from typer.testing import CliRunner + from specify_cli import app + + monkeypatch.chdir(project_dir) + data = self.WORKFLOW_YAML.format(version="1.0.0").encode() + seen: dict[str, object] = {} + + def fake_open(url, timeout=None, extra_headers=None, redirect_validator=None): + seen["validator"] = redirect_validator + return self._FakeResponse(data, url) + + runner = CliRunner() + with patch("specify_cli.authentication.http.open_url", side_effect=fake_open): + result = runner.invoke( + app, + ["workflow", "add", "align-wf", "--from", "https://example.com/workflow.yml"], + input="y\n", + ) + assert result.exit_code == 0, result.output + from specify_cli.workflows._commands import _reject_insecure_download_redirect + + assert seen["validator"] is _reject_insecure_download_redirect + + @pytest.mark.skipif(sys.platform == "win32", reason="chmod mode bits not reliable on Windows") + def test_registry_save_failure_preserves_file_on_disk(self, project_dir, monkeypatch): + """A failed dump must not truncate the persisted registry, and must + not alter its on-disk mode either -- the chmod-to-match-existing-mode + step operates on the temp file, never the target, so a failed save + (which never reaches os.replace) cannot touch the original's mode.""" + from specify_cli.workflows.catalog import WorkflowRegistry + + registry = WorkflowRegistry(project_dir) + registry.add("align-wf", {"version": "1.0.0", "source": "catalog"}) + registry.registry_path.chmod(0o644) + + import specify_cli.workflows.catalog as catalog_mod + + def boom(*args, **kwargs): + raise OSError("disk full") + + monkeypatch.setattr(catalog_mod.json, "dump", boom) + with pytest.raises(OSError): + registry.add("align-wf", {"version": "2.0.0", "source": "catalog"}) + monkeypatch.undo() + + fresh = WorkflowRegistry(project_dir) + assert fresh.get("align-wf")["version"] == "1.0.0" + assert stat.S_IMODE(registry.registry_path.stat().st_mode) == 0o644 + assert not list(registry.workflows_dir.glob("*.tmp")) + + def test_update_mixed_targets_does_not_claim_all_up_to_date(self, project_dir, monkeypatch): + """Skipped targets must not be presented as verified up to date.""" + from typer.testing import CliRunner + from specify_cli import app + from specify_cli.workflows.catalog import WorkflowCatalog, WorkflowRegistry + + monkeypatch.chdir(project_dir) + runner = CliRunner() + self._install_dev(runner, app, project_dir) # local source → skipped + WorkflowRegistry(project_dir).add("catalog-wf", { + "name": "Catalog Workflow", + "version": "1.0.0", + "description": "", + "source": "catalog", + "url": "https://example.com/workflow.yml", + }) + monkeypatch.setattr( + WorkflowCatalog, + "get_workflow_info", + lambda self, wid: { + "id": wid, + "version": "1.0.0", + "url": "https://example.com/workflow.yml", + "_install_allowed": True, + }, + ) + result = runner.invoke(app, ["workflow", "update"]) + assert result.exit_code == 0, result.output + assert "All workflows are up to date!" not in result.output + assert "All checked workflows are up to date" in result.output + assert "skipped" in result.output + + def test_run_refuses_falsy_non_bool_enabled(self, project_dir, monkeypatch): + """A falsy non-bool "enabled" (0) shows as disabled in list — run must agree.""" + import json as json_mod + + from typer.testing import CliRunner + from specify_cli import app + from specify_cli.workflows.catalog import WorkflowRegistry + + monkeypatch.chdir(project_dir) + runner = CliRunner() + self._install_dev(runner, app, project_dir) + + registry = WorkflowRegistry(project_dir) + registry.data["workflows"]["align-wf"]["enabled"] = 0 + registry.registry_path.write_text(json_mod.dumps(registry.data), encoding="utf-8") + + result = runner.invoke(app, ["workflow", "run", "align-wf"]) + assert result.exit_code != 0 + assert "disabled" in result.output + + def test_update_installs_newer_catalog_version(self, project_dir, monkeypatch): + from unittest.mock import patch + from typer.testing import CliRunner + from specify_cli import app + from specify_cli.workflows.catalog import WorkflowCatalog, WorkflowRegistry + + monkeypatch.chdir(project_dir) + registry = WorkflowRegistry(project_dir) + registry.add("align-wf", { + "name": "Align Workflow", + "version": "1.0.0", + "description": "CLI alignment test workflow", + "source": "catalog", + "catalog_name": "test-catalog", + "url": "https://example.com/workflow.yml", + }) + wf_dir = project_dir / ".specify" / "workflows" / "align-wf" + wf_dir.mkdir(parents=True) + (wf_dir / "workflow.yml").write_text( + self.WORKFLOW_YAML.format(version="1.0.0"), encoding="utf-8" + ) + + monkeypatch.setattr( + WorkflowCatalog, + "get_workflow_info", + lambda self, wid: { + "id": wid, + "name": "Align Workflow", + "version": "2.0.0", + "url": "https://example.com/workflow.yml", + "_install_allowed": True, + "_catalog_name": "test-catalog", + }, + ) + data = self.WORKFLOW_YAML.format(version="2.0.0").encode() + runner = CliRunner() + with patch( + "specify_cli.authentication.http.open_url", + side_effect=lambda url, timeout=None, extra_headers=None, redirect_validator=None: self._FakeResponse(data, url), + ): + result = runner.invoke(app, ["workflow", "update"], input="y\n") + assert result.exit_code == 0, result.output + assert "1.0.0" in result.output and "2.0.0" in result.output + meta = WorkflowRegistry(project_dir).get("align-wf") + assert meta["version"] == "2.0.0" + assert "2.0.0" in (wf_dir / "workflow.yml").read_text(encoding="utf-8") + + def test_update_downloaded_invalid_yaml_escapes_rich_markup(self, project_dir, monkeypatch): + """A malformed downloaded workflow can quote the offending line verbatim; escape it before printing.""" + from unittest.mock import patch + from typer.testing import CliRunner + from specify_cli import app + from specify_cli.workflows.catalog import WorkflowCatalog, WorkflowRegistry + from specify_cli.workflows.engine import WorkflowDefinition + + monkeypatch.chdir(project_dir) + registry = WorkflowRegistry(project_dir) + registry.add("align-wf", { + "name": "Align Workflow", + "version": "1.0.0", + "description": "CLI alignment test workflow", + "source": "catalog", + "catalog_name": "test-catalog", + "url": "https://example.com/workflow.yml", + }) + wf_dir = project_dir / ".specify" / "workflows" / "align-wf" + wf_dir.mkdir(parents=True) + (wf_dir / "workflow.yml").write_text( + self.WORKFLOW_YAML.format(version="1.0.0"), encoding="utf-8" + ) + + monkeypatch.setattr( + WorkflowCatalog, + "get_workflow_info", + lambda self, wid: { + "id": wid, + "name": "Align Workflow", + "version": "2.0.0", + "url": "https://example.com/workflow.yml", + "_install_allowed": True, + "_catalog_name": "test-catalog", + }, + ) + runner = CliRunner() + with patch( + "specify_cli.authentication.http.open_url", + side_effect=lambda url, timeout=None, extra_headers=None, redirect_validator=None: self._FakeResponse(b"", url), + ), patch.object( + WorkflowDefinition, + "from_string", + side_effect=ValueError('bad snippet: "New [Feature]"'), + ): + result = runner.invoke(app, ["workflow", "update"], input="y\n") + assert 'bad snippet: "New [Feature]"' in result.output + assert "Failed to update" in result.output + # The previously installed workflow must survive a failed update. + assert "1.0.0" in (wf_dir / "workflow.yml").read_text(encoding="utf-8") + + def test_update_malformed_catalog_url_fails_cleanly(self, project_dir, monkeypatch): + """An unparseable catalog URL (unbalanced IPv6 literal) must not abort the whole update.""" + from typer.testing import CliRunner + from specify_cli import app + from specify_cli.workflows.catalog import WorkflowCatalog, WorkflowRegistry + + monkeypatch.chdir(project_dir) + registry = WorkflowRegistry(project_dir) + registry.add("align-wf", { + "name": "Align Workflow", + "version": "1.0.0", + "description": "CLI alignment test workflow", + "source": "catalog", + "catalog_name": "test-catalog", + "url": "https://[::1/workflow.yml", + }) + wf_dir = project_dir / ".specify" / "workflows" / "align-wf" + wf_dir.mkdir(parents=True) + (wf_dir / "workflow.yml").write_text( + self.WORKFLOW_YAML.format(version="1.0.0"), encoding="utf-8" + ) + + monkeypatch.setattr( + WorkflowCatalog, + "get_workflow_info", + lambda self, wid: { + "id": wid, + "name": "Align Workflow", + "version": "2.0.0", + "url": "https://[::1/workflow.yml", + "_install_allowed": True, + "_catalog_name": "test-catalog", + }, + ) + runner = CliRunner() + result = runner.invoke(app, ["workflow", "update"], input="y\n") + assert "malformed install URL" in result.output + assert "Failed to update" in result.output + # The previously installed workflow must survive. + assert "1.0.0" in (wf_dir / "workflow.yml").read_text(encoding="utf-8") + + def test_add_non_string_catalog_url_fails_cleanly(self, project_dir, monkeypatch): + """A truthy non-string catalog URL must hit the clean error path, not AttributeError.""" + from typer.testing import CliRunner + from specify_cli import app + from specify_cli.workflows.catalog import WorkflowCatalog + + monkeypatch.chdir(project_dir) + monkeypatch.setattr( + WorkflowCatalog, + "get_workflow_info", + lambda self, wid: { + "id": wid, + "name": "Align Workflow", + "version": "1.0.0", + "url": 123, + "_install_allowed": True, + "_catalog_name": "test-catalog", + }, + ) + runner = CliRunner() + result = runner.invoke(app, ["workflow", "add", "align-wf"]) + assert result.exit_code != 0 + assert result.exception is None or isinstance(result.exception, SystemExit) + assert "malformed install URL" in result.output + + def test_enable_failed_save_leaves_workflow_disabled(self, project_dir, monkeypatch): + from typer.testing import CliRunner + from specify_cli import app + from specify_cli.workflows.catalog import WorkflowRegistry + + monkeypatch.chdir(project_dir) + runner = CliRunner() + self._install_dev(runner, app, project_dir) + result = runner.invoke(app, ["workflow", "disable", "align-wf"]) + assert result.exit_code == 0, result.output + + def boom(self): + raise OSError("disk full") + + with pytest.MonkeyPatch.context() as mp: + mp.setattr(WorkflowRegistry, "save", boom) + result = runner.invoke(app, ["workflow", "enable", "align-wf"]) + assert result.exit_code != 0 + + assert WorkflowRegistry(project_dir).get("align-wf")["enabled"] is False + result = runner.invoke(app, ["workflow", "enable", "align-wf"]) + assert result.exit_code == 0, result.output + assert WorkflowRegistry(project_dir).get("align-wf")["enabled"] is True + + @pytest.mark.parametrize("command", ["enable", "disable"]) + def test_enable_disable_save_failure_gives_clean_output( + self, project_dir, monkeypatch, command + ): + """A save() failure in enable/disable must produce a clean escaped CLI + error, not surface the raw OSError as an unhandled exception. Shared + root behavior: both call registry.add() with a fresh mapping and must + catch its deliberate OSError the same way.""" + from typer.testing import CliRunner + from specify_cli import app + from specify_cli.workflows.catalog import WorkflowRegistry + + monkeypatch.chdir(project_dir) + runner = CliRunner() + self._install_dev(runner, app, project_dir) + # disable starts from the enabled default; enable needs a prior disable. + starting_enabled = command == "disable" + if command == "enable": + pre = runner.invoke(app, ["workflow", "disable", "align-wf"]) + assert pre.exit_code == 0, pre.output + + def boom(self): + raise OSError("disk full") + + with pytest.MonkeyPatch.context() as mp: + mp.setattr(WorkflowRegistry, "save", boom) + result = runner.invoke(app, ["workflow", command, "align-wf"]) + + assert result.exit_code != 0 + assert result.exception is None or isinstance(result.exception, SystemExit) + assert result.output.strip() != "" + assert ( + WorkflowRegistry(project_dir).get("align-wf").get("enabled", True) + is starting_enabled + ) + + def test_update_rejects_version_mismatch_from_stale_url(self, project_dir, monkeypatch): + """A URL serving a different version than the catalog advertised must fail the update.""" + from unittest.mock import patch + from typer.testing import CliRunner + from specify_cli import app + from specify_cli.workflows.catalog import WorkflowCatalog, WorkflowRegistry + + monkeypatch.chdir(project_dir) + WorkflowRegistry(project_dir).add("align-wf", { + "name": "Align Workflow", + "version": "1.0.0", + "description": "CLI alignment test workflow", + "source": "catalog", + "catalog_name": "test-catalog", + "url": "https://example.com/workflow.yml", + }) + wf_dir = project_dir / ".specify" / "workflows" / "align-wf" + wf_dir.mkdir(parents=True) + (wf_dir / "workflow.yml").write_text( + self.WORKFLOW_YAML.format(version="1.0.0"), encoding="utf-8" + ) + + monkeypatch.setattr( + WorkflowCatalog, + "get_workflow_info", + lambda self, wid: { + "id": wid, + "name": "Align Workflow", + "version": "2.0.0", + "url": "https://example.com/workflow.yml", + "_install_allowed": True, + "_catalog_name": "test-catalog", + }, + ) + # The URL still serves the old 1.0.0 payload. + data = self.WORKFLOW_YAML.format(version="1.0.0").encode() + runner = CliRunner() + with patch( + "specify_cli.authentication.http.open_url", + side_effect=lambda url, timeout=None, extra_headers=None, redirect_validator=None: self._FakeResponse(data, url), + ): + result = runner.invoke(app, ["workflow", "update"], input="y\n") + assert "does not match the catalog version" in result.output + assert "Failed to update" in result.output + meta = WorkflowRegistry(project_dir).get("align-wf") + assert meta["version"] == "1.0.0" + assert "1.0.0" in (wf_dir / "workflow.yml").read_text(encoding="utf-8") + + def test_update_preserves_disabled_state(self, project_dir, monkeypatch): + from unittest.mock import patch + from typer.testing import CliRunner + from specify_cli import app + from specify_cli.workflows.catalog import WorkflowCatalog, WorkflowRegistry + + monkeypatch.chdir(project_dir) + WorkflowRegistry(project_dir).add("align-wf", { + "name": "Align Workflow", + "version": "1.0.0", + "description": "", + "source": "catalog", + "url": "https://example.com/workflow.yml", + "enabled": False, + }) + monkeypatch.setattr( + WorkflowCatalog, + "get_workflow_info", + lambda self, wid: { + "id": wid, + "version": "2.0.0", + "url": "https://example.com/workflow.yml", + "_install_allowed": True, + }, + ) + data = self.WORKFLOW_YAML.format(version="2.0.0").encode() + runner = CliRunner() + with patch( + "specify_cli.authentication.http.open_url", + side_effect=lambda url, timeout=None, extra_headers=None, redirect_validator=None: self._FakeResponse(data, url), + ): + result = runner.invoke(app, ["workflow", "update"], input="y\n") + assert result.exit_code == 0, result.output + meta = WorkflowRegistry(project_dir).get("align-wf") + assert meta["version"] == "2.0.0" + assert meta["enabled"] is False + + @pytest.mark.skipif(os.name == "nt", reason="POSIX permission bits") + def test_update_preserves_workflow_file_mode(self, project_dir, monkeypatch): + import stat + from unittest.mock import patch + from typer.testing import CliRunner + from specify_cli import app + from specify_cli.workflows.catalog import WorkflowCatalog, WorkflowRegistry + + monkeypatch.chdir(project_dir) + WorkflowRegistry(project_dir).add("align-wf", { + "name": "Align Workflow", + "version": "1.0.0", + "description": "", + "source": "catalog", + "url": "https://example.com/workflow.yml", + }) + workflow_file = ( + project_dir + / ".specify" + / "workflows" + / "align-wf" + / "workflow.yml" + ) + workflow_file.parent.mkdir(parents=True) + workflow_file.write_text( + self.WORKFLOW_YAML.format(version="1.0.0"), encoding="utf-8" + ) + workflow_file.chmod(0o640) + + monkeypatch.setattr( + WorkflowCatalog, + "get_workflow_info", + lambda self, wid: { + "id": wid, + "version": "2.0.0", + "url": "https://example.com/workflow.yml", + "_install_allowed": True, + }, + ) + data = self.WORKFLOW_YAML.format(version="2.0.0").encode() + with patch( + "specify_cli.authentication.http.open_url", + side_effect=lambda url, timeout=None, extra_headers=None, + redirect_validator=None: self._FakeResponse(data, url), + ): + result = CliRunner().invoke( + app, ["workflow", "update"], input="y\n" + ) + + assert result.exit_code == 0, result.output + assert stat.S_IMODE(workflow_file.stat().st_mode) == 0o640 + + def test_update_skips_corrupted_registry_entry(self, project_dir, monkeypatch): + import json + from typer.testing import CliRunner + from specify_cli import app + from specify_cli.workflows.catalog import WorkflowRegistry + + monkeypatch.chdir(project_dir) + registry_path = WorkflowRegistry(project_dir).registry_path + registry_path.parent.mkdir(parents=True, exist_ok=True) + registry_path.write_text( + json.dumps({"schema_version": "1.0", "workflows": {"broken": "not-a-dict"}}), + encoding="utf-8", + ) + runner = CliRunner() + result = runner.invoke(app, ["workflow", "update"]) + assert result.exit_code == 0, result.output + assert "corrupted" in result.output + + def test_list_skips_corrupted_registry_entry(self, project_dir, monkeypatch): + import json + from typer.testing import CliRunner + from specify_cli import app + from specify_cli.workflows.catalog import WorkflowRegistry + + monkeypatch.chdir(project_dir) + registry_path = WorkflowRegistry(project_dir).registry_path + registry_path.parent.mkdir(parents=True, exist_ok=True) + registry_path.write_text( + json.dumps( + { + "schema_version": "1.0", + "workflows": { + "broken": "not-a-dict", + "ok": {"name": "OK Workflow", "version": "1.0.0"}, + }, + } + ), + encoding="utf-8", + ) + runner = CliRunner() + result = runner.invoke(app, ["workflow", "list"]) + assert result.exit_code == 0, result.output + assert "corrupted" in result.output + assert "OK Workflow" in result.output + + def test_list_unreadable_registry_fails_closed_with_clean_error( + self, project_dir, monkeypatch + ): + """An unreadable registry file must produce a clean CLI error, not a + raw traceback and not a silent "nothing installed" list -- the latter + is exactly the fail-open state a caller could otherwise mistake for + "safe to (re)install", overwriting real files. Covers the read/query + boundary fix required at every WorkflowRegistry call site.""" + import builtins + from typer.testing import CliRunner + from specify_cli import app + from specify_cli.workflows.catalog import WorkflowRegistry + + monkeypatch.chdir(project_dir) + runner = CliRunner() + self._install_dev(runner, app, project_dir) + + registry_path = WorkflowRegistry(project_dir).registry_path.resolve() + real_open = builtins.open + + def _raising_open(file, mode="r", *args, **kwargs): + if Path(file).resolve() == registry_path and "r" in mode: + raise OSError("simulated read failure") + return real_open(file, mode, *args, **kwargs) + + monkeypatch.setattr(builtins, "open", _raising_open) + result = runner.invoke(app, ["workflow", "list"]) + assert result.exit_code != 0 + assert result.exception is None or isinstance(result.exception, SystemExit) + assert "Error" in result.output + + def test_list_escapes_rich_markup_in_registry_fields(self, project_dir, monkeypatch): + """User-editable name/description/id fields must not be parsed as Rich markup.""" + import json + from typer.testing import CliRunner + from specify_cli import app + from specify_cli.workflows.catalog import WorkflowRegistry + + monkeypatch.chdir(project_dir) + registry_path = WorkflowRegistry(project_dir).registry_path + registry_path.parent.mkdir(parents=True, exist_ok=True) + registry_path.write_text( + json.dumps( + { + "schema_version": "1.0", + "workflows": { + "ok": { + "name": "Bracket [Test]", + "version": "1.0.0", + "description": "desc [with] brackets", + }, + }, + } + ), + encoding="utf-8", + ) + runner = CliRunner() + result = runner.invoke(app, ["workflow", "list"]) + assert result.exit_code == 0, result.output + assert "Bracket [Test]" in result.output + assert "desc [with] brackets" in result.output + + def test_update_reports_unsafe_registry_id_per_workflow(self, project_dir, monkeypatch): + """An unsafe workflow id in the registry must fail that one entry, not abort the whole update.""" + import json + from typer.testing import CliRunner + from specify_cli import app + from specify_cli.workflows.catalog import WorkflowRegistry, WorkflowCatalog + + monkeypatch.chdir(project_dir) + registry_path = WorkflowRegistry(project_dir).registry_path + registry_path.parent.mkdir(parents=True, exist_ok=True) + registry_path.write_text( + json.dumps( + { + "schema_version": "1.0", + "workflows": { + "../evil": { + "name": "Bad", + "version": "0.0.1", + "source": "catalog", + "url": "https://example.com/evil.yml", + }, + }, + } + ), + encoding="utf-8", + ) + monkeypatch.setattr( + WorkflowCatalog, + "get_workflow_info", + lambda self, wid: {"version": "9.9.9", "url": "https://example.com/evil.yml", "_install_allowed": True}, + ) + runner = CliRunner() + result = runner.invoke(app, ["workflow", "update"], input="y\n") + assert result.exit_code != 0 + assert "Failed to update" in result.output + + def test_update_registry_save_failure_restores_prior_file_without_redundant_write( + self, project_dir, monkeypatch + ): + """A registry.add() save failure during `workflow update` must be + fully restored by _install_workflow_from_catalog's own atomic + rollback (rename-based, not a byte-level rewrite). The outer + workflow_update loop must not perform any redundant write of its + own onto the destination file -- that write happened only after + typer.Exit already unwound, could itself fail/truncate the safely + preserved file, and is provably unnecessary here since the inner + transaction already restored it via rename.""" + from typer.testing import CliRunner + from specify_cli import app + from specify_cli.workflows.catalog import WorkflowCatalog, WorkflowRegistry + + monkeypatch.chdir(project_dir) + monkeypatch.setattr( + WorkflowCatalog, + "get_workflow_info", + lambda self, wid: { + "id": wid, + "name": "Align Workflow", + "version": "1.0.0", + "url": "https://example.com/workflow.yml", + "_install_allowed": True, + "_catalog_name": "test-catalog", + }, + ) + source_data = self.WORKFLOW_YAML.format(version="1.0.0").encode() + runner = CliRunner() + with pytest.MonkeyPatch.context() as mp: + mp.setattr( + "specify_cli.authentication.http.open_url", + lambda url, timeout=None, extra_headers=None, redirect_validator=None: self._FakeResponse( + source_data, url + ), + ) + result = runner.invoke(app, ["workflow", "add", "align-wf"]) + assert result.exit_code == 0, result.output + + dest_file = project_dir / ".specify" / "workflows" / "align-wf" / "workflow.yml" + original_data = dest_file.read_bytes() + + new_data = self.WORKFLOW_YAML.format(version="2.0.0").encode() + + def boom_save(self): + raise OSError("disk full") + + dest_writes: list[bytes] = [] + real_write_bytes = Path.write_bytes + resolved_dest_file = dest_file.resolve() + + def tracking_write_bytes(self_path, data, *args, **kwargs): + if self_path.resolve() == resolved_dest_file: + dest_writes.append(data) + return real_write_bytes(self_path, data, *args, **kwargs) + + with pytest.MonkeyPatch.context() as mp: + mp.setattr( + WorkflowCatalog, + "get_workflow_info", + lambda self, wid: { + "id": wid, + "name": "Align Workflow", + "version": "2.0.0", + "url": "https://example.com/workflow.yml", + "_install_allowed": True, + "_catalog_name": "test-catalog", + }, + ) + mp.setattr( + "specify_cli.authentication.http.open_url", + lambda url, timeout=None, extra_headers=None, redirect_validator=None: self._FakeResponse( + new_data, url + ), + ) + mp.setattr(WorkflowRegistry, "save", boom_save) + mp.setattr(Path, "write_bytes", tracking_write_bytes) + result = runner.invoke(app, ["workflow", "update"], input="y\n") + + assert result.exit_code != 0 + assert "Failed to update" in result.output + # No redundant/second write of the destination file was attempted -- + # the inner atomic commit/rollback (rename-based) is the only thing + # that ever touches it. + assert dest_writes == [] + assert dest_file.read_bytes() == original_data + registry = WorkflowRegistry(project_dir) + assert registry.is_installed("align-wf") + assert registry.get("align-wf")["version"] == "1.0.0" + + def test_update_non_json_description_restores_prior_file_and_registry( + self, project_dir, monkeypatch + ): + from unittest.mock import patch + + from typer.testing import CliRunner + from specify_cli import app + from specify_cli.workflows.catalog import WorkflowCatalog, WorkflowRegistry + + monkeypatch.chdir(project_dir) + registry = WorkflowRegistry(project_dir) + registry.add( + "align-wf", + { + "name": "Align Workflow", + "version": "1.0.0", + "description": "", + "source": "catalog", + "url": "https://example.com/workflow.yml", + }, + ) + workflow_file = ( + project_dir + / ".specify" + / "workflows" + / "align-wf" + / "workflow.yml" + ) + workflow_file.parent.mkdir(parents=True) + original_data = self.WORKFLOW_YAML.format(version="1.0.0").encode() + workflow_file.write_bytes(original_data) + + monkeypatch.setattr( + WorkflowCatalog, + "get_workflow_info", + lambda self, wid: { + "id": wid, + "name": "Align Workflow", + "version": "2.0.0", + "url": "https://example.com/workflow.yml", + "_install_allowed": True, + "_catalog_name": "test-catalog", + }, + ) + invalid_data = self.WORKFLOW_YAML.format(version="2.0.0").replace( + 'description: "CLI alignment test workflow"', + "description: 2026-01-02", + ).encode() + with patch( + "specify_cli.authentication.http.open_url", + side_effect=lambda url, timeout=None, extra_headers=None, + redirect_validator=None: self._FakeResponse(invalid_data, url), + ): + result = CliRunner().invoke( + app, ["workflow", "update", "align-wf"], input="y\n" + ) + + assert result.exit_code != 0 + assert result.exception is None or isinstance(result.exception, SystemExit) + assert "Failed to update workflow registry" in result.output + assert workflow_file.read_bytes() == original_data + current = WorkflowRegistry(project_dir).get("align-wf") + assert current["version"] == "1.0.0" + leftovers = [ + path.name + for path in workflow_file.parent.iterdir() + if path.name != "workflow.yml" + ] + assert leftovers == [] + + def test_commit_failure_reports_unrestored_backup_location( + self, tmp_path, monkeypatch + ): + from specify_cli.workflows import _commands + + dest_dir = tmp_path / "align-wf" + dest_dir.mkdir() + dest_file = dest_dir / "workflow.yml" + staged_file = dest_dir / ".workflow.yml.staged" + dest_file.write_text("original", encoding="utf-8") + staged_file.write_text("replacement", encoding="utf-8") + + real_replace = os.replace + calls = 0 + backup_file = None + + def fail_commit_and_restore(src, dst): + nonlocal backup_file, calls + calls += 1 + if calls == 1: + backup_file = Path(dst) + return real_replace(src, dst) + if calls == 2: + raise OSError("commit denied") + raise OSError("restore denied") + + monkeypatch.setattr(os, "replace", fail_commit_and_restore) + with pytest.raises(OSError) as exc_info: + _commands._commit_workflow_file( + staged_file, dest_file, existed_before=True + ) + + message = str(exc_info.value) + assert "commit denied" in message + assert "restore denied" in message + assert backup_file is not None + assert str(backup_file) in message + assert not dest_file.exists() + assert backup_file.read_text(encoding="utf-8") == "original" + + def test_commit_keyboard_interrupt_restores_prior_file( + self, tmp_path, monkeypatch + ): + from specify_cli.workflows import _commands + + dest_dir = tmp_path / "align-wf" + dest_dir.mkdir() + dest_file = dest_dir / "workflow.yml" + staged_file = dest_dir / ".workflow.yml.staged" + dest_file.write_text("original", encoding="utf-8") + staged_file.write_text("replacement", encoding="utf-8") + + real_replace = os.replace + calls = 0 + + def interrupt_commit(src, dst): + nonlocal calls + calls += 1 + if calls == 2: + raise KeyboardInterrupt + return real_replace(src, dst) + + monkeypatch.setattr(os, "replace", interrupt_commit) + with pytest.raises(KeyboardInterrupt): + _commands._commit_workflow_file( + staged_file, dest_file, existed_before=True + ) + + assert dest_file.read_text(encoding="utf-8") == "original" + assert staged_file.read_text(encoding="utf-8") == "replacement" + assert list(dest_dir.glob("*.bak")) == [] + + def test_commit_interrupt_after_first_rename_restores_prior_file( + self, tmp_path, monkeypatch + ): + from specify_cli.workflows import _commands + + dest_dir = tmp_path / "align-wf" + dest_dir.mkdir() + dest_file = dest_dir / "workflow.yml" + staged_file = dest_dir / ".workflow.yml.staged" + dest_file.write_text("original", encoding="utf-8") + staged_file.write_text("replacement", encoding="utf-8") + + real_replace = os.replace + calls = 0 + + def interrupt_after_replace(src, dst): + nonlocal calls + calls += 1 + result = real_replace(src, dst) + if calls == 1: + raise KeyboardInterrupt + return result + + monkeypatch.setattr(os, "replace", interrupt_after_replace) + with pytest.raises(KeyboardInterrupt): + _commands._commit_workflow_file( + staged_file, dest_file, existed_before=True + ) + + assert dest_file.read_text(encoding="utf-8") == "original" + assert staged_file.read_text(encoding="utf-8") == "replacement" + assert list(dest_dir.glob("*.bak")) == [] + + def test_commit_uses_unique_backup_without_overwriting_existing_sibling( + self, tmp_path + ): + from specify_cli.workflows import _commands + + dest_dir = tmp_path / "align-wf" + dest_dir.mkdir() + dest_file = dest_dir / "workflow.yml" + staged_file = dest_dir / ".workflow.yml.staged" + fixed_backup = dest_dir / "workflow.yml.bak" + dest_file.write_text("original", encoding="utf-8") + staged_file.write_text("replacement", encoding="utf-8") + fixed_backup.write_text("diagnostic copy", encoding="utf-8") + + backup_file = _commands._commit_workflow_file( + staged_file, dest_file, existed_before=True + ) + + assert backup_file is not None + assert backup_file != fixed_backup + assert backup_file.read_text(encoding="utf-8") == "original" + assert fixed_backup.read_text(encoding="utf-8") == "diagnostic copy" + assert dest_file.read_text(encoding="utf-8") == "replacement" + + @pytest.mark.parametrize( + ("replacement_source", "replacement_version"), + [("local", "1.0.0"), ("catalog", "1.5.0")], + ) + def test_update_rechecks_registry_after_confirmation( + self, project_dir, monkeypatch, replacement_source, replacement_version + ): + from unittest.mock import patch + + from typer.testing import CliRunner + from specify_cli import app + from specify_cli.workflows import _commands + from specify_cli.workflows.catalog import WorkflowCatalog, WorkflowRegistry + + monkeypatch.chdir(project_dir) + registry = WorkflowRegistry(project_dir) + registry.add( + "align-wf", + { + "name": "Align Workflow", + "version": "1.0.0", + "description": "", + "source": "catalog", + "url": "https://example.com/workflow.yml", + }, + ) + workflow_file = ( + project_dir + / ".specify" + / "workflows" + / "align-wf" + / "workflow.yml" + ) + workflow_file.parent.mkdir(parents=True) + workflow_file.write_text( + self.WORKFLOW_YAML.format(version="1.0.0"), encoding="utf-8" + ) + replacement_data = self.WORKFLOW_YAML.format( + version=replacement_version + ).replace( + 'description: "CLI alignment test workflow"', + 'description: "concurrent replacement"', + ).encode() + + monkeypatch.setattr( + WorkflowCatalog, + "get_workflow_info", + lambda self, wid: { + "id": wid, + "name": "Align Workflow", + "version": "2.0.0", + "url": "https://example.com/workflow.yml", + "_install_allowed": True, + "_catalog_name": "test-catalog", + }, + ) + + def replace_while_confirming(*args, **kwargs): + WorkflowRegistry(project_dir).add( + "align-wf", + { + "name": "Concurrent replacement", + "version": replacement_version, + "description": "", + "source": replacement_source, + }, + ) + workflow_file.write_bytes(replacement_data) + return True + + monkeypatch.setattr(_commands.typer, "confirm", replace_while_confirming) + catalog_data = self.WORKFLOW_YAML.format(version="2.0.0").encode() + with patch( + "specify_cli.authentication.http.open_url", + side_effect=lambda url, timeout=None, extra_headers=None, + redirect_validator=None: self._FakeResponse(catalog_data, url), + ): + result = CliRunner().invoke(app, ["workflow", "update", "align-wf"]) + + assert result.exit_code != 0 + assert "changed during update" in result.output + assert workflow_file.read_bytes() == replacement_data + current = WorkflowRegistry(project_dir).get("align-wf") + assert current["source"] == replacement_source + assert current["version"] == replacement_version + + @pytest.mark.skipif(not hasattr(os, "symlink"), reason="symlinks are unavailable") + def test_resume_rejects_symlinked_cross_project_owner_root( + self, project_dir, tmp_path + ): + from specify_cli.workflows import _commands + + real_owner = tmp_path / "real-owner" + real_owner.mkdir() + owner_link = tmp_path / "owner-link" + owner_link.symlink_to(real_owner, target_is_directory=True) + + with pytest.raises(ValueError, match="unavailable"): + _commands._resolve_run_owner_root(str(owner_link), project_dir) + + @pytest.mark.skipif(not hasattr(os, "symlink"), reason="symlinks are unavailable") + def test_resume_rejects_cross_project_owner_with_symlinked_ancestor( + self, project_dir, tmp_path + ): + from specify_cli.workflows import _commands + + real_parent = tmp_path / "real-parent" + owner = real_parent / "owner" + owner.mkdir(parents=True) + parent_link = tmp_path / "parent-link" + parent_link.symlink_to(real_parent, target_is_directory=True) + + with pytest.raises(ValueError, match="unavailable"): + _commands._resolve_run_owner_root( + str(parent_link / "owner"), project_dir + ) + + def test_enable_disable_corrupted_registry_entry_errors(self, project_dir, monkeypatch): + import json + from typer.testing import CliRunner + from specify_cli import app + from specify_cli.workflows.catalog import WorkflowRegistry + + monkeypatch.chdir(project_dir) + registry_path = WorkflowRegistry(project_dir).registry_path + registry_path.parent.mkdir(parents=True, exist_ok=True) + registry_path.write_text( + json.dumps({"schema_version": "1.0", "workflows": {"broken": "not-a-dict"}}), + encoding="utf-8", + ) + runner = CliRunner() + for cmd in ("enable", "disable"): + result = runner.invoke(app, ["workflow", cmd, "broken"]) + assert result.exit_code != 0 + assert "corrupted" in result.output + + def test_update_up_to_date_reports_and_exits_zero(self, project_dir, monkeypatch): + from typer.testing import CliRunner + from specify_cli import app + from specify_cli.workflows.catalog import WorkflowCatalog, WorkflowRegistry + + monkeypatch.chdir(project_dir) + WorkflowRegistry(project_dir).add("align-wf", { + "name": "Align Workflow", + "version": "1.0.0", + "description": "", + "source": "catalog", + "url": "https://example.com/workflow.yml", + }) + monkeypatch.setattr( + WorkflowCatalog, + "get_workflow_info", + lambda self, wid: { + "id": wid, + "version": "1.0.0", + "url": "https://example.com/workflow.yml", + "_install_allowed": True, + }, + ) + runner = CliRunner() + result = runner.invoke(app, ["workflow", "update"]) + assert result.exit_code == 0, result.output + assert "Up to date" in result.output + assert "All workflows are up to date!" in result.output + + def test_update_restores_backup_on_failed_download(self, project_dir, monkeypatch): + from unittest.mock import patch + from typer.testing import CliRunner + from specify_cli import app + from specify_cli.workflows.catalog import WorkflowCatalog, WorkflowRegistry + + monkeypatch.chdir(project_dir) + WorkflowRegistry(project_dir).add("align-wf", { + "name": "Align Workflow", + "version": "1.0.0", + "description": "", + "source": "catalog", + "url": "https://example.com/workflow.yml", + }) + wf_dir = project_dir / ".specify" / "workflows" / "align-wf" + wf_dir.mkdir(parents=True) + original = self.WORKFLOW_YAML.format(version="1.0.0") + (wf_dir / "workflow.yml").write_text(original, encoding="utf-8") + + monkeypatch.setattr( + WorkflowCatalog, + "get_workflow_info", + lambda self, wid: { + "id": wid, + "version": "2.0.0", + "url": "https://example.com/workflow.yml", + "_install_allowed": True, + }, + ) + + def boom(url, timeout=None, extra_headers=None, redirect_validator=None): + raise OSError("network down") + + runner = CliRunner() + with patch("specify_cli.authentication.http.open_url", side_effect=boom): + result = runner.invoke(app, ["workflow", "update"], input="y\n") + assert result.exit_code != 0 + assert "Failed to update" in result.output + # Working copy and registry version are untouched + assert (wf_dir / "workflow.yml").read_text(encoding="utf-8") == original + assert WorkflowRegistry(project_dir).get("align-wf")["version"] == "1.0.0" + + # -- enable / disable ------------------------------------------------ + + def test_disable_blocks_run_enable_restores(self, project_dir, monkeypatch): + from typer.testing import CliRunner + from specify_cli import app + from specify_cli.workflows.catalog import WorkflowRegistry + + monkeypatch.chdir(project_dir) + runner = CliRunner() + self._install_dev(runner, app, project_dir) + + result = runner.invoke(app, ["workflow", "disable", "align-wf"]) + assert result.exit_code == 0, result.output + assert WorkflowRegistry(project_dir).get("align-wf")["enabled"] is False + + result = runner.invoke(app, ["workflow", "run", "align-wf"]) + assert result.exit_code != 0 + assert "disabled" in result.output + + result = runner.invoke(app, ["workflow", "enable", "align-wf"]) + assert result.exit_code == 0, result.output + assert WorkflowRegistry(project_dir).get("align-wf")["enabled"] is True + + result = runner.invoke(app, ["workflow", "run", "align-wf"]) + assert result.exit_code == 0, result.output + + def test_run_rejects_corrupted_registry_entry(self, project_dir, monkeypatch): + from typer.testing import CliRunner + from specify_cli import app + from specify_cli.workflows.catalog import WorkflowRegistry + + monkeypatch.chdir(project_dir) + runner = CliRunner() + self._install_dev(runner, app, project_dir) + + registry = WorkflowRegistry(project_dir) + registry.data["workflows"]["align-wf"] = "corrupted" + registry.save() + + result = runner.invoke(app, ["workflow", "run", "align-wf"]) + assert result.exit_code != 0 + assert "corrupted" in result.output + + def test_run_rejects_corrupt_registry_file(self, project_dir, monkeypatch): + from typer.testing import CliRunner + from specify_cli import app + from specify_cli.workflows.catalog import WorkflowRegistry + + monkeypatch.chdir(project_dir) + runner = CliRunner() + self._install_dev(runner, app, project_dir) + + registry_path = WorkflowRegistry(project_dir).registry_path + registry_path.write_text("not json", encoding="utf-8") + + result = runner.invoke(app, ["workflow", "run", "align-wf"]) + + assert result.exit_code != 0 + assert "registry" in result.output.lower() + assert "corrupt" in result.output.lower() + + def test_disable_blocks_case_variant_installed_path( + self, project_dir, monkeypatch + ): + from typer.testing import CliRunner + from specify_cli import app + + monkeypatch.chdir(project_dir) + runner = CliRunner() + self._install_dev(runner, app, project_dir) + + result = runner.invoke(app, ["workflow", "disable", "align-wf"]) + assert result.exit_code == 0, result.output + + case_variant = ( + project_dir + / ".SPECIFY" + / "WORKFLOWS" + / "ALIGN-WF" + / "workflow.yml" + ) + if not case_variant.is_file(): + pytest.skip("filesystem is case-sensitive") + + result = runner.invoke( + app, ["workflow", "run", str(case_variant)] + ) + + assert result.exit_code != 0 + assert "disabled" in result.output + + def test_disable_blocks_run_via_path_equivalent_id(self, project_dir, monkeypatch): + """Path-equivalent and newline IDs must not dodge the registry lookup.""" + from typer.testing import CliRunner + from specify_cli import app + + monkeypatch.chdir(project_dir) + runner = CliRunner() + self._install_dev(runner, app, project_dir) + + result = runner.invoke(app, ["workflow", "disable", "align-wf"]) + assert result.exit_code == 0, result.output + + for spelling in ("align-wf/", "align-wf/.", "align-wf\n"): + result = runner.invoke(app, ["workflow", "run", spelling]) + assert result.exit_code != 0, spelling + assert "Invalid workflow ID" in result.output, spelling + + # Direct path to the installed workflow's own YAML must also refuse. + installed_yaml = ".specify/workflows/align-wf/workflow.yml" + assert (project_dir / installed_yaml).is_file() + result = runner.invoke(app, ["workflow", "run", installed_yaml]) + assert result.exit_code != 0 + assert "disabled" in result.output + + # Same guard must hold when invoked from outside the project. + outside = project_dir.parent / "outside-cwd" + outside.mkdir(exist_ok=True) + monkeypatch.chdir(outside) + result = runner.invoke( + app, ["workflow", "run", str(project_dir / installed_yaml)] + ) + assert result.exit_code != 0 + assert "disabled" in result.output + + @pytest.mark.skipif(not hasattr(os, "symlink"), reason="symlinks are unavailable") + def test_disable_blocks_run_when_installed_yaml_is_symlinked( + self, project_dir, monkeypatch + ): + """A disabled workflow's own workflow.yml being replaced with a symlink + must not bypass the disabled check. Resolving the path before mapping + it back to its registry owner would follow the symlink out of + .specify/workflows, fail to find an owner, and let engine.load_workflow + run the original symlink target anyway -- ownership must be + determined from the normalized *lexical* path (not resolve()), and a + symlinked path component in the installed tree must be refused.""" + from typer.testing import CliRunner + from specify_cli import app + + monkeypatch.chdir(project_dir) + runner = CliRunner() + self._install_dev(runner, app, project_dir) + + result = runner.invoke(app, ["workflow", "disable", "align-wf"]) + assert result.exit_code == 0, result.output + + installed_yaml = project_dir / ".specify" / "workflows" / "align-wf" / "workflow.yml" + external_target = project_dir / "external-workflow.yml" + external_target.write_text( + self.WORKFLOW_YAML.format(version="9.9.9"), encoding="utf-8" + ) + installed_yaml.unlink() + installed_yaml.symlink_to(external_target) + + result = runner.invoke(app, ["workflow", "run", str(installed_yaml)]) + assert result.exit_code != 0 + assert result.exception is None or isinstance(result.exception, SystemExit) + assert "disabled" in result.output or "symlink" in result.output.lower() + + @pytest.mark.skipif(not hasattr(os, "symlink"), reason="symlinks are unavailable") + def test_alias_rejects_symlinked_workflow_storage_before_resolve( + self, project_dir, temp_dir + ): + import shutil + import typer + from specify_cli.workflows import _commands + + specify_dir = project_dir / ".specify" + shutil.rmtree(specify_dir) + redirected = temp_dir / "redirected-storage" + workflow_file = redirected / "workflows" / "evil" / "workflow.yml" + workflow_file.parent.mkdir(parents=True) + workflow_file.write_text( + self.WORKFLOW_YAML.format(version="1.0.0"), encoding="utf-8" + ) + specify_dir.symlink_to(redirected, target_is_directory=True) + alias = temp_dir / "workflow-alias.yml" + alias.symlink_to( + project_dir + / ".specify" + / "workflows" + / "evil" + / "workflow.yml" + ) + + with pytest.raises(typer.Exit): + _commands._resolve_installed_workflow_ownership( + alias, _commands.err_console + ) + + @pytest.mark.skipif(not hasattr(os, "symlink"), reason="symlinks are unavailable") + def test_run_refuses_symlinked_specify_dir_hiding_disabled_workflow( + self, temp_dir, monkeypatch + ): + """A victim project's own .specify directory being a symlink to an + attacker-controlled tree must not bypass the disabled-workflow guard. + _reject_unsafe_workflow_storage only checks the *cwd's* project root + (unrelated here); the id/leaf symlink-component loop only checks + components from the id directory onward, missing .specify/ + .specify/workflows themselves. The ownership check must reject an + unsafe .specify/.specify-workflows for the actual path-derived + registry root before ever consulting the registry -- it must not + rely on WorkflowRegistry's own symlinked-parent handling, which + raises a generic OSError; the ownership guard should surface the + specific unsafe-storage error before registry construction.""" + from typer.testing import CliRunner + from specify_cli import app + + victim = temp_dir / "victim" + victim.mkdir() + attacker_real = temp_dir / "attacker-real" + (attacker_real / "workflows" / "evil").mkdir(parents=True) + (attacker_real / "workflows" / "evil" / "workflow.yml").write_text( + self.WORKFLOW_YAML.format(version="1.0.0"), encoding="utf-8" + ) + (attacker_real / "workflows" / "workflow-registry.json").write_text( + json.dumps( + { + "schema_version": "1.0", + "workflows": { + "evil": { + "name": "Evil", + "version": "1.0.0", + "source": "dev", + "enabled": False, + } + }, + } + ), + encoding="utf-8", + ) + (victim / ".specify").symlink_to(attacker_real) + + unrelated_cwd = temp_dir / "unrelated-cwd" + unrelated_cwd.mkdir() + monkeypatch.chdir(unrelated_cwd) + + runner = CliRunner() + target = victim / ".specify" / "workflows" / "evil" / "workflow.yml" + result = runner.invoke(app, ["workflow", "run", str(target)]) + assert result.exit_code != 0 + assert result.exception is None or isinstance(result.exception, SystemExit) + assert "symlink" in result.output.lower() + + def test_run_nested_installed_paths_uses_nearest_owner( + self, temp_dir, monkeypatch + ): + """A direct workflow.yml path whose lexical segments contain + .specify/workflows more than once (an unrelated nested project + happens to live beneath an outer installed workflow's own + directory tree, reusing the same segment names) must be attributed + to its *nearest* (innermost) owning project/ID -- scanning from the + start of the path and stopping at the first match would pick the + outer project and the wrong workflow ID, gating the run on an + unrelated workflow's disabled state instead of the real owner's.""" + from typer.testing import CliRunner + from specify_cli import app + + def _write_registry(workflows_dir, workflow_id, enabled): + workflows_dir.mkdir(parents=True, exist_ok=True) + (workflows_dir / "workflow-registry.json").write_text( + json.dumps( + { + "schema_version": "1.0", + "workflows": { + workflow_id: { + "name": workflow_id, + "version": "1.0.0", + "source": "dev", + "enabled": enabled, + } + }, + } + ), + encoding="utf-8", + ) + + outer_workflows = temp_dir / "outer-proj" / ".specify" / "workflows" + outer_wf_dir = outer_workflows / "outer-wf" + outer_wf_dir.mkdir(parents=True) + (outer_wf_dir / "workflow.yml").write_text( + self.WORKFLOW_YAML.format(version="1.0.0"), encoding="utf-8" + ) + _write_registry(outer_workflows, "outer-wf", enabled=False) + + # An unrelated nested project lives inside the outer workflow's own + # directory tree, with its own separate installed workflow. + inner_workflows = outer_wf_dir / "nested-proj" / ".specify" / "workflows" + inner_wf_dir = inner_workflows / "inner-wf" + inner_wf_dir.mkdir(parents=True) + (inner_wf_dir / "workflow.yml").write_text( + self.WORKFLOW_YAML.format(version="1.0.0"), encoding="utf-8" + ) + _write_registry(inner_workflows, "inner-wf", enabled=True) + + unrelated_cwd = temp_dir / "unrelated-cwd" + unrelated_cwd.mkdir() + monkeypatch.chdir(unrelated_cwd) + + runner = CliRunner() + target = inner_wf_dir / "workflow.yml" + result = runner.invoke(app, ["workflow", "run", str(target)]) + # inner-wf (the actual nearest owner) is enabled -- must run, not + # be blocked by the unrelated outer-wf's disabled state. + assert result.exit_code == 0, result.output + + # The inverse proves this isn't just ignoring nesting: disabling + # the true (nearest) owner must actually block this exact path. + _write_registry(inner_workflows, "inner-wf", enabled=False) + result = runner.invoke(app, ["workflow", "run", str(target)]) + assert result.exit_code != 0 + assert "disabled" in result.output + + @pytest.mark.skipif(not hasattr(os, "symlink"), reason="symlinks are unavailable") + def test_run_blocks_disabled_workflow_via_outward_alias_symlink( + self, project_dir, monkeypatch + ): + """The inverse of the existing inward-symlink case: a path with no + .specify/workflows segments at all (e.g. /tmp/alias.yml) that is + itself a symlink resolving *into* installed storage must still + receive the disabled check. Only checking the lexical path's own + segments misses this alias entirely, since it has no such segments + to begin with, and would let engine.load_workflow follow the + symlink to the disabled workflow's real content unchecked.""" + from typer.testing import CliRunner + from specify_cli import app + + monkeypatch.chdir(project_dir) + runner = CliRunner() + self._install_dev(runner, app, project_dir) + + result = runner.invoke(app, ["workflow", "disable", "align-wf"]) + assert result.exit_code == 0, result.output + + installed_yaml = ( + project_dir / ".specify" / "workflows" / "align-wf" / "workflow.yml" + ) + external_dir = project_dir / "outside-alias" + external_dir.mkdir() + alias = external_dir / "alias.yml" + alias.symlink_to(installed_yaml) + + result = runner.invoke(app, ["workflow", "run", str(alias)]) + assert result.exit_code != 0 + assert "disabled" in result.output + + result = runner.invoke(app, ["workflow", "enable", "align-wf"]) + assert result.exit_code == 0, result.output + result = runner.invoke(app, ["workflow", "run", str(alias)]) + assert result.exit_code == 0, result.output + + _GATED_WORKFLOW_YAML = """ +schema_version: "1.0" +workflow: + id: "gated-wf" + name: "Gated Workflow" + version: "1.0.0" +steps: + - id: ask + type: gate + message: "Review" + options: [approve, reject] +""" + + def _install_and_run_gated(self, runner, app, project_dir): + """Install a gate-step workflow and run it to a paused state. + + Returns the run_id. The gate step pauses without any interactive + input, giving a resumable run tied to an installed workflow ID. + """ + src = project_dir / "gated-src" + src.mkdir(exist_ok=True) + (src / "workflow.yml").write_text(self._GATED_WORKFLOW_YAML, encoding="utf-8") + result = runner.invoke(app, ["workflow", "add", str(src), "--dev"]) + assert result.exit_code == 0, result.output + + result = runner.invoke(app, ["workflow", "run", "gated-wf", "--json"]) + assert result.exit_code == 0, result.output + payload = json.loads(result.stdout) + assert payload["status"] == "paused" + return payload["run_id"] + + def test_unregistered_workflow_shaped_path_is_not_persisted_as_owner( + self, project_dir, temp_dir, monkeypatch + ): + """A direct file is not installed merely because its path resembles + installed storage; only registry membership establishes ownership.""" + import shutil + from typer.testing import CliRunner + from specify_cli import app + + standalone_root = temp_dir / "standalone-project" + workflows_dir = standalone_root / ".specify" / "workflows" + workflow_file = workflows_dir / "gated-wf" / "workflow.yml" + workflow_file.parent.mkdir(parents=True) + workflow_file.write_text(self._GATED_WORKFLOW_YAML, encoding="utf-8") + + monkeypatch.chdir(project_dir) + runner = CliRunner() + run_ids = [] + for _ in range(2): + result = runner.invoke( + app, ["workflow", "run", str(workflow_file), "--json"] + ) + assert result.exit_code == 0, result.output + run_ids.append(json.loads(result.stdout)["run_id"]) + + for run_id in run_ids: + state_path = ( + project_dir + / ".specify" + / "workflows" + / "runs" + / run_id + / "state.json" + ) + state = json.loads(state_path.read_text(encoding="utf-8")) + assert state["installed_workflow_id"] is None + assert state["installed_registry_root"] is None + + shutil.rmtree(standalone_root) + result = runner.invoke( + app, ["workflow", "resume", run_ids[0], "--json"] + ) + assert result.exit_code == 0, result.output + + workflows_dir.mkdir(parents=True) + (workflows_dir / "workflow-registry.json").write_text( + json.dumps( + { + "schema_version": "1.0", + "workflows": { + "gated-wf": { + "name": "Unrelated workflow", + "version": "9.9.9", + "source": "dev", + "enabled": False, + } + }, + } + ), + encoding="utf-8", + ) + result = runner.invoke( + app, ["workflow", "resume", run_ids[1], "--json"] + ) + assert result.exit_code == 0, result.output + + def test_resume_blocks_when_installed_workflow_disabled( + self, project_dir, monkeypatch + ): + """A run started from an installed workflow must not resume once + that workflow is disabled. engine.resume() replays the persisted + run directly from disk with no registry awareness at all, so the + installed workflow's origin (id + owning registry root) is + persisted at run start and re-checked against the registry's + *current* state before resuming, mirroring `workflow run`'s + disabled guard.""" + from typer.testing import CliRunner + from specify_cli import app + + monkeypatch.chdir(project_dir) + runner = CliRunner() + run_id = self._install_and_run_gated(runner, app, project_dir) + + result = runner.invoke(app, ["workflow", "disable", "gated-wf"]) + assert result.exit_code == 0, result.output + + result = runner.invoke(app, ["workflow", "resume", run_id]) + assert result.exit_code != 0 + assert "disabled" in result.output + + # Re-enabling must unblock the exact same run. + result = runner.invoke(app, ["workflow", "enable", "gated-wf"]) + assert result.exit_code == 0, result.output + result = runner.invoke(app, ["workflow", "resume", run_id, "--json"]) + assert result.exit_code == 0, result.output + resumed = json.loads(result.stdout) + assert resumed["run_id"] == run_id + + def test_resume_rejects_corrupted_registry_entry( + self, project_dir, monkeypatch + ): + from typer.testing import CliRunner + from specify_cli import app + from specify_cli.workflows.catalog import WorkflowRegistry + + monkeypatch.chdir(project_dir) + runner = CliRunner() + run_id = self._install_and_run_gated(runner, app, project_dir) + + registry = WorkflowRegistry(project_dir) + registry.data["workflows"]["gated-wf"] = "corrupted" + registry.save() + + result = runner.invoke(app, ["workflow", "resume", run_id]) + assert result.exit_code != 0 + assert "corrupted" in result.output + + def test_resume_preload_io_error_is_reported_cleanly( + self, project_dir, monkeypatch + ): + from unittest.mock import patch + from typer.testing import CliRunner + from specify_cli import app + from specify_cli.workflows.engine import RunState + + monkeypatch.chdir(project_dir) + with patch.object( + RunState, "load", side_effect=OSError("permission [denied]") + ): + result = CliRunner().invoke( + app, ["workflow", "resume", "unreadable-run"] + ) + + assert result.exit_code != 0 + assert result.exception is None or isinstance(result.exception, SystemExit) + assert "Resume failed" in result.output + assert "permission [denied]" in result.output + + @pytest.mark.parametrize("malformation", ["non-object", "missing-run-id"]) + def test_resume_preload_rejects_malformed_state_cleanly( + self, project_dir, monkeypatch, malformation + ): + from typer.testing import CliRunner + from specify_cli import app + + monkeypatch.chdir(project_dir) + runner = CliRunner() + run_id = self._install_and_run_gated(runner, app, project_dir) + state_path = ( + project_dir / ".specify" / "workflows" / "runs" / run_id / "state.json" + ) + + if malformation == "non-object": + state_path.write_text("[]", encoding="utf-8") + else: + data = json.loads(state_path.read_text(encoding="utf-8")) + data.pop("run_id") + state_path.write_text(json.dumps(data), encoding="utf-8") + + result = runner.invoke(app, ["workflow", "resume", run_id]) + + assert result.exit_code != 0 + assert result.exception is None or isinstance(result.exception, SystemExit) + assert "Invalid run state" in result.output + + def test_resume_legacy_run_respects_current_disabled_state( + self, project_dir, monkeypatch + ): + """Legacy runs infer same-project registry ownership before resume.""" + from typer.testing import CliRunner + from specify_cli import app + + monkeypatch.chdir(project_dir) + runner = CliRunner() + run_id = self._install_and_run_gated(runner, app, project_dir) + + state_path = ( + project_dir / ".specify" / "workflows" / "runs" / run_id / "state.json" + ) + data = json.loads(state_path.read_text(encoding="utf-8")) + data.pop("installed_workflow_id", None) + data.pop("installed_registry_root", None) + state_path.write_text(json.dumps(data), encoding="utf-8") + + result = runner.invoke(app, ["workflow", "disable", "gated-wf"]) + assert result.exit_code == 0, result.output + + result = runner.invoke(app, ["workflow", "resume", run_id, "--json"]) + assert result.exit_code != 0 + assert "disabled" in result.output + + def test_resume_migrates_legacy_installed_origin_metadata( + self, project_dir, monkeypatch + ): + from typer.testing import CliRunner + from specify_cli import app + + monkeypatch.chdir(project_dir) + runner = CliRunner() + run_id = self._install_and_run_gated(runner, app, project_dir) + + state_path = ( + project_dir / ".specify" / "workflows" / "runs" / run_id / "state.json" + ) + data = json.loads(state_path.read_text(encoding="utf-8")) + data.pop("installed_workflow_id", None) + data.pop("installed_registry_root", None) + state_path.write_text(json.dumps(data), encoding="utf-8") + + result = runner.invoke(app, ["workflow", "resume", run_id, "--json"]) + assert result.exit_code == 0, result.output + + migrated = json.loads(state_path.read_text(encoding="utf-8")) + assert migrated["installed_workflow_id"] == "gated-wf" + assert migrated["installed_registry_root"] is None + + def test_resume_blocks_after_project_moved_following_disable( + self, temp_dir, monkeypatch + ): + """Renaming/moving the entire project after starting a run must not + let a subsequent disable-then-resume bypass the guard. Persisting + the run's *creation-time absolute* project path would make resume + open a now-nonexistent old root (WorkflowRegistry falls back to an + empty default there), missing the disabled entry that actually + lives in the *current* (moved) project's registry. The common, + same-project case must instead re-derive the owning root from the + project's current location on every resume.""" + from typer.testing import CliRunner + from specify_cli import app + import shutil + + project_v1 = temp_dir / "project-v1" + (project_v1 / ".specify" / "workflows").mkdir(parents=True) + monkeypatch.chdir(project_v1) + runner = CliRunner() + run_id = self._install_and_run_gated(runner, app, project_v1) + + project_v2 = temp_dir / "project-v2" + monkeypatch.chdir(temp_dir) + shutil.move(str(project_v1), str(project_v2)) + monkeypatch.chdir(project_v2) + + result = runner.invoke(app, ["workflow", "disable", "gated-wf"]) + assert result.exit_code == 0, result.output + + result = runner.invoke(app, ["workflow", "resume", run_id]) + assert result.exit_code != 0 + assert "disabled" in result.output + + def test_resume_after_project_moved_still_works_when_enabled( + self, temp_dir, monkeypatch + ): + """The inverse of the move regression: an enabled workflow's run + must still resume normally after the project is moved -- the + current-project fallback must not itself block legitimate + resumes.""" + from typer.testing import CliRunner + from specify_cli import app + import shutil + + project_v1 = temp_dir / "project-v1-ok" + (project_v1 / ".specify" / "workflows").mkdir(parents=True) + monkeypatch.chdir(project_v1) + runner = CliRunner() + run_id = self._install_and_run_gated(runner, app, project_v1) + + project_v2 = temp_dir / "project-v2-ok" + monkeypatch.chdir(temp_dir) + shutil.move(str(project_v1), str(project_v2)) + monkeypatch.chdir(project_v2) + + result = runner.invoke(app, ["workflow", "resume", run_id, "--json"]) + assert result.exit_code == 0, result.output + + @pytest.mark.skipif(not hasattr(os, "symlink"), reason="symlinks are unavailable") + def test_resume_respects_cross_project_registry_root( + self, temp_dir, monkeypatch + ): + """A run started via a direct workflow.yml path belonging to a + different project than the cwd used for `workflow run`/`workflow + resume` must still gate resuming on *that* owning project's + registry, not the cwd project's (which has no entry for this ID + at all). This is the genuine cross-project case that must remain + unaffected by only special-casing the common same-project one.""" + from typer.testing import CliRunner + from specify_cli import app + + owner_project = temp_dir / "owner-project" + (owner_project / ".specify" / "workflows").mkdir(parents=True) + monkeypatch.chdir(owner_project) + runner = CliRunner() + src = owner_project / "gated-src" + src.mkdir() + (src / "workflow.yml").write_text(self._GATED_WORKFLOW_YAML, encoding="utf-8") + result = runner.invoke(app, ["workflow", "add", str(src), "--dev"]) + assert result.exit_code == 0, result.output + + unrelated_cwd = temp_dir / "unrelated-cwd" + unrelated_cwd.mkdir() + monkeypatch.chdir(unrelated_cwd) + + owner_alias = temp_dir / "owner-project-alias" + owner_alias.symlink_to(owner_project, target_is_directory=True) + target = owner_alias / ".specify" / "workflows" / "gated-wf" / "workflow.yml" + result = runner.invoke(app, ["workflow", "run", str(target), "--json"]) + assert result.exit_code == 0, result.output + run_id = json.loads(result.stdout)["run_id"] + state_path = ( + unrelated_cwd + / ".specify" + / "workflows" + / "runs" + / run_id + / "state.json" + ) + state = json.loads(state_path.read_text(encoding="utf-8")) + assert state["installed_registry_root"] == str(owner_project.resolve()) + + monkeypatch.chdir(owner_project) + result = runner.invoke(app, ["workflow", "disable", "gated-wf"]) + assert result.exit_code == 0, result.output + + # Resume must run from unrelated_cwd (where this run's own + # state.json actually lives) yet still be blocked by the owner + # project's disabled entry. + monkeypatch.chdir(unrelated_cwd) + result = runner.invoke(app, ["workflow", "resume", run_id]) + assert result.exit_code != 0 + assert "disabled" in result.output + + def test_resume_rejects_missing_cross_project_owner_root( + self, temp_dir, monkeypatch + ): + """A vanished explicit cross-project owner cannot be safely + rediscovered, so resume must fail closed instead of consulting the + unrelated project that stores the run state.""" + from typer.testing import CliRunner + from specify_cli import app + import shutil + + owner_project = temp_dir / "owner-project-2" + (owner_project / ".specify" / "workflows").mkdir(parents=True) + monkeypatch.chdir(owner_project) + runner = CliRunner() + src = owner_project / "gated-src" + src.mkdir() + (src / "workflow.yml").write_text(self._GATED_WORKFLOW_YAML, encoding="utf-8") + result = runner.invoke(app, ["workflow", "add", str(src), "--dev"]) + assert result.exit_code == 0, result.output + + unrelated_cwd = temp_dir / "unrelated-cwd-2" + unrelated_cwd.mkdir() + monkeypatch.chdir(unrelated_cwd) + + target = owner_project / ".specify" / "workflows" / "gated-wf" / "workflow.yml" + result = runner.invoke(app, ["workflow", "run", str(target), "--json"]) + assert result.exit_code == 0, result.output + run_id = json.loads(result.stdout)["run_id"] + + # owner_project vanishes entirely -- its persisted absolute root + # is now dangling. + shutil.rmtree(owner_project) + + result = runner.invoke(app, ["workflow", "resume", run_id]) + assert result.exit_code != 0 + assert "owner" in result.output.lower() + assert "unavailable" in result.output.lower() + + @pytest.mark.parametrize( + "field, bad_value", + [ + ("installed_workflow_id", 123), + ("installed_workflow_id", ["gated-wf"]), + ("installed_workflow_id", {"id": "gated-wf"}), + ("installed_workflow_id", True), + ("installed_workflow_id", ""), + ("installed_workflow_id", "gated-wf\n"), + ("installed_registry_root", 123), + ("installed_registry_root", ["."]), + ("installed_registry_root", {"root": "."}), + ("installed_registry_root", False), + ("installed_registry_root", ""), + ("installed_registry_root", "relative-owner"), + ], + ) + def test_resume_rejects_malformed_run_state_origin_fields( + self, project_dir, monkeypatch, field, bad_value + ): + """RunState.load() rejects malformed or unsafe origin metadata + before registry/path lookups and reports a clean CLI error.""" + from typer.testing import CliRunner + from specify_cli import app + + monkeypatch.chdir(project_dir) + runner = CliRunner() + run_id = self._install_and_run_gated(runner, app, project_dir) + + state_path = ( + project_dir / ".specify" / "workflows" / "runs" / run_id / "state.json" + ) + data = json.loads(state_path.read_text(encoding="utf-8")) + data[field] = bad_value + state_path.write_text(json.dumps(data), encoding="utf-8") + + result = runner.invoke(app, ["workflow", "resume", run_id]) + assert result.exit_code != 0 + assert result.exception is None or isinstance(result.exception, SystemExit) + assert "Error" in result.output + + @pytest.mark.parametrize("command", ["resume", "status"]) + def test_state_load_errors_escape_rich_markup( + self, project_dir, monkeypatch, command + ): + from typer.testing import CliRunner + from specify_cli import app + + monkeypatch.chdir(project_dir) + runner = CliRunner() + run_id = self._install_and_run_gated(runner, app, project_dir) + + state_path = ( + project_dir / ".specify" / "workflows" / "runs" / run_id / "state.json" + ) + data = json.loads(state_path.read_text(encoding="utf-8")) + malicious_status = "[bold red]forged[/bold red]" + data["status"] = malicious_status + state_path.write_text(json.dumps(data), encoding="utf-8") + + result = runner.invoke(app, ["workflow", command, run_id]) + + assert result.exit_code != 0 + assert malicious_status in result.output + + @pytest.mark.parametrize( + "installed_workflow_id, installed_registry_root", + [ + (None, None), + ("gated-wf", None), + ], + ) + def test_resume_accepts_valid_run_state_origin_fields( + self, project_dir, monkeypatch, installed_workflow_id, installed_registry_root + ): + """Valid installed-origin values continue to load and resume.""" + from typer.testing import CliRunner + from specify_cli import app + + monkeypatch.chdir(project_dir) + runner = CliRunner() + run_id = self._install_and_run_gated(runner, app, project_dir) + + state_path = ( + project_dir / ".specify" / "workflows" / "runs" / run_id / "state.json" + ) + data = json.loads(state_path.read_text(encoding="utf-8")) + data["installed_workflow_id"] = installed_workflow_id + data["installed_registry_root"] = installed_registry_root + state_path.write_text(json.dumps(data), encoding="utf-8") + + result = runner.invoke(app, ["workflow", "resume", run_id, "--json"]) + assert result.exit_code == 0, result.output + + @pytest.mark.parametrize( + "field, bad_value", + [ + ("installed_workflow_id", 123), + ("installed_workflow_id", ["gated-wf"]), + ("installed_registry_root", 123), + ("installed_registry_root", ["."]), + ], + ) + def test_status_rejects_malformed_run_state_origin_fields( + self, project_dir, monkeypatch, field, bad_value + ): + """`workflow status ` calls RunState.load() same as resume, + but only caught FileNotFoundError -- the new type validation there + (int/list instead of str-or-null) raises ValueError, which leaked + as a raw unhandled traceback instead of `workflow resume`'s clean + `[red]Error:[/red] {exc}` + exit 1. Must get the identical clean + boundary, leaving the no-run-id list path (and FileNotFoundError + behavior) unchanged.""" + from typer.testing import CliRunner + from specify_cli import app + + monkeypatch.chdir(project_dir) + runner = CliRunner() + run_id = self._install_and_run_gated(runner, app, project_dir) + + state_path = ( + project_dir / ".specify" / "workflows" / "runs" / run_id / "state.json" + ) + data = json.loads(state_path.read_text(encoding="utf-8")) + data[field] = bad_value + state_path.write_text(json.dumps(data), encoding="utf-8") + + result = runner.invoke(app, ["workflow", "status", run_id]) + assert result.exit_code != 0 + assert result.exception is None or isinstance(result.exception, SystemExit) + assert "Error" in result.output + + def test_status_run_not_found_unchanged(self, project_dir, monkeypatch): + """FileNotFoundError behavior for a nonexistent run_id must remain + exactly as before this fix.""" + from typer.testing import CliRunner + from specify_cli import app + + monkeypatch.chdir(project_dir) + (project_dir / ".specify" / "workflows").mkdir(parents=True, exist_ok=True) + runner = CliRunner() + result = runner.invoke(app, ["workflow", "status", "nonexistent-run"]) + assert result.exit_code != 0 + assert "Run not found: nonexistent-run" in result.output + + def test_status_no_run_id_list_path_unaffected(self, project_dir, monkeypatch): + """The no-run-id list-all-runs path must remain unaffected by the + new single-run ValueError boundary.""" + from typer.testing import CliRunner + from specify_cli import app + + monkeypatch.chdir(project_dir) + (project_dir / ".specify" / "workflows").mkdir(parents=True, exist_ok=True) + runner = CliRunner() + result = runner.invoke(app, ["workflow", "status"]) + assert result.exit_code == 0, result.output + + def test_disable_shows_marker_in_list(self, project_dir, monkeypatch): + from typer.testing import CliRunner + from specify_cli import app + + monkeypatch.chdir(project_dir) + runner = CliRunner() + self._install_dev(runner, app, project_dir) + runner.invoke(app, ["workflow", "disable", "align-wf"]) + result = runner.invoke(app, ["workflow", "list"]) + assert result.exit_code == 0, result.output + assert "[disabled]" in result.output + + def test_enable_disable_not_installed_errors(self, project_dir, monkeypatch): + from typer.testing import CliRunner + from specify_cli import app + + monkeypatch.chdir(project_dir) + runner = CliRunner() + for cmd in ("enable", "disable"): + result = runner.invoke(app, ["workflow", cmd, "ghost"]) + assert result.exit_code != 0 + assert "not installed" in result.output + + def test_enable_disable_idempotent_warnings(self, project_dir, monkeypatch): + from typer.testing import CliRunner + from specify_cli import app + + monkeypatch.chdir(project_dir) + runner = CliRunner() + self._install_dev(runner, app, project_dir) + + result = runner.invoke(app, ["workflow", "enable", "align-wf"]) + assert result.exit_code == 0 + assert "already enabled" in result.output + + runner.invoke(app, ["workflow", "disable", "align-wf"]) + result = runner.invoke(app, ["workflow", "disable", "align-wf"]) + assert result.exit_code == 0 + assert "already disabled" in result.output From d83b8d11881405fef6b761c17a15becd79294805 Mon Sep 17 00:00:00 2001 From: Vincent Lee <5388077+vincentclee@users.noreply.github.com> Date: Tue, 14 Jul 2026 09:05:00 -0400 Subject: [PATCH 37/45] fix: add trailing newline to init-options.json output (#3509) `save_init_options()` omitted a final newline, causing `end-of-file-fixer` from .pre-commit-config.yaml (#3430) to flag a diff on every `specify integration upgrade` run. Append `\n` to the `json.dumps()` output to match POSIX expectations and align with `integration_state.py` which already includes the trailing newline. Ref: https://github.com/github/spec-kit/pull/3430 --- src/specify_cli/_init_options.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/specify_cli/_init_options.py b/src/specify_cli/_init_options.py index 6acf4a000b..dd225d8251 100644 --- a/src/specify_cli/_init_options.py +++ b/src/specify_cli/_init_options.py @@ -14,7 +14,7 @@ def save_init_options(project_path: Path, options: dict[str, Any]) -> None: dest = project_path / INIT_OPTIONS_FILE dest.parent.mkdir(parents=True, exist_ok=True) dest.write_text( - json.dumps(options, indent=2, sort_keys=True, ensure_ascii=False), + json.dumps(options, indent=2, sort_keys=True, ensure_ascii=False) + "\n", encoding="utf-8", ) From 73093954e283f700cb772b1719f552c0113d824a Mon Sep 17 00:00:00 2001 From: Noor ul ain Date: Tue, 14 Jul 2026 18:07:33 +0500 Subject: [PATCH 38/45] fix(workflows): evaluate 'in'/'not in' safely on a non-iterable right operand (#3447) (#3468) * fix(workflows): evaluate 'in'/'not in' safely on a non-iterable right operand (#3447) The `in` / `not in` operators in `_evaluate_simple_expression` only guarded `right is not None`, but `left in right` also raises `TypeError` for any other non-iterable right operand (int, bool, float). So a workflow condition like `{{ inputs.tag in inputs.count }}` where `count` is a number leaked a raw `TypeError: argument of type 'int' is not iterable` and crashed the whole run, instead of evaluating like the None case beside it. This was asymmetric with `_safe_compare`, which already swallows `TypeError` and returns False for the ordering operators. Add a `_safe_contains` helper (mirroring `_safe_compare`) that treats both a None and a non-container right operand as "nothing is contained": `in` -> False, `not in` -> True. Add a regression test covering int/bool/float/None right operands and asserting genuine containment against iterables still works. Co-Authored-By: Claude Opus 4.8 (1M context) * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * fix(workflows): address review feedback on #3468 #3447 was fixed independently by #3448 (merged first), which added the same _safe_membership helper this branch introduced. Per Copilot review: - Revert the redundant _safe_contains rename in expressions.py so the file matches main; the working membership guard already lives there. - Drop the duplicate test_in_operator_non_iterable_right_operand test and fold its only new coverage (not in against float/bool/None right operands, which the base test only checked for the int case) into the existing test_membership_against_non_iterable_is_false_not_error. Also merges latest upstream/main into the branch. Co-Authored-By: Claude Opus 4.8 (1M context) --------- Co-authored-by: Claude Opus 4.8 (1M context) Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> --- tests/test_workflows.py | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/tests/test_workflows.py b/tests/test_workflows.py index d5a6d26971..d129257dbd 100644 --- a/tests/test_workflows.py +++ b/tests/test_workflows.py @@ -475,11 +475,11 @@ def test_membership_against_non_iterable_is_false_not_error(self): # previous `right is not None` guard and mirrors _safe_compare, which # already swallows TypeError for the ordering operators. ctx = StepContext(inputs={"tag": "x", "count": 5, "ratio": 1.5, "flag": True}) - assert evaluate_expression("{{ inputs.tag in inputs.count }}", ctx) is False - assert evaluate_expression("{{ inputs.tag not in inputs.count }}", ctx) is True - assert evaluate_expression("{{ 'a' in inputs.ratio }}", ctx) is False - assert evaluate_expression("{{ 'a' in inputs.flag }}", ctx) is False - assert evaluate_expression("{{ inputs.tag in inputs.missing }}", ctx) is False + # `in` -> False and `not in` -> True for every non-iterable right + # operand (int, float, bool, None), so neither operator can drift. + for right in ("count", "ratio", "flag", "missing"): + assert evaluate_expression(f"{{{{ inputs.tag in inputs.{right} }}}}", ctx) is False + assert evaluate_expression(f"{{{{ inputs.tag not in inputs.{right} }}}}", ctx) is True # A condition that would otherwise crash the run now evaluates cleanly. assert evaluate_condition("{{ inputs.tag in inputs.count }}", ctx) is False From e742b8010af5f0d973e49db0c84ea815ce355b6b Mon Sep 17 00:00:00 2001 From: Noor ul ain Date: Tue, 14 Jul 2026 18:11:23 +0500 Subject: [PATCH 39/45] fix(workflows): raise catalog error, not raw ValueError, on a malformed catalog URL (#3484) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * fix(workflows): raise catalog error, not raw ValueError, on a malformed catalog URL The four catalog URL validators in `workflows/catalog.py` (`WorkflowCatalog`/`StepCatalog` `_validate_catalog_url`, and the nested fetch-path validators) accessed `urlparse(url).hostname` unguarded. A malformed authority — e.g. an unterminated IPv6 bracket `https://[::1` or a bracketed non-IP host `https://[not-an-ip]` — makes urlparse / hostname raise `ValueError`. Each validator's contract is to raise a domain error (`WorkflowValidationError` / `StepValidationError` / `WorkflowCatalogError` / `StepCatalogError`), and the command handlers catch only those. So `specify workflow catalog add "https://[::1"` surfaced an uncaught `ValueError` traceback instead of the clean `Error: Catalog URL is malformed` + exit 1 that a bad URL should give. The fetch-path validators also run on the post-redirect `resp.geturl()`, so a hostile redirect target could crash the fetch the same way. Guard each `urlparse`/`.hostname` access with `try/except ValueError -> domain error`, mirroring the fixes already applied to `specify_cli.catalogs` (#3435) and the bundler adapters (#3433). Also read `hostname` once and reuse it for the host check, matching those siblings. Co-Authored-By: Claude Opus 4.8 (1M context) * test(workflows): cover post-redirect malformed-URL guard (#3484 review) Copilot review asked for regression tests on the fetch-path validators that re-check resp.geturl() after redirects — the branch that turns a malformed redirect target into a domain error instead of a raw ValueError. - test_fetch_malformed_redirect_target_raises_catalog_error on both TestWorkflowCatalog and TestStepCatalog: stub open_url with a response whose geturl() is malformed (https://[::1 / https://[not-an-ip]/x) while entry.url is valid, so validation only trips on the redirect target, and assert _fetch_single_catalog raises WorkflowCatalogError / StepCatalogError with a "malformed" message (force_refresh + fresh project_dir so no cache masks it). - Test-the-test: both fail on pre-fix source (raw ValueError re-wrapped as "...Invalid IPv6 URL", no "malformed" match) and pass with the guard. Also merges latest upstream/main into the branch. Co-Authored-By: Claude Opus 4.8 (1M context) --------- Co-authored-by: Claude Opus 4.8 (1M context) --- src/specify_cli/workflows/catalog.py | 70 ++++++++++--- tests/test_workflows.py | 146 +++++++++++++++++++++++++++ 2 files changed, 204 insertions(+), 12 deletions(-) diff --git a/src/specify_cli/workflows/catalog.py b/src/specify_cli/workflows/catalog.py index 654fa76e8b..57cc502834 100644 --- a/src/specify_cli/workflows/catalog.py +++ b/src/specify_cli/workflows/catalog.py @@ -299,8 +299,20 @@ def _validate_catalog_url(self, url: str) -> None: """Validate that a catalog URL uses HTTPS (localhost HTTP allowed).""" from urllib.parse import urlparse - parsed = urlparse(url) - is_localhost = parsed.hostname in ("localhost", "127.0.0.1", "::1") + # A malformed authority (e.g. an unterminated IPv6 bracket + # "https://[::1") makes urlparse / hostname access raise ValueError. + # This validator's contract is to raise WorkflowValidationError for a + # bad URL, so surface that rather than leaking a raw ValueError past the + # command handler (which only catches WorkflowValidationError). Mirrors + # specify_cli.catalogs (#3435). + try: + parsed = urlparse(url) + hostname = parsed.hostname + except ValueError: + raise WorkflowValidationError( + f"Catalog URL is malformed: {url}" + ) from None + is_localhost = hostname in ("localhost", "127.0.0.1", "::1") if parsed.scheme != "https" and not ( parsed.scheme == "http" and is_localhost ): @@ -308,7 +320,7 @@ def _validate_catalog_url(self, url: str) -> None: f"Catalog URL must use HTTPS (got {parsed.scheme}://). " "HTTP is only allowed for localhost." ) - if not parsed.hostname: + if not hostname: raise WorkflowValidationError( "Catalog URL must be a valid URL with a host." ) @@ -474,15 +486,26 @@ def _fetch_single_catalog( from specify_cli.authentication.http import open_url as _open_url def _validate_catalog_url(url: str) -> None: - parsed = urlparse(url) - is_localhost = parsed.hostname in ("localhost", "127.0.0.1", "::1") + # A malformed authority (e.g. "https://[::1") makes urlparse / + # hostname access raise ValueError; treat it as a refused fetch + # rather than leaking a raw ValueError (this also validates the + # post-redirect resp.geturl(), so a hostile redirect target cannot + # crash the fetch either). + try: + parsed = urlparse(url) + hostname = parsed.hostname + except ValueError: + raise WorkflowCatalogError( + f"Refusing to fetch catalog from malformed URL: {url}" + ) from None + is_localhost = hostname in ("localhost", "127.0.0.1", "::1") if parsed.scheme != "https" and not ( parsed.scheme == "http" and is_localhost ): raise WorkflowCatalogError( f"Refusing to fetch catalog from non-HTTPS URL: {url}" ) - if not parsed.hostname: + if not hostname: raise WorkflowCatalogError( f"Refusing to fetch catalog from URL with no hostname: {url}" ) @@ -921,8 +944,20 @@ def _validate_catalog_url(self, url: str) -> None: """Validate that a catalog URL uses HTTPS (localhost HTTP allowed).""" from urllib.parse import urlparse - parsed = urlparse(url) - is_localhost = parsed.hostname in ("localhost", "127.0.0.1", "::1") + # A malformed authority (e.g. an unterminated IPv6 bracket + # "https://[::1") makes urlparse / hostname access raise ValueError. + # This validator's contract is to raise StepValidationError for a bad + # URL, so surface that rather than leaking a raw ValueError past the + # command handler (which only catches StepValidationError). Mirrors + # specify_cli.catalogs (#3435). + try: + parsed = urlparse(url) + hostname = parsed.hostname + except ValueError: + raise StepValidationError( + f"Catalog URL is malformed: {url}" + ) from None + is_localhost = hostname in ("localhost", "127.0.0.1", "::1") if parsed.scheme != "https" and not ( parsed.scheme == "http" and is_localhost ): @@ -930,7 +965,7 @@ def _validate_catalog_url(self, url: str) -> None: f"Catalog URL must use HTTPS (got {parsed.scheme}://). " "HTTP is only allowed for localhost." ) - if not parsed.hostname: + if not hostname: raise StepValidationError( "Catalog URL must be a valid URL with a host." ) @@ -1096,15 +1131,26 @@ def _fetch_single_catalog( from specify_cli.authentication.http import open_url as _open_url def _validate_url(url: str) -> None: - parsed = urlparse(url) - is_localhost = parsed.hostname in ("localhost", "127.0.0.1", "::1") + # A malformed authority (e.g. "https://[::1") makes urlparse / + # hostname access raise ValueError; treat it as a refused fetch + # rather than leaking a raw ValueError (this also validates the + # post-redirect resp.geturl(), so a hostile redirect target cannot + # crash the fetch either). + try: + parsed = urlparse(url) + hostname = parsed.hostname + except ValueError: + raise StepCatalogError( + f"Refusing to fetch catalog from malformed URL: {url}" + ) from None + is_localhost = hostname in ("localhost", "127.0.0.1", "::1") if parsed.scheme != "https" and not ( parsed.scheme == "http" and is_localhost ): raise StepCatalogError( f"Refusing to fetch catalog from non-HTTPS URL: {url}" ) - if not parsed.hostname: + if not hostname: raise StepCatalogError( f"Refusing to fetch catalog from URL with no hostname: {url}" ) diff --git a/tests/test_workflows.py b/tests/test_workflows.py index d129257dbd..8fefad740e 100644 --- a/tests/test_workflows.py +++ b/tests/test_workflows.py @@ -5246,6 +5246,83 @@ def test_validate_url_localhost_http_allowed(self, project_dir): # Should not raise catalog._validate_catalog_url("http://localhost:8080/catalog.json") + @pytest.mark.parametrize( + "url", + [ + "https://[::1", # unterminated IPv6 bracket + "https://[not-an-ip]/x", # bracketed non-IP host + ], + ) + def test_validate_url_malformed_raises_validation_error(self, project_dir, url): + """A malformed authority must raise WorkflowValidationError, not leak a + raw ValueError. + + ``urlparse``/``.hostname`` raise ValueError on a malformed IPv6 + authority. The command handler only catches WorkflowValidationError, + so a raw ValueError would surface as an uncaught traceback instead of a + clean 'Error:' message + exit 1. Mirrors specify_cli.catalogs (#3435). + """ + from specify_cli.workflows.catalog import ( + WorkflowCatalog, + WorkflowValidationError, + ) + + catalog = WorkflowCatalog(project_dir) + with pytest.raises(WorkflowValidationError, match="malformed"): + catalog._validate_catalog_url(url) + + def test_fetch_malformed_redirect_target_raises_catalog_error( + self, project_dir, monkeypatch + ): + """A malformed post-redirect URL must raise WorkflowCatalogError, not a + raw ValueError. + + The fetch path re-validates ``resp.geturl()`` after following redirects, + so a hostile/broken redirect to a malformed authority + (``https://[::1``) hits ``urlparse``/``.hostname`` and raises + ``ValueError``. Without the guard that ValueError is re-wrapped by the + broad ``except`` as ``Failed to fetch catalog ...: Invalid IPv6 URL``; + the guard turns it into a clean ``... malformed URL ...`` refusal. The + initial ``entry.url`` is valid so validation only trips on the redirect + target. Mirrors specify_cli.catalogs (#3435). + """ + from specify_cli.workflows.catalog import ( + WorkflowCatalog, + WorkflowCatalogEntry, + WorkflowCatalogError, + ) + from specify_cli.authentication import http as auth_http + + class _FakeResponse: + def __enter__(self): + return self + + def __exit__(self, exc_type, exc, tb): + return False + + def read(self): + return b"{}" + + def geturl(self): + # A redirect landing on a malformed IPv6 authority. + return "https://[::1" + + monkeypatch.setattr( + auth_http, "open_url", lambda url, timeout=30: _FakeResponse() + ) + + catalog = WorkflowCatalog(project_dir) + entry = WorkflowCatalogEntry( + url="https://example.com/catalog.json", + name="test", + priority=1, + install_allowed=True, + ) + # A fresh project_dir has no cache to fall back to, so the error + # propagates instead of being masked by a stale-cache read. + with pytest.raises(WorkflowCatalogError, match="malformed"): + catalog._fetch_single_catalog(entry, force_refresh=True) + def test_add_catalog(self, project_dir): from specify_cli.workflows.catalog import WorkflowCatalog @@ -5692,6 +5769,75 @@ def test_validate_url_localhost_http_allowed(self, project_dir): # Should not raise catalog._validate_catalog_url("http://localhost:8080/step-catalog.json") + @pytest.mark.parametrize( + "url", + [ + "https://[::1", # unterminated IPv6 bracket + "https://[not-an-ip]/x", # bracketed non-IP host + ], + ) + def test_validate_url_malformed_raises_validation_error(self, project_dir, url): + """A malformed authority must raise StepValidationError, not leak a raw + ValueError past the command handler (which only catches + StepValidationError). Mirrors specify_cli.catalogs (#3435). + """ + from specify_cli.workflows.catalog import StepCatalog, StepValidationError + + catalog = StepCatalog(project_dir) + with pytest.raises(StepValidationError, match="malformed"): + catalog._validate_catalog_url(url) + + def test_fetch_malformed_redirect_target_raises_catalog_error( + self, project_dir, monkeypatch + ): + """A malformed post-redirect URL must raise StepCatalogError, not a raw + ValueError. + + The fetch path re-validates ``resp.geturl()`` after redirects, so a + broken redirect to a bracketed non-IP host (``https://[not-an-ip]/x``) + makes ``urlparse``/``.hostname`` raise ``ValueError``. Without the guard + that leaks out as ``... Invalid IPv6 URL`` re-wrapping; the guard turns + it into a clean ``... malformed URL ...`` refusal. The initial + ``entry.url`` is valid so validation only trips on the redirect target. + Mirrors specify_cli.catalogs (#3435). + """ + from specify_cli.workflows.catalog import ( + StepCatalog, + StepCatalogEntry, + StepCatalogError, + ) + from specify_cli.authentication import http as auth_http + + class _FakeResponse: + def __enter__(self): + return self + + def __exit__(self, exc_type, exc, tb): + return False + + def read(self): + return b"{}" + + def geturl(self): + # A redirect landing on a bracketed non-IP authority. + return "https://[not-an-ip]/x" + + monkeypatch.setattr( + auth_http, "open_url", lambda url, timeout=30: _FakeResponse() + ) + + catalog = StepCatalog(project_dir) + entry = StepCatalogEntry( + url="https://example.com/steps.json", + name="test", + priority=1, + install_allowed=True, + ) + # A fresh project_dir has no cache to fall back to, so the error + # propagates instead of being masked by a stale-cache read. + with pytest.raises(StepCatalogError, match="malformed"): + catalog._fetch_single_catalog(entry, force_refresh=True) + def test_add_catalog(self, project_dir): from specify_cli.workflows.catalog import StepCatalog From 99a3b7ccabdf0a6e227ff065eedb40b61dfa22c4 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Tue, 14 Jul 2026 09:49:05 -0500 Subject: [PATCH 40/45] Update Autonomous Run Governance preset to v0.1.4 (#3511) Update autonomous-run-governance preset submitted by @hindermath to: - presets/catalog.community.json (version, download_url, documentation, provides) - docs/community/presets.md community presets table Closes #3510 Assisted-by: GitHub Copilot (model: claude-sonnet-4.6, autonomous) Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- docs/community/presets.md | 2 +- presets/catalog.community.json | 15 ++++++++------- 2 files changed, 9 insertions(+), 8 deletions(-) diff --git a/docs/community/presets.md b/docs/community/presets.md index 81c01933aa..ae5489e209 100644 --- a/docs/community/presets.md +++ b/docs/community/presets.md @@ -11,7 +11,7 @@ The following community-contributed presets customize how Spec Kit behaves — o | Agent Parity Governance | Adds shared-guidance parity, audit-ready Spec-Kit run evidence, and agent-neutral model-routing guidance across a project's declared AI-agent instruction surfaces so agent guidance does not drift. | 6 templates, 3 commands | — | [spec-kit-preset-agent-parity-governance](https://github.com/hindermath/spec-kit-preset-agent-parity-governance) | | AIDE In-Place Migration | Adapts the AIDE extension workflow for in-place technology migrations (X → Y pattern) — adds migration objectives, verification gates, knowledge documents, and behavioral equivalence criteria | 2 templates, 8 commands | AIDE extension | [spec-kit-presets](https://github.com/mnriem/spec-kit-presets) | | Architecture Governance | Adds secure software architecture, STRIDE+CAPEC threat modeling, arc42 security cross-cutting concepts, S-ADRs, Zero Trust applicability, OWASP SAMM governance, BSI C3A cloud autonomy, BSI C5 cloud compliance assurance, and audit-ready Spec Kit run evidence | 13 templates, 3 commands | — | [spec-kit-preset-architecture-governance](https://github.com/hindermath/spec-kit-preset-architecture-governance) | -| Autonomous Run Governance | Adds permission-bounded, evidence-first governance for autonomous Spec Kit delivery, convergence, resume, closeout, and retrospective learning. | 10 templates, 2 commands | — | [spec-kit-preset-autonomous-run-governance](https://github.com/hindermath/spec-kit-preset-autonomous-run-governance) | +| Autonomous Run Governance | Adds permission-bounded, evidence-first governance for autonomous Spec Kit delivery, convergence, resume, closeout, and retrospective learning. | 12 templates, 2 commands, 2 scripts | — | [spec-kit-preset-autonomous-run-governance](https://github.com/hindermath/spec-kit-preset-autonomous-run-governance) | | Canon Core | Adapts original Spec Kit workflow to work together with Canon extension | 2 templates, 8 commands | — | [spec-kit-canon](https://github.com/maximiliamus/spec-kit-canon) | | Claude AskUserQuestion | Upgrades `/speckit.clarify` and `/speckit.checklist` on Claude Code from Markdown-table prompts to the native AskUserQuestion picker, with a recommended option and reasoning on every question | 2 commands | — | [spec-kit-preset-claude-ask-questions](https://github.com/0xrafasec/spec-kit-preset-claude-ask-questions) | | Command Density | Compacts the nine core Spec Kit command prompts while preserving scripts, handoffs, placeholders, hook output blocks, and rule structure | 9 commands | — | [spec-kit-preset-command-density](https://github.com/Xopoko/spec-kit-preset-command-density) | diff --git a/presets/catalog.community.json b/presets/catalog.community.json index b6bed5dc78..94573625d5 100644 --- a/presets/catalog.community.json +++ b/presets/catalog.community.json @@ -1,6 +1,6 @@ { "schema_version": "1.0", - "updated_at": "2026-07-13T00:00:00Z", + "updated_at": "2026-07-14T00:00:00Z", "catalog_url": "https://raw.githubusercontent.com/github/spec-kit/main/presets/catalog.community.json", "presets": { "a11y-governance": { @@ -134,20 +134,21 @@ "autonomous-run-governance": { "name": "Autonomous Run Governance", "id": "autonomous-run-governance", - "version": "0.1.1", + "version": "0.1.4", "description": "Adds permission-bounded, evidence-first governance for autonomous Spec Kit delivery, convergence, resume, closeout, and retrospective learning.", "author": "Thorsten Hindermann", "repository": "https://github.com/hindermath/spec-kit-preset-autonomous-run-governance", - "download_url": "https://github.com/hindermath/spec-kit-preset-autonomous-run-governance/archive/refs/tags/v0.1.1.zip", + "download_url": "https://github.com/hindermath/spec-kit-preset-autonomous-run-governance/archive/refs/tags/v0.1.4.zip", "homepage": "https://github.com/hindermath/spec-kit-preset-autonomous-run-governance", - "documentation": "https://github.com/hindermath/spec-kit-preset-autonomous-run-governance/blob/main/README.md", + "documentation": "https://github.com/hindermath/spec-kit-preset-autonomous-run-governance/blob/v0.1.4/README.md", "license": "MIT", "requires": { "speckit_version": ">=0.8.3" }, "provides": { - "templates": 10, - "commands": 2 + "templates": 12, + "commands": 2, + "scripts": 2 }, "tags": [ "autonomous", @@ -157,7 +158,7 @@ "retrospective" ], "created_at": "2026-07-13T00:00:00Z", - "updated_at": "2026-07-13T00:00:00Z" + "updated_at": "2026-07-14T00:00:00Z" }, "canon-core": { "name": "Canon Core", From ab825719995c90bbf1856362fa6d2ed09642d7f5 Mon Sep 17 00:00:00 2001 From: Manfred Riem <15701806+mnriem@users.noreply.github.com> Date: Tue, 14 Jul 2026 09:50:31 -0500 Subject: [PATCH 41/45] chore: release 0.12.15, begin 0.12.16.dev0 development (#3513) * chore: bump version to 0.12.15 * chore: begin 0.12.16.dev0 development --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> --- CHANGELOG.md | 15 +++++++++++++++ pyproject.toml | 2 +- 2 files changed, 16 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 61360b0f93..bbc91becb2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,21 @@ +## [0.12.15] - 2026-07-14 + +### Changed + +- Update Autonomous Run Governance preset to v0.1.4 (#3511) +- fix(workflows): raise catalog error, not raw ValueError, on a malformed catalog URL (#3484) +- fix(workflows): evaluate 'in'/'not in' safely on a non-iterable right operand (#3447) (#3468) +- fix: add trailing newline to init-options.json output (#3509) +- feat(workflows): align workflow CLI with extension command surface (#3419) +- fix(extensions): stop env-var config leaking across prefix-colliding extension IDs (#3497) +- fix(integrations): escape control characters in goose recipe YAML renderer (#3384) +- [extension] Update DocGuard — CDD Enforcement extension to v0.32.0 (#3489) +- [extension] Add Multi-Repo Branch Sync extension to community catalog (#3411) +- chore: release 0.12.14, begin 0.12.15.dev0 development (#3506) + ## [0.12.14] - 2026-07-13 ### Changed diff --git a/pyproject.toml b/pyproject.toml index 933c35eadf..e5c50400cc 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,6 +1,6 @@ [project] name = "specify-cli" -version = "0.12.15.dev0" +version = "0.12.16.dev0" description = "Specify CLI, part of GitHub Spec Kit. A tool to bootstrap your projects for Spec-Driven Development (SDD)." readme = "README.md" requires-python = ">=3.11" From 91839fba5015036c03f3a6c9efe1eebb5aaac3ff Mon Sep 17 00:00:00 2001 From: Marsel Safin <179933638+marcelsafin@users.noreply.github.com> Date: Tue, 14 Jul 2026 16:56:10 +0200 Subject: [PATCH 42/45] feat(extensions): port git extension scripts to Python (#3400) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * feat(extensions): port git extension scripts to Python Ports git-common, initialize-repo, auto-commit, and create-new-feature-branch to extensions/git/scripts/python/, mirroring the bash/PowerShell twins. Parity tests run each bash script and its Python twin in identical projects and compare output, exit codes, and resulting git state. Fixes #3282 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: match bash error message for whitespace-only descriptions Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Handle unreadable git-config.yml and assert stderr parity An unreadable config file raised OSError with a full traceback from _parse_auto_commit_config. Treat it like a missing config: auto-commit stays disabled. Covered by a chmod-000 test (skipped on non-POSIX and as root). _assert_parity now also compares stderr so warning or usage-text regressions between the bash and Python twins fail the suite. All existing parity tests pass with the stricter assertion. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(extensions/git): pass script path to core.get_repo_root for cwd-outside-repo callers Without script_file, core.get_repo_root() falls back to Path.cwd() when SPECIFY_INIT_DIR is unset and no .specify root is found upward — the bash twin instead falls back to the script's install location (.specify/scripts/...). Pass script_file so both twins resolve the same repo_root; TypeError fallback keeps older cores working. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * test: exercise SPECIFY_INIT_DIR from outside the project Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(extensions/git): handle UnicodeDecodeError and USER/USERNAME fallback - Catch (OSError, UnicodeDecodeError) when reading git-config.yml in create_new_feature_branch.py, initialize_repo.py, and auto_commit.py so invalid UTF-8 config falls back to defaults instead of crashing with a traceback. - Fall back to USERNAME (then "unknown") when USER is unset when deriving the branch author token, matching the PowerShell twin's Windows-friendly fallback chain. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(extensions/git): platform-aware persist hint and stronger SPECIFY_INIT_DIR test - Add a shared _persist_hint() helper in create_new_feature_branch.py and use it for both the JSON-mode stderr hint and the human-readable stdout hint, so there is a single place emitting the SPECIFY_FEATURE persistence guidance. On Windows (os.name == "nt") it prints PowerShell $env:VAR = "..." syntax; elsewhere it keeps the existing POSIX export VAR=... syntax (parity with the bash twin). - Rework test_specify_init_dir_resolves_target_project so SPECIFY_INIT_DIR is the only thing that can produce the observed result: the script now runs from a separate host_proj (no existing specs, so script/cwd-based discovery would yield 001) while SPECIFY_INIT_DIR points at a different target_proj that already has an existing spec (007-existing, so the override must yield 008). The old version pointed SPECIFY_INIT_DIR at the same project the script was installed in, so it passed even if the env var were ignored. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(extensions): tolerate missing Git executable Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(extensions): quote PowerShell persist hint Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(git): match bash persist hint escaping Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(git): ignore unterminated config record Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * test(git): handle Windows persist hint parity Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(init): install Python shared scripts Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * test(git): normalize Windows persistence hints Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- extensions/git/scripts/python/auto_commit.py | 187 +++++ .../python/create_new_feature_branch.py | 634 +++++++++++++++++ extensions/git/scripts/python/git_common.py | 81 +++ .../git/scripts/python/initialize_repo.py | 89 +++ .../git/test_git_extension_python_parity.py | 647 ++++++++++++++++++ tests/integrations/test_cli.py | 12 + 6 files changed, 1650 insertions(+) create mode 100644 extensions/git/scripts/python/auto_commit.py create mode 100644 extensions/git/scripts/python/create_new_feature_branch.py create mode 100644 extensions/git/scripts/python/git_common.py create mode 100644 extensions/git/scripts/python/initialize_repo.py create mode 100644 tests/extensions/git/test_git_extension_python_parity.py diff --git a/extensions/git/scripts/python/auto_commit.py b/extensions/git/scripts/python/auto_commit.py new file mode 100644 index 0000000000..ebf23a9454 --- /dev/null +++ b/extensions/git/scripts/python/auto_commit.py @@ -0,0 +1,187 @@ +#!/usr/bin/env python3 +"""Git extension: auto_commit.py + +Automatically commit changes after a Spec Kit command completes. +Python port of ``auto-commit.sh`` / ``auto-commit.ps1``. +Checks per-command config keys in git-config.yml before committing. + +Usage: auto_commit.py + e.g.: auto_commit.py after_specify +""" + +from __future__ import annotations + +import re +import shutil +import subprocess +import sys +from pathlib import Path + + +def _find_project_root(start: Path) -> Path | None: + current = start + while True: + if (current / ".specify").is_dir() or (current / ".git").exists(): + return current + if current.parent == current: + return None + current = current.parent + + +def _value_after_colon(line: str) -> str: + return re.sub(r"^[^:]*:\s*", "", line) + + +def _strip_quotes(value: str) -> str: + """Strip one leading quote and all trailing quotes, mirroring the bash sed.""" + value = re.sub(r"^[\"']", "", value) + return re.sub(r"[\"']*$", "", value) + + +def _parse_auto_commit_config( + config_file: Path, event_name: str +) -> tuple[bool, str]: + """Parse the auto_commit section for this event, mirroring the bash line parser. + + Returns (enabled, commit_msg). Looks for auto_commit..enabled + and .message, with auto_commit.default as fallback. + """ + enabled = False + commit_msg = "" + default_enabled = False + in_auto_commit = False + in_event = False + + try: + content = config_file.read_text(encoding="utf-8") + except (OSError, UnicodeDecodeError): + # Unreadable or non-UTF-8 config is treated like a missing one: + # auto-commit stays disabled instead of crashing with a traceback. + return False, "" + for record in content.splitlines(keepends=True): + if not record.endswith("\n"): + break + line = record[:-1] + if line.startswith("auto_commit:"): + in_auto_commit = True + in_event = False + continue + + # Exit auto_commit section on next top-level key + if in_auto_commit and re.match(r"^[a-z]", line): + break + + if not in_auto_commit: + continue + + if re.match(r"^\s+default:\s", line): + value = re.sub(r"\s", "", _value_after_colon(line)).lower() + if value == "true": + default_enabled = True + + if re.match(rf"^\s+{re.escape(event_name)}:", line): + in_event = True + continue + + if in_event: + # Exit on next sibling key (same indent level as event name) + if re.match(r"^\s{2}[a-z]", line) and not re.match(r"^\s{4}", line): + in_event = False + continue + if re.search(r"\s+enabled:", line): + value = re.sub(r"\s", "", _value_after_colon(line)).lower() + if value == "true": + enabled = True + elif value == "false": + enabled = False + if re.search(r"\s+message:", line): + commit_msg = _strip_quotes(_value_after_colon(line)) + + # If event-specific key not found, use default — but only if the event + # section didn't exist at all (an explicit false must win). + if not enabled and default_enabled: + if not re.search(rf"^\s*{re.escape(event_name)}:", content, re.MULTILINE): + enabled = True + + return enabled, commit_msg + + +def main(argv: list[str]) -> int: + event_name = argv[0] if argv else "" + if not event_name: + print(f"Usage: {Path(sys.argv[0]).name} ", file=sys.stderr) + return 1 + + script_dir = Path(__file__).resolve().parent + repo_root = _find_project_root(script_dir) or Path.cwd() + + if shutil.which("git") is None: + print("[specify] Warning: Git not found; skipped auto-commit", file=sys.stderr) + return 0 + + probe = subprocess.run( + ["git", "rev-parse", "--is-inside-work-tree"], + cwd=repo_root, + capture_output=True, + text=True, + ) + if probe.returncode != 0: + print( + "[specify] Warning: Not a Git repository; skipped auto-commit", + file=sys.stderr, + ) + return 0 + + config_file = repo_root / ".specify" / "extensions" / "git" / "git-config.yml" + if not config_file.is_file(): + # No config file — auto-commit disabled by default + return 0 + + enabled, commit_msg = _parse_auto_commit_config(config_file, event_name) + if not enabled: + return 0 + + # Check if there are changes to commit + def _quiet(*args: str) -> bool: + return ( + subprocess.run( + ["git", *args], cwd=repo_root, capture_output=True, text=True + ).returncode + == 0 + ) + + untracked = subprocess.run( + ["git", "ls-files", "--others", "--exclude-standard"], + cwd=repo_root, + capture_output=True, + text=True, + ).stdout.strip() + if _quiet("diff", "--quiet", "HEAD") and _quiet("diff", "--cached", "--quiet") and not untracked: + print(f"[specify] No changes to commit after {event_name}", file=sys.stderr) + return 0 + + # Derive a human-readable command name from the event + # e.g., after_specify -> specify, before_plan -> plan + command_name = re.sub(r"^(after_|before_)", "", event_name) + phase = "before" if event_name.startswith("before_") else "after" + + if not commit_msg: + commit_msg = f"[Spec Kit] Auto-commit {phase} {command_name}" + + steps = [ + (["git", "add", "."], "git add"), + (["git", "commit", "-q", "-m", commit_msg], "git commit"), + ] + for cmd, label in steps: + result = subprocess.run(cmd, cwd=repo_root, capture_output=True, text=True) + if result.returncode != 0: + output = (result.stdout + result.stderr).strip() + print(f"[specify] Error: {label} failed: {output}", file=sys.stderr) + return 1 + + print(f"[OK] Changes committed {phase} {command_name}", file=sys.stderr) + return 0 + + +if __name__ == "__main__": + raise SystemExit(main(sys.argv[1:])) diff --git a/extensions/git/scripts/python/create_new_feature_branch.py b/extensions/git/scripts/python/create_new_feature_branch.py new file mode 100644 index 0000000000..e4b4135150 --- /dev/null +++ b/extensions/git/scripts/python/create_new_feature_branch.py @@ -0,0 +1,634 @@ +#!/usr/bin/env python3 +"""Git extension: create_new_feature_branch.py + +Creates a git feature branch only. The feature directory and spec file are +created by the core create-new-feature script. Python port of +``create-new-feature-branch.sh`` / ``create-new-feature-branch.ps1``. + +Loads the core Python helpers from the project's installed scripts when +available, falling back to the minimal git helpers next to this script. +""" + +from __future__ import annotations + +import importlib.util +import json +import os +import re +import shutil +import subprocess +import sys +from dataclasses import dataclass, field +from datetime import datetime +from pathlib import Path + +SCRIPT_DIR = Path(__file__).resolve().parent +MAX_BRANCH_LENGTH = 244 # GitHub enforces a 244-byte limit on branch names + +USAGE = ( + "Usage: create_new_feature_branch.py [--json] [--dry-run] " + "[--allow-existing-branch] [--short-name ] [--number N] " + "[--timestamp] " +) + +HELP_TEXT = f"""{USAGE} + +Options: + --json Output in JSON format + --dry-run Compute branch name without creating the branch + --allow-existing-branch Switch to branch if it already exists instead of failing + --short-name Provide a custom short name (2-4 words) for the branch + --number N Specify branch number manually (overrides auto-detection) + --timestamp Use timestamp prefix (YYYYMMDD-HHMMSS) instead of sequential numbering + --help, -h Show this help message + +Environment variables: + GIT_BRANCH_NAME Use this exact branch name, bypassing all prefix/suffix generation + +Configuration: + branch_template Optional git-config.yml template with {{author}}, {{app}}, {{number}}, {{slug}} + branch_prefix Optional shorthand namespace expanded before {{number}}-{{slug}} + +Examples: + create_new_feature_branch.py 'Add user authentication system' --short-name 'user-auth' + create_new_feature_branch.py 'Implement OAuth2 integration for API' --number 5 + create_new_feature_branch.py --timestamp --short-name 'user-auth' 'Add user authentication' + GIT_BRANCH_NAME=my-branch create_new_feature_branch.py 'feature description' +""" + +STOP_WORDS = frozenset( + "i a an the to for of in on at by with from is are was were be been being " + "have has had do does did will would should could can may might must shall " + "this that these those my your our their want need add get set".split() +) + + +def _err(message: str) -> None: + print(message, file=sys.stderr) + + +def _persist_hint(var_name: str, value: str) -> str: + """Shell-appropriate guidance for persisting an env var in the caller's shell.""" + if os.name == "nt": + escaped_value = value.replace("'", "''") + return f"$env:{var_name} = '{escaped_value}'" + escaped_value = re.sub(r"([^\w@%+=:,./-])", r"\\\1", value) + return f"export {var_name}={escaped_value}" + + +@dataclass +class Args: + json_mode: bool = False + dry_run: bool = False + allow_existing: bool = False + short_name: str = "" + branch_number: str = "" + use_timestamp: bool = False + description_parts: list[str] = field(default_factory=list) + + +def parse_args(argv: list[str]) -> Args: + args = Args() + i = 0 + while i < len(argv): + arg = argv[i] + if arg == "--json": + args.json_mode = True + elif arg == "--dry-run": + args.dry_run = True + elif arg == "--allow-existing-branch": + args.allow_existing = True + elif arg == "--short-name": + if i + 1 >= len(argv) or argv[i + 1].startswith("--"): + _err("Error: --short-name requires a value") + raise SystemExit(1) + i += 1 + args.short_name = argv[i] + elif arg == "--number": + if i + 1 >= len(argv) or argv[i + 1].startswith("--"): + _err("Error: --number requires a value") + raise SystemExit(1) + i += 1 + args.branch_number = argv[i] + if not re.fullmatch(r"[0-9]+", args.branch_number): + _err("Error: --number must be a non-negative integer") + raise SystemExit(1) + elif arg == "--timestamp": + args.use_timestamp = True + elif arg in ("--help", "-h"): + print(HELP_TEXT) + raise SystemExit(0) + else: + args.description_parts.append(arg) + i += 1 + return args + + +# ── Core helpers loading ───────────────────────────────────────────────────── + + +def _find_project_root(start: Path) -> Path | None: + current = start + while True: + if (current / ".specify").is_dir() or (current / ".git").exists(): + return current + if current.parent == current: + return None + current = current.parent + + +def _load_core_common(project_root: Path | None): + """Load the core common.py from the project's installed scripts. + + Search locations in priority order, mirroring the bash script: + 1. .specify/scripts/python/common.py (installed project) + 2. scripts/python/common.py (source checkout fallback) + Returns the loaded module or None. + """ + if project_root is None: + return None + for relative in (".specify/scripts/python/common.py", "scripts/python/common.py"): + candidate = project_root / relative + if candidate.is_file(): + spec = importlib.util.spec_from_file_location("speckit_core_common", candidate) + if spec is None or spec.loader is None: + continue + module = importlib.util.module_from_spec(spec) + sys.modules[spec.name] = module + spec.loader.exec_module(module) + return module + return None + + +def _local_has_git(repo_root: Path) -> bool: + git_marker = repo_root / ".git" + if not (git_marker.is_dir() or git_marker.is_file()): + return False + if shutil.which("git") is None: + return False + return ( + subprocess.run( + ["git", "-C", str(repo_root), "rev-parse", "--is-inside-work-tree"], + capture_output=True, + text=True, + ).returncode + == 0 + ) + + +# ── Numbering ──────────────────────────────────────────────────────────────── + + +def get_highest_from_specs(specs_dir: Path) -> int: + highest = 0 + if specs_dir.is_dir(): + for entry in specs_dir.iterdir(): + if not entry.is_dir(): + continue + name = entry.name + # Match sequential prefixes (>=3 digits), but skip timestamp dirs. + if re.match(r"^[0-9]{3,}-", name) and not re.match( + r"^[0-9]{8}-[0-9]{6}-", name + ): + number = int(re.match(r"^[0-9]+", name).group(0)) + highest = max(highest, number) + return highest + + +def _extract_highest_number(names: list[str], scope_prefix: str) -> int: + """Extract the highest sequential feature number from a list of ref names.""" + highest = 0 + for name in names: + if not name: + continue + if scope_prefix: + if not name.startswith(scope_prefix): + continue + name = name[len(scope_prefix) :] + name = name.rsplit("/", 1)[-1] + if ( + re.match(r"^[0-9]{3,}-", name) + and not re.match(r"^[0-9]{8}-[0-9]{6}-", name) + and not re.match(r"^[0-9]{7}-[0-9]{6}-", name) + and not re.fullmatch(r"[0-9]{7,8}-[0-9]{6}", name) + ): + match = re.match(r"^([0-9]{3,})-", name) + number = int(match.group(1)) if match else 0 + highest = max(highest, number) + return highest + + +def _git_lines(repo_root: Path, *args: str, env_extra: dict | None = None) -> list[str]: + if shutil.which("git") is None: + return [] + env = {**os.environ, **(env_extra or {})} + result = subprocess.run( + ["git", *args], cwd=repo_root, capture_output=True, text=True, env=env + ) + if result.returncode != 0: + return [] + return result.stdout.splitlines() + + +def get_highest_from_branches(repo_root: Path, scope_prefix: str) -> int: + names = [] + for line in _git_lines(repo_root, "branch", "-a"): + line = re.sub(r"^[+*]\s+", "", line) + line = line.lstrip() + line = re.sub(r"^remotes/[^/]*/", "", line) + names.append(line) + return _extract_highest_number(names, scope_prefix) + + +def get_highest_from_remote_refs(repo_root: Path, scope_prefix: str) -> int: + """Highest number from remote branches without fetching (side-effect-free).""" + highest = 0 + for remote in _git_lines(repo_root, "remote"): + refs = _git_lines( + repo_root, + "ls-remote", + "--heads", + remote, + env_extra={"GIT_TERMINAL_PROMPT": "0"}, + ) + names = [re.sub(r".*refs/heads/", "", ref) for ref in refs] + highest = max(highest, _extract_highest_number(names, scope_prefix)) + return highest + + +def check_existing_branches( + repo_root: Path, specs_dir: Path, skip_fetch: bool, scope_prefix: str +) -> int: + """Check existing branches and return the next available number.""" + if skip_fetch: + highest_branch = max( + get_highest_from_remote_refs(repo_root, scope_prefix), + get_highest_from_branches(repo_root, scope_prefix), + ) + else: + subprocess.run( + ["git", "fetch", "--all", "--prune"], + cwd=repo_root, + capture_output=True, + text=True, + ) + highest_branch = get_highest_from_branches(repo_root, scope_prefix) + + return max(highest_branch, get_highest_from_specs(specs_dir)) + 1 + + +# ── Branch naming ──────────────────────────────────────────────────────────── + + +def clean_branch_name(name: str) -> str: + name = re.sub(r"[^a-z0-9]", "-", name.lower()) + name = re.sub(r"-+", "-", name) + return name.strip("-") + + +def generate_branch_name(description: str) -> str: + """Generate a branch suffix from the description with stop word filtering.""" + clean_name = re.sub(r"[^a-z0-9]", " ", description.lower()) + + meaningful_words = [] + for word in clean_name.split(): + if word in STOP_WORDS: + continue + if len(word) >= 3: + meaningful_words.append(word) + # Keep short words only when they appear uppercased in the original + # description (acronyms like "API" or "DB"). + elif re.search(rf"\b{re.escape(word.upper())}\b", description): + meaningful_words.append(word) + + if meaningful_words: + max_words = 4 if len(meaningful_words) == 4 else 3 + return "-".join(meaningful_words[:max_words]) + + cleaned = clean_branch_name(description) + return "-".join([part for part in cleaned.split("-") if part][:3]) + + +def branch_token(value: str, fallback: str) -> str: + cleaned = clean_branch_name(value) + return cleaned if cleaned else fallback + + +def get_author_token(repo_root: Path) -> str: + author = "" + if shutil.which("git") is not None: + lines = _git_lines(repo_root, "config", "user.name") + author = lines[0] if lines else "" + if not author: + lines = _git_lines(repo_root, "config", "user.email") + email = lines[0] if lines else "" + author = email.split("@")[0] + if not author: + author = os.environ.get("USER") or os.environ.get("USERNAME") or "unknown" + return branch_token(author, "unknown") + + +def get_app_token(repo_root: Path) -> str: + return branch_token(repo_root.name, "app") + + +def read_git_config_value(config_file: Path, key: str) -> str: + if not config_file.is_file(): + return "" + try: + lines = config_file.read_text(encoding="utf-8").splitlines() + except (OSError, UnicodeDecodeError): + return "" + for line in lines: + if re.match(rf"^\s*{re.escape(key)}:", line): + value = re.sub(rf"^\s*{re.escape(key)}:\s*", "", line) + value = re.sub(r"\s+#.*$", "", value) + value = value.strip() + value = re.sub(r'^"|"$', "", value) + value = re.sub(r"^'|'$", "", value) + return value + return "" + + +def resolve_branch_template(config_file: Path) -> str: + template = read_git_config_value(config_file, "branch_template") + if template: + return template + + prefix = read_git_config_value(config_file, "branch_prefix") + if not prefix: + return "" + if prefix.endswith("/"): + return f"{prefix}{{number}}-{{slug}}" + return f"{prefix}/{{number}}-{{slug}}" + + +def validate_branch_template(template: str) -> None: + if not template: + return + if "{number}" not in template: + _err( + "Error: branch_template must include the {number} token so generated " + "branches remain valid feature branches." + ) + raise SystemExit(1) + slug_index = template.find("{slug}") + if slug_index != -1 and "{number}" in template[slug_index:]: + _err( + "Error: branch_template must not place {slug} before {number}; " + "use {slug} only in the final feature segment." + ) + raise SystemExit(1) + feature_segment = template.rsplit("/", 1)[-1] + if not feature_segment.startswith("{number}-"): + _err( + "Error: branch_template must put {number}- at the start of the final " + "path segment so generated branches remain valid feature branches." + ) + raise SystemExit(1) + + +def render_branch_template( + template: str, feature_num: str, branch_suffix: str, author_token: str, app_token: str +) -> str: + rendered = template + rendered = rendered.replace("{author}", author_token) + rendered = rendered.replace("{app}", app_token) + rendered = rendered.replace("{number}", feature_num) + rendered = rendered.replace("{slug}", branch_suffix) + return rendered + + +def extract_feature_num_from_branch(branch_name: str) -> str: + feature_segment = branch_name.rsplit("/", 1)[-1] + match = re.match(r"^[0-9]{8}-[0-9]{6}-", feature_segment) + if match: + return match.group(0).rstrip("-") + match = re.match(r"^[0-9]+-", feature_segment) + if match: + return match.group(0).rstrip("-") + return branch_name + + +def _byte_length(value: str) -> int: + return len(value.encode("utf-8")) + + +# ── Main ───────────────────────────────────────────────────────────────────── + + +def main(argv: list[str]) -> int: + args = parse_args(argv) + + feature_description = " ".join(args.description_parts) + if not feature_description: + _err(USAGE) + return 1 + feature_description = feature_description.strip() + if not feature_description: + _err("Error: Feature description cannot be empty or contain only whitespace") + return 1 + + project_root = _find_project_root(SCRIPT_DIR) + core = _load_core_common(project_root) + + # SPECIFY_INIT_DIR is resolved (and validated) by the core resolver. If the + # core helpers were not found, refuse rather than silently falling back to + # the wrong root. + if os.environ.get("SPECIFY_INIT_DIR") and ( + core is None or not hasattr(core, "resolve_specify_init_dir") + ): + _err( + "Error: SPECIFY_INIT_DIR requires updated Spec Kit core scripts " + "(common.py with resolve_specify_init_dir), which were not found." + ) + return 1 + + if core is not None and hasattr(core, "get_repo_root"): + # Pass script path so cwd-outside-repo callers land on the same + # fallback the bash twin does. Older cores don't accept the kwarg — + # fall back to the no-arg call for compatibility. + try: + repo_root = core.get_repo_root(script_file=Path(__file__)) + except TypeError: + repo_root = core.get_repo_root() + else: + toplevel = _git_lines(Path.cwd(), "rev-parse", "--show-toplevel") + if toplevel: + repo_root = Path(toplevel[0]) + elif project_root is not None: + repo_root = project_root + else: + _err("Error: Could not determine repository root.") + return 1 + repo_root = Path(repo_root) + + has_git_repo = _local_has_git(repo_root) + + specs_dir = repo_root / "specs" + config_file = repo_root / ".specify" / "extensions" / "git" / "git-config.yml" + + author_token = get_author_token(repo_root) + app_token = get_app_token(repo_root) + branch_template = resolve_branch_template(config_file) + validate_branch_template(branch_template) + + def build_branch_name(feature_num: str, branch_suffix: str) -> str: + if branch_template: + return render_branch_template( + branch_template, feature_num, branch_suffix, author_token, app_token + ) + return f"{feature_num}-{branch_suffix}" + + branch_number = args.branch_number + + # Check for GIT_BRANCH_NAME env var override (exact name, no prefix/suffix) + env_branch_name = os.environ.get("GIT_BRANCH_NAME", "") + if env_branch_name: + branch_name = env_branch_name + feature_num = extract_feature_num_from_branch(branch_name) + branch_suffix = branch_name + else: + if args.short_name: + branch_suffix = clean_branch_name(args.short_name) + else: + branch_suffix = generate_branch_name(feature_description) + + if args.use_timestamp and branch_number: + _err("[specify] Warning: --number is ignored when --timestamp is used") + branch_number = "" + + if args.use_timestamp: + feature_num = datetime.now().strftime("%Y%m%d-%H%M%S") + branch_name = build_branch_name(feature_num, branch_suffix) + else: + scope_prefix = "" + if branch_template: + prefix_template = branch_template.split("{number}")[0] + scope_prefix = render_branch_template( + prefix_template, "", branch_suffix, author_token, app_token + ) + if not branch_number: + if args.dry_run and has_git_repo: + branch_number = check_existing_branches( + repo_root, specs_dir, True, scope_prefix + ) + elif args.dry_run: + branch_number = get_highest_from_specs(specs_dir) + 1 + elif has_git_repo: + branch_number = check_existing_branches( + repo_root, specs_dir, False, scope_prefix + ) + else: + branch_number = get_highest_from_specs(specs_dir) + 1 + + feature_num = f"{int(branch_number):03d}" + branch_name = build_branch_name(feature_num, branch_suffix) + + branch_byte_len = _byte_length(branch_name) + if env_branch_name and branch_byte_len > MAX_BRANCH_LENGTH: + _err( + "Error: GIT_BRANCH_NAME must be 244 bytes or fewer in UTF-8. " + f"Provided value is {branch_byte_len} bytes." + ) + return 1 + if branch_byte_len > MAX_BRANCH_LENGTH: + original_branch_name = branch_name + truncated_suffix = branch_suffix + while _byte_length(branch_name) > MAX_BRANCH_LENGTH and truncated_suffix: + truncated_suffix = truncated_suffix[:-1] + truncated_suffix = truncated_suffix.rstrip("-") + branch_name = build_branch_name(feature_num, truncated_suffix) + if _byte_length(branch_name) > MAX_BRANCH_LENGTH: + _err("Error: Branch template prefix exceeds GitHub's 244-byte branch name limit.") + return 1 + + _err("[specify] Warning: Branch name exceeded GitHub's 244-byte limit") + _err( + f"[specify] Original: {original_branch_name} " + f"({_byte_length(original_branch_name)} bytes)" + ) + _err(f"[specify] Truncated to: {branch_name} ({_byte_length(branch_name)} bytes)") + + if not args.dry_run: + if has_git_repo: + create = subprocess.run( + ["git", "checkout", "-q", "-b", branch_name], + cwd=repo_root, + capture_output=True, + text=True, + ) + if create.returncode != 0: + current_branch_lines = _git_lines( + repo_root, "rev-parse", "--abbrev-ref", "HEAD" + ) + current_branch = current_branch_lines[0] if current_branch_lines else "" + branch_exists = bool( + _git_lines(repo_root, "branch", "--list", branch_name) + ) + if branch_exists: + if args.allow_existing: + if current_branch != branch_name: + switch = subprocess.run( + ["git", "checkout", "-q", branch_name], + cwd=repo_root, + capture_output=True, + text=True, + ) + if switch.returncode != 0: + _err( + f"Error: Failed to switch to existing branch '{branch_name}'. " + "Please resolve any local changes or conflicts and try again." + ) + if switch.stderr.strip(): + _err(switch.stderr.strip()) + return 1 + elif args.use_timestamp: + _err( + f"Error: Branch '{branch_name}' already exists. Rerun to get " + "a new timestamp or use a different --short-name." + ) + return 1 + else: + _err( + f"Error: Branch '{branch_name}' already exists. Please use a " + "different feature name or specify a different number with --number." + ) + return 1 + else: + _err(f"Error: Failed to create git branch '{branch_name}'.") + if create.stderr.strip(): + _err(create.stderr.strip()) + else: + _err("Please check your git configuration and try again.") + return 1 + else: + _err( + "[specify] Warning: Git repository not detected; skipped branch " + f"creation for {branch_name}" + ) + + _err(f"# To persist: {_persist_hint('SPECIFY_FEATURE', branch_name)}") + + if args.json_mode: + payload: dict[str, object] = { + "BRANCH_NAME": branch_name, + "FEATURE_NUM": feature_num, + } + if args.dry_run: + payload["DRY_RUN"] = True + print(json.dumps(payload, ensure_ascii=False, separators=(",", ":"))) + else: + print(f"BRANCH_NAME: {branch_name}") + print(f"FEATURE_NUM: {feature_num}") + if not args.dry_run: + print( + "# To persist in your shell: " + f"{_persist_hint('SPECIFY_FEATURE', branch_name)}" + ) + + return 0 + + +if __name__ == "__main__": + raise SystemExit(main(sys.argv[1:])) diff --git a/extensions/git/scripts/python/git_common.py b/extensions/git/scripts/python/git_common.py new file mode 100644 index 0000000000..9b23fbaa06 --- /dev/null +++ b/extensions/git/scripts/python/git_common.py @@ -0,0 +1,81 @@ +#!/usr/bin/env python3 +"""Git-specific common helpers for the git extension. + +Python port of ``git-common.sh`` / ``git-common.ps1`` — contains only +git-specific branch validation and detection logic. +""" + +from __future__ import annotations + +import re +import shutil +import subprocess +import sys +from pathlib import Path + + +def has_git(repo_root: Path | None = None) -> bool: + """Check if we have git available at the repo root.""" + root = Path(repo_root) if repo_root is not None else Path.cwd() + git_marker = root / ".git" + if not (git_marker.is_dir() or git_marker.is_file()): + return False + if shutil.which("git") is None: + return False + result = subprocess.run( + ["git", "-C", str(root), "rev-parse", "--is-inside-work-tree"], + capture_output=True, + text=True, + ) + return result.returncode == 0 + + +def effective_branch_name(raw: str) -> str: + """Strip a single optional path segment (e.g. gitflow "feat/004-name" -> "004-name"). + + Only when the full name is exactly two slash-free segments; otherwise + returns the raw name. + """ + match = re.fullmatch(r"([^/]+)/([^/]+)", raw) + if match: + return match.group(2) + return raw + + +def check_feature_branch(raw: str, has_git_repo: bool) -> bool: + """Validate that a branch name matches the expected feature branch pattern. + + Accepts sequential (###-* with >=3 digits) or timestamp (YYYYMMDD-HHMMSS-*) + formats, either at the start of the branch or after path-style namespace + prefixes. Logic aligned with the bash/PowerShell twins. + """ + if not has_git_repo: + print( + "[specify] Warning: Git repository not detected; skipped branch validation", + file=sys.stderr, + ) + return True + + branch = effective_branch_name(raw) + feature_segment = branch.rsplit("/", 1)[-1] + + # Accept sequential prefix (3+ digits) but exclude malformed timestamps: + # 7-or-8 digit date + 6-digit time with no trailing slug. + is_sequential = bool( + re.match(r"^[0-9]{3,}-", feature_segment) + and not re.match(r"^[0-9]{7}-[0-9]{6}-", feature_segment) + and not re.fullmatch(r"[0-9]{7,8}-[0-9]{6}", feature_segment) + ) + is_timestamp = bool(re.match(r"^[0-9]{8}-[0-9]{6}-", feature_segment)) + + if not is_sequential and not is_timestamp: + print(f"ERROR: Not on a feature branch. Current branch: {raw}", file=sys.stderr) + print( + "Feature branches should be named like: 001-feature-name, " + "1234-feature-name, 20260319-143022-feature-name, or " + "/001-feature-name", + file=sys.stderr, + ) + return False + + return True diff --git a/extensions/git/scripts/python/initialize_repo.py b/extensions/git/scripts/python/initialize_repo.py new file mode 100644 index 0000000000..eaf604c673 --- /dev/null +++ b/extensions/git/scripts/python/initialize_repo.py @@ -0,0 +1,89 @@ +#!/usr/bin/env python3 +"""Git extension: initialize_repo.py + +Initialize a Git repository with an initial commit. +Python port of ``initialize-repo.sh`` / ``initialize-repo.ps1``. +Customizable — replace this script to add .gitignore templates, +default branch config, git-flow, LFS, signing, etc. +""" + +from __future__ import annotations + +import re +import shutil +import subprocess +import sys +from pathlib import Path + + +def _find_project_root(start: Path) -> Path | None: + current = start + while True: + if (current / ".specify").is_dir() or (current / ".git").exists(): + return current + if current.parent == current: + return None + current = current.parent + + +def _read_commit_message(repo_root: Path) -> str: + """Read init_commit_message from git-config.yml, mirroring the bash sed pipeline.""" + default = "[Spec Kit] Initial commit" + config_file = repo_root / ".specify" / "extensions" / "git" / "git-config.yml" + if not config_file.is_file(): + return default + try: + lines = config_file.read_text(encoding="utf-8").splitlines() + except (OSError, UnicodeDecodeError): + return default + for line in lines: + if line.startswith("init_commit_message:"): + value = re.sub(r"^init_commit_message:\s*", "", line) + value = re.sub(r"^[\"']", "", value) + value = re.sub(r"[\"']*$", "", value) + if value: + return value + return default + + +def main() -> int: + script_dir = Path(__file__).resolve().parent + repo_root = _find_project_root(script_dir) or Path.cwd() + + commit_msg = _read_commit_message(repo_root) + + if shutil.which("git") is None: + print( + "[specify] Warning: Git not found; skipped repository initialization", + file=sys.stderr, + ) + return 0 + + probe = subprocess.run( + ["git", "rev-parse", "--is-inside-work-tree"], + cwd=repo_root, + capture_output=True, + text=True, + ) + if probe.returncode == 0: + print("[specify] Git repository already initialized; skipping", file=sys.stderr) + return 0 + + steps = [ + (["git", "init", "-q"], "git init"), + (["git", "add", "."], "git add"), + (["git", "commit", "--allow-empty", "-q", "-m", commit_msg], "git commit"), + ] + for cmd, label in steps: + result = subprocess.run(cmd, cwd=repo_root, capture_output=True, text=True) + if result.returncode != 0: + output = (result.stdout + result.stderr).strip() + print(f"[specify] Error: {label} failed: {output}", file=sys.stderr) + return 1 + + print("[OK] Git repository initialized", file=sys.stderr) + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/tests/extensions/git/test_git_extension_python_parity.py b/tests/extensions/git/test_git_extension_python_parity.py new file mode 100644 index 0000000000..894571fcb3 --- /dev/null +++ b/tests/extensions/git/test_git_extension_python_parity.py @@ -0,0 +1,647 @@ +""" +Parity tests for the Python port of the git extension scripts (extensions/git/scripts/python/). + +Each test runs the bash script and its Python twin in identical twin projects +and asserts matching output, exit codes, and resulting git state. +""" + +import json +import os +import re +import runpy +import shutil +import subprocess +import sys +from pathlib import Path + +import pytest + +from tests.conftest import requires_bash +from tests.extensions.git.test_git_extension import ( + _GIT_ENV, + _init_git, + _run_bash, + _setup_project, + _write_config, +) + +PROJECT_ROOT = Path(__file__).resolve().parent.parent.parent.parent +EXT_PY = PROJECT_ROOT / "extensions" / "git" / "scripts" / "python" +CORE_COMMON_PY = PROJECT_ROOT / "scripts" / "python" / "common.py" + +PY_SCRIPTS = { + "create-new-feature-branch": "create_new_feature_branch.py", + "initialize-repo": "initialize_repo.py", + "auto-commit": "auto_commit.py", +} + + +def _setup_py_project(tmp_path: Path, *, git: bool = True) -> Path: + """Twin of _setup_project that also installs the Python scripts.""" + project = _setup_project(tmp_path, git=git) + + py_core = project / ".specify" / "scripts" / "python" + py_core.mkdir(parents=True, exist_ok=True) + shutil.copy(CORE_COMMON_PY, py_core / "common.py") + + ext_py = project / ".specify" / "extensions" / "git" / "scripts" / "python" + ext_py.mkdir(parents=True, exist_ok=True) + for f in EXT_PY.iterdir(): + if f.suffix == ".py": + shutil.copy(f, ext_py / f.name) + return project + + +def _run_py( + script_name: str, + cwd: Path, + *args: str, + env_extra: dict | None = None, + run_cwd: Path | None = None, +) -> subprocess.CompletedProcess: + """Run an extension Python script. + + ``run_cwd`` overrides the working directory while the script path is + still resolved against ``cwd``, for tests that invoke a project's script + from outside that project. + """ + script = ( + cwd / ".specify" / "extensions" / "git" / "scripts" / "python" / PY_SCRIPTS[script_name] + ) + env = {**os.environ, **_GIT_ENV, **(env_extra or {})} + return subprocess.run( + [sys.executable, str(script), *args], + cwd=run_cwd or cwd, + capture_output=True, + text=True, + env=env, + ) + + +def _twin_projects(tmp_path: Path, *, git: bool = True) -> tuple[Path, Path]: + """Two identically named projects so {app} tokens match.""" + bash_proj = _setup_py_project(tmp_path / "bash" / "proj", git=git) + py_proj = _setup_py_project(tmp_path / "py" / "proj", git=git) + return bash_proj, py_proj + + +def _assert_parity( + bash_result: subprocess.CompletedProcess, + py_result: subprocess.CompletedProcess, + *, + stdout: bool = True, + stderr: bool = True, +) -> None: + assert py_result.returncode == bash_result.returncode, ( + f"exit codes diverge: bash={bash_result.returncode} py={py_result.returncode}\n" + f"bash stderr: {bash_result.stderr}\npy stderr: {py_result.stderr}" + ) + if stdout: + assert py_result.stdout == bash_result.stdout + if stderr: + py_stderr = py_result.stderr + bash_stderr = bash_result.stderr + if os.name == "nt": + py_stderr = _without_persist_hint(py_stderr) + bash_stderr = _without_persist_hint(bash_stderr) + assert py_stderr == bash_stderr + + +def _without_persist_hint(stderr: str) -> str: + return "".join( + line + for line in stderr.splitlines(keepends=True) + if not line.startswith("# To persist: ") + ) + + +@requires_bash +class TestCreateFeatureBranchParity: + def test_sequential_branch_json(self, tmp_path: Path): + bash_proj, py_proj = _twin_projects(tmp_path) + b = _run_bash("create-new-feature-branch.sh", bash_proj, "--json", "Add user authentication") + p = _run_py("create-new-feature-branch", py_proj, "--json", "Add user authentication") + _assert_parity(b, p) + data = json.loads(p.stdout) + assert data == {"BRANCH_NAME": "001-user-authentication", "FEATURE_NUM": "001"} + branch = subprocess.run( + ["git", "rev-parse", "--abbrev-ref", "HEAD"], + cwd=py_proj, + capture_output=True, + text=True, + ).stdout.strip() + assert branch == "001-user-authentication" + + def test_slug_generation_stop_words_and_acronyms(self, tmp_path: Path): + bash_proj, py_proj = _twin_projects(tmp_path) + description = "I want to add DB caching for the API layer" + b = _run_bash("create-new-feature-branch.sh", bash_proj, "--json", "--dry-run", description) + p = _run_py("create-new-feature-branch", py_proj, "--json", "--dry-run", description) + _assert_parity(b, p) + + def test_short_name_cleaning(self, tmp_path: Path): + bash_proj, py_proj = _twin_projects(tmp_path) + # Single separator runs only: the bash twin's collapse step + # (sed 's/-\+/-/g') is a GNU-ism that BSD sed treats literally. + b = _run_bash( + "create-new-feature-branch.sh", bash_proj, + "--json", "--dry-run", "--short-name", "User_Auth!", "desc", + ) + p = _run_py( + "create-new-feature-branch", py_proj, + "--json", "--dry-run", "--short-name", "User_Auth!", "desc", + ) + _assert_parity(b, p) + assert json.loads(p.stdout)["BRANCH_NAME"] == "001-user-auth" + + def test_numbering_from_specs_and_branches(self, tmp_path: Path): + bash_proj, py_proj = _twin_projects(tmp_path) + for proj in (bash_proj, py_proj): + (proj / "specs" / "007-existing").mkdir(parents=True) + (proj / "specs" / "20260101-120000-timestamped").mkdir(parents=True) + subprocess.run(["git", "branch", "012-in-branch"], cwd=proj, check=True) + b = _run_bash("create-new-feature-branch.sh", bash_proj, "--json", "--dry-run", "next feature") + p = _run_py("create-new-feature-branch", py_proj, "--json", "--dry-run", "next feature") + _assert_parity(b, p) + assert json.loads(p.stdout)["FEATURE_NUM"] == "013" + + def test_explicit_number(self, tmp_path: Path): + bash_proj, py_proj = _twin_projects(tmp_path) + b = _run_bash("create-new-feature-branch.sh", bash_proj, "--json", "--number", "42", "some feature") + p = _run_py("create-new-feature-branch", py_proj, "--json", "--number", "42", "some feature") + _assert_parity(b, p) + assert json.loads(p.stdout)["FEATURE_NUM"] == "042" + + def test_timestamp_mode_format(self, tmp_path: Path): + _, py_proj = _twin_projects(tmp_path) + p = _run_py( + "create-new-feature-branch", py_proj, + "--json", "--timestamp", "--short-name", "user-auth", "desc", + ) + assert p.returncode == 0 + data = json.loads(p.stdout) + assert re.fullmatch(r"[0-9]{8}-[0-9]{6}", data["FEATURE_NUM"]) + assert data["BRANCH_NAME"] == f"{data['FEATURE_NUM']}-user-auth" + + def test_timestamp_with_number_warns(self, tmp_path: Path): + bash_proj, py_proj = _twin_projects(tmp_path) + b = _run_bash( + "create-new-feature-branch.sh", bash_proj, + "--json", "--dry-run", "--timestamp", "--number", "5", "desc word", + ) + p = _run_py( + "create-new-feature-branch", py_proj, + "--json", "--dry-run", "--timestamp", "--number", "5", "desc word", + ) + assert p.returncode == b.returncode == 0 + warning = "[specify] Warning: --number is ignored when --timestamp is used" + assert warning in b.stderr + assert warning in p.stderr + + def test_branch_template_author_app(self, tmp_path: Path): + bash_proj, py_proj = _twin_projects(tmp_path) + for proj in (bash_proj, py_proj): + _write_config(proj, 'branch_template: "{author}/{app}/{number}-{slug}"\n') + b = _run_bash("create-new-feature-branch.sh", bash_proj, "--json", "--dry-run", "new payment flow") + p = _run_py("create-new-feature-branch", py_proj, "--json", "--dry-run", "new payment flow") + _assert_parity(b, p) + assert json.loads(p.stdout)["BRANCH_NAME"] == "test-user/proj/001-new-payment-flow" + + def test_branch_prefix_shorthand(self, tmp_path: Path): + bash_proj, py_proj = _twin_projects(tmp_path) + for proj in (bash_proj, py_proj): + _write_config(proj, "branch_prefix: feat\n") + b = _run_bash("create-new-feature-branch.sh", bash_proj, "--json", "--dry-run", "new payment flow") + p = _run_py("create-new-feature-branch", py_proj, "--json", "--dry-run", "new payment flow") + _assert_parity(b, p) + assert json.loads(p.stdout)["BRANCH_NAME"] == "feat/001-new-payment-flow" + + def test_template_scopes_existing_branch_numbers(self, tmp_path: Path): + bash_proj, py_proj = _twin_projects(tmp_path) + for proj in (bash_proj, py_proj): + _write_config(proj, 'branch_template: "{author}/{number}-{slug}"\n') + subprocess.run(["git", "branch", "test-user/008-scoped"], cwd=proj, check=True) + subprocess.run(["git", "branch", "other-user/030-unscoped"], cwd=proj, check=True) + b = _run_bash("create-new-feature-branch.sh", bash_proj, "--json", "--dry-run", "next thing") + p = _run_py("create-new-feature-branch", py_proj, "--json", "--dry-run", "next thing") + _assert_parity(b, p) + assert json.loads(p.stdout)["FEATURE_NUM"] == "009" + + @pytest.mark.parametrize( + "template", + [ + 'branch_template: "feat/{slug}"\n', + 'branch_template: "{slug}/{number}-x"\n', + 'branch_template: "{number}/{slug}-x"\n', + ], + ) + def test_invalid_template_rejected(self, tmp_path: Path, template: str): + bash_proj, py_proj = _twin_projects(tmp_path) + for proj in (bash_proj, py_proj): + _write_config(proj, template) + b = _run_bash("create-new-feature-branch.sh", bash_proj, "--json", "--dry-run", "desc word") + p = _run_py("create-new-feature-branch", py_proj, "--json", "--dry-run", "desc word") + assert b.returncode == p.returncode == 1 + assert p.stderr.strip() == b.stderr.strip() + + def test_git_branch_name_override(self, tmp_path: Path): + bash_proj, py_proj = _twin_projects(tmp_path) + env = {"GIT_BRANCH_NAME": "team/042-exact-name"} + b = _run_bash("create-new-feature-branch.sh", bash_proj, "--json", "desc word", env_extra=env) + p = _run_py("create-new-feature-branch", py_proj, "--json", "desc word", env_extra=env) + _assert_parity(b, p) + assert json.loads(p.stdout) == {"BRANCH_NAME": "team/042-exact-name", "FEATURE_NUM": "042"} + + def test_git_branch_name_override_persist_hint_matches_bash( + self, tmp_path: Path + ): + bash_proj, py_proj = _twin_projects(tmp_path) + env = {"GIT_BRANCH_NAME": "feature/$value's-\"quoted\""} + b = _run_bash( + "create-new-feature-branch.sh", + bash_proj, + "--json", + "desc word", + env_extra=env, + ) + p = _run_py( + "create-new-feature-branch", + py_proj, + "--json", + "desc word", + env_extra=env, + ) + _assert_parity(b, p) + + def test_long_branch_name_truncated_to_244_bytes(self, tmp_path: Path): + bash_proj, py_proj = _twin_projects(tmp_path) + long_name = "-".join(["word"] * 60) + b = _run_bash( + "create-new-feature-branch.sh", bash_proj, + "--json", "--dry-run", "--short-name", long_name, "desc", + ) + p = _run_py( + "create-new-feature-branch", py_proj, + "--json", "--dry-run", "--short-name", long_name, "desc", + ) + _assert_parity(b, p) + assert len(json.loads(p.stdout)["BRANCH_NAME"].encode()) <= 244 + + def test_existing_branch_errors_without_flag(self, tmp_path: Path): + bash_proj, py_proj = _twin_projects(tmp_path) + for proj in (bash_proj, py_proj): + subprocess.run(["git", "branch", "001-user-auth"], cwd=proj, check=True) + args = ("--json", "--number", "1", "--short-name", "user-auth", "desc") + b = _run_bash("create-new-feature-branch.sh", bash_proj, *args) + p = _run_py("create-new-feature-branch", py_proj, *args) + assert b.returncode == p.returncode == 1 + assert p.stderr.strip() == b.stderr.strip() + + def test_existing_branch_switches_with_allow_flag(self, tmp_path: Path): + bash_proj, py_proj = _twin_projects(tmp_path) + for proj in (bash_proj, py_proj): + subprocess.run(["git", "branch", "001-user-auth"], cwd=proj, check=True) + args = ("--json", "--number", "1", "--short-name", "user-auth", "--allow-existing-branch", "desc") + b = _run_bash("create-new-feature-branch.sh", bash_proj, *args) + p = _run_py("create-new-feature-branch", py_proj, *args) + _assert_parity(b, p) + for proj in (bash_proj, py_proj): + branch = subprocess.run( + ["git", "rev-parse", "--abbrev-ref", "HEAD"], + cwd=proj, + capture_output=True, + text=True, + ).stdout.strip() + assert branch == "001-user-auth" + + def test_no_git_graceful_degradation(self, tmp_path: Path): + bash_proj, py_proj = _twin_projects(tmp_path, git=False) + b = _run_bash("create-new-feature-branch.sh", bash_proj, "--json", "offline feature") + p = _run_py("create-new-feature-branch", py_proj, "--json", "offline feature") + _assert_parity(b, p) + assert "skipped branch creation" in p.stderr + + def test_missing_git_executable_gracefully_degrades( + self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch + ): + module = runpy.run_path(str(EXT_PY / "create_new_feature_branch.py")) + monkeypatch.setenv("PATH", "") + + assert module["_git_lines"](tmp_path, "status") == [] + + def test_windows_persist_hint_quotes_branch_name( + self, monkeypatch: pytest.MonkeyPatch + ): + module = runpy.run_path(str(EXT_PY / "create_new_feature_branch.py")) + monkeypatch.setattr(module["os"], "name", "nt") + + result = module["_persist_hint"]( + "GIT_BRANCH_NAME", "feature/$value's-\"quoted\"" + ) + + assert ( + result + == "$env:GIT_BRANCH_NAME = 'feature/$value''s-\"quoted\"'" + ) + + def test_shell_specific_persist_hint_can_be_ignored_for_parity(self): + bash_stderr = ( + "[specify] Warning\n" + "# To persist: export SPECIFY_FEATURE=feature/name\n" + ) + windows_stderr = ( + "[specify] Warning\n" + "# To persist: $env:SPECIFY_FEATURE = 'feature/name'\n" + ) + + assert _without_persist_hint(bash_stderr) == _without_persist_hint( + windows_stderr + ) + + def test_assert_parity_ignores_windows_persist_hint( + self, monkeypatch: pytest.MonkeyPatch + ): + monkeypatch.setattr(os, "name", "nt") + bash_result = subprocess.CompletedProcess( + args=[], returncode=0, stdout="", stderr=( + "[specify] Warning\n" + "# To persist: export SPECIFY_FEATURE=feature/name\n" + ) + ) + py_result = subprocess.CompletedProcess( + args=[], returncode=0, stdout="", stderr=( + "[specify] Warning\n" + "# To persist: $env:SPECIFY_FEATURE = 'feature/name'\n" + ) + ) + + _assert_parity(bash_result, py_result) + + def test_empty_description_errors(self, tmp_path: Path): + bash_proj, py_proj = _twin_projects(tmp_path) + b = _run_bash("create-new-feature-branch.sh", bash_proj, "--json", " ") + p = _run_py("create-new-feature-branch", py_proj, "--json", " ") + assert b.returncode == p.returncode == 1 + assert p.stderr.strip() == b.stderr.strip() + assert "cannot be empty or contain only whitespace" in p.stderr + + def test_specify_init_dir_resolves_target_project(self, tmp_path: Path): + # The script is installed under host_proj, so script_file-based + # discovery (and cwd-based discovery, since we run from elsewhere) + # would resolve host_proj, not target_proj. host_proj has no specs + # (next number 001); target_proj already has 007-existing (next + # number 008). Only honoring SPECIFY_INIT_DIR produces 008, so this + # proves the env var -- not script location or cwd -- controls + # resolution. + host_proj = _setup_py_project(tmp_path / "host") + target_proj = _setup_py_project(tmp_path / "target") + (target_proj / "specs" / "007-existing").mkdir(parents=True) + elsewhere = tmp_path / "elsewhere" + elsewhere.mkdir() + p = _run_py( + "create-new-feature-branch", host_proj, + "--json", "--dry-run", "init dir feature", + env_extra={"SPECIFY_INIT_DIR": str(target_proj)}, + run_cwd=elsewhere, + ) + assert p.returncode == 0 + assert json.loads(p.stdout)["FEATURE_NUM"] == "008" + + def test_specify_init_dir_without_core_errors(self, tmp_path: Path): + _, py_proj = _twin_projects(tmp_path) + ( + py_proj / ".specify" / "scripts" / "python" / "common.py" + ).unlink() + p = _run_py( + "create-new-feature-branch", py_proj, + "--json", "desc word", + env_extra={"SPECIFY_INIT_DIR": str(py_proj)}, + ) + assert p.returncode == 1 + assert "SPECIFY_INIT_DIR requires updated Spec Kit core scripts" in p.stderr + + +@requires_bash +class TestInitializeRepoParity: + def test_initializes_repo_with_default_message(self, tmp_path: Path): + bash_proj, py_proj = _twin_projects(tmp_path, git=False) + b = _run_bash("initialize-repo.sh", bash_proj) + p = _run_py("initialize-repo", py_proj) + _assert_parity(b, p) + assert p.stderr.strip() == b.stderr.strip() + for proj in (bash_proj, py_proj): + message = subprocess.run( + ["git", "log", "-1", "--format=%s"], + cwd=proj, + capture_output=True, + text=True, + ).stdout.strip() + assert message == "[Spec Kit] Initial commit" + + def test_custom_commit_message(self, tmp_path: Path): + bash_proj, py_proj = _twin_projects(tmp_path, git=False) + for proj in (bash_proj, py_proj): + _write_config(proj, 'init_commit_message: "Custom initial commit"\n') + b = _run_bash("initialize-repo.sh", bash_proj) + p = _run_py("initialize-repo", py_proj) + _assert_parity(b, p) + for proj in (bash_proj, py_proj): + message = subprocess.run( + ["git", "log", "-1", "--format=%s"], + cwd=proj, + capture_output=True, + text=True, + ).stdout.strip() + assert message == "Custom initial commit" + + def test_skips_existing_repo(self, tmp_path: Path): + bash_proj, py_proj = _twin_projects(tmp_path) + b = _run_bash("initialize-repo.sh", bash_proj) + p = _run_py("initialize-repo", py_proj) + _assert_parity(b, p) + assert p.stderr.strip() == b.stderr.strip() + assert "already initialized" in p.stderr + + +@requires_bash +class TestAutoCommitParity: + def _dirty(self, proj: Path) -> None: + (proj / "change.txt").write_text("dirty\n", encoding="utf-8") + + def _last_message(self, proj: Path) -> str: + return subprocess.run( + ["git", "log", "-1", "--format=%s"], + cwd=proj, + capture_output=True, + text=True, + ).stdout.strip() + + def test_disabled_by_default(self, tmp_path: Path): + bash_proj, py_proj = _twin_projects(tmp_path) + for proj in (bash_proj, py_proj): + _write_config(proj, "auto_commit:\n after_specify:\n enabled: false\n") + self._dirty(proj) + b = _run_bash("auto-commit.sh", bash_proj, "after_specify") + p = _run_py("auto-commit", py_proj, "after_specify") + _assert_parity(b, p) + assert self._last_message(py_proj) == "seed" + + def test_ignores_unterminated_final_config_line(self, tmp_path: Path): + bash_proj, py_proj = _twin_projects(tmp_path) + config = "auto_commit:\n after_specify:\n enabled: true" + for proj in (bash_proj, py_proj): + _write_config(proj, config) + self._dirty(proj) + b = _run_bash("auto-commit.sh", bash_proj, "after_specify") + p = _run_py("auto-commit", py_proj, "after_specify") + _assert_parity(b, p) + assert self._last_message(py_proj) == "seed" + + def test_enabled_per_command_with_custom_message(self, tmp_path: Path): + bash_proj, py_proj = _twin_projects(tmp_path) + config = ( + "auto_commit:\n" + " default: false\n" + " after_specify:\n" + " enabled: true\n" + ' message: "spec done"\n' + ) + for proj in (bash_proj, py_proj): + _write_config(proj, config) + self._dirty(proj) + b = _run_bash("auto-commit.sh", bash_proj, "after_specify") + p = _run_py("auto-commit", py_proj, "after_specify") + _assert_parity(b, p) + assert p.stderr.strip() == b.stderr.strip() + assert self._last_message(bash_proj) == self._last_message(py_proj) == "spec done" + + def test_default_true_applies_to_unlisted_event(self, tmp_path: Path): + bash_proj, py_proj = _twin_projects(tmp_path) + for proj in (bash_proj, py_proj): + _write_config(proj, "auto_commit:\n default: true\n") + self._dirty(proj) + b = _run_bash("auto-commit.sh", bash_proj, "after_plan") + p = _run_py("auto-commit", py_proj, "after_plan") + _assert_parity(b, p) + expected = "[Spec Kit] Auto-commit after plan" + assert self._last_message(bash_proj) == self._last_message(py_proj) == expected + + def test_explicit_false_beats_default_true(self, tmp_path: Path): + bash_proj, py_proj = _twin_projects(tmp_path) + config = "auto_commit:\n default: true\n after_specify:\n enabled: false\n" + for proj in (bash_proj, py_proj): + _write_config(proj, config) + self._dirty(proj) + b = _run_bash("auto-commit.sh", bash_proj, "after_specify") + p = _run_py("auto-commit", py_proj, "after_specify") + _assert_parity(b, p) + assert self._last_message(py_proj) == "seed" + + def test_before_event_message(self, tmp_path: Path): + bash_proj, py_proj = _twin_projects(tmp_path) + for proj in (bash_proj, py_proj): + _write_config(proj, "auto_commit:\n before_plan:\n enabled: true\n") + self._dirty(proj) + b = _run_bash("auto-commit.sh", bash_proj, "before_plan") + p = _run_py("auto-commit", py_proj, "before_plan") + _assert_parity(b, p) + expected = "[Spec Kit] Auto-commit before plan" + assert self._last_message(bash_proj) == self._last_message(py_proj) == expected + + def test_no_changes_skips(self, tmp_path: Path): + bash_proj, py_proj = _twin_projects(tmp_path) + for proj in (bash_proj, py_proj): + _write_config(proj, "auto_commit:\n after_specify:\n enabled: true\n") + subprocess.run(["git", "add", "-A"], cwd=proj, check=True) + subprocess.run( + ["git", "commit", "-q", "-m", "clean"], + cwd=proj, + check=True, + env={**os.environ, **_GIT_ENV}, + ) + b = _run_bash("auto-commit.sh", bash_proj, "after_specify") + p = _run_py("auto-commit", py_proj, "after_specify") + _assert_parity(b, p) + assert p.stderr.strip() == b.stderr.strip() + assert "No changes to commit" in p.stderr + + def test_no_config_file_skips(self, tmp_path: Path): + bash_proj, py_proj = _twin_projects(tmp_path) + for proj in (bash_proj, py_proj): + self._dirty(proj) + b = _run_bash("auto-commit.sh", bash_proj, "after_specify") + p = _run_py("auto-commit", py_proj, "after_specify") + _assert_parity(b, p) + assert self._last_message(py_proj) == "seed" + + @pytest.mark.skipif(os.name != "posix", reason="POSIX file permissions") + def test_unreadable_config_skips_auto_commit(self, tmp_path: Path): + """An unreadable config behaves like a missing one: no traceback, no commit.""" + if os.geteuid() == 0: + pytest.skip("root bypasses file permissions") + proj = _setup_py_project(tmp_path / "proj") + config = _write_config( + proj, "auto_commit:\n after_specify:\n enabled: true\n" + ) + self._dirty(proj) + config.chmod(0o000) + try: + p = _run_py("auto-commit", proj, "after_specify") + finally: + config.chmod(0o644) + assert p.returncode == 0 + assert "Traceback" not in p.stderr + assert self._last_message(proj) == "seed" + + def test_missing_event_argument_errors(self, tmp_path: Path): + bash_proj, py_proj = _twin_projects(tmp_path) + b = _run_bash("auto-commit.sh", bash_proj) + p = _run_py("auto-commit", py_proj) + assert b.returncode == p.returncode == 1 + + def test_not_a_repo_skips(self, tmp_path: Path): + bash_proj, py_proj = _twin_projects(tmp_path, git=False) + b = _run_bash("auto-commit.sh", bash_proj, "after_specify") + p = _run_py("auto-commit", py_proj, "after_specify") + _assert_parity(b, p) + assert "Not a Git repository" in p.stderr + + +class TestGitCommonPython: + """Unit tests for git_common.py (imported directly).""" + + @pytest.fixture() + def git_common(self): + sys.path.insert(0, str(EXT_PY)) + try: + import git_common + + yield git_common + finally: + sys.path.remove(str(EXT_PY)) + sys.modules.pop("git_common", None) + + def test_has_git(self, git_common, tmp_path: Path): + assert git_common.has_git(tmp_path) is False + _init_git(tmp_path) + assert git_common.has_git(tmp_path) is True + + @pytest.mark.parametrize( + ("branch", "expected"), + [ + ("001-feature-name", True), + ("1234-feature-name", True), + ("20260319-143022-feature-name", True), + ("feat/004-name", True), + ("main", False), + ("2026031-143022", False), + ("20260319-143022", False), + ("2026031-143022-slug", False), + ], + ) + def test_check_feature_branch(self, git_common, branch: str, expected: bool): + assert git_common.check_feature_branch(branch, True) is expected + + def test_check_feature_branch_no_git_warns_but_passes(self, git_common, capsys): + assert git_common.check_feature_branch("main", False) is True + assert "skipped branch validation" in capsys.readouterr().err diff --git a/tests/integrations/test_cli.py b/tests/integrations/test_cli.py index 7b6461b4da..377232f234 100644 --- a/tests/integrations/test_cli.py +++ b/tests/integrations/test_cli.py @@ -314,6 +314,18 @@ def test_shared_infra_overwrites_existing_files_with_force(self, tmp_path): assert (scripts_dir / "setup-plan.sh").exists() assert (templates_dir / "plan-template.md").exists() + def test_shared_infra_installs_python_scripts_for_py(self, tmp_path): + from specify_cli import _install_shared_infra + + project = tmp_path / "python-scripts" + project.mkdir() + + _install_shared_infra(project, "py") + + assert ( + project / ".specify" / "scripts" / "python" / "common.py" + ).exists() + def test_shared_infra_removes_stale_managed_script(self, tmp_path): """A managed script the core no longer ships (e.g. the legacy update-agent-context.sh, superseded by the agent-context extension) is From faeb956664dbe7319aceb8c8b25eece11984aeed Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Tue, 14 Jul 2026 10:08:42 -0500 Subject: [PATCH 43/45] Add PatchWarden Evidence Pack extension to community catalog (#3514) Add patchwarden-evidence extension submitted by @jiezeng2004-design to: - extensions/catalog.community.json (alphabetical order) - docs/community/extensions.md community extensions table Closes #3512 Assisted-by: GitHub Copilot (model: claude-sonnet-4.6, autonomous) Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- docs/community/extensions.md | 1 + extensions/catalog.community.json | 42 ++++++++++++++++++++++++++++++- 2 files changed, 42 insertions(+), 1 deletion(-) diff --git a/docs/community/extensions.md b/docs/community/extensions.md index 30f0e5cd37..ebd49f380d 100644 --- a/docs/community/extensions.md +++ b/docs/community/extensions.md @@ -91,6 +91,7 @@ The following community-contributed extensions are available in [`catalog.commun | Optimize | Audit and optimize AI governance for context efficiency — token budgets, rule health, interpretability, compression, coherence, and echo detection | `process` | Read+Write | [spec-kit-optimize](https://github.com/sakitA/spec-kit-optimize) | | Orchestration Task Context Management | Adds subagent work-unit orchestration to generated Spec Kit task files | `process` | Read+Write | [spec-kit-orchestration-task-context-management](https://github.com/benizzio/spec-kit-orchestration-task-context-management) | | OWASP LLM Threat Model | OWASP Top 10 for LLM Applications 2025 threat analysis on agent artifacts | `code` | Read-only | [spec-kit-threatmodel](https://github.com/NaviaSamal/spec-kit-threatmodel) | +| PatchWarden Evidence Pack | Map Spec Kit tasks into a guarded PatchWarden Goal and export bounded, traceable evidence for an accepted lineage. | `process` | Read+Write | [spec-kit-patchwarden](https://github.com/jiezeng2004-design/spec-kit-patchwarden) | | Plan Review Gate | Require spec.md and plan.md to be merged via MR/PR before allowing task generation | `process` | Read-only | [spec-kit-plan-review-gate](https://github.com/luno/spec-kit-plan-review-gate) | | PR Bridge | Auto-generate pull request descriptions, checklists, and summaries from spec artifacts | `process` | Read-only | [spec-kit-pr-bridge-](https://github.com/Quratulain-bilal/spec-kit-pr-bridge-) | | Presetify | Create and validate presets and preset catalogs | `process` | Read+Write | [presetify](https://github.com/mnriem/spec-kit-extensions/tree/main/presetify) | diff --git a/extensions/catalog.community.json b/extensions/catalog.community.json index 505c71c676..d10c680be4 100644 --- a/extensions/catalog.community.json +++ b/extensions/catalog.community.json @@ -1,6 +1,6 @@ { "schema_version": "1.0", - "updated_at": "2026-07-13T00:00:00Z", + "updated_at": "2026-07-14T00:00:00Z", "catalog_url": "https://raw.githubusercontent.com/github/spec-kit/main/extensions/catalog.community.json", "extensions": { "aide": { @@ -2677,6 +2677,46 @@ "created_at": "2026-04-24T14:00:00Z", "updated_at": "2026-04-24T14:00:00Z" }, + "patchwarden-evidence": { + "name": "PatchWarden Evidence Pack", + "id": "patchwarden-evidence", + "description": "Map Spec Kit tasks into a guarded PatchWarden Goal and export bounded, traceable evidence for an accepted lineage.", + "author": "Zengjie", + "version": "1.0.1", + "download_url": "https://github.com/jiezeng2004-design/spec-kit-patchwarden/archive/refs/tags/v1.0.1.zip", + "repository": "https://github.com/jiezeng2004-design/spec-kit-patchwarden", + "homepage": "https://github.com/jiezeng2004-design/spec-kit-patchwarden", + "documentation": "https://github.com/jiezeng2004-design/spec-kit-patchwarden/blob/main/README.md", + "changelog": "https://github.com/jiezeng2004-design/spec-kit-patchwarden/blob/main/CHANGELOG.md", + "license": "MIT", + "category": "process", + "effect": "read-write", + "requires": { + "speckit_version": ">=0.1.0", + "tools": [ + { + "name": "patchwarden", + "version": ">=1.5.1", + "required": true + } + ] + }, + "provides": { + "commands": 2, + "hooks": 2 + }, + "tags": [ + "verification", + "evidence", + "traceability", + "security" + ], + "verified": false, + "downloads": 0, + "stars": 0, + "created_at": "2026-07-14T00:00:00Z", + "updated_at": "2026-07-14T00:00:00Z" + }, "plan-review-gate": { "name": "Plan Review Gate", "id": "plan-review-gate", From 77ebd5fcea6eb0ced9072fcc0656e3209a1fe6a1 Mon Sep 17 00:00:00 2001 From: Noor ul ain Date: Wed, 15 Jul 2026 01:16:36 +0500 Subject: [PATCH 44/45] fix(workflows): fail while/do-while steps on non-list steps instead of crashing (#3519) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `WhileStep.validate()` and `DoWhileStep.validate()` already reject a non-list `steps` body, but the engine's `execute()` path does not auto-validate (see `WorkflowEngine.load_workflow`, whose docstring notes the definition is "not yet validated"). On an unvalidated run the body is returned as `next_steps`, and the engine feeds it straight into `_execute_steps`, which iterates it as step mappings. A non-list `steps` — a single mapping or scalar authoring mistake — was iterated element-wise (a dict yields its string keys, a str its characters) and raised `AttributeError` on `.get()`, taking down the whole run; the engine invokes `step_impl.execute()` with no surrounding try/except. Guard both `execute` paths to return a FAILED StepResult naming the type error instead, mirroring the if/switch non-list-branch and fan-out non-list `items` handling. The do-while body always dispatches on the first call, so its guard is unconditional; the while body only dispatches when the condition is truthy, so its guard fires only then — a false condition leaves a non-list `steps` benign and the step completes, unchanged. The condition/expression is still evaluated first, so its result is surfaced in the step output for downstream context. Co-authored-by: Claude Opus 4.8 (1M context) --- .../workflows/steps/do_while/__init__.py | 24 +++++++ .../workflows/steps/while_loop/__init__.py | 26 ++++++++ tests/test_workflows.py | 65 +++++++++++++++++++ 3 files changed, 115 insertions(+) diff --git a/src/specify_cli/workflows/steps/do_while/__init__.py b/src/specify_cli/workflows/steps/do_while/__init__.py index f69a682140..ca6047a57a 100644 --- a/src/specify_cli/workflows/steps/do_while/__init__.py +++ b/src/specify_cli/workflows/steps/do_while/__init__.py @@ -27,6 +27,30 @@ def execute(self, config: dict[str, Any], context: StepContext) -> StepResult: nested_steps = config.get("steps", []) condition = config.get("condition", "false") + # The engine does not auto-validate step config (see + # ``WorkflowEngine.load_workflow``) and feeds ``next_steps`` straight + # into ``_execute_steps``, which iterates them as step mappings. A + # non-list ``steps`` (a single mapping or scalar authoring mistake) + # would otherwise be iterated element-wise — a dict yields its string + # keys, a str its characters — and crash the whole run with + # AttributeError on ``.get()``. ``validate`` already rejects a non-list + # ``steps``; fail this step loudly on an unvalidated run instead, + # mirroring the if/switch/fan-out steps. The body always runs on the + # first call, so unlike the while step this guard is unconditional. + if not isinstance(nested_steps, list): + return StepResult( + status=StepStatus.FAILED, + output={ + "condition": condition, + "max_iterations": max_iterations, + "loop_type": "do-while", + }, + error=( + f"Do-while step {config.get('id', '?')!r}: 'steps' must be " + f"a list of steps, got {type(nested_steps).__name__}." + ), + ) + # Always execute body at least once; the engine layer evaluates # `condition` after each iteration to decide whether to loop. return StepResult( diff --git a/src/specify_cli/workflows/steps/while_loop/__init__.py b/src/specify_cli/workflows/steps/while_loop/__init__.py index ea272543b6..e2dbb19305 100644 --- a/src/specify_cli/workflows/steps/while_loop/__init__.py +++ b/src/specify_cli/workflows/steps/while_loop/__init__.py @@ -26,6 +26,32 @@ def execute(self, config: dict[str, Any], context: StepContext) -> StepResult: nested_steps = config.get("steps", []) result = evaluate_condition(condition, context) + + # The engine does not auto-validate step config (see + # ``WorkflowEngine.load_workflow``) and feeds ``next_steps`` straight + # into ``_execute_steps``, which iterates them as step mappings. A + # non-list ``steps`` (a single mapping or scalar authoring mistake) + # would otherwise be iterated element-wise — a dict yields its string + # keys, a str its characters — and crash the whole run with + # AttributeError on ``.get()``. ``validate`` already rejects a non-list + # ``steps``; fail this step loudly on an unvalidated run instead, + # mirroring the if/switch/fan-out steps. The guard fires only when the + # body would actually be dispatched (condition truthy). The condition is + # still evaluated first, so its result is surfaced for downstream context. + if result and not isinstance(nested_steps, list): + return StepResult( + status=StepStatus.FAILED, + output={ + "condition_result": True, + "max_iterations": max_iterations, + "loop_type": "while", + }, + error=( + f"While step {config.get('id', '?')!r}: 'steps' must be a " + f"list of steps, got {type(nested_steps).__name__}." + ), + ) + if result: return StepResult( status=StepStatus.COMPLETED, diff --git a/tests/test_workflows.py b/tests/test_workflows.py index 8fefad740e..a766283119 100644 --- a/tests/test_workflows.py +++ b/tests/test_workflows.py @@ -2260,6 +2260,47 @@ def test_execute_condition_false(self): assert result.output["condition_result"] is False assert result.next_steps == [] + @pytest.mark.parametrize("bad_steps", [{"id": "x"}, "oops", 5]) + def test_execute_non_list_steps_fails_loudly(self, bad_steps): + """A non-list ``steps`` reached at runtime must fail the step, not crash. + + ``validate`` rejects a non-list ``steps``, but the engine does not + auto-validate (see ``WorkflowEngine.load_workflow``) and feeds + ``next_steps`` straight into ``_execute_steps``, which iterates them as + step mappings. The while body only dispatches when the condition is + truthy, so a non-list ``steps`` reaches ``next_steps`` and would crash + the engine's step iteration on an unvalidated run. Mirrors the + if/switch/fan-out non-list handling. + """ + from specify_cli.workflows.steps.while_loop import WhileStep + from specify_cli.workflows.base import StepContext, StepStatus + + step = WhileStep() + ctx = StepContext(inputs={}) + result = step.execute( + {"id": "retry", "condition": "true", "steps": bad_steps}, ctx + ) + assert result.status == StepStatus.FAILED + assert "'steps' must be a list of steps" in (result.error or "") + assert result.next_steps == [] + + @pytest.mark.parametrize("bad_steps", [{"id": "x"}, "oops", 5]) + def test_execute_non_list_steps_ok_when_condition_false(self, bad_steps): + """A false condition never dispatches the body, so a non-list ``steps`` + stays benign — the step completes without touching ``next_steps``. + """ + from specify_cli.workflows.steps.while_loop import WhileStep + from specify_cli.workflows.base import StepContext, StepStatus + + step = WhileStep() + ctx = StepContext(inputs={}) + result = step.execute( + {"id": "retry", "condition": "false", "steps": bad_steps}, ctx + ) + assert result.status == StepStatus.COMPLETED + assert result.output["condition_result"] is False + assert result.next_steps == [] + def test_validate_missing_fields(self): from specify_cli.workflows.steps.while_loop import WhileStep @@ -2350,6 +2391,30 @@ def test_execute_empty_steps(self): assert result.next_steps == [] assert result.status.value == "completed" + @pytest.mark.parametrize("bad_steps", [{"id": "x"}, "oops", 5]) + def test_execute_non_list_steps_fails_loudly(self, bad_steps): + """A non-list ``steps`` must fail the step, not crash the run. + + ``validate`` rejects a non-list ``steps``, but the engine does not + auto-validate (see ``WorkflowEngine.load_workflow``) and feeds + ``next_steps`` straight into ``_execute_steps``, which iterates them as + step mappings. The do-while body always dispatches on the first call + regardless of condition, so a non-list ``steps`` always reaches + ``next_steps`` and would crash the engine's step iteration on an + unvalidated run. Mirrors the if/switch/fan-out non-list handling. + """ + from specify_cli.workflows.steps.do_while import DoWhileStep + from specify_cli.workflows.base import StepContext, StepStatus + + step = DoWhileStep() + ctx = StepContext(inputs={}) + result = step.execute( + {"id": "cycle", "condition": "false", "steps": bad_steps}, ctx + ) + assert result.status == StepStatus.FAILED + assert "'steps' must be a list of steps" in (result.error or "") + assert result.next_steps == [] + def test_validate_missing_fields(self): from specify_cli.workflows.steps.do_while import DoWhileStep From ad601e5d52251f9131220c621d1cbbb7d61bebee Mon Sep 17 00:00:00 2001 From: Manfred Riem <15701806+mnriem@users.noreply.github.com> Date: Tue, 14 Jul 2026 16:05:15 -0500 Subject: [PATCH 45/45] docs: add PyPI as second supported install route (#3425) (#3516) * docs: add PyPI as second supported install route (#3425) The specify-cli package is now officially published to PyPI via the publish-pypi.yml trusted-publishing workflow. Document PyPI as a supported install route alongside the GitHub source install: - Revise the outdated "not affiliated" warning in installation.md to reflect that specify-cli on PyPI is an official, maintained channel. - Add an "Install from PyPI" section and list PyPI under alternative package managers. - Add a dedicated docs/install/pypi.md guide (install, pin version, verify, upgrade, uninstall). - Add the PyPI guide to the docs TOC. - Mention the PyPI route in the README quick start. Assisted-by: GitHub Copilot (model: Claude Opus 4.8, autonomous) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * docs: refine PyPI install guidance from review (#3516) Address review feedback for the PyPI install documentation: - Reword the verification guidance so `specify version` is described as a local version/runtime check rather than proof of package provenance. - Clarify that upgrading a pinned `uv tool` install to the newest PyPI release requires an unpinned reinstall command. - Note that `specify self upgrade` rebuilds `uv tool` and `pipx` installs from the GitHub source release URL rather than preserving a PyPI-based installation. Assisted-by: GitHub Copilot (model: Claude Opus 4.8, autonomous) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * docs: clarify PyPI verification and upgrade guidance Assisted-by: GitHub Copilot (model: MAI-Code-1-Flash, autonomous) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * docs: point PyPI provenance check to source metadata Address review feedback: version/list commands do not reveal install provenance. Direct readers to the source metadata their package manager records (pipx list --json, PEP 610 direct_url.json) to confirm whether an install came from PyPI or a Git URL, and note pip show cannot see uv/pipx-managed environments. Assisted-by: GitHub Copilot (model: Claude Opus 4.8, autonomous) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- README.md | 6 ++++ docs/install/pypi.md | 83 ++++++++++++++++++++++++++++++++++++++++++++ docs/installation.md | 31 ++++++++++++++--- docs/toc.yml | 2 ++ 4 files changed, 118 insertions(+), 4 deletions(-) create mode 100644 docs/install/pypi.md diff --git a/README.md b/README.md index 1386905297..77ef313f2d 100644 --- a/README.md +++ b/README.md @@ -51,6 +51,12 @@ Requires **[uv](https://docs.astral.sh/uv/)** ([install uv](./docs/install/uv.md uv tool install specify-cli --from git+https://github.com/github/spec-kit.git@vX.Y.Z ``` +Prefer installing from PyPI? The `specify-cli` package is also published there: + +```bash +uv tool install specify-cli +``` + See the [Installation Guide](./docs/installation.md) for alternative methods, verification, upgrade, and troubleshooting. ### 2. Initialize a project diff --git a/docs/install/pypi.md b/docs/install/pypi.md new file mode 100644 index 0000000000..1b89d78e44 --- /dev/null +++ b/docs/install/pypi.md @@ -0,0 +1,83 @@ +# Installing from PyPI + +Spec Kit is published to PyPI as [`specify-cli`](https://pypi.org/project/specify-cli/), maintained by the Spec Kit maintainers. Installing from PyPI is the second supported install route alongside installing from the [GitHub source](../installation.md#install-from-source--persistent-installation-recommended). Use whichever fits your workflow — both provide the same `specify` CLI. + +> [!NOTE] +> The PyPI release version tracks the GitHub release tags (for example, PyPI `0.12.11` corresponds to the `v0.12.11` tag). `specify version` is only a local version/runtime sanity check — it reports the installed version but not where the `specify` executable came from, so it cannot distinguish a PyPI install from a Git install. To confirm the install source, inspect the source metadata your package manager records: `pipx list --json` reports the exact install specification for each tool, and for uv/pip installs you can check the package's [PEP 610](https://peps.python.org/pep-0610/) `direct_url.json` inside its `*.dist-info` directory (a Git or URL install records the repository/archive URL there, while a plain PyPI index install does not create that file). Note that `pip show specify-cli` only prints package metadata and will not see uv/pipx-managed environments from the host interpreter. + +## Install Specify CLI + +Use whichever Python tool you already have: + +```bash +# Using uv (recommended) +uv tool install specify-cli + +# Or using pipx +pipx install specify-cli + +# Or using pip +pip install specify-cli +``` + +### Install a specific release + +Pin an exact version for reproducible installs (check [PyPI](https://pypi.org/project/specify-cli/#history) or [Releases](https://github.com/github/spec-kit/releases) for available versions): + +```bash +# Using uv +uv tool install specify-cli==0.12.11 + +# Or using pipx +pipx install specify-cli==0.12.11 + +# Or using pip +pip install specify-cli==0.12.11 +``` + +## Verify + +```bash +specify version +``` + +## Initialize a project + +```bash +specify init --integration copilot +``` + +## Upgrade + +Upgrade by reinstalling the package through the same tool you used for the original install. If you originally pinned a version, note that `uv tool upgrade` preserves that pin; to move to the newest PyPI release, use an unpinned install command so you do not keep the existing version pin: + +```bash +# Using uv +uv tool install --force specify-cli + +# Or using pipx +pipx install --force specify-cli + +# Or using pip +pip install --upgrade specify-cli +``` + +> [!NOTE] +> `specify self upgrade` currently rebuilds `uv tool` and `pipx` installs from the GitHub source release URL rather than preserving a PyPI-based installation. If you want to stay on the PyPI route, use the package-manager commands above. A plain `pip install specify-cli` is treated as an unmanaged install — upgrade it with `pip install --upgrade specify-cli`. See the [Upgrade Guide](../upgrade.md) for details. + +## Uninstall + +```bash +# Using uv +uv tool uninstall specify-cli + +# Or using pipx +pipx uninstall specify-cli + +# Or using pip +pip uninstall specify-cli +``` + +## Next steps + +Head to the [Quick Start](../quickstart.md) to initialize your first project. diff --git a/docs/installation.md b/docs/installation.md index 509c353829..9fb2bf519f 100644 --- a/docs/installation.md +++ b/docs/installation.md @@ -11,9 +11,14 @@ ## Installation > [!IMPORTANT] -> The only official, maintained packages for Spec Kit come from the [github/spec-kit](https://github.com/github/spec-kit) GitHub repository. Any packages with the same name available on PyPI (e.g. `specify-cli` on pypi.org) are **not** affiliated with this project and are not maintained by the Spec Kit maintainers. For normal installs, use the GitHub-based commands shown below. For offline or air-gapped environments, locally built wheels created from this repository are also valid. +> Spec Kit is distributed through two official channels, both published and maintained by the Spec Kit maintainers: the [github/spec-kit](https://github.com/github/spec-kit) GitHub repository (source installs) and the [`specify-cli`](https://pypi.org/project/specify-cli/) package on [PyPI](https://pypi.org/project/specify-cli/). Either route is supported for normal installs — use the commands shown below. After installing, run `specify version` as a local version/runtime sanity check. It confirms that the `specify` command is available and reports its version, but it does not prove whether the executable came from PyPI or GitHub. For offline or air-gapped environments, locally built wheels created from this repository are also valid. -### Persistent Installation (Recommended) +Spec Kit supports two install routes: + +1. **Install from source (GitHub)** — the recommended route, pinned to a release tag. +2. **Install from PyPI** — install the published `specify-cli` package with your usual Python tooling. + +### Install from Source — Persistent Installation (Recommended) Install once and use everywhere. Replace `vX.Y.Z` with a release tag from [Releases](https://github.com/github/spec-kit/releases) — keep the leading `v` (for example, `v0.12.11`, not `0.12.11`): @@ -30,12 +35,30 @@ Then initialize a project: specify init --integration copilot ``` +### Install from PyPI + +Spec Kit is also published to PyPI as [`specify-cli`](https://pypi.org/project/specify-cli/), so you can install it with your preferred Python package manager without referencing the Git URL: + +```bash +# Using uv (recommended) +uv tool install specify-cli + +# Or using pipx +pipx install specify-cli + +# Or using pip +pip install specify-cli +``` + +To install a specific release, pin the version — for example `uv tool install specify-cli==0.12.11`. See the [PyPI installation guide](install/pypi.md) for details, including how to upgrade. + ### One-time Usage Run directly without installing — see the [One-time usage (uvx)](install/one-time.md) guide. ### Alternative Package Managers +- **PyPI** — see the [PyPI installation guide](install/pypi.md) - **pipx** — see the [pipx installation guide](install/pipx.md) - **Enterprise / Air-Gapped** — see the [air-gapped installation guide](install/air-gapped.md) @@ -81,13 +104,13 @@ specify init --integration claude --ignore-agent-tools ## Verification -After installation, run the following command to confirm the correct version is installed: +After installation, run the following command as a local version/runtime check: ```bash specify version ``` -This helps verify you are running the official Spec Kit build from GitHub, not an unrelated package with the same name. +This confirms that the `specify` command is available and reporting the expected version. It does not prove whether that executable came from PyPI or GitHub. **Stay current:** Run `specify self check` periodically to learn whether a newer release is available — it is read-only and never modifies your installation. When you are ready to upgrade, follow the [Upgrade Guide](./upgrade.md). diff --git a/docs/toc.yml b/docs/toc.yml index ca9fba235d..6e348f6e66 100644 --- a/docs/toc.yml +++ b/docs/toc.yml @@ -13,6 +13,8 @@ href: upgrade.md - name: Install uv href: install/uv.md + - name: Install from PyPI + href: install/pypi.md - name: Install with pipx href: install/pipx.md - name: One-time Usage (uvx)