diff --git a/.agents/skills/switchyard-docs/SKILL.md b/.agents/skills/switchyard-docs/SKILL.md index 731b1f7c..b4241871 100644 --- a/.agents/skills/switchyard-docs/SKILL.md +++ b/.agents/skills/switchyard-docs/SKILL.md @@ -1,46 +1,191 @@ --- name: switchyard-docs -description: Edit or debug the published Switchyard MkDocs site. Use for docs pages, mkdocs.yml navigation, mkdocs_hooks.py source links, strict build failures, local previews, or .github/workflows/docs.yml. +description: Use when adding or editing Switchyard docs or contributor guidance under `docs/`, changing Fern configuration or navigation, debugging `fern check`, previewing or publishing docs, or reviewing the Fern GitHub Actions workflows. --- -# Switchyard Documentation +# Switchyard Fern Docs -The published site is the subset of `docs/` selected by `mkdocs.yml`. Strict MkDocs warnings fail -CI, so fix warnings rather than weakening validation. +## Overview -## Source Of Truth +Published documentation is authored as MDX at the top level of `docs/`. Fern infrastructure lives +under `docs/fern/`; do not put nightly content under `docs/fern/versions/nightly/pages/`. -| Concern | File | +`docs/README.md` is the contributor entry point. Scoped instructions live in `docs/AGENTS.md` and +`docs/fern/AGENTS.md`; neither guidance file is published. Keep those files synchronized with this +skill when the authoring, validation, redirect, or publishing workflow changes. + +The public page set, order, and labels come from `docs/fern/versions/nightly.yml`. Markdown files +under `docs/internal/` are intentionally unpublished design and operations notes. Do not add them to +the Fern navigation unless the content is deliberately being promoted to public documentation. + +`fern check` is the contract. Run it locally through `docs/Makefile`; CI runs the same CLI version +pinned in `docs/fern/fern.config.json`. + +## Quick Reference + +| Situation | Command | |---|---| -| Navigation, exclusions, theme | `mkdocs.yml` | -| Source-link rewriting | `mkdocs_hooks.py` | -| Local commands | `docs/Makefile` | -| Build, preview, deployment | `.github/workflows/docs.yml` | -| Documentation dependencies | `pyproject.toml` `docs` group | - -## Workflow - -1. Decide whether a new page is public. Public pages go in `nav`; internal notes go in - `exclude_docs`. -2. Match the surrounding file naming and documentation style. -3. Use relative links. For repository files outside `docs/`, use paths relative to the Markdown - file and let `mkdocs_hooks.py` produce the source URL. -4. Verify public examples use supported public imports and current CLI syntax. -5. Run the strict build: - -```bash -cd docs -make publish -``` - -## CI Constraints - -- Keep the docs workflow path-filtered. -- Keep default permissions read-only and grant write access only to deployment jobs. -- Keep PR previews limited to same-repository pull requests. -- Do not use `pull_request_target` to run untrusted documentation code with write permissions. -- Build once and pass the `site/` artifact to preview and deployment jobs. -- Preserve `keep_files: true` so a main deployment does not remove PR previews. - -Common strict-build fixes are direct: add or remove a missing nav entry, repair the relative link, -or explicitly exclude an internal page. Do not disable strict mode. +| Validate the site | `cd docs && make check` | +| Test routes and redirects | `uv run pytest tests/test_fern_docs.py -v` | +| Run a local preview | `cd docs && make preview` | +| Remove local Fern artifacts | `cd docs && make clean` | +| List published pages | `sed -n '1,240p' docs/fern/versions/nightly.yml` | +| Find unpublished notes | `find docs/internal -type f -name '*.md' -print` | +| Check MDX safety | `cd docs && make check` | +| Check internal Fern links | `uv run pytest tests/test_fern_docs.py::test_internal_mdx_links_resolve_to_navigation_routes -v` | + +## Where Things Live + +- **Published content** → `docs/**/*.mdx`, excluding `docs/fern/`. +- **Unpublished notes** → `docs/internal/**/*.md`. +- **Docs contributor guide and scoped instructions** → `docs/README.md` and `docs/AGENTS.md`. +- **Fern infrastructure guide and scoped instructions** → `docs/fern/README.md` and + `docs/fern/AGENTS.md`. +- **Site and product configuration** → `docs/fern/docs.yml`. +- **Nightly navigation** → `docs/fern/versions/nightly.yml`. +- **Fern CLI pin** → `docs/fern/fern.config.json`. +- **Local commands** → `docs/Makefile`. +- **CI validation** → `.github/workflows/fern-docs-ci.yml`. +- **PR preview collection** → `.github/workflows/fern-docs-preview-build.yml`. +- **PR preview publishing and comment** → `.github/workflows/fern-docs-preview-comment.yml`. +- **Release publishing** → `.github/workflows/publish-fern-docs.yml`. +- **GitHub Pages redirect generator** → `docs/fern/generate_legacy_redirect_site.py`. +- **Route/redirect regression tests** → `tests/test_fern_docs.py`. +- **Shared NVIDIA presentation** → `global-theme: nvidia`; theme assets are not vendored here. + +## Adding a Published Page + +1. Create `docs/
/.mdx` with concise frontmatter, including `title` and `position`. +2. Add it to `docs/fern/versions/nightly.yml` in the intended sidebar location. Preserve existing + labels and order unless the change explicitly redesigns navigation. +3. Point `path:` from the version file back to the authored page with `../../`, for example: + + ```yaml + - page: Health-aware Routing + path: ../../routing_algorithms/health_aware_routing.mdx + ``` + +4. Use version-agnostic Fern URLs for published-page links. Fern URLs derive from the slugified + `page:` and `section:` labels, not filenames. For example, `page: Core Concepts` is + `/concepts/core-concepts` inside every version. +5. Link repository files outside `docs/` with an absolute GitHub URL. They are not part of Fern's + content tree. +6. Import examples from `switchyard`, not `switchyard.lib.*`; public examples must use exports from + `switchyard/__init__.py`. +7. Run `cd docs && make check` before pushing. + +## Editing Navigation + +- Treat `docs/fern/versions/nightly.yml` as the source of truth for public scope and order. +- Use `section:` for labeled groups with nested `contents:`. Do not combine `folder:` with + `contents:`; Fern rejects that schema. +- Keep Switchyard sections label-only. Do not add a `path:` to a section and then repeat that same + file as a child page; Fern publishes the section landing route instead of the intended child + route. Every authored page in this site belongs under `contents:` and resolves as + `/section/page`. +- Keep the first navigation entry as the home page. +- Paths in nightly navigation point to `../../*.mdx`. Frozen releases, when added, own independent + copies under `docs/fern/versions//pages/`. +- Do not add `latest.yml` as a copy of nightly. A future `latest` alias must point at a frozen GA + tree. + +## Links and MDX + +- MDX treats bare angle-bracket text as JSX. Escape comparisons such as `<100ms` as `<100ms` + and wrap placeholders such as `` in backticks. +- HTML void tags must self-close, for example ``. +- Use MDX comments (`{/* note */}`), not HTML comments. +- Built-in Fern components do not require imports. Add product-specific components only when a page + needs them, then register their directory in `docs/fern/docs.yml`. +- Do not link unpublished `docs/internal/` notes from published MDX. +- Preserve titled MkDocs admonitions with the Fern callout `title` prop, for example + ``. +- Give every published page a non-empty `description` in frontmatter. + +## Redirects + +- Use full `/nemo/switchyard/...` paths in both `source` and `destination`. +- Add exact mappings when old filenames and new navigation slugs differ. A generic `index.html` + catch-all cannot translate underscores or changed section hierarchy. +- Cover both the directory route and its generated `index.html` form for every former MkDocs page. +- Keep `docs/fern/generate_legacy_redirect_site.py` synchronized with `docs/fern/docs.yml`. The + generator owns redirects on `nvidia-nemo.github.io/Switchyard`; Fern redirects only own requests + that already reach the Fern domain. +- Run `uv run pytest tests/test_fern_docs.py -v` after changing navigation, links, or redirects. + +## CI Hygiene + +The Fern workflows follow the same split as NeMo Curator: + +- `fern-docs-ci.yml` is a reusable `fern check` workflow called by the repository-wide CI. Its job + is included in the required `CI Success` aggregate, so Fern failures cannot merge as an optional + check. +- `fern-docs-preview-build.yml` runs only for same-repository PR branches, uses no secrets, and + uploads the complete `docs/` tree plus a changed-MDX hint. Fork PRs stop after required Fern + validation. The complete tree is required because nightly navigation under `docs/fern/` + references MDX pages through `../../` paths. It uses PR-numbered concurrency and cancels + superseded runs. +- `fern-docs-preview-comment.yml` runs through `workflow_run`, downloads the collector artifact, + rejects fork-originated runs, derives the target PR from the trusted event, reads the Fern version + from a separate default-branch checkout, and generates a PR-numbered preview with + `DOCS_FERN_TOKEN`. Commenting runs in a separate job with only pull-request write permission. It + must never check out the PR branch, run PR-provided scripts, or trust artifact metadata for the + target PR/tool version. +- `publish-fern-docs.yml` publishes only for `docs/v*` tags or manual dispatch, then replaces the + old GitHub Pages content with static redirects to the custom Fern domain. The workflow verifies + the custom-domain home page first; if it remains unavailable, GitHub Pages stays untouched. + Publishing is serialized, and the repository-write redirect job never receives the Fern token. + A merge to `main` validates the site but does not publish either destination. + +Every workflow installs the exact Fern CLI version from `docs/fern/fern.config.json` and runs Fern +from `docs/fern/`. The privileged preview reads that pin from the trusted default branch rather +than the artifact. Preview generation and publishing require the `DOCS_FERN_TOKEN` organization +secret. All Fern workflow actions use immutable commit pins and supported Node LTS; secret-bearing +jobs disable package-manager caching and validate the Fern version before installing it. Checkout +steps do not persist credentials. Do not collapse the collector and trusted preview phases or use +`pull_request_target`. + +## Failure Map + +| Symptom | Fix | +|---|---| +| `Path does not exist` | Correct the `../../` path in `versions/nightly.yml` or restore the referenced MDX file. | +| Navigation object does not match schema | Use `section` with `contents`, or a standalone `folder`; do not combine `folder` and `contents`. | +| Unsupported JSX tag | Replace it with a built-in Fern component or register the product component directory. | +| MDX parse error near `<` | Escape prose comparisons/placeholders or self-close the HTML element. | +| Link works on GitHub but not Fern | Rewrite it to the canonical Fern URL or an absolute GitHub URL for repository-only files. | +| Child page returns 404 but section works | Remove the duplicate section `path`; keep the authored page only under `contents`. | +| Legacy redirect lands on another 404 | Add an exact old-path → current-slug mapping before broad redirect patterns. | +| `global-theme` authentication failure | Verify NVIDIA Fern authentication; do not vendor a local copy of the shared theme. | + +## Anti-Patterns + +- Adding unpublished design notes to navigation because they happen to live under `docs/`. +- Adding `README.md` or `AGENTS.md` contributor guidance to published navigation. +- Recreating `docs/fern/versions/nightly/pages/`; nightly content is authored at `docs/*.mdx`. +- Deriving links from filenames instead of the navigation title/slug. +- Assigning the same MDX file to a section `path` and one of its child pages. +- Hardcoding `/nightly/` in shared page links. +- Vendoring NVIDIA theme CSS, logos, favicon, or footer components. +- Reintroducing MkDocs configuration, hooks, or a parallel docs build. +- Running a secret-bearing Fern preview directly from a `pull_request` job. +- Publishing production docs on every merge to `main` instead of a docs tag or manual dispatch. + +## References + +- `docs/fern/docs.yml` — Fern instance, theme, versions, and redirects. +- `docs/README.md` — contributor entry point for docs content. +- `docs/AGENTS.md` — scoped instructions for the full docs tree. +- `docs/fern/README.md` — Fern infrastructure and CI guide. +- `docs/fern/AGENTS.md` — scoped instructions for Fern configuration and redirects. +- `docs/fern/versions/nightly.yml` — public navigation and authored-page paths. +- `docs/fern/fern.config.json` — Fern CLI pin. +- `docs/Makefile` — local check and preview commands. +- `docs/fern/generate_legacy_redirect_site.py` — redirect-only GitHub Pages output. +- `tests/test_fern_docs.py` — navigation, link, metadata, and redirect invariants. +- `.github/workflows/fern-docs-ci.yml` — CI-equivalent validation. +- `.github/workflows/fern-docs-preview-build.yml` — same-repository preview source collection. +- `.github/workflows/fern-docs-preview-comment.yml` — trusted preview generation and PR comment. +- `.github/workflows/publish-fern-docs.yml` — tag/manual production publishing. +- [`switchyard-testing-ci`](../switchyard-testing-ci/SKILL.md) — broader repository validation. +- [`switchyard-codebase-exploration`](../switchyard-codebase-exploration/SKILL.md) — impact mapping for docs that cite code or APIs. diff --git a/.agents/skills/switchyard-testing-ci/SKILL.md b/.agents/skills/switchyard-testing-ci/SKILL.md index 9aed5cc4..38a5baca 100644 --- a/.agents/skills/switchyard-testing-ci/SKILL.md +++ b/.agents/skills/switchyard-testing-ci/SKILL.md @@ -1,80 +1,322 @@ --- name: switchyard-testing-ci -description: Select or debug Switchyard validation and GitHub CI. Use when asked which tests to run, whether a change is ready, why a CI job failed, how to reproduce CI, or whether live provider tests are appropriate. +description: Use when validating a Switchyard change, preparing or reviewing a PR, debugging CI failures (ruff, mypy, SPDX, pytest, slim-install), choosing which local tests to run, or diagnosing dependency, optional-extra, stale-name, CLI, server, translation, routing, stats, or live e2e failures. Triggers on phrases like "is this ready to merge", "CI is failing", "which tests should I run", or "how do I reproduce this locally". --- -# Switchyard Testing And CI +# Switchyard Testing and CI -Read `.github/workflows/ci.yml` before claiming local CI equivalence. Select tests from the current -diff and symbol references rather than a static subsystem inventory. +## Overview -## Validation Matrix +**Pick the smallest local gate that is genuinely equivalent to what CI will run, then map failures +to targeted fixes.** The two goals — move fast locally, never report a green run weaker than the CI +gate you are claiming — usually conflict; this skill resolves that conflict from the current diff +rather than from memory. -| Change | Minimum trustworthy validation | +If your change also touches code you have not yet read, run [`switchyard-codebase-exploration`](../switchyard-codebase-exploration/SKILL.md) first so the validation set covers the real impact, not just the diff surface. + +Rust crates under `crates/` (`switchyard-core`, `switchyard-translation`, +`switchyard-components`, `switchyard-server`, `switchyard-py`) are active on the Rust core branch. +For Rust-touching diffs, include `cargo fmt --all --check`, `cargo clippy --workspace +--all-targets -- -D warnings`, and the smallest trustworthy `cargo test` scope; +use `cargo test --workspace` before calling a broad Rust MR ready. + +## Quick Reference + +| Situation | Command | |---|---| -| Markdown or skill Markdown only | Parse skill frontmatter when applicable; `git diff --check` | -| Python implementation | Focused pytest, `uv run ruff check .`, `uv run mypy switchyard` | -| Broad Python or public API | Hermetic full pytest plus lint and mypy | -| One Rust crate | `cargo fmt --all --check`, workspace clippy, focused crate tests | -| Shared Rust/API/FFI | Rust workspace tests plus affected Python tests | -| Packaging, extras, top-level imports | The slim-install smoke commands from the current CI workflow | -| Live provider behavior | Focused integration test only after explicit authorization | +| Map the current diff | `git status -sb && git diff --stat && git diff --name-only` | +| Pre-PR hermetic gate (default) | `uv run ruff check . && uv run mypy switchyard && env -u OPENROUTER_API_KEY -u NVIDIA_API_KEY -u OPENAI_API_KEY -u ANTHROPIC_API_KEY uv run pytest tests/ -v -m "not integration"` | +| Mirror CI pytest with no live creds | `env -u OPENROUTER_API_KEY -u NVIDIA_API_KEY -u OPENAI_API_KEY -u ANTHROPIC_API_KEY uv run pytest tests/ -v -m "not integration"` | +| Rust component crate change | `cargo fmt --all --check && cargo clippy --workspace --all-targets -- -D warnings && cargo test -p switchyard-components` | +| Rust server crate change | `cargo fmt --all --check && cargo clippy --workspace --all-targets -- -D warnings && cargo test -p switchyard-server` | +| Broad Rust crate change | `cargo fmt --all --check && cargo clippy --workspace --all-targets -- -D warnings && cargo test --workspace` | +| Docs/Fern workflow change | `make -C docs check && uv run pytest tests/test_fern_docs.py -v -o addopts=` | +| Select focused validation from uncommitted paths | `python scripts/select_validation.py --changed` | +| Select validation for committed branch work | `python scripts/select_validation.py --base origin/main` | +| Any GitHub Actions workflow change | Run the selector, then `actionlint && zizmor --pedantic .github/workflows`; review `scripts/ci/README.md` | +| Slim-install regression guard | see [Slim-install smoke gate](#slim-install-smoke-gate) | +| Live e2e (only on explicit user request) | `NVIDIA_API_KEY=… uv run pytest tests/e2e/ -v -m integration -o addopts= --maxfail=10` | +| Skill/docs change only | YAML frontmatter check + `git diff --check` (see [Skill/docs-only gate](#skilldocs-only-gate)) | + +## Diff-Based Selection First + +From the repo root, inspect the diff and map each changed area to the focused gates below: + +```bash +cd "$(git rev-parse --show-toplevel)" +git status -sb +git diff --stat +``` + +Use the current diff as the starting plan, then add any tests revealed by code search or by the +failure map in this skill. + +## CI Gates and Local Equivalence + +The hard GitHub Actions gates live in `.github/workflows/ci.yml`: -## Standard Gates +- `uv run ruff check .` +- SPDX header check for every Python file found by CI, including `.agents/skills/**/scripts/*.py` +- `uv run pytest tests/ -v -m "not integration"` on Python 3.12 through 3.14 +- reusable Fern validation (`fern check`) called from `.github/workflows/ci.yml` and included in + the required `CI Success` aggregate +- Rust workspace gate: `cargo fmt --all --check`, `cargo clippy --workspace --all-targets -- -D warnings`, + and `cargo test --workspace` +- slim-install smoke: isolated `uv run --with` default install/import checks, heavy-package + absence, and CLI help checks with `nemo-switchyard[cli,server] @ file://...` +- `uv run mypy switchyard` is a signal job (`continue-on-error`) but should still be run for typed + package code, public APIs, profiles, route bundles, backends, request/response models, and translation changes. -Python pre-PR gate: +The local pre-commit config also runs the Rust fmt/clippy gate when Rust files, +`Cargo.toml`, or `Cargo.lock` change. Treat those hooks as hard failures; do not +bypass them with `--no-verify`. + +A CI-equivalent `ruff` claim means a clean checkout or generated artifacts removed. Do not treat +`uv run ruff check . --exclude docs/.venv-docs --exclude docs/_build` as canonical PR validation; +that is only a dirty-workspace workaround. + +If generated docs artifacts exist locally, prefer: + +```bash +rm -rf docs/.venv-docs docs/_build site/_build +git status -sb +uv run ruff check . +``` + +## Standard Command Sets + +### Fast local loop + +```bash +uv run pytest tests/test_.py -v +uv run pytest tests/test_.py::test_name -v +uv run ruff check . +``` + +Pytest uses `addopts = "-x"`; use `-o addopts= --maxfail=10` when finding all failures: + +```bash +uv run pytest tests/ -v -o addopts= --maxfail=10 -m "not integration" +``` + +### Pre-PR package-code gate + +Use the hermetic test gate by default so local credentials cannot trigger live provider calls: ```bash uv run ruff check . uv run mypy switchyard -env -u OPENROUTER_API_KEY -u NVIDIA_API_KEY -u OPENAI_API_KEY \ - -u ANTHROPIC_API_KEY uv run pytest tests/ -v -m "not integration" +env -u OPENROUTER_API_KEY -u NVIDIA_API_KEY -u OPENAI_API_KEY -u ANTHROPIC_API_KEY uv run pytest tests/ -v -m "not integration" ``` -Rust workspace gate: +If you need to mirror the CI pytest command locally, run from a clean checkout or explicitly remove +provider credentials first: ```bash -cargo fmt --all --check -cargo clippy --workspace --all-targets -- -D warnings -cargo test --workspace +env -u OPENROUTER_API_KEY -u NVIDIA_API_KEY -u OPENAI_API_KEY -u ANTHROPIC_API_KEY uv run pytest tests/ -v -m "not integration" ``` -Use `-o addopts=` when the repository's pytest defaults would stop failure collection too early. -Run focused tests first, then broaden only when the changed contract or shared ownership warrants it. +### Live integration/e2e gate + +Run this only when the user asked for live validation and accepts upstream API calls: + +```bash +NVIDIA_API_KEY=... uv run pytest tests/e2e/ -v -m integration -o addopts= --maxfail=10 +``` -## Live Tests +`tests/e2e/conftest.py` and per-file markers/skips own credential behavior. Do not run live e2e as +a default PR gate. -Integration tests can spend money, require credentials, and contact real services. Never run them -as an automatic pre-PR step. Provision `NVIDIA_API_KEY` through the environment or a secret manager -before running the user-authorized path, and state which provider was called: +### Slim-install smoke gate + +Run after touching `pyproject.toml`, lockfiles, top-level imports, optional extras, +`switchyard/__init__.py`, CLI/server exports, or anything that could pull heavy packages into the +default install: ```bash -uv run pytest tests/e2e/.py \ - -v -m integration -o addopts= +SWITCHYARD_ROOT="$(git rev-parse --show-toplevel)" +SWITCHYARD_DEFAULT_PACKAGE="nemo-switchyard @ file://${SWITCHYARD_ROOT}" +SWITCHYARD_EXTRAS_PACKAGE="nemo-switchyard[cli,server] @ file://${SWITCHYARD_ROOT}" +cd /tmp +uv run --isolated --no-project --python 3.12 \ + --with "${SWITCHYARD_DEFAULT_PACKAGE}" \ + python - <<'PY' +import importlib.util +import switchyard +print('import OK, version:', switchyard.__version__) +from switchyard import RandomRoutingPresets +print('preset import OK:', sorted(RandomRoutingPresets.PRESETS)) +forbidden = [ + 'torch', 'transformers', 'huggingface_hub', 'litellm', + 'datasets', 'tokenizers', 'safetensors', 'pyarrow', 'agents', 'mcp', 'docx', +] +extras = [m for m in forbidden if importlib.util.find_spec(m) is not None] +if extras: + raise SystemExit(f'FAIL: heavy packages pulled into slim install: {extras}') +print('OK: heavy packages absent') +PY +uv run --isolated --no-project \ + --python 3.12 \ + --with "${SWITCHYARD_EXTRAS_PACKAGE}" \ + switchyard launch claude --help >/dev/null +uv run --isolated --no-project \ + --python 3.12 \ + --with "${SWITCHYARD_EXTRAS_PACKAGE}" \ + switchyard launch claude --help 2>&1 \ + | grep -q -- '--routing-profiles' || { echo 'FAIL: --routing-profiles flag missing from help'; exit 1; } +uv run --isolated --no-project \ + --python 3.12 \ + --with "${SWITCHYARD_EXTRAS_PACKAGE}" \ + switchyard serve --help >/dev/null ``` -## Skill Validation +### Skill/docs-only gate -Each retained skill must have YAML frontmatter with `name` and `description`: +For `.agents/skills/` changes, validate YAML frontmatter, whitespace, and any Python helper scripts: ```bash -uv run python - <<'PY' +python - <<'PY' from pathlib import Path - import yaml - -for path in Path(".agents/skills").glob("*/SKILL.md"): - parts = path.read_text().split("---", 2) - data = yaml.safe_load(parts[1]) if len(parts) == 3 else None - if not isinstance(data, dict) or not data.get("name") or not data.get("description"): - raise SystemExit(f"{path}: missing name or description") -print("skill frontmatter is valid") +for p in Path('.agents/skills').glob('*/SKILL.md'): + text = p.read_text(encoding='utf-8') + if not text.startswith('---\n'): + raise SystemExit(f'{p}: missing frontmatter start') + end = text.find('\n---\n', 4) + if end == -1: + raise SystemExit(f'{p}: missing frontmatter end') + data = yaml.safe_load(text[4:end]) + if not isinstance(data, dict) or not data.get('name') or not data.get('description'): + raise SystemExit(f'{p}: missing name/description') +print('OK: skill frontmatter parses') PY git diff --check +find .agents/skills -path '*/scripts/*.py' -print +``` + +For any listed skill Python helper, run targeted lint and SPDX checks: + +```bash +.venv/bin/ruff check --isolated .agents/skills//scripts/.py +python - <<'PY' +from importlib.machinery import SourceFileLoader +from pathlib import Path +spdx = SourceFileLoader('spdx', '.hooks/add_spdx_headers.py').load_module() +paths = [Path('.agents/skills//scripts/.py')] +bad = [str(p) for p in paths if not spdx.has_required_header(p)] +if bad: + raise SystemExit('missing SPDX header: ' + ', '.join(bad)) +print('OK: SPDX headers present') +PY +``` + +`.agents` is gitignored, so stage new or modified skills explicitly: + +```bash +git status --ignored -sb .agents/skills +git add -f .agents/skills/ +``` + +## Failure Map + +### Ruff + +- Generated docs/venv paths (`docs/.venv-docs/`, `docs/_build/`, `site/_build/`) are local debris; + remove them rather than editing vendored/generated files. +- Use autofix narrowly, then inspect the diff: + + ```bash + uv run ruff check --fix . + uv run ruff check . + ``` + +### SPDX + +CI requires this exact header in every Python file, after an optional shebang: + +```python +# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 +``` + +Fix with: + +```bash +python .hooks/add_spdx_headers.py path/to/file.py +uv run ruff check . +``` + +### Pytest + +Classify before editing: + +- Optional package import (`torch`, `fastapi`, `prompt_toolkit`, `nemo_platform`) → keep + optional deps out of top-level/default install paths; use lazy imports and clear runtime errors. +- Unit test makes network call → mock with `respx`, `httpx.ASGITransport`, or a fake backend. +- Translation mismatch → run translation-focused tests before touching broad plumbing. +- Streaming failure → inspect streaming response accumulator, SSE helpers, and translation-stream tests. +- CLI/config regression → run launcher/config/model/verify tests. +- Routing/stats regression → run the random-routing suite. + +Useful focused groups: + +```bash +uv run pytest tests/test_anthropic_openai_translation.py tests/test_responses_openai_translation.py tests/test_translation_engine_chaos.py -v -o addopts= +uv run pytest tests/test_launch_claude.py tests/test_launch_codex.py tests/test_launch_route_builder.py tests/test_user_config.py tests/test_model_discovery.py tests/test_verify.py -v -o addopts= +uv run pytest tests/test_random_routing_config.py tests/test_random_routing_llm_backend.py tests/test_random_routing_presets.py tests/test_random_routing_profile.py -v -o addopts= +``` + +### Mypy + +Mypy is strict on `switchyard/` and ignores tests. Prefer concrete request/response subclasses after +`isinstance` narrowing, `TYPE_CHECKING` imports for circular or optional types, and aliases from +`switchyard/lib/roles.py` instead of broad `Any`. Use targeted ignores with error codes only when an +external incompatibility is localized. + +### Slim install / dependencies + +Default install must not pull heavyweight packages: + +- `torch`, `transformers`, `huggingface_hub`, `litellm`, `datasets`, `tokenizers`, + `safetensors`, `pyarrow`, `agents`, `mcp`, `docx` + +Keep them in optional extras or dev groups, not `[project.dependencies]`. Dependency changes require +explicit user intent; after changing dependencies run `uv lock`, `uv sync`, and the selected tests. + +### Stale package names + +The distribution package is `nemo-switchyard`; imports and the CLI are `switchyard`. Do not +introduce `nemo_switchyard` imports or publish docs that imply `pip install switchyard`. Check: + +```bash +uv run pytest tests/test_cli_stale_names.py tests/test_no_stale_module_paths.py tests/test_version_package_name.py -v -o addopts= +``` + +## Common Mistakes + +| Mistake | Reality | +|---|---| +| Running `uv run ruff check . --exclude docs/.venv-docs --exclude docs/_build` and calling it CI-equivalent | That is a dirty-workspace workaround, not the CI command. Either remove the generated artifacts (`rm -rf docs/.venv-docs docs/_build site/_build`) or run from a clean checkout. | +| Running `uv run pytest tests/ -v` with `OPENROUTER_API_KEY`/`NVIDIA_API_KEY`/`OPENAI_API_KEY`/`ANTHROPIC_API_KEY` set in your shell | Tests that mock providers can accidentally hit live endpoints. Default to `-m "not integration"`, or strip credentials with `env -u`. | +| Treating mypy as optional because CI marks it `continue-on-error` | Mypy still catches real bugs in `switchyard/` typed code. Run it for any change to profiles, route bundles, backends, request/response models, or translation. | +| Skipping the slim-install smoke gate after a dependency or top-level import change | This is the *actual* hard CI gate that catches `torch`/`transformers` accidentally landing in the default install. | +| Claiming validation from `scripts/select_validation.py --changed` after committing changes | `--changed` covers staged, unstaged, and untracked paths. Use `--base origin/main` to include committed branch changes, or pass `--path` explicitly. | +| Adding `# noqa` or per-line ignores to make ruff green | Ruff is a hard CI gate. Fix the code or, if the rule is wrong here, lift the ignore to the file or project level with a one-line justification. | +| Editing a generated artifact (e.g., `docs/.venv-docs/...`) because ruff complained about it | Delete the artifact instead. CI does not have it; you should not either. | + +## Report Format + +Report validation with exact commands and live-call status: + +```markdown +Validation: +- `uv run ruff check .` — pass +- `uv run mypy switchyard` — pass / not run () +- `uv run pytest tests/ -v -m "not integration"` — pass / not run () +- Slim install smoke — pass / not run () +- Live provider tests — not run / pass () ``` -## Reporting +## Related Skills -Report exact commands, pass/fail status, and whether any live network call occurred. Do not call a -focused test run "full CI", and do not treat mypy or clippy warnings as unrelated without checking -whether the diff introduced them. +- [`switchyard-codebase-exploration`](../switchyard-codebase-exploration/SKILL.md) — run this **before** the testing-ci skill on any non-trivial change, so the validation set you pick covers every file the change actually touches (importers, profile builders, tests), not just the diff surface. diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 64af8df4..fedb3cd4 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -132,6 +132,12 @@ jobs: - name: mypy run: uv run mypy switchyard + # Keep Fern validation inside the required CI Success aggregate so a docs + # configuration failure cannot be merged as an optional check. + fern-docs: + name: Fern docs + uses: ./.github/workflows/fern-docs-ci.yml + # Unit tests on supported Python versions. Integration/e2e tests are # intentionally deselected here because they may need Docker, live provider # credentials, or network egress. @@ -234,7 +240,7 @@ jobs: ci-success: name: CI Success if: always() - needs: [changes, lint, spdx-headers, rust, typecheck, test, slim-install-smoke] + needs: [changes, lint, spdx-headers, rust, typecheck, fern-docs, test, slim-install-smoke] runs-on: ubuntu-latest steps: - name: Verify all jobs succeeded diff --git a/.github/workflows/docs.yml b/.github/workflows/docs.yml deleted file mode 100644 index d8611564..00000000 --- a/.github/workflows/docs.yml +++ /dev/null @@ -1,100 +0,0 @@ -name: Documentation - -on: - pull_request: - paths: - - "docs/**" - - "mkdocs_hooks.py" - - "mkdocs.yml" - - "pyproject.toml" - - "uv.lock" - - ".github/workflows/docs.yml" - push: - branches: [main] - paths: - - "docs/**" - - "mkdocs_hooks.py" - - "mkdocs.yml" - - "pyproject.toml" - - "uv.lock" - - ".github/workflows/docs.yml" - -# Cancel superseded PR runs; keep main builds so gh-pages writes don't race. -concurrency: - group: ${{ github.workflow }}-${{ github.ref }} - cancel-in-progress: ${{ github.event_name == 'pull_request' }} - -env: - MKDOCS_SOURCE_REF: ${{ github.event.pull_request.head.sha || github.sha }} - -permissions: - contents: read - -jobs: - build: - name: Build docs - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v4 - - uses: astral-sh/setup-uv@v6 - with: - python-version: "3.12" - enable-cache: true - cache-dependency-glob: "uv.lock" - - run: uv sync --only-group docs --locked - - name: Build documentation - run: uv run --only-group docs mkdocs build --strict - - name: Upload site artifact - uses: actions/upload-artifact@v4 - with: - name: docs-site - path: site/ - retention-days: 1 - - deploy: - name: Deploy to GitHub Pages - needs: build - if: github.event_name == 'push' && github.ref == 'refs/heads/main' - runs-on: ubuntu-latest - permissions: - contents: write - steps: - - name: Download site artifact - uses: actions/download-artifact@v4 - with: - name: docs-site - path: site/ - - name: Deploy to GitHub Pages - uses: peaceiris/actions-gh-pages@v4 - with: - github_token: ${{ secrets.GITHUB_TOKEN }} - publish_dir: ./site - keep_files: true - - # Fork PRs can't write to gh-pages with GITHUB_TOKEN, so the preview is - # gated to same-repo PRs. - preview: - name: Deploy PR preview - needs: build - if: >- - github.event_name == 'pull_request' && - github.event.pull_request.head.repo.full_name == github.repository - runs-on: ubuntu-latest - permissions: - contents: write - pull-requests: write - steps: - - uses: actions/checkout@v4 - - name: Download site artifact - uses: actions/download-artifact@v4 - with: - name: docs-site - path: site/ - - name: Deploy PR preview - uses: rossjrw/pr-preview-action@v1 - with: - source-dir: ./site - preview-branch: gh-pages - umbrella-dir: pr-preview - action: auto - qr-code: false diff --git a/.github/workflows/fern-docs-ci.yml b/.github/workflows/fern-docs-ci.yml new file mode 100644 index 00000000..12ee6203 --- /dev/null +++ b/.github/workflows/fern-docs-ci.yml @@ -0,0 +1,53 @@ +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# Reusable Fern validation gate called by the repository CI workflow. + +name: Fern docs (check) + +on: + workflow_call: + +permissions: + contents: read + +jobs: + check: + name: Check Fern docs + runs-on: ubuntu-latest + steps: + - name: Checkout repository + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + persist-credentials: false + + - name: Setup Node.js + uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0 + with: + node-version: "24" + package-manager-cache: false + + - name: Install Fern CLI + run: | # zizmor: ignore[adhoc-packages] Fern is exact-version pinned in fern.config.json. + FERN_VERSION="$(jq -r '.version' docs/fern/fern.config.json)" + if ! [[ "$FERN_VERSION" =~ ^[0-9]+\.[0-9]+\.[0-9]+(-[A-Za-z0-9.]+)?$ ]]; then + echo "::error::Refusing to install unexpected Fern CLI version: $FERN_VERSION" + exit 1 + fi + npm install -g "fern-api@${FERN_VERSION}" + + - name: Validate Fern configuration + working-directory: ./docs/fern + run: fern check diff --git a/.github/workflows/fern-docs-preview-build.yml b/.github/workflows/fern-docs-preview-build.yml new file mode 100644 index 00000000..e874d637 --- /dev/null +++ b/.github/workflows/fern-docs-preview-build.yml @@ -0,0 +1,67 @@ +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# Workflow 1 of 2 for Fern doc previews. +# +# Collects docs sources and changed-file metadata from same-repository PR +# branches. No secrets are used here. Fork PRs receive the required Fern check +# but do not enter the hosted-preview path. The companion workflow builds the +# preview and posts the PR comment. + +name: "Preview Fern Docs: Build" + +on: + pull_request: + paths: + - "docs/**" + - ".github/workflows/fern-docs-preview-build.yml" + +permissions: + contents: read + +concurrency: + group: fern-docs-preview-${{ github.event.pull_request.number }} + cancel-in-progress: true + +jobs: + collect: + name: Collect preview sources + if: ${{ github.event.pull_request.head.repo.full_name == github.repository }} + runs-on: ubuntu-latest + steps: + - name: Checkout PR + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + fetch-depth: 0 + persist-credentials: false + + - name: Save changed MDX paths + env: + BASE_REF: ${{ github.base_ref }} + run: | + mkdir -p .preview-metadata + git diff --name-only "origin/${BASE_REF}...HEAD" -- '*.mdx' > .preview-metadata/changed_mdx_files 2>/dev/null || true + + # The nightly navigation reaches from docs/fern/ into docs/**/*.mdx, + # so the artifact must preserve the complete docs tree. + - name: Upload docs sources and metadata + uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 + with: + name: fern-preview + path: | + docs/ + .preview-metadata/ + include-hidden-files: true + retention-days: 1 diff --git a/.github/workflows/fern-docs-preview-comment.yml b/.github/workflows/fern-docs-preview-comment.yml new file mode 100644 index 00000000..51195770 --- /dev/null +++ b/.github/workflows/fern-docs-preview-comment.yml @@ -0,0 +1,191 @@ +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# Workflow 2 of 2 for Fern doc previews. +# +# Runs after the same-repository collector succeeds, downloads only its artifact, +# creates a Fern preview with DOCS_FERN_TOKEN, and upserts a PR comment. PR +# identity and the Fern CLI version come from trusted GitHub/default-branch +# state. The workflow never checks out or executes scripts from the PR branch. + +name: "Preview Fern Docs: Comment" + +on: + workflow_run: # zizmor: ignore[dangerous-triggers] Same-repository runs only; identity/tooling are trusted. + workflows: ["Preview Fern Docs: Build"] + types: [completed] + +permissions: {} + +concurrency: + group: fern-docs-preview-${{ github.event.workflow_run.pull_requests[0].number }} + cancel-in-progress: true + +jobs: + preview: + name: Publish preview + runs-on: ubuntu-latest + if: >- + ${{ + github.event.workflow_run.conclusion == 'success' && + github.event.workflow_run.event == 'pull_request' && + github.event.workflow_run.head_repository.full_name == github.repository && + github.event.workflow_run.pull_requests[0].number != null + }} + permissions: + # Download the triggering artifact and read only the trusted default branch. + actions: read # Required to download the triggering workflow artifact. + contents: read + outputs: + page_links: ${{ steps.page-links.outputs.page_links }} + pr_number: ${{ steps.metadata.outputs.pr_number }} + preview_url: ${{ steps.generate-docs.outputs.preview_url }} + steps: + - name: Checkout trusted Fern configuration + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + ref: ${{ github.event.repository.default_branch }} + path: trusted + persist-credentials: false + + - name: Download docs sources and metadata + uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 + with: + name: fern-preview + run-id: ${{ github.event.workflow_run.id }} + github-token: ${{ secrets.GITHUB_TOKEN }} + path: preview-source + + - name: Read trusted workflow metadata + id: metadata + env: + PR_NUMBER: ${{ github.event.workflow_run.pull_requests[0].number }} + run: | + if ! [[ "$PR_NUMBER" =~ ^[0-9]+$ ]]; then + echo "::error::workflow_run did not identify a pull request." + exit 1 + fi + echo "pr_number=$PR_NUMBER" >> "$GITHUB_OUTPUT" + + - name: Read trusted Fern CLI version + id: fern-version + run: | + FERN_VERSION="$(jq -r '.version' trusted/docs/fern/fern.config.json)" + if ! [[ "$FERN_VERSION" =~ ^[0-9]+\.[0-9]+\.[0-9]+(-[A-Za-z0-9.]+)?$ ]]; then + echo "::error::Refusing to install unexpected Fern CLI version: $FERN_VERSION" + exit 1 + fi + echo "version=$FERN_VERSION" >> "$GITHUB_OUTPUT" + + - name: Setup Node.js + uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0 + with: + node-version: "24" + package-manager-cache: false + + - name: Install pinned Fern CLI + env: + FERN_VERSION: ${{ steps.fern-version.outputs.version }} + run: npm install -g "fern-api@$FERN_VERSION" # zizmor: ignore[adhoc-packages] Trusted exact version. + + - name: Generate preview URL + id: generate-docs + env: + FERN_TOKEN: ${{ secrets.DOCS_FERN_TOKEN }} + PR_NUMBER: ${{ steps.metadata.outputs.pr_number }} + working-directory: ./preview-source/docs/fern + run: | + OUTPUT=$(fern generate --docs --preview --id "pr-$PR_NUMBER" --force --no-prompt 2>&1) + echo "$OUTPUT" + URL=$(echo "$OUTPUT" | grep -oP 'Published docs to \K.*(?= \()') + if [ -z "$URL" ]; then + echo "::error::Failed to generate preview URL. See fern output above." + exit 1 + fi + echo "preview_url=$URL" >> "$GITHUB_OUTPUT" + + - name: Build page links for changed MDX files + id: page-links + env: + FERN_TOKEN: ${{ secrets.DOCS_FERN_TOKEN }} + PREVIEW_URL: ${{ steps.generate-docs.outputs.preview_url }} + run: | + CHANGED_FILES="" + if [ -f preview-source/.preview-metadata/changed_mdx_files ]; then + CHANGED_FILES=$(cat preview-source/.preview-metadata/changed_mdx_files) + fi + + if [ -z "$CHANGED_FILES" ] || [ -z "$PREVIEW_URL" ]; then + echo "page_links=" >> "$GITHUB_OUTPUT" + exit 0 + fi + + BASE_URL=$(echo "$PREVIEW_URL" | grep -oP 'https?://[^/]+') + FILES_PARAM=$(echo "$CHANGED_FILES" | tr '\n' ',' | sed 's/,$//' \ + | python3 -c "import sys, urllib.parse; print(urllib.parse.quote(sys.stdin.read().strip(), safe=',/'))") + RESPONSE=$(curl -sf -H "FERN_TOKEN: $FERN_TOKEN" "${PREVIEW_URL}/api/fern-docs/get-slug-for-file?files=${FILES_PARAM}" 2>/dev/null) || { + echo "page_links=" >> "$GITHUB_OUTPUT" + exit 0 + } + + PAGE_LINKS=$(echo "$RESPONSE" | jq -r --arg url "$BASE_URL" \ + '.mappings[] | select(.slug != null) | "- [\(.slug)](\($url)/\(.slug))"') + + if [ -n "$PAGE_LINKS" ]; then + { echo "page_links<> "$GITHUB_OUTPUT" + else + echo "page_links=" >> "$GITHUB_OUTPUT" + fi + + comment: + name: Update pull request comment + needs: preview + runs-on: ubuntu-latest + permissions: + # The comment job receives no Fern token or repository content access. + pull-requests: write # Required to create or update the preview comment. + steps: + - name: Post or update PR comment + env: + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} + PAGE_LINKS: ${{ needs.preview.outputs.page_links }} + PREVIEW_URL: ${{ needs.preview.outputs.preview_url }} + PR_NUMBER: ${{ needs.preview.outputs.pr_number }} + REPOSITORY: ${{ github.repository }} + run: | + BODY=":herb: **Preview your docs:** <${PREVIEW_URL}>" + if [ -n "${PAGE_LINKS}" ]; then + BODY="${BODY} + + Here are the documentation pages you've updated: + ${PAGE_LINKS}" + fi + + MARKER="" + BODY="${BODY} + + ${MARKER}" + + COMMENT_ID=$(gh api "repos/$REPOSITORY/issues/$PR_NUMBER/comments" \ + --jq '.[] | select(.user.login == "github-actions[bot]" and (.body | contains(""))) | .id' \ + | head -1) + + if [ -n "$COMMENT_ID" ]; then + gh api "repos/$REPOSITORY/issues/comments/$COMMENT_ID" \ + -X PATCH -f body="$BODY" + else + gh api "repos/$REPOSITORY/issues/$PR_NUMBER/comments" \ + -f body="$BODY" + fi diff --git a/.github/workflows/getting-started.yml b/.github/workflows/getting-started.yml index 748929ec..7410acf0 100644 --- a/.github/workflows/getting-started.yml +++ b/.github/workflows/getting-started.yml @@ -1,6 +1,6 @@ name: Getting Started -# Executable coverage for docs/getting_started.md. +# Executable coverage for docs/getting_started.mdx. # # Fails when the guide drifts from the product — renamed CLI subcommand, # YAML schema change, broken Python snippet, /health regression, etc. The @@ -10,7 +10,7 @@ name: Getting Started on: pull_request: paths: - - "docs/getting_started.md" + - "docs/getting_started.mdx" - "switchyard/**" - "switchyard_rust/**" - "tests/getting_started/**" @@ -20,7 +20,7 @@ on: push: branches: [main] paths: - - "docs/getting_started.md" + - "docs/getting_started.mdx" - "switchyard/**" - "switchyard_rust/**" - "tests/getting_started/**" @@ -54,7 +54,7 @@ jobs: # the guide. The local mock-upstream fixture lives in # tests/getting_started/conftest.py so it only loads when collecting # from that subtree. - run: uv run pytest tests/getting_started --markdown-docs docs/getting_started.md -v + run: uv run pytest tests/getting_started --markdown-docs docs/getting_started.mdx -v env: # The server-lifecycle test serves a type: model route pointed at a # local in-process mock OpenAI upstream; these keys are never sent diff --git a/.github/workflows/publish-fern-docs.yml b/.github/workflows/publish-fern-docs.yml new file mode 100644 index 00000000..890a74fd --- /dev/null +++ b/.github/workflows/publish-fern-docs.yml @@ -0,0 +1,112 @@ +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# Publishes the Fern site for docs/v* tags or a manual workflow dispatch. +# Required configuration: organization secret DOCS_FERN_TOKEN. + +name: Publish Fern Docs + +on: + push: + tags: + - "docs/v*" + workflow_dispatch: + +permissions: {} + +concurrency: + group: fern-docs-website + cancel-in-progress: false + +jobs: + publish: + name: Publish Fern site + runs-on: ubuntu-latest + permissions: + contents: read + steps: + - name: Checkout repository + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + persist-credentials: false + + - name: Setup Node.js + uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0 + with: + node-version: "24" + package-manager-cache: false + + - name: Install pinned Fern CLI + run: | # zizmor: ignore[adhoc-packages] Fern is exact-version pinned in fern.config.json. + FERN_VERSION="$(jq -r '.version' docs/fern/fern.config.json)" + if ! [[ "$FERN_VERSION" =~ ^[0-9]+\.[0-9]+\.[0-9]+(-[A-Za-z0-9.]+)?$ ]]; then + echo "::error::Refusing to install unexpected Fern CLI version: $FERN_VERSION" + exit 1 + fi + npm install -g "fern-api@${FERN_VERSION}" + + - name: Publish docs + env: + FERN_TOKEN: ${{ secrets.DOCS_FERN_TOKEN }} + working-directory: ./docs/fern + run: fern generate --docs + + - name: Verify custom-domain publication + run: | + for attempt in {1..12}; do + if curl --fail --silent --show-error --location --max-time 20 \ + --output /dev/null https://docs.nvidia.com/nemo/switchyard/home; then + exit 0 + fi + echo "Custom domain is not ready (attempt ${attempt}/12); retrying in 10 seconds." + sleep 10 + done + echo "::error::Fern published, but the custom domain did not become ready." + exit 1 + + - name: Generate legacy GitHub Pages redirects + run: python docs/fern/generate_legacy_redirect_site.py --output legacy-docs-site + + - name: Upload legacy redirects + uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 + with: + name: legacy-docs-site + path: legacy-docs-site/ + include-hidden-files: true + retention-days: 1 + + redirects: + name: Deploy legacy redirects + needs: publish + runs-on: ubuntu-latest + permissions: + # This job receives the generated artifact, but never the Fern token. + actions: read # Required to download redirects generated by the publish job. + contents: write # Required only to replace the gh-pages branch. + steps: + - name: Download legacy redirects + uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 + with: + name: legacy-docs-site + path: legacy-docs-site + + - name: Deploy legacy redirects to GitHub Pages + uses: peaceiris/actions-gh-pages@84c30a85c19949d7eee79c4ff27748b70285e453 # v4 + with: + github_token: ${{ secrets.GITHUB_TOKEN }} + publish_branch: gh-pages + publish_dir: ./legacy-docs-site + keep_files: false + full_commit_message: "docs: redirect GitHub Pages to Fern" diff --git a/AGENTS.md b/AGENTS.md index ab5b7feb..f220c5b9 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -128,7 +128,7 @@ testing, and review do not require loading a skill. |---|---| | `publish-python-release` | Python wheel artifacts, PyPI releases, and release workflow changes | | `switchyard-coding-agent-launchers` | Claude Code, Codex, or OpenClaw launcher behavior | -| `switchyard-docs` | Published MkDocs pages, strict builds, previews, and docs CI | +| `switchyard-docs` | Published Fern docs, `fern check`, previews, and docs CI | | `switchyard-rust-review` | Focused review of Rust, PyO3, async, streaming, and crate boundaries | | `switchyard-stage-router-scorer` | Replaying benchmark trajectories through the stage-router scorer and picker | | `switchyard-testing-ci` | Selecting non-obvious validation or diagnosing CI failures | @@ -258,7 +258,7 @@ and their transitives never appear in downstream vulnerability scans. export OPENAI_API_KEY="sk-..." # or NVIDIA_API_KEY / ANTHROPIC_API_KEY where supported export OPENROUTER_API_KEY="sk-or-..." # pass with --api-key or save via configure -# Serve a routing bundle. Routes live in YAML; see docs/routing_algorithms/overview.md. +# Serve a routing bundle. Routes live in YAML; see docs/routing_algorithms/overview.mdx. switchyard --routing-profiles routes.yaml -- serve --port 4000 # One-command launchers — single-model passthrough via --model @@ -369,7 +369,7 @@ uvicorn.run(build_switchyard_app(switchyard), port=4000) - Export new public classes from `switchyard/__init__.py` with `__all__`. - Write unit tests for new roles and bug fixes. - Use `ProxyContext.metadata` for cross-component state within a request. -- In a new `LLMBackend`, map upstream context-window 4xx to `SwitchyardError::ContextWindowExceeded` (Rust) — the chain executor uses it for evict-and-retry. See [Context-Window Handling](docs/context_window.md). +- In a new `LLMBackend`, map upstream context-window 4xx to `SwitchyardError::ContextWindowExceeded` (Rust) — the chain executor uses it for evict-and-retry. See [Context-Window Handling](docs/operations/context_window.mdx). ### Ask first - Modifying `pyproject.toml` dependencies. diff --git a/CHANGELOG.md b/CHANGELOG.md index 56661edf..cbc3e7e7 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -32,7 +32,7 @@ traffic that sits between client applications and LLM backends. - **Four-role chain** — `RequestProcessor → LLMBackend → ResponseProcessor → TranslationEngine`, executed by the Rust-backed core. See - [Architecture](docs/ARCHITECTURE.md). + [Architecture](docs/architecture.mdx). - **Protocol translation** — convert between OpenAI Chat Completions, Anthropic Messages, and OpenAI Responses wire formats, so each client keeps speaking its native API regardless of the upstream backend. @@ -41,7 +41,7 @@ traffic that sits between client applications and LLM backends. `passthrough`, `random_routing`, `stage_router`, `deterministic` (LLM-as-classifier), `latency_service`, and `noop`. - **Routing strategies** — weighted random split, signal-driven **stage-router** - escalation (see [Stage-Router Routing](docs/stage_router_routing.md)), + escalation (see [Stage-Router Routing](docs/routing_algorithms/stage_router_routing.mdx)), LLM-as-classifier strong/weak routing, and latency-aware multi-endpoint failover. - **One-command launchers** — `switchyard launch claude`, `launch codex`, and @@ -52,7 +52,7 @@ traffic that sits between client applications and LLM backends. `--list-models`), and `verify` / `launch --smoke` round-trip checks. - **Observability** — Prometheus `/metrics`, a JSON `/v1/stats` (`/v1/routing/stats` alias), and per-request cost/token/latency stats. See - [Metrics Reference](docs/METRICS_REFERENCE.md). + [Metrics Reference](docs/internal/metrics_reference.md). - **Python library** — `ProfileSwitchyard` driven by typed profile configs (`PassthroughProfileConfig`, `RandomRoutingProfileConfig`, `StageRouterProfileConfig`, …) and typed `ChatRequest` / `ChatResponse` diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index bc7a44ab..f0fae63e 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -81,6 +81,12 @@ These commands run in CI on every push. **Fix linting errors locally before push uv run ruff check --fix . ``` +For a path-based validation plan, run `python scripts/select_validation.py --changed` (or use +`--base origin/main` after committing). Maintainers adding or changing workflows should also use +the [CI options and workflow design guide](scripts/ci/README.md), which covers required-check +aggregation, fork trust boundaries, concurrency, permissions, immutable action pins, and workflow +security checks. + ### 3. Commit messages Commit messages must follow diff --git a/DEVELOPMENT.md b/DEVELOPMENT.md index 7c92bb70..55460504 100644 --- a/DEVELOPMENT.md +++ b/DEVELOPMENT.md @@ -5,7 +5,7 @@ Switchyard itself. If you only want to **use** the package, see [README](README.md). For deeper architectural docs, see [Agents](AGENTS.md) and -[Architecture](docs/ARCHITECTURE.md). +[Architecture](docs/architecture.mdx). ## Setup diff --git a/README.md b/README.md index 46ae31bb..1e66b46c 100644 --- a/README.md +++ b/README.md @@ -81,7 +81,7 @@ switchyard --routing-profiles routes.yaml -- launch claude > **Bedrock-backed profile caveat (Claude Code + MCP):** Bedrock enforces a 64-character `toolSpec.name` cap. Claude Code's MCP bridge can auto-inject longer tool names, producing `BedrockException` 400s on tool-bearing requests. If you use a Bedrock-backed route and hit this, swap to an OpenAI-compatible model with `--model openai/gpt-4o` or a routing-profile YAML. -See [Agent Launchers](docs/guides/agent_launchers.md) for supported harness +See [Agent Launchers](docs/guides/agent_launchers.mdx) for supported harness versions, model requirements, troubleshooting, and Claude Code `/model` picker aliasing. @@ -133,15 +133,15 @@ LLM clients, targets, and libsy algorithms. See [`crates/switchyard-server/README.md`](crates/switchyard-server/README.md). For profile selection and full configuration examples, start with -[Routing Overview](docs/routing_algorithms/overview.md), then open the +[Routing Overview](docs/routing_algorithms/overview.mdx), then open the strategy-specific page: -- [Random Routing](docs/routing_algorithms/random_routing.md) -- [LLM Classifier Routing](docs/routing_algorithms/llm_classifier_routing.md) -- [Stage-Router Routing](docs/routing_algorithms/stage_router_routing.md) +- [Random Routing](docs/routing_algorithms/random_routing.mdx) +- [LLM Classifier Routing](docs/routing_algorithms/llm_classifier_routing.mdx) +- [Stage-Router Routing](docs/routing_algorithms/stage_router_routing.mdx) For multi-turn classifier sessions, see -[Session Affinity (Sticky Routing)](docs/routing_algorithms/sticky_routing.md). +[Session Affinity (Sticky Routing)](docs/routing_algorithms/sticky_routing.mdx). ### 3. Use as a Python library @@ -184,7 +184,7 @@ flowchart LR Clients keep their native OpenAI or Anthropic API format. Switchyard picks a configured backend, forwards the request in that backend's own format, and translates the response back into the shape the client expects. See -[Architecture](docs/architecture.md) for the system context and the full +[Architecture](docs/architecture.mdx) for the system context and the full request flow. ## Installation Options @@ -207,11 +207,11 @@ See [Installation](INSTALLATION.md) for a full breakdown of what each extra adds ## Documentation -- **[Getting Started](docs/getting_started.md)**: step-by-step setup, first request, troubleshooting -- **[Known Issues](docs/known_issues.md)**: known issues in 0.1.0 -- **[Agent Launchers](docs/guides/agent_launchers.md)**: Claude Code, Codex, and OpenClaw launcher behavior -- **[Cli Reference](docs/cli_reference.md)**: canonical reference for every `switchyard` subcommand and flag -- **[Architecture](docs/architecture.md)**: system context and end-to-end request flow +- **[Getting Started](docs/getting_started.mdx)**: step-by-step setup, first request, troubleshooting +- **[Known Issues](docs/known_issues.mdx)**: known issues in 0.1.0 +- **[Agent Launchers](docs/guides/agent_launchers.mdx)**: Claude Code, Codex, and OpenClaw launcher behavior +- **[Cli Reference](docs/cli_reference.mdx)**: canonical reference for every `switchyard` subcommand and flag +- **[Architecture](docs/architecture.mdx)**: system context and end-to-end request flow - **[Routing Algorithms](docs/routing_algorithms/)**: signal-driven weak/strong stage-router routing: picker layers, signal dimensions, and calibration data. - **[Contributing](CONTRIBUTING.md)**: dev setup, testing, CI gates, PR process - **[Development](DEVELOPMENT.md)**: project structure, benchmarks, conventions diff --git a/docs/AGENTS.md b/docs/AGENTS.md new file mode 100644 index 00000000..6a312742 --- /dev/null +++ b/docs/AGENTS.md @@ -0,0 +1,48 @@ +# Documentation instructions + +These instructions apply to everything under `docs/`. More specific rules in +`docs/fern/AGENTS.md` also apply inside `docs/fern/`. + +## Before editing + +- Read `.agents/skills/switchyard-docs/SKILL.md` completely. +- Read `.agents/skills/switchyard-codebase-exploration/SKILL.md` when documentation cites code, + commands, configuration, or public APIs. +- Treat `docs/fern/versions/nightly.yml` as the source of truth for the published page set, order, + labels, and routes. + +## Content boundaries + +- Published pages are `docs/**/*.mdx`, excluding `docs/fern/`. +- `docs/internal/**/*.md` files are unpublished. Do not add them to Fern navigation or link to them + from published pages unless the task explicitly promotes that content. +- Keep contributor guidance in `README.md` and `AGENTS.md`; do not add either file to Fern + navigation. +- Use public `switchyard` imports in examples, not `switchyard.lib.*` internals. +- Link repository files outside `docs/` with absolute GitHub URLs. + +## Fern authoring + +- Give every published page non-empty `title` and `description` frontmatter. +- Keep navigation sections label-only and place authored pages under `contents:`. Do not assign the + same MDX page to both a section `path:` and a child page. +- Use canonical, version-agnostic Fern routes derived from navigation labels for internal links. +- Do not link unpublished internal notes from published MDX. +- Use Fern callout components instead of GitHub admonition syntax. + +## Validation + +Run the smallest relevant checks after every docs change: + +```bash +cd docs && make check +uv run pytest tests/test_fern_docs.py -v +``` + +Before pushing, also follow the repository-wide validation requirements in the root `AGENTS.md`. +If a docs workflow, route invariant, or ownership boundary changes, update +`.agents/skills/switchyard-docs/SKILL.md` in the same change. + +Fern validation is part of the required `CI Success` aggregate. Keep preview concurrency scoped to +the PR number, and never trust PR-artifact metadata for the target PR or Fern CLI version. Hosted +previews are limited to same-repository branches; fork PRs receive validation only. diff --git a/docs/Makefile b/docs/Makefile index c710ed75..ed98c7a1 100644 --- a/docs/Makefile +++ b/docs/Makefile @@ -1,25 +1,19 @@ -.PHONY: env html publish live clean help +.PHONY: check preview clean help -REPO_ROOT := $(dir $(abspath $(lastword $(MAKEFILE_LIST)))).. +FERN_DIR := $(dir $(abspath $(lastword $(MAKEFILE_LIST))))fern +FERN_VERSION := $(shell jq -r .version $(FERN_DIR)/fern.config.json) +FERN := npx -y fern-api@$(FERN_VERSION) help: - @echo "make env : sync the docs dependency group via uv" - @echo "make html : build HTML site (incremental)" - @echo "make publish : build HTML site (strict, fails on warnings)" - @echo "make live : live-reload server at http://127.0.0.1:8000" - @echo "make clean : remove build artefacts" + @echo "make check : validate Fern configuration and content" + @echo "make preview : serve a local Fern preview" + @echo "make clean : remove local Fern build artifacts" -env: - cd $(REPO_ROOT) && uv sync --only-group docs +check: + cd $(FERN_DIR) && $(FERN) check -html: env - cd $(REPO_ROOT) && uv run --only-group docs mkdocs build - -publish: env - cd $(REPO_ROOT) && uv run --only-group docs mkdocs build --strict - -live: env - cd $(REPO_ROOT) && uv run --only-group docs mkdocs serve +preview: + cd $(FERN_DIR) && $(FERN) docs dev clean: - rm -rf $(REPO_ROOT)/site + rm -rf $(FERN_DIR)/.fern $(FERN_DIR)/node_modules diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 00000000..b384f39d --- /dev/null +++ b/docs/README.md @@ -0,0 +1,53 @@ +# NeMo Switchyard docs + +Switchyard documentation is authored as [Fern](https://buildwithfern.com/) MDX in this directory. +The current Fern site is available at +**[nemo-switchyard.docs.buildwithfern.com/nemo/switchyard](https://nemo-switchyard.docs.buildwithfern.com/nemo/switchyard/home)**. + +## Where to make changes + +| Change | Location | +|---|---| +| Published page | `docs/**/*.mdx` | +| Public navigation | [`fern/versions/nightly.yml`](fern/versions/nightly.yml) | +| Site settings and redirects | [`fern/docs.yml`](fern/docs.yml) | +| Internal, unpublished notes | `docs/internal/**/*.md` | +| Fern build and publishing details | [`fern/README.md`](fern/README.md) | + +Markdown files under `internal/` are intentionally excluded from the public navigation. Promote one +to MDX only when it is ready to become public documentation. + +## Common commands + +Run these from the repository root: + +```bash +cd docs && make check +uv run pytest tests/test_fern_docs.py -v +cd docs && make preview +``` + +`make check` uses the Fern CLI version pinned in +[`fern/fern.config.json`](fern/fern.config.json). `make preview` starts a local server and does not +publish the site. + +## Add or edit a page + +1. Add or update the MDX page under `docs/`. +2. Add new pages to [`fern/versions/nightly.yml`](fern/versions/nightly.yml) in the intended order. +3. Use canonical Fern routes derived from the navigation labels for internal links. +4. Run the Fern check and route regression tests above. + +Every pull request runs Fern validation inside the required `CI Success` aggregate. Docs changes +from repository branches also start a PR-numbered hosted preview: the collector uploads content +without secrets, while the trusted workflow derives PR identity and the Fern CLI version from +GitHub/default-branch state before publishing. Preview generation and PR commenting use separate +permission scopes. Fork PRs receive Fern validation but do not publish hosted previews. + +Legacy GitHub Pages paths are handled in two places: Fern redirects live in +[`fern/docs.yml`](fern/docs.yml), while +[`fern/generate_legacy_redirect_site.py`](fern/generate_legacy_redirect_site.py) builds the static +redirect-only GitHub Pages site. Keep both mappings synchronized. + +For agent-specific editing rules, see [`AGENTS.md`](AGENTS.md) and the +[`switchyard-docs`](../.agents/skills/switchyard-docs/SKILL.md) skill. diff --git a/docs/architecture.md b/docs/architecture.mdx similarity index 88% rename from docs/architecture.md rename to docs/architecture.mdx index ccf0a46d..953f1d70 100644 --- a/docs/architecture.md +++ b/docs/architecture.mdx @@ -1,5 +1,8 @@ -# Switchyard Architecture - +--- +title: "Architecture" +description: "Understand how NeMo Switchyard routes requests, translates provider formats, and handles responses end to end." +position: 2 +--- Switchyard is an LLM traffic proxy that sits between clients and model backends. It keeps the client-facing API stable while applying routing policy, translating provider formats, and handling configured fallbacks. @@ -36,7 +39,7 @@ flowchart TB Routing policy determines which model or endpoint receives a request. Depending on the selected strategy, that decision can use fixed weights, a classifier, request signals, or conversation affinity. See the -[Routing Overview](routing_algorithms/overview.md) for the available strategies. +[Routing Overview](/routing/overview) for the available strategies. ## Backend Wire Format @@ -97,7 +100,7 @@ upstream format. > `SWITCHYARD_STARTUP_TIMING=1`), which prints each probe on its own line. ## Related Documentation -- [Getting Started](getting_started.md): install Switchyard and run a first request -- [Agent Launchers](guides/agent_launchers.md): run coding agents through a local proxy -- [Routing Overview](routing_algorithms/overview.md): choose and configure a routing strategy -- [CLI Reference](cli_reference.md): configure and operate Switchyard from the command line +- [Getting Started with Switchyard](/get-started/getting-started): install Switchyard and run a first request +- [Agent Launchers](/guides/agent-launchers): run coding agents through a local proxy +- [Routing Overview](/routing/overview): choose and configure a routing strategy +- [CLI Reference](/reference/cli-reference): configure and operate Switchyard from the command line diff --git a/docs/cli_reference.md b/docs/cli_reference.mdx similarity index 96% rename from docs/cli_reference.md rename to docs/cli_reference.mdx index bc4df727..9e38c709 100644 --- a/docs/cli_reference.md +++ b/docs/cli_reference.mdx @@ -1,6 +1,9 @@ -# CLI Reference - -This page is the canonical reference for every `switchyard` subcommand. It mirrors the output of `switchyard --help` and `switchyard --help`. If you spot drift, file a docs ticket. Tutorials and recipes live in [Getting Started](getting_started.md); this page is reference material only. +--- +title: "CLI Reference" +description: "Reference every NeMo Switchyard command, option, configuration path, and environment variable." +position: 1 +--- +This page is the canonical reference for every `switchyard` subcommand. It mirrors the output of `switchyard --help` and `switchyard --help`. If you spot drift, file a docs ticket. Tutorials and recipes live in [Getting Started with Switchyard](/get-started/getting-started); this page is reference material only. ## Verbs at a glance @@ -504,16 +507,17 @@ switchyard configure --disable-skill-distillation switchyard configure --reset ``` -!!! note "Non-interactive / CI usage" - **`--target all` (the default) requires launcher models in no-TTY mode.** Without a - TTY or `--claude-model` / `--codex-model` / `--openclaw-model`, the command exits with - `No Claude model configured or discovered`. Pass `--target provider` to save only - provider credentials, or supply explicit `--claude-model` / `--codex-model` / `--openclaw-model` values. + +**`--target all` (the default) requires launcher models in no-TTY mode.** Without a +TTY or `--claude-model` / `--codex-model` / `--openclaw-model`, the command exits with +`No Claude model configured or discovered`. Pass `--target provider` to save only +provider credentials, or supply explicit `--claude-model` / `--codex-model` / `--openclaw-model` values. - **`configure` requires an explicit `--api-key` flag.** It does not read `OPENROUTER_API_KEY` - (or any other `*_API_KEY` environment variable) and does not read `api_key` from a - routing-profile bundle (`--routing-profiles`). Always pass `--api-key` when running - non-interactively. +**`configure` requires an explicit `--api-key` flag.** It does not read `OPENROUTER_API_KEY` +(or any other `*_API_KEY` environment variable) and does not read `api_key` from a +routing-profile bundle (`--routing-profiles`). Always pass `--api-key` when running +non-interactively. + ## `switchyard verify` @@ -571,5 +575,5 @@ switchyard verify --api-key "$OPENROUTER_API_KEY" --base-url https://openrouter. ## See also -- [Getting Started](getting_started.md): install Switchyard and run your first request. -- [Architecture](architecture.md): system context and end-to-end request flow. +- [Getting Started with Switchyard](/get-started/getting-started): install Switchyard and run your first request. +- [Switchyard Architecture](/concepts/architecture): system context and end-to-end request flow. diff --git a/docs/core_concepts.md b/docs/core_concepts.mdx similarity index 73% rename from docs/core_concepts.md rename to docs/core_concepts.mdx index 3e452ab8..426f5d25 100644 --- a/docs/core_concepts.md +++ b/docs/core_concepts.mdx @@ -1,8 +1,11 @@ -# Core Concepts - +--- +title: "Core Concepts" +description: "Learn NeMo Switchyard routes, targets, routing tiers, model IDs, and format translation." +position: 1 +--- This page explains the vocabulary used throughout the documentation. For setup, -see [Getting Started](getting_started.md). For the request lifecycle, see -[Architecture](architecture.md). +see [Getting Started with Switchyard](/get-started/getting-started). For the request lifecycle, see +[Switchyard Architecture](/concepts/architecture). ## How it fits together @@ -30,7 +33,7 @@ client-facing name and a `type` that chooses its behavior. Depending on that type, the route owns one target, two routing tiers, or a larger endpoint list. Shared `defaults:` can provide a base URL, API key, and backend format without -repeating them in every target. The [Routing Overview](routing_algorithms/overview.md) +repeating them in every target. The [Routing Overview](/routing/overview) has a complete runnable bundle. ## Model IDs @@ -52,12 +55,12 @@ A route's `type` sets the strategy. `model` calls one target. classifier model to pick a tier. `stage_router` uses agent-progress signals, with an optional classifier fallback. `escalation_router` starts on the weak tier and uses an LLM judge to escalate when needed. The -[Routing Overview](routing_algorithms/overview.md) explains when to use each. +[Routing Overview](/routing/overview) explains when to use each. Session affinity pins a conversation to one tier so later turns reuse it instead of being classified again. It belongs to deterministic classifier routing and is not a strategy of its own. Random and stage-router routing make -a decision for every request. [Sticky Routing](routing_algorithms/sticky_routing.md) +a decision for every request. [Sticky Routing](/routing/sticky-routing) covers affinity in full. ## Formats and translation @@ -69,7 +72,7 @@ Switchyard translates the request on the way out and the response on the way back. That translation lets Claude Code, which speaks Anthropic Messages, run against -an OpenAI-compatible model. The [Architecture](architecture.md) page documents +an OpenAI-compatible model. The [Switchyard Architecture](/concepts/architecture) page documents the supported formats and request lifecycle. ## Programmatic Python profiles @@ -84,12 +87,12 @@ and request metadata, while the profile protocols define `run`, `process`, and The `switchyard-server` binary is built directly on libsy. Its TOML file explicitly defines LLM clients, targets, and algorithm routes; it does not load Python route bundles. See the -[Rust server README](../crates/switchyard-server/README.md) for details. +[Rust server README](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/switchyard-server/README.md) for details. ## Where to go next -- [Getting Started](getting_started.md) to install and send a first request. -- [Routing Overview](routing_algorithms/overview.md) to choose and tune a strategy. -- [Agent Launchers](guides/agent_launchers.md) to run Claude Code, Codex, or OpenClaw. -- [Architecture](architecture.md) to see a request travel end to end. -- [CLI Reference](cli_reference.md) for flags and environment variables. +- [Getting Started with Switchyard](/get-started/getting-started) to install and send a first request. +- [Routing Overview](/routing/overview) to choose and tune a strategy. +- [Agent Launchers](/guides/agent-launchers) to run Claude Code, Codex, or OpenClaw. +- [Switchyard Architecture](/concepts/architecture) to see a request travel end to end. +- [CLI Reference](/reference/cli-reference) for flags and environment variables. diff --git a/docs/fern/.gitignore b/docs/fern/.gitignore new file mode 100644 index 00000000..ff9bdf6f --- /dev/null +++ b/docs/fern/.gitignore @@ -0,0 +1,2 @@ +.fern/ +node_modules/ diff --git a/docs/fern/AGENTS.md b/docs/fern/AGENTS.md new file mode 100644 index 00000000..b0982045 --- /dev/null +++ b/docs/fern/AGENTS.md @@ -0,0 +1,49 @@ +# Fern infrastructure instructions + +These instructions apply to `docs/fern/` in addition to `docs/AGENTS.md` and the repository root +`AGENTS.md`. + +## Safety + +- Read `.agents/skills/switchyard-docs/SKILL.md` before changing Fern configuration, navigation, + redirects, or workflows. +- `cd docs && make check` and `cd docs && make preview` are local-only. +- A hosted Fern preview and production publication both change external state. Run either only when + the user explicitly asks. +- Never expose `DOCS_FERN_TOKEN` to a `pull_request` job or check out untrusted PR code in the + trusted `workflow_run` job. +- Derive the preview PR number from the trusted `workflow_run` payload and the Fern CLI version from + the default branch. PR artifacts may provide docs content and changed-file hints only. +- Reject fork-originated runs before collecting or processing preview artifacts. Fork PRs receive + required Fern validation but no secret-bearing hosted preview. +- Keep previews PR-numbered and cancel superseded runs. Serialize production publishing, and + isolate repository write permission from Fern CLI execution. + +## Site structure + +- Keep published nightly MDX at `docs/**/*.mdx`, outside `docs/fern/`. +- Do not create `docs/fern/versions/nightly/pages/`. +- Keep `versions/nightly.yml` paths relative to that file; current authored pages use `../../`. +- Keep Switchyard navigation sections label-only so pages resolve as `/section/page`. +- In `docs.yml`, use full `/nemo/switchyard/...` paths for redirect sources and destinations. + +## Redirect ownership + +- Fern redirects in `docs.yml` handle URLs that already reach the Fern domain. +- `generate_legacy_redirect_site.py` handles the former GitHub Pages domain. +- When a legacy route changes, update both redirect owners and the regression expectations in + `tests/test_fern_docs.py` when needed. +- Do not replace the old Pages site until the publishing workflow has verified the Fern custom + domain home page. + +## Validation + +After changing anything in this directory, run: + +```bash +cd docs && make check +uv run pytest tests/test_fern_docs.py -v +``` + +Before pushing, follow the root `AGENTS.md` requirements. Update +`.agents/skills/switchyard-docs/SKILL.md` whenever these workflows or invariants change. diff --git a/docs/fern/README.md b/docs/fern/README.md new file mode 100644 index 00000000..1ca842b8 --- /dev/null +++ b/docs/fern/README.md @@ -0,0 +1,79 @@ +# NeMo Switchyard Fern docs + +This directory contains the Fern configuration, navigation, redirect tooling, and CI-facing inputs +for the [NeMo Switchyard documentation](https://nemo-switchyard.docs.buildwithfern.com/nemo/switchyard/home). +Published MDX pages live one level up under `docs/`; they are not duplicated under +`docs/fern/versions/nightly/pages/`. + +## Layout + +```text +docs/ +├── Makefile # pinned local check and preview commands +├── **/*.mdx # published documentation pages +├── internal/**/*.md # unpublished project notes +└── fern/ + ├── README.md # this guide + ├── AGENTS.md # scoped agent instructions + ├── fern.config.json # Fern organization and CLI version pin + ├── docs.yml # instance, theme, versions, and redirects + ├── generate_legacy_redirect_site.py + └── versions/nightly.yml # public navigation and page paths +``` + +## Local development + +Run the supported commands from the repository root: + +```bash +cd docs && make check # validate Fern configuration, navigation, links, and MDX +cd docs && make preview # serve a local preview at http://localhost:3000 +cd docs && make clean # remove local Fern artifacts +``` + +The Makefile reads the CLI version from `fern.config.json` and runs it with `npx`, so a global Fern +installation is not required. + +Run the route and redirect regression tests separately: + +```bash +uv run pytest tests/test_fern_docs.py -v +``` + +## Navigation and routes + +`versions/nightly.yml` defines the published page set and sidebar. Its page paths point back to MDX +files under `docs/` with `../../` paths. Navigation labels determine the public slugs; filenames do +not. + +Keep sections label-only and put pages under `contents:`. Assigning an MDX file to a section and a +child page changes the route Fern publishes and can make the expected child URL return 404. + +## Redirects + +`docs.yml` owns redirects for requests that reach the Fern domain. Use complete +`/nemo/switchyard/...` source and destination paths and add exact mappings when legacy filenames or +section names do not match current slugs. + +`generate_legacy_redirect_site.py` produces the redirect-only site served from the former GitHub +Pages domain. Keep its redirect map aligned with `docs.yml`; tests enforce the shared mappings. + +## CI and publishing + +| Workflow | Purpose | +|---|---| +| `.github/workflows/fern-docs-ci.yml` | Reusable `fern check` gate called by required repository CI | +| `.github/workflows/fern-docs-preview-build.yml` | Collect same-repository PR docs without secrets | +| `.github/workflows/fern-docs-preview-comment.yml` | Build a trusted PR-numbered preview, then comment with separate permissions | +| `.github/workflows/publish-fern-docs.yml` | Serialize tagged/manual publishing and deploy redirects in a write-only job | + +Hosted previews and production publishing require the `DOCS_FERN_TOKEN` organization secret. Do +not run a secret-bearing preview from a `pull_request` job, and do not publish production docs from +an ordinary merge to `main`. The trusted preview reads PR identity from `workflow_run`, reads the +Fern version from the default branch, and processes the downloaded PR docs in an isolated +directory. Fork PRs stop after required Fern validation. Preview runs are cancelled when +superseded; production runs are serialized. + +For content authoring, start with [`../README.md`](../README.md). For scoped agent rules, see +[`AGENTS.md`](AGENTS.md) and the +[`switchyard-docs`](../../.agents/skills/switchyard-docs/SKILL.md) skill. diff --git a/docs/fern/docs.yml b/docs/fern/docs.yml new file mode 100644 index 00000000..6a60a95e --- /dev/null +++ b/docs/fern/docs.yml @@ -0,0 +1,104 @@ +# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +instances: + - url: nemo-switchyard.docs.buildwithfern.com/nemo/switchyard + custom-domain: docs.nvidia.com/nemo/switchyard + multi-source: true + +title: NVIDIA NeMo Switchyard + +global-theme: nvidia + +logo: + href: /nemo/switchyard/home + right-text: NeMo Switchyard + +navbar-links: + - type: github + value: https://github.com/NVIDIA-NeMo/Switchyard + +versions: + - display-name: nightly + path: versions/nightly.yml + slug: nightly + +redirects: + # Root URLs from the former GitHub Pages site. + - source: "/nemo/switchyard/index.html" + destination: "/nemo/switchyard/home" + - source: "/nemo/switchyard/index" + destination: "/nemo/switchyard/home" + + # Section landing URLs from the initial Fern preview. Sections are now + # label-only so every authored page has a stable /section/page route. + - source: "/nemo/switchyard/get-started" + destination: "/nemo/switchyard/get-started/getting-started" + - source: "/nemo/switchyard/guides" + destination: "/nemo/switchyard/guides/agent-launchers" + - source: "/nemo/switchyard/concepts" + destination: "/nemo/switchyard/concepts/core-concepts" + - source: "/nemo/switchyard/routing" + destination: "/nemo/switchyard/routing/overview" + - source: "/nemo/switchyard/operations" + destination: "/nemo/switchyard/operations/context-window-handling" + - source: "/nemo/switchyard/reference" + destination: "/nemo/switchyard/reference/cli-reference" + + # Exact mappings from MkDocs directory URLs to Fern navigation slugs. + - source: "/nemo/switchyard/getting_started" + destination: "/nemo/switchyard/get-started/getting-started" + - source: "/nemo/switchyard/getting_started/index.html" + destination: "/nemo/switchyard/get-started/getting-started" + - source: "/nemo/switchyard/known_issues" + destination: "/nemo/switchyard/get-started/known-issues-in-0-1-0" + - source: "/nemo/switchyard/known_issues/index.html" + destination: "/nemo/switchyard/get-started/known-issues-in-0-1-0" + - source: "/nemo/switchyard/guides/agent_launchers" + destination: "/nemo/switchyard/guides/agent-launchers" + - source: "/nemo/switchyard/guides/agent_launchers/index.html" + destination: "/nemo/switchyard/guides/agent-launchers" + - source: "/nemo/switchyard/skill_distillation" + destination: "/nemo/switchyard/guides/skill-distillation" + - source: "/nemo/switchyard/skill_distillation/index.html" + destination: "/nemo/switchyard/guides/skill-distillation" + - source: "/nemo/switchyard/core_concepts" + destination: "/nemo/switchyard/concepts/core-concepts" + - source: "/nemo/switchyard/core_concepts/index.html" + destination: "/nemo/switchyard/concepts/core-concepts" + - source: "/nemo/switchyard/architecture" + destination: "/nemo/switchyard/concepts/architecture" + - source: "/nemo/switchyard/architecture/index.html" + destination: "/nemo/switchyard/concepts/architecture" + - source: "/nemo/switchyard/routing_algorithms/overview" + destination: "/nemo/switchyard/routing/overview" + - source: "/nemo/switchyard/routing_algorithms/overview/index.html" + destination: "/nemo/switchyard/routing/overview" + - source: "/nemo/switchyard/routing_algorithms/random_routing" + destination: "/nemo/switchyard/routing/random-routing" + - source: "/nemo/switchyard/routing_algorithms/random_routing/index.html" + destination: "/nemo/switchyard/routing/random-routing" + - source: "/nemo/switchyard/routing_algorithms/llm_classifier_routing" + destination: "/nemo/switchyard/routing/llm-classifier-routing" + - source: "/nemo/switchyard/routing_algorithms/llm_classifier_routing/index.html" + destination: "/nemo/switchyard/routing/llm-classifier-routing" + - source: "/nemo/switchyard/routing_algorithms/sticky_routing" + destination: "/nemo/switchyard/routing/sticky-routing" + - source: "/nemo/switchyard/routing_algorithms/sticky_routing/index.html" + destination: "/nemo/switchyard/routing/sticky-routing" + - source: "/nemo/switchyard/routing_algorithms/stage_router_routing" + destination: "/nemo/switchyard/routing/stage-router-routing" + - source: "/nemo/switchyard/routing_algorithms/stage_router_routing/index.html" + destination: "/nemo/switchyard/routing/stage-router-routing" + - source: "/nemo/switchyard/routing_algorithms/escalation_router_routing" + destination: "/nemo/switchyard/routing/escalation-router-routing" + - source: "/nemo/switchyard/routing_algorithms/escalation_router_routing/index.html" + destination: "/nemo/switchyard/routing/escalation-router-routing" + - source: "/nemo/switchyard/operations/context_window" + destination: "/nemo/switchyard/operations/context-window-handling" + - source: "/nemo/switchyard/operations/context_window/index.html" + destination: "/nemo/switchyard/operations/context-window-handling" + - source: "/nemo/switchyard/cli_reference" + destination: "/nemo/switchyard/reference/cli-reference" + - source: "/nemo/switchyard/cli_reference/index.html" + destination: "/nemo/switchyard/reference/cli-reference" diff --git a/docs/fern/fern.config.json b/docs/fern/fern.config.json new file mode 100644 index 00000000..d859bfbe --- /dev/null +++ b/docs/fern/fern.config.json @@ -0,0 +1,4 @@ +{ + "organization": "nvidia", + "version": "5.75.7" +} diff --git a/docs/fern/generate_legacy_redirect_site.py b/docs/fern/generate_legacy_redirect_site.py new file mode 100644 index 00000000..46ec6ae0 --- /dev/null +++ b/docs/fern/generate_legacy_redirect_site.py @@ -0,0 +1,79 @@ +# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +"""Generate static redirects from the former GitHub Pages site to Fern.""" + +from __future__ import annotations + +import argparse +import html +import json +from pathlib import Path + +CANONICAL_BASE = "https://docs.nvidia.com/nemo/switchyard" + +LEGACY_REDIRECTS = { + "": "/home", + "getting_started": "/get-started/getting-started", + "known_issues": "/get-started/known-issues-in-0-1-0", + "guides/agent_launchers": "/guides/agent-launchers", + "skill_distillation": "/guides/skill-distillation", + "core_concepts": "/concepts/core-concepts", + "architecture": "/concepts/architecture", + "routing_algorithms/overview": "/routing/overview", + "routing_algorithms/random_routing": "/routing/random-routing", + "routing_algorithms/llm_classifier_routing": "/routing/llm-classifier-routing", + "routing_algorithms/sticky_routing": "/routing/sticky-routing", + "routing_algorithms/stage_router_routing": "/routing/stage-router-routing", + "routing_algorithms/escalation_router_routing": "/routing/escalation-router-routing", + "operations/context_window": "/operations/context-window-handling", + "cli_reference": "/reference/cli-reference", +} + + +def render_redirect_page(destination: str) -> str: + """Return a static redirect page for one canonical Fern destination.""" + target = f"{CANONICAL_BASE}{destination}" + escaped_target = html.escape(target, quote=True) + javascript_target = json.dumps(target) + return f""" + + + + + + + NeMo Switchyard documentation moved + + + +

This documentation moved to {escaped_target}.

+ + +""" + + +def generate_redirect_site(output: Path) -> None: + """Write one GitHub Pages redirect document for every former MkDocs route.""" + output.mkdir(parents=True, exist_ok=True) + (output / ".nojekyll").touch() + for legacy_path, destination in LEGACY_REDIRECTS.items(): + page = output / legacy_path / "index.html" + page.parent.mkdir(parents=True, exist_ok=True) + page.write_text(render_redirect_page(destination), encoding="utf-8") + + +def main() -> None: + """Parse CLI arguments and generate the redirect-only GitHub Pages site.""" + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--output", type=Path, required=True) + args = parser.parse_args() + generate_redirect_site(args.output) + + +if __name__ == "__main__": + main() diff --git a/docs/fern/versions/nightly.yml b/docs/fern/versions/nightly.yml new file mode 100644 index 00000000..9fb04b53 --- /dev/null +++ b/docs/fern/versions/nightly.yml @@ -0,0 +1,46 @@ +# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +navigation: + - page: "Home" + path: ../../index.mdx + - section: "Get Started" + contents: + - page: "Getting Started" + path: ../../getting_started.mdx + - page: "Known Issues in 0.1.0" + path: ../../known_issues.mdx + - section: "Guides" + contents: + - page: "Agent Launchers" + path: ../../guides/agent_launchers.mdx + - page: "Skill Distillation" + path: ../../skill_distillation.mdx + - section: "Concepts" + contents: + - page: "Core Concepts" + path: ../../core_concepts.mdx + - page: "Architecture" + path: ../../architecture.mdx + - section: "Routing" + contents: + - page: "Overview" + path: ../../routing_algorithms/overview.mdx + - page: "Random Routing" + path: ../../routing_algorithms/random_routing.mdx + - page: "LLM Classifier Routing" + path: ../../routing_algorithms/llm_classifier_routing.mdx + - page: "Sticky Routing" + path: ../../routing_algorithms/sticky_routing.mdx + - page: "Stage-Router Routing" + path: ../../routing_algorithms/stage_router_routing.mdx + - page: "Escalation-Router Routing" + path: ../../routing_algorithms/escalation_router_routing.mdx + - section: "Operations" + contents: + - page: "Context-Window Handling" + path: ../../operations/context_window.mdx + - section: "Reference" + contents: + - page: "CLI Reference" + path: ../../cli_reference.mdx diff --git a/docs/getting_started.md b/docs/getting_started.mdx similarity index 82% rename from docs/getting_started.md rename to docs/getting_started.mdx index d0b645f3..a215da87 100644 --- a/docs/getting_started.md +++ b/docs/getting_started.mdx @@ -1,11 +1,14 @@ -# Getting Started with Switchyard - +--- +title: "Getting Started" +description: "Install NeMo Switchyard, configure an LLM provider, start the proxy, and send your first routed request." +position: 1 +--- ## Prerequisites - Python 3.12 or later - macOS, Linux, or Windows - An API key for OpenRouter, OpenAI, Anthropic, or another OpenAI-compatible endpoint. - To use OpenRouter, create an account at [openrouter.ai](https://openrouter.ai/) + To use OpenRouter, create an account at [openrouter.ai](https://openrouter.ai) and generate a key from the [OpenRouter keys page](https://openrouter.ai/keys). ## Install @@ -115,7 +118,7 @@ switchyard launch openclaw # OpenClaw ``` Each launcher reads the routing bundle and provider credentials saved by -`switchyard configure`. See [Agent Launchers](guides/agent_launchers.md) for +`switchyard configure`. See [Agent Launchers](/guides/agent-launchers) for supported harness versions, model requirements, and Claude Code `/model` picker aliasing. @@ -142,17 +145,17 @@ another route type when the routing decision needs different inputs: | Algorithm | Use it when | Config | |---|---|---| -| [Random Routing](routing_algorithms/random_routing.md) | You need a fixed strong/weak split for A/B tests or baselines. | `random_routing` | -| [LLM Classifier Routing](routing_algorithms/llm_classifier_routing.md) | Request content should decide whether to use `weak` or `strong`. | `deterministic` | -| [Stage-Router Routing](routing_algorithms/stage_router_routing.md) | Tool-result and progress signals should route most turns without an extra classifier call. | `stage_router` | +| [Random Routing](/routing/random-routing) | You need a fixed strong/weak split for A/B tests or baselines. | `random_routing` | +| [LLM Classifier Routing](/routing/llm-classifier-routing) | Request content should decide whether to use `weak` or `strong`. | `deterministic` | +| [Stage-Router Routing](/routing/stage-router-routing) | Tool-result and progress signals should route most turns without an extra classifier call. | `stage_router` | LLM classifier routes can also enable -[Session Affinity (Sticky Routing)](routing_algorithms/sticky_routing.md) to pin +[Session Affinity (Sticky Routing)](/routing/sticky-routing) to pin multi-turn conversations to one tier. A single YAML file can declare multiple routes. Each route becomes a model id on `GET /v1/models`; the first declared route is the launcher's initial model. See -[Routing Overview](routing_algorithms/overview.md) for route selection and the +[Routing Overview](/routing/overview) for route selection and the strategy-specific pages for full examples and tuning notes. --- @@ -241,11 +244,11 @@ uv run mypy switchyard ## Next steps -- [CLI Reference](cli_reference.md): full flag reference for every verb -- [Agent Launchers](guides/agent_launchers.md): Claude Code, Codex, and OpenClaw launcher details -- [Architecture](architecture.md): system context and end-to-end request flow -- [Routing Overview](routing_algorithms/overview.md): choose the right routing strategy -- [Random Routing](routing_algorithms/random_routing.md): fixed strong/weak split routing -- [LLM Classifier Routing](routing_algorithms/llm_classifier_routing.md): classifier-driven strong/weak routing -- [Stage-Router Routing](routing_algorithms/stage_router_routing.md): picker layers, signal dimensions, calibration -- [Sticky Routing](routing_algorithms/sticky_routing.md): conversation-level route affinity +- [CLI Reference](/reference/cli-reference): full flag reference for every verb +- [Agent Launchers](/guides/agent-launchers): Claude Code, Codex, and OpenClaw launcher details +- [Switchyard Architecture](/concepts/architecture): system context and end-to-end request flow +- [Routing Overview](/routing/overview): choose the right routing strategy +- [Random Routing](/routing/random-routing): fixed strong/weak split routing +- [LLM Classifier Routing](/routing/llm-classifier-routing): classifier-driven strong/weak routing +- [Stage-Router Routing](/routing/stage-router-routing): picker layers, signal dimensions, calibration +- [Sticky Routing](/routing/sticky-routing): conversation-level route affinity diff --git a/docs/guides/agent_launchers.md b/docs/guides/agent_launchers.mdx similarity index 97% rename from docs/guides/agent_launchers.md rename to docs/guides/agent_launchers.mdx index f22562c7..653d6fbd 100644 --- a/docs/guides/agent_launchers.md +++ b/docs/guides/agent_launchers.mdx @@ -1,5 +1,8 @@ -# Agent Launchers - +--- +title: "Agent Launchers" +description: "Launch Claude Code, Codex, and OpenClaw through a local NeMo Switchyard proxy." +position: 1 +--- Switchyard launchers start a local proxy, configure the target coding agent to use it, and shut the proxy down when the agent exits. Use them when you want Claude Code, Codex, or OpenClaw to run through the same routing and translation diff --git a/docs/index.md b/docs/index.mdx similarity index 76% rename from docs/index.md rename to docs/index.mdx index 3801dd0a..3a4104bc 100644 --- a/docs/index.md +++ b/docs/index.mdx @@ -1,5 +1,8 @@ -# Switchyard Documentation - +--- +title: "Home" +description: "Route, translate, and operate LLM traffic across OpenAI, Anthropic, and compatible backends with NVIDIA NeMo Switchyard." +position: 1 +--- Switchyard is a typed control plane for LLM traffic. It sits between client applications and model backends, translates OpenAI Chat / Anthropic Messages / OpenAI Responses formats, and routes each request through profile-backed chains. @@ -27,7 +30,7 @@ clients -> compatible API surface -> routing and resilience -> model backends ``` For system context and request lifecycle diagrams, see -[Architecture](architecture.md). +[Switchyard Architecture](/concepts/architecture). ## First Run @@ -38,46 +41,29 @@ switchyard launch claude ``` For source installs, non-interactive configuration, and a curl sanity check, use -[Getting Started](getting_started.md). +[Getting Started with Switchyard](/get-started/getting-started). ## Main Workflows -
- -- **Run coding agents** - + + Launch Claude Code, Codex, or OpenClaw through a local Switchyard proxy. - - [Agent Launchers](guides/agent_launchers.md) - -- **Configure routing** - + + Pick between fixed splits, classifier routing, and stage-router routing, with optional session affinity for classifier-driven conversations. - - [Routing Overview](routing_algorithms/overview.md) - -- **Prepare skill distillation** - + + Save a namespace for the intended automatic skill learning flow without changing request routing. - - [Skill Distillation](skill_distillation.md) - -- **Understand the system** - - See how clients, routing policy, model backends, and operations fit - together. - - [Architecture](architecture.md) - -- **Operate the proxy** - + + + See how clients, routing policy, model backends, and operations fit together. + + Understand context-window overflow handling and fallback behavior. - - [Context-Window Handling](operations/context_window.md) - -
+ + ## Configuration Model @@ -120,22 +106,22 @@ Non-interactive `configure` does not read provider credentials from the routing bundle; pass `--api-key` explicitly when persisting the bundle for CI. Route types, launcher use, and persistence are covered in -[Routing Overview](routing_algorithms/overview.md). The Rust server uses its +[Routing Overview](/routing/overview). The Rust server uses its own explicit TOML configuration for LLM clients, targets, and libsy algorithms. ## Routing Reference | Need | Read | |---|---| -| Fixed strong/weak traffic split for baselines or A/B tests | [Random Routing](routing_algorithms/random_routing.md) | -| Per-request strong/weak decisions from a classifier model | [LLM Classifier Routing](routing_algorithms/llm_classifier_routing.md) | -| Signal-driven weak/strong escalation with optional classifier fallback | [Stage-Router Routing](routing_algorithms/stage_router_routing.md) | -| Conversation-level affinity for cache reuse | [Sticky Routing](routing_algorithms/sticky_routing.md) | +| Fixed strong/weak traffic split for baselines or A/B tests | [Random Routing](/routing/random-routing) | +| Per-request strong/weak decisions from a classifier model | [LLM Classifier Routing](/routing/llm-classifier-routing) | +| Signal-driven weak/strong escalation with optional classifier fallback | [Stage-Router Routing](/routing/stage-router-routing) | +| Conversation-level affinity for cache reuse | [Sticky Routing](/routing/sticky-routing) | ## Operations and Reference | Topic | Read | |---|---| -| Known limitations and workarounds for 0.1.0 | [Known Issues](known_issues.md) | -| CLI syntax, flags, resolution rules, and environment variables | [CLI Reference](cli_reference.md) | -| Context-window overflow retry and fallback behavior | [Context-Window Handling](operations/context_window.md) | +| Known limitations and workarounds for 0.1.0 | [Known Issues](/get-started/known-issues-in-0-1-0) | +| CLI syntax, flags, resolution rules, and environment variables | [CLI Reference](/reference/cli-reference) | +| Context-window overflow retry and fallback behavior | [Context-Window Handling](/operations/context-window-handling) | diff --git a/docs/known_issues.md b/docs/known_issues.mdx similarity index 65% rename from docs/known_issues.md rename to docs/known_issues.mdx index d2b3b361..a157af5d 100644 --- a/docs/known_issues.md +++ b/docs/known_issues.mdx @@ -1,5 +1,8 @@ -# Known Issues - +--- +title: "Known Issues in 0.1.0" +description: "Review known limitations and workarounds for NeMo Switchyard 0.1.0." +position: 2 +--- ## 0.1.0 1. Completed Codex Responses tasks may record `0` token usage in `GET /v1/stats` and `GET /v1/routing/stats`. 2. Tool-bearing Codex requests may fail when Switchyard routes them to an upstream that accepts only a fixed set of tool names or schemas. diff --git a/docs/operations/context_window.md b/docs/operations/context_window.mdx similarity index 87% rename from docs/operations/context_window.md rename to docs/operations/context_window.mdx index a6504e15..66d7cd71 100644 --- a/docs/operations/context_window.md +++ b/docs/operations/context_window.mdx @@ -1,5 +1,8 @@ -# Context-Window Handling - +--- +title: "Context-Window Handling" +description: "Handle upstream context-window overflows with target eviction, fallback routing, and one retry." +position: 1 +--- When an upstream rejects a request because the prompt exceeds the model's context window, Switchyard evicts that target for the current request, reroutes to the configured fallback target, and retries once. If the fallback diff --git a/docs/routing_algorithms/escalation_router_routing.md b/docs/routing_algorithms/escalation_router_routing.mdx similarity index 91% rename from docs/routing_algorithms/escalation_router_routing.md rename to docs/routing_algorithms/escalation_router_routing.mdx index 696512c3..e4599b4e 100644 --- a/docs/routing_algorithms/escalation_router_routing.md +++ b/docs/routing_algorithms/escalation_router_routing.mdx @@ -1,4 +1,8 @@ -# Escalation-Router Routing +--- +title: "Escalation-Router Routing" +description: "Start agent tasks on a weak model and latch them to a strong model when an LLM judge detects sustained trouble." +position: 6 +--- Escalation-router routing starts each conversation on a cheaper `weak` model. An LLM judge watches how the work progresses and moves the conversation to a @@ -9,7 +13,7 @@ the task. Use it for multi-turn agent workloads where a weak model can handle routine work but may need rescue after repeated errors, loops, drift, false progress, or premature completion. Unlike -[LLM Classifier Routing](llm_classifier_routing.md), which predicts how +[LLM Classifier Routing](/routing/llm-classifier-routing), which predicts how difficult a request looks, escalation routing judges whether the run is actually going well. @@ -18,7 +22,7 @@ actually going well. The router keeps one state value per conversation: whether that conversation has escalated. Conversations are identified from the stable system prompt and first user message, using the same bounded in-memory store described in -[Sticky Routing](sticky_routing.md). +[Sticky Routing](/routing/sticky-routing). For each turn, Switchyard: @@ -118,7 +122,7 @@ a client-selectable model. If the selected tier exceeds its context window, Switchyard retries once on `fallback_target_on_evict`, which must match one of the configured tier ids (`strong` / `weak` unless the targets set their own `id`). See -[Context-Window Handling](../operations/context_window.md). +[Context-Window Handling](/operations/context-window-handling). ## Useful options @@ -183,12 +187,12 @@ mid-session prefix rewrites can change a deep key and lose the existing latch. ## When not to use escalation routing - **One-shot requests.** There is no trajectory to judge. Use - [LLM Classifier Routing](llm_classifier_routing.md) when the initial request + [LLM Classifier Routing](/routing/llm-classifier-routing) when the initial request should determine the tier. - **Fixed traffic experiments.** Use - [Random Routing](random_routing.md) for A/B splits and gradual traffic ramps. + [Random Routing](/routing/random-routing) for A/B splits and gradual traffic ramps. - **Per-turn stage optimization.** Use - [Stage-Router Routing](stage_router_routing.md) when tool-result signals + [Stage-Router Routing](/routing/stage-router-routing) when tool-result signals should move individual turns in both directions. - **Latency-critical traffic.** Eligible unlatched turns wait for the judge before the selected backend call. @@ -197,8 +201,8 @@ mid-session prefix rewrites can change a deep key and lose the existing latch. ## Related -- [Routing Overview](overview.md): compare all supported routing strategies. -- [Sticky Routing](sticky_routing.md): session-key derivation and affinity +- [Routing Overview](/routing/overview): compare all supported routing strategies. +- [Sticky Routing](/routing/sticky-routing): session-key derivation and affinity behavior. -- [Architecture](../architecture.md): the end-to-end request lifecycle and +- [Architecture](/concepts/architecture): the end-to-end request lifecycle and system boundaries. diff --git a/docs/routing_algorithms/llm_classifier_routing.md b/docs/routing_algorithms/llm_classifier_routing.mdx similarity index 90% rename from docs/routing_algorithms/llm_classifier_routing.md rename to docs/routing_algorithms/llm_classifier_routing.mdx index f513589c..436fc8dd 100644 --- a/docs/routing_algorithms/llm_classifier_routing.md +++ b/docs/routing_algorithms/llm_classifier_routing.mdx @@ -1,5 +1,8 @@ -# LLM Classifier Routing - +--- +title: "LLM Classifier Routing" +description: "Use an LLM classifier to route each request to a strong or weak model target." +position: 3 +--- LLM classifier routing asks a classifier model to evaluate each request, then sends the request to a `weak` or `strong` backend. Use it when routing should depend on request content, tool use, context needs, or risk level instead of a @@ -87,7 +90,7 @@ prompt determine the verdict. For a self-hosted strong, weak, or classifier target, configure it like any other OpenAI-compatible endpoint. See -[Self-hosted targets](overview.md#self-hosted-targets). +[Self-hosted targets](/routing/overview#self-hosted-targets). ## Session affinity @@ -99,10 +102,10 @@ reuse that tier before classification, so they skip the classifier call; abstain, low-confidence, missing-signal, and fail-open decisions do not pin. Configure these fields on the `type: deterministic` entry in the `routes:` -bundle. See [Session Affinity](sticky_routing.md) for YAML and -[How session affinity composes](overview.md#how-session-affinity-composes) for +bundle. See [Session Affinity](/routing/sticky-routing) for YAML and +[How session affinity composes](/routing/overview#how-session-affinity-composes) for the interaction with routing decisions. If the per-request classifier cost is too high, use -[Stage-Router Routing](stage_router_routing.md), which can route many turns from tool and +[Stage-Router Routing](/routing/stage-router-routing), which can route many turns from tool and agent-progress signals without an extra classifier call. diff --git a/docs/routing_algorithms/overview.md b/docs/routing_algorithms/overview.mdx similarity index 79% rename from docs/routing_algorithms/overview.md rename to docs/routing_algorithms/overview.mdx index 96b8d4cb..72af2292 100644 --- a/docs/routing_algorithms/overview.md +++ b/docs/routing_algorithms/overview.mdx @@ -1,5 +1,8 @@ -# Routing Overview - +--- +title: "Overview" +description: "Choose and configure passthrough, random, classifier, stage-router, escalation-router, and sticky routing behavior." +position: 1 +--- The Python CLI loads routing policies from a YAML bundle. Each key under `routes:` becomes a model ID available through OpenAI Chat Completions, Anthropic Messages, and OpenAI Responses requests: @@ -15,13 +18,15 @@ configuration and tuning. | Strategy | Use it when | Route `type` | |---|---|---| -| [Random Routing](random_routing.md) | You need a fixed strong/weak split for A/B tests, baselines, or cost experiments. | `random_routing` | -| [LLM Classifier Routing](llm_classifier_routing.md) | Request content should decide whether a turn needs the weak or strong tier. | `deterministic` | -| [Stage-Router Routing](stage_router_routing.md) | Tool-result and agent-progress signals should route most turns without an extra classifier call. | `stage_router` | -| [Escalation-Router Routing](escalation_router_routing.md) | Start every task on the weak tier and escalate to strong when an LLM judge detects trouble. | `escalation_router` | +| [Random Routing](/routing/random-routing) | You need a fixed strong/weak split for A/B tests, baselines, or cost experiments. | `random_routing` | +| [LLM Classifier Routing](/routing/llm-classifier-routing) | Request content should decide whether a turn needs the weak or strong tier. | `deterministic` | +| [Stage-Router Routing](/routing/stage-router-routing) | Tool-result and agent-progress signals should route most turns without an extra classifier call. | `stage_router` | +| [Escalation-Router Routing](/routing/escalation-router-routing) | Start every task on the weak tier and escalate to strong when an LLM judge detects trouble. | `escalation_router` | -[Session Affinity (Sticky Routing)](sticky_routing.md) is an opt-in feature of -LLM classifier routing, not a standalone routing strategy. +[Session Affinity (Sticky Routing)](/routing/sticky-routing) is an opt-in feature of +LLM classifier routing, not a standalone routing strategy. See +[How session affinity composes](#how-session-affinity-composes) for the exact +behavior. ## Common route shape @@ -145,7 +150,7 @@ decisions never pin. Later turns reuse the tier and skip the classifier call. Pins use a bounded in-process LRU keyed from the stable conversation prefix. They are not shared across workers or restarts. See -[Sticky Routing](sticky_routing.md) for configuration and key derivation. +[Sticky Routing](/routing/sticky-routing) for configuration and key derivation. Random and stage-router routing make a fresh routing decision for each request. The escalation router instead uses affinity as a one-way escalation latch: only @@ -156,5 +161,5 @@ the strong tier is pinned, and the judge runs until that latch fires. The `switchyard-server` binary has a separate TOML schema that explicitly constructs LLM clients, targets, and libsy algorithms. It does not load Python route bundles. See the -[Rust server README](../../crates/switchyard-server/README.md) for its supported +[Rust server README](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/switchyard-server/README.md) for its supported configuration. diff --git a/docs/routing_algorithms/random_routing.md b/docs/routing_algorithms/random_routing.mdx similarity index 81% rename from docs/routing_algorithms/random_routing.md rename to docs/routing_algorithms/random_routing.mdx index d7b85bae..7ac6ce23 100644 --- a/docs/routing_algorithms/random_routing.md +++ b/docs/routing_algorithms/random_routing.mdx @@ -1,12 +1,15 @@ -# Random Routing - +--- +title: "Random Routing" +description: "Split traffic between strong and weak model targets with a fixed probability." +position: 2 +--- Random routing sends a fixed percentage of traffic to a `strong` model and the rest to a `weak` model. It does not inspect the prompt. Use it for A/B tests, benchmark baselines, and gradual traffic ramps. If routing should depend on task difficulty, use -[LLM Classifier Routing](llm_classifier_routing.md) or -[Stage-Router Routing](stage_router_routing.md). +[LLM Classifier Routing](/routing/llm-classifier-routing) or +[Stage-Router Routing](/routing/stage-router-routing). ## Algorithm @@ -25,7 +28,7 @@ repeatable sequence for tests or benchmarks. Random routing is per request. A multi-turn conversation can move between models from turn to turn. If you need conversation-level stickiness, use -[Sticky Routing](sticky_routing.md) where it is supported, or select a direct +[Sticky Routing](/routing/sticky-routing) where it is supported, or select a direct tier model instead of the routing model. ## Enable It diff --git a/docs/routing_algorithms/stage_router_routing.md b/docs/routing_algorithms/stage_router_routing.mdx similarity index 96% rename from docs/routing_algorithms/stage_router_routing.md rename to docs/routing_algorithms/stage_router_routing.mdx index 589e9cff..c8e9104f 100644 --- a/docs/routing_algorithms/stage_router_routing.md +++ b/docs/routing_algorithms/stage_router_routing.mdx @@ -1,5 +1,8 @@ -# Stage-Router Routing - +--- +title: "Stage-Router Routing" +description: "Route agent turns with progress signals, tool results, and an optional LLM classifier fallback." +position: 5 +--- Stage-router routing sends each request to either a **capable** model or a cheaper **efficient** one, depending on where the agent is in its run. The goal is to spend the capable model on the turns that need it (exploration, error @@ -11,7 +14,7 @@ plus an optional LLM classifier. If the selected backend hits a context-window overflow, the router retries once against `fallback_target_on_evict`; a second overflow surfaces a -context-pool-exhausted error (see [Context-Window Handling](../operations/context_window.md)). +context-pool-exhausted error (see [Context-Window Handling](/operations/context-window-handling)). ## How it works @@ -221,7 +224,7 @@ If you omit `confidence_threshold`, the config default of `0.5` applies; the example sets it explicitly. `fallback_target_on_evict` is required and must reference one of the -declared target ids. See [Context-Window Handling](../operations/context_window.md) for +declared target ids. See [Context-Window Handling](/operations/context-window-handling) for exception types and error envelopes. ### Optional: add an LLM classifier @@ -282,7 +285,7 @@ curl -s http://localhost:4000/v1/stats > routing_stats_final.json - **Single-model deployments.** Use a `model` route instead. - **Probabilistic A/B splits.** Use - [Random Routing](random_routing.md) (`type: random_routing`). + [Random Routing](/routing/random-routing) (`type: random_routing`). The stage-router's signals are wasted on a fixed traffic ratio. - **No tool-result history.** Stage-router needs meaningful tool-call traffic to populate the tool-result signal. For pure chat-completion workloads every @@ -290,5 +293,5 @@ curl -s http://localhost:4000/v1/stats > routing_stats_final.json ## Related -- [Architecture](../architecture.md): the end-to-end request lifecycle and +- [Switchyard Architecture](/concepts/architecture): the end-to-end request lifecycle and system boundaries. diff --git a/docs/routing_algorithms/sticky_routing.md b/docs/routing_algorithms/sticky_routing.mdx similarity index 95% rename from docs/routing_algorithms/sticky_routing.md rename to docs/routing_algorithms/sticky_routing.mdx index 404d798a..8865d481 100644 --- a/docs/routing_algorithms/sticky_routing.md +++ b/docs/routing_algorithms/sticky_routing.mdx @@ -1,5 +1,8 @@ -# Sticky Routing - +--- +title: "Sticky Routing" +description: "Keep multi-turn conversations on one model tier to preserve upstream prompt and KV caches." +position: 4 +--- By default, routing is recomputed on every request, so a single task can hop between models mid-conversation. This throws away the upstream prompt/KV cache and, on the classifier router, pays for an LLM classifier call every turn. diff --git a/docs/skill_distillation.md b/docs/skill_distillation.mdx similarity index 97% rename from docs/skill_distillation.md rename to docs/skill_distillation.mdx index c66d5017..162952b2 100644 --- a/docs/skill_distillation.md +++ b/docs/skill_distillation.mdx @@ -1,5 +1,8 @@ -# Skill Distillation - +--- +title: "Skill Distillation" +description: "Configure skill distillation to turn agent sessions into reusable skills without retraining the model." +position: 2 +--- Skill distillation turns agent sessions into a reusable skill for the same kind of work. The model is not retrained. Switchyard saves the session history, uses it to update a `SKILL.md`, and makes the active skill available to later diff --git a/docs/stylesheets/nvidia.css b/docs/stylesheets/nvidia.css deleted file mode 100644 index 69d0270e..00000000 --- a/docs/stylesheets/nvidia.css +++ /dev/null @@ -1,15 +0,0 @@ -/* NVIDIA brand colors */ -:root, -[data-md-color-scheme="default"] { - --md-primary-fg-color: #76B900; - --md-primary-fg-color--light: #93d400; - --md-primary-fg-color--dark: #5a8c00; - --md-accent-fg-color: #76B900; -} - -[data-md-color-scheme="slate"] { - --md-primary-fg-color: #76B900; - --md-primary-fg-color--light: #93d400; - --md-primary-fg-color--dark: #5a8c00; - --md-accent-fg-color: #76B900; -} diff --git a/mkdocs.yml b/mkdocs.yml deleted file mode 100644 index 0409a52e..00000000 --- a/mkdocs.yml +++ /dev/null @@ -1,107 +0,0 @@ -site_name: Switchyard -site_description: Typed, composable LLM routing and format translation for Python -repo_url: https://github.com/NVIDIA-NeMo/Switchyard -repo_name: NVIDIA-NeMo/Switchyard -edit_uri: edit/main/docs/ -extra: - source_ref: !ENV [MKDOCS_SOURCE_REF, main] -copyright: "Copyright © 2025-2026 NVIDIA Corporation" - -docs_dir: docs -site_dir: site -use_directory_urls: true -strict: true - -hooks: - - mkdocs_hooks.py - -nav: - - Home: index.md - - Get Started: - - Getting Started: getting_started.md - - Known Issues in 0.1.0: known_issues.md - - Guides: - - Agent Launchers: guides/agent_launchers.md - - Skill Distillation: skill_distillation.md - - Concepts: - - Core Concepts: core_concepts.md - - Architecture: architecture.md - - Routing: - - Overview: routing_algorithms/overview.md - - Random Routing: routing_algorithms/random_routing.md - - LLM Classifier Routing: routing_algorithms/llm_classifier_routing.md - - Sticky Routing: routing_algorithms/sticky_routing.md - - Stage-Router Routing: routing_algorithms/stage_router_routing.md - - Escalation-Router Routing: routing_algorithms/escalation_router_routing.md - - Operations: - - Context-Window Handling: operations/context_window.md - - Reference: - - CLI Reference: cli_reference.md - -exclude_docs: | - internal/** - research/** - vllm-serve-hidden-state.md - -theme: - name: material - palette: - - media: "(prefers-color-scheme: light)" - scheme: default - primary: custom - accent: custom - toggle: - icon: material/weather-sunny - name: Switch to dark mode - - media: "(prefers-color-scheme: dark)" - scheme: slate - primary: custom - accent: custom - toggle: - icon: material/weather-night - name: Switch to light mode - font: - text: Inter - code: JetBrains Mono - features: - - navigation.top - - navigation.sections - - search.suggest - - search.highlight - - content.code.copy - - toc.follow - -plugins: - - search: - lang: en - -markdown_extensions: - - admonition - - pymdownx.details - - pymdownx.superfences: - custom_fences: - - name: mermaid - class: mermaid - format: !!python/name:pymdownx.superfences.fence_code_format - - pymdownx.highlight: - anchor_linenums: true - line_spans: __span - pygments_lang_class: true - - pymdownx.inlinehilite - - pymdownx.snippets: - base_path: [docs] - check_paths: true - - pymdownx.emoji: - emoji_index: !!python/name:material.extensions.emoji.twemoji - emoji_generator: !!python/name:material.extensions.emoji.to_svg - - attr_list - - md_in_html - - tables - - toc: - permalink: true - title: On this page - - pymdownx.tasklist: - custom_checkbox: true - -extra_css: - - stylesheets/nvidia.css diff --git a/mkdocs_hooks.py b/mkdocs_hooks.py deleted file mode 100644 index d9f7bdd1..00000000 --- a/mkdocs_hooks.py +++ /dev/null @@ -1,310 +0,0 @@ -# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. -# SPDX-License-Identifier: Apache-2.0 - -"""MkDocs hooks for repository-relative source links.""" - -import re -from collections.abc import Mapping -from pathlib import Path -from typing import Protocol -from urllib.parse import quote, unquote, urlsplit - -_FENCE_RE = re.compile( - r"^(?P[> \t]*)(?:(?P[-+*]|\d{1,9}[.)])" - r"(?P[ \t]{1,4}))?(?P`{3,}|~{3,})(?P[^\r\n]*)" -) -_BACKTICK_RUN_RE = re.compile(r"`+") -_LINK_DESTINATION_RE = re.compile( - r"(?P\]\(\s*)(?:<(?P[^>\r\n]+)>|(?P[^\s)\r\n]+))" -) - - -class _PageFile(Protocol): - abs_src_path: str - - -class _Page(Protocol): - file: _PageFile - - -class _MkDocsConfig(Protocol): - config_file_path: str - - def __getitem__(self, key: str) -> object: ... - - -def _target_url( - destination: str, - *, - source_path: Path, - docs_dir: Path, - repo_root: Path, - repo_url: str, - source_ref: str, -) -> str | None: - parsed = urlsplit(destination) - if parsed.scheme or parsed.netloc or not parsed.path or parsed.path.startswith(("/", "\\")): - return None - - try: - target = (source_path.parent / unquote(parsed.path)).resolve() - except (OSError, RuntimeError): - return None - - if target.is_relative_to(docs_dir) or not target.is_relative_to(repo_root): - return None - if not target.exists(): - return None - - object_type = "tree" if target.is_dir() else "blob" - relative_target = quote(target.relative_to(repo_root).as_posix(), safe="/") - encoded_ref = quote(source_ref, safe="/") - suffix = destination[len(parsed.path) :] - return f"{repo_url.rstrip('/')}/{object_type}/{encoded_ref}/{relative_target}{suffix}" - - -def _rewrite_link_destinations( - markdown: str, - *, - source_path: Path, - docs_dir: Path, - repo_root: Path, - repo_url: str, - source_ref: str, -) -> str: - def replace(match: re.Match[str]) -> str: - destination = match.group("angled") or match.group("plain") - rewritten = _target_url( - destination, - source_path=source_path, - docs_dir=docs_dir, - repo_root=repo_root, - repo_url=repo_url, - source_ref=source_ref, - ) - if rewritten is None: - return match.group(0) - if match.group("angled") is not None: - rewritten = f"<{rewritten}>" - return f"{match.group('prefix')}{rewritten}" - - return _LINK_DESTINATION_RE.sub(replace, markdown) - - -def _rewrite_outside_code_spans( - markdown: str, - *, - source_path: Path, - docs_dir: Path, - repo_root: Path, - repo_url: str, - source_ref: str, -) -> str: - def rewrite(text: str) -> str: - return _rewrite_link_destinations( - text, - source_path=source_path, - docs_dir=docs_dir, - repo_root=repo_root, - repo_url=repo_url, - source_ref=source_ref, - ) - - output: list[str] = [] - cursor = 0 - while opening := _BACKTICK_RUN_RE.search(markdown, cursor): - output.append(rewrite(markdown[cursor : opening.start()])) - closing = next( - ( - candidate - for candidate in _BACKTICK_RUN_RE.finditer(markdown, opening.end()) - if len(candidate.group(0)) == len(opening.group(0)) - ), - None, - ) - if closing is None: - output.append(rewrite(markdown[opening.start() :])) - return "".join(output) - output.append(markdown[opening.start() : closing.end()]) - cursor = closing.end() - output.append(rewrite(markdown[cursor:])) - return "".join(output) - - -def _container_position(line: str, quote_depth: int) -> tuple[int, int] | None: - cursor = 0 - for _ in range(quote_depth): - padding = 0 - while cursor < len(line) and line[cursor] == " " and padding < 3: - cursor += 1 - padding += 1 - if cursor >= len(line) or line[cursor] != ">": - return None - cursor += 1 - if cursor < len(line) and line[cursor] in " \t": - cursor += 1 - - indentation_start = cursor - while cursor < len(line) and line[cursor] in " \t": - cursor += 1 - indentation = len(line[indentation_start:cursor].expandtabs(4)) - return indentation, cursor - - -def _fence_container(match: re.Match[str]) -> tuple[int, int] | None: - prefix = match.group("prefix") - quote_depth = prefix.count(">") - position = _container_position(prefix, quote_depth) - if position is None: - return None - - indentation, _ = position - list_marker = match.group("list_marker") - if list_marker is not None: - indentation += len(list_marker) + len(match.group("list_padding").expandtabs(4)) - elif indentation <= 3: - indentation = 0 - return quote_depth, indentation - - -def _line_is_in_container(line: str, quote_depth: int, indentation: int) -> bool: - if not line.strip(): - return True - position = _container_position(line, quote_depth) - if position is None: - return False - line_indentation, content_start = position - return not line[content_start:].strip() or line_indentation >= indentation - - -def _is_closing_fence( - match: re.Match[str], - *, - fence_character: str, - fence_length: int, - quote_depth: int, - indentation: int, -) -> bool: - marker = match.group("marker") - if ( - match.group("list_marker") is not None - or marker[0] != fence_character - or len(marker) < fence_length - or match.group("rest").strip() - or match.group("prefix").count(">") != quote_depth - ): - return False - - position = _container_position(match.group("prefix"), quote_depth) - if position is None: - return False - closing_indentation, _ = position - return closing_indentation <= 3 if indentation == 0 else closing_indentation == indentation - - -def rewrite_repository_links( - markdown: str, - *, - source_path: Path, - docs_dir: Path, - repo_root: Path, - repo_url: str, - source_ref: str, -) -> str: - """Rewrite valid links outside docs_dir to repository source URLs.""" - source_path = source_path.resolve() - docs_dir = docs_dir.resolve() - repo_root = repo_root.resolve() - output: list[str] = [] - pending_markdown: list[str] = [] - fence_lines: list[str] = [] - fence_character: str | None = None - fence_length = 0 - fence_quote_depth = 0 - fence_indentation = 0 - - def flush_pending_markdown() -> None: - if not pending_markdown: - return - output.append( - _rewrite_outside_code_spans( - "".join(pending_markdown), - source_path=source_path, - docs_dir=docs_dir, - repo_root=repo_root, - repo_url=repo_url, - source_ref=source_ref, - ) - ) - pending_markdown.clear() - - for line in markdown.splitlines(keepends=True): - if fence_character is not None and not _line_is_in_container( - line, fence_quote_depth, fence_indentation - ): - pending_markdown.extend(fence_lines) - fence_lines.clear() - fence_character = None - fence_length = 0 - fence_quote_depth = 0 - fence_indentation = 0 - - fence = _FENCE_RE.match(line) - if fence_character is None: - container = _fence_container(fence) if fence is not None else None - if fence is None or container is None: - pending_markdown.append(line) - continue - flush_pending_markdown() - marker = fence.group("marker") - fence_character = marker[0] - fence_length = len(marker) - fence_quote_depth, fence_indentation = container - fence_lines.append(line) - continue - - fence_lines.append(line) - if fence is not None and _is_closing_fence( - fence, - fence_character=fence_character, - fence_length=fence_length, - quote_depth=fence_quote_depth, - indentation=fence_indentation, - ): - output.extend(fence_lines) - fence_lines.clear() - fence_character = None - fence_length = 0 - fence_quote_depth = 0 - fence_indentation = 0 - - output.extend(fence_lines) - flush_pending_markdown() - return "".join(output) - - -def on_page_markdown( - markdown: str, - *, - page: _Page, - config: _MkDocsConfig, - files: object, -) -> str: - """Rewrite repository-relative links before MkDocs validates Markdown links.""" - del files - docs_dir = config["docs_dir"] - repo_url = config["repo_url"] - extra = config["extra"] - if not isinstance(docs_dir, str) or not isinstance(repo_url, str): - raise ValueError("MkDocs docs_dir and repo_url must be configured") - if not isinstance(extra, Mapping) or not isinstance(extra.get("source_ref"), str): - raise ValueError("MkDocs extra.source_ref must be configured") - - return rewrite_repository_links( - markdown, - source_path=Path(page.file.abs_src_path), - docs_dir=Path(docs_dir), - repo_root=Path(config.config_file_path).resolve().parent, - repo_url=repo_url, - source_ref=extra["source_ref"], - ) diff --git a/scripts/ci/README.md b/scripts/ci/README.md new file mode 100644 index 00000000..f4098bb0 --- /dev/null +++ b/scripts/ci/README.md @@ -0,0 +1,106 @@ +# CI Options and Workflow Design + +This directory is the maintainer-facing entry point for choosing local validation and designing +GitHub Actions workflows. The defaults below incorporate patterns used in NVIDIA NeMo Data +Designer and NeMo Curator, adapted to Switchyard's required `CI Success` aggregate and public-fork +threat model. + +## Pattern Provenance + +- **NeMo Curator** informed the two-stage preview model: build from the pull request without + credentials, then perform privileged work from trusted workflow metadata. +- **NeMo Data Designer** informed same-repository preview policy, PR-scoped cancellation, bot + comment filtering, serialized publishing, and keeping action/runtime pins current. +- **Switchyard** adds the stable `CI Success` aggregate, job-level permission separation, trusted + default-branch tool configuration, and regression tests for workflow security invariants. + +## Choose a Validation Scope + +Use the selector to turn changed paths into the smallest relevant local command set: + +```bash +# Uncommitted work, including staged and untracked files +python scripts/select_validation.py --changed + +# The whole branch, including committed work +git fetch origin main +python scripts/select_validation.py --base origin/main + +# A proposed path before editing +python scripts/select_validation.py --path .github/workflows/example.yml +``` + +`--json` provides the same plan to editor or agent integrations. The selector never schedules live +provider tests; those require explicit intent and credentials. Before pushing, use the full hard +gate in `.github/workflows/ci.yml` when the change crosses multiple ownership areas or affects +packaging, shared runtime behavior, or the required aggregate. + +## Choose a Workflow Shape + +| Need | Preferred option | Why | +| --- | --- | --- | +| Required PR validation | Call a reusable workflow from `ci.yml` and add its job to `CI Success.needs` | Branch protection keeps one stable required check while new gates cannot become accidentally optional. | +| Path-filtered or advisory validation | Use a separate `pull_request` workflow | It can skip irrelevant changes without making the required aggregate wait for a check that never appears. | +| Secretless preview build | Run on `pull_request` with read-only permissions | Fork code can be checked without exposing secrets or write tokens. | +| Secret-bearing preview or PR comment | Use a trusted `workflow_run` follow-up for same-repository PRs only | Untrusted PR code and trusted credentials never execute in the same job. | +| Production publishing | Trigger from the default branch or a release tag and serialize runs | Prevents an older deployment from racing and overwriting a newer one. | +| Expensive release matrix | Use tag or manual dispatch | Keeps routine PR feedback fast while preserving an explicit full-matrix option. | + +## Required Design Checks + +### Stable required aggregation + +- Every hard gate must be a direct or transitive dependency of `CI Success`. +- Keep branch protection pointed at `CI Success`, not a changing list of matrix jobs. +- Do not put path filters on a required workflow unless a stable always-running aggregate accounts + for the skipped result. + +### Trust boundaries + +- Treat pull-request source, artifacts, cache contents, and artifact filenames as untrusted. +- Derive PR identity, repository identity, and tool versions from trusted event metadata or the + default-branch checkout—not from a downloaded artifact. +- Restrict secret-bearing previews to same-repository PRs. Fork PRs receive validation only. +- A `workflow_run` workflow must exist on the default branch before GitHub will trigger it. + +### Concurrency + +- For PR work, group by workflow plus PR number and cancel superseded runs. +- For production publishing, use one stable deployment group and do not cancel an active publish. +- Avoid grouping only by branch when multiple PRs or deployment targets can share it. + +### Permissions and secrets + +- Start privileged workflows with `permissions: {}` and grant only the permissions each job needs. +- Separate generation, commenting, and repository deployment jobs when they use different secrets + or write scopes. +- Keep `pull-requests: write`, `contents: write`, and publishing tokens out of jobs that execute PR + code. +- When upserting bot comments, match both a unique marker and the expected bot author. + +### Supply chain and toolchains + +- Pin third-party actions to immutable commit SHAs and retain a version comment for update tools and + reviewers. +- Pin command-line tools exactly when their output or hosted behavior is release-sensitive. +- Use runtime versions still supported by the action ecosystem; upgrade intentionally and test the + full workflow path. +- Run `actionlint` for workflow syntax and `zizmor --pedantic .github/workflows` for security. Keep + suppressions narrow, inline, and justified. + +## Review Checklist + +Before merging a workflow change, answer all of these: + +1. Is the check required, advisory, privileged follow-up, or release-only? +2. If required, does failure reach `CI Success`? +3. Can fork-controlled code, metadata, artifacts, or caches reach a secret or write permission? +4. Is superseded PR work cancelled and production work serialized? +5. Are permissions job-scoped and external actions immutable? +6. Are tool versions selected from trusted configuration? +7. Do regression tests encode the security and aggregation invariants? +8. Do `actionlint`, `zizmor`, and the path-selected local validations pass? + +Workflow-specific operational guidance remains with its owner, such as the +[release workflow](../../docs/internal/release_workflow.md) for package publication and the +[Switchyard docs skill](../../.agents/skills/switchyard-docs/SKILL.md) for Fern publishing. diff --git a/scripts/select_validation.py b/scripts/select_validation.py new file mode 100644 index 00000000..c65041e4 --- /dev/null +++ b/scripts/select_validation.py @@ -0,0 +1,225 @@ +#!/usr/bin/env python3 +# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +"""Select local validation commands from changed repository paths.""" + +from __future__ import annotations + +import argparse +import json +import subprocess +import sys +from collections.abc import Iterable +from dataclasses import asdict, dataclass +from pathlib import Path + + +@dataclass(frozen=True) +class Validation: + """One local validation command and the CI behavior it covers.""" + + name: str + command: str + reason: str + + +VALIDATIONS = { + "python": Validation( + "Python quality gates", + "uv run ruff check . && uv run mypy switchyard", + "Python, tests, scripts, or package metadata changed.", + ), + "tests": Validation( + "Offline test suite", + 'env -u OPENROUTER_API_KEY -u NVIDIA_API_KEY -u OPENAI_API_KEY ' + '-u ANTHROPIC_API_KEY uv run pytest tests/ -v -m "not integration"', + "Runtime code, tests, or package metadata changed.", + ), + "rust": Validation( + "Rust quality gates", + "cargo fmt --all --check && cargo clippy --workspace --all-targets -- -D warnings " + "&& cargo test --workspace", + "Rust sources or Cargo metadata changed.", + ), + "fern": Validation( + "Fern docs", + "make -C docs check && uv run pytest tests/test_fern_docs.py -v -o addopts=", + "Fern content, configuration, or docs workflows changed.", + ), + "readme": Validation( + "README examples", + "OPENAI_API_KEY=sk-test NVIDIA_API_KEY=nvapi-test ANTHROPIC_API_KEY=sk-ant-test " + "uv run pytest tests/readme --markdown-docs README.md -v", + "README examples or their executable coverage changed.", + ), + "getting-started": Validation( + "Getting Started examples", + "OPENAI_API_KEY=sk-test NVIDIA_API_KEY=nvapi-test ANTHROPIC_API_KEY=sk-ant-test " + "uv run pytest tests/getting_started --markdown-docs docs/getting_started.mdx -v", + "The Getting Started guide or its executable coverage changed.", + ), + "workflow-security": Validation( + "Workflow syntax and security", + "actionlint && zizmor --pedantic .github/workflows", + "A GitHub Actions workflow changed.", + ), + "package": Validation( + "Package build", + "uv build", + "Package metadata, lockfiles, native code, or release automation changed.", + ), +} + + +def _normalize(path: str) -> str: + """Return a repository-relative path with POSIX separators.""" + value = Path(path).as_posix().removeprefix("./") + return value.rstrip("/") + + +def select_validations(paths: Iterable[str]) -> list[Validation]: + """Return the stable, deduplicated validation plan for ``paths``.""" + normalized = {_normalize(path) for path in paths if _normalize(path)} + selected: set[str] = set() + + python_changed = any( + path.endswith(".py") + or path in {"pyproject.toml", "uv.lock"} + or path.startswith("switchyard/") + or path.startswith("switchyard_rust/") + for path in normalized + ) + if python_changed: + selected.update({"python", "tests"}) + + if any( + path in {"Cargo.toml", "Cargo.lock"} + or path.startswith("crates/") + or path.startswith("switchyard_rust/") + for path in normalized + ): + selected.add("rust") + + if any(path == "README.md" or path.startswith("tests/readme/") for path in normalized): + selected.add("readme") + + if any( + path == "docs/getting_started.mdx" or path.startswith("tests/getting_started/") + for path in normalized + ): + selected.add("getting-started") + + fern_workflows = { + ".github/workflows/ci.yml", + ".github/workflows/fern-docs-ci.yml", + ".github/workflows/fern-docs-preview-build.yml", + ".github/workflows/fern-docs-preview-comment.yml", + ".github/workflows/publish-fern-docs.yml", + } + if any( + path.startswith("docs/") + or path == "tests/test_fern_docs.py" + or path in fern_workflows + for path in normalized + ): + selected.add("fern") + + if any(path.startswith(".github/workflows/") for path in normalized): + selected.add("workflow-security") + + if any( + path in {"pyproject.toml", "uv.lock", "Cargo.toml", "Cargo.lock"} + or path.startswith("crates/") + or path.startswith("scripts/release/") + or path in { + ".github/workflows/package-portability.yml", + ".github/workflows/publish.yml", + } + for path in normalized + ): + selected.add("package") + + return [validation for key, validation in VALIDATIONS.items() if key in selected] + + +def _git_paths(*args: str) -> set[str]: + """Return paths printed by one Git command.""" + result = subprocess.run( + ("git", *args), + check=True, + capture_output=True, + text=True, + ) + return {line for line in result.stdout.splitlines() if line} + + +def changed_paths(base: str | None) -> set[str]: + """Return committed, staged, unstaged, and untracked paths for this checkout.""" + paths: set[str] = set() + if base is not None: + paths.update(_git_paths("diff", "--name-only", f"{base}...HEAD")) + paths.update(_git_paths("diff", "--name-only", "HEAD")) + paths.update(_git_paths("ls-files", "--others", "--exclude-standard")) + return paths + + +def main(argv: list[str] | None = None) -> int: + """Print a focused validation plan for explicit or Git-derived paths.""" + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--path", action="append", default=[], help="Changed path; repeat as needed") + parser.add_argument( + "--changed", + action="store_true", + help="Include staged, unstaged, and untracked paths", + ) + parser.add_argument( + "--base", + help="Also include committed changes since the merge base with this ref", + ) + parser.add_argument("--json", action="store_true", help="Emit machine-readable JSON") + args = parser.parse_args(argv) + + if not args.path and not args.changed and args.base is None: + parser.error("pass --path, --changed, or --base") + + paths = {_normalize(path) for path in args.path} + try: + if args.changed or args.base is not None: + paths.update(changed_paths(args.base)) + except subprocess.CalledProcessError as exc: + print(exc.stderr.strip() or "git path discovery failed", file=sys.stderr) + return 2 + + validations = select_validations(paths) + if args.json: + print( + json.dumps( + { + "paths": sorted(paths), + "validations": [asdict(validation) for validation in validations], + }, + indent=2, + ) + ) + return 0 + + if not paths: + print("No changed paths found.") + return 0 + + print("Changed paths:") + for path in sorted(paths): + print(f" - {path}") + print("\nRecommended validation:") + if not validations: + print(" - No focused gate mapped; review scripts/ci/README.md before pushing.") + for validation in validations: + print(f" - {validation.name}: {validation.command}") + print(f" {validation.reason}") + print("\nLive provider tests are never selected automatically.") + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/switchyard/lib/profiles/stage_router_config.py b/switchyard/lib/profiles/stage_router_config.py index 0092bb59..4ae3346a 100644 --- a/switchyard/lib/profiles/stage_router_config.py +++ b/switchyard/lib/profiles/stage_router_config.py @@ -1,7 +1,10 @@ # SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 -"""Declarative config for stage_router profiles. See ``docs/stage_router_routing.md``.""" +"""Declarative config for stage_router profiles. + +See ``docs/routing_algorithms/stage_router_routing.mdx``. +""" from __future__ import annotations diff --git a/tests/getting_started/test_getting_started.py b/tests/getting_started/test_getting_started.py index 2d560846..7f7ac9b8 100644 --- a/tests/getting_started/test_getting_started.py +++ b/tests/getting_started/test_getting_started.py @@ -1,7 +1,7 @@ # SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 -"""Executable coverage for ``docs/getting_started.md``.""" +"""Executable coverage for ``docs/getting_started.mdx``.""" from __future__ import annotations @@ -30,7 +30,7 @@ from tests._mock_openai_server import _MockOpenAIServer REPO_ROOT = Path(__file__).resolve().parents[2] -GUIDE_PATH = REPO_ROOT / "docs" / "getting_started.md" +GUIDE_PATH = REPO_ROOT / "docs" / "getting_started.mdx" #: How long to wait for ``switchyard serve`` to become ready on /health. STARTUP_TIMEOUT_S: float = 30.0 @@ -156,7 +156,7 @@ def test_all_yaml_blocks_in_guide_validate_as_route_bundles( build_route_bundle_table(payload) except RouteBundleConfigError as exc: raise AssertionError( - f"YAML block {idx} in getting_started.md failed to parse " + f"YAML block {idx} in getting_started.mdx failed to parse " f"as a route bundle: {exc}\n\nBlock:\n{block}" ) from exc diff --git a/tests/readme/test_readme.py b/tests/readme/test_readme.py index 788312b8..651e2efc 100644 --- a/tests/readme/test_readme.py +++ b/tests/readme/test_readme.py @@ -26,10 +26,10 @@ REPO_ROOT = Path(__file__).resolve().parents[2] README_PATH = REPO_ROOT / "README.md" ROUTING_DOC_PATHS = ( - REPO_ROOT / "docs" / "routing_algorithms" / "overview.md", - REPO_ROOT / "docs" / "routing_algorithms" / "random_routing.md", - REPO_ROOT / "docs" / "routing_algorithms" / "llm_classifier_routing.md", - REPO_ROOT / "docs" / "routing_algorithms" / "stage_router_routing.md", + REPO_ROOT / "docs" / "routing_algorithms" / "overview.mdx", + REPO_ROOT / "docs" / "routing_algorithms" / "random_routing.mdx", + REPO_ROOT / "docs" / "routing_algorithms" / "llm_classifier_routing.mdx", + REPO_ROOT / "docs" / "routing_algorithms" / "stage_router_routing.mdx", ) diff --git a/tests/test_cli_reference_docs.py b/tests/test_cli_reference_docs.py index 119e65bc..07689dc8 100644 --- a/tests/test_cli_reference_docs.py +++ b/tests/test_cli_reference_docs.py @@ -12,7 +12,7 @@ from switchyard.cli.switchyard_cli import _build_parser REPO_ROOT = Path(__file__).resolve().parents[1] -CLI_REFERENCE = REPO_ROOT / "docs" / "cli_reference.md" +CLI_REFERENCE = REPO_ROOT / "docs" / "cli_reference.mdx" _MARKDOWN = MarkdownIt() @@ -37,7 +37,7 @@ def _markdown_section(text: str, heading: str) -> str: end = next_token.map[0] break return "\n".join(lines[start:end]) - raise AssertionError(f"docs/cli_reference.md missing section {heading!r}") + raise AssertionError(f"docs/cli_reference.mdx missing section {heading!r}") def _long_options(parser: argparse.ArgumentParser) -> set[str]: @@ -59,7 +59,7 @@ def test_cli_reference_documents_every_serve_flag() -> None: missing = sorted(flag for flag in expected if flag not in serve_section) assert not missing, ( - "docs/cli_reference.md serve section missing flags: " + ", ".join(missing) + "docs/cli_reference.mdx serve section missing flags: " + ", ".join(missing) ) @@ -73,7 +73,7 @@ def test_cli_reference_documents_every_configure_flag() -> None: missing = sorted(flag for flag in expected if flag not in configure_section) assert not missing, ( - "docs/cli_reference.md configure section missing flags: " + "docs/cli_reference.mdx configure section missing flags: " + ", ".join(missing) ) @@ -94,7 +94,7 @@ def test_cli_reference_omits_unsupported_skill_distillation_flags() -> None: stale = sorted(flag for flag in unsupported_flags if flag in configure_section) assert not stale, ( - "docs/cli_reference.md configure section documents unsupported flags: " + "docs/cli_reference.mdx configure section documents unsupported flags: " + ", ".join(stale) ) @@ -136,7 +136,12 @@ def test_cli_reference_documents_shared_intake_flags_once() -> None: def _public_cli_doc_paths() -> list[Path]: - return [REPO_ROOT / "README.md", *sorted((REPO_ROOT / "docs").glob("*.md"))] + docs_root = REPO_ROOT / "docs" + fern_root = docs_root / "fern" + published_pages = sorted( + path for path in docs_root.rglob("*.mdx") if fern_root not in path.parents + ) + return [REPO_ROOT / "README.md", *published_pages] def _markdown_command_lines(path: Path) -> list[tuple[int, str]]: diff --git a/tests/test_fern_docs.py b/tests/test_fern_docs.py new file mode 100644 index 00000000..4637a08a --- /dev/null +++ b/tests/test_fern_docs.py @@ -0,0 +1,214 @@ +# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +"""Regression tests for Fern navigation, links, metadata, and legacy redirects.""" + +from __future__ import annotations + +import re +import runpy +from pathlib import Path +from typing import Any + +import yaml + +REPO_ROOT = Path(__file__).resolve().parents[1] +DOCS_ROOT = REPO_ROOT / "docs" +FERN_ROOT = DOCS_ROOT / "fern" +NAVIGATION_PATH = FERN_ROOT / "versions" / "nightly.yml" +DOCS_CONFIG_PATH = FERN_ROOT / "docs.yml" +GENERATOR_PATH = FERN_ROOT / "generate_legacy_redirect_site.py" +BASE_PATH = "/nemo/switchyard" +WORKFLOW_ROOT = REPO_ROOT / ".github" / "workflows" +CI_WORKFLOW_PATH = WORKFLOW_ROOT / "ci.yml" +FERN_CI_WORKFLOW_PATH = WORKFLOW_ROOT / "fern-docs-ci.yml" +FERN_PREVIEW_BUILD_PATH = WORKFLOW_ROOT / "fern-docs-preview-build.yml" +FERN_PREVIEW_COMMENT_PATH = WORKFLOW_ROOT / "fern-docs-preview-comment.yml" +FERN_PUBLISH_PATH = WORKFLOW_ROOT / "publish-fern-docs.yml" + + +def _load_yaml(path: Path) -> dict[str, Any]: + """Load one YAML mapping used by the Fern project.""" + data = yaml.safe_load(path.read_text(encoding="utf-8")) + assert isinstance(data, dict) + return data + + +def _load_workflow(path: Path) -> dict[str, Any]: + """Load a GitHub workflow without treating the `on` key as a YAML boolean.""" + data = yaml.load(path.read_text(encoding="utf-8"), Loader=yaml.BaseLoader) + assert isinstance(data, dict) + return data + + +def _slug(value: str) -> str: + """Return the Fern-style slug for the navigation labels used in this site.""" + return re.sub(r"[^a-z0-9]+", "-", value.lower()).strip("-") + + +def _published_routes() -> set[str]: + """Derive unversioned published routes from nightly navigation.""" + navigation = _load_yaml(NAVIGATION_PATH)["navigation"] + routes: set[str] = set() + for item in navigation: + if "page" in item: + routes.add(f"/{_slug(item['page'])}") + continue + + assert "section" in item + assert "path" not in item, "section paths must not duplicate child page paths" + section_slug = _slug(item["section"]) + for child in item["contents"]: + assert "page" in child + routes.add(f"/{section_slug}/{_slug(child['page'])}") + return routes + + +def test_internal_mdx_links_resolve_to_navigation_routes() -> None: + """Every root-relative MDX link must name a route that Fern publishes.""" + routes = _published_routes() + broken: list[str] = [] + patterns = (r"\]\((/[^)\s]+)", r'href="(/[^"]+)"') + for path in sorted(DOCS_ROOT.rglob("*.mdx")): + if FERN_ROOT in path.parents: + continue + text = path.read_text(encoding="utf-8") + for pattern in patterns: + for target in re.findall(pattern, text): + route = target.split("#", 1)[0] + if route not in routes: + broken.append(f"{path.relative_to(REPO_ROOT)}: {target}") + assert not broken, "broken Fern links:\n" + "\n".join(broken) + + +def test_published_pages_have_descriptions_and_titled_callouts() -> None: + """Published pages must retain SEO descriptions and converted admonition titles.""" + failures: list[str] = [] + for path in sorted(DOCS_ROOT.rglob("*.mdx")): + if FERN_ROOT in path.parents: + continue + text = path.read_text(encoding="utf-8") + frontmatter = text.split("---", 2)[1] + metadata = yaml.safe_load(frontmatter) + if not metadata.get("description"): + failures.append(f"{path.relative_to(REPO_ROOT)}: missing description") + if not metadata.get("title"): + failures.append(f"{path.relative_to(REPO_ROOT)}: missing title") + for tag in re.findall(r"<(?:Note|Warning)(?:\s+[^>]*)?>", text): + if "title=" not in tag: + failures.append(f"{path.relative_to(REPO_ROOT)}: untitled {tag}") + assert not failures, "\n".join(failures) + + +def test_fern_redirects_cover_every_github_pages_route() -> None: + """Fern must map every former MkDocs route to its current navigation route.""" + generator = runpy.run_path(str(GENERATOR_PATH)) + legacy_redirects = generator["LEGACY_REDIRECTS"] + configured = { + item["source"]: item["destination"] + for item in _load_yaml(DOCS_CONFIG_PATH)["redirects"] + } + routes = _published_routes() + + for legacy_path, destination in legacy_redirects.items(): + assert destination in routes + full_destination = f"{BASE_PATH}{destination}" + if not legacy_path: + assert configured[f"{BASE_PATH}/index.html"] == full_destination + continue + source = f"{BASE_PATH}/{legacy_path}" + assert configured[source] == full_destination + assert configured[f"{source}/index.html"] == full_destination + + +def test_legacy_github_pages_site_generation(tmp_path: Path) -> None: + """The generated GitHub Pages site must redirect every legacy route to Fern.""" + generator = runpy.run_path(str(GENERATOR_PATH)) + generator["generate_redirect_site"](tmp_path) + + assert (tmp_path / ".nojekyll").is_file() + for legacy_path, destination in generator["LEGACY_REDIRECTS"].items(): + page = tmp_path / legacy_path / "index.html" + target = f"{generator['CANONICAL_BASE']}{destination}" + assert page.is_file() + content = page.read_text(encoding="utf-8") + assert f'' in content + assert "window.location.search + window.location.hash" in content + + +def test_fern_validation_is_part_of_required_ci_success() -> None: + """The required CI aggregate must fail when reusable Fern validation fails.""" + ci = _load_workflow(CI_WORKFLOW_PATH) + jobs = ci["jobs"] + + assert jobs["fern-docs"]["uses"] == "./.github/workflows/fern-docs-ci.yml" + assert "fern-docs" in jobs["ci-success"]["needs"] + + fern_ci = _load_workflow(FERN_CI_WORKFLOW_PATH) + assert set(fern_ci["on"]) == {"workflow_call"} + + +def test_fern_preview_uses_trusted_identity_and_pr_scoped_concurrency() -> None: + """Preview artifacts must not choose the target PR, preview identifier, or tool version.""" + build = _load_workflow(FERN_PREVIEW_BUILD_PATH) + comment = _load_workflow(FERN_PREVIEW_COMMENT_PATH) + build_text = FERN_PREVIEW_BUILD_PATH.read_text(encoding="utf-8") + comment_text = FERN_PREVIEW_COMMENT_PATH.read_text(encoding="utf-8") + + assert build["concurrency"] == { + "group": "fern-docs-preview-${{ github.event.pull_request.number }}", + "cancel-in-progress": "true", + } + assert comment["concurrency"] == { + "group": "fern-docs-preview-${{ github.event.workflow_run.pull_requests[0].number }}", + "cancel-in-progress": "true", + } + assert "github.event.workflow_run.pull_requests[0].number" in comment_text + assert "github.event.pull_request.head.repo.full_name == github.repository" in build_text + assert "github.event.workflow_run.head_repository.full_name == github.repository" in comment_text + assert "github.event.repository.default_branch" in comment_text + assert "working-directory: ./preview-source/docs/fern" in comment_text + assert '--id "pr-$PR_NUMBER"' in comment_text + assert ".preview-metadata/pr_number" not in comment_text + assert ".preview-metadata/head_ref" not in comment_text + assert '.user.login == "github-actions[bot]"' in comment_text + + +def test_fern_publish_serializes_runs_and_isolates_write_permission() -> None: + """Only the redirect deployment may receive repository write permission.""" + workflow = _load_workflow(FERN_PUBLISH_PATH) + jobs = workflow["jobs"] + + assert workflow["permissions"] == {} + assert workflow["concurrency"] == { + "group": "fern-docs-website", + "cancel-in-progress": "false", + } + assert jobs["publish"]["permissions"] == {"contents": "read"} + assert jobs["redirects"]["permissions"] == { + "actions": "read", + "contents": "write", + } + assert "DOCS_FERN_TOKEN" in yaml.safe_dump(jobs["publish"]) + assert "DOCS_FERN_TOKEN" not in yaml.safe_dump(jobs["redirects"]) + + +def test_fern_workflows_pin_actions_and_use_supported_node() -> None: + """Secret-bearing and docs validation workflows use immutable action pins and Node 24.""" + paths = ( + FERN_CI_WORKFLOW_PATH, + FERN_PREVIEW_BUILD_PATH, + FERN_PREVIEW_COMMENT_PATH, + FERN_PUBLISH_PATH, + ) + action_pattern = re.compile(r"^\s*uses:\s+([^#\s]+)", re.MULTILINE) + + for path in paths: + text = path.read_text(encoding="utf-8") + for action in action_pattern.findall(text): + if action.startswith("./"): + continue + assert re.fullmatch(r"[^@]+@[0-9a-f]{40}", action), f"{path}: {action}" + + for path in (FERN_CI_WORKFLOW_PATH, FERN_PREVIEW_COMMENT_PATH, FERN_PUBLISH_PATH): + assert 'node-version: "24"' in path.read_text(encoding="utf-8") diff --git a/tests/test_mkdocs_repository_links.py b/tests/test_mkdocs_repository_links.py deleted file mode 100644 index 38952c68..00000000 --- a/tests/test_mkdocs_repository_links.py +++ /dev/null @@ -1,209 +0,0 @@ -# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. -# SPDX-License-Identifier: Apache-2.0 - -"""Tests for repository-relative links in MkDocs pages.""" - -import re -from pathlib import Path -from types import SimpleNamespace - -from mkdocs_hooks import on_page_markdown, rewrite_repository_links - -REPO_ROOT = Path(__file__).resolve().parents[1] -REPO_URL = "https://github.com/example/switchyard" -SOURCE_REF = "docs-ref" - - -class _Config(dict[str, object]): - def __init__(self, repo_root: Path, docs_dir: Path) -> None: - super().__init__( - docs_dir=str(docs_dir), - repo_url=REPO_URL, - extra={"source_ref": SOURCE_REF}, - ) - self.config_file_path = str(repo_root / "mkdocs.yml") - - -def _repository( - tmp_path: Path, page_relative_path: str = "docs/guides/page.md" -) -> tuple[Path, Path, Path]: - repo_root = tmp_path / "repo" - docs_dir = repo_root / "docs" - page_path = repo_root / page_relative_path - page_path.parent.mkdir(parents=True) - page_path.write_text("# Page\n", encoding="utf-8") - (repo_root / "AGENTS.md").write_text("# Instructions\n", encoding="utf-8") - (repo_root / "examples" / "prometheus").mkdir(parents=True) - (docs_dir / "index.md").write_text("# Home\n", encoding="utf-8") - return repo_root, docs_dir, page_path - - -def _rewrite(markdown: str, repo_root: Path, docs_dir: Path, page_path: Path) -> str: - return rewrite_repository_links( - markdown, - source_path=page_path, - docs_dir=docs_dir, - repo_root=repo_root, - repo_url=REPO_URL, - source_ref=SOURCE_REF, - ) - - -def test_repository_file_and_directory_links_use_blob_and_tree_urls(tmp_path: Path) -> None: - repo_root, docs_dir, page_path = _repository(tmp_path) - markdown = "[file](../../AGENTS.md)\n[directory](../../examples/prometheus/)\n" - - rewritten = _rewrite(markdown, repo_root, docs_dir, page_path) - - assert rewritten == ( - f"[file]({REPO_URL}/blob/{SOURCE_REF}/AGENTS.md)\n" - f"[directory]({REPO_URL}/tree/{SOURCE_REF}/examples/prometheus)\n" - ) - - -def test_hook_resolves_links_from_a_nested_page_and_uses_mkdocs_config(tmp_path: Path) -> None: - repo_root, docs_dir, page_path = _repository(tmp_path, "docs/routing/deep/page.md") - page = SimpleNamespace(file=SimpleNamespace(abs_src_path=str(page_path))) - - rewritten = on_page_markdown( - "[instructions](../../../AGENTS.md)", - page=page, - config=_Config(repo_root, docs_dir), - files=object(), - ) - - assert rewritten == f"[instructions]({REPO_URL}/blob/{SOURCE_REF}/AGENTS.md)" - - -def test_repository_links_preserve_query_strings_and_fragments(tmp_path: Path) -> None: - repo_root, docs_dir, page_path = _repository(tmp_path) - - rewritten = _rewrite( - "[instructions](../../AGENTS.md?plain=1&view=raw#setup)", - repo_root, - docs_dir, - page_path, - ) - - assert rewritten == ( - f"[instructions]({REPO_URL}/blob/{SOURCE_REF}/AGENTS.md?plain=1&view=raw#setup)" - ) - - -def test_repository_links_in_fenced_code_blocks_are_not_rewritten(tmp_path: Path) -> None: - repo_root, docs_dir, page_path = _repository(tmp_path) - markdown = ( - "```markdown\n" - "[backtick fence](../../AGENTS.md)\n" - "```\n" - "~~~markdown\n" - "[tilde fence](../../AGENTS.md)\n" - "~~~\n" - "[live link](../../AGENTS.md)\n" - ) - - rewritten = _rewrite(markdown, repo_root, docs_dir, page_path) - - assert rewritten == markdown.replace( - "[live link](../../AGENTS.md)", - f"[live link]({REPO_URL}/blob/{SOURCE_REF}/AGENTS.md)", - ) - - -def test_repository_links_in_blockquote_fences_are_not_rewritten(tmp_path: Path) -> None: - repo_root, docs_dir, page_path = _repository(tmp_path) - markdown = ( - "> ~~~markdown\n" - "> [fenced link](../../AGENTS.md)\n" - "> ~~~\n" - "> Ordinary prose with ~~~ inline.\n" - "> [live link](../../AGENTS.md)\n" - ) - - rewritten = _rewrite(markdown, repo_root, docs_dir, page_path) - - assert rewritten == markdown.replace( - "[live link](../../AGENTS.md)", - f"[live link]({REPO_URL}/blob/{SOURCE_REF}/AGENTS.md)", - ) - - -def test_repository_links_in_list_item_fences_are_not_rewritten(tmp_path: Path) -> None: - repo_root, docs_dir, page_path = _repository(tmp_path) - markdown = ( - "- Example:\n" - "\n" - " ~~~markdown\n" - " [fenced link](../../AGENTS.md)\n" - " ~~~\n" - "\n" - " Ordinary prose with ~~~ inline.\n" - " [live link](../../AGENTS.md)\n" - ) - - rewritten = _rewrite(markdown, repo_root, docs_dir, page_path) - - assert rewritten == markdown.replace( - "[live link](../../AGENTS.md)", - f"[live link]({REPO_URL}/blob/{SOURCE_REF}/AGENTS.md)", - ) - - -def test_indented_fence_markers_do_not_hide_outdented_prose_links(tmp_path: Path) -> None: - repo_root, docs_dir, page_path = _repository(tmp_path) - markdown = ( - " ~~~\n" - "[live link](../../AGENTS.md)\n" - " ~~~\n" - ) - - rewritten = _rewrite(markdown, repo_root, docs_dir, page_path) - - assert rewritten == markdown.replace( - "[live link](../../AGENTS.md)", - f"[live link]({REPO_URL}/blob/{SOURCE_REF}/AGENTS.md)", - ) - - -def test_missing_repository_target_remains_relative(tmp_path: Path) -> None: - repo_root, docs_dir, page_path = _repository(tmp_path) - markdown = "[missing](../../missing.md?plain=1#section)" - - assert _rewrite(markdown, repo_root, docs_dir, page_path) == markdown - - -def test_existing_target_outside_repository_remains_relative(tmp_path: Path) -> None: - repo_root, docs_dir, page_path = _repository(tmp_path) - (tmp_path / "outside.md").write_text("# Outside\n", encoding="utf-8") - markdown = "[outside](../../../outside.md)" - - assert _rewrite(markdown, repo_root, docs_dir, page_path) == markdown - - -def test_links_inside_docs_dir_and_non_relative_links_are_unchanged(tmp_path: Path) -> None: - repo_root, docs_dir, page_path = _repository(tmp_path) - markdown = ( - "[docs page](../index.md)\n" - "[external](https://example.com/docs)\n" - "[page anchor](#section)\n" - ) - - assert _rewrite(markdown, repo_root, docs_dir, page_path) == markdown - - -def test_docs_do_not_hard_code_same_repository_blob_or_tree_links() -> None: - config = (REPO_ROOT / "mkdocs.yml").read_text(encoding="utf-8") - repo_url_match = re.search(r"^repo_url:\s*(\S+)\s*$", config, flags=re.MULTILINE) - assert repo_url_match is not None - repo_url = repo_url_match.group(1).strip("'\"").rstrip("/") - source_link = re.compile(rf"{re.escape(repo_url)}/(?:blob|tree)/") - markdown_paths = [REPO_ROOT / "README.md", *(REPO_ROOT / "docs").rglob("*.md")] - - offenders = [ - f"{path.relative_to(REPO_ROOT)}:{line_number}: {line.strip()}" - for path in markdown_paths - for line_number, line in enumerate(path.read_text(encoding="utf-8").splitlines(), start=1) - if source_link.search(line) - ] - - assert not offenders, "Hard-coded same-repository source links:\n" + "\n".join(offenders) diff --git a/tests/test_select_validation.py b/tests/test_select_validation.py new file mode 100644 index 00000000..c4debf22 --- /dev/null +++ b/tests/test_select_validation.py @@ -0,0 +1,68 @@ +# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +"""Tests for path-based local validation selection and its contributor guidance.""" + +from __future__ import annotations + +import runpy +from pathlib import Path +from typing import Any + +REPO_ROOT = Path(__file__).resolve().parents[1] +SCRIPT_PATH = REPO_ROOT / "scripts" / "select_validation.py" +GUIDE_PATH = REPO_ROOT / "scripts" / "ci" / "README.md" + + +def _load_script() -> dict[str, Any]: + """Load the validation selector without making scripts a Python package.""" + return runpy.run_path(str(SCRIPT_PATH)) + + +def _names(*paths: str) -> set[str]: + """Return validation names selected for the supplied paths.""" + module = _load_script() + return {item.name for item in module["select_validations"](paths)} + + +def test_fern_workflow_selects_docs_and_security_validation() -> None: + """Fern workflow edits need both product checks and workflow security checks.""" + names = _names(".github/workflows/fern-docs-preview-comment.yml") + assert names == {"Fern docs", "Workflow syntax and security"} + + +def test_native_or_package_changes_select_all_affected_gates() -> None: + """Native package metadata must cover Python, Rust, tests, and package construction.""" + names = _names("Cargo.lock", "switchyard_rust/__init__.py") + assert names == { + "Offline test suite", + "Package build", + "Python quality gates", + "Rust quality gates", + } + + +def test_docs_only_change_does_not_select_runtime_or_live_tests() -> None: + """Ordinary docs changes stay focused and never opt into provider calls.""" + validations = _load_script()["select_validations"](["docs/operations/context_window.mdx"]) + assert [item.name for item in validations] == ["Fern docs"] + assert all("integration" not in item.command for item in validations) + + +def test_ci_guide_records_required_security_invariants() -> None: + """The guide must retain the cross-repository workflow design lessons.""" + guide = GUIDE_PATH.read_text(encoding="utf-8") + for invariant in ( + "Stable required aggregation", + "Trust boundaries", + "Concurrency", + "Permissions and secrets", + "Supply chain and toolchains", + "same-repository PRs", + "CI Success", + "actionlint", + "zizmor", + "NeMo Curator", + "NeMo Data Designer", + ): + assert invariant in guide