From a5b88ad82a60c2460a7c71361569b8e9a6871d87 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" Date: Mon, 27 Jul 2026 10:07:21 -0400 Subject: [PATCH] Add shared parallel spector scenario compiler (@azure-tools/spector-runner) Add a shared, parallel spector scenario compiler in the new @azure-tools/spector-runner package, plus a wrapper-free CLI with a declarative spector.config.yaml (specsRoot/outputDir/compileConfig + postCompile/postCompileDeclarations hooks, selectable via --phase all|compile|declarations) and a defineConfig module mode for emitters needing more. Move the Go, TypeScript and Python regenerators onto this one parallel-compile + reporting + lifecycle engine; output is byte-identical to the previous per-emitter drivers. Fixes #5022 --- ...r-scenario-compiler-2026-07-24-11-00-00.md | 9 + .../instructions/typespec-ts.instructions.md | 64 ++-- packages/spector-runner/README.md | 199 ++++++++++ packages/spector-runner/cmd/spector-runner.js | 13 + packages/spector-runner/package.json | 8 + .../spector-runner/schema/spector-config.tsp | 34 ++ packages/spector-runner/src/cli.ts | 195 ++++++++++ packages/spector-runner/src/compiler.ts | 260 +++++++++++++ packages/spector-runner/src/concurrency.ts | 52 +++ packages/spector-runner/src/config.ts | 176 +++++++++ packages/spector-runner/src/hooks.ts | 116 ++++++ packages/spector-runner/src/index.ts | 22 ++ packages/spector-runner/src/loader.ts | 43 ++- packages/spector-runner/src/run.ts | 186 +++++++++ packages/spector-runner/src/task-runner.ts | 48 +++ packages/spector-runner/src/types.ts | 43 ++- packages/spector-runner/test/cli.test.ts | 43 +++ packages/spector-runner/test/compiler.test.ts | 161 ++++++++ packages/spector-runner/test/config.test.ts | 122 ++++++ packages/spector-runner/test/hooks.test.ts | 67 ++++ packages/spector-runner/test/loader.test.ts | 38 ++ packages/spector-runner/test/run.test.ts | 206 ++++++++++ .../.scripts/azure-rest-api-specs.json | 2 +- packages/typespec-go/.scripts/go-test.js | 2 +- packages/typespec-go/.scripts/lint-go.js | 2 +- packages/typespec-go/.scripts/semaphore.js | 104 ----- .../.scripts/spector-runner.config.js | 183 +++++++++ .../.scripts/sync-azure-rest-api-specs.js | 2 +- packages/typespec-go/.scripts/tspcompile.js | 284 -------------- packages/typespec-go/.scripts/tspconfig.yaml | 2 +- packages/typespec-go/package.json | 2 +- .../eng/scripts/ci/regenerate.ts | 120 ++++-- packages/typespec-ts/package.json | 6 +- packages/typespec-ts/spector.config.yaml | 22 ++ .../test/commands/emit-declarations.ts | 174 +++++++++ .../typespec-ts/test/commands/gen-spector.js | 71 ---- .../typespec-ts/test/commands/post-compile.js | 55 +++ packages/typespec-ts/test/commands/run.ts | 355 ------------------ pnpm-lock.yaml | 3 + 39 files changed, 2606 insertions(+), 888 deletions(-) create mode 100644 .chronus/changes/spector-scenario-compiler-2026-07-24-11-00-00.md create mode 100755 packages/spector-runner/cmd/spector-runner.js create mode 100644 packages/spector-runner/src/cli.ts create mode 100644 packages/spector-runner/src/compiler.ts create mode 100644 packages/spector-runner/src/concurrency.ts create mode 100644 packages/spector-runner/src/config.ts create mode 100644 packages/spector-runner/src/hooks.ts create mode 100644 packages/spector-runner/src/run.ts create mode 100644 packages/spector-runner/src/task-runner.ts create mode 100644 packages/spector-runner/test/cli.test.ts create mode 100644 packages/spector-runner/test/compiler.test.ts create mode 100644 packages/spector-runner/test/config.test.ts create mode 100644 packages/spector-runner/test/hooks.test.ts create mode 100644 packages/spector-runner/test/run.test.ts delete mode 100644 packages/typespec-go/.scripts/semaphore.js create mode 100644 packages/typespec-go/.scripts/spector-runner.config.js delete mode 100644 packages/typespec-go/.scripts/tspcompile.js create mode 100644 packages/typespec-ts/test/commands/emit-declarations.ts delete mode 100644 packages/typespec-ts/test/commands/gen-spector.js create mode 100644 packages/typespec-ts/test/commands/post-compile.js delete mode 100644 packages/typespec-ts/test/commands/run.ts diff --git a/.chronus/changes/spector-scenario-compiler-2026-07-24-11-00-00.md b/.chronus/changes/spector-scenario-compiler-2026-07-24-11-00-00.md new file mode 100644 index 0000000000..503fe9a76a --- /dev/null +++ b/.chronus/changes/spector-scenario-compiler-2026-07-24-11-00-00.md @@ -0,0 +1,9 @@ +--- +changeKind: internal +packages: + - "@azure-tools/typespec-go" + - "@azure-tools/typespec-ts" + - "@azure-tools/typespec-python" +--- + +Add a shared, parallel spector scenario compiler to the `@azure-tools/spector-runner` package, plus a wrapper-free `spector-runner` CLI with a declarative config + hook system, and move the Go, TypeScript (`@azure-tools/typespec-ts`) and Python (`@azure-tools/typespec-python`) regenerators onto it. `compileScenarios` spawns one `tsp compile` subprocess per scenario (up to N at a time, N = CPU count by default) and reports progress through a `TaskRunner`/`runWithConcurrency`/`=== Summary ===` engine shared with `@typespec/tsp-integration`, so the two can be merged later. The `spector-runner` CLI (and its `runSpectorRunner`/`buildScenarios` programmatic API) builds every scenario in one or more `spector.config.yaml` files and compiles them in parallel for a given emitter — driven entirely by flags (`--config`, `--specs-root`, `--emit`, `--output-dir`, `--option`, `--cwd`, ...). The `spector.config.yaml` schema can itself declare the output layout (`specsRoot`, `outputDir`, `compileConfig`) and per-instance lifecycle hooks (`hooks.postCompile`, `hooks.postCompileDeclarations`) as shell commands, selectable with `--phase all|compile|declarations`; hook commands receive `SPECTOR_OUTPUT_DIR`/`SPECTOR_SPEC_PATH`/`SPECTOR_SPEC_NAME`/`SPECTOR_PHASE`. This lets the TypeScript regenerator drop its bespoke driver/config JS entirely: `pnpm generate-tsp-only` is now just `spector-runner --config spector.config.yaml`, with two short hook scripts for the client post-processing and the api-extractor declarations phase. For emitters that need more (extra spec roots, local specs, cross-cutting pre/post steps), a `spector-runner.config.{js,ts}` module (`defineConfig`) declares the whole run — `scenarios` plus `preRun`/`postScenario`/`postRun` hooks — and the CLI drives it end-to-end via `--config-file` (or `runConfig`/`loadConfigFile` programmatically). This lets the per-emitter regenerators shrink (Go and TS now run entirely through the bare CLI; Python keeps only its upstream-synced option tables + flavor/baseline setup in a `--config-file` module). A `CompileScenario` may also set a per-scenario `cwd` (used by the TS regenerator, whose committed per-output `tspconfig.yaml` is resolved relative to each output folder). The Go, TS and Python regenerators now share this one parallel-compile + reporting + lifecycle engine; output is byte-identical to the previous per-emitter drivers. diff --git a/.github/instructions/typespec-ts.instructions.md b/.github/instructions/typespec-ts.instructions.md index 0167551cc8..1e0d772a19 100644 --- a/.github/instructions/typespec-ts.instructions.md +++ b/.github/instructions/typespec-ts.instructions.md @@ -71,18 +71,25 @@ Integration (spector) tests generate real clients from specs, then assert on the though the rest of `test/integration/` is gone). It also copies assets to `./temp/assets`. It is a cross-platform Node script (`test/commands/copy-typespec.ts`) and runs the same on Windows, macOS, and Linux. -2. `test/commands/gen-spector.js` picks the `azureModularTsps` list from - `test/commands/spector-list.js` (the only spec set) and runs `test/commands/run.ts` - for each entry, emitting into `test/azure-modular-integration/generated/`. +2. `spector-runner --config spector.config.yaml` (the shared + `@azure-tools/spector-runner` CLI) reads the opt-in spec list from + `spector.config.yaml` and compiles each in parallel, emitting into + `test/azure-modular-integration/generated/`. Each spec compiles + using the committed `tspconfig.yaml` in its output folder (which names the + emitter + per-package options), then the config's per-instance hooks run. Generation is split into two phases via `--phase` (default `all` runs both): - - `--phase=client` emits the `src/*.ts` sources the tests import. It compiles via - `node <@typespec/compiler>/cmd/tsp.js compile` (resolved once per process) rather - than `npx tsp`, which avoids ~5 s/spec of npx re-resolution while keeping one fresh - subprocess per compile. - - `--phase=declarations` emits the tracked `src/index.d.ts` baseline (tsc `.d.ts` + - api-extractor rollup). Only `check:tree` consumes these, so the e2e script - `generate-and-run` runs the vitest suite in parallel with the - declaration regen, keeping the (slow) api-extractor work off the test critical path. + - `--phase compile` emits the `src/*.ts` sources the tests import (one fresh + `tsp compile` subprocess per spec), then runs the `postCompile` hook + (`test/commands/post-compile.js`) to write each package's `.gitignore` and a + self-contained test `tsconfig.json`. + - `--phase declarations` runs the `postCompileDeclarations` hook + (`test/commands/emit-declarations.ts`) per spec — tsc `.d.ts` + + api-extractor rollup into the tracked `src/index.d.ts` — **without + recompiling**. Only `check:tree` consumes these, so the e2e script + `generate-and-run` runs the vitest suite in parallel with the declaration + regen, keeping the (slow) api-extractor work off the test critical path. + Each hook is a short script that reads `SPECTOR_OUTPUT_DIR` (the spec's output + folder); the shared CLI sets it per instance. 3. The vitest `integration-azure-modular` project then runs the `*.test.ts` assertions. `pnpm regen-test-baselines` (alias of `generate-tsp-only`, which runs both phases) @@ -94,9 +101,9 @@ Each generated package writes a `.gitignore` that ignores everything except `src/index.d.ts`, `.gitignore`, and `tspconfig.yaml`. So a generated folder is full of files on disk (`src/*.ts`, `types/`, `temp/`), but git only tracks the rolled-up `src/index.d.ts` (produced by the api-extractor "dtsRollup" pass in the `declarations` -phase of `run.ts`). The `client` phase rewrites `src/` and therefore _removes_ -`src/index.d.ts`; the `declarations` phase restores it byte-for-byte — so both phases -must run before `check:tree`. +phase, `test/commands/emit-declarations.ts`). The `compile` phase rewrites `src/` and +therefore _removes_ `src/index.d.ts`; the `declarations` phase restores it byte-for-byte +— so both phases must run before `check:tree`. ## CI: `e2e-test` job in `.github/workflows/ci-typescript.yml` @@ -109,18 +116,21 @@ So a baseline that doesn't match freshly generated output (changed, missing, or ## Gotchas - **Command scripts run on `node`, not `tsx`.** The `test/commands/*` scripts (including - `copy:typespec`, `gen-spector.js`, `check:tree`, and `gen:scenario-suites`) are executed - directly with `node`, which strips TypeScript types natively — this requires **Node >= - 22.18**. When adding or editing one: import sibling `.ts` files with an explicit `.ts` - specifier (node does not remap `.js` -> `.ts` the way tsx did), and use `import type` for - type-only names from CommonJS deps such as `typescript` (`CompilerOptions`) and - `@microsoft/api-extractor` (`IExtractorConfigPrepareOptions`) — otherwise node tries to - load them as runtime named exports and throws. `copy:typespec` is cross-platform, so the - old Windows workaround (replicating Unix `rm`/`cp` by hand) is no longer needed. + `copy:typespec`, `check:tree`, `post-compile.js`, `emit-declarations.ts`, and + `gen:scenario-suites`) are executed directly with `node`, which strips TypeScript types + natively — this requires **Node >= 22.18**. When adding or editing one: import sibling + `.ts` files with an explicit `.ts` specifier (node does not remap `.js` -> `.ts` the way + tsx did), and use `import type` for type-only names from CommonJS deps such as + `typescript` (`CompilerOptions`) and `@microsoft/api-extractor` + (`IExtractorConfigPrepareOptions`) — otherwise node tries to load them as runtime named + exports and throws. The spector hooks are invoked by the shared `spector-runner` CLI as + shell commands (see `hooks` in `spector.config.yaml`); each reads `SPECTOR_OUTPUT_DIR`. + `copy:typespec` is cross-platform, so the old Windows workaround (replicating Unix + `rm`/`cp` by hand) is no longer needed. - **Never `git add -A` after regenerating baselines.** The api-extractor rollup step can intermittently fail to (re)write `src/index.d.ts` for a few specs (concurrent workers in - gen-spector share an api-extractor temp workspace per package). The folder still - regenerates, but the tracked rollup goes missing — which `git add -A` silently stages as - a deletion. Review `git status` for _unexpected_ deletions/additions and stage baseline - changes by explicit path before committing. If a baseline went missing this way, restore - it from the previous commit rather than re-deleting it. + the `declarations` phase share an api-extractor temp workspace per package). The folder + still regenerates, but the tracked rollup goes missing — which `git add -A` silently + stages as a deletion. Review `git status` for _unexpected_ deletions/additions and stage + baseline changes by explicit path before committing. If a baseline went missing this way, + restore it from the previous commit rather than re-deleting it. diff --git a/packages/spector-runner/README.md b/packages/spector-runner/README.md index 08b93eebd1..627f2c7cce 100644 --- a/packages/spector-runner/README.md +++ b/packages/spector-runner/README.md @@ -44,3 +44,202 @@ for (const { path, options } of resolveSpecs(config)) { // compile `path` with `options` } ``` + +## Compiling scenarios + +`compileScenarios` builds a set of scenarios in parallel by spawning one +`tsp compile` subprocess per scenario (up to `jobs` at a time, defaulting to the +number of CPUs). It resolves the local `tsp` CLI from `cwd`, reports progress +through a `TaskRunner` (shared with `@typespec/tsp-integration`), and prints a +`=== Summary ===` block at the end. + +```ts +import { compileScenarios, resolveSpecEntrypoint } from "@azure-tools/spector-runner"; + +const scenarios = resolveSpecs(config).map(({ path, options }) => ({ + name: path, + entrypoint: resolveSpecEntrypoint(specsRoot, path), + emit: ["@azure-tools/typespec-go"], + options: { "@azure-tools/typespec-go": options }, +})); + +const summary = await compileScenarios(scenarios, { + jobs: 8, + cwd: emitterPackageRoot, // where `tsp` and the emitter are installed + config: "path/to/stub/tspconfig.yaml", // optional: passed as --config to every compile + onScenarioComplete: (result) => { + // per-scenario post-processing (runs after each subprocess exits) + }, +}); + +console.log(`${summary.succeeded}/${summary.results.length} succeeded`); +``` + +`resolveSpecEntrypoint` maps a config spec-path key to its entry file (an +explicit `.tsp` key, otherwise `client.tsp` preferred over `main.tsp`). + +Each `CompileScenario` may set a per-scenario `cwd`, overriding the run-wide +`cwd` for that one compile. Use it when scenarios must run in their own output +folder — e.g. the TypeScript regenerator, where every generated folder ships a +committed `tspconfig.yaml` resolved relative to it (`--config tspconfig.yaml`). +The `tsp` CLI is still located from the run-wide `cwd`. + +`TaskRunner` and `runWithConcurrency` are also exported for reuse. + +The Go (`typespec-go/.scripts/spector-runner.config.js`), TypeScript +(`typespec-ts/test/commands/spector-runner.config.js`) and Python +(`typespec-python/eng/scripts/ci/regenerate.ts`) regenerators all drive their +spec compilation through this engine — see [Config files & hooks](#config-files--hooks). + +## `spector-runner` CLI + +For the common case you don't need any wrapper script — the `spector-runner` +bin builds every scenario in a `spector.config.yaml` and compiles them in +parallel for a given emitter: + +```bash +spector-runner \ + --config path/to/spector.config.yaml \ + --specs-root path/to/specs \ + --emit . \ + --output-dir "generated/{parentDir}/{options.module}" +``` + +Key flags (see `spector-runner --help` for the full list): + +- **`--config `** — a `spector.config.yaml` (repeatable to merge several). +- **`--specs-root `** — root the spec-path keys are relative to. Defaults to + the config's `specsRoot`. +- **`--emit, --emitter `** — emitter package name or path (`.` for the local + package; repeatable). Path-like values are resolved and their `package.json` + `name` is used to namespace options. Optional when the config sets + `compileConfig` (the emitter then comes from each committed `tspconfig.yaml`). +- **`--output-dir