Azure · timotheeguerin · Jul 27, 2026
@@ -0,0 +1,9 @@
+---
+changeKind: internal
+packages:
+  - "@azure-tools/typespec-go"
+  - "@azure-tools/typespec-ts"
+  - "@azure-tools/typespec-python"
+---
+
+Add a shared, parallel spector scenario compiler to the `@azure-tools/spector-runner` package, plus a wrapper-free `spector-runner` CLI with a declarative config + hook system, and move the Go, TypeScript (`@azure-tools/typespec-ts`) and Python (`@azure-tools/typespec-python`) regenerators onto it. `compileScenarios` spawns one `tsp compile` subprocess per scenario (up to N at a time, N = CPU count by default) and reports progress through a `TaskRunner`/`runWithConcurrency`/`=== Summary ===` engine shared with `@typespec/tsp-integration`, so the two can be merged later. The `spector-runner` CLI (and its `runSpectorRunner`/`buildScenarios` programmatic API) builds every scenario in one or more `spector.config.yaml` files and compiles them in parallel for a given emitter — driven entirely by flags (`--config`, `--specs-root`, `--emit`, `--output-dir`, `--option`, `--cwd`, ...). The `spector.config.yaml` schema can itself declare the output layout (`specsRoot`, `outputDir`, `compileConfig`) and per-instance lifecycle hooks (`hooks.postCompile`, `hooks.postCompileDeclarations`) as shell commands, selectable with `--phase all|compile|declarations`; hook commands receive `SPECTOR_OUTPUT_DIR`/`SPECTOR_SPEC_PATH`/`SPECTOR_SPEC_NAME`/`SPECTOR_PHASE`. This lets the TypeScript regenerator drop its bespoke driver/config JS entirely: `pnpm generate-tsp-only` is now just `spector-runner --config spector.config.yaml`, with two short hook scripts for the client post-processing and the api-extractor declarations phase. For emitters that need more (extra spec roots, local specs, cross-cutting pre/post steps), a `spector-runner.config.{js,ts}` module (`defineConfig`) declares the whole run — `scenarios` plus `preRun`/`postScenario`/`postRun` hooks — and the CLI drives it end-to-end via `--config-file` (or `runConfig`/`loadConfigFile` programmatically). This lets the per-emitter regenerators shrink (Go and TS now run entirely through the bare CLI; Python keeps only its upstream-synced option tables + flavor/baseline setup in a `--config-file` module). A `CompileScenario` may also set a per-scenario `cwd` (used by the TS regenerator, whose committed per-output `tspconfig.yaml` is resolved relative to each output folder). The Go, TS and Python regenerators now share this one parallel-compile + reporting + lifecycle engine; output is byte-identical to the previous per-emitter drivers.
@@ -71,18 +71,25 @@ Integration (spector) tests generate real clients from specs, then assert on the
    though the rest of `test/integration/` is gone). It also copies assets to `./temp/assets`.
    It is a cross-platform Node script (`test/commands/copy-typespec.ts`) and runs the same
    on Windows, macOS, and Linux.
-2. `test/commands/gen-spector.js` picks the `azureModularTsps` list from
-   `test/commands/spector-list.js` (the only spec set) and runs `test/commands/run.ts`
-   for each entry, emitting into `test/azure-modular-integration/generated/<outputPath>`.
+2. `spector-runner --config spector.config.yaml` (the shared
+   `@azure-tools/spector-runner` CLI) reads the opt-in spec list from
+   `spector.config.yaml` and compiles each in parallel, emitting into
+   `test/azure-modular-integration/generated/<outputPath>`. Each spec compiles
+   using the committed `tspconfig.yaml` in its output folder (which names the
+   emitter + per-package options), then the config's per-instance hooks run.
    Generation is split into two phases via `--phase` (default `all` runs both):
-   - `--phase=client` emits the `src/*.ts` sources the tests import. It compiles via
-     `node <@typespec/compiler>/cmd/tsp.js compile` (resolved once per process) rather
-     than `npx tsp`, which avoids ~5 s/spec of npx re-resolution while keeping one fresh
-     subprocess per compile.
-   - `--phase=declarations` emits the tracked `src/index.d.ts` baseline (tsc `.d.ts` +
-     api-extractor rollup). Only `check:tree` consumes these, so the e2e script
-     `generate-and-run` runs the vitest suite in parallel with the
-     declaration regen, keeping the (slow) api-extractor work off the test critical path.
+   - `--phase compile` emits the `src/*.ts` sources the tests import (one fresh
+     `tsp compile` subprocess per spec), then runs the `postCompile` hook
+     (`test/commands/post-compile.js`) to write each package's `.gitignore` and a
+     self-contained test `tsconfig.json`.
+   - `--phase declarations` runs the `postCompileDeclarations` hook
+     (`test/commands/emit-declarations.ts`) per spec — tsc `.d.ts` +
+     api-extractor rollup into the tracked `src/index.d.ts` — **without
+     recompiling**. Only `check:tree` consumes these, so the e2e script
+     `generate-and-run` runs the vitest suite in parallel with the declaration
+     regen, keeping the (slow) api-extractor work off the test critical path.
+   Each hook is a short script that reads `SPECTOR_OUTPUT_DIR` (the spec's output
+   folder); the shared CLI sets it per instance.
 3. The vitest `integration-azure-modular` project then runs the `*.test.ts` assertions.
 
 `pnpm regen-test-baselines` (alias of `generate-tsp-only`, which runs both phases)
@@ -94,9 +101,9 @@ Each generated package writes a `.gitignore` that ignores everything except
 `src/index.d.ts`, `.gitignore`, and `tspconfig.yaml`. So a generated folder is full of
 files on disk (`src/*.ts`, `types/`, `temp/`), but git only tracks the rolled-up
 `src/index.d.ts` (produced by the api-extractor "dtsRollup" pass in the `declarations`
-phase of `run.ts`). The `client` phase rewrites `src/` and therefore _removes_
-`src/index.d.ts`; the `declarations` phase restores it byte-for-byte — so both phases
-must run before `check:tree`.
+phase, `test/commands/emit-declarations.ts`). The `compile` phase rewrites `src/` and
+therefore _removes_ `src/index.d.ts`; the `declarations` phase restores it byte-for-byte
+— so both phases must run before `check:tree`.
 
 ## CI: `e2e-test` job in `.github/workflows/ci-typescript.yml`
 
@@ -109,18 +116,21 @@ So a baseline that doesn't match freshly generated output (changed, missing, or
 ## Gotchas
 
 - **Command scripts run on `node`, not `tsx`.** The `test/commands/*` scripts (including
-  `copy:typespec`, `gen-spector.js`, `check:tree`, and `gen:scenario-suites`) are executed
-  directly with `node`, which strips TypeScript types natively — this requires **Node >=
-  22.18**. When adding or editing one: import sibling `.ts` files with an explicit `.ts`
-  specifier (node does not remap `.js` -> `.ts` the way tsx did), and use `import type` for
-  type-only names from CommonJS deps such as `typescript` (`CompilerOptions`) and
-  `@microsoft/api-extractor` (`IExtractorConfigPrepareOptions`) — otherwise node tries to
-  load them as runtime named exports and throws. `copy:typespec` is cross-platform, so the
-  old Windows workaround (replicating Unix `rm`/`cp` by hand) is no longer needed.
+  `copy:typespec`, `check:tree`, `post-compile.js`, `emit-declarations.ts`, and
+  `gen:scenario-suites`) are executed directly with `node`, which strips TypeScript types
+  natively — this requires **Node >= 22.18**. When adding or editing one: import sibling
+  `.ts` files with an explicit `.ts` specifier (node does not remap `.js` -> `.ts` the way
+  tsx did), and use `import type` for type-only names from CommonJS deps such as
+  `typescript` (`CompilerOptions`) and `@microsoft/api-extractor`
+  (`IExtractorConfigPrepareOptions`) — otherwise node tries to load them as runtime named
+  exports and throws. The spector hooks are invoked by the shared `spector-runner` CLI as
+  shell commands (see `hooks` in `spector.config.yaml`); each reads `SPECTOR_OUTPUT_DIR`.
+  `copy:typespec` is cross-platform, so the old Windows workaround (replicating Unix
+  `rm`/`cp` by hand) is no longer needed.
 - **Never `git add -A` after regenerating baselines.** The api-extractor rollup step can
   intermittently fail to (re)write `src/index.d.ts` for a few specs (concurrent workers in
-  gen-spector share an api-extractor temp workspace per package). The folder still
-  regenerates, but the tracked rollup goes missing — which `git add -A` silently stages as
-  a deletion. Review `git status` for _unexpected_ deletions/additions and stage baseline
-  changes by explicit path before committing. If a baseline went missing this way, restore
-  it from the previous commit rather than re-deleting it.
+  the `declarations` phase share an api-extractor temp workspace per package). The folder
+  still regenerates, but the tracked rollup goes missing — which `git add -A` silently
+  stages as a deletion. Review `git status` for _unexpected_ deletions/additions and stage
+  baseline changes by explicit path before committing. If a baseline went missing this way,
+  restore it from the previous commit rather than re-deleting it.
@@ -44,3 +44,202 @@ for (const { path, options } of resolveSpecs(config)) {
   // compile `path` with `options`
 }
 ```
+
+## Compiling scenarios
+
+`compileScenarios` builds a set of scenarios in parallel by spawning one
+`tsp compile` subprocess per scenario (up to `jobs` at a time, defaulting to the
+number of CPUs). It resolves the local `tsp` CLI from `cwd`, reports progress
+through a `TaskRunner` (shared with `@typespec/tsp-integration`), and prints a
+`=== Summary ===` block at the end.
+
+```ts
+import { compileScenarios, resolveSpecEntrypoint } from "@azure-tools/spector-runner";
+
+const scenarios = resolveSpecs(config).map(({ path, options }) => ({
+  name: path,
+  entrypoint: resolveSpecEntrypoint(specsRoot, path),
+  emit: ["@azure-tools/typespec-go"],
+  options: { "@azure-tools/typespec-go": options },
+}));
+
+const summary = await compileScenarios(scenarios, {
+  jobs: 8,
+  cwd: emitterPackageRoot, // where `tsp` and the emitter are installed
+  config: "path/to/stub/tspconfig.yaml", // optional: passed as --config to every compile
+  onScenarioComplete: (result) => {
+    // per-scenario post-processing (runs after each subprocess exits)
+  },
+});
+
+console.log(`${summary.succeeded}/${summary.results.length} succeeded`);
+```
+
+`resolveSpecEntrypoint` maps a config spec-path key to its entry file (an
+explicit `.tsp` key, otherwise `client.tsp` preferred over `main.tsp`).
+
+Each `CompileScenario` may set a per-scenario `cwd`, overriding the run-wide
+`cwd` for that one compile. Use it when scenarios must run in their own output
+folder — e.g. the TypeScript regenerator, where every generated folder ships a
+committed `tspconfig.yaml` resolved relative to it (`--config tspconfig.yaml`).
+The `tsp` CLI is still located from the run-wide `cwd`.
+
+`TaskRunner` and `runWithConcurrency` are also exported for reuse.
+
+The Go (`typespec-go/.scripts/spector-runner.config.js`), TypeScript
+(`typespec-ts/test/commands/spector-runner.config.js`) and Python
+(`typespec-python/eng/scripts/ci/regenerate.ts`) regenerators all drive their
+spec compilation through this engine — see [Config files & hooks](#config-files--hooks).
+
+## `spector-runner` CLI
+
+For the common case you don't need any wrapper script — the `spector-runner`
+bin builds every scenario in a `spector.config.yaml` and compiles them in
+parallel for a given emitter:
+
+```bash
+spector-runner \
+  --config path/to/spector.config.yaml \
+  --specs-root path/to/specs \
+  --emit . \
+  --output-dir "generated/{parentDir}/{options.module}"
+```
+
+Key flags (see `spector-runner --help` for the full list):
+
+- **`--config <path>`** — a `spector.config.yaml` (repeatable to merge several).
+- **`--specs-root <dir>`** — root the spec-path keys are relative to. Defaults to
+  the config's `specsRoot`.
+- **`--emit, --emitter <e>`** — emitter package name or path (`.` for the local
+  package; repeatable). Path-like values are resolved and their `package.json`
+  `name` is used to namespace options. Optional when the config sets
+  `compileConfig` (the emitter then comes from each committed `tspconfig.yaml`).
+- **`--output-dir <template>`** — `emitter-output-dir` template. Placeholders:
+  `{path}` (spec key), `{dir}` (its directory, `.tsp` stripped), `{parentDir}`
+  (`{dir}` minus its last segment), `{outputPath}` (the `outputPath` option,
+  defaulting to `{path}`) and `{options.NAME}` (a spec option value). Defaults to
+  the config's `outputDir`.
+- **`--option <key=value>`** — default emitter option (repeatable); per-spec
+  config options override these by key.
+- **`--tspconfig <path>`** — stub `tspconfig.yaml` passed as `--config` to every
+  compile (prevents option bleed from an upstream config next to the specs).
+- **`--phase <all|compile|declarations>`** — which lifecycle phase(s) to run when
+  the config declares [hooks](#hooks-in-spectorconfigyaml). Default `all`.
+- **`--cwd <dir>`** — directory to run compiles/hooks in and resolve the local
+  `tsp` CLI + emitter. Defaults to `process.cwd()`.
+- **`--jobs <n>`**, **`--filter <regex>`**, **`--verbose`**.
+
+The same logic is available programmatically as `runSpectorRunner` (and the
+lower-level `buildScenarios`), which additionally accept `extraScenarios` and an
+`onScenarioComplete` hook so an emitter can add its own local specs and
+post-processing on top of the config-driven ones:
+
+```ts
+import { runSpectorRunner } from "@azure-tools/spector-runner";
+
+const summary = await runSpectorRunner({
+  config: ["http.spector.config.yaml", "azure.spector.config.yaml"],
+  specsRoot,
+  emit: ["."],
+  outputDir: "test/http-specs/{parentDir}/{options.module}",
+  cwd: emitterPackageRoot,
+  extraScenarios: [...emitterLocalSpecs],
+  onScenarioComplete: ({ scenario, success }) => {
+    // metadata patch / cleanup
+  },
+});
+```
+
+## Hooks in `spector.config.yaml`
+
+For the common case an emitter needs no wrapper module at all: declare the
+output layout and per-instance pre/post steps directly in `spector.config.yaml`
+and drive it with the bare CLI. Each spec compiles using the committed
+`tspconfig.yaml` in its output folder, then the hooks run.
+
+```yaml
+# spector.config.yaml
+specsRoot: temp/specs # default for --specs-root
+outputDir: generated/{outputPath} # {outputPath} = the outputPath option, defaulting to the spec key
+compileConfig: tspconfig.yaml # compile using this committed config inside each output folder
+
+hooks:
+  # A shell command run once per spec. SPECTOR_OUTPUT_DIR points at the output
+  # folder; SPECTOR_SPEC_PATH / SPECTOR_SPEC_NAME / SPECTOR_PHASE are also set.
+  postCompile: node ./scripts/post-compile.js
+  postCompileDeclarations: node ./scripts/emit-declarations.ts
+
+specs:
+  routes: true
+  versioning/removed:
+    - { options: { outputPath: versioning/removed/v1 } }
+    - { options: { outputPath: versioning/removed/v2 } }
+```
+
+```bash
+spector-runner --config spector.config.yaml --phase all
+```
+
+- **`compileConfig`** — when set, each compile runs with its output folder as the
+  cwd and this file as `--config`, so the emitter and per-package options come
+  from the committed config (no synthesized `--emit`/`--option`). Requires
+  `outputDir` to locate each folder.
+- **`hooks.postCompile`** — runs after each successful compile (the `compile` and
+  `all` phases). Keep it a short, reusable script that reads `SPECTOR_OUTPUT_DIR`.
+- **`hooks.postCompileDeclarations`** — runs per spec in the `declarations` phase
+  (and at the end of `all`) **without recompiling**, for a follow-up build step
+  over already-generated sources.
+- **`--phase <all|compile|declarations>`** — pick which phase(s) to run. This lets
+  a heavy declarations pass overlap with, say, an integration-test run:
+
+  ```jsonc
+  "generate-and-run": "npm run gen:client && concurrently \"npm run test\" \"npm run gen:declarations\"",
+  "gen:client": "spector-runner --config spector.config.yaml --phase compile",
+  "gen:declarations": "spector-runner --config spector.config.yaml --phase declarations",
+  ```
+
+## Config files & hooks
+
+Emitters that need more than one spec root, local specs, or pre/post steps can
+declare the **whole run** in a `spector-runner.config.{js,ts}` module instead of
+a bespoke driver script. The `spector-runner` CLI loads it with `--config-file`
+and runs it end-to-end: `preRun` → all scenarios compiled in parallel →
+`postRun`, with `postScenario` after each compile.
+
+```bash
+spector-runner --config-file .scripts/spector-runner.config.js [--filter <regex>] [--jobs <n>] [--verbose]
+```
+
+```js
+import { defineConfig, buildScenarios } from "@azure-tools/spector-runner";
+
+export default defineConfig({
+  cwd: emitterPackageRoot,
+  jobs: 8,
+  tspconfig: ".scripts/tspconfig.yaml", // stub --config for every compile
+
+  // Either an array or a builder invoked with the run context. Compose several
+  // spec roots by calling buildScenarios more than once and concatenating.
+  scenarios: (ctx) => [
+    ...buildScenarios({ config: "spector.config.http.yaml", specsRoot: httpSpecs, emit: ["."], outputDir }),
+    ...buildScenarios({ config: "spector.config.azure.yaml", specsRoot: azureSpecs, emit: ["."], outputDir }),
+    ...localSpecs, // hand-built CompileScenario[]
+  ],
+
+  // Lifecycle hooks (all optional, all awaited):
+  preRun: (ctx) => syncExternalSpecs(),                 // one-time setup: sync/clone/clean
+  postScenario: (result, ctx) => patchOrCleanup(result), // per-scenario post-processing
+  postRun: (summary, ctx) => rollUpDeclarations(),      // follow-up step (not a tsp compile)
+});
+```
+
+Hook and builder context (`RunHookContext`): `{ cwd, jobs, verbose, filter }` —
+already reflecting CLI overrides, so a builder can skip work or a `preRun` can
+adapt when the run is filtered. The CLI `--filter <regex>` filters the built
+scenarios by `name` centrally.
+
+`defineConfig` is an identity helper for type-checking/completion. `runConfig`
+runs a config programmatically (`runConfig(config, { jobs, cwd, verbose, filter })`),
+and `loadConfigFile(path)` imports a config module — useful when a `.ts` config
+must run through `tsx` (e.g. the Python regenerator, whose config imports its
+upstream-synced option tables).
@@ -0,0 +1,13 @@
+#!/usr/bin/env node
+// Copyright (c) Microsoft Corporation. All rights reserved.
+// Licensed under the MIT License. See License.txt in the project root for license information.
+/* eslint-disable no-console */
+import { main } from "../dist/src/cli.js";
+
+main().then(
+  (code) => process.exit(code),
+  (error) => {
+    console.error(error instanceof Error ? error.message : error);
+    process.exit(1);
+  },
+);
@@ -18,13 +18,17 @@
   },
   "main": "dist/src/index.js",
   "types": "dist/src/index.d.ts",
+  "bin": {
+    "spector-runner": "./cmd/spector-runner.js"
+  },
   "exports": {
     ".": {
       "import": "./dist/src/index.js"
     },
     "./schema": "./schema/dist/SpectorConfig.json"
   },
   "files": [
+    "cmd/**",
     "dist/**",
     "!dist/test/**",
     "schema/dist/SpectorConfig.json"
@@ -41,8 +45,12 @@
     "test:ci": "vitest run --coverage --reporter=junit --reporter=default"
   },
   "dependencies": {
+    "picocolors": "catalog:",
     "yaml": "catalog:"
   },
+  "peerDependencies": {
+    "@typespec/compiler": "workspace:^"
+  },
   "devDependencies": {
     "@typespec/compiler": "workspace:^",
     "@typespec/json-schema": "workspace:^",