From 174b2e5dcda227ef1745cef8f9b3d47e9f8ea491 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 12 Jun 2026 01:54:48 +0000 Subject: [PATCH 01/36] process: Refine forkable-docs plan with Phase 0 docs and golden maps Add a docs-and-tests-first Phase 0 to the forkable-docs spec, per review: - Documentation Contract Changes: per-doc before/after map for every tbd doc (tbd-docs.md, tbd-design.md, README, development.md, docs-overview.md, skill-baseline, welcome-user) plus the new reference docs, playbook shortcut, and generated layout/index files. - Golden-Test Maps: a console-output style contract (icons, color roles, bracket state markers, stderr provenance, JSON/docmap shape) plus expected output for every new and changed command against one canonical fixture. - Resolve the tbd docs surface gap: explicitly re-home all four existing behaviors (bare manual, --section, --list, --all) and catalog the existing goldens that break (notably cli-help-all.tryscript.md), each as a bead blocked on its behavior's phase. - Add resolved decisions 16-17 and open question 3 (per-kind JSON shape); gate the new tbd status Docs line on forks existing so zero-fork output stays byte-identical. https://claude.ai/code/session_01X8S12JzmmxEfLpYzgH8Y7E --- .../plan-2026-06-11-eject-forkable-docs.md | 559 +++++++++++++++++- 1 file changed, 555 insertions(+), 4 deletions(-) diff --git a/docs/project/specs/active/plan-2026-06-11-eject-forkable-docs.md b/docs/project/specs/active/plan-2026-06-11-eject-forkable-docs.md index 7b9a0e12..ffe2ca38 100644 --- a/docs/project/specs/active/plan-2026-06-11-eject-forkable-docs.md +++ b/docs/project/specs/active/plan-2026-06-11-eject-forkable-docs.md @@ -193,6 +193,29 @@ Notes: The sibling viewers `tbd readme` and `tbd design` are unchanged for now (candidates for the same treatment later). +#### Disposition of today’s `tbd docs` surface + +`tbd docs` already carries **four** behaviors today, and the kernel must re-home all of +them explicitly — not just the bare viewer — so no current capability is silently +dropped and the new `tbd docs list` verb does not collide with today’s +`tbd docs --list`: + +| Today (f04) | Current behavior | Under f05 | +| --- | --- | --- | +| `tbd docs` (bare) | renders the full `tbd-docs.md` manual | **status overview** of managed docs (the scope’s landing page) | +| `tbd docs ` / `--section ` | jumps to a manual section | `tbd docs show tbd-docs --section ` (the manual is now a `reference` doc) | +| `tbd docs --list` | lists the manual’s **sections** | retired as a top-level flag; `tbd docs list` now lists **docs across kinds**. Section navigation moves to `tbd docs show tbd-docs` | +| `tbd docs --all` | the “tbd Documentation Resources” orientation card | folded into the bare `tbd docs` overview, whose “menu” block (browse / fork / learn-more pointers) carries the same orientation value | + +The one real hazard is the `--list` meaning flip (sections → docs). +The f05 gate makes older CLIs refuse to run against an f05 repo (see “Format bump: +f05”), so there is no window where a user gets the old meaning against a new layout. +The flip is recorded in Backward Compatibility within the `tbd docs` reorganization (CLI +change 1), and **every doc and golden test that exercised the old `tbd docs` / `--list` +/ `--all` / `--section` surface is rewritten in the same release** — the affected tests +are catalogued exactly in [Golden-Test Maps](#golden-test-maps), the largest single one +being `cli-help-all.tryscript.md` (seven `tbd docs` assertions tied to the old surface). + ### Three kinds of sync, kept deliberately separate tbd now has three update surfaces, and they stay distinct. @@ -741,9 +764,13 @@ The skill routing table gets matching rows, e.g.: the `DocCache`/`DocSync` extensions may refactor freely). - **Library APIs**: N/A (nothing exported). - **CLI surface**: three deliberate 0.x changes. - (1) Bare `tbd docs` is repurposed from manual viewer to status overview; the manual - stays reachable as `tbd docs show tbd-docs` / `tbd docs manual` (DO NOT MAINTAIN the - old bare behavior — confirmed; update all routing docs in the same release). + (1) The whole `tbd docs` surface is reorganized (see “Disposition of today’s + `tbd docs` surface”): bare `tbd docs` becomes the status overview; `tbd docs ` + / `--section` and the section-listing `--list` move onto `tbd docs show tbd-docs`; + `--all` folds into the overview; `tbd docs list` now lists docs across kinds. + DO NOT MAINTAIN any of the old `tbd docs` behaviors — confirmed; the manual stays + reachable as `tbd docs show tbd-docs` / `tbd docs manual`, and all routing docs and + goldens are updated in the same release. (2) `tbd sync --docs` becomes a deprecated alias of `tbd docs sync` (KEEP DEPRECATED until the next format cut). (3) The `tbd setup --interactive` flag is removed (DO NOT MAINTAIN — it never had @@ -803,51 +830,64 @@ Settled during design review (2026-06-11): 1. **One noun-scoped command group, `tbd docs `**, replaces the earlier top-level `tbd eject`/`tbd uneject` + verb-flags design (see Alternatives #7). Scope split: `tbd sync` = project data; `tbd docs` = the doc layer. + 2. **Verb pair is `fork`/`unfork`** (not shadcn-style `add`, and not the original `eject`/`uneject` — see decision 14 for the rename rationale). + 3. **`tbd docs sync` absorbs `tbd sync --docs`** (old flag kept as a deprecated alias). + 4. **tbd self-docs use a reserved `tbd-` name prefix** as regular docs in the system (kind `reference`), rather than a `tbd docs self` subcommand or dedicated viewers; bare `tbd docs` becomes the status overview and the manual is `tbd docs show tbd-docs` (alias `tbd docs manual`). No backward compatibility for the old self-doc viewer behavior. + 5. **Update semantics**: clean three-way merges apply by default (tracked files make git the undo); non-clean updates require an explicit strategy — `--merge` (combine with conflict markers) or `--rebase` (keep local content, advance the fork point) — and tracked files are mutated only by explicit `tbd docs` verbs, never by setup or background sync. The earlier standalone `rebase` subcommand is folded into `update --rebase`. + 6. **Default fork dir is `docs/tbd/`**, surfaced as an editable customization during setup and persisted to `docs_cache.fork_dir` when changed. + 7. **Generated README index ships in v1**: explains what the docs are, lists them, and points to `npx get-tbd@latest docs` for further info. + 8. **All fork state lives under one committed directory, `.tbd/doc-forks/`** — `forks.yml` (manifest) plus `base/` (snapshots) — revised from the earlier separate `.tbd/ejected.yml` + `.tbd/eject-base/` pair so the layout is self-describing and `.tbd/docs/` remains purely the cache. + 9. **Format bump to f05** with a metadata-only f04→f05 migration, following the f04 precedent: gitignore template refresh, format-history and layout-doc updates; older CLIs prompt to upgrade on encountering f05. + 10. **docref everywhere, as a hard rule.** Every document reference in tbd — config values, the fork manifest, CLI arguments, JSON output, our own docs — is a docref string, with no exceptions. Parser ported from the #117 branch; the spec ships as a bundled `reference` doc. + 11. **docmap is redefined as a minimal inventory format** (docmap/0.1: per-doc identity, location, and metadata — no bundles, lockfiles, or sync semantics). The reference doc is authored fresh for this spec and does not depend on the speculative #117 design, which is cited only as exploratory background. `tbd docs list/status --json` emit it; hand-authored docmaps are valid; source machinery is deferred as future “operations over docmaps.” + 12. **Three sync surfaces, no universal sync.** `tbd sync` = issues; `tbd setup` = installation/integrations (may invoke a docs-cache sync and report pending doc updates); `tbd docs sync`/`update` = the doc layer. A combined “sync everything” command was rejected: doc updates can involve merges and tracked-file mutation, unlike the others. The taxonomy table is documented in `tbd-docs.md`. + 13. **No interactive setup.** The unused `--interactive` flag is removed; setup is agent-first and non-interactive, with self-documenting summary output (including a read-only `--relevant` detection preview) and conversational onboarding via the skill and `welcome-user`. + 14. **Terminology: fork/unfork, upstream, built-in.** The original `eject`/`uneject` vocabulary (create-react-app heritage: a one-way escape from a managed bundle) fit when every doc came from inside tbd; with docref sources, docs are multi-origin and @@ -859,6 +899,7 @@ Settled during design review (2026-06-11): “Eject” stays as a routing synonym in the skill table. (For reference, shadcn names the model — “open code,” “copy and own” — but its verb is just `add`, with no update story.) + 15. **The `.tbd/` layout contract is explicit and documented in-place.** A generated `.tbd/README.md` (written by setup/migration) documents what each entry is — see “The `.tbd/` layout contract” — and the same contract lands in `tbd-docs.md` and @@ -866,6 +907,26 @@ Settled during design review (2026-06-11): config dirs, this layout reorganization and the CLI-semantics changes are safe to ship together with no old/new coexistence hazards. +16. **Phase 0 writes the doc and golden contracts first.** Before any feature code, the + refinement pins (a) the per-doc change map + ([Documentation Contract Changes](#documentation-contract-changes)) and (b) the + exact console output of every new/changed command + ([Golden-Test Maps](#golden-test-maps)), against one canonical fixture and a shared + console-output style contract. + Writing the expected output first is the design tool that keeps the command surface, + the docmap `--json` shape, and cross-command style consistent — and it turns the + `tbd docs` surface migration into a reviewed, test-backed change rather than an + afterthought. + +17. **All four of today’s `tbd docs` behaviors are re-homed explicitly** (decision 4 + named only the bare viewer): bare → status overview; ``/`--section` and the + section-listing `--list` → `tbd docs show tbd-docs`; `--all` → folded into the + overview; `tbd docs list`/`status` take the verbs. + This is part of CLI change 1 in Backward Compatibility, safe because the f05 gate + blocks old/new coexistence. + The `--list` meaning flip (sections → docs) is the one behavior change to call out + in release notes. + ## Open Questions 1. **Should `--relevant` ever become the fresh-setup default?** Recommended: no for now @@ -873,12 +934,73 @@ Settled during design review (2026-06-11): feedback. 2. **Pack definitions in code vs doc frontmatter tags**: code const now (recommended); migrate to frontmatter `tags:` if packs grow or third-party doc sources arrive. +3. **Should the per-kind list JSON migrate to docmap too?** + `tbd guidelines/shortcut/template --list --json` emit a flat array today. + Recommended: no for now — keep the array (backward-compat; it predates docmap) and + let `tbd docs list`/`status --json` be the docmap contract. + Revisit if consumers want one shape everywhere. ## Implementation Plan Phases are ordered so each lands independently shippable. Each numbered item is intended to be one bead. +**Phase 0 comes first by design.** Before any code, we pin the two contracts the rest of +the work implements against: (1) exactly how each tbd doc changes +([Documentation Contract Changes](#documentation-contract-changes)), and (2) the exact +console output every new and changed command must produce +([Golden-Test Maps](#golden-test-maps)). Writing the expected output first is itself a +design tool — it forces the command surface, the JSON/docmap shape, and the +cross-command style to be consistent before they are built, and it turns each later +phase into “make this golden block real.” +Phase 0 is the refinement this spec asks for; Phases 1–5 then fill it in. + +### Phase 0: Documentation contracts and golden-test maps + +No production code. +This phase produces the two reviewable contracts and updates the docs +whose wording does not depend on the code (it does not stamp the format id — that lands +with the f05 code in Phase 1 — but it fixes the *text* those code changes must match). +Each item is one bead (labeled `0.1`–`0.5` so it sits before the global `1`–`20` +numbering the later phases use): + +- **0.1 — Author the contracts** (this spec): the + [Documentation Contract Changes](#documentation-contract-changes) and + [Golden-Test Maps](#golden-test-maps) sections — the per-doc before/after table, the + console output style contract, and the per-command expected-output blocks. + This is the design artifact the rest of Phase 0 and all later phases consume; it is + the deliverable of the current refinement. +- **0.2 — Doc edits independent of the new code** (the contract’s “Phase 0” rows): + rewrite `tbd-docs.md` (the `tbd docs` group, the three-sync taxonomy table, the + `.tbd/` layout contract, the managed-docs / fork / update / states sections, the + `docs_cache` config reference); update `tbd-design.md` (layout + CLI-group + format + narrative) and this repo’s `development.md` / `docs-overview.md` path and doc-command + sections. These land now because their wording is fixed by the contract, not by the + implementation. +- **0.3 — New bundled reference docs** `references/docref-format.md` (adopted from the + #117 branch, marked adopted v0.1) and `references/docmap-format.md` (authored fresh + and minimal per the Design section). + Authoring them now lets the docmap/0.1 schema in the golden `--json` maps be reviewed + against its own spec before any code emits it. +- **0.4 — New playbook shortcut** `shortcuts/standard/suggest-upstream-improvements.md` + (follows `new-guideline.md` conventions; references `tbd docs status --json` / + `tbd docs diff`). Pure documentation, so it lands in Phase 0; the skill/README routing + rows that point at it land with the agent surface in Phase 5 (kept together to avoid a + half-wired routing table). +- **0.5 — Lock the golden-test maps against reality.** For every *existing* command the + maps reference unchanged (`tbd guidelines --list`, `tbd status` with zero forks, the + `--json` doc shape), capture the current built-CLI output and paste it verbatim into + the maps, so the “consistency baseline” is real output, not a guess. + Catalog every existing golden/tryscript test the feature will break (see + [Existing golden tests that change](#existing-golden-tests-that-change)) and file a + bead per rewrite, each blocked on the phase that ships the corresponding behavior — so + no golden is left silently failing. + +Phase 0 ships as a docs-only PR. Its only test impact is that the new reference docs and +the new shortcut must resolve — which `doc-references.test.ts` already enforces once its +extractor learns the `reference` kind (the extractor change itself is tracked with the +rest of the test wiring in Phase 5). + ### Phase 1: Format bump and fork kernel 1. **f05 format bump**: `CURRENT_FORMAT = 'f05'`, f04→f05 step in `migrateToLatest()` @@ -968,6 +1090,422 @@ Each numbered item is intended to be one bead. ("Forkable guidelines: fork them into your repo"), `tbd prime` mention if warranted. 20. **CHANGELOG + release notes** per `release-notes-guidelines`. +## Documentation Contract Changes + +This is the per-doc map the user asked for: exactly what changes in each tbd doc, and +when. “Phase 0” rows are wording fixed by this design and land in the docs-only Phase 0 +PR; “with code” rows are wording coupled to an implementation phase (e.g. a stamped +format id) and land with that phase, but still follow the contract written here. +Blocks already specified in the Design section (the three-sync taxonomy, the `.tbd/` +layout contract, the doc-states table, the update decision table) are **referenced, not +duplicated** — the contract is that those exact blocks appear in the named docs. + +| Doc | Lands | Contract change | +| --- | --- | --- | +| `packages/tbd/docs/tbd-docs.md` (the CLI manual; also the `tbd-docs` reference doc) | Phase 0 | Replace the “Documentation Commands” section with the `tbd docs` group (per “The `tbd docs` command group”); add a “Managing forked docs” section (fork/unfork/update/diff/status + the doc-states and update-decision tables); add the three-sync taxonomy table and the `.tbd/` layout contract verbatim; extend “Configuration Reference” with `docs_cache.fork_dir` and `docs_cache.local_dirs` and the note that `files`/`source`/`local_dirs` values are docrefs; cross-link `docref-format` and `docmap-format`. | +| `packages/tbd/docs/tbd-design.md` | Phase 0 (narrative) + Phase 1 (format id) | §2 File Layer + path conventions: document `.tbd/doc-forks/` (manifest + base snapshots), the external fork dir, and the resolution precedence (fork dir → `local_dirs` → cache, first-match-wins). §4 CLI Layer: add the `tbd docs` group and the three-sync taxonomy. Format narrative: add f05 alongside f04. docref/docmap named as the addressing conventions. | +| `packages/tbd/src/lib/tbd-format.ts` (`FORMAT_HISTORY`) | Phase 1 (with code) | Add the `f05` entry: `introduced` (next minor), description “Adds forkable-docs layout”, `changes` = [`docs_cache.fork_dir`, `docs_cache.local_dirs`, `.tbd/doc-forks/forks.yml` + `base/`, generated `.tbd/README.md`], `migration` = “metadata-only: stamp f05, refresh `.tbd/.gitignore`, write `.tbd/README.md` layout contract”. `CURRENT_FORMAT = 'f05'`. This file is the authoritative format history; its wording is the contract Phase 0 references but does not edit. | +| `docs/development.md` (this repo) | Phase 0 | “Path Conventions” block: add `.tbd/doc-forks/` (committed) and note the fork dir lives **outside** `.tbd/` (default `docs/tbd/`). Add a “Testing forkable docs” pointer to the new e2e/tryscript files. | +| `docs/docs-overview.md` (this repo) | Phase 0 | “tbd CLI Documentation Commands” + “Adding external docs by URL”: replace with the `tbd docs` group; `tbd docs add `; add a line on forking docs into a visible `docs/tbd/`. | +| `README.md` | Phase 0 | “Shortcuts, Guidelines, and Templates”: add a “Forkable: see them in your repo” paragraph with a `tbd docs fork --relevant` example. “Documentation” block: `tbd docs` is now an overview; the manual is `tbd docs show tbd-docs`. Per-kind `--add` lines annotated as aliases for `tbd docs add`. | +| `packages/tbd/docs/shortcuts/system/skill-baseline.md` (injected agent skill) | Phase 5 | Add the fork/update/upstream rows to “User Request → Agent Action” (the rows in “Upstream-contribution playbook”); add `tbd docs list` / `tbd docs fork` to the “Documentation” command table; one-line “Forkable docs” note. Kept within the skill’s size budget; lands with the rest of the agent surface so routing is never half-wired. | +| `packages/tbd/docs/install/claude-header.md` | none | **No change.** Its `allowed-tools: Bash(tbd:*)` already covers `tbd docs`. Stated here so the audit is explicit. | +| `packages/tbd/docs/shortcuts/standard/welcome-user.md` | Phase 5 | Add the onboarding offer after the status summary ("Want tbd’s guidelines visible in your repo? Which languages?") and a “make guidelines visible” row routing to `tbd docs fork --relevant`. | +| `packages/tbd/docs/shortcuts/standard/suggest-upstream-improvements.md` (**new**) | Phase 0 | The upstream-contribution playbook (per “Upstream-contribution playbook”). Pure docs. | +| `packages/tbd/docs/references/docref-format.md` (**new**) | Phase 0 | Adopted from the #117 branch, marked adopted v0.1. First doc of the new `reference` kind. | +| `packages/tbd/docs/references/docmap-format.md` (**new**) | Phase 0 | Authored fresh, minimal (docmap/0.1 inventory only) per the Design section; #117 cited as background. | +| Generated `.tbd/README.md` (**new**, by setup/migration) | Phase 1 (with code) | The `.tbd/` layout contract block, written in place; kept current like the gitignore. | +| Generated `/README.md` (**new**, by fork/unfork/update) | Phase 1 (with code) | The fork-dir index: what these docs are, one line per doc with its description, “managed by `tbd docs fork`”, and the `npx get-tbd@latest docs` pointer. | +| `packages/tbd/CHANGELOG.md` + release notes | Phase 5 | f05 entry per `release-notes-guidelines`. | +| `tbd --help` (the `docs` command `.description()` + subcommand help, in `src/cli/commands/docs.ts`) | Phase 1 (with code) | Description changes from “Display CLI documentation (use tbd sync --docs …)” to a managed-docs summary; subcommand help strings per the [Golden-Test Maps](#golden-test-maps). User-facing text, so it is part of the contract. | + +Two consistency points the contract pins down: + +- **The per-kind list JSON is unchanged.** `tbd guidelines --list --json` keeps emitting + a flat array (today’s shape). + Only `tbd docs list --json` / `tbd docs status --json` emit the new docmap object, + with tbd’s state fields as extension fields. + We deliberately do *not* migrate the per-kind arrays to docmap (backward-compat, and + they predate the format) — see Open Question 3. +- **`tbd-docs.md` is both a rendered manual and a forkable `reference` doc.** Its + bundled source stays at the docs root (`packages/tbd/docs/tbd-docs.md`); the + cache/kind-dir is `references/` and the lookup name is `tbd-docs`. The doc-sync map + records that root-to-`references/` mapping (a small `doc-sync.ts` detail, Phase 5 item + 18). + +## Golden-Test Maps + +These are the expected console outputs the new and changed commands must produce. +They are written here (not as live test files) because they run against the built CLI +and would fail until the feature ships; each block is **lift-ready** — the +implementation phase pastes it into the named harness. +Two harnesses are in use, matching today’s repo: **tryscript** (`*.tryscript.md`, +`NO_COLOR=1`, `[..]` matches intra-line, `...` matches whole lines, custom `[PATTERN]`s +for unstable fields) and **vitest inline snapshots** (`golden-output.test.ts`, +`FORCE_COLOR=0`). All maps below are shown as captured with color disabled (the state +the golden files store), so bold/dim render as plain text. + +Unstable fields use placeholders that become tryscript patterns: `[SIZE]` = +`\([0-9.]+ .B, ~[0-9.]+k? tok\)`, `[PATH]`, `[HASH]`, `[VERSION]`. Per +`golden-testing-guidelines`, everything else (names, kinds, states, counts, ordering) is +shown literally — no patterns on values we control. + +### Console output style contract + +Every `tbd docs` command obeys the conventions already used across tbd (verified against +`output.ts`, `sections.ts`, and `doc-command-handler.ts`). This is the consistency +anchor the user asked for; the maps below are just this contract applied: + +- **Section headers** are `formatHeading()` — UPPERCASE, bold (`INTEGRATIONS`, + `HEALTH CHECKS`); bodies indent two spaces. + The `tbd docs` overview reuses this. +- **Icons** come only from `ICONS` — `✓` success/closed, `✗` error, `⚠` warn, `•` + notice, `○` open, `◐` in_progress, `●` blocked. + No new glyphs are invented for doc states. +- **Color roles** (from `createColors`): `id`=cyan for doc names, `dim` for + metadata/sizes/paths, `bold` for names and headers, `success`/`warn`/`error` for the + matching icons. Forked doc names render with the `id` role, like issue IDs. +- **Doc-state markers are dim bracket tags** appended to the list entry, exactly like + the existing `[shadowed]` tag: `[forked]`, `[forked, customized]`, `[local]`. State + *icons* are never used for doc states — brackets are the established convention. + (Staleness and `conflicted` are lifecycle facts shown by `tbd docs status` / the + summary line, not list markers — list markers describe ownership.) +- **Provenance on serve** is one dim line to **stderr** (so piped stdout stays clean), + reusing the existing stderr-note channel: `(serving forked copy: [PATH])`. +- **Outcomes**: success → `✓ ` green to stdout; refusal/error → `✗ ` red to + stderr + non-zero exit; preview → `[DRY-RUN] ` yellow (the `output.dryRun` form). +- **`--json`** goes through `output.data(...)`: docmap object for `list`/`status`, + per-doc object for `show`; no ANSI; consumers ignore unknown fields. +- **Footer** uses `renderFooter`: `Use 'cmd' for X, 'cmd' for Y.` +- **Width** wraps at 88 columns (`getTerminalWidth`). +- **Command/scope parallels** kept on purpose: `tbd docs` (overview/summary) is to + `tbd docs status` (the per-doc table) as `tbd status` is to `tbd stats`; the bare + overview’s “menu” block and the `tbd setup --auto` Docs summary share identical + wording. + +A single canonical fixture is used across every map below (a repo just upgraded, so +upstream has moved): + +| name | kind | customized | stale | merge on update | +| --- | --- | --- | --- | --- | +| `python-rules` | guideline | yes | yes | clean | +| `acme-style` | guideline | yes | yes | conflicts | +| `review-code` | shortcut | no | yes | n/a (refresh) | +| `tbd-docs` | reference | no | no | untouched | + +### `tbd docs` (bare overview) + +Mirrors `tbd status`: a summary plus pointers, never the full table. +Zero-fork case is the default and stays the orientation card the old `tbd docs --all` +provided: + +```text +$ tbd docs # no docs forked yet +tbd docs — managed documentation + + 37 available in cache (.tbd/docs/, gitignored); none forked into the repo. + + Browse: tbd docs list + Make visible: tbd docs fork --relevant (detected: typescript, python → 13 docs into docs/tbd/) + tbd docs fork --all (everything) + Customize one: tbd docs fork + Learn more: tbd docs show tbd-docs +? 0 +``` + +With forks present: + +```text +$ tbd docs +tbd docs — managed documentation + + 37 available (33 upstream, 4 forked into docs/tbd/) + 4 forked: 2 customized, 3 with upstream updates — run 'tbd docs update' + + Inspect: tbd docs status + Browse: tbd docs list + Update: tbd docs update + Learn more: tbd docs show tbd-docs +? 0 +``` + +### `tbd docs list` + +Grouped by kind (bold header), each entry in the established two-line form +(`name [SIZE]` then 3-space-indented `Title: Description`), with dim ownership markers +appended: + +```text +$ tbd docs list +guideline + acme-style [SIZE] [forked, customized] + ACME House Style: Internal style overrides for ACME repos + python-rules [SIZE] [forked, customized] + Python Coding Rules: Type hints, docstrings, exception handling, resource management + typescript-rules [SIZE] + TypeScript Rules: Strictness, module boundaries, and error handling for TypeScript + [.. remaining guidelines ..] +shortcut + review-code [SIZE] [forked] + Review Code: Comprehensive code review across uncommitted, branch, or PR scopes + [.. remaining shortcuts ..] +reference + docmap-format [SIZE] + Docmap Format: A minimal inventory format for a collection of documents + docref-format [SIZE] + Docref Grammar: A single-string, URI-like address for any document + tbd-docs [SIZE] [forked] + tbd CLI Documentation: Command reference for the tbd CLI +? 0 +``` + +`tbd docs list --kind=guideline` filters to one group (no kind header needed). +JSON is the docmap object (see “docmap” in Design); the array form is the per-kind +commands’ contract, not this one: + +```text +$ tbd docs list --json +{ + "docmap": "docmap/0.1", + "name": "tbd-docs", + "documents": [ + { + "name": "python-rules", + "type": "guideline", + "path": "docs/tbd/guidelines/python-rules.md", + "source": "internal:guidelines/python-rules.md", + "title": "Python Coding Rules", + "description": "Type hints, docstrings, exception handling, resource management", + "word_count": [..], + "state": "customized", + "stale": true + } + [.. one entry per doc; upstream docs have state "upstream" and no fork fields ..] + ] +} +? 0 +``` + +### `tbd docs show` / `tbd docs manual` + +Kind-agnostic read; `reference` docs carry no agent header (unlike guidelines). +The manual moves here from the old bare `tbd docs`: + +```text +$ tbd docs show tbd-docs | head -3 +# tbd CLI Documentation + +Git-native issue tracking for AI agents and humans. +? 0 + +$ tbd docs manual | head -1 # alias for: tbd docs show tbd-docs +# tbd CLI Documentation +? 0 + +$ tbd docs show python-rules # serves the forked copy; provenance to stderr +[.. forked file content on stdout ..] +# stderr: (serving forked copy: docs/tbd/guidelines/python-rules.md) +? 0 +``` + +### `tbd docs fork` + +```text +$ tbd docs fork python-rules +✓ Forked python-rules → docs/tbd/guidelines/python-rules.md + Recorded base in .tbd/doc-forks/ (source: internal:guidelines/python-rules.md) + Regenerated docs/tbd/README.md + +Edit it in place — tbd now serves your copy everywhere it served the upstream one. +? 0 + +$ tbd docs fork --relevant --dry-run +[DRY-RUN] Would fork 13 docs into docs/tbd/ (packs: core, typescript, python) + guideline general-eng-agent-principles + guideline general-coding-rules + [.. 9 more ..] + guideline python-rules +No files written. Re-run without --dry-run to apply. +? 0 + +$ tbd docs fork python-rules # target exists and is not an unmodified fork +✗ docs/tbd/guidelines/python-rules.md already exists and is not an unmodified fork. + Refusing to overwrite it. Options: + tbd docs diff python-rules # see how it differs + tbd docs fork python-rules --force # overwrite with upstream +? 1 +``` + +### `tbd docs unfork` + +```text +$ tbd docs unfork python-rules # customized → refuse +✗ python-rules has local customizations (differs from its base). + Refusing to discard them. Options: + tbd docs diff python-rules # review your changes + tbd docs unfork python-rules --force # discard and fall back to upstream +? 1 + +$ tbd docs unfork review-code # unmodified → succeeds +✓ Unforked review-code — served from upstream again. + Removed the forked file, its base snapshot, and its manifest entry. + Regenerated docs/tbd/README.md +? 0 +``` + +### `tbd docs status` + +The per-doc table (dim header row, `output.table` convention). +The closing summary line matches the bare overview and the `tbd status` Docs line +exactly: + +```text +$ tbd docs status +NAME KIND STATE SOURCE +acme-style guideline customized, stale github:acme/eng-docs@main//guidelines/style.md +python-rules guideline customized, stale internal:guidelines/python-rules.md +review-code shortcut stale internal:shortcuts/standard/review-code.md +tbd-docs reference forked internal:tbd-docs.md + +4 forked: 2 customized, 3 with upstream updates — run 'tbd docs update' +? 0 +``` + +### `tbd docs diff` + +Git-style, no network (`git diff --no-index` against the relevant copy): + +```text +$ tbd docs diff python-rules # your file vs current upstream (the net fork) +--- upstream:guidelines/python-rules.md ++++ docs/tbd/guidelines/python-rules.md +@@ +[.. unified diff hunks ..] +? 0 + +$ tbd docs diff python-rules --base # your file vs its base (what you changed) +$ tbd docs diff python-rules --upstream # base vs current upstream (incoming changes) +``` + +### `tbd docs update` + +Default run on the canonical fixture: refresh the unmodified-stale doc, apply the clean +merge, and *list* the conflict for a decision (never touch it by default): + +```text +$ tbd docs update +Updated 2 forked docs: + ✓ review-code refreshed to upstream (was unmodified) + ✓ python-rules merged upstream cleanly (review with: git diff) + +1 doc needs a decision: + ⚠ acme-style your changes conflict with upstream + re-run with one of: + tbd docs update acme-style --merge # combine, then resolve conflict markers + tbd docs update acme-style --rebase # keep your version, advance the fork point +? 0 + +$ tbd docs update acme-style --merge +✓ acme-style wrote merged content with conflict markers; base advanced. + Resolve the <<<<<<< / ======= / >>>>>>> markers, then the doc returns to 'customized'. +? 0 + +$ tbd docs update acme-style --rebase +✓ acme-style kept your version; fork point advanced to current upstream. +? 0 + +$ tbd docs update --dry-run +[DRY-RUN] 2 docs would update (1 refresh, 1 clean merge); 1 would conflict (acme-style). +No files written. +? 0 +``` + +### `tbd docs add` + +Aligned with today’s `--add` output, restated for docrefs and the new pointers: + +```text +$ tbd docs add github:acme/eng-docs@main//guidelines/style.md --kind=guideline --name=acme-style +Adding guideline: acme-style + Source: github:acme/eng-docs@main//guidelines/style.md +✓ Added to .tbd/docs/guidelines/acme-style.md + Config updated (docs_cache.files): github:acme/eng-docs@main//guidelines/style.md + +Run 'tbd docs list' to verify, or 'tbd docs fork acme-style' to make it visible. +? 0 +``` + +### `tbd status` (Docs line) and `tbd setup --auto` (Docs summary) + +`tbd status` gains a Docs line **only when forks exist** — so with zero forks the output +is byte-identical to today’s `cli-orientation-golden.tryscript.md` (honoring the +“default behavior unchanged when nothing is forked” guarantee). +When forks exist it appears after the Worktree line, before the footer: + +```text +$ tbd status # excerpt, forks present +[.. INTEGRATIONS, Worktree as today ..] + +Docs: 4 forked (2 customized, 3 with upstream updates — run 'tbd docs update') + +Use 'tbd stats' for issue statistics, 'tbd doctor' for health checks. +? 0 +``` + +`tbd setup --auto` prints the Docs summary (zero-fork = the menu; with forks = a +pending-update report); setup never writes the fork dir: + +```text +# zero forks +Docs: 37 available in cache (.tbd/docs/, gitignored); none forked into the repo. + Browse: tbd docs list + Make visible: tbd docs fork --relevant (detected: typescript, python → 13 docs into docs/tbd/) + tbd docs fork --all (everything) + Customize one: tbd docs fork + +# after an upgrade, forks present +Docs: 4 forked into docs/tbd/. 3 have upstream updates — run 'tbd docs update'. +``` + +### `tbd doctor` (new HEALTH CHECKS) + +Appended to the existing `HEALTH CHECKS` list, following doctor’s `✓`/`⚠` + `Run:` +convention (icon at column 0, no indent): + +```text +$ tbd doctor # excerpt +[.. existing health checks ..] +✓ Forked docs - 4 forked, base snapshots intact +⚠ Forked docs - 1 unresolved merge conflict (acme-style) + Run: resolve the conflict markers, then re-run tbd docs update +✓ Fork dir - docs/tbd/ tracked in git (not gitignored) +✓ Reserved tbd- names - no user docs claim the prefix +? 0 +``` + +### Existing golden tests that change + +The plan’s original Testing Strategy listed only *new* tests. +These *existing* goldens break and must be rewritten in the same release; each is one +bead, blocked on the phase that ships the behavior: + +| Test | Change | Phase | +| --- | --- | --- | +| `cli-help-all.tryscript.md` (≈7 `tbd docs` blocks: `--help` `[topic]`/`--section`, `--list` sections, positional topic, `--section` content, `--list --json`, bare manual) | **Rewrite** to the new surface: `docs` subcommand help; no top-level `--section`/section-`--list`; section nav becomes `tbd docs show tbd-docs --section`. Largest single change. | 1–2 | +| `cli-doc-output.tryscript.md` ("Docs Command" block: `tbd docs --list` → “Available documentation sections:”) | **Rewrite** to `tbd docs list` (cross-kind, state markers) + `--json` docmap. | 2 | +| `golden-output.test.ts` (`tbd docs --all` inline snapshot) | **Replace** with the bare `tbd docs` overview snapshot (`--all` folded into the overview). | 2 | +| `golden-output.test.ts` ("post-setup What’s Next") | **Extend** to assert the Docs menu lines. | 4 | +| `cli-orientation-golden.tryscript.md` (`tbd status`) | **Unchanged** for zero forks (verifies the guarantee); **add** a new forked-state status golden in a fixture with a fork. | 1 | +| `setup-flows.test.ts` | **Extend** for the Docs summary (menu + pending-update report). | 4 | +| `doc-references.test.ts` | **Extend** the extractor: add `tbd docs ` and the `reference` kind; remove the `reference`/`prefix:` skips so `tbd docs show tbd-docs`, `suggest-upstream-improvements`, and the new reference docs all resolve. | 5 (extractor); 0 (the new docs it must resolve) | +| `doc-add-e2e.test.ts` | **Keep** (per-kind `--add` stays an alias) and **extend** with `tbd docs add `. | 2 | + +New golden/e2e files (named for the phases that add them): `fork-manifest` + +state-matrix units (Phase 1); a `cli-docs-fork.tryscript.md` lifecycle (fork → list +marker → serve → edit → unfork refuse → `--force`) (Phase 1); `fork-update` +decision-table units + a `cli-docs-update.tryscript.md` upgrade/merge scenario (Phase +3); `doc-packs` detection units + a `fork --relevant` fixture e2e (Phase 4). + ## Testing Strategy - **Unit (vitest)**: manifest round-trip; hash normalization (CRLF/LF); the full state @@ -992,7 +1530,20 @@ Each numbered item is intended to be one bead. `fork --relevant` in a fixture repo with `pyproject.toml`; collision/overwrite refusal; doctor findings. - **Docs-reference test**: extend `doc-references.test.ts` so every command named in the - new shortcut/docs resolves. + new shortcut/docs resolves (extractor learns `tbd docs ` and the + `reference` kind; the `reference`/`prefix:` skips are removed). +- **Golden output**: the expected console output for every new and changed command is + specified in [Golden-Test Maps](#golden-test-maps) (the consistency contract plus a + per-command map against one canonical fixture). + Each map is lifted into its harness (tryscript or vitest inline snapshot) by the phase + that ships the command. +- **Existing goldens that break** are catalogued in + [Existing golden tests that change](#existing-golden-tests-that-change) — notably + `cli-help-all.tryscript.md`, `cli-doc-output.tryscript.md`, and the `tbd docs --all` + snapshot in `golden-output.test.ts`. Each rewrite is a bead blocked on the behavior’s + phase, so CI never carries a silently-failing golden. + A red-then-green pass on these (run the rewritten golden against the old binary to + confirm it fails, then against the new one) verifies the surface actually changed. ## Relationship to PR #117 From 3caea0365f16317465ca0bf352913e8347bd3fd1 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 12 Jun 2026 07:15:03 +0000 Subject: [PATCH 02/36] process: Tighten forkable-docs plan: docmap everywhere, --keep-ours Apply review decisions: - Rename the update strategy flag --rebase -> --keep-ours (--rebase collided with git-rebase's meaning); keep the historical references in Alternatives #7 and decision 5 accurate. - Forked-doc serve note is on by default (not gated behind --verbose); the extra context helps agents track customized docs. - docmap becomes the single data model for ALL doc output, no backward-compat carve-out: per-kind --list --json switches from a flat array to docmap; one shared renderer drives both text and JSON so they cannot drift. Resolves the former OQ3. - Make output consistency structural: a single rendering layer owns list/table/overview/marker formatting; the style contract is authoritative; pre-existing status/doctor drift tracked separately. - docref + docmap ship as standalone, dependency-free, fully-tested modules designed for later extraction into their own package. Adds resolved decisions 18-23, removes OQ3, updates the golden-test catalog (per-kind --list --json change) and Phase 1/2 module items. https://claude.ai/code/session_01X8S12JzmmxEfLpYzgH8Y7E --- .../plan-2026-06-11-eject-forkable-docs.md | 175 +++++++++++++----- 1 file changed, 125 insertions(+), 50 deletions(-) diff --git a/docs/project/specs/active/plan-2026-06-11-eject-forkable-docs.md b/docs/project/specs/active/plan-2026-06-11-eject-forkable-docs.md index ffe2ca38..b6908d77 100644 --- a/docs/project/specs/active/plan-2026-06-11-eject-forkable-docs.md +++ b/docs/project/specs/active/plan-2026-06-11-eject-forkable-docs.md @@ -39,8 +39,8 @@ layout revision (format `f05`, a metadata-only migration from f04): 3. **`tbd docs update`** — after a tbd upgrade, pull upstream changes into forked docs: unmodified copies refresh in place; customized copies get a git three-way merge that applies automatically when clean; conflicting docs are listed until you choose a - resolution: `--merge` (combine, with standard conflict markers) or `--rebase` (keep - your version and advance the fork point). + resolution: `--merge` (combine, with standard conflict markers) or `--keep-ours` + (keep your version and advance the fork point). 4. **A small committed manifest plus stored merge bases** (`.tbd/doc-forks/forks.yml` + `.tbd/doc-forks/base/`) recording, for each forked doc, exactly which upstream content it forked from — making “customized?”, “stale vs upstream?”, and three-way @@ -169,7 +169,7 @@ tbd docs show # read any doc by name, kind-agnostic tbd docs sync # refresh the gitignored cache (absorbs `tbd sync --docs`) tbd docs add # register a single external doc (URL, github:, local path) tbd docs fork / unfork # fork into the repo / return to upstream -tbd docs update # reconcile forks with upstream (--merge / --rebase) +tbd docs update # reconcile forks with upstream (--merge / --keep-ours) tbd docs diff / status # inspect ``` @@ -502,8 +502,10 @@ documents: word_count: 2400 ``` -Producers may *generate* a docmap (as tbd does: `tbd docs list --json` / `status --json` -emit exactly this, with tbd’s state fields as extension fields) or *hand-author* one. +Producers may *generate* a docmap (as tbd does: **every** list/inventory command emits +exactly this — `tbd docs list` / `tbd docs status` and the per-kind +`tbd guidelines/shortcut/template --list`, in both `--json` and, via one shared +renderer, text — with tbd’s state fields as extension fields) or *hand-author* one. Consumers must ignore unknown fields. That producer-agnosticism is what makes the concept useful beyond tbd: any repo can commit a docmap to advertise its doc collection, and a future source framework would @@ -559,20 +561,20 @@ conflict count, standard markers, no repo state touched): When a doc isn’t cleanly updatable, the user chooses one of two resolution strategies (mutually exclusive flags): **`--merge`** combines both sides, writing standard conflict -markers to resolve by editing; **`--rebase`** keeps the local version untouched and just -advances the fork point to current upstream (“my fork supersedes this upstream change” / -“I already folded it in by hand”). +markers to resolve by editing; **`--keep-ours`** keeps the local version untouched and +just advances the fork point to current upstream (“my fork supersedes this upstream +change” / “I already folded it in by hand”). -| Doc state | `update` (default) | `update --merge` | `update --rebase` | +| Doc state | `update` (default) | `update --merge` | `update --keep-ours` | | --- | --- | --- | --- | | `forked` (unmodified) + stale | replace file with new upstream; advance base | same | same | | `customized` + stale, three-way merge is clean | apply merged result; advance base | same | keep file as-is; advance base only | -| `customized` + stale, merge conflicts | **skip**; warn and list the docs: “re-run with `--merge` (combine, resolve markers) or `--rebase` (keep your version)” | write conflict markers into the file; advance base; set `conflicted` | keep file as-is; advance base only | +| `customized` + stale, merge conflicts | **skip**; warn and list the docs: “re-run with `--merge` (combine, resolve markers) or `--keep-ours` (keep your version)” | write conflict markers into the file; advance base; set `conflicted` | keep file as-is; advance base only | | `customized`, not stale | no-op | no-op | no-op | | `conflicted` (unresolved markers) | skip + warn: resolve first | skip + warn | skip + warn | | `orphaned` | skip + note (upstream removed the doc; keep your copy or `unfork`) | same | same | | `missing` / `local` | skip (doctor’s problem / nothing upstream) | same | same | -| base file missing (manual deletion) | cannot merge; skip + point at `--rebase` | same | re-establish base from current upstream (repair) | +| base file missing (manual deletion) | cannot merge; skip + point at `--keep-ours` | same | re-establish base from current upstream (repair) | Design points: @@ -584,11 +586,12 @@ Design points: conflicted `--merge`), the base becomes the new upstream content. So post-resolution, the doc is simply “a customized fork of current upstream” — states stay coherent with no extra bookkeeping. -- **`--rebase` is not git-rebase content semantics** (for single files, replaying your - diff onto the new base is just the same three-way merge). - Here it means *re-base the fork point*: your content stands, upstream’s change is - acknowledged, staleness clears, and future updates diff against the new base. - It also repairs a missing base file. +- **`--keep-ours` keeps your content and advances the fork point.** For a single file + there is no diff to replay — keeping your version *is* the operation; upstream’s + change is acknowledged, staleness clears, and future updates diff against the new + base. It also repairs a missing base file. + (This was `--rebase` in an earlier draft, renamed because the operation is not + git-rebase content semantics — it does not replay your diff, it keeps it.) - **Only the explicit command mutates tracked files.** `tbd setup --auto` and the 24-hour doc auto-sync refresh the gitignored cache as today and then *report* pending updates (`2 forked docs have upstream updates — run 'tbd docs update'`), but never @@ -634,7 +637,7 @@ tbd docs diff python-rules --upstream # base vs current upstream (incoming tbd docs update # refresh unmodified + apply clean merges; list conflicts tbd docs update python-rules # limit to specific docs tbd docs update --merge # conflicts: combine, write conflict markers to resolve -tbd docs update --rebase # conflicts: keep your version, advance the fork point +tbd docs update --keep-ours # conflicts: keep your version, advance the fork point tbd docs update --dry-run # preview, including which docs would conflict # Reverse @@ -753,7 +756,7 @@ The skill routing table gets matching rows, e.g.: | “Put all of tbd’s docs in my repo” | `tbd docs fork --all` | | “Stop customizing X / go back to the default” | `tbd docs unfork X` (`--force` only after confirming) | | “Eject the guidelines …” (legacy term) | treat as fork: `tbd docs fork …` | -| “Update the guidelines to the latest” (or after `tbd setup` reports pending updates) | `tbd docs update`; if conflicts are listed, ask the user, then `--merge` (combine + resolve) or `--rebase` (keep ours) | +| “Update the guidelines to the latest” (or after `tbd setup` reports pending updates) | `tbd docs update`; if conflicts are listed, ask the user, then `--merge` (combine + resolve) or `--keep-ours` (keep ours) | | “Could we contribute these improvements back?” | `tbd shortcut suggest-upstream-improvements` | ## Backward Compatibility @@ -844,10 +847,10 @@ Settled during design review (2026-06-11): 5. **Update semantics**: clean three-way merges apply by default (tracked files make git the undo); non-clean updates require an explicit strategy — `--merge` (combine with - conflict markers) or `--rebase` (keep local content, advance the fork point) — and + conflict markers) or `--keep-ours` (keep local content, advance the fork point) — and tracked files are mutated only by explicit `tbd docs` verbs, never by setup or background sync. The earlier standalone `rebase` subcommand is folded into - `update --rebase`. + `update --keep-ours`. 6. **Default fork dir is `docs/tbd/`**, surfaced as an editable customization during setup and persisted to `docs_cache.fork_dir` when changed. @@ -891,7 +894,7 @@ Settled during design review (2026-06-11): 14. **Terminology: fork/unfork, upstream, built-in.** The original `eject`/`uneject` vocabulary (create-react-app heritage: a one-way escape from a managed bundle) fit when every doc came from inside tbd; with docref sources, docs are multi-origin and - the fork lifecycle (fork point, merge, rebase, unfork, contribute back) is the + the fork lifecycle (fork point, merge, advance base, unfork, contribute back) is the accurate model. Renames: verbs `fork`/`unfork`; state `forked`; config `docs_cache.fork_dir`; the former `bundled` state is now `upstream` (not forked; served from its upstream via the cache); “bundled”/“built-in” is reserved for @@ -927,6 +930,45 @@ Settled during design review (2026-06-11): The `--list` meaning flip (sections → docs) is the one behavior change to call out in release notes. +Settled during refinement (2026-06-12): + +18. **The forked-doc serve note is on by default.** When a read command serves a + forked/local copy it prints the one-line stderr provenance note without `--verbose` + (suppressed only by `--quiet`/`--json`). The extra context is deliberate — it helps + agents recall which docs are customized in a session. + +19. **`tbd docs --all` is removed, not aliased.** Its orientation value moves into the + bare `tbd docs` overview; no compatibility shim, since the f05 gate prevents old/new + confusion. + +20. **The update strategy flag is `--keep-ours`, not `--rebase`.** `--rebase` collided + with git-rebase’s meaning, while the operation simply keeps the local version and + advances the fork point. + It pairs with `--merge` (combine, with conflict markers). + +21. **docmap is the single data model for all doc output; no backward-compat + carve-out.** Every list/inventory command builds one docmap and renders it to text + or `--json`; the per-kind `--list --json` therefore changes from today’s flat array + to a docmap (accepted — no backward-compat requirement). + Single-doc reads emit one docmap document entry plus `content`. Text and JSON derive + from the same structure, so they cannot drift and any field/state addition is a + one-place change. This supersedes the earlier OQ3. + +22. **Strict output-style consistency is structural.** All doc-command rendering goes + through one shared rendering layer (no per-command `console.log` formatting); the + Console output style contract is authoritative, and a cross-command contract/golden + test pins it. The mandate is “everything is consistent and systematic, easy to + update.” Pre-existing cross-command drift *outside* the docs surface (e.g. the + status/doctor INTEGRATIONS divergence documented in + `cli-orientation-golden.tryscript.md`) is real but out of scope here; it is tracked + as a separate follow-up bead rather than silently folded into this spec. + +23. **docref and docmap ship as standalone, fully-tested, dependency-free modules** + (`src/docref/`, `src/docmap/`) designed for later extraction into their own package: + no tbd-internal imports, a public API surface, and their own spec-mirror test suites + (the docref parser comes over from the #117 branch already in this shape). + tbd consumes them; they do not consume tbd. + ## Open Questions 1. **Should `--relevant` ever become the fresh-setup default?** Recommended: no for now @@ -934,11 +976,10 @@ Settled during design review (2026-06-11): feedback. 2. **Pack definitions in code vs doc frontmatter tags**: code const now (recommended); migrate to frontmatter `tags:` if packs grow or third-party doc sources arrive. -3. **Should the per-kind list JSON migrate to docmap too?** - `tbd guidelines/shortcut/template --list --json` emit a flat array today. - Recommended: no for now — keep the array (backward-compat; it predates docmap) and - let `tbd docs list`/`status --json` be the docmap contract. - Revisit if consumers want one shape everywhere. + +(The earlier OQ3 — whether the per-kind list JSON should also be docmap — is resolved: +yes, docmap everywhere. +See Resolved Decision 21.) ## Implementation Plan @@ -1008,9 +1049,16 @@ rest of the test wiring in Phase 5). `.tbd/README.md` layout contract), format-history and layout-doc updates (`tbd-design.md`, `development.md`), older-CLI newer-format-detection behavior verified with a test. -2. **docref module port** (`src/docref/`): bring the parser, types, and spec-mirror - tests over from the #117 branch (standalone, already fully covered); wire docref - normalization into config source strings and URL handling. +2. **docref + docmap modules — standalone, fully tested, extraction-ready** + (`src/docref/`, `src/docmap/`): bring the docref parser, types, and spec-mirror tests + over from the #117 branch, and add a `src/docmap/` module (the docmap/0.1 type, + (de)serialize, and validate, authored against `references/docmap-format.md`). Both + are **dependency-free** (no tbd-internal imports), expose a small public API, and + carry their own full test suites — structured so they can move to a standalone + package later without change (Resolved Decision 23); tbd imports them, never the + reverse. Wire docref normalization into config source strings and the `--add` / + `docs add` URL handling, and make `DocMap` the type every list/inventory command + produces (Resolved Decisions 21–22). 3. **Manifest + base module** (`src/file/fork-manifest.ts`): zod schema, read/write of `.tbd/doc-forks/forks.yml`, base copies under `.tbd/doc-forks/base/`, LF-normalized SHA-256 hashing, state computation @@ -1039,9 +1087,14 @@ rest of the test wiring in Phase 5). 8. **`tbd docs status` (+ bare `tbd docs`)** with `--json` per the docmap map schema; one-line summary in `tbd status`. -9. **`tbd docs list` and `tbd docs show `**: cross-kind listing with state markers - (`--json` per the docmap map schema); kind-agnostic read (per-kind readers - unchanged). +9. **Shared docmap renderer + `tbd docs list` / `show `**: build the single + rendering layer (Resolved Decision 22) that turns a `DocMap` into either text + (grouped list, status table, overview, single-entry read) or `--json`, then implement + cross-kind `list` (with state markers) and kind-agnostic `show` on top of it. + **Migrate the per-kind readers (`guidelines`/`shortcut`/`template`) onto the same + renderer**: their `--list --json` switches from the flat array to a docmap (Resolved + Decision 21) and their text output becomes the one canonical format. + The reader *commands* stay; only their shared rendering path changes. 10. **`docs_cache.local_dirs` + `tbd docs add ` + grouped sync**: local-dirs wiring into the effective lookup order; `add` consolidating the per-kind `--add` flags (kept as aliases) with docref normalization replacing the ad-hoc blob-URL @@ -1060,10 +1113,10 @@ rest of the test wiring in Phase 5). exit-code handling, dry-run via `-p`), the per-state decision logic from the update table, base advancement, `conflicted` flag set/auto-clear. Unit tests cover every row × strategy of the table. -14. **`tbd docs update` command surface** with mutually exclusive `--merge` / `--rebase` - strategy flags: name filtering, `--dry-run` preview with conflict listing, the - skip-warning naming both strategies, summary counts; pending-update reporting wired - into `tbd setup --auto` output and `tbd status`. +14. **`tbd docs update` command surface** with mutually exclusive `--merge` / + `--keep-ours` strategy flags: name filtering, `--dry-run` preview with conflict + listing, the skip-warning naming both strategies, summary counts; pending-update + reporting wired into `tbd setup --auto` output and `tbd status`. ### Phase 4: Setup and packs @@ -1121,12 +1174,19 @@ duplicated** — the contract is that those exact blocks appear in the named doc Two consistency points the contract pins down: -- **The per-kind list JSON is unchanged.** `tbd guidelines --list --json` keeps emitting - a flat array (today’s shape). - Only `tbd docs list --json` / `tbd docs status --json` emit the new docmap object, - with tbd’s state fields as extension fields. - We deliberately do *not* migrate the per-kind arrays to docmap (backward-compat, and - they predate the format) — see Open Question 3. +- **One data model, one renderer — docmap is it (no backward-compat carve-outs).** Every + command that lists or inventories docs builds a single in-memory **docmap** and then + renders it; `--json` serializes that docmap, and text mode runs it through one shared + renderer. This applies uniformly: `tbd docs list` / `tbd docs status`, the per-kind + `tbd guidelines/shortcut/template --list` (whose `--json` **changes** from today’s + flat array to a docmap — accepted, since we have no backward-compat requirement here), + and the bare `tbd docs` overview (a docmap rendered in summary form). + Single-doc reads (`tbd docs show`, the per-kind `` readers) emit one docmap + *document entry* plus a `content` field — the same entry shape, so list and read never + drift. Because text and JSON derive from the same structure, they cannot disagree, and + adding a field or a state is a one-place change. + This is the systematic consistency the design optimizes for; see Resolved Decisions + 21–22. - **`tbd-docs.md` is both a rendered manual and a forkable `reference` doc.** Its bundled source stays at the docs root (`packages/tbd/docs/tbd-docs.md`); the cache/kind-dir is `references/` and the lookup name is `tbd-docs`. The doc-sync map @@ -1152,9 +1212,15 @@ shown literally — no patterns on values we control. ### Console output style contract -Every `tbd docs` command obeys the conventions already used across tbd (verified against -`output.ts`, `sections.ts`, and `doc-command-handler.ts`). This is the consistency -anchor the user asked for; the maps below are just this contract applied: +This contract is **authoritative and enforced structurally, not by convention**. Doc +commands must not hand-roll `console.log` formatting; all of them route through a single +shared rendering layer (extend `output.ts` / `sections.ts`, or a small `docs-render.ts` +that builds on them) that owns list-entry layout, state markers, the summary line, +tables, and the overview. +One docmap in, one renderer out (see the consistency point above), so there is exactly +one place to change any of these and no command can drift from another. +Every rule below is verified against today’s `output.ts`, `sections.ts`, and +`doc-command-handler.ts`; the maps that follow are just this contract applied: - **Section headers** are `formatHeading()` — UPPERCASE, bold (`INTEGRATIONS`, `HEALTH CHECKS`); bodies indent two spaces. @@ -1171,7 +1237,10 @@ anchor the user asked for; the maps below are just this contract applied: (Staleness and `conflicted` are lifecycle facts shown by `tbd docs status` / the summary line, not list markers — list markers describe ownership.) - **Provenance on serve** is one dim line to **stderr** (so piped stdout stays clean), - reusing the existing stderr-note channel: `(serving forked copy: [PATH])`. + reusing the existing stderr-note channel: `(serving forked copy: [PATH])`. It is **on + by default** (suppressed only by `--quiet`/`--json`), deliberately not gated behind + `--verbose` — the small amount of extra context helps an agent remember which docs are + customized in a session. - **Outcomes**: success → `✓ ` green to stdout; refusal/error → `✗ ` red to stderr + non-zero exit; preview → `[DRY-RUN] ` yellow (the `output.dryRun` form). - **`--json`** goes through `output.data(...)`: docmap object for `list`/`status`, @@ -1286,6 +1355,11 @@ $ tbd docs list --json ? 0 ``` +`tbd guidelines --list --json` (and `shortcut`/`template`) emit the **same** docmap +object, filtered to that kind — the consistency the design mandates: one shape, one +renderer, so the only difference between these commands’ output is the set of documents +in it. + ### `tbd docs show` / `tbd docs manual` Kind-agnostic read; `reference` docs carry no agent header (unlike guidelines). @@ -1402,7 +1476,7 @@ Updated 2 forked docs: ⚠ acme-style your changes conflict with upstream re-run with one of: tbd docs update acme-style --merge # combine, then resolve conflict markers - tbd docs update acme-style --rebase # keep your version, advance the fork point + tbd docs update acme-style --keep-ours # keep your version, advance the fork point ? 0 $ tbd docs update acme-style --merge @@ -1410,7 +1484,7 @@ $ tbd docs update acme-style --merge Resolve the <<<<<<< / ======= / >>>>>>> markers, then the doc returns to 'customized'. ? 0 -$ tbd docs update acme-style --rebase +$ tbd docs update acme-style --keep-ours ✓ acme-style kept your version; fork point advanced to current upstream. ? 0 @@ -1493,6 +1567,7 @@ bead, blocked on the phase that ships the behavior: | --- | --- | --- | | `cli-help-all.tryscript.md` (≈7 `tbd docs` blocks: `--help` `[topic]`/`--section`, `--list` sections, positional topic, `--section` content, `--list --json`, bare manual) | **Rewrite** to the new surface: `docs` subcommand help; no top-level `--section`/section-`--list`; section nav becomes `tbd docs show tbd-docs --section`. Largest single change. | 1–2 | | `cli-doc-output.tryscript.md` ("Docs Command" block: `tbd docs --list` → “Available documentation sections:”) | **Rewrite** to `tbd docs list` (cross-kind, state markers) + `--json` docmap. | 2 | +| `cli-doc-output.tryscript.md` ("Guidelines --json returns structured data" block: flat `[ { … } ]` array) | **Rewrite**: per-kind `--list --json` now emits a docmap object, not an array (Resolved Decision 21). The per-kind `--list` *text* blocks stay (same canonical format), so only the JSON assertion changes. | 2 | | `golden-output.test.ts` (`tbd docs --all` inline snapshot) | **Replace** with the bare `tbd docs` overview snapshot (`--all` folded into the overview). | 2 | | `golden-output.test.ts` ("post-setup What’s Next") | **Extend** to assert the Docs menu lines. | 4 | | `cli-orientation-golden.tryscript.md` (`tbd status`) | **Unchanged** for zero forks (verifies the guarantee); **add** a new forked-state status golden in a fixture with a fork. | 1 | @@ -1511,9 +1586,9 @@ decision-table units + a `cli-docs-update.tryscript.md` upgrade/merge scenario ( - **Unit (vitest)**: manifest round-trip; hash normalization (CRLF/LF); the full state matrix as a table-driven test (base hash × file hash × cache hash × conflicted flag → state); the update decision table row by row and per strategy (replace, clean - three-way, conflict skip, `--merge` markers + base advance, `--rebase` keep-file + + three-way, conflict skip, `--merge` markers + base advance, `--keep-ours` keep-file + base advance, strategy-flag mutual exclusion, conflicted-pending skip, missing-base - repair via `--rebase`, orphaned); marker auto-clear; f04→f05 migration; the ported + repair via `--keep-ours`, orphaned); marker auto-clear; f04→f05 migration; the ported docref spec-mirror tests; local_dirs precedence ordering; source-root grouping (N docs from one repo → one fetch; per-group failure isolation; cache preserved on fetch failure); git revision/tag capture in the manifest; `--json` output validating against From 9811114a8e8afadba31c7e92dc57a8eb617728df Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 12 Jun 2026 08:10:07 +0000 Subject: [PATCH 03/36] process: forkable-docs plan: themes (no detection), 2-axis onboarding Apply review decisions and rename the spec off "eject": - Drop the --relevant flag, repo auto-detection, and the hard-coded pack->doc map. Themes come from each doc's frontmatter `category` (retiring the brittle inferGuidelineCategory name-inference, which mis-files convex-rules and has no convex/electron theme). The agent forks the general theme plus the repo's languages/frameworks from a clear list; fork accepts names, --category, or --all. - Onboarding now presents two explicit axes: scope (all standard guidelines, recommended, or a theme subset) and visibility (hidden cache "magic" vs forked into docs/tbd/). Both make the same guidelines active; forking only adds visibility and customizability. - Handle out-of-band deletion of a forked file: serving falls back to upstream, status reports `missing` with restore/finalize options, and doctor --fix finalizes the unfork. Added a golden map and E2E coverage. - Rename the spec file eject-forkable-docs.md -> forkable-docs.md and remove lingering current-usage "eject" (the routing synonym); keep only the rename-rationale references in Alternatives #7 and decisions 1/2/14. - Resolved decisions 24-26; closed all open questions; corrected the Phase 1/2/4/5 beads (themes, shared renderer, deletion handling, two-axis onboarding, skill-baseline routing). https://claude.ai/code/session_01X8S12JzmmxEfLpYzgH8Y7E --- ...cs.md => plan-2026-06-11-forkable-docs.md} | 305 +++++++++++++----- 1 file changed, 219 insertions(+), 86 deletions(-) rename docs/project/specs/active/{plan-2026-06-11-eject-forkable-docs.md => plan-2026-06-11-forkable-docs.md} (86%) diff --git a/docs/project/specs/active/plan-2026-06-11-eject-forkable-docs.md b/docs/project/specs/active/plan-2026-06-11-forkable-docs.md similarity index 86% rename from docs/project/specs/active/plan-2026-06-11-eject-forkable-docs.md rename to docs/project/specs/active/plan-2026-06-11-forkable-docs.md index b6908d77..c96dcf86 100644 --- a/docs/project/specs/active/plan-2026-06-11-eject-forkable-docs.md +++ b/docs/project/specs/active/plan-2026-06-11-forkable-docs.md @@ -30,9 +30,9 @@ layout revision (format `f05`, a metadata-only migration from f04): 1. **A `tbd docs` command group** scoped to managed docs, following tbd’s existing noun-verb convention (`dep add`, `label add`, `attic restore`): `tbd docs fork` - copies any bundled doc (or all of them, or a language-relevant pack) into a visible, - git-tracked folder in the repo (default `docs/tbd/`). Forked docs shadow the bundled - copies in all lookups, so customizing them Just Works. + copies any bundled doc (one, a theme, or all of them) into a visible, git-tracked + folder in the repo (default `docs/tbd/`). Forked docs shadow the bundled copies in + all lookups, so customizing them Just Works. `tbd sync` keeps its scope (project data); `tbd docs sync` takes over cache refresh. 2. **`tbd docs unfork`** — remove a forked copy and fall back to the upstream version, refusing to discard customizations unless `--force` is given. @@ -47,7 +47,9 @@ layout revision (format `f05`, a metadata-only migration from f04): merging cheap, exact, offline operations. 5. **Agent-first setup opt-in** — no interactive prompts (agents are the operators): `tbd setup --auto` keeps current behavior and prints a self-documenting summary of - the visibility options with a repo-aware `--relevant` preview, while + the two choices — *scope* (all standard guidelines active, recommended, or a subset + by theme) and *visibility* (leave them in the hidden cache, the “magic” path, or fork + into `docs/tbd/` for explicit, customizable, git-tracked copies) — while `welcome-user`/the skill teach agents to offer the choice conversationally and run fork themselves. 6. **An upstream-contribution playbook** — a bundled shortcut that walks an agent @@ -74,8 +76,9 @@ lack. Everything here is forward-compatible with the larger #117 design (see - **G5. Setup choice, without interactivity:** Setup surfaces how visible docs can be — via self-documenting output and agent-led conversation, never prompts; the default remains exactly the current behavior (hidden cache). -- **G6. Language relevance:** Users can say what language(s) they work in and fork just - the relevant guidelines (with auto-detection as a convenience). +- **G6. Theme-based selection:** Docs are organized by theme (frontmatter `category`), + so an agent forks the general guidelines plus the themes for the repo’s languages and + frameworks — from a clear list, with no auto-detection and no hard-coded pack map. - **G7. Upstream loop:** A documented, low-ceremony path from “I improved a guideline” to “an issue with the diff is filed on jlevy/tbd.” - **G8. Agent-operable:** Every step is a plain CLI command with `--json` output, and @@ -551,6 +554,29 @@ The `conflicted` flag is set only by `update --merge` and clears automatically o standard markers (`<<<<<<<`/`=======`/`>>>>>>>`) are gone; scanning for markers only in flagged docs avoids false positives on docs that legitimately discuss merge conflicts. +**Out-of-band deletion — the user removes a forked file (or the whole fork dir) without +telling tbd.** This is an expected case, not an error: the fork dir is ordinary repo +files, and people delete files. +Because lookups fall through the precedence list, a deleted forked file **transparently +falls back to the upstream cache copy** — the doc keeps working and +`tbd guidelines ` still serves it (from upstream now, with no provenance note, +since nothing is forked there anymore). +The dangling manifest entry surfaces as `missing`, and tbd offers exactly two clean +resolutions everywhere it is reported: + +- **restore** it — `tbd docs fork --force` re-creates the file from the recorded + base (your fork point), or +- **finalize** the removal — `tbd docs unfork ` clears the manifest entry and base + snapshot. + +`tbd docs status` lists `missing` docs with those two options; `tbd doctor` flags them +and `tbd doctor --fix` **finalizes the unfork** (the deletion is read as intent to stop +forking — it removes the orphaned manifest entry and base, leaving the doc served from +upstream); `tbd docs update` skips `missing` docs. +Deleting the entire fork dir is just this case in bulk: every entry becomes `missing`, +all serving falls back to upstream, and `doctor --fix` clears the manifest. +Nothing is ever silently re-created against the user’s deletion. + ### Updating forked docs after a tbd upgrade The most common lifecycle event: you forked docs, upgraded tbd (or `tbd docs sync` @@ -622,8 +648,8 @@ tbd docs add github:org/repo@main//docs/rules.md --kind=guideline # any docref tbd docs fork python-rules # one doc (name resolution as in `tbd guidelines`) tbd docs fork python-rules review-code # several tbd docs fork --kind=guideline typescript # disambiguate if a name exists in two kinds -tbd docs fork --pack=python # curated pack (repeatable: --pack=python --pack=core) -tbd docs fork --relevant # packs chosen by repo detection (see below) +tbd docs fork --category=python # a whole theme (reads frontmatter; repeatable) +tbd docs fork --category=general --category=typescript # general + a language tbd docs fork --all # everything tbd docs fork --all --dry-run # preview what would be written @@ -674,25 +700,47 @@ Behavior details: collisions, unreachable sources (per source group, serving last cached copy), fork dir covered by a `.gitignore` (defeats the purpose — warn), manifest/dir drift. -### Packs and language detection +### Doc themes and the fork recommendation + +There is **no `--relevant` flag, no repo auto-detection, and no hard-coded pack→doc +map.** Detection rules and a central pack list both drift out of sync with the docs and +substitute brittle logic for an agent’s judgment. +Instead, each doc declares a **theme** via its frontmatter `category`, so a doc joins a +theme by setting one field — nothing central to keep in sync — and the agent picks based +on what the repo actually is. +(This also retires the ad-hoc name-based `inferGuidelineCategory` inference, which today +mis-files docs like `convex-rules` as `general` and has no `convex`/`electron` theme at +all; Phase 0 curates the frontmatter so each doc lands in the right theme.) -A small constant map in code (e.g. `src/file/doc-packs.ts`), not config — easy to test, -easy to extend later if we move it to frontmatter tags: +The basic themes: -| Pack | Contents (guidelines unless noted) | +| Theme (`category`) | What’s in it | | --- | --- | -| `core` | general-eng-agent-principles, general-coding-rules, general-comment-rules, error-handling-rules, general-tdd-guidelines, general-testing-rules, commit-conventions, common-doc-guidelines | -| `typescript` | typescript-rules, typescript-cli-tool-rules, typescript-sorting-patterns, typescript-yaml-handling-rules, typescript-code-coverage, pnpm-monorepo-patterns, bun-monorepo-patterns | -| `python` | python-rules, python-modern-guidelines, python-cli-patterns | -| `convex` | convex-rules, convex-limits-best-practices | -| `electron` | electron-app-development-patterns | - -`--relevant` = `core` plus packs chosen by cheap repo detection: `package.json`/ -`tsconfig.json` → `typescript`; `pyproject.toml`/`uv.lock`/`requirements.txt` → -`python`; `convex/` dir or `convex` dependency → `convex`; `electron` dependency → -`electron`. Detection is a pure function over the repo root — trivially unit-testable. -Shortcuts/templates are not in packs v1 (fork them by name or `--all`); revisit if there -is demand. +| **general** | The foundational guidelines that apply to every repo — the `general-*` rules plus coding, comment, error-handling, TDD/testing, commit, and doc guidelines. | +| **typescript** | TypeScript rules, including CLI tooling (and the sorting / YAML / coverage / monorepo rules). | +| **python** | Python rules, including CLI patterns. | +| **convex** | Convex rules and limits / best-practices. | +| **electron** | Electron app development patterns. | + +`tbd docs list` shows every doc grouped by theme, so the choices are visible. +The **recommendation** — stated in the bare `tbd docs` overview, the setup summary, the +skill, and `welcome-user`, and kept identical across all of them — is simply: **fork the +general guidelines, plus the themes for whatever languages and frameworks the repo +uses.** An agent that knows the project applies it directly (general + typescript for a +TypeScript CLI; general + python + convex for a Convex/Python backend) with no detection +table to maintain. + +Selection reuses the existing `category` metadata — no new construct, no central map: + +```bash +tbd docs fork python-rules review-code # by name +tbd docs fork --category=typescript # a whole theme (reads frontmatter; repeatable) +tbd docs fork --category=general --category=python # general + a language +tbd docs fork --all # everything +``` + +Themes are guidelines-oriented; shortcuts and templates are forked by name or with +`--all`. ### Setup integration (agent-first, non-interactive) @@ -702,28 +750,38 @@ today but has never had prompts (`setup.ts:1281`), is removed rather than built Setup is instead designed to be excellent non-interactively: - **`tbd setup --auto`: unchanged behavior, self-documenting output.** Cache-only - remains the default. - The summary *is* the menu — copy-paste commands with a repo-aware preview, since pack - detection can run read-only during setup: + remains the default (guidelines are active either way). + The summary *is* the menu, and it states the two choices explicitly so an agent — or a + user reading the output — can make them deliberately: ``` Docs: 37 available in cache (.tbd/docs/, gitignored); none forked into the repo. - Browse: tbd docs list - Make visible: tbd docs fork --relevant (detected: typescript, python → 13 docs into docs/tbd/) - tbd docs fork --all (everything) - Customize one: tbd docs fork + Guidelines are active from the cache. To make them visible and customizable, fork + them into docs/tbd/ (same behavior — just explicit and git-tracked): + + Scope: all standard guidelines (recommended), or a theme: + general, typescript, python, convex, electron + Make visible: tbd docs fork --category=general --category= + tbd docs fork --all (everything) + Browse / read: tbd docs list / tbd docs show ``` When forked docs exist with pending upstream updates (typically right after an upgrade), the summary reports the count and suggests `tbd docs update` — but setup itself never modifies files in the fork dir. -- **Agent-led onboarding is the choice mechanism.** `welcome-user` and the skill docs - instruct the agent to ask the user conversationally ("Do you want tbd’s guidelines - visible in your repo? - Which languages do you use?") and then run `tbd docs fork --pack=…` / `--relevant` / - `--all` itself. `tbd docs fork --relevant --dry-run` gives agents a zero-risk preview - to show before acting. +- **Agent-led onboarding makes both choices explicit.** `welcome-user` and the skill + instruct the agent to put two questions to the user conversationally: + 1. **Scope** — keep *all* standard guidelines active (recommended), or just the themes + for your stack (general plus your languages/frameworks)? + 2. **Visibility** — leave them in tbd’s hidden cache (they just work — the “magic” + path), or fork them into `docs/tbd/` so they are visible on GitHub, reviewable, and + customizable (checked into git)? + + The agent explains that forking changes nothing about how guidelines work — both paths + make the same guidelines active — it only makes them explicit and editable. + It then runs `tbd docs fork --category=…` / `--all` (or leaves the cache as-is) + accordingly, using `--dry-run` to preview first. No setup flags needed: `tbd docs` *is* the API. ### Upstream-contribution playbook @@ -751,11 +809,11 @@ The skill routing table gets matching rows, e.g.: | User says | Agent runs | | --- | --- | | “What guidelines are there?” | `tbd docs list` | -| “Show me / let me browse the guidelines in this repo” | `tbd docs fork --relevant` (after confirming) | +| “Make the guidelines visible / put the relevant ones in my repo” | `tbd docs fork --category=general` plus the repo’s languages, after confirming scope + visibility | | “I want to customize the Python guidelines” | `tbd docs fork python-rules` then edit | | “Put all of tbd’s docs in my repo” | `tbd docs fork --all` | | “Stop customizing X / go back to the default” | `tbd docs unfork X` (`--force` only after confirming) | -| “Eject the guidelines …” (legacy term) | treat as fork: `tbd docs fork …` | +| “I deleted a forked guideline file” | `tbd docs status` shows it `missing`; `tbd docs fork X --force` to restore or `tbd docs unfork X` to finalize | | “Update the guidelines to the latest” (or after `tbd setup` reports pending updates) | `tbd docs update`; if conflicts are listed, ask the user, then `--merge` (combine + resolve) or `--keep-ours` (keep ours) | | “Could we contribute these improvements back?” | `tbd shortcut suggest-upstream-improvements` | @@ -859,9 +917,9 @@ Settled during design review (2026-06-11): points to `npx get-tbd@latest docs` for further info. 8. **All fork state lives under one committed directory, `.tbd/doc-forks/`** — - `forks.yml` (manifest) plus `base/` (snapshots) — revised from the earlier separate - `.tbd/ejected.yml` + `.tbd/eject-base/` pair so the layout is self-describing and - `.tbd/docs/` remains purely the cache. + `forks.yml` (manifest) plus `base/` (snapshots) — consolidated into one + self-describing directory (rather than scattered top-level files) so `.tbd/docs/` + remains purely the cache. 9. **Format bump to f05** with a metadata-only f04→f05 migration, following the f04 precedent: gitignore template refresh, format-history and layout-doc updates; older @@ -887,9 +945,9 @@ Settled during design review (2026-06-11): The taxonomy table is documented in `tbd-docs.md`. 13. **No interactive setup.** The unused `--interactive` flag is removed; setup is - agent-first and non-interactive, with self-documenting summary output (including a - read-only `--relevant` detection preview) and conversational onboarding via the - skill and `welcome-user`. + agent-first and non-interactive, with self-documenting summary output (naming the + scope and visibility choices) and conversational onboarding via the skill and + `welcome-user`. 14. **Terminology: fork/unfork, upstream, built-in.** The original `eject`/`uneject` vocabulary (create-react-app heritage: a one-way escape from a managed bundle) fit @@ -899,7 +957,6 @@ Settled during design review (2026-06-11): `docs_cache.fork_dir`; the former `bundled` state is now `upstream` (not forked; served from its upstream via the cache); “bundled”/“built-in” is reserved for tbd-shipped `internal:` docs, where it remains literally true. - “Eject” stays as a routing synonym in the skill table. (For reference, shadcn names the model — “open code,” “copy and own” — but its verb is just `add`, with no update story.) @@ -969,17 +1026,41 @@ Settled during refinement (2026-06-12): (the docref parser comes over from the #117 branch already in this shape). tbd consumes them; they do not consume tbd. +24. **No detection, no hard-coded pack map; themes are frontmatter `category`.** The + `--relevant` flag, repo auto-detection, and a central pack→doc list are all dropped + — they drift out of sync with the docs and replace agent judgment with brittle + rules. Each doc declares its theme in frontmatter (`general`, `typescript`, `python`, + `convex`, `electron`), the name-based `inferGuidelineCategory` inference is retired, + and the agent forks the general theme plus the repo’s languages/frameworks. + `tbd docs fork` accepts names, `--category`, or `--all`. + +25. **Onboarding presents two explicit axes: scope and visibility.** Setup and + `welcome-user` make clear that (a) all standard guidelines are active by default + (recommended; can be subset by theme), and (b) the user can keep them in the hidden + cache (the “magic” path) or fork them into `docs/tbd/` for explicit, customizable, + git-tracked copies. Both paths make the *same* guidelines active — forking only adds + visibility and editability — and that equivalence is stated wherever the choice is + offered. + +26. **Out-of-band deletion of a forked file is a supported state, not an error.** + Serving falls back to upstream automatically; `tbd docs status` reports `missing` + with two resolutions (restore via `fork --force`, or finalize via `unfork`); + `tbd doctor --fix` finalizes the unfork; deleting the whole fork dir is the same + case in bulk. Covered by a golden test. + ## Open Questions -1. **Should `--relevant` ever become the fresh-setup default?** Recommended: no for now - — current behavior stays default per the explicit product call; revisit with usage - feedback. -2. **Pack definitions in code vs doc frontmatter tags**: code const now (recommended); - migrate to frontmatter `tags:` if packs grow or third-party doc sources arrive. +The questions raised during earlier reviews are all now resolved: -(The earlier OQ3 — whether the per-kind list JSON should also be docmap — is resolved: -yes, docmap everywhere. -See Resolved Decision 21.) +- *Should `--relevant` become the fresh-setup default?* — Moot: the `--relevant` flag + and repo auto-detection are removed entirely (Resolved Decision 24); selection is by + theme, chosen by the agent. +- *Pack definitions in code vs frontmatter tags?* — Resolved: themes are each doc’s + frontmatter `category`, with no central pack map (Resolved Decision 24). +- *Should the per-kind list JSON also be docmap?* — Resolved: yes, docmap everywhere + (Resolved Decision 21). + +No open questions remain; new ones will be surfaced here as implementation proceeds. ## Implementation Plan @@ -1080,8 +1161,9 @@ rest of the test wiring in Phase 5). `[forked]`/`[customized]`/`[local]` markers in `--list`. 7. **E2E tests** (pattern of `doc-add-e2e.test.ts`): fork → list marker → serve shows forked content → edit → unfork refuses → `--force` succeeds → upstream serving - restored; `tbd setup --auto` refresh leaves forked files untouched; f04 repo migrates - to f05 on setup. + restored; **delete a forked file out-of-band → serve transparently falls back to + upstream and `tbd docs status` reports `missing`**; `tbd setup --auto` refresh leaves + forked files untouched; f04 repo migrates to f05 on setup. ### Phase 2: Status, browse, diff, doctor @@ -1103,9 +1185,10 @@ rest of the test wiring in Phase 5). for manifest provenance; no cache pruning on fetch failure). 11. **`tbd docs diff `** with `--base` / `--upstream` variants (`git diff --no-index` style output against base and cache copies, no network). -12. **`tbd doctor` checks**: missing/orphaned entries, base missing/corrupt, unresolved - conflicts, reserved `tbd-` names, gitignored fork dir warning, `--fix` for manifest - cleanup. +12. **`tbd doctor` checks**: missing (deleted) / orphaned entries, base missing/corrupt, + unresolved conflicts, reserved `tbd-` names, gitignored fork dir warning; `--fix` + finalizes the unfork for `missing` forked files (treating an out-of-band deletion as + intent to stop forking) and cleans orphaned manifest entries. ### Phase 3: Update and merge @@ -1118,12 +1201,19 @@ rest of the test wiring in Phase 5). listing, the skip-warning naming both strategies, summary counts; pending-update reporting wired into `tbd setup --auto` output and `tbd status`. -### Phase 4: Setup and packs - -15. **Packs + detection** (`src/file/doc-packs.ts`): pack map, `--pack`, `--relevant`, - detection function with unit tests. -16. **Setup integration**: self-documenting `--auto` summary (fork options as copy-paste - commands with read-only pack-detection preview; pending-update count); removal of +### Phase 4: Setup and themes + +15. **Themes via frontmatter `category`** (no `doc-packs.ts`, no detection function): + curate the `category` field on the bundled docs so each lands in exactly one theme + (`general`, `typescript`, `python`, `convex`, `electron`), retire the name-based + `inferGuidelineCategory` inference in favor of the declared field, and add + `--category` selection to `tbd docs fork` (reusing the existing `--list --category` + metadata — no new map). + Unit tests: category-based fork selection, and that every bundled doc resolves to + exactly one theme. +16. **Setup integration**: self-documenting `--auto` summary naming the two choices — + *scope* (all standard guidelines, recommended, or a theme subset) and *visibility* + (hidden cache vs fork into `docs/tbd/`) — plus the pending-update count; removal of the unused `--interactive` flag; the fork dir documented as a config customization (`docs_cache.fork_dir`); the sync-taxonomy table added to `tbd-docs.md`. @@ -1138,9 +1228,13 @@ rest of the test wiring in Phase 5). as exploratory background only) as reference docs, retire the bare-`tbd docs` manual viewer in favor of `tbd docs show tbd-docs` + the `tbd docs manual` alias (`tbd readme`/`tbd design` untouched). -19. **Agent docs**: routing rows + fork/update section in `tbd-docs.md`, skill header - (`install/claude-header.md`), `welcome-user` onboarding question, README section - ("Forkable guidelines: fork them into your repo"), `tbd prime` mention if warranted. +19. **Agent docs**: routing rows (fork / update / upstream + the missing-file row) in + the skill (`shortcuts/system/skill-baseline.md`) and a fork/update section in + `tbd-docs.md`; the two-axis (*scope* + *visibility*) `welcome-user` onboarding; + README section ("Forkable guidelines: fork them into your repo"); `tbd prime` + mention if warranted. + (`install/claude-header.md` needs no change — `Bash(tbd:*)` already covers + `tbd docs`.) 20. **CHANGELOG + release notes** per `release-notes-guidelines`. ## Documentation Contract Changes @@ -1160,10 +1254,10 @@ duplicated** — the contract is that those exact blocks appear in the named doc | `packages/tbd/src/lib/tbd-format.ts` (`FORMAT_HISTORY`) | Phase 1 (with code) | Add the `f05` entry: `introduced` (next minor), description “Adds forkable-docs layout”, `changes` = [`docs_cache.fork_dir`, `docs_cache.local_dirs`, `.tbd/doc-forks/forks.yml` + `base/`, generated `.tbd/README.md`], `migration` = “metadata-only: stamp f05, refresh `.tbd/.gitignore`, write `.tbd/README.md` layout contract”. `CURRENT_FORMAT = 'f05'`. This file is the authoritative format history; its wording is the contract Phase 0 references but does not edit. | | `docs/development.md` (this repo) | Phase 0 | “Path Conventions” block: add `.tbd/doc-forks/` (committed) and note the fork dir lives **outside** `.tbd/` (default `docs/tbd/`). Add a “Testing forkable docs” pointer to the new e2e/tryscript files. | | `docs/docs-overview.md` (this repo) | Phase 0 | “tbd CLI Documentation Commands” + “Adding external docs by URL”: replace with the `tbd docs` group; `tbd docs add `; add a line on forking docs into a visible `docs/tbd/`. | -| `README.md` | Phase 0 | “Shortcuts, Guidelines, and Templates”: add a “Forkable: see them in your repo” paragraph with a `tbd docs fork --relevant` example. “Documentation” block: `tbd docs` is now an overview; the manual is `tbd docs show tbd-docs`. Per-kind `--add` lines annotated as aliases for `tbd docs add`. | +| `README.md` | Phase 0 | “Shortcuts, Guidelines, and Templates”: add a “Forkable: see them in your repo” paragraph with a `tbd docs fork --category=general` example. “Documentation” block: `tbd docs` is now an overview; the manual is `tbd docs show tbd-docs`. Per-kind `--add` lines annotated as aliases for `tbd docs add`. | | `packages/tbd/docs/shortcuts/system/skill-baseline.md` (injected agent skill) | Phase 5 | Add the fork/update/upstream rows to “User Request → Agent Action” (the rows in “Upstream-contribution playbook”); add `tbd docs list` / `tbd docs fork` to the “Documentation” command table; one-line “Forkable docs” note. Kept within the skill’s size budget; lands with the rest of the agent surface so routing is never half-wired. | | `packages/tbd/docs/install/claude-header.md` | none | **No change.** Its `allowed-tools: Bash(tbd:*)` already covers `tbd docs`. Stated here so the audit is explicit. | -| `packages/tbd/docs/shortcuts/standard/welcome-user.md` | Phase 5 | Add the onboarding offer after the status summary ("Want tbd’s guidelines visible in your repo? Which languages?") and a “make guidelines visible” row routing to `tbd docs fork --relevant`. | +| `packages/tbd/docs/shortcuts/standard/welcome-user.md` | Phase 5 | Add the two-axis onboarding offer after the status summary — *scope* (all guidelines vs a theme subset) and *visibility* (hidden cache vs forked into `docs/tbd/`) — plus a “make guidelines visible” row routing to `tbd docs fork --category=…`. | | `packages/tbd/docs/shortcuts/standard/suggest-upstream-improvements.md` (**new**) | Phase 0 | The upstream-contribution playbook (per “Upstream-contribution playbook”). Pure docs. | | `packages/tbd/docs/references/docref-format.md` (**new**) | Phase 0 | Adopted from the #117 branch, marked adopted v0.1. First doc of the new `reference` kind. | | `packages/tbd/docs/references/docmap-format.md` (**new**) | Phase 0 | Authored fresh, minimal (docmap/0.1 inventory only) per the Design section; #117 cited as background. | @@ -1273,12 +1367,14 @@ $ tbd docs # no docs forked yet tbd docs — managed documentation 37 available in cache (.tbd/docs/, gitignored); none forked into the repo. - - Browse: tbd docs list - Make visible: tbd docs fork --relevant (detected: typescript, python → 13 docs into docs/tbd/) - tbd docs fork --all (everything) - Customize one: tbd docs fork - Learn more: tbd docs show tbd-docs + Guidelines are active from the cache. Fork them into docs/tbd/ to make them + visible and customizable (same behavior — just explicit and git-tracked): + + Scope: all standard guidelines (recommended), or a theme: + general, typescript, python, convex, electron + Make visible: tbd docs fork --category=general --category= + tbd docs fork --all (everything) + Browse / read: tbd docs list / tbd docs show ? 0 ``` @@ -1393,11 +1489,11 @@ $ tbd docs fork python-rules Edit it in place — tbd now serves your copy everywhere it served the upstream one. ? 0 -$ tbd docs fork --relevant --dry-run -[DRY-RUN] Would fork 13 docs into docs/tbd/ (packs: core, typescript, python) +$ tbd docs fork --category=general --category=python --dry-run +[DRY-RUN] Would fork 11 docs into docs/tbd/ (themes: general, python) guideline general-eng-agent-principles guideline general-coding-rules - [.. 9 more ..] + [.. 7 more ..] guideline python-rules No files written. Re-run without --dry-run to apply. ? 0 @@ -1445,6 +1541,37 @@ tbd-docs reference forked internal:tbd-docs.md ? 0 ``` +### Removed forked file (out-of-band deletion) + +The user deletes a forked file directly (`rm docs/tbd/guidelines/review-code.md`) +without telling tbd. +Serving keeps working (falls back to upstream), `status` reports `missing` with both +resolutions, and `doctor --fix` finalizes the unfork: + +```text +$ rm docs/tbd/shortcuts/review-code.md + +$ tbd docs show review-code # still works — falls back to upstream +[.. upstream review-code content on stdout, no provenance note ..] +? 0 + +$ tbd docs status +NAME KIND STATE SOURCE +acme-style guideline customized, stale github:acme/eng-docs@main//guidelines/style.md +python-rules guideline customized, stale internal:guidelines/python-rules.md +review-code shortcut missing internal:shortcuts/standard/review-code.md +tbd-docs reference forked internal:tbd-docs.md + +1 doc is missing (forked file deleted): + review-code restore with 'tbd docs fork review-code --force', or finalize with 'tbd docs unfork review-code' +? 0 + +$ tbd doctor --fix # excerpt +⚠ Forked docs - 1 missing (review-code: forked file deleted) + Fixed: finalized unfork (removed manifest entry + base); now served from upstream +? 0 +``` + ### `tbd docs diff` Git-style, no network (`git diff --no-index` against the relevant copy): @@ -1532,10 +1659,13 @@ pending-update report); setup never writes the fork dir: ```text # zero forks Docs: 37 available in cache (.tbd/docs/, gitignored); none forked into the repo. - Browse: tbd docs list - Make visible: tbd docs fork --relevant (detected: typescript, python → 13 docs into docs/tbd/) - tbd docs fork --all (everything) - Customize one: tbd docs fork + Guidelines are active from the cache. Fork them into docs/tbd/ to make them + visible and customizable (same behavior — just explicit and git-tracked): + Scope: all standard guidelines (recommended), or a theme: + general, typescript, python, convex, electron + Make visible: tbd docs fork --category=general --category= + tbd docs fork --all (everything) + Browse / read: tbd docs list / tbd docs show # after an upgrade, forks present Docs: 4 forked into docs/tbd/. 3 have upstream updates — run 'tbd docs update'. @@ -1579,7 +1709,9 @@ New golden/e2e files (named for the phases that add them): `fork-manifest` + state-matrix units (Phase 1); a `cli-docs-fork.tryscript.md` lifecycle (fork → list marker → serve → edit → unfork refuse → `--force`) (Phase 1); `fork-update` decision-table units + a `cli-docs-update.tryscript.md` upgrade/merge scenario (Phase -3); `doc-packs` detection units + a `fork --relevant` fixture e2e (Phase 4). +3); category-selection units + a `fork --category` e2e (Phase 4); a deleted-fork +scenario in the Phase 1 lifecycle test (serve falls back to upstream, status `missing`, +`doctor --fix` finalizes). ## Testing Strategy @@ -1592,7 +1724,7 @@ decision-table units + a `cli-docs-update.tryscript.md` upgrade/merge scenario ( docref spec-mirror tests; local_dirs precedence ordering; source-root grouping (N docs from one repo → one fetch; per-group failure isolation; cache preserved on fetch failure); git revision/tag capture in the manifest; `--json` output validating against - the docmap map schema; pack detection; fork path mapping (incl. + the docmap schema; category-based fork selection; fork path mapping (incl. shortcuts flattening); README index generation. - **E2E (spawn against built CLI, like `doc-add-e2e.test.ts`)**: the Phase 1 scenario above; precedence (forked shadows upstream; local file with no entry is served); an @@ -1602,7 +1734,8 @@ decision-table units + a `cli-docs-update.tryscript.md` upgrade/merge scenario ( `customized`); convergence-unfork (upstream adopts the customization → `update` → plain `unfork` succeeds); group surface (bare `tbd docs` shows status, `tbd docs show tbd-docs` serves the manual, `tbd sync --docs` alias still works); - `fork --relevant` in a fixture repo with `pyproject.toml`; collision/overwrite + `fork --category=python`; out-of-band deletion of a forked file (serve falls back to + upstream, status `missing`, `doctor --fix` finalizes the unfork); collision/overwrite refusal; doctor findings. - **Docs-reference test**: extend `doc-references.test.ts` so every command named in the new shortcut/docs resolves (extractor learns `tbd docs ` and the From f70cad5c69b675050675b412a11702bcdb132f9b Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 12 Jun 2026 08:22:23 +0000 Subject: [PATCH 04/36] process: forkable-docs plan: use "category" not "themes" MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Rename the guideline-grouping concept from "theme(s)" to "category"/ "categories" throughout the spec. "Theme" was ambiguous (suggests UX appearance/styling), and "category" matches the existing `--category` flag and frontmatter field — one consistent term. Tidy the few spots that would otherwise read redundantly (e.g. "category (frontmatter category)"). https://claude.ai/code/session_01X8S12JzmmxEfLpYzgH8Y7E --- .../active/plan-2026-06-11-forkable-docs.md | 102 +++++++++--------- 1 file changed, 52 insertions(+), 50 deletions(-) diff --git a/docs/project/specs/active/plan-2026-06-11-forkable-docs.md b/docs/project/specs/active/plan-2026-06-11-forkable-docs.md index c96dcf86..d8be8796 100644 --- a/docs/project/specs/active/plan-2026-06-11-forkable-docs.md +++ b/docs/project/specs/active/plan-2026-06-11-forkable-docs.md @@ -30,7 +30,7 @@ layout revision (format `f05`, a metadata-only migration from f04): 1. **A `tbd docs` command group** scoped to managed docs, following tbd’s existing noun-verb convention (`dep add`, `label add`, `attic restore`): `tbd docs fork` - copies any bundled doc (one, a theme, or all of them) into a visible, git-tracked + copies any bundled doc (one, a category, or all of them) into a visible, git-tracked folder in the repo (default `docs/tbd/`). Forked docs shadow the bundled copies in all lookups, so customizing them Just Works. `tbd sync` keeps its scope (project data); `tbd docs sync` takes over cache refresh. @@ -48,8 +48,8 @@ layout revision (format `f05`, a metadata-only migration from f04): 5. **Agent-first setup opt-in** — no interactive prompts (agents are the operators): `tbd setup --auto` keeps current behavior and prints a self-documenting summary of the two choices — *scope* (all standard guidelines active, recommended, or a subset - by theme) and *visibility* (leave them in the hidden cache, the “magic” path, or fork - into `docs/tbd/` for explicit, customizable, git-tracked copies) — while + by category) and *visibility* (leave them in the hidden cache, the “magic” path, or + fork into `docs/tbd/` for explicit, customizable, git-tracked copies) — while `welcome-user`/the skill teach agents to offer the choice conversationally and run fork themselves. 6. **An upstream-contribution playbook** — a bundled shortcut that walks an agent @@ -76,9 +76,9 @@ lack. Everything here is forward-compatible with the larger #117 design (see - **G5. Setup choice, without interactivity:** Setup surfaces how visible docs can be — via self-documenting output and agent-led conversation, never prompts; the default remains exactly the current behavior (hidden cache). -- **G6. Theme-based selection:** Docs are organized by theme (frontmatter `category`), - so an agent forks the general guidelines plus the themes for the repo’s languages and - frameworks — from a clear list, with no auto-detection and no hard-coded pack map. +- **G6. Category-based selection:** Docs are organized by their frontmatter `category`, + so an agent forks the general guidelines plus the categories for the repo’s languages + and frameworks — from a clear list, with no auto-detection and no hard-coded pack map. - **G7. Upstream loop:** A documented, low-ceremony path from “I improved a guideline” to “an issue with the diff is filed on jlevy/tbd.” - **G8. Agent-operable:** Every step is a plain CLI command with `--json` output, and @@ -648,7 +648,7 @@ tbd docs add github:org/repo@main//docs/rules.md --kind=guideline # any docref tbd docs fork python-rules # one doc (name resolution as in `tbd guidelines`) tbd docs fork python-rules review-code # several tbd docs fork --kind=guideline typescript # disambiguate if a name exists in two kinds -tbd docs fork --category=python # a whole theme (reads frontmatter; repeatable) +tbd docs fork --category=python # a whole category (reads frontmatter; repeatable) tbd docs fork --category=general --category=typescript # general + a language tbd docs fork --all # everything tbd docs fork --all --dry-run # preview what would be written @@ -700,21 +700,21 @@ Behavior details: collisions, unreachable sources (per source group, serving last cached copy), fork dir covered by a `.gitignore` (defeats the purpose — warn), manifest/dir drift. -### Doc themes and the fork recommendation +### Doc categories and the fork recommendation There is **no `--relevant` flag, no repo auto-detection, and no hard-coded pack→doc map.** Detection rules and a central pack list both drift out of sync with the docs and substitute brittle logic for an agent’s judgment. -Instead, each doc declares a **theme** via its frontmatter `category`, so a doc joins a -theme by setting one field — nothing central to keep in sync — and the agent picks based -on what the repo actually is. +Instead, each doc declares its **category** in frontmatter, so a doc joins a category by +setting one field — nothing central to keep in sync — and the agent picks based on what +the repo actually is. (This also retires the ad-hoc name-based `inferGuidelineCategory` inference, which today -mis-files docs like `convex-rules` as `general` and has no `convex`/`electron` theme at -all; Phase 0 curates the frontmatter so each doc lands in the right theme.) +mis-files docs like `convex-rules` as `general` and has no `convex`/`electron` category +at all; Phase 0 curates the frontmatter so each doc lands in the right category.) -The basic themes: +The basic categories: -| Theme (`category`) | What’s in it | +| Category | What’s in it | | --- | --- | | **general** | The foundational guidelines that apply to every repo — the `general-*` rules plus coding, comment, error-handling, TDD/testing, commit, and doc guidelines. | | **typescript** | TypeScript rules, including CLI tooling (and the sorting / YAML / coverage / monorepo rules). | @@ -722,10 +722,10 @@ The basic themes: | **convex** | Convex rules and limits / best-practices. | | **electron** | Electron app development patterns. | -`tbd docs list` shows every doc grouped by theme, so the choices are visible. +`tbd docs list` shows every doc grouped by category, so the choices are visible. The **recommendation** — stated in the bare `tbd docs` overview, the setup summary, the skill, and `welcome-user`, and kept identical across all of them — is simply: **fork the -general guidelines, plus the themes for whatever languages and frameworks the repo +general guidelines, plus the categories for whatever languages and frameworks the repo uses.** An agent that knows the project applies it directly (general + typescript for a TypeScript CLI; general + python + convex for a Convex/Python backend) with no detection table to maintain. @@ -734,12 +734,12 @@ Selection reuses the existing `category` metadata — no new construct, no centr ```bash tbd docs fork python-rules review-code # by name -tbd docs fork --category=typescript # a whole theme (reads frontmatter; repeatable) +tbd docs fork --category=typescript # a whole category (reads frontmatter; repeatable) tbd docs fork --category=general --category=python # general + a language tbd docs fork --all # everything ``` -Themes are guidelines-oriented; shortcuts and templates are forked by name or with +Categories are guidelines-oriented; shortcuts and templates are forked by name or with `--all`. ### Setup integration (agent-first, non-interactive) @@ -759,7 +759,7 @@ Setup is instead designed to be excellent non-interactively: Guidelines are active from the cache. To make them visible and customizable, fork them into docs/tbd/ (same behavior — just explicit and git-tracked): - Scope: all standard guidelines (recommended), or a theme: + Scope: all standard guidelines (recommended), or a category: general, typescript, python, convex, electron Make visible: tbd docs fork --category=general --category= tbd docs fork --all (everything) @@ -772,8 +772,8 @@ Setup is instead designed to be excellent non-interactively: - **Agent-led onboarding makes both choices explicit.** `welcome-user` and the skill instruct the agent to put two questions to the user conversationally: - 1. **Scope** — keep *all* standard guidelines active (recommended), or just the themes - for your stack (general plus your languages/frameworks)? + 1. **Scope** — keep *all* standard guidelines active (recommended), or just the + categories for your stack (general plus your languages/frameworks)? 2. **Visibility** — leave them in tbd’s hidden cache (they just work — the “magic” path), or fork them into `docs/tbd/` so they are visible on GitHub, reviewable, and customizable (checked into git)? @@ -1026,21 +1026,22 @@ Settled during refinement (2026-06-12): (the docref parser comes over from the #117 branch already in this shape). tbd consumes them; they do not consume tbd. -24. **No detection, no hard-coded pack map; themes are frontmatter `category`.** The +24. **No detection, no hard-coded pack map; categories come from doc frontmatter.** The `--relevant` flag, repo auto-detection, and a central pack→doc list are all dropped — they drift out of sync with the docs and replace agent judgment with brittle - rules. Each doc declares its theme in frontmatter (`general`, `typescript`, `python`, - `convex`, `electron`), the name-based `inferGuidelineCategory` inference is retired, - and the agent forks the general theme plus the repo’s languages/frameworks. + rules. Each doc declares its category in frontmatter (`general`, `typescript`, + `python`, `convex`, `electron`), the name-based `inferGuidelineCategory` inference + is retired, and the agent forks the general category plus the repo’s + languages/frameworks. `tbd docs fork` accepts names, `--category`, or `--all`. 25. **Onboarding presents two explicit axes: scope and visibility.** Setup and `welcome-user` make clear that (a) all standard guidelines are active by default - (recommended; can be subset by theme), and (b) the user can keep them in the hidden - cache (the “magic” path) or fork them into `docs/tbd/` for explicit, customizable, - git-tracked copies. Both paths make the *same* guidelines active — forking only adds - visibility and editability — and that equivalence is stated wherever the choice is - offered. + (recommended; can be subset by category), and (b) the user can keep them in the + hidden cache (the “magic” path) or fork them into `docs/tbd/` for explicit, + customizable, git-tracked copies. + Both paths make the *same* guidelines active — forking only adds visibility and + editability — and that equivalence is stated wherever the choice is offered. 26. **Out-of-band deletion of a forked file is a supported state, not an error.** Serving falls back to upstream automatically; `tbd docs status` reports `missing` @@ -1054,9 +1055,9 @@ The questions raised during earlier reviews are all now resolved: - *Should `--relevant` become the fresh-setup default?* — Moot: the `--relevant` flag and repo auto-detection are removed entirely (Resolved Decision 24); selection is by - theme, chosen by the agent. -- *Pack definitions in code vs frontmatter tags?* — Resolved: themes are each doc’s - frontmatter `category`, with no central pack map (Resolved Decision 24). + category, chosen by the agent. +- *Pack definitions in code vs frontmatter tags?* — Resolved: categories come from each + doc’s frontmatter, with no central pack map (Resolved Decision 24). - *Should the per-kind list JSON also be docmap?* — Resolved: yes, docmap everywhere (Resolved Decision 21). @@ -1201,21 +1202,22 @@ rest of the test wiring in Phase 5). listing, the skip-warning naming both strategies, summary counts; pending-update reporting wired into `tbd setup --auto` output and `tbd status`. -### Phase 4: Setup and themes +### Phase 4: Setup and categories -15. **Themes via frontmatter `category`** (no `doc-packs.ts`, no detection function): - curate the `category` field on the bundled docs so each lands in exactly one theme - (`general`, `typescript`, `python`, `convex`, `electron`), retire the name-based - `inferGuidelineCategory` inference in favor of the declared field, and add - `--category` selection to `tbd docs fork` (reusing the existing `--list --category` - metadata — no new map). +15. **Categories from doc frontmatter** (no `doc-packs.ts`, no detection function): + curate the `category` field on the bundled docs so each lands in exactly one + category (`general`, `typescript`, `python`, `convex`, `electron`), retire the + name-based `inferGuidelineCategory` inference in favor of the declared field, and + add `--category` selection to `tbd docs fork` (reusing the existing + `--list --category` metadata — no new map). Unit tests: category-based fork selection, and that every bundled doc resolves to - exactly one theme. + exactly one category. 16. **Setup integration**: self-documenting `--auto` summary naming the two choices — - *scope* (all standard guidelines, recommended, or a theme subset) and *visibility* - (hidden cache vs fork into `docs/tbd/`) — plus the pending-update count; removal of - the unused `--interactive` flag; the fork dir documented as a config customization - (`docs_cache.fork_dir`); the sync-taxonomy table added to `tbd-docs.md`. + *scope* (all standard guidelines, recommended, or a category subset) and + *visibility* (hidden cache vs fork into `docs/tbd/`) — plus the pending-update + count; removal of the unused `--interactive` flag; the fork dir documented as a + config customization (`docs_cache.fork_dir`); the sync-taxonomy table added to + `tbd-docs.md`. ### Phase 5: Docs and agent surface @@ -1257,7 +1259,7 @@ duplicated** — the contract is that those exact blocks appear in the named doc | `README.md` | Phase 0 | “Shortcuts, Guidelines, and Templates”: add a “Forkable: see them in your repo” paragraph with a `tbd docs fork --category=general` example. “Documentation” block: `tbd docs` is now an overview; the manual is `tbd docs show tbd-docs`. Per-kind `--add` lines annotated as aliases for `tbd docs add`. | | `packages/tbd/docs/shortcuts/system/skill-baseline.md` (injected agent skill) | Phase 5 | Add the fork/update/upstream rows to “User Request → Agent Action” (the rows in “Upstream-contribution playbook”); add `tbd docs list` / `tbd docs fork` to the “Documentation” command table; one-line “Forkable docs” note. Kept within the skill’s size budget; lands with the rest of the agent surface so routing is never half-wired. | | `packages/tbd/docs/install/claude-header.md` | none | **No change.** Its `allowed-tools: Bash(tbd:*)` already covers `tbd docs`. Stated here so the audit is explicit. | -| `packages/tbd/docs/shortcuts/standard/welcome-user.md` | Phase 5 | Add the two-axis onboarding offer after the status summary — *scope* (all guidelines vs a theme subset) and *visibility* (hidden cache vs forked into `docs/tbd/`) — plus a “make guidelines visible” row routing to `tbd docs fork --category=…`. | +| `packages/tbd/docs/shortcuts/standard/welcome-user.md` | Phase 5 | Add the two-axis onboarding offer after the status summary — *scope* (all guidelines vs a category subset) and *visibility* (hidden cache vs forked into `docs/tbd/`) — plus a “make guidelines visible” row routing to `tbd docs fork --category=…`. | | `packages/tbd/docs/shortcuts/standard/suggest-upstream-improvements.md` (**new**) | Phase 0 | The upstream-contribution playbook (per “Upstream-contribution playbook”). Pure docs. | | `packages/tbd/docs/references/docref-format.md` (**new**) | Phase 0 | Adopted from the #117 branch, marked adopted v0.1. First doc of the new `reference` kind. | | `packages/tbd/docs/references/docmap-format.md` (**new**) | Phase 0 | Authored fresh, minimal (docmap/0.1 inventory only) per the Design section; #117 cited as background. | @@ -1370,7 +1372,7 @@ tbd docs — managed documentation Guidelines are active from the cache. Fork them into docs/tbd/ to make them visible and customizable (same behavior — just explicit and git-tracked): - Scope: all standard guidelines (recommended), or a theme: + Scope: all standard guidelines (recommended), or a category: general, typescript, python, convex, electron Make visible: tbd docs fork --category=general --category= tbd docs fork --all (everything) @@ -1490,7 +1492,7 @@ Edit it in place — tbd now serves your copy everywhere it served the upstream ? 0 $ tbd docs fork --category=general --category=python --dry-run -[DRY-RUN] Would fork 11 docs into docs/tbd/ (themes: general, python) +[DRY-RUN] Would fork 11 docs into docs/tbd/ (categories: general, python) guideline general-eng-agent-principles guideline general-coding-rules [.. 7 more ..] @@ -1661,7 +1663,7 @@ pending-update report); setup never writes the fork dir: Docs: 37 available in cache (.tbd/docs/, gitignored); none forked into the repo. Guidelines are active from the cache. Fork them into docs/tbd/ to make them visible and customizable (same behavior — just explicit and git-tracked): - Scope: all standard guidelines (recommended), or a theme: + Scope: all standard guidelines (recommended), or a category: general, typescript, python, convex, electron Make visible: tbd docs fork --category=general --category= tbd docs fork --all (everything) From c182fbd6a546dee0f1d5e50e9f12d226a031c99b Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 12 Jun 2026 08:53:09 +0000 Subject: [PATCH 05/36] feat: Add standalone docref and docmap modules Add the two dependency-free, extraction-ready modules the forkable-docs plan (plan-2026-06-11-forkable-docs.md) builds on: - docref: the single URI-like address grammar for documents (internal:, local paths, URLs, and github:/gitlab:/git: with ref + path), plus normalization of github/gitlab blob and raw URLs to the canonical scheme. Parser, formatter, normalizer, and helpers with a spec-mirror test suite (23 tests). - docmap (docmap/0.1): a minimal document-inventory format (identity = type + name, location, presentation metadata) with create/parse/validate and query helpers. Unknown fields are preserved so producers can attach extension fields. 13 tests. Neither module imports tbd-internal code, per the standalone-module decision in the spec. Implements part of tbd-ljqx. https://claude.ai/code/session_01X8S12JzmmxEfLpYzgH8Y7E --- packages/tbd/src/docmap/docmap.ts | 135 ++++++++++++++++++ packages/tbd/src/docmap/index.ts | 22 +++ packages/tbd/src/docref/docref.ts | 226 ++++++++++++++++++++++++++++++ packages/tbd/src/docref/index.ts | 18 +++ packages/tbd/tests/docmap.test.ts | 116 +++++++++++++++ packages/tbd/tests/docref.test.ts | 148 +++++++++++++++++++ 6 files changed, 665 insertions(+) create mode 100644 packages/tbd/src/docmap/docmap.ts create mode 100644 packages/tbd/src/docmap/index.ts create mode 100644 packages/tbd/src/docref/docref.ts create mode 100644 packages/tbd/src/docref/index.ts create mode 100644 packages/tbd/tests/docmap.test.ts create mode 100644 packages/tbd/tests/docref.test.ts diff --git a/packages/tbd/src/docmap/docmap.ts b/packages/tbd/src/docmap/docmap.ts new file mode 100644 index 00000000..7e0e61af --- /dev/null +++ b/packages/tbd/src/docmap/docmap.ts @@ -0,0 +1,135 @@ +/** + * docmap — a minimal, machine-readable inventory of a collection of documents. + * + * A docmap is a "sitemap for docs": one entry per document, each with an identity + * (`type` + `name`, unique within the map), a location (`path` and/or a provenance + * `source` docref), and presentation metadata (`title`, `description`, `word_count`). + * It describes a collection; it says nothing about how the collection is assembled, + * fetched, or kept fresh. + * + * This is the docmap/0.1 format. The module is standalone and dependency-free (no + * tbd-internal imports) so it can move to its own package later. Consumers MUST + * ignore unknown fields, so producers (such as tbd) may attach extension fields — + * for example tbd's `state`/`stale` — without breaking other readers. + */ + +import { z } from 'zod'; + +/** Current docmap format version tag. */ +export const DOCMAP_VERSION = 'docmap/0.1' as const; + +/** + * One document in a docmap. Unknown fields are preserved (extension fields). + */ +export const DocMapEntrySchema = z + .object({ + /** Identity, unique within the map together with `type`. */ + name: z.string().min(1), + /** Identity, e.g. "guideline" | "shortcut" | "template" | "reference". */ + type: z.string().min(1), + /** Location within the collection (repo-relative or collection-relative). */ + path: z.string().optional(), + /** Provenance: a docref string for where the doc came from. */ + source: z.string().optional(), + title: z.string().optional(), + description: z.string().optional(), + word_count: z.number().int().nonnegative().optional(), + }) + .passthrough(); + +export type DocMapEntry = z.infer; + +/** + * A document inventory. `documents` entries are unique by (`type`, `name`). + */ +export const DocMapSchema = z + .object({ + docmap: z.string(), + name: z.string().optional(), + documents: z.array(DocMapEntrySchema), + }) + .passthrough(); + +export type DocMap = z.infer; + +/** Error thrown when an object is not a valid docmap. */ +export class DocMapError extends Error { + constructor(detail: string) { + super(`Invalid docmap: ${detail}`); + this.name = 'DocMapError'; + } +} + +/** Stable key for an entry's identity. */ +export function entryKey(entry: Pick): string { + return `${entry.type}:${entry.name}`; +} + +/** Assert that entries are unique by (type, name); throws {@link DocMapError}. */ +function assertUniqueIdentities(documents: DocMapEntry[]): void { + const seen = new Set(); + for (const entry of documents) { + const key = entryKey(entry); + if (seen.has(key)) { + throw new DocMapError(`duplicate entry identity ${JSON.stringify(key)}`); + } + seen.add(key); + } +} + +/** + * Build a docmap from entries. Validates each entry and identity uniqueness. + */ +export function createDocMap(documents: DocMapEntry[], options: { name?: string } = {}): DocMap { + const parsed = documents.map((d) => DocMapEntrySchema.parse(d)); + assertUniqueIdentities(parsed); + const map: DocMap = { docmap: DOCMAP_VERSION, documents: parsed }; + if (options.name !== undefined) { + map.name = options.name; + } + return map; +} + +/** + * Parse and validate an unknown value as a docmap (e.g. from YAML/JSON). + * Verifies the version tag and identity uniqueness. + */ +export function parseDocMap(value: unknown): DocMap { + const result = DocMapSchema.safeParse(value); + if (!result.success) { + throw new DocMapError(result.error.issues.map((i) => i.message).join('; ')); + } + const map = result.data; + if (!map.docmap.startsWith('docmap/')) { + throw new DocMapError(`unrecognized version tag ${JSON.stringify(map.docmap)}`); + } + assertUniqueIdentities(map.documents); + return map; +} + +/** Find an entry by name, optionally constrained to a type. */ +export function findEntry(map: DocMap, name: string, type?: string): DocMapEntry | undefined { + return map.documents.find((d) => d.name === name && (type === undefined || d.type === type)); +} + +/** Group entries by their `type`, preserving order within each group. */ +export function groupByType(map: DocMap): Map { + const groups = new Map(); + for (const entry of map.documents) { + const list = groups.get(entry.type); + if (list) { + list.push(entry); + } else { + groups.set(entry.type, [entry]); + } + } + return groups; +} + +/** Return a new docmap containing only entries of the given type. */ +export function filterByType(map: DocMap, type: string): DocMap { + const documents = map.documents.filter((d) => d.type === type); + return map.name !== undefined + ? { docmap: map.docmap, name: map.name, documents } + : { docmap: map.docmap, documents }; +} diff --git a/packages/tbd/src/docmap/index.ts b/packages/tbd/src/docmap/index.ts new file mode 100644 index 00000000..f0643f77 --- /dev/null +++ b/packages/tbd/src/docmap/index.ts @@ -0,0 +1,22 @@ +/** + * Public API for the standalone docmap module (docmap/0.1). + * + * A minimal document-inventory format with no tbd-internal dependencies, structured + * for extraction into its own package. tbd's list/inventory commands build a docmap + * and render it to text or `--json`. + */ + +export { + type DocMap, + type DocMapEntry, + DOCMAP_VERSION, + DocMapSchema, + DocMapEntrySchema, + DocMapError, + entryKey, + createDocMap, + parseDocMap, + findEntry, + groupByType, + filterByType, +} from './docmap.js'; diff --git a/packages/tbd/src/docref/docref.ts b/packages/tbd/src/docref/docref.ts new file mode 100644 index 00000000..6961a9c8 --- /dev/null +++ b/packages/tbd/src/docref/docref.ts @@ -0,0 +1,226 @@ +/** + * docref — a single-string, URI-like address for any document. + * + * This module is intentionally standalone and dependency-free (no tbd-internal + * imports) so it can move to its own package later. It is the one address syntax + * used everywhere a doc's source or location is named: config source strings, the + * fork manifest's `source` field, `tbd docs add` arguments, and `local_dirs` entries. + * + * Supported forms: + * internal:guidelines/python-rules.md bundled doc shipped inside tbd + * ./docs/general/ in-repo path (local) + * /abs/path/file.md absolute local path + * https://example.com/style.md plain URL + * github:owner/repo@ref//path/to/file.md git-hosted (also gitlab:, git:) + * + * Web URLs that point at a known git host are normalized to the `github:`/`gitlab:` + * form so there is one canonical address for a given file: + * https://github.com/o/r/blob/main/f.md -> github:o/r@main//f.md + * https://raw.githubusercontent.com/o/r/main/f.md -> github:o/r@main//f.md + */ + +/** Scheme of a git-hosted docref. */ +export type GitHost = 'github' | 'gitlab' | 'git'; + +/** A parsed document reference. */ +export type DocRef = + | { readonly kind: 'internal'; readonly path: string } + | { readonly kind: 'local'; readonly path: string } + | { readonly kind: 'url'; readonly url: string } + | { + readonly kind: 'git'; + readonly host: GitHost; + readonly owner: string; + readonly repo: string; + readonly ref?: string; + readonly path: string; + }; + +/** Error thrown when a string is not a valid docref. */ +export class DocRefError extends Error { + constructor( + public readonly input: string, + detail: string, + ) { + super(`Invalid docref ${JSON.stringify(input)}: ${detail}`); + this.name = 'DocRefError'; + } +} + +const GIT_SCHEMES: readonly GitHost[] = ['github', 'gitlab', 'git']; + +/** True for strings that address a local filesystem path rather than a scheme. */ +function looksLocal(input: string): boolean { + return ( + input.startsWith('./') || + input.startsWith('../') || + input.startsWith('/') || + input.startsWith('~/') + ); +} + +/** Strip a single leading `./` for tidy comparison; other forms are left as-is. */ +function tidyLocal(path: string): string { + return path.startsWith('./') ? path.slice(2) : path; +} + +/** + * Parse a `host:owner/repo[@ref]//path` body (everything after the scheme). + */ +function parseGitBody(host: GitHost, body: string, input: string): DocRef { + const sep = body.indexOf('//'); + if (sep === -1) { + throw new DocRefError(input, `git docref must contain "//" separating repo from path`); + } + const repoPart = body.slice(0, sep); + const path = body.slice(sep + 2); + if (!path) { + throw new DocRefError(input, 'git docref has an empty path'); + } + + const atIndex = repoPart.indexOf('@'); + const ownerRepo = atIndex === -1 ? repoPart : repoPart.slice(0, atIndex); + const ref = atIndex === -1 ? undefined : repoPart.slice(atIndex + 1) || undefined; + + const slash = ownerRepo.indexOf('/'); + if (slash === -1) { + throw new DocRefError(input, 'git docref must be "owner/repo"'); + } + const owner = ownerRepo.slice(0, slash); + const repo = ownerRepo.slice(slash + 1); + if (!owner || !repo) { + throw new DocRefError(input, 'git docref must be "owner/repo"'); + } + + return ref === undefined + ? { kind: 'git', host, owner, repo, path } + : { kind: 'git', host, owner, repo, ref, path }; +} + +/** + * If `url` points at a known git host's file view, return the equivalent git + * docref; otherwise return null (caller keeps it as a plain URL). + */ +function gitRefFromUrl(url: string): DocRef | null { + let parsed: URL; + try { + parsed = new URL(url); + } catch { + return null; + } + const segments = parsed.pathname.split('/').filter(Boolean); + + // https://github.com/{owner}/{repo}/blob/{ref}/{path...} + if (parsed.hostname === 'github.com' && segments[2] === 'blob' && segments.length >= 5) { + const [owner, repo, , ref, ...rest] = segments; + return { kind: 'git', host: 'github', owner: owner!, repo: repo!, ref, path: rest.join('/') }; + } + // https://raw.githubusercontent.com/{owner}/{repo}/{ref}/{path...} + if (parsed.hostname === 'raw.githubusercontent.com' && segments.length >= 4) { + const [owner, repo, ref, ...rest] = segments; + return { kind: 'git', host: 'github', owner: owner!, repo: repo!, ref, path: rest.join('/') }; + } + // https://gitlab.com/{owner}/{repo}/-/blob/{ref}/{path...} + if (parsed.hostname === 'gitlab.com' && segments[2] === '-' && segments[3] === 'blob') { + const [owner, repo, , , ref, ...rest] = segments; + if (owner && repo && ref && rest.length > 0) { + return { kind: 'git', host: 'gitlab', owner, repo, ref, path: rest.join('/') }; + } + } + return null; +} + +/** + * Parse a docref string into a structured {@link DocRef}. + * Throws {@link DocRefError} if the string is not a valid docref. + */ +export function parseDocRef(input: string): DocRef { + const raw = input.trim(); + if (!raw) { + throw new DocRefError(input, 'empty'); + } + + // Internal bundled docs. + if (raw.startsWith('internal:')) { + const path = raw.slice('internal:'.length); + if (!path) { + throw new DocRefError(input, 'internal docref has an empty path'); + } + return { kind: 'internal', path }; + } + + // Git-hosted schemes. + for (const host of GIT_SCHEMES) { + const prefix = `${host}:`; + if (raw.startsWith(prefix)) { + return parseGitBody(host, raw.slice(prefix.length), input); + } + } + + // Web URLs — normalize known git hosts, otherwise keep as a plain URL. + if (raw.startsWith('http://') || raw.startsWith('https://')) { + return gitRefFromUrl(raw) ?? { kind: 'url', url: raw }; + } + + // Local filesystem paths. + if (looksLocal(raw)) { + return { kind: 'local', path: raw }; + } + + // A scheme-less, non-URL string with no path markers is treated as a local + // relative path (e.g. "guidelines/python-rules.md"). A stray scheme is rejected. + if (/^[a-z][a-z0-9+.-]*:/i.test(raw)) { + throw new DocRefError(input, 'unknown scheme'); + } + return { kind: 'local', path: raw }; +} + +/** Parse a docref, returning null instead of throwing on invalid input. */ +export function tryParseDocRef(input: string): DocRef | null { + try { + return parseDocRef(input); + } catch { + return null; + } +} + +/** Serialize a {@link DocRef} back to its canonical string form. */ +export function formatDocRef(ref: DocRef): string { + switch (ref.kind) { + case 'internal': + return `internal:${ref.path}`; + case 'local': + return ref.path; + case 'url': + return ref.url; + case 'git': { + const refPart = ref.ref ? `@${ref.ref}` : ''; + return `${ref.host}:${ref.owner}/${ref.repo}${refPart}//${ref.path}`; + } + } +} + +/** + * Normalize a docref string to its canonical form (parse + re-format). + * Rationalizes git web URLs into the `github:`/`gitlab:` scheme. + */ +export function normalizeDocRef(input: string): string { + return formatDocRef(parseDocRef(input)); +} + +/** True if `input` parses as a valid docref. */ +export function isDocRef(input: string): boolean { + return tryParseDocRef(input) !== null; +} + +/** + * Whether two docrefs address the same document, ignoring a leading `./` on + * local paths. Useful for de-duping config entries. + */ +export function docRefsEqual(a: DocRef, b: DocRef): boolean { + if (a.kind !== b.kind) return false; + if (a.kind === 'local' && b.kind === 'local') { + return tidyLocal(a.path) === tidyLocal(b.path); + } + return formatDocRef(a) === formatDocRef(b); +} diff --git a/packages/tbd/src/docref/index.ts b/packages/tbd/src/docref/index.ts new file mode 100644 index 00000000..755e50cc --- /dev/null +++ b/packages/tbd/src/docref/index.ts @@ -0,0 +1,18 @@ +/** + * Public API for the standalone docref module. + * + * docref is the single address grammar for documents across tbd. It has no + * tbd-internal dependencies and is structured for extraction into its own package. + */ + +export { + type DocRef, + type GitHost, + DocRefError, + parseDocRef, + tryParseDocRef, + formatDocRef, + normalizeDocRef, + isDocRef, + docRefsEqual, +} from './docref.js'; diff --git a/packages/tbd/tests/docmap.test.ts b/packages/tbd/tests/docmap.test.ts new file mode 100644 index 00000000..19a71567 --- /dev/null +++ b/packages/tbd/tests/docmap.test.ts @@ -0,0 +1,116 @@ +/** + * Tests for the standalone docmap/0.1 inventory module. + */ + +import { describe, it, expect } from 'vitest'; +import { + DOCMAP_VERSION, + DocMapError, + createDocMap, + parseDocMap, + findEntry, + groupByType, + filterByType, + entryKey, +} from '../src/docmap/index.js'; + +const sample = [ + { + name: 'python-rules', + type: 'guideline', + path: 'docs/tbd/guidelines/python-rules.md', + source: 'internal:guidelines/python-rules.md', + title: 'Python Coding Rules', + description: 'Type hints, docstrings, exception handling', + word_count: 2400, + }, + { name: 'review-code', type: 'shortcut', source: 'internal:shortcuts/standard/review-code.md' }, + { name: 'tbd-docs', type: 'reference', source: 'internal:tbd-docs.md' }, +]; + +describe('createDocMap', () => { + it('builds a docmap with the version tag and entries', () => { + const map = createDocMap(sample, { name: 'tbd-docs' }); + expect(map.docmap).toBe(DOCMAP_VERSION); + expect(map.name).toBe('tbd-docs'); + expect(map.documents).toHaveLength(3); + }); + + it('omits name when not provided', () => { + const map = createDocMap(sample); + expect('name' in map).toBe(false); + }); + + it('preserves extension fields (e.g. tbd state)', () => { + const map = createDocMap([{ name: 'x', type: 'guideline', state: 'customized', stale: true }]); + expect(map.documents[0]).toMatchObject({ state: 'customized', stale: true }); + }); + + it('rejects duplicate (type, name) identities', () => { + expect(() => + createDocMap([ + { name: 'dup', type: 'guideline' }, + { name: 'dup', type: 'guideline' }, + ]), + ).toThrow(DocMapError); + }); + + it('allows the same name under different types', () => { + const map = createDocMap([ + { name: 'typescript', type: 'guideline' }, + { name: 'typescript', type: 'template' }, + ]); + expect(map.documents).toHaveLength(2); + }); +}); + +describe('parseDocMap', () => { + it('round-trips a created docmap', () => { + const map = createDocMap(sample, { name: 'tbd-docs' }); + expect(parseDocMap(map)).toEqual(map); + }); + + it('rejects a missing/wrong version tag', () => { + expect(() => parseDocMap({ documents: [] })).toThrow(DocMapError); + expect(() => parseDocMap({ docmap: 'sitemap/1', documents: [] })).toThrow(DocMapError); + }); + + it('rejects entries missing identity fields', () => { + expect(() => parseDocMap({ docmap: DOCMAP_VERSION, documents: [{ name: 'x' }] })).toThrow( + DocMapError, + ); + }); + + it('accepts and preserves unknown top-level and entry fields', () => { + const map = parseDocMap({ + docmap: DOCMAP_VERSION, + generated_by: 'tbd', + documents: [{ name: 'x', type: 'guideline', state: 'forked' }], + }); + expect(map.documents[0]).toMatchObject({ state: 'forked' }); + }); +}); + +describe('queries', () => { + const map = createDocMap(sample); + + it('finds entries by name and optional type', () => { + expect(findEntry(map, 'python-rules')?.type).toBe('guideline'); + expect(findEntry(map, 'python-rules', 'shortcut')).toBeUndefined(); + }); + + it('groups entries by type', () => { + const groups = groupByType(map); + expect([...groups.keys()]).toEqual(['guideline', 'shortcut', 'reference']); + }); + + it('filters to a single type and keeps it a valid docmap', () => { + const guidelines = filterByType(map, 'guideline'); + expect(guidelines.documents).toHaveLength(1); + expect(parseDocMap(guidelines)).toEqual(guidelines); + }); + + it('entryKey is stable and identity-based', () => { + expect(entryKey({ type: 'guideline', name: 'python-rules' })).toBe('guideline:python-rules'); + }); +}); diff --git a/packages/tbd/tests/docref.test.ts b/packages/tbd/tests/docref.test.ts new file mode 100644 index 00000000..7504f840 --- /dev/null +++ b/packages/tbd/tests/docref.test.ts @@ -0,0 +1,148 @@ +/** + * Tests for the standalone docref grammar module. + * + * Acts as a spec-mirror: every supported form and normalization is exercised here, + * so the module can be lifted to its own package with its behavior pinned. + */ + +import { describe, it, expect } from 'vitest'; +import { + type DocRef, + DocRefError, + parseDocRef, + tryParseDocRef, + formatDocRef, + normalizeDocRef, + isDocRef, + docRefsEqual, +} from '../src/docref/index.js'; + +describe('parseDocRef', () => { + it('parses internal bundled docs', () => { + expect(parseDocRef('internal:guidelines/python-rules.md')).toEqual({ + kind: 'internal', + path: 'guidelines/python-rules.md', + }); + }); + + it('parses local paths (./ , ../ , absolute, scheme-less)', () => { + expect(parseDocRef('./docs/general/')).toEqual({ kind: 'local', path: './docs/general/' }); + expect(parseDocRef('../shared/rules.md')).toEqual({ + kind: 'local', + path: '../shared/rules.md', + }); + expect(parseDocRef('/abs/path/file.md')).toEqual({ kind: 'local', path: '/abs/path/file.md' }); + expect(parseDocRef('guidelines/python-rules.md')).toEqual({ + kind: 'local', + path: 'guidelines/python-rules.md', + }); + }); + + it('parses plain URLs', () => { + expect(parseDocRef('https://example.com/style.md')).toEqual({ + kind: 'url', + url: 'https://example.com/style.md', + }); + }); + + it('parses github: scheme with a ref', () => { + expect(parseDocRef('github:acme/eng-docs@main//guidelines/style.md')).toEqual({ + kind: 'git', + host: 'github', + owner: 'acme', + repo: 'eng-docs', + ref: 'main', + path: 'guidelines/style.md', + }); + }); + + it('parses github: scheme without a ref', () => { + expect(parseDocRef('github:acme/eng-docs//guidelines/style.md')).toEqual({ + kind: 'git', + host: 'github', + owner: 'acme', + repo: 'eng-docs', + path: 'guidelines/style.md', + }); + }); + + it('parses gitlab: and git: schemes', () => { + expect(parseDocRef('gitlab:org/repo@v1.0//a/b.md').kind).toBe('git'); + expect(parseDocRef('git:org/repo@sha//a/b.md')).toMatchObject({ host: 'git', ref: 'sha' }); + }); + + it('trims surrounding whitespace', () => { + expect(parseDocRef(' internal:a.md ')).toEqual({ kind: 'internal', path: 'a.md' }); + }); + + it('rejects empty, scheme-only, and malformed git refs', () => { + expect(() => parseDocRef('')).toThrow(DocRefError); + expect(() => parseDocRef('internal:')).toThrow(DocRefError); + expect(() => parseDocRef('github:owner-only//path.md')).toThrow(DocRefError); + expect(() => parseDocRef('github:owner/repo/no-double-slash.md')).toThrow(DocRefError); + expect(() => parseDocRef('github:owner/repo@main//')).toThrow(DocRefError); + expect(() => parseDocRef('mailto:someone@example.com')).toThrow(DocRefError); + }); +}); + +describe('normalizeDocRef', () => { + it('normalizes a github blob URL to the github: scheme', () => { + expect(normalizeDocRef('https://github.com/o/r/blob/main/f.md')).toBe('github:o/r@main//f.md'); + }); + + it('normalizes a raw.githubusercontent.com URL to the github: scheme', () => { + expect(normalizeDocRef('https://raw.githubusercontent.com/o/r/main/dir/f.md')).toBe( + 'github:o/r@main//dir/f.md', + ); + }); + + it('normalizes a gitlab blob URL to the gitlab: scheme', () => { + expect(normalizeDocRef('https://gitlab.com/o/r/-/blob/main/f.md')).toBe( + 'gitlab:o/r@main//f.md', + ); + }); + + it('leaves non-git URLs and other forms unchanged', () => { + expect(normalizeDocRef('https://example.com/x.md')).toBe('https://example.com/x.md'); + expect(normalizeDocRef('internal:a/b.md')).toBe('internal:a/b.md'); + }); +}); + +describe('formatDocRef round-trips', () => { + const cases: string[] = [ + 'internal:guidelines/python-rules.md', + './docs/general/', + '/abs/file.md', + 'https://example.com/style.md', + 'github:acme/eng-docs@main//guidelines/style.md', + 'github:acme/eng-docs//guidelines/style.md', + 'gitlab:org/repo@v1.0//a/b.md', + ]; + it.each(cases)('parse->format is identity for %s', (input) => { + expect(formatDocRef(parseDocRef(input))).toBe(input); + }); +}); + +describe('helpers', () => { + it('isDocRef reflects validity', () => { + expect(isDocRef('internal:a.md')).toBe(true); + expect(isDocRef('mailto:x@y.com')).toBe(false); + }); + + it('tryParseDocRef returns null on invalid input', () => { + expect(tryParseDocRef('')).toBeNull(); + expect(tryParseDocRef('internal:a.md')).not.toBeNull(); + }); + + it('docRefsEqual ignores a leading ./ on local paths', () => { + const a = parseDocRef('./guidelines/x.md'); + const b = parseDocRef('guidelines/x.md'); + expect(docRefsEqual(a, b)).toBe(true); + }); + + it('docRefsEqual distinguishes different kinds', () => { + const a: DocRef = parseDocRef('internal:x.md'); + const b: DocRef = parseDocRef('https://e.com/x.md'); + expect(docRefsEqual(a, b)).toBe(false); + }); +}); From 6c122d67ff44ffbc0f5b0f7b6c25fd85b60ad144 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 12 Jun 2026 08:59:17 +0000 Subject: [PATCH 06/36] feat: Add fork-manifest module (state, hashing, base snapshots) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Add src/file/fork-manifest.ts — the committed fork state under .tbd/doc-forks/ (forks.yml manifest + base//.md snapshots) for forkable docs. - Pure functions: LF-normalized sha256 hashing, conflict-marker detection (requires all three standard markers so prose mentioning one isn't flagged), and computeForkStatus, a total function mapping base/file/cache hashes plus the conflicted flag to one of upstream/forked/customized/stale/conflicted/ local/missing/orphaned, with customized and stale as combinable modifiers. - Manifest helpers (find/upsert/remove) and a zod schema. - Filesystem I/O: read/write the manifest (yaml) and base snapshots. Full unit coverage including the table-driven state matrix (22 tests). Implements tbd-hgf3. https://claude.ai/code/session_01X8S12JzmmxEfLpYzgH8Y7E --- packages/tbd/src/file/fork-manifest.ts | 293 +++++++++++++++++++++++ packages/tbd/tests/fork-manifest.test.ts | 251 +++++++++++++++++++ 2 files changed, 544 insertions(+) create mode 100644 packages/tbd/src/file/fork-manifest.ts create mode 100644 packages/tbd/tests/fork-manifest.test.ts diff --git a/packages/tbd/src/file/fork-manifest.ts b/packages/tbd/src/file/fork-manifest.ts new file mode 100644 index 00000000..bbfca663 --- /dev/null +++ b/packages/tbd/src/file/fork-manifest.ts @@ -0,0 +1,293 @@ +/** + * Fork manifest and base snapshots for the forkable-docs workflow. + * + * All fork state lives under one committed directory, `.tbd/doc-forks/`: + * forks.yml — the manifest (provenance, hashes, revisions) + * base//.md — verbatim base snapshots (the merge bases) + * + * The base of every fork is a stored snapshot of the upstream content the fork + * was created from. Together with `base_hash`, this makes "customized?", + * "stale vs upstream?", and three-way merging cheap, exact, offline operations. + * + * The hashing and state computation here are pure functions (see + * {@link hashContent} and {@link computeForkStatus}); only the read/write helpers + * touch the filesystem. + */ + +import { createHash } from 'node:crypto'; +import { readFile, mkdir, rm } from 'node:fs/promises'; +import { dirname, join } from 'node:path'; + +import { writeFile } from 'atomically'; +import { parse as parseYaml } from 'yaml'; +import { z } from 'zod'; + +import { stringifyYaml } from '../utils/yaml-utils.js'; + +/** Directory (repo-relative under `.tbd/`) holding all fork state. */ +export const DOC_FORKS_DIR = '.tbd/doc-forks'; +/** Manifest filename within {@link DOC_FORKS_DIR}. */ +export const FORKS_FILE = 'forks.yml'; +/** Subdirectory within {@link DOC_FORKS_DIR} holding base snapshots. */ +export const BASE_SUBDIR = 'base'; + +/** Doc kinds that can be forked. */ +export const FORK_KINDS = ['guideline', 'shortcut', 'template', 'reference'] as const; +export type ForkKind = (typeof FORK_KINDS)[number]; + +// ============================================================================= +// Schema +// ============================================================================= + +export const ForkEntrySchema = z.object({ + /** Doc name (e.g. "python-rules"). */ + name: z.string().min(1), + /** Doc kind (guideline/shortcut/template/reference). */ + kind: z.string().min(1), + /** Repo-relative path of the forked file (e.g. "docs/tbd/guidelines/python-rules.md"). */ + path: z.string().min(1), + /** Provenance docref the fork was created from. */ + source: z.string().min(1), + /** sha256: of the LF-normalized base content. */ + base_hash: z.string().min(1), + /** tbd version when the base was last set. */ + tbd_version: z.string().optional(), + /** Upstream commit at base time (git-hosted sources only). */ + source_revision: z.string().optional(), + /** Exact/matching tag at base time, when one exists. */ + source_tag: z.string().optional(), + /** Set by `update --merge` when it leaves conflict markers; auto-clears. */ + conflicted: z.boolean().optional(), +}); + +export type ForkEntry = z.infer; + +export const ForkManifestSchema = z.object({ + forks: z.array(ForkEntrySchema).default([]), +}); + +export type ForkManifest = z.infer; + +/** An empty manifest (no forks). */ +export function emptyManifest(): ForkManifest { + return { forks: [] }; +} + +// ============================================================================= +// Hashing (pure) +// ============================================================================= + +/** Normalize line endings to LF so hashes are stable across platforms. */ +export function normalizeLineEndings(content: string): string { + return content.replace(/\r\n/g, '\n').replace(/\r/g, '\n'); +} + +/** sha256: hash of the LF-normalized content. */ +export function hashContent(content: string): string { + const hex = createHash('sha256').update(normalizeLineEndings(content), 'utf8').digest('hex'); + return `sha256:${hex}`; +} + +/** + * Whether `content` still contains git conflict markers. Requires all three + * standard markers so prose that merely discusses one marker is not flagged. + */ +export function hasConflictMarkers(content: string): boolean { + return /^<{7}/m.test(content) && /^={7}\s*$/m.test(content) && /^>{7}/m.test(content); +} + +// ============================================================================= +// State computation (pure) +// ============================================================================= + +export type DocState = + | 'upstream' + | 'forked' + | 'customized' + | 'stale' + | 'conflicted' + | 'local' + | 'missing' + | 'orphaned'; + +/** Inputs for {@link computeForkStatus}; all comparisons are by hash. */ +export interface ForkStatusInput { + /** Whether a manifest entry exists for this doc. */ + inManifest: boolean; + /** Whether the forked file is present on disk. */ + forkFileExists: boolean; + /** sha256: of the current forked file (if present). */ + forkHash?: string; + /** Recorded `base_hash` from the manifest entry. */ + baseHash?: string; + /** sha256: of the current upstream/cache content; undefined if the source is gone. */ + cacheHash?: string; + /** Manifest `conflicted` flag. */ + conflictedFlag?: boolean; + /** Whether conflict markers are present in the current file. */ + markersPresent?: boolean; +} + +/** + * Computed lifecycle status of a doc. `state` is the headline for display; the + * booleans are orthogonal modifiers (`customized` and `stale` can combine). + */ +export interface ForkStatus { + state: DocState; + customized: boolean; + stale: boolean; + conflicted: boolean; + orphaned: boolean; +} + +/** + * Compute a doc's fork status from hashes and flags. Pure and total — every + * combination of inputs maps to exactly one {@link ForkStatus}. + */ +export function computeForkStatus(input: ForkStatusInput): ForkStatus { + const none = { customized: false, stale: false, conflicted: false, orphaned: false }; + + if (!input.inManifest) { + // A file in the fork dir with no manifest entry is a hand-authored local doc; + // otherwise the doc is simply served from upstream via the cache. + return { state: input.forkFileExists ? 'local' : 'upstream', ...none }; + } + + if (!input.forkFileExists) { + return { state: 'missing', ...none }; + } + + const customized = input.forkHash !== input.baseHash; + const orphaned = input.cacheHash === undefined; + const stale = !orphaned && input.cacheHash !== input.baseHash; + const conflicted = Boolean(input.conflictedFlag && input.markersPresent); + + // Headline state for display. `customized` and `stale` can combine; the + // renderer composes "customized, stale" from state + the stale modifier. + let state: DocState; + if (conflicted) { + state = 'conflicted'; + } else if (orphaned) { + state = 'orphaned'; + } else if (customized) { + state = 'customized'; + } else if (stale) { + state = 'stale'; + } else { + state = 'forked'; + } + + return { state, customized, stale, conflicted, orphaned }; +} + +// ============================================================================= +// Manifest helpers (pure) +// ============================================================================= + +/** Find a fork entry by name, optionally constrained to a kind. */ +export function findFork( + manifest: ForkManifest, + name: string, + kind?: string, +): ForkEntry | undefined { + return manifest.forks.find((f) => f.name === name && (kind === undefined || f.kind === kind)); +} + +/** Return a new manifest with `entry` inserted or replaced (matched by kind+name). */ +export function upsertFork(manifest: ForkManifest, entry: ForkEntry): ForkManifest { + const forks = manifest.forks.filter((f) => !(f.name === entry.name && f.kind === entry.kind)); + forks.push(entry); + forks.sort((a, b) => a.kind.localeCompare(b.kind) || a.name.localeCompare(b.name)); + return { forks }; +} + +/** Return a new manifest with the matching entry removed. */ +export function removeFork(manifest: ForkManifest, name: string, kind?: string): ForkManifest { + return { + forks: manifest.forks.filter( + (f) => !(f.name === name && (kind === undefined || f.kind === kind)), + ), + }; +} + +// ============================================================================= +// Paths +// ============================================================================= + +export function forksDir(tbdRoot: string): string { + return join(tbdRoot, DOC_FORKS_DIR); +} + +export function forksFilePath(tbdRoot: string): string { + return join(forksDir(tbdRoot), FORKS_FILE); +} + +export function baseFilePath(tbdRoot: string, kind: string, name: string): string { + return join(forksDir(tbdRoot), BASE_SUBDIR, kind, `${name}.md`); +} + +// ============================================================================= +// Filesystem I/O +// ============================================================================= + +function isNotFound(err: unknown): boolean { + return (err as NodeJS.ErrnoException | undefined)?.code === 'ENOENT'; +} + +/** Read the fork manifest, returning an empty manifest if none exists. */ +export async function readForkManifest(tbdRoot: string): Promise { + try { + const content = await readFile(forksFilePath(tbdRoot), 'utf-8'); + const data = parseYaml(content) as unknown; + return ForkManifestSchema.parse(data ?? { forks: [] }); + } catch (err) { + if (isNotFound(err)) { + return emptyManifest(); + } + throw err; + } +} + +/** Write the fork manifest (creating `.tbd/doc-forks/` as needed). */ +export async function writeForkManifest(tbdRoot: string, manifest: ForkManifest): Promise { + await mkdir(forksDir(tbdRoot), { recursive: true }); + const yaml = stringifyYaml(manifest, { lineWidth: 0, sortMapEntries: false }); + await writeFile(forksFilePath(tbdRoot), yaml); +} + +/** Read a base snapshot's content, or null if it is absent. */ +export async function readBaseContent( + tbdRoot: string, + kind: string, + name: string, +): Promise { + try { + return await readFile(baseFilePath(tbdRoot, kind, name), 'utf-8'); + } catch (err) { + if (isNotFound(err)) { + return null; + } + throw err; + } +} + +/** Write a base snapshot verbatim (creating parent dirs as needed). */ +export async function writeBaseContent( + tbdRoot: string, + kind: string, + name: string, + content: string, +): Promise { + const path = baseFilePath(tbdRoot, kind, name); + await mkdir(dirname(path), { recursive: true }); + await writeFile(path, content); +} + +/** Remove a base snapshot if present. */ +export async function removeBaseContent( + tbdRoot: string, + kind: string, + name: string, +): Promise { + await rm(baseFilePath(tbdRoot, kind, name), { force: true }); +} diff --git a/packages/tbd/tests/fork-manifest.test.ts b/packages/tbd/tests/fork-manifest.test.ts new file mode 100644 index 00000000..2bc7b673 --- /dev/null +++ b/packages/tbd/tests/fork-manifest.test.ts @@ -0,0 +1,251 @@ +/** + * Tests for the fork manifest, base snapshots, hashing, and state computation. + * + * The state computation is covered as a table-driven matrix (base/file/cache + * hashes + conflicted flag -> state), per the spec's testing strategy. + */ + +import { describe, it, expect, beforeEach, afterEach } from 'vitest'; +import { mkdtemp, rm } from 'node:fs/promises'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; + +import { + type ForkEntry, + type DocState, + hashContent, + normalizeLineEndings, + hasConflictMarkers, + computeForkStatus, + findFork, + upsertFork, + removeFork, + emptyManifest, + readForkManifest, + writeForkManifest, + readBaseContent, + writeBaseContent, + removeBaseContent, +} from '../src/file/fork-manifest.js'; + +describe('hashContent', () => { + it('is stable across CRLF/CR/LF line endings', () => { + const lf = 'line one\nline two\n'; + const crlf = 'line one\r\nline two\r\n'; + const cr = 'line one\rline two\r'; + expect(hashContent(lf)).toBe(hashContent(crlf)); + expect(hashContent(lf)).toBe(hashContent(cr)); + }); + + it('produces a sha256: prefixed hex digest', () => { + expect(hashContent('x')).toMatch(/^sha256:[0-9a-f]{64}$/); + }); + + it('differs for different content', () => { + expect(hashContent('a')).not.toBe(hashContent('b')); + }); + + it('normalizeLineEndings collapses CRLF and CR to LF', () => { + expect(normalizeLineEndings('a\r\nb\rc')).toBe('a\nb\nc'); + }); +}); + +describe('hasConflictMarkers', () => { + it('detects a real three-marker conflict', () => { + const conflict = ['<<<<<<< ours', 'mine', '=======', 'theirs', '>>>>>>> upstream'].join('\n'); + expect(hasConflictMarkers(conflict)).toBe(true); + }); + + it('does not flag prose that merely mentions one marker', () => { + expect(hasConflictMarkers('Resolve the <<<<<<< marker by editing.')).toBe(false); + expect(hasConflictMarkers('A line of ======= in a table border.')).toBe(false); + }); +}); + +describe('computeForkStatus matrix', () => { + const BASE = hashContent('base'); + const EDITED = hashContent('edited'); + const MOVED = hashContent('moved'); + + interface Row { + label: string; + input: Parameters[0]; + expected: Partial> & { state: DocState }; + } + + const rows: Row[] = [ + { + label: 'not in manifest, no file -> upstream', + input: { inManifest: false, forkFileExists: false }, + expected: { state: 'upstream', customized: false, stale: false }, + }, + { + label: 'file present, no manifest entry -> local', + input: { inManifest: false, forkFileExists: true }, + expected: { state: 'local' }, + }, + { + label: 'manifest entry, file deleted -> missing', + input: { inManifest: true, forkFileExists: false, baseHash: BASE }, + expected: { state: 'missing' }, + }, + { + label: 'file == base, cache == base -> forked', + input: { + inManifest: true, + forkFileExists: true, + forkHash: BASE, + baseHash: BASE, + cacheHash: BASE, + }, + expected: { state: 'forked', customized: false, stale: false }, + }, + { + label: 'file != base, cache == base -> customized (not stale)', + input: { + inManifest: true, + forkFileExists: true, + forkHash: EDITED, + baseHash: BASE, + cacheHash: BASE, + }, + expected: { state: 'customized', customized: true, stale: false }, + }, + { + label: 'file == base, cache != base -> stale (unmodified)', + input: { + inManifest: true, + forkFileExists: true, + forkHash: BASE, + baseHash: BASE, + cacheHash: MOVED, + }, + expected: { state: 'stale', customized: false, stale: true }, + }, + { + label: 'file != base, cache != base -> customized + stale', + input: { + inManifest: true, + forkFileExists: true, + forkHash: EDITED, + baseHash: BASE, + cacheHash: MOVED, + }, + expected: { state: 'customized', customized: true, stale: true }, + }, + { + label: 'cache absent (source gone) -> orphaned', + input: { + inManifest: true, + forkFileExists: true, + forkHash: BASE, + baseHash: BASE, + cacheHash: undefined, + }, + expected: { state: 'orphaned', orphaned: true, stale: false }, + }, + { + label: 'conflicted flag + markers present -> conflicted', + input: { + inManifest: true, + forkFileExists: true, + forkHash: EDITED, + baseHash: BASE, + cacheHash: MOVED, + conflictedFlag: true, + markersPresent: true, + }, + expected: { state: 'conflicted', conflicted: true }, + }, + { + label: 'conflicted flag but markers resolved -> not conflicted (customized)', + input: { + inManifest: true, + forkFileExists: true, + forkHash: EDITED, + baseHash: BASE, + cacheHash: MOVED, + conflictedFlag: true, + markersPresent: false, + }, + expected: { state: 'customized', conflicted: false }, + }, + ]; + + it.each(rows)('$label', ({ input, expected }) => { + expect(computeForkStatus(input)).toMatchObject(expected); + }); +}); + +describe('manifest helpers', () => { + const entry = (name: string, kind = 'guideline'): ForkEntry => ({ + name, + kind, + path: `docs/tbd/${kind}s/${name}.md`, + source: `internal:${kind}s/${name}.md`, + base_hash: hashContent(name), + }); + + it('findFork matches by name and optional kind', () => { + const m = { forks: [entry('a'), entry('b', 'shortcut')] }; + expect(findFork(m, 'a')?.kind).toBe('guideline'); + expect(findFork(m, 'b', 'guideline')).toBeUndefined(); + expect(findFork(m, 'b', 'shortcut')?.name).toBe('b'); + }); + + it('upsertFork inserts, replaces by kind+name, and keeps sorted order', () => { + let m = emptyManifest(); + m = upsertFork(m, entry('zeta')); + m = upsertFork(m, entry('alpha')); + expect(m.forks.map((f) => f.name)).toEqual(['alpha', 'zeta']); + + const replaced = { ...entry('alpha'), base_hash: 'sha256:changed' }; + m = upsertFork(m, replaced); + expect(m.forks).toHaveLength(2); + expect(findFork(m, 'alpha')?.base_hash).toBe('sha256:changed'); + }); + + it('removeFork drops the matching entry', () => { + const m = { forks: [entry('a'), entry('b')] }; + expect(removeFork(m, 'a').forks.map((f) => f.name)).toEqual(['b']); + }); +}); + +describe('filesystem round-trip', () => { + let dir: string; + beforeEach(async () => { + dir = await mkdtemp(join(tmpdir(), 'tbd-fork-manifest-')); + }); + afterEach(async () => { + await rm(dir, { recursive: true, force: true }); + }); + + it('reads an empty manifest when none exists', async () => { + expect(await readForkManifest(dir)).toEqual({ forks: [] }); + }); + + it('round-trips a manifest through write/read', async () => { + const manifest = { + forks: [ + { + name: 'python-rules', + kind: 'guideline', + path: 'docs/tbd/guidelines/python-rules.md', + source: 'internal:guidelines/python-rules.md', + base_hash: hashContent('python base'), + tbd_version: '0.2.3', + }, + ], + }; + await writeForkManifest(dir, manifest); + expect(await readForkManifest(dir)).toEqual(manifest); + }); + + it('round-trips base content verbatim and removes it', async () => { + const content = '# Python Rules\n\nbase content\n'; + await writeBaseContent(dir, 'guideline', 'python-rules', content); + expect(await readBaseContent(dir, 'guideline', 'python-rules')).toBe(content); + await removeBaseContent(dir, 'guideline', 'python-rules'); + expect(await readBaseContent(dir, 'guideline', 'python-rules')).toBeNull(); + }); +}); From d74958ffd83e7be6d778877780d493a7a0056f61 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 12 Jun 2026 09:07:10 +0000 Subject: [PATCH 07/36] feat: Add fork operations (forkDoc/unforkDoc/forkStatusFor) Add src/file/doc-fork.ts: the fork/unfork/status operations on top of the fork-manifest module, using the default docs/tbd/ fork dir laid out as //.md. - forkDoc: write the forked file + base snapshot + manifest entry; refuse to overwrite a target that exists and is not an unmodified fork (unless --force); re-forking an unmodified fork refreshes it and advances the base. - unforkDoc: remove file + base + entry; refuse to discard local customizations unless forced; clean up a missing-file entry without complaint. - forkStatusFor: compute live state by reading the forked file and base and comparing to current upstream content (incl. out-of-band deletion -> missing and source-gone -> orphaned). 9 tests. CLI wiring follows. Part of tbd-q04x. Note: this slice uses the default docs/tbd/ dir; configurable fork_dir and the f05 format bump are deferred to a finalization step (see tbd-z1b5). https://claude.ai/code/session_01X8S12JzmmxEfLpYzgH8Y7E --- packages/tbd/src/file/doc-fork.ts | 250 ++++++++++++++++++++++++++++ packages/tbd/tests/doc-fork.test.ts | 218 ++++++++++++++++++++++++ 2 files changed, 468 insertions(+) create mode 100644 packages/tbd/src/file/doc-fork.ts create mode 100644 packages/tbd/tests/doc-fork.test.ts diff --git a/packages/tbd/src/file/doc-fork.ts b/packages/tbd/src/file/doc-fork.ts new file mode 100644 index 00000000..19cac250 --- /dev/null +++ b/packages/tbd/src/file/doc-fork.ts @@ -0,0 +1,250 @@ +/** + * Fork operations: copy a managed doc into a visible, git-tracked folder, return + * it to upstream, and report fork state. + * + * Forked files live outside `.tbd/` in the fork dir (default `docs/tbd/`), laid out + * as `//.md`. Provenance and the merge base live in the + * committed manifest under `.tbd/doc-forks/` (see fork-manifest.ts) so the forked + * files themselves stay clean, diffable, and forkable. + * + * These operations take the upstream content and current manifest as inputs and do + * the filesystem writes; resolving which doc/source to fork is the caller's job. + */ + +import { readFile, rm, mkdir } from 'node:fs/promises'; +import { dirname, join, relative } from 'node:path'; + +import { writeFile } from 'atomically'; + +import { + type ForkEntry, + type ForkKind, + type ForkManifest, + type ForkStatus, + computeForkStatus, + findFork, + hashContent, + hasConflictMarkers, + readBaseContent, + removeBaseContent, + upsertFork, + removeFork, + writeBaseContent, +} from './fork-manifest.js'; + +/** Default fork directory, relative to the repo/tbd root. */ +export const DEFAULT_FORK_DIR = 'docs/tbd'; + +/** Map a doc kind to its plural directory name within the fork dir. */ +export const KIND_DIR: Record = { + guideline: 'guidelines', + shortcut: 'shortcuts', + template: 'templates', + reference: 'references', +}; + +/** Absolute path of a forked file. */ +export function forkFilePath( + tbdRoot: string, + forkDir: string, + kind: ForkKind, + name: string, +): string { + return join(tbdRoot, forkDir, KIND_DIR[kind], `${name}.md`); +} + +/** Repo-relative path recorded in the manifest (always POSIX-style forward slashes). */ +export function forkRelPath(forkDir: string, kind: ForkKind, name: string): string { + return `${forkDir}/${KIND_DIR[kind]}/${name}.md`; +} + +async function pathExists(path: string): Promise { + try { + await readFile(path); + return true; + } catch { + return false; + } +} + +/** Error raised when a fork/unfork would lose user content; carries a reason code. */ +export class ForkConflictError extends Error { + constructor( + public readonly code: 'overwrite' | 'customized' | 'not-forked', + message: string, + ) { + super(message); + this.name = 'ForkConflictError'; + } +} + +export interface ForkDocParams { + tbdRoot: string; + forkDir: string; + manifest: ForkManifest; + kind: ForkKind; + name: string; + /** Provenance docref (e.g. "internal:guidelines/python-rules.md"). */ + source: string; + /** Upstream content to fork (becomes both the file and the base snapshot). */ + content: string; + tbdVersion?: string; + force?: boolean; +} + +export interface ForkDocResult { + manifest: ForkManifest; + relPath: string; + action: 'created' | 'refreshed'; +} + +/** + * Fork a doc into the fork dir, recording its base snapshot and manifest entry. + * + * Refuses to overwrite a target that exists and is not an unmodified fork (e.g. a + * pre-existing user file or a customized fork), unless `force` is set. Re-forking an + * unmodified fork refreshes it to the supplied upstream content and advances the base. + */ +export async function forkDoc(params: ForkDocParams): Promise { + const { tbdRoot, forkDir, kind, name, source, content, force } = params; + const absPath = forkFilePath(tbdRoot, forkDir, kind, name); + const relPath = forkRelPath(forkDir, kind, name); + const existingEntry = findFork(params.manifest, name, kind); + + let action: ForkDocResult['action'] = 'created'; + + if (await pathExists(absPath)) { + const current = await readFile(absPath, 'utf-8'); + const isUnmodifiedFork = hashContent(current) === existingEntry?.base_hash; + if (isUnmodifiedFork) { + action = 'refreshed'; + } else if (!force) { + throw new ForkConflictError( + 'overwrite', + `${relPath} already exists and is not an unmodified fork`, + ); + } + } + + await mkdir(dirname(absPath), { recursive: true }); + await writeFile(absPath, content); + await writeBaseContent(tbdRoot, kind, name, content); + + const entry: ForkEntry = { + name, + kind, + path: relPath, + source, + base_hash: hashContent(content), + ...(params.tbdVersion ? { tbd_version: params.tbdVersion } : {}), + }; + + return { manifest: upsertFork(params.manifest, entry), relPath, action }; +} + +export interface UnforkDocParams { + tbdRoot: string; + forkDir: string; + manifest: ForkManifest; + name: string; + kind?: ForkKind; + force?: boolean; +} + +export interface UnforkDocResult { + manifest: ForkManifest; + relPath: string; + /** True when the forked file was deleted (false when it was already missing). */ + fileRemoved: boolean; +} + +/** + * Remove a fork and fall back to upstream. Refuses to discard local customizations + * (file differs from its base) unless `force` is set. Cleans up a `missing` entry + * (file already deleted) without complaint. + */ +export async function unforkDoc(params: UnforkDocParams): Promise { + const { tbdRoot, forkDir, manifest, name, kind, force } = params; + const entry = findFork(manifest, name, kind); + if (!entry) { + throw new ForkConflictError('not-forked', `${name} is not a forked doc`); + } + const entryKind = entry.kind as ForkKind; + const absPath = forkFilePath(tbdRoot, forkDir, entryKind, name); + const relPath = entry.path; + + let fileRemoved = false; + if (await pathExists(absPath)) { + const current = await readFile(absPath, 'utf-8'); + if (hashContent(current) !== entry.base_hash && !force) { + throw new ForkConflictError( + 'customized', + `${name} has local customizations (differs from its base)`, + ); + } + await rm(absPath, { force: true }); + fileRemoved = true; + } + + await removeBaseContent(tbdRoot, entryKind, name); + return { manifest: removeFork(manifest, name, entryKind), relPath, fileRemoved }; +} + +/** + * Compute the live {@link ForkStatus} of a manifest entry by reading its forked + * file and base, and comparing against the current upstream/cache content. + * + * @param cacheContent current upstream content, or null/undefined if the source is + * gone from the cache (orphaned). + */ +export async function forkStatusFor( + tbdRoot: string, + forkDir: string, + entry: ForkEntry, + cacheContent: string | null | undefined, +): Promise { + const kind = entry.kind as ForkKind; + const absPath = forkFilePath(tbdRoot, forkDir, kind, entry.name); + let forkContent: string | null = null; + try { + forkContent = await readFile(absPath, 'utf-8'); + } catch { + forkContent = null; + } + + return computeForkStatus({ + inManifest: true, + forkFileExists: forkContent !== null, + forkHash: forkContent !== null ? hashContent(forkContent) : undefined, + baseHash: entry.base_hash, + cacheHash: cacheContent == null ? undefined : hashContent(cacheContent), + conflictedFlag: entry.conflicted, + markersPresent: forkContent !== null ? hasConflictMarkers(forkContent) : false, + }); +} + +/** Read the forked file content for an entry, or null if it is missing. */ +export async function readForkFile( + tbdRoot: string, + forkDir: string, + entry: ForkEntry, +): Promise { + try { + return await readFile( + forkFilePath(tbdRoot, forkDir, entry.kind as ForkKind, entry.name), + 'utf-8', + ); + } catch { + return null; + } +} + +/** Read the stored base snapshot for an entry, or null if it is missing. */ +export async function readForkBase(tbdRoot: string, entry: ForkEntry): Promise { + return readBaseContent(tbdRoot, entry.kind, entry.name); +} + +/** Compute the repo-relative path for a fork dir given an absolute tbd root. */ +export function relativeForkDir(tbdRoot: string, absForkDir: string): string { + return relative(tbdRoot, absForkDir); +} diff --git a/packages/tbd/tests/doc-fork.test.ts b/packages/tbd/tests/doc-fork.test.ts new file mode 100644 index 00000000..b111a8bd --- /dev/null +++ b/packages/tbd/tests/doc-fork.test.ts @@ -0,0 +1,218 @@ +/** + * Tests for fork operations (forkDoc / unforkDoc / forkStatusFor) against a temp dir. + */ + +import { describe, it, expect, beforeEach, afterEach } from 'vitest'; +import { mkdtemp, rm, readFile, writeFile } from 'node:fs/promises'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; + +import { + forkDoc, + unforkDoc, + forkStatusFor, + forkFilePath, + forkRelPath, + ForkConflictError, + DEFAULT_FORK_DIR, +} from '../src/file/doc-fork.js'; +import { emptyManifest, findFork, readBaseContent } from '../src/file/fork-manifest.js'; + +const FORK_DIR = DEFAULT_FORK_DIR; +const UPSTREAM = '# Python Rules\n\nUpstream content.\n'; + +describe('forkDoc', () => { + let root: string; + beforeEach(async () => { + root = await mkdtemp(join(tmpdir(), 'tbd-doc-fork-')); + }); + afterEach(async () => { + await rm(root, { recursive: true, force: true }); + }); + + async function fork(content = UPSTREAM, force = false) { + return forkDoc({ + tbdRoot: root, + forkDir: FORK_DIR, + manifest: emptyManifest(), + kind: 'guideline', + name: 'python-rules', + source: 'internal:guidelines/python-rules.md', + content, + force, + }); + } + + it('writes the forked file, base snapshot, and manifest entry', async () => { + const result = await fork(); + expect(result.action).toBe('created'); + expect(result.relPath).toBe('docs/tbd/guidelines/python-rules.md'); + + const fileContent = await readFile( + forkFilePath(root, FORK_DIR, 'guideline', 'python-rules'), + 'utf-8', + ); + expect(fileContent).toBe(UPSTREAM); + expect(await readBaseContent(root, 'guideline', 'python-rules')).toBe(UPSTREAM); + + const entry = findFork(result.manifest, 'python-rules'); + expect(entry).toMatchObject({ + name: 'python-rules', + kind: 'guideline', + path: 'docs/tbd/guidelines/python-rules.md', + source: 'internal:guidelines/python-rules.md', + }); + }); + + it('refuses to overwrite a pre-existing non-fork file', async () => { + const abs = forkFilePath(root, FORK_DIR, 'guideline', 'python-rules'); + await import('node:fs/promises').then((fs) => + fs.mkdir(join(root, FORK_DIR, 'guidelines'), { recursive: true }), + ); + await writeFile(abs, 'pre-existing user content\n'); + + await expect(fork()).rejects.toThrow(ForkConflictError); + // --force overwrites. + const forced = await fork(UPSTREAM, true); + expect(forced.action).toBe('created'); + expect(await readFile(abs, 'utf-8')).toBe(UPSTREAM); + }); + + it('refreshes an unmodified fork to new upstream and advances the base', async () => { + const first = await fork(); + const NEW_UPSTREAM = '# Python Rules\n\nNew upstream.\n'; + const refreshed = await forkDoc({ + tbdRoot: root, + forkDir: FORK_DIR, + manifest: first.manifest, + kind: 'guideline', + name: 'python-rules', + source: 'internal:guidelines/python-rules.md', + content: NEW_UPSTREAM, + }); + expect(refreshed.action).toBe('refreshed'); + expect(await readBaseContent(root, 'guideline', 'python-rules')).toBe(NEW_UPSTREAM); + }); +}); + +describe('forkStatusFor', () => { + let root: string; + beforeEach(async () => { + root = await mkdtemp(join(tmpdir(), 'tbd-doc-fork-status-')); + }); + afterEach(async () => { + await rm(root, { recursive: true, force: true }); + }); + + it('reports forked / customized / stale / missing correctly', async () => { + const { manifest } = await forkDoc({ + tbdRoot: root, + forkDir: FORK_DIR, + manifest: emptyManifest(), + kind: 'guideline', + name: 'python-rules', + source: 'internal:guidelines/python-rules.md', + content: UPSTREAM, + }); + const entry = findFork(manifest, 'python-rules')!; + const abs = forkFilePath(root, FORK_DIR, 'guideline', 'python-rules'); + + // Unmodified, upstream unchanged -> forked. + expect((await forkStatusFor(root, FORK_DIR, entry, UPSTREAM)).state).toBe('forked'); + + // Upstream moved, file unmodified -> stale. + expect((await forkStatusFor(root, FORK_DIR, entry, UPSTREAM + 'more\n')).state).toBe('stale'); + + // Edit the file -> customized. + await writeFile(abs, UPSTREAM + 'my edit\n'); + const customized = await forkStatusFor(root, FORK_DIR, entry, UPSTREAM); + expect(customized.state).toBe('customized'); + expect(customized.customized).toBe(true); + + // Source gone from cache -> orphaned. + expect((await forkStatusFor(root, FORK_DIR, entry, null)).state).toBe('orphaned'); + + // Delete the file out-of-band -> missing. + await rm(abs, { force: true }); + expect((await forkStatusFor(root, FORK_DIR, entry, UPSTREAM)).state).toBe('missing'); + }); +}); + +describe('unforkDoc', () => { + let root: string; + beforeEach(async () => { + root = await mkdtemp(join(tmpdir(), 'tbd-doc-unfork-')); + }); + afterEach(async () => { + await rm(root, { recursive: true, force: true }); + }); + + async function setup() { + return forkDoc({ + tbdRoot: root, + forkDir: FORK_DIR, + manifest: emptyManifest(), + kind: 'guideline', + name: 'python-rules', + source: 'internal:guidelines/python-rules.md', + content: UPSTREAM, + }); + } + + it('removes file, base, and entry for an unmodified fork', async () => { + const { manifest } = await setup(); + const result = await unforkDoc({ + tbdRoot: root, + forkDir: FORK_DIR, + manifest, + name: 'python-rules', + }); + expect(result.fileRemoved).toBe(true); + expect(findFork(result.manifest, 'python-rules')).toBeUndefined(); + expect(await readBaseContent(root, 'guideline', 'python-rules')).toBeNull(); + }); + + it('refuses to unfork a customized doc unless forced', async () => { + const { manifest } = await setup(); + const abs = forkFilePath(root, FORK_DIR, 'guideline', 'python-rules'); + await writeFile(abs, UPSTREAM + 'edits\n'); + + await expect( + unforkDoc({ tbdRoot: root, forkDir: FORK_DIR, manifest, name: 'python-rules' }), + ).rejects.toThrow(ForkConflictError); + + const forced = await unforkDoc({ + tbdRoot: root, + forkDir: FORK_DIR, + manifest, + name: 'python-rules', + force: true, + }); + expect(forced.fileRemoved).toBe(true); + }); + + it('cleans up a missing-file entry without complaint', async () => { + const { manifest } = await setup(); + await rm(forkFilePath(root, FORK_DIR, 'guideline', 'python-rules'), { force: true }); + const result = await unforkDoc({ + tbdRoot: root, + forkDir: FORK_DIR, + manifest, + name: 'python-rules', + }); + expect(result.fileRemoved).toBe(false); + expect(findFork(result.manifest, 'python-rules')).toBeUndefined(); + }); + + it('errors when the doc is not forked', async () => { + await expect( + unforkDoc({ tbdRoot: root, forkDir: FORK_DIR, manifest: emptyManifest(), name: 'nope' }), + ).rejects.toThrow(ForkConflictError); + }); + + it('forkRelPath uses plural kind dirs', () => { + expect(forkRelPath(FORK_DIR, 'shortcut', 'review-code')).toBe( + 'docs/tbd/shortcuts/review-code.md', + ); + }); +}); From b462dd9fff1f8cd5c23578b8fab85a972b4eb6de Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 12 Jun 2026 09:24:04 +0000 Subject: [PATCH 08/36] feat: Add tbd docs fork/unfork/status commands + serving precedence Wire the forkable-docs CLI as subcommands of `tbd docs`. The existing manual viewer stays the default action, so `tbd docs`, `--list`, `--all`, and `` are unchanged (verified against the existing goldens). - tbd docs fork [names...] [--kind] [--all] [--force] [--dry-run]: resolve docs from the pristine cache and copy them into docs/tbd//, recording base snapshots + manifest; overwrite refusal; cross-kind ambiguity needs --kind. - tbd docs unfork [names...] [--all] [--force]: remove forks; refuse to discard customizations without --force; clean up missing-file entries. - tbd docs status [--json]: per-doc state table, docmap --json, summary line. - Serving precedence: prepend the fork dir to guidelines/shortcut/template lookup paths so forked docs shadow the cache (missing dirs are skipped, so repos with no forks behave identically). Adds a 12-step fork-lifecycle e2e tryscript (fork -> status -> unfork -> out-of-band deletion -> upstream fallback -> missing). A parent/child --all collision is handled by reading merged options. Implements tbd-q04x, tbd-d31n, tbd-h7ft; partial tbd-i49m (status). Bare `tbd docs` stays the manual viewer for now; the overview reorg + f05 are deferred (tbd-z1b5). https://claude.ai/code/session_01X8S12JzmmxEfLpYzgH8Y7E --- packages/tbd/src/cli/commands/docs-fork.ts | 387 ++++++++++++++++++ packages/tbd/src/cli/commands/docs.ts | 5 + packages/tbd/src/lib/paths.ts | 39 +- packages/tbd/tests/cli-docs-fork.tryscript.md | 157 +++++++ 4 files changed, 582 insertions(+), 6 deletions(-) create mode 100644 packages/tbd/src/cli/commands/docs-fork.ts create mode 100644 packages/tbd/tests/cli-docs-fork.tryscript.md diff --git a/packages/tbd/src/cli/commands/docs-fork.ts b/packages/tbd/src/cli/commands/docs-fork.ts new file mode 100644 index 00000000..cce8005d --- /dev/null +++ b/packages/tbd/src/cli/commands/docs-fork.ts @@ -0,0 +1,387 @@ +/** + * `tbd docs fork` / `unfork` / `status` — the forkable-docs command handlers. + * + * These are added as subcommands of `tbd docs`. Resolution uses the gitignored + * cache (the pristine upstream) so forking copies upstream content into the visible + * fork dir; serving precedence (fork dir shadows cache) is handled by the lookup + * paths in paths.ts. + */ + +import type { Command } from 'commander'; +import { join, relative, sep } from 'node:path'; + +import { BaseCommand } from '../lib/base-command.js'; +import { requireInit } from '../lib/errors.js'; +import { CLIError } from '../lib/errors.js'; +import { VERSION } from '../lib/version.js'; +import { readConfig } from '../../file/config.js'; +import { DocCache } from '../../file/doc-cache.js'; +import { + CACHE_GUIDELINES_PATHS, + CACHE_SHORTCUT_PATHS, + CACHE_TEMPLATE_PATHS, + FORK_DIR, + TBD_DOCS_DIR, +} from '../../lib/paths.js'; +import { + type ForkEntry, + type ForkKind, + readForkManifest, + writeForkManifest, +} from '../../file/fork-manifest.js'; +import { forkDoc, unforkDoc, forkStatusFor, ForkConflictError } from '../../file/doc-fork.js'; +import { createDocMap, type DocMapEntry } from '../../docmap/index.js'; + +/** Kinds that can be resolved from the cache and forked today. */ +const RESOLVABLE_KINDS: ForkKind[] = ['guideline', 'shortcut', 'template']; + +const KIND_CACHE_PATHS: Record = { + guideline: CACHE_GUIDELINES_PATHS, + shortcut: CACHE_SHORTCUT_PATHS, + template: CACHE_TEMPLATE_PATHS, +}; + +interface ResolvedDoc { + kind: ForkKind; + name: string; + source: string; + content: string; +} + +/** Build a cache over a kind's cache-only paths (pristine upstream). */ +async function buildKindCache(kind: ForkKind, tbdRoot: string): Promise { + const cache = new DocCache(KIND_CACHE_PATHS[kind] ?? [], tbdRoot); + await cache.load({ quiet: true }); + return cache; +} + +/** Derive the provenance docref for a cached doc from config, defaulting to internal:. */ +function sourceDocRef( + tbdRoot: string, + files: Record | undefined, + docPath: string, +): string { + const cacheRoot = join(tbdRoot, TBD_DOCS_DIR); + const rel = relative(cacheRoot, docPath).split(sep).join('/'); + return files?.[rel] ?? `internal:${rel}`; +} + +interface ForkOptions { + kind?: string; + all?: boolean; + force?: boolean; + dryRun?: boolean; + json?: boolean; +} + +class DocsForkHandler extends BaseCommand { + async run(names: string[], options: ForkOptions): Promise { + await this.execute(async () => { + const tbdRoot = await requireInit(); + const config = await readConfig(tbdRoot); + const files = config.docs_cache?.files; + + const targets = await this.resolveTargets(tbdRoot, files, names, options); + if (targets.length === 0) { + throw new CLIError( + 'No matching docs to fork. Run `tbd docs status` or `tbd guidelines --list`.', + ); + } + + if (options.dryRun) { + this.output.dryRun(`Would fork ${targets.length} doc(s) into ${FORK_DIR}/`, { + docs: targets.map((t) => `${t.kind}/${t.name}`), + }); + if (!this.ctx.json) { + for (const t of targets) { + console.log(` ${t.kind.padEnd(11)} ${t.name}`); + } + console.log('No files written. Re-run without --dry-run to apply.'); + } + return; + } + + let manifest = await readForkManifest(tbdRoot); + const forked: string[] = []; + for (const t of targets) { + const result = await forkDoc({ + tbdRoot, + forkDir: FORK_DIR, + manifest, + kind: t.kind, + name: t.name, + source: t.source, + content: t.content, + tbdVersion: VERSION, + force: options.force, + }); + manifest = result.manifest; + forked.push(result.relPath); + if (!this.ctx.json) { + this.output.success(`Forked ${t.name} → ${result.relPath}`); + } + } + await writeForkManifest(tbdRoot, manifest); + + if (this.ctx.json) { + this.output.data({ forked }); + } else { + console.log(''); + console.log('Edit in place — tbd now serves your copy wherever it served upstream.'); + } + }, 'Failed to fork'); + } + + private async resolveTargets( + tbdRoot: string, + files: Record | undefined, + names: string[], + options: ForkOptions, + ): Promise { + const kinds = options.kind ? [options.kind as ForkKind] : RESOLVABLE_KINDS; + + if (options.all) { + const targets: ResolvedDoc[] = []; + for (const kind of kinds) { + const cache = await buildKindCache(kind, tbdRoot); + for (const doc of cache.list()) { + // Skip tbd-internal system shortcuts (skill-baseline etc.). + if (kind === 'shortcut' && doc.sourceDir.endsWith('system')) continue; + targets.push({ + kind, + name: doc.name, + source: sourceDocRef(tbdRoot, files, doc.path), + content: doc.content, + }); + } + } + return targets; + } + + const caches = new Map(); + for (const kind of kinds) { + caches.set(kind, await buildKindCache(kind, tbdRoot)); + } + + const targets: ResolvedDoc[] = []; + for (const name of names) { + const matches: ResolvedDoc[] = []; + for (const kind of kinds) { + const hit = caches.get(kind)!.get(name); + if (hit) { + matches.push({ + kind, + name: hit.doc.name, + source: sourceDocRef(tbdRoot, files, hit.doc.path), + content: hit.doc.content, + }); + } + } + if (matches.length === 0) { + throw new CLIError( + `No doc found named "${name}". Run \`tbd guidelines --list\` to see names.`, + ); + } + if (matches.length > 1) { + const kindList = matches.map((m) => m.kind).join(', '); + throw new CLIError( + `"${name}" exists in multiple kinds (${kindList}). Use --kind to disambiguate.`, + ); + } + targets.push(matches[0]!); + } + return targets; + } +} + +interface UnforkOptions { + kind?: string; + all?: boolean; + force?: boolean; + json?: boolean; +} + +class DocsUnforkHandler extends BaseCommand { + async run(names: string[], options: UnforkOptions): Promise { + await this.execute(async () => { + const tbdRoot = await requireInit(); + let manifest = await readForkManifest(tbdRoot); + + const targetNames = options.all ? manifest.forks.map((f) => f.name) : names; + if (targetNames.length === 0) { + throw new CLIError('Specify a doc name to unfork, or use --all.'); + } + + const removed: string[] = []; + for (const name of targetNames) { + try { + const result = await unforkDoc({ + tbdRoot, + forkDir: FORK_DIR, + manifest, + name, + kind: options.kind as ForkKind | undefined, + force: options.force, + }); + manifest = result.manifest; + removed.push(name); + if (!this.ctx.json) { + this.output.success(`Unforked ${name} — served from upstream again.`); + } + } catch (err) { + if (err instanceof ForkConflictError && err.code === 'customized') { + throw new CLIError( + `${name} has local customizations. Review with \`tbd docs status\`, then ` + + `re-run with --force to discard them and fall back to upstream.`, + ); + } + throw err; + } + } + await writeForkManifest(tbdRoot, manifest); + if (this.ctx.json) { + this.output.data({ unforked: removed }); + } + }, 'Failed to unfork'); + } +} + +/** Compose the display label for a doc's state, combining customized + stale. */ +function stateLabel(state: string, stale: boolean): string { + if ((state === 'customized' || state === 'orphaned' || state === 'conflicted') && stale) { + return `${state}, stale`; + } + return state; +} + +class DocsStatusHandler extends BaseCommand { + async run(options: { json?: boolean }): Promise { + await this.execute(async () => { + const tbdRoot = await requireInit(); + const manifest = await readForkManifest(tbdRoot); + const colors = this.output.getColors(); + + // Resolve upstream (cache) content per entry for staleness. + const caches = new Map(); + const rows: { + entry: ForkEntry; + label: string; + customized: boolean; + stale: boolean; + conflicted: boolean; + }[] = []; + + for (const entry of manifest.forks) { + const kind = entry.kind as ForkKind; + if (!caches.has(kind)) caches.set(kind, await buildKindCache(kind, tbdRoot)); + const cacheHit = caches.get(kind)!.get(entry.name); + const status = await forkStatusFor(tbdRoot, FORK_DIR, entry, cacheHit?.doc.content ?? null); + rows.push({ + entry, + label: stateLabel(status.state, status.stale), + customized: status.customized, + stale: status.stale, + conflicted: status.conflicted, + }); + } + + if (this.ctx.json) { + const docs: DocMapEntry[] = rows.map((r) => ({ + name: r.entry.name, + type: r.entry.kind, + path: r.entry.path, + source: r.entry.source, + state: r.label, + stale: r.stale, + })); + this.output.data(createDocMap(docs, { name: 'tbd-forks' })); + return; + } + + if (rows.length === 0) { + console.log('No docs forked into the repo.'); + console.log( + `Make some visible: ${colors.bold('tbd docs fork --category=general')} (and your languages)`, + ); + return; + } + + const nameW = Math.max(4, ...rows.map((r) => r.entry.name.length)); + const kindW = Math.max(4, ...rows.map((r) => r.entry.kind.length)); + const stateW = Math.max(5, ...rows.map((r) => r.label.length)); + const header = `${'NAME'.padEnd(nameW)} ${'KIND'.padEnd(kindW)} ${'STATE'.padEnd(stateW)} SOURCE`; + console.log(colors.dim(header)); + for (const r of rows) { + const line = `${r.entry.name.padEnd(nameW)} ${r.entry.kind.padEnd(kindW)} ${r.label.padEnd(stateW)} ${r.entry.source}`; + console.log(line); + } + + const customizedCount = rows.filter((r) => r.customized).length; + const staleCount = rows.filter((r) => r.stale).length; + const conflictCount = rows.filter((r) => r.conflicted).length; + const parts = [`${customizedCount} customized`]; + if (staleCount > 0) parts.push(`${staleCount} with upstream updates — run 'tbd docs update'`); + if (conflictCount > 0) parts.push(`${conflictCount} conflict pending`); + console.log(''); + console.log(`${rows.length} forked: ${parts.join(', ')}`); + }, 'Failed to read docs status'); + } +} + +/** + * Merge a subcommand's local options with globals/ancestors. The parent `docs` + * command also declares `--all` (its manual-viewer listing), so reading the local + * option alone is unreliable; fall back to the merged view. + */ +function mergedForkOptions(local: ForkOptions, command: Command): ForkOptions { + const g = command.optsWithGlobals(); + return { + all: local.all ?? g.all, + kind: local.kind ?? g.kind, + force: local.force ?? g.force, + dryRun: local.dryRun ?? g.dryRun, + json: g.json, + }; +} + +/** Register fork/unfork/status subcommands onto the `docs` command. */ +export function registerForkSubcommands(docs: Command): void { + docs + .command('fork') + .description( + 'Fork managed docs into the repo (default docs/tbd/) so they are visible and editable', + ) + .argument('[names...]', 'doc name(s) to fork') + .option('--kind ', 'restrict to a kind (guideline|shortcut|template)') + .option('--all', 'fork all available docs') + .option('--force', 'overwrite an existing non-fork file') + .action(async (names: string[], options: ForkOptions, command: Command) => { + await new DocsForkHandler(command).run(names, mergedForkOptions(options, command)); + }); + + docs + .command('unfork') + .description( + 'Remove a fork and fall back to upstream (refuses to discard edits without --force)', + ) + .argument('[names...]', 'doc name(s) to unfork') + .option('--kind ', 'restrict to a kind') + .option('--all', 'unfork all forked docs') + .option('--force', 'discard local customizations') + .action(async (names: string[], options: UnforkOptions, command: Command) => { + const m = mergedForkOptions(options, command); + await new DocsUnforkHandler(command).run(names, { + all: m.all, + kind: m.kind, + force: m.force, + json: m.json, + }); + }); + + docs + .command('status') + .description('Show forked docs and their states') + .action(async (_options: { json?: boolean }, command: Command) => { + await new DocsStatusHandler(command).run({ json: command.optsWithGlobals().json === true }); + }); +} diff --git a/packages/tbd/src/cli/commands/docs.ts b/packages/tbd/src/cli/commands/docs.ts index d31d07ac..ca419b55 100644 --- a/packages/tbd/src/cli/commands/docs.ts +++ b/packages/tbd/src/cli/commands/docs.ts @@ -14,6 +14,7 @@ import { fileURLToPath } from 'node:url'; import { dirname, join } from 'node:path'; import { BaseCommand } from '../lib/base-command.js'; +import { registerForkSubcommands } from './docs-fork.js'; import { shouldUseInteractiveOutput } from '../lib/context.js'; import { CLIError, NotFoundError } from '../lib/errors.js'; import { renderMarkdown, paginateOutput } from '../lib/output.js'; @@ -247,3 +248,7 @@ export const docsCommand = new Command('docs') const handler = new DocsHandler(command); await handler.run(topic, options); }); + +// Forkable-docs operations (fork / unfork / status) are added as subcommands. +// The existing manual-viewer behavior remains the default action above. +registerForkSubcommands(docsCommand); diff --git a/packages/tbd/src/lib/paths.ts b/packages/tbd/src/lib/paths.ts index 7bb3d98c..075ca41b 100644 --- a/packages/tbd/src/lib/paths.ts +++ b/packages/tbd/src/lib/paths.ts @@ -351,28 +351,55 @@ export const INSTALL_DIR = 'install'; /** Built-in install source path (relative to package docs/) */ export const BUILTIN_INSTALL_DIR = INSTALL_DIR; +// ============================================================================= +// Forkable docs: the visible, git-tracked fork directory +// ============================================================================= + +/** References directory name (tbd self-docs and format references). */ +export const REFERENCES_DIR = 'references'; + +/** Default fork directory (repo-relative), where forked docs are made visible. */ +export const FORK_DIR = join(DOCS_DIR, 'tbd'); // docs/tbd/ + +/** Fork-dir kind subdirectories (repo-relative). */ +export const FORK_SHORTCUTS_DIR = join(FORK_DIR, SHORTCUTS_DIR); // docs/tbd/shortcuts/ +export const FORK_GUIDELINES_DIR = join(FORK_DIR, GUIDELINES_DIR); // docs/tbd/guidelines/ +export const FORK_TEMPLATES_DIR = join(FORK_DIR, TEMPLATES_DIR); // docs/tbd/templates/ +export const FORK_REFERENCES_DIR = join(FORK_DIR, REFERENCES_DIR); // docs/tbd/references/ + +/** + * Cache-only lookup paths (the gitignored `.tbd/docs/` cache), used when forking + * needs the pristine upstream content rather than a possibly-forked copy. + */ +export const CACHE_SHORTCUT_PATHS = [TBD_SHORTCUTS_SYSTEM, TBD_SHORTCUTS_STANDARD]; +export const CACHE_GUIDELINES_PATHS = [TBD_GUIDELINES_DIR]; +export const CACHE_TEMPLATE_PATHS = [TBD_TEMPLATES_DIR]; + /** * Default shortcut lookup paths (searched in order, relative to tbd root). - * Earlier paths take precedence over later paths. - * Note: Guidelines and templates are now separate top-level directories. + * Earlier paths take precedence: the fork dir shadows the cache, so a forked doc + * is served wherever the upstream one was. Missing dirs are skipped, so repos with + * no forks behave exactly as before. */ export const DEFAULT_SHORTCUT_PATHS = [ - TBD_SHORTCUTS_SYSTEM, // .tbd/docs/shortcuts/system/ - TBD_SHORTCUTS_STANDARD, // .tbd/docs/shortcuts/standard/ + FORK_SHORTCUTS_DIR, // docs/tbd/shortcuts/ (forked, highest precedence) + ...CACHE_SHORTCUT_PATHS, ]; /** * Default guidelines lookup paths (relative to tbd root). */ export const DEFAULT_GUIDELINES_PATHS = [ - TBD_GUIDELINES_DIR, // .tbd/docs/guidelines/ + FORK_GUIDELINES_DIR, // docs/tbd/guidelines/ (forked, highest precedence) + ...CACHE_GUIDELINES_PATHS, ]; /** * Default template lookup paths (relative to tbd root). */ export const DEFAULT_TEMPLATE_PATHS = [ - TBD_TEMPLATES_DIR, // .tbd/docs/templates/ + FORK_TEMPLATES_DIR, // docs/tbd/templates/ (forked, highest precedence) + ...CACHE_TEMPLATE_PATHS, ]; /** diff --git a/packages/tbd/tests/cli-docs-fork.tryscript.md b/packages/tbd/tests/cli-docs-fork.tryscript.md new file mode 100644 index 00000000..fdc16a2f --- /dev/null +++ b/packages/tbd/tests/cli-docs-fork.tryscript.md @@ -0,0 +1,157 @@ +--- +sandbox: true +env: + NO_COLOR: '1' + FORCE_COLOR: '0' +path: + - ../dist +timeout: 60000 +patterns: + VERSION: 'v[0-9]+\.[0-9]+\.[0-9]+(-[a-z0-9.-]+)?' +before: | + git init --initial-branch=main + git config user.email "test@example.com" + git config user.name "Test User" + git config commit.gpgsign false + echo "# Test repo" > README.md + git add README.md + git commit -m "Initial commit" + tbd setup --auto --prefix=fk --quiet +--- +# tbd docs: Fork Lifecycle Golden Test + +End-to-end fork lifecycle for the `tbd docs` command group: status, fork, status again, +unfork. Serving precedence, customized-refusal, and out-of-band deletion are covered by +the `doc-fork` unit tests; these blocks pin the CLI surface. + +* * * + +## Status with nothing forked + +# Test: status reports no forks + +```console +$ tbd docs status +No docs forked into the repo. +Make some visible: tbd docs fork --category=general (and your languages) +? 0 +``` + +* * * + +## Fork a guideline + +# Test: fork python-rules writes the file and records the manifest + +```console +$ tbd docs fork python-rules +✓ Forked python-rules → docs/tbd/guidelines/python-rules.md + +Edit in place — tbd now serves your copy wherever it served upstream. +? 0 +``` + +# Test: the forked file is present in the repo + +```console +$ test -f docs/tbd/guidelines/python-rules.md && echo present +present +? 0 +``` + +# Test: the base snapshot is recorded under .tbd/doc-forks/ + +```console +$ test -f .tbd/doc-forks/base/guideline/python-rules.md && echo present +present +? 0 +``` + +* * * + +## Status shows the fork + +# Test: status lists the forked doc + +```console +$ tbd docs status +NAME KIND STATE SOURCE +python-rules guideline forked internal:guidelines/python-rules.md + +1 forked: 0 customized +? 0 +``` + +* * * + +## Forking everything available + +# Test: --dry-run previews without writing + +```console +$ tbd docs fork --all --dry-run +[DRY-RUN] Would fork [..] doc(s) into docs/tbd/ +... +No files written. Re-run without --dry-run to apply. +? 0 +``` + +* * * + +## Unfork restores upstream + +# Test: unfork an unmodified fork + +```console +$ tbd docs unfork python-rules +✓ Unforked python-rules — served from upstream again. +? 0 +``` + +# Test: status reports no forks again + +```console +$ tbd docs status +No docs forked into the repo. +... +? 0 +``` + +* * * + +## Out-of-band deletion falls back to upstream + +# Test: deleting a forked file still serves the guideline from upstream + +```console +$ tbd docs fork python-rules +✓ Forked python-rules → docs/tbd/guidelines/python-rules.md +... +? 0 +``` + +# Test: remove the forked file directly + +```console +$ rm docs/tbd/guidelines/python-rules.md +? 0 +``` + +# Test: the guideline still resolves (served from the cache) + +```console +$ tbd guidelines python-rules +... +? 0 +``` + +# Test: status reports the dangling fork as missing + +```console +$ tbd docs status +NAME KIND STATE SOURCE +python-rules guideline missing internal:guidelines/python-rules.md + +1 forked: 0 customized +? 0 +``` From 7112f3623529d2ae0194e0e0f776aeb2081cb005 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 12 Jun 2026 09:33:38 +0000 Subject: [PATCH 09/36] feat: Add tbd docs update (three-way merge + decision table) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Add the "keep forks current" loop the spec calls the most common lifecycle event. - src/file/fork-update.ts: a git merge-file wrapper (captures the conflict-count exit code, standard markers, no repo state touched) and updateOne — the per-state decision logic across default / --merge / --keep-ours (refresh unmodified, clean three-way merge, conflict skip+list, --merge markers + base advance + conflicted flag, --keep-ours keep+advance, missing-base repair, and orphaned/missing/not-stale skips). 14 unit tests cover the table. - tbd docs update [names...] [--merge|--keep-ours] [--dry-run]: iterate forks, read fork/base/upstream content, apply, advance bases, set/clear the conflicted flag, and print the applied + needs-decision summary. Adds cli-docs-update.tryscript.md (refresh, conflict skip, --merge markers, status conflicted, mutual exclusion). Implements tbd-jme1, tbd-f8bu. https://claude.ai/code/session_01X8S12JzmmxEfLpYzgH8Y7E --- packages/tbd/src/cli/commands/docs-fork.ts | 148 ++++++++++- packages/tbd/src/file/fork-update.ts | 240 ++++++++++++++++++ .../tbd/tests/cli-docs-update.tryscript.md | 107 ++++++++ packages/tbd/tests/fork-update.test.ts | 193 ++++++++++++++ 4 files changed, 686 insertions(+), 2 deletions(-) create mode 100644 packages/tbd/src/file/fork-update.ts create mode 100644 packages/tbd/tests/cli-docs-update.tryscript.md create mode 100644 packages/tbd/tests/fork-update.test.ts diff --git a/packages/tbd/src/cli/commands/docs-fork.ts b/packages/tbd/src/cli/commands/docs-fork.ts index cce8005d..b9b905c0 100644 --- a/packages/tbd/src/cli/commands/docs-fork.ts +++ b/packages/tbd/src/cli/commands/docs-fork.ts @@ -8,7 +8,10 @@ */ import type { Command } from 'commander'; -import { join, relative, sep } from 'node:path'; +import { dirname, join, relative, sep } from 'node:path'; +import { mkdir } from 'node:fs/promises'; + +import { writeFile } from 'atomically'; import { BaseCommand } from '../lib/base-command.js'; import { requireInit } from '../lib/errors.js'; @@ -26,10 +29,22 @@ import { import { type ForkEntry, type ForkKind, + hashContent, readForkManifest, writeForkManifest, + writeBaseContent, + upsertFork, } from '../../file/fork-manifest.js'; -import { forkDoc, unforkDoc, forkStatusFor, ForkConflictError } from '../../file/doc-fork.js'; +import { + forkDoc, + unforkDoc, + forkStatusFor, + forkFilePath, + readForkFile, + readForkBase, + ForkConflictError, +} from '../../file/doc-fork.js'; +import { updateOne, type UpdateStrategy } from '../../file/fork-update.js'; import { createDocMap, type DocMapEntry } from '../../docmap/index.js'; /** Kinds that can be resolved from the cache and forked today. */ @@ -328,6 +343,119 @@ class DocsStatusHandler extends BaseCommand { } } +interface UpdateOptions { + merge?: boolean; + keepOurs?: boolean; + dryRun?: boolean; + json?: boolean; +} + +class DocsUpdateHandler extends BaseCommand { + async run(names: string[], options: UpdateOptions): Promise { + await this.execute(async () => { + if (options.merge && options.keepOurs) { + throw new CLIError('--merge and --keep-ours are mutually exclusive.'); + } + const strategy: UpdateStrategy = options.merge + ? 'merge' + : options.keepOurs + ? 'keep-ours' + : 'default'; + + const tbdRoot = await requireInit(); + let manifest = await readForkManifest(tbdRoot); + const selected = + names.length > 0 ? manifest.forks.filter((f) => names.includes(f.name)) : manifest.forks; + + const caches = new Map(); + const upstreamFor = async (entry: ForkEntry): Promise => { + const kind = entry.kind as ForkKind; + if (!caches.has(kind)) caches.set(kind, await buildKindCache(kind, tbdRoot)); + return caches.get(kind)!.get(entry.name)?.doc.content ?? null; + }; + + const applied: { entry: ForkEntry; message: string }[] = []; + const decisions: string[] = []; + + for (const entry of selected) { + const result = await updateOne({ + entry, + forkContent: await readForkFile(tbdRoot, FORK_DIR, entry), + baseContent: await readForkBase(tbdRoot, entry), + upstreamContent: await upstreamFor(entry), + strategy, + }); + + const { newFileContent, newBaseContent } = result; + if (newFileContent === undefined && newBaseContent === undefined) { + if (result.needsDecision) decisions.push(result.message); + continue; + } + + if (!options.dryRun) { + if (newFileContent !== undefined) { + const abs = forkFilePath(tbdRoot, FORK_DIR, entry.kind as ForkKind, entry.name); + await mkdir(dirname(abs), { recursive: true }); + await writeFile(abs, newFileContent); + } + const updated: ForkEntry = { ...entry }; + if (newBaseContent !== undefined) { + await writeBaseContent(tbdRoot, entry.kind, entry.name, newBaseContent); + updated.base_hash = hashContent(newBaseContent); + } + if (result.setConflicted) { + updated.conflicted = true; + } else { + delete updated.conflicted; + } + manifest = upsertFork(manifest, updated); + } + applied.push({ entry, message: result.message }); + } + + if (!options.dryRun) { + await writeForkManifest(tbdRoot, manifest); + } + + if (this.ctx.json) { + this.output.data({ + dryRun: Boolean(options.dryRun), + updated: applied.map((a) => a.entry.name), + needsDecision: decisions, + }); + return; + } + + const colors = this.output.getColors(); + if (applied.length === 0 && decisions.length === 0) { + console.log('All forked docs are up to date.'); + return; + } + if (applied.length > 0) { + const verb = options.dryRun ? 'Would update' : 'Updated'; + console.log(`${verb} ${applied.length} forked doc(s):`); + for (const a of applied) { + console.log(` ${colors.success('✓')} ${a.message}`); + } + } + if (decisions.length > 0) { + console.log(''); + console.log(`${decisions.length} doc(s) need a decision:`); + for (const msg of decisions) { + console.log(` ${colors.warn('⚠')} ${msg}`); + } + console.log(' re-run with one of:'); + console.log( + ' tbd docs update --merge # combine, then resolve conflict markers', + ); + console.log( + ' tbd docs update --keep-ours # keep your version, advance the fork point', + ); + } + }, 'Failed to update forked docs'); + } +} + /** * Merge a subcommand's local options with globals/ancestors. The parent `docs` * command also declares `--all` (its manual-viewer listing), so reading the local @@ -384,4 +512,20 @@ export function registerForkSubcommands(docs: Command): void { .action(async (_options: { json?: boolean }, command: Command) => { await new DocsStatusHandler(command).run({ json: command.optsWithGlobals().json === true }); }); + + docs + .command('update') + .description('Reconcile forked docs with upstream after an upgrade (--merge / --keep-ours)') + .argument('[names...]', 'doc name(s) to update (default: all)') + .option('--merge', 'on conflict: combine and write conflict markers to resolve') + .option('--keep-ours', 'on conflict: keep your version and advance the fork point') + .action(async (names: string[], options: UpdateOptions, command: Command) => { + const g = command.optsWithGlobals(); + await new DocsUpdateHandler(command).run(names, { + merge: options.merge, + keepOurs: options.keepOurs, + dryRun: g.dryRun === true, + json: g.json === true, + }); + }); } diff --git a/packages/tbd/src/file/fork-update.ts b/packages/tbd/src/file/fork-update.ts new file mode 100644 index 00000000..4cf5bbdc --- /dev/null +++ b/packages/tbd/src/file/fork-update.ts @@ -0,0 +1,240 @@ +/** + * Updating forked docs after upstream moves: three-way merge plus the per-state + * decision logic from the spec's update table. + * + * The merge itself is outsourced to git (`git merge-file`), which works on plain + * files, reports the conflict count via exit code, and uses standard conflict + * markers. Nothing in the git repo state is touched. + */ + +import { execFile } from 'node:child_process'; +import { mkdtemp, rm, readFile } from 'node:fs/promises'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; + +import { writeFile } from 'atomically'; + +import { type ForkEntry, hashContent, hasConflictMarkers } from './fork-manifest.js'; + +/** Result of a three-way merge. */ +export interface MergeResult { + merged: string; + /** Number of conflict hunks (0 = clean). */ + conflicts: number; +} + +const MERGE_MAX_BUFFER = 16 * 1024 * 1024; + +/** + * Three-way merge of `current` and `other` against their common `base`, via + * `git merge-file -p`. Returns the merged text (with standard conflict markers + * when conflicts arise) and the conflict count. Pure with respect to repo state: + * it only uses temporary files. + */ +export async function mergeContents( + current: string, + base: string, + other: string, + labels: { current?: string; base?: string; other?: string } = {}, +): Promise { + const dir = await mkdtemp(join(tmpdir(), 'tbd-merge-')); + const currentPath = join(dir, 'current'); + const basePath = join(dir, 'base'); + const otherPath = join(dir, 'other'); + try { + await Promise.all([ + writeFile(currentPath, current), + writeFile(basePath, base), + writeFile(otherPath, other), + ]); + + const args = [ + 'merge-file', + '-p', + '-L', + labels.current ?? 'ours (your fork)', + '-L', + labels.base ?? 'base (fork point)', + '-L', + labels.other ?? 'theirs (upstream)', + currentPath, + basePath, + otherPath, + ]; + + return await new Promise((resolve, reject) => { + execFile('git', args, { maxBuffer: MERGE_MAX_BUFFER }, (error, stdout) => { + if (error) { + const code = (error as NodeJS.ErrnoException & { code?: number }).code; + // git merge-file exits with the number of conflicts (>0); negative/other + // codes are real errors. + if (typeof code === 'number' && code > 0) { + resolve({ merged: stdout, conflicts: code }); + return; + } + reject(error instanceof Error ? error : new Error('git merge-file failed')); + return; + } + resolve({ merged: stdout, conflicts: 0 }); + }); + }); + } finally { + await rm(dir, { recursive: true, force: true }); + } +} + +/** Update strategy chosen by the user for non-clean cases. */ +export type UpdateStrategy = 'default' | 'merge' | 'keep-ours'; + +/** What an update did (or why it was skipped) for one doc. */ +export type UpdateAction = + | 'noop' + | 'replaced' + | 'merged-clean' + | 'merged-conflict' + | 'kept' + | 'repaired' + | 'skip-not-stale' + | 'skip-conflict-listed' + | 'skip-unresolved' + | 'skip-orphaned' + | 'skip-missing' + | 'skip-no-base'; + +export interface UpdateOneInput { + entry: ForkEntry; + /** Current forked file content, or null if the file is missing. */ + forkContent: string | null; + /** Stored base snapshot, or null if the base file is missing. */ + baseContent: string | null; + /** Current upstream/cache content, or null if the source is gone (orphaned). */ + upstreamContent: string | null; + strategy: UpdateStrategy; +} + +export interface UpdateOneResult { + action: UpdateAction; + /** New forked-file content to write, when the action changes the file. */ + newFileContent?: string; + /** New base content to write (advances the fork point), when applicable. */ + newBaseContent?: string; + /** Set the manifest `conflicted` flag (markers written). */ + setConflicted?: boolean; + /** Whether this doc needs a strategy decision (default run could not proceed). */ + needsDecision?: boolean; + /** Human-readable one-line explanation. */ + message: string; +} + +/** + * Decide and (when needed) perform the three-way merge for a single forked doc, + * implementing the spec's update decision table across the default / --merge / + * --keep-ours strategies. Pure aside from the git merge-file call. + */ +export async function updateOne(input: UpdateOneInput): Promise { + const { entry, forkContent, baseContent, upstreamContent, strategy } = input; + const name = entry.name; + + if (forkContent === null) { + return { action: 'skip-missing', message: `${name}: forked file is missing (doctor's job)` }; + } + if (upstreamContent === null) { + return { + action: 'skip-orphaned', + message: `${name}: upstream removed this doc — keep your copy or 'tbd docs unfork ${name}'`, + }; + } + // An unresolved conflicted doc must be resolved before any update. + if (entry.conflicted && hasConflictMarkers(forkContent)) { + return { + action: 'skip-unresolved', + message: `${name}: unresolved conflict markers — resolve them first`, + }; + } + + if (baseContent === null) { + if (strategy === 'keep-ours') { + // Repair: re-establish the base from current upstream, keep the file. + return { + action: 'repaired', + newBaseContent: upstreamContent, + message: `${name}: re-established missing base from upstream (file kept)`, + }; + } + return { + action: 'skip-no-base', + needsDecision: true, + message: `${name}: base snapshot missing — cannot merge; re-run with --keep-ours to repair`, + }; + } + + const customized = hashContent(forkContent) !== hashContent(baseContent); + const stale = hashContent(upstreamContent) !== hashContent(baseContent); + + if (!stale) { + return { action: 'skip-not-stale', message: `${name}: already up to date` }; + } + + // Unmodified fork that is stale: refresh to upstream (default/merge), or keep + // the local copy and just advance the base (keep-ours). + if (!customized) { + if (strategy === 'keep-ours') { + return { + action: 'kept', + newBaseContent: upstreamContent, + message: `${name}: kept your version; fork point advanced`, + }; + } + return { + action: 'replaced', + newFileContent: upstreamContent, + newBaseContent: upstreamContent, + message: `${name}: refreshed to upstream (was unmodified)`, + }; + } + + // Customized and stale. + if (strategy === 'keep-ours') { + return { + action: 'kept', + newBaseContent: upstreamContent, + message: `${name}: kept your version; fork point advanced`, + }; + } + + const merge = await mergeContents(forkContent, baseContent, upstreamContent); + if (merge.conflicts === 0) { + return { + action: 'merged-clean', + newFileContent: merge.merged, + newBaseContent: upstreamContent, + message: `${name}: merged upstream cleanly (review with: git diff)`, + }; + } + + // Conflicts. + if (strategy === 'merge') { + return { + action: 'merged-conflict', + newFileContent: merge.merged, + newBaseContent: upstreamContent, + setConflicted: true, + message: `${name}: wrote merged content with conflict markers; resolve them, then it returns to 'customized'`, + }; + } + // Default: skip and surface the decision. + return { + action: 'skip-conflict-listed', + needsDecision: true, + message: `${name}: your changes conflict with upstream`, + }; +} + +/** Read a file's content, or null if absent. */ +export async function readMaybe(path: string): Promise { + try { + return await readFile(path, 'utf-8'); + } catch { + return null; + } +} diff --git a/packages/tbd/tests/cli-docs-update.tryscript.md b/packages/tbd/tests/cli-docs-update.tryscript.md new file mode 100644 index 00000000..1bae3c47 --- /dev/null +++ b/packages/tbd/tests/cli-docs-update.tryscript.md @@ -0,0 +1,107 @@ +--- +sandbox: true +env: + NO_COLOR: '1' + FORCE_COLOR: '0' +path: + - ../dist +timeout: 60000 +before: | + git init --initial-branch=main + git config user.email "test@example.com" + git config user.name "Test User" + git config commit.gpgsign false + echo "# Test repo" > README.md + git add README.md + git commit -m "Initial commit" + tbd setup --auto --prefix=up --quiet + tbd docs fork python-rules +--- +# tbd docs update: Upgrade/Merge Golden Test + +Simulates a tbd upgrade by editing the gitignored cache copy (the upstream), then +exercises `tbd docs update` across the refresh, clean-merge, and conflict paths. +The decision table itself is covered exhaustively by the `fork-update` unit tests. + +* * * + +## Unmodified fork + upstream change → refresh + +# Test: simulate an upstream change to the cache copy + +```console +$ printf '\n\n' >> .tbd/docs/guidelines/python-rules.md +? 0 +``` + +# Test: update refreshes the unmodified fork + +```console +$ tbd docs update +Updated 1 forked doc(s): + ✓ python-rules: refreshed to upstream (was unmodified) +? 0 +``` + +* * * + +## Customized fork + overlapping upstream change → conflict + +# Test: customize the fork’s first line + +```console +$ sed -i '1c\' docs/tbd/guidelines/python-rules.md +? 0 +``` + +# Test: change the same line upstream + +```console +$ sed -i '1c\' .tbd/docs/guidelines/python-rules.md +? 0 +``` + +# Test: update skips conflicts by default and names both strategies + +```console +$ tbd docs update +... +1 doc(s) need a decision: + ⚠ python-rules: your changes conflict with upstream + re-run with one of: + tbd docs update --merge # combine, then resolve conflict markers + tbd docs update --keep-ours # keep your version, advance the fork point +? 0 +``` + +# Test: --merge writes conflict markers and flags the doc conflicted + +```console +$ tbd docs update --merge +Updated 1 forked doc(s): + ✓ python-rules: wrote merged content with conflict markers; resolve them, then it returns to 'customized' +? 0 +``` + +# Test: status reports the doc as conflicted + +```console +$ tbd docs status +NAME KIND STATE SOURCE +python-rules guideline conflicted internal:guidelines/python-rules.md + +1 forked: 1 customized, 1 conflict pending +? 0 +``` + +* * * + +## --merge and --keep-ours are mutually exclusive + +# Test: passing both is an error + +```console +$ tbd docs update --merge --keep-ours 2>&1 +[..]mutually exclusive[..] +? 1 +``` diff --git a/packages/tbd/tests/fork-update.test.ts b/packages/tbd/tests/fork-update.test.ts new file mode 100644 index 00000000..aa044010 --- /dev/null +++ b/packages/tbd/tests/fork-update.test.ts @@ -0,0 +1,193 @@ +/** + * Tests for the merge wrapper (git merge-file) and the update decision table. + */ + +import { describe, it, expect } from 'vitest'; + +import { mergeContents, updateOne, type UpdateStrategy } from '../src/file/fork-update.js'; +import { type ForkEntry, hashContent, hasConflictMarkers } from '../src/file/fork-manifest.js'; + +const BASE = 'line one\nline two\nline three\n'; + +function entry(overrides: Partial = {}): ForkEntry { + return { + name: 'python-rules', + kind: 'guideline', + path: 'docs/tbd/guidelines/python-rules.md', + source: 'internal:guidelines/python-rules.md', + base_hash: hashContent(BASE), + ...overrides, + }; +} + +describe('mergeContents', () => { + it('merges non-overlapping edits cleanly', async () => { + const ours = 'line ONE\nline two\nline three\n'; // edited line 1 + const theirs = 'line one\nline two\nline THREE\n'; // edited line 3 + const result = await mergeContents(ours, BASE, theirs); + expect(result.conflicts).toBe(0); + expect(result.merged).toBe('line ONE\nline two\nline THREE\n'); + }); + + it('reports conflicts and writes markers for overlapping edits', async () => { + const ours = 'line one\nMINE\nline three\n'; + const theirs = 'line one\nTHEIRS\nline three\n'; + const result = await mergeContents(ours, BASE, theirs); + expect(result.conflicts).toBeGreaterThan(0); + expect(hasConflictMarkers(result.merged)).toBe(true); + }); +}); + +describe('updateOne decision table', () => { + const EDITED = 'line ONE\nline two\nline three\n'; // diverges from BASE on line 1 + const UPSTREAM_NONCONFLICT = 'line one\nline two\nline THREE\n'; // line 3 + const UPSTREAM_CONFLICT = 'line ONE-theirs\nline two\nline three\n'; // line 1, conflicts with EDITED + + it('skips a missing forked file', async () => { + const r = await updateOne({ + entry: entry(), + forkContent: null, + baseContent: BASE, + upstreamContent: UPSTREAM_NONCONFLICT, + strategy: 'default', + }); + expect(r.action).toBe('skip-missing'); + }); + + it('skips an orphaned doc (upstream gone)', async () => { + const r = await updateOne({ + entry: entry(), + forkContent: BASE, + baseContent: BASE, + upstreamContent: null, + strategy: 'default', + }); + expect(r.action).toBe('skip-orphaned'); + }); + + it('skips an unresolved conflicted doc', async () => { + const withMarkers = '<<<<<<< ours\na\n=======\nb\n>>>>>>> theirs\n'; + const r = await updateOne({ + entry: entry({ conflicted: true }), + forkContent: withMarkers, + baseContent: BASE, + upstreamContent: UPSTREAM_NONCONFLICT, + strategy: 'default', + }); + expect(r.action).toBe('skip-unresolved'); + }); + + it('is a no-op when not stale', async () => { + const r = await updateOne({ + entry: entry(), + forkContent: EDITED, + baseContent: BASE, + upstreamContent: BASE, // upstream == base => not stale + strategy: 'default', + }); + expect(r.action).toBe('skip-not-stale'); + }); + + it('replaces an unmodified stale fork (default)', async () => { + const r = await updateOne({ + entry: entry(), + forkContent: BASE, // unmodified + baseContent: BASE, + upstreamContent: UPSTREAM_NONCONFLICT, // stale + strategy: 'default', + }); + expect(r.action).toBe('replaced'); + expect(r.newFileContent).toBe(UPSTREAM_NONCONFLICT); + expect(r.newBaseContent).toBe(UPSTREAM_NONCONFLICT); + }); + + it('applies a clean three-way merge (default)', async () => { + const r = await updateOne({ + entry: entry(), + forkContent: EDITED, + baseContent: BASE, + upstreamContent: UPSTREAM_NONCONFLICT, + strategy: 'default', + }); + expect(r.action).toBe('merged-clean'); + expect(r.newFileContent).toBe('line ONE\nline two\nline THREE\n'); + expect(r.newBaseContent).toBe(UPSTREAM_NONCONFLICT); + }); + + it('skips a conflicting merge by default and asks for a decision', async () => { + const r = await updateOne({ + entry: entry(), + forkContent: EDITED, + baseContent: BASE, + upstreamContent: UPSTREAM_CONFLICT, + strategy: 'default', + }); + expect(r.action).toBe('skip-conflict-listed'); + expect(r.needsDecision).toBe(true); + }); + + it('writes conflict markers and advances base with --merge', async () => { + const r = await updateOne({ + entry: entry(), + forkContent: EDITED, + baseContent: BASE, + upstreamContent: UPSTREAM_CONFLICT, + strategy: 'merge', + }); + expect(r.action).toBe('merged-conflict'); + expect(r.setConflicted).toBe(true); + expect(hasConflictMarkers(r.newFileContent ?? '')).toBe(true); + expect(r.newBaseContent).toBe(UPSTREAM_CONFLICT); + }); + + it('keeps the local version and advances base with --keep-ours', async () => { + const r = await updateOne({ + entry: entry(), + forkContent: EDITED, + baseContent: BASE, + upstreamContent: UPSTREAM_CONFLICT, + strategy: 'keep-ours', + }); + expect(r.action).toBe('kept'); + expect(r.newFileContent).toBeUndefined(); // file untouched + expect(r.newBaseContent).toBe(UPSTREAM_CONFLICT); + }); + + it('repairs a missing base with --keep-ours', async () => { + const r = await updateOne({ + entry: entry(), + forkContent: EDITED, + baseContent: null, // base file gone + upstreamContent: UPSTREAM_NONCONFLICT, + strategy: 'keep-ours', + }); + expect(r.action).toBe('repaired'); + expect(r.newBaseContent).toBe(UPSTREAM_NONCONFLICT); + }); + + it('skips a missing base by default and points at --keep-ours', async () => { + const r = await updateOne({ + entry: entry(), + forkContent: EDITED, + baseContent: null, + upstreamContent: UPSTREAM_NONCONFLICT, + strategy: 'default', + }); + expect(r.action).toBe('skip-no-base'); + expect(r.needsDecision).toBe(true); + }); + + it('keep-ours on an unmodified stale fork advances base without touching the file', async () => { + const strategy: UpdateStrategy = 'keep-ours'; + const r = await updateOne({ + entry: entry(), + forkContent: BASE, + baseContent: BASE, + upstreamContent: UPSTREAM_NONCONFLICT, + strategy, + }); + expect(r.action).toBe('kept'); + expect(r.newFileContent).toBeUndefined(); + expect(r.newBaseContent).toBe(UPSTREAM_NONCONFLICT); + }); +}); From 952abb59409c872e288b59870648d26d9d426eae Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 12 Jun 2026 09:38:58 +0000 Subject: [PATCH 10/36] feat: Add tbd docs diff (fork vs upstream / base / incoming) - diffContents: a git diff --no-index wrapper with clean labels (runs from a temp dir with --no-prefix so headers read "--- upstream / +++ ours", not temp paths). - tbd docs diff : your file vs current upstream (the net fork); --base = your file vs its base (what you changed); --upstream = base vs current upstream (incoming changes). Errors clearly when a side is unavailable. Used by the upstream-contribution workflow. Adds diffContents unit tests. Implements tbd-pghj. https://claude.ai/code/session_01X8S12JzmmxEfLpYzgH8Y7E --- packages/tbd/src/cli/commands/docs-fork.ts | 72 +++++++++++++++++++++- packages/tbd/src/file/fork-update.ts | 40 ++++++++++++ packages/tbd/tests/fork-update.test.ts | 24 +++++++- 3 files changed, 134 insertions(+), 2 deletions(-) diff --git a/packages/tbd/src/cli/commands/docs-fork.ts b/packages/tbd/src/cli/commands/docs-fork.ts index b9b905c0..f968acdd 100644 --- a/packages/tbd/src/cli/commands/docs-fork.ts +++ b/packages/tbd/src/cli/commands/docs-fork.ts @@ -29,6 +29,7 @@ import { import { type ForkEntry, type ForkKind, + findFork, hashContent, readForkManifest, writeForkManifest, @@ -44,7 +45,7 @@ import { readForkBase, ForkConflictError, } from '../../file/doc-fork.js'; -import { updateOne, type UpdateStrategy } from '../../file/fork-update.js'; +import { updateOne, diffContents, type UpdateStrategy } from '../../file/fork-update.js'; import { createDocMap, type DocMapEntry } from '../../docmap/index.js'; /** Kinds that can be resolved from the cache and forked today. */ @@ -456,6 +457,62 @@ class DocsUpdateHandler extends BaseCommand { } } +interface DiffOptions { + base?: boolean; + upstream?: boolean; + kind?: string; +} + +class DocsDiffHandler extends BaseCommand { + async run(name: string, options: DiffOptions): Promise { + await this.execute(async () => { + const tbdRoot = await requireInit(); + const manifest = await readForkManifest(tbdRoot); + const entry = findFork(manifest, name, options.kind as ForkKind | undefined); + if (!entry) { + throw new CLIError(`${name} is not a forked doc. Run \`tbd docs status\` to see forks.`); + } + + const forkContent = await readForkFile(tbdRoot, FORK_DIR, entry); + const baseContent = await readForkBase(tbdRoot, entry); + const cache = await buildKindCache(entry.kind as ForkKind, tbdRoot); + const upstreamContent = cache.get(entry.name)?.doc.content ?? null; + + // Default: your file vs current upstream (the net fork). + let left = upstreamContent; + let right = forkContent; + let labels = { left: 'upstream', right: 'ours' }; + if (options.base) { + left = baseContent; + right = forkContent; + labels = { left: 'base', right: 'ours' }; + } else if (options.upstream) { + left = baseContent; + right = upstreamContent; + labels = { left: 'base', right: 'upstream' }; + } + + if (left === null || right === null) { + throw new CLIError( + `Cannot diff ${name}: one side is unavailable ` + + `(forked file missing, base missing, or upstream gone).`, + ); + } + + const diff = await diffContents(left, right, labels); + if (this.ctx.json) { + this.output.data({ name: entry.name, kind: entry.kind, diff }); + return; + } + if (!diff.trim()) { + console.log(`No differences (${labels.left} vs ${labels.right}).`); + return; + } + console.log(diff.trimEnd()); + }, 'Failed to diff'); + } +} + /** * Merge a subcommand's local options with globals/ancestors. The parent `docs` * command also declares `--all` (its manual-viewer listing), so reading the local @@ -528,4 +585,17 @@ export function registerForkSubcommands(docs: Command): void { json: g.json === true, }); }); + + docs + .command('diff') + .description( + 'Diff a forked doc against upstream (default), its base (--base), or incoming (--upstream)', + ) + .argument('', 'forked doc name') + .option('--base', 'diff your file against its base (what you changed)') + .option('--upstream', 'diff the base against current upstream (incoming changes)') + .option('--kind ', 'restrict to a kind') + .action(async (name: string, options: DiffOptions, command: Command) => { + await new DocsDiffHandler(command).run(name, options); + }); } diff --git a/packages/tbd/src/file/fork-update.ts b/packages/tbd/src/file/fork-update.ts index 4cf5bbdc..e3b53ace 100644 --- a/packages/tbd/src/file/fork-update.ts +++ b/packages/tbd/src/file/fork-update.ts @@ -83,6 +83,46 @@ export async function mergeContents( } } +/** + * Unified diff of two contents via `git diff --no-index`. Returns the diff text + * (empty string when identical). Uses temporary files and touches no repo state. + */ +export async function diffContents( + left: string, + right: string, + labels: { left?: string; right?: string } = {}, +): Promise { + const dir = await mkdtemp(join(tmpdir(), 'tbd-diff-')); + // Name the temp files after the labels and run from the temp dir with --no-prefix + // so the diff header reads e.g. "--- upstream / +++ ours" instead of temp paths. + const leftName = labels.left ?? 'a'; + const rightName = labels.right ?? 'b'; + try { + await Promise.all([ + writeFile(join(dir, leftName), left), + writeFile(join(dir, rightName), right), + ]); + return await new Promise((resolve, reject) => { + const args = ['diff', '--no-index', '--no-color', '--no-prefix', leftName, rightName]; + execFile('git', args, { cwd: dir, maxBuffer: MERGE_MAX_BUFFER }, (error, stdout) => { + if (error) { + const code = (error as NodeJS.ErrnoException & { code?: number }).code; + // git diff exits 1 when the files differ — that's the normal case. + if (code === 1) { + resolve(stdout); + return; + } + reject(error instanceof Error ? error : new Error('git diff failed')); + return; + } + resolve(stdout); + }); + }); + } finally { + await rm(dir, { recursive: true, force: true }); + } +} + /** Update strategy chosen by the user for non-clean cases. */ export type UpdateStrategy = 'default' | 'merge' | 'keep-ours'; diff --git a/packages/tbd/tests/fork-update.test.ts b/packages/tbd/tests/fork-update.test.ts index aa044010..b4ad2e98 100644 --- a/packages/tbd/tests/fork-update.test.ts +++ b/packages/tbd/tests/fork-update.test.ts @@ -4,7 +4,12 @@ import { describe, it, expect } from 'vitest'; -import { mergeContents, updateOne, type UpdateStrategy } from '../src/file/fork-update.js'; +import { + mergeContents, + diffContents, + updateOne, + type UpdateStrategy, +} from '../src/file/fork-update.js'; import { type ForkEntry, hashContent, hasConflictMarkers } from '../src/file/fork-manifest.js'; const BASE = 'line one\nline two\nline three\n'; @@ -38,6 +43,23 @@ describe('mergeContents', () => { }); }); +describe('diffContents', () => { + it('returns empty for identical content', async () => { + expect(await diffContents(BASE, BASE)).toBe(''); + }); + + it('shows changed lines with the given labels', async () => { + const diff = await diffContents(BASE, 'line one\nCHANGED\nline three\n', { + left: 'upstream', + right: 'ours', + }); + expect(diff).toContain('--- upstream'); + expect(diff).toContain('+++ ours'); + expect(diff).toContain('+CHANGED'); + expect(diff).toContain('-line two'); + }); +}); + describe('updateOne decision table', () => { const EDITED = 'line ONE\nline two\nline three\n'; // diverges from BASE on line 1 const UPSTREAM_NONCONFLICT = 'line one\nline two\nline THREE\n'; // line 3 From 118cd4a80b0a04501e04ec164e74db7ca84d93fc Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 12 Jun 2026 09:43:49 +0000 Subject: [PATCH 11/36] feat: Add tbd docs list (cross-kind listing with state markers) tbd docs list [--kind]: lists docs across guideline/shortcut/template grouped by kind, with dim [forked] / [forked, customized] / [local] markers (matching the existing [shadowed] convention) and the standard two-line name + size / title: description format. --json emits a docmap. Forks are reflected because listing uses the serving lookup paths (fork dir prepended). Partial tbd-wzqp (list done; kind-agnostic show and the per-kind --list -> docmap renderer migration remain). https://claude.ai/code/session_01X8S12JzmmxEfLpYzgH8Y7E --- packages/tbd/src/cli/commands/docs-fork.ts | 107 +++++++++++++++++++++ 1 file changed, 107 insertions(+) diff --git a/packages/tbd/src/cli/commands/docs-fork.ts b/packages/tbd/src/cli/commands/docs-fork.ts index f968acdd..9f61edb8 100644 --- a/packages/tbd/src/cli/commands/docs-fork.ts +++ b/packages/tbd/src/cli/commands/docs-fork.ts @@ -23,9 +23,13 @@ import { CACHE_GUIDELINES_PATHS, CACHE_SHORTCUT_PATHS, CACHE_TEMPLATE_PATHS, + DEFAULT_GUIDELINES_PATHS, + DEFAULT_SHORTCUT_PATHS, + DEFAULT_TEMPLATE_PATHS, FORK_DIR, TBD_DOCS_DIR, } from '../../lib/paths.js'; +import { formatDocSize } from '../../lib/format-utils.js'; import { type ForkEntry, type ForkKind, @@ -457,6 +461,98 @@ class DocsUpdateHandler extends BaseCommand { } } +/** Serving lookup paths per kind (fork dir prepended, so forks are reflected). */ +const KIND_SERVE_PATHS: Record = { + guideline: DEFAULT_GUIDELINES_PATHS, + shortcut: DEFAULT_SHORTCUT_PATHS, + template: DEFAULT_TEMPLATE_PATHS, +}; + +interface ListOptions { + kind?: string; + json?: boolean; +} + +class DocsListHandler extends BaseCommand { + async run(options: ListOptions): Promise { + await this.execute(async () => { + const tbdRoot = await requireInit(); + const manifest = await readForkManifest(tbdRoot); + const kinds = options.kind ? [options.kind as ForkKind] : RESOLVABLE_KINDS; + const colors = this.output.getColors(); + + interface Row { + name: string; + title?: string; + description?: string; + sizeInfo: string; + marker: string; + state: string; + path: string; + } + const grouped: { kind: ForkKind; rows: Row[] }[] = []; + const docmapEntries: DocMapEntry[] = []; + + for (const kind of kinds) { + const cache = new DocCache(KIND_SERVE_PATHS[kind] ?? [], tbdRoot); + await cache.load({ quiet: true }); + const rows: Row[] = []; + for (const doc of cache.list()) { + const fork = findFork(manifest, doc.name, kind); + const isLocal = !fork && doc.sourceDir.startsWith(FORK_DIR); + let state = 'upstream'; + let marker = ''; + if (fork) { + const customized = hashContent(doc.content) !== fork.base_hash; + state = customized ? 'customized' : 'forked'; + marker = customized ? '[forked, customized]' : '[forked]'; + } else if (isLocal) { + state = 'local'; + marker = '[local]'; + } + rows.push({ + name: doc.name, + title: doc.frontmatter?.title, + description: doc.frontmatter?.description, + sizeInfo: formatDocSize(doc.sizeBytes, doc.approxTokens), + marker, + state, + path: fork?.path ?? doc.sourceDir + '/' + doc.name + '.md', + }); + docmapEntries.push({ + name: doc.name, + type: kind, + path: fork?.path, + source: fork?.source, + title: doc.frontmatter?.title, + description: doc.frontmatter?.description, + state, + }); + } + grouped.push({ kind, rows }); + } + + if (this.ctx.json) { + this.output.data(createDocMap(docmapEntries, { name: 'tbd-docs' })); + return; + } + + for (const { kind, rows } of grouped) { + if (rows.length === 0) continue; + if (!options.kind) console.log(colors.bold(kind)); + for (const r of rows) { + const indent = options.kind ? '' : ' '; + const markerStr = r.marker ? ` ${colors.dim(r.marker)}` : ''; + console.log(`${indent}${colors.bold(r.name)} ${colors.dim(r.sizeInfo)}${markerStr}`); + const desc = + r.title && r.description ? `${r.title}: ${r.description}` : (r.title ?? r.description); + if (desc) console.log(`${indent} ${desc}`); + } + } + }, 'Failed to list docs'); + } +} + interface DiffOptions { base?: boolean; upstream?: boolean; @@ -563,6 +659,17 @@ export function registerForkSubcommands(docs: Command): void { }); }); + docs + .command('list') + .description('List all docs across kinds, with [forked]/[customized]/[local] markers') + .option('--kind ', 'restrict to a kind (guideline|shortcut|template)') + .action(async (options: ListOptions, command: Command) => { + await new DocsListHandler(command).run({ + kind: options.kind, + json: command.optsWithGlobals().json === true, + }); + }); + docs .command('status') .description('Show forked docs and their states') From 1a4940fb51f8be38bd3f9aad78a891d331472499 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 12 Jun 2026 15:42:42 +0000 Subject: [PATCH 12/36] feat: Bump tbd_format to f05 with seamless layout co-migration Land the forkable-docs format gate (plan-2026-06-11-forkable-docs.md): - f05 in FORMAT_HISTORY + migrate_f04_to_f05 (metadata-only stamp; fork artifacts appear lazily when `tbd docs fork` is first used). The history entry documents the revert recipe: restore .tbd/config.yml and delete $GIT_COMMON_DIR/tbd/layout.yml (it regenerates from the config). - New layout co-migration: an older-but-compatible layout.yml next to a newer-format config is the normal mid-migration state, not an error. isLayoutUpgradeable() + ensureCommonDirLayout re-stamp it in place (preserving created_at) under the shared lock, and the data-context probe routes that state through the locked ensure path instead of failing validation. This is what f03->f04 never needed (no layout predated f04) and what makes the f04->f05 upgrade a clean single command. - Old-client contract: 0.2.3-era clients refuse f05 repos with the standard upgrade message (verified against the published 0.2.3 binary). Tests: new f04->f05 migrate->revert->repeat e2e (two rounds + steady-state silence), read-path layout upgrade preserving created_at, metadata-only stamp unit test, old-client gate (isFormatCompatibleWithSupported), and all format-asserting goldens updated to f05. Full suite: 1262 vitest + 863 tryscript tests pass. Implements tbd-z1b5, tbd-ns1b. https://claude.ai/code/session_01X8S12JzmmxEfLpYzgH8Y7E --- packages/tbd/src/cli/lib/data-context.ts | 10 +- packages/tbd/src/file/common-dir-layout.ts | 33 ++++++- packages/tbd/src/lib/tbd-format.ts | 51 +++++++++- packages/tbd/tests/cli-advanced.tryscript.md | 2 +- .../cli-format-compatibility.tryscript.md | 2 +- ...li-shared-common-dir-worktree.tryscript.md | 22 ++--- .../tests/common-dir-layout-doctor.test.ts | 94 +++++++++++++++---- packages/tbd/tests/config.test.ts | 2 +- packages/tbd/tests/setup-flows.test.ts | 4 +- packages/tbd/tests/tbd-format.test.ts | 78 +++++++++++---- 10 files changed, 243 insertions(+), 55 deletions(-) diff --git a/packages/tbd/src/cli/lib/data-context.ts b/packages/tbd/src/cli/lib/data-context.ts index 1da8b2c0..42c828ae 100644 --- a/packages/tbd/src/cli/lib/data-context.ts +++ b/packages/tbd/src/cli/lib/data-context.ts @@ -28,6 +28,7 @@ import { checkWorktreeHealth, repairWorktree } from '../../file/git.js'; import type { WorktreeHealth, WorktreeStatus } from '../../file/git.js'; import { ensureCommonDirLayout, + isLayoutUpgradeable, readCommonDirLayout, validateCommonDirLayout, withSharedDataSyncLock, @@ -94,13 +95,18 @@ async function probeDataSyncReadiness(tbdRoot: string): Promise { const { config, migrated, fromFormat } = await readConfigWithMigration(tbdRoot); const sharedPaths = await resolveSharedTbdPaths(tbdRoot); const layout = await readCommonDirLayout(sharedPaths.sharedLayoutPath); - if (layout) { + // A layout stamped with an older (compatible) format than the config is the + // normal mid-migration state — it is upgraded under the lock by + // ensureCommonDirLayout, so it must not fail validation here, only mark the + // probe as not ready. + const layoutNeedsUpgrade = layout !== null && isLayoutUpgradeable(layout, config); + if (layout && !layoutNeedsUpgrade) { // Validate eagerly even on the read path so future-format / mismatched // layouts fail closed before any I/O the caller might perform. validateCommonDirLayout(layout, config); } const health = await checkWorktreeHealth(tbdRoot, config.sync.branch); - const ready = !migrated && layout !== null && health.valid; + const ready = !migrated && layout !== null && !layoutNeedsUpgrade && health.valid; return { config, migrated, fromFormat, sharedPaths, layout, health, ready }; } diff --git a/packages/tbd/src/file/common-dir-layout.ts b/packages/tbd/src/file/common-dir-layout.ts index e30092a9..aa14e795 100644 --- a/packages/tbd/src/file/common-dir-layout.ts +++ b/packages/tbd/src/file/common-dir-layout.ts @@ -14,7 +14,14 @@ import { resolveSharedTbdPaths, type SharedTbdPaths, } from '../lib/paths.js'; -import { CURRENT_FORMAT, formatUpgradeMessage, isCompatibleFormat } from '../lib/tbd-format.js'; +import { + CURRENT_FORMAT, + formatUpgradeMessage, + isCompatibleFormat, + isFormatCompatibleWithSupported, + FORMAT_HISTORY, + type FormatVersion, +} from '../lib/tbd-format.js'; import { sortKeys, stringifyYaml } from '../utils/yaml-utils.js'; import { now } from '../utils/time-utils.js'; import { DATA_SYNC_LOCK_OPTIONS, withLockfile } from '../utils/lockfile.js'; @@ -48,6 +55,24 @@ export async function readCommonDirLayout(layoutPath: string): Promise { const existing = await readCommonDirLayout(paths.sharedLayoutPath); if (existing) { + // An older-format layout next to a newer-format config is the normal + // mid-migration state: re-stamp it from the config (preserving created_at) + // instead of failing validation. See isLayoutUpgradeable. + if (isLayoutUpgradeable(existing, config)) { + return writeCommonDirLayout(paths, config, existing); + } validateCommonDirLayout(existing, config); return existing; } diff --git a/packages/tbd/src/lib/tbd-format.ts b/packages/tbd/src/lib/tbd-format.ts index 77839a7a..e44f9689 100644 --- a/packages/tbd/src/lib/tbd-format.ts +++ b/packages/tbd/src/lib/tbd-format.ts @@ -41,7 +41,7 @@ * Current format version. * Bump this ONLY for breaking changes that require migration. */ -export const CURRENT_FORMAT = 'f04'; +export const CURRENT_FORMAT = 'f05'; /** * Initial format version for configs that don't have tbd_format field. @@ -99,6 +99,20 @@ export const FORMAT_HISTORY = { migration: 'Initializes shared common-dir sync layout before writing config.yml with tbd_format f04', }, + f05: { + introduced: '0.3.0', + description: 'Adds the forkable-docs layout (committed fork manifest + visible fork dir)', + changes: [ + 'Added committed fork state under .tbd/doc-forks/ (forks.yml manifest + base/ snapshots)', + 'Added the visible fork dir (docs/tbd/) which shadows the doc cache in lookups', + 'Doc commands (tbd docs fork/unfork/status/update/diff/list) manage that state', + ], + migration: + 'Metadata-only: stamps tbd_format f05 (fork artifacts appear only when fork is first ' + + 'used). The shared common-dir layout.yml is re-stamped in place on the next data ' + + 'command. Revert: restore .tbd/config.yml (git checkout) and delete ' + + '$GIT_COMMON_DIR/tbd/layout.yml; it regenerates from the config.', + }, } as const; export type FormatVersion = keyof typeof FORMAT_HISTORY; @@ -262,6 +276,29 @@ function migrate_f03_to_f04(config: RawConfig): MigrationResult { }; } +/** + * Migrate from f04 to f05. + * - Metadata-only stamp: the forkable-docs artifacts (.tbd/doc-forks/, docs/tbd/) + * are created lazily by `tbd docs fork`, not by migration. + * - The bump gates older clients: they must not serve upstream copies of docs that + * this repo has forked and customized. + */ +function migrate_f04_to_f05(config: RawConfig): MigrationResult { + const changes: string[] = []; + const migrated = { ...config }; + + migrated.tbd_format = 'f05'; + changes.push('Updated tbd_format: f05'); + + return { + config: migrated, + fromFormat: 'f04', + toFormat: 'f05', + changed: changes.length > 0, + changes, + }; +} + // ============================================================================= // Public API // ============================================================================= @@ -339,6 +376,13 @@ export function migrateToLatest(config: RawConfig): MigrationResult { allChanges.push(...result.changes); } + if (currentFormat === 'f04') { + const result = migrate_f04_to_f05(current); + current = result.config; + currentFormat = 'f05' as FormatVersion; + allChanges.push(...result.changes); + } + return { config: current, fromFormat, @@ -417,5 +461,10 @@ export function describeMigration(fromFormat: FormatVersion): string[] { current = 'f04'; } + if (current === 'f04') { + descriptions.push('f04 → f05: Add forkable-docs layout (stamp only)'); + current = 'f05'; + } + return descriptions; } diff --git a/packages/tbd/tests/cli-advanced.tryscript.md b/packages/tbd/tests/cli-advanced.tryscript.md index 143c1c83..7626d4ee 100644 --- a/packages/tbd/tests/cli-advanced.tryscript.md +++ b/packages/tbd/tests/cli-advanced.tryscript.md @@ -275,7 +275,7 @@ settings: ```console $ tbd config show --json { - "tbd_format": "f04", + "tbd_format": "f05", "tbd_version": "[..]", "sync": { "branch": "tbd-sync", diff --git a/packages/tbd/tests/cli-format-compatibility.tryscript.md b/packages/tbd/tests/cli-format-compatibility.tryscript.md index bd3d9bd7..5e15abca 100644 --- a/packages/tbd/tests/cli-format-compatibility.tryscript.md +++ b/packages/tbd/tests/cli-format-compatibility.tryscript.md @@ -44,7 +44,7 @@ Golden tests for fail-closed format compatibility behavior. $ tbd list 2>&1 Error: This repository requires a newer version of tbd. Config format 'f99' is from a newer tbd version. -This tbd version supports up to format 'f04'. +This tbd version supports up to format 'f05'. Upgrade tbd: npm install -g get-tbd@latest ? 1 ``` diff --git a/packages/tbd/tests/cli-shared-common-dir-worktree.tryscript.md b/packages/tbd/tests/cli-shared-common-dir-worktree.tryscript.md index 1702d077..e374cb1a 100644 --- a/packages/tbd/tests/cli-shared-common-dir-worktree.tryscript.md +++ b/packages/tbd/tests/cli-shared-common-dir-worktree.tryscript.md @@ -86,7 +86,7 @@ design: 1. An f03 per-checkout sync worktree owns `tbd-sync`. 2. A linked worktree cannot create its own per-checkout sync worktree because Git rejects the duplicate branch checkout. -3. A new tbd write migrates the repository to the f04 common-dir layout. +3. A new tbd write migrates the repository to the current (f05) common-dir layout. 4. The main checkout and linked worktree both create issues through the same shared sync worktree. @@ -119,17 +119,17 @@ own point-of-use notice (#135). ```console $ tbd create "Main checkout issue" --type=task -• tbd_format f03 → f04: .tbd/config.yml updated in this checkout. Commit on this branch or merge main to publish the format upgrade. +• tbd_format f03 → f05: .tbd/config.yml updated in this checkout. Commit on this branch or merge main to publish the format upgrade. • tbd-sync worktree was missing — auto-materialized it (fresh clone, or the worktree was removed). ✓ Created test-[SHORTID]: Main checkout issue ? 0 ``` -# Test: Top-level config was migrated to f04 with common-dir storage +# Test: Top-level config was migrated to f05 with common-dir storage ```console $ cat .tbd/config.yml -tbd_format: f04 +tbd_format: f05 tbd_version: legacy display: id_prefix: test @@ -144,11 +144,11 @@ settings: ? 0 ``` -# Test: Common-dir layout uses the same f04 format ID +# Test: Common-dir layout uses the same f05 format ID ```console $ cat "$(git rev-parse --path-format=absolute --git-common-dir)/tbd/layout.yml" -tbd_format: f04 +tbd_format: f05 sync_storage: git-common-dir-v1 data_sync_worktree: data-sync-worktree lock_profile: data-sync-v1 @@ -187,13 +187,13 @@ same git common dir # Test: Linked worktree create also succeeds -The linked worktree’s `.tbd/config.yml` is still on `f03` because the f04 bump commit +The linked worktree’s `.tbd/config.yml` is still on `f03` because the format bump commit only landed on `main`; the first mutating command in the linked checkout therefore fires the same `tbd-afjh` notice as the main checkout did before bumping in place. ```console $ (cd agent && tbd create "Linked worktree issue" --type=bug) -• tbd_format f03 → f04: .tbd/config.yml updated in this checkout. Commit on this branch or merge main to publish the format upgrade. +• tbd_format f03 → f05: .tbd/config.yml updated in this checkout. Commit on this branch or merge main to publish the format upgrade. ✓ Created test-[SHORTID]: Linked worktree issue ? 0 ``` @@ -231,15 +231,15 @@ Main checkout issue | task | open ## Older Client Compatibility Guard -# Test: An f03-era client would reject the migrated f04 repository +# Test: An f03-era client would reject the migrated f05 repository This uses the same format ordering contract as tbd itself: a client that only supports -up to f03 must fail closed when it sees the f04 common-dir layout. +up to f03 must fail closed when it sees the f05 common-dir layout. ```console $ node -e 'const fs=require("fs"); const format=fs.readFileSync(".tbd/config.yml","utf8").match(/^tbd_format: (\S+)/m)?.[1]; const supported="f03"; if (format !== undefined && format > supported) { console.error("This repository requires a newer version of tbd."); console.error("Config format '"'"'"+format+"'"'"' is from a newer tbd version."); console.error("This tbd version supports up to format '"'"'"+supported+"'"'"'."); console.error("Upgrade tbd: npm install -g get-tbd@latest"); process.exit(1); }' This repository requires a newer version of tbd. -Config format 'f04' is from a newer tbd version. +Config format 'f05' is from a newer tbd version. This tbd version supports up to format 'f03'. Upgrade tbd: npm install -g get-tbd@latest ? 1 diff --git a/packages/tbd/tests/common-dir-layout-doctor.test.ts b/packages/tbd/tests/common-dir-layout-doctor.test.ts index 637d58bc..f9ece2a9 100644 --- a/packages/tbd/tests/common-dir-layout-doctor.test.ts +++ b/packages/tbd/tests/common-dir-layout-doctor.test.ts @@ -6,7 +6,7 @@ * surfaces the future-format upgrade message instead of attempting repair. * - H3: tbd doctor surfaces an IncompatibleFormatError config as a newer-tbd * error instead of a generic "Invalid config file". - * - H1: a read command on an f04 repo with missing layout.yml regenerates the + * - H1: a read command on an f04+ repo with missing layout.yml regenerates the * layout under the shared lock and never writes a direct .tbd/data-sync/ path. */ @@ -100,10 +100,10 @@ describeUnlessWindows('common-dir layout via CLI', { timeout: 30000 }, () => { it('repairs a layout/config tbd_format mismatch under the shared lock', async () => { const layoutPath = join(dir, '.git', 'tbd', 'layout.yml'); const original = await readFile(layoutPath, 'utf-8'); - expect(original).toContain('tbd_format: f04'); + expect(original).toContain('tbd_format: f05'); // Simulate a partial migration / manual edit by downgrading the layout. - await writeFile(layoutPath, original.replace('tbd_format: f04', 'tbd_format: f03')); + await writeFile(layoutPath, original.replace('tbd_format: f05', 'tbd_format: f03')); // Plain doctor reports it as fixable. The mismatch is a ✗ finding so the // exit is 1 (per tbd-r7rt). @@ -116,20 +116,20 @@ describeUnlessWindows('common-dir layout via CLI', { timeout: 30000 }, () => { const fix = runTbd(dir, ['doctor', '--fix']); expect(fix.status).toBe(0); const repaired = await readFile(layoutPath, 'utf-8'); - expect(repaired).toContain('tbd_format: f04'); + expect(repaired).toContain('tbd_format: f05'); }); it('surfaces future-format layout as needing a newer tbd (no fix attempted)', async () => { const layoutPath = join(dir, '.git', 'tbd', 'layout.yml'); const original = await readFile(layoutPath, 'utf-8'); - await writeFile(layoutPath, original.replace('tbd_format: f04', 'tbd_format: f99')); + await writeFile(layoutPath, original.replace('tbd_format: f05', 'tbd_format: f99')); const fix = runTbd(dir, ['doctor', '--fix']); // Future-format markers are ✗ findings: scripts/CI must see exit 1 (tbd-r7rt). expect(fix.status).toBe(1); const out = fix.stdout + fix.stderr; expect(out).toMatch(/newer tbd|f99/i); - // Layout was not silently rewritten back to f04. + // Layout was not silently rewritten back to f05. const layout = await readFile(layoutPath, 'utf-8'); expect(layout).toContain('tbd_format: f99'); }); @@ -137,7 +137,7 @@ describeUnlessWindows('common-dir layout via CLI', { timeout: 30000 }, () => { it('surfaces future-format config as a newer-tbd error in checkConfig', async () => { const configPath = join(dir, '.tbd', 'config.yml'); const original = await readFile(configPath, 'utf-8'); - await writeFile(configPath, original.replace('tbd_format: f04', 'tbd_format: f99')); + await writeFile(configPath, original.replace('tbd_format: f05', 'tbd_format: f99')); const out = runTbd(dir, ['doctor']); // Future-format config is a ✗ finding: exit 1 (tbd-r7rt). @@ -174,7 +174,7 @@ describeUnlessWindows('common-dir layout via CLI', { timeout: 30000 }, () => { expect(await exists(worktreePath)).toBe(true); expect(await exists(layoutPath)).toBe(true); const layout = await readFile(layoutPath, 'utf-8'); - expect(layout).toContain('tbd_format: f04'); + expect(layout).toContain('tbd_format: f05'); }); it('serializes concurrent doctor --fix init under the shared data-sync lock (tbd-p6zo)', async () => { @@ -207,7 +207,7 @@ describeUnlessWindows('common-dir layout via CLI', { timeout: 30000 }, () => { expect(await exists(worktreePath)).toBe(true); expect(await exists(layoutPath)).toBe(true); const layout = await readFile(layoutPath, 'utf-8'); - expect(layout).toContain('tbd_format: f04'); + expect(layout).toContain('tbd_format: f05'); const worktreeList = await gitIn(dir, 'worktree', 'list', '--porcelain'); const sharedWorktreeLines = worktreeList @@ -221,21 +221,21 @@ describeUnlessWindows('common-dir layout via CLI', { timeout: 30000 }, () => { it('prints a one-time stderr notice when this checkout migrates .tbd/config.yml to a newer tbd_format', async () => { const configPath = join(dir, '.tbd', 'config.yml'); const original = await readFile(configPath, 'utf-8'); - // The setup is f04; "downgrade" the on-disk format marker so the next mutating + // The setup is f05; "downgrade" the on-disk format marker so the next mutating // command sees a stale per-checkout config and migrates it back in place. This // matches a real sibling worktree on a branch that did not yet pick up the // main checkout's f03 → f04 commit. - await writeFile(configPath, original.replace('tbd_format: f04', 'tbd_format: f03')); + await writeFile(configPath, original.replace('tbd_format: f05', 'tbd_format: f03')); const create = runTbd(dir, ['create', 'sibling-bump probe', '--type', 'task', '--no-sync']); expect(create.status).toBe(0); // The notice goes to stderr so it cannot pollute JSON output on stdout. expect(create.stderr).toContain('tbd_format'); - expect(create.stderr).toContain('→ f04'); + expect(create.stderr).toContain('→ f05'); expect(create.stderr).toMatch(/commit on this branch or merge main/i); - // The on-disk config is now back at f04 — the migration ran. + // The on-disk config is now at f05 — the migration ran. const after = await readFile(configPath, 'utf-8'); - expect(after).toContain('tbd_format: f04'); + expect(after).toContain('tbd_format: f05'); // Second mutating call must NOT re-emit the notice: nothing left to migrate. const second = runTbd(dir, ['create', 'sibling-bump probe 2', '--type', 'task', '--no-sync']); @@ -244,6 +244,66 @@ describeUnlessWindows('common-dir layout via CLI', { timeout: 30000 }, () => { }); }); + describe('f04 → f05 upgrade (forkable-docs gate)', () => { + it('upgrades config and layout in place; the loop is revertible and repeatable', async () => { + const configPath = join(dir, '.tbd', 'config.yml'); + const layoutPath = join(dir, '.git', 'tbd', 'layout.yml'); + + const migrateOnceFromF04 = async (round: number) => { + // Rewind both files to the genuine pre-upgrade state — exactly what + // reverting the config bump commit + downgrading (or deleting) the + // machine-local layout looks like. + await writeFile( + configPath, + (await readFile(configPath, 'utf-8')).replace('tbd_format: f05', 'tbd_format: f04'), + ); + await writeFile( + layoutPath, + (await readFile(layoutPath, 'utf-8')).replace('tbd_format: f05', 'tbd_format: f04'), + ); + + // A plain data command must succeed (NOT fail with a layout/config + // mismatch), migrate the config, re-stamp the layout, and emit the + // one-time migration notice. + const create = runTbd(dir, [ + 'create', + `upgrade probe ${round}`, + '--type', + 'task', + '--no-sync', + ]); + expect(create.status).toBe(0); + expect(create.stderr).toContain('f04 → f05'); + expect(await readFile(configPath, 'utf-8')).toContain('tbd_format: f05'); + expect(await readFile(layoutPath, 'utf-8')).toContain('tbd_format: f05'); + }; + + await migrateOnceFromF04(1); + // Revert and repeat: migrating from the restored f04 state is idempotent. + await migrateOnceFromF04(2); + + // Steady state afterwards: no further migration notices. + const steady = runTbd(dir, ['create', 'steady probe', '--type', 'task', '--no-sync']); + expect(steady.status).toBe(0); + expect(steady.stderr).not.toContain('tbd_format'); + }); + + it('read commands upgrade an older layout under the lock, preserving created_at', async () => { + // Config already f05 (e.g. a teammate committed the bump) but this + // machine's layout is still f04: a read must auto-upgrade, not error. + const layoutPath = join(dir, '.git', 'tbd', 'layout.yml'); + const before = await readFile(layoutPath, 'utf-8'); + const createdAt = before.split('\n').find((l) => l.startsWith('created_at:')); + await writeFile(layoutPath, before.replace('tbd_format: f05', 'tbd_format: f04')); + + const list = runTbd(dir, ['list', '--json']); + expect(list.status).toBe(0); + const after = await readFile(layoutPath, 'utf-8'); + expect(after).toContain('tbd_format: f05'); + expect(after).toContain(createdAt); + }); + }); + describe('read fast-path (H1)', () => { it('regenerates a missing layout.yml on first read without writing direct data path', async () => { const layoutPath = join(dir, '.git', 'tbd', 'layout.yml'); @@ -258,7 +318,7 @@ describeUnlessWindows('common-dir layout via CLI', { timeout: 30000 }, () => { ); const directDataSync = join(dir, '.tbd', 'data-sync', 'issues'); - // Simulate an f04 checkout where layout.yml has not been initialized yet. + // Simulate a checkout where layout.yml has not been initialized yet. await rm(layoutPath); expect(await exists(layoutPath)).toBe(false); @@ -267,9 +327,9 @@ describeUnlessWindows('common-dir layout via CLI', { timeout: 30000 }, () => { expect(list.status).toBe(0); expect(await exists(layoutPath)).toBe(true); const layout = await readFile(layoutPath, 'utf-8'); - expect(layout).toContain('tbd_format: f04'); + expect(layout).toContain('tbd_format: f05'); - // No direct .tbd/data-sync/ leakage: f04 must fail closed, not fall back. + // No direct .tbd/data-sync/ leakage: f04+ must fail closed, not fall back. expect(await exists(sharedDataSync)).toBe(true); expect(await exists(directDataSync)).toBe(false); }); diff --git a/packages/tbd/tests/config.test.ts b/packages/tbd/tests/config.test.ts index 5dc035b3..67b522bb 100644 --- a/packages/tbd/tests/config.test.ts +++ b/packages/tbd/tests/config.test.ts @@ -90,7 +90,7 @@ describe('config operations', () => { await expect(readConfig(tempDir)).rejects.toThrow( 'This repository requires a newer version of tbd.\n' + "Config format 'f99' is from a newer tbd version.\n" + - "This tbd version supports up to format 'f04'.\n" + + "This tbd version supports up to format 'f05'.\n" + 'Upgrade tbd: npm install -g get-tbd@latest', ); }); diff --git a/packages/tbd/tests/setup-flows.test.ts b/packages/tbd/tests/setup-flows.test.ts index 5cd67958..f1d982a6 100644 --- a/packages/tbd/tests/setup-flows.test.ts +++ b/packages/tbd/tests/setup-flows.test.ts @@ -161,7 +161,7 @@ describe('setup flows', { timeout: setupFlowTestTimeout }, () => { expect(result.status).toBe(0); const agents = await readFile(join(tempDir, 'AGENTS.md'), 'utf-8'); - expect(agents).toContain(''); + expect(agents).toContain(''); expect(agents).toContain('tbd prime'); const block = agents.slice( @@ -259,7 +259,7 @@ describe('setup flows', { timeout: setupFlowTestTimeout }, () => { const agents = await readFile(join(tempDir, 'AGENTS.md'), 'utf-8'); // Upgraded to the versioned compact block... - expect(agents).toContain('format=f04'); + expect(agents).toContain('format=f05'); // ...while preserving user content outside the managed region. expect(agents).toContain('## My Notes'); expect(agents).toContain('Keep me.'); diff --git a/packages/tbd/tests/tbd-format.test.ts b/packages/tbd/tests/tbd-format.test.ts index 365d8ad3..5947f659 100644 --- a/packages/tbd/tests/tbd-format.test.ts +++ b/packages/tbd/tests/tbd-format.test.ts @@ -20,7 +20,7 @@ import { describe('tbd-format', () => { describe('constants', () => { it('has current format', () => { - expect(CURRENT_FORMAT).toBe('f04'); + expect(CURRENT_FORMAT).toBe('f05'); }); it('has initial format', () => { @@ -32,6 +32,7 @@ describe('tbd-format', () => { expect(FORMAT_HISTORY.f02).toBeDefined(); expect(FORMAT_HISTORY.f03).toBeDefined(); expect(FORMAT_HISTORY.f04).toBeDefined(); + expect(FORMAT_HISTORY.f05).toBeDefined(); }); }); @@ -78,7 +79,7 @@ describe('tbd-format', () => { }); describe('migrateToLatest', () => { - it('migrates f01 to f04 through all format steps', () => { + it('migrates f01 to f05 through all format steps', () => { const config: RawConfig = { tbd_version: '0.1.0', display: { id_prefix: 'test' }, @@ -89,9 +90,9 @@ describe('tbd-format', () => { const result = migrateToLatest(config); expect(result.fromFormat).toBe('f01'); - expect(result.toFormat).toBe('f04'); + expect(result.toFormat).toBe('f05'); expect(result.changed).toBe(true); - expect(result.config.tbd_format).toBe('f04'); + expect(result.config.tbd_format).toBe('f05'); expect(result.config.sync?.storage).toBe('git-common-dir-v1'); expect(result.config.settings?.doc_auto_sync_hours).toBe(24); expect(result.changes).toContain('Added tbd_format: f02'); @@ -99,9 +100,10 @@ describe('tbd-format', () => { expect(result.changes).toContain('Updated tbd_format: f03'); expect(result.changes).toContain('Updated tbd_format: f04'); expect(result.changes).toContain('Added sync.storage: git-common-dir-v1'); + expect(result.changes).toContain('Updated tbd_format: f05'); }); - it('migrates f02 to f04', () => { + it('migrates f02 to f05', () => { const config: RawConfig = { tbd_format: 'f02', tbd_version: '0.1.5', @@ -114,9 +116,9 @@ describe('tbd-format', () => { const result = migrateToLatest(config); expect(result.fromFormat).toBe('f02'); - expect(result.toFormat).toBe('f04'); + expect(result.toFormat).toBe('f05'); expect(result.changed).toBe(true); - expect(result.config.tbd_format).toBe('f04'); + expect(result.config.tbd_format).toBe('f05'); expect(result.config.sync?.storage).toBe('git-common-dir-v1'); // doc_cache moved to docs_cache.files expect(result.config.doc_cache).toBeUndefined(); @@ -133,8 +135,8 @@ describe('tbd-format', () => { it('does not modify already current config', () => { const config: RawConfig = { - tbd_format: 'f04', - tbd_version: '0.2.0', + tbd_format: 'f05', + tbd_version: '0.3.0', sync: { branch: 'tbd-sync', remote: 'origin', storage: 'git-common-dir-v1' }, display: { id_prefix: 'test' }, settings: { auto_sync: false, doc_auto_sync_hours: 12 }, @@ -146,15 +148,15 @@ describe('tbd-format', () => { const result = migrateToLatest(config); - expect(result.fromFormat).toBe('f04'); - expect(result.toFormat).toBe('f04'); + expect(result.fromFormat).toBe('f05'); + expect(result.toFormat).toBe('f05'); expect(result.changed).toBe(false); expect(result.changes).toHaveLength(0); expect(result.config.settings?.doc_auto_sync_hours).toBe(12); expect(result.config.sync?.storage).toBe('git-common-dir-v1'); }); - it('migrates f03 to f04 by adding sync storage marker', () => { + it('migrates f03 through f04 (sync storage marker) to f05', () => { const config: RawConfig = { tbd_format: 'f03', tbd_version: '0.1.6', @@ -166,9 +168,9 @@ describe('tbd-format', () => { const result = migrateToLatest(config); expect(result.fromFormat).toBe('f03'); - expect(result.toFormat).toBe('f04'); + expect(result.toFormat).toBe('f05'); expect(result.changed).toBe(true); - expect(result.config.tbd_format).toBe('f04'); + expect(result.config.tbd_format).toBe('f05'); expect(result.config.sync).toEqual({ branch: 'custom-sync', remote: 'upstream', @@ -176,6 +178,29 @@ describe('tbd-format', () => { }); }); + it('migrates f04 to f05 as a metadata-only stamp', () => { + const config: RawConfig = { + tbd_format: 'f04', + tbd_version: '0.2.3', + display: { id_prefix: 'test' }, + sync: { branch: 'tbd-sync', remote: 'origin', storage: 'git-common-dir-v1' }, + settings: { auto_sync: false, doc_auto_sync_hours: 24 }, + docs_cache: { + files: { 'guidelines/x.md': 'internal:guidelines/x.md' }, + lookup_path: ['.tbd/docs/shortcuts/system'], + }, + }; + + const result = migrateToLatest(config); + + expect(result.fromFormat).toBe('f04'); + expect(result.toFormat).toBe('f05'); + expect(result.changed).toBe(true); + expect(result.changes).toEqual(['Updated tbd_format: f05']); + // Stamp only: every other field is preserved verbatim. + expect(result.config).toEqual({ ...config, tbd_format: 'f05' }); + }); + it('preserves existing settings when migrating', () => { const config: RawConfig = { tbd_version: '0.1.0', @@ -210,6 +235,10 @@ describe('tbd-format', () => { expect(isCompatibleFormat('f04')).toBe(true); }); + it('returns true for f05', () => { + expect(isCompatibleFormat('f05')).toBe(true); + }); + it('returns false for unknown future format', () => { expect(isCompatibleFormat('f99')).toBe(false); }); @@ -220,9 +249,14 @@ describe('tbd-format', () => { expect(isFormatCompatibleWithSupported('f04', 'f03')).toBe(false); }); + it('models old f04 clients rejecting f05 repositories (the forkable-docs gate)', () => { + expect(isFormatCompatibleWithSupported('f05', 'f04')).toBe(false); + }); + it('allows old clients to read older formats they know how to migrate', () => { expect(isFormatCompatibleWithSupported('f01', 'f03')).toBe(true); expect(isFormatCompatibleWithSupported('f03', 'f03')).toBe(true); + expect(isFormatCompatibleWithSupported('f04', 'f05')).toBe(true); }); }); @@ -238,23 +272,31 @@ describe('tbd-format', () => { }); describe('describeMigration', () => { - it('describes f01 migration (three steps)', () => { + it('describes f01 migration (four steps)', () => { const descriptions = describeMigration('f01'); - expect(descriptions).toHaveLength(3); + expect(descriptions).toHaveLength(4); expect(descriptions[0]).toContain('f01 → f02'); expect(descriptions[1]).toContain('f02 → f03'); expect(descriptions[2]).toContain('f03 → f04'); + expect(descriptions[3]).toContain('f04 → f05'); }); it('describes f02 migration', () => { const descriptions = describeMigration('f02'); - expect(descriptions).toHaveLength(2); + expect(descriptions).toHaveLength(3); expect(descriptions[0]).toContain('f02 → f03'); expect(descriptions[1]).toContain('f03 → f04'); + expect(descriptions[2]).toContain('f04 → f05'); }); - it('returns empty for current format', () => { + it('describes f04 migration (one step)', () => { const descriptions = describeMigration('f04'); + expect(descriptions).toHaveLength(1); + expect(descriptions[0]).toContain('f04 → f05'); + }); + + it('returns empty for current format', () => { + const descriptions = describeMigration('f05'); expect(descriptions).toHaveLength(0); }); }); From d96071de9dd4ab8d4c84cb3fd6bde36466848e28 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 12 Jun 2026 15:42:49 +0000 Subject: [PATCH 13/36] chore: Migrate this repo to tbd_format f05 The publish step of the f05 bump: .tbd/config.yml stamped f05 (via the migration itself) and the generated agent surfaces regenerated with the format=f05 marker via `tbd setup --auto`. Verified the full loop on this repo before committing: migrate -> new build works (fork/unfork/status/list/sync) -> published tbd 0.2.3 refuses with the upgrade message -> revert (git checkout .tbd/config.yml + rm layout.yml) -> 0.2.3 works again -> re-migrate -> steady state silent. Note: f04-era clients (including npx get-tbd@0.2.3, which the session hook uses) will refuse this branch until upgraded - that is the gate working. https://claude.ai/code/session_01X8S12JzmmxEfLpYzgH8Y7E --- .agents/skills/tbd/SKILL.md | 2 +- .claude/skills/tbd/SKILL.md | 2 +- .tbd/config.yml | 8 ++++---- AGENTS.md | 2 +- 4 files changed, 7 insertions(+), 7 deletions(-) diff --git a/.agents/skills/tbd/SKILL.md b/.agents/skills/tbd/SKILL.md index a1f8bb5c..c56b4950 100644 --- a/.agents/skills/tbd/SKILL.md +++ b/.agents/skills/tbd/SKILL.md @@ -6,7 +6,7 @@ description: |- Invoke when user mentions: tbd, beads, bd, shortcuts, issues, bugs, tasks, features, epics, todo, tracking, specs, planning, implementation, validation, guidelines, templates, commit, PR, pull request, code review, testing, TDD, test-driven, golden testing, snapshot testing, TypeScript, Python, Convex, monorepo, cleanup, dead code, refactor, handoff, research, architecture, labels, search, checkout library, source code review, or any workflow shortcut. name: tbd --- - diff --git a/.claude/skills/tbd/SKILL.md b/.claude/skills/tbd/SKILL.md index a1f8bb5c..c56b4950 100644 --- a/.claude/skills/tbd/SKILL.md +++ b/.claude/skills/tbd/SKILL.md @@ -6,7 +6,7 @@ description: |- Invoke when user mentions: tbd, beads, bd, shortcuts, issues, bugs, tasks, features, epics, todo, tracking, specs, planning, implementation, validation, guidelines, templates, commit, PR, pull request, code review, testing, TDD, test-driven, golden testing, snapshot testing, TypeScript, Python, Convex, monorepo, cleanup, dead code, refactor, handoff, research, architecture, labels, search, checkout library, source code review, or any workflow shortcut. name: tbd --- - diff --git a/.tbd/config.yml b/.tbd/config.yml index cb7a36d2..1ced77c6 100644 --- a/.tbd/config.yml +++ b/.tbd/config.yml @@ -1,4 +1,4 @@ -tbd_format: f04 +tbd_format: f05 tbd_version: development display: id_prefix: tbd @@ -23,9 +23,6 @@ settings: # Auto-sync: Docs are automatically synced when stale (default: every 24 hours). # Configure with settings.doc_auto_sync_hours (0 = disabled). docs_cache: - lookup_path: - - .tbd/docs/shortcuts/system - - .tbd/docs/shortcuts/standard files: shortcuts/system/shortcut-explanation.md: internal:shortcuts/system/shortcut-explanation.md shortcuts/system/skill-baseline.md: internal:shortcuts/system/skill-baseline.md @@ -92,3 +89,6 @@ docs_cache: templates/plan-spec.md: internal:templates/plan-spec.md templates/qa-playbook.md: internal:templates/qa-playbook.md templates/research-brief.md: internal:templates/research-brief.md + lookup_path: + - .tbd/docs/shortcuts/system + - .tbd/docs/shortcuts/standard diff --git a/AGENTS.md b/AGENTS.md index f43c0321..b290ce1b 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -33,7 +33,7 @@ Work is NOT complete until `git push` succeeds. - NEVER say “ready to push when you are” - YOU must push - If push fails, resolve and retry until it succeeds - + ## tbd This repository uses **tbd** for git-native issue tracking (beads), spec-driven From d0263ea8d857e32a9b46defba78dc47c660aec48 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 12 Jun 2026 15:57:49 +0000 Subject: [PATCH 14/36] docs: Document the format-upgrade abort path; pin interrupted states Address the "what if an upgrade goes wrong" review concern with a complete, tested abort story: - tbd-docs.md Troubleshooting gains "Aborting a Format Upgrade": a state inventory table (tracked vs machine-local vs never-touched), the abort recipe (git-restore tracked files + delete the machine-local layout.yml, which regenerates from the config), and safety notes. The nuclear option (deleting all of $GIT_COMMON_DIR/tbd/) is documented with its real caveat, verified live: recoverable for synced data, but --no-sync changes since the last sync live as uncommitted files in the worktree and would be lost - so the recipe deletes only layout.yml. - development.md cross-references the inventory, recipe, and tests. - Two new e2e tests pin the crash windows and the recipe: an interrupted upgrade (layout f05 + config f04, the data-command window) completes on the next command; the documented abort (restore config + rm layout.yml) restores the exact pre-upgrade state and re-upgrading from it works. The setup-path window (config f05 + layout f04) was already pinned by the read-path upgrade test, now annotated as such. https://claude.ai/code/session_01X8S12JzmmxEfLpYzgH8Y7E --- docs/development.md | 12 +++++ packages/tbd/docs/tbd-docs.md | 54 +++++++++++++++++++ .../tests/common-dir-layout-doctor.test.ts | 45 ++++++++++++++++ 3 files changed, 111 insertions(+) diff --git a/docs/development.md b/docs/development.md index 1dd344e5..27a390cc 100644 --- a/docs/development.md +++ b/docs/development.md @@ -371,6 +371,18 @@ $GIT_COMMON_DIR/tbd/ # Shared by all linked worktrees of this rep (`.tbd/data-sync/issues/`). The direct path is gitignored and exists only as a legacy diagnostic/migration location. +### Format Upgrades and Rollback + +A `tbd_format` bump writes exactly two stamps: the tracked `.tbd/config.yml` and the +machine-local `$GIT_COMMON_DIR/tbd/layout.yml` (plus, only when `tbd setup --auto` is +run, the tracked agent-surface markers). +It never touches issue data, so any upgrade can be aborted: restore the tracked files +from git and delete `layout.yml` (it regenerates from the config). +The full state inventory and abort recipe are user-facing in `tbd-docs.md` +§Troubleshooting → “Aborting a Format Upgrade”; the migrate → revert → repeat loop and +both interrupted-upgrade partial states are pinned by tests in +`tests/common-dir-layout-doctor.test.ts` (“f04 → f05 upgrade”). + ### Key Source Files - `packages/tbd/src/lib/paths.ts` - Path constants and `resolveDataSyncDir()` diff --git a/packages/tbd/docs/tbd-docs.md b/packages/tbd/docs/tbd-docs.md index f7bfcc26..308c2ef5 100644 --- a/packages/tbd/docs/tbd-docs.md +++ b/packages/tbd/docs/tbd-docs.md @@ -576,6 +576,7 @@ Options: > **Note:** `tbd import --from-beads` is deprecated. > Use `tbd setup --auto` or `tbd setup --from-beads` instead for migrating from Beads. + - `--validate` - Validate existing import against Beads source ### beads @@ -1118,6 +1119,59 @@ ls "$(git rev-parse --path-format=absolute --git-common-dir)/tbd/data-sync-workt ls "$(git rev-parse --path-format=absolute --git-common-dir)/tbd/data-sync-worktree/.tbd/data-sync/issues/" | sort ``` +### Aborting a Format Upgrade + +Upgrading tbd can bump the repository format (`tbd_format` in `.tbd/config.yml`, e.g. +f04 → f05). The bump happens automatically on the first command after upgrading, and +older tbd versions then refuse the repository until they are upgraded. +If an upgrade hits unexpected bugs, you can cleanly abort and return to the previous +version. This is everything a format upgrade can touch: + +| State | Location | In git? | Written by | Revert | +| --- | --- | --- | --- | --- | +| Project config | `.tbd/config.yml` | tracked | the migration (format stamp) | `git checkout -- .tbd/config.yml`, or `git revert` the bump commit | +| Agent surfaces | `AGENTS.md`, `.claude/`, `.agents/`, `.codex/` | tracked | only `tbd setup --auto` (marker refresh) | `git checkout --` those paths | +| Shared layout stamp | `$GIT_COMMON_DIR/tbd/layout.yml` | machine-local, not in git | the migration (re-stamp) | delete it — it regenerates from whatever the config says | +| Forked docs (f05) | `docs/tbd/`, `.tbd/doc-forks/` | tracked once committed | only `tbd docs fork` | `git checkout --`/`git revert` if committed; delete if never committed | +| Docs cache | `.tbd/docs/` | gitignored | doc sync (unchanged by migration) | none needed — always safe to delete and re-sync | +| Issue data | `tbd-sync` branch + `$GIT_COMMON_DIR/tbd/data-sync-worktree/` | git branch | **never touched by migration** | none needed — the worktree re-materializes from the branch | + +**Abort recipe** (works from any state, including a crash mid-upgrade): + +```bash +# 1. Restore the tracked files (or `git revert` the format-bump commit): +git checkout -- .tbd/config.yml +git checkout -- AGENTS.md .claude .agents .codex # only if `tbd setup --auto` ran + +# 2. Delete the machine-local format stamp (regenerates from the config): +rm "$(git rev-parse --path-format=absolute --git-common-dir)/tbd/layout.yml" + +# 3. Only if docs were forked and never committed: +rm -rf docs/tbd .tbd/doc-forks +``` + +After this, the previous tbd version works again, and re-running the upgrade later is +safe — the migration is idempotent from any of these states. + +Notes: + +- **The migration never writes issue data**, so the recipe above cannot lose issues — it + touches only the two stamps and tracked files. + A bigger hammer also exists: deleting the entire `$GIT_COMMON_DIR/tbd/` directory is + recoverable (layout and the data-sync worktree re-materialize from the config and the + `tbd-sync` branch on the next command, or via `tbd doctor --fix`) — **but only for + synced data**. Issue changes made with `--no-sync` since the last `tbd sync` live as + uncommitted files inside that worktree and would be lost, so run `tbd sync` first if + you must delete it. This is why the recipe deletes only `layout.yml`, never the whole + directory. +- **Interrupted upgrades self-heal.** If the process dies between the two stamp writes + (layout updated but not config, or config but not layout), the next command with the + new version completes the migration; the abort recipe above also works from either + partial state. +- Teammates each migrate their own machine-local stamp automatically; only the + `.tbd/config.yml` change is shared (via your branch), so reverting that commit is the + team-wide rollback. + ### Performance For large repositories with many issues: diff --git a/packages/tbd/tests/common-dir-layout-doctor.test.ts b/packages/tbd/tests/common-dir-layout-doctor.test.ts index f9ece2a9..33d4786a 100644 --- a/packages/tbd/tests/common-dir-layout-doctor.test.ts +++ b/packages/tbd/tests/common-dir-layout-doctor.test.ts @@ -291,6 +291,8 @@ describeUnlessWindows('common-dir layout via CLI', { timeout: 30000 }, () => { it('read commands upgrade an older layout under the lock, preserving created_at', async () => { // Config already f05 (e.g. a teammate committed the bump) but this // machine's layout is still f04: a read must auto-upgrade, not error. + // This is also the setup-path crash window (setup writes config before + // the layout). const layoutPath = join(dir, '.git', 'tbd', 'layout.yml'); const before = await readFile(layoutPath, 'utf-8'); const createdAt = before.split('\n').find((l) => l.startsWith('created_at:')); @@ -302,6 +304,49 @@ describeUnlessWindows('common-dir layout via CLI', { timeout: 30000 }, () => { expect(after).toContain('tbd_format: f05'); expect(after).toContain(createdAt); }); + + it('completes an interrupted upgrade (layout already f05, config still f04)', async () => { + // The data-command crash window: ensureSharedDataSyncLayout re-stamps the + // layout BEFORE writeConfig persists the config. A crash there leaves + // layout f05 + config f04; the next command must finish the migration, + // not error on the mismatch. + const configPath = join(dir, '.tbd', 'config.yml'); + await writeFile( + configPath, + (await readFile(configPath, 'utf-8')).replace('tbd_format: f05', 'tbd_format: f04'), + ); + expect(await readFile(join(dir, '.git', 'tbd', 'layout.yml'), 'utf-8')).toContain( + 'tbd_format: f05', + ); + + const create = runTbd(dir, ['create', 'resume probe', '--type', 'task', '--no-sync']); + expect(create.status).toBe(0); + expect(create.stderr).toContain('f04 → f05'); + expect(await readFile(configPath, 'utf-8')).toContain('tbd_format: f05'); + }); + + it('aborts via the documented recipe: restore config, delete layout.yml', async () => { + // The tbd-docs "Aborting a Format Upgrade" recipe: git-restore the config + // (simulated by rewriting the stamp) and delete the machine-local layout. + const configPath = join(dir, '.tbd', 'config.yml'); + const layoutPath = join(dir, '.git', 'tbd', 'layout.yml'); + await writeFile( + configPath, + (await readFile(configPath, 'utf-8')).replace('tbd_format: f05', 'tbd_format: f04'), + ); + await rm(layoutPath); + + // The on-disk state is now exactly pre-upgrade: an f04 config and no + // layout — the state an f04-era client accepts and regenerates from. + expect(await readFile(configPath, 'utf-8')).toContain('tbd_format: f04'); + + // Re-running the upgrade from the aborted state works cleanly: the next + // command migrates the config and regenerates the layout at f05. + const list = runTbd(dir, ['list', '--json']); + expect(list.status).toBe(0); + expect(await readFile(configPath, 'utf-8')).toContain('tbd_format: f05'); + expect(await readFile(layoutPath, 'utf-8')).toContain('tbd_format: f05'); + }); }); describe('read fast-path (H1)', () => { From b23ac26e423f033461cb6f7fa2d5338c0ea76622 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 12 Jun 2026 16:11:04 +0000 Subject: [PATCH 15/36] docs: Make install vs upgrade agent ergonomics explicit Both paths are now stated concisely where users and agents actually look: - README Quick Start gains the third tell-your-agent prompt: upgrading an existing installation ("upgrade tbd, run tbd setup --auto, and commit the changes"), alongside the existing fresh-machine and fresh-cloud prompts. - README gains an "Upgrading" section: the same two commands as install, the automatic format migration + commit-the-diff guidance, what teammates on older versions see, and the pointer to the abort recipe in tbd docs. - skill-baseline.md (the injected agent skill) labels npm install as install-or-upgrade, marks `tbd setup --auto` as the upgrade step that applies format migrations, and tells agents what to do when they hit the "requires a newer version of tbd" refusal. Installed SKILL.md surfaces regenerated. The upgrade flow was verified end-to-end on an f04 repo: setup --auto migrates both stamps, prints the commit guidance, and refreshes surfaces. https://claude.ai/code/session_01X8S12JzmmxEfLpYzgH8Y7E --- .agents/skills/tbd/SKILL.md | 7 ++++-- .claude/skills/tbd/SKILL.md | 7 ++++-- README.md | 25 ++++++++++++++++--- .../docs/shortcuts/system/skill-baseline.md | 7 ++++-- skills/tbd/SKILL.md | 7 ++++-- 5 files changed, 42 insertions(+), 11 deletions(-) diff --git a/.agents/skills/tbd/SKILL.md b/.agents/skills/tbd/SKILL.md index c56b4950..c793dad7 100644 --- a/.agents/skills/tbd/SKILL.md +++ b/.agents/skills/tbd/SKILL.md @@ -25,12 +25,15 @@ Run 'tbd setup' to update. ## Installation ```bash -npm install -g get-tbd@latest +npm install -g get-tbd@latest # Install or upgrade the CLI (same command for both) tbd setup --auto --prefix= # Fresh project (--prefix is REQUIRED: 2-8 alphabetic chars recommended. ALWAYS ASK THE USER FOR THE PREFIX; do not guess it) -tbd setup --auto # Existing tbd project (prefix already set) +tbd setup --auto # Existing tbd project — also the upgrade step (applies any format migration; commit the diff it reports) tbd setup --from-beads # Migration from .beads/ if `bd` has been used ``` +If tbd refuses with “This repository requires a newer version of tbd”, run the two +install/upgrade commands above. + ## Routine Commands ```bash diff --git a/.claude/skills/tbd/SKILL.md b/.claude/skills/tbd/SKILL.md index c56b4950..c793dad7 100644 --- a/.claude/skills/tbd/SKILL.md +++ b/.claude/skills/tbd/SKILL.md @@ -25,12 +25,15 @@ Run 'tbd setup' to update. ## Installation ```bash -npm install -g get-tbd@latest +npm install -g get-tbd@latest # Install or upgrade the CLI (same command for both) tbd setup --auto --prefix= # Fresh project (--prefix is REQUIRED: 2-8 alphabetic chars recommended. ALWAYS ASK THE USER FOR THE PREFIX; do not guess it) -tbd setup --auto # Existing tbd project (prefix already set) +tbd setup --auto # Existing tbd project — also the upgrade step (applies any format migration; commit the diff it reports) tbd setup --from-beads # Migration from .beads/ if `bd` has been used ``` +If tbd refuses with “This repository requires a newer version of tbd”, run the two +install/upgrade commands above. + ## Routine Commands ```bash diff --git a/README.md b/README.md index 67f9af6c..c1f88b79 100644 --- a/README.md +++ b/README.md @@ -37,7 +37,6 @@ will work in Cursor, Codex, or any agent environment that can use the `tbd` CLI. ## Quick Start > [!TIP] -> > If running on your own machine, install the `tbd` CLI yourself: > > **`npm install -g get-tbd@latest`** @@ -50,6 +49,11 @@ will work in Cursor, Codex, or any agent environment that can use the `tbd` CLI. > > ***“install tbd (npm install -g get-tbd@latest) and run tbd prime for instructions to > set up this project”*** +> +> If tbd is already set up in the repo and you want the latest version, tell the agent: +> +> ***“upgrade tbd (npm install -g get-tbd@latest), run tbd setup --auto, and commit the +> changes”*** That’s it. Running `tbd prime` gives agents full workflow context on how to use `tbd` and @@ -122,7 +126,6 @@ agents handling different aspects that I manage) is slower, because it forces yo design, but it gives higher quality results. > [!NOTE] -> > We use *Beads* (capitalized) to refer to Steve Yegge’s original > [`bd` tool](https://github.com/steveyegge/beads). > Lowercase “beads” refers generically to the issues stored in `tbd` or `bd`. @@ -225,7 +228,6 @@ practices. These aren’t generic tips; they’re mostly my own detailed and som opinionated rules with concrete examples, built from months of heavy agentic coding. > [!TIP] -> > An example: I *strongly* believe there are much better ways to do testing > proliferating hundreds of unit and integration tests. > So (with help from some Opus 4.5 and GPT-5 Pro) I wrote a multi-page brief about @@ -292,6 +294,23 @@ tbd setup --from-beads > **Tip:** Run `tbd setup --auto` anytime to refresh skill files, hooks, and configs > with the latest shortcuts, guidelines, and templates. +### Upgrading + +Upgrading an existing installation is the same two commands, run by you or your agent: + +```bash +npm install -g get-tbd@latest # Upgrade the CLI +tbd setup --auto # Refresh skills/hooks and apply any format migration +``` + +If the new version bumps the repository format (`tbd_format` in `.tbd/config.yml`), +setup migrates it automatically and prints a notice — **commit the resulting diff** to +publish the upgrade to your team. +Teammates still on an older tbd then see “This repository requires a newer version of +tbd” until they run the same two commands. +Issue data is never touched by an upgrade, and the migration is revertible: see +“Aborting a Format Upgrade” under Troubleshooting in `tbd docs`. + ### Team Setup `tbd` is designed for teams where one person sets up the project and others join later. diff --git a/packages/tbd/docs/shortcuts/system/skill-baseline.md b/packages/tbd/docs/shortcuts/system/skill-baseline.md index b863d178..5d51e7c1 100644 --- a/packages/tbd/docs/shortcuts/system/skill-baseline.md +++ b/packages/tbd/docs/shortcuts/system/skill-baseline.md @@ -17,12 +17,15 @@ description: Full tbd workflow guide for agents ## Installation ```bash -npm install -g get-tbd@latest +npm install -g get-tbd@latest # Install or upgrade the CLI (same command for both) tbd setup --auto --prefix= # Fresh project (--prefix is REQUIRED: 2-8 alphabetic chars recommended. ALWAYS ASK THE USER FOR THE PREFIX; do not guess it) -tbd setup --auto # Existing tbd project (prefix already set) +tbd setup --auto # Existing tbd project — also the upgrade step (applies any format migration; commit the diff it reports) tbd setup --from-beads # Migration from .beads/ if `bd` has been used ``` +If tbd refuses with “This repository requires a newer version of tbd”, run the two +install/upgrade commands above. + ## Routine Commands ```bash diff --git a/skills/tbd/SKILL.md b/skills/tbd/SKILL.md index f20e6ed6..b6d8135d 100644 --- a/skills/tbd/SKILL.md +++ b/skills/tbd/SKILL.md @@ -31,12 +31,15 @@ allowed-tools: Bash(tbd:*), Read, Write ## Installation ```bash -npm install -g get-tbd@latest +npm install -g get-tbd@latest # Install or upgrade the CLI (same command for both) tbd setup --auto --prefix= # Fresh project (--prefix is REQUIRED: 2-8 alphabetic chars recommended. ALWAYS ASK THE USER FOR THE PREFIX; do not guess it) -tbd setup --auto # Existing tbd project (prefix already set) +tbd setup --auto # Existing tbd project — also the upgrade step (applies any format migration; commit the diff it reports) tbd setup --from-beads # Migration from .beads/ if `bd` has been used ``` +If tbd refuses with “This repository requires a newer version of tbd”, run the two +install/upgrade commands above. + ## Routine Commands ```bash From 72fd65c53f41a8ba23b6ee932628591d7e27a038 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 12 Jun 2026 16:29:05 +0000 Subject: [PATCH 16/36] feat: Make fork-dir drift legible (README index, local files, sync notice) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Systematic handling for everything a user or agent can do to docs/tbd/ after forking, per review. Two invariants make it predictable: names are identity (/.md, flat — nested folders are not scanned), and tracking is derived from content hashes, so no git operation can desynchronize tbd from the folder. - Generated docs/tbd/README.md index (the spec'd orientation layer): what the folder is, the layout rules, one line per doc with its description; local files marked; regenerated on every fork/unfork/update; removed (and empty dirs pruned) when the last fork is removed, so unfork --all leaves the repo pristine. - tbd docs status now shows hand-authored `local` files (covers adds, the new half of a rename, and a deleted manifest wholesale) and prints restore/ finalize hints for `missing` docs - a rename reads as an explicit missing+local pair with resolutions. - tbd sync prints a one-line drift notice (stale / conflicted / missing counts) right after the doc-cache refresh - awareness for agents running routine syncs, with mutation still reserved to tbd docs update. - tbd-docs.md documents the full drift matrix (edit / delete / rename / add / subfolder-move / manifest-deletion / git operations) and the derived-tracking guarantee. Tested: 3 new unit tests (local listing ignores nested dirs; drift summary counts stale/missing/local; README generation + pruning) and 11 new tryscript golden steps (README content, rename pair legibility, [local] list marker, sync notice, fork-dir pruning). All verified live first. https://claude.ai/code/session_01X8S12JzmmxEfLpYzgH8Y7E --- packages/tbd/docs/tbd-docs.md | 35 ++++ packages/tbd/src/cli/commands/docs-fork.ts | 79 +++++-- packages/tbd/src/cli/commands/sync.ts | 34 +++- packages/tbd/src/file/doc-fork.ts | 192 +++++++++++++++++- packages/tbd/tests/cli-docs-fork.tryscript.md | 99 +++++++++ packages/tbd/tests/doc-fork.test.ts | 83 ++++++++ 6 files changed, 504 insertions(+), 18 deletions(-) diff --git a/packages/tbd/docs/tbd-docs.md b/packages/tbd/docs/tbd-docs.md index 308c2ef5..a73bd4f1 100644 --- a/packages/tbd/docs/tbd-docs.md +++ b/packages/tbd/docs/tbd-docs.md @@ -715,6 +715,41 @@ On HTTP 403, fetching falls back to `gh api` for authenticated access. User-added shortcuts go to `shortcuts/custom/` (separate from bundled `shortcuts/standard/`). +### Forked Docs in Your Repo (docs/tbd/) + +`tbd docs fork` copies managed docs into `docs/tbd/`, laid out **by kind, flat within +each kind**, with a generated `README.md` index (regenerated on every +fork/unfork/update): + +``` +docs/tbd/ +├── README.md # generated index — what this folder is, one line per doc +├── guidelines/.md +├── shortcuts/.md +└── templates/.md +``` + +Two rules make everything below predictable: **names are identity** (a doc is +`/.md`; nested subfolders are not scanned), and **tracking is derived, not +stored** — every doc’s state is recomputed from content hashes (your file vs its +recorded base vs current upstream), so no git operation can desynchronize tbd from the +folder. Whatever you or your agent do to these files, `tbd docs status` gives a defined +answer: + +| You (or your agent)… | State | What happens / what to do | +| --- | --- | --- | +| Edit a forked file | `customized` | Served as-is; `tbd docs update` three-way merges upstream changes in | +| Delete a forked file | `missing` | Serving falls back to upstream; restore with `tbd docs fork --force` or finalize with `tbd docs unfork ` | +| Rename a forked file | `missing` + `local` | A rename is delete + add: finalize the old name (`unfork`), keep the new file as `local` | +| Add a new `.md` file | `local` | Served with top precedence; nothing to update or unfork (no upstream) | +| Move a file into a subfolder | invisible | Subfolders are not scanned — keep files at `/.md` | +| Delete `.tbd/doc-forks/` (the manifest) | all `local` | Files keep being served; re-fork with `--force` to re-establish update tracking (overwrites with upstream — re-apply edits after) | +| Commit / pull / merge / revert any of it | recomputed | States derive from content, so collaborators see the same answers from the same files | + +Awareness without surprise mutations: `tbd sync` prints a one-line notice when forked +docs are stale, conflicted, or missing, and `tbd docs status` shows the full picture — +but only the explicit `tbd docs update` ever modifies tracked files. + ### uninstall Remove tbd from a repository. diff --git a/packages/tbd/src/cli/commands/docs-fork.ts b/packages/tbd/src/cli/commands/docs-fork.ts index 9f61edb8..7d6e68b3 100644 --- a/packages/tbd/src/cli/commands/docs-fork.ts +++ b/packages/tbd/src/cli/commands/docs-fork.ts @@ -47,6 +47,8 @@ import { forkFilePath, readForkFile, readForkBase, + listLocalForkFiles, + regenerateForkDirReadme, ForkConflictError, } from '../../file/doc-fork.js'; import { updateOne, diffContents, type UpdateStrategy } from '../../file/fork-update.js'; @@ -142,10 +144,13 @@ class DocsForkHandler extends BaseCommand { } } await writeForkManifest(tbdRoot, manifest); + await regenerateForkDirReadme(tbdRoot, FORK_DIR, manifest); if (this.ctx.json) { this.output.data({ forked }); } else { + const colors = this.output.getColors(); + console.log(colors.dim(` Regenerated ${FORK_DIR}/README.md`)); console.log(''); console.log('Edit in place — tbd now serves your copy wherever it served upstream.'); } @@ -259,6 +264,7 @@ class DocsUnforkHandler extends BaseCommand { } } await writeForkManifest(tbdRoot, manifest); + await regenerateForkDirReadme(tbdRoot, FORK_DIR, manifest); if (this.ctx.json) { this.output.data({ unforked: removed }); } @@ -283,13 +289,18 @@ class DocsStatusHandler extends BaseCommand { // Resolve upstream (cache) content per entry for staleness. const caches = new Map(); - const rows: { - entry: ForkEntry; + interface StatusRow { + name: string; + kind: string; label: string; + path: string; + source: string; customized: boolean; stale: boolean; conflicted: boolean; - }[] = []; + missing: boolean; + } + const rows: StatusRow[] = []; for (const entry of manifest.forks) { const kind = entry.kind as ForkKind; @@ -297,20 +308,42 @@ class DocsStatusHandler extends BaseCommand { const cacheHit = caches.get(kind)!.get(entry.name); const status = await forkStatusFor(tbdRoot, FORK_DIR, entry, cacheHit?.doc.content ?? null); rows.push({ - entry, + name: entry.name, + kind: entry.kind, label: stateLabel(status.state, status.stale), + path: entry.path, + source: entry.source, customized: status.customized, stale: status.stale, conflicted: status.conflicted, + missing: status.state === 'missing', + }); + } + + // Hand-authored fork-dir files with no manifest entry (state `local`). + // These cover adds, the new half of a rename, and a deleted manifest. + const locals = await listLocalForkFiles(tbdRoot, FORK_DIR, manifest); + for (const l of locals) { + rows.push({ + name: l.name, + kind: l.kind, + label: 'local', + path: l.relPath, + source: '—', + customized: false, + stale: false, + conflicted: false, + missing: false, }); } + rows.sort((a, b) => a.name.localeCompare(b.name)); if (this.ctx.json) { const docs: DocMapEntry[] = rows.map((r) => ({ - name: r.entry.name, - type: r.entry.kind, - path: r.entry.path, - source: r.entry.source, + name: r.name, + type: r.kind, + path: r.path, + ...(r.source !== '—' ? { source: r.source } : {}), state: r.label, stale: r.stale, })); @@ -326,24 +359,37 @@ class DocsStatusHandler extends BaseCommand { return; } - const nameW = Math.max(4, ...rows.map((r) => r.entry.name.length)); - const kindW = Math.max(4, ...rows.map((r) => r.entry.kind.length)); + const nameW = Math.max(4, ...rows.map((r) => r.name.length)); + const kindW = Math.max(4, ...rows.map((r) => r.kind.length)); const stateW = Math.max(5, ...rows.map((r) => r.label.length)); const header = `${'NAME'.padEnd(nameW)} ${'KIND'.padEnd(kindW)} ${'STATE'.padEnd(stateW)} SOURCE`; console.log(colors.dim(header)); for (const r of rows) { - const line = `${r.entry.name.padEnd(nameW)} ${r.entry.kind.padEnd(kindW)} ${r.label.padEnd(stateW)} ${r.entry.source}`; - console.log(line); + const line = `${r.name.padEnd(nameW)} ${r.kind.padEnd(kindW)} ${r.label.padEnd(stateW)} ${r.source}`; + console.log(r.label === 'local' ? colors.dim(line) : line); } - const customizedCount = rows.filter((r) => r.customized).length; - const staleCount = rows.filter((r) => r.stale).length; - const conflictCount = rows.filter((r) => r.conflicted).length; + const forkedRows = rows.filter((r) => r.label !== 'local'); + const customizedCount = forkedRows.filter((r) => r.customized).length; + const staleCount = forkedRows.filter((r) => r.stale).length; + const conflictCount = forkedRows.filter((r) => r.conflicted).length; + const missingRows = forkedRows.filter((r) => r.missing); const parts = [`${customizedCount} customized`]; if (staleCount > 0) parts.push(`${staleCount} with upstream updates — run 'tbd docs update'`); if (conflictCount > 0) parts.push(`${conflictCount} conflict pending`); + if (locals.length > 0) parts.push(`${locals.length} local`); console.log(''); - console.log(`${rows.length} forked: ${parts.join(', ')}`); + console.log(`${forkedRows.length} forked: ${parts.join(', ')}`); + + if (missingRows.length > 0) { + console.log(''); + console.log(`${missingRows.length} doc(s) missing (forked file deleted or renamed):`); + for (const r of missingRows) { + console.log( + ` ${r.name} restore with 'tbd docs fork ${r.name} --force', or finalize with 'tbd docs unfork ${r.name}'`, + ); + } + } }, 'Failed to read docs status'); } } @@ -420,6 +466,7 @@ class DocsUpdateHandler extends BaseCommand { if (!options.dryRun) { await writeForkManifest(tbdRoot, manifest); + await regenerateForkDirReadme(tbdRoot, FORK_DIR, manifest); } if (this.ctx.json) { diff --git a/packages/tbd/src/cli/commands/sync.ts b/packages/tbd/src/cli/commands/sync.ts index b845daa0..d3efbd11 100644 --- a/packages/tbd/src/cli/commands/sync.ts +++ b/packages/tbd/src/cli/commands/sync.ts @@ -24,7 +24,9 @@ import { type ConflictEntry, type PushResult, } from '../../file/git.js'; -import { DATA_SYNC_DIR } from '../../lib/paths.js'; +import { DATA_SYNC_DIR, FORK_DIR } from '../../lib/paths.js'; +import { readForkManifest } from '../../file/fork-manifest.js'; +import { computeForkDriftSummary } from '../../file/doc-fork.js'; import { basename, join } from 'node:path'; import { access, readFile } from 'node:fs/promises'; import { writeFile } from 'atomically'; @@ -189,9 +191,39 @@ class SyncHandler extends BaseCommand { // Report results this.showDocSyncResult(result); + await this.notifyForkDrift(); return result; } + /** + * One-line awareness notice for forked docs (docs/tbd/): the cache refresh + * above is exactly when forks can become stale, and agents run `tbd sync` + * routinely — so drift is surfaced here, but never acted on (only the + * explicit `tbd docs update` mutates tracked files). + */ + private async notifyForkDrift(): Promise { + try { + const manifest = await readForkManifest(this.tbdRoot); + const drift = await computeForkDriftSummary(this.tbdRoot, FORK_DIR, manifest); + if (drift.forks === 0) return; + const parts: string[] = []; + if (drift.stale > 0) { + parts.push(`${drift.stale} forked doc(s) have upstream updates — run 'tbd docs update'`); + } + if (drift.conflicted > 0) { + parts.push(`${drift.conflicted} with unresolved conflict markers`); + } + if (drift.missing > 0) { + parts.push(`${drift.missing} missing (deleted/renamed) — see 'tbd docs status'`); + } + if (parts.length > 0) { + process.stderr.write(`• Docs: ${parts.join('; ')}\n`); + } + } catch { + // Drift awareness is best-effort; never fail a sync over it. + } + } + /** * Show doc sync status (what would change). */ diff --git a/packages/tbd/src/file/doc-fork.ts b/packages/tbd/src/file/doc-fork.ts index 19cac250..9394ebb7 100644 --- a/packages/tbd/src/file/doc-fork.ts +++ b/packages/tbd/src/file/doc-fork.ts @@ -11,10 +11,18 @@ * the filesystem writes; resolving which doc/source to fork is the caller's job. */ -import { readFile, rm, mkdir } from 'node:fs/promises'; +import { readFile, readdir, rm, rmdir, mkdir } from 'node:fs/promises'; import { dirname, join, relative } from 'node:path'; import { writeFile } from 'atomically'; +import matter from 'gray-matter'; + +import { DocCache } from './doc-cache.js'; +import { + CACHE_GUIDELINES_PATHS, + CACHE_SHORTCUT_PATHS, + CACHE_TEMPLATE_PATHS, +} from '../lib/paths.js'; import { type ForkEntry, @@ -244,6 +252,188 @@ export async function readForkBase(tbdRoot: string, entry: ForkEntry): Promise//*.md` layout is scanned — names are + * identity, so nested folders are deliberately not searched (documented). + */ +export async function listLocalForkFiles( + tbdRoot: string, + forkDir: string, + manifest: ForkManifest, +): Promise { + const locals: LocalForkFile[] = []; + for (const kind of Object.keys(KIND_DIR) as ForkKind[]) { + let entries: string[]; + try { + entries = await readdir(join(tbdRoot, forkDir, KIND_DIR[kind])); + } catch { + continue; // Kind dir absent — nothing forked or added there. + } + for (const entry of entries) { + if (!entry.endsWith('.md')) continue; + const name = entry.slice(0, -3); + if (!findFork(manifest, name, kind)) { + locals.push({ kind, name, relPath: forkRelPath(forkDir, kind, name) }); + } + } + } + locals.sort((a, b) => a.kind.localeCompare(b.kind) || a.name.localeCompare(b.name)); + return locals; +} + +/** Cache-only lookup paths per kind (the pristine upstream copies). */ +const KIND_CACHE_PATHS: Record = { + guideline: CACHE_GUIDELINES_PATHS, + shortcut: CACHE_SHORTCUT_PATHS, + template: CACHE_TEMPLATE_PATHS, +}; + +/** Aggregate drift counts across all forked docs, plus local files. */ +export interface ForkDriftSummary { + forks: number; + customized: number; + /** Upstream moved since the fork point (run `tbd docs update`). */ + stale: number; + conflicted: number; + /** Manifest entries whose forked file was deleted out-of-band. */ + missing: number; + /** Fork-dir files with no manifest entry. */ + local: number; +} + +/** + * Compute the drift summary for awareness surfaces (`tbd sync` notice, status). + * Reads the manifest, the fork dir, and the doc cache; safe to call when nothing + * is forked (all zeros, no cache loads). + */ +export async function computeForkDriftSummary( + tbdRoot: string, + forkDir: string, + manifest: ForkManifest, +): Promise { + const summary: ForkDriftSummary = { + forks: manifest.forks.length, + customized: 0, + stale: 0, + conflicted: 0, + missing: 0, + local: 0, + }; + summary.local = (await listLocalForkFiles(tbdRoot, forkDir, manifest)).length; + if (manifest.forks.length === 0) { + return summary; + } + + const caches = new Map(); + for (const entry of manifest.forks) { + let cache = caches.get(entry.kind); + if (!cache) { + cache = new DocCache(KIND_CACHE_PATHS[entry.kind] ?? [], tbdRoot); + await cache.load({ quiet: true }); + caches.set(entry.kind, cache); + } + const status = await forkStatusFor( + tbdRoot, + forkDir, + entry, + cache.get(entry.name)?.doc.content ?? null, + ); + if (status.customized) summary.customized++; + if (status.stale) summary.stale++; + if (status.conflicted) summary.conflicted++; + if (status.state === 'missing') summary.missing++; + } + return summary; +} + +/** First frontmatter description (or title) of a doc file, for the README index. */ +async function docBlurb(absPath: string): Promise { + try { + const data = matter(await readFile(absPath, 'utf-8')).data as Record; + const description = typeof data.description === 'string' ? data.description : undefined; + const title = typeof data.title === 'string' ? data.title : undefined; + return description ?? title; + } catch { + return undefined; + } +} + +/** + * Regenerate the fork dir's `README.md` index (what this folder is, who manages + * it, and one line per doc). Called after every fork/unfork/update. When nothing + * is forked and no local files remain, the README is removed and empty kind dirs + * are pruned so `unfork --all` leaves the repo pristine. + */ +export async function regenerateForkDirReadme( + tbdRoot: string, + forkDir: string, + manifest: ForkManifest, +): Promise { + const readmePath = join(tbdRoot, forkDir, 'README.md'); + const locals = await listLocalForkFiles(tbdRoot, forkDir, manifest); + + if (manifest.forks.length === 0 && locals.length === 0) { + await rm(readmePath, { force: true }); + for (const kindDir of Object.values(KIND_DIR)) { + await rmdir(join(tbdRoot, forkDir, kindDir)).catch(() => undefined); + } + await rmdir(join(tbdRoot, forkDir)).catch(() => undefined); + return; + } + + interface IndexRow { + kind: string; + name: string; + relPath: string; + suffix: string; + } + const rows: IndexRow[] = [ + ...manifest.forks.map((f) => ({ kind: f.kind, name: f.name, relPath: f.path, suffix: '' })), + ...locals.map((l) => ({ ...l, suffix: ' *(local — not from an upstream)*' })), + ].sort((a, b) => a.kind.localeCompare(b.kind) || a.name.localeCompare(b.name)); + + const lines: string[] = [ + '', + '', + '# tbd Docs (forked into this repo)', + '', + 'Engineering guidelines, shortcuts, and templates managed by', + '[tbd](https://github.com/jlevy/tbd), forked here so they are visible, reviewable,', + 'and editable. tbd serves these copies instead of its built-in versions.', + '', + '- Edit any doc in place — your copy is what tbd serves.', + '- `tbd docs status` shows each doc’s state; `tbd docs update` pulls in upstream', + ' changes (three-way merge); `tbd docs unfork ` returns a doc to the', + ' built-in version.', + '- Keep files at `/.md`: names are how tbd identifies docs, nested', + ' folders are not scanned, and renaming a file counts as delete + add.', + '', + ]; + let currentKind = ''; + for (const row of rows) { + if (row.kind !== currentKind) { + currentKind = row.kind; + lines.push(`## ${KIND_DIR[row.kind as ForkKind] ?? row.kind}`, ''); + } + const blurb = await docBlurb(join(tbdRoot, row.relPath)); + const fileName = row.relPath.split('/').slice(-2).join('/'); + lines.push(`- [**${row.name}**](./${fileName})${blurb ? ` — ${blurb}` : ''}${row.suffix}`); + } + lines.push(''); + await mkdir(join(tbdRoot, forkDir), { recursive: true }); + await writeFile(readmePath, lines.join('\n')); +} + /** Compute the repo-relative path for a fork dir given an absolute tbd root. */ export function relativeForkDir(tbdRoot: string, absForkDir: string): string { return relative(tbdRoot, absForkDir); diff --git a/packages/tbd/tests/cli-docs-fork.tryscript.md b/packages/tbd/tests/cli-docs-fork.tryscript.md index fdc16a2f..d4687c24 100644 --- a/packages/tbd/tests/cli-docs-fork.tryscript.md +++ b/packages/tbd/tests/cli-docs-fork.tryscript.md @@ -46,6 +46,7 @@ Make some visible: tbd docs fork --category=general (and your languages) ```console $ tbd docs fork python-rules ✓ Forked python-rules → docs/tbd/guidelines/python-rules.md + Regenerated docs/tbd/README.md Edit in place — tbd now serves your copy wherever it served upstream. ? 0 @@ -126,6 +127,7 @@ No docs forked into the repo. ```console $ tbd docs fork python-rules ✓ Forked python-rules → docs/tbd/guidelines/python-rules.md + Regenerated docs/tbd/README.md ... ? 0 ``` @@ -153,5 +155,102 @@ NAME KIND STATE SOURCE python-rules guideline missing internal:guidelines/python-rules.md 1 forked: 0 customized + +1 doc(s) missing (forked file deleted or renamed): + python-rules restore with 'tbd docs fork python-rules --force', or finalize with 'tbd docs unfork python-rules' +? 0 +``` + +* * * + +## Drift: README index, local adds, renames, and pruning + +# Test: the generated README index marks itself generated and states the layout rules + +```console +$ head -3 docs/tbd/README.md + + +# tbd Docs (forked into this repo) +? 0 +``` + +# Test: finalize the deletion from the previous section + +```console +$ tbd docs unfork python-rules +✓ Unforked python-rules — served from upstream again. +? 0 +``` + +# Test: re-fork, add a hand-authored local doc, and rename the fork + +```console +$ tbd docs fork python-rules +✓ Forked python-rules → docs/tbd/guidelines/python-rules.md + Regenerated docs/tbd/README.md +... +? 0 +``` + +```console +$ printf -- '---\ntitle: Team Rules\ndescription: Our own rules\n---\n# Team Rules\n' > docs/tbd/guidelines/team-rules.md +? 0 +``` + +```console +$ mv docs/tbd/guidelines/python-rules.md docs/tbd/guidelines/py-rules.md +? 0 +``` + +# Test: status makes the whole drift legible — local add, rename pair, and resolutions + +```console +$ tbd docs status +NAME KIND STATE SOURCE +py-rules guideline local — +python-rules guideline missing internal:guidelines/python-rules.md +team-rules guideline local — + +1 forked: 0 customized, 2 local + +1 doc(s) missing (forked file deleted or renamed): + python-rules restore with 'tbd docs fork python-rules --force', or finalize with 'tbd docs unfork python-rules' +? 0 +``` + +# Test: the local add is served and marked in the cross-kind list + +```console +$ tbd docs list --kind=guideline 2>/dev/null | grep team-rules +team-rules (66 B, ~19 tok) [local] +? 0 +``` + +# Test: tbd sync surfaces doc drift without acting on it + +```console +$ tbd sync --docs 2>&1 +✓ Docs up to date +• Docs: 1 missing (deleted/renamed) — see 'tbd docs status' +? 0 +``` + +# Test: removing the local files and finalizing the last fork prunes the fork dir + +```console +$ rm docs/tbd/guidelines/py-rules.md docs/tbd/guidelines/team-rules.md +? 0 +``` + +```console +$ tbd docs unfork python-rules +✓ Unforked python-rules — served from upstream again. +? 0 +``` + +```console +$ test ! -e docs/tbd && echo "fork dir pruned" +fork dir pruned ? 0 ``` diff --git a/packages/tbd/tests/doc-fork.test.ts b/packages/tbd/tests/doc-fork.test.ts index b111a8bd..a4d5b17a 100644 --- a/packages/tbd/tests/doc-fork.test.ts +++ b/packages/tbd/tests/doc-fork.test.ts @@ -13,6 +13,9 @@ import { forkStatusFor, forkFilePath, forkRelPath, + listLocalForkFiles, + computeForkDriftSummary, + regenerateForkDirReadme, ForkConflictError, DEFAULT_FORK_DIR, } from '../src/file/doc-fork.js'; @@ -216,3 +219,83 @@ describe('unforkDoc', () => { ); }); }); + +describe('drift helpers (local files, summary, README index)', () => { + let root: string; + beforeEach(async () => { + root = await mkdtemp(join(tmpdir(), 'tbd-doc-drift-')); + // A minimal upstream cache copy so staleness can be computed. + await import('node:fs/promises').then((fs) => + fs.mkdir(join(root, '.tbd', 'docs', 'guidelines'), { recursive: true }), + ); + await writeFile(join(root, '.tbd', 'docs', 'guidelines', 'python-rules.md'), UPSTREAM); + }); + afterEach(async () => { + await rm(root, { recursive: true, force: true }); + }); + + async function forkOne() { + return forkDoc({ + tbdRoot: root, + forkDir: FORK_DIR, + manifest: emptyManifest(), + kind: 'guideline', + name: 'python-rules', + source: 'internal:guidelines/python-rules.md', + content: UPSTREAM, + }); + } + + it('listLocalForkFiles finds stray files but ignores nested folders', async () => { + const { manifest } = await forkOne(); + const dir = join(root, FORK_DIR, 'guidelines'); + await writeFile(join(dir, 'team-rules.md'), '# Team\n'); + await import('node:fs/promises').then((fs) => + fs.mkdir(join(dir, 'nested'), { recursive: true }), + ); + await writeFile(join(dir, 'nested', 'hidden.md'), '# Hidden\n'); + + const locals = await listLocalForkFiles(root, FORK_DIR, manifest); + expect(locals).toEqual([ + { kind: 'guideline', name: 'team-rules', relPath: 'docs/tbd/guidelines/team-rules.md' }, + ]); + }); + + it('computeForkDriftSummary reports stale, missing, and local counts', async () => { + const { manifest } = await forkOne(); + + // Fresh fork, cache matches base: no drift. + let s = await computeForkDriftSummary(root, FORK_DIR, manifest); + expect(s).toMatchObject({ forks: 1, stale: 0, missing: 0, local: 0 }); + + // Upstream (cache) moves: stale. + await writeFile(join(root, '.tbd', 'docs', 'guidelines', 'python-rules.md'), UPSTREAM + 'v2\n'); + s = await computeForkDriftSummary(root, FORK_DIR, manifest); + expect(s.stale).toBe(1); + + // Forked file deleted out-of-band: missing. A stray file: local. + await rm(join(root, FORK_DIR, 'guidelines', 'python-rules.md')); + await writeFile(join(root, FORK_DIR, 'guidelines', 'team-rules.md'), '# Team\n'); + s = await computeForkDriftSummary(root, FORK_DIR, manifest); + expect(s).toMatchObject({ missing: 1, local: 1 }); + }); + + it('regenerateForkDirReadme writes the index and prunes when empty', async () => { + const { manifest } = await forkOne(); + await regenerateForkDirReadme(root, FORK_DIR, manifest); + const readme = await readFile(join(root, FORK_DIR, 'README.md'), 'utf-8'); + expect(readme).toContain('DO NOT EDIT'); + expect(readme).toContain('python-rules'); + expect(readme).toContain('nested\n folders are not scanned'); + + // Unfork everything: README and empty dirs are pruned. + const { manifest: empty } = await unforkDoc({ + tbdRoot: root, + forkDir: FORK_DIR, + manifest, + name: 'python-rules', + }); + await regenerateForkDirReadme(root, FORK_DIR, empty); + await expect(readFile(join(root, FORK_DIR, 'README.md'), 'utf-8')).rejects.toThrow(); + }); +}); From 25bc0083ab00f28c18f32bdca0f006c954358a58 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 12 Jun 2026 16:45:23 +0000 Subject: [PATCH 17/36] feat: Version-skew guard for updates; specify the doc model in tbd-design.md MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit First-principles review of the doc-synchronization design surfaced one real hole and produced the canonical design-doc specification: - Version-skew guard: within the f05 era, a teammate on an OLDER tbd sees a fork whose base was advanced by a newer tbd as "stale", and update would silently downgrade it to their older bundled content (and ping-pong the base across the team). The manifest's tbd_version field now does the job it was designed for: update refreshes it on every base advance and refuses (under every strategy) to touch a doc whose fork point is newer than the running tbd, with an upgrade message. Loose semver compare handles dev versions and never guards on unparseable versions. 6 new unit tests. - tbd-design.md gains the canonical model: §2.9 "Managed Docs: Copies, Forks, and Synchronization" (the four copies + manifest, the seven invariants incl. derived tracking and the format gate, the who-writes-what flow table, drift and degraded modes) and §4.13 "Docs Commands" (the command group, the three-sync taxonomy, update semantics). Directory-structure listings and development.md path conventions updated; ToC entries added. Also verified during review (no code change needed): an empty cache on a fresh clone self-heals via doc auto-sync before states are computed, so teammates see correct fork states immediately (tested live). https://claude.ai/code/session_01X8S12JzmmxEfLpYzgH8Y7E --- docs/development.md | 1 + packages/tbd/docs/tbd-design.md | 129 +++++++++++++++++++++ packages/tbd/src/cli/commands/docs-fork.ts | 4 + packages/tbd/src/file/fork-update.ts | 47 +++++++- packages/tbd/tests/fork-update.test.ts | 65 +++++++++++ 5 files changed, 245 insertions(+), 1 deletion(-) diff --git a/docs/development.md b/docs/development.md index 27a390cc..962e5f1f 100644 --- a/docs/development.md +++ b/docs/development.md @@ -345,6 +345,7 @@ the full specification. │ Committed to the repo: ├── config.yml # Project configuration ├── .gitignore # Controls what's gitignored below +├── doc-forks/ # Fork manifest + base snapshots (f05; tbd-design.md §2.9) ├── workspaces/ # Persistent state (outbox, named workspaces) │ └── outbox/ # Sync failure recovery data │ diff --git a/packages/tbd/docs/tbd-design.md b/packages/tbd/docs/tbd-design.md index 4635cf99..e8ec9989 100644 --- a/packages/tbd/docs/tbd-design.md +++ b/packages/tbd/docs/tbd-design.md @@ -71,6 +71,7 @@ agents. - [2.8.5 Comparison with Beads](#285-comparison-with-beads) - [2.8.6 Future Dependency Types](#286-future-dependency-types) - [2.8.7 Future: Transitive Blocking Option](#287-future-transitive-blocking-option) + - [2.9 Managed Docs: Copies, Forks, and Synchronization](#29-managed-docs-copies-forks-and-synchronization) - [3. Git Layer](#3-git-layer) - [3.1 Overview](#31-overview) - [3.2 Sync Branch Architecture](#32-sync-branch-architecture) @@ -119,6 +120,7 @@ agents. - [4.10 Global Options](#410-global-options) - [4.11 Attic Commands](#411-attic-commands) - [4.12 Output Formats](#412-output-formats) + - [4.13 Docs Commands](#413-docs-commands) - [5. Beads Compatibility](#5-beads-compatibility) - [5.1 Import Strategy](#51-import-strategy) - [5.1.1 Import Command](#511-import-command) @@ -1968,6 +1970,95 @@ Add transitive blocking as an opt-in feature if users request it after real-worl * * * +### 2.9 Managed Docs: Copies, Forks, and Synchronization + +tbd manages documentation (guidelines, shortcuts, templates) alongside issues. +With forkable docs (format f05, `plan-2026-06-11-forkable-docs.md`), a doc can exist as +up to **four copies plus a manifest**, each with a distinct owner and lifecycle. +This section is the canonical statement of that model and the invariants that make every +combination of user actions safe. + +#### The copies + +| Copy | Location | In git? | Written by | Role | +| --- | --- | --- | --- | --- | +| Bundled | npm package (`dist/docs/`) | n/a (per tbd version) | tbd releases | The upstream for `internal:` docs; immutable per installed version | +| Cache | `.tbd/docs/` | gitignored | doc sync only | Complete, pristine, machine-local mirror of all upstream docs (bundled + URL sources); disposable | +| Fork | `docs/tbd//.md` | tracked | `tbd docs fork/update` + the user/agent | The editable copy that tbd serves; optional, per-doc | +| Base | `.tbd/doc-forks/base//.md` | tracked | `tbd docs fork/update` | Verbatim upstream snapshot at the fork point; the three-way merge base | +| Manifest | `.tbd/doc-forks/forks.yml` | tracked | `tbd docs fork/unfork/update` | Provenance per fork: source docref, `base_hash` (LF-normalized sha256), `tbd_version` at fork point, `conflicted` flag | + +A doc’s identity is **kind + name**; paths follow fixed conventions +(`/.md`, flat — nested folders are not scanned). + +#### Invariants + +1. **Serving precedence**: the fork dir is prepended to every kind’s lookup path, so a + forked (or hand-authored local) file shadows the cache copy by name. + With nothing forked, lookup paths reduce to the cache and behavior is byte-identical + to pre-f05. +2. **Cache completeness**: doc sync always installs *all* upstream docs into the cache, + including forked ones. + The cache copy is the pristine reference — the staleness comparator, the “theirs” + side of every update merge, and the fallback after unfork. +3. **The cache is never authored**: only doc sync writes it, nothing else reads from + anywhere else for upstream content, and deleting it is always safe (auto-sync + regenerates it on the next doc access, including on fresh clones). +4. **Tracked files mutate only via explicit `tbd docs` verbs** (fork/unfork/update). + Setup, `tbd sync`, and background auto-sync refresh the cache and *report* drift but + never write the fork dir, bases, or manifest. +5. **Tracking is derived, not stored**: every doc state + (`upstream/forked/customized/stale/conflicted/local/missing/orphaned`) is a pure + function of (manifest entry present?, file hash, base hash, cache hash, conflicted + flag, markers present). + There is no hidden database, so no sequence of git operations — commit, pull, merge, + revert, partial commits — can desynchronize tbd from the files: collaborators + recompute identical states from identical content. +6. **The base is the fork point**: advanced only by fork (refresh) and update; with the + stored snapshot, `customized` (file ≠ base), `stale` (cache ≠ base), and three-way + merging are exact, offline operations for every collaborator regardless of which tbd + version created the fork. +7. **The format gate**: forking bumps nothing at runtime, but the f05 `tbd_format` in + `config.yml` (mirrored into the machine-local common-dir `layout.yml`) makes pre-f05 + clients refuse the repo — they would otherwise serve upstream copies of docs the team + has customized. Within the f05 era, the manifest’s per-entry `tbd_version` guards the + remaining skew: `tbd docs update` refuses to touch a doc whose fork point was + advanced by a newer tbd than the one running, since that client’s bundled “upstream” + is older than the fork point and updating would silently downgrade the doc. + +#### Who writes what (synchronization flows) + +| Flow | Reads | Writes | Tracked files touched | +| --- | --- | --- | --- | +| npm upgrade | — | bundle | none | +| Doc sync (`tbd sync --docs`, auto-sync, setup) | bundle + URL sources | cache | none | +| `tbd docs fork` | cache | fork file, base, manifest, fork-dir README | yes (explicit) | +| `tbd docs update` | cache, base, fork file | fork file and/or base, manifest, README | yes (explicit) | +| `tbd docs unfork` | base, fork file | removes fork artifacts, README | yes (explicit) | +| User/agent edits | — | fork dir (any way they like) | yes (theirs) | +| git operations | — | any tracked artifact | yes (theirs) | + +Staleness appears exactly when doc sync moves the cache past a fork’s base; awareness +surfaces (`tbd docs status`, the one-line `tbd sync` drift notice) report it, and only +the explicit `tbd docs update` acts on it. + +#### Drift and degraded modes + +Because of invariant 5, arbitrary user actions in the fork dir resolve to defined +states: edits → `customized`; deletion → `missing` (serving falls back to the cache); +rename → `missing` + `local`; new files → `local` (served, no upstream); deleting the +manifest → everything `local` (serving unaffected); moving into subfolders → not scanned +(documented). Degraded modes fail soft: an unreachable URL source keeps serving the +last-good cache copy; an empty cache self-heals via auto-sync; a deleted base blocks +merging only for that doc (repairable via `update --keep-ours`); the upgrade abort path +is specified in `tbd-docs.md` §Troubleshooting. +The drift matrix with resolutions is user-facing in `tbd-docs.md` §“Forked Docs in Your +Repo”; the state matrix, update decision table, and drift scenarios are pinned by +`fork-manifest`/`fork-update`/`doc-fork` unit tests and the `cli-docs-fork`/ +`cli-docs-update` golden tryscripts. + +* * * + ## 3. Git Layer ### 3.1 Overview @@ -2034,6 +2125,8 @@ main branch: tbd-sync branch: .tbd/.gitignore # Controls what's gitignored below .tbd/.gitattributes # Merge strategies (merge=union for ids.yml) .tbd/workspaces/ # Persistent state (outbox, named workspaces) +.tbd/doc-forks/ # Fork manifest + base snapshots (see §2.9; f05+) +docs/tbd/ # Forked docs, outside .tbd/ (see §2.9; only when fork is used) ``` #### Files Gitignored (local only) @@ -3660,6 +3753,42 @@ tbd attic restore proj-a1b2 2025-01-07T10-30-00Z * * * +### 4.13 Docs Commands + +Operations on managed docs (see §2.9 for the data model) live under the noun-scoped +`tbd docs` group, alongside the existing per-kind readers (`tbd guidelines`, +`tbd shortcut`, `tbd template`, which serve forked copies transparently via lookup +precedence): + +```bash +tbd docs fork [names...] [--kind] [--all] [--force] [--dry-run] # copy into docs/tbd/ +tbd docs unfork [names...] [--all] [--force] # back to upstream; refuses to drop edits +tbd docs status [--json] # per-doc states + missing/local hints +tbd docs update [names...] [--merge|--keep-ours] [--dry-run] # reconcile with upstream +tbd docs diff [--base|--upstream] # net fork / your changes / incoming +tbd docs list [--kind] [--json] # cross-kind list with state markers +``` + +tbd has three deliberately separate update surfaces — they differ in scope, risk, and +failure mode, and doc updates are the only one that can merge and mutate tracked files: + +| Command | Scope | Touches | Modifies tracked files? | +| --- | --- | --- | --- | +| `tbd sync` | project data (issues) | sync worktree + `tbd-sync` branch; refreshes the doc cache and *reports* fork drift | never | +| `tbd setup --auto` | installation + integrations | skills, hooks, settings, `AGENTS.md`; invokes a docs-cache sync | only generated integration files | +| `tbd docs update` | forked docs | fork dir + bases + manifest (offline, against the cache) | **yes — the only doc command that does** | + +Update semantics (the full decision table is unit-tested row by row): an unmodified +stale fork is replaced; a customized stale fork gets a `git merge-file` three-way merge +that applies automatically when clean; conflicts are skipped by default and listed with +the two explicit strategies — `--merge` (combine, standard conflict markers, sets the +`conflicted` flag until markers are resolved) and `--keep-ours` (keep the local content, +advance the fork point). +Forked files are git-tracked, so every applied update is reviewable in `git diff` and +revertible — git is the undo. + +* * * + ## 5. Beads Compatibility ### 5.1 Import Strategy diff --git a/packages/tbd/src/cli/commands/docs-fork.ts b/packages/tbd/src/cli/commands/docs-fork.ts index 7d6e68b3..633de593 100644 --- a/packages/tbd/src/cli/commands/docs-fork.ts +++ b/packages/tbd/src/cli/commands/docs-fork.ts @@ -435,6 +435,7 @@ class DocsUpdateHandler extends BaseCommand { baseContent: await readForkBase(tbdRoot, entry), upstreamContent: await upstreamFor(entry), strategy, + runningVersion: VERSION, }); const { newFileContent, newBaseContent } = result; @@ -453,6 +454,9 @@ class DocsUpdateHandler extends BaseCommand { if (newBaseContent !== undefined) { await writeBaseContent(tbdRoot, entry.kind, entry.name, newBaseContent); updated.base_hash = hashContent(newBaseContent); + // The base records its fork point's tbd version so older clients + // can detect (and refuse) a downgrade — see the update guard. + updated.tbd_version = VERSION; } if (result.setConflicted) { updated.conflicted = true; diff --git a/packages/tbd/src/file/fork-update.ts b/packages/tbd/src/file/fork-update.ts index e3b53ace..c0884ae3 100644 --- a/packages/tbd/src/file/fork-update.ts +++ b/packages/tbd/src/file/fork-update.ts @@ -123,6 +123,26 @@ export async function diffContents( } } +/** + * Loose semver comparison on major.minor.patch (prerelease ignored). + * Returns null when either version cannot be parsed — callers must not guard + * on an unparseable version. + */ +export function compareVersionsLoose(a: string, b: string): -1 | 0 | 1 | null { + const parse = (v: string): number[] | null => { + const m = /^(\d+)\.(\d+)\.(\d+)/.exec(v.trim()); + return m ? [Number(m[1]), Number(m[2]), Number(m[3])] : null; + }; + const pa = parse(a); + const pb = parse(b); + if (!pa || !pb) return null; + for (let i = 0; i < 3; i++) { + if (pa[i]! < pb[i]!) return -1; + if (pa[i]! > pb[i]!) return 1; + } + return 0; +} + /** Update strategy chosen by the user for non-clean cases. */ export type UpdateStrategy = 'default' | 'merge' | 'keep-ours'; @@ -139,7 +159,8 @@ export type UpdateAction = | 'skip-unresolved' | 'skip-orphaned' | 'skip-missing' - | 'skip-no-base'; + | 'skip-no-base' + | 'skip-newer-base'; export interface UpdateOneInput { entry: ForkEntry; @@ -150,6 +171,13 @@ export interface UpdateOneInput { /** Current upstream/cache content, or null if the source is gone (orphaned). */ upstreamContent: string | null; strategy: UpdateStrategy; + /** + * The running tbd version. When the entry's base was advanced by a NEWER tbd + * (entry.tbd_version > runningVersion), this client's "upstream" is older than + * the fork point and an update would silently downgrade the doc — so the doc + * is skipped under every strategy until the client upgrades. + */ + runningVersion?: string; } export interface UpdateOneResult { @@ -192,6 +220,23 @@ export async function updateOne(input: UpdateOneInput): Promise }; } + // Version-skew guard: if the base was advanced by a newer tbd than this one, + // this client's bundled "upstream" is OLDER than the fork point. Updating + // would downgrade the doc (and ping-pong the base across the team), so the + // doc is skipped under every strategy until this client upgrades. + if ( + input.runningVersion !== undefined && + entry.tbd_version !== undefined && + compareVersionsLoose(input.runningVersion, entry.tbd_version) === -1 + ) { + return { + action: 'skip-newer-base', + message: + `${name}: fork point was set by tbd ${entry.tbd_version} (you have ` + + `${input.runningVersion}) — upgrade tbd before updating this doc`, + }; + } + if (baseContent === null) { if (strategy === 'keep-ours') { // Repair: re-establish the base from current upstream, keep the file. diff --git a/packages/tbd/tests/fork-update.test.ts b/packages/tbd/tests/fork-update.test.ts index b4ad2e98..6b003b48 100644 --- a/packages/tbd/tests/fork-update.test.ts +++ b/packages/tbd/tests/fork-update.test.ts @@ -213,3 +213,68 @@ describe('updateOne decision table', () => { expect(r.newBaseContent).toBe(UPSTREAM_NONCONFLICT); }); }); + +describe('version-skew guard', () => { + const UPSTREAM_OLDER = 'line one\nline two\n'; // this client's (older) bundle + + it('skips a doc whose base was advanced by a newer tbd, under every strategy', async () => { + for (const strategy of ['default', 'merge', 'keep-ours'] as const) { + const r = await updateOne({ + entry: entry({ tbd_version: '0.9.0' }), + forkContent: BASE, + baseContent: BASE, + upstreamContent: UPSTREAM_OLDER, // differs from base -> would look "stale" + strategy, + runningVersion: '0.3.0', + }); + expect(r.action).toBe('skip-newer-base'); + expect(r.newFileContent).toBeUndefined(); + expect(r.newBaseContent).toBeUndefined(); + expect(r.message).toContain('upgrade tbd'); + } + }); + + it('proceeds when the running tbd is the same or newer than the fork point', async () => { + for (const v of ['0.9.0', '1.0.0']) { + const r = await updateOne({ + entry: entry({ tbd_version: '0.9.0' }), + forkContent: BASE, + baseContent: BASE, + upstreamContent: 'line one\nline two\nline three plus\n', + strategy: 'default', + runningVersion: v, + }); + expect(r.action).toBe('replaced'); + } + }); + + it('does not guard on unparseable or absent versions', async () => { + const noEntryVersion = await updateOne({ + entry: entry(), + forkContent: BASE, + baseContent: BASE, + upstreamContent: 'changed\n', + strategy: 'default', + runningVersion: '0.3.0', + }); + expect(noEntryVersion.action).toBe('replaced'); + + const weird = await updateOne({ + entry: entry({ tbd_version: 'development' }), + forkContent: BASE, + baseContent: BASE, + upstreamContent: 'changed\n', + strategy: 'default', + runningVersion: '0.3.0', + }); + expect(weird.action).toBe('replaced'); + }); + + it('compareVersionsLoose ignores prerelease and rejects garbage', async () => { + const { compareVersionsLoose } = await import('../src/file/fork-update.js'); + expect(compareVersionsLoose('0.2.3-dev.333.abc', '0.2.3')).toBe(0); + expect(compareVersionsLoose('0.2.3', '0.10.0')).toBe(-1); + expect(compareVersionsLoose('1.0.0', '0.9.9')).toBe(1); + expect(compareVersionsLoose('development', '0.1.0')).toBeNull(); + }); +}); From 79ab5a66a5ea7d62ef962da9a525bbc0cc767680 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 12 Jun 2026 17:33:21 +0000 Subject: [PATCH 18/36] process: Add senior review doc for PR #169 (forkable docs f05) Full review: correctness findings (3 release blockers), phase-completion audit, holistic docs review, fork/export and docmap-vs-search-path design analysis, and DocRef/DocMap abstraction review. https://claude.ai/code/session_01QPsCSYGtwR8JtX2R1aaxyh --- .../review-2026-06-12-pr169-forkable-docs.md | 496 ++++++++++++++++++ 1 file changed, 496 insertions(+) create mode 100644 docs/project/reviews/review-2026-06-12-pr169-forkable-docs.md diff --git a/docs/project/reviews/review-2026-06-12-pr169-forkable-docs.md b/docs/project/reviews/review-2026-06-12-pr169-forkable-docs.md new file mode 100644 index 00000000..8c6fe1b2 --- /dev/null +++ b/docs/project/reviews/review-2026-06-12-pr169-forkable-docs.md @@ -0,0 +1,496 @@ +--- +title: 'Senior Review: PR #169 — Forkable Docs (f05) Spec and Kernel' +description: Adversarial senior engineering review of PR #169, plus a holistic docs review, analysis of the fork/export and doc-map-vs-search-path design questions, and a deep review of the DocRef and DocMap abstractions +author: Review session operated by Joshua Levy with LLM assistance +--- +# Senior Review: PR #169 — Forkable Docs (f05) Spec and Kernel + +**PR:** https://github.com/jlevy/tbd/pull/169 (branch `claude/friendly-lamport-ojs6zm`) + +**Reviewed commit:** `25bc008` + +**Date:** 2026-06-12 + +**Scope:** (1) adversarial senior review of the spec and the shipped code; (2) holistic +review of documentation state across the new surface area; (3) analysis of two open +design questions — copy-all-plus-gitignore vs export-only forking, and doc-map registry +vs search-path resolution; (4) deep review of the DocRef and DocMap abstractions. + +All critical findings were verified empirically against the CLI built from this branch +(`pnpm build`, fresh sandbox repo), not just by code reading. + +## 1. PR Review + +**CI:** all 5 checks green (ubuntu/macos/windows tests, benchmark, coverage+lint). +DeepSource grade A. No unresolved human review comments (only bot comments). +Unit coverage of the new core modules is 89–97%; the new CLI layer is ~3% under vitest +but covered by tryscripts — **which CI runs only on ubuntu** (`ci.yml` matrix runs +`vitest run` per-OS; tryscripts only run inside the ubuntu Coverage & Lint job). + +**Summary:** The spec is genuinely strong — the doc model in `tbd-design.md` §2.9 +(copies table, seven invariants, derived-not-stored state) is the best artifact in the +PR, and the merge/update design (stored bases, decision table, version-skew guard) is +sound and exhaustively unit-tested (129 tests, verified locally). +**Scope note (settled during review):** this PR is committed to being the complete, full +f05 experience and will ship as the next release — there is no separate shipping step. +The spec’s phases therefore serve as the strict validation checklist for driving this +branch to done (finding 4 audits the branch against them). +Within that frame, there are two user-facing bugs (verified end-to-end against the built +CLI) plus one data-clobbering edge case that are release blockers. + +### Critical findings (verified empirically) + +**1. Forked shortcuts are never served — the feature’s core promise (G2) is broken for +one of three kinds.** `doc-sync.ts:561–566` persists `docs_cache.lookup_path` into +**every** repo’s config on setup (verified in a fresh sandbox repo). +`shortcut.ts:78` then does `config.docs_cache?.lookup_path ?? DEFAULT_SHORTCUT_PATHS` — +the persisted key wins, and it contains only cache paths. +Result, reproduced with the built PR CLI: fork `review-code`, edit it, and +`tbd docs list` shows `[forked, customized]` while `tbd shortcut review-code` serves the +**upstream** copy. Guidelines and templates work because they don’t honor `lookup_path`. +This is precisely the “lookup_path zombie” the spec itself cites as a lesson, and it +falsifies §2.9 invariant 1 as written. +The tryscripts miss it because they only fork a guideline. +Fix: prepend the fork dir structurally in `shortcut.ts` regardless of config +(`[FORK_SHORTCUTS_DIR, ...(lookup_path ?? CACHE_SHORTCUT_PATHS)]`), and add a +fork-a-shortcut serve assertion to the tryscript. + +**2. The CLI recommends a flag that doesn’t exist — and the golden test pins it as +correct.** `docs-fork.ts:357` (zero-fork `tbd docs status`) prints +`Make some visible: tbd docs fork --category=general (and your languages)`. `--category` +is a Phase 4 feature; running the suggested command errors with +`unknown option '--category=general'` (verified). +`cli-docs-fork.tryscript.md:36` golden-tests this hint, so CI enshrines a broken +recommendation that agents will follow verbatim. +Fix: either implement `--category` now (the spec says it reuses existing frontmatter +metadata, so it’s small) or change the hint to name-based forking until Phase 4. + +**3. `git merge-file` error exits are misread as conflict counts → can overwrite a +customized doc with empty content.** `fork-update.ts:66–79` treats any positive exit +code as a conflict count. +Verified: `git merge-file` exits **255** on errors (e.g. binary-content refusal), and +git documents the conflict count as truncated to 127 — so exit 255 means error, not “255 +conflicts”. On that path stdout is empty, `updateOne` returns `merged-conflict`, and the +handler writes **empty content** over the user’s customized fork (git-recoverable, but +still a silent clobber). +Fix: treat `code > 127` as an error; one-line guard plus a unit test. + +### Major findings + +**4. Phase-completion audit: the branch is partway through every phase — use the phases +as strict completion gates, and reconcile the golden maps.** This PR delivers the +complete f05 experience as the next release, so the spec’s phases are the in-PR +validation checklist rather than separate shipping steps. +Audited against the spec, the branch currently stands: + +- **Phase 0 (contracts + docs):** spec contracts authored ✔; `tbd-design.md` §2.9/§4.13 + ✔; `tbd-docs.md` drift table + abort recipe ✔. Missing: the `tbd docs` command-group + manual section, the three-sync taxonomy table and `.tbd/` layout contract in + `tbd-docs.md`, the `docs-overview.md` and README forkable-docs rows, + `references/docref-format.md`, `references/docmap-format.md`, + `suggest-upstream-improvements.md`, and item 0.5 (lock golden maps against real + output). +- **Phase 1 (format + kernel):** f05 stamp + migration ✔ (but stamp-only: no + `.tbd/.gitignore` refresh, no generated `.tbd/README.md`, and `FORMAT_HISTORY.f05` + omits the `docs_cache.fork_dir`/`local_dirs` keys the spec defines — finding 5); + docref/docmap modules ✔ (but wired into nothing — see §4); fork-manifest ✔; + fork/unfork ✔; precedence wiring ✔ for guidelines/templates, **broken for shortcuts** + (finding 1). Missing: `tbd docs sync` subcommand, the serve provenance note (Decision + 18), per-kind `--list` markers. +- **Phase 2 (status/browse/doctor):** `status`, `list`, `diff` ✔. Missing: bare + `tbd docs` overview (old viewer still the default action), `show`/`manual`, the shared + docmap renderer + per-kind reader migration, `local_dirs`, `docs add `, + grouped sync, all doctor checks, the `tbd status` Docs line. +- **Phase 3 (update/merge):** merge module + update command ✔ (with the exit-code bug, + finding 3); pending-update reporting in setup missing; the `tbd sync` drift notice is + a good addition not in the spec — add it there. +- **Phase 4 (categories/setup):** nothing landed yet, but `--category` is already + recommended by shipped output (finding 2); setup Docs summary and `--interactive` + removal pending. +- **Phase 5 (agent surface):** nothing landed (skill upgrade hints only); reference kind + \+ self-docs, skill routing rows, `welcome-user` onboarding, and CHANGELOG pending. + +Two consequences. +First, **the golden maps and the shipped output must be reconciled into +one source of truth now**, because they already disagree in a dozen places (e.g. spec: +`Updated 2 forked docs:` / impl: `Updated 1 forked doc(s):`; spec: +`1 doc is missing (forked file deleted):` / impl: +`1 doc(s) missing (forked file deleted or renamed):`; spec fork output has a +`Recorded base in .tbd/doc-forks/…` line the impl never prints; spec unfork refusal +points at `tbd docs diff`, impl points at `tbd docs status`; spec list `--json` includes +`stale` and `word_count`, impl emits neither). +Recommendation: update the spec maps to the (mostly better) shipped wording, then treat +them as binding for the remaining phases — validating each phase against its golden +block as it lands. Second, the PR title/description should state this scope (full +experience, next release) and carry the phase checklist; as of `25bc008` they still +described a spec-only PR. + +**5. The f05 format definition drifts from its own spec, and a release cut now would +ship a confusing hybrid.** + +- `FORMAT_HISTORY.f05` (`tbd-format.ts`) omits `docs_cache.fork_dir`, `local_dirs`, and + the generated `.tbd/README.md` that the spec’s contract table defines as part of f05; + the migration is stamp-only (no `.tbd/.gitignore` refresh, no `.tbd/README.md`), and + `fork_dir` is not configurable at all — `FORK_DIR` is a hard constant + (`paths.ts:361`), contradicting Resolved Decision 6. Additive later landing is + probably fine, but then amend the spec so f05’s definition matches what f05 actually + stamps. +- The old docs surface coexists with the new: `tbd docs --list` (sections) and + `tbd docs list` (docs) both work with different meanings; the command description + still says “use tbd sync --docs”. + The spec’s safety argument is “everything ships in the same release behind the f05 + gate” — and since this PR is that release vehicle, the argument holds only once the + surface re-homing lands here too. + Concretely: #1, #2, and the disposition of all four old `tbd docs` behaviors are + release bars for this PR. +- The config migration also reorders keys (`lookup_path` moved to the bottom of + `docs_cache` in this repo’s own diff) — harmless but contradicts “metadata-only stamp” + minimal-churn expectations. + +**6. Windows is untested for the whole feature and has at least one real defect.** +`FORK_DIR = join(DOCS_DIR, 'tbd')` is `docs\tbd` on Windows; `forkRelPath()` then +records `docs\tbd/guidelines/x.md` (mixed separators) into the committed manifest and +CLI output. Meanwhile the unit tests use a *different* constant — +`DEFAULT_FORK_DIR = 'docs/tbd'` (POSIX literal in `doc-fork.ts:44`) — so Windows CI +green proves nothing about production paths: tests exercise a value production never +uses. And tryscripts don’t run on Windows at all. +Fix: one POSIX-string constant for repo-relative semantics (join only at fs boundaries), +delete the duplicate, and run the fork tryscripts in the OS matrix. +(Related: `docref` rejects `C:/...` as “unknown scheme” — see §4.) + +### Minor findings + +7. **`conflicted` never clears in the stored manifest.** After resolving markers, state + computes correctly (flag AND markers), but the committed `forks.yml` keeps + `conflicted: true` until some later update writes the entry (`docs-fork.ts:461–465` + only clears when an update applies). + Spec says “auto-clears”. + Cosmetic but confusing in a committed file. +8. **`tbd docs update ` silently reports “All forked docs are up to date”** — + unknown names are filtered out without error (`docs-fork.ts:418–419`). +9. **Path-traversal hardening**: `unforkDoc`/`updateOne` compute fs paths from committed + manifest `name`/`kind` without validating for separators/`..` — a hostile `forks.yml` + in a cloned repo can direct `rm`/writes outside the fork dir. + Validate names (no `/`, `\`, `..`) on manifest read. +10. **`pathExists` reads the whole file to test existence, then callers re-read it** + (`doc-fork.ts:69–76`); it also swallows non-ENOENT errors. + Use `stat`, or read once and branch on ENOENT. +11. **Fork-conflict error is not actionable** (raw “already exists and is not an + unmodified fork”, no options) — violates the spec golden and `error-handling-rules` + ("tell users what to do next"); contrast unfork, which does it well. +12. `tbd docs list --kind=bogus` / `fork --kind=bogus` silently produce empty results or + a misleading “No doc found” (`KIND_CACHE_PATHS[kind] ?? []`). Validate the kind. +13. `FORK_KINDS` includes `'reference'` but `KIND_CACHE_PATHS` doesn’t — a forked + reference doc (Phase 5) would permanently read `orphaned`. Latent trap; add a + comment or a guard now. +14. The sync drift notice writes via `process.stderr.write` (`sync.ts` ~218), bypassing + the output layer the spec’s style contract mandates; `update --json` returns prose + strings in `needsDecision` rather than names. +15. `UpdateAction` includes a never-returned `'noop'` member; the update tryscript uses + GNU-only `sed -i` (fails if run locally on macOS). +16. **Spec is behind the code in one place**: the version-skew guard (`skip-newer-base`, + a good design addition, documented in §2.9 invariant 7) has no row in the spec’s + update decision table. + Add it. + +### Strengths worth keeping + +The derived-state design (no stored tracking ⇒ git can’t desync it) is the right call +and §2.9 articulates it precisely; the stored-base three-way merge is the correct answer +to the shadcn-has-no-update-story problem and Alternatives #5 justifies the cost +honestly; out-of-band deletion as a supported state with exactly two resolutions is +excellent UX thinking; the decision-table unit tests cover every row × strategy +including the skew guard; the abort-upgrade recipe with its state-inventory table is the +kind of operational doc most projects never write; and the drift tryscript (rename → +`missing`+`local`, prune-on-empty) tests realistic mess, not just happy paths. + +## 2. Holistic Documentation Review + +**Surface inventory after this PR** (maturity in parens): + +| Surface | Audience | State | +| --- | --- | --- | +| `tbd-design.md` §2.9 + §4.13 | contributors/design | **Strong, new canonical doc model** | +| `tbd-docs.md` “Forked Docs in Your Repo” + “Aborting a Format Upgrade” | users | Good, but see inversion below | +| README “Upgrading” + skill-baseline upgrade hints | users/agents | Good; upgrade ergonomics now first-class | +| `development.md` format-upgrade section | contributors | Good | +| The plan spec | design | Strong but now diverges from code (§1, finding 4) | +| `docs-overview.md` | contributors | **Stale** — still describes only the old `--add` flags; its Phase 0 contract row was not executed | +| `tbd-docs.md` command reference | users | **Missing the new commands entirely** — fork/unfork/status/update/diff/list ship in this PR but are documented nowhere in the manual; discovery is `--help` only | +| `welcome-user.md`, skill routing rows, `suggest-upstream-improvements.md`, `docref-format.md`, `docmap-format.md` | agents/users | Absent (Phases 0/5 promised, not landed) | + +**Code currently leads docs on this branch**: the commands exist but the manual, the +onboarding, and the agent routing don’t yet — today an agent discovers `tbd docs fork` +via `--help` and the (broken) `--category` hint with zero guidance. +Since this PR is the complete experience, the docs must catch up before release; +concretely: + +1. **Re-couple agent surface to command surface.** The minimal skill routing rows ("make + guidelines visible" → fork; “update the guidelines” → update; “I deleted a forked + file” → status/restore/finalize) and a short `tbd-docs.md` “Managing docs” section + are part of this PR’s release bar — a command without its routing row is invisible to + the primary operator (agents). +2. **One first-principles “Managing docs” chapter in `tbd-docs.md`**, opening with the + two-mode model users actually face — *hidden cache* (default: docs live in gitignored + `.tbd/docs/`, always active, zero repo footprint) vs *forked* (tracked in + `docs/tbd/`, visible on GitHub, editable, mergeable) — then the scope axis (all vs by + category), then commands, then the drift table. + The content exists today but is scattered across the spec (two-axis framing), §2.9 + (model), and tbd-docs (drift table). + The spec’s Documentation Contract table already prescribes exactly this; execute it. +3. **Avoid the dual-drift-table trap.** The user-action table in `tbd-docs.md` and the + drift matrix in §2.9 describe the same truths in different words for different + audiences — fine per the ownership/audience rule, but make one canonical (suggest + §2.9) and have the other cite it, so the next state addition doesn’t fork them. +4. **Upgrade workflow is now well covered** (README → manual troubleshooting → + development.md → §2.9, correctly layered by audience). + One gap: the README “Upgrading” section should add one line: “if you’ve forked docs, + `tbd sync` will tell you when upstream moved — run `tbd docs update`.” +5. **`docs-overview.md`** needs its promised rewrite (the `tbd docs` group, docref-based + `add`, fork mention) — it’s the repo’s own orientation doc and currently teaches the + superseded surface. +6. **The three-sync taxonomy** (sync vs setup vs docs sync vs docs update) is the single + most confusion-prone area for users; the spec’s 4-row table is the right artifact and + should land in `tbd-docs.md` verbatim as the contract says (today a 3-row variant + lives only in design §4.13). + +## 3. The Two Design Questions + +### Q1: Copy-all-and-gitignore-some vs export-only-the-forked + +**Recommendation: keep the current export-only model. +Don’t build the gitignore-workflow variant, even as an option.** + +The decisive observations: + +- **Gitignored mirrors don’t actually deliver the visibility goal.** The original + complaint is “can’t browse them *on GitHub*, can’t check them in.” + Gitignored files appear in neither GitHub nor PRs — so under copy-all, the unforked + majority is exactly as invisible to the team as `.tbd/docs/` is today; the cache has + only moved to a prettier path. + The only incremental benefit is local-editor browsing, which `.tbd/docs/` (plain local + files) already provides. +- **It creates the worst silent failure mode in the design space.** A user or agent + *will* edit a gitignored mirror file (they sit right next to tracked ones, and agents + grep first and check ignore rules never). + The edit works locally, is served (fork dir has top precedence), and silently never + reaches the team — no commit, no PR review, no record. + The current design makes visibility an explicit act (fork = start tracking), so + divergence is impossible without a git-visible artifact. + That property is load-bearing; the copy-all option destroys it. +- **Gitignore-as-state-machine is fragile in well-known ways**: ignoring an + already-tracked file does nothing (the #1 gitignore confusion); `git add -f` + accidents; “is doc X forked?” + becomes a predicate over *two* systems (index + ignore rules) instead of one manifest + — and the derived-state achievement (§2.9 invariant 5: no sequence of git operations + can desync tbd) stops holding, because ignore rules are not content. +- **Upgrade and deletion semantics get contradictory.** Today “nothing is ever silently + re-created against the user’s deletion” is a clean principle. + With a mirror, `tbd docs sync` must re-write unforked mirror files on every upgrade — + so deleting one either resurrects (violating the principle) or requires tombstones + (new state machinery). +- **“See everything, then choose” is already served** — and better: `tbd docs list` + (with sizes/descriptions) is the catalog; `tbd docs show ` (Phase 2 — worth + pulling earlier, since it’s the browse-without-forking command); and crucially + `tbd docs fork --all` *is* the copy-all option in tracked form: every doc visible in + `docs/tbd/`, on GitHub, with the manifest tracking all of them — and `unfork` (or just + not committing) is the undo. + A user who wants the all-visible experience can have it today with one command and + real visibility, instead of a fake one. + +The fallback intuition — tbd should always use `docs/tbd` as the first source and fall +back to its internal cached copies — is exactly what’s implemented (the precedence list, +§2.9 invariant 1) and is correct *independent of this choice*; it’s also what makes +out-of-band deletion degrade gracefully. +The action item from §1 is just to make it true for shortcuts. + +One simplification to consider: since `fork --all` is the sanctioned “show me +everything” path, make the zero-fork `tbd docs` overview present **three** named +postures — hidden (default), curated (`--category`), everything (`--all`) — so the +catalog→choice flow is explicit without any gitignore machinery. + +### Q2: Granular doc-map registry vs search path + +**What the PR actually builds is a hybrid, and the split is right — but it should be +stated as a principle, in the docs, because right now it has to be reverse-engineered.** +The principle is: + +> **Resolve by convention; track only what cannot be derived; publish the inventory as a +> generated view.** + +- **Resolution = search path** (fork dir → [future `local_dirs`] → cache, + first-match-wins, names-are-identity, flat kind dirs). + Right because: zero registration ceremony (drop a file → served, the `local` state for + free), and *a registry can’t be wrong if there is no registry* — every stale-registry + failure mode (file says X, disk says Y) is structurally impossible. + The lookup_path bug found in §1 is the cautionary tale for the alternative: the one + place resolution *is* config-state-driven is exactly where the feature broke. +- **The manifest tracks only the non-derivable fact**: which upstream a fork came from + and the base snapshot at the fork point. + A merge base cannot be recomputed from disk; everything else + (customized/stale/missing/local) is derived by hashing. + So the manifest is minimal by construction — it’s not a doc registry, it’s a + provenance ledger, and only for docs that have an upstream relationship. +- **docmap = generated view**, so it’s always true. + Making it *authoritative* (file-by-file management in docmap format) would mean: every + add/rename/delete must update the map (agents and humans will skip this), map merge + conflicts, and the stale-registry failure class — in exchange for capabilities not yet + needed. + +**What is given up, honestly stated**: + +1. **Arbitrary layouts.** A team with an existing guidelines tree in non-tbd layout + can’t map names→paths; they must move files into `/.md` (or wait for + `local_dirs`, which only adds more convention-shaped dirs). + An authoritative map could redirect per-doc. +2. **Per-doc metadata overrides and ordering** (title/description live in frontmatter + only). +3. **A committed machine-readable inventory** for external consumers to read from the + repo at rest (today they’d have to run `tbd docs list --json`). + +**Why the loss is acceptable and reversible**: the docmap *format* already carries +everything an authoritative registry would need (`type`, `name`, `path`, `source`). So +the future move — “tbd can also *read* a committed docmap as a doc source” (#117’s +framework, as ‘operations over docmaps’) — adds a consumer without changing the format +or breaking anything shipped. +The option stays open at zero cost. +What should *not* happen is making the docmap authoritative preemptively: wait for a +concrete user with the arbitrary-layout problem. + +Two refinements to make now so the story is crisp (both feed §4): + +- Fix the location inconsistency: tbd’s own `docs list --json` currently emits upstream + entries with **neither `path` nor `source`** (`docs-fork.ts:573–581`), while the + format’s definition says every entry has a location. + Emit `source: internal:…` for upstream docs (already computed for forking). + Then the docmap is genuinely usable as an inventory by third parties. +- Write the one-paragraph “resolution is by search path; the docmap is a view, not an + input (today)” statement into `docmap-format.md` and §2.9, so nobody later “fixes” the + system into registry-driven resolution by accident. + +## 4. Deep Review: DocRef and DocMap Abstractions + +Both modules are well-built as code: dependency-free as claimed, small public APIs, +spec-mirror tests, extraction-ready. +This section is about the *abstractions* — where they’re exactly right, and where v0.1 +currently overcommits or under-specifies in ways that are cheap to fix now and expensive +later. Verdict: **DocRef needs four cuts/clarifications to be the universal grammar +intended; DocMap needs two tightenings and one deletion. +Neither needs more features.** + +### DocRef (`src/docref/docref.ts`) + +**What’s right and should be preserved**: single-string, totally-parsed, typed result; +the `//` repo/path separator is genuinely better than GitHub’s own blob-URL ambiguity +(branch names containing `/` parse correctly — `github:o/r@feature/x//path` works +because the separator is unambiguous); normalization of `blob`/`raw` web URLs to one +canonical form is exactly the right kind of opinion; idempotent `parse∘format` +round-trips are tested. + +**Issue 1 — cut the `git:` scheme (overcommitment).** `git:owner/repo//path` has no +hostname, so it’s unresolvable — there’s nothing a consumer can fetch. +Worse, the natural reading `git:host.com/owner/repo//path` mis-parses today +(owner=`host.com`, repo=`owner`, path swallows the rest). +Shipping an unresolvable, mis-parsing scheme in a v0.1 grammar is exactly the +“complexity that might not be correct later” risk: once any manifest contains a `git:` +ref it must be supported forever. +Cut it from v0.1; add a host-bearing form (`git:host/owner/repo@ref//path`) when a +non-GitHub/GitLab need actually appears. + +**Issue 2 — decide the bare-path question deliberately (currently the grammar validates +almost nothing).** `parseDocRef('hello world')` succeeds as a local path, so +`isDocRef()` is true for nearly any string and validation is toothless. +For a *universal* address format the grammar should be strict — local paths must start +with `./`, `../`, or `/` — letting each consumer decide to coerce bare strings at its +own boundary (tbd can keep accepting `guidelines/python-rules.md` in config by +prepending `./` before parse). +Strict grammar + lenient consumers composes; lenient grammar can never be tightened. +If the lenient rule stays, document it as a deliberate decision in `docref-format.md`, +because it surprises. + +**Issue 3 — two local-path holes**: (a) `~/` parses as local but no expansion semantics +are defined anywhere — define ("consumers expand to the user home") or reject in v0.1 +(recommend reject; it’s a config-file convenience that can come later); (b) Windows +drive-letter paths (`C:/Users/...`) hit the `unknown scheme` rejection (`docref.ts:172`) +since `C:` matches the scheme regex. +CI runs Windows; a Windows user’s absolute path is a legitimate address. +Either special-case `^[A-Za-z]:[\\/]` as local, or document “absolute paths are +POSIX-style” — but choose explicitly. + +**Issue 4 — URL fragments are silently dropped during normalization.** +`https://github.com/o/r/blob/main/f.md#testing` normalizes to `github:o/r@main//f.md` — +the `#testing` vanishes (`gitRefFromUrl` reads `pathname` only). +For *documents*, fragments are meaningful (tbd itself has `--section`). v0.1 can rule +fragments out of scope, but silent data loss in a *normalizer* is the one behavior a +format can’t afford. +Either preserve (add an optional `fragment` to the git/url kinds — small) or reject refs +with fragments with a clear error. +Recommend preserve: it’s one optional field and it future-proofs section addressing. + +**Smaller notes**: `docRefsEqual` is syntactic — fine, but say so in the format doc +(GitHub owners are case-insensitive; case is deliberately not normalized). +`internal:` is fine to keep in the universal grammar *if* the format doc defines it +app-relatively ("the consuming tool’s bundled collection") rather than as tbd-specific. +And the reference doc should cite purl (package-url) as prior art and say why it doesn’t +fit (package-centric identity, no good in-repo file story) — reviewers will ask. + +**The biggest DocRef gap is not in the code**: `references/docref-format.md` doesn’t +exist, so the grammar’s only spec is a module docstring — while Resolved Decision 10 +declares it a “hard rule with no exceptions” and *nothing in the shipped code parses a +docref anywhere* (zero imports outside the module; manifest `source` strings are built +by string concatenation, `docs-fork.ts:81–89`). Before the next release: write the +reference doc, and wire `tryParseDocRef` validation into at least manifest read and +`docs status` so the hard rule is enforced somewhere real. + +### DocMap (`src/docmap/docmap.ts`) + +**What’s right**: one object, one entry shape; `passthrough()` for extension fields with +“consumers must ignore unknown fields”; identity uniqueness enforced; self-identifying +version tag. This is the right size for v0.1. + +**Tighten 1 — require a location.** The format’s own definition says each entry has “a +location (`path`, and/or a provenance `source`)” but the schema makes both optional and +tbd’s first producer emits entries with neither (§3 Q2). Add a zod refinement: at least +one of `path`/`source` per entry. +An inventory whose entries can’t be located isn’t an inventory; this is the single +change that makes hand-authored docmaps in other repos actually consumable. + +**Tighten 2 — pin path-relativity.** Nothing says what `path` is relative to. +For a committed docmap file the only sane answer is *relative to the docmap file’s own +directory* (the sitemap convention); for generated/streamed docmaps (tbd’s `--json`), +relative to a stated collection root. +One paragraph in `docmap-format.md`; without it, two consumers will disagree on day one. + +**Delete 1 — drop `word_count` from the core format.** It’s the only presentation field +with a unit opinion baked in, and tbd — the format’s first and only producer — doesn’t +emit it (it renders bytes + approx tokens instead, `docs-fork.ts:568`). A core field the +reference implementation skips is a credibility leak. +Let size/length metrics be extension fields (`size_bytes`, `approx_tokens`, `word_count` +— whatever a producer has); keep core = identity + location + `title`/`description`. + +**Smaller notes**: `parseDocMap` accepts any `docmap/*` version — fine for 0.x, but +state the policy ("readers accept `docmap/0.*`, warn on others"). +`entryKey`/`groupByType`/`filterByType` are good minimal helpers; resist adding more (no +merge/diff helpers until an operation needs them — that’s the #117 layer, deliberately +deferred). + +**Using this use case to refine the format**: the fork feature is teaching the right +lesson — tbd needed `state`/`stale` and put them in extension fields, not the core. +That’s the pattern holding up well. +The two places the use case exposes real format gaps are exactly location-requiredness +and path-relativity above; fix those and docmap/0.1 is something another tool could +adopt as-is. + +## 5. Suggested Next Steps + +This PR ships the complete f05 experience as the next release; in priority order: + +1. Fix the three correctness blockers — shortcut serving precedence, the `--category` + hint, the merge-file exit-code guard. +2. Reconcile the golden maps with shipped output (one source of truth); from then on + validate each phase against its golden block as it lands — the Phase 0.5 discipline, + applied in-PR. +3. Complete the remaining phase items per the audit in finding 4; the old-surface + re-homing (Phase 2) and the agent surface (Phase 5) are the release-critical tail. +4. Apply the DocRef/DocMap tightenings (§4) while nothing depends on them. +5. Keep the PR title/description current with the phase checklist as items complete. From 6b4d2662cfa7b647a0f3ce08e46becc9e532a9f1 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 12 Jun 2026 17:37:39 +0000 Subject: [PATCH 19/36] fix: Harden forkable docs against adversarial-review findings MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Address the Tier 1 + Tier 2 findings from the forkable-docs adversarial review, plus doc-accuracy corrections, each with unit/golden coverage. Code: - doctor (F1): an older-but-compatible common-dir layout is now a pending migration (warning, exit 0), not a mismatch error (was exit 1, which broke CI on un-migrated repos). `doctor --fix` routes through prepareDataSyncContext so it migrates BOTH config and layout (never a layout-only half-migration); a corrupt layout.yml is rewritten from config. - manifest (S1): serialize forks.yml read-modify-write under a shared lock so concurrent fork/unfork/update cannot drop entries to last-writer-wins. - manifest (S2/S8): validate doc names (isSafeDocName) and parse per-entry, dropping unsafe/malformed entries with a warning instead of aborting the whole read — a crafted name can no longer escape the fork dir, and one bad entry no longer takes down status/update for the rest. - merge (S5): LF-normalize all three inputs before git merge-file so a CRLF fork against an LF base/upstream does not report a spurious whole-file conflict. - conflicts (S7): detect *unresolved* conflicts by tbd's own marker labels, so a doc that legitimately contains conflict-marker examples is not stuck conflicted. - README injection (S6): sanitize fork names/paths written into the generated fork-dir README. - update (S3): surface skipped docs (conflicted/orphaned/missing/no-base/ newer-base) instead of silently swallowing them. - version-skew guard: skip a doc whose base was advanced by a newer tbd, under every strategy, until the client upgrades. - status hint (D3): point an empty repo at `tbd docs fork ` / `--all`. Docs: - tbd-docs.md abort recipe: warn that a concurrent tbd write re-stamps the layout and can undo an abort (revert config in step 1 before deleting the stamp in step 2); clarify that reverting config alone drops the format gate even when forks were already committed. - development.md: layout.yml mirrors the config's tbd_format (was a stale "f04"). https://claude.ai/code/session_01X8S12JzmmxEfLpYzgH8Y7E --- docs/development.md | 2 +- packages/tbd/docs/tbd-docs.md | 14 ++ packages/tbd/src/cli/commands/docs-fork.ts | 232 ++++++++++-------- packages/tbd/src/cli/commands/doctor.ts | 46 +++- packages/tbd/src/file/doc-fork.ts | 73 +++++- packages/tbd/src/file/fork-manifest.ts | 117 ++++++++- packages/tbd/src/file/fork-update.ts | 51 ++-- packages/tbd/tests/cli-docs-fork.tryscript.md | 2 +- .../tests/common-dir-layout-doctor.test.ts | 61 ++++- packages/tbd/tests/fork-manifest.test.ts | 98 +++++++- packages/tbd/tests/fork-update.test.ts | 33 ++- 11 files changed, 555 insertions(+), 174 deletions(-) diff --git a/docs/development.md b/docs/development.md index 962e5f1f..4a721781 100644 --- a/docs/development.md +++ b/docs/development.md @@ -355,7 +355,7 @@ the full specification. └── backups/ # Legacy local backups $GIT_COMMON_DIR/tbd/ # Shared by all linked worktrees of this repo -├── layout.yml # Common-dir layout metadata (same f04 format ID) +├── layout.yml # Common-dir layout metadata (mirrors config's tbd_format) ├── locks/ │ └── data-sync.lock/ # mkdir-based repo-scoped lock ├── backups/ # Shared migration/repair backups diff --git a/packages/tbd/docs/tbd-docs.md b/packages/tbd/docs/tbd-docs.md index a73bd4f1..b39f45d2 100644 --- a/packages/tbd/docs/tbd-docs.md +++ b/packages/tbd/docs/tbd-docs.md @@ -1188,6 +1188,12 @@ rm -rf docs/tbd .tbd/doc-forks After this, the previous tbd version works again, and re-running the upgrade later is safe — the migration is idempotent from any of these states. +Reverting `.tbd/config.yml` is enough to drop the format gate even if forks were already +committed: compatibility is decided only by `tbd_format` in the config, not by the +presence of `docs/tbd/` or `.tbd/doc-forks/`. Committed fork files simply become inert +`local` docs under the older version — harmless to leave in place, so step 3 is only for +cleanup, never required to abort. + Notes: - **The migration never writes issue data**, so the recipe above cannot lose issues — it @@ -1203,6 +1209,14 @@ Notes: (layout updated but not config, or config but not layout), the next command with the new version completes the migration; the abort recipe above also works from either partial state. +- **Quiesce other tbd processes first.** The same self-healing re-stamp that completes + an interrupted upgrade can also undo an abort. + Any concurrent `tbd` write (another worktree, a background agent, an editor hook) + re-stamps `layout.yml` from whatever `.tbd/config.yml` currently says. + If you delete `layout.yml` while the config is still on the new format — or before the + config revert in step 1 has landed — the next write recreates the stamp and reopens + the migration. Stop other agents and worktrees, do step 1 (revert the config) before + step 2 (delete the stamp), and the abort sticks. - Teammates each migrate their own machine-local stamp automatically; only the `.tbd/config.yml` change is shared (via your branch), so reverting that commit is the team-wide rollback. diff --git a/packages/tbd/src/cli/commands/docs-fork.ts b/packages/tbd/src/cli/commands/docs-fork.ts index 633de593..f132bf84 100644 --- a/packages/tbd/src/cli/commands/docs-fork.ts +++ b/packages/tbd/src/cli/commands/docs-fork.ts @@ -39,6 +39,7 @@ import { writeForkManifest, writeBaseContent, upsertFork, + withForkManifestLock, } from '../../file/fork-manifest.js'; import { forkDoc, @@ -123,28 +124,30 @@ class DocsForkHandler extends BaseCommand { return; } - let manifest = await readForkManifest(tbdRoot); const forked: string[] = []; - for (const t of targets) { - const result = await forkDoc({ - tbdRoot, - forkDir: FORK_DIR, - manifest, - kind: t.kind, - name: t.name, - source: t.source, - content: t.content, - tbdVersion: VERSION, - force: options.force, - }); - manifest = result.manifest; - forked.push(result.relPath); - if (!this.ctx.json) { - this.output.success(`Forked ${t.name} → ${result.relPath}`); + await withForkManifestLock(tbdRoot, async () => { + let manifest = await readForkManifest(tbdRoot); + for (const t of targets) { + const result = await forkDoc({ + tbdRoot, + forkDir: FORK_DIR, + manifest, + kind: t.kind, + name: t.name, + source: t.source, + content: t.content, + tbdVersion: VERSION, + force: options.force, + }); + manifest = result.manifest; + forked.push(result.relPath); + if (!this.ctx.json) { + this.output.success(`Forked ${t.name} → ${result.relPath}`); + } } - } - await writeForkManifest(tbdRoot, manifest); - await regenerateForkDirReadme(tbdRoot, FORK_DIR, manifest); + await writeForkManifest(tbdRoot, manifest); + await regenerateForkDirReadme(tbdRoot, FORK_DIR, manifest); + }); if (this.ctx.json) { this.output.data({ forked }); @@ -230,41 +233,43 @@ class DocsUnforkHandler extends BaseCommand { async run(names: string[], options: UnforkOptions): Promise { await this.execute(async () => { const tbdRoot = await requireInit(); - let manifest = await readForkManifest(tbdRoot); + const removed: string[] = []; + await withForkManifestLock(tbdRoot, async () => { + let manifest = await readForkManifest(tbdRoot); - const targetNames = options.all ? manifest.forks.map((f) => f.name) : names; - if (targetNames.length === 0) { - throw new CLIError('Specify a doc name to unfork, or use --all.'); - } + const targetNames = options.all ? manifest.forks.map((f) => f.name) : names; + if (targetNames.length === 0) { + throw new CLIError('Specify a doc name to unfork, or use --all.'); + } - const removed: string[] = []; - for (const name of targetNames) { - try { - const result = await unforkDoc({ - tbdRoot, - forkDir: FORK_DIR, - manifest, - name, - kind: options.kind as ForkKind | undefined, - force: options.force, - }); - manifest = result.manifest; - removed.push(name); - if (!this.ctx.json) { - this.output.success(`Unforked ${name} — served from upstream again.`); + for (const name of targetNames) { + try { + const result = await unforkDoc({ + tbdRoot, + forkDir: FORK_DIR, + manifest, + name, + kind: options.kind as ForkKind | undefined, + force: options.force, + }); + manifest = result.manifest; + removed.push(name); + if (!this.ctx.json) { + this.output.success(`Unforked ${name} — served from upstream again.`); + } + } catch (err) { + if (err instanceof ForkConflictError && err.code === 'customized') { + throw new CLIError( + `${name} has local customizations. Review with \`tbd docs status\`, then ` + + `re-run with --force to discard them and fall back to upstream.`, + ); + } + throw err; } - } catch (err) { - if (err instanceof ForkConflictError && err.code === 'customized') { - throw new CLIError( - `${name} has local customizations. Review with \`tbd docs status\`, then ` + - `re-run with --force to discard them and fall back to upstream.`, - ); - } - throw err; } - } - await writeForkManifest(tbdRoot, manifest); - await regenerateForkDirReadme(tbdRoot, FORK_DIR, manifest); + await writeForkManifest(tbdRoot, manifest); + await regenerateForkDirReadme(tbdRoot, FORK_DIR, manifest); + }); if (this.ctx.json) { this.output.data({ unforked: removed }); } @@ -303,7 +308,7 @@ class DocsStatusHandler extends BaseCommand { const rows: StatusRow[] = []; for (const entry of manifest.forks) { - const kind = entry.kind as ForkKind; + const kind = entry.kind; if (!caches.has(kind)) caches.set(kind, await buildKindCache(kind, tbdRoot)); const cacheHit = caches.get(kind)!.get(entry.name); const status = await forkStatusFor(tbdRoot, FORK_DIR, entry, cacheHit?.doc.content ?? null); @@ -354,7 +359,7 @@ class DocsStatusHandler extends BaseCommand { if (rows.length === 0) { console.log('No docs forked into the repo.'); console.log( - `Make some visible: ${colors.bold('tbd docs fork --category=general')} (and your languages)`, + `Make some visible: ${colors.bold('tbd docs fork ')} or ${colors.bold('tbd docs fork --all')}`, ); return; } @@ -414,76 +419,86 @@ class DocsUpdateHandler extends BaseCommand { : 'default'; const tbdRoot = await requireInit(); - let manifest = await readForkManifest(tbdRoot); - const selected = - names.length > 0 ? manifest.forks.filter((f) => names.includes(f.name)) : manifest.forks; - - const caches = new Map(); - const upstreamFor = async (entry: ForkEntry): Promise => { - const kind = entry.kind as ForkKind; - if (!caches.has(kind)) caches.set(kind, await buildKindCache(kind, tbdRoot)); - return caches.get(kind)!.get(entry.name)?.doc.content ?? null; - }; - const applied: { entry: ForkEntry; message: string }[] = []; const decisions: string[] = []; + const skipped: string[] = []; + + await withForkManifestLock(tbdRoot, async () => { + let manifest = await readForkManifest(tbdRoot); + const selected = + names.length > 0 ? manifest.forks.filter((f) => names.includes(f.name)) : manifest.forks; + + const caches = new Map(); + const upstreamFor = async (entry: ForkEntry): Promise => { + const kind = entry.kind; + if (!caches.has(kind)) caches.set(kind, await buildKindCache(kind, tbdRoot)); + return caches.get(kind)!.get(entry.name)?.doc.content ?? null; + }; + + for (const entry of selected) { + const result = await updateOne({ + entry, + forkContent: await readForkFile(tbdRoot, FORK_DIR, entry), + baseContent: await readForkBase(tbdRoot, entry), + upstreamContent: await upstreamFor(entry), + strategy, + runningVersion: VERSION, + }); - for (const entry of selected) { - const result = await updateOne({ - entry, - forkContent: await readForkFile(tbdRoot, FORK_DIR, entry), - baseContent: await readForkBase(tbdRoot, entry), - upstreamContent: await upstreamFor(entry), - strategy, - runningVersion: VERSION, - }); + const { newFileContent, newBaseContent } = result; + if (newFileContent === undefined && newBaseContent === undefined) { + if (result.needsDecision) { + decisions.push(result.message); + } else if (result.action !== 'skip-not-stale' && result.action !== 'noop') { + // Conflicted / orphaned / missing / version-skewed: actionable but + // not applied here — surface, never silently swallow. + skipped.push(result.message); + } + continue; + } - const { newFileContent, newBaseContent } = result; - if (newFileContent === undefined && newBaseContent === undefined) { - if (result.needsDecision) decisions.push(result.message); - continue; + if (!options.dryRun) { + if (newFileContent !== undefined) { + const abs = forkFilePath(tbdRoot, FORK_DIR, entry.kind, entry.name); + await mkdir(dirname(abs), { recursive: true }); + await writeFile(abs, newFileContent); + } + const updated: ForkEntry = { ...entry }; + if (newBaseContent !== undefined) { + await writeBaseContent(tbdRoot, entry.kind, entry.name, newBaseContent); + updated.base_hash = hashContent(newBaseContent); + // The base records its fork point's tbd version so older clients + // can detect (and refuse) a downgrade — see the update guard. + updated.tbd_version = VERSION; + } + if (result.setConflicted) { + updated.conflicted = true; + } else { + delete updated.conflicted; + } + manifest = upsertFork(manifest, updated); + } + applied.push({ entry, message: result.message }); } if (!options.dryRun) { - if (newFileContent !== undefined) { - const abs = forkFilePath(tbdRoot, FORK_DIR, entry.kind as ForkKind, entry.name); - await mkdir(dirname(abs), { recursive: true }); - await writeFile(abs, newFileContent); - } - const updated: ForkEntry = { ...entry }; - if (newBaseContent !== undefined) { - await writeBaseContent(tbdRoot, entry.kind, entry.name, newBaseContent); - updated.base_hash = hashContent(newBaseContent); - // The base records its fork point's tbd version so older clients - // can detect (and refuse) a downgrade — see the update guard. - updated.tbd_version = VERSION; - } - if (result.setConflicted) { - updated.conflicted = true; - } else { - delete updated.conflicted; - } - manifest = upsertFork(manifest, updated); + await writeForkManifest(tbdRoot, manifest); + await regenerateForkDirReadme(tbdRoot, FORK_DIR, manifest); } - applied.push({ entry, message: result.message }); - } - - if (!options.dryRun) { - await writeForkManifest(tbdRoot, manifest); - await regenerateForkDirReadme(tbdRoot, FORK_DIR, manifest); - } + }); // end withForkManifestLock if (this.ctx.json) { this.output.data({ dryRun: Boolean(options.dryRun), updated: applied.map((a) => a.entry.name), needsDecision: decisions, + skipped, }); return; } const colors = this.output.getColors(); - if (applied.length === 0 && decisions.length === 0) { + if (applied.length === 0 && decisions.length === 0 && skipped.length === 0) { console.log('All forked docs are up to date.'); return; } @@ -494,6 +509,13 @@ class DocsUpdateHandler extends BaseCommand { console.log(` ${colors.success('✓')} ${a.message}`); } } + if (skipped.length > 0) { + console.log(''); + console.log(`${skipped.length} doc(s) skipped:`); + for (const msg of skipped) { + console.log(` ${colors.warn('⚠')} ${msg}`); + } + } if (decisions.length > 0) { console.log(''); console.log(`${decisions.length} doc(s) need a decision:`); @@ -622,7 +644,7 @@ class DocsDiffHandler extends BaseCommand { const forkContent = await readForkFile(tbdRoot, FORK_DIR, entry); const baseContent = await readForkBase(tbdRoot, entry); - const cache = await buildKindCache(entry.kind as ForkKind, tbdRoot); + const cache = await buildKindCache(entry.kind, tbdRoot); const upstreamContent = cache.get(entry.name)?.doc.content ?? null; // Default: your file vs current upstream (the net fork). diff --git a/packages/tbd/src/cli/commands/doctor.ts b/packages/tbd/src/cli/commands/doctor.ts index 8915a91d..79da25bf 100644 --- a/packages/tbd/src/cli/commands/doctor.ts +++ b/packages/tbd/src/cli/commands/doctor.ts @@ -53,6 +53,7 @@ import { } from '../../file/git.js'; import { CommonDirLayoutError, + isLayoutUpgradeable, readCommonDirLayout, validateCommonDirLayout, withSharedDataSyncLock, @@ -1208,11 +1209,27 @@ class DoctorHandler extends BaseCommand { try { layout = await readCommonDirLayout(layoutPath); } catch (error) { + // A corrupt/unparseable layout is machine-local and regenerable from the + // config — make it fixable rather than a dead-end error. + if (fix && !this.checkDryRun('Rewrite corrupt common-dir layout from config')) { + const configRef = this.config; + await withSharedDataSyncLock(this.cwd, async () => + writeCommonDirLayout(sharedPaths, configRef), + ); + return { + name: 'Common-dir layout', + status: 'ok', + message: 'rewritten from config (was unreadable)', + path: layoutPath, + }; + } return { name: 'Common-dir layout', status: 'error', - message: error instanceof Error ? error.message : String(error), + message: `${error instanceof Error ? error.message : String(error)}`, path: layoutPath, + fixable: true, + suggestion: `Run: tbd doctor --fix (rewrites it from config), or delete ${layoutPath}`, }; } if (!layout) { @@ -1233,6 +1250,33 @@ class DoctorHandler extends BaseCommand { suggestion: 'Upgrade: npm install -g get-tbd@latest', }; } + // A layout from an older but compatible format than the (in-memory, + // already-migrated) config is the normal mid-migration state, not a + // mismatch: the format bump applies on the next data command. Surface it as + // an informational warning (exit 0, so CI on un-migrated f04 repos is not + // broken); --fix applies the FULL migration (config + layout) via the locked + // data-context path — never just the layout, which would half-migrate the + // repo and lock out older clients with nothing to commit. + if (isLayoutUpgradeable(layout, this.config)) { + if (fix && !this.checkDryRun('Apply pending format migration')) { + await prepareDataSyncContext(this.cwd); + return { + name: 'Common-dir layout', + status: 'ok', + message: `format migration applied (${layout.tbd_format} → ${this.config.tbd_format})`, + path: layoutPath, + }; + } + return { + name: 'Common-dir layout', + status: 'warn', + message: `format migration pending (${layout.tbd_format} → ${this.config.tbd_format}); applies on next write or 'tbd doctor --fix'`, + path: layoutPath, + fixable: true, + suggestion: 'Run: tbd doctor --fix (or any write command) to apply', + }; + } + try { validateCommonDirLayout(layout, this.config); return { name: 'Common-dir layout', status: 'ok', path: layoutPath }; diff --git a/packages/tbd/src/file/doc-fork.ts b/packages/tbd/src/file/doc-fork.ts index 9394ebb7..ac8cf897 100644 --- a/packages/tbd/src/file/doc-fork.ts +++ b/packages/tbd/src/file/doc-fork.ts @@ -29,10 +29,11 @@ import { type ForkKind, type ForkManifest, type ForkStatus, + compareVersionsLoose, computeForkStatus, findFork, hashContent, - hasConflictMarkers, + hasUnresolvedConflict, readBaseContent, removeBaseContent, upsertFork, @@ -78,7 +79,7 @@ async function pathExists(path: string): Promise { /** Error raised when a fork/unfork would lose user content; carries a reason code. */ export class ForkConflictError extends Error { constructor( - public readonly code: 'overwrite' | 'customized' | 'not-forked', + public readonly code: 'overwrite' | 'customized' | 'not-forked' | 'version-skew', message: string, ) { super(message); @@ -125,6 +126,22 @@ export async function forkDoc(params: ForkDocParams): Promise { const current = await readFile(absPath, 'utf-8'); const isUnmodifiedFork = hashContent(current) === existingEntry?.base_hash; if (isUnmodifiedFork) { + // Refreshing an unmodified fork advances its base to this client's cache. + // Refuse if the fork point was set by a NEWER tbd: refreshing would + // downgrade the doc to our older bundled content (the same hazard the + // update guard prevents). --force overrides. + if ( + !force && + existingEntry?.tbd_version !== undefined && + params.tbdVersion !== undefined && + compareVersionsLoose(params.tbdVersion, existingEntry.tbd_version) === -1 + ) { + throw new ForkConflictError( + 'version-skew', + `${name}: fork point was set by tbd ${existingEntry.tbd_version} (you have ` + + `${params.tbdVersion}) — upgrade tbd before re-forking, or use --force to downgrade`, + ); + } action = 'refreshed'; } else if (!force) { throw new ForkConflictError( @@ -177,7 +194,7 @@ export async function unforkDoc(params: UnforkDocParams): Promise { - const kind = entry.kind as ForkKind; + const kind = entry.kind; const absPath = forkFilePath(tbdRoot, forkDir, kind, entry.name); let forkContent: string | null = null; try { @@ -227,7 +244,7 @@ export async function forkStatusFor( baseHash: entry.base_hash, cacheHash: cacheContent == null ? undefined : hashContent(cacheContent), conflictedFlag: entry.conflicted, - markersPresent: forkContent !== null ? hasConflictMarkers(forkContent) : false, + markersPresent: forkContent !== null ? hasUnresolvedConflict(forkContent) : false, }); } @@ -238,10 +255,7 @@ export async function readForkFile( entry: ForkEntry, ): Promise { try { - return await readFile( - forkFilePath(tbdRoot, forkDir, entry.kind as ForkKind, entry.name), - 'utf-8', - ); + return await readFile(forkFilePath(tbdRoot, forkDir, entry.kind, entry.name), 'utf-8'); } catch { return null; } @@ -356,13 +370,38 @@ export async function computeForkDriftSummary( return summary; } +/** + * Sanitize untrusted text (a frontmatter blurb or doc name) for safe inclusion + * in the generated README list. The blurb comes from forked/local file content, + * so it must not be able to inject markdown structure, links, or raw HTML into a + * committed file rendered on GitHub: take the first line and strip the + * characters that break a single list-item context. + */ +function sanitizeForReadme(text: string): string { + const firstLine = text.split(/\r?\n/)[0] ?? ''; + return firstLine + .replace(/[<>[\]`|]/g, '') + .replace(/\s+/g, ' ') + .trim() + .slice(0, 200); +} + +/** Percent-encode each path segment so odd local filenames make valid links. */ +function readmeLinkPath(relPath: string): string { + return relPath + .split('/') + .map((seg) => encodeURIComponent(seg)) + .join('/'); +} + /** First frontmatter description (or title) of a doc file, for the README index. */ async function docBlurb(absPath: string): Promise { try { const data = matter(await readFile(absPath, 'utf-8')).data as Record; const description = typeof data.description === 'string' ? data.description : undefined; const title = typeof data.title === 'string' ? data.title : undefined; - return description ?? title; + const blurb = description ?? title; + return blurb ? sanitizeForReadme(blurb) : undefined; } catch { return undefined; } @@ -398,7 +437,12 @@ export async function regenerateForkDirReadme( suffix: string; } const rows: IndexRow[] = [ - ...manifest.forks.map((f) => ({ kind: f.kind, name: f.name, relPath: f.path, suffix: '' })), + ...manifest.forks.map((f) => ({ + kind: f.kind, + name: f.name, + relPath: forkRelPath(forkDir, f.kind, f.name), + suffix: '', + })), ...locals.map((l) => ({ ...l, suffix: ' *(local — not from an upstream)*' })), ].sort((a, b) => a.kind.localeCompare(b.kind) || a.name.localeCompare(b.name)); @@ -427,7 +471,12 @@ export async function regenerateForkDirReadme( } const blurb = await docBlurb(join(tbdRoot, row.relPath)); const fileName = row.relPath.split('/').slice(-2).join('/'); - lines.push(`- [**${row.name}**](./${fileName})${blurb ? ` — ${blurb}` : ''}${row.suffix}`); + // Local filenames are arbitrary on disk; escape the link text and encode the + // link target so a name like `xy.md` or `a b.md` can't break the README. + const label = sanitizeForReadme(row.name) || row.name.replace(/[<>[\]`|]/g, ''); + lines.push( + `- [**${label}**](./${readmeLinkPath(fileName)})${blurb ? ` — ${blurb}` : ''}${row.suffix}`, + ); } lines.push(''); await mkdir(join(tbdRoot, forkDir), { recursive: true }); diff --git a/packages/tbd/src/file/fork-manifest.ts b/packages/tbd/src/file/fork-manifest.ts index bbfca663..b31758b1 100644 --- a/packages/tbd/src/file/fork-manifest.ts +++ b/packages/tbd/src/file/fork-manifest.ts @@ -23,6 +23,8 @@ import { parse as parseYaml } from 'yaml'; import { z } from 'zod'; import { stringifyYaml } from '../utils/yaml-utils.js'; +import { withLockfile } from '../utils/lockfile.js'; +import { resolveSharedTbdPaths } from '../lib/paths.js'; /** Directory (repo-relative under `.tbd/`) holding all fork state. */ export const DOC_FORKS_DIR = '.tbd/doc-forks'; @@ -39,11 +41,25 @@ export type ForkKind = (typeof FORK_KINDS)[number]; // Schema // ============================================================================= +/** + * A safe doc name: no path separators, no `..`, no leading dot, no NUL. + * Names are used to build filesystem paths (and are a doc's identity), so a + * crafted manifest must not be able to escape the fork dir (e.g. via a + * `../../../../victim` name through `unfork --force`). Allows the punctuation + * real doc names use (letters, digits, `.`, `_`, `-`). + */ +const SAFE_DOC_NAME = /^[A-Za-z0-9][A-Za-z0-9._-]*$/; +export function isSafeDocName(name: string): boolean { + return SAFE_DOC_NAME.test(name) && !name.includes('..') && !name.endsWith('.md'); +} + export const ForkEntrySchema = z.object({ - /** Doc name (e.g. "python-rules"). */ - name: z.string().min(1), - /** Doc kind (guideline/shortcut/template/reference). */ - kind: z.string().min(1), + /** Doc name (e.g. "python-rules"). Constrained so it cannot escape the fork dir. */ + name: z.string().min(1).refine(isSafeDocName, { + message: 'invalid doc name (no path separators, "..", or leading dot)', + }), + /** Doc kind — must be one of the known fork kinds. */ + kind: z.enum(FORK_KINDS), /** Repo-relative path of the forked file (e.g. "docs/tbd/guidelines/python-rules.md"). */ path: z.string().min(1), /** Provenance docref the fork was created from. */ @@ -96,6 +112,61 @@ export function hasConflictMarkers(content: string): boolean { return /^<{7}/m.test(content) && /^={7}\s*$/m.test(content) && /^>{7}/m.test(content); } +/** + * The labels tbd writes into its three-way merge conflict markers. Detection of + * an *unresolved* conflict keys off these specific labels (not generic marker + * lines) so a forked doc that legitimately contains conflict-marker examples + * (e.g. a git tutorial, or our own golden-testing guideline) is not stuck + * `conflicted` forever after one unrelated `update --merge`. + */ +export const CONFLICT_LABELS = { + ours: 'ours (your fork)', + base: 'base (fork point)', + theirs: 'theirs (upstream)', +} as const; + +/** Whether `content` still carries tbd's own unresolved conflict markers. */ +export function hasUnresolvedConflict(content: string): boolean { + return ( + content.includes(`<<<<<<< ${CONFLICT_LABELS.ours}`) && + content.includes(`>>>>>>> ${CONFLICT_LABELS.theirs}`) + ); +} + +/** + * Loose semver comparison on major.minor.patch (prerelease ignored). Returns + * null when either version is unparseable — callers must not guard on a version + * they cannot parse (treat null as "do not block"). + */ +export function compareVersionsLoose(a: string, b: string): -1 | 0 | 1 | null { + const parse = (v: string): [number, number, number] | null => { + const m = /^(\d+)\.(\d+)\.(\d+)/.exec(v.trim()); + return m ? [Number(m[1]), Number(m[2]), Number(m[3])] : null; + }; + const pa = parse(a); + const pb = parse(b); + if (!pa || !pb) return null; + for (let i = 0; i < 3; i++) { + const x = pa[i]!; + const y = pb[i]!; + if (x < y) return -1; + if (x > y) return 1; + } + return 0; +} + +/** + * Run `fn` while holding the doc-forks manifest lock, serializing the + * read-modify-write of `forks.yml` across concurrent fork/unfork/update so + * entries are not lost to last-writer-wins. The lock lives in the machine-local + * git common dir (never committed), alongside the data-sync lock. + */ +export async function withForkManifestLock(tbdRoot: string, fn: () => Promise): Promise { + const paths = await resolveSharedTbdPaths(tbdRoot); + await mkdir(paths.sharedLocksDir, { recursive: true }); + return withLockfile(join(paths.sharedLocksDir, 'doc-forks.lock'), fn); +} + // ============================================================================= // State computation (pure) // ============================================================================= @@ -234,18 +305,48 @@ function isNotFound(err: unknown): boolean { return (err as NodeJS.ErrnoException | undefined)?.code === 'ENOENT'; } -/** Read the fork manifest, returning an empty manifest if none exists. */ +/** The outer manifest shape, before per-entry validation. */ +const ForkManifestEnvelopeSchema = z.object({ + forks: z.array(z.unknown()).default([]), +}); + +/** + * Read the fork manifest, returning an empty manifest if none exists. + * + * Parsing is tolerant per entry: a malformed or unsafe entry (bad name/kind, + * path-traversal attempt) is dropped with a warning rather than aborting the + * whole read. This both fails closed on a crafted entry (it is never returned, + * so commands never act on it — no out-of-tree deletes) and keeps one corrupt + * entry from taking down status/update for every other fork. + */ export async function readForkManifest(tbdRoot: string): Promise { + let content: string; try { - const content = await readFile(forksFilePath(tbdRoot), 'utf-8'); - const data = parseYaml(content) as unknown; - return ForkManifestSchema.parse(data ?? { forks: [] }); + content = await readFile(forksFilePath(tbdRoot), 'utf-8'); } catch (err) { if (isNotFound(err)) { return emptyManifest(); } throw err; } + const envelope = ForkManifestEnvelopeSchema.parse(parseYaml(content) ?? { forks: [] }); + const forks: ForkEntry[] = []; + let dropped = 0; + for (const raw of envelope.forks) { + const parsed = ForkEntrySchema.safeParse(raw); + if (parsed.success) { + forks.push(parsed.data); + } else { + dropped++; + } + } + if (dropped > 0) { + process.stderr.write( + `• Ignored ${dropped} invalid fork manifest entr${dropped === 1 ? 'y' : 'ies'} ` + + `in ${FORKS_FILE} (bad name/kind or unsafe path). Run 'tbd doctor' to review.\n`, + ); + } + return { forks }; } /** Write the fork manifest (creating `.tbd/doc-forks/` as needed). */ diff --git a/packages/tbd/src/file/fork-update.ts b/packages/tbd/src/file/fork-update.ts index c0884ae3..ad5be248 100644 --- a/packages/tbd/src/file/fork-update.ts +++ b/packages/tbd/src/file/fork-update.ts @@ -14,7 +14,14 @@ import { join } from 'node:path'; import { writeFile } from 'atomically'; -import { type ForkEntry, hashContent, hasConflictMarkers } from './fork-manifest.js'; +import { + type ForkEntry, + CONFLICT_LABELS, + compareVersionsLoose, + hashContent, + hasUnresolvedConflict, + normalizeLineEndings, +} from './fork-manifest.js'; /** Result of a three-way merge. */ export interface MergeResult { @@ -42,21 +49,25 @@ export async function mergeContents( const basePath = join(dir, 'base'); const otherPath = join(dir, 'other'); try { + // Normalize line endings before the merge. Hashing is LF-normalized, so a + // CRLF fork file vs an LF base/upstream would otherwise make git merge-file + // see every line as changed and report a spurious whole-file conflict. The + // merged output is LF (matching the hash basis). await Promise.all([ - writeFile(currentPath, current), - writeFile(basePath, base), - writeFile(otherPath, other), + writeFile(currentPath, normalizeLineEndings(current)), + writeFile(basePath, normalizeLineEndings(base)), + writeFile(otherPath, normalizeLineEndings(other)), ]); const args = [ 'merge-file', '-p', '-L', - labels.current ?? 'ours (your fork)', + labels.current ?? CONFLICT_LABELS.ours, '-L', - labels.base ?? 'base (fork point)', + labels.base ?? CONFLICT_LABELS.base, '-L', - labels.other ?? 'theirs (upstream)', + labels.other ?? CONFLICT_LABELS.theirs, currentPath, basePath, otherPath, @@ -123,26 +134,6 @@ export async function diffContents( } } -/** - * Loose semver comparison on major.minor.patch (prerelease ignored). - * Returns null when either version cannot be parsed — callers must not guard - * on an unparseable version. - */ -export function compareVersionsLoose(a: string, b: string): -1 | 0 | 1 | null { - const parse = (v: string): number[] | null => { - const m = /^(\d+)\.(\d+)\.(\d+)/.exec(v.trim()); - return m ? [Number(m[1]), Number(m[2]), Number(m[3])] : null; - }; - const pa = parse(a); - const pb = parse(b); - if (!pa || !pb) return null; - for (let i = 0; i < 3; i++) { - if (pa[i]! < pb[i]!) return -1; - if (pa[i]! > pb[i]!) return 1; - } - return 0; -} - /** Update strategy chosen by the user for non-clean cases. */ export type UpdateStrategy = 'default' | 'merge' | 'keep-ours'; @@ -212,8 +203,10 @@ export async function updateOne(input: UpdateOneInput): Promise message: `${name}: upstream removed this doc — keep your copy or 'tbd docs unfork ${name}'`, }; } - // An unresolved conflicted doc must be resolved before any update. - if (entry.conflicted && hasConflictMarkers(forkContent)) { + // An unresolved conflicted doc must be resolved before any update. Keys off + // tbd's own labeled markers so a doc that legitimately contains conflict-marker + // examples is not blocked forever. + if (entry.conflicted && hasUnresolvedConflict(forkContent)) { return { action: 'skip-unresolved', message: `${name}: unresolved conflict markers — resolve them first`, diff --git a/packages/tbd/tests/cli-docs-fork.tryscript.md b/packages/tbd/tests/cli-docs-fork.tryscript.md index d4687c24..cf320a70 100644 --- a/packages/tbd/tests/cli-docs-fork.tryscript.md +++ b/packages/tbd/tests/cli-docs-fork.tryscript.md @@ -33,7 +33,7 @@ the `doc-fork` unit tests; these blocks pin the CLI surface. ```console $ tbd docs status No docs forked into the repo. -Make some visible: tbd docs fork --category=general (and your languages) +Make some visible: tbd docs fork or tbd docs fork --all ? 0 ``` diff --git a/packages/tbd/tests/common-dir-layout-doctor.test.ts b/packages/tbd/tests/common-dir-layout-doctor.test.ts index 33d4786a..8e8a57f1 100644 --- a/packages/tbd/tests/common-dir-layout-doctor.test.ts +++ b/packages/tbd/tests/common-dir-layout-doctor.test.ts @@ -97,26 +97,69 @@ describeUnlessWindows('common-dir layout via CLI', { timeout: 30000 }, () => { }); describe('doctor --fix (H3)', () => { - it('repairs a layout/config tbd_format mismatch under the shared lock', async () => { + it('treats an older-format layout as a pending migration and applies it on --fix', async () => { const layoutPath = join(dir, '.git', 'tbd', 'layout.yml'); const original = await readFile(layoutPath, 'utf-8'); expect(original).toContain('tbd_format: f05'); - // Simulate a partial migration / manual edit by downgrading the layout. + // A layout behind the config (downgraded to f03 here) is the normal + // mid-migration state, not a mismatch. await writeFile(layoutPath, original.replace('tbd_format: f05', 'tbd_format: f03')); - // Plain doctor reports it as fixable. The mismatch is a ✗ finding so the - // exit is 1 (per tbd-r7rt). + // Plain doctor reports a pending migration: a warning (exit 0, so CI on + // un-migrated repos is not broken), fixable. const diagnose = runTbd(dir, ['doctor']); - expect(diagnose.status).toBe(1); + expect(diagnose.status).toBe(0); expect(diagnose.stdout + diagnose.stderr).toMatch(/Common-dir layout/i); - expect(diagnose.stdout + diagnose.stderr).toMatch(/mismatched|doctor --fix/i); + expect(diagnose.stdout + diagnose.stderr).toMatch(/migration pending|doctor --fix/i); + + // doctor --fix applies the migration; layout is re-stamped to current. + const fix = runTbd(dir, ['doctor', '--fix']); + expect(fix.status).toBe(0); + expect(await readFile(layoutPath, 'utf-8')).toContain('tbd_format: f05'); + }); + + it('does not false-positive on a healthy older repo; --fix fully migrates (never half)', async () => { + // Downgrade BOTH config and layout to f04: a consistent pre-migration + // repo, exactly what an f05 client sees before the first write. + const configPath = join(dir, '.tbd', 'config.yml'); + const layoutPath = join(dir, '.git', 'tbd', 'layout.yml'); + await writeFile( + configPath, + (await readFile(configPath, 'utf-8')).replace('tbd_format: f05', 'tbd_format: f04'), + ); + await writeFile( + layoutPath, + (await readFile(layoutPath, 'utf-8')).replace('tbd_format: f05', 'tbd_format: f04'), + ); - // doctor --fix rewrites layout.yml from config; resulting state is clean. + // Plain doctor must NOT error on a healthy un-migrated repo (was exit 1). + const diagnose = runTbd(dir, ['doctor']); + expect(diagnose.status).toBe(0); + // The config marker on disk is untouched by a read-only doctor run. + expect(await readFile(configPath, 'utf-8')).toContain('tbd_format: f04'); + + // --fix must produce a CONSISTENT f05 repo: BOTH config and layout + // migrated, never layout-only (which would lock out older clients with + // nothing to commit). const fix = runTbd(dir, ['doctor', '--fix']); expect(fix.status).toBe(0); - const repaired = await readFile(layoutPath, 'utf-8'); - expect(repaired).toContain('tbd_format: f05'); + expect(await readFile(configPath, 'utf-8')).toContain('tbd_format: f05'); + expect(await readFile(layoutPath, 'utf-8')).toContain('tbd_format: f05'); + }); + + it('rewrites a corrupt layout.yml from config on --fix', async () => { + const layoutPath = join(dir, '.git', 'tbd', 'layout.yml'); + await writeFile(layoutPath, 'this: is: not: valid: yaml: [\n'); + + // Plain doctor surfaces it as fixable with remediation (not a dead end). + const diagnose = runTbd(dir, ['doctor']); + expect(diagnose.status).toBe(1); + expect(diagnose.stdout + diagnose.stderr).toMatch(/doctor --fix|delete/i); + + const fix = runTbd(dir, ['doctor', '--fix']); + expect(fix.status).toBe(0); + expect(await readFile(layoutPath, 'utf-8')).toContain('tbd_format: f05'); }); it('surfaces future-format layout as needing a newer tbd (no fix attempted)', async () => { diff --git a/packages/tbd/tests/fork-manifest.test.ts b/packages/tbd/tests/fork-manifest.test.ts index 2bc7b673..587847b4 100644 --- a/packages/tbd/tests/fork-manifest.test.ts +++ b/packages/tbd/tests/fork-manifest.test.ts @@ -5,17 +5,22 @@ * hashes + conflicted flag -> state), per the spec's testing strategy. */ -import { describe, it, expect, beforeEach, afterEach } from 'vitest'; -import { mkdtemp, rm } from 'node:fs/promises'; +import { describe, it, expect, beforeEach, afterEach, vi } from 'vitest'; +import { mkdtemp, rm, mkdir, writeFile as writeFileRaw } from 'node:fs/promises'; import { tmpdir } from 'node:os'; -import { join } from 'node:path'; +import { join, dirname } from 'node:path'; import { type ForkEntry, + type ForkKind, + type ForkManifest, type DocState, + CONFLICT_LABELS, hashContent, normalizeLineEndings, hasConflictMarkers, + hasUnresolvedConflict, + isSafeDocName, computeForkStatus, findFork, upsertFork, @@ -23,6 +28,7 @@ import { emptyManifest, readForkManifest, writeForkManifest, + forksFilePath, readBaseContent, writeBaseContent, removeBaseContent, @@ -62,6 +68,52 @@ describe('hasConflictMarkers', () => { }); }); +describe('isSafeDocName', () => { + it('accepts the punctuation real doc names use', () => { + for (const n of ['python-rules', 'a', 'tbd.design', 'foo_bar', 'v1', 'A1._-']) { + expect(isSafeDocName(n)).toBe(true); + } + }); + + it('rejects path-traversal and otherwise-unsafe names (S2)', () => { + // These must never round-trip through the manifest: a crafted name is how a + // doc could otherwise escape the fork dir (e.g. unfork --force deleting an + // out-of-tree file). + for (const n of [ + '', + '..', + '../victim', + '../../etc/passwd', + 'a/b', + 'foo/../bar', + '/abs', + '.hidden', + 'name.md', // the ".md" is added by path construction, not part of identity + 'has space', + 'tab\tname', + 'nul\u0000name', + ]) { + expect(isSafeDocName(n)).toBe(false); + } + }); +}); + +describe('hasUnresolvedConflict', () => { + it("is true only for tbd's own labeled markers (S7)", () => { + const tbd = + `<<<<<<< ${CONFLICT_LABELS.ours}\nmine\n=======\n` + + `theirs\n>>>>>>> ${CONFLICT_LABELS.theirs}\n`; + expect(hasUnresolvedConflict(tbd)).toBe(true); + }); + + it('is false for generic/example conflict markers and plain prose', () => { + // A git tutorial (or our own golden-testing guideline) that shows generic + // <<<<<<< HEAD markers must not be treated as an unresolved tbd conflict. + expect(hasUnresolvedConflict('<<<<<<< HEAD\na\n=======\nb\n>>>>>>> branch\n')).toBe(false); + expect(hasUnresolvedConflict('no markers at all')).toBe(false); + }); +}); + describe('computeForkStatus matrix', () => { const BASE = hashContent('base'); const EDITED = hashContent('edited'); @@ -178,7 +230,7 @@ describe('computeForkStatus matrix', () => { }); describe('manifest helpers', () => { - const entry = (name: string, kind = 'guideline'): ForkEntry => ({ + const entry = (name: string, kind: ForkKind = 'guideline'): ForkEntry => ({ name, kind, path: `docs/tbd/${kind}s/${name}.md`, @@ -224,8 +276,44 @@ describe('filesystem round-trip', () => { expect(await readForkManifest(dir)).toEqual({ forks: [] }); }); + it('drops invalid/unsafe entries on read and warns, keeping good ones (S2/S8)', async () => { + const raw = [ + 'forks:', + ' - name: good-doc', + ' kind: guideline', + ' path: docs/tbd/guidelines/good-doc.md', + ' source: internal:guidelines/good-doc.md', + ' base_hash: sha256:abc', + ' - name: ../../../etc/evil', // path traversal -> dropped + ' kind: guideline', + ' path: whatever', + ' source: internal:x', + ' base_hash: sha256:def', + ' - name: bad-kind-doc', // unknown kind -> dropped + ' kind: not-a-kind', + ' path: docs/tbd/x/bad.md', + ' source: internal:x', + ' base_hash: sha256:ghi', + '', + ].join('\n'); + await mkdir(dirname(forksFilePath(dir)), { recursive: true }); + await writeFileRaw(forksFilePath(dir), raw, 'utf-8'); + + const writes: string[] = []; + const spy = vi.spyOn(process.stderr, 'write').mockImplementation((chunk: unknown) => { + writes.push(String(chunk)); + return true; + }); + const manifest = await readForkManifest(dir); + spy.mockRestore(); + + // Only the valid entry survives; the crafted/unsafe ones never reach callers. + expect(manifest.forks.map((f) => f.name)).toEqual(['good-doc']); + expect(writes.join('')).toContain('Ignored 2 invalid'); + }); + it('round-trips a manifest through write/read', async () => { - const manifest = { + const manifest: ForkManifest = { forks: [ { name: 'python-rules', diff --git a/packages/tbd/tests/fork-update.test.ts b/packages/tbd/tests/fork-update.test.ts index 6b003b48..1f0f804d 100644 --- a/packages/tbd/tests/fork-update.test.ts +++ b/packages/tbd/tests/fork-update.test.ts @@ -41,6 +41,18 @@ describe('mergeContents', () => { expect(result.conflicts).toBeGreaterThan(0); expect(hasConflictMarkers(result.merged)).toBe(true); }); + + it('merges a CRLF fork against LF base/upstream without spurious conflict (S5)', async () => { + // Without LF-normalization, a CRLF fork vs an LF base/upstream makes git + // merge-file see every line as changed and report a whole-file conflict. + // Edits are on non-adjacent lines (1 and 3) so the only thing under test is + // the line-ending mismatch, not git's adjacent-hunk conflict behavior. + const ours = 'line ONE\r\nline two\r\nline three\r\n'; // CRLF, edited line 1 + const theirs = 'line one\nline two\nline THREE\n'; // LF, edited line 3 + const result = await mergeContents(ours, BASE, theirs); + expect(result.conflicts).toBe(0); + expect(result.merged).toBe('line ONE\nline two\nline THREE\n'); + }); }); describe('diffContents', () => { @@ -87,8 +99,8 @@ describe('updateOne decision table', () => { expect(r.action).toBe('skip-orphaned'); }); - it('skips an unresolved conflicted doc', async () => { - const withMarkers = '<<<<<<< ours\na\n=======\nb\n>>>>>>> theirs\n'; + it('skips an unresolved conflicted doc (tbd-labeled markers)', async () => { + const withMarkers = '<<<<<<< ours (your fork)\na\n=======\nb\n>>>>>>> theirs (upstream)\n'; const r = await updateOne({ entry: entry({ conflicted: true }), forkContent: withMarkers, @@ -99,6 +111,21 @@ describe('updateOne decision table', () => { expect(r.action).toBe('skip-unresolved'); }); + it('does NOT treat generic/legit conflict-marker text as unresolved (S7)', async () => { + // A doc that merely contains example markers (e.g. a git tutorial) with the + // conflicted flag still set must not be stuck — only tbd's own labels count. + const generic = '<<<<<<< HEAD\na\n=======\nb\n>>>>>>> branch\n'; + const r = await updateOne({ + entry: entry({ conflicted: true }), + forkContent: generic, // unmodified vs base + baseContent: generic, + upstreamContent: generic + 'upstream line\n', // stale, non-conflicting + strategy: 'default', + }); + expect(r.action).not.toBe('skip-unresolved'); + expect(r.action).toBe('replaced'); + }); + it('is a no-op when not stale', async () => { const r = await updateOne({ entry: entry(), @@ -271,7 +298,7 @@ describe('version-skew guard', () => { }); it('compareVersionsLoose ignores prerelease and rejects garbage', async () => { - const { compareVersionsLoose } = await import('../src/file/fork-update.js'); + const { compareVersionsLoose } = await import('../src/file/fork-manifest.js'); expect(compareVersionsLoose('0.2.3-dev.333.abc', '0.2.3')).toBe(0); expect(compareVersionsLoose('0.2.3', '0.10.0')).toBe(-1); expect(compareVersionsLoose('1.0.0', '0.9.9')).toBe(1); From cf5beae000f224ded92d640fd9da393510ebd655 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 12 Jun 2026 17:43:04 +0000 Subject: [PATCH 20/36] test: Cover fork-refresh version-skew guard (S4) and README injection (S6) The code fixes for these two findings shipped without dedicated unit tests: - S4: forkDoc refuses to refresh an unmodified fork when the fork point was set by a newer tbd (would silently downgrade), with --force as the escape hatch; and proceeds when the running tbd is the same or newer. - S6: a hand-authored local doc's frontmatter blurb and (unvalidated) filename are sanitized/encoded before going into the generated fork-dir README, so they cannot inject markdown, links, raw HTML, or a broken link target. https://claude.ai/code/session_01X8S12JzmmxEfLpYzgH8Y7E --- packages/tbd/tests/doc-fork.test.ts | 89 +++++++++++++++++++++++++++++ 1 file changed, 89 insertions(+) diff --git a/packages/tbd/tests/doc-fork.test.ts b/packages/tbd/tests/doc-fork.test.ts index a4d5b17a..0d207dfd 100644 --- a/packages/tbd/tests/doc-fork.test.ts +++ b/packages/tbd/tests/doc-fork.test.ts @@ -96,6 +96,72 @@ describe('forkDoc', () => { expect(refreshed.action).toBe('refreshed'); expect(await readBaseContent(root, 'guideline', 'python-rules')).toBe(NEW_UPSTREAM); }); + + it('blocks a refresh when the fork point was set by a newer tbd (S4 version-skew)', async () => { + // The first fork records its fork point at a tbd newer than the one we now run. + const first = await forkDoc({ + tbdRoot: root, + forkDir: FORK_DIR, + manifest: emptyManifest(), + kind: 'guideline', + name: 'python-rules', + source: 'internal:guidelines/python-rules.md', + content: UPSTREAM, + tbdVersion: '0.9.0', + }); + + const OLDER_BUNDLE = '# Python Rules\n\nOlder bundled content.\n'; + const reForkOlder = (force = false) => + forkDoc({ + tbdRoot: root, + forkDir: FORK_DIR, + manifest: first.manifest, + kind: 'guideline', + name: 'python-rules', + source: 'internal:guidelines/python-rules.md', + content: OLDER_BUNDLE, + tbdVersion: '0.3.0', // older than the 0.9.0 fork point + force, + }); + + // Refreshing here would silently downgrade the doc to our older bundle — blocked. + const err = await reForkOlder().catch((e: unknown) => e); + expect(err).toBeInstanceOf(ForkConflictError); + expect((err as ForkConflictError).code).toBe('version-skew'); + // The base snapshot is untouched by the blocked refresh. + expect(await readBaseContent(root, 'guideline', 'python-rules')).toBe(UPSTREAM); + + // --force is the explicit downgrade escape hatch. + const forced = await reForkOlder(true); + expect(forced.action).toBe('refreshed'); + expect(await readBaseContent(root, 'guideline', 'python-rules')).toBe(OLDER_BUNDLE); + }); + + it('allows a refresh when the running tbd is the same or newer than the fork point', async () => { + const first = await forkDoc({ + tbdRoot: root, + forkDir: FORK_DIR, + manifest: emptyManifest(), + kind: 'guideline', + name: 'python-rules', + source: 'internal:guidelines/python-rules.md', + content: UPSTREAM, + tbdVersion: '0.9.0', + }); + const NEWER = '# Python Rules\n\nNewer upstream.\n'; + const refreshed = await forkDoc({ + tbdRoot: root, + forkDir: FORK_DIR, + manifest: first.manifest, + kind: 'guideline', + name: 'python-rules', + source: 'internal:guidelines/python-rules.md', + content: NEWER, + tbdVersion: '1.0.0', // newer than the fork point + }); + expect(refreshed.action).toBe('refreshed'); + expect(await readBaseContent(root, 'guideline', 'python-rules')).toBe(NEWER); + }); }); describe('forkStatusFor', () => { @@ -298,4 +364,27 @@ describe('drift helpers (local files, summary, README index)', () => { await regenerateForkDirReadme(root, FORK_DIR, empty); await expect(readFile(join(root, FORK_DIR, 'README.md'), 'utf-8')).rejects.toThrow(); }); + + it('sanitizes a local doc’s blurb and link target in the README index (S6)', async () => { + const { manifest } = await forkOne(); + + // A hand-authored local doc (no manifest entry, so its name is NOT + // isSafeDocName-validated): a filename with a space and a frontmatter + // description carrying markdown/HTML/link injection. + const evil = '---\n' + 'description: "Evil x [l](http://e) | `c`"\n' + '---\n# Local\n'; + await writeFile(join(root, FORK_DIR, 'guidelines', 'team rules.md'), evil); + + await regenerateForkDirReadme(root, FORK_DIR, manifest); + const readme = await readFile(join(root, FORK_DIR, 'README.md'), 'utf-8'); + + // The blurb's structure-breaking characters are stripped: no raw HTML, no + // injected link/code span, no table pipe. + expect(readme).not.toContain(''); + expect(readme).not.toContain(''); + expect(readme).not.toContain('[l]'); + expect(readme).not.toContain('`c`'); + // The link target for the spaced filename is percent-encoded so it stays a + // single valid link. + expect(readme).toContain('team%20rules.md'); + }); }); From 430122037f13906202510d9a2d6fccaf67b5acd3 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 12 Jun 2026 18:05:34 +0000 Subject: [PATCH 21/36] fix: Serve forked shortcuts past lookup_path; reject merge-file error exits Two release blockers from the senior review (PR comment): - Forked shortcuts were never served: setup persists docs_cache.lookup_path into every repo config, and the shortcut command let it replace the fork-dir-prepended defaults, so a forked shortcut showed [forked] in 'tbd docs list' while 'tbd shortcut ' served the upstream copy. The fork dir is now prepended structurally (tbd-design.md 2.9 invariant 1); golden-tested by a new fork-a-shortcut serve block. Closes tbd-62qe. - git merge-file exits with the conflict count truncated to 127; error exits (255, e.g. binary input) were misread as conflict counts with empty stdout, which would overwrite a customized fork with empty content. Exits above 127 now reject with stderr context. Closes tbd-xbpe. https://claude.ai/code/session_01QPsCSYGtwR8JtX2R1aaxyh --- packages/tbd/src/cli/commands/shortcut.ts | 13 ++++-- packages/tbd/src/file/fork-update.ts | 16 ++++--- packages/tbd/tests/cli-docs-fork.tryscript.md | 44 +++++++++++++++++++ packages/tbd/tests/fork-update.test.ts | 10 +++++ 4 files changed, 74 insertions(+), 9 deletions(-) diff --git a/packages/tbd/src/cli/commands/shortcut.ts b/packages/tbd/src/cli/commands/shortcut.ts index d2d7c564..b52fecb0 100644 --- a/packages/tbd/src/cli/commands/shortcut.ts +++ b/packages/tbd/src/cli/commands/shortcut.ts @@ -16,7 +16,7 @@ import { requireInit, CLIError } from '../lib/errors.js'; import { DocCache, SCORE_PREFIX_MATCH } from '../../file/doc-cache.js'; import { addDoc } from '../../file/doc-add.js'; import { readConfig } from '../../file/config.js'; -import { DEFAULT_SHORTCUT_PATHS } from '../../lib/paths.js'; +import { CACHE_SHORTCUT_PATHS, FORK_SHORTCUTS_DIR } from '../../lib/paths.js'; import { truncate } from '../../lib/truncate.js'; import { formatDocSize } from '../../lib/format-utils.js'; import { getTerminalWidth } from '../lib/output.js'; @@ -73,9 +73,16 @@ class ShortcutHandler extends BaseCommand { // Get tbd root (supports running from subdirectories) const tbdRoot = await requireInit(); - // Read config to get lookup paths (fall back to defaults) + // Read config to get lookup paths (fall back to the cache defaults). The + // fork dir is prepended structurally, not via config: a persisted + // lookup_path (setup writes one into every repo) must not be able to turn + // off fork shadowing (tbd-design.md §2.9 invariant 1). const config = await readConfig(tbdRoot); - const lookupPaths = config.docs_cache?.lookup_path ?? DEFAULT_SHORTCUT_PATHS; + const configured = config.docs_cache?.lookup_path ?? CACHE_SHORTCUT_PATHS; + const lookupPaths = [ + FORK_SHORTCUTS_DIR, + ...configured.filter((p) => p !== FORK_SHORTCUTS_DIR), + ]; // Create and load the doc cache with proper base directory const cache = new DocCache(lookupPaths, tbdRoot); diff --git a/packages/tbd/src/file/fork-update.ts b/packages/tbd/src/file/fork-update.ts index ad5be248..64e717a8 100644 --- a/packages/tbd/src/file/fork-update.ts +++ b/packages/tbd/src/file/fork-update.ts @@ -74,16 +74,21 @@ export async function mergeContents( ]; return await new Promise((resolve, reject) => { - execFile('git', args, { maxBuffer: MERGE_MAX_BUFFER }, (error, stdout) => { + execFile('git', args, { maxBuffer: MERGE_MAX_BUFFER }, (error, stdout, stderr) => { if (error) { const code = (error as NodeJS.ErrnoException & { code?: number }).code; - // git merge-file exits with the number of conflicts (>0); negative/other - // codes are real errors. - if (typeof code === 'number' && code > 0) { + // git merge-file exits with the number of conflicts, truncated to 127. + // Anything above that (255 = real error, e.g. binary input) must NOT be + // read as a conflict count: stdout is empty on errors, and writing it + // would clobber the forked file. + if (typeof code === 'number' && code > 0 && code <= 127) { resolve({ merged: stdout, conflicts: code }); return; } - reject(error instanceof Error ? error : new Error('git merge-file failed')); + const detail = String(stderr ?? '').trim(); + reject( + new Error(`git merge-file failed${detail ? `: ${detail}` : ''}`, { cause: error }), + ); return; } resolve({ merged: stdout, conflicts: 0 }); @@ -139,7 +144,6 @@ export type UpdateStrategy = 'default' | 'merge' | 'keep-ours'; /** What an update did (or why it was skipped) for one doc. */ export type UpdateAction = - | 'noop' | 'replaced' | 'merged-clean' | 'merged-conflict' diff --git a/packages/tbd/tests/cli-docs-fork.tryscript.md b/packages/tbd/tests/cli-docs-fork.tryscript.md index cf320a70..9de0a10e 100644 --- a/packages/tbd/tests/cli-docs-fork.tryscript.md +++ b/packages/tbd/tests/cli-docs-fork.tryscript.md @@ -254,3 +254,47 @@ $ test ! -e docs/tbd && echo "fork dir pruned" fork dir pruned ? 0 ``` + +* * * + +## Forked shortcuts shadow the cache (serving precedence) + +Config persists a `lookup_path` for shortcuts, so precedence must be structural: the +fork dir wins regardless of config (tbd-design.md §2.9 invariant 1). + +# Test: fork a shortcut and customize it + +```console +$ tbd docs fork review-code +✓ Forked review-code → docs/tbd/shortcuts/review-code.md + Regenerated docs/tbd/README.md +... +? 0 +``` + +```console +$ printf '\nFORK-SERVE-CHECK\n' >> docs/tbd/shortcuts/review-code.md +? 0 +``` + +# Test: the shortcut command serves the forked copy + +```console +$ tbd shortcut review-code 2>/dev/null | grep FORK-SERVE-CHECK +FORK-SERVE-CHECK +? 0 +``` + +# Test: unfork restores upstream serving + +```console +$ tbd docs unfork review-code --force +✓ Unforked review-code — served from upstream again. +? 0 +``` + +```console +$ tbd shortcut review-code 2>/dev/null | grep -c FORK-SERVE-CHECK +0 +? 1 +``` diff --git a/packages/tbd/tests/fork-update.test.ts b/packages/tbd/tests/fork-update.test.ts index 1f0f804d..72d9ad33 100644 --- a/packages/tbd/tests/fork-update.test.ts +++ b/packages/tbd/tests/fork-update.test.ts @@ -53,6 +53,16 @@ describe('mergeContents', () => { expect(result.conflicts).toBe(0); expect(result.merged).toBe('line ONE\nline two\nline THREE\n'); }); + + it('rejects on git merge-file errors instead of reading exit 255 as a conflict count', async () => { + // Binary input makes git merge-file refuse with exit 255. Error exits are not + // conflict counts (counts are truncated to 127); misreading one would write + // empty merged output over the user's forked file. + const binary = 'line one\u0000\nline two\n'; + await expect(mergeContents(binary, BASE, 'line one\nline two\nx\n')).rejects.toThrow( + /merge-file failed/, + ); + }); }); describe('diffContents', () => { From 6b6949e50fbf4fdeb1466e9b653330f677322e97 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 12 Jun 2026 18:06:02 +0000 Subject: [PATCH 22/36] fix: Harden docref v0.1 grammar (drop git:, strict locals, fragments) Per the DocRef review (epic tbd-3neh), tightening v0.1 before anything depends on it: - Drop the git: scheme: it had no hostname (unresolvable) and mis-parsed host-bearing forms. Additional protocols may be added in future versions. Closes tbd-s6tb. - Strict local paths: must be anchored with ./, ../, /, or a Windows drive letter; bare relative strings and ~ are rejected with actionable errors (consumers may coerce at their own boundary). Closes tbd-z9hs. - Windows drive-letter paths (C:/, C:\) parse as local instead of hitting the unknown-scheme rejection. Closes tbd-devl. - URL fragments are preserved through parsing and blob-URL normalization (new optional fragment field on git refs) instead of being silently dropped. Closes tbd-0n4l. - docRefsEqual documented as purely syntactic. https://claude.ai/code/session_01QPsCSYGtwR8JtX2R1aaxyh --- packages/tbd/src/docref/docref.ts | 102 +++++++++++++++++++++++------- packages/tbd/tests/docref.test.ts | 50 +++++++++++++-- 2 files changed, 122 insertions(+), 30 deletions(-) diff --git a/packages/tbd/src/docref/docref.ts b/packages/tbd/src/docref/docref.ts index 6961a9c8..4732d497 100644 --- a/packages/tbd/src/docref/docref.ts +++ b/packages/tbd/src/docref/docref.ts @@ -6,12 +6,23 @@ * used everywhere a doc's source or location is named: config source strings, the * fork manifest's `source` field, `tbd docs add` arguments, and `local_dirs` entries. * - * Supported forms: - * internal:guidelines/python-rules.md bundled doc shipped inside tbd - * ./docs/general/ in-repo path (local) - * /abs/path/file.md absolute local path - * https://example.com/style.md plain URL - * github:owner/repo@ref//path/to/file.md git-hosted (also gitlab:, git:) + * Supported forms (docref v0.1): + * internal:guidelines/python-rules.md bundled doc shipped inside the consuming + * tool (app-relative, not tbd-specific) + * ./docs/general/ ../shared/ /abs/f.md local paths — must be anchored with + * "./", "../", "/", or a Windows drive + * letter (C:/ or C:\) + * https://example.com/style.md plain URL + * github:owner/repo@ref//path/to/file.md git-hosted (also gitlab:) + * github:owner/repo@ref//file.md#section optional fragment, preserved + * + * The grammar is deliberately strict: bare relative strings ("guidelines/x.md") and + * home-relative paths ("~/x.md") are NOT valid docrefs. Consumers that want lenient + * input may coerce at their own boundary (e.g. prepend "./") before parsing — a + * strict grammar plus lenient consumers composes; the reverse cannot be tightened. + * + * Additional protocols (for example a host-bearing git scheme for forges beyond + * GitHub/GitLab) may be added in future versions. * * Web URLs that point at a known git host are normalized to the `github:`/`gitlab:` * form so there is one canonical address for a given file: @@ -20,7 +31,7 @@ */ /** Scheme of a git-hosted docref. */ -export type GitHost = 'github' | 'gitlab' | 'git'; +export type GitHost = 'github' | 'gitlab'; /** A parsed document reference. */ export type DocRef = @@ -34,6 +45,8 @@ export type DocRef = readonly repo: string; readonly ref?: string; readonly path: string; + /** Optional in-document anchor (e.g. a heading slug), preserved verbatim. */ + readonly fragment?: string; }; /** Error thrown when a string is not a valid docref. */ @@ -47,15 +60,18 @@ export class DocRefError extends Error { } } -const GIT_SCHEMES: readonly GitHost[] = ['github', 'gitlab', 'git']; +const GIT_SCHEMES: readonly GitHost[] = ['github', 'gitlab']; -/** True for strings that address a local filesystem path rather than a scheme. */ +/** + * True for strings that address a local filesystem path: anchored relative + * (`./`, `../`), absolute (`/`), or a Windows drive-letter path (`C:/`, `C:\`). + */ function looksLocal(input: string): boolean { return ( input.startsWith('./') || input.startsWith('../') || input.startsWith('/') || - input.startsWith('~/') + /^[A-Za-z]:[\\/]/.test(input) ); } @@ -65,7 +81,7 @@ function tidyLocal(path: string): string { } /** - * Parse a `host:owner/repo[@ref]//path` body (everything after the scheme). + * Parse a `host:owner/repo[@ref]//path[#fragment]` body (everything after the scheme). */ function parseGitBody(host: GitHost, body: string, input: string): DocRef { const sep = body.indexOf('//'); @@ -73,7 +89,11 @@ function parseGitBody(host: GitHost, body: string, input: string): DocRef { throw new DocRefError(input, `git docref must contain "//" separating repo from path`); } const repoPart = body.slice(0, sep); - const path = body.slice(sep + 2); + const pathPart = body.slice(sep + 2); + + const hashIndex = pathPart.indexOf('#'); + const path = hashIndex === -1 ? pathPart : pathPart.slice(0, hashIndex); + const fragment = hashIndex === -1 ? undefined : pathPart.slice(hashIndex + 1) || undefined; if (!path) { throw new DocRefError(input, 'git docref has an empty path'); } @@ -92,14 +112,21 @@ function parseGitBody(host: GitHost, body: string, input: string): DocRef { throw new DocRefError(input, 'git docref must be "owner/repo"'); } - return ref === undefined - ? { kind: 'git', host, owner, repo, path } - : { kind: 'git', host, owner, repo, ref, path }; + return { + kind: 'git', + host, + owner, + repo, + path, + ...(ref !== undefined ? { ref } : {}), + ...(fragment !== undefined ? { fragment } : {}), + }; } /** * If `url` points at a known git host's file view, return the equivalent git * docref; otherwise return null (caller keeps it as a plain URL). + * URL fragments are preserved — normalization must never silently drop data. */ function gitRefFromUrl(url: string): DocRef | null { let parsed: URL; @@ -109,22 +136,40 @@ function gitRefFromUrl(url: string): DocRef | null { return null; } const segments = parsed.pathname.split('/').filter(Boolean); + const fragment = parsed.hash ? parsed.hash.slice(1) || undefined : undefined; + const frag = fragment !== undefined ? { fragment } : {}; // https://github.com/{owner}/{repo}/blob/{ref}/{path...} if (parsed.hostname === 'github.com' && segments[2] === 'blob' && segments.length >= 5) { const [owner, repo, , ref, ...rest] = segments; - return { kind: 'git', host: 'github', owner: owner!, repo: repo!, ref, path: rest.join('/') }; + return { + kind: 'git', + host: 'github', + owner: owner!, + repo: repo!, + ref, + path: rest.join('/'), + ...frag, + }; } // https://raw.githubusercontent.com/{owner}/{repo}/{ref}/{path...} if (parsed.hostname === 'raw.githubusercontent.com' && segments.length >= 4) { const [owner, repo, ref, ...rest] = segments; - return { kind: 'git', host: 'github', owner: owner!, repo: repo!, ref, path: rest.join('/') }; + return { + kind: 'git', + host: 'github', + owner: owner!, + repo: repo!, + ref, + path: rest.join('/'), + ...frag, + }; } // https://gitlab.com/{owner}/{repo}/-/blob/{ref}/{path...} if (parsed.hostname === 'gitlab.com' && segments[2] === '-' && segments[3] === 'blob') { const [owner, repo, , , ref, ...rest] = segments; if (owner && repo && ref && rest.length > 0) { - return { kind: 'git', host: 'gitlab', owner, repo, ref, path: rest.join('/') }; + return { kind: 'git', host: 'gitlab', owner, repo, ref, path: rest.join('/'), ...frag }; } } return null; @@ -162,17 +207,24 @@ export function parseDocRef(input: string): DocRef { return gitRefFromUrl(raw) ?? { kind: 'url', url: raw }; } - // Local filesystem paths. + // Local filesystem paths (anchored; includes Windows drive letters). if (looksLocal(raw)) { return { kind: 'local', path: raw }; } - // A scheme-less, non-URL string with no path markers is treated as a local - // relative path (e.g. "guidelines/python-rules.md"). A stray scheme is rejected. + if (raw.startsWith('~')) { + throw new DocRefError( + input, + 'home-relative (~) paths are not supported; use an absolute or ./-relative path', + ); + } if (/^[a-z][a-z0-9+.-]*:/i.test(raw)) { throw new DocRefError(input, 'unknown scheme'); } - return { kind: 'local', path: raw }; + throw new DocRefError( + input, + 'local paths must start with "./", "../", or "/" (bare relative paths are not valid docrefs)', + ); } /** Parse a docref, returning null instead of throwing on invalid input. */ @@ -195,7 +247,8 @@ export function formatDocRef(ref: DocRef): string { return ref.url; case 'git': { const refPart = ref.ref ? `@${ref.ref}` : ''; - return `${ref.host}:${ref.owner}/${ref.repo}${refPart}//${ref.path}`; + const fragPart = ref.fragment ? `#${ref.fragment}` : ''; + return `${ref.host}:${ref.owner}/${ref.repo}${refPart}//${ref.path}${fragPart}`; } } } @@ -215,7 +268,8 @@ export function isDocRef(input: string): boolean { /** * Whether two docrefs address the same document, ignoring a leading `./` on - * local paths. Useful for de-duping config entries. + * local paths. Comparison is purely syntactic — no case normalization of hosts + * or owners. Useful for de-duping config entries. */ export function docRefsEqual(a: DocRef, b: DocRef): boolean { if (a.kind !== b.kind) return false; diff --git a/packages/tbd/tests/docref.test.ts b/packages/tbd/tests/docref.test.ts index 7504f840..5a302586 100644 --- a/packages/tbd/tests/docref.test.ts +++ b/packages/tbd/tests/docref.test.ts @@ -25,17 +25,26 @@ describe('parseDocRef', () => { }); }); - it('parses local paths (./ , ../ , absolute, scheme-less)', () => { + it('parses anchored local paths (./ , ../ , absolute, drive letter)', () => { expect(parseDocRef('./docs/general/')).toEqual({ kind: 'local', path: './docs/general/' }); expect(parseDocRef('../shared/rules.md')).toEqual({ kind: 'local', path: '../shared/rules.md', }); expect(parseDocRef('/abs/path/file.md')).toEqual({ kind: 'local', path: '/abs/path/file.md' }); - expect(parseDocRef('guidelines/python-rules.md')).toEqual({ + expect(parseDocRef('C:/Users/x/file.md')).toEqual({ kind: 'local', - path: 'guidelines/python-rules.md', + path: 'C:/Users/x/file.md', }); + expect(parseDocRef('c:\\docs\\file.md')).toEqual({ kind: 'local', path: 'c:\\docs\\file.md' }); + }); + + it('rejects bare relative and home-relative paths (strict grammar)', () => { + // Bare strings are not docrefs; consumers may coerce by prepending "./" at + // their own boundary. Strict-now can loosen later; the reverse cannot. + expect(() => parseDocRef('guidelines/python-rules.md')).toThrow(DocRefError); + expect(() => parseDocRef('hello world')).toThrow(DocRefError); + expect(() => parseDocRef('~/docs/rules.md')).toThrow(/home-relative/); }); it('parses plain URLs', () => { @@ -66,9 +75,26 @@ describe('parseDocRef', () => { }); }); - it('parses gitlab: and git: schemes', () => { + it('parses a fragment on the path and keeps it separate', () => { + expect(parseDocRef('github:acme/eng-docs@main//guidelines/style.md#naming')).toEqual({ + kind: 'git', + host: 'github', + owner: 'acme', + repo: 'eng-docs', + ref: 'main', + path: 'guidelines/style.md', + fragment: 'naming', + }); + }); + + it('parses gitlab: scheme', () => { expect(parseDocRef('gitlab:org/repo@v1.0//a/b.md').kind).toBe('git'); - expect(parseDocRef('git:org/repo@sha//a/b.md')).toMatchObject({ host: 'git', ref: 'sha' }); + }); + + it('rejects the dropped git: scheme as unknown', () => { + // v0.1 supports github:/gitlab: only; a host-bearing git: form may be added + // in a future version. + expect(() => parseDocRef('git:org/repo@sha//a/b.md')).toThrow(/unknown scheme/); }); it('trims surrounding whitespace', () => { @@ -81,6 +107,7 @@ describe('parseDocRef', () => { expect(() => parseDocRef('github:owner-only//path.md')).toThrow(DocRefError); expect(() => parseDocRef('github:owner/repo/no-double-slash.md')).toThrow(DocRefError); expect(() => parseDocRef('github:owner/repo@main//')).toThrow(DocRefError); + expect(() => parseDocRef('github:owner/repo@main//#frag-only')).toThrow(DocRefError); expect(() => parseDocRef('mailto:someone@example.com')).toThrow(DocRefError); }); }); @@ -102,8 +129,15 @@ describe('normalizeDocRef', () => { ); }); + it('preserves URL fragments through normalization (never silently dropped)', () => { + expect(normalizeDocRef('https://github.com/o/r/blob/main/f.md#testing')).toBe( + 'github:o/r@main//f.md#testing', + ); + }); + it('leaves non-git URLs and other forms unchanged', () => { expect(normalizeDocRef('https://example.com/x.md')).toBe('https://example.com/x.md'); + expect(normalizeDocRef('https://example.com/x.md#frag')).toBe('https://example.com/x.md#frag'); expect(normalizeDocRef('internal:a/b.md')).toBe('internal:a/b.md'); }); }); @@ -113,9 +147,11 @@ describe('formatDocRef round-trips', () => { 'internal:guidelines/python-rules.md', './docs/general/', '/abs/file.md', + 'C:/Users/x/file.md', 'https://example.com/style.md', 'github:acme/eng-docs@main//guidelines/style.md', 'github:acme/eng-docs//guidelines/style.md', + 'github:acme/eng-docs@main//guidelines/style.md#naming', 'gitlab:org/repo@v1.0//a/b.md', ]; it.each(cases)('parse->format is identity for %s', (input) => { @@ -127,6 +163,7 @@ describe('helpers', () => { it('isDocRef reflects validity', () => { expect(isDocRef('internal:a.md')).toBe(true); expect(isDocRef('mailto:x@y.com')).toBe(false); + expect(isDocRef('bare-string.md')).toBe(false); }); it('tryParseDocRef returns null on invalid input', () => { @@ -136,7 +173,8 @@ describe('helpers', () => { it('docRefsEqual ignores a leading ./ on local paths', () => { const a = parseDocRef('./guidelines/x.md'); - const b = parseDocRef('guidelines/x.md'); + // A consumer-coerced bare path: equal modulo the "./" anchor. + const b: DocRef = { kind: 'local', path: 'guidelines/x.md' }; expect(docRefsEqual(a, b)).toBe(true); }); From a3a5b370fc1b158845768f6df4d15caad9d751bc Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 12 Jun 2026 18:06:07 +0000 Subject: [PATCH 23/36] fix: Tighten docmap v0.1 and make tbd docs list --json conform MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Per the DocMap review (epic tbd-ss3p): - Every entry must carry a location (path and/or source) — an inventory whose entries cannot be located is not an inventory. Closes tbd-qylm. - Drop word_count from the core schema; size metrics are producer extension fields (tbd renders bytes + approx tokens and never emitted it). Closes tbd-7mxx. - Readers accept docmap/0.* only and reject other majors with a clear unsupported-version error. Closes tbd-hayb. - tbd docs list --json now emits the provenance source docref for upstream entries (and a path for local files), so every entry is locatable and the output is consumable as a real inventory. Closes tbd-xnbl. - Path-relativity convention (relative to the docmap's own location) documented in the module; the docmap-format.md reference doc tracks the rest (tbd-arsr). Also in docs-fork.ts (epic tbd-5wv9 polish): - Validate --kind across fork/unfork/list/diff (closes tbd-00wl) - Error on unknown names in docs update instead of silently reporting up-to-date (closes tbd-fywy) - Clear the stored conflicted flag once markers are resolved, so the committed manifest matches computed state (closes tbd-y85r) - Actionable overwrite refusal naming diff/--force options (closes tbd-y1kp) - Reference-kind guard comment; CLI cannot create reference entries until Phase 5 (closes tbd-roz6) https://claude.ai/code/session_01QPsCSYGtwR8JtX2R1aaxyh --- packages/tbd/src/cli/commands/docs-fork.ts | 100 +++++++++++++++++---- packages/tbd/src/docmap/docmap.ts | 32 +++++-- packages/tbd/tests/docmap.test.ts | 39 ++++++-- 3 files changed, 134 insertions(+), 37 deletions(-) diff --git a/packages/tbd/src/cli/commands/docs-fork.ts b/packages/tbd/src/cli/commands/docs-fork.ts index f132bf84..9a730f54 100644 --- a/packages/tbd/src/cli/commands/docs-fork.ts +++ b/packages/tbd/src/cli/commands/docs-fork.ts @@ -35,6 +35,7 @@ import { type ForkKind, findFork, hashContent, + hasUnresolvedConflict, readForkManifest, writeForkManifest, writeBaseContent, @@ -51,6 +52,7 @@ import { listLocalForkFiles, regenerateForkDirReadme, ForkConflictError, + KIND_DIR, } from '../../file/doc-fork.js'; import { updateOne, diffContents, type UpdateStrategy } from '../../file/fork-update.js'; import { createDocMap, type DocMapEntry } from '../../docmap/index.js'; @@ -58,6 +60,21 @@ import { createDocMap, type DocMapEntry } from '../../docmap/index.js'; /** Kinds that can be resolved from the cache and forked today. */ const RESOLVABLE_KINDS: ForkKind[] = ['guideline', 'shortcut', 'template']; +/** + * Validate a user-supplied --kind value. Without this, an unknown kind silently + * produces an empty cache and misleading "no docs" output. + */ +function parseKindOption(kind: string | undefined): ForkKind | undefined { + if (kind === undefined) return undefined; + if (!(RESOLVABLE_KINDS as string[]).includes(kind)) { + throw new CLIError(`Unknown kind "${kind}". Valid kinds: ${RESOLVABLE_KINDS.join(', ')}.`); + } + return kind as ForkKind; +} + +// 'reference' joins with Phase 5 (references/ cache dir); until then a manifest +// entry of that kind resolves as orphaned by design, and parseKindOption keeps +// the CLI from creating one. const KIND_CACHE_PATHS: Record = { guideline: CACHE_GUIDELINES_PATHS, shortcut: CACHE_SHORTCUT_PATHS, @@ -128,17 +145,29 @@ class DocsForkHandler extends BaseCommand { await withForkManifestLock(tbdRoot, async () => { let manifest = await readForkManifest(tbdRoot); for (const t of targets) { - const result = await forkDoc({ - tbdRoot, - forkDir: FORK_DIR, - manifest, - kind: t.kind, - name: t.name, - source: t.source, - content: t.content, - tbdVersion: VERSION, - force: options.force, - }); + let result; + try { + result = await forkDoc({ + tbdRoot, + forkDir: FORK_DIR, + manifest, + kind: t.kind, + name: t.name, + source: t.source, + content: t.content, + tbdVersion: VERSION, + force: options.force, + }); + } catch (err) { + if (err instanceof ForkConflictError && err.code === 'overwrite') { + throw new CLIError( + `${err.message}. Refusing to overwrite it. Options:\n` + + ` tbd docs diff ${t.name} # see how it differs\n` + + ` tbd docs fork ${t.name} --force # overwrite with upstream`, + ); + } + throw err; + } manifest = result.manifest; forked.push(result.relPath); if (!this.ctx.json) { @@ -166,7 +195,8 @@ class DocsForkHandler extends BaseCommand { names: string[], options: ForkOptions, ): Promise { - const kinds = options.kind ? [options.kind as ForkKind] : RESOLVABLE_KINDS; + const parsedKind = parseKindOption(options.kind); + const kinds = parsedKind ? [parsedKind] : RESOLVABLE_KINDS; if (options.all) { const targets: ResolvedDoc[] = []; @@ -249,7 +279,7 @@ class DocsUnforkHandler extends BaseCommand { forkDir: FORK_DIR, manifest, name, - kind: options.kind as ForkKind | undefined, + kind: parseKindOption(options.kind), force: options.force, }); manifest = result.manifest; @@ -425,6 +455,15 @@ class DocsUpdateHandler extends BaseCommand { await withForkManifestLock(tbdRoot, async () => { let manifest = await readForkManifest(tbdRoot); + if (names.length > 0) { + const known = new Set(manifest.forks.map((f) => f.name)); + const unknown = names.filter((n) => !known.has(n)); + if (unknown.length > 0) { + throw new CLIError( + `Not forked: ${unknown.join(', ')}. Run \`tbd docs status\` to see forked docs.`, + ); + } + } const selected = names.length > 0 ? manifest.forks.filter((f) => names.includes(f.name)) : manifest.forks; @@ -436,9 +475,10 @@ class DocsUpdateHandler extends BaseCommand { }; for (const entry of selected) { + const forkContent = await readForkFile(tbdRoot, FORK_DIR, entry); const result = await updateOne({ entry, - forkContent: await readForkFile(tbdRoot, FORK_DIR, entry), + forkContent, baseContent: await readForkBase(tbdRoot, entry), upstreamContent: await upstreamFor(entry), strategy, @@ -447,9 +487,21 @@ class DocsUpdateHandler extends BaseCommand { const { newFileContent, newBaseContent } = result; if (newFileContent === undefined && newBaseContent === undefined) { + // The conflicted flag auto-clears once markers are resolved (spec); + // persist the clear so the committed manifest matches computed state. + if ( + !options.dryRun && + entry.conflicted && + forkContent !== null && + !hasUnresolvedConflict(forkContent) + ) { + const cleared: ForkEntry = { ...entry }; + delete cleared.conflicted; + manifest = upsertFork(manifest, cleared); + } if (result.needsDecision) { decisions.push(result.message); - } else if (result.action !== 'skip-not-stale' && result.action !== 'noop') { + } else if (result.action !== 'skip-not-stale') { // Conflicted / orphaned / missing / version-skewed: actionable but // not applied here — surface, never silently swallow. skipped.push(result.message); @@ -551,7 +603,10 @@ class DocsListHandler extends BaseCommand { await this.execute(async () => { const tbdRoot = await requireInit(); const manifest = await readForkManifest(tbdRoot); - const kinds = options.kind ? [options.kind as ForkKind] : RESOLVABLE_KINDS; + const config = await readConfig(tbdRoot); + const files = config.docs_cache?.files; + const parsedKind = parseKindOption(options.kind); + const kinds = parsedKind ? [parsedKind] : RESOLVABLE_KINDS; const colors = this.output.getColors(); interface Row { @@ -592,11 +647,18 @@ class DocsListHandler extends BaseCommand { state, path: fork?.path ?? doc.sourceDir + '/' + doc.name + '.md', }); + // Every docmap entry must carry a location (path and/or source): + // forked docs have both; local files have a path but no upstream; + // upstream docs are located by their provenance docref. + const localPath = `${FORK_DIR}/${KIND_DIR[kind]}/${doc.name}.md`; docmapEntries.push({ name: doc.name, type: kind, - path: fork?.path, - source: fork?.source, + ...(fork + ? { path: fork.path, source: fork.source } + : isLocal + ? { path: localPath } + : { source: sourceDocRef(tbdRoot, files, doc.path) }), title: doc.frontmatter?.title, description: doc.frontmatter?.description, state, @@ -637,7 +699,7 @@ class DocsDiffHandler extends BaseCommand { await this.execute(async () => { const tbdRoot = await requireInit(); const manifest = await readForkManifest(tbdRoot); - const entry = findFork(manifest, name, options.kind as ForkKind | undefined); + const entry = findFork(manifest, name, parseKindOption(options.kind)); if (!entry) { throw new CLIError(`${name} is not a forked doc. Run \`tbd docs status\` to see forks.`); } diff --git a/packages/tbd/src/docmap/docmap.ts b/packages/tbd/src/docmap/docmap.ts index 7e0e61af..a0017bd0 100644 --- a/packages/tbd/src/docmap/docmap.ts +++ b/packages/tbd/src/docmap/docmap.ts @@ -3,14 +3,20 @@ * * A docmap is a "sitemap for docs": one entry per document, each with an identity * (`type` + `name`, unique within the map), a location (`path` and/or a provenance - * `source` docref), and presentation metadata (`title`, `description`, `word_count`). - * It describes a collection; it says nothing about how the collection is assembled, - * fetched, or kept fresh. + * `source` docref — at least one is required), and presentation metadata (`title`, + * `description`). It describes a collection; it says nothing about how the + * collection is assembled, fetched, or kept fresh — a docmap is a generated VIEW + * of a collection, never an input to resolution. + * + * Path convention: for a docmap committed as a file, `path` is relative to the + * docmap file's own directory (the sitemap convention); generated docmaps state + * their collection root out of band. * * This is the docmap/0.1 format. The module is standalone and dependency-free (no * tbd-internal imports) so it can move to its own package later. Consumers MUST * ignore unknown fields, so producers (such as tbd) may attach extension fields — - * for example tbd's `state`/`stale` — without breaking other readers. + * for example tbd's `state`/`stale`, or size metrics like `word_count` / + * `size_bytes` — without breaking other readers; core fields stay minimal. */ import { z } from 'zod'; @@ -20,6 +26,8 @@ export const DOCMAP_VERSION = 'docmap/0.1' as const; /** * One document in a docmap. Unknown fields are preserved (extension fields). + * Every entry must carry a location: `path` and/or `source` — an inventory whose + * entries cannot be located is not an inventory. */ export const DocMapEntrySchema = z .object({ @@ -27,15 +35,17 @@ export const DocMapEntrySchema = z name: z.string().min(1), /** Identity, e.g. "guideline" | "shortcut" | "template" | "reference". */ type: z.string().min(1), - /** Location within the collection (repo-relative or collection-relative). */ + /** Location within the collection (relative to the docmap's own location). */ path: z.string().optional(), /** Provenance: a docref string for where the doc came from. */ source: z.string().optional(), title: z.string().optional(), description: z.string().optional(), - word_count: z.number().int().nonnegative().optional(), }) - .passthrough(); + .passthrough() + .refine((entry) => entry.path !== undefined || entry.source !== undefined, { + message: 'docmap entry must have a location: path and/or source', + }); export type DocMapEntry = z.infer; @@ -100,8 +110,12 @@ export function parseDocMap(value: unknown): DocMap { throw new DocMapError(result.error.issues.map((i) => i.message).join('; ')); } const map = result.data; - if (!map.docmap.startsWith('docmap/')) { - throw new DocMapError(`unrecognized version tag ${JSON.stringify(map.docmap)}`); + // Readers accept docmap/0.* only: a different major may change field semantics, + // so failing fast beats misreading. + if (!map.docmap.startsWith('docmap/0.')) { + throw new DocMapError( + `unsupported docmap version ${JSON.stringify(map.docmap)} (this reader supports docmap/0.*)`, + ); } assertUniqueIdentities(map.documents); return map; diff --git a/packages/tbd/tests/docmap.test.ts b/packages/tbd/tests/docmap.test.ts index 19a71567..c4fbf5cb 100644 --- a/packages/tbd/tests/docmap.test.ts +++ b/packages/tbd/tests/docmap.test.ts @@ -22,7 +22,6 @@ const sample = [ source: 'internal:guidelines/python-rules.md', title: 'Python Coding Rules', description: 'Type hints, docstrings, exception handling', - word_count: 2400, }, { name: 'review-code', type: 'shortcut', source: 'internal:shortcuts/standard/review-code.md' }, { name: 'tbd-docs', type: 'reference', source: 'internal:tbd-docs.md' }, @@ -41,24 +40,33 @@ describe('createDocMap', () => { expect('name' in map).toBe(false); }); - it('preserves extension fields (e.g. tbd state)', () => { - const map = createDocMap([{ name: 'x', type: 'guideline', state: 'customized', stale: true }]); - expect(map.documents[0]).toMatchObject({ state: 'customized', stale: true }); + it('preserves extension fields (e.g. tbd state, size metrics)', () => { + const map = createDocMap([ + { + name: 'x', + type: 'guideline', + source: 'internal:x.md', + state: 'customized', + stale: true, + word_count: 2400, + }, + ]); + expect(map.documents[0]).toMatchObject({ state: 'customized', stale: true, word_count: 2400 }); }); it('rejects duplicate (type, name) identities', () => { expect(() => createDocMap([ - { name: 'dup', type: 'guideline' }, - { name: 'dup', type: 'guideline' }, + { name: 'dup', type: 'guideline', source: 'internal:a.md' }, + { name: 'dup', type: 'guideline', source: 'internal:b.md' }, ]), ).toThrow(DocMapError); }); it('allows the same name under different types', () => { const map = createDocMap([ - { name: 'typescript', type: 'guideline' }, - { name: 'typescript', type: 'template' }, + { name: 'typescript', type: 'guideline', source: 'internal:g/ts.md' }, + { name: 'typescript', type: 'template', source: 'internal:t/ts.md' }, ]); expect(map.documents).toHaveLength(2); }); @@ -75,17 +83,30 @@ describe('parseDocMap', () => { expect(() => parseDocMap({ docmap: 'sitemap/1', documents: [] })).toThrow(DocMapError); }); + it('accepts docmap/0.x and rejects other majors', () => { + expect(parseDocMap({ docmap: 'docmap/0.2', documents: [] }).docmap).toBe('docmap/0.2'); + expect(() => parseDocMap({ docmap: 'docmap/1.0', documents: [] })).toThrow( + /supports docmap\/0/, + ); + }); + it('rejects entries missing identity fields', () => { expect(() => parseDocMap({ docmap: DOCMAP_VERSION, documents: [{ name: 'x' }] })).toThrow( DocMapError, ); }); + it('rejects entries without a location (path and/or source required)', () => { + expect(() => + parseDocMap({ docmap: DOCMAP_VERSION, documents: [{ name: 'x', type: 'guideline' }] }), + ).toThrow(/location/); + }); + it('accepts and preserves unknown top-level and entry fields', () => { const map = parseDocMap({ docmap: DOCMAP_VERSION, generated_by: 'tbd', - documents: [{ name: 'x', type: 'guideline', state: 'forked' }], + documents: [{ name: 'x', type: 'guideline', source: 'internal:x.md', state: 'forked' }], }); expect(map.documents[0]).toMatchObject({ state: 'forked' }); }); From 83fe4bb605924cc2458e8a34d7497c53868e44e4 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 12 Jun 2026 18:06:32 +0000 Subject: [PATCH 24/36] refactor: stat-based existence check; drop dead noop update action - pathExists used readFile (full content read, then callers re-read) and swallowed non-ENOENT errors; now stat-based, propagating real failures. Closes tbd-znnn. - UpdateAction 'noop' was never returned; removed (with its one reference). Part of tbd-d9l0; the tryscript sed portability half lands with the docs commit. https://claude.ai/code/session_01QPsCSYGtwR8JtX2R1aaxyh --- packages/tbd/src/file/doc-fork.ts | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/packages/tbd/src/file/doc-fork.ts b/packages/tbd/src/file/doc-fork.ts index ac8cf897..30f94632 100644 --- a/packages/tbd/src/file/doc-fork.ts +++ b/packages/tbd/src/file/doc-fork.ts @@ -11,7 +11,7 @@ * the filesystem writes; resolving which doc/source to fork is the caller's job. */ -import { readFile, readdir, rm, rmdir, mkdir } from 'node:fs/promises'; +import { readFile, readdir, rm, rmdir, mkdir, stat } from 'node:fs/promises'; import { dirname, join, relative } from 'node:path'; import { writeFile } from 'atomically'; @@ -69,10 +69,13 @@ export function forkRelPath(forkDir: string, kind: ForkKind, name: string): stri async function pathExists(path: string): Promise { try { - await readFile(path); + await stat(path); return true; - } catch { - return false; + } catch (err) { + if ((err as NodeJS.ErrnoException).code === 'ENOENT') { + return false; + } + throw err; } } From e8b5112ba6e11e00987c73db449b30ae9a1f875f Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 12 Jun 2026 18:06:37 +0000 Subject: [PATCH 25/36] process: Capture review rationale; sync spec with implemented guards MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Spec Alternatives: add the copy-all-and-gitignore fork-dir variant as considered and rejected (gitignored mirrors are invisible on GitHub/PRs; edits to them diverge silently; fork --all is the tracked all-visible posture). Closes the rationale-capture half of tbd-a3pj. - tbd-design.md 2.9: state the resolution principle — resolve by convention; track only what cannot be derived; publish the inventory as a generated view — with the gitignore-mirror rejection noted. Closes tbd-a3pj. - Spec update decision table: add the version-skew guard row and design point (update + re-fork refresh refuse when the fork point was set by a newer tbd), and note the tbd sync drift notice. Closes tbd-69g0. - Spec docmap section: location required per entry, word_count moved to extension fields, path-relativity convention; golden list --json block updated to match (upstream entries carry source). - Spec docref mentions: github:/gitlab: only, future-protocols note. - tbd-docs.md drift table cites tbd-design 2.9 as the canonical model (closes tbd-koe4); README Upgrading gains the forked-docs update line (closes tbd-jznb). - cli-docs-update.tryscript.md: portable perl -pi instead of GNU-only sed -i (closes tbd-d9l0 with the noop removal). https://claude.ai/code/session_01QPsCSYGtwR8JtX2R1aaxyh --- README.md | 4 +- .../active/plan-2026-06-11-forkable-docs.md | 58 +++++++++++++++---- packages/tbd/docs/tbd-design.md | 12 ++++ packages/tbd/docs/tbd-docs.md | 9 +-- .../tbd/tests/cli-docs-update.tryscript.md | 4 +- 5 files changed, 70 insertions(+), 17 deletions(-) diff --git a/README.md b/README.md index c1f88b79..8bd9d4bf 100644 --- a/README.md +++ b/README.md @@ -309,7 +309,9 @@ publish the upgrade to your team. Teammates still on an older tbd then see “This repository requires a newer version of tbd” until they run the same two commands. Issue data is never touched by an upgrade, and the migration is revertible: see -“Aborting a Format Upgrade” under Troubleshooting in `tbd docs`. +“Aborting a Format Upgrade” under Troubleshooting in `tbd docs`. If you have forked docs +in `docs/tbd/`, `tbd sync` prints a notice when their upstream versions moved — run +`tbd docs update` to merge the changes in. ### Team Setup diff --git a/docs/project/specs/active/plan-2026-06-11-forkable-docs.md b/docs/project/specs/active/plan-2026-06-11-forkable-docs.md index d8be8796..8c399949 100644 --- a/docs/project/specs/active/plan-2026-06-11-forkable-docs.md +++ b/docs/project/specs/active/plan-2026-06-11-forkable-docs.md @@ -307,8 +307,9 @@ advanced by `tbd docs update`. This is the *base* of every three-way merge: committed so any collaborator (or CI) can run `tbd docs update` later with full fidelity, regardless of which tbd version originally forked the doc. - **Git provenance is recorded when the source is git-hosted.** For `github:` / - `gitlab:` / `git:` docrefs, fork and every base advance also record the upstream - commit (`source_revision`) and, when the pinned ref is a tag or the commit matches one + `gitlab:` docrefs (the git schemes in docref v0.1; additional protocols may be added + in future versions), fork and every base advance also record the upstream commit + (`source_revision`) and, when the pinned ref is a tag or the commit matches one exactly, `source_tag`. Non-git sources (`internal:`, bare URLs) have no revision to record — which is precisely why bases are *snapshots* rather than pointers: the stored copy is the universal provenance fallback that works for every source kind, with @@ -486,10 +487,14 @@ Stripped to its core, the simple, well-defined, reusable concept is just the > A **docmap** is a machine-readable inventory of a collection of documents: one entry > per doc, each with an identity (`type` + `name`, unique within the map), a location -> (`path`, and/or a provenance `source` docref), and presentation metadata (`title`, -> `description`, `word_count`). It describes a doc collection; it says nothing about how -> the collection is assembled, fetched, or kept fresh. - +> (`path`, and/or a provenance `source` docref — every entry carries at least one), and +> presentation metadata (`title`, `description`). It describes a doc collection; it says +> nothing about how the collection is assembled, fetched, or kept fresh — a docmap is a +> generated *view* of a collection, never an input to resolution. + +For a docmap committed as a file, `path` is relative to the docmap file’s own directory +(the sitemap convention); size metrics (`word_count`, `size_bytes`, token estimates) are +producer extension fields, not core. A sitemap for docs, with docref as its addressing primitive: ```yaml @@ -499,10 +504,9 @@ documents: - name: python-rules type: guideline path: guidelines/python-rules.md # location within the collection - source: internal:guidelines/python-rules.md # provenance docref (optional) + source: internal:guidelines/python-rules.md # provenance docref title: Python Coding Rules description: Type hints, docstrings, exception handling, resource management - word_count: 2400 ``` Producers may *generate* a docmap (as tbd does: **every** list/inventory command emits @@ -601,32 +605,45 @@ change” / “I already folded it in by hand”). | `orphaned` | skip + note (upstream removed the doc; keep your copy or `unfork`) | same | same | | `missing` / `local` | skip (doctor’s problem / nothing upstream) | same | same | | base file missing (manual deletion) | cannot merge; skip + point at `--keep-ours` | same | re-establish base from current upstream (repair) | +| fork point set by a **newer tbd** (`tbd_version` > running version) | skip + warn: upgrade tbd first (this client’s bundled “upstream” is older than the fork point; updating would silently downgrade the doc) | same | same | Design points: +- **Version-skew guard.** The manifest’s per-entry `tbd_version` records which tbd + advanced the fork point; `update` (every strategy) and a re-fork refresh both refuse + to act when the running tbd is older than that, since their bundled content predates + the fork point (re-fork accepts `--force` as the explicit downgrade escape hatch). + - **Clean merges apply by default deliberately.** The forked file is git-tracked, so every auto-merge is fully visible in `git diff` and trivially revertible — git is the undo. Conflicted docs are never touched by default; the listing names the two strategies and the user (or agent) re-runs with one. + - **Base advance happens at merge time.** After any update (replace, clean merge, or conflicted `--merge`), the base becomes the new upstream content. So post-resolution, the doc is simply “a customized fork of current upstream” — states stay coherent with no extra bookkeeping. + - **`--keep-ours` keeps your content and advances the fork point.** For a single file there is no diff to replay — keeping your version *is* the operation; upstream’s change is acknowledged, staleness clears, and future updates diff against the new base. It also repairs a missing base file. (This was `--rebase` in an earlier draft, renamed because the operation is not git-rebase content semantics — it does not replay your diff, it keeps it.) + - **Only the explicit command mutates tracked files.** `tbd setup --auto` and the 24-hour doc auto-sync refresh the gitignored cache as today and then *report* pending updates (`2 forked docs have upstream updates — run 'tbd docs update'`), but never write into the fork dir. + `tbd sync` likewise prints a one-line drift notice (stale / conflicted / missing + counts) after its cache refresh — awareness only, never action. Background paths rewriting committed files would be surprising and hard to audit. + - **Convergence is the unfork path.** If you customized a doc, upstream later adopted your change, and `update` merges cleanly such that the file now equals upstream, the doc returns to plain `forked` (unmodified) — and `tbd docs unfork` works without `--force`. + - `--dry-run` previews all of the above, including which docs would conflict. ### CLI surface @@ -854,16 +871,20 @@ forked. visibility — but every tbd upgrade churns dozens of files in user repos, generated and authored content become indistinguishable, and there are no fork/unfork semantics. Rejected. + 2. **Finish the PR #117 f05 framework first.** Right long-term shape, but blocked on five open architecture questions and a format migration; delivers user value months later. Rejected as the *first* step; this spec is its forward-compatible kernel. + 3. **Symlinks from `docs/` into `.tbd/docs/`.** Not portable (Windows), breaks on GitHub rendering, and git-tracking symlinked generated content is worse than either mode. Rejected. + 4. **Frontmatter provenance stamps instead of a manifest.** Self-contained files, but fork would modify content on copy (breaking clean diffs against upstream) and “customized” detection would need fragile frontmatter-stripping/normalization. Rejected in favor of verbatim copies + manifest. + 5. **Updates without stored bases (hash-only provenance).** Smaller footprint, but a hash cannot drive a three-way merge, so *every* upstream change to a customized doc would surface as a wall-of-conflicts two-way diff — clean merges become impossible @@ -871,11 +892,13 @@ forked. Reconstructing bases from old npm versions (network, unpublished versions, `development` builds) or from git history (squashes, mixed commits) is unreliable. Rejected: committed base copies are the price of a real update story. + 6. **Auto-merging during `tbd setup --auto` or background doc auto-sync.** Tempting (zero extra commands) but background paths rewriting committed files is surprising and unauditable; doc auto-sync can trigger from any read command. Rejected: setup/status only *report*; `tbd docs update` is the single explicit mutation point. + 7. **Verbs as flags on a top-level `tbd eject` command** (`--update`, `--status`, `--rebase`, plus a separate `tbd uneject` — the original verb names; see decision 14). The original draft of this spec. @@ -884,6 +907,21 @@ forked. The noun-scoped `tbd docs` group matches existing tbd conventions (`dep`/`label`/`attic`/`config`) and gives sync/show/list a coherent home. +8. **Copy *all* docs into the fork dir and gitignore the unforked ones** (fork = + flipping a gitignore entry). + Full local browsability, but it fails the actual goal: gitignored mirrors are exactly + as invisible on GitHub and in PRs as the cache is, so only the tracked subset gains + visibility either way. + It also creates the worst silent failure mode available — an edit to a gitignored + mirror is served locally (top precedence) but never reaches the team, with no + git-visible artifact; fork state becomes a predicate over two systems (git index + + ignore rules, with the classic ignored-but-tracked confusions); and cache refresh + would either resurrect deleted mirror files or need tombstones, contradicting + “nothing is ever silently re-created against the user’s deletion.” + `tbd docs fork --all` already provides the all-visible posture in tracked form, with + `unfork` as the undo. + Rejected. + ## Resolved Decisions Settled during design review (2026-06-11): @@ -1443,11 +1481,11 @@ $ tbd docs list --json "source": "internal:guidelines/python-rules.md", "title": "Python Coding Rules", "description": "Type hints, docstrings, exception handling, resource management", - "word_count": [..], "state": "customized", "stale": true } - [.. one entry per doc; upstream docs have state "upstream" and no fork fields ..] + [.. one entry per doc; upstream docs have state "upstream" and a source docref + (their provenance) but no path — every entry carries a location ..] ] } ? 0 diff --git a/packages/tbd/docs/tbd-design.md b/packages/tbd/docs/tbd-design.md index e8ec9989..20bb6f64 100644 --- a/packages/tbd/docs/tbd-design.md +++ b/packages/tbd/docs/tbd-design.md @@ -1991,6 +1991,18 @@ combination of user actions safe. A doc’s identity is **kind + name**; paths follow fixed conventions (`/.md`, flat — nested folders are not scanned). +The model follows one principle: **resolve by convention; track only what cannot be +derived; publish the inventory as a generated view.** Lookup is a fixed search path over +conventional locations — no registry that can drift from disk. +The only stored tracking is the fork manifest, which records the one fact that cannot be +recomputed from the files: each fork’s upstream source and base snapshot. +The docmap that doc commands emit is a generated view of this state, never an input to +resolution. (A copy-all-and-gitignore variant of the fork dir was considered and +rejected: gitignored mirrors are invisible on GitHub and in PRs, and edits to them +diverge silently with no team-visible artifact — `tbd docs fork --all` provides the +all-visible posture in tracked form. +See the spec’s Alternatives.) + #### Invariants 1. **Serving precedence**: the fork dir is prepended to every kind’s lookup path, so a diff --git a/packages/tbd/docs/tbd-docs.md b/packages/tbd/docs/tbd-docs.md index b39f45d2..fb86e325 100644 --- a/packages/tbd/docs/tbd-docs.md +++ b/packages/tbd/docs/tbd-docs.md @@ -731,10 +731,11 @@ docs/tbd/ Two rules make everything below predictable: **names are identity** (a doc is `/.md`; nested subfolders are not scanned), and **tracking is derived, not -stored** — every doc’s state is recomputed from content hashes (your file vs its -recorded base vs current upstream), so no git operation can desynchronize tbd from the -folder. Whatever you or your agent do to these files, `tbd docs status` gives a defined -answer: +stored** (the canonical model — copies, invariants, flows — is `tbd-design.md` §2.9; +this table is its user-facing summary) — every doc’s state is recomputed from content +hashes (your file vs its recorded base vs current upstream), so no git operation can +desynchronize tbd from the folder. +Whatever you or your agent do to these files, `tbd docs status` gives a defined answer: | You (or your agent)… | State | What happens / what to do | | --- | --- | --- | diff --git a/packages/tbd/tests/cli-docs-update.tryscript.md b/packages/tbd/tests/cli-docs-update.tryscript.md index 1bae3c47..d1b36f06 100644 --- a/packages/tbd/tests/cli-docs-update.tryscript.md +++ b/packages/tbd/tests/cli-docs-update.tryscript.md @@ -50,14 +50,14 @@ Updated 1 forked doc(s): # Test: customize the fork’s first line ```console -$ sed -i '1c\' docs/tbd/guidelines/python-rules.md +$ perl -pi -e '$_ = "\n" if $. == 1' docs/tbd/guidelines/python-rules.md ? 0 ``` # Test: change the same line upstream ```console -$ sed -i '1c\' .tbd/docs/guidelines/python-rules.md +$ perl -pi -e '$_ = "\n" if $. == 1' .tbd/docs/guidelines/python-rules.md ? 0 ``` From 7f35ba8e753337490fa656942af2ba406cdbe1c9 Mon Sep 17 00:00:00 2001 From: Test User Date: Fri, 12 Jun 2026 19:44:35 +0000 Subject: [PATCH 26/36] fix: Prevent hook-inherited GIT_DIR from redirecting tests onto the real repo Critical infrastructure fix (tbd-a1lc). Git exports GIT_DIR (plus GIT_PREFIX etc.) into hook environments; pushing from a linked worktree therefore runs the lefthook pre-push test suite with GIT_DIR pointing at the real repo's gitdir. An absolute GIT_DIR overrides cwd-based repo discovery in every git and tbd subprocess the tests spawn, so test fixtures executed their git init/commit/branch and data-corruption scenarios against the REAL repository: in the observed incident this rewrote local main to fixture commits, created fixture branches, flipped core.bare, stamped the shared layout.yml, and overwrote the live data-sync ids.yml with the corrupted-data test fixture (merge-refs.test.ts and corrupted-data.test.ts identified as the writers). Two independent layers, either of which closes the hole: - tests/scrub-git-env.ts (wired via vitest setupFiles) deletes the git location vars in every worker before any test spawns a subprocess, so the pervasive { ...process.env } spawn pattern across ~60 test sites is safe regardless of how the runner was invoked. - scripts/scrub-git-env.mjs wraps every lefthook pre-push command, so no current or future hook command (including non-vitest ones) inherits the poisoned env. Verified red/green against a sacrificial victim repo using the real suite: pre-fix config + hook-style env mutates the victim's refs (reproducing the incident exactly: ours/theirs fixture branches appear); fixed config leaves the victim byte-identical. Wrapper verified to scrub, propagate exit codes, and run the real hook commands. https://claude.ai/code/session_01QPsCSYGtwR8JtX2R1aaxyh --- lefthook.yml | 11 +++++-- packages/tbd/tests/scrub-git-env.ts | 33 ++++++++++++++++++++ packages/tbd/vitest.config.ts | 5 +++ scripts/scrub-git-env.mjs | 47 +++++++++++++++++++++++++++++ 4 files changed, 93 insertions(+), 3 deletions(-) create mode 100644 packages/tbd/tests/scrub-git-env.ts create mode 100644 scripts/scrub-git-env.mjs diff --git a/lefthook.yml b/lefthook.yml index fa17f395..5e085759 100644 --- a/lefthook.yml +++ b/lefthook.yml @@ -52,19 +52,24 @@ pre-commit: priority: 3 pre-push: + # Every command runs behind scripts/scrub-git-env.mjs: git exports GIT_DIR into + # hook environments (always when pushing from a linked worktree), which would + # otherwise redirect any git/tbd subprocess the suite spawns onto the REAL + # repository — test fixtures then rewrite real refs and tbd data (tbd-a1lc). + # The vitest suite self-scrubs too (tests/scrub-git-env.ts); both layers stay. commands: # Build if needed - skips if dist is up-to-date (integration tests require dist/bin.mjs) build: - run: pnpm build:check + run: node scripts/scrub-git-env.mjs pnpm build:check priority: 1 # Then run tests test: - run: pnpm test + run: node scripts/scrub-git-env.mjs pnpm test priority: 2 # Enforce 14-day package-age rule on dependency pins. # See packages/tbd/docs/guidelines/pnpm-monorepo-patterns.md#supply-chain-mitigation # Only runs when a package.json is staged so day-to-day pushes aren't slowed by registry lookups. package-age: glob: '{package.json,packages/*/package.json}' - run: pnpm check:package-age + run: node scripts/scrub-git-env.mjs pnpm check:package-age priority: 3 diff --git a/packages/tbd/tests/scrub-git-env.ts b/packages/tbd/tests/scrub-git-env.ts new file mode 100644 index 00000000..2ba60cc5 --- /dev/null +++ b/packages/tbd/tests/scrub-git-env.ts @@ -0,0 +1,33 @@ +/** + * Vitest per-worker setup: strip inherited git-location environment variables + * before any test spawns a subprocess. + * + * Tests create throwaway git repos in temp dirs and run `git` / the tbd CLI with + * `cwd` set there, relying on git and tbd discovering the repo from `cwd`. But git + * exports GIT_DIR (and friends) into hook environments: running `git push` from a + * linked worktree invokes the pre-push test suite with GIT_DIR pointing at the + * real repository's gitdir. An absolute GIT_DIR overrides cwd-based discovery, so a + * fixture's `git init` / `commit` / `checkout -b` and tbd's `--git-common-dir` + * resolution would all operate on the REAL repo — rewriting its branches and + * corrupting its data-sync worktree. (Running vitest directly never set GIT_DIR, + * which is why only hook-invoked runs were affected.) + * + * Deleting these here, once per worker before any test, makes every + * `{ ...process.env }` spawn in the suite safe regardless of how the runner was + * invoked. The pre-push hook also scrubs them (see lefthook.yml) as defense in + * depth; either layer alone closes the hole, but both are cheap. + */ +const GIT_LOCATION_VARS = [ + 'GIT_DIR', + 'GIT_WORK_TREE', + 'GIT_INDEX_FILE', + 'GIT_COMMON_DIR', + 'GIT_OBJECT_DIRECTORY', + 'GIT_ALTERNATE_OBJECT_DIRECTORIES', + 'GIT_PREFIX', + 'GIT_NAMESPACE', +]; + +for (const name of GIT_LOCATION_VARS) { + delete process.env[name]; +} diff --git a/packages/tbd/vitest.config.ts b/packages/tbd/vitest.config.ts index 9c2e74a4..1d89da55 100644 --- a/packages/tbd/vitest.config.ts +++ b/packages/tbd/vitest.config.ts @@ -10,6 +10,11 @@ export default defineConfig({ test: { include: ['tests/**/*.test.ts'], globalSetup: ['tests/global-setup.ts'], + // Strip inherited GIT_DIR (and friends) in every worker before tests spawn + // subprocesses, so an ambient git env (e.g. from a pre-push hook run in a + // linked worktree) cannot redirect fixture git/tbd onto the real repo. + // See tests/scrub-git-env.ts and the tbd-a1lc incident. + setupFiles: ['tests/scrub-git-env.ts'], hookTimeout: isWindows ? 30000 : 10000, coverage: { provider: 'v8', diff --git a/scripts/scrub-git-env.mjs b/scripts/scrub-git-env.mjs new file mode 100644 index 00000000..d9589b95 --- /dev/null +++ b/scripts/scrub-git-env.mjs @@ -0,0 +1,47 @@ +#!/usr/bin/env node +/* global process, console */ +/** + * Run a command line with git's repo-location environment variables removed. + * + * Git exports GIT_DIR (and related vars) into hook environments — notably, + * `git push` from a linked worktree runs the pre-push hook with GIT_DIR pointing + * at the real repository's gitdir. Any git or tbd subprocess spawned by the + * hook's commands inherits it, and an absolute GIT_DIR overrides cwd-based repo + * discovery: test fixtures that `git init` in temp dirs then commit, branch, and + * corrupt data against the REAL repository instead. This destroyed local refs + * and the tbd data-sync mappings in one incident (tbd-a1lc). + * + * Usage (from lefthook.yml): node scripts/scrub-git-env.mjs + * Arguments are joined with spaces and run through the platform shell — the same + * semantics as lefthook's own `run:` line, minus the git location env. + * + * The vitest suite also self-scrubs (packages/tbd/tests/scrub-git-env.ts); + * this wrapper protects every other current or future hook command the same way. + */ + +import { spawnSync } from 'node:child_process'; + +const GIT_LOCATION_VARS = [ + 'GIT_DIR', + 'GIT_WORK_TREE', + 'GIT_INDEX_FILE', + 'GIT_COMMON_DIR', + 'GIT_OBJECT_DIRECTORY', + 'GIT_ALTERNATE_OBJECT_DIRECTORIES', + 'GIT_PREFIX', + 'GIT_NAMESPACE', +]; + +const commandLine = process.argv.slice(2).join(' '); +if (!commandLine) { + console.error('usage: scrub-git-env.mjs '); + process.exit(2); +} + +const env = { ...process.env }; +for (const name of GIT_LOCATION_VARS) { + delete env[name]; +} + +const result = spawnSync(commandLine, { stdio: 'inherit', env, shell: true }); +process.exit(result.status ?? 1); From 3c718c009157425cf5afdebc56289b787d72bcca Mon Sep 17 00:00:00 2001 From: Test User Date: Fri, 12 Jun 2026 20:12:55 +0000 Subject: [PATCH 27/36] feat: Re-home the tbd docs surface per the f05 disposition table MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Implements the spec's four-row disposition (closes tbd-g0hu), plus the review's remaining UX beads, and reconciles the spec's golden maps with shipped output (closes tbd-8gt5): - Bare 'tbd docs' is the managed-docs status overview, presenting the three postures — hidden / curated / everything (closes tbd-3h1s) — and degrades to a manual pointer before init. The old [topic], --section, --list, and --all viewer flags are retired. - 'tbd docs show ' is the kind-agnostic reader: serves any doc through fork-dir-precedence paths with a '(serving forked copy: ...)' stderr provenance note (spec Decision 18 for show), and serves the bundled manual as the reserved tbd-docs name with --section/--sections navigation. 'tbd docs manual [topic]' is its alias. - 'tbd docs sync' is the canonical cache refresh (spec Phase 1 item 4); 'tbd sync --docs' stays as a deprecated alias rendering through one shared module (docs-sync-output.ts), so the surfaces cannot drift. The fork drift notice goes through the output layer, and 'docs update --json' carries structured {name, message} entries (closes tbd-cab2). - Unfork refusal now lists options (diff / --force), matching fork. - Goldens rewritten: cli-help-all docs blocks, cli-doc-output sections block, golden-output bare-overview snapshot (count masked), cli-setup top-level help; manual's Documentation Commands section updated. - Spec golden maps reconciled to shipped output with a shipped-vs-phase- contract status convention; f05 documented as the stamp-only migration it is, with fork_dir configurability and .tbd/README.md noted as in-era additions (closes tbd-wngu). https://claude.ai/code/session_01QPsCSYGtwR8JtX2R1aaxyh --- .../active/plan-2026-06-11-forkable-docs.md | 165 +++--- packages/tbd/docs/tbd-docs.md | 17 +- packages/tbd/src/cli/commands/docs-fork.ts | 31 +- packages/tbd/src/cli/commands/docs.ts | 511 +++++++++++------- packages/tbd/src/cli/commands/sync.ts | 75 +-- packages/tbd/src/cli/lib/docs-sync-output.ts | 107 ++++ .../tbd/tests/cli-doc-output.tryscript.md | 6 +- packages/tbd/tests/cli-help-all.tryscript.md | 54 +- packages/tbd/tests/cli-setup.tryscript.md | 4 +- packages/tbd/tests/golden-output.test.ts | 58 +- 10 files changed, 606 insertions(+), 422 deletions(-) create mode 100644 packages/tbd/src/cli/lib/docs-sync-output.ts diff --git a/docs/project/specs/active/plan-2026-06-11-forkable-docs.md b/docs/project/specs/active/plan-2026-06-11-forkable-docs.md index 8c399949..ea4a7ee6 100644 --- a/docs/project/specs/active/plan-2026-06-11-forkable-docs.md +++ b/docs/project/specs/active/plan-2026-06-11-forkable-docs.md @@ -256,9 +256,10 @@ docs/tbd/ # default; configurable └── plan-spec.md ``` -- Default: `docs/tbd/`. The location is surfaced as an explicit, editable customization - during setup (and changeable any time); a non-default choice is persisted to the new - config key `docs_cache.fork_dir` (part of the f05 layout). +- Default: `docs/tbd/`, fixed for the initial f05 release. + Making the location configurable (persisted to a `docs_cache.fork_dir` key, surfaced + during setup) is planned within the f05 era as an additive, optional config key; until + then the constant in `paths.ts` is the single source of truth. - Layout is `//.md`. The bundled `shortcuts/system|standard|custom` subdivision is **flattened** to `shortcuts/` on fork — that split is an implementation detail; the manifest preserves the original @@ -339,10 +340,13 @@ The new committed layout artifacts (the `.tbd/doc-forks/` directory and the `tbd_format` bumps to **f05** with a step in the existing migration chain (`src/lib/tbd-format.ts`, following the f03→f04 precedent): -- The f04→f05 migration is metadata-only: stamp the format id, refresh the - `.tbd/.gitignore` template/comments, and write the `.tbd/README.md` layout contract - (below). No files move; `.tbd/docs/` is untouched and fork artifacts appear only when - fork is first used — the upgrade path is exactly as smooth as f03→f04. +- The f04→f05 migration is metadata-only: stamp the format id. + No files move; `.tbd/docs/` is untouched and fork artifacts appear only when fork is + first used — the upgrade path is exactly as smooth as f03→f04. (The `.tbd/.gitignore` + template refresh and the generated `.tbd/README.md` layout contract are additive setup + outputs planned within the f05 era, not part of the migration stamp; + `FORMAT_HISTORY.f05` in `tbd-format.ts` is the authoritative record of what the stamp + does.) - Older CLIs encountering an f05 repo detect the newer format id via the existing compatibility machinery and refuse to run, prompting an upgrade — an explicit signal, rather than silently serving the upstream copies while upgraded teammates see @@ -1344,6 +1348,11 @@ Unstable fields use placeholders that become tryscript patterns: `[SIZE]` = `golden-testing-guidelines`, everything else (names, kinds, states, counts, ordering) is shown literally — no patterns on values we control. +Status convention: blocks for **shipped** commands are captured from the built CLI and +match the live goldens; blocks for commands that have **not landed yet** are marked +*(Phase N contract)* and must be re-captured against the real CLI when that phase ships +(the Phase 0.5 discipline, applied per phase). + ### Console output style contract This contract is **authoritative and enforced structurally, not by convention**. Doc @@ -1406,25 +1415,29 @@ provided: $ tbd docs # no docs forked yet tbd docs — managed documentation - 37 available in cache (.tbd/docs/, gitignored); none forked into the repo. - Guidelines are active from the cache. Fork them into docs/tbd/ to make them - visible and customizable (same behavior — just explicit and git-tracked): + [..] docs available in the cache (.tbd/docs/, gitignored); none forked into the repo. + Guidelines are active from the cache. Three postures, all serving the same docs: + + Hidden (default): keep the cache as-is — zero repo footprint + Curated: tbd docs fork [...] fork chosen docs into docs/tbd/ + Everything: tbd docs fork --all all docs, visible and editable - Scope: all standard guidelines (recommended), or a category: - general, typescript, python, convex, electron - Make visible: tbd docs fork --category=general --category= - tbd docs fork --all (everything) Browse / read: tbd docs list / tbd docs show + Learn more: tbd docs show tbd-docs (the manual; alias: tbd docs manual) ? 0 ``` +When Phase 4 lands `--category`, the Curated line additionally names +`tbd docs fork --category=` with the category list — the menu must only name +selectors that exist (the `--category` hint shipped before the flag once; never again). + With forks present: ```text $ tbd docs tbd docs — managed documentation - 37 available (33 upstream, 4 forked into docs/tbd/) + [..] available ([..] upstream, 4 forked into docs/tbd/) 4 forked: 2 customized, 3 with upstream updates — run 'tbd docs update' Inspect: tbd docs status @@ -1518,48 +1531,59 @@ $ tbd docs show python-rules # serves the forked copy; provenance to stderr ? 0 ``` +### `tbd docs sync` + +Refreshes the gitignored cache; `tbd sync --docs` remains a deprecated alias with +identical output (both render through one shared module, `docs-sync-output.ts`): + +```text +$ tbd docs sync +✓ Docs up to date +? 0 + +$ tbd docs sync # after an upgrade changed bundled docs; forks stale +✓ Synced docs: ~1 doc(s) +• Docs: 1 forked doc(s) have upstream updates — run 'tbd docs update' +? 0 +``` + ### `tbd docs fork` ```text $ tbd docs fork python-rules ✓ Forked python-rules → docs/tbd/guidelines/python-rules.md - Recorded base in .tbd/doc-forks/ (source: internal:guidelines/python-rules.md) Regenerated docs/tbd/README.md -Edit it in place — tbd now serves your copy everywhere it served the upstream one. +Edit in place — tbd now serves your copy wherever it served upstream. ? 0 -$ tbd docs fork --category=general --category=python --dry-run -[DRY-RUN] Would fork 11 docs into docs/tbd/ (categories: general, python) - guideline general-eng-agent-principles - guideline general-coding-rules - [.. 7 more ..] - guideline python-rules +$ tbd docs fork --all --dry-run +[DRY-RUN] Would fork [..] doc(s) into docs/tbd/ +... No files written. Re-run without --dry-run to apply. ? 0 $ tbd docs fork python-rules # target exists and is not an unmodified fork -✗ docs/tbd/guidelines/python-rules.md already exists and is not an unmodified fork. - Refusing to overwrite it. Options: - tbd docs diff python-rules # see how it differs - tbd docs fork python-rules --force # overwrite with upstream +Error: docs/tbd/guidelines/python-rules.md already exists and is not an unmodified fork. Refusing to overwrite it. Options: + tbd docs diff python-rules # see how it differs + tbd docs fork python-rules --force # overwrite with upstream ? 1 ``` +*(Phase 4 contract)* `tbd docs fork --category=general --category=python --dry-run` +previews the same way with a `(categories: general, python)` suffix on the summary line. + ### `tbd docs unfork` ```text $ tbd docs unfork python-rules # customized → refuse -✗ python-rules has local customizations (differs from its base). - Refusing to discard them. Options: - tbd docs diff python-rules # review your changes - tbd docs unfork python-rules --force # discard and fall back to upstream +Error: python-rules has local customizations (differs from its base). Refusing to discard them. Options: + tbd docs diff python-rules # review your changes + tbd docs unfork python-rules --force # discard and fall back to upstream ? 1 $ tbd docs unfork review-code # unmodified → succeeds ✓ Unforked review-code — served from upstream again. - Removed the forked file, its base snapshot, and its manifest entry. - Regenerated docs/tbd/README.md ? 0 ``` @@ -1602,11 +1626,11 @@ python-rules guideline customized, stale internal:guidelines/python-rules.m review-code shortcut missing internal:shortcuts/standard/review-code.md tbd-docs reference forked internal:tbd-docs.md -1 doc is missing (forked file deleted): +1 doc(s) missing (forked file deleted or renamed): review-code restore with 'tbd docs fork review-code --force', or finalize with 'tbd docs unfork review-code' ? 0 -$ tbd doctor --fix # excerpt +$ tbd doctor --fix # excerpt — (Phase 2 contract: doctor checks) ⚠ Forked docs - 1 missing (review-code: forked file deleted) Fixed: finalized unfork (removed manifest entry + base); now served from upstream ? 0 @@ -1635,33 +1659,36 @@ merge, and *list* the conflict for a decision (never touch it by default): ```text $ tbd docs update -Updated 2 forked docs: - ✓ review-code refreshed to upstream (was unmodified) - ✓ python-rules merged upstream cleanly (review with: git diff) - -1 doc needs a decision: - ⚠ acme-style your changes conflict with upstream - re-run with one of: - tbd docs update acme-style --merge # combine, then resolve conflict markers - tbd docs update acme-style --keep-ours # keep your version, advance the fork point +Updated 2 forked doc(s): + ✓ review-code: refreshed to upstream (was unmodified) + ✓ python-rules: merged upstream cleanly (review with: git diff) + +1 doc(s) need a decision: + ⚠ acme-style: your changes conflict with upstream + re-run with one of: + tbd docs update --merge # combine, then resolve conflict markers + tbd docs update --keep-ours # keep your version, advance the fork point ? 0 $ tbd docs update acme-style --merge -✓ acme-style wrote merged content with conflict markers; base advanced. - Resolve the <<<<<<< / ======= / >>>>>>> markers, then the doc returns to 'customized'. +Updated 1 forked doc(s): + ✓ acme-style: wrote merged content with conflict markers; resolve them, then it returns to 'customized' ? 0 $ tbd docs update acme-style --keep-ours -✓ acme-style kept your version; fork point advanced to current upstream. +Updated 1 forked doc(s): + ✓ acme-style: kept your version; fork point advanced ? 0 $ tbd docs update --dry-run -[DRY-RUN] 2 docs would update (1 refresh, 1 clean merge); 1 would conflict (acme-style). -No files written. +Would update 2 forked doc(s): + ✓ review-code: refreshed to upstream (was unmodified) + ✓ python-rules: merged upstream cleanly (review with: git diff) +... ? 0 ``` -### `tbd docs add` +### `tbd docs add` *(Phase 2 contract — not yet implemented)* Aligned with today’s `--add` output, restated for docrefs and the new pointers: @@ -1676,7 +1703,7 @@ Run 'tbd docs list' to verify, or 'tbd docs fork acme-style' to make it visible. ? 0 ``` -### `tbd status` (Docs line) and `tbd setup --auto` (Docs summary) +### `tbd status` (Docs line) and `tbd setup --auto` (Docs summary) *(Phase 3/4 contracts — not yet implemented)* `tbd status` gains a Docs line **only when forks exist** — so with zero forks the output is byte-identical to today’s `cli-orientation-golden.tryscript.md` (honoring the @@ -1697,21 +1724,23 @@ Use 'tbd stats' for issue statistics, 'tbd doctor' for health checks. pending-update report); setup never writes the fork dir: ```text -# zero forks -Docs: 37 available in cache (.tbd/docs/, gitignored); none forked into the repo. - Guidelines are active from the cache. Fork them into docs/tbd/ to make them - visible and customizable (same behavior — just explicit and git-tracked): - Scope: all standard guidelines (recommended), or a category: - general, typescript, python, convex, electron - Make visible: tbd docs fork --category=general --category= - tbd docs fork --all (everything) +# zero forks — same three-posture menu as the bare overview, prefixed Docs: +Docs: [..] available in the cache (.tbd/docs/, gitignored); none forked into the repo. + Guidelines are active from the cache. Three postures, all serving the same docs: + Hidden (default): keep the cache as-is — zero repo footprint + Curated: tbd docs fork [...] fork chosen docs into docs/tbd/ + Everything: tbd docs fork --all all docs, visible and editable Browse / read: tbd docs list / tbd docs show # after an upgrade, forks present Docs: 4 forked into docs/tbd/. 3 have upstream updates — run 'tbd docs update'. ``` -### `tbd doctor` (new HEALTH CHECKS) +The setup menu and the bare-overview menu share wording by construction; when Phase 4 +adds `--category` both gain the category line together (never name a selector that does +not exist). + +### `tbd doctor` (new HEALTH CHECKS) *(Phase 2 contract — not yet implemented)* Appended to the existing `HEALTH CHECKS` list, following doctor’s `✓`/`⚠` + `Run:` convention (icon at column 0, no indent): @@ -1735,15 +1764,15 @@ bead, blocked on the phase that ships the behavior: | Test | Change | Phase | | --- | --- | --- | -| `cli-help-all.tryscript.md` (≈7 `tbd docs` blocks: `--help` `[topic]`/`--section`, `--list` sections, positional topic, `--section` content, `--list --json`, bare manual) | **Rewrite** to the new surface: `docs` subcommand help; no top-level `--section`/section-`--list`; section nav becomes `tbd docs show tbd-docs --section`. Largest single change. | 1–2 | -| `cli-doc-output.tryscript.md` ("Docs Command" block: `tbd docs --list` → “Available documentation sections:”) | **Rewrite** to `tbd docs list` (cross-kind, state markers) + `--json` docmap. | 2 | -| `cli-doc-output.tryscript.md` ("Guidelines --json returns structured data" block: flat `[ { … } ]` array) | **Rewrite**: per-kind `--list --json` now emits a docmap object, not an array (Resolved Decision 21). The per-kind `--list` *text* blocks stay (same canonical format), so only the JSON assertion changes. | 2 | -| `golden-output.test.ts` (`tbd docs --all` inline snapshot) | **Replace** with the bare `tbd docs` overview snapshot (`--all` folded into the overview). | 2 | -| `golden-output.test.ts` ("post-setup What’s Next") | **Extend** to assert the Docs menu lines. | 4 | +| `cli-help-all.tryscript.md` (≈7 `tbd docs` blocks: `--help` `[topic]`/`--section`, `--list` sections, positional topic, `--section` content, `--list --json`, bare manual) | **Done (this PR).** Rewrite to the new surface: `docs` subcommand help; no top-level `--section`/section-`--list`; section nav becomes `tbd docs show tbd-docs --section`. Largest single change. | 1–2 | +| `cli-doc-output.tryscript.md` ("Docs Command" block: `tbd docs --list` → “Available documentation sections:”) | **Done (this PR).** Section listing retargeted to `tbd docs show tbd-docs --sections`; the cross-kind `tbd docs list` golden lives in `cli-docs-fork.tryscript.md`. | 2 | +| `cli-doc-output.tryscript.md` ("Guidelines --json returns structured data" block: flat `[ { … } ]` array) | **Pending (Phase 2, renderer migration).** Rewrite: per-kind `--list --json` now emits a docmap object, not an array (Resolved Decision 21). The per-kind `--list` *text* blocks stay (same canonical format), so only the JSON assertion changes. | 2 | +| `golden-output.test.ts` (`tbd docs --all` inline snapshot) | **Done (this PR).** Replace with the bare `tbd docs` overview snapshot (`--all` folded into the overview). | 2 | +| `golden-output.test.ts` ("post-setup What’s Next") | **Pending (Phase 4).** Extend to assert the Docs menu lines. | 4 | | `cli-orientation-golden.tryscript.md` (`tbd status`) | **Unchanged** for zero forks (verifies the guarantee); **add** a new forked-state status golden in a fixture with a fork. | 1 | -| `setup-flows.test.ts` | **Extend** for the Docs summary (menu + pending-update report). | 4 | -| `doc-references.test.ts` | **Extend** the extractor: add `tbd docs ` and the `reference` kind; remove the `reference`/`prefix:` skips so `tbd docs show tbd-docs`, `suggest-upstream-improvements`, and the new reference docs all resolve. | 5 (extractor); 0 (the new docs it must resolve) | -| `doc-add-e2e.test.ts` | **Keep** (per-kind `--add` stays an alias) and **extend** with `tbd docs add `. | 2 | +| `setup-flows.test.ts` | **Pending (Phase 4).** Extend for the Docs summary (menu + pending-update report). | 4 | +| `doc-references.test.ts` | **Pending (Phases 0/5).** Extend the extractor: add `tbd docs ` and the `reference` kind; remove the `reference`/`prefix:` skips so `tbd docs show tbd-docs`, `suggest-upstream-improvements`, and the new reference docs all resolve. | 5 (extractor); 0 (the new docs it must resolve) | +| `doc-add-e2e.test.ts` | **Pending (Phase 2).** Keep (per-kind `--add` stays an alias) and **extend** with `tbd docs add `. | 2 | New golden/e2e files (named for the phases that add them): `fork-manifest` + state-matrix units (Phase 1); a `cli-docs-fork.tryscript.md` lifecycle (fork → list diff --git a/packages/tbd/docs/tbd-docs.md b/packages/tbd/docs/tbd-docs.md index fb86e325..7b453d52 100644 --- a/packages/tbd/docs/tbd-docs.md +++ b/packages/tbd/docs/tbd-docs.md @@ -675,12 +675,23 @@ and skips reinstallation. ### Documentation Commands -Built-in documentation viewers: +Managed docs (the `tbd docs` group): + +```bash +tbd docs # Status overview of managed docs +tbd docs list # All docs across kinds, with state markers +tbd docs show # Read any doc by name (kind-agnostic) +tbd docs show tbd-docs # The CLI manual (alias: tbd docs manual) +tbd docs show tbd-docs --sections # List the manual's sections +tbd docs show tbd-docs --section # Read one manual section +tbd docs sync # Refresh the gitignored docs cache +tbd docs fork / unfork / update / diff / status # Forked docs (see below) +``` + +Other built-in viewers: ```bash tbd readme # Display README (same as GitHub landing page) -tbd docs # Display CLI reference documentation -tbd docs --list # List available documentation sections tbd design # Display design documentation tbd design --list # List design doc sections tbd closing # Display session closing protocol reminder diff --git a/packages/tbd/src/cli/commands/docs-fork.ts b/packages/tbd/src/cli/commands/docs-fork.ts index 9a730f54..ca07b8aa 100644 --- a/packages/tbd/src/cli/commands/docs-fork.ts +++ b/packages/tbd/src/cli/commands/docs-fork.ts @@ -58,13 +58,13 @@ import { updateOne, diffContents, type UpdateStrategy } from '../../file/fork-up import { createDocMap, type DocMapEntry } from '../../docmap/index.js'; /** Kinds that can be resolved from the cache and forked today. */ -const RESOLVABLE_KINDS: ForkKind[] = ['guideline', 'shortcut', 'template']; +export const RESOLVABLE_KINDS: ForkKind[] = ['guideline', 'shortcut', 'template']; /** * Validate a user-supplied --kind value. Without this, an unknown kind silently * produces an empty cache and misleading "no docs" output. */ -function parseKindOption(kind: string | undefined): ForkKind | undefined { +export function parseKindOption(kind: string | undefined): ForkKind | undefined { if (kind === undefined) return undefined; if (!(RESOLVABLE_KINDS as string[]).includes(kind)) { throw new CLIError(`Unknown kind "${kind}". Valid kinds: ${RESOLVABLE_KINDS.join(', ')}.`); @@ -290,8 +290,10 @@ class DocsUnforkHandler extends BaseCommand { } catch (err) { if (err instanceof ForkConflictError && err.code === 'customized') { throw new CLIError( - `${name} has local customizations. Review with \`tbd docs status\`, then ` + - `re-run with --force to discard them and fall back to upstream.`, + `${name} has local customizations (differs from its base). ` + + `Refusing to discard them. Options:\n` + + ` tbd docs diff ${name} # review your changes\n` + + ` tbd docs unfork ${name} --force # discard and fall back to upstream`, ); } throw err; @@ -450,8 +452,8 @@ class DocsUpdateHandler extends BaseCommand { const tbdRoot = await requireInit(); const applied: { entry: ForkEntry; message: string }[] = []; - const decisions: string[] = []; - const skipped: string[] = []; + const decisions: { name: string; message: string }[] = []; + const skipped: { name: string; message: string }[] = []; await withForkManifestLock(tbdRoot, async () => { let manifest = await readForkManifest(tbdRoot); @@ -500,11 +502,11 @@ class DocsUpdateHandler extends BaseCommand { manifest = upsertFork(manifest, cleared); } if (result.needsDecision) { - decisions.push(result.message); + decisions.push({ name: entry.name, message: result.message }); } else if (result.action !== 'skip-not-stale') { // Conflicted / orphaned / missing / version-skewed: actionable but // not applied here — surface, never silently swallow. - skipped.push(result.message); + skipped.push({ name: entry.name, message: result.message }); } continue; } @@ -564,15 +566,15 @@ class DocsUpdateHandler extends BaseCommand { if (skipped.length > 0) { console.log(''); console.log(`${skipped.length} doc(s) skipped:`); - for (const msg of skipped) { - console.log(` ${colors.warn('⚠')} ${msg}`); + for (const sk of skipped) { + console.log(` ${colors.warn('⚠')} ${sk.message}`); } } if (decisions.length > 0) { console.log(''); console.log(`${decisions.length} doc(s) need a decision:`); - for (const msg of decisions) { - console.log(` ${colors.warn('⚠')} ${msg}`); + for (const d of decisions) { + console.log(` ${colors.warn('⚠')} ${d.message}`); } console.log(' re-run with one of:'); console.log( @@ -745,9 +747,8 @@ class DocsDiffHandler extends BaseCommand { } /** - * Merge a subcommand's local options with globals/ancestors. The parent `docs` - * command also declares `--all` (its manual-viewer listing), so reading the local - * option alone is unreliable; fall back to the merged view. + * Merge a subcommand's local options with globals/ancestors (e.g. the global + * --dry-run and --json), preferring the subcommand's own values. */ function mergedForkOptions(local: ForkOptions, command: Command): ForkOptions { const g = command.optsWithGlobals(); diff --git a/packages/tbd/src/cli/commands/docs.ts b/packages/tbd/src/cli/commands/docs.ts index ca419b55..f5f5b0f1 100644 --- a/packages/tbd/src/cli/commands/docs.ts +++ b/packages/tbd/src/cli/commands/docs.ts @@ -1,254 +1,359 @@ /** - * `tbd docs` - Display CLI documentation. + * `tbd docs` — manage tbd-served docs: browse, fork into the repo, sync the + * cache, and pull upstream updates into forks. * - * Shows the bundled documentation for tbd CLI. - * Documentation can be filtered by section. - * - * Note: Doc cache sync functionality has moved to `tbd sync --docs`. - * See: docs/project/specs/active/plan-2026-01-29-unified-sync-command.md + * Surface (the f05 reorganization of the old manual viewer): + * tbd docs status overview of managed docs (landing page) + * tbd docs show read any doc by name (kind-agnostic; --section) + * tbd docs show tbd-docs the CLI manual (old bare `tbd docs`) + * tbd docs manual [topic] alias for `tbd docs show tbd-docs` + * tbd docs sync refresh the gitignored cache (canonical form of + * the deprecated `tbd sync --docs` alias) + * tbd docs list/status/fork/unfork/update/diff (see docs-fork.ts) */ import { Command } from 'commander'; import { readFile } from 'node:fs/promises'; import { fileURLToPath } from 'node:url'; -import { dirname, join } from 'node:path'; +import { dirname, join, relative } from 'node:path'; import { BaseCommand } from '../lib/base-command.js'; -import { registerForkSubcommands } from './docs-fork.js'; +import { registerForkSubcommands, parseKindOption, RESOLVABLE_KINDS } from './docs-fork.js'; import { shouldUseInteractiveOutput } from '../lib/context.js'; -import { CLIError, NotFoundError } from '../lib/errors.js'; +import { CLIError, NotFoundError, NotInitializedError, requireInit } from '../lib/errors.js'; import { renderMarkdown, paginateOutput } from '../lib/output.js'; +import { + printDocSyncResult, + printDocSyncStatus, + printForkDriftNotice, +} from '../lib/docs-sync-output.js'; +import { syncDocsWithDefaults } from '../../file/doc-sync.js'; +import { DocCache } from '../../file/doc-cache.js'; +import { + DEFAULT_GUIDELINES_PATHS, + DEFAULT_SHORTCUT_PATHS, + DEFAULT_TEMPLATE_PATHS, + FORK_DIR, +} from '../../lib/paths.js'; +import { readForkManifest, type ForkKind } from '../../file/fork-manifest.js'; +import { computeForkDriftSummary } from '../../file/doc-fork.js'; import type { DocSection } from '../../lib/types.js'; import GithubSlugger from 'github-slugger'; -/** - * Get the path to the bundled docs file. - * The docs file is copied to dist/docs/ during build. - */ -function getDocsPath(): string { - const __filename = fileURLToPath(import.meta.url); - const __dirname = dirname(__filename); - // When bundled, runs from dist/bin.mjs or dist/cli.mjs - // Docs are at dist/docs/tbd-docs.md (same level as the bundle) - return join(__dirname, 'docs', 'tbd-docs.md'); -} +/** Reserved name that serves the bundled CLI manual (`tbd-docs.md`). */ +const MANUAL_DOC_NAME = 'tbd-docs'; -interface DocsOptions { - section?: string; - list?: boolean; - all?: boolean; -} +/** Serving lookup paths per kind (fork dir first, so forks shadow the cache). */ +const SHOW_PATHS: Record = { + guideline: DEFAULT_GUIDELINES_PATHS, + shortcut: DEFAULT_SHORTCUT_PATHS, + template: DEFAULT_TEMPLATE_PATHS, + reference: [], +}; -class DocsHandler extends BaseCommand { - async run(topic: string | undefined, options: DocsOptions): Promise { - let content: string; +/** + * Path to the bundled manual. The docs file is copied to dist/docs/ during + * build; in development it is read from the package docs/ directory. + */ +async function readManualContent(): Promise { + const __dirname = dirname(fileURLToPath(import.meta.url)); + try { + return await readFile(join(__dirname, 'docs', 'tbd-docs.md'), 'utf-8'); + } catch { try { - content = await readFile(getDocsPath(), 'utf-8'); + return await readFile(join(__dirname, '..', '..', '..', 'docs', 'tbd-docs.md'), 'utf-8'); } catch { - // Fallback: try to read from source location during development - try { - const __filename = fileURLToPath(import.meta.url); - const __dirname = dirname(__filename); - // During development: src/cli/commands -> packages/tbd/docs - const devPath = join(__dirname, '..', '..', '..', 'docs', 'tbd-docs.md'); - content = await readFile(devPath, 'utf-8'); - } catch { - throw new CLIError('Documentation file not found. Please rebuild the CLI.'); - } + throw new CLIError('Documentation file not found. Please rebuild the CLI.'); } + } +} - const sections = this.extractSections(content); +/** Extract `## ` section metadata (title + GitHub slug) from a markdown doc. */ +function extractSections(content: string): DocSection[] { + const sections: DocSection[] = []; + const slugger = new GithubSlugger(); + for (const line of content.split('\n')) { + if (line.startsWith('## ')) { + const title = line.slice(3).trim(); + sections.push({ title, slug: slugger.slug(title) }); + } + } + return sections; +} + +/** + * Extract one `## ` section (header through the next header), matching by slug + * or partial title. Returns null when no section matches. + */ +function extractSection(content: string, sections: DocSection[], query: string): string | null { + const lowerQuery = query.toLowerCase(); + const matched = + sections.find((s) => s.slug === lowerQuery) ?? + sections.find((s) => s.title.toLowerCase().includes(lowerQuery)); + if (!matched) { + return null; + } - // Show comprehensive documentation listing - if (options.all) { - await this.showComprehensiveListing(); - return; + const lines = content.split('\n'); + let inSection = false; + const sectionLines: string[] = []; + for (const line of lines) { + if (line.startsWith('## ')) { + if (inSection) break; + if (line.slice(3).trim() === matched.title) { + inSection = true; + sectionLines.push(line); + } + } else if (inSection) { + sectionLines.push(line); } + } + while (sectionLines.length > 0 && sectionLines[sectionLines.length - 1]?.trim() === '') { + sectionLines.pop(); + } + return sectionLines.length > 0 ? sectionLines.join('\n') : null; +} - // List available sections - if (options.list) { - this.output.data(sections, () => { +/** + * Bare `tbd docs`: the status overview / landing page for managed docs. + * Mirrors `tbd status`: a summary plus pointers, never the full table. + */ +class DocsOverviewHandler extends BaseCommand { + async run(): Promise { + await this.execute(async () => { + let tbdRoot: string; + try { + tbdRoot = await requireInit(); + } catch (err) { + if (!(err instanceof NotInitializedError)) throw err; + // The overview stays useful before setup (the old viewer worked + // anywhere): point at the bundled manual and at initialization. const colors = this.output.getColors(); - console.log(colors.bold('Available documentation sections:')); + console.log(`${colors.bold('tbd docs')} — managed documentation`); console.log(''); - // Calculate max slug length for alignment - const maxSlugLen = Math.max(...sections.map((s) => s.slug.length)); - for (const section of sections) { - const paddedSlug = section.slug.padEnd(maxSlugLen); - console.log(` ${colors.id(paddedSlug)} ${section.title}`); - } + console.log(' tbd is not initialized in this repo (run: tbd setup --auto).'); + console.log(' The CLI manual is bundled and always available:'); console.log(''); - console.log(`Use ${colors.dim('tbd docs ')} to view a specific section.`); - }); - return; - } - - // Determine which section to show (positional topic takes precedence) - const sectionQuery = topic ?? options.section; + console.log(' Learn more: tbd docs show tbd-docs (alias: tbd docs manual)'); + return; + } + const manifest = await readForkManifest(tbdRoot); + const drift = await computeForkDriftSummary(tbdRoot, FORK_DIR, manifest); + + let total = 0; + for (const kind of RESOLVABLE_KINDS) { + const cache = new DocCache(SHOW_PATHS[kind], tbdRoot); + await cache.load({ quiet: true }); + total += cache.list().length; + } - // Filter by section if specified - if (sectionQuery) { - const sectionContent = this.extractSection(content, sections, sectionQuery); - if (!sectionContent) { - throw new NotFoundError( - 'Section', - `"${sectionQuery}" (use --list to see available sections)`, - ); + if (this.ctx.json) { + this.output.data({ available: total, ...drift }); + return; } - content = sectionContent; - } - // Output the documentation with Markdown colorization and pagination for interactive - if (shouldUseInteractiveOutput(this.ctx)) { - const rendered = renderMarkdown(content, this.ctx.color); - await paginateOutput(rendered, true); - } else { - console.log(content); - } - } + const colors = this.output.getColors(); + console.log(`${colors.bold('tbd docs')} — managed documentation`); + console.log(''); - /** - * Extract section metadata from the documentation. - * Sections are top-level headers (## ). - * Returns title and slugified ID for each section. - */ - private extractSections(content: string): DocSection[] { - const sections: DocSection[] = []; - const lines = content.split('\n'); - const slugger = new GithubSlugger(); - - for (const line of lines) { - if (line.startsWith('## ')) { - const title = line.slice(3).trim(); - const slug = slugger.slug(title); - sections.push({ title, slug }); + if (drift.forks === 0) { + console.log( + ` ${total} docs available in the cache (.tbd/docs/, gitignored); none forked into the repo.`, + ); + console.log( + ' Guidelines are active from the cache. Three postures, all serving the same docs:', + ); + console.log(''); + console.log(' Hidden (default): keep the cache as-is — zero repo footprint'); + console.log( + ` Curated: ${colors.bold('tbd docs fork [...]')} fork chosen docs into ${FORK_DIR}/`, + ); + console.log( + ` Everything: ${colors.bold('tbd docs fork --all')} all docs, visible and editable`, + ); + console.log(''); + console.log(` Browse / read: tbd docs list / tbd docs show `); + console.log( + ` Learn more: tbd docs show tbd-docs (the manual; alias: tbd docs manual)`, + ); + return; } - } - return sections; + const upstream = total - drift.forks; + console.log( + ` ${total} available (${upstream} upstream, ${drift.forks} forked into ${FORK_DIR}/)`, + ); + const parts = [`${drift.customized} customized`]; + if (drift.stale > 0) { + parts.push(`${drift.stale} with upstream updates — run 'tbd docs update'`); + } + if (drift.conflicted > 0) parts.push(`${drift.conflicted} conflict pending`); + if (drift.missing > 0) parts.push(`${drift.missing} missing — see 'tbd docs status'`); + if (drift.local > 0) parts.push(`${drift.local} local`); + console.log(` ${drift.forks} forked: ${parts.join(', ')}`); + console.log(''); + console.log(' Inspect: tbd docs status'); + console.log(' Browse: tbd docs list'); + console.log(' Update: tbd docs update'); + console.log(' Learn more: tbd docs show tbd-docs'); + }, 'Failed to read docs overview'); } +} - /** - * Extract a specific section from the documentation. - * Matches by slug or partial title match. - * Returns content from the section header to the next section header. - */ - private extractSection(content: string, sections: DocSection[], query: string): string | null { - const lowerQuery = query.toLowerCase(); - - // Find matching section - first try exact slug match, then partial title match - const matchedSection = - sections.find((s) => s.slug === lowerQuery) ?? - sections.find((s) => s.title.toLowerCase().includes(lowerQuery)); - - if (!matchedSection) { - return null; - } - - const lines = content.split('\n'); - let inSection = false; - const sectionLines: string[] = []; +interface ShowOptions { + section?: string; + sections?: boolean; + kind?: string; +} - for (const line of lines) { - if (line.startsWith('## ')) { - if (inSection) { - // End of our section - break; +/** + * `tbd docs show `: kind-agnostic read of any managed doc. The reserved + * `tbd-docs` name serves the bundled CLI manual (with `--section` navigation, + * relocated here from the old bare `tbd docs` viewer). + */ +class DocsShowHandler extends BaseCommand { + async run(name: string, options: ShowOptions): Promise { + await this.execute(async () => { + let content: string; + let provenance: string | null = null; + + if (name === MANUAL_DOC_NAME) { + content = await readManualContent(); + } else { + const tbdRoot = await requireInit(); + const requestedKind = parseKindOption(options.kind); + const kinds = requestedKind ? [requestedKind] : RESOLVABLE_KINDS; + const matches: { kind: ForkKind; content: string; sourceDir: string; path: string }[] = []; + for (const kind of kinds) { + const cache = new DocCache(SHOW_PATHS[kind], tbdRoot); + await cache.load({ quiet: true }); + const hit = cache.get(name); + if (hit) { + matches.push({ + kind, + content: hit.doc.content, + sourceDir: hit.doc.sourceDir, + path: hit.doc.path, + }); + } } - const currentTitle = line.slice(3).trim(); - if (currentTitle === matchedSection.title) { - inSection = true; - sectionLines.push(line); + if (matches.length === 0) { + throw new NotFoundError('Doc', `"${name}" (run \`tbd docs list\` to see names)`); + } + if (matches.length > 1) { + const kindList = matches.map((m) => m.kind).join(', '); + throw new CLIError( + `"${name}" exists in multiple kinds (${kindList}). Use --kind to disambiguate.`, + ); + } + const match = matches[0]!; + content = match.content; + if (match.sourceDir.startsWith(FORK_DIR)) { + provenance = relative(tbdRoot, match.path).split('\\').join('/'); } - } else if (inSection) { - sectionLines.push(line); } - } - if (sectionLines.length === 0) { - return null; - } + const sections = extractSections(content); + + if (options.sections) { + this.output.data(sections, () => { + const colors = this.output.getColors(); + console.log(colors.bold(`Sections in ${name}:`)); + console.log(''); + const maxSlugLen = Math.max(...sections.map((s) => s.slug.length)); + for (const section of sections) { + console.log(` ${colors.id(section.slug.padEnd(maxSlugLen))} ${section.title}`); + } + console.log(''); + console.log( + `Use ${colors.dim(`tbd docs show ${name} --section `)} to view a section.`, + ); + }); + return; + } - // Trim trailing empty lines - while (sectionLines.length > 0) { - const lastLine = sectionLines[sectionLines.length - 1]; - if (lastLine?.trim() === '') { - sectionLines.pop(); - } else { - break; + if (options.section) { + const sectionContent = extractSection(content, sections, options.section); + if (!sectionContent) { + throw new NotFoundError( + 'Section', + `"${options.section}" (use --sections to see available sections)`, + ); + } + content = sectionContent; + } + + // Provenance to stderr so piped stdout stays clean (on by default; + // the extra context helps agents recall which docs are customized). + if (provenance && !this.ctx.quiet && !this.ctx.json) { + process.stderr.write(`(serving forked copy: ${provenance})\n`); } - } - return sectionLines.join('\n'); + if (shouldUseInteractiveOutput(this.ctx)) { + const rendered = renderMarkdown(content, this.ctx.color); + await paginateOutput(rendered, true); + } else { + console.log(content); + } + }, 'Failed to show doc'); } +} - /** - * Show a comprehensive listing of all documentation resources organized by purpose. - */ - private async showComprehensiveListing(): Promise { - const colors = this.output.getColors(); - - console.log(colors.bold('=== tbd Documentation Resources ===')); - console.log(''); - - // Getting Started - console.log(colors.bold('Getting Started:')); - console.log(' tbd Full orientation and project status'); - console.log(' tbd prime Workflow context and guidance'); - console.log(' tbd prime --brief Quick reference (~35 lines)'); - console.log(' tbd --help CLI command reference'); - console.log(''); - - // Workflows (Shortcuts) - console.log(colors.bold('Workflows (Shortcuts):')); - console.log(' tbd shortcut --list List all available shortcuts'); - console.log(' tbd shortcut new-plan-spec Plan a new feature'); - console.log(' tbd shortcut code-review-and-commit Commit code properly'); - console.log(' tbd shortcut create-or-update-pr-simple Create a pull request'); - console.log(''); - - // Guidelines - console.log(colors.bold('Guidelines (Coding Standards):')); - console.log(' tbd guidelines --list List all available guidelines'); - console.log(' tbd guidelines typescript-rules TypeScript best practices'); - console.log(' tbd guidelines general-tdd-guidelines Test-driven development'); - console.log(' tbd guidelines golden-testing-guidelines Snapshot/golden testing'); - console.log(''); - - // Templates - console.log(colors.bold('Templates:')); - console.log(' tbd template --list List all available templates'); - console.log(' tbd template plan-spec Feature planning template'); - console.log(' tbd template architecture-doc Architecture document template'); - console.log(''); - - // Design & Reference - console.log(colors.bold('Design & Reference:')); - console.log(' tbd docs --list List documentation sections'); - console.log(' tbd design tbd design document'); - console.log(' tbd closing Session closing protocol'); - console.log(''); - - // Quick Tips - console.log(colors.bold('Quick Tips:')); - console.log(' - Run tbd ready to see what issues are available to work on'); - console.log(' - Run tbd shortcut to get step-by-step instructions'); - console.log(' - Run tbd guidelines to get coding standards'); - console.log(' - Always run tbd sync at the end of a session'); +/** `tbd docs sync`: refresh the gitignored docs cache (canonical command). */ +class DocsSyncHandler extends BaseCommand { + async run(): Promise { + await this.execute(async () => { + const tbdRoot = await requireInit(); + if (this.ctx.dryRun) { + const result = await syncDocsWithDefaults(tbdRoot, { dryRun: true }); + printDocSyncStatus(this.output, result); + return; + } + const spinner = this.output.spinner('Syncing docs...'); + const result = await syncDocsWithDefaults(tbdRoot); + spinner.stop(); + printDocSyncResult(this.output, result); + await printForkDriftNotice(this.output, tbdRoot); + }, 'Failed to sync docs'); } } export const docsCommand = new Command('docs') - .description('Display CLI documentation (use tbd sync --docs for doc cache sync)') - .argument('[topic]', 'Topic to display (e.g., "commands", "id-system")') - .option('--section ', 'Show specific section (e.g., "commands", "workflows")') - .option('--list', 'List available sections') - .option('--all', 'Show comprehensive listing of all documentation resources') - .action(async (topic: string | undefined, options: DocsOptions, command: Command) => { - const handler = new DocsHandler(command); - await handler.run(topic, options); + .description('Manage tbd-served docs: browse, fork into your repo, and pull upstream updates') + .action(async (_options: unknown, command: Command) => { + await new DocsOverviewHandler(command).run(); + }); + +docsCommand + .command('show') + .description('Read any managed doc by name (tbd-docs is the CLI manual)') + .argument('', 'doc name (e.g. python-rules, tbd-docs)') + .option('--section ', 'show one section of the doc') + .option('--sections', 'list the doc’s sections') + .option('--kind ', 'restrict to a kind (guideline|shortcut|template)') + .action(async (name: string, options: ShowOptions, command: Command) => { + await new DocsShowHandler(command).run(name, options); + }); + +docsCommand + .command('manual') + .description('Show the tbd CLI manual (alias for: tbd docs show tbd-docs)') + .argument('[topic]', 'section to display (e.g. "commands", "id-system")') + .option('--section ', 'show one section of the manual') + .option('--sections', 'list the manual’s sections') + .action(async (topic: string | undefined, options: ShowOptions, command: Command) => { + await new DocsShowHandler(command).run(MANUAL_DOC_NAME, { + ...options, + section: options.section ?? topic, + }); + }); + +docsCommand + .command('sync') + .description('Refresh the gitignored docs cache (.tbd/docs/) from bundled and URL sources') + .action(async (_options: unknown, command: Command) => { + await new DocsSyncHandler(command).run(); }); -// Forkable-docs operations (fork / unfork / status) are added as subcommands. -// The existing manual-viewer behavior remains the default action above. +// Fork lifecycle operations (fork / unfork / status / list / update / diff). registerForkSubcommands(docsCommand); diff --git a/packages/tbd/src/cli/commands/sync.ts b/packages/tbd/src/cli/commands/sync.ts index d3efbd11..8f0f7b4b 100644 --- a/packages/tbd/src/cli/commands/sync.ts +++ b/packages/tbd/src/cli/commands/sync.ts @@ -24,9 +24,12 @@ import { type ConflictEntry, type PushResult, } from '../../file/git.js'; -import { DATA_SYNC_DIR, FORK_DIR } from '../../lib/paths.js'; -import { readForkManifest } from '../../file/fork-manifest.js'; -import { computeForkDriftSummary } from '../../file/doc-fork.js'; +import { DATA_SYNC_DIR } from '../../lib/paths.js'; +import { + printDocSyncResult, + printDocSyncStatus, + printForkDriftNotice, +} from '../lib/docs-sync-output.js'; import { basename, join } from 'node:path'; import { access, readFile } from 'node:fs/promises'; import { writeFile } from 'atomically'; @@ -181,7 +184,7 @@ class SyncHandler extends BaseCommand { if (statusOnly) { // Show status without making changes const result = await syncDocsWithDefaults(this.tbdRoot, { dryRun: true }); - this.showDocStatus(result); + printDocSyncStatus(this.output, result); return result; } @@ -190,71 +193,11 @@ class SyncHandler extends BaseCommand { spinner.stop(); // Report results - this.showDocSyncResult(result); - await this.notifyForkDrift(); + printDocSyncResult(this.output, result); + await printForkDriftNotice(this.output, this.tbdRoot); return result; } - /** - * One-line awareness notice for forked docs (docs/tbd/): the cache refresh - * above is exactly when forks can become stale, and agents run `tbd sync` - * routinely — so drift is surfaced here, but never acted on (only the - * explicit `tbd docs update` mutates tracked files). - */ - private async notifyForkDrift(): Promise { - try { - const manifest = await readForkManifest(this.tbdRoot); - const drift = await computeForkDriftSummary(this.tbdRoot, FORK_DIR, manifest); - if (drift.forks === 0) return; - const parts: string[] = []; - if (drift.stale > 0) { - parts.push(`${drift.stale} forked doc(s) have upstream updates — run 'tbd docs update'`); - } - if (drift.conflicted > 0) { - parts.push(`${drift.conflicted} with unresolved conflict markers`); - } - if (drift.missing > 0) { - parts.push(`${drift.missing} missing (deleted/renamed) — see 'tbd docs status'`); - } - if (parts.length > 0) { - process.stderr.write(`• Docs: ${parts.join('; ')}\n`); - } - } catch { - // Drift awareness is best-effort; never fail a sync over it. - } - } - - /** - * Show doc sync status (what would change). - */ - private showDocStatus(result: SyncDocsResult): void { - const colors = this.output.getColors(); - const hasChanges = - result.added.length > 0 || - result.updated.length > 0 || - result.removed.length > 0 || - result.pruned.length > 0; - - if (!hasChanges) { - this.output.success('Docs up to date'); - return; - } - - console.log(colors.bold('Docs:')); - if (result.added.length > 0) { - console.log(` ${colors.success(`+${result.added.length}`)} new doc(s) available`); - } - if (result.updated.length > 0) { - console.log(` ${colors.warn(`~${result.updated.length}`)} doc(s) to update`); - } - if (result.removed.length > 0) { - console.log(` ${colors.error(`-${result.removed.length}`)} doc(s) to remove`); - } - if (result.pruned.length > 0) { - console.log(` ${colors.dim(`${result.pruned.length}`)} stale config entry/entries`); - } - } - /** * Show doc sync result after sync. */ diff --git a/packages/tbd/src/cli/lib/docs-sync-output.ts b/packages/tbd/src/cli/lib/docs-sync-output.ts new file mode 100644 index 00000000..77d31b0f --- /dev/null +++ b/packages/tbd/src/cli/lib/docs-sync-output.ts @@ -0,0 +1,107 @@ +/** + * Shared rendering for docs-cache sync results and the forked-docs drift notice. + * + * Used by both `tbd docs sync` (the canonical command) and `tbd sync --docs` + * (kept as a deprecated alias), so the two surfaces cannot drift apart. + */ + +import type { OutputManager } from './output.js'; +import type { SyncDocsResult } from '../../file/doc-sync.js'; +import { FORK_DIR } from '../../lib/paths.js'; +import { readForkManifest } from '../../file/fork-manifest.js'; +import { computeForkDriftSummary } from '../../file/doc-fork.js'; + +/** Print the result of a docs-cache sync (writes applied). */ +export function printDocSyncResult(output: OutputManager, result: SyncDocsResult): void { + const hasChanges = + result.added.length > 0 || + result.updated.length > 0 || + result.removed.length > 0 || + result.pruned.length > 0; + + if (!hasChanges) { + output.success('Docs up to date'); + return; + } + + const parts: string[] = []; + if (result.added.length > 0) { + parts.push(`+${result.added.length}`); + } + if (result.updated.length > 0) { + parts.push(`~${result.updated.length}`); + } + if (result.removed.length > 0) { + parts.push(`-${result.removed.length}`); + } + + if (parts.length > 0) { + output.success(`Synced docs: ${parts.join(' ')} doc(s)`); + } + + if (result.pruned.length > 0) { + output.info(`Removed ${result.pruned.length} stale config entry/entries`); + } + + for (const err of result.errors) { + output.warn(`Doc sync error: ${err.path}: ${err.error}`); + } +} + +/** Print what a docs-cache sync would change (dry-run / status view). */ +export function printDocSyncStatus(output: OutputManager, result: SyncDocsResult): void { + const colors = output.getColors(); + const hasChanges = + result.added.length > 0 || + result.updated.length > 0 || + result.removed.length > 0 || + result.pruned.length > 0; + + if (!hasChanges) { + output.success('Docs up to date'); + return; + } + + console.log(colors.bold('Docs:')); + if (result.added.length > 0) { + console.log(` ${colors.success(`+${result.added.length}`)} new doc(s) available`); + } + if (result.updated.length > 0) { + console.log(` ${colors.warn(`~${result.updated.length}`)} doc(s) to update`); + } + if (result.removed.length > 0) { + console.log(` ${colors.error(`-${result.removed.length}`)} doc(s) to remove`); + } + if (result.pruned.length > 0) { + console.log(` ${colors.dim(`${result.pruned.length}`)} stale config entry/entries`); + } +} + +/** + * One-line awareness notice for forked docs: a cache refresh is exactly when + * forks become stale, so drift is surfaced here — but never acted on (only the + * explicit `tbd docs update` mutates tracked files). Best-effort: never fails + * the surrounding sync. + */ +export async function printForkDriftNotice(output: OutputManager, tbdRoot: string): Promise { + try { + const manifest = await readForkManifest(tbdRoot); + const drift = await computeForkDriftSummary(tbdRoot, FORK_DIR, manifest); + if (drift.forks === 0) return; + const parts: string[] = []; + if (drift.stale > 0) { + parts.push(`${drift.stale} forked doc(s) have upstream updates — run 'tbd docs update'`); + } + if (drift.conflicted > 0) { + parts.push(`${drift.conflicted} with unresolved conflict markers`); + } + if (drift.missing > 0) { + parts.push(`${drift.missing} missing (deleted/renamed) — see 'tbd docs status'`); + } + if (parts.length > 0) { + output.notice(`Docs: ${parts.join('; ')}`); + } + } catch { + // Drift awareness is best-effort; never fail a sync over it. + } +} diff --git a/packages/tbd/tests/cli-doc-output.tryscript.md b/packages/tbd/tests/cli-doc-output.tryscript.md index 5ec96319..2a8d3301 100644 --- a/packages/tbd/tests/cli-doc-output.tryscript.md +++ b/packages/tbd/tests/cli-doc-output.tryscript.md @@ -118,11 +118,11 @@ description: Welcome message for users after tbd installation or setup ## Docs Command -# Test: Docs command with --color=never produces clean output +# Test: Manual section listing with --color=never produces clean output ```console -$ tbd --color=never docs --list | head -3 -Available documentation sections: +$ tbd --color=never docs show tbd-docs --sections | head -3 +Sections in tbd-docs: [..] ? 0 diff --git a/packages/tbd/tests/cli-help-all.tryscript.md b/packages/tbd/tests/cli-help-all.tryscript.md index 6154342a..26c2ea3b 100644 --- a/packages/tbd/tests/cli-help-all.tryscript.md +++ b/packages/tbd/tests/cli-help-all.tryscript.md @@ -219,58 +219,54 @@ $ tbd import --help | grep -c "\-\-validate" ## Documentation Command Help -# Test: docs --help shows topic argument +# Test: docs --help shows the managed-docs subcommands ```console -$ tbd docs --help | grep -c "\[topic\]" -1 +$ tbd docs --help | grep -c "manual" +2 ? 0 ``` -# Test: docs --help shows section option +# Test: the old viewer flags are retired from the docs command ```console $ tbd docs --help | grep -c "\-\-section" -1 -? 0 +0 +? 1 ``` -# Test: docs --list shows slugified IDs +# Test: section listing lives on show --sections ```console -$ tbd docs --list | grep -c "id-system" +$ tbd docs show tbd-docs --sections | grep -c "id-system" 0 ? 1 ``` -# Test: docs --list shows available sections - ```console -$ tbd docs --list | grep -c "Quick Reference" +$ tbd docs show tbd-docs --sections | grep -c "Quick Reference" 1 ? 0 ``` -# Test: docs positional topic argument works +# Test: section navigation lives on show --section ```console -$ tbd docs id-system 2>&1 -Error: Section not found: "id-system" (use --list to see available sections) +$ tbd docs show tbd-docs --section id-system 2>&1 +Error: Section not found: "id-system" (use --sections to see available sections) ? 1 ``` -# Test: docs --section shows filtered content - ```console -$ tbd docs --section "ID System" 2>&1 -Error: Section not found: "ID System" (use --list to see available sections) +$ tbd docs show tbd-docs --section "ID System" 2>&1 +Error: Section not found: "ID System" (use --sections to see available sections) ? 1 ``` -# Test: docs --list --json outputs array with slugs +# Test: show --sections --json outputs array with slugs ```console -$ tbd docs --list --json +$ tbd docs show tbd-docs --sections --json [ { "title": "Key Design Features", @@ -344,10 +340,24 @@ $ tbd docs --list --json ? 0 ``` -# Test: docs shows full documentation +# Test: the manual is served by show tbd-docs and the manual alias + +```console +$ tbd docs show tbd-docs | grep -c "tbd CLI Documentation" +1 +? 0 +``` + +```console +$ tbd docs manual | grep -c "tbd CLI Documentation" +1 +? 0 +``` + +# Test: bare docs is the managed-docs overview (works before init) ```console -$ tbd docs | grep -c "tbd CLI Documentation" +$ tbd docs | grep -c "managed documentation" 1 ? 0 ``` diff --git a/packages/tbd/tests/cli-setup.tryscript.md b/packages/tbd/tests/cli-setup.tryscript.md index e3c03618..035ed43f 100644 --- a/packages/tbd/tests/cli-setup.tryscript.md +++ b/packages/tbd/tests/cli-setup.tryscript.md @@ -56,8 +56,8 @@ Documentation: guidelines [options] [query] Find and output coding guidelines template [options] [query] Find and output document templates closing Display the session closing protocol reminder - docs [options] [topic] Display CLI documentation (use tbd sync --docs - for doc cache sync) + docs Manage tbd-served docs: browse, fork into your + repo, and pull upstream updates design [options] [topic] Display design documentation and Beads comparison diff --git a/packages/tbd/tests/golden-output.test.ts b/packages/tbd/tests/golden-output.test.ts index 9d3c4114..2abf6b1d 100644 --- a/packages/tbd/tests/golden-output.test.ts +++ b/packages/tbd/tests/golden-output.test.ts @@ -54,49 +54,27 @@ describe('golden output tests', { timeout: isWindows ? 60000 : 15000 }, () => { runTbd(['init', '--prefix=test']); } - describe('tbd docs --all', () => { - it('shows comprehensive documentation listing', () => { + describe('tbd docs (bare overview)', () => { + it('shows the managed-docs overview with the three postures', () => { initGitAndTbd(); - const result = runTbd(['docs', '--all']); + const result = runTbd(['docs']); expect(result.status).toBe(0); - // Use inline snapshot to capture the exact format - expect(result.stdout).toMatchInlineSnapshot(` - "=== tbd Documentation Resources === - - Getting Started: - tbd Full orientation and project status - tbd prime Workflow context and guidance - tbd prime --brief Quick reference (~35 lines) - tbd --help CLI command reference - - Workflows (Shortcuts): - tbd shortcut --list List all available shortcuts - tbd shortcut new-plan-spec Plan a new feature - tbd shortcut code-review-and-commit Commit code properly - tbd shortcut create-or-update-pr-simple Create a pull request - - Guidelines (Coding Standards): - tbd guidelines --list List all available guidelines - tbd guidelines typescript-rules TypeScript best practices - tbd guidelines general-tdd-guidelines Test-driven development - tbd guidelines golden-testing-guidelines Snapshot/golden testing - - Templates: - tbd template --list List all available templates - tbd template plan-spec Feature planning template - tbd template architecture-doc Architecture document template - - Design & Reference: - tbd docs --list List documentation sections - tbd design tbd design document - tbd closing Session closing protocol - - Quick Tips: - - Run tbd ready to see what issues are available to work on - - Run tbd shortcut to get step-by-step instructions - - Run tbd guidelines to get coding standards - - Always run tbd sync at the end of a session + // The bundled-doc count grows over time; mask it so the golden pins + // the format, not the inventory size. + const stdout = result.stdout.replace(/\d+ docs available/, '[N] docs available'); + expect(stdout).toMatchInlineSnapshot(` + "tbd docs — managed documentation + + [N] docs available in the cache (.tbd/docs/, gitignored); none forked into the repo. + Guidelines are active from the cache. Three postures, all serving the same docs: + + Hidden (default): keep the cache as-is — zero repo footprint + Curated: tbd docs fork [...] fork chosen docs into docs/tbd/ + Everything: tbd docs fork --all all docs, visible and editable + + Browse / read: tbd docs list / tbd docs show + Learn more: tbd docs show tbd-docs (the manual; alias: tbd docs manual) " `); }); From e5ce02827db313da2df68025be71112da7d0d9ef Mon Sep 17 00:00:00 2001 From: Test User Date: Fri, 12 Jun 2026 20:23:26 +0000 Subject: [PATCH 28/36] feat: Cross-platform fork paths, docref/docmap reference docs, agent surface MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Closes the remaining tractable review beads: - One POSIX fork-dir constant (closes tbd-iqm1, with the tryscript-on-Windows half explicitly blocked upstream): FORK_DIR and its kind subdirs are POSIX literals (committed manifest paths and output must be platform-identical; fs access joins them), the duplicate DEFAULT_FORK_DIR is removed so tests exercise the production value, and a new cross-platform vitest e2e (fork-cross-platform-e2e.test.ts) pins POSIX manifest paths, fork-dir shadowing, show provenance, and unfork restoration on every CI OS — tryscript goldens cannot run on Windows because tryscript executes blocks via the platform shell (cmd), which is an upstream limitation. - Manifest source fields are validated as docrefs at read (the docref-everywhere rule now enforced somewhere real), and the docref and docmap formats ship as reference docs (references/docref-format.md, references/docmap-format.md) covering strictness, normalization, equality, purl prior art, location requirements, path relativity, version policy, and the view-not-input principle (closes tbd-vu3d, tbd-arsr; serving them via a reference kind remains Phase 5). - Agent surface: skill routing rows for list/fork/update/missing-file plus the docs command table entries (closes tbd-msh3), regenerated into all skill copies via setup --auto. - Manual: "Managing Docs: Two Modes" chapter (hidden cache vs forked, from first principles) with the four-row sync taxonomy table (closes tbd-j7im, tbd-low8); docs-overview.md rewritten for the tbd docs group (closes tbd-slcn). https://claude.ai/code/session_01QPsCSYGtwR8JtX2R1aaxyh --- .agents/skills/tbd/SKILL.md | 6 ++ .claude/skills/tbd/SKILL.md | 6 ++ docs/docs-overview.md | 28 ++++-- packages/tbd/docs/references/docmap-format.md | 64 +++++++++++++ packages/tbd/docs/references/docref-format.md | 71 ++++++++++++++ .../docs/shortcuts/system/skill-baseline.md | 6 ++ packages/tbd/docs/tbd-docs.md | 26 ++++++ packages/tbd/src/file/doc-fork.ts | 3 - packages/tbd/src/file/fork-manifest.ts | 5 +- packages/tbd/src/lib/paths.ts | 17 ++-- packages/tbd/tests/doc-fork.test.ts | 3 +- .../tbd/tests/fork-cross-platform-e2e.test.ts | 93 +++++++++++++++++++ skills/tbd/SKILL.md | 6 ++ 13 files changed, 311 insertions(+), 23 deletions(-) create mode 100644 packages/tbd/docs/references/docmap-format.md create mode 100644 packages/tbd/docs/references/docref-format.md create mode 100644 packages/tbd/tests/fork-cross-platform-e2e.test.ts diff --git a/.agents/skills/tbd/SKILL.md b/.agents/skills/tbd/SKILL.md index c793dad7..5bbf7e0d 100644 --- a/.agents/skills/tbd/SKILL.md +++ b/.agents/skills/tbd/SKILL.md @@ -93,6 +93,10 @@ or want help → run `tbd shortcut welcome-user` | **Documentation** | | | “Research this topic” | `tbd shortcut new-research-brief` | | “Document architecture” | `tbd shortcut new-architecture-doc` | +| “What guidelines/docs are there?” | `tbd docs list` | +| “Make the guidelines visible / customize doc X” | `tbd docs fork ` (or `--all`), then edit in `docs/tbd/` | +| “Update the guidelines to the latest” | `tbd docs update`; on conflicts ask the user, then `--merge` or `--keep-ours` | +| “I deleted a forked doc file” | `tbd docs status` shows it `missing`; restore with `tbd docs fork --force` or finalize with `tbd docs unfork ` | | **Cleanup & Maintenance** | | | “Clean up this code” / “Remove dead code” | `tbd shortcut code-cleanup-all` | | “Fix repository problems” | `tbd doctor --fix` | @@ -182,6 +186,8 @@ working branch. See `tbd guidelines tbd-sync-troubleshooting` for details. | `tbd guidelines ` | Load coding guidelines | | `tbd guidelines --list` | List guidelines | | `tbd template ` | Output a template | +| `tbd docs` / `tbd docs list` | Managed-docs overview / cross-kind list with state markers | +| `tbd docs fork/unfork/update ` | Fork docs into `docs/tbd/`, return to upstream, pull upstream updates | ## Quick Reference diff --git a/.claude/skills/tbd/SKILL.md b/.claude/skills/tbd/SKILL.md index c793dad7..5bbf7e0d 100644 --- a/.claude/skills/tbd/SKILL.md +++ b/.claude/skills/tbd/SKILL.md @@ -93,6 +93,10 @@ or want help → run `tbd shortcut welcome-user` | **Documentation** | | | “Research this topic” | `tbd shortcut new-research-brief` | | “Document architecture” | `tbd shortcut new-architecture-doc` | +| “What guidelines/docs are there?” | `tbd docs list` | +| “Make the guidelines visible / customize doc X” | `tbd docs fork ` (or `--all`), then edit in `docs/tbd/` | +| “Update the guidelines to the latest” | `tbd docs update`; on conflicts ask the user, then `--merge` or `--keep-ours` | +| “I deleted a forked doc file” | `tbd docs status` shows it `missing`; restore with `tbd docs fork --force` or finalize with `tbd docs unfork ` | | **Cleanup & Maintenance** | | | “Clean up this code” / “Remove dead code” | `tbd shortcut code-cleanup-all` | | “Fix repository problems” | `tbd doctor --fix` | @@ -182,6 +186,8 @@ working branch. See `tbd guidelines tbd-sync-troubleshooting` for details. | `tbd guidelines ` | Load coding guidelines | | `tbd guidelines --list` | List guidelines | | `tbd template ` | Output a template | +| `tbd docs` / `tbd docs list` | Managed-docs overview / cross-kind list with state markers | +| `tbd docs fork/unfork/update ` | Fork docs into `docs/tbd/`, return to upstream, pull upstream updates | ## Quick Reference diff --git a/docs/docs-overview.md b/docs/docs-overview.md index 720bccd5..f90d395c 100644 --- a/docs/docs-overview.md +++ b/docs/docs-overview.md @@ -50,17 +50,24 @@ Project-specific specifications, architecture, and research docs: ### tbd CLI Documentation Commands -In addition to these repository docs, tbd provides built-in documentation via CLI: +In addition to these repository docs, tbd provides managed documentation via the +`tbd docs` group and per-kind readers: -- `tbd shortcut --list` / `tbd shortcut `—Workflow shortcuts (new-plan-spec, - code-review-and-commit, review-code-typescript, etc.) -- `tbd guidelines --list` / `tbd guidelines `—Coding guidelines (typescript-rules, - python-rules, general-tdd-guidelines, etc.) -- `tbd template --list` / `tbd template `—Document templates (plan-spec, - research-brief, architecture) +- `tbd docs`—Status overview of managed docs; `tbd docs list` shows every doc across + kinds with `[forked]`/`[customized]`/`[local]` markers +- `tbd docs show `—Read any doc by name; `tbd docs show tbd-docs` is the CLI + manual (alias `tbd docs manual`) +- `tbd shortcut ` / `tbd guidelines ` / `tbd template `—Per-kind + readers (with `--list`) +- `tbd docs sync`—Refresh the gitignored `.tbd/docs/` cache (also run by setup) -These CLI-provided docs are installed locally in `.tbd/docs/` during `tbd setup --auto` -and can be refreshed anytime by re-running setup. +#### Forking docs into the repo + +`tbd docs fork ` (or `--all`) copies managed docs into a visible, git-tracked +`docs/tbd/` folder; tbd then serves your copies everywhere it served the upstream ones. +`tbd docs update` three-way merges upstream changes into forked copies after an upgrade; +`tbd docs status` shows each doc’s state. +See “Managing Docs” in `tbd docs show tbd-docs` for the full model. #### Adding external docs by URL @@ -76,7 +83,8 @@ GitHub blob URLs are automatically converted to raw URLs. If direct fetch returns HTTP 403, the system falls back to `gh api` for authenticated access. User-added shortcuts are stored in `shortcuts/custom/` to keep them separate from -bundled docs. +bundled docs. (A unified `tbd docs add ` form is planned; the per-kind flags +remain as aliases.) diff --git a/packages/tbd/docs/references/docref-format.md b/packages/tbd/docs/references/docref-format.md new file mode 100644 index 00000000..ad4b82d5 --- /dev/null +++ b/packages/tbd/docs/references/docref-format.md @@ -0,0 +1,71 @@ +--- +title: Docref Format +description: A single-string, URI-like address for any document — the one source-address grammar used across tbd +author: Joshua Levy (github.com/jlevy) with LLM assistance +category: general +--- +# Docref Format (v0.1) + +A **docref** is a single-string, URI-like address for a document. +It is the one address syntax used everywhere tbd names where a doc comes from or lives: +`docs_cache.files` values, the fork manifest’s `source` field, and future `tbd docs add` +arguments. The grammar is tool-agnostic: any application can adopt it, and the reference +implementation (`src/docref/` in tbd) is standalone and dependency-free. + +## Forms + +| Form | Example | Meaning | +| --- | --- | --- | +| internal | `internal:guidelines/python-rules.md` | A doc bundled inside the consuming tool. App-relative: each tool resolves it against its own bundled collection. | +| local | `./docs/general/`, `../shared/rules.md`, `/abs/f.md`, `C:/docs/f.md` | A filesystem path. Must be **anchored**: `./`, `../`, `/`, or a Windows drive letter. | +| url | `https://example.com/style.md` | A plain URL, kept verbatim. | +| git | `github:owner/repo@ref//path/to/file.md` | A file in a git host’s repo. `gitlab:` likewise. `@ref` is optional; `//` separates repo from path. | +| git + fragment | `github:o/r@main//f.md#naming` | Optional in-document anchor, preserved verbatim. | + +The `//` separator makes refs with slashes unambiguous: +`github:o/r@feature/x//docs/f.md` pins ref `feature/x` — unlike GitHub blob URLs, where +ref and path cannot be split reliably. + +## Strictness + +The grammar is deliberately strict; consumers may be lenient at their own boundary: + +- **Bare relative strings are not docrefs** (`guidelines/x.md` is invalid). + A consumer that wants to accept them may prepend `./` before parsing. + A strict grammar plus lenient consumers composes; the reverse can never be tightened. +- **Home-relative paths (`~/…`) are rejected** in v0.1 (no portable expansion + semantics). +- **Unknown schemes are rejected** (`mailto:…`, `git:…`). Additional protocols — for + example a host-bearing git scheme for forges beyond GitHub/GitLab — may be added in + future versions. + +## Normalization + +Web URLs that point at a known git host’s file view normalize to the canonical scheme, +so one file has one address: + +- `https://github.com/o/r/blob/main/f.md` → `github:o/r@main//f.md` +- `https://raw.githubusercontent.com/o/r/main/f.md` → `github:o/r@main//f.md` +- `https://gitlab.com/o/r/-/blob/main/f.md` → `gitlab:o/r@main//f.md` + +URL fragments are preserved through normalization — a normalizer must never silently +drop data. + +## Equality + +Two docrefs are equal when their canonical forms are identical, except that local paths +compare with a single leading `./` ignored. +Equality is purely syntactic: hosts and owners are not case-normalized, and no network +or filesystem is consulted. + +## Prior Art + +[purl](https://github.com/package-url/purl-spec) addresses *packages* +(`pkg:type/namespace/name@version`); its identity is the package, with file paths as an +awkward suffix. docref’s identity is the *document*, with in-repo paths and anchored +local files as first-class forms — which is why a separate small grammar exists rather +than a purl profile. + + diff --git a/packages/tbd/docs/shortcuts/system/skill-baseline.md b/packages/tbd/docs/shortcuts/system/skill-baseline.md index 5d51e7c1..608269b9 100644 --- a/packages/tbd/docs/shortcuts/system/skill-baseline.md +++ b/packages/tbd/docs/shortcuts/system/skill-baseline.md @@ -85,6 +85,10 @@ or want help → run `tbd shortcut welcome-user` | **Documentation** | | | “Research this topic” | `tbd shortcut new-research-brief` | | “Document architecture” | `tbd shortcut new-architecture-doc` | +| “What guidelines/docs are there?” | `tbd docs list` | +| “Make the guidelines visible / customize doc X” | `tbd docs fork ` (or `--all`), then edit in `docs/tbd/` | +| “Update the guidelines to the latest” | `tbd docs update`; on conflicts ask the user, then `--merge` or `--keep-ours` | +| “I deleted a forked doc file” | `tbd docs status` shows it `missing`; restore with `tbd docs fork --force` or finalize with `tbd docs unfork ` | | **Cleanup & Maintenance** | | | “Clean up this code” / “Remove dead code” | `tbd shortcut code-cleanup-all` | | “Fix repository problems” | `tbd doctor --fix` | @@ -174,6 +178,8 @@ working branch. See `tbd guidelines tbd-sync-troubleshooting` for details. | `tbd guidelines ` | Load coding guidelines | | `tbd guidelines --list` | List guidelines | | `tbd template ` | Output a template | +| `tbd docs` / `tbd docs list` | Managed-docs overview / cross-kind list with state markers | +| `tbd docs fork/unfork/update ` | Fork docs into `docs/tbd/`, return to upstream, pull upstream updates | ## Quick Reference diff --git a/packages/tbd/docs/tbd-docs.md b/packages/tbd/docs/tbd-docs.md index 7b453d52..b713feb3 100644 --- a/packages/tbd/docs/tbd-docs.md +++ b/packages/tbd/docs/tbd-docs.md @@ -726,6 +726,32 @@ On HTTP 403, fetching falls back to `gh api` for authenticated access. User-added shortcuts go to `shortcuts/custom/` (separate from bundled `shortcuts/standard/`). +### Managing Docs: Two Modes + +Every managed doc is served through one search path; where the file lives is a per-doc +choice between two modes that serve identical content: + +- **Hidden cache (the default).** Docs live in the gitignored `.tbd/docs/` cache — + always active, zero repo footprint, refreshed by `tbd docs sync` (and by setup). +- **Forked.** `tbd docs fork ` (or `--all`) copies a doc into `docs/tbd/`, tracked + in git: visible on GitHub, reviewable in PRs, and editable — your copy shadows the + cache everywhere the upstream one was served. + `tbd docs unfork` returns to the cache; `tbd docs update` three-way merges upstream + changes into your copy after an upgrade. + +Forking changes nothing about how docs work — it only makes them explicit and editable. +Four update surfaces stay deliberately separate: + +| Command | Scope | Touches | Modifies tracked files? | +| --- | --- | --- | --- | +| `tbd sync` | project data (issues/beads) | sync worktree + `tbd-sync` branch; also refreshes the doc cache and *reports* fork drift | never | +| `tbd setup --auto` | installation + integrations | skills, hooks, settings, `AGENTS.md`; invokes a docs-cache sync | only generated integration files | +| `tbd docs sync` | doc cache | gitignored `.tbd/docs/` only | never | +| `tbd docs update` | your forked docs | fork dir + bases + manifest (offline, against the cache) | **yes — the only doc command that does** | + +Disambiguation worth stating once: `tbd update ` is an issue operation, +`tbd docs update` a doc operation — the noun scope always disambiguates. + ### Forked Docs in Your Repo (docs/tbd/) `tbd docs fork` copies managed docs into `docs/tbd/`, laid out **by kind, flat within diff --git a/packages/tbd/src/file/doc-fork.ts b/packages/tbd/src/file/doc-fork.ts index 30f94632..93328446 100644 --- a/packages/tbd/src/file/doc-fork.ts +++ b/packages/tbd/src/file/doc-fork.ts @@ -41,9 +41,6 @@ import { writeBaseContent, } from './fork-manifest.js'; -/** Default fork directory, relative to the repo/tbd root. */ -export const DEFAULT_FORK_DIR = 'docs/tbd'; - /** Map a doc kind to its plural directory name within the fork dir. */ export const KIND_DIR: Record = { guideline: 'guidelines', diff --git a/packages/tbd/src/file/fork-manifest.ts b/packages/tbd/src/file/fork-manifest.ts index b31758b1..548bb04e 100644 --- a/packages/tbd/src/file/fork-manifest.ts +++ b/packages/tbd/src/file/fork-manifest.ts @@ -25,6 +25,7 @@ import { z } from 'zod'; import { stringifyYaml } from '../utils/yaml-utils.js'; import { withLockfile } from '../utils/lockfile.js'; import { resolveSharedTbdPaths } from '../lib/paths.js'; +import { isDocRef } from '../docref/index.js'; /** Directory (repo-relative under `.tbd/`) holding all fork state. */ export const DOC_FORKS_DIR = '.tbd/doc-forks'; @@ -62,8 +63,8 @@ export const ForkEntrySchema = z.object({ kind: z.enum(FORK_KINDS), /** Repo-relative path of the forked file (e.g. "docs/tbd/guidelines/python-rules.md"). */ path: z.string().min(1), - /** Provenance docref the fork was created from. */ - source: z.string().min(1), + /** Provenance docref the fork was created from (docref-everywhere rule). */ + source: z.string().min(1).refine(isDocRef, { message: 'source must be a valid docref' }), /** sha256: of the LF-normalized base content. */ base_hash: z.string().min(1), /** tbd version when the base was last set. */ diff --git a/packages/tbd/src/lib/paths.ts b/packages/tbd/src/lib/paths.ts index 075ca41b..51e4e890 100644 --- a/packages/tbd/src/lib/paths.ts +++ b/packages/tbd/src/lib/paths.ts @@ -358,14 +358,19 @@ export const BUILTIN_INSTALL_DIR = INSTALL_DIR; /** References directory name (tbd self-docs and format references). */ export const REFERENCES_DIR = 'references'; -/** Default fork directory (repo-relative), where forked docs are made visible. */ -export const FORK_DIR = join(DOCS_DIR, 'tbd'); // docs/tbd/ +/** + * Default fork directory (repo-relative), where forked docs are made visible. + * A POSIX literal, not join()'d: this value is committed (manifest paths) and + * printed, so it must be identical on every platform; fs access joins it with + * the root via join(), which accepts forward slashes on Windows. + */ +export const FORK_DIR = 'docs/tbd'; /** Fork-dir kind subdirectories (repo-relative). */ -export const FORK_SHORTCUTS_DIR = join(FORK_DIR, SHORTCUTS_DIR); // docs/tbd/shortcuts/ -export const FORK_GUIDELINES_DIR = join(FORK_DIR, GUIDELINES_DIR); // docs/tbd/guidelines/ -export const FORK_TEMPLATES_DIR = join(FORK_DIR, TEMPLATES_DIR); // docs/tbd/templates/ -export const FORK_REFERENCES_DIR = join(FORK_DIR, REFERENCES_DIR); // docs/tbd/references/ +export const FORK_SHORTCUTS_DIR = `${FORK_DIR}/${SHORTCUTS_DIR}`; // docs/tbd/shortcuts/ +export const FORK_GUIDELINES_DIR = `${FORK_DIR}/${GUIDELINES_DIR}`; // docs/tbd/guidelines/ +export const FORK_TEMPLATES_DIR = `${FORK_DIR}/${TEMPLATES_DIR}`; // docs/tbd/templates/ +export const FORK_REFERENCES_DIR = `${FORK_DIR}/${REFERENCES_DIR}`; // docs/tbd/references/ /** * Cache-only lookup paths (the gitignored `.tbd/docs/` cache), used when forking diff --git a/packages/tbd/tests/doc-fork.test.ts b/packages/tbd/tests/doc-fork.test.ts index 0d207dfd..441282b8 100644 --- a/packages/tbd/tests/doc-fork.test.ts +++ b/packages/tbd/tests/doc-fork.test.ts @@ -17,11 +17,10 @@ import { computeForkDriftSummary, regenerateForkDirReadme, ForkConflictError, - DEFAULT_FORK_DIR, } from '../src/file/doc-fork.js'; import { emptyManifest, findFork, readBaseContent } from '../src/file/fork-manifest.js'; -const FORK_DIR = DEFAULT_FORK_DIR; +import { FORK_DIR } from '../src/lib/paths.js'; const UPSTREAM = '# Python Rules\n\nUpstream content.\n'; describe('forkDoc', () => { diff --git a/packages/tbd/tests/fork-cross-platform-e2e.test.ts b/packages/tbd/tests/fork-cross-platform-e2e.test.ts new file mode 100644 index 00000000..3f82e058 --- /dev/null +++ b/packages/tbd/tests/fork-cross-platform-e2e.test.ts @@ -0,0 +1,93 @@ +/** + * Cross-platform e2e for the fork surface, run against the built CLI on every + * CI OS (unlike the tryscript goldens, which run only where a POSIX shell is + * available — tryscript executes blocks via the platform shell, cmd on + * Windows). Pins the Windows-sensitive behaviors: committed manifest paths are + * POSIX regardless of platform, and fork-dir shadowing serves the forked copy. + * See tbd-iqm1. + */ + +import { describe, it, expect, beforeEach, afterEach } from 'vitest'; +import { mkdtemp, rm, readFile, writeFile } from 'node:fs/promises'; +import { tmpdir } from 'node:os'; +import { join, dirname } from 'node:path'; +import { fileURLToPath } from 'node:url'; +import { execSync, spawnSync } from 'node:child_process'; + +const __dirname = dirname(fileURLToPath(import.meta.url)); + +describe('fork surface cross-platform e2e', { timeout: 120_000 }, () => { + let tempDir: string; + const tbdBin = join(__dirname, '..', 'dist', 'bin.mjs'); + + beforeEach(async () => { + tempDir = await mkdtemp(join(tmpdir(), 'tbd-fork-xplat-')); + execSync('git init --initial-branch=main', { cwd: tempDir }); + execSync('git config user.email "test@example.com"', { cwd: tempDir }); + execSync('git config user.name "Test"', { cwd: tempDir }); + execSync('git config commit.gpgsign false', { cwd: tempDir }); + runTbd(['init', '--prefix=fx']); + }); + + afterEach(async () => { + await rm(tempDir, { recursive: true, force: true }); + }); + + function runTbd(args: string[]): { stdout: string; stderr: string; status: number } { + const result = spawnSync('node', [tbdBin, ...args], { + cwd: tempDir, + encoding: 'utf-8', + env: { ...process.env, FORCE_COLOR: '0', NO_COLOR: '1' }, + timeout: 60000, + }); + return { + stdout: result.stdout || '', + stderr: result.stderr || '', + status: result.status ?? 1, + }; + } + + it('records POSIX manifest paths and serves the forked copy on every platform', async () => { + const fork = runTbd(['docs', 'fork', 'python-rules']); + expect(fork.status).toBe(0); + expect(fork.stdout).toContain('Forked python-rules → docs/tbd/guidelines/python-rules.md'); + + // The committed manifest must be platform-independent: forward slashes only. + const manifest = await readFile(join(tempDir, '.tbd', 'doc-forks', 'forks.yml'), 'utf-8'); + expect(manifest).toContain('path: docs/tbd/guidelines/python-rules.md'); + expect(manifest).not.toContain('\\'); + + // Customize the fork; the per-kind reader must serve the forked copy. + const forkedPath = join(tempDir, 'docs', 'tbd', 'guidelines', 'python-rules.md'); + const content = await readFile(forkedPath, 'utf-8'); + await writeFile(forkedPath, `${content}\nXPLAT-FORK-MARKER\n`); + + const served = runTbd(['guidelines', 'python-rules']); + expect(served.status).toBe(0); + expect(served.stdout).toContain('XPLAT-FORK-MARKER'); + + const list = runTbd(['docs', 'list', '--kind=guideline']); + expect(list.status).toBe(0); + expect(list.stdout).toMatch(/python-rules .*\[forked, customized\]/); + }); + + it('show serves forked copies with a POSIX provenance note; unfork restores upstream', async () => { + runTbd(['docs', 'fork', 'review-code']); + const forkedPath = join(tempDir, 'docs', 'tbd', 'shortcuts', 'review-code.md'); + const content = await readFile(forkedPath, 'utf-8'); + await writeFile(forkedPath, `${content}\nXPLAT-SHOW-MARKER\n`); + + const show = runTbd(['docs', 'show', 'review-code']); + expect(show.status).toBe(0); + expect(show.stdout).toContain('XPLAT-SHOW-MARKER'); + expect(show.stderr).toContain('(serving forked copy: docs/tbd/shortcuts/review-code.md)'); + + const unfork = runTbd(['docs', 'unfork', 'review-code', '--force']); + expect(unfork.status).toBe(0); + + const after = runTbd(['docs', 'show', 'review-code']); + expect(after.status).toBe(0); + expect(after.stdout).not.toContain('XPLAT-SHOW-MARKER'); + expect(after.stderr).not.toContain('serving forked copy'); + }); +}); diff --git a/skills/tbd/SKILL.md b/skills/tbd/SKILL.md index b6d8135d..1d647d52 100644 --- a/skills/tbd/SKILL.md +++ b/skills/tbd/SKILL.md @@ -99,6 +99,10 @@ or want help → run `tbd shortcut welcome-user` | **Documentation** | | | “Research this topic” | `tbd shortcut new-research-brief` | | “Document architecture” | `tbd shortcut new-architecture-doc` | +| “What guidelines/docs are there?” | `tbd docs list` | +| “Make the guidelines visible / customize doc X” | `tbd docs fork ` (or `--all`), then edit in `docs/tbd/` | +| “Update the guidelines to the latest” | `tbd docs update`; on conflicts ask the user, then `--merge` or `--keep-ours` | +| “I deleted a forked doc file” | `tbd docs status` shows it `missing`; restore with `tbd docs fork --force` or finalize with `tbd docs unfork ` | | **Cleanup & Maintenance** | | | “Clean up this code” / “Remove dead code” | `tbd shortcut code-cleanup-all` | | “Fix repository problems” | `tbd doctor --fix` | @@ -188,6 +192,8 @@ working branch. See `tbd guidelines tbd-sync-troubleshooting` for details. | `tbd guidelines ` | Load coding guidelines | | `tbd guidelines --list` | List guidelines | | `tbd template ` | Output a template | +| `tbd docs` / `tbd docs list` | Managed-docs overview / cross-kind list with state markers | +| `tbd docs fork/unfork/update ` | Fork docs into `docs/tbd/`, return to upstream, pull upstream updates | ## Quick Reference From a6f54f5f4b6662495d110629380e5be36212e4f1 Mon Sep 17 00:00:00 2001 From: Test User Date: Fri, 12 Jun 2026 21:14:42 +0000 Subject: [PATCH 29/36] feat: Reference kind, one-docmap rendering across all doc surfaces, status Docs line MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Phase 2/5 items from the forkable-docs spec: - reference kind is live (closes tbd-f233): references/ is scanned into the cache (copy-docs now bundles it), docref-format/docmap-format are served, and the self-docs tbd-docs + tbd-design are registered as reference docs under their existing names — served from the cache so forks shadow them, with a bundled fallback so they stay readable before init or first sync. fork/unfork/list/show/update all work on references; the doc-references extractor now validates `tbd docs show ` mentions and drops the stale `tbd reference` skip. - One data model, one renderer (tbd-wzqp): servedEntryFor() in lib/doc-serve.ts is the single point of docmap-entry construction; docs list, the bare overview (--json now emits the docmap itself), kind- agnostic show (--json emits entry + content), and the per-kind readers all build entries there. Per-kind --list --json switches from the flat array to the same docmap filtered to that kind (spec Decision 21); per-kind reads emit entry + score + content; per-kind text serving gains the '(serving forked copy: …)' stderr provenance note (Decision 18). cli-doc-output goldens rewritten per the spec's existing-goldens table. - tbd status gains a Docs drift line only when forks exist (tbd-i49m), computed in the async gather phase into StatusData so rendering stays pure; zero-fork output is byte-identical (orientation golden unchanged). https://claude.ai/code/session_01QPsCSYGtwR8JtX2R1aaxyh --- .tbd/config.yml | 10 +- packages/tbd/scripts/copy-docs.mjs | 6 ++ packages/tbd/src/cli/commands/docs-fork.ts | 43 ++------- packages/tbd/src/cli/commands/docs.ts | 92 ++++++++++++++---- packages/tbd/src/cli/commands/guidelines.ts | 22 ++--- packages/tbd/src/cli/commands/status.ts | 25 ++++- .../tbd/src/cli/lib/doc-command-handler.ts | 82 +++++++++------- packages/tbd/src/cli/lib/doc-serve.ts | 96 +++++++++++++++++++ packages/tbd/src/file/doc-sync.ts | 6 ++ packages/tbd/src/lib/paths.ts | 12 +++ .../tbd/tests/cli-doc-output.tryscript.md | 19 ++-- packages/tbd/tests/doc-references.test.ts | 12 +-- 12 files changed, 304 insertions(+), 121 deletions(-) create mode 100644 packages/tbd/src/cli/lib/doc-serve.ts diff --git a/.tbd/config.yml b/.tbd/config.yml index 1ced77c6..5d7033b0 100644 --- a/.tbd/config.yml +++ b/.tbd/config.yml @@ -23,7 +23,12 @@ settings: # Auto-sync: Docs are automatically synced when stale (default: every 24 hours). # Configure with settings.doc_auto_sync_hours (0 = disabled). docs_cache: + lookup_path: + - .tbd/docs/shortcuts/system + - .tbd/docs/shortcuts/standard files: + references/tbd-docs.md: internal:tbd-docs.md + references/tbd-design.md: internal:tbd-design.md shortcuts/system/shortcut-explanation.md: internal:shortcuts/system/shortcut-explanation.md shortcuts/system/skill-baseline.md: internal:shortcuts/system/skill-baseline.md shortcuts/system/skill-brief.md: internal:shortcuts/system/skill-brief.md @@ -89,6 +94,5 @@ docs_cache: templates/plan-spec.md: internal:templates/plan-spec.md templates/qa-playbook.md: internal:templates/qa-playbook.md templates/research-brief.md: internal:templates/research-brief.md - lookup_path: - - .tbd/docs/shortcuts/system - - .tbd/docs/shortcuts/standard + references/docmap-format.md: internal:references/docmap-format.md + references/docref-format.md: internal:references/docref-format.md diff --git a/packages/tbd/scripts/copy-docs.mjs b/packages/tbd/scripts/copy-docs.mjs index c3043a3c..20f6f10d 100644 --- a/packages/tbd/scripts/copy-docs.mjs +++ b/packages/tbd/scripts/copy-docs.mjs @@ -35,6 +35,7 @@ const SHORTCUTS_DIR = join(DOCS_DIR, 'shortcuts'); const SHORTCUTS_SYSTEM_DIR = join(SHORTCUTS_DIR, 'system'); const GUIDELINES_DIR = join(DOCS_DIR, 'guidelines'); const TEMPLATES_DIR = join(DOCS_DIR, 'templates'); +const REFERENCES_DIR = join(DOCS_DIR, 'references'); /** * Packaged documentation files (in packages/tbd/docs/). @@ -135,6 +136,11 @@ if (phase === 'prebuild') { await copyDir(TEMPLATES_DIR, join(distDocs, 'templates')); } + // Copy reference docs (docref/docmap formats) to dist/docs + if (existsSync(REFERENCES_DIR)) { + await copyDir(REFERENCES_DIR, join(distDocs, 'references')); + } + // Copy install directory to dist/docs (headers for composing skill files) await copyDir(INSTALL_DIR, join(distDocs, 'install')); diff --git a/packages/tbd/src/cli/commands/docs-fork.ts b/packages/tbd/src/cli/commands/docs-fork.ts index ca07b8aa..0bb236f7 100644 --- a/packages/tbd/src/cli/commands/docs-fork.ts +++ b/packages/tbd/src/cli/commands/docs-fork.ts @@ -21,9 +21,11 @@ import { readConfig } from '../../file/config.js'; import { DocCache } from '../../file/doc-cache.js'; import { CACHE_GUIDELINES_PATHS, + CACHE_REFERENCE_PATHS, CACHE_SHORTCUT_PATHS, CACHE_TEMPLATE_PATHS, DEFAULT_GUIDELINES_PATHS, + DEFAULT_REFERENCE_PATHS, DEFAULT_SHORTCUT_PATHS, DEFAULT_TEMPLATE_PATHS, FORK_DIR, @@ -52,13 +54,13 @@ import { listLocalForkFiles, regenerateForkDirReadme, ForkConflictError, - KIND_DIR, } from '../../file/doc-fork.js'; import { updateOne, diffContents, type UpdateStrategy } from '../../file/fork-update.js'; import { createDocMap, type DocMapEntry } from '../../docmap/index.js'; +import { servedEntryFor } from '../lib/doc-serve.js'; /** Kinds that can be resolved from the cache and forked today. */ -export const RESOLVABLE_KINDS: ForkKind[] = ['guideline', 'shortcut', 'template']; +export const RESOLVABLE_KINDS: ForkKind[] = ['guideline', 'shortcut', 'template', 'reference']; /** * Validate a user-supplied --kind value. Without this, an unknown kind silently @@ -72,13 +74,11 @@ export function parseKindOption(kind: string | undefined): ForkKind | undefined return kind as ForkKind; } -// 'reference' joins with Phase 5 (references/ cache dir); until then a manifest -// entry of that kind resolves as orphaned by design, and parseKindOption keeps -// the CLI from creating one. const KIND_CACHE_PATHS: Record = { guideline: CACHE_GUIDELINES_PATHS, shortcut: CACHE_SHORTCUT_PATHS, template: CACHE_TEMPLATE_PATHS, + reference: CACHE_REFERENCE_PATHS, }; interface ResolvedDoc { @@ -593,6 +593,7 @@ const KIND_SERVE_PATHS: Record = { guideline: DEFAULT_GUIDELINES_PATHS, shortcut: DEFAULT_SHORTCUT_PATHS, template: DEFAULT_TEMPLATE_PATHS, + reference: DEFAULT_REFERENCE_PATHS, }; interface ListOptions { @@ -628,18 +629,7 @@ class DocsListHandler extends BaseCommand { await cache.load({ quiet: true }); const rows: Row[] = []; for (const doc of cache.list()) { - const fork = findFork(manifest, doc.name, kind); - const isLocal = !fork && doc.sourceDir.startsWith(FORK_DIR); - let state = 'upstream'; - let marker = ''; - if (fork) { - const customized = hashContent(doc.content) !== fork.base_hash; - state = customized ? 'customized' : 'forked'; - marker = customized ? '[forked, customized]' : '[forked]'; - } else if (isLocal) { - state = 'local'; - marker = '[local]'; - } + const { entry, state, marker } = servedEntryFor(tbdRoot, kind, doc, manifest, files); rows.push({ name: doc.name, title: doc.frontmatter?.title, @@ -647,24 +637,9 @@ class DocsListHandler extends BaseCommand { sizeInfo: formatDocSize(doc.sizeBytes, doc.approxTokens), marker, state, - path: fork?.path ?? doc.sourceDir + '/' + doc.name + '.md', - }); - // Every docmap entry must carry a location (path and/or source): - // forked docs have both; local files have a path but no upstream; - // upstream docs are located by their provenance docref. - const localPath = `${FORK_DIR}/${KIND_DIR[kind]}/${doc.name}.md`; - docmapEntries.push({ - name: doc.name, - type: kind, - ...(fork - ? { path: fork.path, source: fork.source } - : isLocal - ? { path: localPath } - : { source: sourceDocRef(tbdRoot, files, doc.path) }), - title: doc.frontmatter?.title, - description: doc.frontmatter?.description, - state, + path: entry.path ?? doc.sourceDir + '/' + doc.name + '.md', }); + docmapEntries.push(entry); } grouped.push({ kind, rows }); } diff --git a/packages/tbd/src/cli/commands/docs.ts b/packages/tbd/src/cli/commands/docs.ts index f5f5b0f1..a50003a9 100644 --- a/packages/tbd/src/cli/commands/docs.ts +++ b/packages/tbd/src/cli/commands/docs.ts @@ -31,37 +31,50 @@ import { syncDocsWithDefaults } from '../../file/doc-sync.js'; import { DocCache } from '../../file/doc-cache.js'; import { DEFAULT_GUIDELINES_PATHS, + DEFAULT_REFERENCE_PATHS, DEFAULT_SHORTCUT_PATHS, DEFAULT_TEMPLATE_PATHS, FORK_DIR, } from '../../lib/paths.js'; import { readForkManifest, type ForkKind } from '../../file/fork-manifest.js'; import { computeForkDriftSummary } from '../../file/doc-fork.js'; +import { servedEntryFor, loadServeContext } from '../lib/doc-serve.js'; +import { createDocMap, type DocMapEntry } from '../../docmap/index.js'; import type { DocSection } from '../../lib/types.js'; import GithubSlugger from 'github-slugger'; -/** Reserved name that serves the bundled CLI manual (`tbd-docs.md`). */ +/** Reserved name that serves the CLI manual (`tbd-docs.md`). */ const MANUAL_DOC_NAME = 'tbd-docs'; +/** + * Self-docs: served as `reference` docs from the cache like everything else + * (so forks shadow them), but with a bundled fallback so they stay readable + * before init or before the first cache sync. + */ +const BUNDLED_ROOT_DOCS: Record = { + [MANUAL_DOC_NAME]: 'tbd-docs.md', + 'tbd-design': 'tbd-design.md', +}; + /** Serving lookup paths per kind (fork dir first, so forks shadow the cache). */ const SHOW_PATHS: Record = { guideline: DEFAULT_GUIDELINES_PATHS, shortcut: DEFAULT_SHORTCUT_PATHS, template: DEFAULT_TEMPLATE_PATHS, - reference: [], + reference: DEFAULT_REFERENCE_PATHS, }; /** - * Path to the bundled manual. The docs file is copied to dist/docs/ during - * build; in development it is read from the package docs/ directory. + * Read a doc bundled at the docs root. Copied to dist/docs/ during build; in + * development read from the package docs/ directory. */ -async function readManualContent(): Promise { +async function readBundledRootDoc(filename: string): Promise { const __dirname = dirname(fileURLToPath(import.meta.url)); try { - return await readFile(join(__dirname, 'docs', 'tbd-docs.md'), 'utf-8'); + return await readFile(join(__dirname, 'docs', filename), 'utf-8'); } catch { try { - return await readFile(join(__dirname, '..', '..', '..', 'docs', 'tbd-docs.md'), 'utf-8'); + return await readFile(join(__dirname, '..', '..', '..', 'docs', filename), 'utf-8'); } catch { throw new CLIError('Documentation file not found. Please rebuild the CLI.'); } @@ -140,15 +153,25 @@ class DocsOverviewHandler extends BaseCommand { const manifest = await readForkManifest(tbdRoot); const drift = await computeForkDriftSummary(tbdRoot, FORK_DIR, manifest); + const { files } = await loadServeContext(tbdRoot); let total = 0; + const entries: DocMapEntry[] = []; for (const kind of RESOLVABLE_KINDS) { const cache = new DocCache(SHOW_PATHS[kind], tbdRoot); await cache.load({ quiet: true }); - total += cache.list().length; + const docs = cache.list(); + total += docs.length; + if (this.ctx.json) { + for (const doc of docs) { + entries.push(servedEntryFor(tbdRoot, kind, doc, manifest, files).entry); + } + } } if (this.ctx.json) { - this.output.data({ available: total, ...drift }); + // The overview is the docmap rendered in summary form; --json emits + // the docmap itself (one data model, one renderer). + this.output.data(createDocMap(entries, { name: 'tbd-docs' })); return; } @@ -217,10 +240,22 @@ class DocsShowHandler extends BaseCommand { let content: string; let provenance: string | null = null; - if (name === MANUAL_DOC_NAME) { - content = await readManualContent(); + const bundledFallback = BUNDLED_ROOT_DOCS[name]; + let tbdRoot: string | null = null; + if (bundledFallback) { + try { + tbdRoot = await requireInit(); + } catch (err) { + if (!(err instanceof NotInitializedError)) throw err; + } + } else { + tbdRoot = await requireInit(); + } + + if (tbdRoot === null) { + // Self-docs stay readable before init. + content = await readBundledRootDoc(bundledFallback!); } else { - const tbdRoot = await requireInit(); const requestedKind = parseKindOption(options.kind); const kinds = requestedKind ? [requestedKind] : RESOLVABLE_KINDS; const matches: { kind: ForkKind; content: string; sourceDir: string; path: string }[] = []; @@ -238,18 +273,35 @@ class DocsShowHandler extends BaseCommand { } } if (matches.length === 0) { - throw new NotFoundError('Doc', `"${name}" (run \`tbd docs list\` to see names)`); - } - if (matches.length > 1) { + if (bundledFallback) { + // Initialized but cache not yet synced with the self-docs. + content = await readBundledRootDoc(bundledFallback); + } else { + throw new NotFoundError('Doc', `"${name}" (run \`tbd docs list\` to see names)`); + } + } else if (matches.length > 1) { const kindList = matches.map((m) => m.kind).join(', '); throw new CLIError( `"${name}" exists in multiple kinds (${kindList}). Use --kind to disambiguate.`, ); - } - const match = matches[0]!; - content = match.content; - if (match.sourceDir.startsWith(FORK_DIR)) { - provenance = relative(tbdRoot, match.path).split('\\').join('/'); + } else { + const match = matches[0]!; + content = match.content; + if (this.ctx.json && !options.sections) { + const { manifest, files } = await loadServeContext(tbdRoot); + const { entry } = servedEntryFor( + tbdRoot, + match.kind, + { name, content, sourceDir: match.sourceDir, path: match.path }, + manifest, + files, + ); + this.output.data({ ...entry, content }); + return; + } + if (match.sourceDir.startsWith(FORK_DIR)) { + provenance = relative(tbdRoot, match.path).split('\\').join('/'); + } } } diff --git a/packages/tbd/src/cli/commands/guidelines.ts b/packages/tbd/src/cli/commands/guidelines.ts index 22ba26e5..6df6556e 100644 --- a/packages/tbd/src/cli/commands/guidelines.ts +++ b/packages/tbd/src/cli/commands/guidelines.ts @@ -8,6 +8,7 @@ import { Command } from 'commander'; import pc from 'picocolors'; +import { createDocMap } from '../../docmap/index.js'; import { DocCommandHandler, type DocCommandOptions } from '../lib/doc-command-handler.js'; import { CLIError } from '../lib/errors.js'; import { DEFAULT_GUIDELINES_PATHS } from '../../lib/paths.js'; @@ -118,19 +119,14 @@ class GuidelinesHandler extends DocCommandHandler { } if (this.ctx.json) { - this.output.data( - docs.map((d) => ({ - name: d.name, - title: d.frontmatter?.title, - description: d.frontmatter?.description, - category: inferGuidelineCategory(d.name), - path: d.path, - sourceDir: d.sourceDir, - sizeBytes: d.sizeBytes, - approxTokens: d.approxTokens, - shadowed: this.cache!.isShadowed(d), - })), - ); + // Same docmap as `tbd docs list`, filtered to guidelines, with the + // category as a per-entry extension field. + const entries = await this.docMapEntries(docs); + const withCategory = entries.map((e) => ({ + ...e, + category: inferGuidelineCategory(e.name), + })); + this.output.data(createDocMap(withCategory, { name: 'tbd-docs' })); return; } diff --git a/packages/tbd/src/cli/commands/status.ts b/packages/tbd/src/cli/commands/status.ts index fe87629f..05b8ed02 100644 --- a/packages/tbd/src/cli/commands/status.ts +++ b/packages/tbd/src/cli/commands/status.ts @@ -27,7 +27,9 @@ import { type IntegrationCheck, } from '../lib/sections.js'; import { readConfig, findTbdRoot } from '../../file/config.js'; -import { resolveSharedTbdPaths } from '../../lib/paths.js'; +import { resolveSharedTbdPaths, FORK_DIR } from '../../lib/paths.js'; +import { readForkManifest } from '../../file/fork-manifest.js'; +import { computeForkDriftSummary, type ForkDriftSummary } from '../../file/doc-fork.js'; import { getClaudePaths, getAgentsMdPath, @@ -72,6 +74,7 @@ interface StatusData { worktree_healthy: boolean | null; worktree_status: WorktreeStatus | null; workspaces: string[]; + docs_drift: ForkDriftSummary | null; // Integrations integrations: { @@ -117,6 +120,7 @@ class StatusHandler extends BaseCommand { worktree_healthy: null, worktree_status: null, workspaces: [], + docs_drift: null, integrations: { portable_skill: false, portable_skill_path: AGENTS_SKILL_DISPLAY, @@ -156,6 +160,12 @@ class StatusHandler extends BaseCommand { if (statusData.initialized && tbdRoot) { // Load config and issue info await this.loadPostInitInfo(tbdRoot, statusData); + try { + const manifest = await readForkManifest(tbdRoot); + statusData.docs_drift = await computeForkDriftSummary(tbdRoot, FORK_DIR, manifest); + } catch { + // Docs awareness is best-effort; never fail status over it. + } } this.output.data(statusData, () => { @@ -329,6 +339,7 @@ class StatusHandler extends BaseCommand { ); // INTEGRATIONS section (shared with doctor) + // (docs drift summary is appended after Worktree; see renderDocsLine) const integrationChecks: IntegrationCheck[] = [ { name: 'Portable Agent Skill', @@ -363,6 +374,18 @@ class StatusHandler extends BaseCommand { renderWorktreeStatus(data.worktree_path, data.worktree_status, colors); } + // Docs line — only when forks exist, so zero-fork output stays + // byte-identical to the pre-f05 status (the orientation golden pins it). + if (data.docs_drift && data.docs_drift.forks > 0) { + const d = data.docs_drift; + const parts: string[] = [`${d.customized} customized`]; + if (d.stale > 0) parts.push(`${d.stale} with upstream updates — run 'tbd docs update'`); + if (d.conflicted > 0) parts.push(`${d.conflicted} conflict pending`); + if (d.missing > 0) parts.push(`${d.missing} missing — see 'tbd docs status'`); + console.log(''); + console.log(`${colors.bold('Docs:')} ${d.forks} forked (${parts.join(', ')})`); + } + // Workspaces (only show if there are any) if (data.workspaces.length > 0) { console.log(''); diff --git a/packages/tbd/src/cli/lib/doc-command-handler.ts b/packages/tbd/src/cli/lib/doc-command-handler.ts index 6121201d..cd54d63a 100644 --- a/packages/tbd/src/cli/lib/doc-command-handler.ts +++ b/packages/tbd/src/cli/lib/doc-command-handler.ts @@ -12,8 +12,12 @@ import { BaseCommand } from './base-command.js'; import { shouldUseInteractiveOutput } from './context.js'; import { GUIDELINES_AGENT_HEADER } from './doc-prompts.js'; import { requireInit } from './errors.js'; -import { DocCache, SCORE_PREFIX_MATCH } from '../../file/doc-cache.js'; +import { DocCache, SCORE_PREFIX_MATCH, type CachedDoc } from '../../file/doc-cache.js'; import { addDoc, type DocType } from '../../file/doc-add.js'; +import { servedEntryFor, loadServeContext } from './doc-serve.js'; +import { createDocMap, type DocMapEntry } from '../../docmap/index.js'; +import { FORK_DIR } from '../../lib/paths.js'; +import { relative } from 'node:path'; import { truncate } from '../../lib/truncate.js'; import { formatDocSize } from '../../lib/format-utils.js'; import { getTerminalWidth, renderMarkdownWithFrontmatter, paginateOutput } from './output.js'; @@ -86,18 +90,9 @@ export abstract class DocCommandHandler extends BaseCommand { const docs = this.cache.list(includeAll); if (this.ctx.json) { - this.output.data( - docs.map((d) => ({ - name: d.name, - title: d.frontmatter?.title, - description: d.frontmatter?.description, - path: d.path, - sourceDir: d.sourceDir, - sizeBytes: d.sizeBytes, - approxTokens: d.approxTokens, - shadowed: this.cache!.isShadowed(d), - })), - ); + // One data model: the per-kind list emits the same docmap object as + // `tbd docs list`, filtered to this kind (spec Decision 21). + this.output.data(createDocMap(await this.docMapEntries(docs), { name: 'tbd-docs' })); return; } @@ -135,6 +130,45 @@ export abstract class DocCommandHandler extends BaseCommand { } } + /** + * Build docmap entries for this kind's docs through the shared constructor, + * with per-kind extensions (size metrics; shadowed flag when applicable). + */ + protected async docMapEntries(docs: CachedDoc[]): Promise { + const { manifest, files } = await loadServeContext(this.tbdRoot); + return docs.map((d) => { + const { entry } = servedEntryFor(this.tbdRoot, this.config.docType, d, manifest, files); + return { + ...entry, + sizeBytes: d.sizeBytes, + approxTokens: d.approxTokens, + ...(this.cache!.isShadowed(d) ? { shadowed: true } : {}), + }; + }); + } + + /** + * Emit one doc: docmap entry + content in JSON mode (the one-entry read + * shape, spec Decision 22); in text mode, a forked-copy provenance note on + * stderr (Decision 18) before the content. + */ + protected async emitDoc(doc: CachedDoc, score?: number): Promise { + if (this.ctx.json) { + const [entry] = await this.docMapEntries([doc]); + this.output.data({ + ...entry, + ...(score !== undefined ? { score } : {}), + content: doc.content, + }); + return; + } + if (doc.sourceDir.startsWith(FORK_DIR) && !this.ctx.quiet) { + const rel = relative(this.tbdRoot, doc.path).split('\\').join('/'); + process.stderr.write(`(serving forked copy: ${rel})\n`); + } + await this.outputDocContent(doc.content); + } + /** * Handle no query: show explanation + help. */ @@ -186,16 +220,7 @@ export abstract class DocCommandHandler extends BaseCommand { // Try exact match first const exactMatch = this.cache.get(query); if (exactMatch) { - if (this.ctx.json) { - this.output.data({ - name: exactMatch.doc.name, - title: exactMatch.doc.frontmatter?.title, - score: exactMatch.score, - content: exactMatch.doc.content, - }); - } else { - await this.outputDocContent(exactMatch.doc.content); - } + await this.emitDoc(exactMatch.doc, exactMatch.score); return; } @@ -222,16 +247,7 @@ export abstract class DocCommandHandler extends BaseCommand { } // Good fuzzy match - output it - if (this.ctx.json) { - this.output.data({ - name: best.doc.name, - title: best.doc.frontmatter?.title, - score: best.score, - content: best.doc.content, - }); - } else { - await this.outputDocContent(best.doc.content); - } + await this.emitDoc(best.doc, best.score); } /** diff --git a/packages/tbd/src/cli/lib/doc-serve.ts b/packages/tbd/src/cli/lib/doc-serve.ts new file mode 100644 index 00000000..25b1d4d1 --- /dev/null +++ b/packages/tbd/src/cli/lib/doc-serve.ts @@ -0,0 +1,96 @@ +/** + * Shared construction of docmap entries for served docs — the one-model, + * one-renderer contract (spec Decisions 21/22): `tbd docs list`/`show`, the + * bare overview, and the per-kind readers all build entries here, so their + * JSON output cannot drift. + */ + +import { readConfig } from '../../file/config.js'; +import { + type ForkKind, + findFork, + hashContent, + readForkManifest, +} from '../../file/fork-manifest.js'; +import { KIND_DIR } from '../../file/doc-fork.js'; +import { FORK_DIR, TBD_DOCS_DIR } from '../../lib/paths.js'; +import { join, relative, sep } from 'node:path'; +import type { DocMapEntry } from '../../docmap/index.js'; + +/** A served doc's docmap entry plus its derived fork-state presentation. */ +export interface ServedEntryInfo { + entry: DocMapEntry; + state: string; + marker: string; +} + +/** Minimal doc fields needed to derive a docmap entry. */ +export interface ServedDocLike { + name: string; + content: string; + sourceDir: string; + path: string; + frontmatter?: { title?: string; description?: string }; +} + +/** + * Build the docmap entry (+ state marker) for one served doc. Single point of + * construction for every docs inventory and read surface (Decision 21/22). + * Every entry carries a location: forked docs have path+source, local files a + * path, upstream docs their provenance docref. + */ +export function servedEntryFor( + tbdRoot: string, + kind: ForkKind, + doc: ServedDocLike, + manifest: Awaited>, + files: Record | undefined, +): ServedEntryInfo { + const fork = findFork(manifest, doc.name, kind); + const isLocal = !fork && doc.sourceDir.startsWith(FORK_DIR); + let state = 'upstream'; + let marker = ''; + if (fork) { + const customized = hashContent(doc.content) !== fork.base_hash; + state = customized ? 'customized' : 'forked'; + marker = customized ? '[forked, customized]' : '[forked]'; + } else if (isLocal) { + state = 'local'; + marker = '[local]'; + } + const localPath = `${FORK_DIR}/${KIND_DIR[kind]}/${doc.name}.md`; + const entry: DocMapEntry = { + name: doc.name, + type: kind, + ...(fork + ? { path: fork.path, source: fork.source } + : isLocal + ? { path: localPath } + : { source: sourceDocRefFor(tbdRoot, files, doc.path) }), + title: doc.frontmatter?.title, + description: doc.frontmatter?.description, + state, + }; + return { entry, state, marker }; +} + +/** Load the shared context servedEntryFor needs (manifest + config file map). */ +export async function loadServeContext(tbdRoot: string): Promise<{ + manifest: Awaited>; + files: Record | undefined; +}> { + const manifest = await readForkManifest(tbdRoot); + const config = await readConfig(tbdRoot); + return { manifest, files: config.docs_cache?.files }; +} + +/** Derive the provenance docref for a cached doc from config, defaulting to internal:. */ +function sourceDocRefFor( + tbdRoot: string, + files: Record | undefined, + docPath: string, +): string { + const cacheRoot = join(tbdRoot, TBD_DOCS_DIR); + const rel = relative(cacheRoot, docPath).split(sep).join('/'); + return files?.[rel] ?? `internal:${rel}`; +} diff --git a/packages/tbd/src/file/doc-sync.ts b/packages/tbd/src/file/doc-sync.ts index 0236d50d..5f07a214 100644 --- a/packages/tbd/src/file/doc-sync.ts +++ b/packages/tbd/src/file/doc-sync.ts @@ -338,8 +338,14 @@ export async function generateDefaultDocCacheConfig(): Promise (not --list or flags) - // Name can be: simple (foo-bar) or prefixed (prefix:foo-bar) + // Match per-kind readers (tbd shortcut|guidelines|template ) and the + // kind-agnostic reader (tbd docs show ). Names can be simple (foo-bar) + // or prefixed (prefix:foo-bar). const pattern = - /tbd (shortcut|guidelines|template|reference) ([a-z][a-z0-9-]*(?::[a-z][a-z0-9-]*)?)/g; + /tbd (shortcut|guidelines|template|docs show) ([a-z][a-z0-9-]*(?::[a-z][a-z0-9-]*)?)/g; const commands: string[] = []; let match; @@ -71,11 +72,6 @@ function extractDocCommands(content: string): string[] { if (name.includes(':')) { continue; } - // Skip reference commands - command not yet implemented (planned for f04) - // TODO: Remove this check once tbd reference command is implemented - if (cmd === 'reference') { - continue; - } commands.push(`tbd ${cmd} ${name}`); } } From c83d2fc037ea0c829b522de09b2695887bd27fd6 Mon Sep 17 00:00:00 2001 From: Test User Date: Fri, 12 Jun 2026 21:23:50 +0000 Subject: [PATCH 30/36] docs: declare a category in every bundled guideline's frontmatter Each guideline now carries exactly one category (general, typescript, python, convex, electron) in its YAML frontmatter, per the forkable-docs spec (Phase 4 item 15, tbd-jme1): categories live on the docs themselves, not in a central map. References already declared category: general and are unchanged. Adds tests/doc-categories.test.ts, which walks the source-level packages/tbd/docs/guidelines/*.md, parses frontmatter with gray-matter, and asserts every guideline declares exactly one category from the allowed set. The CLI-side category constant lands separately. https://claude.ai/code/session_01QPsCSYGtwR8JtX2R1aaxyh --- .../backward-compatibility-rules.md | 1 + .../docs/guidelines/bun-monorepo-patterns.md | 1 + .../guidelines/cli-agent-skill-patterns.md | 1 + .../tbd/docs/guidelines/commit-conventions.md | 1 + .../docs/guidelines/common-doc-guidelines.md | 1 + .../convex-limits-best-practices.md | 1 + packages/tbd/docs/guidelines/convex-rules.md | 1 + .../electron-app-development-patterns.md | 1 + .../docs/guidelines/error-handling-rules.md | 1 + .../docs/guidelines/general-coding-rules.md | 1 + .../docs/guidelines/general-comment-rules.md | 1 + .../general-eng-agent-principles.md | 1 + .../docs/guidelines/general-tdd-guidelines.md | 1 + .../docs/guidelines/general-testing-rules.md | 1 + .../guidelines/golden-testing-guidelines.md | 1 + .../docs/guidelines/pnpm-monorepo-patterns.md | 1 + .../docs/guidelines/python-cli-patterns.md | 1 + .../guidelines/python-modern-guidelines.md | 1 + packages/tbd/docs/guidelines/python-rules.md | 1 + .../guidelines/release-notes-guidelines.md | 1 + .../docs/guidelines/supply-chain-hardening.md | 5 +- .../guidelines/tbd-sync-troubleshooting.md | 1 + .../guidelines/typescript-cli-tool-rules.md | 1 + .../guidelines/typescript-code-coverage.md | 1 + .../tbd/docs/guidelines/typescript-rules.md | 1 + .../guidelines/typescript-sorting-patterns.md | 1 + .../typescript-yaml-handling-rules.md | 1 + packages/tbd/tests/doc-categories.test.ts | 76 +++++++++++++++++++ 28 files changed, 105 insertions(+), 2 deletions(-) create mode 100644 packages/tbd/tests/doc-categories.test.ts diff --git a/packages/tbd/docs/guidelines/backward-compatibility-rules.md b/packages/tbd/docs/guidelines/backward-compatibility-rules.md index cf4be0a4..cae8f1da 100644 --- a/packages/tbd/docs/guidelines/backward-compatibility-rules.md +++ b/packages/tbd/docs/guidelines/backward-compatibility-rules.md @@ -2,6 +2,7 @@ title: Backward Compatibility Rules description: Guidelines for maintaining backward compatibility across code, APIs, file formats, and database schemas author: Joshua Levy (github.com/jlevy) with LLM assistance +category: general --- ## Backward Compatibility Guidelines diff --git a/packages/tbd/docs/guidelines/bun-monorepo-patterns.md b/packages/tbd/docs/guidelines/bun-monorepo-patterns.md index d7b80e97..1213ae91 100644 --- a/packages/tbd/docs/guidelines/bun-monorepo-patterns.md +++ b/packages/tbd/docs/guidelines/bun-monorepo-patterns.md @@ -2,6 +2,7 @@ title: Bun Monorepo Patterns description: Modern patterns for Bun-based TypeScript monorepo architecture author: Joshua Levy (github.com/jlevy) with LLM assistance +category: typescript --- # Bun Monorepo Patterns diff --git a/packages/tbd/docs/guidelines/cli-agent-skill-patterns.md b/packages/tbd/docs/guidelines/cli-agent-skill-patterns.md index 3ba21c60..00ca7201 100644 --- a/packages/tbd/docs/guidelines/cli-agent-skill-patterns.md +++ b/packages/tbd/docs/guidelines/cli-agent-skill-patterns.md @@ -2,6 +2,7 @@ title: Agent Skills and CLI Integration Patterns description: How to write skills and agent-integrated CLIs that work across Claude Code, Codex, and the broader coding-agent ecosystem—a simple baseline plus references for advanced, multi-subcommand tools author: Joshua Levy (github.com/jlevy) with LLM assistance +category: general --- # Agent Skills and CLI Integration Patterns diff --git a/packages/tbd/docs/guidelines/commit-conventions.md b/packages/tbd/docs/guidelines/commit-conventions.md index 0af7e892..a6422cd6 100644 --- a/packages/tbd/docs/guidelines/commit-conventions.md +++ b/packages/tbd/docs/guidelines/commit-conventions.md @@ -2,6 +2,7 @@ title: Commit Conventions description: Conventional Commits format with extensions for agentic workflows author: Joshua Levy (github.com/jlevy) with LLM assistance +category: general --- # Commit Conventions diff --git a/packages/tbd/docs/guidelines/common-doc-guidelines.md b/packages/tbd/docs/guidelines/common-doc-guidelines.md index c32f7735..bd212700 100644 --- a/packages/tbd/docs/guidelines/common-doc-guidelines.md +++ b/packages/tbd/docs/guidelines/common-doc-guidelines.md @@ -2,6 +2,7 @@ title: Common Documentation Guidelines description: Common cross-project standards for writing and organizing docs, code comments, and text files—how to organize, structure, write, and format documents, plus the guideline footer convention. Downstream of github.com/jlevy/practical-prose. Use whenever writing or editing any documentation, README, guideline, or design doc. author: Joshua Levy (github.com/jlevy) with LLM assistance +category: general --- # Common Documentation Guidelines diff --git a/packages/tbd/docs/guidelines/convex-limits-best-practices.md b/packages/tbd/docs/guidelines/convex-limits-best-practices.md index f7635666..aa5cdf7d 100644 --- a/packages/tbd/docs/guidelines/convex-limits-best-practices.md +++ b/packages/tbd/docs/guidelines/convex-limits-best-practices.md @@ -2,6 +2,7 @@ title: Convex Limits and Best Practices description: Comprehensive reference for Convex platform limits, workarounds, and performance best practices author: Joshua Levy (github.com/jlevy) with LLM assistance +category: convex --- # Research Brief: Convex Database Limits, Best Practices, and Workarounds diff --git a/packages/tbd/docs/guidelines/convex-rules.md b/packages/tbd/docs/guidelines/convex-rules.md index 03d332bb..00b99593 100644 --- a/packages/tbd/docs/guidelines/convex-rules.md +++ b/packages/tbd/docs/guidelines/convex-rules.md @@ -2,6 +2,7 @@ title: Convex Rules description: Guidelines and best practices for building Convex projects, including database schema design, queries, mutations, and real-world examples author: Convex team +category: convex --- # Convex Guidelines diff --git a/packages/tbd/docs/guidelines/electron-app-development-patterns.md b/packages/tbd/docs/guidelines/electron-app-development-patterns.md index 600beb3f..685d3498 100644 --- a/packages/tbd/docs/guidelines/electron-app-development-patterns.md +++ b/packages/tbd/docs/guidelines/electron-app-development-patterns.md @@ -1,6 +1,7 @@ --- title: Electron App Development Patterns description: Guidelines for Electron development ecosystems including npm, pnpm, and Bun, with security baselines and framework comparisons +category: electron --- # Electron App Development Patterns diff --git a/packages/tbd/docs/guidelines/error-handling-rules.md b/packages/tbd/docs/guidelines/error-handling-rules.md index 4d29df0c..fc4f4183 100644 --- a/packages/tbd/docs/guidelines/error-handling-rules.md +++ b/packages/tbd/docs/guidelines/error-handling-rules.md @@ -2,6 +2,7 @@ title: Error Handling Rules description: Rules for handling errors, failures, and exceptional conditions author: Joshua Levy (github.com/jlevy) with LLM assistance +category: general --- # Error Handling Rules diff --git a/packages/tbd/docs/guidelines/general-coding-rules.md b/packages/tbd/docs/guidelines/general-coding-rules.md index e10707f5..b48d3c2b 100644 --- a/packages/tbd/docs/guidelines/general-coding-rules.md +++ b/packages/tbd/docs/guidelines/general-coding-rules.md @@ -2,6 +2,7 @@ title: General Coding Rules description: Rules for constants, magic numbers, and general coding practices author: Joshua Levy (github.com/jlevy) with LLM assistance +category: general --- # General Coding Rules diff --git a/packages/tbd/docs/guidelines/general-comment-rules.md b/packages/tbd/docs/guidelines/general-comment-rules.md index def30012..8f393aac 100644 --- a/packages/tbd/docs/guidelines/general-comment-rules.md +++ b/packages/tbd/docs/guidelines/general-comment-rules.md @@ -2,6 +2,7 @@ title: General Comment Rules description: Language-agnostic rules for writing clean, maintainable comments author: Joshua Levy (github.com/jlevy) with LLM assistance +category: general --- # General Comment Rules diff --git a/packages/tbd/docs/guidelines/general-eng-agent-principles.md b/packages/tbd/docs/guidelines/general-eng-agent-principles.md index 21281415..743c6f8b 100644 --- a/packages/tbd/docs/guidelines/general-eng-agent-principles.md +++ b/packages/tbd/docs/guidelines/general-eng-agent-principles.md @@ -2,6 +2,7 @@ title: Engineering Agent Principles description: Core principles for AI agents acting as senior engineers—objectivity and communication conduct plus the engineering process (detailed understanding, verification, end-to-end ownership, scope discipline, tracking future work, and acting versus seeking clarification) author: Joshua Levy (github.com/jlevy) with LLM assistance +category: general --- # Engineering Agent Principles diff --git a/packages/tbd/docs/guidelines/general-tdd-guidelines.md b/packages/tbd/docs/guidelines/general-tdd-guidelines.md index 5db475d3..f9a1c585 100644 --- a/packages/tbd/docs/guidelines/general-tdd-guidelines.md +++ b/packages/tbd/docs/guidelines/general-tdd-guidelines.md @@ -2,6 +2,7 @@ title: TDD Guidelines description: Test-Driven Development methodology and best practices author: Joshua Levy (github.com/jlevy) with LLM assistance +category: general --- # Test-Driven Development (TDD) Guidelines diff --git a/packages/tbd/docs/guidelines/general-testing-rules.md b/packages/tbd/docs/guidelines/general-testing-rules.md index f1372bef..bad21e4c 100644 --- a/packages/tbd/docs/guidelines/general-testing-rules.md +++ b/packages/tbd/docs/guidelines/general-testing-rules.md @@ -2,6 +2,7 @@ title: General Testing Rules description: Rules for writing minimal, effective tests with maximum coverage author: Joshua Levy (github.com/jlevy) with LLM assistance +category: general --- ## General Testing Rules diff --git a/packages/tbd/docs/guidelines/golden-testing-guidelines.md b/packages/tbd/docs/guidelines/golden-testing-guidelines.md index c79b1075..4159d1c9 100644 --- a/packages/tbd/docs/guidelines/golden-testing-guidelines.md +++ b/packages/tbd/docs/guidelines/golden-testing-guidelines.md @@ -2,6 +2,7 @@ title: Golden Testing Guidelines description: Guidelines for implementing golden/snapshot testing for complex systems author: Joshua Levy (github.com/jlevy) with LLM assistance +category: general --- # Golden Testing Guidelines diff --git a/packages/tbd/docs/guidelines/pnpm-monorepo-patterns.md b/packages/tbd/docs/guidelines/pnpm-monorepo-patterns.md index 8da94df7..ae3456d1 100644 --- a/packages/tbd/docs/guidelines/pnpm-monorepo-patterns.md +++ b/packages/tbd/docs/guidelines/pnpm-monorepo-patterns.md @@ -2,6 +2,7 @@ title: pnpm Monorepo Patterns description: Modern patterns for pnpm-based TypeScript monorepo architecture author: Joshua Levy (github.com/jlevy) with LLM assistance +category: typescript --- # pnpm Monorepo Patterns diff --git a/packages/tbd/docs/guidelines/python-cli-patterns.md b/packages/tbd/docs/guidelines/python-cli-patterns.md index d53c4dd8..974fef60 100644 --- a/packages/tbd/docs/guidelines/python-cli-patterns.md +++ b/packages/tbd/docs/guidelines/python-cli-patterns.md @@ -2,6 +2,7 @@ title: Python CLI Patterns description: Modern patterns for Python CLI application architecture author: Joshua Levy (github.com/jlevy) with LLM assistance +category: python --- # Python CLI Patterns diff --git a/packages/tbd/docs/guidelines/python-modern-guidelines.md b/packages/tbd/docs/guidelines/python-modern-guidelines.md index 93700ab1..c46df4db 100644 --- a/packages/tbd/docs/guidelines/python-modern-guidelines.md +++ b/packages/tbd/docs/guidelines/python-modern-guidelines.md @@ -2,6 +2,7 @@ title: Python Modern Guidelines description: Guidelines for modern Python projects using uv, with a few more opinionated practices author: Joshua Levy (github.com/jlevy) with LLM assistance +category: python --- # Python Modern Guidelines diff --git a/packages/tbd/docs/guidelines/python-rules.md b/packages/tbd/docs/guidelines/python-rules.md index db71f67e..a271601c 100644 --- a/packages/tbd/docs/guidelines/python-rules.md +++ b/packages/tbd/docs/guidelines/python-rules.md @@ -2,6 +2,7 @@ title: Python Rules description: General Python coding rules and best practices author: Joshua Levy (github.com/jlevy) with LLM assistance +category: python --- # Python Rules diff --git a/packages/tbd/docs/guidelines/release-notes-guidelines.md b/packages/tbd/docs/guidelines/release-notes-guidelines.md index db8bb52a..79ebb9ed 100644 --- a/packages/tbd/docs/guidelines/release-notes-guidelines.md +++ b/packages/tbd/docs/guidelines/release-notes-guidelines.md @@ -1,6 +1,7 @@ --- title: Release Notes Guidelines description: Guidelines for writing clear, accurate release notes +category: general --- # Release Notes Guidelines diff --git a/packages/tbd/docs/guidelines/supply-chain-hardening.md b/packages/tbd/docs/guidelines/supply-chain-hardening.md index f1bdb1f2..70212b48 100644 --- a/packages/tbd/docs/guidelines/supply-chain-hardening.md +++ b/packages/tbd/docs/guidelines/supply-chain-hardening.md @@ -2,6 +2,7 @@ title: Supply-Chain Hardening description: Strongly recommended for EVERY repo—apply it if a repo has not been hardened yet. Cross-ecosystem policy for installing dependencies safely (the 14-day cool-off, disabled install scripts, lockfile discipline, untrusted-repo handling). Use whenever a user mentions hardening, security, supply chain, or setting up a new repo; before adding/upgrading dependencies; when auditing for compromised packages; or when reviewing install/build/run commands across npm/pnpm, PyPI, Cargo, or Go. author: Joshua Levy (github.com/jlevy) with LLM assistance +category: general --- # Supply-Chain Hardening @@ -244,8 +245,8 @@ process.exit(violations > 0 ? 1 : 0); **Exception bookkeeping**: when you pin a fresh version under the exception process, leave a marker next to the pin (JSONC comment in `package.json`, or a `CHANGELOG.md` -note for strict JSON parsers): `// Exception: CVE-2026-XXXX patch within 14d window. -Reviewed .` +note for strict JSON parsers): +`// Exception: CVE-2026-XXXX patch within 14d window. Reviewed .` ## Untrusted Repos and Modes diff --git a/packages/tbd/docs/guidelines/tbd-sync-troubleshooting.md b/packages/tbd/docs/guidelines/tbd-sync-troubleshooting.md index fe666f8c..101f9101 100644 --- a/packages/tbd/docs/guidelines/tbd-sync-troubleshooting.md +++ b/packages/tbd/docs/guidelines/tbd-sync-troubleshooting.md @@ -2,6 +2,7 @@ title: tbd Sync and Workspace Troubleshooting description: Common issues and solutions for tbd sync and workspace operations author: Joshua Levy (github.com/jlevy) with LLM assistance +category: general --- ## Common Sync Issues diff --git a/packages/tbd/docs/guidelines/typescript-cli-tool-rules.md b/packages/tbd/docs/guidelines/typescript-cli-tool-rules.md index 9d3f5c79..29204a77 100644 --- a/packages/tbd/docs/guidelines/typescript-cli-tool-rules.md +++ b/packages/tbd/docs/guidelines/typescript-cli-tool-rules.md @@ -2,6 +2,7 @@ title: TypeScript CLI Tool Rules description: Rules for building CLI tools with Commander.js, picocolors, and TypeScript author: Joshua Levy (github.com/jlevy) with LLM assistance +category: typescript --- # CLI Tool Development Rules diff --git a/packages/tbd/docs/guidelines/typescript-code-coverage.md b/packages/tbd/docs/guidelines/typescript-code-coverage.md index ebd0b483..de7625ad 100644 --- a/packages/tbd/docs/guidelines/typescript-code-coverage.md +++ b/packages/tbd/docs/guidelines/typescript-code-coverage.md @@ -2,6 +2,7 @@ title: TypeScript Code Coverage description: Best practices for code coverage in TypeScript with Vitest and v8 provider author: Joshua Levy (github.com/jlevy) with LLM assistance +category: typescript --- # Code Coverage Best Practices for TypeScript with Vitest diff --git a/packages/tbd/docs/guidelines/typescript-rules.md b/packages/tbd/docs/guidelines/typescript-rules.md index 3506bfe9..c0a015e1 100644 --- a/packages/tbd/docs/guidelines/typescript-rules.md +++ b/packages/tbd/docs/guidelines/typescript-rules.md @@ -4,6 +4,7 @@ description: TypeScript coding rules and best practices author: Joshua Levy (github.com/jlevy) with LLM assistance globs: "*.ts" alwaysApply: true +category: typescript --- # TypeScript Rules diff --git a/packages/tbd/docs/guidelines/typescript-sorting-patterns.md b/packages/tbd/docs/guidelines/typescript-sorting-patterns.md index 52d75f39..80573da1 100644 --- a/packages/tbd/docs/guidelines/typescript-sorting-patterns.md +++ b/packages/tbd/docs/guidelines/typescript-sorting-patterns.md @@ -2,6 +2,7 @@ title: TypeScript Sorting Patterns description: Deterministic sorting patterns and comparison chains for TypeScript author: Joshua Levy (github.com/jlevy) with LLM assistance +category: typescript --- # TypeScript Sorting Patterns diff --git a/packages/tbd/docs/guidelines/typescript-yaml-handling-rules.md b/packages/tbd/docs/guidelines/typescript-yaml-handling-rules.md index 7e5e1629..db804cbb 100644 --- a/packages/tbd/docs/guidelines/typescript-yaml-handling-rules.md +++ b/packages/tbd/docs/guidelines/typescript-yaml-handling-rules.md @@ -3,6 +3,7 @@ title: TypeScript YAML Handling Rules description: Best practices for parsing and serializing YAML in TypeScript author: Joshua Levy (github.com/jlevy) with LLM assistance globs: "*.ts" +category: typescript --- # TypeScript YAML Handling Rules diff --git a/packages/tbd/tests/doc-categories.test.ts b/packages/tbd/tests/doc-categories.test.ts new file mode 100644 index 00000000..2bb66371 --- /dev/null +++ b/packages/tbd/tests/doc-categories.test.ts @@ -0,0 +1,76 @@ +/** + * Tests that every bundled guideline doc declares exactly one valid category + * in its YAML frontmatter. Categories drive grouped doc listings and + * category-based selection (e.g. `tbd docs fork --category=...`), so each + * guideline must land in exactly one category. + * + * Walks the SOURCE-level docs (packages/tbd/docs/guidelines) relative to this + * test file, like doc-references.test.ts does for bundled docs. + */ + +import { describe, it, expect } from 'vitest'; +import { readFile, readdir } from 'node:fs/promises'; +import { join, dirname, extname } from 'node:path'; +import { fileURLToPath } from 'node:url'; +import matter from 'gray-matter'; + +const __dirname = dirname(fileURLToPath(import.meta.url)); +const GUIDELINES_DIR = join(__dirname, '..', 'docs', 'guidelines'); + +/** + * The allowed guideline categories. Every bundled guideline must declare + * exactly one of these in its frontmatter `category` field. + * (Mirrors the category set in the forkable-docs spec; the CLI-side constant + * lands separately.) + */ +export const GUIDELINE_CATEGORIES = [ + 'general', + 'typescript', + 'python', + 'convex', + 'electron', +] as const; + +describe('guideline doc categories', () => { + it('every bundled guideline declares exactly one valid category', async () => { + const entries = await readdir(GUIDELINES_DIR); + const files = entries.filter((f) => extname(f) === '.md').sort(); + + // Sanity check: the bundled guidelines must actually be there. + expect(files.length).toBeGreaterThan(0); + + const failures: string[] = []; + + for (const file of files) { + const raw = await readFile(join(GUIDELINES_DIR, file), 'utf-8'); + + let category: unknown; + try { + // gray-matter (js-yaml) rejects duplicated keys, so a doc declaring + // `category:` twice fails here rather than silently picking one. + const parsed = matter(raw); + category = parsed.data.category; + } catch (error) { + failures.push(`${file}: frontmatter failed to parse (${(error as Error).message})`); + continue; + } + + if (category === undefined || category === null) { + failures.push(`${file}: missing frontmatter \`category\``); + } else if (typeof category !== 'string') { + // e.g. a YAML list — must be exactly one category, as a single string + failures.push( + `${file}: \`category\` must be a single string, got ${JSON.stringify(category)}`, + ); + } else if (!(GUIDELINE_CATEGORIES as readonly string[]).includes(category)) { + failures.push( + `${file}: invalid category "${category}" (allowed: ${GUIDELINE_CATEGORIES.join(', ')})`, + ); + } + } + + if (failures.length > 0) { + expect.fail(`Guideline category failures:\n ${failures.join('\n ')}`); + } + }); +}); From b15ac193d390b71d766e1beec47767a86a12eeca Mon Sep 17 00:00:00 2001 From: Test User Date: Fri, 12 Jun 2026 21:30:14 +0000 Subject: [PATCH 31/36] =?UTF-8?q?docs:=20Forkable-docs=20agent=20surface?= =?UTF-8?q?=20=E2=80=94=20upstream=20playbook,=20onboarding,=20CHANGELOG?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Phase 5 docs items (17/19/20) plus contract-table rows for the forkable-docs feature (beads tbd-e12s, tbd-lxab, tbd-5bvs): - New suggest-upstream-improvements shortcut: the playbook for reviewing fork customizations (tbd docs status --json, tbd docs diff --base), deciding what generalizes, filing upstream, and re-syncing via tbd docs update once merged. - welcome-user: the two-axis onboarding offer (scope + visibility) and a make-guidelines-visible routing row (tbd docs fork / --all). - README: Forkable paragraph in Shortcuts/Guidelines/Templates, tbd docs as the managed-docs overview with the manual at tbd docs show tbd-docs, --add lines annotated as aliases for the upcoming tbd docs add . - development.md: .tbd/doc-forks/ tracking-state note (fork dir lives outside .tbd/, default docs/tbd/) and a Testing Forkable Docs pointer. - CHANGELOG: Unreleased (0.3.0) f05 entry per release-notes-guidelines. - tbd-docs.md Configuration Reference: docs_cache example with docref values, planned fork_dir/local_dirs keys, docref-format/docmap-format cross-links. https://claude.ai/code/session_01QPsCSYGtwR8JtX2R1aaxyh --- README.md | 23 +++++-- docs/development.md | 11 +++ packages/tbd/CHANGELOG.md | 69 +++++++++++++++++++ .../standard/suggest-upstream-improvements.md | 69 +++++++++++++++++++ .../docs/shortcuts/standard/welcome-user.md | 14 ++++ packages/tbd/docs/tbd-docs.md | 22 ++++++ 6 files changed, 202 insertions(+), 6 deletions(-) create mode 100644 packages/tbd/docs/shortcuts/standard/suggest-upstream-improvements.md diff --git a/README.md b/README.md index 8bd9d4bf..ce7ba98e 100644 --- a/README.md +++ b/README.md @@ -163,8 +163,8 @@ You just talk naturally. > [!NOTE] > For full technical details, see the [reference docs](packages/tbd/docs/tbd-docs.md) -> (run `tbd docs`) or the full [design doc](packages/tbd/docs/tbd-design.md) -> (`tbd design`). +> (run `tbd docs show tbd-docs`) or the full +> [design doc](packages/tbd/docs/tbd-design.md) (`tbd design`). - **Git-native:** Beads live in your repo, synced to a separate, dedicated `tbd-sync` branch. Your code history stays clean—no bead churn polluting your logs. @@ -309,9 +309,9 @@ publish the upgrade to your team. Teammates still on an older tbd then see “This repository requires a newer version of tbd” until they run the same two commands. Issue data is never touched by an upgrade, and the migration is revertible: see -“Aborting a Format Upgrade” under Troubleshooting in `tbd docs`. If you have forked docs -in `docs/tbd/`, `tbd sync` prints a notice when their upstream versions moved — run -`tbd docs update` to merge the changes in. +“Aborting a Format Upgrade” under Troubleshooting in the CLI manual (`tbd docs manual`). +If you have forked docs in `docs/tbd/`, `tbd sync` prints a notice when their upstream +versions moved — run `tbd docs update` to merge the changes in. ### Team Setup @@ -415,11 +415,21 @@ tbd template --list # List all templates tbd template plan-spec # Get a plan spec template # Add your own from any URL +# (per-kind aliases for the upcoming unified `tbd docs add `) tbd guidelines --add= --name= tbd shortcut --add= --name= tbd template --add= --name= ``` +**Forkable: see them in your repo.** By default these docs are served from a hidden, +gitignored cache. Fork any of them into `docs/tbd/` and they become visible on GitHub, +reviewable in PRs, and editable in place—tbd serves your copy instead, and +`tbd docs update` merges upstream improvements into it after an upgrade: + +```bash +tbd docs fork --all # Or fork by name: tbd docs fork [...] +``` + **Available shortcuts:** | Category | Shortcut | Purpose | @@ -501,7 +511,8 @@ Every command supports these flags for automation: ```bash tbd # Full orientation and workflow guidance tbd readme # This file -tbd docs # Full CLI reference +tbd docs # Managed-docs overview (cached, forked, and local docs) +tbd docs show tbd-docs # Full CLI reference (the manual; alias: tbd docs manual) ``` Or read online: diff --git a/docs/development.md b/docs/development.md index 4a721781..c820650f 100644 --- a/docs/development.md +++ b/docs/development.md @@ -367,6 +367,11 @@ $GIT_COMMON_DIR/tbd/ # Shared by all linked worktrees of this rep └── meta.yml ``` +`.tbd/doc-forks/` is committed and holds only fork *tracking state*: the `forks.yml` +manifest plus `base/` snapshots that `tbd docs update` three-way merges against. +The doc **fork dir** itself lives deliberately *outside* `.tbd/` — default `docs/tbd/`, +tracked in git like any other docs. + **CRITICAL**: Issues must be written to the **worktree path** (`$GIT_COMMON_DIR/tbd/data-sync-worktree/.tbd/data-sync/issues/`), NOT the direct path (`.tbd/data-sync/issues/`). The direct path is gitignored and exists only as a legacy @@ -453,6 +458,12 @@ Run the e2e worktree scenarios: npx tryscript run tests/cli-sync-worktree-scenarios.tryscript.md ``` +### Testing Forkable Docs + +Forkable-docs behavior (fork/unfork/update/diff/status) is covered by +`tests/cli-docs-fork.tryscript.md`, `tests/cli-docs-update.tryscript.md`, and +`tests/fork-cross-platform-e2e.test.ts` (run from `packages/tbd/`). + diff --git a/packages/tbd/CHANGELOG.md b/packages/tbd/CHANGELOG.md index 11cb0e1d..4864b59d 100644 --- a/packages/tbd/CHANGELOG.md +++ b/packages/tbd/CHANGELOG.md @@ -1,5 +1,74 @@ # get-tbd +## Unreleased (0.3.0) + +The headline is **forkable docs**: every doc tbd serves (guidelines, shortcuts, +templates, and the new reference docs) can now be forked into your repo as visible, +git-tracked files, edited in place, and reconciled with upstream after upgrades. +The repository format bumps `f04` → `f05`; the migration is **stamp-only** (it stamps +`tbd_format: f05` in `.tbd/config.yml` and refreshes generated metadata — fork artifacts +appear only when you first fork something), so upgrading is safe and revertible. + +### Features + +- **Forkable docs** (`tbd docs fork` / `unfork` / `update` / `diff` / `status`): fork + any managed doc into a visible fork dir (`docs/tbd/`, laid out by kind with a + generated `README.md` index, tracked in git). + Your copy shadows the hidden cache everywhere the doc is served; forking changes + nothing about how docs work — it only makes them explicit and editable. + - `tbd docs update` three-way merges upstream changes into your copies after an + upgrade: `--merge` combines and writes conflict markers to resolve, `--keep-ours` + keeps your version and advances the fork point; `--dry-run` previews and lists + conflicts. + - `tbd docs diff ` compares your copy against upstream (default), against its + recorded base (`--base`, what you changed), or base against upstream (`--upstream`, + incoming changes). + - `tbd docs status` reports every fork’s state (`forked`, `customized`, `conflicted`, + `missing`, plus `local` for your own files in the fork dir). + States are recomputed from content hashes — no git operation can desynchronize + tracking. Fork state lives in the committed `.tbd/doc-forks/` (a `forks.yml` manifest + plus `base/` snapshots); the fork dir itself stays outside `.tbd/`. +- **The `tbd docs` surface is re-homed around managed docs**: bare `tbd docs` is now the + status overview, and `tbd docs list` lists all docs across kinds with + `[forked]`/`[customized]`/`[local]` markers. + The CLI manual moved to `tbd docs show tbd-docs` (alias: `tbd docs manual`); the old + `tbd docs --list` / `--all` / `--section` flags are retired in favor of + `tbd docs show tbd-docs --sections` / `--section `. `tbd docs show ` reads + any doc by name, kind-agnostically. + `tbd docs sync` refreshes the gitignored docs cache (`tbd sync --docs` remains as a + deprecated alias). +- **docref + docmap formats, and a new `reference` doc kind**: every doc source is + addressed by a **docref** — one URI-like grammar (`internal:…`, anchored local paths, + URLs, `github:owner/repo@ref//path`) used for `docs_cache.files` values and + fork-manifest `source` values alike — and every doc listing is one **docmap** + (`docmap/0.1`) rendered as text or `--json`: `tbd docs list` / `tbd docs status`, the + bare-`tbd docs` overview, and the per-kind `--list` (whose `--json` output changes + from a flat array to a docmap). + Both formats ship as docs of the new `reference` kind — read them with + `tbd docs show docref-format` and `tbd docs show docmap-format` — alongside the manual + (`tbd-docs`) and design doc (`tbd-design`), which are now managed docs too. +- **Fork drift is visible, never auto-fixed**: `tbd status` gains a `Docs:` line when + forks exist (forked/customized counts, pending upstream updates, conflicts, missing + files), and `tbd sync` prints a one-line notice when forked docs are stale, + conflicted, or missing. + Only the explicit `tbd docs update` ever modifies tracked files. + +### Guidelines and content + +These ship inside the package and are read by agents via `tbd docs show …`, +`tbd shortcut …`, and `tbd setup`: + +- **New `suggest-upstream-improvements` shortcut**: the playbook for reviewing fork + customizations (`tbd docs status --json`, `tbd docs diff --base`), deciding + what generalizes, contributing it upstream, and re-syncing with `tbd docs update` once + merged. +- **New `docref-format` and `docmap-format` reference docs**: the specifications for the + two formats above, forkable like any other doc. +- **Onboarding and agent surface updated for forkable docs**: `welcome-user` now makes + the two-axis offer (scope: all standard guidelines or a stack subset; visibility: + hidden cache or forked into `docs/tbd/`), and the agent skill routes fork, update, and + missing-file requests to the new commands. + ## 0.2.3 A drop-in patch on top of v0.2.2. **No on-disk format change** (`f04` stays `f04`), so diff --git a/packages/tbd/docs/shortcuts/standard/suggest-upstream-improvements.md b/packages/tbd/docs/shortcuts/standard/suggest-upstream-improvements.md new file mode 100644 index 00000000..b4012c9b --- /dev/null +++ b/packages/tbd/docs/shortcuts/standard/suggest-upstream-improvements.md @@ -0,0 +1,69 @@ +--- +title: Suggest Upstream Improvements +description: Review local doc-fork customizations and contribute the generally useful changes back upstream +category: meta +author: Joshua Levy (github.com/jlevy) with LLM assistance +--- +This shortcut reviews the docs this project has forked and customized (in `docs/tbd/`), +decides which changes generalize beyond this project, and proposes them upstream — to +the tbd repo for tbd’s bundled docs, or to your org’s docs repo for docs added by URL. + +## When to Use + +- Forked guidelines or shortcuts have accumulated edits that would help other projects. +- Before unforking: upstream the good parts first, so nothing is lost when the fork goes + away. +- After `tbd docs update` keeps producing the same conflict because upstream lacks an + improvement you made locally. + +## Instructions + +Create a to-do list with the following items then perform all of them: + +1. **Collect customizations**: Run `tbd docs status --json` and collect every doc in + `customized` state (including customized docs that also have upstream updates + pending). + +2. **Review each diff**: For each doc, run `tbd docs diff --base` (your file vs + its recorded base — exactly what this project changed). + Classify each hunk: + - **Generally applicable**: fixes, clarifications, better examples, rules any project + would benefit from. Candidates for upstreaming. + - **Project-specific**: team conventions, internal links, stack-specific overrides. + These stay in the fork; do not propose them. + +3. **Identify the upstream target** for each doc with generalizable hunks: + - Bundled docs (manifest `source` starts with `internal:`): the tbd repo + (`jlevy/tbd`), where doc sources live under `packages/tbd/docs/`. + - URL-added docs: the repo their docref points at (for example, your org’s shared + docs repo). + +4. **Draft the proposal**: For each doc, draft an issue or PR body containing: + - Which doc (name and kind) and why the change is generally useful. + - Only the generalizable diff hunks, in fenced code blocks. + - Brief project context: what prompted the change here. + +5. **Confirm, then file**: Show the draft to the user and get confirmation. + Then: + - Issue: `gh issue create -R jlevy/tbd` (or the org’s repo) with the drafted body. + - PR (preferred when the change is small and ready): apply the generalizable hunks to + the upstream source file on a branch and open a PR with `gh pr create`. + +6. **Close the loop (after upstream merges)**: Once the change ships upstream and tbd is + upgraded, run `tbd docs update`. If upstream adopted the customization, the merge + converges and the doc returns to unmodified `forked` state — then a plain + `tbd docs unfork ` (no `--force` needed) completes the cleanup, or keep the + fork for future edits. + +## Notes + +- Only propose hunks you would accept as a maintainer: each should stand alone, with + rationale. +- Never file an issue or PR without showing the user the draft first. +- Diff views: `tbd docs diff ` (no flag) compares your copy against current + upstream; `--base` is the right view for “what did we change”; `--upstream` shows + incoming upstream changes. + + diff --git a/packages/tbd/docs/shortcuts/standard/welcome-user.md b/packages/tbd/docs/shortcuts/standard/welcome-user.md index a2c3feba..e9e1d25c 100644 --- a/packages/tbd/docs/shortcuts/standard/welcome-user.md +++ b/packages/tbd/docs/shortcuts/standard/welcome-user.md @@ -13,6 +13,19 @@ project. First, run `tbd status` to check the current state. Give a brief summary of the status (repository, sync status, integrations). +Then make the two-axis guidelines offer — one short question per axis: + +1. **Scope** — keep **all** standard guidelines active (recommended), or just a subset + for this project’s languages and stack? +2. **Visibility** — leave them in tbd’s hidden cache (the default — they just work), or + fork them into `docs/tbd/` so they are visible on GitHub, reviewable in PRs, and + editable (checked into git)? + +Explain that forking changes nothing about how guidelines work — both paths serve the +same guidelines — it only makes them visible and customizable. +If the user wants visible docs, run `tbd docs fork [...]` for the chosen +subset or `tbd docs fork --all` for everything (preview with `--dry-run` first). + Then show the welcome message: * * * @@ -50,6 +63,7 @@ Here are examples of things you can say and what happens: | “I’m building a TypeScript CLI” | Applies TypeScript CLI guidelines | | “Help me set up better testing” | Applies testing guidelines | | “What are the Python best practices?” | Applies Python guidelines | +| “Make the guidelines visible in my repo” | Forks them into `docs/tbd/` (`tbd docs fork ` or `tbd docs fork --all`) | **Tips:** diff --git a/packages/tbd/docs/tbd-docs.md b/packages/tbd/docs/tbd-docs.md index b713feb3..290e7fe2 100644 --- a/packages/tbd/docs/tbd-docs.md +++ b/packages/tbd/docs/tbd-docs.md @@ -1096,8 +1096,30 @@ sync: branch: tbd-sync # Sync branch name remote: origin # Remote name auto_sync: true # Auto-sync after writes + +docs_cache: + files: # Docs synced into the cache: destination -> docref + guidelines/python-rules.md: internal:guidelines/python-rules.md + guidelines/my-team-rules.md: github:my-org/docs@main//rules.md + lookup_path: # Search paths for doc lookup (earlier wins) + - .tbd/docs/shortcuts/system + - .tbd/docs/shortcuts/standard ``` +`docs_cache.files` values — like the fork manifest’s `source` values in +`.tbd/doc-forks/forks.yml` — are **docrefs**: one URI-like address grammar +(`internal:…`, anchored local paths, URLs, `github:owner/repo@ref//path`). For the full +grammar see `tbd docs show docref-format`; for the docmap structure that doc listings +and their `--json` output follow, see `tbd docs show docmap-format`. + +Two further `docs_cache` keys are reserved in the f05 format era but **planned, not yet +read**: + +- `docs_cache.fork_dir` — override the fork-dir location (currently fixed at + `docs/tbd/`). +- `docs_cache.local_dirs` — extra local doc directories (as docrefs) served between the + fork dir and the cache. + ## Priority Scale | Value | Alias | Meaning | From 374fb10e09699745867184b29960d27116839de7 Mon Sep 17 00:00:00 2001 From: Test User Date: Fri, 12 Jun 2026 21:31:23 +0000 Subject: [PATCH 32/36] feat: tbd doctor 'Forked docs' check group with --fix finalize-unfork (tbd-5xt0) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Append a fork-consistency check group to doctor's HEALTH CHECKS, per the forkable-docs spec (Phase 2 item 12, folding tbd-nt2c): - forks.yml totally unparseable: report ⚠ naming the file instead of crashing - missing forked files (deleted out-of-band): ⚠; --fix finalizes the unfork (removes manifest entry + base, regenerates the fork-dir README) and the doc is served from upstream again - orphaned entries (upstream/cache doc gone): ⚠; --fix removes entry + base, keeping the file as a local doc - base snapshot missing or hash-mismatched: ⚠ with re-fork/unfork remediation (no auto-fix - choosing would guess at user intent) - unresolved tbd conflict markers in forked files (flag-independent) - reserved tbd-* names claimed by non-manifest files in the fork dir - fork dir gitignored (git check-ignore, only when forks exist) Zero forks and no fork dir: the group contributes nothing, so doctor output is byte-identical for repos that never touched forking (golden-output and cli-setup goldens verified unchanged). All forks healthy: one line, '✓ Forked docs - N forked, base snapshots intact' plus the Fork dir ✓. KIND_CACHE_PATHS is replicated locally in doctor.ts rather than exported from doc-fork.ts (kept out of that module's API on purpose). E2e coverage in tests/doctor-fork-checks.test.ts against the built CLI: healthy headline, missing+--fix finalize, orphaned+--fix, conflict markers, reserved names, base tamper, gitignored fork dir, corrupt manifest, and zero-fork silence. https://claude.ai/code/session_01QPsCSYGtwR8JtX2R1aaxyh --- packages/tbd/src/cli/commands/doctor.ts | 323 ++++++++++++++++++ packages/tbd/tests/doctor-fork-checks.test.ts | 219 ++++++++++++ 2 files changed, 542 insertions(+) create mode 100644 packages/tbd/tests/doctor-fork-checks.test.ts diff --git a/packages/tbd/src/cli/commands/doctor.ts b/packages/tbd/src/cli/commands/doctor.ts index 79da25bf..d8759464 100644 --- a/packages/tbd/src/cli/commands/doctor.ts +++ b/packages/tbd/src/cli/commands/doctor.ts @@ -22,7 +22,13 @@ import { resolveSharedTbdPaths, TBD_DIR, DATA_SYNC_DIR, + FORK_DIR, + CACHE_GUIDELINES_PATHS, + CACHE_REFERENCE_PATHS, + CACHE_SHORTCUT_PATHS, + CACHE_TEMPLATE_PATHS, } from '../../lib/paths.js'; +import type { ForkEntry } from '../../file/fork-manifest.js'; import { detectDuplicateYamlKeys } from '../../utils/yaml-utils.js'; import { getClaudePaths, @@ -355,6 +361,24 @@ class DoctorHandler extends BaseCommand { // Check 15: Sync consistency (worktree matches local, ahead/behind counts) healthChecks.push(await this.safeCheck('Sync consistency', () => this.checkSyncConsistency())); + // Check 16: Forked docs (manifest ↔ base snapshots ↔ fork dir consistency). + // A check *group*: contributes zero lines when nothing is forked and no + // fork dir exists (doctor output for non-fork users must not grow), one ✓ + // line when all forks are healthy, and one ⚠ line per issue category + // otherwise. Unexpected throws degrade to one error finding (safeCheck + // semantics, adapted for a multi-result check). + try { + healthChecks.push(...(await this.checkForkedDocs(options.fix))); + } catch (error) { + healthChecks.push({ + name: 'Forked docs', + status: 'error', + message: `check could not complete: ${ + error instanceof Error ? error.message : String(error) + }`, + }); + } + // Run integration checks (optional IDE/agent integrations) const integrationChecks: DiagnosticResult[] = []; @@ -1782,6 +1806,305 @@ class DoctorHandler extends BaseCommand { }; } } + + /** + * Check 16 group: Forked docs (`.tbd/doc-forks/` ↔ base snapshots ↔ fork dir). + * + * Validates the forkable-docs state per the f05 spec (Phase 2 doctor checks): + * manifest readability, missing forked files (--fix finalizes the unfork), + * orphaned entries whose upstream doc is gone (--fix removes the entry), + * base snapshot presence/hash integrity (no auto-fix — re-fork vs unfork is + * the user's call), unresolved tbd conflict markers, user docs claiming the + * reserved `tbd-` name prefix, and a gitignored fork dir. + * + * Returns zero findings when nothing is forked and no fork dir exists, so + * doctor output is byte-identical for repos that never touched forking. + * See: docs/project/specs/active/plan-2026-06-11-forkable-docs.md §`tbd doctor`. + */ + private async checkForkedDocs(fix?: boolean): Promise { + const name = 'Forked docs'; + const { + DOC_FORKS_DIR, + FORKS_FILE, + findFork, + hashContent, + hasUnresolvedConflict, + readForkManifest, + removeBaseContent, + removeFork, + withForkManifestLock, + writeForkManifest, + } = await import('../../file/fork-manifest.js'); + const manifestPath = `${DOC_FORKS_DIR}/${FORKS_FILE}`; + + // 16a: manifest readability. readForkManifest tolerates per-entry corruption + // (drops bad entries with a stderr warning) but throws on a totally + // unparseable file — report that instead of crashing the doctor run. + let manifest; + try { + manifest = await readForkManifest(this.cwd); + } catch (error) { + const reason = (error instanceof Error ? error.message : String(error)).split('\n')[0]; + return [ + { + name, + status: 'warn', + message: `fork manifest unreadable: ${reason}`, + path: manifestPath, + suggestion: `Fix or delete ${manifestPath} (forked files stay in place), then re-run tbd doctor`, + }, + ]; + } + + // Zero forks and no fork dir: print nothing. + let forkDirExists = true; + try { + await access(join(this.cwd, FORK_DIR)); + } catch { + forkDirExists = false; + } + if (manifest.forks.length === 0 && !forkDirExists) { + return []; + } + + const { + forkStatusFor, + listLocalForkFiles, + readForkBase, + readForkFile, + regenerateForkDirReadme, + unforkDoc, + } = await import('../../file/doc-fork.js'); + const { DocCache } = await import('../../file/doc-cache.js'); + + // Cache-only lookup paths per kind (the pristine upstream copies). Replica + // of doc-fork.ts's module-private KIND_CACHE_PATHS, which is deliberately + // not exported (doctor owns its own copy rather than widening that API). + const kindCachePaths: Record = { + guideline: CACHE_GUIDELINES_PATHS, + shortcut: CACHE_SHORTCUT_PATHS, + template: CACHE_TEMPLATE_PATHS, + reference: CACHE_REFERENCE_PATHS, + }; + const caches = new Map>(); + + // Classify every manifest entry into at most one issue bucket + // (missing > orphaned > base problem), with unresolved conflict markers + // detected on every fork file that still exists. + const missing: ForkEntry[] = []; + const orphaned: ForkEntry[] = []; + const baseProblems: string[] = []; + const conflicted: string[] = []; + + for (const entry of manifest.forks) { + let cache = caches.get(entry.kind); + if (!cache) { + cache = new DocCache(kindCachePaths[entry.kind] ?? [], this.cwd); + await cache.load({ quiet: true }); + caches.set(entry.kind, cache); + } + const cacheContent = cache.get(entry.name)?.doc.content ?? null; + const status = await forkStatusFor(this.cwd, FORK_DIR, entry, cacheContent); + + if (status.state === 'missing') { + missing.push(entry); + continue; + } + if (status.orphaned) { + orphaned.push(entry); + continue; + } + + // 16d: base snapshot integrity for live forks. + const base = await readForkBase(this.cwd, entry); + if (base === null) { + baseProblems.push(`${entry.name}: missing`); + } else if (hashContent(base) !== entry.base_hash) { + baseProblems.push(`${entry.name}: hash mismatch`); + } + + // 16e: unresolved tbd conflict markers (flag-independent — detect markers + // even when the manifest `conflicted` flag was never set or went stale). + const content = await readForkFile(this.cwd, FORK_DIR, entry); + if (content !== null && hasUnresolvedConflict(content)) { + conflicted.push(entry.name); + } + } + + const results: DiagnosticResult[] = []; + + // 16b: manifest entries whose forked file was deleted out-of-band. The + // deletion is read as intent to stop forking: --fix finalizes the unfork + // (removes manifest entry + base snapshot; the doc is served from upstream). + if (missing.length > 0) { + const message = `${missing.length} missing (${missing.map((e) => e.name).join(', ')}: forked file deleted)`; + if (fix && !this.checkDryRun('Finalize unfork of deleted forked docs')) { + try { + await withForkManifestLock(this.cwd, async () => { + let current = await readForkManifest(this.cwd); + for (const entry of missing) { + if (!findFork(current, entry.name, entry.kind)) continue; + const result = await unforkDoc({ + tbdRoot: this.cwd, + forkDir: FORK_DIR, + manifest: current, + name: entry.name, + kind: entry.kind, + }); + current = result.manifest; + } + await writeForkManifest(this.cwd, current); + await regenerateForkDirReadme(this.cwd, FORK_DIR, current); + }); + results.push({ + name, + status: 'warn', + message, + details: [ + 'Fixed: finalized unfork (removed manifest entry + base); now served from upstream', + ], + }); + } catch (error) { + results.push({ + name, + status: 'error', + message: `failed to finalize unfork: ${ + error instanceof Error ? error.message : String(error) + }`, + }); + } + } else { + results.push({ + name, + status: 'warn', + message, + fixable: true, + suggestion: + 'Run: tbd doctor --fix to finalize the unfork, or tbd docs fork --force to restore', + }); + } + } + + // 16c: orphaned entries (upstream/cache doc no longer exists). --fix removes + // the manifest entry + base but keeps the file (it becomes a local doc — + // upstream is gone, so the file may be the only copy). + if (orphaned.length > 0) { + const message = `${orphaned.length} orphaned (${orphaned.map((e) => e.name).join(', ')}: upstream doc no longer exists)`; + if (fix && !this.checkDryRun('Remove orphaned fork manifest entries')) { + try { + await withForkManifestLock(this.cwd, async () => { + let current = await readForkManifest(this.cwd); + for (const entry of orphaned) { + current = removeFork(current, entry.name, entry.kind); + await removeBaseContent(this.cwd, entry.kind, entry.name); + } + await writeForkManifest(this.cwd, current); + await regenerateForkDirReadme(this.cwd, FORK_DIR, current); + }); + results.push({ + name, + status: 'warn', + message, + details: [ + `Fixed: removed orphaned manifest entr${orphaned.length === 1 ? 'y' : 'ies'} + base; file kept as a local doc`, + ], + }); + } catch (error) { + results.push({ + name, + status: 'error', + message: `failed to remove orphaned entries: ${ + error instanceof Error ? error.message : String(error) + }`, + }); + } + } else { + results.push({ + name, + status: 'warn', + message, + fixable: true, + suggestion: + 'Run: tbd doctor --fix to remove the entry (your file is kept as a local doc)', + }); + } + } + + // 16d findings: no auto-fix — choosing between re-fork and unfork would + // guess at user intent. + if (baseProblems.length > 0) { + results.push({ + name, + status: 'warn', + message: `${baseProblems.length} base snapshot problem${baseProblems.length === 1 ? '' : 's'} (${baseProblems.join(', ')})`, + suggestion: 'Run: tbd docs fork --force to re-fork, or tbd docs unfork ', + }); + } + + // 16e findings. + if (conflicted.length > 0) { + results.push({ + name, + status: 'warn', + message: `${conflicted.length} unresolved merge conflict${conflicted.length === 1 ? '' : 's'} (${conflicted.join(', ')})`, + suggestion: 'Run: resolve the conflict markers, then re-run tbd docs update', + }); + } + + // Healthy headline: exactly one ✓ line when forks exist and 16b–16e found + // nothing (reserved-name and fork-dir findings below have their own names). + if (manifest.forks.length > 0 && results.length === 0) { + results.push({ + name, + status: 'ok', + message: `${manifest.forks.length} forked, base snapshots intact`, + }); + } + + // 16f: user docs claiming the reserved `tbd-` prefix (fork-dir files with + // no manifest entry; forked tbd self-docs legitimately keep their entry). + const locals = await listLocalForkFiles(this.cwd, FORK_DIR, manifest); + const reserved = locals.filter((l) => l.name.startsWith('tbd-')); + if (reserved.length > 0) { + results.push({ + name: 'Reserved tbd- names', + status: 'warn', + message: `${reserved.length} user doc${reserved.length === 1 ? ' claims' : 's claim'} the reserved tbd- prefix`, + details: reserved.map((l) => l.relPath), + suggestion: 'Rename the file(s): the tbd- prefix is reserved for tbd self-docs', + }); + } + + // 16g: fork dir gitignored (only meaningful when forks exist — a gitignored + // fork dir defeats the purpose of forking: the docs would not be committed). + if (manifest.forks.length > 0) { + let ignored = false; + try { + await git('-C', this.cwd, 'check-ignore', '-q', FORK_DIR); + ignored = true; + } catch { + // Exit 1 = not ignored (healthy). Other failures: cannot verify; do not + // warn on a guess. + ignored = false; + } + results.push( + ignored + ? { + name: 'Fork dir', + status: 'warn', + message: `${FORK_DIR}/ is gitignored — forked docs will not be committed`, + suggestion: `Remove the .gitignore rule covering ${FORK_DIR}/ so forks are tracked in git`, + } + : { + name: 'Fork dir', + status: 'ok', + message: `${FORK_DIR}/ tracked in git (not gitignored)`, + }, + ); + } + + return results; + } } export const doctorCommand = new Command('doctor') diff --git a/packages/tbd/tests/doctor-fork-checks.test.ts b/packages/tbd/tests/doctor-fork-checks.test.ts new file mode 100644 index 00000000..a6c63b52 --- /dev/null +++ b/packages/tbd/tests/doctor-fork-checks.test.ts @@ -0,0 +1,219 @@ +/** + * E2e tests for the `tbd doctor` "Forked docs" check group (forkable-docs spec + * Phase 2, tbd-5xt0), run against the built CLI in a temp repo. Pins the + * contract lines from plan-2026-06-11-forkable-docs.md §`tbd doctor`: the + * healthy ✓ headline, the missing-file ⚠ + `--fix` finalize-unfork flow, + * conflict-marker detection, reserved `tbd-` names, base snapshot integrity, + * the gitignored fork dir, corrupt-manifest reporting — and zero-fork silence + * (doctor output for non-fork users must not grow). + */ + +import { describe, it, expect, beforeEach, afterEach } from 'vitest'; +import { mkdtemp, rm, readFile, writeFile, appendFile, mkdir, stat } from 'node:fs/promises'; +import { tmpdir } from 'node:os'; +import { join, dirname } from 'node:path'; +import { fileURLToPath } from 'node:url'; +import { execSync, spawnSync } from 'node:child_process'; + +const __dirname = dirname(fileURLToPath(import.meta.url)); + +describe('doctor fork checks e2e', { timeout: 120_000 }, () => { + let tempDir: string; + const tbdBin = join(__dirname, '..', 'dist', 'bin.mjs'); + + beforeEach(async () => { + tempDir = await mkdtemp(join(tmpdir(), 'tbd-doctor-fork-')); + execSync('git init --initial-branch=main', { cwd: tempDir }); + execSync('git config user.email "test@example.com"', { cwd: tempDir }); + execSync('git config user.name "Test"', { cwd: tempDir }); + execSync('git config commit.gpgsign false', { cwd: tempDir }); + runTbd(['init', '--prefix=fx']); + }); + + afterEach(async () => { + await rm(tempDir, { recursive: true, force: true }); + }); + + function runTbd(args: string[]): { stdout: string; stderr: string; status: number } { + const result = spawnSync('node', [tbdBin, ...args], { + cwd: tempDir, + encoding: 'utf-8', + env: { ...process.env, FORCE_COLOR: '0', NO_COLOR: '1' }, + timeout: 60000, + }); + return { + stdout: result.stdout || '', + stderr: result.stderr || '', + status: result.status ?? 1, + }; + } + + async function exists(path: string): Promise { + try { + await stat(path); + return true; + } catch { + return false; + } + } + + it('zero forks: doctor prints no fork lines at all', () => { + const doctor = runTbd(['doctor']); + expect(doctor.status).toBe(0); + expect(doctor.stdout).not.toContain('Forked docs'); + expect(doctor.stdout).not.toContain('Fork dir'); + expect(doctor.stdout).not.toContain('Reserved tbd- names'); + }); + + it('healthy forks: exactly one ✓ headline plus the fork-dir ✓ line', () => { + expect(runTbd(['docs', 'fork', 'python-rules', 'review-code']).status).toBe(0); + + const doctor = runTbd(['doctor']); + expect(doctor.status).toBe(0); + expect(doctor.stdout).toContain('✓ Forked docs - 2 forked, base snapshots intact'); + expect(doctor.stdout).toContain('✓ Fork dir - docs/tbd/ tracked in git (not gitignored)'); + expect(doctor.stdout.match(/Forked docs/g)).toHaveLength(1); + expect(doctor.stdout).not.toContain('Reserved tbd- names'); + }); + + it('deleted forked file: ⚠ then --fix finalizes the unfork', async () => { + expect(runTbd(['docs', 'fork', 'review-code']).status).toBe(0); + await rm(join(tempDir, 'docs', 'tbd', 'shortcuts', 'review-code.md')); + + const doctor = runTbd(['doctor']); + expect(doctor.status).toBe(0); + expect(doctor.stdout).toContain( + '⚠ Forked docs - 1 missing (review-code: forked file deleted) [fixable]', + ); + expect(doctor.stdout).toContain( + 'Run: tbd doctor --fix to finalize the unfork, or tbd docs fork --force to restore', + ); + + const fixed = runTbd(['doctor', '--fix']); + expect(fixed.status).toBe(0); + expect(fixed.stdout).toContain('⚠ Forked docs - 1 missing (review-code: forked file deleted)'); + expect(fixed.stdout).toContain( + 'Fixed: finalized unfork (removed manifest entry + base); now served from upstream', + ); + + // Manifest entry and base snapshot are gone. + const manifest = await readFile(join(tempDir, '.tbd', 'doc-forks', 'forks.yml'), 'utf-8'); + expect(manifest).not.toContain('review-code'); + expect( + await exists(join(tempDir, '.tbd', 'doc-forks', 'base', 'shortcut', 'review-code.md')), + ).toBe(false); + + // Doc is served from upstream again (no provenance note on stderr). + const show = runTbd(['docs', 'show', 'review-code']); + expect(show.status).toBe(0); + expect(show.stdout.length).toBeGreaterThan(0); + expect(show.stderr).not.toContain('serving forked copy'); + + // With the last fork finalized the fork dir is pruned — doctor goes silent. + const after = runTbd(['doctor']); + expect(after.status).toBe(0); + expect(after.stdout).not.toContain('Forked docs'); + }); + + it('orphaned entry (upstream doc gone): ⚠ then --fix removes the entry, keeps the file', async () => { + expect(runTbd(['docs', 'fork', 'python-rules']).status).toBe(0); + await rm(join(tempDir, '.tbd', 'docs', 'guidelines', 'python-rules.md')); + + const doctor = runTbd(['doctor']); + expect(doctor.status).toBe(0); + expect(doctor.stdout).toContain( + '⚠ Forked docs - 1 orphaned (python-rules: upstream doc no longer exists) [fixable]', + ); + + const fixed = runTbd(['doctor', '--fix']); + expect(fixed.status).toBe(0); + expect(fixed.stdout).toContain( + 'Fixed: removed orphaned manifest entry + base; file kept as a local doc', + ); + const manifest = await readFile(join(tempDir, '.tbd', 'doc-forks', 'forks.yml'), 'utf-8'); + expect(manifest).not.toContain('python-rules'); + expect(await exists(join(tempDir, 'docs', 'tbd', 'guidelines', 'python-rules.md'))).toBe(true); + }); + + it('unresolved conflict markers in a forked file: ⚠ with the update remediation', async () => { + expect(runTbd(['docs', 'fork', 'python-rules']).status).toBe(0); + const forkedPath = join(tempDir, 'docs', 'tbd', 'guidelines', 'python-rules.md'); + const content = await readFile(forkedPath, 'utf-8'); + // tbd's own merge-conflict labels — detection keys off these, not generic markers. + await writeFile( + forkedPath, + `${content}\n<<<<<<< ours (your fork)\nmine\n=======\ntheirs\n>>>>>>> theirs (upstream)\n`, + ); + + const doctor = runTbd(['doctor']); + expect(doctor.status).toBe(0); + expect(doctor.stdout).toContain('⚠ Forked docs - 1 unresolved merge conflict (python-rules)'); + expect(doctor.stdout).toContain( + 'Run: resolve the conflict markers, then re-run tbd docs update', + ); + }); + + it('reserved tbd-* stray file warns, even with zero forks', async () => { + await mkdir(join(tempDir, 'docs', 'tbd', 'references'), { recursive: true }); + await writeFile( + join(tempDir, 'docs', 'tbd', 'references', 'tbd-myhack.md'), + '# Hand-authored, claims the reserved prefix\n', + ); + + const doctor = runTbd(['doctor']); + expect(doctor.status).toBe(0); + expect(doctor.stdout).toContain( + '⚠ Reserved tbd- names - 1 user doc claims the reserved tbd- prefix', + ); + expect(doctor.stdout).toContain('docs/tbd/references/tbd-myhack.md'); + // No manifest entries, so no Forked docs headline and no fork-dir line. + expect(doctor.stdout).not.toContain('Forked docs'); + expect(doctor.stdout).not.toContain('Fork dir'); + }); + + it('base snapshot hash mismatch: ⚠ with re-fork/unfork remediation (no auto-fix)', async () => { + expect(runTbd(['docs', 'fork', 'python-rules']).status).toBe(0); + await appendFile( + join(tempDir, '.tbd', 'doc-forks', 'base', 'guideline', 'python-rules.md'), + '\ntampered\n', + ); + + const doctor = runTbd(['doctor']); + expect(doctor.status).toBe(0); + expect(doctor.stdout).toContain( + '⚠ Forked docs - 1 base snapshot problem (python-rules: hash mismatch)', + ); + expect(doctor.stdout).toContain( + 'Run: tbd docs fork --force to re-fork, or tbd docs unfork ', + ); + + // --fix must not touch it (re-fork vs unfork is the user's call). + const fixed = runTbd(['doctor', '--fix']); + expect(fixed.stdout).toContain('1 base snapshot problem (python-rules: hash mismatch)'); + }); + + it('gitignored fork dir warns when forks exist', async () => { + expect(runTbd(['docs', 'fork', 'python-rules']).status).toBe(0); + await writeFile(join(tempDir, '.gitignore'), 'docs/tbd/\n'); + + const doctor = runTbd(['doctor']); + expect(doctor.status).toBe(0); + expect(doctor.stdout).toContain( + '⚠ Fork dir - docs/tbd/ is gitignored — forked docs will not be committed', + ); + expect(doctor.stdout).not.toContain('tracked in git (not gitignored)'); + }); + + it('totally unparseable forks.yml is reported, not crashed on', async () => { + expect(runTbd(['docs', 'fork', 'python-rules']).status).toBe(0); + await writeFile(join(tempDir, '.tbd', 'doc-forks', 'forks.yml'), '{{{{not yaml: [\n'); + + const doctor = runTbd(['doctor']); + expect(doctor.status).toBe(0); + expect(doctor.stdout).toContain('⚠ Forked docs - fork manifest unreadable:'); + expect(doctor.stdout).toContain('(.tbd/doc-forks/forks.yml)'); + expect(doctor.stdout).toContain( + 'Fix or delete .tbd/doc-forks/forks.yml (forked files stay in place), then re-run tbd doctor', + ); + }); +}); From a4da660a12f724395c6db5d170c627956f4c9402 Mon Sep 17 00:00:00 2001 From: Test User Date: Fri, 12 Jun 2026 21:35:09 +0000 Subject: [PATCH 33/36] feat: setup --auto Docs summary + remove unused --interactive flag (tbd-f8bu) Phase 4 item 16 of the forkable-docs spec, setup side: - tbd setup --auto now ends its summary with a Docs section before the completion banner. Zero forks: the three-posture menu (shared wording with the bare 'tbd docs' overview via new cli/lib/docs-menu.ts) under a 'Docs: N docs available in the cache' lead line. Forks present: one line with the fork count and, when any fork is stale, the pending upstream-update count and a 'tbd docs update' nudge. Reporting only - setup never writes the fork dir. Suppressed under --quiet/--json. - Remove the --interactive flag (never had prompts; agents are the operators). Commander option, help text, and doc comments dropped; 'tbd setup --interactive' now fails with unknown option. - Goldens: extend golden-output post-setup block and setup-flows with the zero-fork menu and fork/stale one-liners; drop --interactive from the pinned 'tbd setup --help' block in cli-setup-commands.tryscript.md. https://claude.ai/code/session_01QPsCSYGtwR8JtX2R1aaxyh --- packages/tbd/src/cli/commands/setup.ts | 78 ++++++++++++++++--- packages/tbd/src/cli/lib/docs-menu.ts | 29 +++++++ .../tbd/tests/cli-setup-commands.tryscript.md | 1 - packages/tbd/tests/golden-output.test.ts | 15 ++++ packages/tbd/tests/setup-flows.test.ts | 69 +++++++++++++++- 5 files changed, 178 insertions(+), 14 deletions(-) create mode 100644 packages/tbd/src/cli/lib/docs-menu.ts diff --git a/packages/tbd/src/cli/commands/setup.ts b/packages/tbd/src/cli/commands/setup.ts index b03c915c..3dd2cc30 100644 --- a/packages/tbd/src/cli/commands/setup.ts +++ b/packages/tbd/src/cli/commands/setup.ts @@ -7,7 +7,6 @@ * * Options: * - `tbd setup --auto` - Non-interactive setup (for agents/scripts) - * - `tbd setup --interactive` - Interactive setup with prompts (for humans) * - `tbd setup --from-beads` - Migrate from Beads to tbd * * See: tbd-design.md §6.4.2 Claude Code Integration @@ -46,6 +45,9 @@ import { DATA_SYNC_DIR_NAME, DEFAULT_SHORTCUT_PATHS, DEFAULT_GUIDELINES_PATHS, + DEFAULT_TEMPLATE_PATHS, + DEFAULT_REFERENCE_PATHS, + FORK_DIR, TBD_SHORTCUTS_SYSTEM, TBD_SHORTCUTS_STANDARD, TBD_GUIDELINES_DIR, @@ -61,6 +63,9 @@ import { AGENT_INTEGRATION_FORMAT, } from '../../lib/integration-paths.js'; import { initWorktree, isInGitRepo, findGitRoot, checkWorktreeHealth } from '../../file/git.js'; +import { readForkManifest } from '../../file/fork-manifest.js'; +import { computeForkDriftSummary } from '../../file/doc-fork.js'; +import { docsPostureMenuLines } from '../lib/docs-menu.js'; import { DocCache, generateShortcutDirectory } from '../../file/doc-cache.js'; import { withSharedDataSyncLock, writeCommonDirLayout } from '../../file/common-dir-layout.js'; import { withDataSyncContext } from '../lib/data-context.js'; @@ -1238,12 +1243,11 @@ class SetupCodexHandler extends BaseCommand { } // ============================================================================ -// Setup Default Handler (for --auto and --interactive modes) +// Setup Default Handler (for --auto mode) // ============================================================================ interface SetupDefaultOptions { auto?: boolean; - interactive?: boolean; fromBeads?: boolean; prefix?: string; force?: boolean; @@ -1251,11 +1255,10 @@ interface SetupDefaultOptions { } /** - * Default handler for `tbd setup` with --auto or --interactive flags. + * Default handler for `tbd setup --auto`. * * This implements the unified onboarding flow: * - `tbd setup --auto`: Non-interactive setup with smart defaults (for agents) - * - `tbd setup --interactive`: Interactive setup with prompts (for humans) * * Decision tree: * 1. Not in git repo → Error (git init first) @@ -1278,7 +1281,6 @@ class SetupDefaultHandler extends BaseCommand { // Determine mode const isAutoMode = options.auto === true; - // Note: options.interactive will be used when we add interactive prompts // Header console.log(colors.bold('tbd: Git-native issue tracking for AI agents and humans')); @@ -1938,6 +1940,61 @@ class SetupAutoHandler extends BaseCommand { console.log(colors.warn(` ! ${r.name}: ${r.error}`)); } } + + // Docs summary: serving posture and pending upstream updates (read-only). + await this.printDocsSummary(cwd); + } + + /** + * Print the Docs summary after the integration sections: how many docs tbd + * serves from the gitignored cache, or — when docs are forked into the repo — + * the fork count plus any pending upstream updates. Reporting only: setup + * never writes the fork dir (only `tbd docs update` modifies forked docs). + * + * Zero-fork wording is shared with the bare `tbd docs` overview via + * {@link docsPostureMenuLines} so the two menus cannot drift. + */ + private async printDocsSummary(cwd: string): Promise { + // Suppressed like the other setup summary sections. + if (this.ctx.quiet || this.ctx.json) { + return; + } + + let lines: string[]; + try { + const manifest = await readForkManifest(cwd); + const drift = await computeForkDriftSummary(cwd, FORK_DIR, manifest); + + if (drift.forks > 0) { + const updates = + drift.stale > 0 ? ` ${drift.stale} have upstream updates — run 'tbd docs update'.` : ''; + lines = [`Docs: ${drift.forks} forked into ${FORK_DIR}/.${updates}`]; + } else { + let total = 0; + for (const paths of [ + DEFAULT_GUIDELINES_PATHS, + DEFAULT_SHORTCUT_PATHS, + DEFAULT_TEMPLATE_PATHS, + DEFAULT_REFERENCE_PATHS, + ]) { + const cache = new DocCache(paths, cwd); + await cache.load({ quiet: true }); + total += cache.list().length; + } + lines = [ + `Docs: ${total} docs available in the cache (.tbd/docs/, gitignored); none forked into the repo.`, + ...docsPostureMenuLines().map((line) => ` ${line}`), + ]; + } + } catch { + // Reporting only — never fail setup because the summary could not be read. + return; + } + + console.log(''); + for (const line of lines) { + console.log(line); + } } /** @@ -2179,7 +2236,6 @@ class SetupAutoHandler extends BaseCommand { export const setupCommand = new Command('setup') .description('Configure tbd integration with editors and tools') .option('--auto', 'Non-interactive mode with smart defaults (for agents/scripts)') - .option('--interactive', 'Interactive mode with prompts (for humans)') .option('--from-beads', 'Migrate from Beads to tbd') .option('--prefix ', 'Project prefix for issue IDs (required for fresh setup)') .option('--force', 'Allow non-recommended prefix format (not 2-8 alphabetic)') @@ -2189,14 +2245,14 @@ export const setupCommand = new Command('setup') 'Comma-separated agent surfaces to install: portable, agents-md, claude, codex (or "all"). Default: all', ) .action(async (options: SetupDefaultOptions, command) => { - // If --auto or --interactive flag is set, run the default handler - if (options.auto || options.interactive) { + // If --auto flag is set, run the default handler + if (options.auto) { const handler = new SetupDefaultHandler(command); await handler.run(options); return; } - // If --from-beads is set without --auto/--interactive, treat as --auto + // If --from-beads is set without --auto, treat as --auto if (options.fromBeads) { const handler = new SetupDefaultHandler(command); await handler.run({ ...options, auto: true }); @@ -2214,7 +2270,6 @@ export const setupCommand = new Command('setup') console.log( ' --auto Non-interactive mode with smart defaults (for agents/scripts)', ); - console.log(' --interactive Interactive mode with prompts (for humans)'); console.log(' --from-beads Migrate from Beads to tbd (implies --auto)'); console.log(''); console.log('Options:'); @@ -2228,7 +2283,6 @@ export const setupCommand = new Command('setup') console.log('Examples:'); console.log(' tbd setup --auto --prefix=tbd # Full automatic setup with prefix'); console.log(' tbd setup --from-beads # Migrate from Beads (uses beads prefix)'); - console.log(' tbd setup --interactive # Interactive setup with prompts'); console.log(''); console.log('For surgical initialization without integrations, see: tbd init --help'); }); diff --git a/packages/tbd/src/cli/lib/docs-menu.ts b/packages/tbd/src/cli/lib/docs-menu.ts new file mode 100644 index 00000000..e576b6e3 --- /dev/null +++ b/packages/tbd/src/cli/lib/docs-menu.ts @@ -0,0 +1,29 @@ +/** + * Shared docs-posture menu for the zero-forks state. + * + * The same "three postures" menu is shown by `tbd setup --auto` (the Docs + * summary) and by the bare `tbd docs` overview, so the wording lives in one + * place and the two surfaces cannot drift. Lines are returned unindented and + * uncolored; callers add their own indentation and formatting. + * + * Contract note (forkable-docs spec): the menu must only name selectors that + * exist — when `tbd docs fork --category` ships, the Curated line gains the + * category form here, in one place. + */ + +import { FORK_DIR } from '../../lib/paths.js'; + +/** + * Menu body lines for the zero-forks state: the three serving postures plus + * the browse/read pointer. Shown under a lead line reporting the count of + * docs available in the cache. + */ +export function docsPostureMenuLines(): string[] { + return [ + 'Guidelines are active from the cache. Three postures, all serving the same docs:', + 'Hidden (default): keep the cache as-is — zero repo footprint', + `Curated: tbd docs fork [...] fork chosen docs into ${FORK_DIR}/`, + 'Everything: tbd docs fork --all all docs, visible and editable', + 'Browse / read: tbd docs list / tbd docs show ', + ]; +} diff --git a/packages/tbd/tests/cli-setup-commands.tryscript.md b/packages/tbd/tests/cli-setup-commands.tryscript.md index 9b582fa0..195ac9a5 100644 --- a/packages/tbd/tests/cli-setup-commands.tryscript.md +++ b/packages/tbd/tests/cli-setup-commands.tryscript.md @@ -39,7 +39,6 @@ Configure tbd integration with editors and tools Options: --auto Non-interactive mode with smart defaults (for agents/scripts) - --interactive Interactive mode with prompts (for humans) --from-beads Migrate from Beads to tbd --prefix Project prefix for issue IDs (required for fresh setup) --force Allow non-recommended prefix format (not 2-8 alphabetic) diff --git a/packages/tbd/tests/golden-output.test.ts b/packages/tbd/tests/golden-output.test.ts index 2abf6b1d..82767eca 100644 --- a/packages/tbd/tests/golden-output.test.ts +++ b/packages/tbd/tests/golden-output.test.ts @@ -90,6 +90,21 @@ describe('golden output tests', { timeout: isWindows ? 60000 : 15000 }, () => { expect(result.status).toBe(0); + // The Docs summary precedes the completion banner: the zero-fork + // three-posture menu, sharing wording with the bare `tbd docs` overview + // (count masked — the bundled-doc inventory grows over time). + const masked = result.stdout.replace(/Docs: \d+ docs available/, 'Docs: [N] docs available'); + expect(masked).toContain( + [ + 'Docs: [N] docs available in the cache (.tbd/docs/, gitignored); none forked into the repo.', + ' Guidelines are active from the cache. Three postures, all serving the same docs:', + ' Hidden (default): keep the cache as-is — zero repo footprint', + ' Curated: tbd docs fork [...] fork chosen docs into docs/tbd/', + ' Everything: tbd docs fork --all all docs, visible and editable', + ' Browse / read: tbd docs list / tbd docs show ', + ].join('\n'), + ); + // Verify What's Next section uses "what you can say" framing expect(result.stdout).toContain("WHAT'S NEXT"); expect(result.stdout).toContain('Try saying things like:'); diff --git a/packages/tbd/tests/setup-flows.test.ts b/packages/tbd/tests/setup-flows.test.ts index f1d982a6..01ea2648 100644 --- a/packages/tbd/tests/setup-flows.test.ts +++ b/packages/tbd/tests/setup-flows.test.ts @@ -62,7 +62,17 @@ describe('setup flows', { timeout: setupFlowTestTimeout }, () => { const result = runTbd(['setup']); expect(result.stdout).toContain('--auto'); - expect(result.stdout).toContain('--interactive'); + expect(result.stdout).toContain('--from-beads'); + // The --interactive flag was removed (never had prompts; agents are the operators). + expect(result.stdout).not.toContain('--interactive'); + }); + + it('rejects the removed --interactive flag', () => { + initGitRepo(); + const result = runTbd(['setup', '--interactive']); + + expect(result.status).not.toBe(0); + expect(result.stderr).toContain("unknown option '--interactive'"); }); }); @@ -351,6 +361,63 @@ describe('setup flows', { timeout: setupFlowTestTimeout }, () => { }); }); + describe('docs summary', () => { + // The zero-fork menu, verbatim. Wording is shared with the bare `tbd docs` + // overview (src/cli/lib/docs-menu.ts); the count is masked because the + // bundled-doc inventory grows over time. + const zeroForkMenu = [ + 'Docs: [N] docs available in the cache (.tbd/docs/, gitignored); none forked into the repo.', + ' Guidelines are active from the cache. Three postures, all serving the same docs:', + ' Hidden (default): keep the cache as-is — zero repo footprint', + ' Curated: tbd docs fork [...] fork chosen docs into docs/tbd/', + ' Everything: tbd docs fork --all all docs, visible and editable', + ' Browse / read: tbd docs list / tbd docs show ', + ].join('\n'); + + it('shows the three-posture menu when nothing is forked', () => { + initGitRepo(); + + const result = runTbd(['setup', '--auto', '--prefix=test']); + expect(result.status).toBe(0); + + const masked = result.stdout.replace(/Docs: \d+ docs available/, 'Docs: [N] docs available'); + expect(masked).toContain(zeroForkMenu); + }); + + it('reports fork count and pending updates instead of the menu', async () => { + initGitRepo(); + runTbd(['setup', '--auto', '--prefix=test']); + + const forkResult = runTbd(['docs', 'fork', 'python-rules']); + expect(forkResult.status).toBe(0); + + // Forks current: one line, no update nudge, no posture menu. + const current = runTbd(['setup', '--auto']); + expect(current.status).toBe(0); + expect(current.stdout).toContain('Docs: 1 forked into docs/tbd/.'); + expect(current.stdout).not.toContain('upstream updates'); + expect(current.stdout).not.toContain('Three postures'); + + // Mark the fork stale (as after a tbd upgrade): its recorded base no + // longer matches the cache content. + const manifestPath = join(tempDir, '.tbd', 'doc-forks', 'forks.yml'); + const manifest = await readFile(manifestPath, 'utf-8'); + await writeFile( + manifestPath, + manifest.replace(/base_hash: sha256:[0-9a-f]+/, `base_hash: sha256:${'0'.repeat(64)}`), + ); + + const stale = runTbd(['setup', '--auto']); + expect(stale.status).toBe(0); + expect(stale.stdout).toContain( + "Docs: 1 forked into docs/tbd/. 1 have upstream updates — run 'tbd docs update'.", + ); + + // Reporting only: setup must never write the fork dir or its manifest. + expect(await readFile(manifestPath, 'utf-8')).toContain(`sha256:${'0'.repeat(64)}`); + }); + }); + describe('beads migration', { timeout: isWindows ? 60000 : 15000 }, () => { it('detects beads and offers migration', async () => { initGitRepo(); From 3406821758cddde0aece2baea4311037c9fa5da3 Mon Sep 17 00:00:00 2001 From: Test User Date: Fri, 12 Jun 2026 22:06:15 +0000 Subject: [PATCH 34/36] =?UTF-8?q?feat:=20Complete=20the=20forkable-docs=20?= =?UTF-8?q?experience=20=E2=80=94=20Phases=202-5=20shipped?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Consolidates four parallel workstreams plus the mainline data-plane work; every numbered item in the spec's Phase 2-5 plan is now implemented, and the spec's golden maps carry captured output with no remaining phase-contract annotations for shipped behavior. Data plane (tbd-ohkj): - `tbd docs add ` unifies the per-kind --add flags (kept as aliases): docref normalization replaces the ad-hoc blob-URL conversion, the canonical docref is what config records, git docrefs require an explicit @ref, and local docrefs work offline. `reference` joins the addable kinds. - `docs_cache.local_dirs` serves additional in-repo doc directories between the fork dir and the cache: first-class for list/show/per-kind reading with state `local` and a "(serving local doc: …)" note; not forkable (no upstream). Fixed the cache refresh dropping sibling docs_cache keys on config rewrite, and an effectiveServePaths double-sanitize that silently emptied the local-dir list. - Sync groups source entries per git repo+ref with per-group failure isolation (one unreachable source skips its own entries instead of timing out per file), one best-effort `git ls-remote` revision capture per group for provenance, and never prunes cached copies on fetch failure. Doctor (tbd-5xt0, folding tbd-nt2c): a "Forked docs" health-check group — corrupt forks.yml, missing forked files (--fix finalizes the unfork), orphaned entries (--fix cleans), base snapshot missing/mismatch, unresolved conflict markers, reserved tbd-* names, gitignored fork dir — silent for non-users, with 9 e2e tests. Categories (tbd-jme1): every bundled guideline declares exactly one category in frontmatter (enforced by a new unit test); the name-based inferGuidelineCategory inference is retired (the old 'testing' value errors clearly); `tbd docs fork --category=` (repeatable) selects by the declared field, with the dry-run summary naming the categories. Setup (tbd-f8bu): the unused --interactive flag is removed everywhere (including the stale help epilog and its six pinned tryscripts); setup --auto prints the Docs summary — the shared three-posture menu (docs-menu.ts, also adopted by the bare overview, now including the --category line since the flag exists) for zero forks, and the pending-update one-liner when forks exist. Setup never writes the fork dir. Experience docs (tbd-lxab, tbd-e12s, tbd-5bvs): the suggest-upstream-improvements playbook shortcut; two-axis (scope × visibility) welcome-user onboarding; README forkable-docs paragraph and managed-docs pointers; development.md path-conventions and testing pointers; CHANGELOG f05 entry per release-notes-guidelines; tbd-docs.md Configuration Reference documents docrefs, shipped local_dirs, and the still-reserved fork_dir. Goldens: per-kind frontmatter block, category filter semantics, setup menus, overview snapshot, and the spec's golden maps recaptured from the built CLI (fork --category dry-run, docs add incl. the offline local form, status Docs line, setup summaries, doctor excerpts). isLayoutUpgradeable left as designed after re-verification against the H3 test (tbd-m72a: F3 verified safe earlier, S9 is correct behavior, F7 intentionally permissive for derived metadata). Full suite: 1,311 unit tests + 881 tryscript blocks green. https://claude.ai/code/session_01QPsCSYGtwR8JtX2R1aaxyh --- .agents/skills/tbd/SKILL.md | 1 + .claude/skills/tbd/SKILL.md | 1 + .tbd/config.yml | 7 +- README.md | 2 +- docs/docs-overview.md | 4 +- .../active/plan-2026-06-11-forkable-docs.md | 64 ++++-- .../guidelines/cli-agent-skill-patterns.md | 2 +- packages/tbd/docs/tbd-docs.md | 15 +- packages/tbd/docs/tbd-prime.md | 1 - packages/tbd/src/cli/commands/docs-fork.ts | 56 +++-- packages/tbd/src/cli/commands/docs.ts | 150 +++++++++---- packages/tbd/src/cli/commands/guidelines.ts | 54 +---- .../tbd/src/cli/lib/doc-command-handler.ts | 32 ++- packages/tbd/src/cli/lib/doc-serve.ts | 72 +++++- packages/tbd/src/cli/lib/docs-menu.ts | 5 +- packages/tbd/src/cli/lib/output.ts | 2 +- packages/tbd/src/file/doc-add.ts | 48 +++- packages/tbd/src/file/doc-sync.ts | 205 ++++++++++++++---- packages/tbd/src/file/github-fetch.ts | 18 ++ packages/tbd/src/lib/doc-categories.ts | 25 +++ packages/tbd/src/lib/schemas.ts | 7 + packages/tbd/tests/cli-advanced.tryscript.md | 8 +- packages/tbd/tests/cli-closing.tryscript.md | 2 +- .../tbd/tests/cli-doc-output.tryscript.md | 2 +- packages/tbd/tests/cli-prime.tryscript.md | 2 +- .../tbd/tests/cli-setup-commands.tryscript.md | 2 +- packages/tbd/tests/cli-setup.tryscript.md | 4 +- packages/tbd/tests/cli-uninstall.tryscript.md | 2 +- .../tbd/tests/docs-add-local-dirs-e2e.test.ts | 153 +++++++++++++ packages/tbd/tests/golden-output.test.ts | 18 +- packages/tbd/tests/setup-flows.test.ts | 1 + 31 files changed, 746 insertions(+), 219 deletions(-) create mode 100644 packages/tbd/src/lib/doc-categories.ts create mode 100644 packages/tbd/tests/docs-add-local-dirs-e2e.test.ts diff --git a/.agents/skills/tbd/SKILL.md b/.agents/skills/tbd/SKILL.md index 5bbf7e0d..789922a1 100644 --- a/.agents/skills/tbd/SKILL.md +++ b/.agents/skills/tbd/SKILL.md @@ -230,6 +230,7 @@ Run `tbd shortcut ` to use any of these shortcuts: | revise-all-architecture-docs | Comprehensive revision of all current architecture documents | | revise-architecture-doc | Update an architecture document to reflect current codebase state | | setup-github-cli | Ensure GitHub CLI (gh) is installed and working | +| suggest-upstream-improvements | Review local doc-fork customizations and contribute the generally useful changes back upstream | | sync-failure-recovery | Handle tbd sync failures by saving to workspace and recovering later | | update-specs-status | Review active specs and sync their status with tbd issues | | welcome-user | Welcome message for users after tbd installation or setup | diff --git a/.claude/skills/tbd/SKILL.md b/.claude/skills/tbd/SKILL.md index 5bbf7e0d..789922a1 100644 --- a/.claude/skills/tbd/SKILL.md +++ b/.claude/skills/tbd/SKILL.md @@ -230,6 +230,7 @@ Run `tbd shortcut ` to use any of these shortcuts: | revise-all-architecture-docs | Comprehensive revision of all current architecture documents | | revise-architecture-doc | Update an architecture document to reflect current codebase state | | setup-github-cli | Ensure GitHub CLI (gh) is installed and working | +| suggest-upstream-improvements | Review local doc-fork customizations and contribute the generally useful changes back upstream | | sync-failure-recovery | Handle tbd sync failures by saving to workspace and recovering later | | update-specs-status | Review active specs and sync their status with tbd issues | | welcome-user | Welcome message for users after tbd installation or setup | diff --git a/.tbd/config.yml b/.tbd/config.yml index 5d7033b0..f686c5c9 100644 --- a/.tbd/config.yml +++ b/.tbd/config.yml @@ -23,9 +23,6 @@ settings: # Auto-sync: Docs are automatically synced when stale (default: every 24 hours). # Configure with settings.doc_auto_sync_hours (0 = disabled). docs_cache: - lookup_path: - - .tbd/docs/shortcuts/system - - .tbd/docs/shortcuts/standard files: references/tbd-docs.md: internal:tbd-docs.md references/tbd-design.md: internal:tbd-design.md @@ -60,6 +57,7 @@ docs_cache: shortcuts/standard/revise-all-architecture-docs.md: internal:shortcuts/standard/revise-all-architecture-docs.md shortcuts/standard/revise-architecture-doc.md: internal:shortcuts/standard/revise-architecture-doc.md shortcuts/standard/setup-github-cli.md: internal:shortcuts/standard/setup-github-cli.md + shortcuts/standard/suggest-upstream-improvements.md: internal:shortcuts/standard/suggest-upstream-improvements.md shortcuts/standard/sync-failure-recovery.md: internal:shortcuts/standard/sync-failure-recovery.md shortcuts/standard/update-specs-status.md: internal:shortcuts/standard/update-specs-status.md shortcuts/standard/welcome-user.md: internal:shortcuts/standard/welcome-user.md @@ -96,3 +94,6 @@ docs_cache: templates/research-brief.md: internal:templates/research-brief.md references/docmap-format.md: internal:references/docmap-format.md references/docref-format.md: internal:references/docref-format.md + lookup_path: + - .tbd/docs/shortcuts/system + - .tbd/docs/shortcuts/standard diff --git a/README.md b/README.md index ce7ba98e..77a6a139 100644 --- a/README.md +++ b/README.md @@ -415,7 +415,7 @@ tbd template --list # List all templates tbd template plan-spec # Get a plan spec template # Add your own from any URL -# (per-kind aliases for the upcoming unified `tbd docs add `) +# (per-kind aliases for `tbd docs add `) tbd guidelines --add= --name= tbd shortcut --add= --name= tbd template --add= --name= diff --git a/docs/docs-overview.md b/docs/docs-overview.md index f90d395c..bb4357c0 100644 --- a/docs/docs-overview.md +++ b/docs/docs-overview.md @@ -83,8 +83,8 @@ GitHub blob URLs are automatically converted to raw URLs. If direct fetch returns HTTP 403, the system falls back to `gh api` for authenticated access. User-added shortcuts are stored in `shortcuts/custom/` to keep them separate from -bundled docs. (A unified `tbd docs add ` form is planned; the per-kind flags -remain as aliases.) +bundled docs. The unified form is `tbd docs add ` (the per-kind flags remain as +aliases), and `docs_cache.local_dirs` can serve additional in-repo doc directories.