diff --git a/.agents/skills/tbd/SKILL.md b/.agents/skills/tbd/SKILL.md index a1f8bb5c..789922a1 100644 --- a/.agents/skills/tbd/SKILL.md +++ b/.agents/skills/tbd/SKILL.md @@ -6,7 +6,7 @@ description: |- Invoke when user mentions: tbd, beads, bd, shortcuts, issues, bugs, tasks, features, epics, todo, tracking, specs, planning, implementation, validation, guidelines, templates, commit, PR, pull request, code review, testing, TDD, test-driven, golden testing, snapshot testing, TypeScript, Python, Convex, monorepo, cleanup, dead code, refactor, handoff, research, architecture, labels, search, checkout library, source code review, or any workflow shortcut. name: tbd --- - @@ -25,12 +25,15 @@ Run 'tbd setup' to update. ## Installation ```bash -npm install -g get-tbd@latest +npm install -g get-tbd@latest # Install or upgrade the CLI (same command for both) tbd setup --auto --prefix= # Fresh project (--prefix is REQUIRED: 2-8 alphabetic chars recommended. ALWAYS ASK THE USER FOR THE PREFIX; do not guess it) -tbd setup --auto # Existing tbd project (prefix already set) +tbd setup --auto # Existing tbd project — also the upgrade step (applies any format migration; commit the diff it reports) tbd setup --from-beads # Migration from .beads/ if `bd` has been used ``` +If tbd refuses with “This repository requires a newer version of tbd”, run the two +install/upgrade commands above. + ## Routine Commands ```bash @@ -90,6 +93,10 @@ or want help → run `tbd shortcut welcome-user` | **Documentation** | | | “Research this topic” | `tbd shortcut new-research-brief` | | “Document architecture” | `tbd shortcut new-architecture-doc` | +| “What guidelines/docs are there?” | `tbd docs list` | +| “Make the guidelines visible / customize doc X” | `tbd docs fork ` (or `--all`), then edit in `docs/tbd/` | +| “Update the guidelines to the latest” | `tbd docs update`; on conflicts ask the user, then `--merge` or `--keep-ours` | +| “I deleted a forked doc file” | `tbd docs status` shows it `missing`; restore with `tbd docs fork --force` or finalize with `tbd docs unfork ` | | **Cleanup & Maintenance** | | | “Clean up this code” / “Remove dead code” | `tbd shortcut code-cleanup-all` | | “Fix repository problems” | `tbd doctor --fix` | @@ -179,6 +186,8 @@ working branch. See `tbd guidelines tbd-sync-troubleshooting` for details. | `tbd guidelines ` | Load coding guidelines | | `tbd guidelines --list` | List guidelines | | `tbd template ` | Output a template | +| `tbd docs` / `tbd docs list` | Managed-docs overview / cross-kind list with state markers | +| `tbd docs fork/unfork/update ` | Fork docs into `docs/tbd/`, return to upstream, pull upstream updates | ## Quick Reference @@ -221,6 +230,7 @@ Run `tbd shortcut ` to use any of these shortcuts: | revise-all-architecture-docs | Comprehensive revision of all current architecture documents | | revise-architecture-doc | Update an architecture document to reflect current codebase state | | setup-github-cli | Ensure GitHub CLI (gh) is installed and working | +| suggest-upstream-improvements | Review local doc-fork customizations and contribute the generally useful changes back upstream | | sync-failure-recovery | Handle tbd sync failures by saving to workspace and recovering later | | update-specs-status | Review active specs and sync their status with tbd issues | | welcome-user | Welcome message for users after tbd installation or setup | diff --git a/.claude/skills/tbd/SKILL.md b/.claude/skills/tbd/SKILL.md index a1f8bb5c..789922a1 100644 --- a/.claude/skills/tbd/SKILL.md +++ b/.claude/skills/tbd/SKILL.md @@ -6,7 +6,7 @@ description: |- Invoke when user mentions: tbd, beads, bd, shortcuts, issues, bugs, tasks, features, epics, todo, tracking, specs, planning, implementation, validation, guidelines, templates, commit, PR, pull request, code review, testing, TDD, test-driven, golden testing, snapshot testing, TypeScript, Python, Convex, monorepo, cleanup, dead code, refactor, handoff, research, architecture, labels, search, checkout library, source code review, or any workflow shortcut. name: tbd --- - @@ -25,12 +25,15 @@ Run 'tbd setup' to update. ## Installation ```bash -npm install -g get-tbd@latest +npm install -g get-tbd@latest # Install or upgrade the CLI (same command for both) tbd setup --auto --prefix= # Fresh project (--prefix is REQUIRED: 2-8 alphabetic chars recommended. ALWAYS ASK THE USER FOR THE PREFIX; do not guess it) -tbd setup --auto # Existing tbd project (prefix already set) +tbd setup --auto # Existing tbd project — also the upgrade step (applies any format migration; commit the diff it reports) tbd setup --from-beads # Migration from .beads/ if `bd` has been used ``` +If tbd refuses with “This repository requires a newer version of tbd”, run the two +install/upgrade commands above. + ## Routine Commands ```bash @@ -90,6 +93,10 @@ or want help → run `tbd shortcut welcome-user` | **Documentation** | | | “Research this topic” | `tbd shortcut new-research-brief` | | “Document architecture” | `tbd shortcut new-architecture-doc` | +| “What guidelines/docs are there?” | `tbd docs list` | +| “Make the guidelines visible / customize doc X” | `tbd docs fork ` (or `--all`), then edit in `docs/tbd/` | +| “Update the guidelines to the latest” | `tbd docs update`; on conflicts ask the user, then `--merge` or `--keep-ours` | +| “I deleted a forked doc file” | `tbd docs status` shows it `missing`; restore with `tbd docs fork --force` or finalize with `tbd docs unfork ` | | **Cleanup & Maintenance** | | | “Clean up this code” / “Remove dead code” | `tbd shortcut code-cleanup-all` | | “Fix repository problems” | `tbd doctor --fix` | @@ -179,6 +186,8 @@ working branch. See `tbd guidelines tbd-sync-troubleshooting` for details. | `tbd guidelines ` | Load coding guidelines | | `tbd guidelines --list` | List guidelines | | `tbd template ` | Output a template | +| `tbd docs` / `tbd docs list` | Managed-docs overview / cross-kind list with state markers | +| `tbd docs fork/unfork/update ` | Fork docs into `docs/tbd/`, return to upstream, pull upstream updates | ## Quick Reference @@ -221,6 +230,7 @@ Run `tbd shortcut ` to use any of these shortcuts: | revise-all-architecture-docs | Comprehensive revision of all current architecture documents | | revise-architecture-doc | Update an architecture document to reflect current codebase state | | setup-github-cli | Ensure GitHub CLI (gh) is installed and working | +| suggest-upstream-improvements | Review local doc-fork customizations and contribute the generally useful changes back upstream | | sync-failure-recovery | Handle tbd sync failures by saving to workspace and recovering later | | update-specs-status | Review active specs and sync their status with tbd issues | | welcome-user | Welcome message for users after tbd installation or setup | diff --git a/.tbd/config.yml b/.tbd/config.yml index cb7a36d2..f686c5c9 100644 --- a/.tbd/config.yml +++ b/.tbd/config.yml @@ -1,4 +1,4 @@ -tbd_format: f04 +tbd_format: f05 tbd_version: development display: id_prefix: tbd @@ -23,10 +23,9 @@ settings: # Auto-sync: Docs are automatically synced when stale (default: every 24 hours). # Configure with settings.doc_auto_sync_hours (0 = disabled). docs_cache: - lookup_path: - - .tbd/docs/shortcuts/system - - .tbd/docs/shortcuts/standard files: + references/tbd-docs.md: internal:tbd-docs.md + references/tbd-design.md: internal:tbd-design.md shortcuts/system/shortcut-explanation.md: internal:shortcuts/system/shortcut-explanation.md shortcuts/system/skill-baseline.md: internal:shortcuts/system/skill-baseline.md shortcuts/system/skill-brief.md: internal:shortcuts/system/skill-brief.md @@ -58,6 +57,7 @@ docs_cache: shortcuts/standard/revise-all-architecture-docs.md: internal:shortcuts/standard/revise-all-architecture-docs.md shortcuts/standard/revise-architecture-doc.md: internal:shortcuts/standard/revise-architecture-doc.md shortcuts/standard/setup-github-cli.md: internal:shortcuts/standard/setup-github-cli.md + shortcuts/standard/suggest-upstream-improvements.md: internal:shortcuts/standard/suggest-upstream-improvements.md shortcuts/standard/sync-failure-recovery.md: internal:shortcuts/standard/sync-failure-recovery.md shortcuts/standard/update-specs-status.md: internal:shortcuts/standard/update-specs-status.md shortcuts/standard/welcome-user.md: internal:shortcuts/standard/welcome-user.md @@ -92,3 +92,8 @@ docs_cache: templates/plan-spec.md: internal:templates/plan-spec.md templates/qa-playbook.md: internal:templates/qa-playbook.md templates/research-brief.md: internal:templates/research-brief.md + references/docmap-format.md: internal:references/docmap-format.md + references/docref-format.md: internal:references/docref-format.md + lookup_path: + - .tbd/docs/shortcuts/system + - .tbd/docs/shortcuts/standard diff --git a/AGENTS.md b/AGENTS.md index f43c0321..b290ce1b 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -33,7 +33,7 @@ Work is NOT complete until `git push` succeeds. - NEVER say “ready to push when you are” - YOU must push - If push fails, resolve and retry until it succeeds - + ## tbd This repository uses **tbd** for git-native issue tracking (beads), spec-driven diff --git a/README.md b/README.md index 67f9af6c..77a6a139 100644 --- a/README.md +++ b/README.md @@ -37,7 +37,6 @@ will work in Cursor, Codex, or any agent environment that can use the `tbd` CLI. ## Quick Start > [!TIP] -> > If running on your own machine, install the `tbd` CLI yourself: > > **`npm install -g get-tbd@latest`** @@ -50,6 +49,11 @@ will work in Cursor, Codex, or any agent environment that can use the `tbd` CLI. > > ***“install tbd (npm install -g get-tbd@latest) and run tbd prime for instructions to > set up this project”*** +> +> If tbd is already set up in the repo and you want the latest version, tell the agent: +> +> ***“upgrade tbd (npm install -g get-tbd@latest), run tbd setup --auto, and commit the +> changes”*** That’s it. Running `tbd prime` gives agents full workflow context on how to use `tbd` and @@ -122,7 +126,6 @@ agents handling different aspects that I manage) is slower, because it forces yo design, but it gives higher quality results. > [!NOTE] -> > We use *Beads* (capitalized) to refer to Steve Yegge’s original > [`bd` tool](https://github.com/steveyegge/beads). > Lowercase “beads” refers generically to the issues stored in `tbd` or `bd`. @@ -160,8 +163,8 @@ You just talk naturally. > [!NOTE] > For full technical details, see the [reference docs](packages/tbd/docs/tbd-docs.md) -> (run `tbd docs`) or the full [design doc](packages/tbd/docs/tbd-design.md) -> (`tbd design`). +> (run `tbd docs show tbd-docs`) or the full +> [design doc](packages/tbd/docs/tbd-design.md) (`tbd design`). - **Git-native:** Beads live in your repo, synced to a separate, dedicated `tbd-sync` branch. Your code history stays clean—no bead churn polluting your logs. @@ -225,7 +228,6 @@ practices. These aren’t generic tips; they’re mostly my own detailed and som opinionated rules with concrete examples, built from months of heavy agentic coding. > [!TIP] -> > An example: I *strongly* believe there are much better ways to do testing > proliferating hundreds of unit and integration tests. > So (with help from some Opus 4.5 and GPT-5 Pro) I wrote a multi-page brief about @@ -292,6 +294,25 @@ tbd setup --from-beads > **Tip:** Run `tbd setup --auto` anytime to refresh skill files, hooks, and configs > with the latest shortcuts, guidelines, and templates. +### Upgrading + +Upgrading an existing installation is the same two commands, run by you or your agent: + +```bash +npm install -g get-tbd@latest # Upgrade the CLI +tbd setup --auto # Refresh skills/hooks and apply any format migration +``` + +If the new version bumps the repository format (`tbd_format` in `.tbd/config.yml`), +setup migrates it automatically and prints a notice — **commit the resulting diff** to +publish the upgrade to your team. +Teammates still on an older tbd then see “This repository requires a newer version of +tbd” until they run the same two commands. +Issue data is never touched by an upgrade, and the migration is revertible: see +“Aborting a Format Upgrade” under Troubleshooting in the CLI manual (`tbd docs manual`). +If you have forked docs in `docs/tbd/`, `tbd sync` prints a notice when their upstream +versions moved — run `tbd docs update` to merge the changes in. + ### Team Setup `tbd` is designed for teams where one person sets up the project and others join later. @@ -394,11 +415,21 @@ tbd template --list # List all templates tbd template plan-spec # Get a plan spec template # Add your own from any URL +# (per-kind aliases for `tbd docs add `) tbd guidelines --add= --name= tbd shortcut --add= --name= tbd template --add= --name= ``` +**Forkable: see them in your repo.** By default these docs are served from a hidden, +gitignored cache. Fork any of them into `docs/tbd/` and they become visible on GitHub, +reviewable in PRs, and editable in place—tbd serves your copy instead, and +`tbd docs update` merges upstream improvements into it after an upgrade: + +```bash +tbd docs fork --all # Or fork by name: tbd docs fork [...] +``` + **Available shortcuts:** | Category | Shortcut | Purpose | @@ -480,7 +511,8 @@ Every command supports these flags for automation: ```bash tbd # Full orientation and workflow guidance tbd readme # This file -tbd docs # Full CLI reference +tbd docs # Managed-docs overview (cached, forked, and local docs) +tbd docs show tbd-docs # Full CLI reference (the manual; alias: tbd docs manual) ``` Or read online: diff --git a/docs/development.md b/docs/development.md index 1dd344e5..c820650f 100644 --- a/docs/development.md +++ b/docs/development.md @@ -345,6 +345,7 @@ the full specification. │ Committed to the repo: ├── config.yml # Project configuration ├── .gitignore # Controls what's gitignored below +├── doc-forks/ # Fork manifest + base snapshots (f05; tbd-design.md §2.9) ├── workspaces/ # Persistent state (outbox, named workspaces) │ └── outbox/ # Sync failure recovery data │ @@ -354,7 +355,7 @@ the full specification. └── backups/ # Legacy local backups $GIT_COMMON_DIR/tbd/ # Shared by all linked worktrees of this repo -├── layout.yml # Common-dir layout metadata (same f04 format ID) +├── layout.yml # Common-dir layout metadata (mirrors config's tbd_format) ├── locks/ │ └── data-sync.lock/ # mkdir-based repo-scoped lock ├── backups/ # Shared migration/repair backups @@ -366,11 +367,28 @@ $GIT_COMMON_DIR/tbd/ # Shared by all linked worktrees of this rep └── meta.yml ``` +`.tbd/doc-forks/` is committed and holds only fork *tracking state*: the `forks.yml` +manifest plus `base/` snapshots that `tbd docs update` three-way merges against. +The doc **fork dir** itself lives deliberately *outside* `.tbd/` — default `docs/tbd/`, +tracked in git like any other docs. + **CRITICAL**: Issues must be written to the **worktree path** (`$GIT_COMMON_DIR/tbd/data-sync-worktree/.tbd/data-sync/issues/`), NOT the direct path (`.tbd/data-sync/issues/`). The direct path is gitignored and exists only as a legacy diagnostic/migration location. +### Format Upgrades and Rollback + +A `tbd_format` bump writes exactly two stamps: the tracked `.tbd/config.yml` and the +machine-local `$GIT_COMMON_DIR/tbd/layout.yml` (plus, only when `tbd setup --auto` is +run, the tracked agent-surface markers). +It never touches issue data, so any upgrade can be aborted: restore the tracked files +from git and delete `layout.yml` (it regenerates from the config). +The full state inventory and abort recipe are user-facing in `tbd-docs.md` +§Troubleshooting → “Aborting a Format Upgrade”; the migrate → revert → repeat loop and +both interrupted-upgrade partial states are pinned by tests in +`tests/common-dir-layout-doctor.test.ts` (“f04 → f05 upgrade”). + ### Key Source Files - `packages/tbd/src/lib/paths.ts` - Path constants and `resolveDataSyncDir()` @@ -440,6 +458,12 @@ Run the e2e worktree scenarios: npx tryscript run tests/cli-sync-worktree-scenarios.tryscript.md ``` +### Testing Forkable Docs + +Forkable-docs behavior (fork/unfork/update/diff/status) is covered by +`tests/cli-docs-fork.tryscript.md`, `tests/cli-docs-update.tryscript.md`, and +`tests/fork-cross-platform-e2e.test.ts` (run from `packages/tbd/`). + diff --git a/docs/docs-overview.md b/docs/docs-overview.md index 720bccd5..bb4357c0 100644 --- a/docs/docs-overview.md +++ b/docs/docs-overview.md @@ -50,17 +50,24 @@ Project-specific specifications, architecture, and research docs: ### tbd CLI Documentation Commands -In addition to these repository docs, tbd provides built-in documentation via CLI: +In addition to these repository docs, tbd provides managed documentation via the +`tbd docs` group and per-kind readers: -- `tbd shortcut --list` / `tbd shortcut `—Workflow shortcuts (new-plan-spec, - code-review-and-commit, review-code-typescript, etc.) -- `tbd guidelines --list` / `tbd guidelines `—Coding guidelines (typescript-rules, - python-rules, general-tdd-guidelines, etc.) -- `tbd template --list` / `tbd template `—Document templates (plan-spec, - research-brief, architecture) +- `tbd docs`—Status overview of managed docs; `tbd docs list` shows every doc across + kinds with `[forked]`/`[customized]`/`[local]` markers +- `tbd docs show `—Read any doc by name; `tbd docs show tbd-docs` is the CLI + manual (alias `tbd docs manual`) +- `tbd shortcut ` / `tbd guidelines ` / `tbd template `—Per-kind + readers (with `--list`) +- `tbd docs sync`—Refresh the gitignored `.tbd/docs/` cache (also run by setup) -These CLI-provided docs are installed locally in `.tbd/docs/` during `tbd setup --auto` -and can be refreshed anytime by re-running setup. +#### Forking docs into the repo + +`tbd docs fork ` (or `--all`) copies managed docs into a visible, git-tracked +`docs/tbd/` folder; tbd then serves your copies everywhere it served the upstream ones. +`tbd docs update` three-way merges upstream changes into forked copies after an upgrade; +`tbd docs status` shows each doc’s state. +See “Managing Docs” in `tbd docs show tbd-docs` for the full model. #### Adding external docs by URL @@ -76,7 +83,8 @@ GitHub blob URLs are automatically converted to raw URLs. If direct fetch returns HTTP 403, the system falls back to `gh api` for authenticated access. User-added shortcuts are stored in `shortcuts/custom/` to keep them separate from -bundled docs. +bundled docs. The unified form is `tbd docs add ` (the per-kind flags remain as +aliases), and `docs_cache.local_dirs` can serve additional in-repo doc directories. diff --git a/docs/project/specs/done/plan-2026-06-11-forkable-docs.md b/docs/project/specs/done/plan-2026-06-11-forkable-docs.md new file mode 100644 index 00000000..f5b2c95a --- /dev/null +++ b/docs/project/specs/done/plan-2026-06-11-forkable-docs.md @@ -0,0 +1,1891 @@ +--- +title: Forkable Docs Workflow +description: Minimal shadcn-style fork/unfork/update workflow so built-in and external guidelines, shortcuts, and templates can be copied into the repo, customized, tracked in git, kept current via merges, and offered back upstream +author: Joshua Levy (github.com/jlevy) with LLM assistance +--- +# Feature: Forkable Docs Workflow + +**Date:** 2026-06-11 + +**Author:** Joshua Levy with LLM assistance + +**Status:** Draft + +## Overview + +tbd bundles 25+ guidelines, 30+ shortcuts, and several templates, and installs them into +a gitignored cache (`.tbd/docs/`). This works well for knowledge injection, but users +report a visibility problem: **the docs that shape their project are invisible in their +repo**. They can’t browse them on GitHub, can’t customize them, can’t check the ones +they care about into git, and don’t have a clear story for contributing improvements +back. + +PR #117 (`plan-2026-05-07-docs-config-redesign.md`) designed a complete framework for +this — ordered source lists, four source types, lockfiles, a DocGraph/DocMap split — and +surfaced five unresolved architectural questions (Q15–Q19). That redesign may still +happen, but it is too much machinery to block on. + +This spec proposes the **minimal kernel** of that vision, implementable now as a small +layout revision (format `f05`, a metadata-only migration from f04): + +1. **A `tbd docs` command group** scoped to managed docs, following tbd’s existing + noun-verb convention (`dep add`, `label add`, `attic restore`): `tbd docs fork` + copies any bundled doc (one, a category, or all of them) into a visible, git-tracked + folder in the repo (default `docs/tbd/`). Forked docs shadow the bundled copies in + all lookups, so customizing them Just Works. + `tbd sync` keeps its scope (project data); `tbd docs sync` takes over cache refresh. +2. **`tbd docs unfork`** — remove a forked copy and fall back to the upstream version, + refusing to discard customizations unless `--force` is given. +3. **`tbd docs update`** — after a tbd upgrade, pull upstream changes into forked docs: + unmodified copies refresh in place; customized copies get a git three-way merge that + applies automatically when clean; conflicting docs are listed until you choose a + resolution: `--merge` (combine, with standard conflict markers) or `--keep-ours` + (keep your version and advance the fork point). +4. **A small committed manifest plus stored merge bases** (`.tbd/doc-forks/forks.yml` + + `.tbd/doc-forks/base/`) recording, for each forked doc, exactly which upstream + content it forked from — making “customized?”, “stale vs upstream?”, and three-way + merging cheap, exact, offline operations. +5. **Agent-first setup opt-in** — no interactive prompts (agents are the operators): + `tbd setup --auto` keeps current behavior and prints a self-documenting summary of + the two choices — *scope* (all standard guidelines active, recommended, or a subset + by category) and *visibility* (leave them in the hidden cache, the “magic” path, or + fork into `docs/tbd/` for explicit, customizable, git-tracked copies) — while + `welcome-user`/the skill teach agents to offer the choice conversationally and run + fork themselves. +6. **An upstream-contribution playbook** — a bundled shortcut that walks an agent + through diffing customized docs against upstream and filing a GitHub issue on + `jlevy/tbd` with the proposed improvements. + Pure documentation, no new code. + +This is the shadcn “copy and own” model: bundled docs are the registry, fork is the copy +step, git is the fork, and a GitHub issue is the upstream PR — plus the update path +(merging upstream changes into your fork) that plain copy-and-own registries famously +lack. Everything here is forward-compatible with the larger #117 design (see +[Relationship to PR #117](#relationship-to-pr-117)). + +## Goals + +- **G1. Visibility:** A user can get any or all bundled docs as plain files in their + repo, browsable on GitHub and checked into git. +- **G2. Forkability:** A forked doc can be edited freely; tbd serves the edited version + everywhere the upstream one was served (CLI lookup, agent shortcuts). +- **G3. Safe reversal:** Unforking an unmodified doc is trivial; unforking a customized + doc is an error unless forced, so customizations are never silently lost. +- **G4. Provenance:** For every forked doc, tbd can answer: where did it come from, has + it been customized, and has upstream changed since fork. +- **G5. Setup choice, without interactivity:** Setup surfaces how visible docs can be — + via self-documenting output and agent-led conversation, never prompts; the default + remains exactly the current behavior (hidden cache). +- **G6. Category-based selection:** Docs are organized by their frontmatter `category`, + so an agent forks the general guidelines plus the categories for the repo’s languages + and frameworks — from a clear list, with no auto-detection and no hard-coded pack map. +- **G7. Upstream loop:** A documented, low-ceremony path from “I improved a guideline” + to “an issue with the diff is filed on jlevy/tbd.” +- **G8. Agent-operable:** Every step is a plain CLI command with `--json` output, and + the tbd skill routes natural requests ("I want to customize the Python guidelines") to + the right commands. +- **G9. Safe updates:** Upgrading tbd flows upstream improvements into forked docs + without losing customizations: trivial and cleanly mergeable updates apply by default, + conflicts are surfaced explicitly and merged only on opt-in, and nothing is ever + silently overwritten. +- **G10. Open location conventions:** Source addresses and the doc index follow small, + documented, tool-agnostic conventions (the docref grammar; the docmap inventory + format), shipped as bundled reference docs — so docs in locations beyond + built-in/forked have a stable addressing story from day one, shared with any future + source framework. + +## Non-Goals + +- **No data-model rework.** The f05 bump is a layout-version stamp plus new committed + artifacts; issue storage, sync, and the docs-cache format are untouched. + The larger source-framework redesign (PR #117) remains future work. +- **No multi-file external sources yet.** The docref *grammar* is adopted now for all + source addresses, single-file `tbd docs add ` generalizes today’s `--add`, and + `docs_cache.local_dirs` serves other in-repo directories — but whole-repo/bundle + sources, lockfiles, and external-source sync (“operations over docmaps”) stay in the + f06+ framework. +- **No automated upstream PRs.** The upstream loop is a playbook the agent follows with + user confirmation (filing an issue), not an automated `tbd upstream` command. +- **No interactive merge-resolution tooling.** Updates use git’s file-level three-way + merge (`git merge-file`) and standard conflict markers; resolving a conflicted doc is + ordinary editing (by the user or their agent), not a custom UI. +- **No changes to issue sync, beads, or the sync worktree.** + +## Background + +### Current behavior (f04) + +- Bundled docs live in + `packages/tbd/docs/{guidelines,shortcuts/{system,standard},templates}` and ship inside + the npm package (`dist/docs/`). +- `tbd setup --auto` / `tbd sync --docs` install them into `.tbd/docs/` via + `syncDocsWithDefaults()` (`src/file/doc-sync.ts:534`): the `docs_cache.files` map in + `.tbd/config.yml` maps destination paths to sources (`internal:` prefix or URL), + defaults are merged with user entries (user wins, `mergeDocCacheConfig()`, + `doc-sync.ts:373`), and files are overwritten on content mismatch. +- **The entire `.tbd/docs/` directory is gitignored** (`.tbd/.gitignore` entry written + by `setup.ts:1642`), which is the root of the visibility complaint. +- Name resolution goes through `DocCache` (`src/file/doc-cache.ts`): an ordered list of + directories is scanned and **the first file with a given name wins; later ones are + marked shadowed** (`doc-cache.ts:228`, `isShadowed()` at `:371`). Shortcuts honor + `docs_cache.lookup_path` from config; guidelines and templates currently hardcode + their default paths (`guidelines.ts:69`, `template.ts:19`). +- URL-added docs (`tbd shortcut --add=`) are written under `.tbd/docs/` too (e.g. + `shortcuts/custom/`) and recorded in `docs_cache.files` — also gitignored, also + invisible. +- There is no fork/override command, no provenance tracking, and no content hashing; doc + kinds are a closed union `'guideline' | 'shortcut' | 'template'` + (`src/file/doc-add.ts:24`). + +### What users asked for + +- “I’d just like to *see* all the guidelines that are relevant” — in the repo, not + behind a CLI. +- Pull bundled docs into the repo, fork them, keep the ones they want as visible, + git-tracked files. +- A way to push good local improvements back upstream without ceremony. + +### Why not finish PR #117 instead + +PR #117 is a design-options spec with 18 goals, 16 workflows, and 20 open questions, +including unresolved architecture (DocGraph vs DocMap, bundle/source cardinality, +lockfile identity). It models fork as one workflow (W8) inside a much larger source +framework. +The kernel below delivers W7–W11’s user value with ~3 small modules and only a +stamp-style format migration, and produces real usage data that will inform the bigger +design if we still want it. + +## Design + +### The `tbd docs` command group + +All operations on managed docs consolidate under one noun-scoped group, matching tbd’s +existing `dep`/`label`/`attic`/`config`/`workspace` convention. +Commands are split by scope: **`tbd sync` moves project data** (issues/beads — +unchanged), **`tbd docs` manages the doc layer** (cache + forked files). +Verbs-as-flags (`--update`, `--status`, …) are avoided; each operation is a subcommand: + +```bash +tbd docs # bare = status overview of managed docs +tbd docs list # all docs across kinds, with [forked]/[customized] markers +tbd docs show # read any doc by name, kind-agnostic +tbd docs sync # refresh the gitignored cache (absorbs `tbd sync --docs`) +tbd docs add # register a single external doc (URL, github:, local path) +tbd docs fork / unfork # fork into the repo / return to upstream +tbd docs update # reconcile forks with upstream (--merge / --keep-ours) +tbd docs diff / status # inspect +``` + +Notes: + +- **`tbd docs sync`** takes over cache refresh; `tbd sync --docs` remains as a + deprecated alias until the next format cut. + Subcommand grammar also removes a hazard the flag design had: a doc named `update` can + never collide with the verb. +- **tbd’s own docs join the system under a `tbd-` name prefix** instead of dedicated + viewer commands. The bundled files already carry these names (`tbd-docs.md`, + `tbd-design.md`); they become regular cached docs of a new kind `reference` (dir + `references/`), so they are listable, readable (`tbd docs show tbd-docs`), and even + forkable like everything else. + The `tbd-` prefix is reserved for tbd self-docs (`tbd doctor` warns on user docs + claiming it). +- **The old bare `tbd docs` manual viewer is repurposed**: bare `tbd docs` now shows the + status overview (the scope’s landing page); the CLI reference it used to display is + `tbd docs show tbd-docs`, with `tbd docs manual` as a convenience alias. + No backward compatibility is kept for the old bare behavior. + The sibling viewers `tbd readme` and `tbd design` are unchanged for now (candidates + for the same treatment later). + +#### Disposition of today’s `tbd docs` surface + +`tbd docs` already carries **four** behaviors today, and the kernel must re-home all of +them explicitly — not just the bare viewer — so no current capability is silently +dropped and the new `tbd docs list` verb does not collide with today’s +`tbd docs --list`: + +| Today (f04) | Current behavior | Under f05 | +| --- | --- | --- | +| `tbd docs` (bare) | renders the full `tbd-docs.md` manual | **status overview** of managed docs (the scope’s landing page) | +| `tbd docs ` / `--section ` | jumps to a manual section | `tbd docs show tbd-docs --section ` (the manual is now a `reference` doc) | +| `tbd docs --list` | lists the manual’s **sections** | retired as a top-level flag; `tbd docs list` now lists **docs across kinds**. Section navigation moves to `tbd docs show tbd-docs` | +| `tbd docs --all` | the “tbd Documentation Resources” orientation card | folded into the bare `tbd docs` overview, whose “menu” block (browse / fork / learn-more pointers) carries the same orientation value | + +The one real hazard is the `--list` meaning flip (sections → docs). +The f05 gate makes older CLIs refuse to run against an f05 repo (see “Format bump: +f05”), so there is no window where a user gets the old meaning against a new layout. +The flip is recorded in Backward Compatibility within the `tbd docs` reorganization (CLI +change 1), and **every doc and golden test that exercised the old `tbd docs` / `--list` +/ `--all` / `--section` surface is rewritten in the same release** — the affected tests +are catalogued exactly in [Golden-Test Maps](#golden-test-maps), the largest single one +being `cli-help-all.tryscript.md` (seven `tbd docs` assertions tied to the old surface). + +### Three kinds of sync, kept deliberately separate + +tbd now has three update surfaces, and they stay distinct. +A universal “sync everything” command was considered and rejected: the three differ in +scope, risk, and failure modes — doc updates can involve merges and tracked-file +mutation; nothing else does: + +| Command | Scope | Touches | Network | Modifies tracked files? | +| --- | --- | --- | --- | --- | +| `tbd sync` | project data (issues/beads) | sync worktree + `tbd-sync` branch | yes | never | +| `tbd setup --auto` | installation + integrations | skills, hooks, settings, `AGENTS.md`; invokes a docs-cache sync as a convenience | yes | only generated integration files | +| `tbd docs sync` | doc cache | gitignored `.tbd/docs/` only | yes (grouped by source, failure-isolated) | never | +| `tbd docs update` | your forked docs | fork dir + bases + manifest | no (offline, against the cache) | **yes — the only doc command that does** | + +Disambiguation worth stating once: `tbd update ` is an issue operation, +`tbd docs update` a doc operation — the noun scope always disambiguates. +Setup may *invoke* a docs-cache sync (as it effectively does today) and *report* pending +doc updates, but the canonical doc commands are `tbd docs sync` / `tbd docs update`, and +only the latter ever writes tracked files. +This table lands in `tbd-docs.md` so the taxonomy is documented for users and agents, +not just in this spec. + +### The fork directory + +One repo-relative directory holds all forked (and any hand-authored) tbd docs: + +``` +docs/tbd/ # default; configurable +├── README.md # generated index (see below) +├── guidelines/ +│ └── python-rules.md +├── shortcuts/ +│ └── review-code.md +└── templates/ + └── plan-spec.md +``` + +- Default: `docs/tbd/`, fixed for the initial f05 release. + Making the location configurable (persisted to a `docs_cache.fork_dir` key, surfaced + during setup) is planned within the f05 era as an additive, optional config key; until + then the constant in `paths.ts` is the single source of truth. +- Layout is `//.md`. The bundled + `shortcuts/system|standard|custom` subdivision is **flattened** to `shortcuts/` on + fork — that split is an implementation detail; the manifest preserves the original + source path for unfork and provenance. +- Files are copied **verbatim** (no frontmatter injection, no stamping). + Provenance lives in the manifest, so forked files stay clean, diffable, and forkable. +- A small `README.md` index is generated into the fork dir (and regenerated on every + fork/unfork/update). + It explains what these docs are (engineering guidelines, shortcuts, and templates + forked from tbd’s built-in set or external sources), lists each doc with its + frontmatter description, notes that the folder is managed by `tbd docs fork`, and + points readers at `npx get-tbd@latest docs` for further info. + This makes the folder self-explanatory when browsed on GitHub. + +### The fork manifest and stored merge bases + +All fork state lives under one committed directory, **`.tbd/doc-forks/`**: the manifest +(`forks.yml`) and the base snapshots (`base/`). (The existing `.tbd/.gitignore` only +excludes specific paths, so the directory is tracked automatically, like `config.yml`.) + +```yaml +# .tbd/doc-forks/forks.yml — managed by `tbd docs fork` / `unfork` / `update`. +forks: + - name: python-rules + kind: guideline + path: docs/tbd/guidelines/python-rules.md # repo-relative + source: internal:guidelines/python-rules.md # provenance docref (any docref form) + tbd_version: 0.2.3 # version when base was last set + base_hash: sha256:9f2c… # hash of the stored base content + conflicted: true # only present after `update --merge` + # left conflict markers; auto-clears + - name: acme-style + kind: guideline + path: docs/tbd/guidelines/acme-style.md + source: github:acme/eng-docs@main//guidelines/style.md + source_revision: 8f31c2d4… # git commit at base time (git sources) + source_tag: v1.4.0 # exact/matching tag, when one exists + base_hash: sha256:77ab… +``` + +Alongside it, **`.tbd/doc-forks/base//.md`** (also committed) stores a +verbatim copy of the upstream content the fork is based on — set at fork time and +advanced by `tbd docs update`. This is the *base* of every three-way merge: + +- A hash alone cannot drive a merge; the actual base content is required, and it must be + committed so any collaborator (or CI) can run `tbd docs update` later with full + fidelity, regardless of which tbd version originally forked the doc. +- **Git provenance is recorded when the source is git-hosted.** For `github:` / + `gitlab:` docrefs (the git schemes in docref v0.1; additional protocols may be added + in future versions), fork and every base advance also record the upstream commit + (`source_revision`) and, when the pinned ref is a tag or the commit matches one + exactly, `source_tag`. Non-git sources (`internal:`, bare URLs) have no revision to + record — which is precisely why bases are *snapshots* rather than pointers: the stored + copy is the universal provenance fallback that works for every source kind, with + revisions as extra precision when the source can provide them. +- `base_hash` is the SHA-256 of the LF-normalized base content (line-ending + normalization avoids false “customized” results from `core.autocrlf` on Windows). + `tbd doctor` verifies the base file still hashes to it. +- This manifest + base pair is the *recorded override edge* that the PR #117 design + review called for (its point 4), in miniature — and it records “enough data to add + three-way merge later without changing the format,” which that review asked for, by + shipping three-way merge now. + It makes these exact, offline checks: + - **customized**: current file hash ≠ `base_hash` + - **stale**: current upstream/cache content hash ≠ `base_hash` (upstream moved since + the base was last advanced — independent of whether the user edited) +- Because manifest and bases are committed, collaborators who pull get identical + resolution, status, and merge behavior. + +The cost is a committed second copy of each forked doc. +That duplication is the price of a real update story (see Alternatives #5); shadcn-style +registries that skip it have no upgrade path at all. + +### Format bump: f05 + +The new committed layout artifacts (the `.tbd/doc-forks/` directory and the +`docs_cache.fork_dir` / `docs_cache.local_dirs` keys) constitute a layout revision, so +`tbd_format` bumps to **f05** with a step in the existing migration chain +(`src/lib/tbd-format.ts`, following the f03→f04 precedent): + +- The f04→f05 migration is metadata-only: stamp the format id. + No files move; `.tbd/docs/` is untouched and fork artifacts appear only when fork is + first used — the upgrade path is exactly as smooth as f03→f04. (The `.tbd/.gitignore` + template refresh and the generated `.tbd/README.md` layout contract are additive setup + outputs planned within the f05 era, not part of the migration stamp; + `FORMAT_HISTORY.f05` in `tbd-format.ts` is the authoritative record of what the stamp + does.) +- Older CLIs encountering an f05 repo detect the newer format id via the existing + compatibility machinery and refuse to run, prompting an upgrade — an explicit signal, + rather than silently serving the upstream copies while upgraded teammates see + customized ones. **This gate is also what makes the rest of this spec safe**: because + an upgrade is enforced before any tbd runs against an f05 config dir, the command + reorganization and semantics changes ship without old/new coexistence hazards. +- Layout documentation updates alongside: format history in `tbd-format.ts`, + `tbd-design.md` layout sections, and the path conventions in `development.md`. +- Naming note: PR #117’s draft called its future format “f05”; since this kernel claims + f05, that redesign would land as f06+. + +### The `.tbd/` layout contract + +With doc forking, `.tbd/` gains one directory — so the contract of every entry is stated +explicitly, both here and in a generated **`.tbd/README.md`** (written by +setup/migration, kept current like the gitignore): + +``` +.tbd/ +├── README.md # generated: this contract, in-place +├── config.yml # committed: project configuration (hand-editable) +├── .gitignore # generated: controls what below is ignored +├── docs/ # GITIGNORED CACHE — unchanged f04 contract: machine-managed +│ # mirror of all upstream docs, regenerated by `tbd docs sync`, +│ # safe to delete anytime, never hand-edit, never committed +├── doc-forks/ # COMMITTED fork state — owned by `tbd docs fork/unfork/update`: +│ ├── forks.yml # manifest (provenance, hashes, revisions) +│ └── base/ # verbatim base snapshots (the merge bases) +├── workspaces/ # committed: persistent state (outbox etc.) — unchanged +└── state.yml # gitignored: local state — unchanged +``` + +The two doc directories answer different questions: `.tbd/docs/` is *what upstream +currently provides* (everything, always, disposable); `.tbd/doc-forks/` is *what this +repo has forked and from where* (a subset, precious, committed). +The user-facing forked files themselves live outside `.tbd/` entirely, in the fork dir +(`docs/tbd/` by default). +`.tbd/docs/` keeps its f04 mechanics verbatim — same path, same regeneration, same +gitignore entry — so the upgrade does not perturb it; `tbd doctor` validates +`doc-forks/` consistency (manifest ↔ base files ↔ fork dir). + +### The cache stays complete; sync is grouped by source + +`tbd setup --auto` / `tbd docs sync` continue to install **all** docs into `.tbd/docs/`, +including ones that are forked. +The cache copy is the pristine reference: it is what `diff` and staleness compare +against, it is the “theirs” side of every `update` merge, and it is what serving falls +back to after `unfork`. Setup and cache sync never touch files in the fork dir — tracked +files change only via the explicit fork/unfork/update commands. + +With docref sources, docs in the cache can have different *source roots*, so sync +operates per source group rather than per file: + +- **Grouping.** `docs_cache.files` entries are grouped by source root: all `internal:` + docs form one group (copied from the installed package); all docs sharing a git repo + + ref (e.g. `github:acme/eng-docs@main//…`) form one group, fetched with **one** network + operation per repo (a single checkout/archive at that revision, files extracted from + it — never N fetches for N docs); each standalone URL is its own group. +- **Failure isolation.** A group that fails — network error, repo or ref gone, URL 404, + moved content — is reported in the sync summary and its docs keep serving from the + last-good cache copy; all other groups proceed. + One bad doc or one vanished source never aborts the rest of the sync. + Cache entries are pruned only when a doc is explicitly removed from config, never on + fetch failure. +- **Update stays offline.** `tbd docs update` merges against the cache, so a source + being unreachable never blocks updating docs from healthy sources — it only means that + group’s staleness information is as fresh as its last successful sync. + `tbd docs status` annotates affected docs + (`source unreachable — serving cached copy from `), and `tbd doctor` reports + unreachable sources per group. + +### Resolution precedence + +Effective lookup order per kind, applied structurally (not by asking users to hand-edit +`lookup_path` — the PR #87 “lookup_path zombie” lesson): + +``` +// # forked + hand-authored local docs (highest) + # other in-repo doc dirs (optional config; see below) + # .tbd/docs/... as today +``` + +- Implemented by prepending the fork-dir path when building each command’s `DocCache` + path list. As part of this, guidelines and templates start honoring the same + config-driven path mechanics that shortcuts already use (small unification of + `guidelines.ts:69` / `template.ts:19`). +- The existing first-match-wins shadowing in `DocCache` then does all the work: a forked + `python-rules.md` shadows the cache copy with no new resolution code. +- **Local docs for free:** any `.md` file a user drops into the fork dir is served with + top precedence even with no manifest entry. + Status reports it as `local` (no provenance, nothing to unfork). + This cheaply delivers “easy project-local docs” (PR #117’s G2) without any + registration ceremony. +- When a reading command serves a forked/local doc, it prints a one-line provenance note + to stderr, e.g. `(serving forked copy: docs/tbd/guidelines/python-rules.md)`, and + `--list` output marks such docs `[forked]` / `[forked, customized]` / `[local]`. + +### Docs in other locations: docref, `local_dirs`, and the docmap view + +The kernel so far assumes two locations: the gitignored cache and the fork dir. +Three measured adoptions from the PR #117 branch generalize *location* without pulling +in the source framework: + +**1. Rule: every document reference, everywhere, is a docref.** Any string in tbd that +says “where a doc comes from” or “where a doc lives” — `docs_cache.files` values, the +fork manifest’s `source:` field, `tbd docs add` arguments, `local_dirs` entries, +provenance fields in JSON output, examples in our own docs — uses the docref grammar: a +single-string, URI-like address (`./path/`, `https://…`, `github:org/repo@ref//path`, +plus consumer-defined schemes — tbd’s `internal:` already fits the grammar). +This is a hard convention with no exceptions, so there is exactly one address syntax to +learn, parse, validate, and document. +Adoption is nearly free: the parser is ~100 lines, standalone, already written and fully +tested on the #117 branch (`src/docref/`), and today’s `internal:` prefixes and URLs are +already valid docrefs. +It also rationalizes the ad-hoc GitHub blob-URL conversion in `--add` into principled +normalization (`https://github.com/o/r/blob/main/f.md` → `github:o/r@main//f.md`), and +any future source framework inherits addresses instead of migrating them. + +**2. `docs_cache.local_dirs` serves docs from other directories in the repo.** An +ordered list of local-path docrefs (`./`-prefixed) naming additional in-repo doc +directories: + +```yaml +docs_cache: + fork_dir: docs/tbd + local_dirs: + - ./docs/general/agent-rules/ # e.g. a team guidelines dir that already exists +``` + +These slot into the effective lookup order between the fork dir and the cache (see +above). Docs found there are first-class for reading — `list`, `show`, serving, with +provenance notes — and report state `local`; they are not forkable or updatable, since +there is no upstream: they already live in the repo. +`DocCache`’s multi-directory shadowing handles this with zero new resolution code; it is +the supported, documented form of what `lookup_path` half-allowed today. + +**3. docmap, reduced to its essence: a doc inventory format.** The #117 draft bundles +several ideas under the “docmap” name — source manifests, lockfiles, sync semantics, +bundles, resolution. +Stripped to its core, the simple, well-defined, reusable concept is just the +*inventory*: + +> A **docmap** is a machine-readable inventory of a collection of documents: one entry +> per doc, each with an identity (`type` + `name`, unique within the map), a location +> (`path`, and/or a provenance `source` docref — every entry carries at least one), and +> presentation metadata (`title`, `description`). It describes a doc collection; it says +> nothing about how the collection is assembled, fetched, or kept fresh — a docmap is a +> generated *view* of a collection, never an input to resolution. + +For a docmap committed as a file, `path` is relative to the docmap file’s own directory +(the sitemap convention); size metrics (`word_count`, `size_bytes`, token estimates) are +producer extension fields, not core. +A sitemap for docs, with docref as its addressing primitive: + +```yaml +docmap: docmap/0.1 +name: tbd-docs # optional collection name +documents: + - name: python-rules + type: guideline + path: guidelines/python-rules.md # location within the collection + source: internal:guidelines/python-rules.md # provenance docref + title: Python Coding Rules + description: Type hints, docstrings, exception handling, resource management +``` + +Producers may *generate* a docmap (as tbd does: **every** list/inventory command emits +exactly this — `tbd docs list` / `tbd docs status` and the per-kind +`tbd guidelines/shortcut/template --list`, in both `--json` and, via one shared +renderer, text — with tbd’s state fields as extension fields) or *hand-author* one. +Consumers must ignore unknown fields. +That producer-agnosticism is what makes the concept useful beyond tbd: any repo can +commit a docmap to advertise its doc collection, and a future source framework would +*discover* external docs by reading one — the inventory format becomes the interface +between doc publishers and doc consumers, with the heavy machinery (manifests, +lockfiles, sync — the speculative #117 layer) defined later as *operations over +docmaps*, not as part of the format. +v0.1 deliberately has no bundles, no lockfile fields, no sync semantics. + +**The line deliberately not crossed:** pulling a *directory* of docs from another repo +(a true external bundle) requires sync, pinning, and cache identity — operations over +docmaps — and stays in the future framework. +Until then the supported answers are: single files via `tbd docs add `, in-repo +directories via `local_dirs`, and (if a team must share a guidelines repo today) +vendoring/submoduling it and pointing `local_dirs` at it. + +Both formats ship as bundled `reference`-kind docs. +`docref-format` is adapted from the #117 branch essentially as-is (marked adopted, +v0.1). `docmap-format` is **authored fresh and minimal** for v0.1 — just the inventory +definition above — citing the #117 draft only as exploratory background, since that PR +is speculative and may never land in its current form. +As regular docs they are listable and forkable like everything else — the conventions +document themselves through the system they describe, and our other docs can +cross-reference them by name. + +### Doc states + +| State | Meaning | Detected by | +| --- | --- | --- | +| `upstream` | not forked; served from its upstream via the cache | not in manifest | +| `forked` | forked, unmodified | file hash == `base_hash` | +| `customized` | forked and edited locally | file hash ≠ `base_hash` | +| `stale` | upstream changed since base was set (orthogonal to customized) | cache hash ≠ `base_hash` | +| `conflicted` | `update --merge` left unresolved conflict markers | manifest `conflicted` flag AND markers still present in file | +| `local` | file in fork dir with no manifest entry, or served from a `local_dirs` directory | file present, no entry | +| `missing` | manifest entry but file deleted | entry present, file absent | +| `orphaned` | manifest entry whose upstream no longer provides the doc | source absent from its group’s last successful sync | + +`customized` and `stale` can combine (`customized+stale`): the user edited *and* +upstream moved — exactly the case the update merge and the upstream-contribution +playbook care about. +The `conflicted` flag is set only by `update --merge` and clears automatically once the +standard markers (`<<<<<<<`/`=======`/`>>>>>>>`) are gone; scanning for markers only in +flagged docs avoids false positives on docs that legitimately discuss merge conflicts. + +**Out-of-band deletion — the user removes a forked file (or the whole fork dir) without +telling tbd.** This is an expected case, not an error: the fork dir is ordinary repo +files, and people delete files. +Because lookups fall through the precedence list, a deleted forked file **transparently +falls back to the upstream cache copy** — the doc keeps working and +`tbd guidelines ` still serves it (from upstream now, with no provenance note, +since nothing is forked there anymore). +The dangling manifest entry surfaces as `missing`, and tbd offers exactly two clean +resolutions everywhere it is reported: + +- **restore** it — `tbd docs fork --force` re-creates the file from the recorded + base (your fork point), or +- **finalize** the removal — `tbd docs unfork ` clears the manifest entry and base + snapshot. + +`tbd docs status` lists `missing` docs with those two options; `tbd doctor` flags them +and `tbd doctor --fix` **finalizes the unfork** (the deletion is read as intent to stop +forking — it removes the orphaned manifest entry and base, leaving the doc served from +upstream); `tbd docs update` skips `missing` docs. +Deleting the entire fork dir is just this case in bulk: every entry becomes `missing`, +all serving falls back to upstream, and `doctor --fix` clears the manifest. +Nothing is ever silently re-created against the user’s deletion. + +### Updating forked docs after a tbd upgrade + +The most common lifecycle event: you forked docs, upgraded tbd (or `tbd docs sync` +pulled fresh content), and the upstream versions moved. +`tbd docs update` reconciles forked copies with upstream, outsourcing the merge itself +to git (`git merge-file current base other` — works on plain files, exit code reports +conflict count, standard markers, no repo state touched): + +When a doc isn’t cleanly updatable, the user chooses one of two resolution strategies +(mutually exclusive flags): **`--merge`** combines both sides, writing standard conflict +markers to resolve by editing; **`--keep-ours`** keeps the local version untouched and +just advances the fork point to current upstream (“my fork supersedes this upstream +change” / “I already folded it in by hand”). + +| Doc state | `update` (default) | `update --merge` | `update --keep-ours` | +| --- | --- | --- | --- | +| `forked` (unmodified) + stale | replace file with new upstream; advance base | same | same | +| `customized` + stale, three-way merge is clean | apply merged result; advance base | same | keep file as-is; advance base only | +| `customized` + stale, merge conflicts | **skip**; warn and list the docs: “re-run with `--merge` (combine, resolve markers) or `--keep-ours` (keep your version)” | write conflict markers into the file; advance base; set `conflicted` | keep file as-is; advance base only | +| `customized`, not stale | no-op | no-op | no-op | +| `conflicted` (unresolved markers) | skip + warn: resolve first | skip + warn | skip + warn | +| `orphaned` | skip + note (upstream removed the doc; keep your copy or `unfork`) | same | same | +| `missing` / `local` | skip (doctor’s problem / nothing upstream) | same | same | +| base file missing (manual deletion) | cannot merge; skip + point at `--keep-ours` | same | re-establish base from current upstream (repair) | +| fork point set by a **newer tbd** (`tbd_version` > running version) | skip + warn: upgrade tbd first (this client’s bundled “upstream” is older than the fork point; updating would silently downgrade the doc) | same | same | + +Design points: + +- **Version-skew guard.** The manifest’s per-entry `tbd_version` records which tbd + advanced the fork point; `update` (every strategy) and a re-fork refresh both refuse + to act when the running tbd is older than that, since their bundled content predates + the fork point (re-fork accepts `--force` as the explicit downgrade escape hatch). + +- **Clean merges apply by default deliberately.** The forked file is git-tracked, so + every auto-merge is fully visible in `git diff` and trivially revertible — git is the + undo. Conflicted docs are never touched by default; the listing names the two + strategies and the user (or agent) re-runs with one. + +- **Base advance happens at merge time.** After any update (replace, clean merge, or + conflicted `--merge`), the base becomes the new upstream content. + So post-resolution, the doc is simply “a customized fork of current upstream” — states + stay coherent with no extra bookkeeping. + +- **`--keep-ours` keeps your content and advances the fork point.** For a single file + there is no diff to replay — keeping your version *is* the operation; upstream’s + change is acknowledged, staleness clears, and future updates diff against the new + base. It also repairs a missing base file. + (This was `--rebase` in an earlier draft, renamed because the operation is not + git-rebase content semantics — it does not replay your diff, it keeps it.) + +- **Only the explicit command mutates tracked files.** `tbd setup --auto` and the + 24-hour doc auto-sync refresh the gitignored cache as today and then *report* pending + updates (`2 forked docs have upstream updates — run 'tbd docs update'`), but never + write into the fork dir. + `tbd sync` likewise prints a one-line drift notice (stale / conflicted / missing + counts) after its cache refresh — awareness only, never action. + Background paths rewriting committed files would be surprising and hard to audit. + +- **Convergence is the unfork path.** If you customized a doc, upstream later adopted + your change, and `update` merges cleanly such that the file now equals upstream, the + doc returns to plain `forked` (unmodified) — and `tbd docs unfork` works without + `--force`. + +- `--dry-run` previews all of the above, including which docs would conflict. + +### CLI surface + +```bash +# Read and browse (works with zero forked docs) +tbd docs # status overview (bare command) +tbd docs list [--kind=guideline] # all docs across kinds, with state markers +tbd docs show python-rules # read any doc by name (kind-agnostic) +tbd docs show tbd-docs # tbd's own manual, via the tbd- prefix +tbd docs manual # alias for `tbd docs show tbd-docs` +tbd docs sync # refresh the gitignored cache + +# Add external docs (single files; consolidates the per-kind --add flags) +tbd docs add https://example.com/style.md --kind=guideline --name=acme-style +tbd docs add github:org/repo@main//docs/rules.md --kind=guideline # any docref form + +# Fork +tbd docs fork python-rules # one doc (name resolution as in `tbd guidelines`) +tbd docs fork python-rules review-code # several +tbd docs fork --kind=guideline typescript # disambiguate if a name exists in two kinds +tbd docs fork --category=python # a whole category (reads frontmatter; repeatable) +tbd docs fork --category=general --category=typescript # general + a language +tbd docs fork --all # everything +tbd docs fork --all --dry-run # preview what would be written + +# Inspect +tbd docs status [--json] # table of all forked docs + states +tbd docs diff python-rules # your file vs current upstream (the net fork) +tbd docs diff python-rules --base # your file vs its base (what you changed) +tbd docs diff python-rules --upstream # base vs current upstream (incoming changes) + +# Update (after a tbd upgrade; see "Updating forked docs" above) +tbd docs update # refresh unmodified + apply clean merges; list conflicts +tbd docs update python-rules # limit to specific docs +tbd docs update --merge # conflicts: combine, write conflict markers to resolve +tbd docs update --keep-ours # conflicts: keep your version, advance the fork point +tbd docs update --dry-run # preview, including which docs would conflict + +# Reverse +tbd docs unfork python-rules # delete file + base + manifest entry; ERROR if customized +tbd docs unfork python-rules --force # discard customizations deliberately +tbd docs unfork --all [--force] +``` + +The existing per-kind readers (`tbd guidelines `, `tbd shortcut `, +`tbd template `) are unchanged — `tbd docs show` is the kind-agnostic superset +that also reaches `reference` docs. + +Behavior details: + +- **Fork refuses to overwrite.** If the target path exists and is not an unmodified + forked copy (e.g. a pre-existing user file), fork errors and lists the conflict; + `--force` overwrites. + Never silently clobber user content. +- **Re-forking an already-forked doc** is just an update: on an *unmodified* doc it + refreshes to current upstream content (same as `update`); on a customized doc it + errors and points at `update` / `diff` (`--force` remains the explicit + start-over-from-upstream escape hatch, discarding customizations). +- **Unfork of a `missing` doc** cleans up the manifest entry (with a note). +- **Added external docs (`tbd docs add `) are forkable too** — the manifest + `source` is the normalized docref and staleness compares against the cache copy, which + `tbd docs sync` already refreshes. + No special casing. The per-kind `--add`/`--name` flags remain as aliases for + `tbd docs add`. +- All subcommands support `--json` and `--dry-run` per the existing CLI conventions. +- `tbd status` gains one summary line (e.g. + `Docs: 4 forked (1 customized, 2 with upstream updates — run 'tbd docs update', 1 conflict pending)`) + and `tbd doctor` gains checks: missing files, orphaned entries, base files + missing/corrupt (hash mismatch), unresolved `conflicted` docs, reserved `tbd-` name + collisions, unreachable sources (per source group, serving last cached copy), fork dir + covered by a `.gitignore` (defeats the purpose — warn), manifest/dir drift. + +### Doc categories and the fork recommendation + +There is **no `--relevant` flag, no repo auto-detection, and no hard-coded pack→doc +map.** Detection rules and a central pack list both drift out of sync with the docs and +substitute brittle logic for an agent’s judgment. +Instead, each doc declares its **category** in frontmatter, so a doc joins a category by +setting one field — nothing central to keep in sync — and the agent picks based on what +the repo actually is. +(This also retires the ad-hoc name-based `inferGuidelineCategory` inference, which today +mis-files docs like `convex-rules` as `general` and has no `convex`/`electron` category +at all; Phase 0 curates the frontmatter so each doc lands in the right category.) + +The basic categories: + +| Category | What’s in it | +| --- | --- | +| **general** | The foundational guidelines that apply to every repo — the `general-*` rules plus coding, comment, error-handling, TDD/testing, commit, and doc guidelines. | +| **typescript** | TypeScript rules, including CLI tooling (and the sorting / YAML / coverage / monorepo rules). | +| **python** | Python rules, including CLI patterns. | +| **convex** | Convex rules and limits / best-practices. | +| **electron** | Electron app development patterns. | + +`tbd docs list` shows every doc grouped by category, so the choices are visible. +The **recommendation** — stated in the bare `tbd docs` overview, the setup summary, the +skill, and `welcome-user`, and kept identical across all of them — is simply: **fork the +general guidelines, plus the categories for whatever languages and frameworks the repo +uses.** An agent that knows the project applies it directly (general + typescript for a +TypeScript CLI; general + python + convex for a Convex/Python backend) with no detection +table to maintain. + +Selection reuses the existing `category` metadata — no new construct, no central map: + +```bash +tbd docs fork python-rules review-code # by name +tbd docs fork --category=typescript # a whole category (reads frontmatter; repeatable) +tbd docs fork --category=general --category=python # general + a language +tbd docs fork --all # everything +``` + +Categories are guidelines-oriented; shortcuts and templates are forked by name or with +`--all`. + +### Setup integration (agent-first, non-interactive) + +tbd is operated almost exclusively by agents, and agents don’t benefit from prompts — so +there is **no interactive visibility menu**. The `--interactive` flag, which exists +today but has never had prompts (`setup.ts:1281`), is removed rather than built out. +Setup is instead designed to be excellent non-interactively: + +- **`tbd setup --auto`: unchanged behavior, self-documenting output.** Cache-only + remains the default (guidelines are active either way). + The summary *is* the menu, and it states the two choices explicitly so an agent — or a + user reading the output — can make them deliberately: + + ``` + Docs: 37 available in cache (.tbd/docs/, gitignored); none forked into the repo. + Guidelines are active from the cache. To make them visible and customizable, fork + them into docs/tbd/ (same behavior — just explicit and git-tracked): + + Scope: all standard guidelines (recommended), or a category: + general, typescript, python, convex, electron + Make visible: tbd docs fork --category=general --category= + tbd docs fork --all (everything) + Browse / read: tbd docs list / tbd docs show + ``` + + When forked docs exist with pending upstream updates (typically right after an + upgrade), the summary reports the count and suggests `tbd docs update` — but setup + itself never modifies files in the fork dir. + +- **Agent-led onboarding makes both choices explicit.** `welcome-user` and the skill + instruct the agent to put two questions to the user conversationally: + 1. **Scope** — keep *all* standard guidelines active (recommended), or just the + categories for your stack (general plus your languages/frameworks)? + 2. **Visibility** — leave them in tbd’s hidden cache (they just work — the “magic” + path), or fork them into `docs/tbd/` so they are visible on GitHub, reviewable, and + customizable (checked into git)? + + The agent explains that forking changes nothing about how guidelines work — both paths + make the same guidelines active — it only makes them explicit and editable. + It then runs `tbd docs fork --category=…` / `--all` (or leaves the cache as-is) + accordingly, using `--dry-run` to preview first. + No setup flags needed: `tbd docs` *is* the API. + +### Upstream-contribution playbook + +New bundled shortcut `shortcuts/standard/suggest-upstream-improvements.md` — pure +documentation, no code. +It instructs the agent to: + +1. Run `tbd docs status --json` and collect docs in `customized` (or `customized+stale`) + state. +2. For each, run `tbd docs diff ` and classify hunks: generally applicable + improvements vs project-specific customizations. +3. Draft an issue body: which doc, why the change is generally useful, the relevant diff + hunks in fenced blocks, and project context. +4. Show the draft to the user for confirmation, then file with + `gh issue create -R jlevy/tbd` (the `gh` integration and `use_gh_cli` setting already + exist). +5. Suggest the follow-up loop: once upstream ships the change and tbd is upgraded, run + `tbd docs update` — if upstream adopted the customization, the merge converges, the + doc returns to unmodified `forked` state, and a plain `tbd docs unfork` (no `--force` + needed) completes the unfork. + +The skill routing table gets matching rows, e.g.: + +| User says | Agent runs | +| --- | --- | +| “What guidelines are there?” | `tbd docs list` | +| “Make the guidelines visible / put the relevant ones in my repo” | `tbd docs fork --category=general` plus the repo’s languages, after confirming scope + visibility | +| “I want to customize the Python guidelines” | `tbd docs fork python-rules` then edit | +| “Put all of tbd’s docs in my repo” | `tbd docs fork --all` | +| “Stop customizing X / go back to the default” | `tbd docs unfork X` (`--force` only after confirming) | +| “I deleted a forked guideline file” | `tbd docs status` shows it `missing`; `tbd docs fork X --force` to restore or `tbd docs unfork X` to finalize | +| “Update the guidelines to the latest” (or after `tbd setup` reports pending updates) | `tbd docs update`; if conflicts are listed, ask the user, then `--merge` (combine + resolve) or `--keep-ours` (keep ours) | +| “Could we contribute these improvements back?” | `tbd shortcut suggest-upstream-improvements` | + +## Backward Compatibility + +**BACKWARD COMPATIBILITY REQUIREMENTS:** + +- **Code types, methods, and function signatures**: DO NOT MAINTAIN (internal modules; + the `DocCache`/`DocSync` extensions may refactor freely). +- **Library APIs**: N/A (nothing exported). +- **CLI surface**: three deliberate 0.x changes. + (1) The whole `tbd docs` surface is reorganized (see “Disposition of today’s + `tbd docs` surface”): bare `tbd docs` becomes the status overview; `tbd docs ` + / `--section` and the section-listing `--list` move onto `tbd docs show tbd-docs`; + `--all` folds into the overview; `tbd docs list` now lists docs across kinds. + DO NOT MAINTAIN any of the old `tbd docs` behaviors — confirmed; the manual stays + reachable as `tbd docs show tbd-docs` / `tbd docs manual`, and all routing docs and + goldens are updated in the same release. + (2) `tbd sync --docs` becomes a deprecated alias of `tbd docs sync` (KEEP DEPRECATED + until the next format cut). + (3) The `tbd setup --interactive` flag is removed (DO NOT MAINTAIN — it never had + prompts; setup is agent-first and non-interactive). + `tbd readme` / `tbd design` / per-kind readers unchanged. +- **Server APIs**: N/A. +- **File formats**: MIGRATE. `tbd_format` bumps f04 → f05 (new committed layout + artifacts: `docs_cache.fork_dir`, `.tbd/doc-forks/forks.yml`, `.tbd/doc-forks/base/`) + via the existing one-shot migration chain — metadata-only, no file moves (see “Format + bump: f05”). Older CLIs detect the newer format id and prompt to upgrade, per the + established format-compatibility policy. +- **Database schemas**: N/A. + +The default behavior of every existing command is unchanged when nothing has been +forked. + +## Alternatives Considered + +1. **Stop gitignoring `.tbd/docs/` (commit the cache).** One-line change, full + visibility — but every tbd upgrade churns dozens of files in user repos, generated + and authored content become indistinguishable, and there are no fork/unfork + semantics. Rejected. + +2. **Finish the PR #117 f05 framework first.** Right long-term shape, but blocked on + five open architecture questions and a format migration; delivers user value months + later. Rejected as the *first* step; this spec is its forward-compatible kernel. + +3. **Symlinks from `docs/` into `.tbd/docs/`.** Not portable (Windows), breaks on GitHub + rendering, and git-tracking symlinked generated content is worse than either mode. + Rejected. + +4. **Frontmatter provenance stamps instead of a manifest.** Self-contained files, but + fork would modify content on copy (breaking clean diffs against upstream) and + “customized” detection would need fragile frontmatter-stripping/normalization. + Rejected in favor of verbatim copies + manifest. + +5. **Updates without stored bases (hash-only provenance).** Smaller footprint, but a + hash cannot drive a three-way merge, so *every* upstream change to a customized doc + would surface as a wall-of-conflicts two-way diff — clean merges become impossible + and `update` degrades to “overwrite or do it by hand.” + Reconstructing bases from old npm versions (network, unpublished versions, + `development` builds) or from git history (squashes, mixed commits) is unreliable. + Rejected: committed base copies are the price of a real update story. + +6. **Auto-merging during `tbd setup --auto` or background doc auto-sync.** Tempting + (zero extra commands) but background paths rewriting committed files is surprising + and unauditable; doc auto-sync can trigger from any read command. + Rejected: setup/status only *report*; `tbd docs update` is the single explicit + mutation point. + +7. **Verbs as flags on a top-level `tbd eject` command** (`--update`, `--status`, + `--rebase`, plus a separate `tbd uneject` — the original verb names; see decision + 14). The original draft of this spec. + Rejected: flags-as-verbs is unintuitive, burns two top-level command slots, and a doc + named `update` would be ambiguous with the verb. + The noun-scoped `tbd docs` group matches existing tbd conventions + (`dep`/`label`/`attic`/`config`) and gives sync/show/list a coherent home. + +8. **Copy *all* docs into the fork dir and gitignore the unforked ones** (fork = + flipping a gitignore entry). + Full local browsability, but it fails the actual goal: gitignored mirrors are exactly + as invisible on GitHub and in PRs as the cache is, so only the tracked subset gains + visibility either way. + It also creates the worst silent failure mode available — an edit to a gitignored + mirror is served locally (top precedence) but never reaches the team, with no + git-visible artifact; fork state becomes a predicate over two systems (git index + + ignore rules, with the classic ignored-but-tracked confusions); and cache refresh + would either resurrect deleted mirror files or need tombstones, contradicting + “nothing is ever silently re-created against the user’s deletion.” + `tbd docs fork --all` already provides the all-visible posture in tracked form, with + `unfork` as the undo. + Rejected. + +## Resolved Decisions + +Settled during design review (2026-06-11): + +1. **One noun-scoped command group, `tbd docs `**, replaces the earlier top-level + `tbd eject`/`tbd uneject` + verb-flags design (see Alternatives #7). Scope split: + `tbd sync` = project data; `tbd docs` = the doc layer. + +2. **Verb pair is `fork`/`unfork`** (not shadcn-style `add`, and not the original + `eject`/`uneject` — see decision 14 for the rename rationale). + +3. **`tbd docs sync` absorbs `tbd sync --docs`** (old flag kept as a deprecated alias). + +4. **tbd self-docs use a reserved `tbd-` name prefix** as regular docs in the system + (kind `reference`), rather than a `tbd docs self` subcommand or dedicated viewers; + bare `tbd docs` becomes the status overview and the manual is + `tbd docs show tbd-docs` (alias `tbd docs manual`). No backward compatibility for the + old self-doc viewer behavior. + +5. **Update semantics**: clean three-way merges apply by default (tracked files make git + the undo); non-clean updates require an explicit strategy — `--merge` (combine with + conflict markers) or `--keep-ours` (keep local content, advance the fork point) — and + tracked files are mutated only by explicit `tbd docs` verbs, never by setup or + background sync. The earlier standalone `rebase` subcommand is folded into + `update --keep-ours`. + +6. **Default fork dir is `docs/tbd/`**, surfaced as an editable customization during + setup and persisted to `docs_cache.fork_dir` when changed. + +7. **Generated README index ships in v1**: explains what the docs are, lists them, and + points to `npx get-tbd@latest docs` for further info. + +8. **All fork state lives under one committed directory, `.tbd/doc-forks/`** — + `forks.yml` (manifest) plus `base/` (snapshots) — consolidated into one + self-describing directory (rather than scattered top-level files) so `.tbd/docs/` + remains purely the cache. + +9. **Format bump to f05** with a metadata-only f04→f05 migration, following the f04 + precedent: gitignore template refresh, format-history and layout-doc updates; older + CLIs prompt to upgrade on encountering f05. + +10. **docref everywhere, as a hard rule.** Every document reference in tbd — config + values, the fork manifest, CLI arguments, JSON output, our own docs — is a docref + string, with no exceptions. + Parser ported from the #117 branch; the spec ships as a bundled `reference` doc. + +11. **docmap is redefined as a minimal inventory format** (docmap/0.1: per-doc identity, + location, and metadata — no bundles, lockfiles, or sync semantics). + The reference doc is authored fresh for this spec and does not depend on the + speculative #117 design, which is cited only as exploratory background. + `tbd docs list/status --json` emit it; hand-authored docmaps are valid; source + machinery is deferred as future “operations over docmaps.” + +12. **Three sync surfaces, no universal sync.** `tbd sync` = issues; `tbd setup` = + installation/integrations (may invoke a docs-cache sync and report pending doc + updates); `tbd docs sync`/`update` = the doc layer. + A combined “sync everything” command was rejected: doc updates can involve merges + and tracked-file mutation, unlike the others. + The taxonomy table is documented in `tbd-docs.md`. + +13. **No interactive setup.** The unused `--interactive` flag is removed; setup is + agent-first and non-interactive, with self-documenting summary output (naming the + scope and visibility choices) and conversational onboarding via the skill and + `welcome-user`. + +14. **Terminology: fork/unfork, upstream, built-in.** The original `eject`/`uneject` + vocabulary (create-react-app heritage: a one-way escape from a managed bundle) fit + when every doc came from inside tbd; with docref sources, docs are multi-origin and + the fork lifecycle (fork point, merge, advance base, unfork, contribute back) is the + accurate model. Renames: verbs `fork`/`unfork`; state `forked`; config + `docs_cache.fork_dir`; the former `bundled` state is now `upstream` (not forked; + served from its upstream via the cache); “bundled”/“built-in” is reserved for + tbd-shipped `internal:` docs, where it remains literally true. + (For reference, shadcn names the model — “open code,” “copy and own” — but its verb + is just `add`, with no update story.) + +15. **The `.tbd/` layout contract is explicit and documented in-place.** A generated + `.tbd/README.md` (written by setup/migration) documents what each entry is — see + “The `.tbd/` layout contract” — and the same contract lands in `tbd-docs.md` and + `tbd-design.md`. Because the f05 gate makes older CLIs refuse to run against newer + config dirs, this layout reorganization and the CLI-semantics changes are safe to + ship together with no old/new coexistence hazards. + +16. **Phase 0 writes the doc and golden contracts first.** Before any feature code, the + refinement pins (a) the per-doc change map + ([Documentation Contract Changes](#documentation-contract-changes)) and (b) the + exact console output of every new/changed command + ([Golden-Test Maps](#golden-test-maps)), against one canonical fixture and a shared + console-output style contract. + Writing the expected output first is the design tool that keeps the command surface, + the docmap `--json` shape, and cross-command style consistent — and it turns the + `tbd docs` surface migration into a reviewed, test-backed change rather than an + afterthought. + +17. **All four of today’s `tbd docs` behaviors are re-homed explicitly** (decision 4 + named only the bare viewer): bare → status overview; ``/`--section` and the + section-listing `--list` → `tbd docs show tbd-docs`; `--all` → folded into the + overview; `tbd docs list`/`status` take the verbs. + This is part of CLI change 1 in Backward Compatibility, safe because the f05 gate + blocks old/new coexistence. + The `--list` meaning flip (sections → docs) is the one behavior change to call out + in release notes. + +Settled during refinement (2026-06-12): + +18. **The forked-doc serve note is on by default.** When a read command serves a + forked/local copy it prints the one-line stderr provenance note without `--verbose` + (suppressed only by `--quiet`/`--json`). The extra context is deliberate — it helps + agents recall which docs are customized in a session. + +19. **`tbd docs --all` is removed, not aliased.** Its orientation value moves into the + bare `tbd docs` overview; no compatibility shim, since the f05 gate prevents old/new + confusion. + +20. **The update strategy flag is `--keep-ours`, not `--rebase`.** `--rebase` collided + with git-rebase’s meaning, while the operation simply keeps the local version and + advances the fork point. + It pairs with `--merge` (combine, with conflict markers). + +21. **docmap is the single data model for all doc output; no backward-compat + carve-out.** Every list/inventory command builds one docmap and renders it to text + or `--json`; the per-kind `--list --json` therefore changes from today’s flat array + to a docmap (accepted — no backward-compat requirement). + Single-doc reads emit one docmap document entry plus `content`. Text and JSON derive + from the same structure, so they cannot drift and any field/state addition is a + one-place change. This supersedes the earlier OQ3. + +22. **Strict output-style consistency is structural.** All doc-command rendering goes + through one shared rendering layer (no per-command `console.log` formatting); the + Console output style contract is authoritative, and a cross-command contract/golden + test pins it. The mandate is “everything is consistent and systematic, easy to + update.” Pre-existing cross-command drift *outside* the docs surface (e.g. the + status/doctor INTEGRATIONS divergence documented in + `cli-orientation-golden.tryscript.md`) is real but out of scope here; it is tracked + as a separate follow-up bead rather than silently folded into this spec. + +23. **docref and docmap ship as standalone, fully-tested, dependency-free modules** + (`src/docref/`, `src/docmap/`) designed for later extraction into their own package: + no tbd-internal imports, a public API surface, and their own spec-mirror test suites + (the docref parser comes over from the #117 branch already in this shape). + tbd consumes them; they do not consume tbd. + +24. **No detection, no hard-coded pack map; categories come from doc frontmatter.** The + `--relevant` flag, repo auto-detection, and a central pack→doc list are all dropped + — they drift out of sync with the docs and replace agent judgment with brittle + rules. Each doc declares its category in frontmatter (`general`, `typescript`, + `python`, `convex`, `electron`), the name-based `inferGuidelineCategory` inference + is retired, and the agent forks the general category plus the repo’s + languages/frameworks. + `tbd docs fork` accepts names, `--category`, or `--all`. + +25. **Onboarding presents two explicit axes: scope and visibility.** Setup and + `welcome-user` make clear that (a) all standard guidelines are active by default + (recommended; can be subset by category), and (b) the user can keep them in the + hidden cache (the “magic” path) or fork them into `docs/tbd/` for explicit, + customizable, git-tracked copies. + Both paths make the *same* guidelines active — forking only adds visibility and + editability — and that equivalence is stated wherever the choice is offered. + +26. **Out-of-band deletion of a forked file is a supported state, not an error.** + Serving falls back to upstream automatically; `tbd docs status` reports `missing` + with two resolutions (restore via `fork --force`, or finalize via `unfork`); + `tbd doctor --fix` finalizes the unfork; deleting the whole fork dir is the same + case in bulk. Covered by a golden test. + +## Open Questions + +The questions raised during earlier reviews are all now resolved: + +- *Should `--relevant` become the fresh-setup default?* — Moot: the `--relevant` flag + and repo auto-detection are removed entirely (Resolved Decision 24); selection is by + category, chosen by the agent. +- *Pack definitions in code vs frontmatter tags?* — Resolved: categories come from each + doc’s frontmatter, with no central pack map (Resolved Decision 24). +- *Should the per-kind list JSON also be docmap?* — Resolved: yes, docmap everywhere + (Resolved Decision 21). + +No open questions remain; new ones will be surfaced here as implementation proceeds. + +## Implementation Plan + +Phases are ordered so each lands independently shippable. +Each numbered item is intended to be one bead. + +**Phase 0 comes first by design.** Before any code, we pin the two contracts the rest of +the work implements against: (1) exactly how each tbd doc changes +([Documentation Contract Changes](#documentation-contract-changes)), and (2) the exact +console output every new and changed command must produce +([Golden-Test Maps](#golden-test-maps)). Writing the expected output first is itself a +design tool — it forces the command surface, the JSON/docmap shape, and the +cross-command style to be consistent before they are built, and it turns each later +phase into “make this golden block real.” +Phase 0 is the refinement this spec asks for; Phases 1–5 then fill it in. + +### Phase 0: Documentation contracts and golden-test maps + +No production code. +This phase produces the two reviewable contracts and updates the docs +whose wording does not depend on the code (it does not stamp the format id — that lands +with the f05 code in Phase 1 — but it fixes the *text* those code changes must match). +Each item is one bead (labeled `0.1`–`0.5` so it sits before the global `1`–`20` +numbering the later phases use): + +- **0.1 — Author the contracts** (this spec): the + [Documentation Contract Changes](#documentation-contract-changes) and + [Golden-Test Maps](#golden-test-maps) sections — the per-doc before/after table, the + console output style contract, and the per-command expected-output blocks. + This is the design artifact the rest of Phase 0 and all later phases consume; it is + the deliverable of the current refinement. +- **0.2 — Doc edits independent of the new code** (the contract’s “Phase 0” rows): + rewrite `tbd-docs.md` (the `tbd docs` group, the three-sync taxonomy table, the + `.tbd/` layout contract, the managed-docs / fork / update / states sections, the + `docs_cache` config reference); update `tbd-design.md` (layout + CLI-group + format + narrative) and this repo’s `development.md` / `docs-overview.md` path and doc-command + sections. These land now because their wording is fixed by the contract, not by the + implementation. +- **0.3 — New bundled reference docs** `references/docref-format.md` (adopted from the + #117 branch, marked adopted v0.1) and `references/docmap-format.md` (authored fresh + and minimal per the Design section). + Authoring them now lets the docmap/0.1 schema in the golden `--json` maps be reviewed + against its own spec before any code emits it. +- **0.4 — New playbook shortcut** `shortcuts/standard/suggest-upstream-improvements.md` + (follows `new-guideline.md` conventions; references `tbd docs status --json` / + `tbd docs diff`). Pure documentation, so it lands in Phase 0; the skill/README routing + rows that point at it land with the agent surface in Phase 5 (kept together to avoid a + half-wired routing table). +- **0.5 — Lock the golden-test maps against reality.** For every *existing* command the + maps reference unchanged (`tbd guidelines --list`, `tbd status` with zero forks, the + `--json` doc shape), capture the current built-CLI output and paste it verbatim into + the maps, so the “consistency baseline” is real output, not a guess. + Catalog every existing golden/tryscript test the feature will break (see + [Existing golden tests that change](#existing-golden-tests-that-change)) and file a + bead per rewrite, each blocked on the phase that ships the corresponding behavior — so + no golden is left silently failing. + +Phase 0 ships as a docs-only PR. Its only test impact is that the new reference docs and +the new shortcut must resolve — which `doc-references.test.ts` already enforces once its +extractor learns the `reference` kind (the extractor change itself is tracked with the +rest of the test wiring in Phase 5). + +### Phase 1: Format bump and fork kernel + +1. **f05 format bump**: `CURRENT_FORMAT = 'f05'`, f04→f05 step in `migrateToLatest()` + (metadata-only: stamp + `.tbd/.gitignore` template refresh + generated + `.tbd/README.md` layout contract), format-history and layout-doc updates + (`tbd-design.md`, `development.md`), older-CLI newer-format-detection behavior + verified with a test. +2. **docref + docmap modules — standalone, fully tested, extraction-ready** + (`src/docref/`, `src/docmap/`): bring the docref parser, types, and spec-mirror tests + over from the #117 branch, and add a `src/docmap/` module (the docmap/0.1 type, + (de)serialize, and validate, authored against `references/docmap-format.md`). Both + are **dependency-free** (no tbd-internal imports), expose a small public API, and + carry their own full test suites — structured so they can move to a standalone + package later without change (Resolved Decision 23); tbd imports them, never the + reverse. Wire docref normalization into config source strings and the `--add` / + `docs add` URL handling, and make `DocMap` the type every list/inventory command + produces (Resolved Decisions 21–22). +3. **Manifest + base module** (`src/file/fork-manifest.ts`): zod schema, read/write of + `.tbd/doc-forks/forks.yml`, base copies under `.tbd/doc-forks/base/`, LF-normalized + SHA-256 hashing, state computation + (`forked`/`customized`/`stale`/`conflicted`/`missing`/`orphaned`/`local`) given + manifest + fork dir + base dir + cache. + Pure functions; full unit coverage including the state matrix. +4. **`tbd docs` group scaffolding + `fork` subcommand** (rework + `src/cli/commands/docs.ts` into the group; old manual-viewer behavior relocates per + item 18): name resolution via `DocCache` (reusing fuzzy-match + suggestions), + `--kind`, multiple names, `--all`, `--dry-run`, `--json`, overwrite refusal + + `--force`, re-fork semantics, README index generation (with the + `npx get-tbd@latest docs` pointer), `docs_cache.fork_dir` support. + `tbd docs sync` subcommand delegating to `syncDocsWithDefaults()` (`tbd sync --docs` + kept as deprecated alias). +5. **`tbd docs unfork` subcommand**: single/multi/`--all`, customized refusal + + `--force`, base-file and missing-entry cleanup, README regeneration. +6. **Precedence wiring**: prepend fork-dir paths in guidelines/shortcut/template lookups + (unifying guidelines/templates onto config-driven paths), provenance line on serve, + `[forked]`/`[customized]`/`[local]` markers in `--list`. +7. **E2E tests** (pattern of `doc-add-e2e.test.ts`): fork → list marker → serve shows + forked content → edit → unfork refuses → `--force` succeeds → upstream serving + restored; **delete a forked file out-of-band → serve transparently falls back to + upstream and `tbd docs status` reports `missing`**; `tbd setup --auto` refresh leaves + forked files untouched; f04 repo migrates to f05 on setup. + +### Phase 2: Status, browse, diff, doctor + +8. **`tbd docs status` (+ bare `tbd docs`)** with `--json` per the docmap map schema; + one-line summary in `tbd status`. +9. **Shared docmap renderer + `tbd docs list` / `show `**: build the single + rendering layer (Resolved Decision 22) that turns a `DocMap` into either text + (grouped list, status table, overview, single-entry read) or `--json`, then implement + cross-kind `list` (with state markers) and kind-agnostic `show` on top of it. + **Migrate the per-kind readers (`guidelines`/`shortcut`/`template`) onto the same + renderer**: their `--list --json` switches from the flat array to a docmap (Resolved + Decision 21) and their text output becomes the one canonical format. + The reader *commands* stay; only their shared rendering path changes. +10. **`docs_cache.local_dirs` + `tbd docs add ` + grouped sync**: local-dirs + wiring into the effective lookup order; `add` consolidating the per-kind `--add` + flags (kept as aliases) with docref normalization replacing the ad-hoc blob-URL + conversion in `doc-add.ts`; sync refactored to group `files` entries by source root + (one fetch per git repo+ref; per-group failure isolation; git revision/tag capture + for manifest provenance; no cache pruning on fetch failure). +11. **`tbd docs diff `** with `--base` / `--upstream` variants + (`git diff --no-index` style output against base and cache copies, no network). +12. **`tbd doctor` checks**: missing (deleted) / orphaned entries, base missing/corrupt, + unresolved conflicts, reserved `tbd-` names, gitignored fork dir warning; `--fix` + finalizes the unfork for `missing` forked files (treating an out-of-band deletion as + intent to stop forking) and cleans orphaned manifest entries. + +### Phase 3: Update and merge + +13. **Merge module** (`src/file/fork-update.ts`): `git merge-file` wrapper (labels, + exit-code handling, dry-run via `-p`), the per-state decision logic from the update + table, base advancement, `conflicted` flag set/auto-clear. + Unit tests cover every row × strategy of the table. +14. **`tbd docs update` command surface** with mutually exclusive `--merge` / + `--keep-ours` strategy flags: name filtering, `--dry-run` preview with conflict + listing, the skip-warning naming both strategies, summary counts; pending-update + reporting wired into `tbd setup --auto` output and `tbd status`. + +### Phase 4: Setup and categories + +15. **Categories from doc frontmatter** (no `doc-packs.ts`, no detection function): + curate the `category` field on the bundled docs so each lands in exactly one + category (`general`, `typescript`, `python`, `convex`, `electron`), retire the + name-based `inferGuidelineCategory` inference in favor of the declared field, and + add `--category` selection to `tbd docs fork` (reusing the existing + `--list --category` metadata — no new map). + Unit tests: category-based fork selection, and that every bundled doc resolves to + exactly one category. +16. **Setup integration**: self-documenting `--auto` summary naming the two choices — + *scope* (all standard guidelines, recommended, or a category subset) and + *visibility* (hidden cache vs fork into `docs/tbd/`) — plus the pending-update + count; removal of the unused `--interactive` flag; the fork dir documented as a + config customization (`docs_cache.fork_dir`); the sync-taxonomy table added to + `tbd-docs.md`. + +### Phase 5: Docs and agent surface + +17. **Playbook shortcut** `suggest-upstream-improvements.md` (follows `new-guideline.md` + conventions; references `tbd docs status --json` and `tbd docs diff`). +18. **Self-docs and format-docs migration**: add kind `reference` (dir `references/`), + register `tbd-docs` and `tbd-design` in the cache under their existing `tbd-` names, + bundle `docref-format` (adapted nearly as-is from the #117 branch, marked adopted) + and `docmap-format` (authored fresh and minimal per the design section, #117 cited + as exploratory background only) as reference docs, retire the bare-`tbd docs` manual + viewer in favor of `tbd docs show tbd-docs` + the `tbd docs manual` alias + (`tbd readme`/`tbd design` untouched). +19. **Agent docs**: routing rows (fork / update / upstream + the missing-file row) in + the skill (`shortcuts/system/skill-baseline.md`) and a fork/update section in + `tbd-docs.md`; the two-axis (*scope* + *visibility*) `welcome-user` onboarding; + README section ("Forkable guidelines: fork them into your repo"); `tbd prime` + mention if warranted. + (`install/claude-header.md` needs no change — `Bash(tbd:*)` already covers + `tbd docs`.) +20. **CHANGELOG + release notes** per `release-notes-guidelines`. + +## Documentation Contract Changes + +This is the per-doc map the user asked for: exactly what changes in each tbd doc, and +when. “Phase 0” rows are wording fixed by this design and land in the docs-only Phase 0 +PR; “with code” rows are wording coupled to an implementation phase (e.g. a stamped +format id) and land with that phase, but still follow the contract written here. +Blocks already specified in the Design section (the three-sync taxonomy, the `.tbd/` +layout contract, the doc-states table, the update decision table) are **referenced, not +duplicated** — the contract is that those exact blocks appear in the named docs. + +| Doc | Lands | Contract change | +| --- | --- | --- | +| `packages/tbd/docs/tbd-docs.md` (the CLI manual; also the `tbd-docs` reference doc) | Phase 0 | Replace the “Documentation Commands” section with the `tbd docs` group (per “The `tbd docs` command group”); add a “Managing forked docs” section (fork/unfork/update/diff/status + the doc-states and update-decision tables); add the three-sync taxonomy table and the `.tbd/` layout contract verbatim; extend “Configuration Reference” with `docs_cache.fork_dir` and `docs_cache.local_dirs` and the note that `files`/`source`/`local_dirs` values are docrefs; cross-link `docref-format` and `docmap-format`. | +| `packages/tbd/docs/tbd-design.md` | Phase 0 (narrative) + Phase 1 (format id) | §2 File Layer + path conventions: document `.tbd/doc-forks/` (manifest + base snapshots), the external fork dir, and the resolution precedence (fork dir → `local_dirs` → cache, first-match-wins). §4 CLI Layer: add the `tbd docs` group and the three-sync taxonomy. Format narrative: add f05 alongside f04. docref/docmap named as the addressing conventions. | +| `packages/tbd/src/lib/tbd-format.ts` (`FORMAT_HISTORY`) | Phase 1 (with code) | Add the `f05` entry: `introduced` (next minor), description “Adds forkable-docs layout”, `changes` = [`docs_cache.fork_dir`, `docs_cache.local_dirs`, `.tbd/doc-forks/forks.yml` + `base/`, generated `.tbd/README.md`], `migration` = “metadata-only: stamp f05, refresh `.tbd/.gitignore`, write `.tbd/README.md` layout contract”. `CURRENT_FORMAT = 'f05'`. This file is the authoritative format history; its wording is the contract Phase 0 references but does not edit. | +| `docs/development.md` (this repo) | Phase 0 | “Path Conventions” block: add `.tbd/doc-forks/` (committed) and note the fork dir lives **outside** `.tbd/` (default `docs/tbd/`). Add a “Testing forkable docs” pointer to the new e2e/tryscript files. | +| `docs/docs-overview.md` (this repo) | Phase 0 | “tbd CLI Documentation Commands” + “Adding external docs by URL”: replace with the `tbd docs` group; `tbd docs add `; add a line on forking docs into a visible `docs/tbd/`. | +| `README.md` | Phase 0 | “Shortcuts, Guidelines, and Templates”: add a “Forkable: see them in your repo” paragraph with a `tbd docs fork --category=general` example. “Documentation” block: `tbd docs` is now an overview; the manual is `tbd docs show tbd-docs`. Per-kind `--add` lines annotated as aliases for `tbd docs add`. | +| `packages/tbd/docs/shortcuts/system/skill-baseline.md` (injected agent skill) | Phase 5 | Add the fork/update/upstream rows to “User Request → Agent Action” (the rows in “Upstream-contribution playbook”); add `tbd docs list` / `tbd docs fork` to the “Documentation” command table; one-line “Forkable docs” note. Kept within the skill’s size budget; lands with the rest of the agent surface so routing is never half-wired. | +| `packages/tbd/docs/install/claude-header.md` | none | **No change.** Its `allowed-tools: Bash(tbd:*)` already covers `tbd docs`. Stated here so the audit is explicit. | +| `packages/tbd/docs/shortcuts/standard/welcome-user.md` | Phase 5 | Add the two-axis onboarding offer after the status summary — *scope* (all guidelines vs a category subset) and *visibility* (hidden cache vs forked into `docs/tbd/`) — plus a “make guidelines visible” row routing to `tbd docs fork --category=…`. | +| `packages/tbd/docs/shortcuts/standard/suggest-upstream-improvements.md` (**new**) | Phase 0 | The upstream-contribution playbook (per “Upstream-contribution playbook”). Pure docs. | +| `packages/tbd/docs/references/docref-format.md` (**new**) | Phase 0 | Adopted from the #117 branch, marked adopted v0.1. First doc of the new `reference` kind. | +| `packages/tbd/docs/references/docmap-format.md` (**new**) | Phase 0 | Authored fresh, minimal (docmap/0.1 inventory only) per the Design section; #117 cited as background. | +| Generated `.tbd/README.md` (**new**, by setup/migration) | Phase 1 (with code) | The `.tbd/` layout contract block, written in place; kept current like the gitignore. | +| Generated `/README.md` (**new**, by fork/unfork/update) | Phase 1 (with code) | The fork-dir index: what these docs are, one line per doc with its description, “managed by `tbd docs fork`”, and the `npx get-tbd@latest docs` pointer. | +| `packages/tbd/CHANGELOG.md` + release notes | Phase 5 | f05 entry per `release-notes-guidelines`. | +| `tbd --help` (the `docs` command `.description()` + subcommand help, in `src/cli/commands/docs.ts`) | Phase 1 (with code) | Description changes from “Display CLI documentation (use tbd sync --docs …)” to a managed-docs summary; subcommand help strings per the [Golden-Test Maps](#golden-test-maps). User-facing text, so it is part of the contract. | + +Two consistency points the contract pins down: + +- **One data model, one renderer — docmap is it (no backward-compat carve-outs).** Every + command that lists or inventories docs builds a single in-memory **docmap** and then + renders it; `--json` serializes that docmap, and text mode runs it through one shared + renderer. This applies uniformly: `tbd docs list` / `tbd docs status`, the per-kind + `tbd guidelines/shortcut/template --list` (whose `--json` **changes** from today’s + flat array to a docmap — accepted, since we have no backward-compat requirement here), + and the bare `tbd docs` overview (a docmap rendered in summary form). + Single-doc reads (`tbd docs show`, the per-kind `` readers) emit one docmap + *document entry* plus a `content` field — the same entry shape, so list and read never + drift. Because text and JSON derive from the same structure, they cannot disagree, and + adding a field or a state is a one-place change. + This is the systematic consistency the design optimizes for; see Resolved Decisions + 21–22. +- **`tbd-docs.md` is both a rendered manual and a forkable `reference` doc.** Its + bundled source stays at the docs root (`packages/tbd/docs/tbd-docs.md`); the + cache/kind-dir is `references/` and the lookup name is `tbd-docs`. The doc-sync map + records that root-to-`references/` mapping (a small `doc-sync.ts` detail, Phase 5 item + 18). + +## Golden-Test Maps + +These are the expected console outputs the new and changed commands must produce. +They are written here (not as live test files) because they run against the built CLI +and would fail until the feature ships; each block is **lift-ready** — the +implementation phase pastes it into the named harness. +Two harnesses are in use, matching today’s repo: **tryscript** (`*.tryscript.md`, +`NO_COLOR=1`, `[..]` matches intra-line, `...` matches whole lines, custom `[PATTERN]`s +for unstable fields) and **vitest inline snapshots** (`golden-output.test.ts`, +`FORCE_COLOR=0`). All maps below are shown as captured with color disabled (the state +the golden files store), so bold/dim render as plain text. + +Unstable fields use placeholders that become tryscript patterns: `[SIZE]` = +`\([0-9.]+ .B, ~[0-9.]+k? tok\)`, `[PATH]`, `[HASH]`, `[VERSION]`. Per +`golden-testing-guidelines`, everything else (names, kinds, states, counts, ordering) is +shown literally — no patterns on values we control. + +Status convention: blocks for **shipped** commands are captured from the built CLI and +match the live goldens; blocks for commands that have **not landed yet** are marked +*(Phase N contract)* and must be re-captured against the real CLI when that phase ships +(the Phase 0.5 discipline, applied per phase). + +### Console output style contract + +This contract is **authoritative and enforced structurally, not by convention**. Doc +commands must not hand-roll `console.log` formatting; all of them route through a single +shared rendering layer (extend `output.ts` / `sections.ts`, or a small `docs-render.ts` +that builds on them) that owns list-entry layout, state markers, the summary line, +tables, and the overview. +One docmap in, one renderer out (see the consistency point above), so there is exactly +one place to change any of these and no command can drift from another. +Every rule below is verified against today’s `output.ts`, `sections.ts`, and +`doc-command-handler.ts`; the maps that follow are just this contract applied: + +- **Section headers** are `formatHeading()` — UPPERCASE, bold (`INTEGRATIONS`, + `HEALTH CHECKS`); bodies indent two spaces. + The `tbd docs` overview reuses this. +- **Icons** come only from `ICONS` — `✓` success/closed, `✗` error, `⚠` warn, `•` + notice, `○` open, `◐` in_progress, `●` blocked. + No new glyphs are invented for doc states. +- **Color roles** (from `createColors`): `id`=cyan for doc names, `dim` for + metadata/sizes/paths, `bold` for names and headers, `success`/`warn`/`error` for the + matching icons. Forked doc names render with the `id` role, like issue IDs. +- **Doc-state markers are dim bracket tags** appended to the list entry, exactly like + the existing `[shadowed]` tag: `[forked]`, `[forked, customized]`, `[local]`. State + *icons* are never used for doc states — brackets are the established convention. + (Staleness and `conflicted` are lifecycle facts shown by `tbd docs status` / the + summary line, not list markers — list markers describe ownership.) +- **Provenance on serve** is one dim line to **stderr** (so piped stdout stays clean), + reusing the existing stderr-note channel: `(serving forked copy: [PATH])`. It is **on + by default** (suppressed only by `--quiet`/`--json`), deliberately not gated behind + `--verbose` — the small amount of extra context helps an agent remember which docs are + customized in a session. +- **Outcomes**: success → `✓ ` green to stdout; refusal/error → `✗ ` red to + stderr + non-zero exit; preview → `[DRY-RUN] ` yellow (the `output.dryRun` form). +- **`--json`** goes through `output.data(...)`: docmap object for `list`/`status`, + per-doc object for `show`; no ANSI; consumers ignore unknown fields. +- **Footer** uses `renderFooter`: `Use 'cmd' for X, 'cmd' for Y.` +- **Width** wraps at 88 columns (`getTerminalWidth`). +- **Command/scope parallels** kept on purpose: `tbd docs` (overview/summary) is to + `tbd docs status` (the per-doc table) as `tbd status` is to `tbd stats`; the bare + overview’s “menu” block and the `tbd setup --auto` Docs summary share identical + wording. + +A single canonical fixture is used across every map below (a repo just upgraded, so +upstream has moved): + +| name | kind | customized | stale | merge on update | +| --- | --- | --- | --- | --- | +| `python-rules` | guideline | yes | yes | clean | +| `acme-style` | guideline | yes | yes | conflicts | +| `review-code` | shortcut | no | yes | n/a (refresh) | +| `tbd-docs` | reference | no | no | untouched | + +### `tbd docs` (bare overview) + +Mirrors `tbd status`: a summary plus pointers, never the full table. +Zero-fork case is the default and stays the orientation card the old `tbd docs --all` +provided: + +```text +$ tbd docs # no docs forked yet +tbd docs — managed documentation + + [..] docs available in the cache (.tbd/docs/, gitignored); none forked into the repo. + Guidelines are active from the cache. Three postures, all serving the same docs: + + Hidden (default): keep the cache as-is — zero repo footprint + Curated: tbd docs fork [...] fork chosen docs into docs/tbd/ + tbd docs fork --category= (general, typescript, python, convex, electron) + Everything: tbd docs fork --all all docs, visible and editable + + Browse / read: tbd docs list / tbd docs show + Learn more: tbd docs show tbd-docs (the manual; alias: tbd docs manual) +? 0 +``` + +The menu body lives in one shared module (`docs-menu.ts`) used by both this overview and +the setup Docs summary, so the two surfaces cannot drift — and the menu only names +selectors that exist (the `--category` hint shipped before the flag once; never again). + +With forks present: + +```text +$ tbd docs +tbd docs — managed documentation + + [..] available ([..] upstream, 4 forked into docs/tbd/) + 4 forked: 2 customized, 3 with upstream updates — run 'tbd docs update' + + Inspect: tbd docs status + Browse: tbd docs list + Update: tbd docs update + Learn more: tbd docs show tbd-docs +? 0 +``` + +### `tbd docs list` + +Grouped by kind (bold header), each entry in the established two-line form +(`name [SIZE]` then 3-space-indented `Title: Description`), with dim ownership markers +appended: + +```text +$ tbd docs list +guideline + acme-style [SIZE] [forked, customized] + ACME House Style: Internal style overrides for ACME repos + python-rules [SIZE] [forked, customized] + Python Coding Rules: Type hints, docstrings, exception handling, resource management + typescript-rules [SIZE] + TypeScript Rules: Strictness, module boundaries, and error handling for TypeScript + [.. remaining guidelines ..] +shortcut + review-code [SIZE] [forked] + Review Code: Comprehensive code review across uncommitted, branch, or PR scopes + [.. remaining shortcuts ..] +reference + docmap-format [SIZE] + Docmap Format: A minimal inventory format for a collection of documents + docref-format [SIZE] + Docref Grammar: A single-string, URI-like address for any document + tbd-docs [SIZE] [forked] + tbd CLI Documentation: Command reference for the tbd CLI +? 0 +``` + +`tbd docs list --kind=guideline` filters to one group (no kind header needed). +JSON is the docmap object (see “docmap” in Design); the array form is the per-kind +commands’ contract, not this one: + +```text +$ tbd docs list --json +{ + "docmap": "docmap/0.1", + "name": "tbd-docs", + "documents": [ + { + "name": "python-rules", + "type": "guideline", + "path": "docs/tbd/guidelines/python-rules.md", + "source": "internal:guidelines/python-rules.md", + "title": "Python Coding Rules", + "description": "Type hints, docstrings, exception handling, resource management", + "state": "customized", + "stale": true + } + [.. one entry per doc; upstream docs have state "upstream" and a source docref + (their provenance) but no path — every entry carries a location ..] + ] +} +? 0 +``` + +`tbd guidelines --list --json` (and `shortcut`/`template`) emit the **same** docmap +object, filtered to that kind — the consistency the design mandates: one shape, one +renderer, so the only difference between these commands’ output is the set of documents +in it. + +### `tbd docs show` / `tbd docs manual` + +Kind-agnostic read; `reference` docs carry no agent header (unlike guidelines). +The manual moves here from the old bare `tbd docs`: + +```text +$ tbd docs show tbd-docs | head -3 +# tbd CLI Documentation + +Git-native issue tracking for AI agents and humans. +? 0 + +$ tbd docs manual | head -1 # alias for: tbd docs show tbd-docs +# tbd CLI Documentation +? 0 + +$ tbd docs show python-rules # serves the forked copy; provenance to stderr +[.. forked file content on stdout ..] +# stderr: (serving forked copy: docs/tbd/guidelines/python-rules.md) +? 0 +``` + +### `tbd docs sync` + +Refreshes the gitignored cache; `tbd sync --docs` remains a deprecated alias with +identical output (both render through one shared module, `docs-sync-output.ts`): + +```text +$ tbd docs sync +✓ Docs up to date +? 0 + +$ tbd docs sync # after an upgrade changed bundled docs; forks stale +✓ Synced docs: ~1 doc(s) +• Docs: 1 forked doc(s) have upstream updates — run 'tbd docs update' +? 0 +``` + +### `tbd docs fork` + +```text +$ tbd docs fork python-rules +✓ Forked python-rules → docs/tbd/guidelines/python-rules.md + Regenerated docs/tbd/README.md + +Edit in place — tbd now serves your copy wherever it served upstream. +? 0 + +$ tbd docs fork --all --dry-run +[DRY-RUN] Would fork [..] doc(s) into docs/tbd/ +... +No files written. Re-run without --dry-run to apply. +? 0 + +$ tbd docs fork python-rules # target exists and is not an unmodified fork +Error: docs/tbd/guidelines/python-rules.md already exists and is not an unmodified fork. Refusing to overwrite it. Options: + tbd docs diff python-rules # see how it differs + tbd docs fork python-rules --force # overwrite with upstream +? 1 +``` + +`--category` selection (shipped with Phase 4; categories come from each doc’s declared +frontmatter, the name-based inference retired): + +```text +$ tbd docs fork --category=python --dry-run +[DRY-RUN] Would fork 3 doc(s) into docs/tbd/ (categories: python) + guideline python-cli-patterns + guideline python-modern-guidelines + guideline python-rules +No files written. Re-run without --dry-run to apply. +? 0 +``` + +### `tbd docs unfork` + +```text +$ tbd docs unfork python-rules # customized → refuse +Error: python-rules has local customizations (differs from its base). Refusing to discard them. Options: + tbd docs diff python-rules # review your changes + tbd docs unfork python-rules --force # discard and fall back to upstream +? 1 + +$ tbd docs unfork review-code # unmodified → succeeds +✓ Unforked review-code — served from upstream again. +? 0 +``` + +### `tbd docs status` + +The per-doc table (dim header row, `output.table` convention). +The closing summary line matches the bare overview and the `tbd status` Docs line +exactly: + +```text +$ tbd docs status +NAME KIND STATE SOURCE +acme-style guideline customized, stale github:acme/eng-docs@main//guidelines/style.md +python-rules guideline customized, stale internal:guidelines/python-rules.md +review-code shortcut stale internal:shortcuts/standard/review-code.md +tbd-docs reference forked internal:tbd-docs.md + +4 forked: 2 customized, 3 with upstream updates — run 'tbd docs update' +? 0 +``` + +### Removed forked file (out-of-band deletion) + +The user deletes a forked file directly (`rm docs/tbd/guidelines/review-code.md`) +without telling tbd. +Serving keeps working (falls back to upstream), `status` reports `missing` with both +resolutions, and `doctor --fix` finalizes the unfork: + +```text +$ rm docs/tbd/shortcuts/review-code.md + +$ tbd docs show review-code # still works — falls back to upstream +[.. upstream review-code content on stdout, no provenance note ..] +? 0 + +$ tbd docs status +NAME KIND STATE SOURCE +acme-style guideline customized, stale github:acme/eng-docs@main//guidelines/style.md +python-rules guideline customized, stale internal:guidelines/python-rules.md +review-code shortcut missing internal:shortcuts/standard/review-code.md +tbd-docs reference forked internal:tbd-docs.md + +1 doc(s) missing (forked file deleted or renamed): + review-code restore with 'tbd docs fork review-code --force', or finalize with 'tbd docs unfork review-code' +? 0 + +$ tbd doctor --fix # excerpt +⚠ Forked docs - 1 missing (review-code: forked file deleted) + Fixed: finalized unfork (removed manifest entry + base); now served from upstream +? 0 +``` + +### `tbd docs diff` + +Git-style, no network (`git diff --no-index` against the relevant copy): + +```text +$ tbd docs diff python-rules # your file vs current upstream (the net fork) +--- upstream:guidelines/python-rules.md ++++ docs/tbd/guidelines/python-rules.md +@@ +[.. unified diff hunks ..] +? 0 + +$ tbd docs diff python-rules --base # your file vs its base (what you changed) +$ tbd docs diff python-rules --upstream # base vs current upstream (incoming changes) +``` + +### `tbd docs update` + +Default run on the canonical fixture: refresh the unmodified-stale doc, apply the clean +merge, and *list* the conflict for a decision (never touch it by default): + +```text +$ tbd docs update +Updated 2 forked doc(s): + ✓ review-code: refreshed to upstream (was unmodified) + ✓ python-rules: merged upstream cleanly (review with: git diff) + +1 doc(s) need a decision: + ⚠ acme-style: your changes conflict with upstream + re-run with one of: + tbd docs update --merge # combine, then resolve conflict markers + tbd docs update --keep-ours # keep your version, advance the fork point +? 0 + +$ tbd docs update acme-style --merge +Updated 1 forked doc(s): + ✓ acme-style: wrote merged content with conflict markers; resolve them, then it returns to 'customized' +? 0 + +$ tbd docs update acme-style --keep-ours +Updated 1 forked doc(s): + ✓ acme-style: kept your version; fork point advanced +? 0 + +$ tbd docs update --dry-run +Would update 2 forked doc(s): + ✓ review-code: refreshed to upstream (was unmodified) + ✓ python-rules: merged upstream cleanly (review with: git diff) +... +? 0 +``` + +### `tbd docs add` + +Aligned with the per-kind `--add` output (kept as aliases), restated for docrefs — the +canonical docref is what config records; git docrefs require an explicit `@ref`: + +```text +$ tbd docs add github:acme/eng-docs@main//guidelines/style.md --kind=guideline --name=acme-style +Adding guideline: acme-style + Source: github:acme/eng-docs@main//guidelines/style.md +✓ Added to .tbd/docs/guidelines/acme-style.md + Config updated (docs_cache.files): github:acme/eng-docs@main//guidelines/style.md + +Run 'tbd docs list' to verify, or 'tbd docs fork acme-style' to make it visible. +? 0 + +$ tbd docs add ./team/team-rules.md --kind=guideline # local docrefs work offline +Adding guideline: team-rules + Source: ./team/team-rules.md +✓ Added to .tbd/docs/guidelines/team-rules.md + Config updated (docs_cache.files): ./team/team-rules.md + +Run 'tbd docs list' to verify, or 'tbd docs fork team-rules' to make it visible. +? 0 +``` + +### `tbd status` (Docs line) and `tbd setup --auto` (Docs summary) + +`tbd status` gains a Docs line **only when forks exist** — so with zero forks the output +is byte-identical to today’s `cli-orientation-golden.tryscript.md` (honoring the +“default behavior unchanged when nothing is forked” guarantee). +When forks exist it appears after the Worktree line, before the footer: + +```text +$ tbd status # excerpt, forks present +[.. INTEGRATIONS, Worktree as today ..] + +Docs: 4 forked (2 customized, 3 with upstream updates — run 'tbd docs update') + +Use 'tbd stats' for issue statistics, 'tbd doctor' for health checks. +? 0 +``` + +`tbd setup --auto` prints the Docs summary (zero-fork = the menu; with forks = a +pending-update report); setup never writes the fork dir: + +```text +# zero forks — same three-posture menu as the bare overview, prefixed Docs: +Docs: [..] docs available in the cache (.tbd/docs/, gitignored); none forked into the repo. + Guidelines are active from the cache. Three postures, all serving the same docs: + Hidden (default): keep the cache as-is — zero repo footprint + Curated: tbd docs fork [...] fork chosen docs into docs/tbd/ + tbd docs fork --category= (general, typescript, python, convex, electron) + Everything: tbd docs fork --all all docs, visible and editable + Browse / read: tbd docs list / tbd docs show + +# after an upgrade, forks present +Docs: 1 forked into docs/tbd/. 1 have upstream updates — run 'tbd docs update'. +``` + +The setup menu and the bare-overview menu share wording by construction +(`docs-menu.ts`), and `tbd status` adds its one Docs line only when forks exist: + +```text +$ tbd status # excerpt, forks present +Docs: 4 forked (2 customized, 3 with upstream updates — run 'tbd docs update') +``` + +### `tbd doctor` (new HEALTH CHECKS) + +Appended to the existing `HEALTH CHECKS` list, following doctor’s `✓`/`⚠` + `Run:` +convention (icon at column 0, no indent): + +```text +$ tbd doctor # excerpt +[.. existing health checks ..] +✓ Forked docs - 4 forked, base snapshots intact +⚠ Forked docs - 1 unresolved merge conflict (acme-style) + Run: resolve the conflict markers, then re-run tbd docs update +✓ Fork dir - docs/tbd/ tracked in git (not gitignored) +✓ Reserved tbd- names - no user docs claim the prefix +? 0 +``` + +### Existing golden tests that change + +The plan’s original Testing Strategy listed only *new* tests. +These *existing* goldens break and must be rewritten in the same release; each is one +bead, blocked on the phase that ships the behavior: + +| Test | Change | Phase | +| --- | --- | --- | +| `cli-help-all.tryscript.md` (≈7 `tbd docs` blocks: `--help` `[topic]`/`--section`, `--list` sections, positional topic, `--section` content, `--list --json`, bare manual) | **Done (this PR).** Rewrite to the new surface: `docs` subcommand help; no top-level `--section`/section-`--list`; section nav becomes `tbd docs show tbd-docs --section`. Largest single change. | 1–2 | +| `cli-doc-output.tryscript.md` ("Docs Command" block: `tbd docs --list` → “Available documentation sections:”) | **Done (this PR).** Section listing retargeted to `tbd docs show tbd-docs --sections`; the cross-kind `tbd docs list` golden lives in `cli-docs-fork.tryscript.md`. | 2 | +| `cli-doc-output.tryscript.md` ("Guidelines --json returns structured data" block: flat `[ { … } ]` array) | **Done (this PR).** Rewrite: per-kind `--list --json` now emits a docmap object, not an array (Resolved Decision 21). The per-kind `--list` *text* blocks stay (same canonical format), so only the JSON assertion changes. | 2 | +| `golden-output.test.ts` (`tbd docs --all` inline snapshot) | **Done (this PR).** Replace with the bare `tbd docs` overview snapshot (`--all` folded into the overview). | 2 | +| `golden-output.test.ts` ("post-setup What’s Next") | **Done (this PR).** Extend to assert the Docs menu lines. | 4 | +| `cli-orientation-golden.tryscript.md` (`tbd status`) | **Unchanged** for zero forks (verifies the guarantee); **add** a new forked-state status golden in a fixture with a fork. | 1 | +| `setup-flows.test.ts` | **Done (this PR).** Extend for the Docs summary (menu + pending-update report). | 4 | +| `doc-references.test.ts` | **Done (this PR).** Extend the extractor: add `tbd docs ` and the `reference` kind; remove the `reference`/`prefix:` skips so `tbd docs show tbd-docs`, `suggest-upstream-improvements`, and the new reference docs all resolve. | 5 (extractor); 0 (the new docs it must resolve) | +| `doc-add-e2e.test.ts` | **Done (this PR).** Keep (per-kind `--add` stays an alias) and **extend** with `tbd docs add `. | 2 | + +New golden/e2e files (named for the phases that add them): `fork-manifest` + +state-matrix units (Phase 1); a `cli-docs-fork.tryscript.md` lifecycle (fork → list +marker → serve → edit → unfork refuse → `--force`) (Phase 1); `fork-update` +decision-table units + a `cli-docs-update.tryscript.md` upgrade/merge scenario (Phase +3); category-selection units + a `fork --category` e2e (Phase 4); a deleted-fork +scenario in the Phase 1 lifecycle test (serve falls back to upstream, status `missing`, +`doctor --fix` finalizes). + +## Testing Strategy + +- **Unit (vitest)**: manifest round-trip; hash normalization (CRLF/LF); the full state + matrix as a table-driven test (base hash × file hash × cache hash × conflicted flag → + state); the update decision table row by row and per strategy (replace, clean + three-way, conflict skip, `--merge` markers + base advance, `--keep-ours` keep-file + + base advance, strategy-flag mutual exclusion, conflicted-pending skip, missing-base + repair via `--keep-ours`, orphaned); marker auto-clear; f04→f05 migration; the ported + docref spec-mirror tests; local_dirs precedence ordering; source-root grouping (N docs + from one repo → one fetch; per-group failure isolation; cache preserved on fetch + failure); git revision/tag capture in the manifest; `--json` output validating against + the docmap schema; category-based fork selection; fork path mapping (incl. + shortcuts flattening); README index generation. +- **E2E (spawn against built CLI, like `doc-add-e2e.test.ts`)**: the Phase 1 scenario + above; precedence (forked shadows upstream; local file with no entry is served); an + upgrade simulation (fork → customize → mutate the cache copy to simulate a new + upstream version → `update` cleanly merges non-overlapping edits, then a conflicting + edit is listed and only merged with `--merge`, resolving markers returns the doc to + `customized`); convergence-unfork (upstream adopts the customization → `update` → + plain `unfork` succeeds); group surface (bare `tbd docs` shows status, + `tbd docs show tbd-docs` serves the manual, `tbd sync --docs` alias still works); + `fork --category=python`; out-of-band deletion of a forked file (serve falls back to + upstream, status `missing`, `doctor --fix` finalizes the unfork); collision/overwrite + refusal; doctor findings. +- **Docs-reference test**: extend `doc-references.test.ts` so every command named in the + new shortcut/docs resolves (extractor learns `tbd docs ` and the + `reference` kind; the `reference`/`prefix:` skips are removed). +- **Golden output**: the expected console output for every new and changed command is + specified in [Golden-Test Maps](#golden-test-maps) (the consistency contract plus a + per-command map against one canonical fixture). + Each map is lifted into its harness (tryscript or vitest inline snapshot) by the phase + that ships the command. +- **Existing goldens that break** are catalogued in + [Existing golden tests that change](#existing-golden-tests-that-change) — notably + `cli-help-all.tryscript.md`, `cli-doc-output.tryscript.md`, and the `tbd docs --all` + snapshot in `golden-output.test.ts`. Each rewrite is a bead blocked on the behavior’s + phase, so CI never carries a silently-failing golden. + A red-then-green pass on these (run the rewritten golden against the old binary to + confirm it fails, then against the new one) verifies the surface actually changed. + +## Relationship to PR #117 + +This spec deliberately implements the smallest forward-compatible slice of that redesign +(note: #117’s draft used “f05” as its format id; this spec’s layout bump now claims f05, +so the full framework would land as f06+): + +| #117 concept | Here | Future-proofing | +| --- | --- | --- | +| W8 fork / G4 local override | `tbd docs fork` + shadowing | fork dir becomes a `local` source in the source framework | +| G10 provenance / review pt. 4 “recorded override edge” | `.tbd/doc-forks/forks.yml` manifest | manifest entries become the framework’s override edges; fields chosen to match (source, hash, version) | +| W9 diff / G6 status | `tbd docs diff`/`status`, doctor | states map onto the framework’s status vocabulary | +| W10/W11 upstream roundtrip + unfork | playbook shortcut + convergence via `update` + `unfork` | can later be automated as `tbd docs upstream` | +| W5 sync vs. update separation; review pt. “record enough for three-way later” | cache refresh stays passive (`tbd docs sync`); `tbd docs update` is the explicit advance, with stored bases enabling three-way merge now | maps onto the framework’s `sync`/`source update` contract | +| G2 local project docs | `local` files in fork dir + `docs_cache.local_dirs` | becomes a first-class local source | +| docref format (design-docref-format.md) | **adopted wholesale**: parser ported, universal source-address grammar, shipped as a reference doc | the framework inherits addresses with no migration | +| docmap format (design-docmap-format.md) | **redefined and reduced**: the minimal inventory format (docmap/0.1) is extracted, freshly specced, and adopted as the `--json` contract | #117’s manifest/lockfile/sync becomes future “operations over docmaps,” consuming the same format | +| Source framework, lockfiles, DocGraph split, doc types as data | **not built** | unchanged decision space; nothing here constrains Q15–Q19 | + +Recommendation: keep PR #117 open as the long-horizon design reference (or convert its +spec to `specs/future/`), and note there that the fork kernel shipped separately. + +## References + +- `docs/project/specs/active/plan-2026-05-07-docs-config-redesign.md` (PR #117 branch + `claude/review-config-format-2wxh8`) — the full framework design and its review + thread. +- `packages/tbd/docs/design-docref-format.md` and + `packages/tbd/docs/design-docmap-format.md` (same #117 branch) — the docref grammar + and the exploratory docmap draft this spec distills into docmap/0.1, along with the + `src/docref/` reference implementation and tests. +- `src/file/doc-sync.ts`, `src/file/doc-cache.ts`, `src/file/doc-add.ts` — current + cache, lookup/shadowing, and add-by-URL implementations this builds on. +- Done specs: `plan-2026-01-22-doc-cache-abstraction.md`, + `plan-2026-01-26-configurable-doc-cache-sync.md`, + `plan-2026-02-02-external-docs-repos.md`. +- [shadcn/ui](https://ui.shadcn.com) — the “open code / copy and own” registry model + this borrows. + + diff --git a/lefthook.yml b/lefthook.yml index fa17f395..5e085759 100644 --- a/lefthook.yml +++ b/lefthook.yml @@ -52,19 +52,24 @@ pre-commit: priority: 3 pre-push: + # Every command runs behind scripts/scrub-git-env.mjs: git exports GIT_DIR into + # hook environments (always when pushing from a linked worktree), which would + # otherwise redirect any git/tbd subprocess the suite spawns onto the REAL + # repository — test fixtures then rewrite real refs and tbd data (tbd-a1lc). + # The vitest suite self-scrubs too (tests/scrub-git-env.ts); both layers stay. commands: # Build if needed - skips if dist is up-to-date (integration tests require dist/bin.mjs) build: - run: pnpm build:check + run: node scripts/scrub-git-env.mjs pnpm build:check priority: 1 # Then run tests test: - run: pnpm test + run: node scripts/scrub-git-env.mjs pnpm test priority: 2 # Enforce 14-day package-age rule on dependency pins. # See packages/tbd/docs/guidelines/pnpm-monorepo-patterns.md#supply-chain-mitigation # Only runs when a package.json is staged so day-to-day pushes aren't slowed by registry lookups. package-age: glob: '{package.json,packages/*/package.json}' - run: pnpm check:package-age + run: node scripts/scrub-git-env.mjs pnpm check:package-age priority: 3 diff --git a/packages/tbd/CHANGELOG.md b/packages/tbd/CHANGELOG.md index 11cb0e1d..4864b59d 100644 --- a/packages/tbd/CHANGELOG.md +++ b/packages/tbd/CHANGELOG.md @@ -1,5 +1,74 @@ # get-tbd +## Unreleased (0.3.0) + +The headline is **forkable docs**: every doc tbd serves (guidelines, shortcuts, +templates, and the new reference docs) can now be forked into your repo as visible, +git-tracked files, edited in place, and reconciled with upstream after upgrades. +The repository format bumps `f04` → `f05`; the migration is **stamp-only** (it stamps +`tbd_format: f05` in `.tbd/config.yml` and refreshes generated metadata — fork artifacts +appear only when you first fork something), so upgrading is safe and revertible. + +### Features + +- **Forkable docs** (`tbd docs fork` / `unfork` / `update` / `diff` / `status`): fork + any managed doc into a visible fork dir (`docs/tbd/`, laid out by kind with a + generated `README.md` index, tracked in git). + Your copy shadows the hidden cache everywhere the doc is served; forking changes + nothing about how docs work — it only makes them explicit and editable. + - `tbd docs update` three-way merges upstream changes into your copies after an + upgrade: `--merge` combines and writes conflict markers to resolve, `--keep-ours` + keeps your version and advances the fork point; `--dry-run` previews and lists + conflicts. + - `tbd docs diff ` compares your copy against upstream (default), against its + recorded base (`--base`, what you changed), or base against upstream (`--upstream`, + incoming changes). + - `tbd docs status` reports every fork’s state (`forked`, `customized`, `conflicted`, + `missing`, plus `local` for your own files in the fork dir). + States are recomputed from content hashes — no git operation can desynchronize + tracking. Fork state lives in the committed `.tbd/doc-forks/` (a `forks.yml` manifest + plus `base/` snapshots); the fork dir itself stays outside `.tbd/`. +- **The `tbd docs` surface is re-homed around managed docs**: bare `tbd docs` is now the + status overview, and `tbd docs list` lists all docs across kinds with + `[forked]`/`[customized]`/`[local]` markers. + The CLI manual moved to `tbd docs show tbd-docs` (alias: `tbd docs manual`); the old + `tbd docs --list` / `--all` / `--section` flags are retired in favor of + `tbd docs show tbd-docs --sections` / `--section `. `tbd docs show ` reads + any doc by name, kind-agnostically. + `tbd docs sync` refreshes the gitignored docs cache (`tbd sync --docs` remains as a + deprecated alias). +- **docref + docmap formats, and a new `reference` doc kind**: every doc source is + addressed by a **docref** — one URI-like grammar (`internal:…`, anchored local paths, + URLs, `github:owner/repo@ref//path`) used for `docs_cache.files` values and + fork-manifest `source` values alike — and every doc listing is one **docmap** + (`docmap/0.1`) rendered as text or `--json`: `tbd docs list` / `tbd docs status`, the + bare-`tbd docs` overview, and the per-kind `--list` (whose `--json` output changes + from a flat array to a docmap). + Both formats ship as docs of the new `reference` kind — read them with + `tbd docs show docref-format` and `tbd docs show docmap-format` — alongside the manual + (`tbd-docs`) and design doc (`tbd-design`), which are now managed docs too. +- **Fork drift is visible, never auto-fixed**: `tbd status` gains a `Docs:` line when + forks exist (forked/customized counts, pending upstream updates, conflicts, missing + files), and `tbd sync` prints a one-line notice when forked docs are stale, + conflicted, or missing. + Only the explicit `tbd docs update` ever modifies tracked files. + +### Guidelines and content + +These ship inside the package and are read by agents via `tbd docs show …`, +`tbd shortcut …`, and `tbd setup`: + +- **New `suggest-upstream-improvements` shortcut**: the playbook for reviewing fork + customizations (`tbd docs status --json`, `tbd docs diff --base`), deciding + what generalizes, contributing it upstream, and re-syncing with `tbd docs update` once + merged. +- **New `docref-format` and `docmap-format` reference docs**: the specifications for the + two formats above, forkable like any other doc. +- **Onboarding and agent surface updated for forkable docs**: `welcome-user` now makes + the two-axis offer (scope: all standard guidelines or a stack subset; visibility: + hidden cache or forked into `docs/tbd/`), and the agent skill routes fork, update, and + missing-file requests to the new commands. + ## 0.2.3 A drop-in patch on top of v0.2.2. **No on-disk format change** (`f04` stays `f04`), so diff --git a/packages/tbd/docs/guidelines/backward-compatibility-rules.md b/packages/tbd/docs/guidelines/backward-compatibility-rules.md index cf4be0a4..cae8f1da 100644 --- a/packages/tbd/docs/guidelines/backward-compatibility-rules.md +++ b/packages/tbd/docs/guidelines/backward-compatibility-rules.md @@ -2,6 +2,7 @@ title: Backward Compatibility Rules description: Guidelines for maintaining backward compatibility across code, APIs, file formats, and database schemas author: Joshua Levy (github.com/jlevy) with LLM assistance +category: general --- ## Backward Compatibility Guidelines diff --git a/packages/tbd/docs/guidelines/bun-monorepo-patterns.md b/packages/tbd/docs/guidelines/bun-monorepo-patterns.md index d7b80e97..1213ae91 100644 --- a/packages/tbd/docs/guidelines/bun-monorepo-patterns.md +++ b/packages/tbd/docs/guidelines/bun-monorepo-patterns.md @@ -2,6 +2,7 @@ title: Bun Monorepo Patterns description: Modern patterns for Bun-based TypeScript monorepo architecture author: Joshua Levy (github.com/jlevy) with LLM assistance +category: typescript --- # Bun Monorepo Patterns diff --git a/packages/tbd/docs/guidelines/cli-agent-skill-patterns.md b/packages/tbd/docs/guidelines/cli-agent-skill-patterns.md index 3ba21c60..5a95865d 100644 --- a/packages/tbd/docs/guidelines/cli-agent-skill-patterns.md +++ b/packages/tbd/docs/guidelines/cli-agent-skill-patterns.md @@ -2,6 +2,7 @@ title: Agent Skills and CLI Integration Patterns description: How to write skills and agent-integrated CLIs that work across Claude Code, Codex, and the broader coding-agent ecosystem—a simple baseline plus references for advanced, multi-subcommand tools author: Joshua Levy (github.com/jlevy) with LLM assistance +category: general --- # Agent Skills and CLI Integration Patterns @@ -622,7 +623,7 @@ Rules: reference commands **explicitly** (`mycli command arg`, never “see the - **`--json` on every command**—one output path that renders human or machine output. - **`--brief`/`--quiet`** for constrained contexts and scripts. - **Idempotent `setup --auto`** (non-interactive) vs. - `setup --interactive` for humans; never let an agent get stuck on a prompt. + a guided setup for humans; never let an agent get stuck on a prompt. - **Actionable errors** that include the next command to run. - **Discoverable help**: an `IMPORTANT:` epilog pointing at a context-restore command (e.g., `mycli prime`), and a “Getting Started” one-liner. diff --git a/packages/tbd/docs/guidelines/commit-conventions.md b/packages/tbd/docs/guidelines/commit-conventions.md index 0af7e892..a6422cd6 100644 --- a/packages/tbd/docs/guidelines/commit-conventions.md +++ b/packages/tbd/docs/guidelines/commit-conventions.md @@ -2,6 +2,7 @@ title: Commit Conventions description: Conventional Commits format with extensions for agentic workflows author: Joshua Levy (github.com/jlevy) with LLM assistance +category: general --- # Commit Conventions diff --git a/packages/tbd/docs/guidelines/common-doc-guidelines.md b/packages/tbd/docs/guidelines/common-doc-guidelines.md index c32f7735..bd212700 100644 --- a/packages/tbd/docs/guidelines/common-doc-guidelines.md +++ b/packages/tbd/docs/guidelines/common-doc-guidelines.md @@ -2,6 +2,7 @@ title: Common Documentation Guidelines description: Common cross-project standards for writing and organizing docs, code comments, and text files—how to organize, structure, write, and format documents, plus the guideline footer convention. Downstream of github.com/jlevy/practical-prose. Use whenever writing or editing any documentation, README, guideline, or design doc. author: Joshua Levy (github.com/jlevy) with LLM assistance +category: general --- # Common Documentation Guidelines diff --git a/packages/tbd/docs/guidelines/convex-limits-best-practices.md b/packages/tbd/docs/guidelines/convex-limits-best-practices.md index f7635666..aa5cdf7d 100644 --- a/packages/tbd/docs/guidelines/convex-limits-best-practices.md +++ b/packages/tbd/docs/guidelines/convex-limits-best-practices.md @@ -2,6 +2,7 @@ title: Convex Limits and Best Practices description: Comprehensive reference for Convex platform limits, workarounds, and performance best practices author: Joshua Levy (github.com/jlevy) with LLM assistance +category: convex --- # Research Brief: Convex Database Limits, Best Practices, and Workarounds diff --git a/packages/tbd/docs/guidelines/convex-rules.md b/packages/tbd/docs/guidelines/convex-rules.md index 03d332bb..00b99593 100644 --- a/packages/tbd/docs/guidelines/convex-rules.md +++ b/packages/tbd/docs/guidelines/convex-rules.md @@ -2,6 +2,7 @@ title: Convex Rules description: Guidelines and best practices for building Convex projects, including database schema design, queries, mutations, and real-world examples author: Convex team +category: convex --- # Convex Guidelines diff --git a/packages/tbd/docs/guidelines/electron-app-development-patterns.md b/packages/tbd/docs/guidelines/electron-app-development-patterns.md index 600beb3f..685d3498 100644 --- a/packages/tbd/docs/guidelines/electron-app-development-patterns.md +++ b/packages/tbd/docs/guidelines/electron-app-development-patterns.md @@ -1,6 +1,7 @@ --- title: Electron App Development Patterns description: Guidelines for Electron development ecosystems including npm, pnpm, and Bun, with security baselines and framework comparisons +category: electron --- # Electron App Development Patterns diff --git a/packages/tbd/docs/guidelines/error-handling-rules.md b/packages/tbd/docs/guidelines/error-handling-rules.md index 4d29df0c..fc4f4183 100644 --- a/packages/tbd/docs/guidelines/error-handling-rules.md +++ b/packages/tbd/docs/guidelines/error-handling-rules.md @@ -2,6 +2,7 @@ title: Error Handling Rules description: Rules for handling errors, failures, and exceptional conditions author: Joshua Levy (github.com/jlevy) with LLM assistance +category: general --- # Error Handling Rules diff --git a/packages/tbd/docs/guidelines/general-coding-rules.md b/packages/tbd/docs/guidelines/general-coding-rules.md index e10707f5..b48d3c2b 100644 --- a/packages/tbd/docs/guidelines/general-coding-rules.md +++ b/packages/tbd/docs/guidelines/general-coding-rules.md @@ -2,6 +2,7 @@ title: General Coding Rules description: Rules for constants, magic numbers, and general coding practices author: Joshua Levy (github.com/jlevy) with LLM assistance +category: general --- # General Coding Rules diff --git a/packages/tbd/docs/guidelines/general-comment-rules.md b/packages/tbd/docs/guidelines/general-comment-rules.md index def30012..8f393aac 100644 --- a/packages/tbd/docs/guidelines/general-comment-rules.md +++ b/packages/tbd/docs/guidelines/general-comment-rules.md @@ -2,6 +2,7 @@ title: General Comment Rules description: Language-agnostic rules for writing clean, maintainable comments author: Joshua Levy (github.com/jlevy) with LLM assistance +category: general --- # General Comment Rules diff --git a/packages/tbd/docs/guidelines/general-eng-agent-principles.md b/packages/tbd/docs/guidelines/general-eng-agent-principles.md index 21281415..743c6f8b 100644 --- a/packages/tbd/docs/guidelines/general-eng-agent-principles.md +++ b/packages/tbd/docs/guidelines/general-eng-agent-principles.md @@ -2,6 +2,7 @@ title: Engineering Agent Principles description: Core principles for AI agents acting as senior engineers—objectivity and communication conduct plus the engineering process (detailed understanding, verification, end-to-end ownership, scope discipline, tracking future work, and acting versus seeking clarification) author: Joshua Levy (github.com/jlevy) with LLM assistance +category: general --- # Engineering Agent Principles diff --git a/packages/tbd/docs/guidelines/general-tdd-guidelines.md b/packages/tbd/docs/guidelines/general-tdd-guidelines.md index 5db475d3..f9a1c585 100644 --- a/packages/tbd/docs/guidelines/general-tdd-guidelines.md +++ b/packages/tbd/docs/guidelines/general-tdd-guidelines.md @@ -2,6 +2,7 @@ title: TDD Guidelines description: Test-Driven Development methodology and best practices author: Joshua Levy (github.com/jlevy) with LLM assistance +category: general --- # Test-Driven Development (TDD) Guidelines diff --git a/packages/tbd/docs/guidelines/general-testing-rules.md b/packages/tbd/docs/guidelines/general-testing-rules.md index f1372bef..bad21e4c 100644 --- a/packages/tbd/docs/guidelines/general-testing-rules.md +++ b/packages/tbd/docs/guidelines/general-testing-rules.md @@ -2,6 +2,7 @@ title: General Testing Rules description: Rules for writing minimal, effective tests with maximum coverage author: Joshua Levy (github.com/jlevy) with LLM assistance +category: general --- ## General Testing Rules diff --git a/packages/tbd/docs/guidelines/golden-testing-guidelines.md b/packages/tbd/docs/guidelines/golden-testing-guidelines.md index c79b1075..4159d1c9 100644 --- a/packages/tbd/docs/guidelines/golden-testing-guidelines.md +++ b/packages/tbd/docs/guidelines/golden-testing-guidelines.md @@ -2,6 +2,7 @@ title: Golden Testing Guidelines description: Guidelines for implementing golden/snapshot testing for complex systems author: Joshua Levy (github.com/jlevy) with LLM assistance +category: general --- # Golden Testing Guidelines diff --git a/packages/tbd/docs/guidelines/pnpm-monorepo-patterns.md b/packages/tbd/docs/guidelines/pnpm-monorepo-patterns.md index 8da94df7..ae3456d1 100644 --- a/packages/tbd/docs/guidelines/pnpm-monorepo-patterns.md +++ b/packages/tbd/docs/guidelines/pnpm-monorepo-patterns.md @@ -2,6 +2,7 @@ title: pnpm Monorepo Patterns description: Modern patterns for pnpm-based TypeScript monorepo architecture author: Joshua Levy (github.com/jlevy) with LLM assistance +category: typescript --- # pnpm Monorepo Patterns diff --git a/packages/tbd/docs/guidelines/python-cli-patterns.md b/packages/tbd/docs/guidelines/python-cli-patterns.md index d53c4dd8..974fef60 100644 --- a/packages/tbd/docs/guidelines/python-cli-patterns.md +++ b/packages/tbd/docs/guidelines/python-cli-patterns.md @@ -2,6 +2,7 @@ title: Python CLI Patterns description: Modern patterns for Python CLI application architecture author: Joshua Levy (github.com/jlevy) with LLM assistance +category: python --- # Python CLI Patterns diff --git a/packages/tbd/docs/guidelines/python-modern-guidelines.md b/packages/tbd/docs/guidelines/python-modern-guidelines.md index 93700ab1..c46df4db 100644 --- a/packages/tbd/docs/guidelines/python-modern-guidelines.md +++ b/packages/tbd/docs/guidelines/python-modern-guidelines.md @@ -2,6 +2,7 @@ title: Python Modern Guidelines description: Guidelines for modern Python projects using uv, with a few more opinionated practices author: Joshua Levy (github.com/jlevy) with LLM assistance +category: python --- # Python Modern Guidelines diff --git a/packages/tbd/docs/guidelines/python-rules.md b/packages/tbd/docs/guidelines/python-rules.md index db71f67e..a271601c 100644 --- a/packages/tbd/docs/guidelines/python-rules.md +++ b/packages/tbd/docs/guidelines/python-rules.md @@ -2,6 +2,7 @@ title: Python Rules description: General Python coding rules and best practices author: Joshua Levy (github.com/jlevy) with LLM assistance +category: python --- # Python Rules diff --git a/packages/tbd/docs/guidelines/release-notes-guidelines.md b/packages/tbd/docs/guidelines/release-notes-guidelines.md index db8bb52a..79ebb9ed 100644 --- a/packages/tbd/docs/guidelines/release-notes-guidelines.md +++ b/packages/tbd/docs/guidelines/release-notes-guidelines.md @@ -1,6 +1,7 @@ --- title: Release Notes Guidelines description: Guidelines for writing clear, accurate release notes +category: general --- # Release Notes Guidelines diff --git a/packages/tbd/docs/guidelines/supply-chain-hardening.md b/packages/tbd/docs/guidelines/supply-chain-hardening.md index f1bdb1f2..70212b48 100644 --- a/packages/tbd/docs/guidelines/supply-chain-hardening.md +++ b/packages/tbd/docs/guidelines/supply-chain-hardening.md @@ -2,6 +2,7 @@ title: Supply-Chain Hardening description: Strongly recommended for EVERY repo—apply it if a repo has not been hardened yet. Cross-ecosystem policy for installing dependencies safely (the 14-day cool-off, disabled install scripts, lockfile discipline, untrusted-repo handling). Use whenever a user mentions hardening, security, supply chain, or setting up a new repo; before adding/upgrading dependencies; when auditing for compromised packages; or when reviewing install/build/run commands across npm/pnpm, PyPI, Cargo, or Go. author: Joshua Levy (github.com/jlevy) with LLM assistance +category: general --- # Supply-Chain Hardening @@ -244,8 +245,8 @@ process.exit(violations > 0 ? 1 : 0); **Exception bookkeeping**: when you pin a fresh version under the exception process, leave a marker next to the pin (JSONC comment in `package.json`, or a `CHANGELOG.md` -note for strict JSON parsers): `// Exception: CVE-2026-XXXX patch within 14d window. -Reviewed .` +note for strict JSON parsers): +`// Exception: CVE-2026-XXXX patch within 14d window. Reviewed .` ## Untrusted Repos and Modes diff --git a/packages/tbd/docs/guidelines/tbd-sync-troubleshooting.md b/packages/tbd/docs/guidelines/tbd-sync-troubleshooting.md index fe666f8c..101f9101 100644 --- a/packages/tbd/docs/guidelines/tbd-sync-troubleshooting.md +++ b/packages/tbd/docs/guidelines/tbd-sync-troubleshooting.md @@ -2,6 +2,7 @@ title: tbd Sync and Workspace Troubleshooting description: Common issues and solutions for tbd sync and workspace operations author: Joshua Levy (github.com/jlevy) with LLM assistance +category: general --- ## Common Sync Issues diff --git a/packages/tbd/docs/guidelines/typescript-cli-tool-rules.md b/packages/tbd/docs/guidelines/typescript-cli-tool-rules.md index 9d3f5c79..29204a77 100644 --- a/packages/tbd/docs/guidelines/typescript-cli-tool-rules.md +++ b/packages/tbd/docs/guidelines/typescript-cli-tool-rules.md @@ -2,6 +2,7 @@ title: TypeScript CLI Tool Rules description: Rules for building CLI tools with Commander.js, picocolors, and TypeScript author: Joshua Levy (github.com/jlevy) with LLM assistance +category: typescript --- # CLI Tool Development Rules diff --git a/packages/tbd/docs/guidelines/typescript-code-coverage.md b/packages/tbd/docs/guidelines/typescript-code-coverage.md index ebd0b483..de7625ad 100644 --- a/packages/tbd/docs/guidelines/typescript-code-coverage.md +++ b/packages/tbd/docs/guidelines/typescript-code-coverage.md @@ -2,6 +2,7 @@ title: TypeScript Code Coverage description: Best practices for code coverage in TypeScript with Vitest and v8 provider author: Joshua Levy (github.com/jlevy) with LLM assistance +category: typescript --- # Code Coverage Best Practices for TypeScript with Vitest diff --git a/packages/tbd/docs/guidelines/typescript-rules.md b/packages/tbd/docs/guidelines/typescript-rules.md index 3506bfe9..c0a015e1 100644 --- a/packages/tbd/docs/guidelines/typescript-rules.md +++ b/packages/tbd/docs/guidelines/typescript-rules.md @@ -4,6 +4,7 @@ description: TypeScript coding rules and best practices author: Joshua Levy (github.com/jlevy) with LLM assistance globs: "*.ts" alwaysApply: true +category: typescript --- # TypeScript Rules diff --git a/packages/tbd/docs/guidelines/typescript-sorting-patterns.md b/packages/tbd/docs/guidelines/typescript-sorting-patterns.md index 52d75f39..80573da1 100644 --- a/packages/tbd/docs/guidelines/typescript-sorting-patterns.md +++ b/packages/tbd/docs/guidelines/typescript-sorting-patterns.md @@ -2,6 +2,7 @@ title: TypeScript Sorting Patterns description: Deterministic sorting patterns and comparison chains for TypeScript author: Joshua Levy (github.com/jlevy) with LLM assistance +category: typescript --- # TypeScript Sorting Patterns diff --git a/packages/tbd/docs/guidelines/typescript-yaml-handling-rules.md b/packages/tbd/docs/guidelines/typescript-yaml-handling-rules.md index 7e5e1629..db804cbb 100644 --- a/packages/tbd/docs/guidelines/typescript-yaml-handling-rules.md +++ b/packages/tbd/docs/guidelines/typescript-yaml-handling-rules.md @@ -3,6 +3,7 @@ title: TypeScript YAML Handling Rules description: Best practices for parsing and serializing YAML in TypeScript author: Joshua Levy (github.com/jlevy) with LLM assistance globs: "*.ts" +category: typescript --- # TypeScript YAML Handling Rules diff --git a/packages/tbd/docs/references/docmap-format.md b/packages/tbd/docs/references/docmap-format.md new file mode 100644 index 00000000..af7ce740 --- /dev/null +++ b/packages/tbd/docs/references/docmap-format.md @@ -0,0 +1,64 @@ +--- +title: Docmap Format +description: A minimal, machine-readable inventory of a collection of documents — a sitemap for docs, with docref as its addressing primitive +author: Joshua Levy (github.com/jlevy) with LLM assistance +category: general +--- +# Docmap Format (docmap/0.1) + +A **docmap** is a machine-readable inventory of a collection of documents: one entry per +doc, each with an identity, a location, and presentation metadata. +It describes a collection; it says nothing about how the collection is assembled, +fetched, or kept fresh. +A docmap is a generated **view** of a collection, never an input to resolution — tools +that serve docs resolve by their own conventions and *emit* a docmap (as +`tbd docs list --json` does); future machinery that consumes docmaps as sources is +defined as operations *over* this format, not as part of it. + +## Shape + +```yaml +docmap: docmap/0.1 +name: tbd-docs # optional collection name +documents: + - name: python-rules + type: guideline + path: guidelines/python-rules.md # location within the collection + source: internal:guidelines/python-rules.md # provenance docref + title: Python Coding Rules + description: Type hints, docstrings, exception handling +``` + +Rules: + +- **Identity**: `type` + `name`, unique within the map. + `type` is an open vocabulary (tbd uses `guideline` / `shortcut` / `template` / + `reference`). +- **Location**: every entry carries `path` and/or `source` (a + [docref](docref-format.md)). An inventory whose entries cannot be located is not an + inventory. +- **Path relativity**: for a docmap committed as a file, `path` is relative to the + docmap file’s own directory (the sitemap convention); generated docmaps state their + collection root out of band (tbd’s `--json` paths are repo-relative). +- **Presentation metadata**: `title` and `description` are the core fields. +- **Extension fields**: producers may attach anything else — tbd adds `state` and + `stale`; size metrics (`word_count`, `size_bytes`, token estimates) are likewise + extensions, not core. + **Consumers must ignore unknown fields.** + +## Versioning + +The `docmap:` value is the format tag. +Readers accept `docmap/0.*` and reject other majors with a clear error: a different +major may change field semantics, and failing fast beats misreading. + +## Reference Implementation + +`src/docmap/` in tbd: standalone, dependency-free schema, validation, and query helpers, +structured for extraction into its own package. +Producers may generate docmaps (every tbd list/inventory command emits one) or +hand-author them — any repo can commit a docmap to advertise its doc collection. + + diff --git a/packages/tbd/docs/references/docref-format.md b/packages/tbd/docs/references/docref-format.md new file mode 100644 index 00000000..ad4b82d5 --- /dev/null +++ b/packages/tbd/docs/references/docref-format.md @@ -0,0 +1,71 @@ +--- +title: Docref Format +description: A single-string, URI-like address for any document — the one source-address grammar used across tbd +author: Joshua Levy (github.com/jlevy) with LLM assistance +category: general +--- +# Docref Format (v0.1) + +A **docref** is a single-string, URI-like address for a document. +It is the one address syntax used everywhere tbd names where a doc comes from or lives: +`docs_cache.files` values, the fork manifest’s `source` field, and future `tbd docs add` +arguments. The grammar is tool-agnostic: any application can adopt it, and the reference +implementation (`src/docref/` in tbd) is standalone and dependency-free. + +## Forms + +| Form | Example | Meaning | +| --- | --- | --- | +| internal | `internal:guidelines/python-rules.md` | A doc bundled inside the consuming tool. App-relative: each tool resolves it against its own bundled collection. | +| local | `./docs/general/`, `../shared/rules.md`, `/abs/f.md`, `C:/docs/f.md` | A filesystem path. Must be **anchored**: `./`, `../`, `/`, or a Windows drive letter. | +| url | `https://example.com/style.md` | A plain URL, kept verbatim. | +| git | `github:owner/repo@ref//path/to/file.md` | A file in a git host’s repo. `gitlab:` likewise. `@ref` is optional; `//` separates repo from path. | +| git + fragment | `github:o/r@main//f.md#naming` | Optional in-document anchor, preserved verbatim. | + +The `//` separator makes refs with slashes unambiguous: +`github:o/r@feature/x//docs/f.md` pins ref `feature/x` — unlike GitHub blob URLs, where +ref and path cannot be split reliably. + +## Strictness + +The grammar is deliberately strict; consumers may be lenient at their own boundary: + +- **Bare relative strings are not docrefs** (`guidelines/x.md` is invalid). + A consumer that wants to accept them may prepend `./` before parsing. + A strict grammar plus lenient consumers composes; the reverse can never be tightened. +- **Home-relative paths (`~/…`) are rejected** in v0.1 (no portable expansion + semantics). +- **Unknown schemes are rejected** (`mailto:…`, `git:…`). Additional protocols — for + example a host-bearing git scheme for forges beyond GitHub/GitLab — may be added in + future versions. + +## Normalization + +Web URLs that point at a known git host’s file view normalize to the canonical scheme, +so one file has one address: + +- `https://github.com/o/r/blob/main/f.md` → `github:o/r@main//f.md` +- `https://raw.githubusercontent.com/o/r/main/f.md` → `github:o/r@main//f.md` +- `https://gitlab.com/o/r/-/blob/main/f.md` → `gitlab:o/r@main//f.md` + +URL fragments are preserved through normalization — a normalizer must never silently +drop data. + +## Equality + +Two docrefs are equal when their canonical forms are identical, except that local paths +compare with a single leading `./` ignored. +Equality is purely syntactic: hosts and owners are not case-normalized, and no network +or filesystem is consulted. + +## Prior Art + +[purl](https://github.com/package-url/purl-spec) addresses *packages* +(`pkg:type/namespace/name@version`); its identity is the package, with file paths as an +awkward suffix. docref’s identity is the *document*, with in-repo paths and anchored +local files as first-class forms — which is why a separate small grammar exists rather +than a purl profile. + + diff --git a/packages/tbd/docs/shortcuts/standard/suggest-upstream-improvements.md b/packages/tbd/docs/shortcuts/standard/suggest-upstream-improvements.md new file mode 100644 index 00000000..b4012c9b --- /dev/null +++ b/packages/tbd/docs/shortcuts/standard/suggest-upstream-improvements.md @@ -0,0 +1,69 @@ +--- +title: Suggest Upstream Improvements +description: Review local doc-fork customizations and contribute the generally useful changes back upstream +category: meta +author: Joshua Levy (github.com/jlevy) with LLM assistance +--- +This shortcut reviews the docs this project has forked and customized (in `docs/tbd/`), +decides which changes generalize beyond this project, and proposes them upstream — to +the tbd repo for tbd’s bundled docs, or to your org’s docs repo for docs added by URL. + +## When to Use + +- Forked guidelines or shortcuts have accumulated edits that would help other projects. +- Before unforking: upstream the good parts first, so nothing is lost when the fork goes + away. +- After `tbd docs update` keeps producing the same conflict because upstream lacks an + improvement you made locally. + +## Instructions + +Create a to-do list with the following items then perform all of them: + +1. **Collect customizations**: Run `tbd docs status --json` and collect every doc in + `customized` state (including customized docs that also have upstream updates + pending). + +2. **Review each diff**: For each doc, run `tbd docs diff --base` (your file vs + its recorded base — exactly what this project changed). + Classify each hunk: + - **Generally applicable**: fixes, clarifications, better examples, rules any project + would benefit from. Candidates for upstreaming. + - **Project-specific**: team conventions, internal links, stack-specific overrides. + These stay in the fork; do not propose them. + +3. **Identify the upstream target** for each doc with generalizable hunks: + - Bundled docs (manifest `source` starts with `internal:`): the tbd repo + (`jlevy/tbd`), where doc sources live under `packages/tbd/docs/`. + - URL-added docs: the repo their docref points at (for example, your org’s shared + docs repo). + +4. **Draft the proposal**: For each doc, draft an issue or PR body containing: + - Which doc (name and kind) and why the change is generally useful. + - Only the generalizable diff hunks, in fenced code blocks. + - Brief project context: what prompted the change here. + +5. **Confirm, then file**: Show the draft to the user and get confirmation. + Then: + - Issue: `gh issue create -R jlevy/tbd` (or the org’s repo) with the drafted body. + - PR (preferred when the change is small and ready): apply the generalizable hunks to + the upstream source file on a branch and open a PR with `gh pr create`. + +6. **Close the loop (after upstream merges)**: Once the change ships upstream and tbd is + upgraded, run `tbd docs update`. If upstream adopted the customization, the merge + converges and the doc returns to unmodified `forked` state — then a plain + `tbd docs unfork ` (no `--force` needed) completes the cleanup, or keep the + fork for future edits. + +## Notes + +- Only propose hunks you would accept as a maintainer: each should stand alone, with + rationale. +- Never file an issue or PR without showing the user the draft first. +- Diff views: `tbd docs diff ` (no flag) compares your copy against current + upstream; `--base` is the right view for “what did we change”; `--upstream` shows + incoming upstream changes. + + diff --git a/packages/tbd/docs/shortcuts/standard/welcome-user.md b/packages/tbd/docs/shortcuts/standard/welcome-user.md index a2c3feba..e9e1d25c 100644 --- a/packages/tbd/docs/shortcuts/standard/welcome-user.md +++ b/packages/tbd/docs/shortcuts/standard/welcome-user.md @@ -13,6 +13,19 @@ project. First, run `tbd status` to check the current state. Give a brief summary of the status (repository, sync status, integrations). +Then make the two-axis guidelines offer — one short question per axis: + +1. **Scope** — keep **all** standard guidelines active (recommended), or just a subset + for this project’s languages and stack? +2. **Visibility** — leave them in tbd’s hidden cache (the default — they just work), or + fork them into `docs/tbd/` so they are visible on GitHub, reviewable in PRs, and + editable (checked into git)? + +Explain that forking changes nothing about how guidelines work — both paths serve the +same guidelines — it only makes them visible and customizable. +If the user wants visible docs, run `tbd docs fork [...]` for the chosen +subset or `tbd docs fork --all` for everything (preview with `--dry-run` first). + Then show the welcome message: * * * @@ -50,6 +63,7 @@ Here are examples of things you can say and what happens: | “I’m building a TypeScript CLI” | Applies TypeScript CLI guidelines | | “Help me set up better testing” | Applies testing guidelines | | “What are the Python best practices?” | Applies Python guidelines | +| “Make the guidelines visible in my repo” | Forks them into `docs/tbd/` (`tbd docs fork ` or `tbd docs fork --all`) | **Tips:** diff --git a/packages/tbd/docs/shortcuts/system/skill-baseline.md b/packages/tbd/docs/shortcuts/system/skill-baseline.md index b863d178..608269b9 100644 --- a/packages/tbd/docs/shortcuts/system/skill-baseline.md +++ b/packages/tbd/docs/shortcuts/system/skill-baseline.md @@ -17,12 +17,15 @@ description: Full tbd workflow guide for agents ## Installation ```bash -npm install -g get-tbd@latest +npm install -g get-tbd@latest # Install or upgrade the CLI (same command for both) tbd setup --auto --prefix= # Fresh project (--prefix is REQUIRED: 2-8 alphabetic chars recommended. ALWAYS ASK THE USER FOR THE PREFIX; do not guess it) -tbd setup --auto # Existing tbd project (prefix already set) +tbd setup --auto # Existing tbd project — also the upgrade step (applies any format migration; commit the diff it reports) tbd setup --from-beads # Migration from .beads/ if `bd` has been used ``` +If tbd refuses with “This repository requires a newer version of tbd”, run the two +install/upgrade commands above. + ## Routine Commands ```bash @@ -82,6 +85,10 @@ or want help → run `tbd shortcut welcome-user` | **Documentation** | | | “Research this topic” | `tbd shortcut new-research-brief` | | “Document architecture” | `tbd shortcut new-architecture-doc` | +| “What guidelines/docs are there?” | `tbd docs list` | +| “Make the guidelines visible / customize doc X” | `tbd docs fork ` (or `--all`), then edit in `docs/tbd/` | +| “Update the guidelines to the latest” | `tbd docs update`; on conflicts ask the user, then `--merge` or `--keep-ours` | +| “I deleted a forked doc file” | `tbd docs status` shows it `missing`; restore with `tbd docs fork --force` or finalize with `tbd docs unfork ` | | **Cleanup & Maintenance** | | | “Clean up this code” / “Remove dead code” | `tbd shortcut code-cleanup-all` | | “Fix repository problems” | `tbd doctor --fix` | @@ -171,6 +178,8 @@ working branch. See `tbd guidelines tbd-sync-troubleshooting` for details. | `tbd guidelines ` | Load coding guidelines | | `tbd guidelines --list` | List guidelines | | `tbd template ` | Output a template | +| `tbd docs` / `tbd docs list` | Managed-docs overview / cross-kind list with state markers | +| `tbd docs fork/unfork/update ` | Fork docs into `docs/tbd/`, return to upstream, pull upstream updates | ## Quick Reference diff --git a/packages/tbd/docs/tbd-design.md b/packages/tbd/docs/tbd-design.md index 4635cf99..20bb6f64 100644 --- a/packages/tbd/docs/tbd-design.md +++ b/packages/tbd/docs/tbd-design.md @@ -71,6 +71,7 @@ agents. - [2.8.5 Comparison with Beads](#285-comparison-with-beads) - [2.8.6 Future Dependency Types](#286-future-dependency-types) - [2.8.7 Future: Transitive Blocking Option](#287-future-transitive-blocking-option) + - [2.9 Managed Docs: Copies, Forks, and Synchronization](#29-managed-docs-copies-forks-and-synchronization) - [3. Git Layer](#3-git-layer) - [3.1 Overview](#31-overview) - [3.2 Sync Branch Architecture](#32-sync-branch-architecture) @@ -119,6 +120,7 @@ agents. - [4.10 Global Options](#410-global-options) - [4.11 Attic Commands](#411-attic-commands) - [4.12 Output Formats](#412-output-formats) + - [4.13 Docs Commands](#413-docs-commands) - [5. Beads Compatibility](#5-beads-compatibility) - [5.1 Import Strategy](#51-import-strategy) - [5.1.1 Import Command](#511-import-command) @@ -1968,6 +1970,107 @@ Add transitive blocking as an opt-in feature if users request it after real-worl * * * +### 2.9 Managed Docs: Copies, Forks, and Synchronization + +tbd manages documentation (guidelines, shortcuts, templates) alongside issues. +With forkable docs (format f05, `plan-2026-06-11-forkable-docs.md`), a doc can exist as +up to **four copies plus a manifest**, each with a distinct owner and lifecycle. +This section is the canonical statement of that model and the invariants that make every +combination of user actions safe. + +#### The copies + +| Copy | Location | In git? | Written by | Role | +| --- | --- | --- | --- | --- | +| Bundled | npm package (`dist/docs/`) | n/a (per tbd version) | tbd releases | The upstream for `internal:` docs; immutable per installed version | +| Cache | `.tbd/docs/` | gitignored | doc sync only | Complete, pristine, machine-local mirror of all upstream docs (bundled + URL sources); disposable | +| Fork | `docs/tbd//.md` | tracked | `tbd docs fork/update` + the user/agent | The editable copy that tbd serves; optional, per-doc | +| Base | `.tbd/doc-forks/base//.md` | tracked | `tbd docs fork/update` | Verbatim upstream snapshot at the fork point; the three-way merge base | +| Manifest | `.tbd/doc-forks/forks.yml` | tracked | `tbd docs fork/unfork/update` | Provenance per fork: source docref, `base_hash` (LF-normalized sha256), `tbd_version` at fork point, `conflicted` flag | + +A doc’s identity is **kind + name**; paths follow fixed conventions +(`/.md`, flat — nested folders are not scanned). + +The model follows one principle: **resolve by convention; track only what cannot be +derived; publish the inventory as a generated view.** Lookup is a fixed search path over +conventional locations — no registry that can drift from disk. +The only stored tracking is the fork manifest, which records the one fact that cannot be +recomputed from the files: each fork’s upstream source and base snapshot. +The docmap that doc commands emit is a generated view of this state, never an input to +resolution. (A copy-all-and-gitignore variant of the fork dir was considered and +rejected: gitignored mirrors are invisible on GitHub and in PRs, and edits to them +diverge silently with no team-visible artifact — `tbd docs fork --all` provides the +all-visible posture in tracked form. +See the spec’s Alternatives.) + +#### Invariants + +1. **Serving precedence**: the fork dir is prepended to every kind’s lookup path, so a + forked (or hand-authored local) file shadows the cache copy by name. + With nothing forked, lookup paths reduce to the cache and behavior is byte-identical + to pre-f05. +2. **Cache completeness**: doc sync always installs *all* upstream docs into the cache, + including forked ones. + The cache copy is the pristine reference — the staleness comparator, the “theirs” + side of every update merge, and the fallback after unfork. +3. **The cache is never authored**: only doc sync writes it, nothing else reads from + anywhere else for upstream content, and deleting it is always safe (auto-sync + regenerates it on the next doc access, including on fresh clones). +4. **Tracked files mutate only via explicit `tbd docs` verbs** (fork/unfork/update). + Setup, `tbd sync`, and background auto-sync refresh the cache and *report* drift but + never write the fork dir, bases, or manifest. +5. **Tracking is derived, not stored**: every doc state + (`upstream/forked/customized/stale/conflicted/local/missing/orphaned`) is a pure + function of (manifest entry present?, file hash, base hash, cache hash, conflicted + flag, markers present). + There is no hidden database, so no sequence of git operations — commit, pull, merge, + revert, partial commits — can desynchronize tbd from the files: collaborators + recompute identical states from identical content. +6. **The base is the fork point**: advanced only by fork (refresh) and update; with the + stored snapshot, `customized` (file ≠ base), `stale` (cache ≠ base), and three-way + merging are exact, offline operations for every collaborator regardless of which tbd + version created the fork. +7. **The format gate**: forking bumps nothing at runtime, but the f05 `tbd_format` in + `config.yml` (mirrored into the machine-local common-dir `layout.yml`) makes pre-f05 + clients refuse the repo — they would otherwise serve upstream copies of docs the team + has customized. Within the f05 era, the manifest’s per-entry `tbd_version` guards the + remaining skew: `tbd docs update` refuses to touch a doc whose fork point was + advanced by a newer tbd than the one running, since that client’s bundled “upstream” + is older than the fork point and updating would silently downgrade the doc. + +#### Who writes what (synchronization flows) + +| Flow | Reads | Writes | Tracked files touched | +| --- | --- | --- | --- | +| npm upgrade | — | bundle | none | +| Doc sync (`tbd sync --docs`, auto-sync, setup) | bundle + URL sources | cache | none | +| `tbd docs fork` | cache | fork file, base, manifest, fork-dir README | yes (explicit) | +| `tbd docs update` | cache, base, fork file | fork file and/or base, manifest, README | yes (explicit) | +| `tbd docs unfork` | base, fork file | removes fork artifacts, README | yes (explicit) | +| User/agent edits | — | fork dir (any way they like) | yes (theirs) | +| git operations | — | any tracked artifact | yes (theirs) | + +Staleness appears exactly when doc sync moves the cache past a fork’s base; awareness +surfaces (`tbd docs status`, the one-line `tbd sync` drift notice) report it, and only +the explicit `tbd docs update` acts on it. + +#### Drift and degraded modes + +Because of invariant 5, arbitrary user actions in the fork dir resolve to defined +states: edits → `customized`; deletion → `missing` (serving falls back to the cache); +rename → `missing` + `local`; new files → `local` (served, no upstream); deleting the +manifest → everything `local` (serving unaffected); moving into subfolders → not scanned +(documented). Degraded modes fail soft: an unreachable URL source keeps serving the +last-good cache copy; an empty cache self-heals via auto-sync; a deleted base blocks +merging only for that doc (repairable via `update --keep-ours`); the upgrade abort path +is specified in `tbd-docs.md` §Troubleshooting. +The drift matrix with resolutions is user-facing in `tbd-docs.md` §“Forked Docs in Your +Repo”; the state matrix, update decision table, and drift scenarios are pinned by +`fork-manifest`/`fork-update`/`doc-fork` unit tests and the `cli-docs-fork`/ +`cli-docs-update` golden tryscripts. + +* * * + ## 3. Git Layer ### 3.1 Overview @@ -2034,6 +2137,8 @@ main branch: tbd-sync branch: .tbd/.gitignore # Controls what's gitignored below .tbd/.gitattributes # Merge strategies (merge=union for ids.yml) .tbd/workspaces/ # Persistent state (outbox, named workspaces) +.tbd/doc-forks/ # Fork manifest + base snapshots (see §2.9; f05+) +docs/tbd/ # Forked docs, outside .tbd/ (see §2.9; only when fork is used) ``` #### Files Gitignored (local only) @@ -3660,6 +3765,42 @@ tbd attic restore proj-a1b2 2025-01-07T10-30-00Z * * * +### 4.13 Docs Commands + +Operations on managed docs (see §2.9 for the data model) live under the noun-scoped +`tbd docs` group, alongside the existing per-kind readers (`tbd guidelines`, +`tbd shortcut`, `tbd template`, which serve forked copies transparently via lookup +precedence): + +```bash +tbd docs fork [names...] [--kind] [--all] [--force] [--dry-run] # copy into docs/tbd/ +tbd docs unfork [names...] [--all] [--force] # back to upstream; refuses to drop edits +tbd docs status [--json] # per-doc states + missing/local hints +tbd docs update [names...] [--merge|--keep-ours] [--dry-run] # reconcile with upstream +tbd docs diff [--base|--upstream] # net fork / your changes / incoming +tbd docs list [--kind] [--json] # cross-kind list with state markers +``` + +tbd has three deliberately separate update surfaces — they differ in scope, risk, and +failure mode, and doc updates are the only one that can merge and mutate tracked files: + +| Command | Scope | Touches | Modifies tracked files? | +| --- | --- | --- | --- | +| `tbd sync` | project data (issues) | sync worktree + `tbd-sync` branch; refreshes the doc cache and *reports* fork drift | never | +| `tbd setup --auto` | installation + integrations | skills, hooks, settings, `AGENTS.md`; invokes a docs-cache sync | only generated integration files | +| `tbd docs update` | forked docs | fork dir + bases + manifest (offline, against the cache) | **yes — the only doc command that does** | + +Update semantics (the full decision table is unit-tested row by row): an unmodified +stale fork is replaced; a customized stale fork gets a `git merge-file` three-way merge +that applies automatically when clean; conflicts are skipped by default and listed with +the two explicit strategies — `--merge` (combine, standard conflict markers, sets the +`conflicted` flag until markers are resolved) and `--keep-ours` (keep the local content, +advance the fork point). +Forked files are git-tracked, so every applied update is reviewable in `git diff` and +revertible — git is the undo. + +* * * + ## 5. Beads Compatibility ### 5.1 Import Strategy diff --git a/packages/tbd/docs/tbd-docs.md b/packages/tbd/docs/tbd-docs.md index f7bfcc26..049306b1 100644 --- a/packages/tbd/docs/tbd-docs.md +++ b/packages/tbd/docs/tbd-docs.md @@ -156,13 +156,11 @@ The recommended way to initialize tbd and configure agent integrations. ```bash tbd setup --auto # Full setup with auto-detection (recommended) -tbd setup --interactive # Interactive setup with prompts tbd setup --from-beads # Migrate from existing Beads setup ``` Options: - `--auto` - Automatic mode: auto-detect prefix, migrate beads if present -- `--interactive` - Interactive mode: prompt for all options - `--from-beads` - Migrate issues from existing Beads setup - `--prefix ` - Override auto-detected prefix @@ -576,6 +574,7 @@ Options: > **Note:** `tbd import --from-beads` is deprecated. > Use `tbd setup --auto` or `tbd setup --from-beads` instead for migrating from Beads. + - `--validate` - Validate existing import against Beads source ### beads @@ -674,12 +673,23 @@ and skips reinstallation. ### Documentation Commands -Built-in documentation viewers: +Managed docs (the `tbd docs` group): + +```bash +tbd docs # Status overview of managed docs +tbd docs list # All docs across kinds, with state markers +tbd docs show # Read any doc by name (kind-agnostic) +tbd docs show tbd-docs # The CLI manual (alias: tbd docs manual) +tbd docs show tbd-docs --sections # List the manual's sections +tbd docs show tbd-docs --section # Read one manual section +tbd docs sync # Refresh the gitignored docs cache +tbd docs fork / unfork / update / diff / status # Forked docs (see below) +``` + +Other built-in viewers: ```bash tbd readme # Display README (same as GitHub landing page) -tbd docs # Display CLI reference documentation -tbd docs --list # List available documentation sections tbd design # Display design documentation tbd design --list # List design doc sections tbd closing # Display session closing protocol reminder @@ -714,6 +724,68 @@ On HTTP 403, fetching falls back to `gh api` for authenticated access. User-added shortcuts go to `shortcuts/custom/` (separate from bundled `shortcuts/standard/`). +### Managing Docs: Two Modes + +Every managed doc is served through one search path; where the file lives is a per-doc +choice between two modes that serve identical content: + +- **Hidden cache (the default).** Docs live in the gitignored `.tbd/docs/` cache — + always active, zero repo footprint, refreshed by `tbd docs sync` (and by setup). +- **Forked.** `tbd docs fork ` (or `--all`) copies a doc into `docs/tbd/`, tracked + in git: visible on GitHub, reviewable in PRs, and editable — your copy shadows the + cache everywhere the upstream one was served. + `tbd docs unfork` returns to the cache; `tbd docs update` three-way merges upstream + changes into your copy after an upgrade. + +Forking changes nothing about how docs work — it only makes them explicit and editable. +Four update surfaces stay deliberately separate: + +| Command | Scope | Touches | Modifies tracked files? | +| --- | --- | --- | --- | +| `tbd sync` | project data (issues/beads) | sync worktree + `tbd-sync` branch; also refreshes the doc cache and *reports* fork drift | never | +| `tbd setup --auto` | installation + integrations | skills, hooks, settings, `AGENTS.md`; invokes a docs-cache sync | only generated integration files | +| `tbd docs sync` | doc cache | gitignored `.tbd/docs/` only | never | +| `tbd docs update` | your forked docs | fork dir + bases + manifest (offline, against the cache) | **yes — the only doc command that does** | + +Disambiguation worth stating once: `tbd update ` is an issue operation, +`tbd docs update` a doc operation — the noun scope always disambiguates. + +### Forked Docs in Your Repo (docs/tbd/) + +`tbd docs fork` copies managed docs into `docs/tbd/`, laid out **by kind, flat within +each kind**, with a generated `README.md` index (regenerated on every +fork/unfork/update): + +``` +docs/tbd/ +├── README.md # generated index — what this folder is, one line per doc +├── guidelines/.md +├── shortcuts/.md +└── templates/.md +``` + +Two rules make everything below predictable: **names are identity** (a doc is +`/.md`; nested subfolders are not scanned), and **tracking is derived, not +stored** (the canonical model — copies, invariants, flows — is `tbd-design.md` §2.9; +this table is its user-facing summary) — every doc’s state is recomputed from content +hashes (your file vs its recorded base vs current upstream), so no git operation can +desynchronize tbd from the folder. +Whatever you or your agent do to these files, `tbd docs status` gives a defined answer: + +| You (or your agent)… | State | What happens / what to do | +| --- | --- | --- | +| Edit a forked file | `customized` | Served as-is; `tbd docs update` three-way merges upstream changes in | +| Delete a forked file | `missing` | Serving falls back to upstream; restore with `tbd docs fork --force` or finalize with `tbd docs unfork ` | +| Rename a forked file | `missing` + `local` | A rename is delete + add: finalize the old name (`unfork`), keep the new file as `local` | +| Add a new `.md` file | `local` | Served with top precedence; nothing to update or unfork (no upstream) | +| Move a file into a subfolder | invisible | Subfolders are not scanned — keep files at `/.md` | +| Delete `.tbd/doc-forks/` (the manifest) | all `local` | Files keep being served; re-fork with `--force` to re-establish update tracking (overwrites with upstream — re-apply edits after) | +| Commit / pull / merge / revert any of it | recomputed | States derive from content, so collaborators see the same answers from the same files | + +Awareness without surprise mutations: `tbd sync` prints a one-line notice when forked +docs are stale, conflicted, or missing, and `tbd docs status` shows the full picture — +but only the explicit `tbd docs update` ever modifies tracked files. + ### uninstall Remove tbd from a repository. @@ -1022,8 +1094,32 @@ sync: branch: tbd-sync # Sync branch name remote: origin # Remote name auto_sync: true # Auto-sync after writes + +docs_cache: + files: # Docs synced into the cache: destination -> docref + guidelines/python-rules.md: internal:guidelines/python-rules.md + guidelines/my-team-rules.md: github:my-org/docs@main//rules.md + lookup_path: # Search paths for doc lookup (earlier wins) + - .tbd/docs/shortcuts/system + - .tbd/docs/shortcuts/standard ``` +`docs_cache.files` values — like the fork manifest’s `source` values in +`.tbd/doc-forks/forks.yml` — are **docrefs**: one URI-like address grammar +(`internal:…`, anchored local paths, URLs, `github:owner/repo@ref//path`). For the full +grammar see `tbd docs show docref-format`; for the docmap structure that doc listings +and their `--json` output follow, see `tbd docs show docmap-format`. + +Two further `docs_cache` keys: + +- `docs_cache.local_dirs` — an ordered list of `./`-prefixed local docrefs naming extra + in-repo doc directories, served between the fork dir and the cache. + Docs found there are first-class for reading (`list`, `show`, the per-kind readers, + with a `(serving local doc: …)` note) and report state `local`; they are not forkable + or updatable — they already live in the repo. +- `docs_cache.fork_dir` — reserved in the f05 format era but **planned, not yet read**: + the fork-dir location is currently fixed at `docs/tbd/`. + ## Priority Scale | Value | Alias | Meaning | @@ -1118,6 +1214,73 @@ ls "$(git rev-parse --path-format=absolute --git-common-dir)/tbd/data-sync-workt ls "$(git rev-parse --path-format=absolute --git-common-dir)/tbd/data-sync-worktree/.tbd/data-sync/issues/" | sort ``` +### Aborting a Format Upgrade + +Upgrading tbd can bump the repository format (`tbd_format` in `.tbd/config.yml`, e.g. +f04 → f05). The bump happens automatically on the first command after upgrading, and +older tbd versions then refuse the repository until they are upgraded. +If an upgrade hits unexpected bugs, you can cleanly abort and return to the previous +version. This is everything a format upgrade can touch: + +| State | Location | In git? | Written by | Revert | +| --- | --- | --- | --- | --- | +| Project config | `.tbd/config.yml` | tracked | the migration (format stamp) | `git checkout -- .tbd/config.yml`, or `git revert` the bump commit | +| Agent surfaces | `AGENTS.md`, `.claude/`, `.agents/`, `.codex/` | tracked | only `tbd setup --auto` (marker refresh) | `git checkout --` those paths | +| Shared layout stamp | `$GIT_COMMON_DIR/tbd/layout.yml` | machine-local, not in git | the migration (re-stamp) | delete it — it regenerates from whatever the config says | +| Forked docs (f05) | `docs/tbd/`, `.tbd/doc-forks/` | tracked once committed | only `tbd docs fork` | `git checkout --`/`git revert` if committed; delete if never committed | +| Docs cache | `.tbd/docs/` | gitignored | doc sync (unchanged by migration) | none needed — always safe to delete and re-sync | +| Issue data | `tbd-sync` branch + `$GIT_COMMON_DIR/tbd/data-sync-worktree/` | git branch | **never touched by migration** | none needed — the worktree re-materializes from the branch | + +**Abort recipe** (works from any state, including a crash mid-upgrade): + +```bash +# 1. Restore the tracked files (or `git revert` the format-bump commit): +git checkout -- .tbd/config.yml +git checkout -- AGENTS.md .claude .agents .codex # only if `tbd setup --auto` ran + +# 2. Delete the machine-local format stamp (regenerates from the config): +rm "$(git rev-parse --path-format=absolute --git-common-dir)/tbd/layout.yml" + +# 3. Only if docs were forked and never committed: +rm -rf docs/tbd .tbd/doc-forks +``` + +After this, the previous tbd version works again, and re-running the upgrade later is +safe — the migration is idempotent from any of these states. + +Reverting `.tbd/config.yml` is enough to drop the format gate even if forks were already +committed: compatibility is decided only by `tbd_format` in the config, not by the +presence of `docs/tbd/` or `.tbd/doc-forks/`. Committed fork files simply become inert +`local` docs under the older version — harmless to leave in place, so step 3 is only for +cleanup, never required to abort. + +Notes: + +- **The migration never writes issue data**, so the recipe above cannot lose issues — it + touches only the two stamps and tracked files. + A bigger hammer also exists: deleting the entire `$GIT_COMMON_DIR/tbd/` directory is + recoverable (layout and the data-sync worktree re-materialize from the config and the + `tbd-sync` branch on the next command, or via `tbd doctor --fix`) — **but only for + synced data**. Issue changes made with `--no-sync` since the last `tbd sync` live as + uncommitted files inside that worktree and would be lost, so run `tbd sync` first if + you must delete it. This is why the recipe deletes only `layout.yml`, never the whole + directory. +- **Interrupted upgrades self-heal.** If the process dies between the two stamp writes + (layout updated but not config, or config but not layout), the next command with the + new version completes the migration; the abort recipe above also works from either + partial state. +- **Quiesce other tbd processes first.** The same self-healing re-stamp that completes + an interrupted upgrade can also undo an abort. + Any concurrent `tbd` write (another worktree, a background agent, an editor hook) + re-stamps `layout.yml` from whatever `.tbd/config.yml` currently says. + If you delete `layout.yml` while the config is still on the new format — or before the + config revert in step 1 has landed — the next write recreates the stamp and reopens + the migration. Stop other agents and worktrees, do step 1 (revert the config) before + step 2 (delete the stamp), and the abort sticks. +- Teammates each migrate their own machine-local stamp automatically; only the + `.tbd/config.yml` change is shared (via your branch), so reverting that commit is the + team-wide rollback. + ### Performance For large repositories with many issues: diff --git a/packages/tbd/docs/tbd-prime.md b/packages/tbd/docs/tbd-prime.md index 64180fd2..0f1003d4 100644 --- a/packages/tbd/docs/tbd-prime.md +++ b/packages/tbd/docs/tbd-prime.md @@ -109,7 +109,6 @@ tbd dep add # Tests depend on feature ## Setup Commands - `tbd setup --auto` - Non-interactive setup with smart defaults (for agents/scripts) -- `tbd setup --interactive` - Interactive setup with prompts (for humans) - `tbd setup --from-beads` - Migrate from Beads to tbd ## Quick Reference diff --git a/packages/tbd/scripts/copy-docs.mjs b/packages/tbd/scripts/copy-docs.mjs index c3043a3c..20f6f10d 100644 --- a/packages/tbd/scripts/copy-docs.mjs +++ b/packages/tbd/scripts/copy-docs.mjs @@ -35,6 +35,7 @@ const SHORTCUTS_DIR = join(DOCS_DIR, 'shortcuts'); const SHORTCUTS_SYSTEM_DIR = join(SHORTCUTS_DIR, 'system'); const GUIDELINES_DIR = join(DOCS_DIR, 'guidelines'); const TEMPLATES_DIR = join(DOCS_DIR, 'templates'); +const REFERENCES_DIR = join(DOCS_DIR, 'references'); /** * Packaged documentation files (in packages/tbd/docs/). @@ -135,6 +136,11 @@ if (phase === 'prebuild') { await copyDir(TEMPLATES_DIR, join(distDocs, 'templates')); } + // Copy reference docs (docref/docmap formats) to dist/docs + if (existsSync(REFERENCES_DIR)) { + await copyDir(REFERENCES_DIR, join(distDocs, 'references')); + } + // Copy install directory to dist/docs (headers for composing skill files) await copyDir(INSTALL_DIR, join(distDocs, 'install')); diff --git a/packages/tbd/src/cli/commands/docs-fork.ts b/packages/tbd/src/cli/commands/docs-fork.ts new file mode 100644 index 00000000..02983690 --- /dev/null +++ b/packages/tbd/src/cli/commands/docs-fork.ts @@ -0,0 +1,833 @@ +/** + * `tbd docs fork` / `unfork` / `status` — the forkable-docs command handlers. + * + * These are added as subcommands of `tbd docs`. Resolution uses the gitignored + * cache (the pristine upstream) so forking copies upstream content into the visible + * fork dir; serving precedence (fork dir shadows cache) is handled by the lookup + * paths in paths.ts. + */ + +import type { Command } from 'commander'; +import { dirname, join, relative, sep } from 'node:path'; +import { mkdir } from 'node:fs/promises'; + +import { writeFile } from 'atomically'; + +import { BaseCommand } from '../lib/base-command.js'; +import { requireInit } from '../lib/errors.js'; +import { CLIError } from '../lib/errors.js'; +import { VERSION } from '../lib/version.js'; +import { readConfig } from '../../file/config.js'; +import { DocCache } from '../../file/doc-cache.js'; +import { + CACHE_GUIDELINES_PATHS, + CACHE_REFERENCE_PATHS, + CACHE_SHORTCUT_PATHS, + CACHE_TEMPLATE_PATHS, + FORK_DIR, + TBD_DOCS_DIR, +} from '../../lib/paths.js'; +import { formatDocSize } from '../../lib/format-utils.js'; +import { + type ForkEntry, + type ForkKind, + findFork, + hashContent, + hasUnresolvedConflict, + readForkManifest, + writeForkManifest, + writeBaseContent, + upsertFork, + withForkManifestLock, +} from '../../file/fork-manifest.js'; +import { + forkDoc, + unforkDoc, + forkStatusFor, + forkFilePath, + readForkFile, + readForkBase, + listLocalForkFiles, + regenerateForkDirReadme, + ForkConflictError, +} from '../../file/doc-fork.js'; +import { updateOne, diffContents, type UpdateStrategy } from '../../file/fork-update.js'; +import { createDocMap, type DocMapEntry } from '../../docmap/index.js'; +import { servedEntryFor, loadServeContext, effectiveServePaths } from '../lib/doc-serve.js'; +import { docCategory, parseCategoryOption } from '../../lib/doc-categories.js'; + +/** Kinds that can be resolved from the cache and forked today. */ +export const RESOLVABLE_KINDS: ForkKind[] = ['guideline', 'shortcut', 'template', 'reference']; + +/** + * Validate a user-supplied --kind value. Without this, an unknown kind silently + * produces an empty cache and misleading "no docs" output. + */ +export function parseKindOption(kind: string | undefined): ForkKind | undefined { + if (kind === undefined) return undefined; + if (!(RESOLVABLE_KINDS as string[]).includes(kind)) { + throw new CLIError(`Unknown kind "${kind}". Valid kinds: ${RESOLVABLE_KINDS.join(', ')}.`); + } + return kind as ForkKind; +} + +const KIND_CACHE_PATHS: Record = { + guideline: CACHE_GUIDELINES_PATHS, + shortcut: CACHE_SHORTCUT_PATHS, + template: CACHE_TEMPLATE_PATHS, + reference: CACHE_REFERENCE_PATHS, +}; + +interface ResolvedDoc { + kind: ForkKind; + name: string; + source: string; + content: string; +} + +/** Build a cache over a kind's cache-only paths (pristine upstream). */ +async function buildKindCache(kind: ForkKind, tbdRoot: string): Promise { + const cache = new DocCache(KIND_CACHE_PATHS[kind] ?? [], tbdRoot); + await cache.load({ quiet: true }); + return cache; +} + +/** Derive the provenance docref for a cached doc from config, defaulting to internal:. */ +function sourceDocRef( + tbdRoot: string, + files: Record | undefined, + docPath: string, +): string { + const cacheRoot = join(tbdRoot, TBD_DOCS_DIR); + const rel = relative(cacheRoot, docPath).split(sep).join('/'); + return files?.[rel] ?? `internal:${rel}`; +} + +interface ForkOptions { + kind?: string; + all?: boolean; + category?: string[]; + force?: boolean; + dryRun?: boolean; + json?: boolean; +} + +class DocsForkHandler extends BaseCommand { + async run(names: string[], options: ForkOptions): Promise { + await this.execute(async () => { + const tbdRoot = await requireInit(); + const config = await readConfig(tbdRoot); + const files = config.docs_cache?.files; + + const targets = await this.resolveTargets(tbdRoot, files, names, options); + if (targets.length === 0) { + throw new CLIError( + 'No matching docs to fork. Run `tbd docs status` or `tbd guidelines --list`.', + ); + } + + if (options.dryRun) { + const catSuffix = + options.category && options.category.length > 0 + ? ` (categories: ${options.category.join(', ')})` + : ''; + this.output.dryRun(`Would fork ${targets.length} doc(s) into ${FORK_DIR}/${catSuffix}`, { + docs: targets.map((t) => `${t.kind}/${t.name}`), + }); + if (!this.ctx.json) { + for (const t of targets) { + console.log(` ${t.kind.padEnd(11)} ${t.name}`); + } + console.log('No files written. Re-run without --dry-run to apply.'); + } + return; + } + + const forked: string[] = []; + await withForkManifestLock(tbdRoot, async () => { + let manifest = await readForkManifest(tbdRoot); + for (const t of targets) { + let result; + try { + result = await forkDoc({ + tbdRoot, + forkDir: FORK_DIR, + manifest, + kind: t.kind, + name: t.name, + source: t.source, + content: t.content, + tbdVersion: VERSION, + force: options.force, + }); + } catch (err) { + if (err instanceof ForkConflictError && err.code === 'overwrite') { + throw new CLIError( + `${err.message}. Refusing to overwrite it. Options:\n` + + ` tbd docs diff ${t.name} # see how it differs\n` + + ` tbd docs fork ${t.name} --force # overwrite with upstream`, + ); + } + throw err; + } + manifest = result.manifest; + forked.push(result.relPath); + if (!this.ctx.json) { + this.output.success(`Forked ${t.name} → ${result.relPath}`); + } + } + await writeForkManifest(tbdRoot, manifest); + await regenerateForkDirReadme(tbdRoot, FORK_DIR, manifest); + }); + + if (this.ctx.json) { + this.output.data({ forked }); + } else { + const colors = this.output.getColors(); + console.log(colors.dim(` Regenerated ${FORK_DIR}/README.md`)); + console.log(''); + console.log('Edit in place — tbd now serves your copy wherever it served upstream.'); + } + }, 'Failed to fork'); + } + + private async resolveTargets( + tbdRoot: string, + files: Record | undefined, + names: string[], + options: ForkOptions, + ): Promise { + const parsedKind = parseKindOption(options.kind); + const kinds = parsedKind ? [parsedKind] : RESOLVABLE_KINDS; + + const categories = (options.category ?? []).map(parseCategoryOption); + if (options.all || categories.length > 0) { + const targets: ResolvedDoc[] = []; + for (const kind of kinds) { + const cache = await buildKindCache(kind, tbdRoot); + for (const doc of cache.list()) { + // Skip tbd-internal system shortcuts (skill-baseline etc.). + if (kind === 'shortcut' && doc.sourceDir.endsWith('system')) continue; + if (categories.length > 0 && !categories.includes(docCategory(doc.frontmatter))) { + continue; + } + targets.push({ + kind, + name: doc.name, + source: sourceDocRef(tbdRoot, files, doc.path), + content: doc.content, + }); + } + } + return targets; + } + + const caches = new Map(); + for (const kind of kinds) { + caches.set(kind, await buildKindCache(kind, tbdRoot)); + } + + const targets: ResolvedDoc[] = []; + for (const name of names) { + const matches: ResolvedDoc[] = []; + for (const kind of kinds) { + const hit = caches.get(kind)!.get(name); + if (hit) { + matches.push({ + kind, + name: hit.doc.name, + source: sourceDocRef(tbdRoot, files, hit.doc.path), + content: hit.doc.content, + }); + } + } + if (matches.length === 0) { + throw new CLIError( + `No doc found named "${name}". Run \`tbd guidelines --list\` to see names.`, + ); + } + if (matches.length > 1) { + const kindList = matches.map((m) => m.kind).join(', '); + throw new CLIError( + `"${name}" exists in multiple kinds (${kindList}). Use --kind to disambiguate.`, + ); + } + targets.push(matches[0]!); + } + return targets; + } +} + +interface UnforkOptions { + kind?: string; + all?: boolean; + force?: boolean; + json?: boolean; +} + +class DocsUnforkHandler extends BaseCommand { + async run(names: string[], options: UnforkOptions): Promise { + await this.execute(async () => { + const tbdRoot = await requireInit(); + const removed: string[] = []; + await withForkManifestLock(tbdRoot, async () => { + let manifest = await readForkManifest(tbdRoot); + + const targetNames = options.all ? manifest.forks.map((f) => f.name) : names; + if (targetNames.length === 0) { + throw new CLIError('Specify a doc name to unfork, or use --all.'); + } + + for (const name of targetNames) { + try { + const result = await unforkDoc({ + tbdRoot, + forkDir: FORK_DIR, + manifest, + name, + kind: parseKindOption(options.kind), + force: options.force, + }); + manifest = result.manifest; + removed.push(name); + if (!this.ctx.json) { + this.output.success(`Unforked ${name} — served from upstream again.`); + } + } catch (err) { + if (err instanceof ForkConflictError && err.code === 'customized') { + throw new CLIError( + `${name} has local customizations (differs from its base). ` + + `Refusing to discard them. Options:\n` + + ` tbd docs diff ${name} # review your changes\n` + + ` tbd docs unfork ${name} --force # discard and fall back to upstream`, + ); + } + throw err; + } + } + await writeForkManifest(tbdRoot, manifest); + await regenerateForkDirReadme(tbdRoot, FORK_DIR, manifest); + }); + if (this.ctx.json) { + this.output.data({ unforked: removed }); + } + }, 'Failed to unfork'); + } +} + +/** Compose the display label for a doc's state, combining customized + stale. */ +function stateLabel(state: string, stale: boolean): string { + if ((state === 'customized' || state === 'orphaned' || state === 'conflicted') && stale) { + return `${state}, stale`; + } + return state; +} + +class DocsStatusHandler extends BaseCommand { + async run(options: { json?: boolean }): Promise { + await this.execute(async () => { + const tbdRoot = await requireInit(); + const manifest = await readForkManifest(tbdRoot); + const colors = this.output.getColors(); + + // Resolve upstream (cache) content per entry for staleness. + const caches = new Map(); + interface StatusRow { + name: string; + kind: string; + label: string; + path: string; + source: string; + customized: boolean; + stale: boolean; + conflicted: boolean; + missing: boolean; + } + const rows: StatusRow[] = []; + + for (const entry of manifest.forks) { + const kind = entry.kind; + if (!caches.has(kind)) caches.set(kind, await buildKindCache(kind, tbdRoot)); + const cacheHit = caches.get(kind)!.get(entry.name); + const status = await forkStatusFor(tbdRoot, FORK_DIR, entry, cacheHit?.doc.content ?? null); + rows.push({ + name: entry.name, + kind: entry.kind, + label: stateLabel(status.state, status.stale), + path: entry.path, + source: entry.source, + customized: status.customized, + stale: status.stale, + conflicted: status.conflicted, + missing: status.state === 'missing', + }); + } + + // Hand-authored fork-dir files with no manifest entry (state `local`). + // These cover adds, the new half of a rename, and a deleted manifest. + const locals = await listLocalForkFiles(tbdRoot, FORK_DIR, manifest); + for (const l of locals) { + rows.push({ + name: l.name, + kind: l.kind, + label: 'local', + path: l.relPath, + source: '—', + customized: false, + stale: false, + conflicted: false, + missing: false, + }); + } + rows.sort((a, b) => a.name.localeCompare(b.name)); + + if (this.ctx.json) { + const docs: DocMapEntry[] = rows.map((r) => ({ + name: r.name, + type: r.kind, + path: r.path, + ...(r.source !== '—' ? { source: r.source } : {}), + state: r.label, + stale: r.stale, + })); + this.output.data(createDocMap(docs, { name: 'tbd-forks' })); + return; + } + + if (rows.length === 0) { + console.log('No docs forked into the repo.'); + console.log( + `Make some visible: ${colors.bold('tbd docs fork ')} or ${colors.bold('tbd docs fork --all')}`, + ); + return; + } + + const nameW = Math.max(4, ...rows.map((r) => r.name.length)); + const kindW = Math.max(4, ...rows.map((r) => r.kind.length)); + const stateW = Math.max(5, ...rows.map((r) => r.label.length)); + const header = `${'NAME'.padEnd(nameW)} ${'KIND'.padEnd(kindW)} ${'STATE'.padEnd(stateW)} SOURCE`; + console.log(colors.dim(header)); + for (const r of rows) { + const line = `${r.name.padEnd(nameW)} ${r.kind.padEnd(kindW)} ${r.label.padEnd(stateW)} ${r.source}`; + console.log(r.label === 'local' ? colors.dim(line) : line); + } + + const forkedRows = rows.filter((r) => r.label !== 'local'); + const customizedCount = forkedRows.filter((r) => r.customized).length; + const staleCount = forkedRows.filter((r) => r.stale).length; + const conflictCount = forkedRows.filter((r) => r.conflicted).length; + const missingRows = forkedRows.filter((r) => r.missing); + const parts = [`${customizedCount} customized`]; + if (staleCount > 0) parts.push(`${staleCount} with upstream updates — run 'tbd docs update'`); + if (conflictCount > 0) parts.push(`${conflictCount} conflict pending`); + if (locals.length > 0) parts.push(`${locals.length} local`); + console.log(''); + console.log(`${forkedRows.length} forked: ${parts.join(', ')}`); + + if (missingRows.length > 0) { + console.log(''); + console.log(`${missingRows.length} doc(s) missing (forked file deleted or renamed):`); + for (const r of missingRows) { + console.log( + ` ${r.name} restore with 'tbd docs fork ${r.name} --force', or finalize with 'tbd docs unfork ${r.name}'`, + ); + } + } + }, 'Failed to read docs status'); + } +} + +interface UpdateOptions { + merge?: boolean; + keepOurs?: boolean; + dryRun?: boolean; + json?: boolean; +} + +class DocsUpdateHandler extends BaseCommand { + async run(names: string[], options: UpdateOptions): Promise { + await this.execute(async () => { + if (options.merge && options.keepOurs) { + throw new CLIError('--merge and --keep-ours are mutually exclusive.'); + } + const strategy: UpdateStrategy = options.merge + ? 'merge' + : options.keepOurs + ? 'keep-ours' + : 'default'; + + const tbdRoot = await requireInit(); + const applied: { entry: ForkEntry; message: string }[] = []; + const decisions: { name: string; message: string }[] = []; + const skipped: { name: string; message: string }[] = []; + + await withForkManifestLock(tbdRoot, async () => { + let manifest = await readForkManifest(tbdRoot); + if (names.length > 0) { + const known = new Set(manifest.forks.map((f) => f.name)); + const unknown = names.filter((n) => !known.has(n)); + if (unknown.length > 0) { + throw new CLIError( + `Not forked: ${unknown.join(', ')}. Run \`tbd docs status\` to see forked docs.`, + ); + } + } + const selected = + names.length > 0 ? manifest.forks.filter((f) => names.includes(f.name)) : manifest.forks; + + const caches = new Map(); + const upstreamFor = async (entry: ForkEntry): Promise => { + const kind = entry.kind; + if (!caches.has(kind)) caches.set(kind, await buildKindCache(kind, tbdRoot)); + return caches.get(kind)!.get(entry.name)?.doc.content ?? null; + }; + + for (const entry of selected) { + const forkContent = await readForkFile(tbdRoot, FORK_DIR, entry); + const result = await updateOne({ + entry, + forkContent, + baseContent: await readForkBase(tbdRoot, entry), + upstreamContent: await upstreamFor(entry), + strategy, + runningVersion: VERSION, + }); + + const { newFileContent, newBaseContent } = result; + if (newFileContent === undefined && newBaseContent === undefined) { + // The conflicted flag auto-clears once markers are resolved (spec); + // persist the clear so the committed manifest matches computed state. + if ( + !options.dryRun && + entry.conflicted && + forkContent !== null && + !hasUnresolvedConflict(forkContent) + ) { + const cleared: ForkEntry = { ...entry }; + delete cleared.conflicted; + manifest = upsertFork(manifest, cleared); + } + if (result.needsDecision) { + decisions.push({ name: entry.name, message: result.message }); + } else if (result.action !== 'skip-not-stale') { + // Conflicted / orphaned / missing / version-skewed: actionable but + // not applied here — surface, never silently swallow. + skipped.push({ name: entry.name, message: result.message }); + } + continue; + } + + if (!options.dryRun) { + if (newFileContent !== undefined) { + const abs = forkFilePath(tbdRoot, FORK_DIR, entry.kind, entry.name); + await mkdir(dirname(abs), { recursive: true }); + await writeFile(abs, newFileContent); + } + const updated: ForkEntry = { ...entry }; + if (newBaseContent !== undefined) { + await writeBaseContent(tbdRoot, entry.kind, entry.name, newBaseContent); + updated.base_hash = hashContent(newBaseContent); + // The base records its fork point's tbd version so older clients + // can detect (and refuse) a downgrade — see the update guard. + updated.tbd_version = VERSION; + } + if (result.setConflicted) { + updated.conflicted = true; + } else { + delete updated.conflicted; + } + manifest = upsertFork(manifest, updated); + } + applied.push({ entry, message: result.message }); + } + + if (!options.dryRun) { + await writeForkManifest(tbdRoot, manifest); + await regenerateForkDirReadme(tbdRoot, FORK_DIR, manifest); + } + }); // end withForkManifestLock + + if (this.ctx.json) { + this.output.data({ + dryRun: Boolean(options.dryRun), + updated: applied.map((a) => a.entry.name), + needsDecision: decisions, + skipped, + }); + return; + } + + const colors = this.output.getColors(); + if (applied.length === 0 && decisions.length === 0 && skipped.length === 0) { + console.log('All forked docs are up to date.'); + return; + } + if (applied.length > 0) { + const verb = options.dryRun ? 'Would update' : 'Updated'; + console.log(`${verb} ${applied.length} forked doc(s):`); + for (const a of applied) { + console.log(` ${colors.success('✓')} ${a.message}`); + } + } + if (skipped.length > 0) { + console.log(''); + console.log(`${skipped.length} doc(s) skipped:`); + for (const sk of skipped) { + console.log(` ${colors.warn('⚠')} ${sk.message}`); + } + } + if (decisions.length > 0) { + console.log(''); + console.log(`${decisions.length} doc(s) need a decision:`); + for (const d of decisions) { + console.log(` ${colors.warn('⚠')} ${d.message}`); + } + console.log(' re-run with one of:'); + console.log( + ' tbd docs update --merge # combine, then resolve conflict markers', + ); + console.log( + ' tbd docs update --keep-ours # keep your version, advance the fork point', + ); + } + }, 'Failed to update forked docs'); + } +} + +interface ListOptions { + kind?: string; + json?: boolean; +} + +class DocsListHandler extends BaseCommand { + async run(options: ListOptions): Promise { + await this.execute(async () => { + const tbdRoot = await requireInit(); + const { manifest, files, localDirs } = await loadServeContext(tbdRoot); + const parsedKind = parseKindOption(options.kind); + const kinds = parsedKind ? [parsedKind] : RESOLVABLE_KINDS; + const colors = this.output.getColors(); + const seenLocal = new Set(); + + interface Row { + name: string; + title?: string; + description?: string; + sizeInfo: string; + marker: string; + state: string; + path: string; + } + const grouped: { kind: ForkKind; rows: Row[] }[] = []; + const docmapEntries: DocMapEntry[] = []; + + for (const kind of kinds) { + const cache = new DocCache(effectiveServePaths(kind, localDirs), tbdRoot); + await cache.load({ quiet: true }); + const rows: Row[] = []; + for (const doc of cache.list()) { + if (localDirs.includes(doc.sourceDir)) { + if (seenLocal.has(doc.path)) continue; + seenLocal.add(doc.path); + } + const { entry, state, marker } = servedEntryFor( + tbdRoot, + kind, + doc, + manifest, + files, + localDirs, + ); + rows.push({ + name: doc.name, + title: doc.frontmatter?.title, + description: doc.frontmatter?.description, + sizeInfo: formatDocSize(doc.sizeBytes, doc.approxTokens), + marker, + state, + path: entry.path ?? doc.sourceDir + '/' + doc.name + '.md', + }); + docmapEntries.push(entry); + } + grouped.push({ kind, rows }); + } + + if (this.ctx.json) { + this.output.data(createDocMap(docmapEntries, { name: 'tbd-docs' })); + return; + } + + for (const { kind, rows } of grouped) { + if (rows.length === 0) continue; + if (!options.kind) console.log(colors.bold(kind)); + for (const r of rows) { + const indent = options.kind ? '' : ' '; + const markerStr = r.marker ? ` ${colors.dim(r.marker)}` : ''; + console.log(`${indent}${colors.bold(r.name)} ${colors.dim(r.sizeInfo)}${markerStr}`); + const desc = + r.title && r.description ? `${r.title}: ${r.description}` : (r.title ?? r.description); + if (desc) console.log(`${indent} ${desc}`); + } + } + }, 'Failed to list docs'); + } +} + +interface DiffOptions { + base?: boolean; + upstream?: boolean; + kind?: string; +} + +class DocsDiffHandler extends BaseCommand { + async run(name: string, options: DiffOptions): Promise { + await this.execute(async () => { + const tbdRoot = await requireInit(); + const manifest = await readForkManifest(tbdRoot); + const entry = findFork(manifest, name, parseKindOption(options.kind)); + if (!entry) { + throw new CLIError(`${name} is not a forked doc. Run \`tbd docs status\` to see forks.`); + } + + const forkContent = await readForkFile(tbdRoot, FORK_DIR, entry); + const baseContent = await readForkBase(tbdRoot, entry); + const cache = await buildKindCache(entry.kind, tbdRoot); + const upstreamContent = cache.get(entry.name)?.doc.content ?? null; + + // Default: your file vs current upstream (the net fork). + let left = upstreamContent; + let right = forkContent; + let labels = { left: 'upstream', right: 'ours' }; + if (options.base) { + left = baseContent; + right = forkContent; + labels = { left: 'base', right: 'ours' }; + } else if (options.upstream) { + left = baseContent; + right = upstreamContent; + labels = { left: 'base', right: 'upstream' }; + } + + if (left === null || right === null) { + throw new CLIError( + `Cannot diff ${name}: one side is unavailable ` + + `(forked file missing, base missing, or upstream gone).`, + ); + } + + const diff = await diffContents(left, right, labels); + if (this.ctx.json) { + this.output.data({ name: entry.name, kind: entry.kind, diff }); + return; + } + if (!diff.trim()) { + console.log(`No differences (${labels.left} vs ${labels.right}).`); + return; + } + console.log(diff.trimEnd()); + }, 'Failed to diff'); + } +} + +/** + * Merge a subcommand's local options with globals/ancestors (e.g. the global + * --dry-run and --json), preferring the subcommand's own values. + */ +function mergedForkOptions(local: ForkOptions, command: Command): ForkOptions { + const g = command.optsWithGlobals(); + return { + all: local.all ?? g.all, + category: local.category, + kind: local.kind ?? g.kind, + force: local.force ?? g.force, + dryRun: local.dryRun ?? g.dryRun, + json: g.json, + }; +} + +/** Register fork/unfork/status subcommands onto the `docs` command. */ +export function registerForkSubcommands(docs: Command): void { + docs + .command('fork') + .description( + 'Fork managed docs into the repo (default docs/tbd/) so they are visible and editable', + ) + .argument('[names...]', 'doc name(s) to fork') + .option('--kind ', 'restrict to a kind (guideline|shortcut|template|reference)') + .option('--all', 'fork all available docs') + .option( + '--category ', + 'fork all docs in a category (repeatable: general|typescript|python|convex|electron)', + (value: string, previous: string[] = []) => [...previous, value], + ) + .option('--force', 'overwrite an existing non-fork file') + .action(async (names: string[], options: ForkOptions, command: Command) => { + await new DocsForkHandler(command).run(names, mergedForkOptions(options, command)); + }); + + docs + .command('unfork') + .description( + 'Remove a fork and fall back to upstream (refuses to discard edits without --force)', + ) + .argument('[names...]', 'doc name(s) to unfork') + .option('--kind ', 'restrict to a kind') + .option('--all', 'unfork all forked docs') + .option('--force', 'discard local customizations') + .action(async (names: string[], options: UnforkOptions, command: Command) => { + const m = mergedForkOptions(options, command); + await new DocsUnforkHandler(command).run(names, { + all: m.all, + kind: m.kind, + force: m.force, + json: m.json, + }); + }); + + docs + .command('list') + .description('List all docs across kinds, with [forked]/[customized]/[local] markers') + .option('--kind ', 'restrict to a kind (guideline|shortcut|template)') + .action(async (options: ListOptions, command: Command) => { + await new DocsListHandler(command).run({ + kind: options.kind, + json: command.optsWithGlobals().json === true, + }); + }); + + docs + .command('status') + .description('Show forked docs and their states') + .action(async (_options: { json?: boolean }, command: Command) => { + await new DocsStatusHandler(command).run({ json: command.optsWithGlobals().json === true }); + }); + + docs + .command('update') + .description('Reconcile forked docs with upstream after an upgrade (--merge / --keep-ours)') + .argument('[names...]', 'doc name(s) to update (default: all)') + .option('--merge', 'on conflict: combine and write conflict markers to resolve') + .option('--keep-ours', 'on conflict: keep your version and advance the fork point') + .action(async (names: string[], options: UpdateOptions, command: Command) => { + const g = command.optsWithGlobals(); + await new DocsUpdateHandler(command).run(names, { + merge: options.merge, + keepOurs: options.keepOurs, + dryRun: g.dryRun === true, + json: g.json === true, + }); + }); + + docs + .command('diff') + .description( + 'Diff a forked doc against upstream (default), its base (--base), or incoming (--upstream)', + ) + .argument('', 'forked doc name') + .option('--base', 'diff your file against its base (what you changed)') + .option('--upstream', 'diff the base against current upstream (incoming changes)') + .option('--kind ', 'restrict to a kind') + .action(async (name: string, options: DiffOptions, command: Command) => { + await new DocsDiffHandler(command).run(name, options); + }); +} diff --git a/packages/tbd/src/cli/commands/docs.ts b/packages/tbd/src/cli/commands/docs.ts index d31d07ac..72fc597d 100644 --- a/packages/tbd/src/cli/commands/docs.ts +++ b/packages/tbd/src/cli/commands/docs.ts @@ -1,249 +1,481 @@ /** - * `tbd docs` - Display CLI documentation. + * `tbd docs` — manage tbd-served docs: browse, fork into the repo, sync the + * cache, and pull upstream updates into forks. * - * Shows the bundled documentation for tbd CLI. - * Documentation can be filtered by section. - * - * Note: Doc cache sync functionality has moved to `tbd sync --docs`. - * See: docs/project/specs/active/plan-2026-01-29-unified-sync-command.md + * Surface (the f05 reorganization of the old manual viewer): + * tbd docs status overview of managed docs (landing page) + * tbd docs show read any doc by name (kind-agnostic; --section) + * tbd docs show tbd-docs the CLI manual (old bare `tbd docs`) + * tbd docs manual [topic] alias for `tbd docs show tbd-docs` + * tbd docs sync refresh the gitignored cache (canonical form of + * the deprecated `tbd sync --docs` alias) + * tbd docs list/status/fork/unfork/update/diff (see docs-fork.ts) */ import { Command } from 'commander'; import { readFile } from 'node:fs/promises'; import { fileURLToPath } from 'node:url'; -import { dirname, join } from 'node:path'; +import { dirname, join, relative } from 'node:path'; import { BaseCommand } from '../lib/base-command.js'; +import { registerForkSubcommands, parseKindOption, RESOLVABLE_KINDS } from './docs-fork.js'; import { shouldUseInteractiveOutput } from '../lib/context.js'; -import { CLIError, NotFoundError } from '../lib/errors.js'; +import { CLIError, NotFoundError, NotInitializedError, requireInit } from '../lib/errors.js'; import { renderMarkdown, paginateOutput } from '../lib/output.js'; +import { + printDocSyncResult, + printDocSyncStatus, + printForkDriftNotice, +} from '../lib/docs-sync-output.js'; +import { syncDocsWithDefaults } from '../../file/doc-sync.js'; +import { addDoc, type DocType } from '../../file/doc-add.js'; +import { parseDocRef } from '../../docref/index.js'; +import { DocCache } from '../../file/doc-cache.js'; +import { FORK_DIR } from '../../lib/paths.js'; +import { readForkManifest, type ForkKind } from '../../file/fork-manifest.js'; +import { computeForkDriftSummary } from '../../file/doc-fork.js'; +import { servedEntryFor, loadServeContext, effectiveServePaths } from '../lib/doc-serve.js'; +import { docsPostureMenuLines } from '../lib/docs-menu.js'; +import { createDocMap, type DocMapEntry } from '../../docmap/index.js'; import type { DocSection } from '../../lib/types.js'; import GithubSlugger from 'github-slugger'; +/** Reserved name that serves the CLI manual (`tbd-docs.md`). */ +const MANUAL_DOC_NAME = 'tbd-docs'; + /** - * Get the path to the bundled docs file. - * The docs file is copied to dist/docs/ during build. + * Self-docs: served as `reference` docs from the cache like everything else + * (so forks shadow them), but with a bundled fallback so they stay readable + * before init or before the first cache sync. */ -function getDocsPath(): string { - const __filename = fileURLToPath(import.meta.url); - const __dirname = dirname(__filename); - // When bundled, runs from dist/bin.mjs or dist/cli.mjs - // Docs are at dist/docs/tbd-docs.md (same level as the bundle) - return join(__dirname, 'docs', 'tbd-docs.md'); -} - -interface DocsOptions { - section?: string; - list?: boolean; - all?: boolean; -} +const BUNDLED_ROOT_DOCS: Record = { + [MANUAL_DOC_NAME]: 'tbd-docs.md', + 'tbd-design': 'tbd-design.md', +}; -class DocsHandler extends BaseCommand { - async run(topic: string | undefined, options: DocsOptions): Promise { - let content: string; +/** + * Read a doc bundled at the docs root. Copied to dist/docs/ during build; in + * development read from the package docs/ directory. + */ +async function readBundledRootDoc(filename: string): Promise { + const __dirname = dirname(fileURLToPath(import.meta.url)); + try { + return await readFile(join(__dirname, 'docs', filename), 'utf-8'); + } catch { try { - content = await readFile(getDocsPath(), 'utf-8'); + return await readFile(join(__dirname, '..', '..', '..', 'docs', filename), 'utf-8'); } catch { - // Fallback: try to read from source location during development - try { - const __filename = fileURLToPath(import.meta.url); - const __dirname = dirname(__filename); - // During development: src/cli/commands -> packages/tbd/docs - const devPath = join(__dirname, '..', '..', '..', 'docs', 'tbd-docs.md'); - content = await readFile(devPath, 'utf-8'); - } catch { - throw new CLIError('Documentation file not found. Please rebuild the CLI.'); - } + throw new CLIError('Documentation file not found. Please rebuild the CLI.'); } + } +} - const sections = this.extractSections(content); +/** Extract `## ` section metadata (title + GitHub slug) from a markdown doc. */ +function extractSections(content: string): DocSection[] { + const sections: DocSection[] = []; + const slugger = new GithubSlugger(); + for (const line of content.split('\n')) { + if (line.startsWith('## ')) { + const title = line.slice(3).trim(); + sections.push({ title, slug: slugger.slug(title) }); + } + } + return sections; +} - // Show comprehensive documentation listing - if (options.all) { - await this.showComprehensiveListing(); - return; +/** + * Extract one `## ` section (header through the next header), matching by slug + * or partial title. Returns null when no section matches. + */ +function extractSection(content: string, sections: DocSection[], query: string): string | null { + const lowerQuery = query.toLowerCase(); + const matched = + sections.find((s) => s.slug === lowerQuery) ?? + sections.find((s) => s.title.toLowerCase().includes(lowerQuery)); + if (!matched) { + return null; + } + + const lines = content.split('\n'); + let inSection = false; + const sectionLines: string[] = []; + for (const line of lines) { + if (line.startsWith('## ')) { + if (inSection) break; + if (line.slice(3).trim() === matched.title) { + inSection = true; + sectionLines.push(line); + } + } else if (inSection) { + sectionLines.push(line); } + } + while (sectionLines.length > 0 && sectionLines[sectionLines.length - 1]?.trim() === '') { + sectionLines.pop(); + } + return sectionLines.length > 0 ? sectionLines.join('\n') : null; +} - // List available sections - if (options.list) { - this.output.data(sections, () => { +/** + * Bare `tbd docs`: the status overview / landing page for managed docs. + * Mirrors `tbd status`: a summary plus pointers, never the full table. + */ +class DocsOverviewHandler extends BaseCommand { + async run(): Promise { + await this.execute(async () => { + let tbdRoot: string; + try { + tbdRoot = await requireInit(); + } catch (err) { + if (!(err instanceof NotInitializedError)) throw err; + // The overview stays useful before setup (the old viewer worked + // anywhere): point at the bundled manual and at initialization. const colors = this.output.getColors(); - console.log(colors.bold('Available documentation sections:')); + console.log(`${colors.bold('tbd docs')} — managed documentation`); console.log(''); - // Calculate max slug length for alignment - const maxSlugLen = Math.max(...sections.map((s) => s.slug.length)); - for (const section of sections) { - const paddedSlug = section.slug.padEnd(maxSlugLen); - console.log(` ${colors.id(paddedSlug)} ${section.title}`); - } + console.log(' tbd is not initialized in this repo (run: tbd setup --auto).'); + console.log(' The CLI manual is bundled and always available:'); console.log(''); - console.log(`Use ${colors.dim('tbd docs ')} to view a specific section.`); - }); - return; - } + console.log(' Learn more: tbd docs show tbd-docs (alias: tbd docs manual)'); + return; + } + const manifest = await readForkManifest(tbdRoot); + const drift = await computeForkDriftSummary(tbdRoot, FORK_DIR, manifest); - // Determine which section to show (positional topic takes precedence) - const sectionQuery = topic ?? options.section; + const { files, localDirs } = await loadServeContext(tbdRoot); + let total = 0; + const entries: DocMapEntry[] = []; + const seenLocal = new Set(); + for (const kind of RESOLVABLE_KINDS) { + const cache = new DocCache(effectiveServePaths(kind, localDirs), tbdRoot); + await cache.load({ quiet: true }); + for (const doc of cache.list()) { + // local_dirs serve every kind; inventory counts each local doc once. + if (localDirs.includes(doc.sourceDir)) { + if (seenLocal.has(doc.path)) continue; + seenLocal.add(doc.path); + } + total += 1; + if (this.ctx.json) { + entries.push(servedEntryFor(tbdRoot, kind, doc, manifest, files, localDirs).entry); + } + } + } - // Filter by section if specified - if (sectionQuery) { - const sectionContent = this.extractSection(content, sections, sectionQuery); - if (!sectionContent) { - throw new NotFoundError( - 'Section', - `"${sectionQuery}" (use --list to see available sections)`, - ); + if (this.ctx.json) { + // The overview is the docmap rendered in summary form; --json emits + // the docmap itself (one data model, one renderer). + this.output.data(createDocMap(entries, { name: 'tbd-docs' })); + return; } - content = sectionContent; - } - // Output the documentation with Markdown colorization and pagination for interactive - if (shouldUseInteractiveOutput(this.ctx)) { - const rendered = renderMarkdown(content, this.ctx.color); - await paginateOutput(rendered, true); - } else { - console.log(content); - } - } + const colors = this.output.getColors(); + console.log(`${colors.bold('tbd docs')} — managed documentation`); + console.log(''); - /** - * Extract section metadata from the documentation. - * Sections are top-level headers (## ). - * Returns title and slugified ID for each section. - */ - private extractSections(content: string): DocSection[] { - const sections: DocSection[] = []; - const lines = content.split('\n'); - const slugger = new GithubSlugger(); - - for (const line of lines) { - if (line.startsWith('## ')) { - const title = line.slice(3).trim(); - const slug = slugger.slug(title); - sections.push({ title, slug }); + if (drift.forks === 0) { + console.log( + ` ${total} docs available in the cache (.tbd/docs/, gitignored); none forked into the repo.`, + ); + // One menu, shared with the setup Docs summary (docs-menu.ts). + const [intro, ...postures] = docsPostureMenuLines(); + console.log(` ${intro}`); + console.log(''); + for (const line of postures.slice(0, -1)) { + console.log(` ${line}`); + } + console.log(''); + console.log(` ${postures[postures.length - 1]}`); + console.log( + ` Learn more: tbd docs show tbd-docs (the manual; alias: tbd docs manual)`, + ); + return; } - } - return sections; + const upstream = total - drift.forks; + console.log( + ` ${total} available (${upstream} upstream, ${drift.forks} forked into ${FORK_DIR}/)`, + ); + const parts = [`${drift.customized} customized`]; + if (drift.stale > 0) { + parts.push(`${drift.stale} with upstream updates — run 'tbd docs update'`); + } + if (drift.conflicted > 0) parts.push(`${drift.conflicted} conflict pending`); + if (drift.missing > 0) parts.push(`${drift.missing} missing — see 'tbd docs status'`); + if (drift.local > 0) parts.push(`${drift.local} local`); + console.log(` ${drift.forks} forked: ${parts.join(', ')}`); + console.log(''); + console.log(' Inspect: tbd docs status'); + console.log(' Browse: tbd docs list'); + console.log(' Update: tbd docs update'); + console.log(' Learn more: tbd docs show tbd-docs'); + }, 'Failed to read docs overview'); } +} - /** - * Extract a specific section from the documentation. - * Matches by slug or partial title match. - * Returns content from the section header to the next section header. - */ - private extractSection(content: string, sections: DocSection[], query: string): string | null { - const lowerQuery = query.toLowerCase(); - - // Find matching section - first try exact slug match, then partial title match - const matchedSection = - sections.find((s) => s.slug === lowerQuery) ?? - sections.find((s) => s.title.toLowerCase().includes(lowerQuery)); - - if (!matchedSection) { - return null; - } +interface ShowOptions { + section?: string; + sections?: boolean; + kind?: string; +} + +/** + * `tbd docs show `: kind-agnostic read of any managed doc. The reserved + * `tbd-docs` name serves the bundled CLI manual (with `--section` navigation, + * relocated here from the old bare `tbd docs` viewer). + */ +class DocsShowHandler extends BaseCommand { + async run(name: string, options: ShowOptions): Promise { + await this.execute(async () => { + let content: string; + let provenance: string | null = null; - const lines = content.split('\n'); - let inSection = false; - const sectionLines: string[] = []; + const bundledFallback = BUNDLED_ROOT_DOCS[name]; + let tbdRoot: string | null = null; + if (bundledFallback) { + try { + tbdRoot = await requireInit(); + } catch (err) { + if (!(err instanceof NotInitializedError)) throw err; + } + } else { + tbdRoot = await requireInit(); + } - for (const line of lines) { - if (line.startsWith('## ')) { - if (inSection) { - // End of our section - break; + if (tbdRoot === null) { + // Self-docs stay readable before init. + content = await readBundledRootDoc(bundledFallback!); + } else { + const requestedKind = parseKindOption(options.kind); + const kinds = requestedKind ? [requestedKind] : RESOLVABLE_KINDS; + const { + manifest: showManifest, + files: showFiles, + localDirs, + } = await loadServeContext(tbdRoot); + const matches: { kind: ForkKind; content: string; sourceDir: string; path: string }[] = []; + for (const kind of kinds) { + const cache = new DocCache(effectiveServePaths(kind, localDirs), tbdRoot); + await cache.load({ quiet: true }); + const hit = cache.get(name); + if (hit) { + matches.push({ + kind, + content: hit.doc.content, + sourceDir: hit.doc.sourceDir, + path: hit.doc.path, + }); + } } - const currentTitle = line.slice(3).trim(); - if (currentTitle === matchedSection.title) { - inSection = true; - sectionLines.push(line); + // local_dirs serve every kind, so one local file matches under each + // kind probe; identical paths are one doc, not an ambiguity. + const uniquePaths = new Set(matches.map((m) => m.path)); + if (uniquePaths.size === 1 && matches.length > 1) { + matches.length = 1; + } + if (matches.length === 0) { + if (bundledFallback) { + // Initialized but cache not yet synced with the self-docs. + content = await readBundledRootDoc(bundledFallback); + } else { + throw new NotFoundError('Doc', `"${name}" (run \`tbd docs list\` to see names)`); + } + } else if (matches.length > 1) { + const kindList = matches.map((m) => m.kind).join(', '); + throw new CLIError( + `"${name}" exists in multiple kinds (${kindList}). Use --kind to disambiguate.`, + ); + } else { + const match = matches[0]!; + content = match.content; + if (this.ctx.json && !options.sections) { + const { entry } = servedEntryFor( + tbdRoot, + match.kind, + { name, content, sourceDir: match.sourceDir, path: match.path }, + showManifest, + showFiles, + localDirs, + ); + this.output.data({ ...entry, content }); + return; + } + if (match.sourceDir.startsWith(FORK_DIR)) { + provenance = `(serving forked copy: ${relative(tbdRoot, match.path).split('\\').join('/')})`; + } else if (localDirs.includes(match.sourceDir)) { + provenance = `(serving local doc: ${relative(tbdRoot, match.path).split('\\').join('/')})`; + } } - } else if (inSection) { - sectionLines.push(line); } - } - if (sectionLines.length === 0) { - return null; - } + const sections = extractSections(content); + + if (options.sections) { + this.output.data(sections, () => { + const colors = this.output.getColors(); + console.log(colors.bold(`Sections in ${name}:`)); + console.log(''); + const maxSlugLen = Math.max(...sections.map((s) => s.slug.length)); + for (const section of sections) { + console.log(` ${colors.id(section.slug.padEnd(maxSlugLen))} ${section.title}`); + } + console.log(''); + console.log( + `Use ${colors.dim(`tbd docs show ${name} --section `)} to view a section.`, + ); + }); + return; + } - // Trim trailing empty lines - while (sectionLines.length > 0) { - const lastLine = sectionLines[sectionLines.length - 1]; - if (lastLine?.trim() === '') { - sectionLines.pop(); + if (options.section) { + const sectionContent = extractSection(content, sections, options.section); + if (!sectionContent) { + throw new NotFoundError( + 'Section', + `"${options.section}" (use --sections to see available sections)`, + ); + } + content = sectionContent; + } + + // Provenance to stderr so piped stdout stays clean (on by default; + // the extra context helps agents recall which docs are customized). + if (provenance && !this.ctx.quiet && !this.ctx.json) { + process.stderr.write(`${provenance}\n`); + } + + if (shouldUseInteractiveOutput(this.ctx)) { + const rendered = renderMarkdown(content, this.ctx.color); + await paginateOutput(rendered, true); } else { - break; + console.log(content); } - } + }, 'Failed to show doc'); + } +} + +interface AddOptions { + kind?: string; + name?: string; +} - return sectionLines.join('\n'); +/** + * `tbd docs add `: register an external doc in the cache. The unified + * form of the per-kind --add flags (which remain as aliases); the canonical + * docref is what config records. + */ +class DocsAddHandler extends BaseCommand { + async run(docref: string, options: AddOptions): Promise { + await this.execute(async () => { + const tbdRoot = await requireInit(); + const kind = parseKindOption(options.kind); + if (!kind) { + throw new CLIError('--kind is required (guideline|shortcut|template|reference)'); + } + const inferredName = (() => { + const parsed = parseDocRef(docref); + const p = + parsed.kind === 'url' + ? new URL(parsed.url).pathname + : 'path' in parsed + ? parsed.path + : ''; + const base = p.split('/').filter(Boolean).pop() ?? ''; + return base.endsWith('.md') ? base.slice(0, -3) : base; + })(); + const finalName = options.name ?? inferredName; + if (!finalName) { + throw new CLIError('--name is required when it cannot be inferred from the docref'); + } + + console.log(`Adding ${kind}: ${finalName}`); + const result = await addDoc(tbdRoot, { + url: docref, + name: finalName, + docType: kind as DocType, + }); + console.log(` Source: ${result.source}`); + if (result.usedGhCli) { + const colors = this.output.getColors(); + console.log(colors.dim(' (fetched via gh CLI due to direct access restriction)')); + } + this.output.success(`Added to .tbd/docs/${result.destPath}`); + console.log(` Config updated (docs_cache.files): ${result.source}`); + console.log(''); + console.log( + `Run 'tbd docs list' to verify, or 'tbd docs fork ${finalName}' to make it visible.`, + ); + }, 'Failed to add doc'); } +} - /** - * Show a comprehensive listing of all documentation resources organized by purpose. - */ - private async showComprehensiveListing(): Promise { - const colors = this.output.getColors(); - - console.log(colors.bold('=== tbd Documentation Resources ===')); - console.log(''); - - // Getting Started - console.log(colors.bold('Getting Started:')); - console.log(' tbd Full orientation and project status'); - console.log(' tbd prime Workflow context and guidance'); - console.log(' tbd prime --brief Quick reference (~35 lines)'); - console.log(' tbd --help CLI command reference'); - console.log(''); - - // Workflows (Shortcuts) - console.log(colors.bold('Workflows (Shortcuts):')); - console.log(' tbd shortcut --list List all available shortcuts'); - console.log(' tbd shortcut new-plan-spec Plan a new feature'); - console.log(' tbd shortcut code-review-and-commit Commit code properly'); - console.log(' tbd shortcut create-or-update-pr-simple Create a pull request'); - console.log(''); - - // Guidelines - console.log(colors.bold('Guidelines (Coding Standards):')); - console.log(' tbd guidelines --list List all available guidelines'); - console.log(' tbd guidelines typescript-rules TypeScript best practices'); - console.log(' tbd guidelines general-tdd-guidelines Test-driven development'); - console.log(' tbd guidelines golden-testing-guidelines Snapshot/golden testing'); - console.log(''); - - // Templates - console.log(colors.bold('Templates:')); - console.log(' tbd template --list List all available templates'); - console.log(' tbd template plan-spec Feature planning template'); - console.log(' tbd template architecture-doc Architecture document template'); - console.log(''); - - // Design & Reference - console.log(colors.bold('Design & Reference:')); - console.log(' tbd docs --list List documentation sections'); - console.log(' tbd design tbd design document'); - console.log(' tbd closing Session closing protocol'); - console.log(''); - - // Quick Tips - console.log(colors.bold('Quick Tips:')); - console.log(' - Run tbd ready to see what issues are available to work on'); - console.log(' - Run tbd shortcut to get step-by-step instructions'); - console.log(' - Run tbd guidelines to get coding standards'); - console.log(' - Always run tbd sync at the end of a session'); +/** `tbd docs sync`: refresh the gitignored docs cache (canonical command). */ +class DocsSyncHandler extends BaseCommand { + async run(): Promise { + await this.execute(async () => { + const tbdRoot = await requireInit(); + if (this.ctx.dryRun) { + const result = await syncDocsWithDefaults(tbdRoot, { dryRun: true }); + printDocSyncStatus(this.output, result); + return; + } + const spinner = this.output.spinner('Syncing docs...'); + const result = await syncDocsWithDefaults(tbdRoot); + spinner.stop(); + printDocSyncResult(this.output, result); + await printForkDriftNotice(this.output, tbdRoot); + }, 'Failed to sync docs'); } } export const docsCommand = new Command('docs') - .description('Display CLI documentation (use tbd sync --docs for doc cache sync)') - .argument('[topic]', 'Topic to display (e.g., "commands", "id-system")') - .option('--section ', 'Show specific section (e.g., "commands", "workflows")') - .option('--list', 'List available sections') - .option('--all', 'Show comprehensive listing of all documentation resources') - .action(async (topic: string | undefined, options: DocsOptions, command: Command) => { - const handler = new DocsHandler(command); - await handler.run(topic, options); + .description('Manage tbd-served docs: browse, fork into your repo, and pull upstream updates') + .action(async (_options: unknown, command: Command) => { + await new DocsOverviewHandler(command).run(); + }); + +docsCommand + .command('show') + .description('Read any managed doc by name (tbd-docs is the CLI manual)') + .argument('', 'doc name (e.g. python-rules, tbd-docs)') + .option('--section ', 'show one section of the doc') + .option('--sections', 'list the doc’s sections') + .option('--kind ', 'restrict to a kind (guideline|shortcut|template)') + .action(async (name: string, options: ShowOptions, command: Command) => { + await new DocsShowHandler(command).run(name, options); + }); + +docsCommand + .command('manual') + .description('Show the tbd CLI manual (alias for: tbd docs show tbd-docs)') + .argument('[topic]', 'section to display (e.g. "commands", "id-system")') + .option('--section ', 'show one section of the manual') + .option('--sections', 'list the manual’s sections') + .action(async (topic: string | undefined, options: ShowOptions, command: Command) => { + await new DocsShowHandler(command).run(MANUAL_DOC_NAME, { + ...options, + section: options.section ?? topic, + }); }); + +docsCommand + .command('add') + .description( + 'Register an external doc in the cache by docref (URL, github:o/r@ref//path, ./local)', + ) + .argument('', 'source docref') + .option('--kind ', 'doc kind (guideline|shortcut|template|reference)') + .option('--name ', 'doc name (defaults to the source filename)') + .action(async (docref: string, options: AddOptions, command: Command) => { + await new DocsAddHandler(command).run(docref, options); + }); + +docsCommand + .command('sync') + .description('Refresh the gitignored docs cache (.tbd/docs/) from bundled and URL sources') + .action(async (_options: unknown, command: Command) => { + await new DocsSyncHandler(command).run(); + }); + +// Fork lifecycle operations (fork / unfork / status / list / update / diff). +registerForkSubcommands(docsCommand); diff --git a/packages/tbd/src/cli/commands/doctor.ts b/packages/tbd/src/cli/commands/doctor.ts index 8915a91d..e4310348 100644 --- a/packages/tbd/src/cli/commands/doctor.ts +++ b/packages/tbd/src/cli/commands/doctor.ts @@ -22,7 +22,13 @@ import { resolveSharedTbdPaths, TBD_DIR, DATA_SYNC_DIR, + FORK_DIR, + CACHE_GUIDELINES_PATHS, + CACHE_REFERENCE_PATHS, + CACHE_SHORTCUT_PATHS, + CACHE_TEMPLATE_PATHS, } from '../../lib/paths.js'; +import type { ForkEntry } from '../../file/fork-manifest.js'; import { detectDuplicateYamlKeys } from '../../utils/yaml-utils.js'; import { getClaudePaths, @@ -53,6 +59,7 @@ import { } from '../../file/git.js'; import { CommonDirLayoutError, + isLayoutUpgradeable, readCommonDirLayout, validateCommonDirLayout, withSharedDataSyncLock, @@ -354,6 +361,24 @@ class DoctorHandler extends BaseCommand { // Check 15: Sync consistency (worktree matches local, ahead/behind counts) healthChecks.push(await this.safeCheck('Sync consistency', () => this.checkSyncConsistency())); + // Check 16: Forked docs (manifest ↔ base snapshots ↔ fork dir consistency). + // A check *group*: contributes zero lines when nothing is forked and no + // fork dir exists (doctor output for non-fork users must not grow), one ✓ + // line when all forks are healthy, and one ⚠ line per issue category + // otherwise. Unexpected throws degrade to one error finding (safeCheck + // semantics, adapted for a multi-result check). + try { + healthChecks.push(...(await this.checkForkedDocs(options.fix))); + } catch (error) { + healthChecks.push({ + name: 'Forked docs', + status: 'error', + message: `check could not complete: ${ + error instanceof Error ? error.message : String(error) + }`, + }); + } + // Run integration checks (optional IDE/agent integrations) const integrationChecks: DiagnosticResult[] = []; @@ -1208,11 +1233,27 @@ class DoctorHandler extends BaseCommand { try { layout = await readCommonDirLayout(layoutPath); } catch (error) { + // A corrupt/unparseable layout is machine-local and regenerable from the + // config — make it fixable rather than a dead-end error. + if (fix && !this.checkDryRun('Rewrite corrupt common-dir layout from config')) { + const configRef = this.config; + await withSharedDataSyncLock(this.cwd, async () => + writeCommonDirLayout(sharedPaths, configRef), + ); + return { + name: 'Common-dir layout', + status: 'ok', + message: 'rewritten from config (was unreadable)', + path: layoutPath, + }; + } return { name: 'Common-dir layout', status: 'error', - message: error instanceof Error ? error.message : String(error), + message: `${error instanceof Error ? error.message : String(error)}`, path: layoutPath, + fixable: true, + suggestion: `Run: tbd doctor --fix (rewrites it from config), or delete ${layoutPath}`, }; } if (!layout) { @@ -1233,6 +1274,33 @@ class DoctorHandler extends BaseCommand { suggestion: 'Upgrade: npm install -g get-tbd@latest', }; } + // A layout from an older but compatible format than the (in-memory, + // already-migrated) config is the normal mid-migration state, not a + // mismatch: the format bump applies on the next data command. Surface it as + // an informational warning (exit 0, so CI on un-migrated f04 repos is not + // broken); --fix applies the FULL migration (config + layout) via the locked + // data-context path — never just the layout, which would half-migrate the + // repo and lock out older clients with nothing to commit. + if (isLayoutUpgradeable(layout, this.config)) { + if (fix && !this.checkDryRun('Apply pending format migration')) { + await prepareDataSyncContext(this.cwd); + return { + name: 'Common-dir layout', + status: 'ok', + message: `format migration applied (${layout.tbd_format} → ${this.config.tbd_format})`, + path: layoutPath, + }; + } + return { + name: 'Common-dir layout', + status: 'warn', + message: `format migration pending (${layout.tbd_format} → ${this.config.tbd_format}); applies on next write or 'tbd doctor --fix'`, + path: layoutPath, + fixable: true, + suggestion: 'Run: tbd doctor --fix (or any write command) to apply', + }; + } + try { validateCommonDirLayout(layout, this.config); return { name: 'Common-dir layout', status: 'ok', path: layoutPath }; @@ -1738,6 +1806,305 @@ class DoctorHandler extends BaseCommand { }; } } + + /** + * Check 16 group: Forked docs (`.tbd/doc-forks/` ↔ base snapshots ↔ fork dir). + * + * Validates the forkable-docs state per the f05 spec (Phase 2 doctor checks): + * manifest readability, missing forked files (--fix finalizes the unfork), + * orphaned entries whose upstream doc is gone (--fix removes the entry), + * base snapshot presence/hash integrity (no auto-fix — re-fork vs unfork is + * the user's call), unresolved tbd conflict markers, user docs claiming the + * reserved `tbd-` name prefix, and a gitignored fork dir. + * + * Returns zero findings when nothing is forked and no fork dir exists, so + * doctor output is byte-identical for repos that never touched forking. + * See: docs/project/specs/done/plan-2026-06-11-forkable-docs.md §`tbd doctor`. + */ + private async checkForkedDocs(fix?: boolean): Promise { + const name = 'Forked docs'; + const { + DOC_FORKS_DIR, + FORKS_FILE, + findFork, + hashContent, + hasUnresolvedConflict, + readForkManifest, + removeBaseContent, + removeFork, + withForkManifestLock, + writeForkManifest, + } = await import('../../file/fork-manifest.js'); + const manifestPath = `${DOC_FORKS_DIR}/${FORKS_FILE}`; + + // 16a: manifest readability. readForkManifest tolerates per-entry corruption + // (drops bad entries with a stderr warning) but throws on a totally + // unparseable file — report that instead of crashing the doctor run. + let manifest; + try { + manifest = await readForkManifest(this.cwd); + } catch (error) { + const reason = (error instanceof Error ? error.message : String(error)).split('\n')[0]; + return [ + { + name, + status: 'warn', + message: `fork manifest unreadable: ${reason}`, + path: manifestPath, + suggestion: `Fix or delete ${manifestPath} (forked files stay in place), then re-run tbd doctor`, + }, + ]; + } + + // Zero forks and no fork dir: print nothing. + let forkDirExists = true; + try { + await access(join(this.cwd, FORK_DIR)); + } catch { + forkDirExists = false; + } + if (manifest.forks.length === 0 && !forkDirExists) { + return []; + } + + const { + forkStatusFor, + listLocalForkFiles, + readForkBase, + readForkFile, + regenerateForkDirReadme, + unforkDoc, + } = await import('../../file/doc-fork.js'); + const { DocCache } = await import('../../file/doc-cache.js'); + + // Cache-only lookup paths per kind (the pristine upstream copies). Replica + // of doc-fork.ts's module-private KIND_CACHE_PATHS, which is deliberately + // not exported (doctor owns its own copy rather than widening that API). + const kindCachePaths: Record = { + guideline: CACHE_GUIDELINES_PATHS, + shortcut: CACHE_SHORTCUT_PATHS, + template: CACHE_TEMPLATE_PATHS, + reference: CACHE_REFERENCE_PATHS, + }; + const caches = new Map>(); + + // Classify every manifest entry into at most one issue bucket + // (missing > orphaned > base problem), with unresolved conflict markers + // detected on every fork file that still exists. + const missing: ForkEntry[] = []; + const orphaned: ForkEntry[] = []; + const baseProblems: string[] = []; + const conflicted: string[] = []; + + for (const entry of manifest.forks) { + let cache = caches.get(entry.kind); + if (!cache) { + cache = new DocCache(kindCachePaths[entry.kind] ?? [], this.cwd); + await cache.load({ quiet: true }); + caches.set(entry.kind, cache); + } + const cacheContent = cache.get(entry.name)?.doc.content ?? null; + const status = await forkStatusFor(this.cwd, FORK_DIR, entry, cacheContent); + + if (status.state === 'missing') { + missing.push(entry); + continue; + } + if (status.orphaned) { + orphaned.push(entry); + continue; + } + + // 16d: base snapshot integrity for live forks. + const base = await readForkBase(this.cwd, entry); + if (base === null) { + baseProblems.push(`${entry.name}: missing`); + } else if (hashContent(base) !== entry.base_hash) { + baseProblems.push(`${entry.name}: hash mismatch`); + } + + // 16e: unresolved tbd conflict markers (flag-independent — detect markers + // even when the manifest `conflicted` flag was never set or went stale). + const content = await readForkFile(this.cwd, FORK_DIR, entry); + if (content !== null && hasUnresolvedConflict(content)) { + conflicted.push(entry.name); + } + } + + const results: DiagnosticResult[] = []; + + // 16b: manifest entries whose forked file was deleted out-of-band. The + // deletion is read as intent to stop forking: --fix finalizes the unfork + // (removes manifest entry + base snapshot; the doc is served from upstream). + if (missing.length > 0) { + const message = `${missing.length} missing (${missing.map((e) => e.name).join(', ')}: forked file deleted)`; + if (fix && !this.checkDryRun('Finalize unfork of deleted forked docs')) { + try { + await withForkManifestLock(this.cwd, async () => { + let current = await readForkManifest(this.cwd); + for (const entry of missing) { + if (!findFork(current, entry.name, entry.kind)) continue; + const result = await unforkDoc({ + tbdRoot: this.cwd, + forkDir: FORK_DIR, + manifest: current, + name: entry.name, + kind: entry.kind, + }); + current = result.manifest; + } + await writeForkManifest(this.cwd, current); + await regenerateForkDirReadme(this.cwd, FORK_DIR, current); + }); + results.push({ + name, + status: 'warn', + message, + details: [ + 'Fixed: finalized unfork (removed manifest entry + base); now served from upstream', + ], + }); + } catch (error) { + results.push({ + name, + status: 'error', + message: `failed to finalize unfork: ${ + error instanceof Error ? error.message : String(error) + }`, + }); + } + } else { + results.push({ + name, + status: 'warn', + message, + fixable: true, + suggestion: + 'Run: tbd doctor --fix to finalize the unfork, or tbd docs fork --force to restore', + }); + } + } + + // 16c: orphaned entries (upstream/cache doc no longer exists). --fix removes + // the manifest entry + base but keeps the file (it becomes a local doc — + // upstream is gone, so the file may be the only copy). + if (orphaned.length > 0) { + const message = `${orphaned.length} orphaned (${orphaned.map((e) => e.name).join(', ')}: upstream doc no longer exists)`; + if (fix && !this.checkDryRun('Remove orphaned fork manifest entries')) { + try { + await withForkManifestLock(this.cwd, async () => { + let current = await readForkManifest(this.cwd); + for (const entry of orphaned) { + current = removeFork(current, entry.name, entry.kind); + await removeBaseContent(this.cwd, entry.kind, entry.name); + } + await writeForkManifest(this.cwd, current); + await regenerateForkDirReadme(this.cwd, FORK_DIR, current); + }); + results.push({ + name, + status: 'warn', + message, + details: [ + `Fixed: removed orphaned manifest entr${orphaned.length === 1 ? 'y' : 'ies'} + base; file kept as a local doc`, + ], + }); + } catch (error) { + results.push({ + name, + status: 'error', + message: `failed to remove orphaned entries: ${ + error instanceof Error ? error.message : String(error) + }`, + }); + } + } else { + results.push({ + name, + status: 'warn', + message, + fixable: true, + suggestion: + 'Run: tbd doctor --fix to remove the entry (your file is kept as a local doc)', + }); + } + } + + // 16d findings: no auto-fix — choosing between re-fork and unfork would + // guess at user intent. + if (baseProblems.length > 0) { + results.push({ + name, + status: 'warn', + message: `${baseProblems.length} base snapshot problem${baseProblems.length === 1 ? '' : 's'} (${baseProblems.join(', ')})`, + suggestion: 'Run: tbd docs fork --force to re-fork, or tbd docs unfork ', + }); + } + + // 16e findings. + if (conflicted.length > 0) { + results.push({ + name, + status: 'warn', + message: `${conflicted.length} unresolved merge conflict${conflicted.length === 1 ? '' : 's'} (${conflicted.join(', ')})`, + suggestion: 'Run: resolve the conflict markers, then re-run tbd docs update', + }); + } + + // Healthy headline: exactly one ✓ line when forks exist and 16b–16e found + // nothing (reserved-name and fork-dir findings below have their own names). + if (manifest.forks.length > 0 && results.length === 0) { + results.push({ + name, + status: 'ok', + message: `${manifest.forks.length} forked, base snapshots intact`, + }); + } + + // 16f: user docs claiming the reserved `tbd-` prefix (fork-dir files with + // no manifest entry; forked tbd self-docs legitimately keep their entry). + const locals = await listLocalForkFiles(this.cwd, FORK_DIR, manifest); + const reserved = locals.filter((l) => l.name.startsWith('tbd-')); + if (reserved.length > 0) { + results.push({ + name: 'Reserved tbd- names', + status: 'warn', + message: `${reserved.length} user doc${reserved.length === 1 ? ' claims' : 's claim'} the reserved tbd- prefix`, + details: reserved.map((l) => l.relPath), + suggestion: 'Rename the file(s): the tbd- prefix is reserved for tbd self-docs', + }); + } + + // 16g: fork dir gitignored (only meaningful when forks exist — a gitignored + // fork dir defeats the purpose of forking: the docs would not be committed). + if (manifest.forks.length > 0) { + let ignored = false; + try { + await git('-C', this.cwd, 'check-ignore', '-q', FORK_DIR); + ignored = true; + } catch { + // Exit 1 = not ignored (healthy). Other failures: cannot verify; do not + // warn on a guess. + ignored = false; + } + results.push( + ignored + ? { + name: 'Fork dir', + status: 'warn', + message: `${FORK_DIR}/ is gitignored — forked docs will not be committed`, + suggestion: `Remove the .gitignore rule covering ${FORK_DIR}/ so forks are tracked in git`, + } + : { + name: 'Fork dir', + status: 'ok', + message: `${FORK_DIR}/ tracked in git (not gitignored)`, + }, + ); + } + + return results; + } } export const doctorCommand = new Command('doctor') diff --git a/packages/tbd/src/cli/commands/guidelines.ts b/packages/tbd/src/cli/commands/guidelines.ts index 22ba26e5..03df8dfc 100644 --- a/packages/tbd/src/cli/commands/guidelines.ts +++ b/packages/tbd/src/cli/commands/guidelines.ts @@ -8,53 +8,15 @@ import { Command } from 'commander'; import pc from 'picocolors'; +import { createDocMap } from '../../docmap/index.js'; import { DocCommandHandler, type DocCommandOptions } from '../lib/doc-command-handler.js'; import { CLIError } from '../lib/errors.js'; import { DEFAULT_GUIDELINES_PATHS } from '../../lib/paths.js'; import { truncate } from '../../lib/truncate.js'; +import { DOC_CATEGORIES, docCategory, parseCategoryOption } from '../../lib/doc-categories.js'; import { formatDocSize } from '../../lib/format-utils.js'; import { getTerminalWidth } from '../lib/output.js'; -/** - * Guideline categories for filtering. - */ -type GuidelineCategory = 'typescript' | 'python' | 'testing' | 'general'; - -/** - * Infer category from guideline name. - */ -function inferGuidelineCategory(name: string): GuidelineCategory | undefined { - // TypeScript guidelines - if (name.startsWith('typescript-')) { - return 'typescript'; - } - - // Python guidelines - if (name.startsWith('python-')) { - return 'python'; - } - - // Testing guidelines - if (name.includes('tdd') || name.includes('testing') || name.includes('golden')) { - return 'testing'; - } - - // General guidelines (everything else starting with general- or other general rules) - if ( - name.startsWith('general-') || - name.includes('rules') || - name.includes('patterns') || - name.startsWith('backward-') || - name.startsWith('convex-') || - name.startsWith('release-') || - name.startsWith('writing-') - ) { - return 'general'; - } - - return undefined; -} - interface GuidelinesOptions extends DocCommandOptions { category?: string; add?: string; @@ -109,35 +71,29 @@ class GuidelinesHandler extends DocCommandHandler { let docs = this.cache.list(includeAll); - // Filter by category if specified + // Filter by the declared frontmatter category (name-based inference retired) if (category) { - docs = docs.filter((d) => { - const docCategory = inferGuidelineCategory(d.name); - return docCategory === category; - }); + const wanted = parseCategoryOption(category); + docs = docs.filter((d) => docCategory(d.frontmatter) === wanted); } if (this.ctx.json) { - this.output.data( - docs.map((d) => ({ - name: d.name, - title: d.frontmatter?.title, - description: d.frontmatter?.description, - category: inferGuidelineCategory(d.name), - path: d.path, - sourceDir: d.sourceDir, - sizeBytes: d.sizeBytes, - approxTokens: d.approxTokens, - shadowed: this.cache!.isShadowed(d), - })), - ); + // Same docmap as `tbd docs list`, filtered to guidelines, with the + // category as a per-entry extension field. + const entries = await this.docMapEntries(docs); + const docByName = new Map(docs.map((d) => [d.name, d])); + const withCategory = entries.map((e) => ({ + ...e, + category: docCategory(docByName.get(e.name)?.frontmatter), + })); + this.output.data(createDocMap(withCategory, { name: 'tbd-docs' })); return; } if (docs.length === 0) { if (category) { console.log(`No guidelines found in category: ${category}`); - console.log('Valid categories: typescript, python, testing, general'); + console.log(`Valid categories: ${DOC_CATEGORIES.join(', ')}`); } else { console.log('No guidelines found.'); console.log('Run `tbd setup --auto` to install built-in guidelines.'); diff --git a/packages/tbd/src/cli/commands/setup.ts b/packages/tbd/src/cli/commands/setup.ts index b03c915c..3dd2cc30 100644 --- a/packages/tbd/src/cli/commands/setup.ts +++ b/packages/tbd/src/cli/commands/setup.ts @@ -7,7 +7,6 @@ * * Options: * - `tbd setup --auto` - Non-interactive setup (for agents/scripts) - * - `tbd setup --interactive` - Interactive setup with prompts (for humans) * - `tbd setup --from-beads` - Migrate from Beads to tbd * * See: tbd-design.md §6.4.2 Claude Code Integration @@ -46,6 +45,9 @@ import { DATA_SYNC_DIR_NAME, DEFAULT_SHORTCUT_PATHS, DEFAULT_GUIDELINES_PATHS, + DEFAULT_TEMPLATE_PATHS, + DEFAULT_REFERENCE_PATHS, + FORK_DIR, TBD_SHORTCUTS_SYSTEM, TBD_SHORTCUTS_STANDARD, TBD_GUIDELINES_DIR, @@ -61,6 +63,9 @@ import { AGENT_INTEGRATION_FORMAT, } from '../../lib/integration-paths.js'; import { initWorktree, isInGitRepo, findGitRoot, checkWorktreeHealth } from '../../file/git.js'; +import { readForkManifest } from '../../file/fork-manifest.js'; +import { computeForkDriftSummary } from '../../file/doc-fork.js'; +import { docsPostureMenuLines } from '../lib/docs-menu.js'; import { DocCache, generateShortcutDirectory } from '../../file/doc-cache.js'; import { withSharedDataSyncLock, writeCommonDirLayout } from '../../file/common-dir-layout.js'; import { withDataSyncContext } from '../lib/data-context.js'; @@ -1238,12 +1243,11 @@ class SetupCodexHandler extends BaseCommand { } // ============================================================================ -// Setup Default Handler (for --auto and --interactive modes) +// Setup Default Handler (for --auto mode) // ============================================================================ interface SetupDefaultOptions { auto?: boolean; - interactive?: boolean; fromBeads?: boolean; prefix?: string; force?: boolean; @@ -1251,11 +1255,10 @@ interface SetupDefaultOptions { } /** - * Default handler for `tbd setup` with --auto or --interactive flags. + * Default handler for `tbd setup --auto`. * * This implements the unified onboarding flow: * - `tbd setup --auto`: Non-interactive setup with smart defaults (for agents) - * - `tbd setup --interactive`: Interactive setup with prompts (for humans) * * Decision tree: * 1. Not in git repo → Error (git init first) @@ -1278,7 +1281,6 @@ class SetupDefaultHandler extends BaseCommand { // Determine mode const isAutoMode = options.auto === true; - // Note: options.interactive will be used when we add interactive prompts // Header console.log(colors.bold('tbd: Git-native issue tracking for AI agents and humans')); @@ -1938,6 +1940,61 @@ class SetupAutoHandler extends BaseCommand { console.log(colors.warn(` ! ${r.name}: ${r.error}`)); } } + + // Docs summary: serving posture and pending upstream updates (read-only). + await this.printDocsSummary(cwd); + } + + /** + * Print the Docs summary after the integration sections: how many docs tbd + * serves from the gitignored cache, or — when docs are forked into the repo — + * the fork count plus any pending upstream updates. Reporting only: setup + * never writes the fork dir (only `tbd docs update` modifies forked docs). + * + * Zero-fork wording is shared with the bare `tbd docs` overview via + * {@link docsPostureMenuLines} so the two menus cannot drift. + */ + private async printDocsSummary(cwd: string): Promise { + // Suppressed like the other setup summary sections. + if (this.ctx.quiet || this.ctx.json) { + return; + } + + let lines: string[]; + try { + const manifest = await readForkManifest(cwd); + const drift = await computeForkDriftSummary(cwd, FORK_DIR, manifest); + + if (drift.forks > 0) { + const updates = + drift.stale > 0 ? ` ${drift.stale} have upstream updates — run 'tbd docs update'.` : ''; + lines = [`Docs: ${drift.forks} forked into ${FORK_DIR}/.${updates}`]; + } else { + let total = 0; + for (const paths of [ + DEFAULT_GUIDELINES_PATHS, + DEFAULT_SHORTCUT_PATHS, + DEFAULT_TEMPLATE_PATHS, + DEFAULT_REFERENCE_PATHS, + ]) { + const cache = new DocCache(paths, cwd); + await cache.load({ quiet: true }); + total += cache.list().length; + } + lines = [ + `Docs: ${total} docs available in the cache (.tbd/docs/, gitignored); none forked into the repo.`, + ...docsPostureMenuLines().map((line) => ` ${line}`), + ]; + } + } catch { + // Reporting only — never fail setup because the summary could not be read. + return; + } + + console.log(''); + for (const line of lines) { + console.log(line); + } } /** @@ -2179,7 +2236,6 @@ class SetupAutoHandler extends BaseCommand { export const setupCommand = new Command('setup') .description('Configure tbd integration with editors and tools') .option('--auto', 'Non-interactive mode with smart defaults (for agents/scripts)') - .option('--interactive', 'Interactive mode with prompts (for humans)') .option('--from-beads', 'Migrate from Beads to tbd') .option('--prefix ', 'Project prefix for issue IDs (required for fresh setup)') .option('--force', 'Allow non-recommended prefix format (not 2-8 alphabetic)') @@ -2189,14 +2245,14 @@ export const setupCommand = new Command('setup') 'Comma-separated agent surfaces to install: portable, agents-md, claude, codex (or "all"). Default: all', ) .action(async (options: SetupDefaultOptions, command) => { - // If --auto or --interactive flag is set, run the default handler - if (options.auto || options.interactive) { + // If --auto flag is set, run the default handler + if (options.auto) { const handler = new SetupDefaultHandler(command); await handler.run(options); return; } - // If --from-beads is set without --auto/--interactive, treat as --auto + // If --from-beads is set without --auto, treat as --auto if (options.fromBeads) { const handler = new SetupDefaultHandler(command); await handler.run({ ...options, auto: true }); @@ -2214,7 +2270,6 @@ export const setupCommand = new Command('setup') console.log( ' --auto Non-interactive mode with smart defaults (for agents/scripts)', ); - console.log(' --interactive Interactive mode with prompts (for humans)'); console.log(' --from-beads Migrate from Beads to tbd (implies --auto)'); console.log(''); console.log('Options:'); @@ -2228,7 +2283,6 @@ export const setupCommand = new Command('setup') console.log('Examples:'); console.log(' tbd setup --auto --prefix=tbd # Full automatic setup with prefix'); console.log(' tbd setup --from-beads # Migrate from Beads (uses beads prefix)'); - console.log(' tbd setup --interactive # Interactive setup with prompts'); console.log(''); console.log('For surgical initialization without integrations, see: tbd init --help'); }); diff --git a/packages/tbd/src/cli/commands/shortcut.ts b/packages/tbd/src/cli/commands/shortcut.ts index d2d7c564..b52fecb0 100644 --- a/packages/tbd/src/cli/commands/shortcut.ts +++ b/packages/tbd/src/cli/commands/shortcut.ts @@ -16,7 +16,7 @@ import { requireInit, CLIError } from '../lib/errors.js'; import { DocCache, SCORE_PREFIX_MATCH } from '../../file/doc-cache.js'; import { addDoc } from '../../file/doc-add.js'; import { readConfig } from '../../file/config.js'; -import { DEFAULT_SHORTCUT_PATHS } from '../../lib/paths.js'; +import { CACHE_SHORTCUT_PATHS, FORK_SHORTCUTS_DIR } from '../../lib/paths.js'; import { truncate } from '../../lib/truncate.js'; import { formatDocSize } from '../../lib/format-utils.js'; import { getTerminalWidth } from '../lib/output.js'; @@ -73,9 +73,16 @@ class ShortcutHandler extends BaseCommand { // Get tbd root (supports running from subdirectories) const tbdRoot = await requireInit(); - // Read config to get lookup paths (fall back to defaults) + // Read config to get lookup paths (fall back to the cache defaults). The + // fork dir is prepended structurally, not via config: a persisted + // lookup_path (setup writes one into every repo) must not be able to turn + // off fork shadowing (tbd-design.md §2.9 invariant 1). const config = await readConfig(tbdRoot); - const lookupPaths = config.docs_cache?.lookup_path ?? DEFAULT_SHORTCUT_PATHS; + const configured = config.docs_cache?.lookup_path ?? CACHE_SHORTCUT_PATHS; + const lookupPaths = [ + FORK_SHORTCUTS_DIR, + ...configured.filter((p) => p !== FORK_SHORTCUTS_DIR), + ]; // Create and load the doc cache with proper base directory const cache = new DocCache(lookupPaths, tbdRoot); diff --git a/packages/tbd/src/cli/commands/status.ts b/packages/tbd/src/cli/commands/status.ts index fe87629f..05b8ed02 100644 --- a/packages/tbd/src/cli/commands/status.ts +++ b/packages/tbd/src/cli/commands/status.ts @@ -27,7 +27,9 @@ import { type IntegrationCheck, } from '../lib/sections.js'; import { readConfig, findTbdRoot } from '../../file/config.js'; -import { resolveSharedTbdPaths } from '../../lib/paths.js'; +import { resolveSharedTbdPaths, FORK_DIR } from '../../lib/paths.js'; +import { readForkManifest } from '../../file/fork-manifest.js'; +import { computeForkDriftSummary, type ForkDriftSummary } from '../../file/doc-fork.js'; import { getClaudePaths, getAgentsMdPath, @@ -72,6 +74,7 @@ interface StatusData { worktree_healthy: boolean | null; worktree_status: WorktreeStatus | null; workspaces: string[]; + docs_drift: ForkDriftSummary | null; // Integrations integrations: { @@ -117,6 +120,7 @@ class StatusHandler extends BaseCommand { worktree_healthy: null, worktree_status: null, workspaces: [], + docs_drift: null, integrations: { portable_skill: false, portable_skill_path: AGENTS_SKILL_DISPLAY, @@ -156,6 +160,12 @@ class StatusHandler extends BaseCommand { if (statusData.initialized && tbdRoot) { // Load config and issue info await this.loadPostInitInfo(tbdRoot, statusData); + try { + const manifest = await readForkManifest(tbdRoot); + statusData.docs_drift = await computeForkDriftSummary(tbdRoot, FORK_DIR, manifest); + } catch { + // Docs awareness is best-effort; never fail status over it. + } } this.output.data(statusData, () => { @@ -329,6 +339,7 @@ class StatusHandler extends BaseCommand { ); // INTEGRATIONS section (shared with doctor) + // (docs drift summary is appended after Worktree; see renderDocsLine) const integrationChecks: IntegrationCheck[] = [ { name: 'Portable Agent Skill', @@ -363,6 +374,18 @@ class StatusHandler extends BaseCommand { renderWorktreeStatus(data.worktree_path, data.worktree_status, colors); } + // Docs line — only when forks exist, so zero-fork output stays + // byte-identical to the pre-f05 status (the orientation golden pins it). + if (data.docs_drift && data.docs_drift.forks > 0) { + const d = data.docs_drift; + const parts: string[] = [`${d.customized} customized`]; + if (d.stale > 0) parts.push(`${d.stale} with upstream updates — run 'tbd docs update'`); + if (d.conflicted > 0) parts.push(`${d.conflicted} conflict pending`); + if (d.missing > 0) parts.push(`${d.missing} missing — see 'tbd docs status'`); + console.log(''); + console.log(`${colors.bold('Docs:')} ${d.forks} forked (${parts.join(', ')})`); + } + // Workspaces (only show if there are any) if (data.workspaces.length > 0) { console.log(''); diff --git a/packages/tbd/src/cli/commands/sync.ts b/packages/tbd/src/cli/commands/sync.ts index b845daa0..8f0f7b4b 100644 --- a/packages/tbd/src/cli/commands/sync.ts +++ b/packages/tbd/src/cli/commands/sync.ts @@ -25,6 +25,11 @@ import { type PushResult, } from '../../file/git.js'; import { DATA_SYNC_DIR } from '../../lib/paths.js'; +import { + printDocSyncResult, + printDocSyncStatus, + printForkDriftNotice, +} from '../lib/docs-sync-output.js'; import { basename, join } from 'node:path'; import { access, readFile } from 'node:fs/promises'; import { writeFile } from 'atomically'; @@ -179,7 +184,7 @@ class SyncHandler extends BaseCommand { if (statusOnly) { // Show status without making changes const result = await syncDocsWithDefaults(this.tbdRoot, { dryRun: true }); - this.showDocStatus(result); + printDocSyncStatus(this.output, result); return result; } @@ -188,41 +193,11 @@ class SyncHandler extends BaseCommand { spinner.stop(); // Report results - this.showDocSyncResult(result); + printDocSyncResult(this.output, result); + await printForkDriftNotice(this.output, this.tbdRoot); return result; } - /** - * Show doc sync status (what would change). - */ - private showDocStatus(result: SyncDocsResult): void { - const colors = this.output.getColors(); - const hasChanges = - result.added.length > 0 || - result.updated.length > 0 || - result.removed.length > 0 || - result.pruned.length > 0; - - if (!hasChanges) { - this.output.success('Docs up to date'); - return; - } - - console.log(colors.bold('Docs:')); - if (result.added.length > 0) { - console.log(` ${colors.success(`+${result.added.length}`)} new doc(s) available`); - } - if (result.updated.length > 0) { - console.log(` ${colors.warn(`~${result.updated.length}`)} doc(s) to update`); - } - if (result.removed.length > 0) { - console.log(` ${colors.error(`-${result.removed.length}`)} doc(s) to remove`); - } - if (result.pruned.length > 0) { - console.log(` ${colors.dim(`${result.pruned.length}`)} stale config entry/entries`); - } - } - /** * Show doc sync result after sync. */ diff --git a/packages/tbd/src/cli/commands/uninstall.ts b/packages/tbd/src/cli/commands/uninstall.ts index d3500f75..1ede1e9f 100644 --- a/packages/tbd/src/cli/commands/uninstall.ts +++ b/packages/tbd/src/cli/commands/uninstall.ts @@ -7,6 +7,8 @@ import { Command } from 'commander'; import { rm, access, readdir, stat } from 'node:fs/promises'; import { execSync } from 'node:child_process'; + +import { gitSafeEnv } from '../../lib/git-env.js'; import { join, relative } from 'node:path'; import { BaseCommand } from '../lib/base-command.js'; @@ -88,6 +90,7 @@ class UninstallHandler extends BaseCommand { let localBranchExists = false; try { execSync(`git rev-parse --verify ${syncBranch}`, { + env: gitSafeEnv(), encoding: 'utf-8', stdio: ['ignore', 'pipe', 'ignore'], }); @@ -104,6 +107,7 @@ class UninstallHandler extends BaseCommand { if (options.removeRemote) { try { execSync(`git rev-parse --verify ${remote}/${syncBranch}`, { + env: gitSafeEnv(), encoding: 'utf-8', stdio: ['ignore', 'pipe', 'ignore'], }); @@ -156,6 +160,7 @@ class UninstallHandler extends BaseCommand { try { // First try to remove the worktree through git execSync(`git worktree remove --force "${worktreePath}"`, { + env: gitSafeEnv(), encoding: 'utf-8', stdio: ['ignore', 'pipe', 'ignore'], }); @@ -174,6 +179,7 @@ class UninstallHandler extends BaseCommand { if (legacyWorktreeExists) { try { execSync(`git worktree remove --force "${legacyWorktreePath}"`, { + env: gitSafeEnv(), encoding: 'utf-8', stdio: ['ignore', 'pipe', 'ignore'], }); @@ -188,6 +194,7 @@ class UninstallHandler extends BaseCommand { if (localBranchExists && !options.keepBranch) { try { execSync(`git branch -D ${syncBranch}`, { + env: gitSafeEnv(), encoding: 'utf-8', stdio: ['ignore', 'pipe', 'ignore'], }); @@ -201,6 +208,7 @@ class UninstallHandler extends BaseCommand { if (remoteBranchExists && options.removeRemote) { try { execSync(`git push ${remote} --delete ${syncBranch}`, { + env: gitSafeEnv(), encoding: 'utf-8', stdio: ['ignore', 'pipe', 'ignore'], }); diff --git a/packages/tbd/src/cli/lib/data-context.ts b/packages/tbd/src/cli/lib/data-context.ts index 1da8b2c0..42c828ae 100644 --- a/packages/tbd/src/cli/lib/data-context.ts +++ b/packages/tbd/src/cli/lib/data-context.ts @@ -28,6 +28,7 @@ import { checkWorktreeHealth, repairWorktree } from '../../file/git.js'; import type { WorktreeHealth, WorktreeStatus } from '../../file/git.js'; import { ensureCommonDirLayout, + isLayoutUpgradeable, readCommonDirLayout, validateCommonDirLayout, withSharedDataSyncLock, @@ -94,13 +95,18 @@ async function probeDataSyncReadiness(tbdRoot: string): Promise { const { config, migrated, fromFormat } = await readConfigWithMigration(tbdRoot); const sharedPaths = await resolveSharedTbdPaths(tbdRoot); const layout = await readCommonDirLayout(sharedPaths.sharedLayoutPath); - if (layout) { + // A layout stamped with an older (compatible) format than the config is the + // normal mid-migration state — it is upgraded under the lock by + // ensureCommonDirLayout, so it must not fail validation here, only mark the + // probe as not ready. + const layoutNeedsUpgrade = layout !== null && isLayoutUpgradeable(layout, config); + if (layout && !layoutNeedsUpgrade) { // Validate eagerly even on the read path so future-format / mismatched // layouts fail closed before any I/O the caller might perform. validateCommonDirLayout(layout, config); } const health = await checkWorktreeHealth(tbdRoot, config.sync.branch); - const ready = !migrated && layout !== null && health.valid; + const ready = !migrated && layout !== null && !layoutNeedsUpgrade && health.valid; return { config, migrated, fromFormat, sharedPaths, layout, health, ready }; } diff --git a/packages/tbd/src/cli/lib/doc-command-handler.ts b/packages/tbd/src/cli/lib/doc-command-handler.ts index 6121201d..f427a233 100644 --- a/packages/tbd/src/cli/lib/doc-command-handler.ts +++ b/packages/tbd/src/cli/lib/doc-command-handler.ts @@ -12,8 +12,12 @@ import { BaseCommand } from './base-command.js'; import { shouldUseInteractiveOutput } from './context.js'; import { GUIDELINES_AGENT_HEADER } from './doc-prompts.js'; import { requireInit } from './errors.js'; -import { DocCache, SCORE_PREFIX_MATCH } from '../../file/doc-cache.js'; +import { DocCache, SCORE_PREFIX_MATCH, type CachedDoc } from '../../file/doc-cache.js'; import { addDoc, type DocType } from '../../file/doc-add.js'; +import { servedEntryFor, loadServeContext, effectiveServePaths } from './doc-serve.js'; +import { createDocMap, type DocMapEntry } from '../../docmap/index.js'; +import { FORK_DIR } from '../../lib/paths.js'; +import { relative } from 'node:path'; import { truncate } from '../../lib/truncate.js'; import { formatDocSize } from '../../lib/format-utils.js'; import { getTerminalWidth, renderMarkdownWithFrontmatter, paginateOutput } from './output.js'; @@ -73,7 +77,13 @@ export abstract class DocCommandHandler extends BaseCommand { */ protected async initCache(): Promise { this.tbdRoot = await requireInit(); - this.cache = new DocCache(this.config.paths, this.tbdRoot); + // local_dirs slot between the fork dir and the cache for every kind. + const { localDirs } = await loadServeContext(this.tbdRoot); + const paths = + localDirs.length > 0 + ? effectiveServePaths(this.config.docType, localDirs) + : this.config.paths; + this.cache = new DocCache(paths, this.tbdRoot); await this.cache.load({ quiet: this.ctx.quiet }); } @@ -86,18 +96,9 @@ export abstract class DocCommandHandler extends BaseCommand { const docs = this.cache.list(includeAll); if (this.ctx.json) { - this.output.data( - docs.map((d) => ({ - name: d.name, - title: d.frontmatter?.title, - description: d.frontmatter?.description, - path: d.path, - sourceDir: d.sourceDir, - sizeBytes: d.sizeBytes, - approxTokens: d.approxTokens, - shadowed: this.cache!.isShadowed(d), - })), - ); + // One data model: the per-kind list emits the same docmap object as + // `tbd docs list`, filtered to this kind (spec Decision 21). + this.output.data(createDocMap(await this.docMapEntries(docs), { name: 'tbd-docs' })); return; } @@ -135,6 +136,57 @@ export abstract class DocCommandHandler extends BaseCommand { } } + /** + * Build docmap entries for this kind's docs through the shared constructor, + * with per-kind extensions (size metrics; shadowed flag when applicable). + */ + protected async docMapEntries(docs: CachedDoc[]): Promise { + const { manifest, files, localDirs } = await loadServeContext(this.tbdRoot); + return docs.map((d) => { + const { entry } = servedEntryFor( + this.tbdRoot, + this.config.docType, + d, + manifest, + files, + localDirs, + ); + return { + ...entry, + sizeBytes: d.sizeBytes, + approxTokens: d.approxTokens, + ...(this.cache!.isShadowed(d) ? { shadowed: true } : {}), + }; + }); + } + + /** + * Emit one doc: docmap entry + content in JSON mode (the one-entry read + * shape, spec Decision 22); in text mode, a forked-copy provenance note on + * stderr (Decision 18) before the content. + */ + protected async emitDoc(doc: CachedDoc, score?: number): Promise { + if (this.ctx.json) { + const [entry] = await this.docMapEntries([doc]); + this.output.data({ + ...entry, + ...(score !== undefined ? { score } : {}), + content: doc.content, + }); + return; + } + const { localDirs } = await loadServeContext(this.tbdRoot); + if (!this.ctx.quiet) { + const rel = relative(this.tbdRoot, doc.path).split('\\').join('/'); + if (doc.sourceDir.startsWith(FORK_DIR)) { + process.stderr.write(`(serving forked copy: ${rel})\n`); + } else if (localDirs.includes(doc.sourceDir)) { + process.stderr.write(`(serving local doc: ${rel})\n`); + } + } + await this.outputDocContent(doc.content); + } + /** * Handle no query: show explanation + help. */ @@ -186,16 +238,7 @@ export abstract class DocCommandHandler extends BaseCommand { // Try exact match first const exactMatch = this.cache.get(query); if (exactMatch) { - if (this.ctx.json) { - this.output.data({ - name: exactMatch.doc.name, - title: exactMatch.doc.frontmatter?.title, - score: exactMatch.score, - content: exactMatch.doc.content, - }); - } else { - await this.outputDocContent(exactMatch.doc.content); - } + await this.emitDoc(exactMatch.doc, exactMatch.score); return; } @@ -222,16 +265,7 @@ export abstract class DocCommandHandler extends BaseCommand { } // Good fuzzy match - output it - if (this.ctx.json) { - this.output.data({ - name: best.doc.name, - title: best.doc.frontmatter?.title, - score: best.score, - content: best.doc.content, - }); - } else { - await this.outputDocContent(best.doc.content); - } + await this.emitDoc(best.doc, best.score); } /** @@ -284,7 +318,7 @@ export abstract class DocCommandHandler extends BaseCommand { } console.log(pc.green(` Added to ${result.destPath}`)); - console.log(pc.green(` Config updated with source: ${result.rawUrl}`)); + console.log(pc.green(` Config updated with source: ${result.source}`)); console.log(''); console.log(`Run \`tbd ${this.config.typeNamePlural} --list\` to verify.`); } diff --git a/packages/tbd/src/cli/lib/doc-serve.ts b/packages/tbd/src/cli/lib/doc-serve.ts new file mode 100644 index 00000000..0a2a3251 --- /dev/null +++ b/packages/tbd/src/cli/lib/doc-serve.ts @@ -0,0 +1,158 @@ +/** + * Shared construction of docmap entries for served docs — the one-model, + * one-renderer contract (spec Decisions 21/22): `tbd docs list`/`show`, the + * bare overview, and the per-kind readers all build entries here, so their + * JSON output cannot drift. + */ + +import { readConfig } from '../../file/config.js'; +import { + type ForkKind, + findFork, + hashContent, + readForkManifest, +} from '../../file/fork-manifest.js'; +import { KIND_DIR } from '../../file/doc-fork.js'; +import { + CACHE_GUIDELINES_PATHS, + CACHE_REFERENCE_PATHS, + CACHE_SHORTCUT_PATHS, + CACHE_TEMPLATE_PATHS, + FORK_DIR, + FORK_GUIDELINES_DIR, + FORK_REFERENCES_DIR, + FORK_SHORTCUTS_DIR, + FORK_TEMPLATES_DIR, + TBD_DOCS_DIR, +} from '../../lib/paths.js'; +import { join, relative, sep } from 'node:path'; +import type { DocMapEntry } from '../../docmap/index.js'; +import { tryParseDocRef } from '../../docref/index.js'; + +const KIND_FORK_DIRS: Record = { + guideline: FORK_GUIDELINES_DIR, + shortcut: FORK_SHORTCUTS_DIR, + template: FORK_TEMPLATES_DIR, + reference: FORK_REFERENCES_DIR, +}; + +const KIND_CACHE_DIRS: Record = { + guideline: CACHE_GUIDELINES_PATHS, + shortcut: CACHE_SHORTCUT_PATHS, + template: CACHE_TEMPLATE_PATHS, + reference: CACHE_REFERENCE_PATHS, +}; + +/** + * Validated repo-relative dirs from docs_cache.local_dirs (./-prefixed local + * docrefs). Invalid entries are skipped — doctor surfaces them. + */ +export function sanitizeLocalDirs(localDirs: string[] | undefined): string[] { + const out: string[] = []; + for (const dir of localDirs ?? []) { + const parsed = tryParseDocRef(dir); + if (parsed?.kind === 'local' && dir.startsWith('./')) { + out.push(parsed.path.replace(/^\.\//, '').replace(/\/+$/, '')); + } + } + return out; +} + +/** + * Effective lookup order for serving one kind: fork dir → local_dirs → cache + * (the resolution precedence the design fixes). local_dirs entries serve for + * every kind so reads find them by name; inventory surfaces dedupe so each + * local doc appears once. + */ +export function effectiveServePaths(kind: ForkKind, localDirs: string[]): string[] { + // localDirs arrive already sanitized (loadServeContext) — repo-relative, + // not raw docrefs — so they must not pass through sanitizeLocalDirs again. + return [KIND_FORK_DIRS[kind], ...localDirs, ...KIND_CACHE_DIRS[kind]]; +} + +/** A served doc's docmap entry plus its derived fork-state presentation. */ +export interface ServedEntryInfo { + entry: DocMapEntry; + state: string; + marker: string; +} + +/** Minimal doc fields needed to derive a docmap entry. */ +export interface ServedDocLike { + name: string; + content: string; + sourceDir: string; + path: string; + frontmatter?: { title?: string; description?: string }; +} + +/** + * Build the docmap entry (+ state marker) for one served doc. Single point of + * construction for every docs inventory and read surface (Decision 21/22). + * Every entry carries a location: forked docs have path+source, local files a + * path, upstream docs their provenance docref. + */ +export function servedEntryFor( + tbdRoot: string, + kind: ForkKind, + doc: ServedDocLike, + manifest: Awaited>, + files: Record | undefined, + localDirs: string[] = [], +): ServedEntryInfo { + const fork = findFork(manifest, doc.name, kind); + const isLocalDir = !fork && localDirs.some((d) => doc.sourceDir === d); + const isLocal = !fork && (doc.sourceDir.startsWith(FORK_DIR) || isLocalDir); + let state = 'upstream'; + let marker = ''; + if (fork) { + const customized = hashContent(doc.content) !== fork.base_hash; + state = customized ? 'customized' : 'forked'; + marker = customized ? '[forked, customized]' : '[forked]'; + } else if (isLocal) { + state = 'local'; + marker = '[local]'; + } + const localPath = isLocalDir + ? `${doc.sourceDir}/${doc.name}.md` + : `${FORK_DIR}/${KIND_DIR[kind]}/${doc.name}.md`; + const entry: DocMapEntry = { + name: doc.name, + type: kind, + ...(fork + ? { path: fork.path, source: fork.source } + : isLocal + ? { path: localPath } + : { source: sourceDocRefFor(tbdRoot, files, doc.path) }), + title: doc.frontmatter?.title, + description: doc.frontmatter?.description, + state, + }; + return { entry, state, marker }; +} + +/** Load the shared context servedEntryFor needs (manifest + config maps). */ +export async function loadServeContext(tbdRoot: string): Promise<{ + manifest: Awaited>; + files: Record | undefined; + localDirs: string[]; +}> { + const manifest = await readForkManifest(tbdRoot); + const config = await readConfig(tbdRoot); + return { + manifest, + files: config.docs_cache?.files, + localDirs: sanitizeLocalDirs(config.docs_cache?.local_dirs), + }; +} + +/** Derive the provenance docref for a cached doc from config, defaulting to internal:. */ +function sourceDocRefFor( + tbdRoot: string, + files: Record | undefined, + docPath: string, +): string { + const cacheRoot = join(tbdRoot, TBD_DOCS_DIR); + const rel = relative(cacheRoot, docPath).split(sep).join('/'); + return files?.[rel] ?? `internal:${rel}`; +} diff --git a/packages/tbd/src/cli/lib/docs-menu.ts b/packages/tbd/src/cli/lib/docs-menu.ts new file mode 100644 index 00000000..7d46d3dc --- /dev/null +++ b/packages/tbd/src/cli/lib/docs-menu.ts @@ -0,0 +1,30 @@ +/** + * Shared docs-posture menu for the zero-forks state. + * + * The same "three postures" menu is shown by `tbd setup --auto` (the Docs + * summary) and by the bare `tbd docs` overview, so the wording lives in one + * place and the two surfaces cannot drift. Lines are returned unindented and + * uncolored; callers add their own indentation and formatting. + * + * Contract note (forkable-docs spec): the menu must only name selectors that + * exist — `--category` appears here because the flag ships with it. + */ + +import { FORK_DIR } from '../../lib/paths.js'; +import { DOC_CATEGORIES } from '../../lib/doc-categories.js'; + +/** + * Menu body lines for the zero-forks state: the three serving postures plus + * the browse/read pointer. Shown under a lead line reporting the count of + * docs available in the cache. + */ +export function docsPostureMenuLines(): string[] { + return [ + 'Guidelines are active from the cache. Three postures, all serving the same docs:', + 'Hidden (default): keep the cache as-is — zero repo footprint', + `Curated: tbd docs fork [...] fork chosen docs into ${FORK_DIR}/`, + ` tbd docs fork --category= (${DOC_CATEGORIES.join(', ')})`, + 'Everything: tbd docs fork --all all docs, visible and editable', + 'Browse / read: tbd docs list / tbd docs show ', + ]; +} diff --git a/packages/tbd/src/cli/lib/docs-sync-output.ts b/packages/tbd/src/cli/lib/docs-sync-output.ts new file mode 100644 index 00000000..77d31b0f --- /dev/null +++ b/packages/tbd/src/cli/lib/docs-sync-output.ts @@ -0,0 +1,107 @@ +/** + * Shared rendering for docs-cache sync results and the forked-docs drift notice. + * + * Used by both `tbd docs sync` (the canonical command) and `tbd sync --docs` + * (kept as a deprecated alias), so the two surfaces cannot drift apart. + */ + +import type { OutputManager } from './output.js'; +import type { SyncDocsResult } from '../../file/doc-sync.js'; +import { FORK_DIR } from '../../lib/paths.js'; +import { readForkManifest } from '../../file/fork-manifest.js'; +import { computeForkDriftSummary } from '../../file/doc-fork.js'; + +/** Print the result of a docs-cache sync (writes applied). */ +export function printDocSyncResult(output: OutputManager, result: SyncDocsResult): void { + const hasChanges = + result.added.length > 0 || + result.updated.length > 0 || + result.removed.length > 0 || + result.pruned.length > 0; + + if (!hasChanges) { + output.success('Docs up to date'); + return; + } + + const parts: string[] = []; + if (result.added.length > 0) { + parts.push(`+${result.added.length}`); + } + if (result.updated.length > 0) { + parts.push(`~${result.updated.length}`); + } + if (result.removed.length > 0) { + parts.push(`-${result.removed.length}`); + } + + if (parts.length > 0) { + output.success(`Synced docs: ${parts.join(' ')} doc(s)`); + } + + if (result.pruned.length > 0) { + output.info(`Removed ${result.pruned.length} stale config entry/entries`); + } + + for (const err of result.errors) { + output.warn(`Doc sync error: ${err.path}: ${err.error}`); + } +} + +/** Print what a docs-cache sync would change (dry-run / status view). */ +export function printDocSyncStatus(output: OutputManager, result: SyncDocsResult): void { + const colors = output.getColors(); + const hasChanges = + result.added.length > 0 || + result.updated.length > 0 || + result.removed.length > 0 || + result.pruned.length > 0; + + if (!hasChanges) { + output.success('Docs up to date'); + return; + } + + console.log(colors.bold('Docs:')); + if (result.added.length > 0) { + console.log(` ${colors.success(`+${result.added.length}`)} new doc(s) available`); + } + if (result.updated.length > 0) { + console.log(` ${colors.warn(`~${result.updated.length}`)} doc(s) to update`); + } + if (result.removed.length > 0) { + console.log(` ${colors.error(`-${result.removed.length}`)} doc(s) to remove`); + } + if (result.pruned.length > 0) { + console.log(` ${colors.dim(`${result.pruned.length}`)} stale config entry/entries`); + } +} + +/** + * One-line awareness notice for forked docs: a cache refresh is exactly when + * forks become stale, so drift is surfaced here — but never acted on (only the + * explicit `tbd docs update` mutates tracked files). Best-effort: never fails + * the surrounding sync. + */ +export async function printForkDriftNotice(output: OutputManager, tbdRoot: string): Promise { + try { + const manifest = await readForkManifest(tbdRoot); + const drift = await computeForkDriftSummary(tbdRoot, FORK_DIR, manifest); + if (drift.forks === 0) return; + const parts: string[] = []; + if (drift.stale > 0) { + parts.push(`${drift.stale} forked doc(s) have upstream updates — run 'tbd docs update'`); + } + if (drift.conflicted > 0) { + parts.push(`${drift.conflicted} with unresolved conflict markers`); + } + if (drift.missing > 0) { + parts.push(`${drift.missing} missing (deleted/renamed) — see 'tbd docs status'`); + } + if (parts.length > 0) { + output.notice(`Docs: ${parts.join('; ')}`); + } + } catch { + // Drift awareness is best-effort; never fail a sync over it. + } +} diff --git a/packages/tbd/src/cli/lib/output.ts b/packages/tbd/src/cli/lib/output.ts index f4770ddf..0d7977d0 100644 --- a/packages/tbd/src/cli/lib/output.ts +++ b/packages/tbd/src/cli/lib/output.ts @@ -133,7 +133,7 @@ export function createHelpEpilog(colorOption: ColorOption = 'auto'): string { '', ' This initializes tbd and configures your coding agents automatically.', ` To refresh setup (idempotent, safe anytime): ${colors.green('`tbd setup --auto`')}`, - ` For interactive setup: ${colors.dim('`tbd setup --interactive`')}`, + ` For managed docs: ${colors.dim('`tbd docs`')}`, '', colors.blue('For more on tbd, see: https://github.com/jlevy/tbd'), ]; diff --git a/packages/tbd/src/docmap/docmap.ts b/packages/tbd/src/docmap/docmap.ts new file mode 100644 index 00000000..a0017bd0 --- /dev/null +++ b/packages/tbd/src/docmap/docmap.ts @@ -0,0 +1,149 @@ +/** + * docmap — a minimal, machine-readable inventory of a collection of documents. + * + * A docmap is a "sitemap for docs": one entry per document, each with an identity + * (`type` + `name`, unique within the map), a location (`path` and/or a provenance + * `source` docref — at least one is required), and presentation metadata (`title`, + * `description`). It describes a collection; it says nothing about how the + * collection is assembled, fetched, or kept fresh — a docmap is a generated VIEW + * of a collection, never an input to resolution. + * + * Path convention: for a docmap committed as a file, `path` is relative to the + * docmap file's own directory (the sitemap convention); generated docmaps state + * their collection root out of band. + * + * This is the docmap/0.1 format. The module is standalone and dependency-free (no + * tbd-internal imports) so it can move to its own package later. Consumers MUST + * ignore unknown fields, so producers (such as tbd) may attach extension fields — + * for example tbd's `state`/`stale`, or size metrics like `word_count` / + * `size_bytes` — without breaking other readers; core fields stay minimal. + */ + +import { z } from 'zod'; + +/** Current docmap format version tag. */ +export const DOCMAP_VERSION = 'docmap/0.1' as const; + +/** + * One document in a docmap. Unknown fields are preserved (extension fields). + * Every entry must carry a location: `path` and/or `source` — an inventory whose + * entries cannot be located is not an inventory. + */ +export const DocMapEntrySchema = z + .object({ + /** Identity, unique within the map together with `type`. */ + name: z.string().min(1), + /** Identity, e.g. "guideline" | "shortcut" | "template" | "reference". */ + type: z.string().min(1), + /** Location within the collection (relative to the docmap's own location). */ + path: z.string().optional(), + /** Provenance: a docref string for where the doc came from. */ + source: z.string().optional(), + title: z.string().optional(), + description: z.string().optional(), + }) + .passthrough() + .refine((entry) => entry.path !== undefined || entry.source !== undefined, { + message: 'docmap entry must have a location: path and/or source', + }); + +export type DocMapEntry = z.infer; + +/** + * A document inventory. `documents` entries are unique by (`type`, `name`). + */ +export const DocMapSchema = z + .object({ + docmap: z.string(), + name: z.string().optional(), + documents: z.array(DocMapEntrySchema), + }) + .passthrough(); + +export type DocMap = z.infer; + +/** Error thrown when an object is not a valid docmap. */ +export class DocMapError extends Error { + constructor(detail: string) { + super(`Invalid docmap: ${detail}`); + this.name = 'DocMapError'; + } +} + +/** Stable key for an entry's identity. */ +export function entryKey(entry: Pick): string { + return `${entry.type}:${entry.name}`; +} + +/** Assert that entries are unique by (type, name); throws {@link DocMapError}. */ +function assertUniqueIdentities(documents: DocMapEntry[]): void { + const seen = new Set(); + for (const entry of documents) { + const key = entryKey(entry); + if (seen.has(key)) { + throw new DocMapError(`duplicate entry identity ${JSON.stringify(key)}`); + } + seen.add(key); + } +} + +/** + * Build a docmap from entries. Validates each entry and identity uniqueness. + */ +export function createDocMap(documents: DocMapEntry[], options: { name?: string } = {}): DocMap { + const parsed = documents.map((d) => DocMapEntrySchema.parse(d)); + assertUniqueIdentities(parsed); + const map: DocMap = { docmap: DOCMAP_VERSION, documents: parsed }; + if (options.name !== undefined) { + map.name = options.name; + } + return map; +} + +/** + * Parse and validate an unknown value as a docmap (e.g. from YAML/JSON). + * Verifies the version tag and identity uniqueness. + */ +export function parseDocMap(value: unknown): DocMap { + const result = DocMapSchema.safeParse(value); + if (!result.success) { + throw new DocMapError(result.error.issues.map((i) => i.message).join('; ')); + } + const map = result.data; + // Readers accept docmap/0.* only: a different major may change field semantics, + // so failing fast beats misreading. + if (!map.docmap.startsWith('docmap/0.')) { + throw new DocMapError( + `unsupported docmap version ${JSON.stringify(map.docmap)} (this reader supports docmap/0.*)`, + ); + } + assertUniqueIdentities(map.documents); + return map; +} + +/** Find an entry by name, optionally constrained to a type. */ +export function findEntry(map: DocMap, name: string, type?: string): DocMapEntry | undefined { + return map.documents.find((d) => d.name === name && (type === undefined || d.type === type)); +} + +/** Group entries by their `type`, preserving order within each group. */ +export function groupByType(map: DocMap): Map { + const groups = new Map(); + for (const entry of map.documents) { + const list = groups.get(entry.type); + if (list) { + list.push(entry); + } else { + groups.set(entry.type, [entry]); + } + } + return groups; +} + +/** Return a new docmap containing only entries of the given type. */ +export function filterByType(map: DocMap, type: string): DocMap { + const documents = map.documents.filter((d) => d.type === type); + return map.name !== undefined + ? { docmap: map.docmap, name: map.name, documents } + : { docmap: map.docmap, documents }; +} diff --git a/packages/tbd/src/docmap/index.ts b/packages/tbd/src/docmap/index.ts new file mode 100644 index 00000000..f0643f77 --- /dev/null +++ b/packages/tbd/src/docmap/index.ts @@ -0,0 +1,22 @@ +/** + * Public API for the standalone docmap module (docmap/0.1). + * + * A minimal document-inventory format with no tbd-internal dependencies, structured + * for extraction into its own package. tbd's list/inventory commands build a docmap + * and render it to text or `--json`. + */ + +export { + type DocMap, + type DocMapEntry, + DOCMAP_VERSION, + DocMapSchema, + DocMapEntrySchema, + DocMapError, + entryKey, + createDocMap, + parseDocMap, + findEntry, + groupByType, + filterByType, +} from './docmap.js'; diff --git a/packages/tbd/src/docref/docref.ts b/packages/tbd/src/docref/docref.ts new file mode 100644 index 00000000..4732d497 --- /dev/null +++ b/packages/tbd/src/docref/docref.ts @@ -0,0 +1,280 @@ +/** + * docref — a single-string, URI-like address for any document. + * + * This module is intentionally standalone and dependency-free (no tbd-internal + * imports) so it can move to its own package later. It is the one address syntax + * used everywhere a doc's source or location is named: config source strings, the + * fork manifest's `source` field, `tbd docs add` arguments, and `local_dirs` entries. + * + * Supported forms (docref v0.1): + * internal:guidelines/python-rules.md bundled doc shipped inside the consuming + * tool (app-relative, not tbd-specific) + * ./docs/general/ ../shared/ /abs/f.md local paths — must be anchored with + * "./", "../", "/", or a Windows drive + * letter (C:/ or C:\) + * https://example.com/style.md plain URL + * github:owner/repo@ref//path/to/file.md git-hosted (also gitlab:) + * github:owner/repo@ref//file.md#section optional fragment, preserved + * + * The grammar is deliberately strict: bare relative strings ("guidelines/x.md") and + * home-relative paths ("~/x.md") are NOT valid docrefs. Consumers that want lenient + * input may coerce at their own boundary (e.g. prepend "./") before parsing — a + * strict grammar plus lenient consumers composes; the reverse cannot be tightened. + * + * Additional protocols (for example a host-bearing git scheme for forges beyond + * GitHub/GitLab) may be added in future versions. + * + * Web URLs that point at a known git host are normalized to the `github:`/`gitlab:` + * form so there is one canonical address for a given file: + * https://github.com/o/r/blob/main/f.md -> github:o/r@main//f.md + * https://raw.githubusercontent.com/o/r/main/f.md -> github:o/r@main//f.md + */ + +/** Scheme of a git-hosted docref. */ +export type GitHost = 'github' | 'gitlab'; + +/** A parsed document reference. */ +export type DocRef = + | { readonly kind: 'internal'; readonly path: string } + | { readonly kind: 'local'; readonly path: string } + | { readonly kind: 'url'; readonly url: string } + | { + readonly kind: 'git'; + readonly host: GitHost; + readonly owner: string; + readonly repo: string; + readonly ref?: string; + readonly path: string; + /** Optional in-document anchor (e.g. a heading slug), preserved verbatim. */ + readonly fragment?: string; + }; + +/** Error thrown when a string is not a valid docref. */ +export class DocRefError extends Error { + constructor( + public readonly input: string, + detail: string, + ) { + super(`Invalid docref ${JSON.stringify(input)}: ${detail}`); + this.name = 'DocRefError'; + } +} + +const GIT_SCHEMES: readonly GitHost[] = ['github', 'gitlab']; + +/** + * True for strings that address a local filesystem path: anchored relative + * (`./`, `../`), absolute (`/`), or a Windows drive-letter path (`C:/`, `C:\`). + */ +function looksLocal(input: string): boolean { + return ( + input.startsWith('./') || + input.startsWith('../') || + input.startsWith('/') || + /^[A-Za-z]:[\\/]/.test(input) + ); +} + +/** Strip a single leading `./` for tidy comparison; other forms are left as-is. */ +function tidyLocal(path: string): string { + return path.startsWith('./') ? path.slice(2) : path; +} + +/** + * Parse a `host:owner/repo[@ref]//path[#fragment]` body (everything after the scheme). + */ +function parseGitBody(host: GitHost, body: string, input: string): DocRef { + const sep = body.indexOf('//'); + if (sep === -1) { + throw new DocRefError(input, `git docref must contain "//" separating repo from path`); + } + const repoPart = body.slice(0, sep); + const pathPart = body.slice(sep + 2); + + const hashIndex = pathPart.indexOf('#'); + const path = hashIndex === -1 ? pathPart : pathPart.slice(0, hashIndex); + const fragment = hashIndex === -1 ? undefined : pathPart.slice(hashIndex + 1) || undefined; + if (!path) { + throw new DocRefError(input, 'git docref has an empty path'); + } + + const atIndex = repoPart.indexOf('@'); + const ownerRepo = atIndex === -1 ? repoPart : repoPart.slice(0, atIndex); + const ref = atIndex === -1 ? undefined : repoPart.slice(atIndex + 1) || undefined; + + const slash = ownerRepo.indexOf('/'); + if (slash === -1) { + throw new DocRefError(input, 'git docref must be "owner/repo"'); + } + const owner = ownerRepo.slice(0, slash); + const repo = ownerRepo.slice(slash + 1); + if (!owner || !repo) { + throw new DocRefError(input, 'git docref must be "owner/repo"'); + } + + return { + kind: 'git', + host, + owner, + repo, + path, + ...(ref !== undefined ? { ref } : {}), + ...(fragment !== undefined ? { fragment } : {}), + }; +} + +/** + * If `url` points at a known git host's file view, return the equivalent git + * docref; otherwise return null (caller keeps it as a plain URL). + * URL fragments are preserved — normalization must never silently drop data. + */ +function gitRefFromUrl(url: string): DocRef | null { + let parsed: URL; + try { + parsed = new URL(url); + } catch { + return null; + } + const segments = parsed.pathname.split('/').filter(Boolean); + const fragment = parsed.hash ? parsed.hash.slice(1) || undefined : undefined; + const frag = fragment !== undefined ? { fragment } : {}; + + // https://github.com/{owner}/{repo}/blob/{ref}/{path...} + if (parsed.hostname === 'github.com' && segments[2] === 'blob' && segments.length >= 5) { + const [owner, repo, , ref, ...rest] = segments; + return { + kind: 'git', + host: 'github', + owner: owner!, + repo: repo!, + ref, + path: rest.join('/'), + ...frag, + }; + } + // https://raw.githubusercontent.com/{owner}/{repo}/{ref}/{path...} + if (parsed.hostname === 'raw.githubusercontent.com' && segments.length >= 4) { + const [owner, repo, ref, ...rest] = segments; + return { + kind: 'git', + host: 'github', + owner: owner!, + repo: repo!, + ref, + path: rest.join('/'), + ...frag, + }; + } + // https://gitlab.com/{owner}/{repo}/-/blob/{ref}/{path...} + if (parsed.hostname === 'gitlab.com' && segments[2] === '-' && segments[3] === 'blob') { + const [owner, repo, , , ref, ...rest] = segments; + if (owner && repo && ref && rest.length > 0) { + return { kind: 'git', host: 'gitlab', owner, repo, ref, path: rest.join('/'), ...frag }; + } + } + return null; +} + +/** + * Parse a docref string into a structured {@link DocRef}. + * Throws {@link DocRefError} if the string is not a valid docref. + */ +export function parseDocRef(input: string): DocRef { + const raw = input.trim(); + if (!raw) { + throw new DocRefError(input, 'empty'); + } + + // Internal bundled docs. + if (raw.startsWith('internal:')) { + const path = raw.slice('internal:'.length); + if (!path) { + throw new DocRefError(input, 'internal docref has an empty path'); + } + return { kind: 'internal', path }; + } + + // Git-hosted schemes. + for (const host of GIT_SCHEMES) { + const prefix = `${host}:`; + if (raw.startsWith(prefix)) { + return parseGitBody(host, raw.slice(prefix.length), input); + } + } + + // Web URLs — normalize known git hosts, otherwise keep as a plain URL. + if (raw.startsWith('http://') || raw.startsWith('https://')) { + return gitRefFromUrl(raw) ?? { kind: 'url', url: raw }; + } + + // Local filesystem paths (anchored; includes Windows drive letters). + if (looksLocal(raw)) { + return { kind: 'local', path: raw }; + } + + if (raw.startsWith('~')) { + throw new DocRefError( + input, + 'home-relative (~) paths are not supported; use an absolute or ./-relative path', + ); + } + if (/^[a-z][a-z0-9+.-]*:/i.test(raw)) { + throw new DocRefError(input, 'unknown scheme'); + } + throw new DocRefError( + input, + 'local paths must start with "./", "../", or "/" (bare relative paths are not valid docrefs)', + ); +} + +/** Parse a docref, returning null instead of throwing on invalid input. */ +export function tryParseDocRef(input: string): DocRef | null { + try { + return parseDocRef(input); + } catch { + return null; + } +} + +/** Serialize a {@link DocRef} back to its canonical string form. */ +export function formatDocRef(ref: DocRef): string { + switch (ref.kind) { + case 'internal': + return `internal:${ref.path}`; + case 'local': + return ref.path; + case 'url': + return ref.url; + case 'git': { + const refPart = ref.ref ? `@${ref.ref}` : ''; + const fragPart = ref.fragment ? `#${ref.fragment}` : ''; + return `${ref.host}:${ref.owner}/${ref.repo}${refPart}//${ref.path}${fragPart}`; + } + } +} + +/** + * Normalize a docref string to its canonical form (parse + re-format). + * Rationalizes git web URLs into the `github:`/`gitlab:` scheme. + */ +export function normalizeDocRef(input: string): string { + return formatDocRef(parseDocRef(input)); +} + +/** True if `input` parses as a valid docref. */ +export function isDocRef(input: string): boolean { + return tryParseDocRef(input) !== null; +} + +/** + * Whether two docrefs address the same document, ignoring a leading `./` on + * local paths. Comparison is purely syntactic — no case normalization of hosts + * or owners. Useful for de-duping config entries. + */ +export function docRefsEqual(a: DocRef, b: DocRef): boolean { + if (a.kind !== b.kind) return false; + if (a.kind === 'local' && b.kind === 'local') { + return tidyLocal(a.path) === tidyLocal(b.path); + } + return formatDocRef(a) === formatDocRef(b); +} diff --git a/packages/tbd/src/docref/index.ts b/packages/tbd/src/docref/index.ts new file mode 100644 index 00000000..755e50cc --- /dev/null +++ b/packages/tbd/src/docref/index.ts @@ -0,0 +1,18 @@ +/** + * Public API for the standalone docref module. + * + * docref is the single address grammar for documents across tbd. It has no + * tbd-internal dependencies and is structured for extraction into its own package. + */ + +export { + type DocRef, + type GitHost, + DocRefError, + parseDocRef, + tryParseDocRef, + formatDocRef, + normalizeDocRef, + isDocRef, + docRefsEqual, +} from './docref.js'; diff --git a/packages/tbd/src/file/common-dir-layout.ts b/packages/tbd/src/file/common-dir-layout.ts index e30092a9..aa14e795 100644 --- a/packages/tbd/src/file/common-dir-layout.ts +++ b/packages/tbd/src/file/common-dir-layout.ts @@ -14,7 +14,14 @@ import { resolveSharedTbdPaths, type SharedTbdPaths, } from '../lib/paths.js'; -import { CURRENT_FORMAT, formatUpgradeMessage, isCompatibleFormat } from '../lib/tbd-format.js'; +import { + CURRENT_FORMAT, + formatUpgradeMessage, + isCompatibleFormat, + isFormatCompatibleWithSupported, + FORMAT_HISTORY, + type FormatVersion, +} from '../lib/tbd-format.js'; import { sortKeys, stringifyYaml } from '../utils/yaml-utils.js'; import { now } from '../utils/time-utils.js'; import { DATA_SYNC_LOCK_OPTIONS, withLockfile } from '../utils/lockfile.js'; @@ -48,6 +55,24 @@ export async function readCommonDirLayout(layoutPath: string): Promise { const existing = await readCommonDirLayout(paths.sharedLayoutPath); if (existing) { + // An older-format layout next to a newer-format config is the normal + // mid-migration state: re-stamp it from the config (preserving created_at) + // instead of failing validation. See isLayoutUpgradeable. + if (isLayoutUpgradeable(existing, config)) { + return writeCommonDirLayout(paths, config, existing); + } validateCommonDirLayout(existing, config); return existing; } diff --git a/packages/tbd/src/file/doc-add.ts b/packages/tbd/src/file/doc-add.ts index 792fc32a..a23b2ffe 100644 --- a/packages/tbd/src/file/doc-add.ts +++ b/packages/tbd/src/file/doc-add.ts @@ -10,9 +10,12 @@ import { join, dirname } from 'node:path'; import { mkdir } from 'node:fs/promises'; import { writeFile } from 'atomically'; +import { readFile } from 'node:fs/promises'; + import { readConfig, writeConfig } from './config.js'; -import { githubBlobToRawUrl, fetchWithGhFallback } from './github-fetch.js'; +import { fetchWithGhFallback, gitDocRefToRawUrl } from './github-fetch.js'; import { TBD_DOCS_DIR } from '../lib/paths.js'; +import { normalizeDocRef, parseDocRef, DocRefError } from '../docref/index.js'; // ============================================================================= // Types @@ -21,13 +24,13 @@ import { TBD_DOCS_DIR } from '../lib/paths.js'; /** * The type of document being added. */ -export type DocType = 'guideline' | 'shortcut' | 'template'; +export type DocType = 'guideline' | 'shortcut' | 'template' | 'reference'; /** * Options for adding a document. */ export interface AddDocOptions { - /** URL to fetch the document from */ + /** Source docref (or URL — normalized to its canonical docref) to add */ url: string; /** Name for the document (without .md extension) */ name: string; @@ -41,7 +44,9 @@ export interface AddDocOptions { export interface AddDocResult { /** The destination path relative to .tbd/docs/ */ destPath: string; - /** The raw URL used to fetch the content */ + /** The canonical docref recorded in config (docref-everywhere rule) */ + source: string; + /** The raw URL used to fetch the content (or the local path read) */ rawUrl: string; /** Whether gh CLI was used as fallback */ usedGhCli: boolean; @@ -97,6 +102,8 @@ export function getDocTypeSubdir(docType: DocType): string { return 'shortcuts/custom'; case 'template': return 'templates'; + case 'reference': + return 'references'; } } @@ -124,10 +131,33 @@ export async function addDoc(tbdRoot: string, options: AddDocOptions): Promise//${parsed.path}`, + ); + } + + let content: string; + let usedGhCli = false; + let rawUrl = source; + if (parsed.kind === 'local') { + rawUrl = parsed.path; + content = await readFile(join(tbdRoot, parsed.path), 'utf-8'); + } else { + rawUrl = parsed.kind === 'git' ? gitDocRefToRawUrl(parsed) : parsed.url; + const fetched = await fetchWithGhFallback(rawUrl); + content = fetched.content; + usedGhCli = fetched.usedGhCli; + } // Validate content validateDocContent(content, cleanName); @@ -141,7 +171,7 @@ export async function addDoc(tbdRoot: string, options: AddDocOptions): Promise//.md`. Provenance and the merge base live in the + * committed manifest under `.tbd/doc-forks/` (see fork-manifest.ts) so the forked + * files themselves stay clean, diffable, and forkable. + * + * These operations take the upstream content and current manifest as inputs and do + * the filesystem writes; resolving which doc/source to fork is the caller's job. + */ + +import { readFile, readdir, rm, rmdir, mkdir, stat } from 'node:fs/promises'; +import { dirname, join, relative } from 'node:path'; + +import { writeFile } from 'atomically'; +import matter from 'gray-matter'; + +import { DocCache } from './doc-cache.js'; +import { + CACHE_GUIDELINES_PATHS, + CACHE_SHORTCUT_PATHS, + CACHE_TEMPLATE_PATHS, +} from '../lib/paths.js'; + +import { + type ForkEntry, + type ForkKind, + type ForkManifest, + type ForkStatus, + compareVersionsLoose, + computeForkStatus, + findFork, + hashContent, + hasUnresolvedConflict, + readBaseContent, + removeBaseContent, + upsertFork, + removeFork, + writeBaseContent, +} from './fork-manifest.js'; + +/** Map a doc kind to its plural directory name within the fork dir. */ +export const KIND_DIR: Record = { + guideline: 'guidelines', + shortcut: 'shortcuts', + template: 'templates', + reference: 'references', +}; + +/** Absolute path of a forked file. */ +export function forkFilePath( + tbdRoot: string, + forkDir: string, + kind: ForkKind, + name: string, +): string { + return join(tbdRoot, forkDir, KIND_DIR[kind], `${name}.md`); +} + +/** Repo-relative path recorded in the manifest (always POSIX-style forward slashes). */ +export function forkRelPath(forkDir: string, kind: ForkKind, name: string): string { + return `${forkDir}/${KIND_DIR[kind]}/${name}.md`; +} + +async function pathExists(path: string): Promise { + try { + await stat(path); + return true; + } catch (err) { + if ((err as NodeJS.ErrnoException).code === 'ENOENT') { + return false; + } + throw err; + } +} + +/** Error raised when a fork/unfork would lose user content; carries a reason code. */ +export class ForkConflictError extends Error { + constructor( + public readonly code: 'overwrite' | 'customized' | 'not-forked' | 'version-skew', + message: string, + ) { + super(message); + this.name = 'ForkConflictError'; + } +} + +export interface ForkDocParams { + tbdRoot: string; + forkDir: string; + manifest: ForkManifest; + kind: ForkKind; + name: string; + /** Provenance docref (e.g. "internal:guidelines/python-rules.md"). */ + source: string; + /** Upstream content to fork (becomes both the file and the base snapshot). */ + content: string; + tbdVersion?: string; + force?: boolean; +} + +export interface ForkDocResult { + manifest: ForkManifest; + relPath: string; + action: 'created' | 'refreshed'; +} + +/** + * Fork a doc into the fork dir, recording its base snapshot and manifest entry. + * + * Refuses to overwrite a target that exists and is not an unmodified fork (e.g. a + * pre-existing user file or a customized fork), unless `force` is set. Re-forking an + * unmodified fork refreshes it to the supplied upstream content and advances the base. + */ +export async function forkDoc(params: ForkDocParams): Promise { + const { tbdRoot, forkDir, kind, name, source, content, force } = params; + const absPath = forkFilePath(tbdRoot, forkDir, kind, name); + const relPath = forkRelPath(forkDir, kind, name); + const existingEntry = findFork(params.manifest, name, kind); + + let action: ForkDocResult['action'] = 'created'; + + if (await pathExists(absPath)) { + const current = await readFile(absPath, 'utf-8'); + const isUnmodifiedFork = hashContent(current) === existingEntry?.base_hash; + if (isUnmodifiedFork) { + // Refreshing an unmodified fork advances its base to this client's cache. + // Refuse if the fork point was set by a NEWER tbd: refreshing would + // downgrade the doc to our older bundled content (the same hazard the + // update guard prevents). --force overrides. + if ( + !force && + existingEntry?.tbd_version !== undefined && + params.tbdVersion !== undefined && + compareVersionsLoose(params.tbdVersion, existingEntry.tbd_version) === -1 + ) { + throw new ForkConflictError( + 'version-skew', + `${name}: fork point was set by tbd ${existingEntry.tbd_version} (you have ` + + `${params.tbdVersion}) — upgrade tbd before re-forking, or use --force to downgrade`, + ); + } + action = 'refreshed'; + } else if (!force) { + throw new ForkConflictError( + 'overwrite', + `${relPath} already exists and is not an unmodified fork`, + ); + } + } + + await mkdir(dirname(absPath), { recursive: true }); + await writeFile(absPath, content); + await writeBaseContent(tbdRoot, kind, name, content); + + const entry: ForkEntry = { + name, + kind, + path: relPath, + source, + base_hash: hashContent(content), + ...(params.tbdVersion ? { tbd_version: params.tbdVersion } : {}), + }; + + return { manifest: upsertFork(params.manifest, entry), relPath, action }; +} + +export interface UnforkDocParams { + tbdRoot: string; + forkDir: string; + manifest: ForkManifest; + name: string; + kind?: ForkKind; + force?: boolean; +} + +export interface UnforkDocResult { + manifest: ForkManifest; + relPath: string; + /** True when the forked file was deleted (false when it was already missing). */ + fileRemoved: boolean; +} + +/** + * Remove a fork and fall back to upstream. Refuses to discard local customizations + * (file differs from its base) unless `force` is set. Cleans up a `missing` entry + * (file already deleted) without complaint. + */ +export async function unforkDoc(params: UnforkDocParams): Promise { + const { tbdRoot, forkDir, manifest, name, kind, force } = params; + const entry = findFork(manifest, name, kind); + if (!entry) { + throw new ForkConflictError('not-forked', `${name} is not a forked doc`); + } + const entryKind = entry.kind; + const absPath = forkFilePath(tbdRoot, forkDir, entryKind, name); + const relPath = entry.path; + + let fileRemoved = false; + if (await pathExists(absPath)) { + const current = await readFile(absPath, 'utf-8'); + if (hashContent(current) !== entry.base_hash && !force) { + throw new ForkConflictError( + 'customized', + `${name} has local customizations (differs from its base)`, + ); + } + await rm(absPath, { force: true }); + fileRemoved = true; + } + + await removeBaseContent(tbdRoot, entryKind, name); + return { manifest: removeFork(manifest, name, entryKind), relPath, fileRemoved }; +} + +/** + * Compute the live {@link ForkStatus} of a manifest entry by reading its forked + * file and base, and comparing against the current upstream/cache content. + * + * @param cacheContent current upstream content, or null/undefined if the source is + * gone from the cache (orphaned). + */ +export async function forkStatusFor( + tbdRoot: string, + forkDir: string, + entry: ForkEntry, + cacheContent: string | null | undefined, +): Promise { + const kind = entry.kind; + const absPath = forkFilePath(tbdRoot, forkDir, kind, entry.name); + let forkContent: string | null = null; + try { + forkContent = await readFile(absPath, 'utf-8'); + } catch { + forkContent = null; + } + + return computeForkStatus({ + inManifest: true, + forkFileExists: forkContent !== null, + forkHash: forkContent !== null ? hashContent(forkContent) : undefined, + baseHash: entry.base_hash, + cacheHash: cacheContent == null ? undefined : hashContent(cacheContent), + conflictedFlag: entry.conflicted, + markersPresent: forkContent !== null ? hasUnresolvedConflict(forkContent) : false, + }); +} + +/** Read the forked file content for an entry, or null if it is missing. */ +export async function readForkFile( + tbdRoot: string, + forkDir: string, + entry: ForkEntry, +): Promise { + try { + return await readFile(forkFilePath(tbdRoot, forkDir, entry.kind, entry.name), 'utf-8'); + } catch { + return null; + } +} + +/** Read the stored base snapshot for an entry, or null if it is missing. */ +export async function readForkBase(tbdRoot: string, entry: ForkEntry): Promise { + return readBaseContent(tbdRoot, entry.kind, entry.name); +} + +/** A hand-authored file in the fork dir with no manifest entry (state `local`). */ +export interface LocalForkFile { + kind: ForkKind; + name: string; + /** Repo-relative path (POSIX separators). */ + relPath: string; +} + +/** + * List fork-dir files that have no manifest entry. These are served (the fork dir + * has top lookup precedence) but have no upstream: nothing to update or unfork. + * Only the flat `//*.md` layout is scanned — names are + * identity, so nested folders are deliberately not searched (documented). + */ +export async function listLocalForkFiles( + tbdRoot: string, + forkDir: string, + manifest: ForkManifest, +): Promise { + const locals: LocalForkFile[] = []; + for (const kind of Object.keys(KIND_DIR) as ForkKind[]) { + let entries: string[]; + try { + entries = await readdir(join(tbdRoot, forkDir, KIND_DIR[kind])); + } catch { + continue; // Kind dir absent — nothing forked or added there. + } + for (const entry of entries) { + if (!entry.endsWith('.md')) continue; + const name = entry.slice(0, -3); + if (!findFork(manifest, name, kind)) { + locals.push({ kind, name, relPath: forkRelPath(forkDir, kind, name) }); + } + } + } + locals.sort((a, b) => a.kind.localeCompare(b.kind) || a.name.localeCompare(b.name)); + return locals; +} + +/** Cache-only lookup paths per kind (the pristine upstream copies). */ +const KIND_CACHE_PATHS: Record = { + guideline: CACHE_GUIDELINES_PATHS, + shortcut: CACHE_SHORTCUT_PATHS, + template: CACHE_TEMPLATE_PATHS, +}; + +/** Aggregate drift counts across all forked docs, plus local files. */ +export interface ForkDriftSummary { + forks: number; + customized: number; + /** Upstream moved since the fork point (run `tbd docs update`). */ + stale: number; + conflicted: number; + /** Manifest entries whose forked file was deleted out-of-band. */ + missing: number; + /** Fork-dir files with no manifest entry. */ + local: number; +} + +/** + * Compute the drift summary for awareness surfaces (`tbd sync` notice, status). + * Reads the manifest, the fork dir, and the doc cache; safe to call when nothing + * is forked (all zeros, no cache loads). + */ +export async function computeForkDriftSummary( + tbdRoot: string, + forkDir: string, + manifest: ForkManifest, +): Promise { + const summary: ForkDriftSummary = { + forks: manifest.forks.length, + customized: 0, + stale: 0, + conflicted: 0, + missing: 0, + local: 0, + }; + summary.local = (await listLocalForkFiles(tbdRoot, forkDir, manifest)).length; + if (manifest.forks.length === 0) { + return summary; + } + + const caches = new Map(); + for (const entry of manifest.forks) { + let cache = caches.get(entry.kind); + if (!cache) { + cache = new DocCache(KIND_CACHE_PATHS[entry.kind] ?? [], tbdRoot); + await cache.load({ quiet: true }); + caches.set(entry.kind, cache); + } + const status = await forkStatusFor( + tbdRoot, + forkDir, + entry, + cache.get(entry.name)?.doc.content ?? null, + ); + if (status.customized) summary.customized++; + if (status.stale) summary.stale++; + if (status.conflicted) summary.conflicted++; + if (status.state === 'missing') summary.missing++; + } + return summary; +} + +/** + * Sanitize untrusted text (a frontmatter blurb or doc name) for safe inclusion + * in the generated README list. The blurb comes from forked/local file content, + * so it must not be able to inject markdown structure, links, or raw HTML into a + * committed file rendered on GitHub: take the first line and strip the + * characters that break a single list-item context. + */ +function sanitizeForReadme(text: string): string { + const firstLine = text.split(/\r?\n/)[0] ?? ''; + return firstLine + .replace(/[<>[\]`|]/g, '') + .replace(/\s+/g, ' ') + .trim() + .slice(0, 200); +} + +/** Percent-encode each path segment so odd local filenames make valid links. */ +function readmeLinkPath(relPath: string): string { + return relPath + .split('/') + .map((seg) => encodeURIComponent(seg)) + .join('/'); +} + +/** First frontmatter description (or title) of a doc file, for the README index. */ +async function docBlurb(absPath: string): Promise { + try { + const data = matter(await readFile(absPath, 'utf-8')).data as Record; + const description = typeof data.description === 'string' ? data.description : undefined; + const title = typeof data.title === 'string' ? data.title : undefined; + const blurb = description ?? title; + return blurb ? sanitizeForReadme(blurb) : undefined; + } catch { + return undefined; + } +} + +/** + * Regenerate the fork dir's `README.md` index (what this folder is, who manages + * it, and one line per doc). Called after every fork/unfork/update. When nothing + * is forked and no local files remain, the README is removed and empty kind dirs + * are pruned so `unfork --all` leaves the repo pristine. + */ +export async function regenerateForkDirReadme( + tbdRoot: string, + forkDir: string, + manifest: ForkManifest, +): Promise { + const readmePath = join(tbdRoot, forkDir, 'README.md'); + const locals = await listLocalForkFiles(tbdRoot, forkDir, manifest); + + if (manifest.forks.length === 0 && locals.length === 0) { + await rm(readmePath, { force: true }); + for (const kindDir of Object.values(KIND_DIR)) { + await rmdir(join(tbdRoot, forkDir, kindDir)).catch(() => undefined); + } + await rmdir(join(tbdRoot, forkDir)).catch(() => undefined); + return; + } + + interface IndexRow { + kind: string; + name: string; + relPath: string; + suffix: string; + } + const rows: IndexRow[] = [ + ...manifest.forks.map((f) => ({ + kind: f.kind, + name: f.name, + relPath: forkRelPath(forkDir, f.kind, f.name), + suffix: '', + })), + ...locals.map((l) => ({ ...l, suffix: ' *(local — not from an upstream)*' })), + ].sort((a, b) => a.kind.localeCompare(b.kind) || a.name.localeCompare(b.name)); + + const lines: string[] = [ + '', + '', + '# tbd Docs (forked into this repo)', + '', + 'Engineering guidelines, shortcuts, and templates managed by', + '[tbd](https://github.com/jlevy/tbd), forked here so they are visible, reviewable,', + 'and editable. tbd serves these copies instead of its built-in versions.', + '', + '- Edit any doc in place — your copy is what tbd serves.', + '- `tbd docs status` shows each doc’s state; `tbd docs update` pulls in upstream', + ' changes (three-way merge); `tbd docs unfork ` returns a doc to the', + ' built-in version.', + '- Keep files at `/.md`: names are how tbd identifies docs, nested', + ' folders are not scanned, and renaming a file counts as delete + add.', + '', + ]; + let currentKind = ''; + for (const row of rows) { + if (row.kind !== currentKind) { + currentKind = row.kind; + lines.push(`## ${KIND_DIR[row.kind as ForkKind] ?? row.kind}`, ''); + } + const blurb = await docBlurb(join(tbdRoot, row.relPath)); + const fileName = row.relPath.split('/').slice(-2).join('/'); + // Local filenames are arbitrary on disk; escape the link text and encode the + // link target so a name like `xy.md` or `a b.md` can't break the README. + const label = sanitizeForReadme(row.name) || row.name.replace(/[<>[\]`|]/g, ''); + lines.push( + `- [**${label}**](./${readmeLinkPath(fileName)})${blurb ? ` — ${blurb}` : ''}${row.suffix}`, + ); + } + lines.push(''); + await mkdir(join(tbdRoot, forkDir), { recursive: true }); + await writeFile(readmePath, lines.join('\n')); +} + +/** Compute the repo-relative path for a fork dir given an absolute tbd root. */ +export function relativeForkDir(tbdRoot: string, absForkDir: string): string { + return relative(tbdRoot, absForkDir); +} diff --git a/packages/tbd/src/file/doc-sync.ts b/packages/tbd/src/file/doc-sync.ts index 0236d50d..1f7d2b66 100644 --- a/packages/tbd/src/file/doc-sync.ts +++ b/packages/tbd/src/file/doc-sync.ts @@ -12,7 +12,9 @@ import { writeFile } from 'atomically'; import { fileURLToPath } from 'node:url'; import { TBD_DOCS_DIR } from '../lib/paths.js'; -import { fetchWithGhFallback } from './github-fetch.js'; +import { fetchWithGhFallback, gitDocRefToRawUrl } from './github-fetch.js'; +import { tryParseDocRef } from '../docref/index.js'; +import { gitSafeEnv } from '../lib/git-env.js'; import { readConfig, writeConfig, updateLocalState } from './config.js'; // ============================================================================= @@ -23,16 +25,95 @@ import { readConfig, writeConfig, updateLocalState } from './config.js'; * A parsed document source. */ export interface DocSource { - /** Source type: internal bundled or external URL */ - type: 'internal' | 'url'; - /** The source location - either a relative path or full URL */ + /** Source type: internal bundled, in-repo local file, or external URL */ + type: 'internal' | 'local' | 'url'; + /** The source location - a relative path or full URL */ location: string; } +/** One group of config entries sharing a source root (git repo+ref, etc.). */ +export interface SourceGroup { + /** Group key: 'internal', 'local', a git 'host:owner/repo@ref', or a URL origin. */ + key: string; + /** Parsed git source shared by the group, when it is a git group. */ + gitSource?: { host: 'github' | 'gitlab'; owner: string; repo: string; ref?: string }; + /** [destPath, sourceStr] config entries in the group. */ + entries: [string, string][]; +} + +/** + * Group docs_cache.files entries by source root: one group per git repo+ref + * (so failures isolate per source and revisions are captured once), with + * internal, local, and ad-hoc URL entries in origin-keyed groups. + */ +export function groupSourceEntries(config: Record): SourceGroup[] { + const groups = new Map(); + for (const [destPath, sourceStr] of Object.entries(config)) { + let key = 'url'; + let gitSource: SourceGroup['gitSource']; + if (sourceStr.startsWith('internal:')) { + key = 'internal'; + } else { + const parsed = tryParseDocRef(sourceStr); + if (parsed?.kind === 'git') { + key = `${parsed.host}:${parsed.owner}/${parsed.repo}@${parsed.ref ?? 'main'}`; + gitSource = { + host: parsed.host, + owner: parsed.owner, + repo: parsed.repo, + ref: parsed.ref, + }; + } else if (parsed?.kind === 'local') { + key = 'local'; + } else { + try { + key = new URL(sourceStr).origin; + } catch { + key = 'url'; + } + } + } + let group = groups.get(key); + if (!group) { + group = { key, gitSource, entries: [] }; + groups.set(key, group); + } + group.entries.push([destPath, sourceStr]); + } + return [...groups.values()]; +} + +/** + * Capture the current revision of a git source's ref (one ls-remote per + * group), for provenance. Best-effort: offline or missing git returns null. + */ +async function captureGitRevision( + src: NonNullable, +): Promise { + const repoUrl = + src.host === 'gitlab' + ? `https://gitlab.com/${src.owner}/${src.repo}.git` + : `https://github.com/${src.owner}/${src.repo}.git`; + try { + const { execFile } = await import('node:child_process'); + const { promisify } = await import('node:util'); + const out = await promisify(execFile)('git', ['ls-remote', repoUrl, src.ref ?? 'HEAD'], { + timeout: 10_000, + env: gitSafeEnv(), + }); + const sha = out.stdout.split('\t')[0]?.trim(); + return sha && /^[0-9a-f]{40}$/.test(sha) ? sha : null; + } catch { + return null; + } +} + /** * Result of a sync operation. */ export interface SyncResult { + /** Captured git revisions per source group key (provenance; best-effort). */ + sourceRevisions: Record; /** Paths of newly downloaded/copied docs */ added: string[]; /** Paths of updated docs (content changed) */ @@ -108,6 +189,16 @@ export class DocSync { }; } + // Git docrefs (github:o/r@ref//path, gitlab:...) fetch via the host's + // raw-content endpoint; the canonical docref is what config stores. + const parsed = tryParseDocRef(source); + if (parsed?.kind === 'git') { + return { type: 'url', location: gitDocRefToRawUrl(parsed) }; + } + if (parsed?.kind === 'local') { + return { type: 'local', location: parsed.path }; + } + // Anything else is treated as a URL return { type: 'url', @@ -124,6 +215,10 @@ export class DocSync { if (source.type === 'internal') { return this.fetchInternalContent(source.location); } + if (source.type === 'local') { + // Local docref: read from the repo, relative to the tbd root. + return await readFile(join(this.tbdRoot, source.location), 'utf-8'); + } return this.fetchUrlContent(source.location); } @@ -204,6 +299,7 @@ export class DocSync { */ async sync(options: DocSyncOptions = {}): Promise { const result: SyncResult = { + sourceRevisions: {}, added: [], updated: [], removed: [], @@ -215,45 +311,79 @@ export class DocSync { const currentPaths = await this.getCurrentState(); const configPaths = new Set(Object.keys(this.config)); - // Process each doc in config - for (const [destPath, sourceStr] of Object.entries(this.config)) { - try { - const source = this.parseSource(sourceStr); - const content = await this.fetchContent(source); - const fullPath = join(this.docsDir, destPath); - - // Check if file exists and compare content - let exists = false; - let existingContent = ''; + // Group entries by source root (one group per git repo+ref; internal, + // local, and ad-hoc URLs each form their own group). Grouping gives + // per-source failure isolation — one unreachable repo skips only its own + // entries instead of timing out on each — and one revision capture per + // git source for provenance. + const groups = groupSourceEntries(this.config); + const failedGroups = new Set(); + for (const group of groups) { + if (group.gitSource) { + const revision = await captureGitRevision(group.gitSource); + if (revision) { + result.sourceRevisions[group.key] = revision; + } + } + } - try { - existingContent = await readFile(fullPath, 'utf-8'); - exists = true; - } catch { - // File doesn't exist + // Process each doc in config (group order; failed groups short-circuit) + for (const group of groups) { + for (const [destPath, sourceStr] of group.entries) { + if (failedGroups.has(group.key)) { + result.errors.push({ + path: destPath, + error: `skipped: source group ${group.key} unreachable`, + }); + result.success = false; + continue; } + try { + const source = this.parseSource(sourceStr); + const content = await this.fetchContent(source); + const fullPath = join(this.docsDir, destPath); + + // Check if file exists and compare content + let exists = false; + let existingContent = ''; + + try { + existingContent = await readFile(fullPath, 'utf-8'); + exists = true; + } catch { + // File doesn't exist + } - if (!exists) { - // New file - if (!options.dryRun) { - await mkdir(dirname(fullPath), { recursive: true }); - await writeFile(fullPath, content); + if (!exists) { + // New file + if (!options.dryRun) { + await mkdir(dirname(fullPath), { recursive: true }); + await writeFile(fullPath, content); + } + result.added.push(destPath); + } else if (existingContent !== content) { + // Content changed + if (!options.dryRun) { + await writeFile(fullPath, content); + } + result.updated.push(destPath); } - result.added.push(destPath); - } else if (existingContent !== content) { - // Content changed - if (!options.dryRun) { - await writeFile(fullPath, content); + // else: unchanged, do nothing + } catch (err) { + const message = (err as Error).message; + result.errors.push({ path: destPath, error: message }); + result.success = false; + // Network-shaped failures poison the whole group: skip its remaining + // entries rather than re-timing-out per file. Cached copies are never + // pruned on fetch failure (removal below only applies to paths that + // left the config). + if ( + group.gitSource && + /fetch|network|ENOTFOUND|ETIMEDOUT|ECONNREFUSED|HTTP/i.test(message) + ) { + failedGroups.add(group.key); } - result.updated.push(destPath); } - // else: unchanged, do nothing - } catch (err) { - result.errors.push({ - path: destPath, - error: (err as Error).message, - }); - result.success = false; } } @@ -338,8 +468,14 @@ export async function generateDefaultDocCacheConfig(): Promise/.md — verbatim base snapshots (the merge bases) + * + * The base of every fork is a stored snapshot of the upstream content the fork + * was created from. Together with `base_hash`, this makes "customized?", + * "stale vs upstream?", and three-way merging cheap, exact, offline operations. + * + * The hashing and state computation here are pure functions (see + * {@link hashContent} and {@link computeForkStatus}); only the read/write helpers + * touch the filesystem. + */ + +import { createHash } from 'node:crypto'; +import { readFile, mkdir, rm } from 'node:fs/promises'; +import { dirname, join } from 'node:path'; + +import { writeFile } from 'atomically'; +import { parse as parseYaml } from 'yaml'; +import { z } from 'zod'; + +import { stringifyYaml } from '../utils/yaml-utils.js'; +import { withLockfile } from '../utils/lockfile.js'; +import { resolveSharedTbdPaths } from '../lib/paths.js'; +import { isDocRef } from '../docref/index.js'; + +/** Directory (repo-relative under `.tbd/`) holding all fork state. */ +export const DOC_FORKS_DIR = '.tbd/doc-forks'; +/** Manifest filename within {@link DOC_FORKS_DIR}. */ +export const FORKS_FILE = 'forks.yml'; +/** Subdirectory within {@link DOC_FORKS_DIR} holding base snapshots. */ +export const BASE_SUBDIR = 'base'; + +/** Doc kinds that can be forked. */ +export const FORK_KINDS = ['guideline', 'shortcut', 'template', 'reference'] as const; +export type ForkKind = (typeof FORK_KINDS)[number]; + +// ============================================================================= +// Schema +// ============================================================================= + +/** + * A safe doc name: no path separators, no `..`, no leading dot, no NUL. + * Names are used to build filesystem paths (and are a doc's identity), so a + * crafted manifest must not be able to escape the fork dir (e.g. via a + * `../../../../victim` name through `unfork --force`). Allows the punctuation + * real doc names use (letters, digits, `.`, `_`, `-`). + */ +const SAFE_DOC_NAME = /^[A-Za-z0-9][A-Za-z0-9._-]*$/; +export function isSafeDocName(name: string): boolean { + return SAFE_DOC_NAME.test(name) && !name.includes('..') && !name.endsWith('.md'); +} + +export const ForkEntrySchema = z.object({ + /** Doc name (e.g. "python-rules"). Constrained so it cannot escape the fork dir. */ + name: z.string().min(1).refine(isSafeDocName, { + message: 'invalid doc name (no path separators, "..", or leading dot)', + }), + /** Doc kind — must be one of the known fork kinds. */ + kind: z.enum(FORK_KINDS), + /** Repo-relative path of the forked file (e.g. "docs/tbd/guidelines/python-rules.md"). */ + path: z.string().min(1), + /** Provenance docref the fork was created from (docref-everywhere rule). */ + source: z.string().min(1).refine(isDocRef, { message: 'source must be a valid docref' }), + /** sha256: of the LF-normalized base content. */ + base_hash: z.string().min(1), + /** tbd version when the base was last set. */ + tbd_version: z.string().optional(), + /** Upstream commit at base time (git-hosted sources only). */ + source_revision: z.string().optional(), + /** Exact/matching tag at base time, when one exists. */ + source_tag: z.string().optional(), + /** Set by `update --merge` when it leaves conflict markers; auto-clears. */ + conflicted: z.boolean().optional(), +}); + +export type ForkEntry = z.infer; + +export const ForkManifestSchema = z.object({ + forks: z.array(ForkEntrySchema).default([]), +}); + +export type ForkManifest = z.infer; + +/** An empty manifest (no forks). */ +export function emptyManifest(): ForkManifest { + return { forks: [] }; +} + +// ============================================================================= +// Hashing (pure) +// ============================================================================= + +/** Normalize line endings to LF so hashes are stable across platforms. */ +export function normalizeLineEndings(content: string): string { + return content.replace(/\r\n/g, '\n').replace(/\r/g, '\n'); +} + +/** sha256: hash of the LF-normalized content. */ +export function hashContent(content: string): string { + const hex = createHash('sha256').update(normalizeLineEndings(content), 'utf8').digest('hex'); + return `sha256:${hex}`; +} + +/** + * Whether `content` still contains git conflict markers. Requires all three + * standard markers so prose that merely discusses one marker is not flagged. + */ +export function hasConflictMarkers(content: string): boolean { + return /^<{7}/m.test(content) && /^={7}\s*$/m.test(content) && /^>{7}/m.test(content); +} + +/** + * The labels tbd writes into its three-way merge conflict markers. Detection of + * an *unresolved* conflict keys off these specific labels (not generic marker + * lines) so a forked doc that legitimately contains conflict-marker examples + * (e.g. a git tutorial, or our own golden-testing guideline) is not stuck + * `conflicted` forever after one unrelated `update --merge`. + */ +export const CONFLICT_LABELS = { + ours: 'ours (your fork)', + base: 'base (fork point)', + theirs: 'theirs (upstream)', +} as const; + +/** Whether `content` still carries tbd's own unresolved conflict markers. */ +export function hasUnresolvedConflict(content: string): boolean { + return ( + content.includes(`<<<<<<< ${CONFLICT_LABELS.ours}`) && + content.includes(`>>>>>>> ${CONFLICT_LABELS.theirs}`) + ); +} + +/** + * Loose semver comparison on major.minor.patch (prerelease ignored). Returns + * null when either version is unparseable — callers must not guard on a version + * they cannot parse (treat null as "do not block"). + */ +export function compareVersionsLoose(a: string, b: string): -1 | 0 | 1 | null { + const parse = (v: string): [number, number, number] | null => { + const m = /^(\d+)\.(\d+)\.(\d+)/.exec(v.trim()); + return m ? [Number(m[1]), Number(m[2]), Number(m[3])] : null; + }; + const pa = parse(a); + const pb = parse(b); + if (!pa || !pb) return null; + for (let i = 0; i < 3; i++) { + const x = pa[i]!; + const y = pb[i]!; + if (x < y) return -1; + if (x > y) return 1; + } + return 0; +} + +/** + * Run `fn` while holding the doc-forks manifest lock, serializing the + * read-modify-write of `forks.yml` across concurrent fork/unfork/update so + * entries are not lost to last-writer-wins. The lock lives in the machine-local + * git common dir (never committed), alongside the data-sync lock. + */ +export async function withForkManifestLock(tbdRoot: string, fn: () => Promise): Promise { + const paths = await resolveSharedTbdPaths(tbdRoot); + await mkdir(paths.sharedLocksDir, { recursive: true }); + return withLockfile(join(paths.sharedLocksDir, 'doc-forks.lock'), fn); +} + +// ============================================================================= +// State computation (pure) +// ============================================================================= + +export type DocState = + | 'upstream' + | 'forked' + | 'customized' + | 'stale' + | 'conflicted' + | 'local' + | 'missing' + | 'orphaned'; + +/** Inputs for {@link computeForkStatus}; all comparisons are by hash. */ +export interface ForkStatusInput { + /** Whether a manifest entry exists for this doc. */ + inManifest: boolean; + /** Whether the forked file is present on disk. */ + forkFileExists: boolean; + /** sha256: of the current forked file (if present). */ + forkHash?: string; + /** Recorded `base_hash` from the manifest entry. */ + baseHash?: string; + /** sha256: of the current upstream/cache content; undefined if the source is gone. */ + cacheHash?: string; + /** Manifest `conflicted` flag. */ + conflictedFlag?: boolean; + /** Whether conflict markers are present in the current file. */ + markersPresent?: boolean; +} + +/** + * Computed lifecycle status of a doc. `state` is the headline for display; the + * booleans are orthogonal modifiers (`customized` and `stale` can combine). + */ +export interface ForkStatus { + state: DocState; + customized: boolean; + stale: boolean; + conflicted: boolean; + orphaned: boolean; +} + +/** + * Compute a doc's fork status from hashes and flags. Pure and total — every + * combination of inputs maps to exactly one {@link ForkStatus}. + */ +export function computeForkStatus(input: ForkStatusInput): ForkStatus { + const none = { customized: false, stale: false, conflicted: false, orphaned: false }; + + if (!input.inManifest) { + // A file in the fork dir with no manifest entry is a hand-authored local doc; + // otherwise the doc is simply served from upstream via the cache. + return { state: input.forkFileExists ? 'local' : 'upstream', ...none }; + } + + if (!input.forkFileExists) { + return { state: 'missing', ...none }; + } + + const customized = input.forkHash !== input.baseHash; + const orphaned = input.cacheHash === undefined; + const stale = !orphaned && input.cacheHash !== input.baseHash; + const conflicted = Boolean(input.conflictedFlag && input.markersPresent); + + // Headline state for display. `customized` and `stale` can combine; the + // renderer composes "customized, stale" from state + the stale modifier. + let state: DocState; + if (conflicted) { + state = 'conflicted'; + } else if (orphaned) { + state = 'orphaned'; + } else if (customized) { + state = 'customized'; + } else if (stale) { + state = 'stale'; + } else { + state = 'forked'; + } + + return { state, customized, stale, conflicted, orphaned }; +} + +// ============================================================================= +// Manifest helpers (pure) +// ============================================================================= + +/** Find a fork entry by name, optionally constrained to a kind. */ +export function findFork( + manifest: ForkManifest, + name: string, + kind?: string, +): ForkEntry | undefined { + return manifest.forks.find((f) => f.name === name && (kind === undefined || f.kind === kind)); +} + +/** Return a new manifest with `entry` inserted or replaced (matched by kind+name). */ +export function upsertFork(manifest: ForkManifest, entry: ForkEntry): ForkManifest { + const forks = manifest.forks.filter((f) => !(f.name === entry.name && f.kind === entry.kind)); + forks.push(entry); + forks.sort((a, b) => a.kind.localeCompare(b.kind) || a.name.localeCompare(b.name)); + return { forks }; +} + +/** Return a new manifest with the matching entry removed. */ +export function removeFork(manifest: ForkManifest, name: string, kind?: string): ForkManifest { + return { + forks: manifest.forks.filter( + (f) => !(f.name === name && (kind === undefined || f.kind === kind)), + ), + }; +} + +// ============================================================================= +// Paths +// ============================================================================= + +export function forksDir(tbdRoot: string): string { + return join(tbdRoot, DOC_FORKS_DIR); +} + +export function forksFilePath(tbdRoot: string): string { + return join(forksDir(tbdRoot), FORKS_FILE); +} + +export function baseFilePath(tbdRoot: string, kind: string, name: string): string { + return join(forksDir(tbdRoot), BASE_SUBDIR, kind, `${name}.md`); +} + +// ============================================================================= +// Filesystem I/O +// ============================================================================= + +function isNotFound(err: unknown): boolean { + return (err as NodeJS.ErrnoException | undefined)?.code === 'ENOENT'; +} + +/** The outer manifest shape, before per-entry validation. */ +const ForkManifestEnvelopeSchema = z.object({ + forks: z.array(z.unknown()).default([]), +}); + +/** + * Read the fork manifest, returning an empty manifest if none exists. + * + * Parsing is tolerant per entry: a malformed or unsafe entry (bad name/kind, + * path-traversal attempt) is dropped with a warning rather than aborting the + * whole read. This both fails closed on a crafted entry (it is never returned, + * so commands never act on it — no out-of-tree deletes) and keeps one corrupt + * entry from taking down status/update for every other fork. + */ +export async function readForkManifest(tbdRoot: string): Promise { + let content: string; + try { + content = await readFile(forksFilePath(tbdRoot), 'utf-8'); + } catch (err) { + if (isNotFound(err)) { + return emptyManifest(); + } + throw err; + } + const envelope = ForkManifestEnvelopeSchema.parse(parseYaml(content) ?? { forks: [] }); + const forks: ForkEntry[] = []; + let dropped = 0; + for (const raw of envelope.forks) { + const parsed = ForkEntrySchema.safeParse(raw); + if (parsed.success) { + forks.push(parsed.data); + } else { + dropped++; + } + } + if (dropped > 0) { + process.stderr.write( + `• Ignored ${dropped} invalid fork manifest entr${dropped === 1 ? 'y' : 'ies'} ` + + `in ${FORKS_FILE} (bad name/kind or unsafe path). Run 'tbd doctor' to review.\n`, + ); + } + return { forks }; +} + +/** Write the fork manifest (creating `.tbd/doc-forks/` as needed). */ +export async function writeForkManifest(tbdRoot: string, manifest: ForkManifest): Promise { + await mkdir(forksDir(tbdRoot), { recursive: true }); + const yaml = stringifyYaml(manifest, { lineWidth: 0, sortMapEntries: false }); + await writeFile(forksFilePath(tbdRoot), yaml); +} + +/** Read a base snapshot's content, or null if it is absent. */ +export async function readBaseContent( + tbdRoot: string, + kind: string, + name: string, +): Promise { + try { + return await readFile(baseFilePath(tbdRoot, kind, name), 'utf-8'); + } catch (err) { + if (isNotFound(err)) { + return null; + } + throw err; + } +} + +/** Write a base snapshot verbatim (creating parent dirs as needed). */ +export async function writeBaseContent( + tbdRoot: string, + kind: string, + name: string, + content: string, +): Promise { + const path = baseFilePath(tbdRoot, kind, name); + await mkdir(dirname(path), { recursive: true }); + await writeFile(path, content); +} + +/** Remove a base snapshot if present. */ +export async function removeBaseContent( + tbdRoot: string, + kind: string, + name: string, +): Promise { + await rm(baseFilePath(tbdRoot, kind, name), { force: true }); +} diff --git a/packages/tbd/src/file/fork-update.ts b/packages/tbd/src/file/fork-update.ts new file mode 100644 index 00000000..30cb7b77 --- /dev/null +++ b/packages/tbd/src/file/fork-update.ts @@ -0,0 +1,334 @@ +/** + * Updating forked docs after upstream moves: three-way merge plus the per-state + * decision logic from the spec's update table. + * + * The merge itself is outsourced to git (`git merge-file`), which works on plain + * files, reports the conflict count via exit code, and uses standard conflict + * markers. Nothing in the git repo state is touched. + */ + +import { execFile } from 'node:child_process'; + +import { gitSafeEnv } from '../lib/git-env.js'; +import { mkdtemp, rm, readFile } from 'node:fs/promises'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; + +import { writeFile } from 'atomically'; + +import { + type ForkEntry, + CONFLICT_LABELS, + compareVersionsLoose, + hashContent, + hasUnresolvedConflict, + normalizeLineEndings, +} from './fork-manifest.js'; + +/** Result of a three-way merge. */ +export interface MergeResult { + merged: string; + /** Number of conflict hunks (0 = clean). */ + conflicts: number; +} + +const MERGE_MAX_BUFFER = 16 * 1024 * 1024; + +/** + * Three-way merge of `current` and `other` against their common `base`, via + * `git merge-file -p`. Returns the merged text (with standard conflict markers + * when conflicts arise) and the conflict count. Pure with respect to repo state: + * it only uses temporary files. + */ +export async function mergeContents( + current: string, + base: string, + other: string, + labels: { current?: string; base?: string; other?: string } = {}, +): Promise { + const dir = await mkdtemp(join(tmpdir(), 'tbd-merge-')); + const currentPath = join(dir, 'current'); + const basePath = join(dir, 'base'); + const otherPath = join(dir, 'other'); + try { + // Normalize line endings before the merge. Hashing is LF-normalized, so a + // CRLF fork file vs an LF base/upstream would otherwise make git merge-file + // see every line as changed and report a spurious whole-file conflict. The + // merged output is LF (matching the hash basis). + await Promise.all([ + writeFile(currentPath, normalizeLineEndings(current)), + writeFile(basePath, normalizeLineEndings(base)), + writeFile(otherPath, normalizeLineEndings(other)), + ]); + + const args = [ + 'merge-file', + '-p', + '-L', + labels.current ?? CONFLICT_LABELS.ours, + '-L', + labels.base ?? CONFLICT_LABELS.base, + '-L', + labels.other ?? CONFLICT_LABELS.theirs, + currentPath, + basePath, + otherPath, + ]; + + return await new Promise((resolve, reject) => { + execFile( + 'git', + args, + { maxBuffer: MERGE_MAX_BUFFER, env: gitSafeEnv() }, + (error, stdout, stderr) => { + if (error) { + const code = (error as NodeJS.ErrnoException & { code?: number }).code; + // git merge-file exits with the number of conflicts, truncated to 127. + // Anything above that (255 = real error, e.g. binary input) must NOT be + // read as a conflict count: stdout is empty on errors, and writing it + // would clobber the forked file. + if (typeof code === 'number' && code > 0 && code <= 127) { + resolve({ merged: stdout, conflicts: code }); + return; + } + const detail = String(stderr ?? '').trim(); + reject( + new Error(`git merge-file failed${detail ? `: ${detail}` : ''}`, { cause: error }), + ); + return; + } + resolve({ merged: stdout, conflicts: 0 }); + }, + ); + }); + } finally { + await rm(dir, { recursive: true, force: true }); + } +} + +/** + * Unified diff of two contents via `git diff --no-index`. Returns the diff text + * (empty string when identical). Uses temporary files and touches no repo state. + */ +export async function diffContents( + left: string, + right: string, + labels: { left?: string; right?: string } = {}, +): Promise { + const dir = await mkdtemp(join(tmpdir(), 'tbd-diff-')); + // Name the temp files after the labels and run from the temp dir with --no-prefix + // so the diff header reads e.g. "--- upstream / +++ ours" instead of temp paths. + const leftName = labels.left ?? 'a'; + const rightName = labels.right ?? 'b'; + try { + await Promise.all([ + writeFile(join(dir, leftName), left), + writeFile(join(dir, rightName), right), + ]); + return await new Promise((resolve, reject) => { + const args = ['diff', '--no-index', '--no-color', '--no-prefix', leftName, rightName]; + execFile( + 'git', + args, + { cwd: dir, maxBuffer: MERGE_MAX_BUFFER, env: gitSafeEnv() }, + (error, stdout) => { + if (error) { + const code = (error as NodeJS.ErrnoException & { code?: number }).code; + // git diff exits 1 when the files differ — that's the normal case. + if (code === 1) { + resolve(stdout); + return; + } + reject(error instanceof Error ? error : new Error('git diff failed')); + return; + } + resolve(stdout); + }, + ); + }); + } finally { + await rm(dir, { recursive: true, force: true }); + } +} + +/** Update strategy chosen by the user for non-clean cases. */ +export type UpdateStrategy = 'default' | 'merge' | 'keep-ours'; + +/** What an update did (or why it was skipped) for one doc. */ +export type UpdateAction = + | 'replaced' + | 'merged-clean' + | 'merged-conflict' + | 'kept' + | 'repaired' + | 'skip-not-stale' + | 'skip-conflict-listed' + | 'skip-unresolved' + | 'skip-orphaned' + | 'skip-missing' + | 'skip-no-base' + | 'skip-newer-base'; + +export interface UpdateOneInput { + entry: ForkEntry; + /** Current forked file content, or null if the file is missing. */ + forkContent: string | null; + /** Stored base snapshot, or null if the base file is missing. */ + baseContent: string | null; + /** Current upstream/cache content, or null if the source is gone (orphaned). */ + upstreamContent: string | null; + strategy: UpdateStrategy; + /** + * The running tbd version. When the entry's base was advanced by a NEWER tbd + * (entry.tbd_version > runningVersion), this client's "upstream" is older than + * the fork point and an update would silently downgrade the doc — so the doc + * is skipped under every strategy until the client upgrades. + */ + runningVersion?: string; +} + +export interface UpdateOneResult { + action: UpdateAction; + /** New forked-file content to write, when the action changes the file. */ + newFileContent?: string; + /** New base content to write (advances the fork point), when applicable. */ + newBaseContent?: string; + /** Set the manifest `conflicted` flag (markers written). */ + setConflicted?: boolean; + /** Whether this doc needs a strategy decision (default run could not proceed). */ + needsDecision?: boolean; + /** Human-readable one-line explanation. */ + message: string; +} + +/** + * Decide and (when needed) perform the three-way merge for a single forked doc, + * implementing the spec's update decision table across the default / --merge / + * --keep-ours strategies. Pure aside from the git merge-file call. + */ +export async function updateOne(input: UpdateOneInput): Promise { + const { entry, forkContent, baseContent, upstreamContent, strategy } = input; + const name = entry.name; + + if (forkContent === null) { + return { action: 'skip-missing', message: `${name}: forked file is missing (doctor's job)` }; + } + if (upstreamContent === null) { + return { + action: 'skip-orphaned', + message: `${name}: upstream removed this doc — keep your copy or 'tbd docs unfork ${name}'`, + }; + } + // An unresolved conflicted doc must be resolved before any update. Keys off + // tbd's own labeled markers so a doc that legitimately contains conflict-marker + // examples is not blocked forever. + if (entry.conflicted && hasUnresolvedConflict(forkContent)) { + return { + action: 'skip-unresolved', + message: `${name}: unresolved conflict markers — resolve them first`, + }; + } + + // Version-skew guard: if the base was advanced by a newer tbd than this one, + // this client's bundled "upstream" is OLDER than the fork point. Updating + // would downgrade the doc (and ping-pong the base across the team), so the + // doc is skipped under every strategy until this client upgrades. + if ( + input.runningVersion !== undefined && + entry.tbd_version !== undefined && + compareVersionsLoose(input.runningVersion, entry.tbd_version) === -1 + ) { + return { + action: 'skip-newer-base', + message: + `${name}: fork point was set by tbd ${entry.tbd_version} (you have ` + + `${input.runningVersion}) — upgrade tbd before updating this doc`, + }; + } + + if (baseContent === null) { + if (strategy === 'keep-ours') { + // Repair: re-establish the base from current upstream, keep the file. + return { + action: 'repaired', + newBaseContent: upstreamContent, + message: `${name}: re-established missing base from upstream (file kept)`, + }; + } + return { + action: 'skip-no-base', + needsDecision: true, + message: `${name}: base snapshot missing — cannot merge; re-run with --keep-ours to repair`, + }; + } + + const customized = hashContent(forkContent) !== hashContent(baseContent); + const stale = hashContent(upstreamContent) !== hashContent(baseContent); + + if (!stale) { + return { action: 'skip-not-stale', message: `${name}: already up to date` }; + } + + // Unmodified fork that is stale: refresh to upstream (default/merge), or keep + // the local copy and just advance the base (keep-ours). + if (!customized) { + if (strategy === 'keep-ours') { + return { + action: 'kept', + newBaseContent: upstreamContent, + message: `${name}: kept your version; fork point advanced`, + }; + } + return { + action: 'replaced', + newFileContent: upstreamContent, + newBaseContent: upstreamContent, + message: `${name}: refreshed to upstream (was unmodified)`, + }; + } + + // Customized and stale. + if (strategy === 'keep-ours') { + return { + action: 'kept', + newBaseContent: upstreamContent, + message: `${name}: kept your version; fork point advanced`, + }; + } + + const merge = await mergeContents(forkContent, baseContent, upstreamContent); + if (merge.conflicts === 0) { + return { + action: 'merged-clean', + newFileContent: merge.merged, + newBaseContent: upstreamContent, + message: `${name}: merged upstream cleanly (review with: git diff)`, + }; + } + + // Conflicts. + if (strategy === 'merge') { + return { + action: 'merged-conflict', + newFileContent: merge.merged, + newBaseContent: upstreamContent, + setConflicted: true, + message: `${name}: wrote merged content with conflict markers; resolve them, then it returns to 'customized'`, + }; + } + // Default: skip and surface the decision. + return { + action: 'skip-conflict-listed', + needsDecision: true, + message: `${name}: your changes conflict with upstream`, + }; +} + +/** Read a file's content, or null if absent. */ +export async function readMaybe(path: string): Promise { + try { + return await readFile(path, 'utf-8'); + } catch { + return null; + } +} diff --git a/packages/tbd/src/file/git.ts b/packages/tbd/src/file/git.ts index 7404980a..85ff1b12 100644 --- a/packages/tbd/src/file/git.ts +++ b/packages/tbd/src/file/git.ts @@ -10,6 +10,8 @@ */ import { execFile } from 'node:child_process'; + +import { gitSafeEnv } from '../lib/git-env.js'; import { mkdir } from 'node:fs/promises'; import { promisify } from 'node:util'; import { basename, dirname, join, normalize } from 'node:path'; @@ -97,7 +99,10 @@ const GIT_MAX_BUFFER = 50 * 1024 * 1024; // 50MB */ export async function git(...args: string[]): Promise { try { - const { stdout } = await execFileAsync('git', args, { maxBuffer: GIT_MAX_BUFFER }); + const { stdout } = await execFileAsync('git', args, { + maxBuffer: GIT_MAX_BUFFER, + env: gitSafeEnv(), + }); return stdout.trim(); } catch (err) { throw GitError.from(err, args); @@ -113,7 +118,7 @@ async function gitNoPrompt(...args: string[]): Promise { try { const { stdout } = await execFileAsync('git', args, { maxBuffer: GIT_MAX_BUFFER, - env: { ...process.env, GIT_TERMINAL_PROMPT: '0' }, + env: gitSafeEnv({ GIT_TERMINAL_PROMPT: '0' }), }); return stdout.trim(); } catch (err) { diff --git a/packages/tbd/src/file/github-fetch.ts b/packages/tbd/src/file/github-fetch.ts index 00425cc5..5befc374 100644 --- a/packages/tbd/src/file/github-fetch.ts +++ b/packages/tbd/src/file/github-fetch.ts @@ -225,3 +225,21 @@ export async function fetchWithGhFallback( const content = await ghCliFetch(rawUrl); return { content, usedGhCli: true }; } + +/** + * Resolve a git docref (github:o/r@ref//path, gitlab:...) to a raw-content URL. + * The ref is required for sync determinism; callers enforce that. + */ +export function gitDocRefToRawUrl(ref: { + host: 'github' | 'gitlab'; + owner: string; + repo: string; + ref?: string; + path: string; +}): string { + const rev = ref.ref ?? 'main'; + if (ref.host === 'gitlab') { + return `https://gitlab.com/${ref.owner}/${ref.repo}/-/raw/${rev}/${ref.path}`; + } + return `https://raw.githubusercontent.com/${ref.owner}/${ref.repo}/${rev}/${ref.path}`; +} diff --git a/packages/tbd/src/lib/doc-categories.ts b/packages/tbd/src/lib/doc-categories.ts new file mode 100644 index 00000000..0711e06e --- /dev/null +++ b/packages/tbd/src/lib/doc-categories.ts @@ -0,0 +1,25 @@ +/** + * The fixed category vocabulary for bundled docs. Every bundled guideline + * declares exactly one in frontmatter (enforced by doc-categories.test.ts); + * the old name-based inference is retired in favor of the declared field. + */ + +export const DOC_CATEGORIES = ['general', 'typescript', 'python', 'convex', 'electron'] as const; + +export type DocCategory = (typeof DOC_CATEGORIES)[number]; + +/** A doc's category: the declared frontmatter field, defaulting to general. */ +export function docCategory(frontmatter: { category?: string } | undefined): DocCategory { + const declared = frontmatter?.category; + return (DOC_CATEGORIES as readonly string[]).includes(declared ?? '') + ? (declared as DocCategory) + : 'general'; +} + +/** Validate a user-supplied --category value. */ +export function parseCategoryOption(value: string): DocCategory { + if (!(DOC_CATEGORIES as readonly string[]).includes(value)) { + throw new Error(`Unknown category "${value}". Valid categories: ${DOC_CATEGORIES.join(', ')}.`); + } + return value as DocCategory; +} diff --git a/packages/tbd/src/lib/git-env.ts b/packages/tbd/src/lib/git-env.ts new file mode 100644 index 00000000..959aa270 --- /dev/null +++ b/packages/tbd/src/lib/git-env.ts @@ -0,0 +1,63 @@ +/** + * Ambient-git-environment isolation for every git subprocess tbd spawns. + * + * tbd is a repo-local tool: it discovers its `.tbd` root by walking up from + * cwd, so its git context must come from cwd too. But git exports GIT_DIR + * (and related vars) into hook environments — always, when a hook runs — and + * an inherited *absolute* GIT_DIR overrides cwd-based discovery in every git + * child process, `-C ` included. A user running tbd inside any git hook + * (post-merge, pre-push, …) would otherwise have tbd resolve the hook's + * repository instead of cwd's and read/write the WRONG repo's tbd data; this + * is the product-level half of the tbd-a1lc incident (tracked as tbd-tgwi), + * where exactly that path rewrote a real repo's data-sync state. + * + * Policy: tbd always operates on the repository containing cwd. Git location + * variables are stripped from every git subprocess, and a one-line warning is + * printed (once per process) when an ambient GIT_DIR was present, so anyone + * setting it intentionally learns it does not redirect tbd. + * + * The same variable list is used by tests/scrub-git-env.ts (vitest worker + * hygiene) and mirrored in scripts/scrub-git-env.mjs (lefthook wrapper). + */ + +/** Environment variables through which git overrides cwd-based discovery. */ +export const GIT_LOCATION_VARS = [ + 'GIT_DIR', + 'GIT_WORK_TREE', + 'GIT_INDEX_FILE', + 'GIT_COMMON_DIR', + 'GIT_OBJECT_DIRECTORY', + 'GIT_ALTERNATE_OBJECT_DIRECTORIES', + 'GIT_PREFIX', + 'GIT_NAMESPACE', +] as const; + +let warnedAmbientGitDir = false; + +/** + * A copy of process.env with git location variables removed (plus optional + * overrides), for use as the `env` of any spawned git process. Emits the + * one-time ambient-GIT_DIR warning as a side effect of first use. + */ +export function gitSafeEnv(extra?: Record): NodeJS.ProcessEnv { + warnIfAmbientGitEnv(); + const env: NodeJS.ProcessEnv = { ...process.env, ...extra }; + for (const name of GIT_LOCATION_VARS) { + delete env[name]; + } + return env; +} + +/** + * Warn once per process when an ambient GIT_DIR/GIT_WORK_TREE is being + * ignored. Stderr keeps piped stdout clean; once keeps hooks readable. + */ +export function warnIfAmbientGitEnv(): void { + if (warnedAmbientGitDir) return; + const ambient = process.env.GIT_DIR ?? process.env.GIT_WORK_TREE; + if (ambient === undefined) return; + warnedAmbientGitDir = true; + process.stderr.write( + `(ignoring inherited GIT_DIR: tbd operates on the repository containing the current directory)\n`, + ); +} diff --git a/packages/tbd/src/lib/paths.ts b/packages/tbd/src/lib/paths.ts index 7bb3d98c..fcb5a81e 100644 --- a/packages/tbd/src/lib/paths.ts +++ b/packages/tbd/src/lib/paths.ts @@ -31,6 +31,8 @@ */ import { execFile } from 'node:child_process'; + +import { gitSafeEnv } from './git-env.js'; import { homedir } from 'node:os'; import { isAbsolute, join, relative, resolve, sep } from 'node:path'; import { promisify } from 'node:util'; @@ -149,6 +151,7 @@ export async function resolveGitCommonDir(cwd: string): Promise { let output: string; try { const { stdout } = await execFileAsync('git', ['-C', cwd, 'rev-parse', '--git-common-dir'], { + env: gitSafeEnv(), maxBuffer: 1024 * 1024, }); output = stdout.trim(); @@ -156,7 +159,7 @@ export async function resolveGitCommonDir(cwd: string): Promise { const { stdout } = await execFileAsync( 'git', ['-C', cwd, 'rev-parse', '--path-format=absolute', '--git-common-dir'], - { maxBuffer: 1024 * 1024 }, + { env: gitSafeEnv(), maxBuffer: 1024 * 1024 }, ); output = stdout.trim(); } @@ -351,28 +354,72 @@ export const INSTALL_DIR = 'install'; /** Built-in install source path (relative to package docs/) */ export const BUILTIN_INSTALL_DIR = INSTALL_DIR; +// ============================================================================= +// Forkable docs: the visible, git-tracked fork directory +// ============================================================================= + +/** References directory name (tbd self-docs and format references). */ +export const REFERENCES_DIR = 'references'; + +/** Cached reference docs directory (docref/docmap formats, tbd-docs, tbd-design). */ +export const TBD_REFERENCES_DIR = join(TBD_DOCS_DIR, REFERENCES_DIR); + +/** + * Default fork directory (repo-relative), where forked docs are made visible. + * A POSIX literal, not join()'d: this value is committed (manifest paths) and + * printed, so it must be identical on every platform; fs access joins it with + * the root via join(), which accepts forward slashes on Windows. + */ +export const FORK_DIR = 'docs/tbd'; + +/** Fork-dir kind subdirectories (repo-relative). */ +export const FORK_SHORTCUTS_DIR = `${FORK_DIR}/${SHORTCUTS_DIR}`; // docs/tbd/shortcuts/ +export const FORK_GUIDELINES_DIR = `${FORK_DIR}/${GUIDELINES_DIR}`; // docs/tbd/guidelines/ +export const FORK_TEMPLATES_DIR = `${FORK_DIR}/${TEMPLATES_DIR}`; // docs/tbd/templates/ +export const FORK_REFERENCES_DIR = `${FORK_DIR}/${REFERENCES_DIR}`; // docs/tbd/references/ + +/** + * Cache-only lookup paths (the gitignored `.tbd/docs/` cache), used when forking + * needs the pristine upstream content rather than a possibly-forked copy. + */ +export const CACHE_SHORTCUT_PATHS = [TBD_SHORTCUTS_SYSTEM, TBD_SHORTCUTS_STANDARD]; +export const CACHE_GUIDELINES_PATHS = [TBD_GUIDELINES_DIR]; +export const CACHE_TEMPLATE_PATHS = [TBD_TEMPLATES_DIR]; +export const CACHE_REFERENCE_PATHS = [TBD_REFERENCES_DIR]; + /** * Default shortcut lookup paths (searched in order, relative to tbd root). - * Earlier paths take precedence over later paths. - * Note: Guidelines and templates are now separate top-level directories. + * Earlier paths take precedence: the fork dir shadows the cache, so a forked doc + * is served wherever the upstream one was. Missing dirs are skipped, so repos with + * no forks behave exactly as before. */ export const DEFAULT_SHORTCUT_PATHS = [ - TBD_SHORTCUTS_SYSTEM, // .tbd/docs/shortcuts/system/ - TBD_SHORTCUTS_STANDARD, // .tbd/docs/shortcuts/standard/ + FORK_SHORTCUTS_DIR, // docs/tbd/shortcuts/ (forked, highest precedence) + ...CACHE_SHORTCUT_PATHS, ]; /** * Default guidelines lookup paths (relative to tbd root). */ export const DEFAULT_GUIDELINES_PATHS = [ - TBD_GUIDELINES_DIR, // .tbd/docs/guidelines/ + FORK_GUIDELINES_DIR, // docs/tbd/guidelines/ (forked, highest precedence) + ...CACHE_GUIDELINES_PATHS, ]; /** * Default template lookup paths (relative to tbd root). */ export const DEFAULT_TEMPLATE_PATHS = [ - TBD_TEMPLATES_DIR, // .tbd/docs/templates/ + FORK_TEMPLATES_DIR, // docs/tbd/templates/ (forked, highest precedence) + ...CACHE_TEMPLATE_PATHS, +]; + +/** + * Default reference-doc lookup paths (relative to tbd root). + */ +export const DEFAULT_REFERENCE_PATHS = [ + FORK_REFERENCES_DIR, // docs/tbd/references/ (forked, highest precedence) + ...CACHE_REFERENCE_PATHS, ]; /** diff --git a/packages/tbd/src/lib/schemas.ts b/packages/tbd/src/lib/schemas.ts index c2150ba7..86505adc 100644 --- a/packages/tbd/src/lib/schemas.ts +++ b/packages/tbd/src/lib/schemas.ts @@ -249,6 +249,13 @@ export const DocsCacheSchema = z.object({ * - Full URL for external docs (e.g., "https://raw.githubusercontent.com/org/repo/main/file.md") */ files: z.record(z.string(), z.string()).optional(), + /** + * Ordered local-path docrefs (./-prefixed) naming additional in-repo doc + * directories. They slot into the effective lookup order between the fork + * dir and the cache; docs found there serve with state `local` and are not + * forkable or updatable (they already live in the repo). + */ + local_dirs: z.array(z.string()).optional(), /** * Search paths for doc lookup (like shell $PATH). * Earlier paths take precedence when names conflict. diff --git a/packages/tbd/src/lib/tbd-format.ts b/packages/tbd/src/lib/tbd-format.ts index 77839a7a..e44f9689 100644 --- a/packages/tbd/src/lib/tbd-format.ts +++ b/packages/tbd/src/lib/tbd-format.ts @@ -41,7 +41,7 @@ * Current format version. * Bump this ONLY for breaking changes that require migration. */ -export const CURRENT_FORMAT = 'f04'; +export const CURRENT_FORMAT = 'f05'; /** * Initial format version for configs that don't have tbd_format field. @@ -99,6 +99,20 @@ export const FORMAT_HISTORY = { migration: 'Initializes shared common-dir sync layout before writing config.yml with tbd_format f04', }, + f05: { + introduced: '0.3.0', + description: 'Adds the forkable-docs layout (committed fork manifest + visible fork dir)', + changes: [ + 'Added committed fork state under .tbd/doc-forks/ (forks.yml manifest + base/ snapshots)', + 'Added the visible fork dir (docs/tbd/) which shadows the doc cache in lookups', + 'Doc commands (tbd docs fork/unfork/status/update/diff/list) manage that state', + ], + migration: + 'Metadata-only: stamps tbd_format f05 (fork artifacts appear only when fork is first ' + + 'used). The shared common-dir layout.yml is re-stamped in place on the next data ' + + 'command. Revert: restore .tbd/config.yml (git checkout) and delete ' + + '$GIT_COMMON_DIR/tbd/layout.yml; it regenerates from the config.', + }, } as const; export type FormatVersion = keyof typeof FORMAT_HISTORY; @@ -262,6 +276,29 @@ function migrate_f03_to_f04(config: RawConfig): MigrationResult { }; } +/** + * Migrate from f04 to f05. + * - Metadata-only stamp: the forkable-docs artifacts (.tbd/doc-forks/, docs/tbd/) + * are created lazily by `tbd docs fork`, not by migration. + * - The bump gates older clients: they must not serve upstream copies of docs that + * this repo has forked and customized. + */ +function migrate_f04_to_f05(config: RawConfig): MigrationResult { + const changes: string[] = []; + const migrated = { ...config }; + + migrated.tbd_format = 'f05'; + changes.push('Updated tbd_format: f05'); + + return { + config: migrated, + fromFormat: 'f04', + toFormat: 'f05', + changed: changes.length > 0, + changes, + }; +} + // ============================================================================= // Public API // ============================================================================= @@ -339,6 +376,13 @@ export function migrateToLatest(config: RawConfig): MigrationResult { allChanges.push(...result.changes); } + if (currentFormat === 'f04') { + const result = migrate_f04_to_f05(current); + current = result.config; + currentFormat = 'f05' as FormatVersion; + allChanges.push(...result.changes); + } + return { config: current, fromFormat, @@ -417,5 +461,10 @@ export function describeMigration(fromFormat: FormatVersion): string[] { current = 'f04'; } + if (current === 'f04') { + descriptions.push('f04 → f05: Add forkable-docs layout (stamp only)'); + current = 'f05'; + } + return descriptions; } diff --git a/packages/tbd/tests/cli-advanced.tryscript.md b/packages/tbd/tests/cli-advanced.tryscript.md index 143c1c83..0857fe5f 100644 --- a/packages/tbd/tests/cli-advanced.tryscript.md +++ b/packages/tbd/tests/cli-advanced.tryscript.md @@ -275,7 +275,7 @@ settings: ```console $ tbd config show --json { - "tbd_format": "f04", + "tbd_format": "f05", "tbd_version": "[..]", "sync": { "branch": "tbd-sync", @@ -487,7 +487,7 @@ Getting Started: This initializes tbd and configures your coding agents automatically. To refresh setup (idempotent, safe anytime): `tbd setup --auto` - For interactive setup: `tbd setup --interactive` + For managed docs: `tbd docs` For more on tbd, see: https://github.com/jlevy/tbd ? 0 @@ -531,7 +531,7 @@ Getting Started: This initializes tbd and configures your coding agents automatically. To refresh setup (idempotent, safe anytime): `tbd setup --auto` - For interactive setup: `tbd setup --interactive` + For managed docs: `tbd docs` For more on tbd, see: https://github.com/jlevy/tbd ? 0 @@ -572,7 +572,7 @@ Getting Started: This initializes tbd and configures your coding agents automatically. To refresh setup (idempotent, safe anytime): `tbd setup --auto` - For interactive setup: `tbd setup --interactive` + For managed docs: `tbd docs` For more on tbd, see: https://github.com/jlevy/tbd ? 0 @@ -614,7 +614,7 @@ Getting Started: This initializes tbd and configures your coding agents automatically. To refresh setup (idempotent, safe anytime): `tbd setup --auto` - For interactive setup: `tbd setup --interactive` + For managed docs: `tbd docs` For more on tbd, see: https://github.com/jlevy/tbd ? 0 diff --git a/packages/tbd/tests/cli-closing.tryscript.md b/packages/tbd/tests/cli-closing.tryscript.md index c29f5ce5..ba83dc5b 100644 --- a/packages/tbd/tests/cli-closing.tryscript.md +++ b/packages/tbd/tests/cli-closing.tryscript.md @@ -120,7 +120,7 @@ Getting Started: This initializes tbd and configures your coding agents automatically. To refresh setup (idempotent, safe anytime): `tbd setup --auto` - For interactive setup: `tbd setup --interactive` + For managed docs: `tbd docs` For more on tbd, see: https://github.com/jlevy/tbd ? 0 diff --git a/packages/tbd/tests/cli-doc-output.tryscript.md b/packages/tbd/tests/cli-doc-output.tryscript.md index 5ec96319..ea7c8b9f 100644 --- a/packages/tbd/tests/cli-doc-output.tryscript.md +++ b/packages/tbd/tests/cli-doc-output.tryscript.md @@ -47,14 +47,15 @@ backward-compatibility-rules [..] ? 0 ``` -# Test: Guidelines --json returns structured data +# Test: Guidelines --list --json emits a docmap (one model across all doc surfaces) ```console -$ tbd guidelines --list --json | head -5 -[ - { - "name": "backward-compatibility-rules", - "title": "Backward Compatibility[..]", +$ tbd guidelines --list --json | head -6 +{ + "docmap": "docmap/0.1", + "documents": [ + { + "name": "backward-compatibility-rules", [..] ? 0 ``` @@ -69,20 +70,20 @@ Agent instructions: You have activated a guidelines document. If a user has aske title: [..]Commit[..] description: Conventional Commits format with extensions for agentic workflows author: Joshua Levy (github.com/jlevy) with LLM assistance +category: general --- -[..] ? 0 ``` -# Test: Guidelines query with --json returns raw content +# Test: Guidelines query with --json returns the docmap entry plus content ```console $ tbd guidelines commit-conventions --json | head -5 { "name": "commit-conventions", + "type": "guideline", + "source": "internal:guidelines/commit-conventions.md", "title": "[..]Commit[..]", - "score": 1, -[..] ? 0 ``` @@ -118,11 +119,11 @@ description: Welcome message for users after tbd installation or setup ## Docs Command -# Test: Docs command with --color=never produces clean output +# Test: Manual section listing with --color=never produces clean output ```console -$ tbd --color=never docs --list | head -3 -Available documentation sections: +$ tbd --color=never docs show tbd-docs --sections | head -3 +Sections in tbd-docs: [..] ? 0 diff --git a/packages/tbd/tests/cli-docs-fork.tryscript.md b/packages/tbd/tests/cli-docs-fork.tryscript.md new file mode 100644 index 00000000..9de0a10e --- /dev/null +++ b/packages/tbd/tests/cli-docs-fork.tryscript.md @@ -0,0 +1,300 @@ +--- +sandbox: true +env: + NO_COLOR: '1' + FORCE_COLOR: '0' +path: + - ../dist +timeout: 60000 +patterns: + VERSION: 'v[0-9]+\.[0-9]+\.[0-9]+(-[a-z0-9.-]+)?' +before: | + git init --initial-branch=main + git config user.email "test@example.com" + git config user.name "Test User" + git config commit.gpgsign false + echo "# Test repo" > README.md + git add README.md + git commit -m "Initial commit" + tbd setup --auto --prefix=fk --quiet +--- +# tbd docs: Fork Lifecycle Golden Test + +End-to-end fork lifecycle for the `tbd docs` command group: status, fork, status again, +unfork. Serving precedence, customized-refusal, and out-of-band deletion are covered by +the `doc-fork` unit tests; these blocks pin the CLI surface. + +* * * + +## Status with nothing forked + +# Test: status reports no forks + +```console +$ tbd docs status +No docs forked into the repo. +Make some visible: tbd docs fork or tbd docs fork --all +? 0 +``` + +* * * + +## Fork a guideline + +# Test: fork python-rules writes the file and records the manifest + +```console +$ tbd docs fork python-rules +✓ Forked python-rules → docs/tbd/guidelines/python-rules.md + Regenerated docs/tbd/README.md + +Edit in place — tbd now serves your copy wherever it served upstream. +? 0 +``` + +# Test: the forked file is present in the repo + +```console +$ test -f docs/tbd/guidelines/python-rules.md && echo present +present +? 0 +``` + +# Test: the base snapshot is recorded under .tbd/doc-forks/ + +```console +$ test -f .tbd/doc-forks/base/guideline/python-rules.md && echo present +present +? 0 +``` + +* * * + +## Status shows the fork + +# Test: status lists the forked doc + +```console +$ tbd docs status +NAME KIND STATE SOURCE +python-rules guideline forked internal:guidelines/python-rules.md + +1 forked: 0 customized +? 0 +``` + +* * * + +## Forking everything available + +# Test: --dry-run previews without writing + +```console +$ tbd docs fork --all --dry-run +[DRY-RUN] Would fork [..] doc(s) into docs/tbd/ +... +No files written. Re-run without --dry-run to apply. +? 0 +``` + +* * * + +## Unfork restores upstream + +# Test: unfork an unmodified fork + +```console +$ tbd docs unfork python-rules +✓ Unforked python-rules — served from upstream again. +? 0 +``` + +# Test: status reports no forks again + +```console +$ tbd docs status +No docs forked into the repo. +... +? 0 +``` + +* * * + +## Out-of-band deletion falls back to upstream + +# Test: deleting a forked file still serves the guideline from upstream + +```console +$ tbd docs fork python-rules +✓ Forked python-rules → docs/tbd/guidelines/python-rules.md + Regenerated docs/tbd/README.md +... +? 0 +``` + +# Test: remove the forked file directly + +```console +$ rm docs/tbd/guidelines/python-rules.md +? 0 +``` + +# Test: the guideline still resolves (served from the cache) + +```console +$ tbd guidelines python-rules +... +? 0 +``` + +# Test: status reports the dangling fork as missing + +```console +$ tbd docs status +NAME KIND STATE SOURCE +python-rules guideline missing internal:guidelines/python-rules.md + +1 forked: 0 customized + +1 doc(s) missing (forked file deleted or renamed): + python-rules restore with 'tbd docs fork python-rules --force', or finalize with 'tbd docs unfork python-rules' +? 0 +``` + +* * * + +## Drift: README index, local adds, renames, and pruning + +# Test: the generated README index marks itself generated and states the layout rules + +```console +$ head -3 docs/tbd/README.md + + +# tbd Docs (forked into this repo) +? 0 +``` + +# Test: finalize the deletion from the previous section + +```console +$ tbd docs unfork python-rules +✓ Unforked python-rules — served from upstream again. +? 0 +``` + +# Test: re-fork, add a hand-authored local doc, and rename the fork + +```console +$ tbd docs fork python-rules +✓ Forked python-rules → docs/tbd/guidelines/python-rules.md + Regenerated docs/tbd/README.md +... +? 0 +``` + +```console +$ printf -- '---\ntitle: Team Rules\ndescription: Our own rules\n---\n# Team Rules\n' > docs/tbd/guidelines/team-rules.md +? 0 +``` + +```console +$ mv docs/tbd/guidelines/python-rules.md docs/tbd/guidelines/py-rules.md +? 0 +``` + +# Test: status makes the whole drift legible — local add, rename pair, and resolutions + +```console +$ tbd docs status +NAME KIND STATE SOURCE +py-rules guideline local — +python-rules guideline missing internal:guidelines/python-rules.md +team-rules guideline local — + +1 forked: 0 customized, 2 local + +1 doc(s) missing (forked file deleted or renamed): + python-rules restore with 'tbd docs fork python-rules --force', or finalize with 'tbd docs unfork python-rules' +? 0 +``` + +# Test: the local add is served and marked in the cross-kind list + +```console +$ tbd docs list --kind=guideline 2>/dev/null | grep team-rules +team-rules (66 B, ~19 tok) [local] +? 0 +``` + +# Test: tbd sync surfaces doc drift without acting on it + +```console +$ tbd sync --docs 2>&1 +✓ Docs up to date +• Docs: 1 missing (deleted/renamed) — see 'tbd docs status' +? 0 +``` + +# Test: removing the local files and finalizing the last fork prunes the fork dir + +```console +$ rm docs/tbd/guidelines/py-rules.md docs/tbd/guidelines/team-rules.md +? 0 +``` + +```console +$ tbd docs unfork python-rules +✓ Unforked python-rules — served from upstream again. +? 0 +``` + +```console +$ test ! -e docs/tbd && echo "fork dir pruned" +fork dir pruned +? 0 +``` + +* * * + +## Forked shortcuts shadow the cache (serving precedence) + +Config persists a `lookup_path` for shortcuts, so precedence must be structural: the +fork dir wins regardless of config (tbd-design.md §2.9 invariant 1). + +# Test: fork a shortcut and customize it + +```console +$ tbd docs fork review-code +✓ Forked review-code → docs/tbd/shortcuts/review-code.md + Regenerated docs/tbd/README.md +... +? 0 +``` + +```console +$ printf '\nFORK-SERVE-CHECK\n' >> docs/tbd/shortcuts/review-code.md +? 0 +``` + +# Test: the shortcut command serves the forked copy + +```console +$ tbd shortcut review-code 2>/dev/null | grep FORK-SERVE-CHECK +FORK-SERVE-CHECK +? 0 +``` + +# Test: unfork restores upstream serving + +```console +$ tbd docs unfork review-code --force +✓ Unforked review-code — served from upstream again. +? 0 +``` + +```console +$ tbd shortcut review-code 2>/dev/null | grep -c FORK-SERVE-CHECK +0 +? 1 +``` diff --git a/packages/tbd/tests/cli-docs-update.tryscript.md b/packages/tbd/tests/cli-docs-update.tryscript.md new file mode 100644 index 00000000..d1b36f06 --- /dev/null +++ b/packages/tbd/tests/cli-docs-update.tryscript.md @@ -0,0 +1,107 @@ +--- +sandbox: true +env: + NO_COLOR: '1' + FORCE_COLOR: '0' +path: + - ../dist +timeout: 60000 +before: | + git init --initial-branch=main + git config user.email "test@example.com" + git config user.name "Test User" + git config commit.gpgsign false + echo "# Test repo" > README.md + git add README.md + git commit -m "Initial commit" + tbd setup --auto --prefix=up --quiet + tbd docs fork python-rules +--- +# tbd docs update: Upgrade/Merge Golden Test + +Simulates a tbd upgrade by editing the gitignored cache copy (the upstream), then +exercises `tbd docs update` across the refresh, clean-merge, and conflict paths. +The decision table itself is covered exhaustively by the `fork-update` unit tests. + +* * * + +## Unmodified fork + upstream change → refresh + +# Test: simulate an upstream change to the cache copy + +```console +$ printf '\n\n' >> .tbd/docs/guidelines/python-rules.md +? 0 +``` + +# Test: update refreshes the unmodified fork + +```console +$ tbd docs update +Updated 1 forked doc(s): + ✓ python-rules: refreshed to upstream (was unmodified) +? 0 +``` + +* * * + +## Customized fork + overlapping upstream change → conflict + +# Test: customize the fork’s first line + +```console +$ perl -pi -e '$_ = "\n" if $. == 1' docs/tbd/guidelines/python-rules.md +? 0 +``` + +# Test: change the same line upstream + +```console +$ perl -pi -e '$_ = "\n" if $. == 1' .tbd/docs/guidelines/python-rules.md +? 0 +``` + +# Test: update skips conflicts by default and names both strategies + +```console +$ tbd docs update +... +1 doc(s) need a decision: + ⚠ python-rules: your changes conflict with upstream + re-run with one of: + tbd docs update --merge # combine, then resolve conflict markers + tbd docs update --keep-ours # keep your version, advance the fork point +? 0 +``` + +# Test: --merge writes conflict markers and flags the doc conflicted + +```console +$ tbd docs update --merge +Updated 1 forked doc(s): + ✓ python-rules: wrote merged content with conflict markers; resolve them, then it returns to 'customized' +? 0 +``` + +# Test: status reports the doc as conflicted + +```console +$ tbd docs status +NAME KIND STATE SOURCE +python-rules guideline conflicted internal:guidelines/python-rules.md + +1 forked: 1 customized, 1 conflict pending +? 0 +``` + +* * * + +## --merge and --keep-ours are mutually exclusive + +# Test: passing both is an error + +```console +$ tbd docs update --merge --keep-ours 2>&1 +[..]mutually exclusive[..] +? 1 +``` diff --git a/packages/tbd/tests/cli-format-compatibility.tryscript.md b/packages/tbd/tests/cli-format-compatibility.tryscript.md index bd3d9bd7..5e15abca 100644 --- a/packages/tbd/tests/cli-format-compatibility.tryscript.md +++ b/packages/tbd/tests/cli-format-compatibility.tryscript.md @@ -44,7 +44,7 @@ Golden tests for fail-closed format compatibility behavior. $ tbd list 2>&1 Error: This repository requires a newer version of tbd. Config format 'f99' is from a newer tbd version. -This tbd version supports up to format 'f04'. +This tbd version supports up to format 'f05'. Upgrade tbd: npm install -g get-tbd@latest ? 1 ``` diff --git a/packages/tbd/tests/cli-help-all.tryscript.md b/packages/tbd/tests/cli-help-all.tryscript.md index 6154342a..26c2ea3b 100644 --- a/packages/tbd/tests/cli-help-all.tryscript.md +++ b/packages/tbd/tests/cli-help-all.tryscript.md @@ -219,58 +219,54 @@ $ tbd import --help | grep -c "\-\-validate" ## Documentation Command Help -# Test: docs --help shows topic argument +# Test: docs --help shows the managed-docs subcommands ```console -$ tbd docs --help | grep -c "\[topic\]" -1 +$ tbd docs --help | grep -c "manual" +2 ? 0 ``` -# Test: docs --help shows section option +# Test: the old viewer flags are retired from the docs command ```console $ tbd docs --help | grep -c "\-\-section" -1 -? 0 +0 +? 1 ``` -# Test: docs --list shows slugified IDs +# Test: section listing lives on show --sections ```console -$ tbd docs --list | grep -c "id-system" +$ tbd docs show tbd-docs --sections | grep -c "id-system" 0 ? 1 ``` -# Test: docs --list shows available sections - ```console -$ tbd docs --list | grep -c "Quick Reference" +$ tbd docs show tbd-docs --sections | grep -c "Quick Reference" 1 ? 0 ``` -# Test: docs positional topic argument works +# Test: section navigation lives on show --section ```console -$ tbd docs id-system 2>&1 -Error: Section not found: "id-system" (use --list to see available sections) +$ tbd docs show tbd-docs --section id-system 2>&1 +Error: Section not found: "id-system" (use --sections to see available sections) ? 1 ``` -# Test: docs --section shows filtered content - ```console -$ tbd docs --section "ID System" 2>&1 -Error: Section not found: "ID System" (use --list to see available sections) +$ tbd docs show tbd-docs --section "ID System" 2>&1 +Error: Section not found: "ID System" (use --sections to see available sections) ? 1 ``` -# Test: docs --list --json outputs array with slugs +# Test: show --sections --json outputs array with slugs ```console -$ tbd docs --list --json +$ tbd docs show tbd-docs --sections --json [ { "title": "Key Design Features", @@ -344,10 +340,24 @@ $ tbd docs --list --json ? 0 ``` -# Test: docs shows full documentation +# Test: the manual is served by show tbd-docs and the manual alias + +```console +$ tbd docs show tbd-docs | grep -c "tbd CLI Documentation" +1 +? 0 +``` + +```console +$ tbd docs manual | grep -c "tbd CLI Documentation" +1 +? 0 +``` + +# Test: bare docs is the managed-docs overview (works before init) ```console -$ tbd docs | grep -c "tbd CLI Documentation" +$ tbd docs | grep -c "managed documentation" 1 ? 0 ``` diff --git a/packages/tbd/tests/cli-prime.tryscript.md b/packages/tbd/tests/cli-prime.tryscript.md index c713ab7a..ed934bcb 100644 --- a/packages/tbd/tests/cli-prime.tryscript.md +++ b/packages/tbd/tests/cli-prime.tryscript.md @@ -235,7 +235,7 @@ Getting Started: This initializes tbd and configures your coding agents automatically. To refresh setup (idempotent, safe anytime): `tbd setup --auto` - For interactive setup: `tbd setup --interactive` + For managed docs: `tbd docs` For more on tbd, see: https://github.com/jlevy/tbd ? 0 diff --git a/packages/tbd/tests/cli-setup-commands.tryscript.md b/packages/tbd/tests/cli-setup-commands.tryscript.md index 9b582fa0..e957375e 100644 --- a/packages/tbd/tests/cli-setup-commands.tryscript.md +++ b/packages/tbd/tests/cli-setup-commands.tryscript.md @@ -39,7 +39,6 @@ Configure tbd integration with editors and tools Options: --auto Non-interactive mode with smart defaults (for agents/scripts) - --interactive Interactive mode with prompts (for humans) --from-beads Migrate from Beads to tbd --prefix Project prefix for issue IDs (required for fresh setup) --force Allow non-recommended prefix format (not 2-8 alphabetic) @@ -66,7 +65,7 @@ Getting Started: This initializes tbd and configures your coding agents automatically. To refresh setup (idempotent, safe anytime): `tbd setup --auto` - For interactive setup: `tbd setup --interactive` + For managed docs: `tbd docs` For more on tbd, see: https://github.com/jlevy/tbd ? 0 diff --git a/packages/tbd/tests/cli-setup.tryscript.md b/packages/tbd/tests/cli-setup.tryscript.md index e3c03618..fa1bb87c 100644 --- a/packages/tbd/tests/cli-setup.tryscript.md +++ b/packages/tbd/tests/cli-setup.tryscript.md @@ -56,8 +56,8 @@ Documentation: guidelines [options] [query] Find and output coding guidelines template [options] [query] Find and output document templates closing Display the session closing protocol reminder - docs [options] [topic] Display CLI documentation (use tbd sync --docs - for doc cache sync) + docs Manage tbd-served docs: browse, fork into your + repo, and pull upstream updates design [options] [topic] Display design documentation and Beads comparison @@ -112,7 +112,7 @@ Getting Started: This initializes tbd and configures your coding agents automatically. To refresh setup (idempotent, safe anytime): `tbd setup --auto` - For interactive setup: `tbd setup --interactive` + For managed docs: `tbd docs` For more on tbd, see: https://github.com/jlevy/tbd ? 0 @@ -160,7 +160,7 @@ Getting Started: This initializes tbd and configures your coding agents automatically. To refresh setup (idempotent, safe anytime): `tbd setup --auto` - For interactive setup: `tbd setup --interactive` + For managed docs: `tbd docs` For more on tbd, see: https://github.com/jlevy/tbd ? 0 diff --git a/packages/tbd/tests/cli-shared-common-dir-worktree.tryscript.md b/packages/tbd/tests/cli-shared-common-dir-worktree.tryscript.md index 1702d077..e374cb1a 100644 --- a/packages/tbd/tests/cli-shared-common-dir-worktree.tryscript.md +++ b/packages/tbd/tests/cli-shared-common-dir-worktree.tryscript.md @@ -86,7 +86,7 @@ design: 1. An f03 per-checkout sync worktree owns `tbd-sync`. 2. A linked worktree cannot create its own per-checkout sync worktree because Git rejects the duplicate branch checkout. -3. A new tbd write migrates the repository to the f04 common-dir layout. +3. A new tbd write migrates the repository to the current (f05) common-dir layout. 4. The main checkout and linked worktree both create issues through the same shared sync worktree. @@ -119,17 +119,17 @@ own point-of-use notice (#135). ```console $ tbd create "Main checkout issue" --type=task -• tbd_format f03 → f04: .tbd/config.yml updated in this checkout. Commit on this branch or merge main to publish the format upgrade. +• tbd_format f03 → f05: .tbd/config.yml updated in this checkout. Commit on this branch or merge main to publish the format upgrade. • tbd-sync worktree was missing — auto-materialized it (fresh clone, or the worktree was removed). ✓ Created test-[SHORTID]: Main checkout issue ? 0 ``` -# Test: Top-level config was migrated to f04 with common-dir storage +# Test: Top-level config was migrated to f05 with common-dir storage ```console $ cat .tbd/config.yml -tbd_format: f04 +tbd_format: f05 tbd_version: legacy display: id_prefix: test @@ -144,11 +144,11 @@ settings: ? 0 ``` -# Test: Common-dir layout uses the same f04 format ID +# Test: Common-dir layout uses the same f05 format ID ```console $ cat "$(git rev-parse --path-format=absolute --git-common-dir)/tbd/layout.yml" -tbd_format: f04 +tbd_format: f05 sync_storage: git-common-dir-v1 data_sync_worktree: data-sync-worktree lock_profile: data-sync-v1 @@ -187,13 +187,13 @@ same git common dir # Test: Linked worktree create also succeeds -The linked worktree’s `.tbd/config.yml` is still on `f03` because the f04 bump commit +The linked worktree’s `.tbd/config.yml` is still on `f03` because the format bump commit only landed on `main`; the first mutating command in the linked checkout therefore fires the same `tbd-afjh` notice as the main checkout did before bumping in place. ```console $ (cd agent && tbd create "Linked worktree issue" --type=bug) -• tbd_format f03 → f04: .tbd/config.yml updated in this checkout. Commit on this branch or merge main to publish the format upgrade. +• tbd_format f03 → f05: .tbd/config.yml updated in this checkout. Commit on this branch or merge main to publish the format upgrade. ✓ Created test-[SHORTID]: Linked worktree issue ? 0 ``` @@ -231,15 +231,15 @@ Main checkout issue | task | open ## Older Client Compatibility Guard -# Test: An f03-era client would reject the migrated f04 repository +# Test: An f03-era client would reject the migrated f05 repository This uses the same format ordering contract as tbd itself: a client that only supports -up to f03 must fail closed when it sees the f04 common-dir layout. +up to f03 must fail closed when it sees the f05 common-dir layout. ```console $ node -e 'const fs=require("fs"); const format=fs.readFileSync(".tbd/config.yml","utf8").match(/^tbd_format: (\S+)/m)?.[1]; const supported="f03"; if (format !== undefined && format > supported) { console.error("This repository requires a newer version of tbd."); console.error("Config format '"'"'"+format+"'"'"' is from a newer tbd version."); console.error("This tbd version supports up to format '"'"'"+supported+"'"'"'."); console.error("Upgrade tbd: npm install -g get-tbd@latest"); process.exit(1); }' This repository requires a newer version of tbd. -Config format 'f04' is from a newer tbd version. +Config format 'f05' is from a newer tbd version. This tbd version supports up to format 'f03'. Upgrade tbd: npm install -g get-tbd@latest ? 1 diff --git a/packages/tbd/tests/cli-uninstall.tryscript.md b/packages/tbd/tests/cli-uninstall.tryscript.md index a5831db7..19cc0d71 100644 --- a/packages/tbd/tests/cli-uninstall.tryscript.md +++ b/packages/tbd/tests/cli-uninstall.tryscript.md @@ -60,7 +60,7 @@ Getting Started: This initializes tbd and configures your coding agents automatically. To refresh setup (idempotent, safe anytime): `tbd setup --auto` - For interactive setup: `tbd setup --interactive` + For managed docs: `tbd docs` For more on tbd, see: https://github.com/jlevy/tbd ? 0 diff --git a/packages/tbd/tests/common-dir-layout-doctor.test.ts b/packages/tbd/tests/common-dir-layout-doctor.test.ts index 637d58bc..8e8a57f1 100644 --- a/packages/tbd/tests/common-dir-layout-doctor.test.ts +++ b/packages/tbd/tests/common-dir-layout-doctor.test.ts @@ -6,7 +6,7 @@ * surfaces the future-format upgrade message instead of attempting repair. * - H3: tbd doctor surfaces an IncompatibleFormatError config as a newer-tbd * error instead of a generic "Invalid config file". - * - H1: a read command on an f04 repo with missing layout.yml regenerates the + * - H1: a read command on an f04+ repo with missing layout.yml regenerates the * layout under the shared lock and never writes a direct .tbd/data-sync/ path. */ @@ -97,39 +97,82 @@ describeUnlessWindows('common-dir layout via CLI', { timeout: 30000 }, () => { }); describe('doctor --fix (H3)', () => { - it('repairs a layout/config tbd_format mismatch under the shared lock', async () => { + it('treats an older-format layout as a pending migration and applies it on --fix', async () => { const layoutPath = join(dir, '.git', 'tbd', 'layout.yml'); const original = await readFile(layoutPath, 'utf-8'); - expect(original).toContain('tbd_format: f04'); + expect(original).toContain('tbd_format: f05'); - // Simulate a partial migration / manual edit by downgrading the layout. - await writeFile(layoutPath, original.replace('tbd_format: f04', 'tbd_format: f03')); + // A layout behind the config (downgraded to f03 here) is the normal + // mid-migration state, not a mismatch. + await writeFile(layoutPath, original.replace('tbd_format: f05', 'tbd_format: f03')); - // Plain doctor reports it as fixable. The mismatch is a ✗ finding so the - // exit is 1 (per tbd-r7rt). + // Plain doctor reports a pending migration: a warning (exit 0, so CI on + // un-migrated repos is not broken), fixable. const diagnose = runTbd(dir, ['doctor']); - expect(diagnose.status).toBe(1); + expect(diagnose.status).toBe(0); expect(diagnose.stdout + diagnose.stderr).toMatch(/Common-dir layout/i); - expect(diagnose.stdout + diagnose.stderr).toMatch(/mismatched|doctor --fix/i); + expect(diagnose.stdout + diagnose.stderr).toMatch(/migration pending|doctor --fix/i); - // doctor --fix rewrites layout.yml from config; resulting state is clean. + // doctor --fix applies the migration; layout is re-stamped to current. const fix = runTbd(dir, ['doctor', '--fix']); expect(fix.status).toBe(0); - const repaired = await readFile(layoutPath, 'utf-8'); - expect(repaired).toContain('tbd_format: f04'); + expect(await readFile(layoutPath, 'utf-8')).toContain('tbd_format: f05'); + }); + + it('does not false-positive on a healthy older repo; --fix fully migrates (never half)', async () => { + // Downgrade BOTH config and layout to f04: a consistent pre-migration + // repo, exactly what an f05 client sees before the first write. + const configPath = join(dir, '.tbd', 'config.yml'); + const layoutPath = join(dir, '.git', 'tbd', 'layout.yml'); + await writeFile( + configPath, + (await readFile(configPath, 'utf-8')).replace('tbd_format: f05', 'tbd_format: f04'), + ); + await writeFile( + layoutPath, + (await readFile(layoutPath, 'utf-8')).replace('tbd_format: f05', 'tbd_format: f04'), + ); + + // Plain doctor must NOT error on a healthy un-migrated repo (was exit 1). + const diagnose = runTbd(dir, ['doctor']); + expect(diagnose.status).toBe(0); + // The config marker on disk is untouched by a read-only doctor run. + expect(await readFile(configPath, 'utf-8')).toContain('tbd_format: f04'); + + // --fix must produce a CONSISTENT f05 repo: BOTH config and layout + // migrated, never layout-only (which would lock out older clients with + // nothing to commit). + const fix = runTbd(dir, ['doctor', '--fix']); + expect(fix.status).toBe(0); + expect(await readFile(configPath, 'utf-8')).toContain('tbd_format: f05'); + expect(await readFile(layoutPath, 'utf-8')).toContain('tbd_format: f05'); + }); + + it('rewrites a corrupt layout.yml from config on --fix', async () => { + const layoutPath = join(dir, '.git', 'tbd', 'layout.yml'); + await writeFile(layoutPath, 'this: is: not: valid: yaml: [\n'); + + // Plain doctor surfaces it as fixable with remediation (not a dead end). + const diagnose = runTbd(dir, ['doctor']); + expect(diagnose.status).toBe(1); + expect(diagnose.stdout + diagnose.stderr).toMatch(/doctor --fix|delete/i); + + const fix = runTbd(dir, ['doctor', '--fix']); + expect(fix.status).toBe(0); + expect(await readFile(layoutPath, 'utf-8')).toContain('tbd_format: f05'); }); it('surfaces future-format layout as needing a newer tbd (no fix attempted)', async () => { const layoutPath = join(dir, '.git', 'tbd', 'layout.yml'); const original = await readFile(layoutPath, 'utf-8'); - await writeFile(layoutPath, original.replace('tbd_format: f04', 'tbd_format: f99')); + await writeFile(layoutPath, original.replace('tbd_format: f05', 'tbd_format: f99')); const fix = runTbd(dir, ['doctor', '--fix']); // Future-format markers are ✗ findings: scripts/CI must see exit 1 (tbd-r7rt). expect(fix.status).toBe(1); const out = fix.stdout + fix.stderr; expect(out).toMatch(/newer tbd|f99/i); - // Layout was not silently rewritten back to f04. + // Layout was not silently rewritten back to f05. const layout = await readFile(layoutPath, 'utf-8'); expect(layout).toContain('tbd_format: f99'); }); @@ -137,7 +180,7 @@ describeUnlessWindows('common-dir layout via CLI', { timeout: 30000 }, () => { it('surfaces future-format config as a newer-tbd error in checkConfig', async () => { const configPath = join(dir, '.tbd', 'config.yml'); const original = await readFile(configPath, 'utf-8'); - await writeFile(configPath, original.replace('tbd_format: f04', 'tbd_format: f99')); + await writeFile(configPath, original.replace('tbd_format: f05', 'tbd_format: f99')); const out = runTbd(dir, ['doctor']); // Future-format config is a ✗ finding: exit 1 (tbd-r7rt). @@ -174,7 +217,7 @@ describeUnlessWindows('common-dir layout via CLI', { timeout: 30000 }, () => { expect(await exists(worktreePath)).toBe(true); expect(await exists(layoutPath)).toBe(true); const layout = await readFile(layoutPath, 'utf-8'); - expect(layout).toContain('tbd_format: f04'); + expect(layout).toContain('tbd_format: f05'); }); it('serializes concurrent doctor --fix init under the shared data-sync lock (tbd-p6zo)', async () => { @@ -207,7 +250,7 @@ describeUnlessWindows('common-dir layout via CLI', { timeout: 30000 }, () => { expect(await exists(worktreePath)).toBe(true); expect(await exists(layoutPath)).toBe(true); const layout = await readFile(layoutPath, 'utf-8'); - expect(layout).toContain('tbd_format: f04'); + expect(layout).toContain('tbd_format: f05'); const worktreeList = await gitIn(dir, 'worktree', 'list', '--porcelain'); const sharedWorktreeLines = worktreeList @@ -221,21 +264,21 @@ describeUnlessWindows('common-dir layout via CLI', { timeout: 30000 }, () => { it('prints a one-time stderr notice when this checkout migrates .tbd/config.yml to a newer tbd_format', async () => { const configPath = join(dir, '.tbd', 'config.yml'); const original = await readFile(configPath, 'utf-8'); - // The setup is f04; "downgrade" the on-disk format marker so the next mutating + // The setup is f05; "downgrade" the on-disk format marker so the next mutating // command sees a stale per-checkout config and migrates it back in place. This // matches a real sibling worktree on a branch that did not yet pick up the // main checkout's f03 → f04 commit. - await writeFile(configPath, original.replace('tbd_format: f04', 'tbd_format: f03')); + await writeFile(configPath, original.replace('tbd_format: f05', 'tbd_format: f03')); const create = runTbd(dir, ['create', 'sibling-bump probe', '--type', 'task', '--no-sync']); expect(create.status).toBe(0); // The notice goes to stderr so it cannot pollute JSON output on stdout. expect(create.stderr).toContain('tbd_format'); - expect(create.stderr).toContain('→ f04'); + expect(create.stderr).toContain('→ f05'); expect(create.stderr).toMatch(/commit on this branch or merge main/i); - // The on-disk config is now back at f04 — the migration ran. + // The on-disk config is now at f05 — the migration ran. const after = await readFile(configPath, 'utf-8'); - expect(after).toContain('tbd_format: f04'); + expect(after).toContain('tbd_format: f05'); // Second mutating call must NOT re-emit the notice: nothing left to migrate. const second = runTbd(dir, ['create', 'sibling-bump probe 2', '--type', 'task', '--no-sync']); @@ -244,6 +287,111 @@ describeUnlessWindows('common-dir layout via CLI', { timeout: 30000 }, () => { }); }); + describe('f04 → f05 upgrade (forkable-docs gate)', () => { + it('upgrades config and layout in place; the loop is revertible and repeatable', async () => { + const configPath = join(dir, '.tbd', 'config.yml'); + const layoutPath = join(dir, '.git', 'tbd', 'layout.yml'); + + const migrateOnceFromF04 = async (round: number) => { + // Rewind both files to the genuine pre-upgrade state — exactly what + // reverting the config bump commit + downgrading (or deleting) the + // machine-local layout looks like. + await writeFile( + configPath, + (await readFile(configPath, 'utf-8')).replace('tbd_format: f05', 'tbd_format: f04'), + ); + await writeFile( + layoutPath, + (await readFile(layoutPath, 'utf-8')).replace('tbd_format: f05', 'tbd_format: f04'), + ); + + // A plain data command must succeed (NOT fail with a layout/config + // mismatch), migrate the config, re-stamp the layout, and emit the + // one-time migration notice. + const create = runTbd(dir, [ + 'create', + `upgrade probe ${round}`, + '--type', + 'task', + '--no-sync', + ]); + expect(create.status).toBe(0); + expect(create.stderr).toContain('f04 → f05'); + expect(await readFile(configPath, 'utf-8')).toContain('tbd_format: f05'); + expect(await readFile(layoutPath, 'utf-8')).toContain('tbd_format: f05'); + }; + + await migrateOnceFromF04(1); + // Revert and repeat: migrating from the restored f04 state is idempotent. + await migrateOnceFromF04(2); + + // Steady state afterwards: no further migration notices. + const steady = runTbd(dir, ['create', 'steady probe', '--type', 'task', '--no-sync']); + expect(steady.status).toBe(0); + expect(steady.stderr).not.toContain('tbd_format'); + }); + + it('read commands upgrade an older layout under the lock, preserving created_at', async () => { + // Config already f05 (e.g. a teammate committed the bump) but this + // machine's layout is still f04: a read must auto-upgrade, not error. + // This is also the setup-path crash window (setup writes config before + // the layout). + const layoutPath = join(dir, '.git', 'tbd', 'layout.yml'); + const before = await readFile(layoutPath, 'utf-8'); + const createdAt = before.split('\n').find((l) => l.startsWith('created_at:')); + await writeFile(layoutPath, before.replace('tbd_format: f05', 'tbd_format: f04')); + + const list = runTbd(dir, ['list', '--json']); + expect(list.status).toBe(0); + const after = await readFile(layoutPath, 'utf-8'); + expect(after).toContain('tbd_format: f05'); + expect(after).toContain(createdAt); + }); + + it('completes an interrupted upgrade (layout already f05, config still f04)', async () => { + // The data-command crash window: ensureSharedDataSyncLayout re-stamps the + // layout BEFORE writeConfig persists the config. A crash there leaves + // layout f05 + config f04; the next command must finish the migration, + // not error on the mismatch. + const configPath = join(dir, '.tbd', 'config.yml'); + await writeFile( + configPath, + (await readFile(configPath, 'utf-8')).replace('tbd_format: f05', 'tbd_format: f04'), + ); + expect(await readFile(join(dir, '.git', 'tbd', 'layout.yml'), 'utf-8')).toContain( + 'tbd_format: f05', + ); + + const create = runTbd(dir, ['create', 'resume probe', '--type', 'task', '--no-sync']); + expect(create.status).toBe(0); + expect(create.stderr).toContain('f04 → f05'); + expect(await readFile(configPath, 'utf-8')).toContain('tbd_format: f05'); + }); + + it('aborts via the documented recipe: restore config, delete layout.yml', async () => { + // The tbd-docs "Aborting a Format Upgrade" recipe: git-restore the config + // (simulated by rewriting the stamp) and delete the machine-local layout. + const configPath = join(dir, '.tbd', 'config.yml'); + const layoutPath = join(dir, '.git', 'tbd', 'layout.yml'); + await writeFile( + configPath, + (await readFile(configPath, 'utf-8')).replace('tbd_format: f05', 'tbd_format: f04'), + ); + await rm(layoutPath); + + // The on-disk state is now exactly pre-upgrade: an f04 config and no + // layout — the state an f04-era client accepts and regenerates from. + expect(await readFile(configPath, 'utf-8')).toContain('tbd_format: f04'); + + // Re-running the upgrade from the aborted state works cleanly: the next + // command migrates the config and regenerates the layout at f05. + const list = runTbd(dir, ['list', '--json']); + expect(list.status).toBe(0); + expect(await readFile(configPath, 'utf-8')).toContain('tbd_format: f05'); + expect(await readFile(layoutPath, 'utf-8')).toContain('tbd_format: f05'); + }); + }); + describe('read fast-path (H1)', () => { it('regenerates a missing layout.yml on first read without writing direct data path', async () => { const layoutPath = join(dir, '.git', 'tbd', 'layout.yml'); @@ -258,7 +406,7 @@ describeUnlessWindows('common-dir layout via CLI', { timeout: 30000 }, () => { ); const directDataSync = join(dir, '.tbd', 'data-sync', 'issues'); - // Simulate an f04 checkout where layout.yml has not been initialized yet. + // Simulate a checkout where layout.yml has not been initialized yet. await rm(layoutPath); expect(await exists(layoutPath)).toBe(false); @@ -267,9 +415,9 @@ describeUnlessWindows('common-dir layout via CLI', { timeout: 30000 }, () => { expect(list.status).toBe(0); expect(await exists(layoutPath)).toBe(true); const layout = await readFile(layoutPath, 'utf-8'); - expect(layout).toContain('tbd_format: f04'); + expect(layout).toContain('tbd_format: f05'); - // No direct .tbd/data-sync/ leakage: f04 must fail closed, not fall back. + // No direct .tbd/data-sync/ leakage: f04+ must fail closed, not fall back. expect(await exists(sharedDataSync)).toBe(true); expect(await exists(directDataSync)).toBe(false); }); diff --git a/packages/tbd/tests/config.test.ts b/packages/tbd/tests/config.test.ts index 5dc035b3..67b522bb 100644 --- a/packages/tbd/tests/config.test.ts +++ b/packages/tbd/tests/config.test.ts @@ -90,7 +90,7 @@ describe('config operations', () => { await expect(readConfig(tempDir)).rejects.toThrow( 'This repository requires a newer version of tbd.\n' + "Config format 'f99' is from a newer tbd version.\n" + - "This tbd version supports up to format 'f04'.\n" + + "This tbd version supports up to format 'f05'.\n" + 'Upgrade tbd: npm install -g get-tbd@latest', ); }); diff --git a/packages/tbd/tests/doc-categories.test.ts b/packages/tbd/tests/doc-categories.test.ts new file mode 100644 index 00000000..2bb66371 --- /dev/null +++ b/packages/tbd/tests/doc-categories.test.ts @@ -0,0 +1,76 @@ +/** + * Tests that every bundled guideline doc declares exactly one valid category + * in its YAML frontmatter. Categories drive grouped doc listings and + * category-based selection (e.g. `tbd docs fork --category=...`), so each + * guideline must land in exactly one category. + * + * Walks the SOURCE-level docs (packages/tbd/docs/guidelines) relative to this + * test file, like doc-references.test.ts does for bundled docs. + */ + +import { describe, it, expect } from 'vitest'; +import { readFile, readdir } from 'node:fs/promises'; +import { join, dirname, extname } from 'node:path'; +import { fileURLToPath } from 'node:url'; +import matter from 'gray-matter'; + +const __dirname = dirname(fileURLToPath(import.meta.url)); +const GUIDELINES_DIR = join(__dirname, '..', 'docs', 'guidelines'); + +/** + * The allowed guideline categories. Every bundled guideline must declare + * exactly one of these in its frontmatter `category` field. + * (Mirrors the category set in the forkable-docs spec; the CLI-side constant + * lands separately.) + */ +export const GUIDELINE_CATEGORIES = [ + 'general', + 'typescript', + 'python', + 'convex', + 'electron', +] as const; + +describe('guideline doc categories', () => { + it('every bundled guideline declares exactly one valid category', async () => { + const entries = await readdir(GUIDELINES_DIR); + const files = entries.filter((f) => extname(f) === '.md').sort(); + + // Sanity check: the bundled guidelines must actually be there. + expect(files.length).toBeGreaterThan(0); + + const failures: string[] = []; + + for (const file of files) { + const raw = await readFile(join(GUIDELINES_DIR, file), 'utf-8'); + + let category: unknown; + try { + // gray-matter (js-yaml) rejects duplicated keys, so a doc declaring + // `category:` twice fails here rather than silently picking one. + const parsed = matter(raw); + category = parsed.data.category; + } catch (error) { + failures.push(`${file}: frontmatter failed to parse (${(error as Error).message})`); + continue; + } + + if (category === undefined || category === null) { + failures.push(`${file}: missing frontmatter \`category\``); + } else if (typeof category !== 'string') { + // e.g. a YAML list — must be exactly one category, as a single string + failures.push( + `${file}: \`category\` must be a single string, got ${JSON.stringify(category)}`, + ); + } else if (!(GUIDELINE_CATEGORIES as readonly string[]).includes(category)) { + failures.push( + `${file}: invalid category "${category}" (allowed: ${GUIDELINE_CATEGORIES.join(', ')})`, + ); + } + } + + if (failures.length > 0) { + expect.fail(`Guideline category failures:\n ${failures.join('\n ')}`); + } + }); +}); diff --git a/packages/tbd/tests/doc-fork.test.ts b/packages/tbd/tests/doc-fork.test.ts new file mode 100644 index 00000000..441282b8 --- /dev/null +++ b/packages/tbd/tests/doc-fork.test.ts @@ -0,0 +1,389 @@ +/** + * Tests for fork operations (forkDoc / unforkDoc / forkStatusFor) against a temp dir. + */ + +import { describe, it, expect, beforeEach, afterEach } from 'vitest'; +import { mkdtemp, rm, readFile, writeFile } from 'node:fs/promises'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; + +import { + forkDoc, + unforkDoc, + forkStatusFor, + forkFilePath, + forkRelPath, + listLocalForkFiles, + computeForkDriftSummary, + regenerateForkDirReadme, + ForkConflictError, +} from '../src/file/doc-fork.js'; +import { emptyManifest, findFork, readBaseContent } from '../src/file/fork-manifest.js'; + +import { FORK_DIR } from '../src/lib/paths.js'; +const UPSTREAM = '# Python Rules\n\nUpstream content.\n'; + +describe('forkDoc', () => { + let root: string; + beforeEach(async () => { + root = await mkdtemp(join(tmpdir(), 'tbd-doc-fork-')); + }); + afterEach(async () => { + await rm(root, { recursive: true, force: true }); + }); + + async function fork(content = UPSTREAM, force = false) { + return forkDoc({ + tbdRoot: root, + forkDir: FORK_DIR, + manifest: emptyManifest(), + kind: 'guideline', + name: 'python-rules', + source: 'internal:guidelines/python-rules.md', + content, + force, + }); + } + + it('writes the forked file, base snapshot, and manifest entry', async () => { + const result = await fork(); + expect(result.action).toBe('created'); + expect(result.relPath).toBe('docs/tbd/guidelines/python-rules.md'); + + const fileContent = await readFile( + forkFilePath(root, FORK_DIR, 'guideline', 'python-rules'), + 'utf-8', + ); + expect(fileContent).toBe(UPSTREAM); + expect(await readBaseContent(root, 'guideline', 'python-rules')).toBe(UPSTREAM); + + const entry = findFork(result.manifest, 'python-rules'); + expect(entry).toMatchObject({ + name: 'python-rules', + kind: 'guideline', + path: 'docs/tbd/guidelines/python-rules.md', + source: 'internal:guidelines/python-rules.md', + }); + }); + + it('refuses to overwrite a pre-existing non-fork file', async () => { + const abs = forkFilePath(root, FORK_DIR, 'guideline', 'python-rules'); + await import('node:fs/promises').then((fs) => + fs.mkdir(join(root, FORK_DIR, 'guidelines'), { recursive: true }), + ); + await writeFile(abs, 'pre-existing user content\n'); + + await expect(fork()).rejects.toThrow(ForkConflictError); + // --force overwrites. + const forced = await fork(UPSTREAM, true); + expect(forced.action).toBe('created'); + expect(await readFile(abs, 'utf-8')).toBe(UPSTREAM); + }); + + it('refreshes an unmodified fork to new upstream and advances the base', async () => { + const first = await fork(); + const NEW_UPSTREAM = '# Python Rules\n\nNew upstream.\n'; + const refreshed = await forkDoc({ + tbdRoot: root, + forkDir: FORK_DIR, + manifest: first.manifest, + kind: 'guideline', + name: 'python-rules', + source: 'internal:guidelines/python-rules.md', + content: NEW_UPSTREAM, + }); + expect(refreshed.action).toBe('refreshed'); + expect(await readBaseContent(root, 'guideline', 'python-rules')).toBe(NEW_UPSTREAM); + }); + + it('blocks a refresh when the fork point was set by a newer tbd (S4 version-skew)', async () => { + // The first fork records its fork point at a tbd newer than the one we now run. + const first = await forkDoc({ + tbdRoot: root, + forkDir: FORK_DIR, + manifest: emptyManifest(), + kind: 'guideline', + name: 'python-rules', + source: 'internal:guidelines/python-rules.md', + content: UPSTREAM, + tbdVersion: '0.9.0', + }); + + const OLDER_BUNDLE = '# Python Rules\n\nOlder bundled content.\n'; + const reForkOlder = (force = false) => + forkDoc({ + tbdRoot: root, + forkDir: FORK_DIR, + manifest: first.manifest, + kind: 'guideline', + name: 'python-rules', + source: 'internal:guidelines/python-rules.md', + content: OLDER_BUNDLE, + tbdVersion: '0.3.0', // older than the 0.9.0 fork point + force, + }); + + // Refreshing here would silently downgrade the doc to our older bundle — blocked. + const err = await reForkOlder().catch((e: unknown) => e); + expect(err).toBeInstanceOf(ForkConflictError); + expect((err as ForkConflictError).code).toBe('version-skew'); + // The base snapshot is untouched by the blocked refresh. + expect(await readBaseContent(root, 'guideline', 'python-rules')).toBe(UPSTREAM); + + // --force is the explicit downgrade escape hatch. + const forced = await reForkOlder(true); + expect(forced.action).toBe('refreshed'); + expect(await readBaseContent(root, 'guideline', 'python-rules')).toBe(OLDER_BUNDLE); + }); + + it('allows a refresh when the running tbd is the same or newer than the fork point', async () => { + const first = await forkDoc({ + tbdRoot: root, + forkDir: FORK_DIR, + manifest: emptyManifest(), + kind: 'guideline', + name: 'python-rules', + source: 'internal:guidelines/python-rules.md', + content: UPSTREAM, + tbdVersion: '0.9.0', + }); + const NEWER = '# Python Rules\n\nNewer upstream.\n'; + const refreshed = await forkDoc({ + tbdRoot: root, + forkDir: FORK_DIR, + manifest: first.manifest, + kind: 'guideline', + name: 'python-rules', + source: 'internal:guidelines/python-rules.md', + content: NEWER, + tbdVersion: '1.0.0', // newer than the fork point + }); + expect(refreshed.action).toBe('refreshed'); + expect(await readBaseContent(root, 'guideline', 'python-rules')).toBe(NEWER); + }); +}); + +describe('forkStatusFor', () => { + let root: string; + beforeEach(async () => { + root = await mkdtemp(join(tmpdir(), 'tbd-doc-fork-status-')); + }); + afterEach(async () => { + await rm(root, { recursive: true, force: true }); + }); + + it('reports forked / customized / stale / missing correctly', async () => { + const { manifest } = await forkDoc({ + tbdRoot: root, + forkDir: FORK_DIR, + manifest: emptyManifest(), + kind: 'guideline', + name: 'python-rules', + source: 'internal:guidelines/python-rules.md', + content: UPSTREAM, + }); + const entry = findFork(manifest, 'python-rules')!; + const abs = forkFilePath(root, FORK_DIR, 'guideline', 'python-rules'); + + // Unmodified, upstream unchanged -> forked. + expect((await forkStatusFor(root, FORK_DIR, entry, UPSTREAM)).state).toBe('forked'); + + // Upstream moved, file unmodified -> stale. + expect((await forkStatusFor(root, FORK_DIR, entry, UPSTREAM + 'more\n')).state).toBe('stale'); + + // Edit the file -> customized. + await writeFile(abs, UPSTREAM + 'my edit\n'); + const customized = await forkStatusFor(root, FORK_DIR, entry, UPSTREAM); + expect(customized.state).toBe('customized'); + expect(customized.customized).toBe(true); + + // Source gone from cache -> orphaned. + expect((await forkStatusFor(root, FORK_DIR, entry, null)).state).toBe('orphaned'); + + // Delete the file out-of-band -> missing. + await rm(abs, { force: true }); + expect((await forkStatusFor(root, FORK_DIR, entry, UPSTREAM)).state).toBe('missing'); + }); +}); + +describe('unforkDoc', () => { + let root: string; + beforeEach(async () => { + root = await mkdtemp(join(tmpdir(), 'tbd-doc-unfork-')); + }); + afterEach(async () => { + await rm(root, { recursive: true, force: true }); + }); + + async function setup() { + return forkDoc({ + tbdRoot: root, + forkDir: FORK_DIR, + manifest: emptyManifest(), + kind: 'guideline', + name: 'python-rules', + source: 'internal:guidelines/python-rules.md', + content: UPSTREAM, + }); + } + + it('removes file, base, and entry for an unmodified fork', async () => { + const { manifest } = await setup(); + const result = await unforkDoc({ + tbdRoot: root, + forkDir: FORK_DIR, + manifest, + name: 'python-rules', + }); + expect(result.fileRemoved).toBe(true); + expect(findFork(result.manifest, 'python-rules')).toBeUndefined(); + expect(await readBaseContent(root, 'guideline', 'python-rules')).toBeNull(); + }); + + it('refuses to unfork a customized doc unless forced', async () => { + const { manifest } = await setup(); + const abs = forkFilePath(root, FORK_DIR, 'guideline', 'python-rules'); + await writeFile(abs, UPSTREAM + 'edits\n'); + + await expect( + unforkDoc({ tbdRoot: root, forkDir: FORK_DIR, manifest, name: 'python-rules' }), + ).rejects.toThrow(ForkConflictError); + + const forced = await unforkDoc({ + tbdRoot: root, + forkDir: FORK_DIR, + manifest, + name: 'python-rules', + force: true, + }); + expect(forced.fileRemoved).toBe(true); + }); + + it('cleans up a missing-file entry without complaint', async () => { + const { manifest } = await setup(); + await rm(forkFilePath(root, FORK_DIR, 'guideline', 'python-rules'), { force: true }); + const result = await unforkDoc({ + tbdRoot: root, + forkDir: FORK_DIR, + manifest, + name: 'python-rules', + }); + expect(result.fileRemoved).toBe(false); + expect(findFork(result.manifest, 'python-rules')).toBeUndefined(); + }); + + it('errors when the doc is not forked', async () => { + await expect( + unforkDoc({ tbdRoot: root, forkDir: FORK_DIR, manifest: emptyManifest(), name: 'nope' }), + ).rejects.toThrow(ForkConflictError); + }); + + it('forkRelPath uses plural kind dirs', () => { + expect(forkRelPath(FORK_DIR, 'shortcut', 'review-code')).toBe( + 'docs/tbd/shortcuts/review-code.md', + ); + }); +}); + +describe('drift helpers (local files, summary, README index)', () => { + let root: string; + beforeEach(async () => { + root = await mkdtemp(join(tmpdir(), 'tbd-doc-drift-')); + // A minimal upstream cache copy so staleness can be computed. + await import('node:fs/promises').then((fs) => + fs.mkdir(join(root, '.tbd', 'docs', 'guidelines'), { recursive: true }), + ); + await writeFile(join(root, '.tbd', 'docs', 'guidelines', 'python-rules.md'), UPSTREAM); + }); + afterEach(async () => { + await rm(root, { recursive: true, force: true }); + }); + + async function forkOne() { + return forkDoc({ + tbdRoot: root, + forkDir: FORK_DIR, + manifest: emptyManifest(), + kind: 'guideline', + name: 'python-rules', + source: 'internal:guidelines/python-rules.md', + content: UPSTREAM, + }); + } + + it('listLocalForkFiles finds stray files but ignores nested folders', async () => { + const { manifest } = await forkOne(); + const dir = join(root, FORK_DIR, 'guidelines'); + await writeFile(join(dir, 'team-rules.md'), '# Team\n'); + await import('node:fs/promises').then((fs) => + fs.mkdir(join(dir, 'nested'), { recursive: true }), + ); + await writeFile(join(dir, 'nested', 'hidden.md'), '# Hidden\n'); + + const locals = await listLocalForkFiles(root, FORK_DIR, manifest); + expect(locals).toEqual([ + { kind: 'guideline', name: 'team-rules', relPath: 'docs/tbd/guidelines/team-rules.md' }, + ]); + }); + + it('computeForkDriftSummary reports stale, missing, and local counts', async () => { + const { manifest } = await forkOne(); + + // Fresh fork, cache matches base: no drift. + let s = await computeForkDriftSummary(root, FORK_DIR, manifest); + expect(s).toMatchObject({ forks: 1, stale: 0, missing: 0, local: 0 }); + + // Upstream (cache) moves: stale. + await writeFile(join(root, '.tbd', 'docs', 'guidelines', 'python-rules.md'), UPSTREAM + 'v2\n'); + s = await computeForkDriftSummary(root, FORK_DIR, manifest); + expect(s.stale).toBe(1); + + // Forked file deleted out-of-band: missing. A stray file: local. + await rm(join(root, FORK_DIR, 'guidelines', 'python-rules.md')); + await writeFile(join(root, FORK_DIR, 'guidelines', 'team-rules.md'), '# Team\n'); + s = await computeForkDriftSummary(root, FORK_DIR, manifest); + expect(s).toMatchObject({ missing: 1, local: 1 }); + }); + + it('regenerateForkDirReadme writes the index and prunes when empty', async () => { + const { manifest } = await forkOne(); + await regenerateForkDirReadme(root, FORK_DIR, manifest); + const readme = await readFile(join(root, FORK_DIR, 'README.md'), 'utf-8'); + expect(readme).toContain('DO NOT EDIT'); + expect(readme).toContain('python-rules'); + expect(readme).toContain('nested\n folders are not scanned'); + + // Unfork everything: README and empty dirs are pruned. + const { manifest: empty } = await unforkDoc({ + tbdRoot: root, + forkDir: FORK_DIR, + manifest, + name: 'python-rules', + }); + await regenerateForkDirReadme(root, FORK_DIR, empty); + await expect(readFile(join(root, FORK_DIR, 'README.md'), 'utf-8')).rejects.toThrow(); + }); + + it('sanitizes a local doc’s blurb and link target in the README index (S6)', async () => { + const { manifest } = await forkOne(); + + // A hand-authored local doc (no manifest entry, so its name is NOT + // isSafeDocName-validated): a filename with a space and a frontmatter + // description carrying markdown/HTML/link injection. + const evil = '---\n' + 'description: "Evil x [l](http://e) | `c`"\n' + '---\n# Local\n'; + await writeFile(join(root, FORK_DIR, 'guidelines', 'team rules.md'), evil); + + await regenerateForkDirReadme(root, FORK_DIR, manifest); + const readme = await readFile(join(root, FORK_DIR, 'README.md'), 'utf-8'); + + // The blurb's structure-breaking characters are stripped: no raw HTML, no + // injected link/code span, no table pipe. + expect(readme).not.toContain(''); + expect(readme).not.toContain(''); + expect(readme).not.toContain('[l]'); + expect(readme).not.toContain('`c`'); + // The link target for the spaced filename is percent-encoded so it stays a + // single valid link. + expect(readme).toContain('team%20rules.md'); + }); +}); diff --git a/packages/tbd/tests/doc-references.test.ts b/packages/tbd/tests/doc-references.test.ts index 429e58c8..91fd4a63 100644 --- a/packages/tbd/tests/doc-references.test.ts +++ b/packages/tbd/tests/doc-references.test.ts @@ -54,10 +54,11 @@ function extractDocCommands(content: string): string[] { // Normalize: join lines, collapse whitespace const normalized = content.replace(/\n/g, ' ').replace(/\s+/g, ' '); - // Match: tbd shortcut|guidelines|template|reference (not --list or flags) - // Name can be: simple (foo-bar) or prefixed (prefix:foo-bar) + // Match per-kind readers (tbd shortcut|guidelines|template ) and the + // kind-agnostic reader (tbd docs show ). Names can be simple (foo-bar) + // or prefixed (prefix:foo-bar). const pattern = - /tbd (shortcut|guidelines|template|reference) ([a-z][a-z0-9-]*(?::[a-z][a-z0-9-]*)?)/g; + /tbd (shortcut|guidelines|template|docs show) ([a-z][a-z0-9-]*(?::[a-z][a-z0-9-]*)?)/g; const commands: string[] = []; let match; @@ -71,11 +72,6 @@ function extractDocCommands(content: string): string[] { if (name.includes(':')) { continue; } - // Skip reference commands - command not yet implemented (planned for f04) - // TODO: Remove this check once tbd reference command is implemented - if (cmd === 'reference') { - continue; - } commands.push(`tbd ${cmd} ${name}`); } } diff --git a/packages/tbd/tests/docmap.test.ts b/packages/tbd/tests/docmap.test.ts new file mode 100644 index 00000000..c4fbf5cb --- /dev/null +++ b/packages/tbd/tests/docmap.test.ts @@ -0,0 +1,137 @@ +/** + * Tests for the standalone docmap/0.1 inventory module. + */ + +import { describe, it, expect } from 'vitest'; +import { + DOCMAP_VERSION, + DocMapError, + createDocMap, + parseDocMap, + findEntry, + groupByType, + filterByType, + entryKey, +} from '../src/docmap/index.js'; + +const sample = [ + { + name: 'python-rules', + type: 'guideline', + path: 'docs/tbd/guidelines/python-rules.md', + source: 'internal:guidelines/python-rules.md', + title: 'Python Coding Rules', + description: 'Type hints, docstrings, exception handling', + }, + { name: 'review-code', type: 'shortcut', source: 'internal:shortcuts/standard/review-code.md' }, + { name: 'tbd-docs', type: 'reference', source: 'internal:tbd-docs.md' }, +]; + +describe('createDocMap', () => { + it('builds a docmap with the version tag and entries', () => { + const map = createDocMap(sample, { name: 'tbd-docs' }); + expect(map.docmap).toBe(DOCMAP_VERSION); + expect(map.name).toBe('tbd-docs'); + expect(map.documents).toHaveLength(3); + }); + + it('omits name when not provided', () => { + const map = createDocMap(sample); + expect('name' in map).toBe(false); + }); + + it('preserves extension fields (e.g. tbd state, size metrics)', () => { + const map = createDocMap([ + { + name: 'x', + type: 'guideline', + source: 'internal:x.md', + state: 'customized', + stale: true, + word_count: 2400, + }, + ]); + expect(map.documents[0]).toMatchObject({ state: 'customized', stale: true, word_count: 2400 }); + }); + + it('rejects duplicate (type, name) identities', () => { + expect(() => + createDocMap([ + { name: 'dup', type: 'guideline', source: 'internal:a.md' }, + { name: 'dup', type: 'guideline', source: 'internal:b.md' }, + ]), + ).toThrow(DocMapError); + }); + + it('allows the same name under different types', () => { + const map = createDocMap([ + { name: 'typescript', type: 'guideline', source: 'internal:g/ts.md' }, + { name: 'typescript', type: 'template', source: 'internal:t/ts.md' }, + ]); + expect(map.documents).toHaveLength(2); + }); +}); + +describe('parseDocMap', () => { + it('round-trips a created docmap', () => { + const map = createDocMap(sample, { name: 'tbd-docs' }); + expect(parseDocMap(map)).toEqual(map); + }); + + it('rejects a missing/wrong version tag', () => { + expect(() => parseDocMap({ documents: [] })).toThrow(DocMapError); + expect(() => parseDocMap({ docmap: 'sitemap/1', documents: [] })).toThrow(DocMapError); + }); + + it('accepts docmap/0.x and rejects other majors', () => { + expect(parseDocMap({ docmap: 'docmap/0.2', documents: [] }).docmap).toBe('docmap/0.2'); + expect(() => parseDocMap({ docmap: 'docmap/1.0', documents: [] })).toThrow( + /supports docmap\/0/, + ); + }); + + it('rejects entries missing identity fields', () => { + expect(() => parseDocMap({ docmap: DOCMAP_VERSION, documents: [{ name: 'x' }] })).toThrow( + DocMapError, + ); + }); + + it('rejects entries without a location (path and/or source required)', () => { + expect(() => + parseDocMap({ docmap: DOCMAP_VERSION, documents: [{ name: 'x', type: 'guideline' }] }), + ).toThrow(/location/); + }); + + it('accepts and preserves unknown top-level and entry fields', () => { + const map = parseDocMap({ + docmap: DOCMAP_VERSION, + generated_by: 'tbd', + documents: [{ name: 'x', type: 'guideline', source: 'internal:x.md', state: 'forked' }], + }); + expect(map.documents[0]).toMatchObject({ state: 'forked' }); + }); +}); + +describe('queries', () => { + const map = createDocMap(sample); + + it('finds entries by name and optional type', () => { + expect(findEntry(map, 'python-rules')?.type).toBe('guideline'); + expect(findEntry(map, 'python-rules', 'shortcut')).toBeUndefined(); + }); + + it('groups entries by type', () => { + const groups = groupByType(map); + expect([...groups.keys()]).toEqual(['guideline', 'shortcut', 'reference']); + }); + + it('filters to a single type and keeps it a valid docmap', () => { + const guidelines = filterByType(map, 'guideline'); + expect(guidelines.documents).toHaveLength(1); + expect(parseDocMap(guidelines)).toEqual(guidelines); + }); + + it('entryKey is stable and identity-based', () => { + expect(entryKey({ type: 'guideline', name: 'python-rules' })).toBe('guideline:python-rules'); + }); +}); diff --git a/packages/tbd/tests/docref.test.ts b/packages/tbd/tests/docref.test.ts new file mode 100644 index 00000000..5a302586 --- /dev/null +++ b/packages/tbd/tests/docref.test.ts @@ -0,0 +1,186 @@ +/** + * Tests for the standalone docref grammar module. + * + * Acts as a spec-mirror: every supported form and normalization is exercised here, + * so the module can be lifted to its own package with its behavior pinned. + */ + +import { describe, it, expect } from 'vitest'; +import { + type DocRef, + DocRefError, + parseDocRef, + tryParseDocRef, + formatDocRef, + normalizeDocRef, + isDocRef, + docRefsEqual, +} from '../src/docref/index.js'; + +describe('parseDocRef', () => { + it('parses internal bundled docs', () => { + expect(parseDocRef('internal:guidelines/python-rules.md')).toEqual({ + kind: 'internal', + path: 'guidelines/python-rules.md', + }); + }); + + it('parses anchored local paths (./ , ../ , absolute, drive letter)', () => { + expect(parseDocRef('./docs/general/')).toEqual({ kind: 'local', path: './docs/general/' }); + expect(parseDocRef('../shared/rules.md')).toEqual({ + kind: 'local', + path: '../shared/rules.md', + }); + expect(parseDocRef('/abs/path/file.md')).toEqual({ kind: 'local', path: '/abs/path/file.md' }); + expect(parseDocRef('C:/Users/x/file.md')).toEqual({ + kind: 'local', + path: 'C:/Users/x/file.md', + }); + expect(parseDocRef('c:\\docs\\file.md')).toEqual({ kind: 'local', path: 'c:\\docs\\file.md' }); + }); + + it('rejects bare relative and home-relative paths (strict grammar)', () => { + // Bare strings are not docrefs; consumers may coerce by prepending "./" at + // their own boundary. Strict-now can loosen later; the reverse cannot. + expect(() => parseDocRef('guidelines/python-rules.md')).toThrow(DocRefError); + expect(() => parseDocRef('hello world')).toThrow(DocRefError); + expect(() => parseDocRef('~/docs/rules.md')).toThrow(/home-relative/); + }); + + it('parses plain URLs', () => { + expect(parseDocRef('https://example.com/style.md')).toEqual({ + kind: 'url', + url: 'https://example.com/style.md', + }); + }); + + it('parses github: scheme with a ref', () => { + expect(parseDocRef('github:acme/eng-docs@main//guidelines/style.md')).toEqual({ + kind: 'git', + host: 'github', + owner: 'acme', + repo: 'eng-docs', + ref: 'main', + path: 'guidelines/style.md', + }); + }); + + it('parses github: scheme without a ref', () => { + expect(parseDocRef('github:acme/eng-docs//guidelines/style.md')).toEqual({ + kind: 'git', + host: 'github', + owner: 'acme', + repo: 'eng-docs', + path: 'guidelines/style.md', + }); + }); + + it('parses a fragment on the path and keeps it separate', () => { + expect(parseDocRef('github:acme/eng-docs@main//guidelines/style.md#naming')).toEqual({ + kind: 'git', + host: 'github', + owner: 'acme', + repo: 'eng-docs', + ref: 'main', + path: 'guidelines/style.md', + fragment: 'naming', + }); + }); + + it('parses gitlab: scheme', () => { + expect(parseDocRef('gitlab:org/repo@v1.0//a/b.md').kind).toBe('git'); + }); + + it('rejects the dropped git: scheme as unknown', () => { + // v0.1 supports github:/gitlab: only; a host-bearing git: form may be added + // in a future version. + expect(() => parseDocRef('git:org/repo@sha//a/b.md')).toThrow(/unknown scheme/); + }); + + it('trims surrounding whitespace', () => { + expect(parseDocRef(' internal:a.md ')).toEqual({ kind: 'internal', path: 'a.md' }); + }); + + it('rejects empty, scheme-only, and malformed git refs', () => { + expect(() => parseDocRef('')).toThrow(DocRefError); + expect(() => parseDocRef('internal:')).toThrow(DocRefError); + expect(() => parseDocRef('github:owner-only//path.md')).toThrow(DocRefError); + expect(() => parseDocRef('github:owner/repo/no-double-slash.md')).toThrow(DocRefError); + expect(() => parseDocRef('github:owner/repo@main//')).toThrow(DocRefError); + expect(() => parseDocRef('github:owner/repo@main//#frag-only')).toThrow(DocRefError); + expect(() => parseDocRef('mailto:someone@example.com')).toThrow(DocRefError); + }); +}); + +describe('normalizeDocRef', () => { + it('normalizes a github blob URL to the github: scheme', () => { + expect(normalizeDocRef('https://github.com/o/r/blob/main/f.md')).toBe('github:o/r@main//f.md'); + }); + + it('normalizes a raw.githubusercontent.com URL to the github: scheme', () => { + expect(normalizeDocRef('https://raw.githubusercontent.com/o/r/main/dir/f.md')).toBe( + 'github:o/r@main//dir/f.md', + ); + }); + + it('normalizes a gitlab blob URL to the gitlab: scheme', () => { + expect(normalizeDocRef('https://gitlab.com/o/r/-/blob/main/f.md')).toBe( + 'gitlab:o/r@main//f.md', + ); + }); + + it('preserves URL fragments through normalization (never silently dropped)', () => { + expect(normalizeDocRef('https://github.com/o/r/blob/main/f.md#testing')).toBe( + 'github:o/r@main//f.md#testing', + ); + }); + + it('leaves non-git URLs and other forms unchanged', () => { + expect(normalizeDocRef('https://example.com/x.md')).toBe('https://example.com/x.md'); + expect(normalizeDocRef('https://example.com/x.md#frag')).toBe('https://example.com/x.md#frag'); + expect(normalizeDocRef('internal:a/b.md')).toBe('internal:a/b.md'); + }); +}); + +describe('formatDocRef round-trips', () => { + const cases: string[] = [ + 'internal:guidelines/python-rules.md', + './docs/general/', + '/abs/file.md', + 'C:/Users/x/file.md', + 'https://example.com/style.md', + 'github:acme/eng-docs@main//guidelines/style.md', + 'github:acme/eng-docs//guidelines/style.md', + 'github:acme/eng-docs@main//guidelines/style.md#naming', + 'gitlab:org/repo@v1.0//a/b.md', + ]; + it.each(cases)('parse->format is identity for %s', (input) => { + expect(formatDocRef(parseDocRef(input))).toBe(input); + }); +}); + +describe('helpers', () => { + it('isDocRef reflects validity', () => { + expect(isDocRef('internal:a.md')).toBe(true); + expect(isDocRef('mailto:x@y.com')).toBe(false); + expect(isDocRef('bare-string.md')).toBe(false); + }); + + it('tryParseDocRef returns null on invalid input', () => { + expect(tryParseDocRef('')).toBeNull(); + expect(tryParseDocRef('internal:a.md')).not.toBeNull(); + }); + + it('docRefsEqual ignores a leading ./ on local paths', () => { + const a = parseDocRef('./guidelines/x.md'); + // A consumer-coerced bare path: equal modulo the "./" anchor. + const b: DocRef = { kind: 'local', path: 'guidelines/x.md' }; + expect(docRefsEqual(a, b)).toBe(true); + }); + + it('docRefsEqual distinguishes different kinds', () => { + const a: DocRef = parseDocRef('internal:x.md'); + const b: DocRef = parseDocRef('https://e.com/x.md'); + expect(docRefsEqual(a, b)).toBe(false); + }); +}); diff --git a/packages/tbd/tests/docs-add-local-dirs-e2e.test.ts b/packages/tbd/tests/docs-add-local-dirs-e2e.test.ts new file mode 100644 index 00000000..a21f9a82 --- /dev/null +++ b/packages/tbd/tests/docs-add-local-dirs-e2e.test.ts @@ -0,0 +1,153 @@ +/** + * E2E for spec Phase 2 item 10 (tbd-ohkj): `tbd docs add ` with + * canonical-docref config recording, `docs_cache.local_dirs` serving with + * state `local`, and source grouping for sync. Network-free: uses local + * docrefs and the bundled cache only. + */ + +import { describe, it, expect, beforeEach, afterEach } from 'vitest'; +import { mkdtemp, rm, mkdir, readFile, writeFile } from 'node:fs/promises'; +import { tmpdir } from 'node:os'; +import { join, dirname } from 'node:path'; +import { fileURLToPath } from 'node:url'; +import { execSync, spawnSync } from 'node:child_process'; + +import { groupSourceEntries } from '../src/file/doc-sync.js'; + +const __dirname = dirname(fileURLToPath(import.meta.url)); + +describe('docs add + local_dirs e2e', { timeout: 120_000 }, () => { + let tempDir: string; + const tbdBin = join(__dirname, '..', 'dist', 'bin.mjs'); + + beforeEach(async () => { + tempDir = await mkdtemp(join(tmpdir(), 'tbd-add-local-')); + execSync('git init --initial-branch=main', { cwd: tempDir }); + execSync('git config user.email "t@t.t"', { cwd: tempDir }); + execSync('git config user.name "T"', { cwd: tempDir }); + execSync('git config commit.gpgsign false', { cwd: tempDir }); + runTbd(['init', '--prefix=al']); + }); + + afterEach(async () => { + await rm(tempDir, { recursive: true, force: true }); + }); + + function runTbd(args: string[]): { stdout: string; stderr: string; status: number } { + const result = spawnSync('node', [tbdBin, ...args], { + cwd: tempDir, + encoding: 'utf-8', + env: { ...process.env, FORCE_COLOR: '0', NO_COLOR: '1' }, + timeout: 60000, + }); + return { + stdout: result.stdout || '', + stderr: result.stderr || '', + status: result.status ?? 1, + }; + } + + it('adds a local docref, records the canonical docref, and resyncs from it', async () => { + await mkdir(join(tempDir, 'team'), { recursive: true }); + const src = join(tempDir, 'team', 'team-rules.md'); + await writeFile(src, '---\ntitle: Team Rules\ndescription: Ours\n---\n# Team Rules\nv1\n'); + + const add = runTbd(['docs', 'add', './team/team-rules.md', '--kind=guideline']); + expect(add.status).toBe(0); + expect(add.stdout).toContain('Adding guideline: team-rules'); + expect(add.stdout).toContain('Config updated (docs_cache.files): ./team/team-rules.md'); + + const config = await readFile(join(tempDir, '.tbd', 'config.yml'), 'utf-8'); + expect(config).toContain('guidelines/team-rules.md: ./team/team-rules.md'); + + // Source changes propagate on docs sync (local sources fetch from the repo). + await writeFile(src, '---\ntitle: Team Rules\ndescription: Ours\n---\n# Team Rules\nv2\n'); + const sync = runTbd(['docs', 'sync']); + expect(sync.status).toBe(0); + const served = runTbd(['guidelines', 'team-rules']); + expect(served.stdout).toContain('v2'); + }); + + it('rejects git docrefs without an explicit ref, and internal sources', () => { + const noRef = runTbd(['docs', 'add', 'github:o/r//docs/f.md', '--kind=guideline', '--name=x']); + expect(noRef.status).not.toBe(0); + expect(noRef.stderr + noRef.stdout).toContain('explicit ref'); + + const internal = runTbd([ + 'docs', + 'add', + 'internal:guidelines/python-rules.md', + '--kind=guideline', + '--name=x', + ]); + expect(internal.status).not.toBe(0); + }); + + it('serves local_dirs docs with state local across read surfaces, listed once', async () => { + await mkdir(join(tempDir, 'docs', 'eng'), { recursive: true }); + await writeFile( + join(tempDir, 'docs', 'eng', 'eng-notes.md'), + '---\ntitle: Eng Notes\ndescription: In-repo\n---\n# Eng Notes\nENGMARK\n', + ); + const configPath = join(tempDir, '.tbd', 'config.yml'); + const config = await readFile(configPath, 'utf-8'); + const localDirsBlock = 'docs_cache:\n local_dirs:\n - ./docs/eng/'; + await writeFile( + configPath, + config.includes('docs_cache:') + ? config.replace('docs_cache:', localDirsBlock) + : `${config}\n${localDirsBlock}\n`, + ); + + const list = runTbd(['docs', 'list', '--json']); + expect(list.status).toBe(0); + const map = JSON.parse(list.stdout) as { + documents: { name: string; state?: string; path?: string }[]; + }; + const locals = map.documents.filter((d) => d.name === 'eng-notes'); + expect(locals).toHaveLength(1); + expect(locals[0]!.state).toBe('local'); + expect(locals[0]!.path).toBe('docs/eng/eng-notes.md'); + + const served = runTbd(['guidelines', 'eng-notes']); + expect(served.status).toBe(0); + expect(served.stdout).toContain('ENGMARK'); + expect(served.stderr).toContain('(serving local doc: docs/eng/eng-notes.md)'); + + const show = runTbd(['docs', 'show', 'eng-notes']); + expect(show.status).toBe(0); + expect(show.stdout).toContain('ENGMARK'); + + // Not forkable: there is no upstream. + const fork = runTbd(['docs', 'fork', 'eng-notes']); + expect(fork.status).not.toBe(0); + }); +}); + +describe('groupSourceEntries', () => { + it('groups by git repo+ref with internal/local/url buckets', () => { + const groups = groupSourceEntries({ + 'guidelines/a.md': 'internal:guidelines/a.md', + 'guidelines/b.md': 'github:acme/docs@main//b.md', + 'guidelines/c.md': 'github:acme/docs@main//c.md', + 'guidelines/d.md': 'github:acme/docs@v2//d.md', + 'guidelines/e.md': './team/e.md', + 'guidelines/f.md': 'https://example.com/f.md', + }); + const byKey = Object.fromEntries(groups.map((g) => [g.key, g.entries.length])); + expect(byKey).toEqual({ + internal: 1, + 'github:acme/docs@main': 2, + 'github:acme/docs@v2': 1, + local: 1, + 'https://example.com': 1, + }); + const mainGroup = groups.find((g) => g.key === 'github:acme/docs@main')!; + expect(mainGroup.gitSource).toEqual({ + host: 'github', + owner: 'acme', + repo: 'docs', + ref: 'main', + }); + }); +}); diff --git a/packages/tbd/tests/doctor-fork-checks.test.ts b/packages/tbd/tests/doctor-fork-checks.test.ts new file mode 100644 index 00000000..a6c63b52 --- /dev/null +++ b/packages/tbd/tests/doctor-fork-checks.test.ts @@ -0,0 +1,219 @@ +/** + * E2e tests for the `tbd doctor` "Forked docs" check group (forkable-docs spec + * Phase 2, tbd-5xt0), run against the built CLI in a temp repo. Pins the + * contract lines from plan-2026-06-11-forkable-docs.md §`tbd doctor`: the + * healthy ✓ headline, the missing-file ⚠ + `--fix` finalize-unfork flow, + * conflict-marker detection, reserved `tbd-` names, base snapshot integrity, + * the gitignored fork dir, corrupt-manifest reporting — and zero-fork silence + * (doctor output for non-fork users must not grow). + */ + +import { describe, it, expect, beforeEach, afterEach } from 'vitest'; +import { mkdtemp, rm, readFile, writeFile, appendFile, mkdir, stat } from 'node:fs/promises'; +import { tmpdir } from 'node:os'; +import { join, dirname } from 'node:path'; +import { fileURLToPath } from 'node:url'; +import { execSync, spawnSync } from 'node:child_process'; + +const __dirname = dirname(fileURLToPath(import.meta.url)); + +describe('doctor fork checks e2e', { timeout: 120_000 }, () => { + let tempDir: string; + const tbdBin = join(__dirname, '..', 'dist', 'bin.mjs'); + + beforeEach(async () => { + tempDir = await mkdtemp(join(tmpdir(), 'tbd-doctor-fork-')); + execSync('git init --initial-branch=main', { cwd: tempDir }); + execSync('git config user.email "test@example.com"', { cwd: tempDir }); + execSync('git config user.name "Test"', { cwd: tempDir }); + execSync('git config commit.gpgsign false', { cwd: tempDir }); + runTbd(['init', '--prefix=fx']); + }); + + afterEach(async () => { + await rm(tempDir, { recursive: true, force: true }); + }); + + function runTbd(args: string[]): { stdout: string; stderr: string; status: number } { + const result = spawnSync('node', [tbdBin, ...args], { + cwd: tempDir, + encoding: 'utf-8', + env: { ...process.env, FORCE_COLOR: '0', NO_COLOR: '1' }, + timeout: 60000, + }); + return { + stdout: result.stdout || '', + stderr: result.stderr || '', + status: result.status ?? 1, + }; + } + + async function exists(path: string): Promise { + try { + await stat(path); + return true; + } catch { + return false; + } + } + + it('zero forks: doctor prints no fork lines at all', () => { + const doctor = runTbd(['doctor']); + expect(doctor.status).toBe(0); + expect(doctor.stdout).not.toContain('Forked docs'); + expect(doctor.stdout).not.toContain('Fork dir'); + expect(doctor.stdout).not.toContain('Reserved tbd- names'); + }); + + it('healthy forks: exactly one ✓ headline plus the fork-dir ✓ line', () => { + expect(runTbd(['docs', 'fork', 'python-rules', 'review-code']).status).toBe(0); + + const doctor = runTbd(['doctor']); + expect(doctor.status).toBe(0); + expect(doctor.stdout).toContain('✓ Forked docs - 2 forked, base snapshots intact'); + expect(doctor.stdout).toContain('✓ Fork dir - docs/tbd/ tracked in git (not gitignored)'); + expect(doctor.stdout.match(/Forked docs/g)).toHaveLength(1); + expect(doctor.stdout).not.toContain('Reserved tbd- names'); + }); + + it('deleted forked file: ⚠ then --fix finalizes the unfork', async () => { + expect(runTbd(['docs', 'fork', 'review-code']).status).toBe(0); + await rm(join(tempDir, 'docs', 'tbd', 'shortcuts', 'review-code.md')); + + const doctor = runTbd(['doctor']); + expect(doctor.status).toBe(0); + expect(doctor.stdout).toContain( + '⚠ Forked docs - 1 missing (review-code: forked file deleted) [fixable]', + ); + expect(doctor.stdout).toContain( + 'Run: tbd doctor --fix to finalize the unfork, or tbd docs fork --force to restore', + ); + + const fixed = runTbd(['doctor', '--fix']); + expect(fixed.status).toBe(0); + expect(fixed.stdout).toContain('⚠ Forked docs - 1 missing (review-code: forked file deleted)'); + expect(fixed.stdout).toContain( + 'Fixed: finalized unfork (removed manifest entry + base); now served from upstream', + ); + + // Manifest entry and base snapshot are gone. + const manifest = await readFile(join(tempDir, '.tbd', 'doc-forks', 'forks.yml'), 'utf-8'); + expect(manifest).not.toContain('review-code'); + expect( + await exists(join(tempDir, '.tbd', 'doc-forks', 'base', 'shortcut', 'review-code.md')), + ).toBe(false); + + // Doc is served from upstream again (no provenance note on stderr). + const show = runTbd(['docs', 'show', 'review-code']); + expect(show.status).toBe(0); + expect(show.stdout.length).toBeGreaterThan(0); + expect(show.stderr).not.toContain('serving forked copy'); + + // With the last fork finalized the fork dir is pruned — doctor goes silent. + const after = runTbd(['doctor']); + expect(after.status).toBe(0); + expect(after.stdout).not.toContain('Forked docs'); + }); + + it('orphaned entry (upstream doc gone): ⚠ then --fix removes the entry, keeps the file', async () => { + expect(runTbd(['docs', 'fork', 'python-rules']).status).toBe(0); + await rm(join(tempDir, '.tbd', 'docs', 'guidelines', 'python-rules.md')); + + const doctor = runTbd(['doctor']); + expect(doctor.status).toBe(0); + expect(doctor.stdout).toContain( + '⚠ Forked docs - 1 orphaned (python-rules: upstream doc no longer exists) [fixable]', + ); + + const fixed = runTbd(['doctor', '--fix']); + expect(fixed.status).toBe(0); + expect(fixed.stdout).toContain( + 'Fixed: removed orphaned manifest entry + base; file kept as a local doc', + ); + const manifest = await readFile(join(tempDir, '.tbd', 'doc-forks', 'forks.yml'), 'utf-8'); + expect(manifest).not.toContain('python-rules'); + expect(await exists(join(tempDir, 'docs', 'tbd', 'guidelines', 'python-rules.md'))).toBe(true); + }); + + it('unresolved conflict markers in a forked file: ⚠ with the update remediation', async () => { + expect(runTbd(['docs', 'fork', 'python-rules']).status).toBe(0); + const forkedPath = join(tempDir, 'docs', 'tbd', 'guidelines', 'python-rules.md'); + const content = await readFile(forkedPath, 'utf-8'); + // tbd's own merge-conflict labels — detection keys off these, not generic markers. + await writeFile( + forkedPath, + `${content}\n<<<<<<< ours (your fork)\nmine\n=======\ntheirs\n>>>>>>> theirs (upstream)\n`, + ); + + const doctor = runTbd(['doctor']); + expect(doctor.status).toBe(0); + expect(doctor.stdout).toContain('⚠ Forked docs - 1 unresolved merge conflict (python-rules)'); + expect(doctor.stdout).toContain( + 'Run: resolve the conflict markers, then re-run tbd docs update', + ); + }); + + it('reserved tbd-* stray file warns, even with zero forks', async () => { + await mkdir(join(tempDir, 'docs', 'tbd', 'references'), { recursive: true }); + await writeFile( + join(tempDir, 'docs', 'tbd', 'references', 'tbd-myhack.md'), + '# Hand-authored, claims the reserved prefix\n', + ); + + const doctor = runTbd(['doctor']); + expect(doctor.status).toBe(0); + expect(doctor.stdout).toContain( + '⚠ Reserved tbd- names - 1 user doc claims the reserved tbd- prefix', + ); + expect(doctor.stdout).toContain('docs/tbd/references/tbd-myhack.md'); + // No manifest entries, so no Forked docs headline and no fork-dir line. + expect(doctor.stdout).not.toContain('Forked docs'); + expect(doctor.stdout).not.toContain('Fork dir'); + }); + + it('base snapshot hash mismatch: ⚠ with re-fork/unfork remediation (no auto-fix)', async () => { + expect(runTbd(['docs', 'fork', 'python-rules']).status).toBe(0); + await appendFile( + join(tempDir, '.tbd', 'doc-forks', 'base', 'guideline', 'python-rules.md'), + '\ntampered\n', + ); + + const doctor = runTbd(['doctor']); + expect(doctor.status).toBe(0); + expect(doctor.stdout).toContain( + '⚠ Forked docs - 1 base snapshot problem (python-rules: hash mismatch)', + ); + expect(doctor.stdout).toContain( + 'Run: tbd docs fork --force to re-fork, or tbd docs unfork ', + ); + + // --fix must not touch it (re-fork vs unfork is the user's call). + const fixed = runTbd(['doctor', '--fix']); + expect(fixed.stdout).toContain('1 base snapshot problem (python-rules: hash mismatch)'); + }); + + it('gitignored fork dir warns when forks exist', async () => { + expect(runTbd(['docs', 'fork', 'python-rules']).status).toBe(0); + await writeFile(join(tempDir, '.gitignore'), 'docs/tbd/\n'); + + const doctor = runTbd(['doctor']); + expect(doctor.status).toBe(0); + expect(doctor.stdout).toContain( + '⚠ Fork dir - docs/tbd/ is gitignored — forked docs will not be committed', + ); + expect(doctor.stdout).not.toContain('tracked in git (not gitignored)'); + }); + + it('totally unparseable forks.yml is reported, not crashed on', async () => { + expect(runTbd(['docs', 'fork', 'python-rules']).status).toBe(0); + await writeFile(join(tempDir, '.tbd', 'doc-forks', 'forks.yml'), '{{{{not yaml: [\n'); + + const doctor = runTbd(['doctor']); + expect(doctor.status).toBe(0); + expect(doctor.stdout).toContain('⚠ Forked docs - fork manifest unreadable:'); + expect(doctor.stdout).toContain('(.tbd/doc-forks/forks.yml)'); + expect(doctor.stdout).toContain( + 'Fix or delete .tbd/doc-forks/forks.yml (forked files stay in place), then re-run tbd doctor', + ); + }); +}); diff --git a/packages/tbd/tests/fork-cross-platform-e2e.test.ts b/packages/tbd/tests/fork-cross-platform-e2e.test.ts new file mode 100644 index 00000000..3f82e058 --- /dev/null +++ b/packages/tbd/tests/fork-cross-platform-e2e.test.ts @@ -0,0 +1,93 @@ +/** + * Cross-platform e2e for the fork surface, run against the built CLI on every + * CI OS (unlike the tryscript goldens, which run only where a POSIX shell is + * available — tryscript executes blocks via the platform shell, cmd on + * Windows). Pins the Windows-sensitive behaviors: committed manifest paths are + * POSIX regardless of platform, and fork-dir shadowing serves the forked copy. + * See tbd-iqm1. + */ + +import { describe, it, expect, beforeEach, afterEach } from 'vitest'; +import { mkdtemp, rm, readFile, writeFile } from 'node:fs/promises'; +import { tmpdir } from 'node:os'; +import { join, dirname } from 'node:path'; +import { fileURLToPath } from 'node:url'; +import { execSync, spawnSync } from 'node:child_process'; + +const __dirname = dirname(fileURLToPath(import.meta.url)); + +describe('fork surface cross-platform e2e', { timeout: 120_000 }, () => { + let tempDir: string; + const tbdBin = join(__dirname, '..', 'dist', 'bin.mjs'); + + beforeEach(async () => { + tempDir = await mkdtemp(join(tmpdir(), 'tbd-fork-xplat-')); + execSync('git init --initial-branch=main', { cwd: tempDir }); + execSync('git config user.email "test@example.com"', { cwd: tempDir }); + execSync('git config user.name "Test"', { cwd: tempDir }); + execSync('git config commit.gpgsign false', { cwd: tempDir }); + runTbd(['init', '--prefix=fx']); + }); + + afterEach(async () => { + await rm(tempDir, { recursive: true, force: true }); + }); + + function runTbd(args: string[]): { stdout: string; stderr: string; status: number } { + const result = spawnSync('node', [tbdBin, ...args], { + cwd: tempDir, + encoding: 'utf-8', + env: { ...process.env, FORCE_COLOR: '0', NO_COLOR: '1' }, + timeout: 60000, + }); + return { + stdout: result.stdout || '', + stderr: result.stderr || '', + status: result.status ?? 1, + }; + } + + it('records POSIX manifest paths and serves the forked copy on every platform', async () => { + const fork = runTbd(['docs', 'fork', 'python-rules']); + expect(fork.status).toBe(0); + expect(fork.stdout).toContain('Forked python-rules → docs/tbd/guidelines/python-rules.md'); + + // The committed manifest must be platform-independent: forward slashes only. + const manifest = await readFile(join(tempDir, '.tbd', 'doc-forks', 'forks.yml'), 'utf-8'); + expect(manifest).toContain('path: docs/tbd/guidelines/python-rules.md'); + expect(manifest).not.toContain('\\'); + + // Customize the fork; the per-kind reader must serve the forked copy. + const forkedPath = join(tempDir, 'docs', 'tbd', 'guidelines', 'python-rules.md'); + const content = await readFile(forkedPath, 'utf-8'); + await writeFile(forkedPath, `${content}\nXPLAT-FORK-MARKER\n`); + + const served = runTbd(['guidelines', 'python-rules']); + expect(served.status).toBe(0); + expect(served.stdout).toContain('XPLAT-FORK-MARKER'); + + const list = runTbd(['docs', 'list', '--kind=guideline']); + expect(list.status).toBe(0); + expect(list.stdout).toMatch(/python-rules .*\[forked, customized\]/); + }); + + it('show serves forked copies with a POSIX provenance note; unfork restores upstream', async () => { + runTbd(['docs', 'fork', 'review-code']); + const forkedPath = join(tempDir, 'docs', 'tbd', 'shortcuts', 'review-code.md'); + const content = await readFile(forkedPath, 'utf-8'); + await writeFile(forkedPath, `${content}\nXPLAT-SHOW-MARKER\n`); + + const show = runTbd(['docs', 'show', 'review-code']); + expect(show.status).toBe(0); + expect(show.stdout).toContain('XPLAT-SHOW-MARKER'); + expect(show.stderr).toContain('(serving forked copy: docs/tbd/shortcuts/review-code.md)'); + + const unfork = runTbd(['docs', 'unfork', 'review-code', '--force']); + expect(unfork.status).toBe(0); + + const after = runTbd(['docs', 'show', 'review-code']); + expect(after.status).toBe(0); + expect(after.stdout).not.toContain('XPLAT-SHOW-MARKER'); + expect(after.stderr).not.toContain('serving forked copy'); + }); +}); diff --git a/packages/tbd/tests/fork-manifest.test.ts b/packages/tbd/tests/fork-manifest.test.ts new file mode 100644 index 00000000..587847b4 --- /dev/null +++ b/packages/tbd/tests/fork-manifest.test.ts @@ -0,0 +1,339 @@ +/** + * Tests for the fork manifest, base snapshots, hashing, and state computation. + * + * The state computation is covered as a table-driven matrix (base/file/cache + * hashes + conflicted flag -> state), per the spec's testing strategy. + */ + +import { describe, it, expect, beforeEach, afterEach, vi } from 'vitest'; +import { mkdtemp, rm, mkdir, writeFile as writeFileRaw } from 'node:fs/promises'; +import { tmpdir } from 'node:os'; +import { join, dirname } from 'node:path'; + +import { + type ForkEntry, + type ForkKind, + type ForkManifest, + type DocState, + CONFLICT_LABELS, + hashContent, + normalizeLineEndings, + hasConflictMarkers, + hasUnresolvedConflict, + isSafeDocName, + computeForkStatus, + findFork, + upsertFork, + removeFork, + emptyManifest, + readForkManifest, + writeForkManifest, + forksFilePath, + readBaseContent, + writeBaseContent, + removeBaseContent, +} from '../src/file/fork-manifest.js'; + +describe('hashContent', () => { + it('is stable across CRLF/CR/LF line endings', () => { + const lf = 'line one\nline two\n'; + const crlf = 'line one\r\nline two\r\n'; + const cr = 'line one\rline two\r'; + expect(hashContent(lf)).toBe(hashContent(crlf)); + expect(hashContent(lf)).toBe(hashContent(cr)); + }); + + it('produces a sha256: prefixed hex digest', () => { + expect(hashContent('x')).toMatch(/^sha256:[0-9a-f]{64}$/); + }); + + it('differs for different content', () => { + expect(hashContent('a')).not.toBe(hashContent('b')); + }); + + it('normalizeLineEndings collapses CRLF and CR to LF', () => { + expect(normalizeLineEndings('a\r\nb\rc')).toBe('a\nb\nc'); + }); +}); + +describe('hasConflictMarkers', () => { + it('detects a real three-marker conflict', () => { + const conflict = ['<<<<<<< ours', 'mine', '=======', 'theirs', '>>>>>>> upstream'].join('\n'); + expect(hasConflictMarkers(conflict)).toBe(true); + }); + + it('does not flag prose that merely mentions one marker', () => { + expect(hasConflictMarkers('Resolve the <<<<<<< marker by editing.')).toBe(false); + expect(hasConflictMarkers('A line of ======= in a table border.')).toBe(false); + }); +}); + +describe('isSafeDocName', () => { + it('accepts the punctuation real doc names use', () => { + for (const n of ['python-rules', 'a', 'tbd.design', 'foo_bar', 'v1', 'A1._-']) { + expect(isSafeDocName(n)).toBe(true); + } + }); + + it('rejects path-traversal and otherwise-unsafe names (S2)', () => { + // These must never round-trip through the manifest: a crafted name is how a + // doc could otherwise escape the fork dir (e.g. unfork --force deleting an + // out-of-tree file). + for (const n of [ + '', + '..', + '../victim', + '../../etc/passwd', + 'a/b', + 'foo/../bar', + '/abs', + '.hidden', + 'name.md', // the ".md" is added by path construction, not part of identity + 'has space', + 'tab\tname', + 'nul\u0000name', + ]) { + expect(isSafeDocName(n)).toBe(false); + } + }); +}); + +describe('hasUnresolvedConflict', () => { + it("is true only for tbd's own labeled markers (S7)", () => { + const tbd = + `<<<<<<< ${CONFLICT_LABELS.ours}\nmine\n=======\n` + + `theirs\n>>>>>>> ${CONFLICT_LABELS.theirs}\n`; + expect(hasUnresolvedConflict(tbd)).toBe(true); + }); + + it('is false for generic/example conflict markers and plain prose', () => { + // A git tutorial (or our own golden-testing guideline) that shows generic + // <<<<<<< HEAD markers must not be treated as an unresolved tbd conflict. + expect(hasUnresolvedConflict('<<<<<<< HEAD\na\n=======\nb\n>>>>>>> branch\n')).toBe(false); + expect(hasUnresolvedConflict('no markers at all')).toBe(false); + }); +}); + +describe('computeForkStatus matrix', () => { + const BASE = hashContent('base'); + const EDITED = hashContent('edited'); + const MOVED = hashContent('moved'); + + interface Row { + label: string; + input: Parameters[0]; + expected: Partial> & { state: DocState }; + } + + const rows: Row[] = [ + { + label: 'not in manifest, no file -> upstream', + input: { inManifest: false, forkFileExists: false }, + expected: { state: 'upstream', customized: false, stale: false }, + }, + { + label: 'file present, no manifest entry -> local', + input: { inManifest: false, forkFileExists: true }, + expected: { state: 'local' }, + }, + { + label: 'manifest entry, file deleted -> missing', + input: { inManifest: true, forkFileExists: false, baseHash: BASE }, + expected: { state: 'missing' }, + }, + { + label: 'file == base, cache == base -> forked', + input: { + inManifest: true, + forkFileExists: true, + forkHash: BASE, + baseHash: BASE, + cacheHash: BASE, + }, + expected: { state: 'forked', customized: false, stale: false }, + }, + { + label: 'file != base, cache == base -> customized (not stale)', + input: { + inManifest: true, + forkFileExists: true, + forkHash: EDITED, + baseHash: BASE, + cacheHash: BASE, + }, + expected: { state: 'customized', customized: true, stale: false }, + }, + { + label: 'file == base, cache != base -> stale (unmodified)', + input: { + inManifest: true, + forkFileExists: true, + forkHash: BASE, + baseHash: BASE, + cacheHash: MOVED, + }, + expected: { state: 'stale', customized: false, stale: true }, + }, + { + label: 'file != base, cache != base -> customized + stale', + input: { + inManifest: true, + forkFileExists: true, + forkHash: EDITED, + baseHash: BASE, + cacheHash: MOVED, + }, + expected: { state: 'customized', customized: true, stale: true }, + }, + { + label: 'cache absent (source gone) -> orphaned', + input: { + inManifest: true, + forkFileExists: true, + forkHash: BASE, + baseHash: BASE, + cacheHash: undefined, + }, + expected: { state: 'orphaned', orphaned: true, stale: false }, + }, + { + label: 'conflicted flag + markers present -> conflicted', + input: { + inManifest: true, + forkFileExists: true, + forkHash: EDITED, + baseHash: BASE, + cacheHash: MOVED, + conflictedFlag: true, + markersPresent: true, + }, + expected: { state: 'conflicted', conflicted: true }, + }, + { + label: 'conflicted flag but markers resolved -> not conflicted (customized)', + input: { + inManifest: true, + forkFileExists: true, + forkHash: EDITED, + baseHash: BASE, + cacheHash: MOVED, + conflictedFlag: true, + markersPresent: false, + }, + expected: { state: 'customized', conflicted: false }, + }, + ]; + + it.each(rows)('$label', ({ input, expected }) => { + expect(computeForkStatus(input)).toMatchObject(expected); + }); +}); + +describe('manifest helpers', () => { + const entry = (name: string, kind: ForkKind = 'guideline'): ForkEntry => ({ + name, + kind, + path: `docs/tbd/${kind}s/${name}.md`, + source: `internal:${kind}s/${name}.md`, + base_hash: hashContent(name), + }); + + it('findFork matches by name and optional kind', () => { + const m = { forks: [entry('a'), entry('b', 'shortcut')] }; + expect(findFork(m, 'a')?.kind).toBe('guideline'); + expect(findFork(m, 'b', 'guideline')).toBeUndefined(); + expect(findFork(m, 'b', 'shortcut')?.name).toBe('b'); + }); + + it('upsertFork inserts, replaces by kind+name, and keeps sorted order', () => { + let m = emptyManifest(); + m = upsertFork(m, entry('zeta')); + m = upsertFork(m, entry('alpha')); + expect(m.forks.map((f) => f.name)).toEqual(['alpha', 'zeta']); + + const replaced = { ...entry('alpha'), base_hash: 'sha256:changed' }; + m = upsertFork(m, replaced); + expect(m.forks).toHaveLength(2); + expect(findFork(m, 'alpha')?.base_hash).toBe('sha256:changed'); + }); + + it('removeFork drops the matching entry', () => { + const m = { forks: [entry('a'), entry('b')] }; + expect(removeFork(m, 'a').forks.map((f) => f.name)).toEqual(['b']); + }); +}); + +describe('filesystem round-trip', () => { + let dir: string; + beforeEach(async () => { + dir = await mkdtemp(join(tmpdir(), 'tbd-fork-manifest-')); + }); + afterEach(async () => { + await rm(dir, { recursive: true, force: true }); + }); + + it('reads an empty manifest when none exists', async () => { + expect(await readForkManifest(dir)).toEqual({ forks: [] }); + }); + + it('drops invalid/unsafe entries on read and warns, keeping good ones (S2/S8)', async () => { + const raw = [ + 'forks:', + ' - name: good-doc', + ' kind: guideline', + ' path: docs/tbd/guidelines/good-doc.md', + ' source: internal:guidelines/good-doc.md', + ' base_hash: sha256:abc', + ' - name: ../../../etc/evil', // path traversal -> dropped + ' kind: guideline', + ' path: whatever', + ' source: internal:x', + ' base_hash: sha256:def', + ' - name: bad-kind-doc', // unknown kind -> dropped + ' kind: not-a-kind', + ' path: docs/tbd/x/bad.md', + ' source: internal:x', + ' base_hash: sha256:ghi', + '', + ].join('\n'); + await mkdir(dirname(forksFilePath(dir)), { recursive: true }); + await writeFileRaw(forksFilePath(dir), raw, 'utf-8'); + + const writes: string[] = []; + const spy = vi.spyOn(process.stderr, 'write').mockImplementation((chunk: unknown) => { + writes.push(String(chunk)); + return true; + }); + const manifest = await readForkManifest(dir); + spy.mockRestore(); + + // Only the valid entry survives; the crafted/unsafe ones never reach callers. + expect(manifest.forks.map((f) => f.name)).toEqual(['good-doc']); + expect(writes.join('')).toContain('Ignored 2 invalid'); + }); + + it('round-trips a manifest through write/read', async () => { + const manifest: ForkManifest = { + forks: [ + { + name: 'python-rules', + kind: 'guideline', + path: 'docs/tbd/guidelines/python-rules.md', + source: 'internal:guidelines/python-rules.md', + base_hash: hashContent('python base'), + tbd_version: '0.2.3', + }, + ], + }; + await writeForkManifest(dir, manifest); + expect(await readForkManifest(dir)).toEqual(manifest); + }); + + it('round-trips base content verbatim and removes it', async () => { + const content = '# Python Rules\n\nbase content\n'; + await writeBaseContent(dir, 'guideline', 'python-rules', content); + expect(await readBaseContent(dir, 'guideline', 'python-rules')).toBe(content); + await removeBaseContent(dir, 'guideline', 'python-rules'); + expect(await readBaseContent(dir, 'guideline', 'python-rules')).toBeNull(); + }); +}); diff --git a/packages/tbd/tests/fork-update.test.ts b/packages/tbd/tests/fork-update.test.ts new file mode 100644 index 00000000..72d9ad33 --- /dev/null +++ b/packages/tbd/tests/fork-update.test.ts @@ -0,0 +1,317 @@ +/** + * Tests for the merge wrapper (git merge-file) and the update decision table. + */ + +import { describe, it, expect } from 'vitest'; + +import { + mergeContents, + diffContents, + updateOne, + type UpdateStrategy, +} from '../src/file/fork-update.js'; +import { type ForkEntry, hashContent, hasConflictMarkers } from '../src/file/fork-manifest.js'; + +const BASE = 'line one\nline two\nline three\n'; + +function entry(overrides: Partial = {}): ForkEntry { + return { + name: 'python-rules', + kind: 'guideline', + path: 'docs/tbd/guidelines/python-rules.md', + source: 'internal:guidelines/python-rules.md', + base_hash: hashContent(BASE), + ...overrides, + }; +} + +describe('mergeContents', () => { + it('merges non-overlapping edits cleanly', async () => { + const ours = 'line ONE\nline two\nline three\n'; // edited line 1 + const theirs = 'line one\nline two\nline THREE\n'; // edited line 3 + const result = await mergeContents(ours, BASE, theirs); + expect(result.conflicts).toBe(0); + expect(result.merged).toBe('line ONE\nline two\nline THREE\n'); + }); + + it('reports conflicts and writes markers for overlapping edits', async () => { + const ours = 'line one\nMINE\nline three\n'; + const theirs = 'line one\nTHEIRS\nline three\n'; + const result = await mergeContents(ours, BASE, theirs); + expect(result.conflicts).toBeGreaterThan(0); + expect(hasConflictMarkers(result.merged)).toBe(true); + }); + + it('merges a CRLF fork against LF base/upstream without spurious conflict (S5)', async () => { + // Without LF-normalization, a CRLF fork vs an LF base/upstream makes git + // merge-file see every line as changed and report a whole-file conflict. + // Edits are on non-adjacent lines (1 and 3) so the only thing under test is + // the line-ending mismatch, not git's adjacent-hunk conflict behavior. + const ours = 'line ONE\r\nline two\r\nline three\r\n'; // CRLF, edited line 1 + const theirs = 'line one\nline two\nline THREE\n'; // LF, edited line 3 + const result = await mergeContents(ours, BASE, theirs); + expect(result.conflicts).toBe(0); + expect(result.merged).toBe('line ONE\nline two\nline THREE\n'); + }); + + it('rejects on git merge-file errors instead of reading exit 255 as a conflict count', async () => { + // Binary input makes git merge-file refuse with exit 255. Error exits are not + // conflict counts (counts are truncated to 127); misreading one would write + // empty merged output over the user's forked file. + const binary = 'line one\u0000\nline two\n'; + await expect(mergeContents(binary, BASE, 'line one\nline two\nx\n')).rejects.toThrow( + /merge-file failed/, + ); + }); +}); + +describe('diffContents', () => { + it('returns empty for identical content', async () => { + expect(await diffContents(BASE, BASE)).toBe(''); + }); + + it('shows changed lines with the given labels', async () => { + const diff = await diffContents(BASE, 'line one\nCHANGED\nline three\n', { + left: 'upstream', + right: 'ours', + }); + expect(diff).toContain('--- upstream'); + expect(diff).toContain('+++ ours'); + expect(diff).toContain('+CHANGED'); + expect(diff).toContain('-line two'); + }); +}); + +describe('updateOne decision table', () => { + const EDITED = 'line ONE\nline two\nline three\n'; // diverges from BASE on line 1 + const UPSTREAM_NONCONFLICT = 'line one\nline two\nline THREE\n'; // line 3 + const UPSTREAM_CONFLICT = 'line ONE-theirs\nline two\nline three\n'; // line 1, conflicts with EDITED + + it('skips a missing forked file', async () => { + const r = await updateOne({ + entry: entry(), + forkContent: null, + baseContent: BASE, + upstreamContent: UPSTREAM_NONCONFLICT, + strategy: 'default', + }); + expect(r.action).toBe('skip-missing'); + }); + + it('skips an orphaned doc (upstream gone)', async () => { + const r = await updateOne({ + entry: entry(), + forkContent: BASE, + baseContent: BASE, + upstreamContent: null, + strategy: 'default', + }); + expect(r.action).toBe('skip-orphaned'); + }); + + it('skips an unresolved conflicted doc (tbd-labeled markers)', async () => { + const withMarkers = '<<<<<<< ours (your fork)\na\n=======\nb\n>>>>>>> theirs (upstream)\n'; + const r = await updateOne({ + entry: entry({ conflicted: true }), + forkContent: withMarkers, + baseContent: BASE, + upstreamContent: UPSTREAM_NONCONFLICT, + strategy: 'default', + }); + expect(r.action).toBe('skip-unresolved'); + }); + + it('does NOT treat generic/legit conflict-marker text as unresolved (S7)', async () => { + // A doc that merely contains example markers (e.g. a git tutorial) with the + // conflicted flag still set must not be stuck — only tbd's own labels count. + const generic = '<<<<<<< HEAD\na\n=======\nb\n>>>>>>> branch\n'; + const r = await updateOne({ + entry: entry({ conflicted: true }), + forkContent: generic, // unmodified vs base + baseContent: generic, + upstreamContent: generic + 'upstream line\n', // stale, non-conflicting + strategy: 'default', + }); + expect(r.action).not.toBe('skip-unresolved'); + expect(r.action).toBe('replaced'); + }); + + it('is a no-op when not stale', async () => { + const r = await updateOne({ + entry: entry(), + forkContent: EDITED, + baseContent: BASE, + upstreamContent: BASE, // upstream == base => not stale + strategy: 'default', + }); + expect(r.action).toBe('skip-not-stale'); + }); + + it('replaces an unmodified stale fork (default)', async () => { + const r = await updateOne({ + entry: entry(), + forkContent: BASE, // unmodified + baseContent: BASE, + upstreamContent: UPSTREAM_NONCONFLICT, // stale + strategy: 'default', + }); + expect(r.action).toBe('replaced'); + expect(r.newFileContent).toBe(UPSTREAM_NONCONFLICT); + expect(r.newBaseContent).toBe(UPSTREAM_NONCONFLICT); + }); + + it('applies a clean three-way merge (default)', async () => { + const r = await updateOne({ + entry: entry(), + forkContent: EDITED, + baseContent: BASE, + upstreamContent: UPSTREAM_NONCONFLICT, + strategy: 'default', + }); + expect(r.action).toBe('merged-clean'); + expect(r.newFileContent).toBe('line ONE\nline two\nline THREE\n'); + expect(r.newBaseContent).toBe(UPSTREAM_NONCONFLICT); + }); + + it('skips a conflicting merge by default and asks for a decision', async () => { + const r = await updateOne({ + entry: entry(), + forkContent: EDITED, + baseContent: BASE, + upstreamContent: UPSTREAM_CONFLICT, + strategy: 'default', + }); + expect(r.action).toBe('skip-conflict-listed'); + expect(r.needsDecision).toBe(true); + }); + + it('writes conflict markers and advances base with --merge', async () => { + const r = await updateOne({ + entry: entry(), + forkContent: EDITED, + baseContent: BASE, + upstreamContent: UPSTREAM_CONFLICT, + strategy: 'merge', + }); + expect(r.action).toBe('merged-conflict'); + expect(r.setConflicted).toBe(true); + expect(hasConflictMarkers(r.newFileContent ?? '')).toBe(true); + expect(r.newBaseContent).toBe(UPSTREAM_CONFLICT); + }); + + it('keeps the local version and advances base with --keep-ours', async () => { + const r = await updateOne({ + entry: entry(), + forkContent: EDITED, + baseContent: BASE, + upstreamContent: UPSTREAM_CONFLICT, + strategy: 'keep-ours', + }); + expect(r.action).toBe('kept'); + expect(r.newFileContent).toBeUndefined(); // file untouched + expect(r.newBaseContent).toBe(UPSTREAM_CONFLICT); + }); + + it('repairs a missing base with --keep-ours', async () => { + const r = await updateOne({ + entry: entry(), + forkContent: EDITED, + baseContent: null, // base file gone + upstreamContent: UPSTREAM_NONCONFLICT, + strategy: 'keep-ours', + }); + expect(r.action).toBe('repaired'); + expect(r.newBaseContent).toBe(UPSTREAM_NONCONFLICT); + }); + + it('skips a missing base by default and points at --keep-ours', async () => { + const r = await updateOne({ + entry: entry(), + forkContent: EDITED, + baseContent: null, + upstreamContent: UPSTREAM_NONCONFLICT, + strategy: 'default', + }); + expect(r.action).toBe('skip-no-base'); + expect(r.needsDecision).toBe(true); + }); + + it('keep-ours on an unmodified stale fork advances base without touching the file', async () => { + const strategy: UpdateStrategy = 'keep-ours'; + const r = await updateOne({ + entry: entry(), + forkContent: BASE, + baseContent: BASE, + upstreamContent: UPSTREAM_NONCONFLICT, + strategy, + }); + expect(r.action).toBe('kept'); + expect(r.newFileContent).toBeUndefined(); + expect(r.newBaseContent).toBe(UPSTREAM_NONCONFLICT); + }); +}); + +describe('version-skew guard', () => { + const UPSTREAM_OLDER = 'line one\nline two\n'; // this client's (older) bundle + + it('skips a doc whose base was advanced by a newer tbd, under every strategy', async () => { + for (const strategy of ['default', 'merge', 'keep-ours'] as const) { + const r = await updateOne({ + entry: entry({ tbd_version: '0.9.0' }), + forkContent: BASE, + baseContent: BASE, + upstreamContent: UPSTREAM_OLDER, // differs from base -> would look "stale" + strategy, + runningVersion: '0.3.0', + }); + expect(r.action).toBe('skip-newer-base'); + expect(r.newFileContent).toBeUndefined(); + expect(r.newBaseContent).toBeUndefined(); + expect(r.message).toContain('upgrade tbd'); + } + }); + + it('proceeds when the running tbd is the same or newer than the fork point', async () => { + for (const v of ['0.9.0', '1.0.0']) { + const r = await updateOne({ + entry: entry({ tbd_version: '0.9.0' }), + forkContent: BASE, + baseContent: BASE, + upstreamContent: 'line one\nline two\nline three plus\n', + strategy: 'default', + runningVersion: v, + }); + expect(r.action).toBe('replaced'); + } + }); + + it('does not guard on unparseable or absent versions', async () => { + const noEntryVersion = await updateOne({ + entry: entry(), + forkContent: BASE, + baseContent: BASE, + upstreamContent: 'changed\n', + strategy: 'default', + runningVersion: '0.3.0', + }); + expect(noEntryVersion.action).toBe('replaced'); + + const weird = await updateOne({ + entry: entry({ tbd_version: 'development' }), + forkContent: BASE, + baseContent: BASE, + upstreamContent: 'changed\n', + strategy: 'default', + runningVersion: '0.3.0', + }); + expect(weird.action).toBe('replaced'); + }); + + it('compareVersionsLoose ignores prerelease and rejects garbage', async () => { + const { compareVersionsLoose } = await import('../src/file/fork-manifest.js'); + expect(compareVersionsLoose('0.2.3-dev.333.abc', '0.2.3')).toBe(0); + expect(compareVersionsLoose('0.2.3', '0.10.0')).toBe(-1); + expect(compareVersionsLoose('1.0.0', '0.9.9')).toBe(1); + expect(compareVersionsLoose('development', '0.1.0')).toBeNull(); + }); +}); diff --git a/packages/tbd/tests/git-env-isolation-e2e.test.ts b/packages/tbd/tests/git-env-isolation-e2e.test.ts new file mode 100644 index 00000000..e2c6cc65 --- /dev/null +++ b/packages/tbd/tests/git-env-isolation-e2e.test.ts @@ -0,0 +1,124 @@ +/** + * Product-level GIT_DIR isolation (tbd-tgwi, the product half of tbd-a1lc): + * running the built tbd CLI inside a git-hook-like environment — ambient + * GIT_DIR/GIT_WORK_TREE pointing at a DIFFERENT repository — must operate on + * the repository containing cwd, warn once that the ambient value is ignored, + * and leave the other repository completely untouched. + */ + +import { describe, it, expect, beforeEach, afterEach } from 'vitest'; +import { mkdtemp, rm, writeFile, readFile } from 'node:fs/promises'; +import { tmpdir } from 'node:os'; +import { join, dirname } from 'node:path'; +import { fileURLToPath } from 'node:url'; +import { execSync, spawnSync } from 'node:child_process'; + +const __dirname = dirname(fileURLToPath(import.meta.url)); +const tbdBin = join(__dirname, '..', 'dist', 'bin.mjs'); + +function initRepo(dir: string, prefix: string): void { + execSync('git init --initial-branch=main', { cwd: dir }); + execSync('git config user.email "t@t.t"', { cwd: dir }); + execSync('git config user.name "T"', { cwd: dir }); + execSync('git config commit.gpgsign false', { cwd: dir }); + const init = spawnSync('node', [tbdBin, 'init', `--prefix=${prefix}`], { + cwd: dir, + encoding: 'utf-8', + env: { ...process.env, FORCE_COLOR: '0', NO_COLOR: '1' }, + timeout: 60000, + }); + if (init.status !== 0) throw new Error(`init failed: ${init.stderr}`); +} + +function snapshotRefs(dir: string): string { + return execSync('git for-each-ref', { cwd: dir, encoding: 'utf-8' }); +} + +describe('ambient GIT_DIR isolation (product)', { timeout: 120_000 }, () => { + let workRepo: string; + let victimRepo: string; + + beforeEach(async () => { + workRepo = await mkdtemp(join(tmpdir(), 'tbd-gitenv-work-')); + victimRepo = await mkdtemp(join(tmpdir(), 'tbd-gitenv-victim-')); + initRepo(workRepo, 'wk'); + initRepo(victimRepo, 'vc'); + await writeFile(join(victimRepo, 'real.txt'), 'real\n'); + execSync('git add -A && git commit -m real', { cwd: victimRepo }); + }); + + afterEach(async () => { + await rm(workRepo, { recursive: true, force: true }); + await rm(victimRepo, { recursive: true, force: true }); + }); + + function runTbdHostile(args: string[]): { stdout: string; stderr: string; status: number } { + // The git-hook condition: absolute GIT_DIR (and work tree) name the OTHER repo. + const result = spawnSync('node', [tbdBin, ...args], { + cwd: workRepo, + encoding: 'utf-8', + env: { + ...process.env, + FORCE_COLOR: '0', + NO_COLOR: '1', + GIT_DIR: join(victimRepo, '.git'), + GIT_WORK_TREE: victimRepo, + }, + timeout: 60000, + }); + return { + stdout: result.stdout || '', + stderr: result.stderr || '', + status: result.status ?? 1, + }; + } + + it('operates on the cwd repository, warns once, and never touches the GIT_DIR repo', async () => { + const victimRefsBefore = snapshotRefs(victimRepo); + const victimIds = await readFile(join(victimRepo, '.git', 'tbd', 'layout.yml'), 'utf-8').catch( + () => null, + ); + + const create = runTbdHostile(['create', 'isolation probe issue', '--no-sync']); + expect(create.status).toBe(0); + expect(create.stderr).toContain('ignoring inherited GIT_DIR'); + // Warned once, not per git call. + expect(create.stderr.match(/ignoring inherited GIT_DIR/g)).toHaveLength(1); + + // The issue landed in the cwd repo… + const listWork = spawnSync('node', [tbdBin, 'list'], { + cwd: workRepo, + encoding: 'utf-8', + env: { ...process.env, FORCE_COLOR: '0', NO_COLOR: '1' }, + timeout: 60000, + }); + expect(listWork.stdout).toContain('isolation probe issue'); + + // …and not in the GIT_DIR repo. + const listVictim = spawnSync('node', [tbdBin, 'list'], { + cwd: victimRepo, + encoding: 'utf-8', + env: { ...process.env, FORCE_COLOR: '0', NO_COLOR: '1' }, + timeout: 60000, + }); + expect(listVictim.stdout).not.toContain('isolation probe issue'); + + // The GIT_DIR repo is byte-identical: refs and shared tbd layout. + expect(snapshotRefs(victimRepo)).toBe(victimRefsBefore); + const victimIdsAfter = await readFile( + join(victimRepo, '.git', 'tbd', 'layout.yml'), + 'utf-8', + ).catch(() => null); + expect(victimIdsAfter).toBe(victimIds); + }); + + it('status and docs commands resolve the cwd repository under ambient GIT_DIR', () => { + const status = runTbdHostile(['status']); + expect(status.status).toBe(0); + expect(status.stdout).toContain('wk'); + + const docs = runTbdHostile(['docs']); + expect(docs.status).toBe(0); + expect(docs.stdout).toContain('managed documentation'); + }); +}); diff --git a/packages/tbd/tests/golden-output.test.ts b/packages/tbd/tests/golden-output.test.ts index 9d3c4114..2c2fddc0 100644 --- a/packages/tbd/tests/golden-output.test.ts +++ b/packages/tbd/tests/golden-output.test.ts @@ -54,49 +54,28 @@ describe('golden output tests', { timeout: isWindows ? 60000 : 15000 }, () => { runTbd(['init', '--prefix=test']); } - describe('tbd docs --all', () => { - it('shows comprehensive documentation listing', () => { + describe('tbd docs (bare overview)', () => { + it('shows the managed-docs overview with the three postures', () => { initGitAndTbd(); - const result = runTbd(['docs', '--all']); + const result = runTbd(['docs']); expect(result.status).toBe(0); - // Use inline snapshot to capture the exact format - expect(result.stdout).toMatchInlineSnapshot(` - "=== tbd Documentation Resources === - - Getting Started: - tbd Full orientation and project status - tbd prime Workflow context and guidance - tbd prime --brief Quick reference (~35 lines) - tbd --help CLI command reference - - Workflows (Shortcuts): - tbd shortcut --list List all available shortcuts - tbd shortcut new-plan-spec Plan a new feature - tbd shortcut code-review-and-commit Commit code properly - tbd shortcut create-or-update-pr-simple Create a pull request - - Guidelines (Coding Standards): - tbd guidelines --list List all available guidelines - tbd guidelines typescript-rules TypeScript best practices - tbd guidelines general-tdd-guidelines Test-driven development - tbd guidelines golden-testing-guidelines Snapshot/golden testing - - Templates: - tbd template --list List all available templates - tbd template plan-spec Feature planning template - tbd template architecture-doc Architecture document template - - Design & Reference: - tbd docs --list List documentation sections - tbd design tbd design document - tbd closing Session closing protocol - - Quick Tips: - - Run tbd ready to see what issues are available to work on - - Run tbd shortcut to get step-by-step instructions - - Run tbd guidelines to get coding standards - - Always run tbd sync at the end of a session + // The bundled-doc count grows over time; mask it so the golden pins + // the format, not the inventory size. + const stdout = result.stdout.replace(/\d+ docs available/, '[N] docs available'); + expect(stdout).toMatchInlineSnapshot(` + "tbd docs — managed documentation + + [N] docs available in the cache (.tbd/docs/, gitignored); none forked into the repo. + Guidelines are active from the cache. Three postures, all serving the same docs: + + Hidden (default): keep the cache as-is — zero repo footprint + Curated: tbd docs fork [...] fork chosen docs into docs/tbd/ + tbd docs fork --category= (general, typescript, python, convex, electron) + Everything: tbd docs fork --all all docs, visible and editable + + Browse / read: tbd docs list / tbd docs show + Learn more: tbd docs show tbd-docs (the manual; alias: tbd docs manual) " `); }); @@ -112,6 +91,22 @@ describe('golden output tests', { timeout: isWindows ? 60000 : 15000 }, () => { expect(result.status).toBe(0); + // The Docs summary precedes the completion banner: the zero-fork + // three-posture menu, sharing wording with the bare `tbd docs` overview + // (count masked — the bundled-doc inventory grows over time). + const masked = result.stdout.replace(/Docs: \d+ docs available/, 'Docs: [N] docs available'); + expect(masked).toContain( + [ + 'Docs: [N] docs available in the cache (.tbd/docs/, gitignored); none forked into the repo.', + ' Guidelines are active from the cache. Three postures, all serving the same docs:', + ' Hidden (default): keep the cache as-is — zero repo footprint', + ' Curated: tbd docs fork [...] fork chosen docs into docs/tbd/', + ' tbd docs fork --category= (general, typescript, python, convex, electron)', + ' Everything: tbd docs fork --all all docs, visible and editable', + ' Browse / read: tbd docs list / tbd docs show ', + ].join('\n'), + ); + // Verify What's Next section uses "what you can say" framing expect(result.stdout).toContain("WHAT'S NEXT"); expect(result.stdout).toContain('Try saying things like:'); @@ -215,11 +210,17 @@ describe('golden output tests', { timeout: isWindows ? 60000 : 15000 }, () => { expect(tsResult.stdout).toContain('typescript-rules'); expect(tsResult.stdout).not.toContain('python-rules'); - // Testing category should include tdd guidelines - const testingResult = runTbd(['guidelines', '--list', '--category', 'testing']); - expect(testingResult.status).toBe(0); - expect(testingResult.stdout).toContain('general-tdd-guidelines'); - expect(testingResult.stdout).not.toContain('typescript-rules'); + // The old name-inferred 'testing' category is retired: declared + // frontmatter categories only, with a clear error for unknown values. + const retired = runTbd(['guidelines', '--list', '--category', 'testing']); + expect(retired.status).not.toBe(0); + expect(retired.stderr + retired.stdout).toContain('Unknown category'); + + // TDD/testing guidelines live in 'general' (declared in frontmatter). + const generalResult = runTbd(['guidelines', '--list', '--category', 'general']); + expect(generalResult.status).toBe(0); + expect(generalResult.stdout).toContain('general-tdd-guidelines'); + expect(generalResult.stdout).not.toContain('typescript-rules'); }); }); diff --git a/packages/tbd/tests/scrub-git-env.ts b/packages/tbd/tests/scrub-git-env.ts new file mode 100644 index 00000000..280136fd --- /dev/null +++ b/packages/tbd/tests/scrub-git-env.ts @@ -0,0 +1,24 @@ +/** + * Vitest per-worker setup: strip inherited git-location environment variables + * before any test spawns a subprocess. + * + * Tests create throwaway git repos in temp dirs and run `git` / the tbd CLI with + * `cwd` set there, relying on git and tbd discovering the repo from `cwd`. But git + * exports GIT_DIR (and friends) into hook environments: running `git push` from a + * linked worktree invokes the pre-push test suite with GIT_DIR pointing at the + * real repository's gitdir. An absolute GIT_DIR overrides cwd-based discovery, so a + * fixture's `git init` / `commit` / `checkout -b` and tbd's `--git-common-dir` + * resolution would all operate on the REAL repo — rewriting its branches and + * corrupting its data-sync worktree. (Running vitest directly never set GIT_DIR, + * which is why only hook-invoked runs were affected.) + * + * Deleting these here, once per worker before any test, makes every + * `{ ...process.env }` spawn in the suite safe regardless of how the runner was + * invoked. The pre-push hook also scrubs them (see lefthook.yml) as defense in + * depth; either layer alone closes the hole, but both are cheap. + */ +import { GIT_LOCATION_VARS } from '../src/lib/git-env.js'; + +for (const name of GIT_LOCATION_VARS) { + delete process.env[name]; +} diff --git a/packages/tbd/tests/setup-flows.test.ts b/packages/tbd/tests/setup-flows.test.ts index 5cd67958..ed0608ef 100644 --- a/packages/tbd/tests/setup-flows.test.ts +++ b/packages/tbd/tests/setup-flows.test.ts @@ -62,7 +62,17 @@ describe('setup flows', { timeout: setupFlowTestTimeout }, () => { const result = runTbd(['setup']); expect(result.stdout).toContain('--auto'); - expect(result.stdout).toContain('--interactive'); + expect(result.stdout).toContain('--from-beads'); + // The --interactive flag was removed (never had prompts; agents are the operators). + expect(result.stdout).not.toContain('--interactive'); + }); + + it('rejects the removed --interactive flag', () => { + initGitRepo(); + const result = runTbd(['setup', '--interactive']); + + expect(result.status).not.toBe(0); + expect(result.stderr).toContain("unknown option '--interactive'"); }); }); @@ -161,7 +171,7 @@ describe('setup flows', { timeout: setupFlowTestTimeout }, () => { expect(result.status).toBe(0); const agents = await readFile(join(tempDir, 'AGENTS.md'), 'utf-8'); - expect(agents).toContain(''); + expect(agents).toContain(''); expect(agents).toContain('tbd prime'); const block = agents.slice( @@ -259,7 +269,7 @@ describe('setup flows', { timeout: setupFlowTestTimeout }, () => { const agents = await readFile(join(tempDir, 'AGENTS.md'), 'utf-8'); // Upgraded to the versioned compact block... - expect(agents).toContain('format=f04'); + expect(agents).toContain('format=f05'); // ...while preserving user content outside the managed region. expect(agents).toContain('## My Notes'); expect(agents).toContain('Keep me.'); @@ -351,6 +361,64 @@ describe('setup flows', { timeout: setupFlowTestTimeout }, () => { }); }); + describe('docs summary', () => { + // The zero-fork menu, verbatim. Wording is shared with the bare `tbd docs` + // overview (src/cli/lib/docs-menu.ts); the count is masked because the + // bundled-doc inventory grows over time. + const zeroForkMenu = [ + 'Docs: [N] docs available in the cache (.tbd/docs/, gitignored); none forked into the repo.', + ' Guidelines are active from the cache. Three postures, all serving the same docs:', + ' Hidden (default): keep the cache as-is — zero repo footprint', + ' Curated: tbd docs fork [...] fork chosen docs into docs/tbd/', + ' tbd docs fork --category= (general, typescript, python, convex, electron)', + ' Everything: tbd docs fork --all all docs, visible and editable', + ' Browse / read: tbd docs list / tbd docs show ', + ].join('\n'); + + it('shows the three-posture menu when nothing is forked', () => { + initGitRepo(); + + const result = runTbd(['setup', '--auto', '--prefix=test']); + expect(result.status).toBe(0); + + const masked = result.stdout.replace(/Docs: \d+ docs available/, 'Docs: [N] docs available'); + expect(masked).toContain(zeroForkMenu); + }); + + it('reports fork count and pending updates instead of the menu', async () => { + initGitRepo(); + runTbd(['setup', '--auto', '--prefix=test']); + + const forkResult = runTbd(['docs', 'fork', 'python-rules']); + expect(forkResult.status).toBe(0); + + // Forks current: one line, no update nudge, no posture menu. + const current = runTbd(['setup', '--auto']); + expect(current.status).toBe(0); + expect(current.stdout).toContain('Docs: 1 forked into docs/tbd/.'); + expect(current.stdout).not.toContain('upstream updates'); + expect(current.stdout).not.toContain('Three postures'); + + // Mark the fork stale (as after a tbd upgrade): its recorded base no + // longer matches the cache content. + const manifestPath = join(tempDir, '.tbd', 'doc-forks', 'forks.yml'); + const manifest = await readFile(manifestPath, 'utf-8'); + await writeFile( + manifestPath, + manifest.replace(/base_hash: sha256:[0-9a-f]+/, `base_hash: sha256:${'0'.repeat(64)}`), + ); + + const stale = runTbd(['setup', '--auto']); + expect(stale.status).toBe(0); + expect(stale.stdout).toContain( + "Docs: 1 forked into docs/tbd/. 1 have upstream updates — run 'tbd docs update'.", + ); + + // Reporting only: setup must never write the fork dir or its manifest. + expect(await readFile(manifestPath, 'utf-8')).toContain(`sha256:${'0'.repeat(64)}`); + }); + }); + describe('beads migration', { timeout: isWindows ? 60000 : 15000 }, () => { it('detects beads and offers migration', async () => { initGitRepo(); diff --git a/packages/tbd/tests/tbd-format.test.ts b/packages/tbd/tests/tbd-format.test.ts index 365d8ad3..5947f659 100644 --- a/packages/tbd/tests/tbd-format.test.ts +++ b/packages/tbd/tests/tbd-format.test.ts @@ -20,7 +20,7 @@ import { describe('tbd-format', () => { describe('constants', () => { it('has current format', () => { - expect(CURRENT_FORMAT).toBe('f04'); + expect(CURRENT_FORMAT).toBe('f05'); }); it('has initial format', () => { @@ -32,6 +32,7 @@ describe('tbd-format', () => { expect(FORMAT_HISTORY.f02).toBeDefined(); expect(FORMAT_HISTORY.f03).toBeDefined(); expect(FORMAT_HISTORY.f04).toBeDefined(); + expect(FORMAT_HISTORY.f05).toBeDefined(); }); }); @@ -78,7 +79,7 @@ describe('tbd-format', () => { }); describe('migrateToLatest', () => { - it('migrates f01 to f04 through all format steps', () => { + it('migrates f01 to f05 through all format steps', () => { const config: RawConfig = { tbd_version: '0.1.0', display: { id_prefix: 'test' }, @@ -89,9 +90,9 @@ describe('tbd-format', () => { const result = migrateToLatest(config); expect(result.fromFormat).toBe('f01'); - expect(result.toFormat).toBe('f04'); + expect(result.toFormat).toBe('f05'); expect(result.changed).toBe(true); - expect(result.config.tbd_format).toBe('f04'); + expect(result.config.tbd_format).toBe('f05'); expect(result.config.sync?.storage).toBe('git-common-dir-v1'); expect(result.config.settings?.doc_auto_sync_hours).toBe(24); expect(result.changes).toContain('Added tbd_format: f02'); @@ -99,9 +100,10 @@ describe('tbd-format', () => { expect(result.changes).toContain('Updated tbd_format: f03'); expect(result.changes).toContain('Updated tbd_format: f04'); expect(result.changes).toContain('Added sync.storage: git-common-dir-v1'); + expect(result.changes).toContain('Updated tbd_format: f05'); }); - it('migrates f02 to f04', () => { + it('migrates f02 to f05', () => { const config: RawConfig = { tbd_format: 'f02', tbd_version: '0.1.5', @@ -114,9 +116,9 @@ describe('tbd-format', () => { const result = migrateToLatest(config); expect(result.fromFormat).toBe('f02'); - expect(result.toFormat).toBe('f04'); + expect(result.toFormat).toBe('f05'); expect(result.changed).toBe(true); - expect(result.config.tbd_format).toBe('f04'); + expect(result.config.tbd_format).toBe('f05'); expect(result.config.sync?.storage).toBe('git-common-dir-v1'); // doc_cache moved to docs_cache.files expect(result.config.doc_cache).toBeUndefined(); @@ -133,8 +135,8 @@ describe('tbd-format', () => { it('does not modify already current config', () => { const config: RawConfig = { - tbd_format: 'f04', - tbd_version: '0.2.0', + tbd_format: 'f05', + tbd_version: '0.3.0', sync: { branch: 'tbd-sync', remote: 'origin', storage: 'git-common-dir-v1' }, display: { id_prefix: 'test' }, settings: { auto_sync: false, doc_auto_sync_hours: 12 }, @@ -146,15 +148,15 @@ describe('tbd-format', () => { const result = migrateToLatest(config); - expect(result.fromFormat).toBe('f04'); - expect(result.toFormat).toBe('f04'); + expect(result.fromFormat).toBe('f05'); + expect(result.toFormat).toBe('f05'); expect(result.changed).toBe(false); expect(result.changes).toHaveLength(0); expect(result.config.settings?.doc_auto_sync_hours).toBe(12); expect(result.config.sync?.storage).toBe('git-common-dir-v1'); }); - it('migrates f03 to f04 by adding sync storage marker', () => { + it('migrates f03 through f04 (sync storage marker) to f05', () => { const config: RawConfig = { tbd_format: 'f03', tbd_version: '0.1.6', @@ -166,9 +168,9 @@ describe('tbd-format', () => { const result = migrateToLatest(config); expect(result.fromFormat).toBe('f03'); - expect(result.toFormat).toBe('f04'); + expect(result.toFormat).toBe('f05'); expect(result.changed).toBe(true); - expect(result.config.tbd_format).toBe('f04'); + expect(result.config.tbd_format).toBe('f05'); expect(result.config.sync).toEqual({ branch: 'custom-sync', remote: 'upstream', @@ -176,6 +178,29 @@ describe('tbd-format', () => { }); }); + it('migrates f04 to f05 as a metadata-only stamp', () => { + const config: RawConfig = { + tbd_format: 'f04', + tbd_version: '0.2.3', + display: { id_prefix: 'test' }, + sync: { branch: 'tbd-sync', remote: 'origin', storage: 'git-common-dir-v1' }, + settings: { auto_sync: false, doc_auto_sync_hours: 24 }, + docs_cache: { + files: { 'guidelines/x.md': 'internal:guidelines/x.md' }, + lookup_path: ['.tbd/docs/shortcuts/system'], + }, + }; + + const result = migrateToLatest(config); + + expect(result.fromFormat).toBe('f04'); + expect(result.toFormat).toBe('f05'); + expect(result.changed).toBe(true); + expect(result.changes).toEqual(['Updated tbd_format: f05']); + // Stamp only: every other field is preserved verbatim. + expect(result.config).toEqual({ ...config, tbd_format: 'f05' }); + }); + it('preserves existing settings when migrating', () => { const config: RawConfig = { tbd_version: '0.1.0', @@ -210,6 +235,10 @@ describe('tbd-format', () => { expect(isCompatibleFormat('f04')).toBe(true); }); + it('returns true for f05', () => { + expect(isCompatibleFormat('f05')).toBe(true); + }); + it('returns false for unknown future format', () => { expect(isCompatibleFormat('f99')).toBe(false); }); @@ -220,9 +249,14 @@ describe('tbd-format', () => { expect(isFormatCompatibleWithSupported('f04', 'f03')).toBe(false); }); + it('models old f04 clients rejecting f05 repositories (the forkable-docs gate)', () => { + expect(isFormatCompatibleWithSupported('f05', 'f04')).toBe(false); + }); + it('allows old clients to read older formats they know how to migrate', () => { expect(isFormatCompatibleWithSupported('f01', 'f03')).toBe(true); expect(isFormatCompatibleWithSupported('f03', 'f03')).toBe(true); + expect(isFormatCompatibleWithSupported('f04', 'f05')).toBe(true); }); }); @@ -238,23 +272,31 @@ describe('tbd-format', () => { }); describe('describeMigration', () => { - it('describes f01 migration (three steps)', () => { + it('describes f01 migration (four steps)', () => { const descriptions = describeMigration('f01'); - expect(descriptions).toHaveLength(3); + expect(descriptions).toHaveLength(4); expect(descriptions[0]).toContain('f01 → f02'); expect(descriptions[1]).toContain('f02 → f03'); expect(descriptions[2]).toContain('f03 → f04'); + expect(descriptions[3]).toContain('f04 → f05'); }); it('describes f02 migration', () => { const descriptions = describeMigration('f02'); - expect(descriptions).toHaveLength(2); + expect(descriptions).toHaveLength(3); expect(descriptions[0]).toContain('f02 → f03'); expect(descriptions[1]).toContain('f03 → f04'); + expect(descriptions[2]).toContain('f04 → f05'); }); - it('returns empty for current format', () => { + it('describes f04 migration (one step)', () => { const descriptions = describeMigration('f04'); + expect(descriptions).toHaveLength(1); + expect(descriptions[0]).toContain('f04 → f05'); + }); + + it('returns empty for current format', () => { + const descriptions = describeMigration('f05'); expect(descriptions).toHaveLength(0); }); }); diff --git a/packages/tbd/vitest.config.ts b/packages/tbd/vitest.config.ts index 9c2e74a4..1d89da55 100644 --- a/packages/tbd/vitest.config.ts +++ b/packages/tbd/vitest.config.ts @@ -10,6 +10,11 @@ export default defineConfig({ test: { include: ['tests/**/*.test.ts'], globalSetup: ['tests/global-setup.ts'], + // Strip inherited GIT_DIR (and friends) in every worker before tests spawn + // subprocesses, so an ambient git env (e.g. from a pre-push hook run in a + // linked worktree) cannot redirect fixture git/tbd onto the real repo. + // See tests/scrub-git-env.ts and the tbd-a1lc incident. + setupFiles: ['tests/scrub-git-env.ts'], hookTimeout: isWindows ? 30000 : 10000, coverage: { provider: 'v8', diff --git a/scripts/scrub-git-env.mjs b/scripts/scrub-git-env.mjs new file mode 100644 index 00000000..d9589b95 --- /dev/null +++ b/scripts/scrub-git-env.mjs @@ -0,0 +1,47 @@ +#!/usr/bin/env node +/* global process, console */ +/** + * Run a command line with git's repo-location environment variables removed. + * + * Git exports GIT_DIR (and related vars) into hook environments — notably, + * `git push` from a linked worktree runs the pre-push hook with GIT_DIR pointing + * at the real repository's gitdir. Any git or tbd subprocess spawned by the + * hook's commands inherits it, and an absolute GIT_DIR overrides cwd-based repo + * discovery: test fixtures that `git init` in temp dirs then commit, branch, and + * corrupt data against the REAL repository instead. This destroyed local refs + * and the tbd data-sync mappings in one incident (tbd-a1lc). + * + * Usage (from lefthook.yml): node scripts/scrub-git-env.mjs + * Arguments are joined with spaces and run through the platform shell — the same + * semantics as lefthook's own `run:` line, minus the git location env. + * + * The vitest suite also self-scrubs (packages/tbd/tests/scrub-git-env.ts); + * this wrapper protects every other current or future hook command the same way. + */ + +import { spawnSync } from 'node:child_process'; + +const GIT_LOCATION_VARS = [ + 'GIT_DIR', + 'GIT_WORK_TREE', + 'GIT_INDEX_FILE', + 'GIT_COMMON_DIR', + 'GIT_OBJECT_DIRECTORY', + 'GIT_ALTERNATE_OBJECT_DIRECTORIES', + 'GIT_PREFIX', + 'GIT_NAMESPACE', +]; + +const commandLine = process.argv.slice(2).join(' '); +if (!commandLine) { + console.error('usage: scrub-git-env.mjs '); + process.exit(2); +} + +const env = { ...process.env }; +for (const name of GIT_LOCATION_VARS) { + delete env[name]; +} + +const result = spawnSync(commandLine, { stdio: 'inherit', env, shell: true }); +process.exit(result.status ?? 1); diff --git a/skills/tbd/SKILL.md b/skills/tbd/SKILL.md index f20e6ed6..1d647d52 100644 --- a/skills/tbd/SKILL.md +++ b/skills/tbd/SKILL.md @@ -31,12 +31,15 @@ allowed-tools: Bash(tbd:*), Read, Write ## Installation ```bash -npm install -g get-tbd@latest +npm install -g get-tbd@latest # Install or upgrade the CLI (same command for both) tbd setup --auto --prefix= # Fresh project (--prefix is REQUIRED: 2-8 alphabetic chars recommended. ALWAYS ASK THE USER FOR THE PREFIX; do not guess it) -tbd setup --auto # Existing tbd project (prefix already set) +tbd setup --auto # Existing tbd project — also the upgrade step (applies any format migration; commit the diff it reports) tbd setup --from-beads # Migration from .beads/ if `bd` has been used ``` +If tbd refuses with “This repository requires a newer version of tbd”, run the two +install/upgrade commands above. + ## Routine Commands ```bash @@ -96,6 +99,10 @@ or want help → run `tbd shortcut welcome-user` | **Documentation** | | | “Research this topic” | `tbd shortcut new-research-brief` | | “Document architecture” | `tbd shortcut new-architecture-doc` | +| “What guidelines/docs are there?” | `tbd docs list` | +| “Make the guidelines visible / customize doc X” | `tbd docs fork ` (or `--all`), then edit in `docs/tbd/` | +| “Update the guidelines to the latest” | `tbd docs update`; on conflicts ask the user, then `--merge` or `--keep-ours` | +| “I deleted a forked doc file” | `tbd docs status` shows it `missing`; restore with `tbd docs fork --force` or finalize with `tbd docs unfork ` | | **Cleanup & Maintenance** | | | “Clean up this code” / “Remove dead code” | `tbd shortcut code-cleanup-all` | | “Fix repository problems” | `tbd doctor --fix` | @@ -185,6 +192,8 @@ working branch. See `tbd guidelines tbd-sync-troubleshooting` for details. | `tbd guidelines ` | Load coding guidelines | | `tbd guidelines --list` | List guidelines | | `tbd template ` | Output a template | +| `tbd docs` / `tbd docs list` | Managed-docs overview / cross-kind list with state markers | +| `tbd docs fork/unfork/update ` | Fork docs into `docs/tbd/`, return to upstream, pull upstream updates | ## Quick Reference diff --git a/tests/qa/release-v0.3.0-forkable-docs.qa.md b/tests/qa/release-v0.3.0-forkable-docs.qa.md new file mode 100644 index 00000000..8f59a627 --- /dev/null +++ b/tests/qa/release-v0.3.0-forkable-docs.qa.md @@ -0,0 +1,119 @@ +--- +title: QA Playbook +description: Manual release-readiness and upgrade validation for get-tbd v0.3.0 (forkable docs, f04 → f05 on-disk format migration) +author: Joshua Levy (github.com/jlevy) with LLM assistance +--- +# QA Playbook: Release v0.3.0 Forkable Docs + +Manual QA playbook for cutting `get-tbd` v0.3.0 — the release that ships the +forkable-docs workflow and the `f04` → `f05` on-disk format stamp (spec: +`docs/project/specs/done/plan-2026-06-11-forkable-docs.md`). + +**Purpose**: Prove that (a) the new client stamps real `f04` repos to `f05` idempotently +and older clients fail closed against `f05`, (b) the fork lifecycle +(fork/edit/update/unfork) behaves on a real repo with real upgrades — not just fixtures, +(c) running tbd inside git hooks is safe (the `GIT_DIR` isolation), and (d) the +published `v0.3.0` works in day-to-day use after the global install is swapped. + +**Estimated Time**: ~60–90 minutes (10 min prep, 35–50 min scenarios, 20 min release + +post-publish + global swap). + +> This is a manual test: too costly / not pass-fail-crisp to fully cover in unit or +> tryscript tests. The automated suites cover the fork kernel, the f05 stamp, the docs +> surface goldens, and GIT_DIR isolation in fixtures; this playbook covers real user +> repos, a real upgrade sequence across releases, and the published release surface +> installed globally. + +* * * + +## Current Status (last update 2026-06-13) + +| Phase | Status | Notes | +| --- | --- | --- | +| Phase 0: Pre-flight — sync, version, decide bump | ⏳ Pending | Version `v0.3.0` proposed (0.x semver: format change ⇒ minor bump). CHANGELOG Unreleased entry written on PR #169. | +| Phase 1: In-repo sanity (build, test, publint) | ✅ Passed | At PR #169 head: 1,313 unit tests + 881 tryscript blocks green; CI green on ubuntu/macos/windows + coverage/lint + benchmark. | +| Phase 2: Real-repo f04 → f05 upgrade scenarios | ⏳ Pending | | +| Phase 3: Fork lifecycle on a real repo | ⏳ Pending | | +| Phase 4: Hook-safety (GIT_DIR) on a real repo | ⏳ Pending | | +| Phase 5: Cut release v0.3.0 | ⏳ Pending | Per docs/publishing.md. | +| Phase 6: Post-publish verification + global swap | ⏳ Pending | | + +**Status Legend**: ✅ Passed | ❌ Failed | ⏳ Pending | ⏸️ Blocked + +* * * + +## Phase 0: Pre-flight + +1. `git fetch && git status` — main is current, working tree clean. +2. `tbd sync` — beads clean; spec epic `tbd-67ek` and all children closed. +3. Confirm version bump: `v0.3.0` (format `f05` ships; 0.x minor signals it). +4. Review CHANGELOG Unreleased entry against the shipped surface. + +## Phase 1: In-repo sanity + +1. `pnpm install && pnpm build && pnpm test` — all green. +2. `pnpm -r exec publint` — clean. +3. `node packages/tbd/dist/bin.mjs docs` — overview renders; `tbd docs list` shows + guidelines/shortcuts/templates/references including `docref-format`, `docmap-format`, + `tbd-docs`, `tbd-design`. + +## Phase 2: Real-repo f04 → f05 upgrade + +Run on at least one real repo currently on `f04` (with real issue history), using the +release candidate build. + +1. **Stamp**: any issue command (e.g. `tbd list`) co-migrates: `.tbd/config.yml` gains + `tbd_format: f05`; the one-line notice names the change. + Re-run — idempotent, no second notice, no diff. +2. **Linked worktrees**: in a sibling worktree of the same repo, run `tbd list` — shared + layout consistent; per-checkout config stamps once per checkout. +3. **Old client fails closed**: with the previous global version (`0.2.x`), run + `tbd list` in the migrated repo — clean refusal naming the format and the upgrade + command; no writes. +4. **Data integrity**: `tbd doctor` green; issue counts identical before/after; + spot-read a few issues. + +## Phase 3: Fork lifecycle on a real repo + +1. `tbd docs` (zero forks) — three-posture menu; `tbd setup --auto` shows the same menu. +2. `tbd docs fork python-rules` → file in `docs/tbd/guidelines/`, manifest committed; + `tbd guidelines python-rules` serves the fork with the stderr provenance note. +3. Edit the fork; `tbd docs status` shows `customized`; `tbd status` shows the Docs + line. +4. `tbd docs fork --category=general --dry-run` — sensible selection, no writes. +5. Upgrade simulation: after installing a build with changed bundled docs (or + hand-editing the cache copy), `tbd docs update` refreshes unmodified forks, merges + clean changes, and lists conflicts without touching them; `--merge` writes markers; + resolving returns the doc to `customized`. +6. `tbd docs unfork --all` equivalents leave the repo pristine (fork dir and README + removed when empty); `git status` clean apart from intended files. +7. `tbd docs add ./.md --kind=guideline` — canonical docref recorded; + `tbd docs sync` re-syncs from it. + +## Phase 4: Hook-safety (GIT_DIR) + +1. In a real repo, add a `post-commit` hook that runs `tbd list > /tmp/tbd-hook-out`. + Commit something: the hook output is this repo’s issues, stderr shows the one-line + “ignoring inherited GIT_DIR” notice, and no other repo is touched. +2. From a linked worktree, `git push` with the lefthook pre-push suite (this repo): refs + and tbd data byte-identical afterwards (the tbd-a1lc regression scenario). + +## Phase 5: Cut release + +Follow `docs/publishing.md` (tag-triggered pipeline). +Confirm the GitHub release notes match the CHANGELOG entry. + +## Phase 6: Post-publish verification + global swap + +1. `npm install -g get-tbd@latest` on this machine; `tbd --version` is `0.3.0`. +2. Re-run Phase 2 step 1 and Phase 3 steps 1–3 with the published binary on a fresh + clone of a real repo. +3. Day-to-day smoke: `tbd prime`, `tbd ready`, create/close an issue, `tbd sync`. + +## Findings + +(Record findings as beads with `tbd create`, link them here.) + +| Finding | Bead | Status | +| --- | --- | --- | +| | | |