diff --git a/.catalog/.gitkeep b/.catalog/.gitkeep new file mode 100644 index 0000000..e02abfc --- /dev/null +++ b/.catalog/.gitkeep @@ -0,0 +1 @@ + diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md index 3dc6645..0012bcc 100644 --- a/.github/copilot-instructions.md +++ b/.github/copilot-instructions.md @@ -3,6 +3,10 @@ > Instruction file lido automaticamente pelo **GitHub Copilot Chat** e **Copilot Workspace / Agent Mode**. Espelha [AGENTS.md](../AGENTS.md) com foco em **Agent Mode workflow**. > > Ao trabalhar em Agent Mode, o Copilot pode delegar pra custom agents em [`.agents/`](../.agents/) (canônico, padrão AGENTS.md ecosystem) e/ou em `.github/copilot/agents/` (mirror lido pelo Copilot Coding Agent). Lista atual: `tdd.agent.md`, `reviewer.agent.md`, `architect.agent.md`. +> +> Canonical pattern spec: [YOOL_TUPLE_HAMT.md](../YOOL_TUPLE_HAMT.md) +> +> Receipt schema reference: [YOOL_TUPLE_HAMT.md §1.8.4](../YOOL_TUPLE_HAMT.md#184-receipt-schema-reference) --- @@ -173,6 +177,32 @@ Detalhes em `.skills/README.md`. --- +## yool / tuple / HAMT + +Spec: [YOOL_TUPLE_HAMT.md](../YOOL_TUPLE_HAMT.md). + +Required agent fields: + +```markdown +- yool_id: `agent.dev.python.v1` +- authority: dev | ops | review | audit +- lane: fast | slow | background +- agent_terms: + cpu_quota_pct: 60 + disk_quota_mb: 100 + timeout_s: 300 +``` + +Receipts live under `.receipts/` and should follow the canonical schema in [YOOL_TUPLE_HAMT.md §1.8.4](../YOOL_TUPLE_HAMT.md#184-receipt-schema-reference). + +Build the HAMT catalog with: + +```bash +node bin/build-hamt-catalog --source AGENTS.md --output .catalog/agents.json +``` + +--- + ## Comandos especiais ### Criar nova ADR diff --git a/.gitignore b/.gitignore index 290be9f..ad2fdd7 100644 --- a/.gitignore +++ b/.gitignore @@ -63,6 +63,10 @@ Thumbs.db *.tar.gz .pnpm-store/ +# Runtime receipts +.receipts/** +!.receipts/.gitkeep + # LLM Project Mapper tracked files .starter-meta.json .claude/settings.local.json @@ -85,10 +89,13 @@ _BOOTSTRAP.md docs/** !docs/YOOL_TUPLE_HAMT.md scripts/** +!scripts/build_hamt.py playwright-report/** tests/** !tests/unit/ !tests/unit/*.test.js +!tests/e2e/ +!tests/e2e/*.spec.ts test-results/** coverage/** bootstrap.ps1 diff --git a/AGENTS.md b/AGENTS.md index a29662a..d6cd806 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,5 +1,9 @@ # AGENTS.md +> Canonical pattern spec: [YOOL_TUPLE_HAMT.md](YOOL_TUPLE_HAMT.md) +> +> Receipt schema reference: [YOOL_TUPLE_HAMT.md §1.8.4](YOOL_TUPLE_HAMT.md#184-receipt-schema-reference) + ## Operational Context Before changing code, agents should check the project-specific operational docs: @@ -327,7 +331,7 @@ npm run lint && npm test -- --coverage && npx playwright test ## yool / tuple / HAMT (capability addressing) -Spec: `docs/YOOL_TUPLE_HAMT.md` (vendored from https://github.com/wesleysimplicio/yool-tuple-hamt, version v0.2). +Spec: [`YOOL_TUPLE_HAMT.md`](YOOL_TUPLE_HAMT.md) (vendored from https://github.com/wesleysimplicio/yool-tuple-hamt, version v0.2). Every agent registered in this repo MUST declare its capability with the following fields (header `### ` followed by frontmatter-style lines): @@ -351,10 +355,33 @@ Why these fields exist: - `agent_terms.cpu_quota_pct` — soft throttle via `os.nice` or cgroups. Per Victor Genaro's guardrail: *"precisa de guardrail pra não fritar o processador."* - `agent_terms.disk_quota_mb` — local disk cap before GC kicks in. Per the same review: *"você precisa de garbage collector também pra não encher 100% do disco."* +### Receipts schema + +Every repo using this pattern should keep execution receipts under `.receipts/` and treat them as append-only execution evidence, not ad-hoc logs. + +Minimum receipt contract: + +```json +{ + "id": "sha256:", + "tuple_id": "sha256:", + "yool_id": "agent.dev.python", + "status": "ok", + "created_at": "2026-05-19T17:30:00Z", + "artifacts": [], + "cost": { + "tokens": 0, + "usd": 0 + } +} +``` + +Canonical source for receipt semantics, retention, and catalog placement: [YOOL_TUPLE_HAMT.md §1.8.4](YOOL_TUPLE_HAMT.md#184-receipt-schema-reference). + Build the HAMT catalog with: ```bash -node bin/build-hamt-catalog --source AGENTS.md --output .catalog/hamt.json +node bin/build-hamt-catalog --source AGENTS.md --output .catalog/agents.json ``` Without all four fields, the catalog build skips the entry. CI gate enforces full population for any new agent declaration. diff --git a/CHANGELOG.md b/CHANGELOG.md index be92d8a..8e191d8 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,16 @@ Format follows [Keep a Changelog 1.1.0](https://keepachangelog.com/en/1.1.0/) an ## [Unreleased] +### Added +- Root-level `YOOL_TUPLE_HAMT.md` vendored alongside the existing `docs/` copy so the canonical pattern spec is reachable directly from the repository root and ships with the npm package. +- `build-hamt-catalog` wrapper plus stdlib-only `scripts/build_hamt.py`, enabling `npx @wesleysimplicio/llm-project-mapper build-hamt-catalog` to emit `.catalog/agents.json`. +- Runtime scaffold defaults for `.catalog/.gitkeep`, `.catalog/agents.json`, `.receipts/.gitkeep`, and optional `mcp/server.{ts,py}` edge adapters via `--mcp-edge`. +- Dedicated docs-site coverage for YOOL / tuple / HAMT, including the public `/yool-tuple-hamt` route and regression tests for the new page. + +### Changed +- `AGENTS.md`, `CLAUDE.md`, and `.github/copilot-instructions.md` now point to the root spec, document the receipts schema, and align the generated catalog output on `.catalog/agents.json`. +- The Node bootstrap path now mirrors the shell/PowerShell runtime scaffold so fresh `npx` installs create the catalog, receipts, and optional MCP edge templates consistently. + ## [0.4.2] - 2026-05-19 ### Added diff --git a/CLAUDE.md b/CLAUDE.md index 3925311..bac57be 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1,6 +1,10 @@ # CLAUDE.md > Este arquivo espelha [AGENTS.md](./AGENTS.md). Edite ambos juntos OU mantenha apenas `AGENTS.md` e symlink `CLAUDE.md` -> `AGENTS.md` (`ln -sf AGENTS.md CLAUDE.md`). O Claude Code lê arquivo regular, não símbolo. +> +> Canonical pattern spec: [YOOL_TUPLE_HAMT.md](YOOL_TUPLE_HAMT.md) +> +> Receipt schema reference: [YOOL_TUPLE_HAMT.md §1.8.4](YOOL_TUPLE_HAMT.md#184-receipt-schema-reference) --- @@ -333,7 +337,7 @@ npm run lint && npm test -- --coverage && npx playwright test ## yool / tuple / HAMT (capability addressing) -Spec: `docs/YOOL_TUPLE_HAMT.md` (vendored from https://github.com/wesleysimplicio/yool-tuple-hamt, version v0.2). +Spec: [`YOOL_TUPLE_HAMT.md`](YOOL_TUPLE_HAMT.md) (vendored from https://github.com/wesleysimplicio/yool-tuple-hamt, version v0.2). Every agent registered in this repo MUST declare its capability with the following fields (header `### ` followed by frontmatter-style lines): @@ -351,10 +355,33 @@ Every agent registered in this repo MUST declare its capability with the followi Guardrails are MANDATORY per Victor Genaro's review: *"precisa de guardrail pra não fritar o processador. Você precisa de garbage collector também pra não encher 100% do disco."* See spec §11. +### Receipts schema + +Every repo using this pattern should keep execution receipts under `.receipts/` and treat them as append-only execution evidence, not ad-hoc logs. + +Minimum receipt contract: + +```json +{ + "id": "sha256:", + "tuple_id": "sha256:", + "yool_id": "agent.dev.python", + "status": "ok", + "created_at": "2026-05-19T17:30:00Z", + "artifacts": [], + "cost": { + "tokens": 0, + "usd": 0 + } +} +``` + +Canonical source for receipt semantics, retention, and catalog placement: [YOOL_TUPLE_HAMT.md §1.8.4](YOOL_TUPLE_HAMT.md#184-receipt-schema-reference). + Build the HAMT catalog with: ```bash -node bin/build-hamt-catalog --source AGENTS.md --output .catalog/hamt.json +node bin/build-hamt-catalog --source AGENTS.md --output .catalog/agents.json ``` --- diff --git a/INIT.md b/INIT.md index 8c52318..bba78a2 100644 --- a/INIT.md +++ b/INIT.md @@ -21,7 +21,8 @@ Antes de qualquer Write/Edit, leia `.starter-meta.json` na raiz do repo. Ele é "domain": "...", "stack": "...", "bootstrapped_at": "2026-05-08T19:45:00Z", - "starter_version": "0.2.0", + "starter_version": "0.4.0", + "mcp_edge_enabled": false, "existing_instruction_files": [".github/copilot-instructions.md"], "init_must_ask": ["team", "domain", "vision_oneliner", "primary_personas"], "init_must_merge": [".github/copilot-instructions.md"], @@ -56,6 +57,21 @@ playwright.config.ts (apenas se ainda não existe ou se é template nosso) Caminho fora dessa whitelist **e** que não é arquivo do template original → não escreve. +## Artefatos criados pelo bootstrap + +Além da whitelist acima, o bootstrap pode preparar estes artefatos de scaffold/runtime: + +- `.catalog/.gitkeep` +- `.catalog/agents.json` — stub inicial do catálogo HAMT/YOOL; a versão populada vem do builder. +- `.receipts/.gitkeep` — diretório local para receipts; o default é ficar gitignored. +- `mcp/server.ts` e `mcp/server.py` — **somente** quando `.starter-meta.json.mcp_edge_enabled == true`. + +Regras específicas: + +- Preserve o bloco de aviso dos arquivos em `mcp/`: **MCP é borda/edge, não o loop interno dos agentes**. +- Se `AGENTS.md` já trouxer campos `yool_id`, `authority`, `lane` e `agent_terms`, trate isso como contrato do catálogo; não remova nem “simplifique”. +- `mcp/` só existe para `snapshot` e `dispatch` de borda sobre `.catalog/agents.json`; não enfie orquestração interna ali. + --- ## Fluxo (5 fases — paraleliza tudo que dá) diff --git a/README.md b/README.md index a3d8052..cca4437 100644 --- a/README.md +++ b/README.md @@ -36,6 +36,15 @@ Fill these files after installing the starter in a real project. The goal is to --- +## Patterns + +- Canonical spec: [YOOL_TUPLE_HAMT.md](YOOL_TUPLE_HAMT.md) +- Receipts schema and storage conventions: [Receipt schema](YOOL_TUPLE_HAMT.md#184-receipt-schema-reference) + +The yool / tuple / HAMT pattern is the capability-addressing model this scaffold is standardizing for multi-agent repos. Keep the root spec vendored so agents can reach it from the repository root in one click. + +--- + ## TL;DR — get going in 60 seconds Pick **one** of the install paths below and run it inside your project folder. The bootstrap now starts an automatic local mapping pass immediately; `INIT.md` becomes an optional refinement step for a stronger agent. diff --git a/README.pt-BR.md b/README.pt-BR.md index 5af0992..62f350e 100644 --- a/README.pt-BR.md +++ b/README.pt-BR.md @@ -35,6 +35,15 @@ Preencha esses arquivos depois de instalar o starter em um projeto real. O objet --- +## Padroes + +- Especificacao canonica: [YOOL_TUPLE_HAMT.md](YOOL_TUPLE_HAMT.md) +- Esquema de receipts e convencoes de armazenamento: [Esquema de receipts](YOOL_TUPLE_HAMT.md#184-receipt-schema-reference) + +O padrao yool / tuple / HAMT define o modelo de enderecamento de capacidades que este scaffold passa a assumir para repositorios multiagente. Mantenha a especificacao vendorizada na raiz para que qualquer agente chegue nela em um clique a partir do repo. + +--- + ## TL;DR — começa em 60 segundos Escolha **um** caminho de instalação abaixo e rode dentro da pasta do projeto. O bootstrap agora inicia automaticamente um mapeamento local e preenche a primeira versão dos arquivos; o `INIT.md` vira uma etapa opcional de refinamento com agente. diff --git a/YOOL_TUPLE_HAMT.md b/YOOL_TUPLE_HAMT.md new file mode 100644 index 0000000..9c6e91c --- /dev/null +++ b/YOOL_TUPLE_HAMT.md @@ -0,0 +1,1166 @@ +# yool · tuple · HAMT — capability addressing for agent systems + +> Canonical specification of the **yool / tuple / HAMT** pattern. +> Cross-project pattern doc. Source of truth lives here: +> https://github.com/wesleysimplicio/yool-tuple-hamt (private). +> Vendored into [SendSprint](https://github.com/wesleysimplicio/SendSprint), +> [llm-project-mapper](https://github.com/wesleysimplicio/llm-project-mapper), +> and any future agent-orchestration project. + +Status: **draft v0.2** · Maintainer: @wesleysimplicio · Last updated: 2026-05-19 + +--- + +## 0. TL;DR + +- **yool** — smallest callable capability atom. An opcode: `agent.dev.python`, `ide.cursor.send`, `fs.read_sector`. +- **tuple** — addressable envelope binding yools to map position, authority, lane, budget, source pointers, receipts. Unit of work. +- **HAMT** — Hash Array Mapped Trie cataloging every yool/tuple/agent/operator. O(log32) lookup, immutable structural sharing. +- **tuple-space** (Linda) — producers `out` tuples; workers `in`/`rd` by pattern. No imperative orchestrator. +- **receipts** — content-addressable execution records. Same input → same hash → cache hit, no recompute. +- **MCP edge** — read-mostly snapshot/dispatch surface. NOT the inner loop. +- **guardrails** — CPU throttle + disk GC are mandatory, not optional (see §11). + +--- + +## 1. The Problem (Why We Built This) + +Every agent system we built had the same five rots: + +1. **Orchestrator accretion** — `flow.py` / `pipeline.py` grow with every new step. Imperative coupling. New step = patching 5 files. +2. **Registry sprawl** — agents/IDEs/operators live in hand-maintained dicts. Adding 1 IDE touches the dict, the dispatcher, the CLI, the docs, the tests. +3. **No cross-run cache** — same input rebuilds because intermediate results have no addressable identity. +4. **Resume/replay is bespoke** — every project serializes run state differently. Crash recovery is "best effort", usually a fresh restart. +5. **Audit is post-hoc** — "what ran, with whose authority, against which input, costing how much" — answered by `grep` over logs. + +Symptom that triggered this spec: the **MCP exposure drift** — exposing every internal call as an MCP tool makes the inner loop slow (latency per call), uncacheable (no addressable identity), and unbounded (no budget). The fix isn't "better MCP" — it's keeping MCP at the edge and rebuilding the inner loop on **capability addressing**. + +--- + +## 2. Vocabulary + +### 2.1 yool + +The atomic, callable capability. **One yool = one opcode = one side-effect or pure computation**. + +#### Examples by category + +``` +# Code-acting agents +agent.dev.python +agent.dev.dotnet +agent.dev.typescript +agent.lint.ruff +agent.lint.eslint +agent.lint.dotnet +agent.test.pytest +agent.test.jest +agent.test.e2e.playwright +agent.security.scan +agent.security.dependabot +agent.pr.create +agent.pr.review + +# IDE bridges +ide.cursor.send +ide.zed.send +ide.vscode.send +ide.jetbrains.send + +# Project operators +op.jira.fetch_sprint +op.azure.fetch_iteration +op.linear.fetch_cycle +op.github.fetch_issues + +# Filesystem & net (primitives) +fs.read_sector +fs.write_receipt +net.fetch +net.post + +# Catalog itself (introspection) +catalog.lookup +catalog.list_by_lane +catalog.diff +``` + +#### Naming rules + +| Rule | Example | +|---|---| +| One verb, one direct object | `agent.lint.ruff` (lint is the verb, ruff is the implementation) | +| Stable identifier (rename = breaking) | `agent.dev.python.v1` then bump to `.v2` instead of renaming | +| Namespace by domain.action.tool | `domain` in {agent, ide, op, fs, net, catalog} | +| Pure or single-effect | Yool either reads OR writes, not both implicitly | +| Returns a receipt | Every yool execution emits exactly one receipt | + +#### Anti-patterns + +``` +# NO - multiple verbs +agent.lint_and_test.python + +# NO - implicit fan-out +agent.deploy.all_envs + +# NO - opaque action +agent.do.thing + +# YES - split into independent yools +agent.lint.python +agent.test.python +agent.deploy.staging +agent.deploy.prod +``` + +A yool is **not** a function. A function might back a yool, but the yool itself is the **addressable symbol + contract**, decoupled from implementation. + +### 2.2 tuple + +The envelope. Wraps a payload of yools with everything needed to route, authorize, budget, audit, and replay. + +#### Canonical schema (v1) + +```jsonc +{ + "id": "sha256:abc123...", + "schema": "yool-tuple/v1", + "map_pos": { + "repo": "EVT", + "branch": "feat/JIRA-456", + "sprint_id": "JIRA-456", + "stack": "dotnet" + }, + "authority": { + "user": "wes", + "agent": "sendsprint", + "ci": false + }, + "lane": "build", + "agent_terms": { + "budget_usd": 0.50, + "budget_tokens": 50000, + "deadline_iso": "2026-05-19T20:00:00Z", + "max_retries": 2, + "cpu_quota_pct": 60, + "disk_quota_mb": 100 + }, + "src_ptr": [ + "jira://EVT/JIRA-456", + "commit://abc123" + ], + "payload": [ + {"yool": "agent.dev.dotnet", "args": {}}, + {"yool": "agent.lint.dotnet", "args": {}}, + {"yool": "agent.test.dotnet", "args": {"filter": "Unit"}} + ], + "receipts": [], + "parent_id": null, + "created_at": "2026-05-19T17:30:00Z" +} +``` + +#### Field semantics + +| Field | Purpose | Mutability | +|---|---|---| +| `id` | Content hash of canonical form (excluding `id` itself) | immutable post-creation | +| `schema` | Version of this schema | immutable | +| `map_pos` | Where in the project graph this tuple lives | immutable | +| `authority` | Who/what authorized this work | immutable | +| `lane` | Routing key for workers | immutable (re-emit to change) | +| `agent_terms` | Budget envelope + guardrails | immutable | +| `src_ptr` | Provenance pointers (issue, commit, doc) | immutable | +| `payload` | Ordered yool program | immutable | +| `receipts` | Receipt ids appended as yools complete | append-only | +| `parent_id` | Parent tuple in DAG | immutable | +| `created_at` | Wall clock at creation | immutable | + +#### Properties + +- **Content-addressable**: `id = sha256(canonical(tuple_without_id))`. Same input -> same id -> free dedupe. +- **Self-describing**: `schema` field allows v1 and v2 to coexist on the bus. +- **Replayable**: every field needed to re-execute is on the envelope. +- **Auditable**: `authority` + `src_ptr` make every action traceable. + +#### 1.8.4 Receipt schema reference + +Receipts are append-only, content-addressed execution records written under a repo-local `.receipts/` tree. + +Minimum receipt shape: + +```json +{ + "id": "sha256:", + "tuple_id": "sha256:", + "yool_id": "agent.dev.python", + "status": "ok", + "created_at": "2026-05-19T17:30:00Z", + "artifacts": [], + "cost": { + "tokens": 0, + "usd": 0 + } +} +``` + +Required properties: + +- `id` is the content hash of the canonical receipt body. +- `tuple_id` links the receipt back to the tuple that dispatched the yool. +- `yool_id` identifies the capability that produced the artifact. +- `status` is terminal (`ok`, `error`, `cached`, `skipped`). +- `artifacts` points to any persisted evidence generated by the run. +- `cost` records token and currency spend when the execution uses metered models or services. + +Recommended layout: + +```text +.receipts//.json +``` + +Projects may mirror or compact receipts into `.catalog/receipts/`, but the canonical repo-level convention for scaffolds is `.receipts/` plus content-addressable file names. + +### 2.3 HAMT (Hash Array Mapped Trie) + +A persistent dictionary structure. Used here to **catalog** all yools, agents, IDEs, operators under a single addressable namespace. + +#### Why HAMT vs flat dict + +| Concern | Flat dict | HAMT | +|---|---|---| +| Lookup | O(1) avg, O(n) worst | O(log32 n) bounded | +| Memory on update | Rewrite-heavy | Structural sharing (only touched path copied) | +| Concurrency | Mutex required | Lock-free reads (immutable nodes) | +| Distribution | Hard to shard | Trivially shardable by top-level slot | +| Audit history | None | Hashes form Merkle chain | +| Size at scale | Memory bloat at >100k | Bounded depth = bounded memory walk | + +#### Parameters used + +- Hash: BLAKE2b-64 truncated to 30 bits +- Bits per level: 5 (branching factor = 32) +- Max levels: 6 +- Address space pre-collision: 2^30 ~= 1.07 billion + +A yool name like `agent.dev.python` is hashed; the 30-bit hash decomposes into 6 slot indices `[s0..s5]` that walk the trie. Collisions beyond level 6 collapse to a `collision` leaf list. + +Reference: Phil Bagwell, *Ideal Hash Trees* — https://lampwww.epfl.ch/papers/idealhashtrees.pdf + +### 2.4 tuple-space + +The coordination substrate. Producers `out` tuples; consumers `in`/`rd` tuples matching a pattern. + +**Linda primitives** (Gelernter 1985): + +| Primitive | Semantics | +|---|---| +| `out(tuple)` | Publish to space. Non-blocking. | +| `in(pattern)` | Remove and return one matching tuple. Blocks until match. | +| `rd(pattern)` | Read (don't remove) one matching tuple. Blocks until match. | +| `eval(template)` | Spawn an active tuple that computes itself, becoming a passive tuple. | + +A worker handling `lane=build`: + +```python +while True: + t = bus.in_({"lane": "build"}) + receipt = run(t) + bus.out_(t.with_receipt(receipt)) +``` + +That's the entire orchestrator. + +Reference: David Gelernter, *Generative Communication in Linda*, ACM TOPLAS 1985. + +### 2.5 receipt + +The output artifact of a yool execution. + +```jsonc +{ + "id": "sha256:def456...", + "yool": "agent.test.pytest", + "tuple_id": "sha256:abc123...", + "started_at": "2026-05-19T17:30:00Z", + "ended_at": "2026-05-19T17:31:12Z", + "exit": 0, + "stdout_sha": "sha256:111...", + "stderr_sha": "sha256:222...", + "artifacts": [ + {"kind": "junit", "path": "evidence/sha:333.../junit.xml"}, + {"kind": "coverage", "path": "evidence/sha:444.../coverage.json"} + ], + "cost": {"usd": 0.012, "tokens_in": 1200, "tokens_out": 800, "wall_ms": 72000, "disk_mb": 4.2} +} +``` + +Receipts are **immutable** and **content-addressable**. Two yool runs with the same input hash MAY share a receipt — that's the cache key. + +### 2.6 MCP edge + +Model Context Protocol surfaces are **read-mostly snapshots** of the tuple-space, not the bus itself. + +| Allowed | Disallowed | +|---|---| +| `catalog.lookup(name)` | `tuple.in()` / `tuple.out()` (inner loop) | +| `catalog.list_by_lane(lane)` | per-yool dispatch | +| `tuple.dispatch(tuple)` (returns receipt id) | streaming raw tuple events without ETag | +| `tuple.observe(id)` (SSE, cacheable) | mutation of catalog/receipts | +| `receipt.get(id)` | | + +Why: latency, cost, cacheability. MCP is fine for snapshot dashboards and external agent observation. Putting inner-loop semantics behind MCP turns every tuple emit into a network call. + +Reference: MCP tools spec — https://modelcontextprotocol.io/specification/draft/server/tools + +--- + +## 3. Architecture + +### 3.1 Static layout + +``` ++------------------------------------------------------------+ +| Capability Catalog | +| (HAMT, addressable) | +| | +| agent.dev.* agent.lint.* agent.test.* | +| agent.security.* agent.pr.* agent.deploy.* | +| ide.* op.* fs.* net.* | +| | +| storage: .catalog/capabilities.json (versioned in repo) | ++------------------------------------------------------------+ + ^ + | resolve(name) -> impl + | ++------------------------------------------------------------+ +| Tuple Space | +| (Linda bus) | +| | +| .catalog/tuples.jsonl (append-only, content-addressable) | +| | +| producers --out--> [tuple] --in--> subscribers | +| ^ | | +| | v | +| .catalog/receipts/ (content-addressable) | ++------------------------------------------------------------+ + ^ + | snapshot + | ++------------------------------------------------------------+ +| MCP Edge | +| catalog.lookup catalog.list tuple.dispatch | +| tuple.observe (SSE) receipt.get | ++------------------------------------------------------------+ + ^ + | + Claude / Codex / Copilot / Dashboard +``` + +### 3.2 Dynamic flow (one item) + +``` +1. user/CI/agent emits tuple T0 + { lane: "build", map_pos: {...}, payload: [yool_a, yool_b, yool_c] } + +2. catalog resolves each yool to an impl + yool_a -> agent.dev.python.v1 + yool_b -> agent.lint.ruff.v3 + yool_c -> agent.test.pytest.v2 + +3. worker pool subscribes lane="build" + worker pulls T0, executes payload sequentially or in parallel per dep graph + +4. each yool emits a receipt R_a, R_b, R_c + each receipt is content-addressed; cache check before recompute + +5. final receipt R_T0 = aggregate(R_a, R_b, R_c) + tuple log appends T0 + R_T0 + +6. MCP snapshot reflects new state + dashboard / Claude observe via SSE +``` + +### 3.3 Failure & resume + +``` +crash mid-flight + | + v +restart reads tuples.jsonl + | + v +filter tuples with no terminal receipt + | + v +re-emit on bus (idempotent: id collision = skip) + | + v +workers reprocess; cached receipts short-circuit +``` + +### 3.4 HAMT trie example + +Sample insertion of 3 yools - trie state after each step. + +``` +Initial: empty Node { bitmap=0, children={} } + +insert(agent.dev.python) hash=011010... slots=[13, 4, 22, 9, 1, 7] + + Node { bitmap=...10000000000000, children={13: Leaf(agent.dev.python)} } + +insert(agent.lint.ruff) hash=000111... slots=[3, 21, 0, 18, 30, 12] + + Node { bitmap=...10000000001000, children={3: Leaf(agent.lint.ruff), + 13: Leaf(agent.dev.python)} } + +insert(agent.dev.dotnet) hash=011010... slots=[13, 7, 1, 28, 4, 19] + collides with agent.dev.python at level 0 (slot 13) + + Node { + bitmap=..., + children={ + 3: Leaf(agent.lint.ruff), + 13: Node { # subnode created + bitmap=..., + children={ + 4: Leaf(agent.dev.python), # at level 1, slot 4 + 7: Leaf(agent.dev.dotnet) # at level 1, slot 7 + } + } + } + } +``` + +### 3.5 Tuple-space example (Linda flow) + +``` +Time Producer Bus Worker(build) +---- --------------- ---------------------------- -------------- +t0 emit(T0) --> [T0{lane:build}] + in({lane:build}) +t1 <-- T0 +t2 run(agent.dev.dotnet) + cache miss; exec + emit receipt R_a +t3 [T0{...receipts:[R_a]}] run(agent.lint.dotnet) + cache HIT; reuse +t4 [T0{...receipts:[R_a,R_b]}] run(agent.test.dotnet) + cache miss; exec + emit receipt R_c +t5 [T0{...receipts:[R_a,R_b,R_c]}] out(T0_done) + aggregate R_T0 +t6 observe(T0) <-- [T0_done] +``` + +--- + +## 4. Algorithms + +### 4.1 yool name hashing + +```python +import hashlib + +def yool_hash(name: str) -> int: + h = hashlib.blake2b(name.encode("utf-8"), digest_size=8).digest() + return int.from_bytes(h, "big") & ((1 << 30) - 1) +``` + +### 4.2 HAMT slot decomposition + +```python +def slots(h: int, levels: int = 6, bits: int = 5) -> list[int]: + mask = (1 << bits) - 1 + return [(h >> ((levels - 1 - lvl) * bits)) & mask for lvl in range(levels)] +``` + +### 4.3 HAMT insert (full) + +```python +def insert(root: Node, leaf: Leaf, level: int = 0) -> None: + if level >= MAX_LEVELS: + slot = leaf.hash & (BRANCH - 1) + existing = root.children.get(slot) + if existing is None: + root.bitmap |= 1 << slot + root.children[slot] = Collision(hash_prefix=leaf.hash, leaves=[leaf]) + elif isinstance(existing, Collision): + existing.leaves.append(leaf) + else: + raise RuntimeError("unexpected node at collision depth") + return + + slot = slot_at(leaf.hash, level) + existing = root.children.get(slot) + + if existing is None: + root.bitmap |= 1 << slot + root.children[slot] = leaf + return + + if isinstance(existing, Leaf): + if existing.hash == leaf.hash and existing.key == leaf.key: + existing.tuple = leaf.tuple + return + sub = Node() + insert(sub, existing, level + 1) + insert(sub, leaf, level + 1) + root.children[slot] = sub + return + + if isinstance(existing, Node): + insert(existing, leaf, level + 1) + return +``` + +### 4.4 HAMT lookup + +```python +def lookup(root: Node, key: str) -> Leaf | None: + h = yool_hash(key) + node = root + for lvl in range(MAX_LEVELS): + slot = slot_at(h, lvl) + child = node.children.get(slot) + if child is None: + return None + if isinstance(child, Leaf): + return child if child.key == key else None + if isinstance(child, Collision): + for leaf in child.leaves: + if leaf.key == key: + return leaf + return None + node = child + return None +``` + +### 4.5 Tuple id + +```python +import json, hashlib + +def tuple_id(t: dict) -> str: + t_no_id = {k: v for k, v in t.items() if k != "id"} + canonical = json.dumps(t_no_id, sort_keys=True, separators=(",", ":"), ensure_ascii=False) + return "sha256:" + hashlib.sha256(canonical.encode("utf-8")).hexdigest() +``` + +### 4.6 Receipt content addressing + +```python +def receipt_id(receipt: dict) -> str: + h = hashlib.sha256() + h.update(receipt["yool"].encode()) + h.update(str(receipt["exit"]).encode()) + h.update(receipt["stdout_sha"].encode()) + h.update(receipt["stderr_sha"].encode()) + for a in receipt.get("artifacts", []): + h.update(a["path"].encode()) + return "sha256:" + h.hexdigest() +``` + +### 4.7 Cache check + +```python +def input_hash(yool: str, args: dict, file_shas: list[str], env_whitelist: dict) -> str: + h = hashlib.sha256() + h.update(yool.encode()) + h.update(json.dumps(args, sort_keys=True).encode()) + for sha in sorted(file_shas): + h.update(sha.encode()) + for k, v in sorted(env_whitelist.items()): + h.update(f"{k}={v}".encode()) + return h.hexdigest() + +def cached_receipt(yool: str, ih: str): + return receipt_store.get(f"{yool}@{ih}") +``` + +--- + +## 5. End-to-End Example: SendSprint Adoption + +### 5.1 Before - imperative pipeline + +```python +class SprintFlow: + def __init__(self, sprint_id, stack): + self.sprint_id = sprint_id + self.stack = stack + + def run(self): + items = jira.fetch_sprint(self.sprint_id) + for item in items: + code = self._dev(item) + self._lint(code, item) + self._test(code, item) + self._security_scan(code, item) + self._create_pr(code, item) + + def _dev(self, item): + if self.stack == "python": + return PythonDevAgent().run(item) + elif self.stack == "dotnet": + return DotnetDevAgent().run(item) + # ... 5 more elifs +``` + +Problems: adding a stack patches 5 methods; no caching; crash mid-sprint restarts from scratch; no audit beyond logs. + +### 5.2 After - yool/tuple/HAMT + +```python +class SprintFlow: + def __init__(self, sprint_id, stack, bus, catalog, receipts): + self.sprint_id = sprint_id + self.stack = stack + self.bus = bus + self.catalog = catalog + self.receipts = receipts + + def run(self): + items = self._emit(yool="op.jira.fetch_sprint", args={"sprint_id": self.sprint_id}) + for item in items: + tuple_ = Tuple( + map_pos={"sprint_id": self.sprint_id, "stack": self.stack, "item": item.id}, + lane="build", + agent_terms={"budget_usd": 0.50, "cpu_quota_pct": 60, "disk_quota_mb": 100}, + payload=[ + {"yool": f"agent.dev.{self.stack}", "args": {"item": item}}, + {"yool": f"agent.lint.{self.stack}", "args": {}}, + {"yool": f"agent.test.{self.stack}", "args": {}}, + {"yool": "agent.security.scan", "args": {}}, + {"yool": "agent.pr.create", "args": {}}, + ], + src_ptr=[f"jira://{item.id}"], + ) + self.bus.out_(tuple_) +``` + +Adding a new stack = add `agent.dev.` to the catalog. Zero touches to `SprintFlow`. + +### 5.3 Worker + +```python +async def worker(lane: str, bus, catalog, receipts): + async for t in bus.subscribe(lane): + for step in t.payload: + yool_name = step["yool"] + args = step["args"] + + ih = input_hash(yool_name, args, file_shas=[], env_whitelist={}) + cached = receipts.find(yool_name, ih) + if cached and cached.status == "ok" and not t.flags.get("no_cache"): + t.receipts.append(cached.id) + continue + + impl = catalog.lookup(yool_name) + if impl is None: + raise UnknownYool(yool_name) + + with cpu_throttle(t.agent_terms["cpu_quota_pct"]): + with disk_quota(t.agent_terms["disk_quota_mb"]): + receipt = await impl.run(args) + + receipts.put(receipt) + t.receipts.append(receipt.id) + + bus.out_(t) +``` + +### 5.4 Cache hit example + +``` +# First run +$ sprint run --sprint-id JIRA-456 +[t=0] op.jira.fetch_sprint MISS exec cost=$0.001 +[t=4s] agent.dev.dotnet MISS exec cost=$0.12 +[t=18s] agent.lint.dotnet MISS exec cost=$0.01 +[t=19s] agent.test.dotnet MISS exec cost=$0.04 +[t=42s] agent.security.scan MISS exec cost=$0.02 +[t=44s] agent.pr.create MISS exec cost=$0.01 + TOTAL: $0.201 + +# Re-run, no source change +$ sprint run --sprint-id JIRA-456 +[t=0] op.jira.fetch_sprint MISS exec cost=$0.001 +[t=4s] agent.dev.dotnet HIT skip cost=$0 +[t=4s] agent.lint.dotnet HIT skip cost=$0 +[t=4s] agent.test.dotnet HIT skip cost=$0 +[t=4s] agent.security.scan HIT skip cost=$0 +[t=4s] agent.pr.create HIT skip cost=$0 + TOTAL: $0.001 +``` + +### 5.5 Crash + resume example + +``` +$ sprint run --sprint-id JIRA-456 +[t=0] op.jira.fetch_sprint MISS exec +[t=4s] agent.dev.dotnet MISS exec +[t=18s] agent.lint.dotnet MISS exec +[t=19s] agent.test.dotnet MISS exec +^C (kill -9) + +$ sprint resume --run-id $(sprint runs list | head -1) +[resume] reading .catalog/tuples.jsonl +[resume] T0 has receipts for [op.jira.fetch_sprint, agent.dev.dotnet, agent.lint.dotnet] +[resume] re-emitting T0 from step 4 (agent.test.dotnet) +[t=0] agent.test.dotnet MISS exec cost=$0.04 +[t=23s] agent.security.scan MISS exec cost=$0.02 +[t=25s] agent.pr.create MISS exec cost=$0.01 + TOTAL: $0.07 +``` + +--- + +## 6. End-to-End Example: llm-project-mapper Adoption + +### 6.1 Catalog from `AGENTS.md` + +llm-project-mapper already defines agents declaratively in `AGENTS.md`. The pattern extends each entry with `yool_id`, `authority`, `lane`, `agent_terms` defaults. + +#### Before + +```markdown +## Agents + +### dev-agent +- Role: implements code from spec +- Triggers: new task in .specs/sprints/ +- Stack: auto-detect + +### lint-agent +- Role: enforces style +- Triggers: post-edit +``` + +#### After + +```markdown +## Agents + +### dev-agent +- yool_id: agent.dev.${stack}.v1 +- authority: [user, ci] +- lane: build +- agent_terms: + budget_usd: 0.50 + cpu_quota_pct: 60 + disk_quota_mb: 100 +- Role: implements code from spec +- Triggers: new task in .specs/sprints/ + +### lint-agent +- yool_id: agent.lint.${stack}.v1 +- authority: [user, ci] +- lane: build +- agent_terms: + budget_usd: 0.05 + cpu_quota_pct: 30 + disk_quota_mb: 10 +- Role: enforces style +- Triggers: post-edit +``` + +### 6.2 `bin/build-hamt-catalog` + +```bash +#!/usr/bin/env bash +# Wrapper: node -> python core. + +set -euo pipefail + +PY=$(command -v python3 || command -v py || { echo "python3 required"; exit 1; }) + +ROOT="${1:-.}" +"$PY" "$(dirname "$0")/../scripts/build_hamt.py" \ + --source "$ROOT/AGENTS.md" \ + --output "$ROOT/.catalog/capabilities.json" +``` + +### 6.3 npx flow + +``` +$ npx @wesleysimplicio/llm-project-mapper my-new-project +[scaffold] writing AGENTS.md, .specs/, .skills/, .catalog/.gitkeep ... +[scaffold] creating .catalog/capabilities.json (stub) +[scaffold] adding .receipts/ to .gitignore +[scaffold] writing bin/build-hamt-catalog + +$ npx llm-project-mapper build-hamt-catalog +[build] parsed 7 agents from AGENTS.md +[build] hashing yools ... 7/7 +[build] inserting into HAMT ... done +[build] wrote .catalog/capabilities.json (4.2 KB) +[build] popcount root: 7/32 +``` + +--- + +## 7. Implementation Checklist + +### CP1 · Capability catalog (HAMT) + +- [ ] Pick storage location: `/.catalog/capabilities.json`. +- [ ] Enumerate existing capabilities. +- [ ] Build catalog generator. +- [ ] Replace existing registry lookups with `catalog.lookup(name)`. +- [ ] Test: lookup unknown name returns explicit error. + +### CP2 · Receipt store (content-addressable) + +- [ ] Directory layout: `/.catalog/receipts//.json`. +- [ ] `receipt_id(receipt)` helper. +- [ ] Re-key artifacts by SHA. +- [ ] Run-scoped index. +- [ ] Garbage collection policy (see §11.2). + +### CP3 · Tuple log + +- [ ] Append-only `/.catalog/tuples.jsonl`. +- [ ] Line per emitted tuple + per receipt. +- [ ] Fsync on terminal receipts. +- [ ] Recovery script reads log + filters incomplete tuples. + +### CP4 · Worker pool & lanes + +- [ ] Define lanes. +- [ ] Worker = `subscribe(lane)` loop with bounded concurrency. +- [ ] Replace imperative orchestrator with emitter. +- [ ] Backpressure: queue depth per lane. +- [ ] Guardrails applied per-step (§11). + +### CP5 · MCP edge + +- [ ] MCP server exposes: `catalog.lookup`, `catalog.list_by_lane`, `tuple.dispatch`, `tuple.observe`, `receipt.get`. +- [ ] No write semantics besides `dispatch`. +- [ ] Snapshot endpoint cacheable. + +### CP6 · Budget enforcement + +- [ ] `agent_terms` on every tuple. +- [ ] Worker computes projected cost; reject above remaining budget. +- [ ] Receipt records actual cost. +- [ ] Aggregator emits alarm tuple on threshold. + +--- + +## 8. Migration Playbook + +| Step | Mechanism | Risk | +|---|---|---| +| 1. Generate catalog | Run builder against current registry. Commit JSON. | none | +| 2. Dual-read | Existing code uses old dict; new code reads catalog. Diff on mismatch. | low | +| 3. Receipt shim | Wrap existing artifact writes to emit content-addressed copy. | low | +| 4. Tuple log shim | Write tuple lines alongside current run state. | low | +| 5. Cut orchestrator | Refactor flow to emit/await. Old path under feature flag. | medium | +| 6. Workers replace direct calls | Subscribe-based execution. | medium | +| 7. Cache lookup | Before running yool, check receipt store. | medium | +| 8. MCP server | Expose snapshot. | low | +| 9. Budget enforcement | Add `agent_terms` to all tuples, enforce in workers. | low | +| 10. Remove old orchestrator | Once stable for N sprints, delete dual paths. | low | + +--- + +## 9. Reference Implementations + +This repo: + +- `scripts/build_hamt.py` — Python HAMT builder. +- `examples/python/minimal_bus.py` — minimal Linda-style tuple-space. +- `examples/python/receipts.py` — content-addressable receipt store. +- `examples/node/build-catalog.mjs` — Node wrapper invoking Python core. +- `guardrails/cpu_throttle.py` — CPU quota enforcement (§11.1). +- `guardrails/disk_gc.py` — receipt store GC (§11.2). + +Adopters: + +- SendSprint — Python: `scripts/build_agent_catalog.py`, `src/sendsprint/bus/`, `src/sendsprint/receipts/`. +- llm-project-mapper — Node + Python: `bin/build-hamt-catalog`, `.catalog/capabilities.json`. + +--- + +## 10. Foundational Literature + +| Concept | Reference | +|---|---| +| Tuple spaces / coordination | Gelernter, *Generative Communication in Linda*, ACM TOPLAS 1985 | +| HAMT / persistent hash trie | Bagwell, *Ideal Hash Trees*, EPFL 2001 | +| Locality-preserving multi-attr indexing | Jagadish, *Linear Clustering of Objects with Multiple Attributes* | +| Hilbert clustering analysis | Moon/Jagadish/Faloutsos/Saltz | +| Information theory base | Shannon, *A Mathematical Theory of Communication* | +| Model Context Protocol tools | MCP spec — https://modelcontextprotocol.io/specification/draft/server/tools | +| Content-addressable storage | Merkle, *Protocols for Public Key Cryptosystems*, IEEE S&P 1980 | +| Persistent data structures | Okasaki, *Purely Functional Data Structures*, 1998 | + +--- + +## 11. Guardrails (MANDATORY) + +> Origin of this section: field observation from Victor "Dev Hermes" Genaro (2026-05-19): +> *"precisa de guardrail pra não fritar o processador. Você precisa de garbage collector também pra não encher 100% do disco."* +> +> Any adopter MUST implement both before going past CP4 (worker pool). Without them, a runaway agent or receipt-store explosion can take down the host. + +### 11.1 CPU throttle (don't fry the CPU) + +#### Problem + +A worker that pulls tuples as fast as it can will pin every available core. Multiple workers compound. Local dev box becomes unusable; cloud VM hits CPU throttling and gets killed. + +#### Policy + +Every tuple carries `agent_terms.cpu_quota_pct` (0-100). Worker MUST enforce this before invoking the yool's implementation. + +#### Reference implementation (Python, POSIX) + +```python +# guardrails/cpu_throttle.py +import os +import contextlib + +@contextlib.contextmanager +def cpu_throttle(quota_pct: int): + """Soft CPU throttle via niceness. For hard throttle, use cgroups (Linux).""" + if quota_pct >= 100: + yield + return + + # Niceness mapping: 60% -> nice 5, 30% -> nice 10, 10% -> nice 15 + nice_delta = max(0, int(round((100 - quota_pct) / 6))) + try: + os.nice(nice_delta) + except OSError: + pass + try: + yield + finally: + try: + os.nice(-nice_delta) + except OSError: + pass +``` + +#### Stricter alternative (cgroups, Linux only) + +```bash +cgcreate -g cpu:/yool-worker-${WORKER_ID} +echo $((quota_pct * 1000)) > /sys/fs/cgroup/yool-worker-${WORKER_ID}/cpu.max +cgexec -g cpu:/yool-worker-${WORKER_ID} python -m sendsprint.worker +``` + +#### macOS alternative + +```bash +taskpolicy -c utility python -m sendsprint.worker +``` + +#### Enforcement points + +1. **Worker startup**: read default `cpu_quota_pct` from project config (`.catalog/policy.yaml`). +2. **Per-tuple**: override with `agent_terms.cpu_quota_pct`. +3. **Per-yool**: implementation MAY further reduce (never raise). + +#### Test + +```python +def test_cpu_throttle_under_quota(): + with cpu_throttle(50): + burn_cpu(seconds=2) + assert measured_cpu_time() < 1.5 +``` + +### 11.2 Disk GC (don't fill 100%) + +#### Problem + +Receipts + tuple logs + cached artifacts grow unbounded. Daily sprint with 100 items × 5 yools × 50 KB artifacts = 25 MB/day = 9 GB/year. Multiply by N projects. + +#### Policy — three retention tiers + +| Tier | What | Retention | Why | +|---|---|---|---| +| **hot** | Last N runs of receipts, tuple log, artifacts | default 30 days | active debugging, cache hits | +| **warm** | Receipts only (not artifacts), pointer index | default 365 days | replay + audit | +| **cold** | Hash + pointer record only (artifacts purged) | forever | provenance trail | + +Receipts themselves are **never deleted**, only their **artifact bodies**. Preserves the immutable Merkle chain. + +#### Reference implementation + +```python +# guardrails/disk_gc.py +import json, os, pathlib +from datetime import datetime, timedelta, timezone + +def gc_run(catalog_dir: pathlib.Path, hot_days: int = 30, warm_days: int = 365, max_total_mb: int = 5000): + """ + Phase 1: artifact body purge for receipts older than hot_days. + Phase 2: hard size cap (max_total_mb): purge oldest until under cap. + Phase 3: rotate tuples.jsonl (daily file, gzip yesterday's). + """ + now = datetime.now(timezone.utc) + hot_cutoff = now - timedelta(days=hot_days) + + receipts_dir = catalog_dir / "receipts" + artifacts_dir = catalog_dir / "artifacts" + + purged_artifacts = 0 + purged_bytes = 0 + + for receipt_file in receipts_dir.rglob("*.json"): + r = json.loads(receipt_file.read_text()) + ts = datetime.fromisoformat(r["ended_at"]) + if ts < hot_cutoff: + for art in r.get("artifacts", []): + p = artifacts_dir / art["path"] + if p.exists(): + purged_bytes += p.stat().st_size + p.unlink() + purged_artifacts += 1 + r["artifacts_purged_at"] = now.isoformat() + receipt_file.write_text(json.dumps(r, indent=2)) + + total_mb = _du_mb(catalog_dir) + while total_mb > max_total_mb: + oldest = _find_oldest_artifact(artifacts_dir) + if oldest is None: + break + purged_bytes += oldest.stat().st_size + oldest.unlink() + purged_artifacts += 1 + total_mb = _du_mb(catalog_dir) + + _rotate_daily(catalog_dir / "tuples.jsonl") + + return { + "artifacts_purged": purged_artifacts, + "bytes_freed": purged_bytes, + "total_mb_after": _du_mb(catalog_dir), + } +``` + +#### Schedule + +```cron +# Cron: nightly at 03:00 +0 3 * * * cd ~/Projetos/SendSprint && python -m sendsprint.gc --hot-days 30 --warm-days 365 --max-mb 5000 +``` + +#### Disk pressure circuit breaker + +```python +def check_disk_pressure(catalog_dir: pathlib.Path, free_mb_floor: int = 1000): + stat = os.statvfs(catalog_dir) + free_mb = (stat.f_bavail * stat.f_frsize) / (1024 * 1024) + if free_mb < free_mb_floor: + bus.out_(Tuple(lane="gc.urgent", payload=[{"yool": "fs.gc.run", "args": {}}])) + raise DiskPressure(f"free={free_mb:.0f}MB below floor={free_mb_floor}MB") +``` + +#### Test + +```python +def test_gc_purges_warm_tier_artifacts(tmp_path): + setup_receipts(tmp_path, recent=50, old=50) + result = gc_run(tmp_path, hot_days=30) + assert result["artifacts_purged"] == 50 + assert len(list((tmp_path / "receipts").rglob("*.json"))) == 100 +``` + +### 11.3 Memory & token guardrails + +Every yool implementation MUST: + +- Stream large inputs/outputs to disk rather than buffer. +- Respect `agent_terms.budget_tokens` (LLM calls). +- Emit incremental cost into receipt as work progresses. + +--- + +## 12. Glossary + +- **address space** — set of distinct identifiers a system can refer to. +- **bitmap** — per-HAMT-node bitfield indicating populated child slots. +- **collision** — two keys hashing to the same path beyond max trie depth. +- **lane** — coarse-grained routing key on a tuple. +- **leaf** — terminal HAMT node holding a key/value pair. +- **map_pos** — semantic coordinates inside a tuple. +- **opcode** — synonym for yool name. +- **payload** — ordered list of yool invocations inside a tuple. +- **popcount** — number of bits set in a HAMT node's bitmap. +- **receipt** — immutable, content-addressed record of one yool execution. +- **slot** — index into a HAMT node's children array. +- **structural sharing** — persistent-data-structure update strategy. +- **tuple space** — Linda-style coordination substrate. + +--- + +## 13. Versioning + +| Version | Date | Changes | +|---|---|---| +| v0.1 | 2026-05-19 | Initial draft | +| v0.2 | 2026-05-19 | Expanded examples (SendSprint, llm-project-mapper); guardrails (§11); end-to-end cache/resume flows; HAMT lookup algorithm. | + +--- + +## Appendix A — FAQ + +**Q: Isn't this just Kafka + a registry?** +A: Kafka is the bus, fine. The pattern adds: (1) HAMT-addressed catalog, (2) content-addressable receipts as cache keys, (3) tuple as canonical unit of work with budget/authority. Kafka alone gives transport, not addressing. + +**Q: Why blake2b for hashing instead of sha256?** +A: HAMT addressing benefits from speed over cryptographic strength. blake2b-64 is faster than sha256 and 30 bits suffices for catalog sizes under ~1M. Receipts use sha256 because content-addressing needs collision resistance. + +**Q: How does this interact with my existing MCP server?** +A: Your MCP server becomes the **edge** in §3.1. Expose `catalog.lookup`, `tuple.dispatch`, `tuple.observe`. Don't expose `tuple.in/out`. + +**Q: How do I migrate without breaking prod?** +A: §8 playbook. Dual-read step is key: catalog runs alongside old dict, diff on every lookup. When diffs zero for N days, flip the switch. + +**Q: What about distributed workers?** +A: Tuple-space scales horizontally — workers on different hosts sharing the bus (Kafka, Redis Streams, NATS). HAMT itself is immutable so distribution is read-trivial. + +**Q: Why force guardrails (§11) if my project is small?** +A: Small projects grow. Guardrail cost is low (<=200 LOC for both). Cost of retrofit after a runaway agent fries a laptop is high. + +--- + +## Appendix B — Diagram Index + +- §3.1 — Static layout (3 layers) +- §3.2 — Dynamic flow (one item end-to-end) +- §3.3 — Failure & resume +- §3.4 — HAMT trie insertion +- §3.5 — Linda tuple-space timeline + +--- + +## Appendix C — Quick Vendor Instructions + +```bash +# Vendor the spec +curl -L https://raw.githubusercontent.com/wesleysimplicio/yool-tuple-hamt/main/YOOL_TUPLE_HAMT.md \ + -o YOOL_TUPLE_HAMT.md + +# Or as a submodule (read-only consumer) +git submodule add https://github.com/wesleysimplicio/yool-tuple-hamt vendor/yool-tuple-hamt + +# Copy reference impls +cp vendor/yool-tuple-hamt/scripts/build_hamt.py scripts/ +cp vendor/yool-tuple-hamt/guardrails/cpu_throttle.py src/guardrails/ +cp vendor/yool-tuple-hamt/guardrails/disk_gc.py src/guardrails/ + +# Generate catalog +python scripts/build_hamt.py --source AGENTS.md --output .catalog/capabilities.json + +# Wire workers to use catalog.lookup + receipts + guardrails + +# Add GC schedule (cron / launchd / systemd) +``` diff --git a/bin/build-hamt-catalog b/bin/build-hamt-catalog new file mode 100644 index 0000000..bb5f9ea --- /dev/null +++ b/bin/build-hamt-catalog @@ -0,0 +1,73 @@ +#!/usr/bin/env node +'use strict'; + +const fs = require('node:fs'); +const path = require('node:path'); +const { spawnSync } = require('node:child_process'); + +const PACKAGE_ROOT = path.resolve(__dirname, '..'); +const PYTHON_SCRIPT = path.join(PACKAGE_ROOT, 'scripts', 'build_hamt.py'); +const argv = process.argv.slice(2); + +function printHelp() { + console.log(`build-hamt-catalog + +Build a YOOL/HAMT catalog from AGENTS.md. + +USAGE + build-hamt-catalog [project-root] [--source ] [--output <.catalog/agents.json>] + +EXAMPLES + build-hamt-catalog + build-hamt-catalog . + build-hamt-catalog --source AGENTS.md --output .catalog/agents.json +`); +} + +function commandExists(command) { + const which = process.platform === 'win32' ? 'where' : 'which'; + const result = spawnSync(which, [command], { encoding: 'utf8' }); + return result.status === 0; +} + +function resolvePython() { + if (commandExists('python3')) return { command: 'python3', args: [] }; + if (process.platform === 'win32' && commandExists('py')) return { command: 'py', args: ['-3'] }; + if (commandExists('python')) return { command: 'python', args: [] }; + return null; +} + +function main() { + if (argv.includes('-h') || argv.includes('--help')) { + printHelp(); + process.exit(0); + } + + if (!fs.existsSync(PYTHON_SCRIPT)) { + console.error(`build_hamt.py not found at ${PYTHON_SCRIPT}`); + process.exit(2); + } + + const python = resolvePython(); + if (!python) { + console.error('Python runtime not found. Install python3, or use the Windows py launcher, then retry.'); + process.exit(2); + } + + const result = spawnSync( + python.command, + [...python.args, PYTHON_SCRIPT, ...argv], + { + cwd: process.cwd(), + stdio: 'inherit', + }, + ); + + if (result.error) { + console.error(`Failed to run ${python.command}: ${result.error.message}`); + process.exit(1); + } + process.exit(result.status ?? 1); +} + +main(); diff --git a/bin/cli.js b/bin/cli.js index 3c3d61c..adc76b8 100755 --- a/bin/cli.js +++ b/bin/cli.js @@ -51,6 +51,7 @@ const TEMPLATE_PATHS = [ 'CLAUDE.md', 'INIT.md', '_BOOTSTRAP.md', + 'YOOL_TUPLE_HAMT.md', 'README.md', 'README.pt-BR.md', 'INSTALL.md', @@ -285,10 +286,25 @@ const PRESETS = { }; const argv = process.argv.slice(2); + +if (argv[0] === 'build-hamt-catalog') { + const wrapper = path.join(__dirname, 'build-hamt-catalog'); + const child = spawnSync(process.execPath, [wrapper, ...argv.slice(1)], { + cwd: process.cwd(), + stdio: 'inherit', + }); + if (child.error) { + console.error(`Failed to run build-hamt-catalog: ${child.error.message}`); + process.exit(1); + } + process.exit(child.status ?? 1); +} + const opts = { yes: false, force: false, dryRun: false, + mcpEdge: false, silent: false, skipMeta: false, update: false, @@ -307,6 +323,7 @@ for (let i = 0; i < argv.length; i++) { case '-f': case '--force': opts.force = true; break; case '--dry-run': opts.dryRun = true; break; + case '--mcp-edge': opts.mcpEdge = true; break; case '--update': opts.update = true; break; case '--silent': opts.silent = true; break; case '--skip-meta': opts.skipMeta = true; break; @@ -359,6 +376,7 @@ An automatic local mapping pass starts immediately after the files are applied. USAGE npx @wesleysimplicio/llm-project-mapper [options] + npx @wesleysimplicio/llm-project-mapper build-hamt-catalog [project-root] [--source ] [--output <.catalog/agents.json>] OPTIONS -y, --yes Non-interactive (defaults: no gitignore append, skip CLI handoff) @@ -367,6 +385,7 @@ OPTIONS .github/copilot-instructions.md, .gitignore) --update Safe update mode: --yes --force --append-gitignore yes --cli skip --dry-run Print actions without writing files + --mcp-edge Create mcp/server.ts and mcp/server.py edge adapters --skip-meta Do not write .starter-meta.json --cli Pick CLI for INIT.md handoff (claude|codex|copilot|cursor| deepseek|kimi|minimax|glm|hermes|openclaw|aider|other|skip) @@ -391,6 +410,7 @@ EXAMPLES npx @wesleysimplicio/llm-project-mapper --yes --preset nextjs npx @wesleysimplicio/llm-project-mapper --preset list npx @wesleysimplicio/llm-project-mapper@latest --update + npx @wesleysimplicio/llm-project-mapper build-hamt-catalog DOCS https://github.com/wesleysimplicio/llm-project-mapper @@ -833,6 +853,134 @@ function upsertGitignore(existing) { return before + '\n\n' + RECOMMENDED_IGNORES + (after ? '\n' + after : '\n'); } +function writeFileIfMissing(relPath, content) { + const abs = path.join(CWD, relPath); + if (fs.existsSync(abs)) { + log(` preserve (exists): ${relPath}`); + return; + } + if (opts.dryRun) { + log(` create (dry): ${relPath}`); + return; + } + fs.mkdirSync(path.dirname(abs), { recursive: true }); + fs.writeFileSync(abs, content); + log(` create: ${relPath}`); +} + +function handleRuntimeScaffold() { + const catalogStub = `${JSON.stringify({ + version: 1, + generated_at: null, + agents: [], + notes: [ + 'Generated from AGENTS.md entries that declare yool_id, authority, lane, and agent_terms.', + 'Refresh with build-hamt-catalog after the wrapper is installed.', + ], + }, null, 2)}\n`; + const mcpServerTs = `#!/usr/bin/env node +/** + * WARNING: MCP is an edge adapter only. + * Do not move the inner agent loop into this file. + */ +import { readFile } from "node:fs/promises"; +import path from "node:path"; +import process from "node:process"; + +const catalogPath = path.resolve(process.cwd(), ".catalog", "agents.json"); + +async function readCatalog() { + const raw = await readFile(catalogPath, "utf8"); + return JSON.parse(raw); +} + +export async function snapshot() { + return { + catalogPath, + catalog: await readCatalog(), + generatedAt: new Date().toISOString(), + }; +} + +export async function dispatch(tuple) { + return { + accepted: false, + reason: "TODO: wire tuple dispatch to the edge transport for your host runtime.", + tuple, + catalogPath, + }; +} + +if (process.argv[1] && import.meta.url === new URL(\`file://\${process.argv[1]}\`).href) { + const command = process.argv[2] ?? "snapshot"; + if (command === "snapshot") { + console.log(JSON.stringify(await snapshot(), null, 2)); + } else if (command === "dispatch") { + const tuple = process.argv[3] ? JSON.parse(process.argv[3]) : {}; + console.log(JSON.stringify(await dispatch(tuple), null, 2)); + } else { + console.error(\`Unknown command: \${command}\`); + process.exit(1); + } +} +`; + const mcpServerPy = `#!/usr/bin/env python3 +""" +WARNING: MCP is an edge adapter only. +Do not move the inner agent loop into this file. +""" +from __future__ import annotations + +import json +import pathlib +import sys +from datetime import datetime, timezone +from typing import Any + +CATALOG_PATH = pathlib.Path.cwd() / ".catalog" / "agents.json" + + +def read_catalog() -> dict[str, Any]: + return json.loads(CATALOG_PATH.read_text(encoding="utf-8")) + + +def snapshot() -> dict[str, Any]: + return { + "catalogPath": str(CATALOG_PATH), + "catalog": read_catalog(), + "generatedAt": datetime.now(timezone.utc).isoformat(), + } + + +def dispatch(tuple_payload: dict[str, Any]) -> dict[str, Any]: + return { + "accepted": False, + "reason": "TODO: wire tuple dispatch to the edge transport for your host runtime.", + "tuple": tuple_payload, + "catalogPath": str(CATALOG_PATH), + } + + +if __name__ == "__main__": + command = sys.argv[1] if len(sys.argv) > 1 else "snapshot" + if command == "snapshot": + print(json.dumps(snapshot(), indent=2)) + elif command == "dispatch": + payload = json.loads(sys.argv[2]) if len(sys.argv) > 2 else {} + print(json.dumps(dispatch(payload), indent=2)) + else: + raise SystemExit(f"Unknown command: {command}") +`; + + writeFileIfMissing(path.join('.catalog', '.gitkeep'), ''); + writeFileIfMissing(path.join('.catalog', 'agents.json'), catalogStub); + writeFileIfMissing(path.join('.receipts', '.gitkeep'), ''); + if (opts.mcpEdge) { + writeFileIfMissing(path.join('mcp', 'server.ts'), mcpServerTs); + writeFileIfMissing(path.join('mcp', 'server.py'), mcpServerPy); + } +} + function looksBinary(buf) { const head = buf.length > 8192 ? buf.subarray(0, 8192) : buf; return head.includes(0); @@ -910,6 +1058,7 @@ function writeMeta(productName, stack, projectMode, projectsList, existingInstru bootstrapped_at: new Date().toISOString(), starter_version: PKG.version, cli: '@wesleysimplicio/llm-project-mapper', + mcp_edge_enabled: opts.mcpEdge, preset: opts.preset || null, existing_instruction_files: existingInstructionFiles, preserved_user_files: preservedUserFiles, @@ -1106,6 +1255,7 @@ async function main() { const rl = readline.createInterface({ input: process.stdin, output: process.stdout }); try { await handleGitignore(rl); + handleRuntimeScaffold(); substitute(productName, stack); writeMeta(productName, stack, projectMode, projectsList, existingInstructionFiles, preservedUserFiles); if (!opts.dryRun) { diff --git a/bootstrap.ps1 b/bootstrap.ps1 index 10d130a..32465d2 100644 --- a/bootstrap.ps1 +++ b/bootstrap.ps1 @@ -30,13 +30,15 @@ .EXAMPLE PS> .\bootstrap.ps1 PS> .\bootstrap.ps1 -NonInteractive -Cli claude -AppendGitignore yes + PS> .\bootstrap.ps1 -NonInteractive -Cli codex -AppendGitignore yes -McpEdge #> [CmdletBinding()] param( [switch]$NonInteractive, [string]$Cli = "", [ValidateSet("yes","no","")] - [string]$AppendGitignore = "" + [string]$AppendGitignore = "", + [switch]$McpEdge ) $ErrorActionPreference = "Stop" @@ -309,6 +311,10 @@ pnpm-debug.log* *.tgz *.tar.gz +# Runtime receipts +.receipts/** +!.receipts/.gitkeep + # LLM Project Mapper tracked files .starter-meta.json .claude/settings.local.json @@ -381,6 +387,152 @@ function Handle-Gitignore { Handle-Gitignore Write-Host "" +# --------------------------------------------------------------------------- +# runtime scaffold (.catalog, .receipts, optional MCP edge starter) +# --------------------------------------------------------------------------- +function Handle-RuntimeScaffold { + New-Item -ItemType Directory -Force -Path ".catalog" | Out-Null + New-Item -ItemType Directory -Force -Path ".receipts" | Out-Null + Set-Content -Path ".catalog/.gitkeep" -Value "" -Encoding UTF8 + Set-Content -Path ".receipts/.gitkeep" -Value "" -Encoding UTF8 + + if (-not (Test-Path ".catalog/agents.json")) { + @" +{ + "version": 1, + "generated_at": null, + "agents": [], + "notes": [ + "Generated from AGENTS.md entries that declare yool_id, authority, lane, and agent_terms.", + "Refresh with bin/build-hamt-catalog after the wrapper is installed." + ] +} +"@ | Set-Content -Path ".catalog/agents.json" -Encoding UTF8 + Write-Host "-> .catalog/agents.json stub created." + } else { + Write-Host "-> .catalog/agents.json already exists (preserved)." + } + + Write-Host "-> .catalog/.gitkeep and .receipts/.gitkeep ensured." +} + +function Handle-McpEdgeScaffold { + if (-not $script:McpEdge) { return } + + New-Item -ItemType Directory -Force -Path "mcp" | Out-Null + + if (-not (Test-Path "mcp/server.ts")) { + @' +#!/usr/bin/env node +/** + * WARNING: MCP is an edge adapter only. + * Do not move the inner agent loop into this file. + */ +import { readFile } from "node:fs/promises"; +import path from "node:path"; +import process from "node:process"; + +const catalogPath = path.resolve(process.cwd(), ".catalog", "agents.json"); + +async function readCatalog() { + const raw = await readFile(catalogPath, "utf8"); + return JSON.parse(raw); +} + +export async function snapshot() { + return { + catalogPath, + catalog: await readCatalog(), + generatedAt: new Date().toISOString() + }; +} + +export async function dispatch(tuple) { + return { + accepted: false, + reason: "TODO: wire tuple dispatch to the edge transport for your host runtime.", + tuple, + catalogPath + }; +} + +if (process.argv[1] && import.meta.url === new URL(`file://${process.argv[1]}`).href) { + const command = process.argv[2] ?? "snapshot"; + if (command === "snapshot") { + console.log(JSON.stringify(await snapshot(), null, 2)); + } else if (command === "dispatch") { + const tuple = process.argv[3] ? JSON.parse(process.argv[3]) : {}; + console.log(JSON.stringify(await dispatch(tuple), null, 2)); + } else { + console.error(`Unknown command: ${command}`); + process.exit(1); + } +} +'@ | Set-Content -Path "mcp/server.ts" -Encoding UTF8 + Write-Host "-> mcp/server.ts created." + } else { + Write-Host "-> mcp/server.ts already exists (preserved)." + } + + if (-not (Test-Path "mcp/server.py")) { + @' +#!/usr/bin/env python3 +""" +WARNING: MCP is an edge adapter only. +Do not move the inner agent loop into this file. +""" +from __future__ import annotations + +import json +import pathlib +import sys +from datetime import datetime, timezone +from typing import Any + +CATALOG_PATH = pathlib.Path.cwd() / ".catalog" / "agents.json" + + +def read_catalog() -> dict[str, Any]: + return json.loads(CATALOG_PATH.read_text(encoding="utf-8")) + + +def snapshot() -> dict[str, Any]: + return { + "catalogPath": str(CATALOG_PATH), + "catalog": read_catalog(), + "generatedAt": datetime.now(timezone.utc).isoformat(), + } + + +def dispatch(tuple_payload: dict[str, Any]) -> dict[str, Any]: + return { + "accepted": False, + "reason": "TODO: wire tuple dispatch to the edge transport for your host runtime.", + "tuple": tuple_payload, + "catalogPath": str(CATALOG_PATH), + } + + +if __name__ == "__main__": + command = sys.argv[1] if len(sys.argv) > 1 else "snapshot" + if command == "snapshot": + print(json.dumps(snapshot(), indent=2)) + elif command == "dispatch": + payload = json.loads(sys.argv[2]) if len(sys.argv) > 2 else {} + print(json.dumps(dispatch(payload), indent=2)) + else: + raise SystemExit(f"Unknown command: {command}") +'@ | Set-Content -Path "mcp/server.py" -Encoding UTF8 + Write-Host "-> mcp/server.py created." + } else { + Write-Host "-> mcp/server.py already exists (preserved)." + } +} + +Handle-RuntimeScaffold +Handle-McpEdgeScaffold +Write-Host "" + # --------------------------------------------------------------------------- # .starter-meta.json (machine-readable handoff for INIT.md) # --------------------------------------------------------------------------- @@ -396,7 +548,8 @@ $meta = [ordered]@{ project_mode = $ProjectMode projects = $ProjectsList bootstrapped_at = (Get-Date).ToUniversalTime().ToString("yyyy-MM-ddTHH:mm:ssZ") - starter_version = "0.3.0" + starter_version = "0.4.0" + mcp_edge_enabled = [bool]$McpEdge existing_instruction_files = $ExistingInstructionFiles init_must_ask = @() init_must_infer = @("team","domain","vision_oneliner","personas_beyond_dev") @@ -416,7 +569,7 @@ $InitPrompt = 'Read INIT.md and execute it. Do NOT modify any user source files $CliOpts = @( @{ Key="claude"; Label="Claude Code"; Cmd="claude" }, @{ Key="codex"; Label="Codex CLI"; Cmd="codex" }, - @{ Key="copilot"; Label="GitHub Copilot CLI (chat — no agent loop)"; Cmd="gh" }, + @{ Key="copilot"; Label="GitHub Copilot CLI (chat - no agent loop)"; Cmd="gh" }, @{ Key="cursor"; Label="Cursor Agent (cursor-agent)"; Cmd="cursor-agent" }, @{ Key="deepseek"; Label="Deepseek (via aider --model deepseek/deepseek-coder)"; Cmd="aider" }, @{ Key="kimi"; Label="Kimi K2.6 (via aider --model openrouter/moonshotai/kimi-k2)"; Cmd="aider" }, @@ -426,7 +579,7 @@ $CliOpts = @( @{ Key="openclaw"; Label="OpenClaw"; Cmd="openclaw" }, @{ Key="aider"; Label="Aider (pick model interactively)"; Cmd="aider" }, @{ Key="other"; Label="Other / manual (copy prompt to clipboard)"; Cmd="" }, - @{ Key="skip"; Label="Skip — I will run INIT.md later"; Cmd="" } + @{ Key="skip"; Label="Skip - I will run INIT.md later"; Cmd="" } ) function Has-Cmd($name) { diff --git a/bootstrap.sh b/bootstrap.sh index 7863691..d263d95 100755 --- a/bootstrap.sh +++ b/bootstrap.sh @@ -37,7 +37,7 @@ # ./bootstrap.sh # # Usage non-interactive (CI): -# ./bootstrap.sh --yes --cli claude --append-gitignore yes +# ./bootstrap.sh --yes --cli claude --append-gitignore yes --mcp-edge set -euo pipefail @@ -47,12 +47,14 @@ set -euo pipefail NON_INTERACTIVE=0 CLI_PRESET="" APPEND_GITIGNORE_PRESET="" +ENABLE_MCP_EDGE=0 while [[ $# -gt 0 ]]; do case "$1" in -y|--yes) NON_INTERACTIVE=1; shift ;; --cli) CLI_PRESET="$2"; shift 2 ;; --append-gitignore) APPEND_GITIGNORE_PRESET="$2"; shift 2 ;; # yes|no + --mcp-edge) ENABLE_MCP_EDGE=1; shift ;; -h|--help) sed -n '2,33p' "$0"; exit 0 ;; *) echo "Unknown flag: $1" >&2; exit 1 ;; esac @@ -382,6 +384,10 @@ pnpm-debug.log* *.tgz *.tar.gz +# Runtime receipts +.receipts/** +!.receipts/.gitkeep + # LLM Project Mapper tracked files .starter-meta.json .claude/settings.local.json @@ -456,35 +462,148 @@ handle_gitignore echo "" # --------------------------------------------------------------------------- -# .catalog/ skeleton (yool/tuple/HAMT pattern, see docs/YOOL_TUPLE_HAMT.md) +# runtime scaffold (.catalog, .receipts, optional MCP edge starter) # --------------------------------------------------------------------------- -handle_catalog_skeleton() { - mkdir -p .catalog/receipts .catalog/artifacts - : > .catalog/receipts/.gitkeep - : > .catalog/artifacts/.gitkeep - if [[ ! -f .catalog/README.md ]]; then - cat > .catalog/README.md <<'CATALOG_EOF' -# .catalog/ - -Runtime catalog for the yool/tuple/HAMT pattern (see `docs/YOOL_TUPLE_HAMT.md`). - -| Path | Purpose | -|---|---| -| `hamt.json` | HAMT registry built from AGENTS.md by `bin/build-hamt-catalog` | -| `tuples.jsonl` | Append-only log of tuple-space operations (`out`/`in`) | -| `receipts/.json` | Content-addressable execution records (immutable) | -| `artifacts/` | Body files referenced by receipts; subject to GC per §11.2 | - -`receipts/` is the immutable Merkle chain — NEVER deleted, only artifact bodies are. -GC runs nightly per the disk guardrail (spec §11.2). +handle_runtime_scaffold() { + mkdir -p .catalog .receipts + : > .catalog/.gitkeep + : > .receipts/.gitkeep + + if [[ ! -f .catalog/agents.json ]]; then + cat > .catalog/agents.json <<'CATALOG_EOF' +{ + "version": 1, + "generated_at": null, + "agents": [], + "notes": [ + "Generated from AGENTS.md entries that declare yool_id, authority, lane, and agent_terms.", + "Refresh with bin/build-hamt-catalog after the wrapper is installed." + ] +} CATALOG_EOF - echo "-> .catalog/ skeleton created." + echo "-> .catalog/agents.json stub created." + else + echo "-> .catalog/agents.json already exists (preserved)." + fi + + echo "-> .catalog/.gitkeep and .receipts/.gitkeep ensured." +} + +handle_mcp_edge_scaffold() { + [[ "$ENABLE_MCP_EDGE" == "1" ]] || return 0 + + mkdir -p mcp + + if [[ ! -f mcp/server.ts ]]; then + cat > mcp/server.ts <<'MCP_TS_EOF' +#!/usr/bin/env node +/** + * WARNING: MCP is an edge adapter only. + * Do not move the inner agent loop into this file. + */ +import { readFile } from "node:fs/promises"; +import path from "node:path"; +import process from "node:process"; + +const catalogPath = path.resolve(process.cwd(), ".catalog", "agents.json"); + +async function readCatalog() { + const raw = await readFile(catalogPath, "utf8"); + return JSON.parse(raw); +} + +export async function snapshot() { + return { + catalogPath, + catalog: await readCatalog(), + generatedAt: new Date().toISOString() + }; +} + +export async function dispatch(tuple) { + return { + accepted: false, + reason: "TODO: wire tuple dispatch to the edge transport for your host runtime.", + tuple, + catalogPath + }; +} + +if (process.argv[1] && import.meta.url === new URL(`file://${process.argv[1]}`).href) { + const command = process.argv[2] ?? "snapshot"; + if (command === "snapshot") { + console.log(JSON.stringify(await snapshot(), null, 2)); + } else if (command === "dispatch") { + const tuple = process.argv[3] ? JSON.parse(process.argv[3]) : {}; + console.log(JSON.stringify(await dispatch(tuple), null, 2)); + } else { + console.error(`Unknown command: ${command}`); + process.exit(1); + } +} +MCP_TS_EOF + echo "-> mcp/server.ts created." + else + echo "-> mcp/server.ts already exists (preserved)." + fi + + if [[ ! -f mcp/server.py ]]; then + cat > mcp/server.py <<'MCP_PY_EOF' +#!/usr/bin/env python3 +""" +WARNING: MCP is an edge adapter only. +Do not move the inner agent loop into this file. +""" +from __future__ import annotations + +import json +import pathlib +import sys +from datetime import datetime, timezone +from typing import Any + +CATALOG_PATH = pathlib.Path.cwd() / ".catalog" / "agents.json" + + +def read_catalog() -> dict[str, Any]: + return json.loads(CATALOG_PATH.read_text(encoding="utf-8")) + + +def snapshot() -> dict[str, Any]: + return { + "catalogPath": str(CATALOG_PATH), + "catalog": read_catalog(), + "generatedAt": datetime.now(timezone.utc).isoformat(), + } + + +def dispatch(tuple_payload: dict[str, Any]) -> dict[str, Any]: + return { + "accepted": False, + "reason": "TODO: wire tuple dispatch to the edge transport for your host runtime.", + "tuple": tuple_payload, + "catalogPath": str(CATALOG_PATH), + } + + +if __name__ == "__main__": + command = sys.argv[1] if len(sys.argv) > 1 else "snapshot" + if command == "snapshot": + print(json.dumps(snapshot(), indent=2)) + elif command == "dispatch": + payload = json.loads(sys.argv[2]) if len(sys.argv) > 2 else {} + print(json.dumps(dispatch(payload), indent=2)) + else: + raise SystemExit(f"Unknown command: {command}") +MCP_PY_EOF + echo "-> mcp/server.py created." else - echo "-> .catalog/ already exists (skeleton skipped)." + echo "-> mcp/server.py already exists (preserved)." fi } -handle_catalog_skeleton +handle_runtime_scaffold +handle_mcp_edge_scaffold echo "" # --------------------------------------------------------------------------- @@ -506,7 +625,8 @@ cat > .starter-meta.json < bin/build-hamt-catalog + -> .catalog/agents.json + -> MCP snapshot / dispatch edge + -> workers subscribe by lane + -> .receipts/ evidence +``` + +## MCP is the edge + +MCP is for read-mostly snapshots and external dispatch entrypoints. Keep tuple matching, scheduling, retries, and worker coordination outside the MCP adapter. + +## Canonical spec + +- [Root spec on GitHub](https://github.com/wesleysimplicio/llm-project-mapper/blob/main/YOOL_TUPLE_HAMT.md) +- [Vendored source in this repo](https://github.com/wesleysimplicio/llm-project-mapper/blob/main/docs/YOOL_TUPLE_HAMT.md) diff --git a/docs-site/docs/intro.md b/docs-site/docs/intro.md index e15c31a..5cb766e 100644 --- a/docs-site/docs/intro.md +++ b/docs-site/docs/intro.md @@ -18,6 +18,7 @@ This docs site is a curated view over the repo's real documentation. The source - **Quickstart**: install the starter in seconds and choose the right bootstrap path. - **Guide**: use the private overlay workflow on top of an existing host project. - **Concepts**: understand the architecture and domain map conventions the starter ships. +- **YOOL / tuple / HAMT**: inspect the capability-addressing pattern used by the scaffold. - **Reference**: inspect CLI flags, the init handoff contract, and the SessionStart hook behavior. - **Community**: contribute improvements and add your real-world showcase. @@ -28,5 +29,6 @@ This docs site is a curated view over the repo's real documentation. The source | [Quickstart](/quickstart/get-going) | Fast install path, prerequisites, and starter overview | | [Guide](/guide/private-overlay) | Private overlay installation on an existing repo | | [Concepts](/concepts/skills-and-agents) | How the starter organizes skills, agents, and shared context | +| [YOOL / tuple / HAMT](/yool-tuple-hamt) | Capability addressing, receipts, and MCP edge guidance | | [Reference](/reference/init-handoff) | Bootstrap contract, CLI flags, hooks, and generated docs | | [Community](/community/showcase) | Showcase entries and contribution workflow | diff --git a/docs-site/docusaurus.config.cjs b/docs-site/docusaurus.config.cjs index 10b110e..9a8ef41 100644 --- a/docs-site/docusaurus.config.cjs +++ b/docs-site/docusaurus.config.cjs @@ -74,6 +74,7 @@ const config = { {to: '/', label: 'Docs Home', position: 'left'}, {to: '/quickstart/get-going', label: 'Quickstart', position: 'left'}, {to: '/guide/private-overlay', label: 'Guide', position: 'left'}, + {to: '/yool-tuple-hamt', label: 'YOOL / tuple / HAMT', position: 'left'}, {to: '/reference/cli-flags', label: 'Reference', position: 'left'}, {type: 'docsVersionDropdown', position: 'right'}, { @@ -100,6 +101,7 @@ const config = { {label: 'Architecture Map', to: '/concepts/architecture-map'}, {label: 'Domain Map', to: '/concepts/domain-map'}, {label: 'Skills and Agents', to: '/concepts/skills-and-agents'}, + {label: 'YOOL / tuple / HAMT', to: '/yool-tuple-hamt'}, ], }, { diff --git a/docs-site/src/pages/index.mdx b/docs-site/src/pages/index.mdx index 9019b47..fc85951 100644 --- a/docs-site/src/pages/index.mdx +++ b/docs-site/src/pages/index.mdx @@ -13,6 +13,7 @@ This landing page keeps the docs hub reachable at the site root while the versio - [Current docs (v0.x)](/next/) - [Stable snapshot](/intro) +- [YOOL / tuple / HAMT pattern](/yool-tuple-hamt) - [GitHub repository](https://github.com/wesleysimplicio/llm-project-mapper) ## What you will find diff --git a/docs-site/src/pages/yool-tuple-hamt.mdx b/docs-site/src/pages/yool-tuple-hamt.mdx new file mode 100644 index 0000000..2448acf --- /dev/null +++ b/docs-site/src/pages/yool-tuple-hamt.mdx @@ -0,0 +1,57 @@ +# YOOL / tuple / HAMT + +The YOOL / tuple / HAMT pattern is the capability-addressing model that LLM Project Mapper is standardizing for multi-agent repositories. + +## Why it exists + +- Agents need stable capability ids, not only friendly names. +- Dispatch needs explicit authority and lane routing. +- Receipts need a canonical place and schema. +- MCP should stay at the edge for snapshot and dispatch, not become the inner loop. + +## Try it + +```bash +npx @wesleysimplicio/llm-project-mapper --mcp-edge +bin/build-hamt-catalog +``` + +That flow gives you: + +- `.catalog/agents.json` as the generated HAMT catalog +- `.receipts/` as the default local receipt store +- `mcp/server.ts` and `mcp/server.py` as edge adapters when `--mcp-edge` is enabled + +## Static layout + +```json +{ + "yool_id": "agent.dev.python.v1", + "authority": "dev", + "lane": "fast", + "agent_terms": { + "cpu_quota_pct": 60, + "disk_quota_mb": 100 + } +} +``` + +## Dynamic flow + +```text +AGENTS.md + -> bin/build-hamt-catalog + -> .catalog/agents.json + -> MCP snapshot / dispatch edge + -> workers subscribe by lane + -> .receipts/ evidence +``` + +## MCP is the edge + +MCP is for read-mostly snapshots and external dispatch entrypoints. Keep tuple matching, scheduling, retries, and worker coordination outside the MCP adapter. + +## Canonical spec + +- [Root spec on GitHub](https://github.com/wesleysimplicio/llm-project-mapper/blob/main/YOOL_TUPLE_HAMT.md) +- [Vendored source in this repo](https://github.com/wesleysimplicio/llm-project-mapper/blob/main/docs/YOOL_TUPLE_HAMT.md) diff --git a/package-lock.json b/package-lock.json index 4109b7f..5eb9c3c 100644 --- a/package-lock.json +++ b/package-lock.json @@ -9,6 +9,7 @@ "version": "0.4.2", "license": "MIT", "bin": { + "build-hamt-catalog": "bin/build-hamt-catalog", "llm-project-mapper": "bin/cli.js" }, "devDependencies": { diff --git a/package.json b/package.json index 758617e..d9dbc27 100644 --- a/package.json +++ b/package.json @@ -4,7 +4,8 @@ "description": "AI-friendly project scaffold with AGENTS.md ecosystem (Claude Code, Codex, Copilot, Cursor, Aider, Hermes, OpenClaw). Specs as code, atomic tasks, automated Definition of Done, reusable skills, multi-agent ready.", "type": "commonjs", "bin": { - "llm-project-mapper": "bin/cli.js" + "llm-project-mapper": "bin/cli.js", + "build-hamt-catalog": "bin/build-hamt-catalog" }, "files": [ "LICENSE", @@ -17,6 +18,7 @@ "INIT.md", "INIT.en.md", "_BOOTSTRAP.md", + "YOOL_TUPLE_HAMT.md", "README.md", "README.pt-BR.md", "INSTALL.md", diff --git a/scripts/build_hamt.py b/scripts/build_hamt.py new file mode 100644 index 0000000..8b0675b --- /dev/null +++ b/scripts/build_hamt.py @@ -0,0 +1,290 @@ +#!/usr/bin/env python3 +"""Build a YOOL/HAMT agent catalog from AGENTS.md. + +This script is stdlib-only by design so the Node wrapper can vendor it into +projects without adding Python dependencies. +""" + +from __future__ import annotations + +import argparse +import hashlib +import json +import pathlib +import re +import sys +from dataclasses import dataclass +from datetime import datetime, timezone +from typing import Any + +BRANCH_BITS = 5 +BRANCH_FACTOR = 1 << BRANCH_BITS +MAX_LEVELS = 6 +HEADING_RE = re.compile(r"^###\s+(?P.+?)\s*$") +FIELD_RE = re.compile(r"^-\s+(?P[a-zA-Z_][\w-]*)\s*:\s*(?P.*)$") +INDENT_FIELD_RE = re.compile(r"^\s{2,}(?P[a-zA-Z_][\w-]*)\s*:\s*(?P.*)$") + + +@dataclass +class Leaf: + key: str + value: dict[str, Any] + hash_value: int + + +def yool_hash(name: str) -> int: + digest = hashlib.blake2b(name.encode("utf-8"), digest_size=8).digest() + return int.from_bytes(digest, "big") & ((1 << (BRANCH_BITS * MAX_LEVELS)) - 1) + + +def hash_hex(name: str) -> str: + return hashlib.blake2b(name.encode("utf-8"), digest_size=8).hexdigest() + + +def slot_path(hash_value: int) -> list[int]: + mask = BRANCH_FACTOR - 1 + return [ + (hash_value >> ((MAX_LEVELS - 1 - level) * BRANCH_BITS)) & mask + for level in range(MAX_LEVELS) + ] + + +def canonical_json(data: Any) -> str: + return json.dumps(data, sort_keys=True, separators=(",", ":"), ensure_ascii=False) + + +def parse_scalar(raw: str) -> Any: + value = raw.strip() + if value.startswith("`") and value.endswith("`") and len(value) >= 2: + value = value[1:-1] + if value.startswith('"') and value.endswith('"') and len(value) >= 2: + return value[1:-1] + if value.startswith("'") and value.endswith("'") and len(value) >= 2: + return value[1:-1] + if value.lower() in {"true", "false"}: + return value.lower() == "true" + if re.fullmatch(r"-?\d+", value): + return int(value) + if re.fullmatch(r"-?\d+\.\d+", value): + return float(value) + if value.startswith("[") and value.endswith("]"): + try: + return json.loads(value.replace("'", '"')) + except json.JSONDecodeError: + parts = [item.strip().strip("`").strip("'").strip('"') for item in value[1:-1].split(",")] + return [item for item in parts if item] + return value + + +def heading_anchor(name: str) -> str: + slug = re.sub(r"[^a-z0-9]+", "-", name.lower()).strip("-") + return slug or "agent" + + +def parse_agent_terms(lines: list[str], start_index: int) -> tuple[dict[str, Any], int]: + terms: dict[str, Any] = {} + index = start_index + while index < len(lines): + line = lines[index] + if not line.strip(): + index += 1 + continue + if line.startswith("### "): + break + if line.startswith("- "): + break + match = INDENT_FIELD_RE.match(line) + if not match: + break + terms[match.group("key")] = parse_scalar(match.group("value")) + index += 1 + return terms, index + + +def parse_agents(markdown: str, source_name: str) -> tuple[list[dict[str, Any]], list[dict[str, Any]]]: + lines = markdown.splitlines() + parsed: list[dict[str, Any]] = [] + skipped: list[dict[str, Any]] = [] + current_name: str | None = None + current: dict[str, Any] | None = None + heading_line = 0 + index = 0 + + def finalize() -> None: + nonlocal current_name, current, heading_line + if not current_name or current is None: + return + required = ["yool_id", "authority", "lane", "agent_terms"] + missing = [field for field in required if not current.get(field)] + entry = { + "name": current_name, + "source": {"file": source_name, "line": heading_line, "anchor": heading_anchor(current_name)}, + **current, + } + if missing: + skipped.append({"name": current_name, "missing": missing, "line": heading_line}) + else: + parsed.append(entry) + current_name = None + current = None + heading_line = 0 + + while index < len(lines): + line = lines[index] + heading = HEADING_RE.match(line) + if heading: + finalize() + current_name = heading.group("name").strip() + current = {} + heading_line = index + 1 + index += 1 + continue + if current is None: + index += 1 + continue + field_match = FIELD_RE.match(line) + if field_match: + key = field_match.group("key") + raw_value = field_match.group("value") + if key == "agent_terms": + terms, index = parse_agent_terms(lines, index + 1) + current[key] = terms + continue + current[key] = parse_scalar(raw_value) + index += 1 + + finalize() + return parsed, skipped + + +def blank_node() -> dict[str, Any]: + return {"bitmap": 0, "children": {}} + + +def insert_leaf(root: dict[str, Any], leaf: Leaf, level: int = 0) -> None: + if level >= MAX_LEVELS: + slot = leaf.hash_value & (BRANCH_FACTOR - 1) + children = root["children"] + existing = children.get(str(slot)) + if existing is None: + root["bitmap"] |= 1 << slot + children[str(slot)] = { + "kind": "collision", + "hash_prefix": f"{leaf.hash_value:08x}", + "leaves": [leaf.value], + } + return + if existing["kind"] == "collision": + existing["leaves"].append(leaf.value) + return + raise RuntimeError("unexpected non-collision node at max depth") + + slot = slot_path(leaf.hash_value)[level] + children = root["children"] + key = str(slot) + existing = children.get(key) + + if existing is None: + root["bitmap"] |= 1 << slot + children[key] = {"kind": "leaf", "entry": leaf.value} + return + + if existing["kind"] == "leaf": + prior_entry = existing["entry"] + if prior_entry["yool_id"] == leaf.key: + existing["entry"] = leaf.value + return + subnode = blank_node() + insert_leaf(subnode, Leaf(prior_entry["yool_id"], prior_entry, yool_hash(prior_entry["yool_id"])), level + 1) + insert_leaf(subnode, leaf, level + 1) + children[key] = {"kind": "node", **subnode} + return + + if existing["kind"] == "node": + insert_leaf(existing, leaf, level + 1) + return + + if existing["kind"] == "collision": + existing["leaves"].append(leaf.value) + return + + raise RuntimeError(f"unknown HAMT node kind: {existing['kind']}") + + +def build_catalog(entries: list[dict[str, Any]], source_file: pathlib.Path) -> dict[str, Any]: + root = blank_node() + leaves: list[dict[str, Any]] = [] + + for entry in entries: + hash_value = yool_hash(entry["yool_id"]) + leaf = { + **entry, + "hash": { + "algorithm": "blake2b-64-truncated-30", + "hex": hash_hex(entry["yool_id"]), + "value": hash_value, + "slots": slot_path(hash_value), + }, + } + leaves.append(leaf) + insert_leaf(root, Leaf(entry["yool_id"], leaf, hash_value)) + + payload = { + "schema": "yool-catalog/v1", + "generated_at": datetime.now(timezone.utc).isoformat(), + "source": str(source_file), + "entries": sorted(leaves, key=lambda item: item["yool_id"]), + "hamt": { + "algorithm": "hamt/blake2b-30/v1", + "branch_bits": BRANCH_BITS, + "branch_factor": BRANCH_FACTOR, + "max_levels": MAX_LEVELS, + "root": root, + }, + } + payload["id"] = "sha256:" + hashlib.sha256(canonical_json(payload).encode("utf-8")).hexdigest() + payload["stats"] = { + "entries": len(leaves), + "root_popcount": root["bitmap"].bit_count(), + } + return payload + + +def parse_args(argv: list[str]) -> argparse.Namespace: + parser = argparse.ArgumentParser(description="Build a HAMT catalog from AGENTS.md") + parser.add_argument("project_root", nargs="?", default=".", help="Project root used to resolve defaults") + parser.add_argument("--source", help="Path to AGENTS.md") + parser.add_argument("--output", help="Path to output catalog JSON") + return parser.parse_args(argv) + + +def main(argv: list[str]) -> int: + args = parse_args(argv) + project_root = pathlib.Path(args.project_root).resolve() + source_file = pathlib.Path(args.source).resolve() if args.source else project_root / "AGENTS.md" + output_file = pathlib.Path(args.output).resolve() if args.output else project_root / ".catalog" / "agents.json" + + if not source_file.exists(): + print(f"[build] AGENTS source not found: {source_file}", file=sys.stderr) + return 2 + + parsed, skipped = parse_agents(source_file.read_text(encoding="utf-8"), source_file.name) + catalog = build_catalog(parsed, source_file) + catalog["stats"]["parsed_agents"] = len(parsed) + catalog["stats"]["skipped_agents"] = len(skipped) + if skipped: + catalog["skipped"] = skipped + + output_file.parent.mkdir(parents=True, exist_ok=True) + output_file.write_text(json.dumps(catalog, indent=2, ensure_ascii=False) + "\n", encoding="utf-8") + + print(f"[build] parsed {len(parsed)} agent(s) from {source_file.name}") + if skipped: + print(f"[build] skipped {len(skipped)} incomplete agent(s)") + print(f"[build] wrote {output_file}") + print(f"[build] root popcount: {catalog['stats']['root_popcount']}/{BRANCH_FACTOR}") + return 0 + + +if __name__ == "__main__": + raise SystemExit(main(sys.argv[1:])) diff --git a/scripts/sync-docs-site.mjs b/scripts/sync-docs-site.mjs index 7dfc99c..5b36314 100644 --- a/scripts/sync-docs-site.mjs +++ b/scripts/sync-docs-site.mjs @@ -53,6 +53,7 @@ This docs site is a curated view over the repo's real documentation. The source - **Quickstart**: install the starter in seconds and choose the right bootstrap path. - **Guide**: use the private overlay workflow on top of an existing host project. - **Concepts**: understand the architecture and domain map conventions the starter ships. +- **YOOL / tuple / HAMT**: inspect the capability-addressing pattern used by the scaffold. - **Reference**: inspect CLI flags, the init handoff contract, and the SessionStart hook behavior. - **Community**: contribute improvements and add your real-world showcase. @@ -63,6 +64,7 @@ This docs site is a curated view over the repo's real documentation. The source | [Quickstart](/quickstart/get-going) | Fast install path, prerequisites, and starter overview | | [Guide](/guide/private-overlay) | Private overlay installation on an existing repo | | [Concepts](/concepts/skills-and-agents) | How the starter organizes skills, agents, and shared context | +| [YOOL / tuple / HAMT](/yool-tuple-hamt) | Capability addressing, receipts, and MCP edge guidance | | [Reference](/reference/init-handoff) | Bootstrap contract, CLI flags, hooks, and generated docs | | [Community](/community/showcase) | Showcase entries and contribution workflow | `, @@ -112,6 +114,71 @@ The repository already contains the source docs. This site simply gives them a n - GitHub Pages deployment on every push to \`main\` For the raw source files, see the repository paths referenced throughout this site. +`, + }, + { + target: 'concepts/yool-tuple-hamt.md', + title: 'YOOL / tuple / HAMT', + description: 'Capability addressing pattern, receipt flow, and MCP edge guidance.', + sidebarPosition: 4, + slug: '/yool-tuple-hamt', + content: `# YOOL / tuple / HAMT + +The YOOL / tuple / HAMT pattern is the capability-addressing model that LLM Project Mapper is standardizing for multi-agent repositories. + +## Why it exists + +- Agents need stable capability ids, not only friendly names. +- Dispatch needs explicit authority and lane routing. +- Receipts need a canonical place and schema. +- MCP should stay at the edge for snapshot and dispatch, not become the inner loop. + +## Try it + +\`\`\`bash +npx @wesleysimplicio/llm-project-mapper --mcp-edge +bin/build-hamt-catalog +\`\`\` + +That flow gives you: + +- \`.catalog/agents.json\` as the generated HAMT catalog +- \`.receipts/\` as the default local receipt store +- \`mcp/server.ts\` and \`mcp/server.py\` as edge adapters when \`--mcp-edge\` is enabled + +## Static layout + +\`\`\`json +{ + "yool_id": "agent.dev.python.v1", + "authority": "dev", + "lane": "fast", + "agent_terms": { + "cpu_quota_pct": 60, + "disk_quota_mb": 100 + } +} +\`\`\` + +## Dynamic flow + +\`\`\`text +AGENTS.md + -> bin/build-hamt-catalog + -> .catalog/agents.json + -> MCP snapshot / dispatch edge + -> workers subscribe by lane + -> .receipts/ evidence +\`\`\` + +## MCP is the edge + +MCP is for read-mostly snapshots and external dispatch entrypoints. Keep tuple matching, scheduling, retries, and worker coordination outside the MCP adapter. + +## Canonical spec + +- [Root spec on GitHub](https://github.com/wesleysimplicio/llm-project-mapper/blob/main/YOOL_TUPLE_HAMT.md) +- [Vendored source in this repo](https://github.com/wesleysimplicio/llm-project-mapper/blob/main/docs/YOOL_TUPLE_HAMT.md) `, }, { diff --git a/tests/e2e/build-hamt-catalog.spec.ts b/tests/e2e/build-hamt-catalog.spec.ts new file mode 100644 index 0000000..de1ce44 --- /dev/null +++ b/tests/e2e/build-hamt-catalog.spec.ts @@ -0,0 +1,85 @@ +import { spawnSync, SpawnSyncReturns } from 'node:child_process'; +import * as fs from 'node:fs'; +import * as os from 'node:os'; +import * as path from 'node:path'; +import { expect, test, type TestInfo } from '@playwright/test'; + +const CLI = path.resolve(__dirname, '..', '..', 'bin', 'cli.js'); +const NODE = process.execPath; + +function mkTmp(): string { + return fs.mkdtempSync(path.join(os.tmpdir(), 'lpm-hamt-e2e-')); +} + +function rmTmp(dir: string): void { + try { + fs.rmSync(dir, { recursive: true, force: true }); + } catch { + // ignore + } +} + +function writeFile(dir: string, rel: string, content: string): void { + const full = path.join(dir, rel); + fs.mkdirSync(path.dirname(full), { recursive: true }); + fs.writeFileSync(full, content); +} + +function runCli(args: string[], cwd: string): SpawnSyncReturns { + return spawnSync(NODE, [CLI, ...args], { + cwd, + encoding: 'utf8', + timeout: 30_000, + }); +} + +async function attachEvidence(testInfo: TestInfo, res: SpawnSyncReturns, outputPath: string) { + await testInfo.attach('stdout.txt', { body: res.stdout ?? '', contentType: 'text/plain' }); + await testInfo.attach('stderr.txt', { body: res.stderr ?? '', contentType: 'text/plain' }); + if (fs.existsSync(outputPath)) { + await testInfo.attach('agents.json', { + path: outputPath, + contentType: 'application/json', + }); + } +} + +test('npx-style CLI subcommand builds a HAMT catalog on a fixture repo', async ({}, testInfo) => { + const dir = mkTmp(); + const agentsMd = `# Agents + +### Dev Agent +- yool_id: \`agent.dev.typescript.v1\` +- authority: dev +- lane: fast +- agent_terms: + cpu_quota_pct: 55 + disk_quota_mb: 120 + +### Lint Agent +- yool_id: \`agent.lint.eslint.v1\` +- authority: review +- lane: background +- agent_terms: + cpu_quota_pct: 15 + disk_quota_mb: 30 +`; + try { + writeFile(dir, 'AGENTS.md', agentsMd); + const res = runCli(['build-hamt-catalog', dir], dir); + const outputPath = path.join(dir, '.catalog', 'agents.json'); + await attachEvidence(testInfo, res, outputPath); + expect(res.status, res.stderr).toBe(0); + expect(fs.existsSync(outputPath)).toBe(true); + const catalog = JSON.parse(fs.readFileSync(outputPath, 'utf8')); + expect(catalog.schema).toBe('yool-catalog/v1'); + expect(catalog.stats.entries).toBe(2); + expect(catalog.entries.map((entry: { yool_id: string }) => entry.yool_id)).toEqual([ + 'agent.dev.typescript.v1', + 'agent.lint.eslint.v1', + ]); + expect(catalog.hamt.root.bitmap).toBeGreaterThan(0); + } finally { + rmTmp(dir); + } +}); diff --git a/tests/unit/build-hamt-catalog.test.js b/tests/unit/build-hamt-catalog.test.js new file mode 100644 index 0000000..49b40a5 --- /dev/null +++ b/tests/unit/build-hamt-catalog.test.js @@ -0,0 +1,109 @@ +'use strict'; + +const { test } = require('node:test'); +const assert = require('node:assert/strict'); +const { spawnSync } = require('node:child_process'); +const fs = require('node:fs'); +const os = require('node:os'); +const path = require('node:path'); + +const NODE = process.execPath; +const WRAPPER = path.resolve(__dirname, '..', '..', 'bin', 'build-hamt-catalog'); +const CLI = path.resolve(__dirname, '..', '..', 'bin', 'cli.js'); + +function mkTmp() { + return fs.mkdtempSync(path.join(os.tmpdir(), 'lpm-hamt-')); +} + +function rmTmp(dir) { + try { + fs.rmSync(dir, { recursive: true, force: true }); + } catch { + // ignore + } +} + +function writeFile(dir, rel, content) { + const full = path.join(dir, rel); + fs.mkdirSync(path.dirname(full), { recursive: true }); + fs.writeFileSync(full, content); +} + +const AGENTS_FIXTURE = `# Agents + +### Dev Agent +- yool_id: \`agent.dev.python.v1\` +- authority: dev +- lane: fast +- agent_terms: + cpu_quota_pct: 60 + disk_quota_mb: 100 + timeout_s: 300 + +### Review Agent +- yool_id: \`agent.review.docs.v1\` +- authority: review +- lane: background +- agent_terms: + cpu_quota_pct: 20 + disk_quota_mb: 25 +`; + +test('wrapper builds .catalog/agents.json from AGENTS.md', () => { + const dir = mkTmp(); + try { + writeFile(dir, 'AGENTS.md', AGENTS_FIXTURE); + const res = spawnSync(NODE, [WRAPPER, dir], { + cwd: dir, + encoding: 'utf8', + timeout: 30000, + }); + assert.equal(res.status, 0, res.stderr); + const outputPath = path.join(dir, '.catalog', 'agents.json'); + assert.equal(fs.existsSync(outputPath), true); + const catalog = JSON.parse(fs.readFileSync(outputPath, 'utf8')); + assert.equal(catalog.schema, 'yool-catalog/v1'); + assert.equal(catalog.stats.entries, 2); + assert.equal(catalog.entries[0].yool_id, 'agent.dev.python.v1'); + assert.deepEqual(catalog.entries[0].hash.slots.length, 6); + assert.match(res.stdout, /wrote/); + } finally { + rmTmp(dir); + } +}); + +test('cli subcommand proxies to build-hamt-catalog', () => { + const dir = mkTmp(); + try { + writeFile(dir, 'AGENTS.md', AGENTS_FIXTURE); + const res = spawnSync(NODE, [CLI, 'build-hamt-catalog', dir], { + cwd: dir, + encoding: 'utf8', + timeout: 30000, + }); + assert.equal(res.status, 0, res.stderr); + const outputPath = path.join(dir, '.catalog', 'agents.json'); + assert.equal(fs.existsSync(outputPath), true); + const catalog = JSON.parse(fs.readFileSync(outputPath, 'utf8')); + assert.equal(catalog.stats.parsed_agents, 2); + } finally { + rmTmp(dir); + } +}); + +test('wrapper reports actionable error when python is unavailable', () => { + const dir = mkTmp(); + try { + writeFile(dir, 'AGENTS.md', AGENTS_FIXTURE); + const res = spawnSync(NODE, [WRAPPER, dir], { + cwd: dir, + encoding: 'utf8', + env: { ...process.env, PATH: '' }, + timeout: 30000, + }); + assert.equal(res.status, 2); + assert.match(res.stderr, /Install python3, or use the Windows py launcher/i); + } finally { + rmTmp(dir); + } +}); diff --git a/tests/unit/cli-args.test.js b/tests/unit/cli-args.test.js index 8e9101a..03e39b4 100644 --- a/tests/unit/cli-args.test.js +++ b/tests/unit/cli-args.test.js @@ -64,6 +64,11 @@ test('--dry-run flag is recognized (not treated as unknown)', () => { assert.doesNotMatch(res.stderr, /Unknown flag/, `--dry-run was rejected as unknown: ${res.stderr}`); }); +test('--mcp-edge flag is recognized (not treated as unknown)', () => { + const res = runCli(['--dry-run', '--yes', '--cli', 'skip', '--append-gitignore', 'no', '--mcp-edge']); + assert.doesNotMatch(res.stderr, /Unknown flag/, `--mcp-edge was rejected as unknown: ${res.stderr}`); +}); + test('--preset list prints the catalog and exits 0', () => { const res = runCli(['--preset', 'list']); assert.equal(res.status, 0); diff --git a/tests/unit/cli-install.test.js b/tests/unit/cli-install.test.js index 1a69ee0..e88d603 100644 --- a/tests/unit/cli-install.test.js +++ b/tests/unit/cli-install.test.js @@ -90,6 +90,8 @@ test('fresh install creates .starter-meta.json + AGENTS.md + .specs/', () => { assert.equal(fs.existsSync(path.join(dir, 'AGENTS.md')), true); assert.equal(fs.existsSync(path.join(dir, 'CLAUDE.md')), true); assert.equal(fs.existsSync(path.join(dir, '.specs')), true); + assert.equal(fs.existsSync(path.join(dir, '.catalog', 'agents.json')), true); + assert.equal(fs.existsSync(path.join(dir, '.receipts', '.gitkeep')), true); assert.equal(fs.existsSync(path.join(dir, 'docs', 'local-setup.md')), true); assert.equal(fs.existsSync(path.join(dir, '.specs', 'journal')), true); } finally { @@ -153,6 +155,39 @@ test('.starter-meta.json has the documented shape', () => { } }); +test('fresh install renders AGENTS.md with yool fields and receipt guidance', () => { + const dir = mkTmp(); + try { + writeFile(dir, 'package.json', '{"name":"my-product","dependencies":{"next":"14.0.0"}}'); + const res = runCli(['--yes', '--cli', 'skip', '--append-gitignore', 'no'], dir); + assert.equal(res.status, 0, `cli failed: ${res.stderr}`); + const agents = fs.readFileSync(path.join(dir, 'AGENTS.md'), 'utf8'); + assert.match(agents, /yool_id:/); + assert.match(agents, /authority:/); + assert.match(agents, /lane:/); + assert.match(agents, /agent_terms:/); + assert.match(agents, /Receipts schema/); + assert.match(agents, /YOOL_TUPLE_HAMT\.md/); + } finally { + rmTmp(dir); + } +}); + +test('--mcp-edge creates edge templates and records the meta flag', () => { + const dir = mkTmp(); + try { + writeFile(dir, 'package.json', '{"name":"my-product","dependencies":{"next":"14.0.0"}}'); + const res = runCli(['--yes', '--cli', 'skip', '--append-gitignore', 'no', '--mcp-edge'], dir); + assert.equal(res.status, 0, `cli failed: ${res.stderr}`); + assert.equal(fs.existsSync(path.join(dir, 'mcp', 'server.ts')), true); + assert.equal(fs.existsSync(path.join(dir, 'mcp', 'server.py')), true); + const meta = JSON.parse(fs.readFileSync(path.join(dir, '.starter-meta.json'), 'utf8')); + assert.equal(meta.mcp_edge_enabled, true); + } finally { + rmTmp(dir); + } +}); + test('detects monorepo when ≥2 manifests live under apps/', () => { const dir = mkTmp(); try { diff --git a/tests/unit/docs-site.test.js b/tests/unit/docs-site.test.js index 6cf7fa2..60aaf1b 100644 --- a/tests/unit/docs-site.test.js +++ b/tests/unit/docs-site.test.js @@ -33,6 +33,22 @@ test('docs site ships a landing page at the site root', () => { const landingPage = read('docs-site/src/pages/index.mdx'); assert.match(landingPage, /\[Current docs \(v0\.x\)\]\(\/next\/\)/); assert.match(landingPage, /\[Stable snapshot\]\(\/intro\)/); + assert.match(landingPage, /\[YOOL \/ tuple \/ HAMT pattern\]\(\/yool-tuple-hamt\)/); +}); + +test('docs home links to the yool tuple hamt page from the root doc', () => { + const introDoc = read('docs-site/docs/intro.md'); + assert.match(introDoc, /\*\*YOOL \/ tuple \/ HAMT\*\*/); + assert.match(introDoc, /\[YOOL \/ tuple \/ HAMT\]\(\/yool-tuple-hamt\)/); +}); + +test('docs site includes a dedicated yool tuple hamt page at the target route', () => { + const patternDoc = read('docs-site/docs/concepts/yool-tuple-hamt.md'); + assert.match(patternDoc, /slug:\s*\/yool-tuple-hamt/); + assert.match(patternDoc, /## Static layout/); + assert.match(patternDoc, /## Dynamic flow/); + assert.match(patternDoc, /npx @wesleysimplicio\/llm-project-mapper/); + assert.match(patternDoc, /bin\/build-hamt-catalog/); }); test('docs site package includes the Mermaid ELK dependency required by the theme bundle', () => { @@ -49,3 +65,9 @@ test('docs deployment workflow publishes to GitHub Pages on main', () => { assert.match(workflow, /branches:\s*\n\s*-\s*main/); assert.match(workflow, /actions\/deploy-pages@v4/); }); + +test('docs site navigation exposes the yool tuple hamt page', () => { + const config = read('docs-site/docusaurus.config.cjs'); + assert.match(config, /label:\s*'YOOL \/ tuple \/ HAMT'/); + assert.match(config, /to:\s*'\/yool-tuple-hamt'/); +});