From 27aaa7233f4e7fffbca9098e095f89f5c23ff78d Mon Sep 17 00:00:00 2001 From: Guy Senpai Date: Sun, 28 Jun 2026 09:37:57 +0200 Subject: [PATCH 01/15] docs(brief): add M1.0.6 milestone brief --- briefs/M1.0.6-prefabs-crossrefs-extensions.md | 239 ++++++++++++++++++ 1 file changed, 239 insertions(+) create mode 100644 briefs/M1.0.6-prefabs-crossrefs-extensions.md diff --git a/briefs/M1.0.6-prefabs-crossrefs-extensions.md b/briefs/M1.0.6-prefabs-crossrefs-extensions.md new file mode 100644 index 0000000..569b54d --- /dev/null +++ b/briefs/M1.0.6-prefabs-crossrefs-extensions.md @@ -0,0 +1,239 @@ +# M1.0.6 — Prefabs, entity cross-references & extensions + +> **Status:** PLANNED +> **Phase:** 1 +> **Branch:** `phase-1/scene/prefabs-crossrefs-extensions` +> **Tag (set after merge by Guy):** `v0.10.6-prefabs-crossrefs-extensions` +> **Dependencies:** M0.1 (Tier-0 ECS: `World` spawn paths, `componentBytes`/`markComponentChangedDyn` dynamic-by-`ComponentId` access, dynamic add-component archetype migration, `Archetype`, `Registry.idOf`, generational `EntityId`), M0.8 (grammar v0.6: `prefab_decl` with `of`/`extends`/`requires`/`on_attach`/`on_detach`, `scene_decl` with `instance of` + `extensions:`), M1.0.2 (`ObserverRegistry` + `dispatchOnSpawned` two-phase pattern — reused for the `on_attach` dispatch seam), M1.0.3 (`writeValueAsBytes`/`readBytesAsValue`, `FieldDesc` metadata, persistent heap now Tier-0), M1.0.4 (`.scene.bin` codec: `src/core/scene/{format,writer,accessor}.zig`; `tools/scene_cook`; the `instance of` rejection this milestone removes), M1.0.5 (runtime loader: `loadFromBytes`/`loadScene`, `LoadResult.uuid_to_entity` seam, the two reserved-table asserts this milestone replaces) +> **Open date:** 2026-06-28 +> **Close date:** — + +--- + +# FROZEN SECTION + +*Authored by Claude.ai. Not modifiable by Claude Code outside a Claude.ai round-trip (see § Accepted deviations).* + +## Context + +Third and final sub-milestone of the scene track (M1.0.4–6) in Phase 1. It is the **second consumer of the M1.0.5 `uuid_to_entity` seam** and delivers three subsystems grafted onto the cooked-scene pipeline: **prefab flattening** (instances expanded at cook with per-field overrides), **entity→entity cross-references** (a component field of type `Entity`, resolved on load), and **extensions** (`prefab … extends …` with `on_attach`/`on_detach`, materialized on load). It validates the plan testable: *"prefab instanced N times with overrides; entities that reference each other."* + +This milestone **implements code surfaces the specs already define but the engine does not yet realize**: the `Entity` builtin is already spec'd as a POD component field (`etch-reference-part1.md §3.2`/§5.5, `etch-grammar.md §2.2`) but `registry.zig` has no matching `FieldKind`; the `.scene.bin` header already reserves `extensions_offset @44` and `crossrefs_offset @48` (written empty in M1.0.4) but their bodies have no defined shape; the v0.6 grammar already parses `prefab_decl`/`instance of` but the cook rejects `instance of` with `error.InstanceOfUnsupported`. The work is filling these reserved slots, not inventing new contracts. + +**This is a large milestone spanning two Claude Code sessions** (session boundary after E4). It is **not** split into two milestones: one branch, one PR, one tag. The session boundary is an operational pause, not a milestone boundary. + +**Verified seams (read against the tag `v0.10.5-scene-load` — governs the implementation):** +- `EntityId = packed struct(u64){ index:u32, generation:u32 }` (`src/core/ecs/entity.zig`) — exactly the Etch `Entity (id:u32, generation:u32)`. Slot = **8 B / 8-align**. Sentinel **`EntityId.dead`** (`index/gen = maxInt`) = "no entity" → unassigned field + dangling ref. +- `World.componentBytes(eid, cid) ?[]u8` (mutable, **dynamic by `ComponentId`**) + `World.markComponentChangedDyn(eid, cid)` exist (`src/core/ecs/world.zig`) → cross-ref resolution needs **no new World API**. +- Dynamic add-component path exists (archetype migration by `cid_new`, `src/core/ecs/world.zig`) → extensions can add components at load with no new primitive. +- Loader order today: `instantiate` (fills `uuid_to_entity`) → `loadResources` → `dispatchSpawnLifecycle`. The `assert(extensions_offset==0)` / `assert(crossrefs_offset==0)` in `loadFromBytes` are the bascule points. Parent ordinals are validated, **link not applied** ("owned by the hierarchy milestone" — stays that way here). +- The field type→`FieldKind` gate is `fieldKindFromTypeName(name, reg_kind)` (`src/etch/interp.zig:3485`, called by `compileTypeDecl`): today `.string_`/`.enum_` are emitted only for `reg_kind == .resource`. The `.entity_` kind is added here, gated to `reg_kind == .component`. +- The codec is `writeValueAsBytes`/`readBytesAsValue` (`src/etch/ecs_bridge.zig`), switch over `FieldKind`; the "side table" pattern is already in use (resource `string` fields are handled by a separate string pass, not the codec). Cross-refs follow it (a crossref pass at cook). +- The cook is `scene_cook.zig` (`cook` → `findScene` → `build`); `.prefab.etch` reuses the **same** `format`/`writer`/`accessor` as `.scene.bin` (a prefab is a mini-scene — `engine-scene-serialization.md §5`). + +## Design decisions (frozen) + +These were taken with Guy before coding. They are the contract; do not re-litigate. One open point (D-B note) is confirmed by reading at the head of E1. + +**D-A — `FieldKind.entity_` realizes the `Entity` type (component-side).** +8 B / 8-align slot = `EntityId`. Unassigned/dangling = `EntityId.dead`. **Component-only** — the exact mirror of `string_`/`enum_` (resource-only): `fieldKindFromTypeName` emits `.entity_` only for `reg_kind == .component` (resource→entity refs are a future additive milestone). Stays POD (8 B, no heap, no teardown) → the component SoA/POD invariant (`engine-spec.md §4`) is untouched. `sizeBytes`/`alignBytes`/`fromZigType` switches in `registry.zig` gain the case. + +**D-B — Cross-references Table (binary shape).** +``` +Cross-references Table @ crossrefs_offset + count: u32 + entries: [count] CrossRefEntry // 16 B, 4-align (extern struct) + source_uuid_ordinal: u32 // entity bearing the field (UUID-table ordinal) + schema_index: u32 // bearing component (file-local Schema Registry index) + field_offset: u32 // byte offset of the Entity field in the component slot + target_uuid_ordinal: u32 // referenced entity (UUID-table ordinal) +``` +- **Cook:** the `Entity` field's 8-byte slot is written `EntityId.dead` in its SoA column, and a `CrossRefEntry` is emitted (crossref pass, mirror of the string pass). **Intra-scene only:** the cook validates that `target` is an entity of the same scene; otherwise `error.UnresolvedCrossRef`. Cross-scene refs are out-of-scope (need a global registry). +- **Load:** new `resolveCrossRefs` pass — for each entry: `src = uuid_to_entity[uuidAt(source)]`, `cid = remap[schema_index]`, `tgt = uuid_to_entity[uuidAt(target)]`, then `@memcpy(componentBytes(src, cid).?[field_offset..][0..8], asBytes(&tgt))` and `markComponentChangedDyn(src, cid)`. The `assert(crossrefs_offset==0)` becomes the real read. Cross-refs resolve **cooked** component fields (base + flattened prefab) only. +- Analogue: Bevy `MapEntities`/`EntityMapper`; UE5 `FObjectInstancingGraph`. +- **Open point confirmed at E1 head:** the reference-value syntax inside `.scene.etch` (by entity-name vs UUID literal) is read from `etch-grammar.md` `scene_decl` + the parser. Leaning: **by entity-name** (resolved to a `uuid_ordinal` at cook). If the grammar diverges from this leaning, STOP and report before E4. + +**D-C — Entity Extensions Table (binary shape).** +``` +Entity Extensions Table @ extensions_offset + count: u32 // entities with >=1 active extension (sparse) + entries: [count] + uuid_ordinal: u32 + extension_count: u32 // u32 (not the §4 note's u8) — no variable padding + extension_ids: [extension_count] u32 // indices into the Prefab ID Table + prefab_id_count: u32 + prefab_name_refs: [prefab_id_count] u32 // string-table offsets (deduplicated extension names) +``` +Sparse (entities with no active extension are absent). The `§4` note's `u8` count is widened to `u32` to avoid variable padding (≈3 B/entity, negligible). + +**D-D — Parent/child hierarchy: deferred.** No runtime hierarchy component in M1.0.6 (risk of the wrong abstraction vs `AttachedToSocket`, and out of testable). Parent ordinals stay validated-not-applied. Hierarchy is a later milestone that will build on D-A/D-B + a hierarchy component. + +**D-E — `on_attach`/`on_detach` execution at load (tier boundary).** `loader.zig` (Tier 0) **must not call the Etch VM** (`src/etch`). Position: mirror the M1.0.2 `dispatchOnSpawned` mechanism — the loader materializes extension components (dynamic add-component, Tier-0, defaults copied from the extension's `.prefab.bin`), then fires `on_attach` through a **Tier-0 callback** (a function pointer registered by the Etch bridge, exactly as observer dispatch is wired). **Confirmed-or-blocker at E5 head:** read the `on_spawned` dispatch wiring + the `activate_extension` entry point in the Etch bridge before committing the callback design; if the seam is not as assumed, this is a Case-2 blocker (return to Claude.ai), not an improvised solution. + +## Scope + +**E1 — Design freeze + seam reconfirmation.** +- Read the seams listed in Context against the tag (no assumption — 3 past mis-rulings came from skipping this). +- Confirm the single open grammar point (D-B: `Entity` reference-value syntax in `scene_decl` — by-name vs UUID literal) against `etch-grammar.md` + the parser. Confirm `prefab_decl`/`instance of`/`extensions:` are already parsed (M0.8). +- → **STOP** — report the confirmed reference-value syntax (and any divergence from the by-name leaning) and the seam reconfirmation; await Guy's review + GO before E2. No production code in E1. + +**E2 — `.prefab.bin`: grammar + cook (standalone + `of`).** +- `scene_cook.zig`: add `cookPrefab` (locate the single `prefab_decl`; reject a `.prefab.etch` that holds a `scene`). Build a `CookModel` for a standalone prefab and for a `prefab "Y" of "X"` variant — `of` resolution loads the base prefab's `.prefab.bin`, applies the variant's per-field overrides (variant inherits all of X, overrides listed fields), producing the flattened component set. Serialize via the existing `writer` to `.prefab.bin` (same format as `.scene.bin`). +- `tools/scene_cook/main.zig`: drive `.prefab.etch` → `.prefab.bin` (cook prefabs before scenes; the scene cook in E3 consumes them). +- Prefab dependency resolution: locate a referenced prefab's cooked `.prefab.bin` by name (a cook-time prefab registry / path map — distinct from Etch `import`, which is M1.0.7). +- → **STOP** — push, await review + GO. + +**E3 — Prefab flattening at scene cook (`instance of` + per-field overrides).** +- `scene_cook.zig`: replace `error.InstanceOfUnsupported`. For `instance of "P" "name" { ComponentBlock {…} Comp.field = v }`: load `P.prefab.bin` (via `accessor`), take its flattened components (base + `of` chain already baked into `P.prefab.bin`), apply the instance's per-field overrides and component-block overrides, feed the resulting components into the archetype sort. `buildComponentBlob` accepts a prefab base in addition to the registry default and the instance overrides. +- An instance with no prefab override is identical to a hand-authored entity carrying the prefab's components (same archetype, same bytes). +- → **STOP** — push, await review + GO. + +**E4 — Entity cross-references.** +- `registry.zig`: add `FieldKind.entity_` (D-A) + `sizeBytes`/`alignBytes` cases. +- `interp.zig`: `fieldKindFromTypeName` emits `.entity_` for `Entity` when `reg_kind == .component`; wire it through `compileTypeDecl`. +- Etch `Value` union (wherever defined): add an `.entity_` variant carrying `EntityId`. `ecs_bridge.zig`: `writeValueAsBytes`/`readBytesAsValue` `.entity_` cases. +- `format.zig`: `CrossRefEntry` (D-B) + a read helper. `writer.zig`: emit the cross-references table from the `CookModel`. `accessor.zig`: `crossrefsCount`/`crossref(i)` getters. +- `scene_cook.zig`: crossref pass — for each `Entity` field on a cooked component, write `EntityId.dead` in the slot and emit a `CrossRefEntry`; validate the target is an entity of the scene (`error.UnresolvedCrossRef` otherwise). Handles `Entity` fields originating from the E3 flattening too. +- `loader.zig`: `resolveCrossRefs` pass (D-B), inserted **after `instantiate`, before `loadResources`**; replace `assert(crossrefs_offset==0)` with the real read. +- → **STOP** — push, await review + GO. **— session 1 / session 2 boundary —** + +**E5 — Extensions: extended design (D-E) + cook + `extends` grammar.** +- Resolve D-E first (read the `on_spawned` dispatch + `activate_extension` wiring; STOP as a Case-2 blocker if the seam differs from D-E's assumption). +- `scene_cook.zig`: cook `prefab "Z" extends "X" requires … { entity {…} on_attach {…} on_detach {…} }` → `.prefab.bin` (extension components + the `on_attach`/`on_detach` hook bytecode + the `requires` list). Validate `requires` (compile-time, against X's `of` chain) and additive conflicts (warning per `etch-reference-part2.md §30.5`: two extensions adding the same component to X). A scene's `extensions: ["Z", …]` clause → Entity Extensions Table + Prefab ID Table. +- `format.zig`/`writer.zig`/`accessor.zig`: `ExtensionEntry` + Prefab ID Table (D-C) — struct, emit, getters. +- → **STOP** — push, await review + GO. + +**E6 — Extensions: load (`applyExtensions` + `on_attach` dispatch).** +- `loader.zig`: `applyExtensions` pass, inserted **after `loadResources`, before `dispatchSpawnLifecycle`**: per entity with active extensions, in table order → dynamic add-component (defaults from the extension's `.prefab.bin`) then fire `on_attach` via the Tier-0 callback (D-E). Replace `assert(extensions_offset==0)` with the real read. +- Etch side: the `activate_extension` runtime entry (reused by the loader path) + the `on_attach` dispatch callback registered into Tier-0 (mirroring observer dispatch). `on_detach`/`deactivate_extension` runtime entry for symmetry. +- Canonical final load order: `instantiate` → `resolveCrossRefs` (E4) → `loadResources` → `applyExtensions` (E6) → `dispatchSpawnLifecycle`. +- `CLAUDE.md` §3.4 update on the branch (`docs(claude-md): update for M1.0.6`): État courant row (scene track complete), Tags row `v0.10.6-prefabs-crossrefs-extensions`, the cross-references / extensions binary shapes now frozen, the `FieldKind.entity_` addition, and the Last-updated date. +- → **STOP** — push, await review + GO. + +## Out-of-scope + +- **`.prefab.bin` runtime loading / `spawn("Prefab")` dynamic instantiation** — the scene cook reads the `.prefab.bin` to flatten; spawning a prefab into a live world at runtime is a distinct, additive use. Not here. +- **Resource→entity references** — `Entity` fields on resources. `.entity_` is component-only in M1.0.6 (D-A); resource-side is a future additive milestone (its own side table + `getMutResource` patch). +- **Cross-scene references** — resolving a `target` UUID not present in the loaded scene. Needs a global identity registry; out of M1.0. Intra-scene only (`error.UnresolvedCrossRef` at cook for an absent target). +- **`Entity` fields on extension-added components, and per-field overrides targeting them** — extension components are added at load, not cooked into the entity, so they carry no cook-emitted `CrossRefEntry`; their `Entity` fields are set by `on_attach`, not by the crossref mechanism. The override-on-extension-component case is a known edge, deferred (additive). +- **Parent/child hierarchy application** (D-D) — owned by the hierarchy milestone. +- **Bulk SoA column-copy instantiation (bulk-spawn)** — YAGNI confirmed at M1.0.5 (bench ~1.05 ms/10k). The per-entity `spawnDynamicWithValues` path stays; do not introduce a bulk path or expose/`appendRowFromBytes`. +- **`override` (`etch-resolver-types.md §14`, §5.6)** — needs a second overridable Tier-1 module; out of M1.0. +- **World partition** (`.cell.bin`/`.layer.bin`/`.manifest.bin`), HLOD, data layers, scene streaming, async scene load. +- **Save/load** (`.sav`), scene hot-reload, `.scene.bin` compression. +- **Auto-registering an unknown component from its on-disk `SchemaEntry`** — Phase 2+; an unknown component is still `error.UnknownComponent`. + +## Specs to read first + +1. `engine-scene-serialization.md` — §5 (prefabs: `of`/`extends`/overrides, the Entity Extensions Table serialization, the `of`-vs-`extends` table), §2 (UUID + name identity, cross-ref resolution), §4 (`.scene.bin` layout + the two reserved tables this milestone fills). PRIMARY. +2. `etch-grammar.md` — `prefab_decl` (`of`/`extends`/`requires`/`on_attach`/`on_detach`), `scene_decl` (`instance of`, `extensions:`, and the **entity reference-value syntax** — needed for E1's open point). +3. `etch-reference-part2.md` — §30 (the `prefab` construct: §30.4 `of`-vs-`extends` differentiation, §30.5 additive-conflict validation). +4. `etch-reference-part1.md` — §3.2 / §5.5 (the `Entity` builtin type; POD-in-component rules). +5. `engine-asset-pipeline.md` — §6.3 (scene/prefab cooking; reconciled — prefab flattening is M1.0.6, the M1.0.4 cook rejects `instance of`). +6. `engine-spec.md` — §19 (scene serialization), §4 (component POD/SoA invariant — governs `FieldKind.entity_`), §3.5 (in-tree discipline). +7. `etch-resolver-types.md` — §14 (the `override` frontier — confirms `override` stays out-of-scope). +8. `engine-zig-conventions.md` — §13 (test rooting / lazy-analysis guard — so `tests/scene/` actually runs), §19 (POD `extern struct`, rules summary). + +## Files to create or modify + +- `src/core/ecs/registry.zig` — **edit** (E4) — `FieldKind.entity_` + `sizeBytes`/`alignBytes`/`fromZigType` cases. +- `src/core/ecs/entity.zig` — **edit if needed** (E4) — only if `EntityId.dead` needs a public accessor for the codec/loader (already public). +- `src/etch/interp.zig` — **edit** (E4) — `fieldKindFromTypeName` (`Entity` → `.entity_`, `.component` gate) + `compileTypeDecl` wiring. +- `src/etch/ecs_bridge.zig` — **edit** (E4) — `.entity_` cases in `writeValueAsBytes`/`readBytesAsValue`. +- `src/etch/value.zig` (or wherever the `Value` union lives — locate at E4) — **edit** (E4) — `.entity_` variant carrying `EntityId`. +- `src/core/scene/format.zig` — **edit** (E4/E5) — `CrossRefEntry` (16 B extern), `ExtensionEntry` + Prefab ID Table layout, read helpers. +- `src/core/scene/writer.zig` — **edit** (E4/E5) — emit the cross-references table and the extensions + Prefab ID tables from the `CookModel`. +- `src/core/scene/accessor.zig` — **edit** (E4/E5) — `crossrefsCount`/`crossref(i)`, `extensionsCount`/`extension(i)`, prefab-id getters. +- `src/core/scene/loader.zig` — **edit** (E4/E6) — `resolveCrossRefs` and `applyExtensions` passes; replace the two reserved-table asserts; canonical load order. +- `src/etch/scene_cook.zig` — **edit** (E2/E3/E4/E5) — `cookPrefab` + `of` resolution (E2); `instance of` flattening + per-field overrides (E3); crossref pass (E4); `extends` cook + `extensions:` clause + `requires`/conflict validation (E5). +- `tools/scene_cook/main.zig` — **edit** (E2) — cook `.prefab.etch` → `.prefab.bin`; prefab-before-scene ordering. +- `src/etch/root.zig` — **edit if needed** (E5/E6) — re-export / §13 pin for any new `extends`/dispatch surface. +- `tests/scene/prefab_cook_test.zig` — **create** (E2) — standalone + `of` cook round-trip via accessor. +- `tests/scene/prefab_flatten_test.zig` — **create** (E3) — `instance of` flattening, N instances, per-field overrides, cook→load→ECS. +- `tests/scene/crossref_test.zig` — **create** (E4) — `Entity` field cook + resolve; dangling → `dead`; absent target → `error.UnresolvedCrossRef` at cook. +- `tests/scene/extensions_test.zig` — **create** (E5/E6) — `extends` cook; scene `extensions:` → non-empty table round-trip; load adds components + `on_attach` effect observable. +- `tests/scene/prefab_integration_test.zig` — **create** (E3/E4/E6) — cross-module Etch cook→load: prefab instanced + cross-ref + extension in one scene. +- `build.zig` — **edit** — wire the new `tests/scene/` targets (rooted per §13); the integration target sees both `weld_core` and `weld_etch`. +- `CLAUDE.md` — **edit** (E6) — §3.4, via `docs(claude-md): update for M1.0.6`. + +## Acceptance criteria + +### Tests + +- `tests/scene/prefab_cook_test.zig` — `test "standalone prefab cooks and reads back"` — a `.prefab.etch` cooks to `.prefab.bin`; the accessor reads its components/entities. `test "variant prefab resolves of-chain"` — `prefab "Y" of "X"` flattens X + Y overrides. +- `tests/scene/prefab_flatten_test.zig` — `test "instance of expands to the prefab's components"` — an `instance of "P"` with no override yields the same archetype + bytes as the hand-authored equivalent. `test "per-field overrides apply over the prefab"` — an overridden field reads the instance value, non-overridden fields read the prefab value. `test "N instances of one prefab"` — N instances load as N entities. +- `tests/scene/crossref_test.zig` — `test "entity field resolves to the target handle on load"` — a `Comp.ref: Entity` pointing at another scene entity reads back that entity's `EntityId`. `test "unset entity field is EntityId.dead"`. `test "cook rejects a reference to an absent entity"` — `error.UnresolvedCrossRef`. +- `tests/scene/extensions_test.zig` — `test "extends prefab cooks with hooks and requires"`. `test "scene extensions clause populates the table"` — round-trips a non-empty Entity Extensions Table + Prefab ID Table. `test "load applies extension components and fires on_attach"` — after load the entity carries the extension's components and the `on_attach` effect is observable (e.g. a boosted `Health.max`). +- `tests/scene/prefab_integration_test.zig` — `test "scene with prefab instances, a cross-ref and an extension loads end to end"` — entity count, an overridden field value, a resolved reference, and an `on_attach` effect all correct. + +### Benchmarks + +- None new. Prefab flattening is cook-time; cross-ref and extension resolution are `O(entries)` over small tables with no runtime hot path. Load time must not regress materially vs M1.0.5 — note it in Closing notes if it moves, do not gate on a threshold. + +### Observable behaviour + +- A runnable scenario (extend `tools/scene_cook`'s harness or the integration test with output): cook a prefab, cook a scene that instances it N times with overrides + a cross-ref + an active extension, load it, and log: instantiated entity count, an overridden field value, the resolved reference handle, and the `on_attach` effect. + +### CI + +- `zig build` clean, zero warnings, on the configured matrix. +- `zig build test` green (Debug + ReleaseSafe). +- `zig fmt --check` green. +- `zig build lint` green (when the custom linter exists). +- `commit-msg` hook green on every commit of the branch. + +## Conventions + +- **Branch**: `phase-1/scene/prefabs-crossrefs-extensions` +- **Final tag**: `v0.10.6-prefabs-crossrefs-extensions` +- **PR title**: `Phase 1 / Scene / Prefabs, entity cross-references & extensions` +- **Commit convention**: Conventional Commits (cf. `engine-development-workflow.md` §4.3) +- **Merge strategy**: squash-and-merge (cf. `engine-development-workflow.md` §4.6) + +## Notes + +- **Format source of truth = code, not §4 prose.** Re-read `src/core/scene/{format,writer,accessor,loader}.zig` (frozen M1.0.4/M1.0.5) before extending them. The reserved offsets `@44`/`@48` and the loader's two asserts are the exact bascule points. +- **`.prefab.bin` is the same format as `.scene.bin`** (a prefab is a mini-scene). Reuse `writer`/`format`/`accessor` — do **not** invent a second binary format. The header's `content_version`/`entity_count` etc. apply unchanged. +- **`of` (variant) flattens at cook; `extends` (extension) materializes at load.** A variant is baked into the instance's archetype (static inheritance). An extension's components are added at load and its `on_attach` runs at load (dynamic composition). Per-field overrides in an instance apply to the **final** entity (base + flattened prefab); overrides targeting extension-added components are out-of-scope (see Out-of-scope). +- **`.entity_` is the mirror of `.string_`/`.enum_`.** Those are resource-only (gated out of components); `.entity_` is component-only (gated out of resources). Same gate site (`fieldKindFromTypeName`), opposite `reg_kind`. +- **Cross-ref slot is `EntityId.dead` on disk.** The 8-byte `Entity` column slot is a placeholder at cook; the side table (`CrossRefEntry`) carries the target; the loader patches it via `componentBytes` + `markComponentChangedDyn`. This mirrors how resource `string` slots are zeroed on disk with the string side table carrying the value. +- **Tier discipline holds.** `loader.zig` (`weld_core`) never imports `weld_etch`; `on_attach` dispatch goes through a Tier-0 callback registered by the Etch bridge (D-E), exactly as `on_spawned` observer dispatch is wired. The crossref pass and the flattening live on the Etch cook side (`scene_cook.zig`, which imports `weld_core.scene`). +- **D-E is the one architectural unknown.** If reading the `on_spawned`/`activate_extension` wiring at E5 head shows the callback seam is not available as assumed, that is a Case-2 blocker (stop, journal, return to Claude.ai) — not an improvised in-loader VM call. +- **Two blockers ⇒ re-scope signal.** Per the prompt protocol, two distinct blockers in this milestone mean it is under-specified or too large; flag it rather than push through. + +--- + +# LIVING SECTION + +*Maintained by Claude Code during the milestone. The journal serves review and post-mortem debugging, not marketing.* + +## Specs read + +- [ ] `engine-scene-serialization.md` (§5, §2, §4) — read +- [ ] `etch-grammar.md` (`prefab_decl`, `scene_decl`) — read +- [ ] `etch-reference-part2.md` (§30) — read +- [ ] `etch-reference-part1.md` (§3.2, §5.5) — read +- [ ] `engine-asset-pipeline.md` (§6.3) — read +- [ ] `engine-spec.md` (§19, §4, §3.5) — read +- [ ] `etch-resolver-types.md` (§14) — read +- [ ] `engine-zig-conventions.md` (§13, §19) — read + +## Execution journal + +- — + +## Accepted deviations + +- — + +## Blockers encountered + +- — resolved by or + +## Closing notes + +- **What worked**: +- **What deviated from the original spec**: +- **What to flag explicitly in review**: +- **Final measurements** (load time vs M1.0.5, binary size, compile time as relevant): +- **Residual risk / tech debt left deliberately**: From 6d8f1874574e28379df5b40c63c0f661f9627149 Mon Sep 17 00:00:00 2001 From: Guy Senpai Date: Sun, 28 Jun 2026 09:40:15 +0200 Subject: [PATCH 02/15] docs(brief): confirm specs read for M1.0.6 --- briefs/M1.0.6-prefabs-crossrefs-extensions.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/briefs/M1.0.6-prefabs-crossrefs-extensions.md b/briefs/M1.0.6-prefabs-crossrefs-extensions.md index 569b54d..0b25a2c 100644 --- a/briefs/M1.0.6-prefabs-crossrefs-extensions.md +++ b/briefs/M1.0.6-prefabs-crossrefs-extensions.md @@ -209,14 +209,14 @@ Sparse (entities with no active extension are absent). The `§4` note's `u8` cou ## Specs read -- [ ] `engine-scene-serialization.md` (§5, §2, §4) — read -- [ ] `etch-grammar.md` (`prefab_decl`, `scene_decl`) — read -- [ ] `etch-reference-part2.md` (§30) — read -- [ ] `etch-reference-part1.md` (§3.2, §5.5) — read -- [ ] `engine-asset-pipeline.md` (§6.3) — read -- [ ] `engine-spec.md` (§19, §4, §3.5) — read -- [ ] `etch-resolver-types.md` (§14) — read -- [ ] `engine-zig-conventions.md` (§13, §19) — read +- [x] `engine-scene-serialization.md` (§5, §2, §4) — read 2026-06-28 09:40 +- [x] `etch-grammar.md` (`prefab_decl`, `scene_decl`) — read 2026-06-28 09:40 +- [x] `etch-reference-part2.md` (§30) — read 2026-06-28 09:40 +- [x] `etch-reference-part1.md` (§3.2, §5.5) — read 2026-06-28 09:40 +- [x] `engine-asset-pipeline.md` (§6.3) — read 2026-06-28 09:40 +- [x] `engine-spec.md` (§19, §4 → POD/SoA en §511-519, §3.5) — read 2026-06-28 09:40 +- [x] `etch-resolver-types.md` (§14) — read 2026-06-28 09:40 +- [x] `engine-zig-conventions.md` (§13, §19) — read 2026-06-28 09:40 ## Execution journal From ed515df02eff82fd77cd7839228684bce81ff995 Mon Sep 17 00:00:00 2001 From: Guy Senpai Date: Sun, 28 Jun 2026 09:40:23 +0200 Subject: [PATCH 03/15] docs(brief): activate M1.0.6 --- briefs/M1.0.6-prefabs-crossrefs-extensions.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/briefs/M1.0.6-prefabs-crossrefs-extensions.md b/briefs/M1.0.6-prefabs-crossrefs-extensions.md index 0b25a2c..9ce41ed 100644 --- a/briefs/M1.0.6-prefabs-crossrefs-extensions.md +++ b/briefs/M1.0.6-prefabs-crossrefs-extensions.md @@ -1,6 +1,6 @@ # M1.0.6 — Prefabs, entity cross-references & extensions -> **Status:** PLANNED +> **Status:** ACTIVE > **Phase:** 1 > **Branch:** `phase-1/scene/prefabs-crossrefs-extensions` > **Tag (set after merge by Guy):** `v0.10.6-prefabs-crossrefs-extensions` From 79ebed14bff95fa0067a59b0528debf8eca23f29 Mon Sep 17 00:00:00 2001 From: Guy Senpai Date: Sun, 28 Jun 2026 10:16:04 +0200 Subject: [PATCH 04/15] feat(scene): cook .prefab.etch to .prefab.bin (standalone + of variant) --- briefs/M1.0.6-prefabs-crossrefs-extensions.md | 20 +- build.zig | 4 + src/etch/scene_cook.zig | 304 +++++++++++++++++- tests/scene/prefab_cook_test.zig | 179 +++++++++++ tools/scene_cook/main.zig | 71 +++- 5 files changed, 563 insertions(+), 15 deletions(-) create mode 100644 tests/scene/prefab_cook_test.zig diff --git a/briefs/M1.0.6-prefabs-crossrefs-extensions.md b/briefs/M1.0.6-prefabs-crossrefs-extensions.md index 9ce41ed..3d1034e 100644 --- a/briefs/M1.0.6-prefabs-crossrefs-extensions.md +++ b/briefs/M1.0.6-prefabs-crossrefs-extensions.md @@ -220,7 +220,23 @@ Sparse (entities with no active extension are absent). The `§4` note's `u8` cou ## Execution journal -- — +- 2026-06-28 09:40 — E1 seam reconfirmation against `v0.10.5-scene-load` (tag = ancestor of HEAD). All Context seams confirmed verbatim: + - `src/core/ecs/entity.zig:30` — `EntityId = packed struct(u64){ index:u32, generation:u32 }`; `pub const dead` (maxInt/maxInt) at `:37`. ✓ + - `src/core/ecs/world.zig` — `componentBytes(*World, EntityId, ComponentId) ?[]u8` `:562`; `markComponentChangedDyn(...) void` `:574`; dynamic add-component is **`addComponentDynamic(*World, gpa, EntityId, cid_new: ComponentId, value_bytes: []const u8) !void`** `:687` (exact name — brief said "addComponentDyn" generically); `spawnDynamicWithValues(...) !EntityId` `:436`; `dispatchOnSpawned(*World, gpa, EntityId) !void` `:263`. ✓ + - `src/core/ecs/registry.zig` — `FieldKind` (`:34`) has **9** variants: `int_ float_ bool_ i32_ u32_ f32_ f64_ string_ enum_`; `sizeBytes`/`alignBytes` cover all 9; `idOf(name) ?ComponentId` `:286`; `componentSize`/`componentAlignment` present. ✓ (`.entity_` to be added in E4.) + - `src/etch/value.zig:70` — the Etch `Value` union **already carries `entity_id: EntityId`** (D-A's "add a Value variant" is largely pre-existing; E4 only adds the `.entity_` **FieldKind** codec cases in `ecs_bridge`). + - `src/etch/interp.zig:3485` — `fieldKindFromTypeName(name, reg_kind)` gates `.string_` to `reg_kind == .resource`; `.enum_` resolved in `compileTypeDecl` (`:3359`). The `.entity_` add (E4) mirrors this, gated to `.component`. + - `src/etch/ecs_bridge.zig` — `readBytesAsValue`/`writeValueAsBytes` switch all FieldKinds; resource-string "side table" pattern confirmed (string handled by a separate pass, `.string_` write returns `error.TypeMismatch`). + - `src/core/ecs/observers.zig` — `ObserverRegistry.dispatchOnSpawned` `:219`; Tier-0 callback seam = `registerOnSpawned(gpa, world, ctx, &observerTrampoline)` (Etch side, `interp.zig:1018`), function-pointer + opaque ctx. This is exactly the seam D-E reuses for `on_attach` (formal D-E confirm stays at E5 head). + - `src/core/scene/format.zig:100-101` — `extensions_offset @44` / `crossrefs_offset @48` reserved (0); no `CrossRefEntry`/`ExtensionEntry` yet (E4/E5). `SceneHeader`, `SchemaEntry`, `columnOffset` confirmed. + - `src/core/scene/loader.zig:180-181` — the two bascule asserts `assert(readU32At(extensions_offset)==0)` / `assert(...crossrefs_offset...==0)`; load order `instantiate → loadResources → dispatchSpawnLifecycle` (`:193-196`); `LoadResult.uuid_to_entity` seam (`:131`). ✓ + - `src/etch/scene_cook.zig:250` — `instance of` rejected with `error.InstanceOfUnsupported`; `Entity`/`Vec3`/`AssetHandle` fields rejected via `InvalidProgram` → `UnsupportedFieldKind`; `parent` resolved **by name** via `name_to_uuid_idx` (`:266`, `:410-413`). +- 2026-06-28 09:40 — **D-B resolved (entity reference-value syntax).** Grammar EBNF §15 + the M0.8 parser represent a component field value as a general `expression` (no `uuid "…"` form — `uuid` is not a keyword/ExprKind). The established intra-scene cross-entity reference (`parent:`) is a **STRING_LITERAL resolved by entity NAME** (`scene_cook` `name_to_uuid_idx`). → an `Entity`-typed field value will be a **STRING_LITERAL = the referenced entity's name**, resolved to a `uuid_ordinal` at cook, mirroring `parent:`. This **matches the by-name leaning**; the `target_npc: uuid "…"` form in `engine-scene-serialization.md` §2 is documentation drift, unsupported by grammar + parser. **No grammar divergence blocking E4.** +- 2026-06-28 09:40 — E1 STOP: pushed, awaiting Guy's review + GO before E2. No production code in E1. +- 2026-06-28 10:30 — E2 implemented. `scene_cook.zig`: `cookPrefab(gpa, source, base_resolver, diag_out)` + `BaseResolver` (name → cooked `.prefab.bin` bytes, the cook-time prefab path map). `Builder.findPrefab` (single prefab; rejects a `scene` source → `SceneNotAllowedInPrefab`), `buildPrefab` (standalone = `buildEntity` over prefab entities; `of` = `reconstructBase` from the base `.prefab.bin` via the accessor + `mergeVariantEntities`), `reconstructBase`/`mergeVariantEntities`/`applyVariantOverrides`/`mergeComponentBlob`/`indexOfId`, shared `versionFromNode`. Serializes through the existing `writer` (prefab = mini-scene, identical format). New `CookError`s: `NoPrefabConstruct`/`MultiplePrefabs`/`SceneNotAllowedInPrefab`/`ExtendsUnsupported`/`PrefabHookNotAllowed`/`BasePrefabMissing`/`BasePrefabCorrupt`/`BaseSchemaMismatch`. `tools/scene_cook/main.zig` NOT yet wired to prefabs (see below). Tests `tests/scene/prefab_cook_test.zig` (6, dedicated step `test-prefab-cook`): standalone read-back, `of` of-chain (field-merge + add-component + inherited), re-cook byte-identity, `extends`/scene/`hooks-on-of`/`no-resolver` rejections. Green Debug + ReleaseSafe; full `zig build test` 828/845 pass (17 skip, 0 fail, exit 0); `zig fmt --check` + `zig build lint` clean. +- 2026-06-28 10:30 — **E2 design clarifications (to confirm at review — implement within the frozen scope, do not modify it):** (1) **Prefab entities require an explicit `uuid:`** exactly like scene entities (`cookPrefab` reuses `buildEntity` verbatim → the frozen no-auto-gen determinism rule; the `.prefab.bin` format is identical to `.scene.bin` and references uuid ordinals). Spec §5 prefab examples omit uuids ("anonymous `entity {`" is shorthand per `ast.zig`); legal fixtures carry them. A variant entity that *overrides* a base entity (matched by name) may omit `uuid:` — it inherits the base identity. (2) **`of` merge granularity** = the grammar-legal `Component { field: value }` form (the `Component.field = value` per-field form is `instance_decl`-only per the parser, §15 l.1610), applied as a **field-merge** onto the inherited base component (shared component → overwrite set fields, keep the rest; new component → add). Base↔variant entities matched **by name**; merged entity keeps the **base's uuid + parent**. Multi-entity prefab hierarchy is reconstructed (parent-by-name preserved) but only single-entity + add-component is exercised by the E2 tests. +- 2026-06-28 10:45 — E2 `tools/scene_cook/main.zig`: detects scene vs prefab by input extension (`.prefab.etch` → `cookPrefab`, `.scene.etch` → `cook`); new optional `--prefab-dir ` flag wires a `BaseResolver` that reads `/.prefab.bin` on demand (the on-disk cook-time prefab path map, by prefab NAME) so `of` variants resolve their base. "Cook prefabs before scenes" is the caller's ordering (cook each prefab into `--prefab-dir`, then the scenes/variants against it) — the single-file CLI just needs the resolver root. Smoke-tested: standalone + `of`-chain cook through the CLI to real files, re-opened with the accessor. +- 2026-06-28 10:45 — E2 STOP: pushed, awaiting Guy's review + GO before E3. ## Accepted deviations @@ -228,7 +244,7 @@ Sparse (entities with no active extension are absent). The `§4` note's `u8` cou ## Blockers encountered -- — resolved by or +- **[E1 finding, affects E5 only — not E2/E4] The scene-side `extensions: [...]` clause is NOT parsed.** The brief E1 task asks to "confirm `prefab_decl`/`instance of`/`extensions:` are already parsed (M0.8)". Result: `prefab_decl` (relation `of`/`extends`, `requires`, `on_attach`/`on_detach`) ✓ and `instance of` ✓ are parsed, but the `entity { extensions: [...] }` / `instance of … { extensions: [...] }` clause (`engine-scene-serialization.md` §3, §5.5.2) is **not** — neither in the grammar EBNF §15 (`entity_decl`/`instance_decl` carry only `uuid` + `parent`), nor in the M0.8 parser (`parseSceneEntity` `parser.zig:3364`, `parseSceneInstance` `:3407` parse only `uuid`/`parent` then require TYPE_IDENT-led `component_instance`; a leading lowercase `extensions` ident → parse error). E5's scope ("a scene's `extensions: ["Z", …]` clause → Entity Extensions Table") therefore needs a grammar + parser + AST addition for the clause, which is **not** in the brief's "Files to create or modify" list (no `parser.zig`/`ast.zig`/grammar entries). **Design/scope question for Guy (Case-2), to resolve before E5:** (a) extend grammar §15 + `parseSceneEntity`/`parseSceneInstance` + `SceneEntity`/`SceneInstance` AST + descriptors to carry `extensions: [string]` (a bounded, additive grammar change — recommended, it is the only way to author active extensions in a scene); or (b) defer scene-authored extension activation and land only the binary table + loader `applyExtensions` in M1.0.6 (table populated by a test fixture / future authoring path), keeping the parser untouched. This does **not** block E2–E4 (prefab cook, flattening, cross-refs). — pending ## Closing notes diff --git a/build.zig b/build.zig index 9e56148..9e0bf45 100644 --- a/build.zig +++ b/build.zig @@ -430,6 +430,10 @@ pub fn build(b: *std.Build) void { .{ .path = "tests/scene/cook_roundtrip_test.zig", .scene = true }, // M1.0.4 / E3 — scene cook negative cases (typed errors, no panic). .{ .path = "tests/scene/cook_errors_test.zig", .scene = true }, + // M1.0.6 / E2 — prefab cook → `.prefab.bin`: standalone + `of` variant + // (base inherited from its cooked `.prefab.bin`, field-merge/add overrides), + // re-cook determinism, and the rejected forms (`extends`, hooks on `of`). + .{ .path = "tests/scene/prefab_cook_test.zig", .scene = true, .dedicated_step = "test-prefab-cook" }, // M1.0.5 / E2 — runtime loader `.scene.bin` → ECS: instantiate every // entity (component bytes verbatim), two-phase `on_spawned` (fires once // per entity, after all entities exist). `weld_core` only (no `.scene` diff --git a/src/etch/scene_cook.zig b/src/etch/scene_cook.zig index 302d613..11a6565 100644 --- a/src/etch/scene_cook.zig +++ b/src/etch/scene_cook.zig @@ -43,6 +43,9 @@ const FieldDesc = weld_core.ecs.registry.FieldDesc; const FieldKind = weld_core.ecs.registry.FieldKind; const archetype = weld_core.ecs.archetype; const format = weld_core.scene.format; +// M1.0.6 E2 — `of` variant resolution reads the base prefab's cooked `.prefab.bin` +// back through the same zero-copy accessor the loader uses. +const accessor = weld_core.scene.accessor; const AstArena = ast_mod.AstArena; const StringId = ast_mod.StringId; @@ -86,6 +89,26 @@ pub const CookError = error{ BadUuid, /// An entity `parent:` name does not match any entity in the scene. ParentNotFound, + // ── M1.0.6 E2 — prefab cook (`cookPrefab`) ── + /// No top-level `prefab` construct in a source cooked as a `.prefab.etch`. + NoPrefabConstruct, + /// More than one `prefab` construct (a `.prefab.etch` holds exactly one). + MultiplePrefabs, + /// A `scene` construct appeared in a source cooked as a `.prefab.etch`. + SceneNotAllowedInPrefab, + /// `prefab "Z" extends "X"` — extension cook is owned by M1.0.6 E5, not E2. + ExtendsUnsupported, + /// `on_attach`/`on_detach`/`requires` on a standalone or `of` prefab — those + /// clauses are valid only with `extends` (`etch-grammar.md` §15 l.1653, §30.5). + PrefabHookNotAllowed, + /// `prefab "Y" of "X"` but the base `X.prefab.bin` could not be resolved + /// (no resolver, or the resolver returned null for the base name). + BasePrefabMissing, + /// The resolved base `.prefab.bin` bytes failed `accessor.open`/`verifyHash`. + BasePrefabCorrupt, + /// A base prefab component's name is unknown to the variant's registry, or + /// its on-disk size disagrees with the variant registry's layout. + BaseSchemaMismatch, OutOfMemory, }; @@ -134,6 +157,61 @@ fn fail(diag_out: ?*[]const u8, err: CookError, msg: []const u8) CookError { return err; } +/// Resolves a referenced prefab name (the target of `of "X"`) to its already +/// cooked `.prefab.bin` bytes, or null if unknown. A `.prefab.bin` is the same +/// format as a `.scene.bin`, so a variant's base is read back through the +/// `accessor`. This is the cook-time prefab registry / path map (distinct from +/// Etch `import`, which is M1.0.7); the driver (`tools/scene_cook`) wires it to +/// the on-disk cook output, and tests wire it to an in-process byte buffer. +pub const BaseResolver = struct { + ctx: *anyopaque, + resolveFn: *const fn (ctx: *anyopaque, name: []const u8) ?[]const u8, + + pub fn resolve(self: BaseResolver, name: []const u8) ?[]const u8 { + return self.resolveFn(self.ctx, name); + } +}; + +/// Cook a `.prefab.etch` source into the neutral model + its registry, the same +/// way `cook` handles `.scene.etch`. A prefab is a mini-scene (one `prefab` +/// construct, body = `{ entity_decl }`, no `resources`/`instance`), serialized to +/// the identical `.scene.bin` format. +/// +/// Three forms (`etch-reference-part2.md` §30): a **standalone** prefab cooks its +/// entities directly; a **variant** `prefab "Y" of "X"` resolves `X`'s cooked +/// `.prefab.bin` via `base_resolver`, inherits all of X's flattened components, +/// and applies Y's per-entity overrides (field-merge on shared components, add on +/// new ones) — producing the fully flattened set. An **extension** `extends` is +/// rejected here (`error.ExtendsUnsupported`) — its cook is M1.0.6 E5. +/// +/// `base_resolver` may be null for a standalone prefab; an `of` prefab requires +/// it. On failure returns a `CookError` and sets `diag_out` (if non-null). +pub fn cookPrefab( + gpa: std.mem.Allocator, + source: []const u8, + base_resolver: ?BaseResolver, + diag_out: ?*[]const u8, +) CookError!Cooked { + const parser = @import("parser.zig"); + var pr = parser.parse(gpa, source) catch return fail(diag_out, error.ParseFailed, "Etch parse failed (allocator error)"); + defer pr.deinit(gpa); + if (pr.diagnostics.len > 0) return fail(diag_out, error.ParseFailed, "Etch parse failed"); + const ast = &pr.ast; + + var registry = Registry.init(); + errdefer registry.deinit(gpa); + + var b = Builder.init(gpa, ast, ®istry); + defer b.deinitScratch(); + errdefer b.arena.deinit(); + + try b.registerDecls(diag_out); + const prefab_decl = try b.findPrefab(diag_out); + const model = try b.buildPrefab(prefab_decl, base_resolver, diag_out); + + return .{ .model = model, .registry = registry }; +} + // ── Builder ────────────────────────────────────────────────────────────────── /// One in-progress entity, accumulated before archetype grouping. `comp_ids` is @@ -242,6 +320,27 @@ const Builder = struct { return found orelse fail(diag_out, error.NoSceneConstruct, "no scene construct in the source"); } + /// Locate the single `prefab` construct, erroring if there are zero or many, + /// or if a `scene` construct is present (a `.prefab.etch` holds exactly one + /// `prefab` and no `scene`). The M1.0.6 E2 twin of `findScene`. + fn findPrefab(self: *Builder, diag_out: ?*[]const u8) CookError!ast_mod.PrefabDecl { + const kinds = self.ast.items.items(.kind); + const datas = self.ast.items.items(.data); + var found: ?ast_mod.PrefabDecl = null; + var i: usize = 0; + while (i < self.ast.items.len) : (i += 1) { + switch (kinds[i]) { + .scene_decl => return fail(diag_out, error.SceneNotAllowedInPrefab, "a scene construct in a prefab source (.prefab.etch holds exactly one prefab)"), + .prefab_decl => { + if (found != null) return fail(diag_out, error.MultiplePrefabs, "more than one prefab construct in the source"); + found = self.ast.prefab_decls.items[datas[i]]; + }, + else => {}, + } + } + return found orelse fail(diag_out, error.NoPrefabConstruct, "no prefab construct in the source"); + } + /// Build the full neutral model from the resolved scene. fn build(self: *Builder, scene_decl: ast_mod.SceneDecl, diag_out: ?*[]const u8) CookError!format.CookModel { // Reject `instance of` up front (M1.0.6 owns prefab flattening). @@ -282,17 +381,208 @@ const Builder = struct { }; } + /// Build the neutral model from a `prefab` construct (the M1.0.6 E2 twin of + /// `build`). Standalone prefabs cook their entities directly; an `of` variant + /// inherits the base prefab's flattened components (read from its cooked + /// `.prefab.bin` via the accessor) then applies per-entity overrides. A prefab + /// body is `{ entity_decl }` only — no `resources`/`instance` (§15 l.1624). + fn buildPrefab(self: *Builder, pd: ast_mod.PrefabDecl, base_resolver: ?BaseResolver, diag_out: ?*[]const u8) CookError!format.CookModel { + if (pd.relation == .extends) return fail(diag_out, error.ExtendsUnsupported, "`extends` (extension) prefab cook is owned by M1.0.6 E5, not E2"); + // `requires`/`on_attach`/`on_detach` are valid only with `extends` (§30.5). + if (pd.requires_len != 0 or pd.has_on_attach or pd.has_on_detach) + return fail(diag_out, error.PrefabHookNotAllowed, "`requires`/`on_attach`/`on_detach` are valid only on an `extends` prefab"); + + const prefab_entities = self.ast.scene_entities.items[pd.entities_start .. pd.entities_start + pd.entities_len]; + + var entities: std.ArrayListUnmanaged(EntityBuild) = .empty; + defer entities.deinit(self.gpa); + + if (pd.relation == .of) { + const base_name = self.ast.strings.slice(pd.relation_target); + const resolver = base_resolver orelse return fail(diag_out, error.BasePrefabMissing, "`of` variant cooked without a base-prefab resolver"); + const base_bytes = resolver.resolve(base_name) orelse return fail(diag_out, error.BasePrefabMissing, "`of` variant references a base prefab the resolver does not know"); + var acc = accessor.Accessor.open(base_bytes) catch return fail(diag_out, error.BasePrefabCorrupt, "base prefab .prefab.bin failed to open (magic/version)"); + if (!acc.verifyHash()) return fail(diag_out, error.BasePrefabCorrupt, "base prefab .prefab.bin content hash mismatch"); + try self.reconstructBase(acc, &entities, diag_out); + try self.mergeVariantEntities(prefab_entities, &entities, diag_out); + } else { + for (prefab_entities) |e| { + const eb = try self.buildEntity(e, diag_out); + try entities.append(self.gpa, eb); + } + } + + // Resolve parent names → uuid indices (same invariant as the scene path). + for (entities.items) |*eb| { + if (eb.parent_name.len != 0 and self.name_to_uuid_idx.get(eb.parent_name) == null) + return fail(diag_out, error.ParentNotFound, "entity parent name does not match any entity in the prefab"); + } + + const archetypes = try self.groupArchetypes(entities.items); + const content_version = try self.versionFromNode(pd.version, diag_out); + + return .{ + .strings = try self.a().dupe([]const u8, self.strings.items), + .uuids = try self.a().dupe([16]u8, self.uuids.items), + .resources = &.{}, + .archetypes = archetypes, + .content_version = content_version, + .arena = self.arena, + }; + } + + /// Reconstruct the base prefab's flattened entities from its cooked + /// `.prefab.bin` (read via the accessor) as `EntityBuild`s, interning their + /// names/uuids into the variant's model tables. Each on-disk schema **name** + /// maps to the variant registry's `ComponentId` (`idOf`) and the column bytes + /// copy verbatim — the variant inherits all of the base's components. Per + /// entity, the column order is re-sorted to the variant registry's id order + /// (the base file is sorted by the *base* registry's ids, which may differ). + fn reconstructBase(self: *Builder, acc: accessor.Accessor, out: *std.ArrayListUnmanaged(EntityBuild), diag_out: ?*[]const u8) CookError!void { + // Pass 1 — intern every base entity's name+uuid first: a parent ordinal + // may point at an entity that lives in a later archetype block. + var uuid_to_name_idx: std.AutoHashMapUnmanaged([16]u8, u32) = .empty; + defer uuid_to_name_idx.deinit(self.gpa); + var ai: u32 = 0; + while (ai < acc.archetypeCount()) : (ai += 1) { + const arch = acc.archetype(ai); + var slot: usize = 0; + while (slot < arch.entity_count) : (slot += 1) { + const name_idx = try self.internString(arch.entityName(slot)); + const uuid_bytes = arch.entityUuid(slot).*; + const uuid_idx = try self.internUuid(uuid_bytes); + try self.name_to_uuid_idx.put(self.gpa, self.strings.items[name_idx], uuid_idx); + try uuid_to_name_idx.put(self.gpa, uuid_bytes, name_idx); + } + } + // Pass 2 — one EntityBuild per base entity (components + parent name). + ai = 0; + while (ai < acc.archetypeCount()) : (ai += 1) { + const arch = acc.archetype(ai); + // Map each column's on-disk schema → variant registry id + validate. + const ids0 = try self.a().alloc(ComponentId, arch.component_count); + var c: usize = 0; + while (c < arch.component_count) : (c += 1) { + const sch = acc.schema(arch.schemaIndex(c)); + const id = self.registry.idOf(sch.name) orelse return fail(diag_out, error.BaseSchemaMismatch, "base prefab uses a component the variant does not declare"); + if (self.registry.componentSize(id) != sch.size) return fail(diag_out, error.BaseSchemaMismatch, "base prefab component size disagrees with the variant registry layout"); + ids0[c] = id; + } + var slot: usize = 0; + while (slot < arch.entity_count) : (slot += 1) { + const ids = try self.a().dupe(ComponentId, ids0); + const blobs = try self.a().alloc([]u8, arch.component_count); + c = 0; + while (c < arch.component_count) : (c += 1) blobs[c] = try self.a().dupe(u8, arch.componentSlot(c, slot)); + sortIdsBlobs(ids, blobs); + + const name_idx = try self.internString(arch.entityName(slot)); + const uuid_idx = try self.internUuid(arch.entityUuid(slot).*); + const parent_ord = arch.entityParent(slot); + const parent_name: []const u8 = if (parent_ord == format.no_parent) "" else blk: { + const pidx = uuid_to_name_idx.get(acc.uuidAt(parent_ord).*) orelse return fail(diag_out, error.BaseSchemaMismatch, "base prefab parent ordinal does not resolve to an entity"); + break :blk self.strings.items[pidx]; + }; + try out.append(self.gpa, .{ + .name_idx = name_idx, + .uuid_idx = uuid_idx, + .parent_name = parent_name, + .comp_ids = ids, + .comp_blobs = blobs, + }); + } + } + } + + /// Apply each variant entity over the inherited base set: an entity whose name + /// matches a base entity field-merges/adds its components onto it (identity — + /// uuid/parent — stays the base's); an entity with a new name is appended as a + /// fresh entity (its `uuid:` required, like a standalone). + fn mergeVariantEntities(self: *Builder, variant_entities: []const ast_mod.SceneEntity, entities: *std.ArrayListUnmanaged(EntityBuild), diag_out: ?*[]const u8) CookError!void { + for (variant_entities) |ve| { + const vname = self.ast.strings.slice(ve.name); + if (self.findEntityIdxByName(entities.items, vname)) |idx| { + try self.applyVariantOverrides(&entities.items[idx], ve, diag_out); + } else { + try entities.append(self.gpa, try self.buildEntity(ve, diag_out)); + } + } + } + + fn findEntityIdxByName(self: *Builder, entities: []const EntityBuild, name: []const u8) ?usize { + for (entities, 0..) |eb, i| { + if (std.mem.eql(u8, self.strings.items[eb.name_idx], name)) return i; + } + return null; + } + + /// Overlay a variant entity's component instances on an inherited base entity: + /// a re-declared component the base already has is field-merged (base bytes + + /// the variant's set fields); a component the base lacks is added (registry + /// default + the variant's fields). Identity (uuid/parent) stays the base's. + fn applyVariantOverrides(self: *Builder, eb: *EntityBuild, ve: ast_mod.SceneEntity, diag_out: ?*[]const u8) CookError!void { + const instances = self.ast.component_instances.items[ve.components_start .. ve.components_start + ve.components_len]; + var ids: std.ArrayListUnmanaged(ComponentId) = .empty; + defer ids.deinit(self.gpa); + var blobs: std.ArrayListUnmanaged([]u8) = .empty; + defer blobs.deinit(self.gpa); + try ids.appendSlice(self.gpa, eb.comp_ids); + try blobs.appendSlice(self.gpa, eb.comp_blobs); + + for (instances) |ci| { + const type_name = self.ast.strings.slice(ci.type_name); + const id = self.registry.idOf(type_name) orelse return fail(diag_out, error.UndeclaredType, "variant entity references an undeclared component type"); + if (indexOfId(ids.items, id)) |ci_idx| { + blobs.items[ci_idx] = try self.mergeComponentBlob(blobs.items[ci_idx], id, ci, diag_out); + } else { + try ids.append(self.gpa, id); + try blobs.append(self.gpa, try self.buildComponentBlob(id, ci, diag_out)); + } + } + + const new_ids = try self.a().dupe(ComponentId, ids.items); + const new_blobs = try self.a().dupe([]u8, blobs.items); + sortIdsBlobs(new_ids, new_blobs); + eb.comp_ids = new_ids; + eb.comp_blobs = new_blobs; + } + + /// Copy `base_blob` and apply the variant instance's set fields over it (the + /// variant override form is `Component { field: value }`; the + /// `Component.field =` per-field form is scene-instance-only per the grammar). + /// Components are POD scalar, so every field is a scalar encode. + fn mergeComponentBlob(self: *Builder, base_blob: []const u8, id: ComponentId, ci: ast_mod.ComponentInstance, diag_out: ?*[]const u8) CookError![]u8 { + const size = self.registry.componentSize(id); + const blob = try self.a().alloc(u8, size); + @memcpy(blob, base_blob); + const fields = self.ast.struct_lit_fields.items[ci.fields_start .. ci.fields_start + ci.fields_len]; + for (fields) |slf| { + if (slf.name == 0) return fail(diag_out, error.SpreadUnsupported, "`..spread` is not supported in a component instance"); + const fname = self.ast.strings.slice(slf.name); + const fd = self.registry.findField(id, fname) orelse return fail(diag_out, error.UnknownField, "variant component instance sets a field the component does not declare"); + try self.encodeScalar(blob, fd, slf.value, diag_out); + } + return blob; + } + /// Const-evaluate the scene's `version:` field to a `u16` (0 if absent). The /// authored content version is otherwise silently lost (it rides through to /// `SceneHeader.content_version`). fn sceneContentVersion(self: *Builder, scene_decl: ast_mod.SceneDecl, diag_out: ?*[]const u8) CookError!u16 { - if (scene_decl.version.isNone()) return 0; - const v = interp.evalConst(self.ast, scene_decl.version) catch return fail(diag_out, error.NonConstValue, "scene version must be a constant int"); + return self.versionFromNode(scene_decl.version, diag_out); + } + + /// Const-evaluate a `version:` expression node to a `u16` (0 if `.none`). + /// Shared by the scene cook and the prefab cook — the authored content + /// version rides through to `SceneHeader.content_version` unchanged. + fn versionFromNode(self: *Builder, version: NodeId, diag_out: ?*[]const u8) CookError!u16 { + if (version.isNone()) return 0; + const v = interp.evalConst(self.ast, version) catch return fail(diag_out, error.NonConstValue, "version must be a constant int"); const x: i64 = switch (v) { .int_ => |n| n, - else => return fail(diag_out, error.NonConstValue, "scene version must be an int"), + else => return fail(diag_out, error.NonConstValue, "version must be an int"), }; - return std.math.cast(u16, x) orelse return fail(diag_out, error.NonConstValue, "scene version out of u16 range"); + return std.math.cast(u16, x) orelse return fail(diag_out, error.NonConstValue, "version out of u16 range"); } fn buildEntity(self: *Builder, e: ast_mod.SceneEntity, diag_out: ?*[]const u8) CookError!EntityBuild { @@ -566,6 +856,12 @@ const Builder = struct { } }; +/// Index of `id` in `ids`, or null. Linear (component counts per entity are tiny). +fn indexOfId(ids: []const ComponentId, id: ComponentId) ?usize { + for (ids, 0..) |x, i| if (x == id) return i; + return null; +} + /// Insertion-sort `(ids, blobs)` jointly by `ids` ascending. Component counts /// per entity are tiny, so insertion sort is both simplest and fastest. fn sortIdsBlobs(ids: []ComponentId, blobs: [][]u8) void { diff --git a/tests/scene/prefab_cook_test.zig b/tests/scene/prefab_cook_test.zig new file mode 100644 index 0000000..c3d4c08 --- /dev/null +++ b/tests/scene/prefab_cook_test.zig @@ -0,0 +1,179 @@ +//! M1.0.6 E2 — `.prefab.etch` → `cookPrefab` → `.prefab.bin` writer → accessor. +//! A prefab is a mini-scene: it cooks to the identical `.scene.bin` format, so it +//! round-trips through the same `writer` + `accessor`. Covers the standalone form +//! and the `of` variant (base inherited from its cooked `.prefab.bin`, field-merge +//! on shared components, add on new ones), plus the rejected forms (`extends`, +//! hooks on a non-`extends` prefab). Components are POD scalar (the cook's only +//! component kind — enum/string are resource-only), so fixtures use f32/i32. + +const std = @import("std"); +const weld_core = @import("weld_core"); +const weld_etch = @import("weld_etch"); + +const scene = weld_core.scene; +const Registry = weld_core.ecs.registry.Registry; +const scene_cook = weld_etch.scene_cook; + +const Accessor = scene.accessor.Accessor; + +/// A one-entry base-prefab resolver over an in-process byte buffer. +const OneResolver = struct { + name: []const u8, + bytes: []const u8, + + fn resolve(ctx: *anyopaque, name: []const u8) ?[]const u8 { + const self: *OneResolver = @ptrCast(@alignCast(ctx)); + return if (std.mem.eql(u8, name, self.name)) self.bytes else null; + } + + fn base(self: *OneResolver) scene_cook.BaseResolver { + return .{ .ctx = self, .resolveFn = OneResolver.resolve }; + } +}; + +fn columnOf(acc: Accessor, arch: Accessor.Archetype, comp: []const u8) ?usize { + var c: usize = 0; + while (c < arch.component_count) : (c += 1) { + if (std.mem.eql(u8, acc.schema(arch.schemaIndex(c)).name, comp)) return c; + } + return null; +} + +fn decodeF32(acc: Accessor, reg: *const Registry, arch: Accessor.Archetype, comp: []const u8, field: []const u8, slot: usize) f32 { + const c = columnOf(acc, arch, comp).?; + const fd = reg.findField(reg.idOf(comp).?, field).?; + return @bitCast(std.mem.readInt(u32, arch.componentSlot(c, slot)[fd.offset..][0..4], .little)); +} + +const standalone_src = + \\component Transform { x: f32 = 0.0, y: f32 = 0.0, z: f32 = 0.0 } + \\component Light { intensity: f32 = 2000.0, radius: f32 = 8.0 } + \\prefab "WallTorch" { + \\ version: 2 + \\ entity "root" { + \\ uuid: "7b3e2f1a-42a3-4f2b-8c9d-a3f2b1c98d4e" + \\ Transform { x: 1.0 } + \\ Light { intensity: 1500.0 } + \\ } + \\} +; + +test "standalone prefab cooks and reads back" { + const gpa = std.testing.allocator; + + var cooked = try scene_cook.cookPrefab(gpa, standalone_src, null, null); + defer cooked.deinit(gpa); + + const bytes = try scene.writer.write(gpa, cooked.model, &cooked.registry); + defer gpa.free(bytes); + + var acc = try Accessor.open(bytes); + try std.testing.expect(acc.verifyHash()); + try std.testing.expectEqual(@as(u16, 2), acc.header.content_version); + try std.testing.expectEqual(@as(u32, 1), acc.archetypeCount()); + + const arch = acc.archetype(0); + try std.testing.expectEqual(@as(u32, 1), arch.entity_count); + try std.testing.expectEqual(@as(u32, 2), arch.component_count); // [Light, Transform] + try std.testing.expectEqualStrings("root", arch.entityName(0)); + try std.testing.expectEqual(scene.format.no_parent, arch.entityParent(0)); + try std.testing.expectEqual(@as(u8, 0x7b), arch.entityUuid(0)[0]); + + // Authored field + inherited default. + try std.testing.expectApproxEqAbs(@as(f32, 1500.0), decodeF32(acc, &cooked.registry, arch, "Light", "intensity", 0), 1e-3); + try std.testing.expectApproxEqAbs(@as(f32, 8.0), decodeF32(acc, &cooked.registry, arch, "Light", "radius", 0), 1e-3); + try std.testing.expectApproxEqAbs(@as(f32, 1.0), decodeF32(acc, &cooked.registry, arch, "Transform", "x", 0), 1e-3); +} + +const variant_src = + \\component Transform { x: f32 = 0.0, y: f32 = 0.0, z: f32 = 0.0 } + \\component Light { intensity: f32 = 2000.0, radius: f32 = 8.0 } + \\component Glow { power: f32 = 1.0 } + \\prefab "WallTorch_Blue" of "WallTorch" { + \\ entity "root" { + \\ Light { intensity: 2500.0 } + \\ Glow { power: 3.0 } + \\ } + \\} +; + +test "variant prefab resolves of-chain" { + const gpa = std.testing.allocator; + + // Cook the base standalone prefab to its `.prefab.bin` bytes. + var base = try scene_cook.cookPrefab(gpa, standalone_src, null, null); + defer base.deinit(gpa); + const base_bytes = try scene.writer.write(gpa, base.model, &base.registry); + defer gpa.free(base_bytes); + + var resolver = OneResolver{ .name = "WallTorch", .bytes = base_bytes }; + var cooked = try scene_cook.cookPrefab(gpa, variant_src, resolver.base(), null); + defer cooked.deinit(gpa); + + const bytes = try scene.writer.write(gpa, cooked.model, &cooked.registry); + defer gpa.free(bytes); + + var acc = try Accessor.open(bytes); + try std.testing.expect(acc.verifyHash()); + try std.testing.expectEqual(@as(u32, 1), acc.archetypeCount()); + + const arch = acc.archetype(0); + try std.testing.expectEqual(@as(u32, 1), arch.entity_count); + try std.testing.expectEqual(@as(u32, 3), arch.component_count); // [Glow, Light, Transform] + try std.testing.expectEqualStrings("root", arch.entityName(0)); + + // Overridden field, inherited field, inherited component, added component. + try std.testing.expectApproxEqAbs(@as(f32, 2500.0), decodeF32(acc, &cooked.registry, arch, "Light", "intensity", 0), 1e-3); + try std.testing.expectApproxEqAbs(@as(f32, 8.0), decodeF32(acc, &cooked.registry, arch, "Light", "radius", 0), 1e-3); + try std.testing.expectApproxEqAbs(@as(f32, 1.0), decodeF32(acc, &cooked.registry, arch, "Transform", "x", 0), 1e-3); + try std.testing.expectApproxEqAbs(@as(f32, 3.0), decodeF32(acc, &cooked.registry, arch, "Glow", "power", 0), 1e-3); +} + +test "prefab re-cook is byte-identical" { + const gpa = std.testing.allocator; + var c1 = try scene_cook.cookPrefab(gpa, standalone_src, null, null); + defer c1.deinit(gpa); + const b1 = try scene.writer.write(gpa, c1.model, &c1.registry); + defer gpa.free(b1); + var c2 = try scene_cook.cookPrefab(gpa, standalone_src, null, null); + defer c2.deinit(gpa); + const b2 = try scene.writer.write(gpa, c2.model, &c2.registry); + defer gpa.free(b2); + try std.testing.expectEqualSlices(u8, b1, b2); +} + +test "cookPrefab rejects extends (M1.0.6 E5) and a scene source" { + const gpa = std.testing.allocator; + const extends_src = + \\component Health { current: i32 = 100, max: i32 = 100 } + \\prefab "CombatModule" extends "BaseCharacter" requires Health { + \\ entity "root" { uuid: "00000000-0000-0000-0000-000000000001" Health { max: 150 } } + \\} + ; + var diag: []const u8 = ""; + try std.testing.expectError(error.ExtendsUnsupported, scene_cook.cookPrefab(gpa, extends_src, null, &diag)); + + const scene_src = + \\component Health { current: i32 = 100, max: i32 = 100 } + \\scene "S" { entity "e" { uuid: "00000000-0000-0000-0000-000000000002" Health {} } } + ; + try std.testing.expectError(error.SceneNotAllowedInPrefab, scene_cook.cookPrefab(gpa, scene_src, null, &diag)); +} + +test "cookPrefab rejects hooks on a non-extends prefab" { + const gpa = std.testing.allocator; + const of_with_hook = + \\component Health { current: i32 = 100, max: i32 = 100 } + \\prefab "Boosted" of "Base" requires Health { + \\ entity "root" { uuid: "00000000-0000-0000-0000-000000000003" Health { max: 200 } } + \\} + ; + var diag: []const u8 = ""; + try std.testing.expectError(error.PrefabHookNotAllowed, scene_cook.cookPrefab(gpa, of_with_hook, null, &diag)); +} + +test "of variant without a resolver errors BasePrefabMissing" { + const gpa = std.testing.allocator; + var diag: []const u8 = ""; + try std.testing.expectError(error.BasePrefabMissing, scene_cook.cookPrefab(gpa, variant_src, null, &diag)); +} diff --git a/tools/scene_cook/main.zig b/tools/scene_cook/main.zig index 19e4541..20ec465 100644 --- a/tools/scene_cook/main.zig +++ b/tools/scene_cook/main.zig @@ -1,8 +1,16 @@ -//! `scene_cook` — thin CLI shim around the M1.0.4 scene cook. Parses args + does -//! file I/O; all real work is `weld_etch.scene_cook.cook` + `weld_core.scene -//! .writer.write` in-process. Mirrors `tools/etch_cook` / `tools/asset_cook`. +//! `scene_cook` — thin CLI shim around the M1.0.4 scene cook and the M1.0.6 +//! prefab cook. Parses args + does file I/O; all real work is +//! `weld_etch.scene_cook.{cook,cookPrefab}` + `weld_core.scene.writer.write` +//! in-process. Mirrors `tools/etch_cook` / `tools/asset_cook`. //! -//! scene_cook --output +//! scene_cook --output +//! scene_cook --output [--prefab-dir ] +//! +//! The input kind is taken from its extension: `*.prefab.etch` → prefab cook, +//! `*.scene.etch` → scene cook. A `prefab "Y" of "X"` variant resolves its base +//! `X.prefab.bin` under `--prefab-dir` (by prefab NAME → `/.prefab.bin`), +//! so the caller cooks prefabs into that dir before cooking variants/scenes that +//! reference them ("cook prefabs before scenes"). const std = @import("std"); @@ -12,6 +20,28 @@ const weld_core = @import("weld_core"); const scene_cook = weld_etch.scene_cook; const writer = weld_core.scene.writer; +/// On-disk base-prefab resolver: maps a prefab name to `/.prefab.bin`, +/// reading it into `arena` on demand (the bytes must outlive the cook). A missing +/// dir or unreadable file resolves to null → the cook reports `BasePrefabMissing`. +const DirResolver = struct { + io: std.Io, + dir: std.Io.Dir, + prefab_dir: ?[]const u8, + arena: std.mem.Allocator, + gpa: std.mem.Allocator, + + fn resolve(ctx: *anyopaque, name: []const u8) ?[]const u8 { + const self: *DirResolver = @ptrCast(@alignCast(ctx)); + const root = self.prefab_dir orelse return null; + const path = std.fmt.allocPrint(self.arena, "{s}/{s}.prefab.bin", .{ root, name }) catch return null; + return readWholeFile(self.arena, self.io, self.dir, path) catch null; + } + + fn base(self: *DirResolver) scene_cook.BaseResolver { + return .{ .ctx = self, .resolveFn = DirResolver.resolve }; + } +}; + pub fn main(init: std.process.Init) !void { const gpa = init.gpa; const io = init.io; @@ -19,6 +49,7 @@ pub fn main(init: std.process.Init) !void { var output: ?[]const u8 = null; var input: ?[]const u8 = null; + var prefab_dir: ?[]const u8 = null; var i: usize = 1; while (i < args.len) : (i += 1) { const a = args[i]; @@ -26,6 +57,10 @@ pub fn main(init: std.process.Init) !void { i += 1; if (i >= args.len) return die(io, "missing path after --output"); output = args[i]; + } else if (std.mem.eql(u8, a, "--prefab-dir")) { + i += 1; + if (i >= args.len) return die(io, "missing path after --prefab-dir"); + prefab_dir = args[i]; } else if (std.mem.startsWith(u8, a, "--")) { return die(io, "unknown flag"); } else { @@ -33,8 +68,8 @@ pub fn main(init: std.process.Init) !void { input = a; } } - const out_path = output orelse return die(io, "missing --output "); - const in_path = input orelse return die(io, "missing "); + const out_path = output orelse return die(io, "missing --output "); + const in_path = input orelse return die(io, "missing "); const dir = std.Io.Dir.cwd(); const source = readWholeFile(gpa, io, dir, in_path) catch |err| { @@ -43,10 +78,28 @@ pub fn main(init: std.process.Init) !void { }; defer gpa.free(source); + const is_prefab = std.mem.endsWith(u8, in_path, ".prefab.etch"); + var diag: []const u8 = ""; - var cooked = scene_cook.cook(gpa, source, &diag) catch |err| { - try printErr(io, "cook failed: ", if (diag.len > 0) diag else @errorName(err)); - return err; + var cooked = blk: { + if (is_prefab) { + var resolver = DirResolver{ + .io = io, + .dir = dir, + .prefab_dir = prefab_dir, + .arena = init.arena.allocator(), + .gpa = gpa, + }; + break :blk scene_cook.cookPrefab(gpa, source, resolver.base(), &diag) catch |err| { + try printErr(io, "prefab cook failed: ", if (diag.len > 0) diag else @errorName(err)); + return err; + }; + } else { + break :blk scene_cook.cook(gpa, source, &diag) catch |err| { + try printErr(io, "cook failed: ", if (diag.len > 0) diag else @errorName(err)); + return err; + }; + } }; defer cooked.deinit(gpa); From 686bd4f9baab22264429074ee1aa1637dc6823e8 Mon Sep 17 00:00:00 2001 From: Guy Senpai Date: Sun, 28 Jun 2026 11:35:51 +0200 Subject: [PATCH 05/15] feat(scene): flatten instance of at scene cook (single-entity prefabs) --- briefs/M1.0.6-prefabs-crossrefs-extensions.md | 3 + build.zig | 4 + src/etch/scene_cook.zig | 142 +++++++++-- tests/scene/cook_errors_test.zig | 9 +- tests/scene/prefab_flatten_test.zig | 236 ++++++++++++++++++ tools/scene_cook/main.zig | 20 +- 6 files changed, 390 insertions(+), 24 deletions(-) create mode 100644 tests/scene/prefab_flatten_test.zig diff --git a/briefs/M1.0.6-prefabs-crossrefs-extensions.md b/briefs/M1.0.6-prefabs-crossrefs-extensions.md index 3d1034e..411d34f 100644 --- a/briefs/M1.0.6-prefabs-crossrefs-extensions.md +++ b/briefs/M1.0.6-prefabs-crossrefs-extensions.md @@ -237,6 +237,9 @@ Sparse (entities with no active extension are absent). The `§4` note's `u8` cou - 2026-06-28 10:30 — **E2 design clarifications (to confirm at review — implement within the frozen scope, do not modify it):** (1) **Prefab entities require an explicit `uuid:`** exactly like scene entities (`cookPrefab` reuses `buildEntity` verbatim → the frozen no-auto-gen determinism rule; the `.prefab.bin` format is identical to `.scene.bin` and references uuid ordinals). Spec §5 prefab examples omit uuids ("anonymous `entity {`" is shorthand per `ast.zig`); legal fixtures carry them. A variant entity that *overrides* a base entity (matched by name) may omit `uuid:` — it inherits the base identity. (2) **`of` merge granularity** = the grammar-legal `Component { field: value }` form (the `Component.field = value` per-field form is `instance_decl`-only per the parser, §15 l.1610), applied as a **field-merge** onto the inherited base component (shared component → overwrite set fields, keep the rest; new component → add). Base↔variant entities matched **by name**; merged entity keeps the **base's uuid + parent**. Multi-entity prefab hierarchy is reconstructed (parent-by-name preserved) but only single-entity + add-component is exercised by the E2 tests. - 2026-06-28 10:45 — E2 `tools/scene_cook/main.zig`: detects scene vs prefab by input extension (`.prefab.etch` → `cookPrefab`, `.scene.etch` → `cook`); new optional `--prefab-dir ` flag wires a `BaseResolver` that reads `/.prefab.bin` on demand (the on-disk cook-time prefab path map, by prefab NAME) so `of` variants resolve their base. "Cook prefabs before scenes" is the caller's ordering (cook each prefab into `--prefab-dir`, then the scenes/variants against it) — the single-file CLI just needs the resolver root. Smoke-tested: standalone + `of`-chain cook through the CLI to real files, re-opened with the accessor. - 2026-06-28 10:45 — E2 STOP: pushed, awaiting Guy's review + GO before E3. +- 2026-06-28 11:30 — E3 implemented (`instance of` flattening at scene cook). `scene_cook.zig`: `cook` is now a resolver-less wrapper over `cookScene(gpa, source, base_resolver, diag_out)`; `build` takes `base_resolver` and dispatches scene children to `buildEntity` (direct) or `buildInstanceEntity` (instance). `buildInstanceEntity` resolves `P.prefab.bin` via the resolver + accessor (`BasePrefabMissing`/`BasePrefabCorrupt`), inherits P's single entity's components (`flattenedPrefabComponents`, size-validated), then applies the instance body's overrides **in declaration order** — both grammar forms: `Comp { field: value }` → field-merge (reuses `mergeComponentBlob`), `Comp.field = value` → per-field write (`encodeScalar`), a component P lacks → add (`buildComponentBlob`). The instance supplies the entity's name + uuid (prefab template uuid discarded); instances are roots. Removed `InstanceOfUnsupported` (the M1.0.4 boundary); added `MultiEntityInstanceUnsupported` + `OverrideTargetMissing`. `tools/scene_cook/main.zig`: the scene branch now calls `cookScene` with the `--prefab-dir` resolver (scenes flatten instances too). `tests/scene/cook_errors_test.zig`: the old `InstanceOfUnsupported` case → `BasePrefabMissing` (resolver-less `cook` + instance). New `tests/scene/prefab_flatten_test.zig` (4 tests, step `test-prefab-flatten`): override-free instance == hand-authored (same archetype + byte-identical slots), both override forms, 3 instances cook→load→ECS via the M1.0.5 loader, multi-entity rejection. Green Debug + ReleaseSafe (10/10 prefab tests); full `zig build test` 832/849 pass (17 skip, 0 fail, exit 0); `zig fmt --check` + `zig build lint` clean. +- 2026-06-28 11:30 — **E3 multi-entity decision (per the E3 cadrage point #2).** `engine-scene-serialization.md` §2/§5 define **no** uuid-remapping scheme for a multi-entity prefab's internal entities at instantiation (the doc's "remapping UUID" refs are the M1.0.5 loader's runtime UUID→handle remap; "UUID v4 auto-generated" is editor authoring of unnamed entities) — verified by grep of §2/§5. Per the GO instruction's "if the spec does NOT define it" branch: M1.0.6 instantiates **single-entity prefabs only**, with an explicit typed `error.MultiEntityInstanceUnsupported`; multi-entity instantiation (and its hierarchy) is deferred to the dedicated hierarchy milestone (consistent with D-D). No Claude.ai round-trip (the spec-undefined branch, not the spec-defined one). +- 2026-06-28 11:30 — E3 STOP: pushed, awaiting Guy's review + GO before E4. ## Accepted deviations diff --git a/build.zig b/build.zig index 9e0bf45..f56f481 100644 --- a/build.zig +++ b/build.zig @@ -434,6 +434,10 @@ pub fn build(b: *std.Build) void { // (base inherited from its cooked `.prefab.bin`, field-merge/add overrides), // re-cook determinism, and the rejected forms (`extends`, hooks on `of`). .{ .path = "tests/scene/prefab_cook_test.zig", .scene = true, .dedicated_step = "test-prefab-cook" }, + // M1.0.6 / E3 — `instance of` flattening at scene cook: override-free + // instance == hand-authored equivalent (same archetype + bytes), both + // override forms, N instances cook→load→ECS, single-entity boundary. + .{ .path = "tests/scene/prefab_flatten_test.zig", .scene = true, .dedicated_step = "test-prefab-flatten" }, // M1.0.5 / E2 — runtime loader `.scene.bin` → ECS: instantiate every // entity (component bytes verbatim), two-phase `on_spawned` (fires once // per entity, after all entities exist). `weld_core` only (no `.scene` diff --git a/src/etch/scene_cook.zig b/src/etch/scene_cook.zig index 11a6565..49e64ce 100644 --- a/src/etch/scene_cook.zig +++ b/src/etch/scene_cook.zig @@ -62,8 +62,6 @@ pub const CookError = error{ NoSceneConstruct, /// More than one `scene` construct (a `.scene.etch` holds exactly one). MultipleScenes, - /// `instance of "Prefab"` — prefab flattening is owned by M1.0.6. - InstanceOfUnsupported, /// A component/resource field has a type the runtime registry rejects /// (`error.InvalidProgram`: e.g. `Vec3`/`Entity`/`AssetHandle`, or `string` /// on a component). @@ -109,6 +107,17 @@ pub const CookError = error{ /// A base prefab component's name is unknown to the variant's registry, or /// its on-disk size disagrees with the variant registry's layout. BaseSchemaMismatch, + // ── M1.0.6 E3 — `instance of` flattening at scene cook ── + /// `instance of "P"` where `P.prefab.bin` holds more than one entity. M1.0.6 + /// instantiates only single-entity prefabs: the instance supplies one uuid and + /// the spec defines no remapping for a multi-entity prefab's internal uuids at + /// instantiation (`engine-scene-serialization.md` §2/§5). Multi-entity + /// instantiation (and its hierarchy) is a dedicated later milestone (D-D). + MultiEntityInstanceUnsupported, + /// A `Comp.field = value` per-field override targets a component the flattened + /// instance does not carry (neither inherited from the prefab nor added by an + /// earlier `Comp { … }` member of the same instance body). + OverrideTargetMissing, OutOfMemory, }; @@ -128,10 +137,26 @@ pub const Cooked = struct { } }; -/// Cook an Etch source string into the neutral scene model + its registry. -/// On failure returns a `CookError` and, if `diag_out` is non-null, sets it to a -/// static human-readable message. No `.scene.bin` is produced on failure. +/// Cook an Etch source string into the neutral scene model + its registry, with +/// no prefab resolver — a scene that uses `instance of` errors `BasePrefabMissing` +/// (use `cookScene` with a resolver to flatten instances). On failure returns a +/// `CookError` and, if `diag_out` is non-null, sets it to a static message. No +/// `.scene.bin` is produced on failure. pub fn cook(gpa: std.mem.Allocator, source: []const u8, diag_out: ?*[]const u8) CookError!Cooked { + return cookScene(gpa, source, null, diag_out); +} + +/// Cook a `.scene.etch` source, resolving each `instance of "P"` by flattening +/// `P.prefab.bin` (located through `base_resolver`) into the instance's entity: +/// the prefab's components are inherited and the instance's overrides applied +/// (M1.0.6 E3). `base_resolver` may be null for a scene with no instances; an +/// instance with a null/unknowing resolver errors `BasePrefabMissing`. +pub fn cookScene( + gpa: std.mem.Allocator, + source: []const u8, + base_resolver: ?BaseResolver, + diag_out: ?*[]const u8, +) CookError!Cooked { const parser = @import("parser.zig"); var pr = parser.parse(gpa, source) catch return fail(diag_out, error.ParseFailed, "Etch parse failed (allocator error)"); defer pr.deinit(gpa); @@ -147,7 +172,7 @@ pub fn cook(gpa: std.mem.Allocator, source: []const u8, diag_out: ?*[]const u8) try b.registerDecls(diag_out); const scene_decl = try b.findScene(diag_out); - const model = try b.build(scene_decl, diag_out); + const model = try b.build(scene_decl, base_resolver, diag_out); return .{ .model = model, .registry = registry }; } @@ -341,21 +366,22 @@ const Builder = struct { return found orelse fail(diag_out, error.NoPrefabConstruct, "no prefab construct in the source"); } - /// Build the full neutral model from the resolved scene. - fn build(self: *Builder, scene_decl: ast_mod.SceneDecl, diag_out: ?*[]const u8) CookError!format.CookModel { - // Reject `instance of` up front (M1.0.6 owns prefab flattening). + /// Build the full neutral model from the resolved scene. Direct entities cook + /// from their component instances; `instance of "P"` children are flattened — + /// the prefab's components are inherited from `P.prefab.bin` (via the resolver) + /// and the instance's overrides applied (M1.0.6 E3). + fn build(self: *Builder, scene_decl: ast_mod.SceneDecl, base_resolver: ?BaseResolver, diag_out: ?*[]const u8) CookError!format.CookModel { const children = self.ast.scene_children.items[scene_decl.children_start .. scene_decl.children_start + scene_decl.children_len]; - for (children) |child| { - if (child.kind == .instance) return fail(diag_out, error.InstanceOfUnsupported, "`instance of` (prefab instances) is not cooked by M1.0.4 (owned by M1.0.6)"); - } // Per-entity build, accumulating the name→uuid-index map for parent // resolution in a second pass. var entities: std.ArrayListUnmanaged(EntityBuild) = .empty; defer entities.deinit(self.gpa); for (children) |child| { - const e = self.ast.scene_entities.items[child.index]; - const eb = try self.buildEntity(e, diag_out); + const eb = switch (child.kind) { + .entity => try self.buildEntity(self.ast.scene_entities.items[child.index], diag_out), + .instance => try self.buildInstanceEntity(self.ast.scene_instances.items[child.index], base_resolver, diag_out), + }; try entities.append(self.gpa, eb); } @@ -621,6 +647,94 @@ const Builder = struct { }; } + /// Flatten an `instance of "P" "name" { … }` into one entity: inherit P's + /// single entity's components (from `P.prefab.bin` via the resolver), then + /// apply the instance body's overrides in declaration order — both forms + /// (`Comp { field: value }` field-merge and `Comp.field = value` per-field) and + /// added components. M1.0.6 instantiates only single-entity prefabs + /// (`MultiEntityInstanceUnsupported` otherwise). The instance supplies the + /// entity's name + uuid (the prefab's template uuid is discarded); instances + /// are roots (the grammar's `instance_decl` has no `parent:`). + fn buildInstanceEntity(self: *Builder, inst: ast_mod.SceneInstance, base_resolver: ?BaseResolver, diag_out: ?*[]const u8) CookError!EntityBuild { + const prefab_name = self.ast.strings.slice(inst.prefab_name); + const resolver = base_resolver orelse return fail(diag_out, error.BasePrefabMissing, "scene `instance of` cooked without a prefab resolver (no --prefab-dir?)"); + const base_bytes = resolver.resolve(prefab_name) orelse return fail(diag_out, error.BasePrefabMissing, "`instance of` references a prefab the resolver does not know"); + var acc = accessor.Accessor.open(base_bytes) catch return fail(diag_out, error.BasePrefabCorrupt, "instanced prefab .prefab.bin failed to open (magic/version)"); + if (!acc.verifyHash()) return fail(diag_out, error.BasePrefabCorrupt, "instanced prefab .prefab.bin content hash mismatch"); + + // Inherit the prefab's single entity's components (variant-registry ids). + var ids: std.ArrayListUnmanaged(ComponentId) = .empty; + defer ids.deinit(self.gpa); + var blobs: std.ArrayListUnmanaged([]u8) = .empty; + defer blobs.deinit(self.gpa); + try self.flattenedPrefabComponents(acc, &ids, &blobs, diag_out); + + // Apply the instance body's overrides in declaration order. + const members = self.ast.scene_instance_members.items[inst.members_start .. inst.members_start + inst.members_len]; + for (members) |m| switch (m.kind) { + .component => { + const ci = self.ast.component_instances.items[m.index]; + const id = self.registry.idOf(self.ast.strings.slice(ci.type_name)) orelse return fail(diag_out, error.UndeclaredType, "instance component references an undeclared component type"); + if (indexOfId(ids.items, id)) |idx| { + blobs.items[idx] = try self.mergeComponentBlob(blobs.items[idx], id, ci, diag_out); + } else { + try ids.append(self.gpa, id); + try blobs.append(self.gpa, try self.buildComponentBlob(id, ci, diag_out)); + } + }, + .field_override => { + const fo = self.ast.field_overrides.items[m.index]; + const id = self.registry.idOf(self.ast.strings.slice(fo.type_name)) orelse return fail(diag_out, error.UndeclaredType, "instance field override references an undeclared component type"); + const idx = indexOfId(ids.items, id) orelse return fail(diag_out, error.OverrideTargetMissing, "per-field override targets a component the instance does not carry"); + const fname = self.ast.strings.slice(fo.field); + const fd = self.registry.findField(id, fname) orelse return fail(diag_out, error.UnknownField, "per-field override sets a field the component does not declare"); + try self.encodeScalar(blobs.items[idx], fd, fo.value, diag_out); + }, + }; + + const comp_ids = try self.a().dupe(ComponentId, ids.items); + const comp_blobs = try self.a().dupe([]u8, blobs.items); + sortIdsBlobs(comp_ids, comp_blobs); + + const name_idx = try self.internString(self.ast.strings.slice(inst.instance_name)); + const uuid_idx = try self.internUuid(try self.parseEntityUuid(inst.uuid, diag_out)); + try self.name_to_uuid_idx.put(self.gpa, self.strings.items[name_idx], uuid_idx); + + return .{ + .name_idx = name_idx, + .uuid_idx = uuid_idx, + .parent_name = "", + .comp_ids = comp_ids, + .comp_blobs = comp_blobs, + }; + } + + /// Extract a single-entity prefab's components from its `.prefab.bin`: each + /// on-disk schema name → the scene registry's `ComponentId` (`idOf`, size + /// validated), with the column bytes copied. Errors + /// `MultiEntityInstanceUnsupported` if the prefab holds ≠ 1 entity. Appends + /// into `ids`/`blobs` (unsorted; the caller sorts after applying overrides). + fn flattenedPrefabComponents(self: *Builder, acc: accessor.Accessor, ids: *std.ArrayListUnmanaged(ComponentId), blobs: *std.ArrayListUnmanaged([]u8), diag_out: ?*[]const u8) CookError!void { + var total: u32 = 0; + var ai: u32 = 0; + while (ai < acc.archetypeCount()) : (ai += 1) total += acc.archetype(ai).entity_count; + if (total != 1) return fail(diag_out, error.MultiEntityInstanceUnsupported, "instancing a multi-entity prefab is not supported in M1.0.6 (single-entity only)"); + + ai = 0; + while (ai < acc.archetypeCount()) : (ai += 1) { + const arch = acc.archetype(ai); + if (arch.entity_count == 0) continue; + var c: usize = 0; + while (c < arch.component_count) : (c += 1) { + const sch = acc.schema(arch.schemaIndex(c)); + const id = self.registry.idOf(sch.name) orelse return fail(diag_out, error.BaseSchemaMismatch, "instanced prefab uses a component the scene does not declare"); + if (self.registry.componentSize(id) != sch.size) return fail(diag_out, error.BaseSchemaMismatch, "instanced prefab component size disagrees with the scene registry layout"); + try ids.append(self.gpa, id); + try blobs.append(self.gpa, try self.a().dupe(u8, arch.componentSlot(c, 0))); + } + } + } + /// Build one component blob (`componentSize` bytes) from the type defaults /// overridden by the instance's fields. Components are POD-strict, so every /// field is a scalar — no `string_`/`enum_` slots to special-case here. diff --git a/tests/scene/cook_errors_test.zig b/tests/scene/cook_errors_test.zig index 6816910..1ebc96b 100644 --- a/tests/scene/cook_errors_test.zig +++ b/tests/scene/cook_errors_test.zig @@ -15,8 +15,13 @@ fn expectCookError(comptime want: anyerror, src: []const u8) !void { try std.testing.expect(msg.len > 0); } -test "instance of is rejected (M1.0.6 boundary)" { - try expectCookError(error.InstanceOfUnsupported, +test "instance of without a prefab resolver errors BasePrefabMissing" { + // M1.0.6 E3 replaced the M1.0.4 `InstanceOfUnsupported` boundary with real + // flattening: `cook` (the resolver-less wrapper) can no longer locate the + // referenced prefab, so an instance now errors `BasePrefabMissing` rather than + // a blanket "unsupported". Flattening with a resolver is covered in + // `tests/scene/prefab_flatten_test.zig`. + try expectCookError(error.BasePrefabMissing, \\scene "S" { \\ instance of "Torch" "T1" { } \\} diff --git a/tests/scene/prefab_flatten_test.zig b/tests/scene/prefab_flatten_test.zig new file mode 100644 index 0000000..18cb304 --- /dev/null +++ b/tests/scene/prefab_flatten_test.zig @@ -0,0 +1,236 @@ +//! M1.0.6 E3 — `instance of` flattening at scene cook. A prefab is cooked to its +//! `.prefab.bin`, then a scene that instances it is cooked with a resolver that +//! hands back those bytes; the instance's entity inherits the prefab's components +//! and applies the instance's overrides (both forms). Covers: an override-free +//! instance equals the hand-authored equivalent (same archetype + bytes), both +//! override forms (`Comp.field = v` and `Comp { field: v }`), N instances loading +//! into the ECS through the M1.0.5 loader, and the single-entity boundary. +//! +//! Components are POD scalar (the cook's only component kind), so fixtures use f32. + +const std = @import("std"); +const weld_core = @import("weld_core"); +const weld_etch = @import("weld_etch"); + +const scene = weld_core.scene; +const ecs = weld_core.ecs; +const World = ecs.World; +const Registry = weld_core.ecs.registry.Registry; +const scene_cook = weld_etch.scene_cook; +const Accessor = scene.accessor.Accessor; + +/// One-entry in-process base-prefab resolver. +const OneResolver = struct { + name: []const u8, + bytes: []const u8, + fn resolve(ctx: *anyopaque, name: []const u8) ?[]const u8 { + const self: *OneResolver = @ptrCast(@alignCast(ctx)); + return if (std.mem.eql(u8, name, self.name)) self.bytes else null; + } + fn base(self: *OneResolver) scene_cook.BaseResolver { + return .{ .ctx = self, .resolveFn = OneResolver.resolve }; + } +}; + +// `Torch`: a mono-entity prefab — Transform{x:1} + Light{intensity:1500, radius:6}. +const torch_prefab = + \\component Transform { x: f32 = 0.0, y: f32 = 0.0, z: f32 = 0.0 } + \\component Light { intensity: f32 = 2000.0, radius: f32 = 8.0 } + \\prefab "Torch" { + \\ entity "root" { + \\ uuid: "7b3e2f1a-42a3-4f2b-8c9d-a3f2b1c98d4e" + \\ Transform { x: 1.0 } + \\ Light { intensity: 1500.0, radius: 6.0 } + \\ } + \\} +; + +const scene_decls = + \\component Transform { x: f32 = 0.0, y: f32 = 0.0, z: f32 = 0.0 } + \\component Light { intensity: f32 = 2000.0, radius: f32 = 8.0 } + \\ +; + +fn cookTorch(gpa: std.mem.Allocator) !scene_cook.Cooked { + return scene_cook.cookPrefab(gpa, torch_prefab, null, null); +} + +fn columnOf(acc: Accessor, arch: Accessor.Archetype, comp: []const u8) ?usize { + var c: usize = 0; + while (c < arch.component_count) : (c += 1) { + if (std.mem.eql(u8, acc.schema(arch.schemaIndex(c)).name, comp)) return c; + } + return null; +} + +fn slotOf(arch: Accessor.Archetype, name: []const u8) ?usize { + var s: usize = 0; + while (s < arch.entity_count) : (s += 1) { + if (std.mem.eql(u8, arch.entityName(s), name)) return s; + } + return null; +} + +fn decodeF32(acc: Accessor, reg: *const Registry, arch: Accessor.Archetype, comp: []const u8, field: []const u8, slot: usize) f32 { + const c = columnOf(acc, arch, comp).?; + const fd = reg.findField(reg.idOf(comp).?, field).?; + return @bitCast(std.mem.readInt(u32, arch.componentSlot(c, slot)[fd.offset..][0..4], .little)); +} + +test "instance of expands to the prefab's components (same archetype + bytes as hand-authored)" { + const gpa = std.testing.allocator; + var torch = try cookTorch(gpa); + defer torch.deinit(gpa); + const torch_bytes = try scene.writer.write(gpa, torch.model, &torch.registry); + defer gpa.free(torch_bytes); + var resolver = OneResolver{ .name = "Torch", .bytes = torch_bytes }; + + // An override-free instance, beside the hand-authored entity carrying the + // prefab's exact components/values. + const src = scene_decls ++ + \\scene "S" { + \\ instance of "Torch" "I" { uuid: "00000000-0000-0000-0000-0000000000a1" } + \\ entity "H" { + \\ uuid: "00000000-0000-0000-0000-0000000000a2" + \\ Transform { x: 1.0 } + \\ Light { intensity: 1500.0, radius: 6.0 } + \\ } + \\} + ; + var cooked = try scene_cook.cookScene(gpa, src, resolver.base(), null); + defer cooked.deinit(gpa); + const bytes = try scene.writer.write(gpa, cooked.model, &cooked.registry); + defer gpa.free(bytes); + + var acc = try Accessor.open(bytes); + try std.testing.expect(acc.verifyHash()); + try std.testing.expectEqual(@as(u32, 1), acc.archetypeCount()); // I and H share [Light, Transform] + const arch = acc.archetype(0); + try std.testing.expectEqual(@as(u32, 2), arch.entity_count); + try std.testing.expectEqual(@as(u32, 2), arch.component_count); + + // Per column, the instance slot and the hand-authored slot are byte-identical. + const si = slotOf(arch, "I").?; + const sh = slotOf(arch, "H").?; + var c: usize = 0; + while (c < arch.component_count) : (c += 1) { + try std.testing.expectEqualSlices(u8, arch.componentSlot(c, sh), arch.componentSlot(c, si)); + } +} + +test "per-field overrides apply over the prefab (both forms)" { + const gpa = std.testing.allocator; + var torch = try cookTorch(gpa); + defer torch.deinit(gpa); + const torch_bytes = try scene.writer.write(gpa, torch.model, &torch.registry); + defer gpa.free(torch_bytes); + var resolver = OneResolver{ .name = "Torch", .bytes = torch_bytes }; + + const src = scene_decls ++ + \\scene "S" { + \\ instance of "Torch" "I1" { uuid: "00000000-0000-0000-0000-0000000000b1" Light.intensity = 3000.0 } + \\ instance of "Torch" "I2" { uuid: "00000000-0000-0000-0000-0000000000b2" Light { intensity: 2500.0 } } + \\} + ; + var cooked = try scene_cook.cookScene(gpa, src, resolver.base(), null); + defer cooked.deinit(gpa); + const bytes = try scene.writer.write(gpa, cooked.model, &cooked.registry); + defer gpa.free(bytes); + + var acc = try Accessor.open(bytes); + try std.testing.expect(acc.verifyHash()); + const arch = acc.archetype(0); + const reg = &cooked.registry; + + // `Comp.field = v` form (I1): intensity overridden, radius + transform inherited. + const s1 = slotOf(arch, "I1").?; + try std.testing.expectApproxEqAbs(@as(f32, 3000.0), decodeF32(acc, reg, arch, "Light", "intensity", s1), 1e-3); + try std.testing.expectApproxEqAbs(@as(f32, 6.0), decodeF32(acc, reg, arch, "Light", "radius", s1), 1e-3); + try std.testing.expectApproxEqAbs(@as(f32, 1.0), decodeF32(acc, reg, arch, "Transform", "x", s1), 1e-3); + + // `Comp { field: v }` form (I2): field-merge — intensity overridden, radius inherited. + const s2 = slotOf(arch, "I2").?; + try std.testing.expectApproxEqAbs(@as(f32, 2500.0), decodeF32(acc, reg, arch, "Light", "intensity", s2), 1e-3); + try std.testing.expectApproxEqAbs(@as(f32, 6.0), decodeF32(acc, reg, arch, "Light", "radius", s2), 1e-3); +} + +test "N instances of one prefab load as N entities (cook -> load -> ECS)" { + const gpa = std.testing.allocator; + var torch = try cookTorch(gpa); + defer torch.deinit(gpa); + const torch_bytes = try scene.writer.write(gpa, torch.model, &torch.registry); + defer gpa.free(torch_bytes); + var resolver = OneResolver{ .name = "Torch", .bytes = torch_bytes }; + + const src = scene_decls ++ + \\scene "S" { + \\ instance of "Torch" "I0" { uuid: "00000000-0000-0000-0000-0000000000c0" } + \\ instance of "Torch" "I1" { uuid: "00000000-0000-0000-0000-0000000000c1" Light.intensity = 4000.0 } + \\ instance of "Torch" "I2" { uuid: "00000000-0000-0000-0000-0000000000c2" } + \\} + ; + var cooked = try scene_cook.cookScene(gpa, src, resolver.base(), null); + defer cooked.deinit(gpa); + const bytes = try scene.writer.write(gpa, cooked.model, &cooked.registry); + defer gpa.free(bytes); + + // Load into a World whose registry mirrors the cook's component layout. + var world = World.init(); + defer world.deinit(gpa); + _ = try world.registry.registerComponentRaw(gpa, .{ .name = "Transform", .size = 12, .alignment = 4, .default_bytes = &[_]u8{0} ** 12, .fields = &.{} }); + _ = try world.registry.registerComponentRaw(gpa, .{ .name = "Light", .size = 8, .alignment = 4, .default_bytes = &[_]u8{0} ** 8, .fields = &.{} }); + + var result = try scene.loader.loadFromBytes(&world, gpa, bytes); + defer result.deinit(gpa); + + try std.testing.expectEqual(@as(usize, 3), world.entityCount()); + const light_id = world.componentId("Light").?; + + const Case = struct { uuid: [16]u8, intensity: f32 }; + const cases = [_]Case{ + .{ .uuid = uuidBytes(0xc0), .intensity = 1500.0 }, + .{ .uuid = uuidBytes(0xc1), .intensity = 4000.0 }, + .{ .uuid = uuidBytes(0xc2), .intensity = 1500.0 }, + }; + for (cases) |cs| { + const eid = result.uuid_to_entity.get(cs.uuid).?; + const lb = world.componentBytes(eid, light_id).?; + try std.testing.expectApproxEqAbs(cs.intensity, @as(f32, @bitCast(std.mem.readInt(u32, lb[0..4], .little))), 1e-3); // intensity @0 + try std.testing.expectApproxEqAbs(@as(f32, 6.0), @as(f32, @bitCast(std.mem.readInt(u32, lb[4..8], .little))), 1e-3); // radius @4 (inherited) + } +} + +/// A canonical UUID `00000000-0000-0000-0000-0000000000XX` as its 16 bytes (the +/// last byte = `last`), matching the `uuid:` strings above. +fn uuidBytes(last: u8) [16]u8 { + var u = [_]u8{0} ** 16; + u[15] = last; + return u; +} + +test "instancing a multi-entity prefab is rejected" { + const gpa = std.testing.allocator; + const multi_prefab = + \\component Light { intensity: f32 = 2000.0 } + \\component Transform { x: f32 = 0.0 } + \\prefab "Multi" { + \\ entity "root" { uuid: "00000000-0000-0000-0000-0000000000d0" Light { } } + \\ entity "child" { uuid: "00000000-0000-0000-0000-0000000000d1" Transform { } } + \\} + ; + var multi = try scene_cook.cookPrefab(gpa, multi_prefab, null, null); + defer multi.deinit(gpa); + const multi_bytes = try scene.writer.write(gpa, multi.model, &multi.registry); + defer gpa.free(multi_bytes); + var resolver = OneResolver{ .name = "Multi", .bytes = multi_bytes }; + + const src = + \\component Light { intensity: f32 = 2000.0 } + \\component Transform { x: f32 = 0.0 } + \\scene "S" { + \\ instance of "Multi" "I" { uuid: "00000000-0000-0000-0000-0000000000d2" } + \\} + ; + var diag: []const u8 = ""; + try std.testing.expectError(error.MultiEntityInstanceUnsupported, scene_cook.cookScene(gpa, src, resolver.base(), &diag)); +} diff --git a/tools/scene_cook/main.zig b/tools/scene_cook/main.zig index 20ec465..0eb2e6d 100644 --- a/tools/scene_cook/main.zig +++ b/tools/scene_cook/main.zig @@ -80,22 +80,26 @@ pub fn main(init: std.process.Init) !void { const is_prefab = std.mem.endsWith(u8, in_path, ".prefab.etch"); + // The base-prefab resolver serves both `of` variants (prefab cook) and + // `instance of` flattening (scene cook); a `--prefab-dir` root is required for + // either to resolve a referenced prefab. + var resolver = DirResolver{ + .io = io, + .dir = dir, + .prefab_dir = prefab_dir, + .arena = init.arena.allocator(), + .gpa = gpa, + }; + var diag: []const u8 = ""; var cooked = blk: { if (is_prefab) { - var resolver = DirResolver{ - .io = io, - .dir = dir, - .prefab_dir = prefab_dir, - .arena = init.arena.allocator(), - .gpa = gpa, - }; break :blk scene_cook.cookPrefab(gpa, source, resolver.base(), &diag) catch |err| { try printErr(io, "prefab cook failed: ", if (diag.len > 0) diag else @errorName(err)); return err; }; } else { - break :blk scene_cook.cook(gpa, source, &diag) catch |err| { + break :blk scene_cook.cookScene(gpa, source, resolver.base(), &diag) catch |err| { try printErr(io, "cook failed: ", if (diag.len > 0) diag else @errorName(err)); return err; }; From 5f38bc37021b943766ca005d9dcc11660f10574b Mon Sep 17 00:00:00 2001 From: Guy Senpai Date: Sun, 28 Jun 2026 13:06:34 +0200 Subject: [PATCH 06/15] feat(scene): resolve entity cross-references at scene load --- briefs/M1.0.6-prefabs-crossrefs-extensions.md | 3 + build.zig | 4 + src/core/ecs/registry.zig | 17 +++ src/core/scene/accessor.zig | 14 ++ src/core/scene/format.zig | 52 +++++++ src/core/scene/loader.zig | 39 +++++- src/core/scene/writer.zig | 21 ++- src/etch/ecs_bridge.zig | 20 +++ src/etch/interp.zig | 16 +++ src/etch/scene_cook.zig | 129 ++++++++++++++--- tests/etch_interp/diff_runner.zig | 12 +- tests/scene/crossref_test.zig | 130 ++++++++++++++++++ 12 files changed, 427 insertions(+), 30 deletions(-) create mode 100644 tests/scene/crossref_test.zig diff --git a/briefs/M1.0.6-prefabs-crossrefs-extensions.md b/briefs/M1.0.6-prefabs-crossrefs-extensions.md index 411d34f..3244b56 100644 --- a/briefs/M1.0.6-prefabs-crossrefs-extensions.md +++ b/briefs/M1.0.6-prefabs-crossrefs-extensions.md @@ -240,6 +240,9 @@ Sparse (entities with no active extension are absent). The `§4` note's `u8` cou - 2026-06-28 11:30 — E3 implemented (`instance of` flattening at scene cook). `scene_cook.zig`: `cook` is now a resolver-less wrapper over `cookScene(gpa, source, base_resolver, diag_out)`; `build` takes `base_resolver` and dispatches scene children to `buildEntity` (direct) or `buildInstanceEntity` (instance). `buildInstanceEntity` resolves `P.prefab.bin` via the resolver + accessor (`BasePrefabMissing`/`BasePrefabCorrupt`), inherits P's single entity's components (`flattenedPrefabComponents`, size-validated), then applies the instance body's overrides **in declaration order** — both grammar forms: `Comp { field: value }` → field-merge (reuses `mergeComponentBlob`), `Comp.field = value` → per-field write (`encodeScalar`), a component P lacks → add (`buildComponentBlob`). The instance supplies the entity's name + uuid (prefab template uuid discarded); instances are roots. Removed `InstanceOfUnsupported` (the M1.0.4 boundary); added `MultiEntityInstanceUnsupported` + `OverrideTargetMissing`. `tools/scene_cook/main.zig`: the scene branch now calls `cookScene` with the `--prefab-dir` resolver (scenes flatten instances too). `tests/scene/cook_errors_test.zig`: the old `InstanceOfUnsupported` case → `BasePrefabMissing` (resolver-less `cook` + instance). New `tests/scene/prefab_flatten_test.zig` (4 tests, step `test-prefab-flatten`): override-free instance == hand-authored (same archetype + byte-identical slots), both override forms, 3 instances cook→load→ECS via the M1.0.5 loader, multi-entity rejection. Green Debug + ReleaseSafe (10/10 prefab tests); full `zig build test` 832/849 pass (17 skip, 0 fail, exit 0); `zig fmt --check` + `zig build lint` clean. - 2026-06-28 11:30 — **E3 multi-entity decision (per the E3 cadrage point #2).** `engine-scene-serialization.md` §2/§5 define **no** uuid-remapping scheme for a multi-entity prefab's internal entities at instantiation (the doc's "remapping UUID" refs are the M1.0.5 loader's runtime UUID→handle remap; "UUID v4 auto-generated" is editor authoring of unnamed entities) — verified by grep of §2/§5. Per the GO instruction's "if the spec does NOT define it" branch: M1.0.6 instantiates **single-entity prefabs only**, with an explicit typed `error.MultiEntityInstanceUnsupported`; multi-entity instantiation (and its hierarchy) is deferred to the dedicated hierarchy milestone (consistent with D-D). No Claude.ai round-trip (the spec-undefined branch, not the spec-defined one). - 2026-06-28 11:30 — E3 STOP: pushed, awaiting Guy's review + GO before E4. +- 2026-06-28 12:45 — E4 implemented (entity cross-references). **registry.zig**: `FieldKind.entity_` (8 B / 8-align, `EntityId` packed u64) + `sizeBytes`/`alignBytes`/`fromZigType` (imports `entity.zig`, acyclic). **interp.zig**: `fieldKindFromTypeName` emits `.entity_` for `Entity` when `reg_kind == .component` (mirror of the `string` gate, opposite origin); `compileTypeDecl` defaults an `.entity_` slot to `0xFF` (`EntityId.dead`, not the zeroed `{0,0}` live handle). **ecs_bridge.zig**: `.entity_` cases in `readBytesAsValue`/`writeValueAsBytes` (`value.zig` `EntityId` = u64, bit pattern of core `EntityId`) — the interp runtime read/write path; the cook never uses it. **format.zig**: `CrossRefEntry` (extern 16 B { source_uuid_ordinal, schema_index, field_offset, target_uuid_ordinal } + `readAt`); neutral-model `CrossRef` (carries `component_id`, not schema index) + `CookModel.cross_refs = &.{}` default. **writer.zig**: emit the Cross-references Table @ `crossrefs_offset` (count + entries), converting `component_id` → file-local schema index (`id_to_index`). **accessor.zig**: `crossrefsCount`/`crossref(i)`. **scene_cook.zig**: two-phase crossref pass — build phase records a `CrossRefPending { source_uuid_idx, component_id, field_offset, target_name }` for each set `.entity_` field (slot left `dead`, routed OUT of `encodeScalar` in `buildComponentBlob`/`mergeComponentBlob`/the instance `field_override`); post-build `resolveCrossRefs` resolves `target_name` → uuid ordinal against the complete `name_to_uuid_idx` (forward refs ok), `UnresolvedCrossRef` for an absent target; gated by `collect_crossrefs` (scene only — prefab cook leaves Entity slots `dead`, no entry). Reference is BY NAME (D-B), no `uuid "…"` form. **loader.zig**: `resolveCrossRefs` pass after `instantiate`, before `loadResources` — patches each cooked slot to the target's runtime handle (`@memcpy` + `markComponentChangedDyn`), bounds/identity validated (`MalformedScene`); the `crossrefs_offset==0` assert removed (extensions assert stays — E6). Tests `tests/scene/crossref_test.zig` (3, step `test-crossref`): cook→dead+entry / forward ref / unset=dead / `UnresolvedCrossRef` / cook→load→ECS resolved handle. Green Debug + ReleaseSafe; full `zig build test` 835/852 (17 skip, 0 fail, exit 0); fmt + lint clean. +- 2026-06-28 12:45 — **Out-of-list edit (justified):** `tests/etch_interp/diff_runner.zig` (not in the brief's file list) — its two exhaustive `FieldKind` switches (`writeFieldValue`/`readFieldValue`) gained a `.entity_ => unreachable` arm. Adding a `FieldKind` variant mechanically requires updating every exhaustive switch over it; the S4 differential corpus is POD-only and never carries an `Entity` component field, so `unreachable` is the correct (proven) arm. No behavior change. +- 2026-06-28 12:45 — E4 STOP — **session 1 / session 2 boundary.** Pushed, awaiting Guy's review + GO. Per the gate plan, the next message relays the KB amendment lot (grammar `extensions:` + the scene-serialization §5 / §30.4 doc drifts) and the FROZEN brief amendment for E5 before session 2 starts. ## Accepted deviations diff --git a/build.zig b/build.zig index f56f481..6846456 100644 --- a/build.zig +++ b/build.zig @@ -438,6 +438,10 @@ pub fn build(b: *std.Build) void { // instance == hand-authored equivalent (same archetype + bytes), both // override forms, N instances cook→load→ECS, single-entity boundary. .{ .path = "tests/scene/prefab_flatten_test.zig", .scene = true, .dedicated_step = "test-prefab-flatten" }, + // M1.0.6 / E4 — entity→entity cross-references: cook writes dead + a + // side-table entry (by-name, two-phase), loader patches the slot to the + // target handle; unset = dead; absent target = UnresolvedCrossRef at cook. + .{ .path = "tests/scene/crossref_test.zig", .scene = true, .dedicated_step = "test-crossref" }, // M1.0.5 / E2 — runtime loader `.scene.bin` → ECS: instantiate every // entity (component bytes verbatim), two-phase `on_spawned` (fires once // per entity, after all entities exist). `weld_core` only (no `.scene` diff --git a/src/core/ecs/registry.zig b/src/core/ecs/registry.zig index 270a671..e1c180a 100644 --- a/src/core/ecs/registry.zig +++ b/src/core/ecs/registry.zig @@ -21,6 +21,11 @@ const std = @import("std"); +/// `EntityId` (`packed struct(u64)`) — the storage type of a `.entity_` field +/// (M1.0.6 E4). Imported only for `FieldKind.fromZigType`; `entity.zig` imports +/// nothing of `registry.zig`, so this is acyclic. +const EntityId = @import("entity.zig").EntityId; + /// Stable identifier assigned at registration. The first registered /// component gets `ComponentId(0)`; subsequent registrations get the next /// integer. Stability across runs is *not* guaranteed (it would require an @@ -55,6 +60,15 @@ pub const FieldKind = enum { /// `FieldDesc.enum_type_name_id` so the Etch bridge can rebuild a typed /// `enum_value{ type_name, variant }` on read. enum_, + /// An `Entity` field slot: an `EntityId` (`packed struct(u64)`, 8 bytes, + /// 8-aligned). POD — no heap, no teardown — so the component SoA/POD invariant + /// (`engine-spec.md` §4) is untouched. **Component-only by construction** + /// (M1.0.6 D-A): the exact mirror of `.string_`/`.enum_` (resource-only) — + /// `fieldKindFromTypeName` emits `.entity_` only for the `.component` origin. + /// An unassigned / dangling slot holds `EntityId.dead` (all-ones); at scene + /// cook the slot is written `dead` and an entity→entity reference is carried by + /// the Cross-references Table, resolved to the target's handle at load. + entity_, pub fn sizeBytes(self: FieldKind) usize { return switch (self) { @@ -69,6 +83,7 @@ pub const FieldKind = enum { // `@sizeOf(persistent.StringSlot)` (asserted in `ecs_bridge.zig`). .string_ => 16, .enum_ => @sizeOf(u32), // declaration-order discriminant + .entity_ => @sizeOf(EntityId), // 8 (packed u64) }; } @@ -83,6 +98,7 @@ pub const FieldKind = enum { .f64_ => @alignOf(f64), .string_ => 8, .enum_ => @alignOf(u32), + .entity_ => @alignOf(EntityId), // 8 }; } @@ -94,6 +110,7 @@ pub const FieldKind = enum { i32 => .i32_, u32 => .u32_, f32 => .f32_, + EntityId => .entity_, else => @compileError("unsupported Zig type for FieldKind: " ++ @typeName(T)), }; } diff --git a/src/core/scene/accessor.zig b/src/core/scene/accessor.zig index b4c0a19..497591d 100644 --- a/src/core/scene/accessor.zig +++ b/src/core/scene/accessor.zig @@ -134,6 +134,20 @@ pub const Accessor = struct { } }; + // ── Cross-references Table (M1.0.6 E4) ── + + /// Number of entity→entity cross-reference entries (`0` for a scene with no + /// `Entity` field references, and for every M1.0.4/M1.0.5 file). + pub fn crossrefsCount(self: Accessor) u32 { + return self.readU32(self.header.crossrefs_offset); + } + + /// The `i`-th `CrossRefEntry` (16 B each, after the `u32` count prefix). + pub fn crossref(self: Accessor, i: u32) format.CrossRefEntry { + const off = self.header.crossrefs_offset + 4 + @as(usize, i) * 16; + return format.CrossRefEntry.readAt(self.bytes, off); + } + // ── Archetypes ── pub fn archetypeCount(self: Accessor) u32 { diff --git a/src/core/scene/format.zig b/src/core/scene/format.zig index a1ab2a6..8d23d85 100644 --- a/src/core/scene/format.zig +++ b/src/core/scene/format.zig @@ -174,6 +174,39 @@ pub const SchemaEntry = extern struct { alignment: u16, }; +/// On-disk Cross-references Table entry (M1.0.6 D-B) — one per entity→entity +/// `Entity` component field that references another entity of the same scene. +/// 16 bytes, 4-aligned. All four fields are file-local ordinals/indices (never +/// runtime ids): the loader resolves them against the UUID table + the schema +/// remap. The bearing field's 8-byte slot is written `EntityId.dead` in its SoA +/// column at cook; the loader patches it to the target's runtime handle. +pub const CrossRefEntry = extern struct { + /// UUID-table ordinal of the entity bearing the field (the reference source). + source_uuid_ordinal: u32, + /// File-local Schema Registry index of the bearing component. + schema_index: u32, + /// Byte offset of the `Entity` field within the component slot. + field_offset: u32, + /// UUID-table ordinal of the referenced (target) entity. + target_uuid_ordinal: u32, + + comptime { + std.debug.assert(@sizeOf(CrossRefEntry) == 16); + std.debug.assert(@alignOf(CrossRefEntry) == 4); + } + + /// Read a `CrossRefEntry` little-endian from `bytes` at `off` (unaligned-safe, + /// the codec discipline — never `@ptrCast`). The accessor's `crossref(i)`. + pub fn readAt(bytes: []const u8, off: usize) CrossRefEntry { + return .{ + .source_uuid_ordinal = std.mem.readInt(u32, bytes[off..][0..4], .little), + .schema_index = std.mem.readInt(u32, bytes[off + 4 ..][0..4], .little), + .field_offset = std.mem.readInt(u32, bytes[off + 8 ..][0..4], .little), + .target_uuid_ordinal = std.mem.readInt(u32, bytes[off + 12 ..][0..4], .little), + }; + } +}; + /// Byte offset (relative to the column region start `region_start`) of column /// `i` within an archetype block, given each column's `sizes`/`aligns` in column /// order and the block's `entity_count`. **Writer and accessor MUST call this** @@ -260,6 +293,21 @@ pub const ArchetypeBlock = struct { entities: []EntityEntry, }; +/// One entity→entity cross-reference in the neutral cook model (M1.0.6 E4). The +/// model carries it in terms of the cook's in-memory `component_id`; the **writer** +/// converts `component_id` → file-local Schema Registry index when emitting the +/// on-disk `CrossRefEntry` (the model never knows file-local schema indices). +pub const CrossRef = struct { + /// `CookModel.uuids` ordinal of the source entity (bears the `Entity` field). + source_uuid: u32, + /// The bearing component's in-memory `ComponentId` (writer maps → schema index). + component_id: ComponentId, + /// Byte offset of the `Entity` field within the component slot. + field_offset: u32, + /// `CookModel.uuids` ordinal of the referenced (target) entity. + target_uuid: u32, +}; + /// The neutral, World-free model the cook produces. Owns every slice via an /// internal arena; `deinit` frees the lot. The E2 writer reads it to emit /// `.scene.bin`; the E1 cook test inspects it directly (no serialization). @@ -270,6 +318,10 @@ pub const CookModel = struct { uuids: [][16]u8, resources: []ResourceEntry, archetypes: []ArchetypeBlock, + /// Entity→entity cross-references (M1.0.6 E4); empty for a scene with no + /// `Entity` field references and for every prefab. Serialized to the + /// Cross-references Table @ `crossrefs_offset`. + cross_refs: []const CrossRef = &.{}, /// The authored scene's `version:` field (0 if absent). Propagated to /// `SceneHeader.content_version` — opaque to the codec, for the game's own /// scene-versioning/migration. diff --git a/src/core/scene/loader.zig b/src/core/scene/loader.zig index db2de1e..0136bae 100644 --- a/src/core/scene/loader.zig +++ b/src/core/scene/loader.zig @@ -174,11 +174,10 @@ pub fn loadFromBytes(world: *World, gpa: std.mem.Allocator, bytes: []const u8) a const remap = try buildSchemaRemap(gpa, world, acc); defer gpa.free(remap); - // The cross-references + extensions tables are M1.0.6's; M1.0.4 writes them - // empty (a zero count prefix). Loading a file that populated them would mean - // a newer cook — assert the M1.0.5 contract. + // The extensions table is M1.0.6 E6's; until then a cook writes it empty (a + // zero count prefix). Loading a populated one would mean a newer cook — assert + // the contract. The cross-references table (M1.0.6 E4) IS read below. std.debug.assert(readU32At(acc, acc.header.extensions_offset) == 0); - std.debug.assert(readU32At(acc, acc.header.crossrefs_offset) == 0); var spawned: std.ArrayListUnmanaged(EntityId) = .empty; errdefer spawned.deinit(gpa); @@ -191,6 +190,9 @@ pub fn loadFromBytes(world: *World, gpa: std.mem.Allocator, bytes: []const u8) a } try instantiate(world, gpa, acc, remap, &spawned, &uuid_to_entity); + // Cross-references after every entity exists (a reference can point forward), + // before resources + on_spawned so a rule sees fully-linked entities. + try resolveCrossRefs(world, acc, remap, uuid_to_entity); // Resources before phase 2 so an `on_spawned` rule can read scene resources. try loadResources(world, gpa, acc, remap, &res_strings); try dispatchSpawnLifecycle(world, gpa, spawned.items); @@ -263,6 +265,35 @@ fn instantiate( } } +/// Resolve the Cross-references Table (M1.0.6 E4): patch each cooked `Entity` +/// field slot (written `EntityId.dead` at cook) to the referenced entity's +/// runtime handle. Per entry: map source/target UUID ordinals → handles via +/// `uuid_to_entity`, map the file-local schema index → runtime `ComponentId` via +/// `remap`, then overwrite the 8-byte field at `field_offset` and flag the +/// component changed. Bounds/identity are validated (`MalformedScene`) — the +/// loader treats the byte image as untrusted input. Runs after `instantiate` +/// (every entity exists) and before `loadResources`. +fn resolveCrossRefs(world: *World, acc: Accessor, remap: []const ComponentId, uuid_to_entity: UuidMap) !void { + const ucount = uuidCount(acc); + const count = acc.crossrefsCount(); + var i: u32 = 0; + while (i < count) : (i += 1) { + const e = acc.crossref(i); + if (e.source_uuid_ordinal >= ucount or e.target_uuid_ordinal >= ucount) return error.MalformedScene; + if (e.schema_index >= remap.len) return error.MalformedScene; + + const src = uuid_to_entity.get(acc.uuidAt(e.source_uuid_ordinal).*) orelse return error.MalformedScene; + const tgt = uuid_to_entity.get(acc.uuidAt(e.target_uuid_ordinal).*) orelse return error.MalformedScene; + const cid = remap[e.schema_index]; + + const slot = world.componentBytes(src, cid) orelse return error.MalformedScene; + const off = e.field_offset; + if (@as(usize, off) + @sizeOf(EntityId) > slot.len) return error.MalformedScene; + @memcpy(slot[off..][0..@sizeOf(EntityId)], std.mem.asBytes(&tgt)); + world.markComponentChangedDyn(src, cid); + } +} + /// Load the resources block (E3) — the load-side mirror of M1.0.3's non-POD /// resource path. For each resource: resolve its schema index → runtime /// `ComponentId` (the E1 remap, already size/alignment-validated), copy the POD diff --git a/src/core/scene/writer.zig b/src/core/scene/writer.zig index 27406d9..1d52077 100644 --- a/src/core/scene/writer.zig +++ b/src/core/scene/writer.zig @@ -87,9 +87,9 @@ const Writer = struct { hdr.archetypes_offset = try self.sectionOffset(); try self.writeArchetypes(); hdr.extensions_offset = try self.sectionOffset(); - try self.appendU32(0); // reserved — empty (M1.0.6) + try self.appendU32(0); // reserved — empty (M1.0.6 E5/E6) hdr.crossrefs_offset = try self.sectionOffset(); - try self.appendU32(0); // reserved — empty (M1.0.6) + try self.writeCrossRefs(); hdr.hash = std.hash.XxHash64.hash(0, self.body.items); @@ -142,6 +142,9 @@ const Writer = struct { for (arch.component_ids) |id| try self.noteSchema(id); } for (self.model.resources) |res| try self.noteSchema(res.schema_id); + // The bearing component of every cross-ref is already on an entity (hence + // in an archetype), but note it explicitly so `id_to_index` is total. + for (self.model.cross_refs) |cr| try self.noteSchema(cr.component_id); // Deterministic order: ascending ComponentId. std.mem.sort(ComponentId, self.schema_ids.items, {}, comptime std.sort.asc(ComponentId)); for (self.schema_ids.items, 0..) |id, idx| try self.id_to_index.put(self.gpa, id, @intCast(idx)); @@ -225,6 +228,20 @@ const Writer = struct { } } } + + /// Cross-references Table @ `crossrefs_offset`: `count: u32` then `count` + /// `CrossRefEntry` (16 B). The model carries `component_id`; here it is + /// converted to the file-local Schema Registry index (`id_to_index`) — the + /// on-disk entry never stores a runtime `ComponentId`. + fn writeCrossRefs(self: *Writer) WriteError!void { + try self.appendU32(try u32From(self.model.cross_refs.len)); + for (self.model.cross_refs) |cr| { + try self.appendU32(cr.source_uuid); + try self.appendU32(self.id_to_index.get(cr.component_id).?); + try self.appendU32(cr.field_offset); + try self.appendU32(cr.target_uuid); + } + } }; fn u32From(v: usize) WriteError!u32 { diff --git a/src/etch/ecs_bridge.zig b/src/etch/ecs_bridge.zig index bbf65c4..e0f9ac3 100644 --- a/src/etch/ecs_bridge.zig +++ b/src/etch/ecs_bridge.zig @@ -305,6 +305,15 @@ pub fn readBytesAsValue(kind: FieldKind, bytes: []const u8) Value { // delegating here, and components never carry `.enum_` (validator-gated). // Proven invariant: this arm is never reached. .enum_ => unreachable, + // Entity field (M1.0.6 E4): decode the 8-byte `EntityId` (`value.zig`'s + // `EntityId` is a `u64` that shares the bit pattern of core `EntityId`, + // packed `struct(u64)`; `invalid_entity`/`dead` == all-ones). The runtime + // interp read path returns it as `Value.entity_id`. + .entity_ => blk: { + var v: value_mod.EntityId = 0; + @memcpy(std.mem.asBytes(&v), bytes[0..@sizeOf(value_mod.EntityId)]); + break :blk .{ .entity_id = v }; + }, }; } @@ -377,6 +386,17 @@ pub fn writeValueAsBytes(kind: FieldKind, bytes: []u8, v: Value) BridgeError!voi }; @memcpy(bytes[0..@sizeOf(u32)], std.mem.asBytes(&disc)); }, + // Entity field (M1.0.6 E4): store the 8-byte `EntityId` (u64 bit pattern). + // The interp runtime write path (e.g. `entity.get_mut(Comp).ref = other`) + // routes here; the scene cook does NOT (it writes `dead` + a cross-ref + // side entry, never an immediate value). + .entity_ => { + const x: value_mod.EntityId = switch (v) { + .entity_id => |e| e, + else => return error.TypeMismatch, + }; + @memcpy(bytes[0..@sizeOf(value_mod.EntityId)], std.mem.asBytes(&x)); + }, } } diff --git a/src/etch/interp.zig b/src/etch/interp.zig index 77ed22e..da57788 100644 --- a/src/etch/interp.zig +++ b/src/etch/interp.zig @@ -3448,6 +3448,16 @@ pub fn compileTypeDecl( } continue; } + if (fd.kind == .entity_) { + // Unassigned `Entity` field = `EntityId.dead` (all-ones), NOT the + // zeroed slot (which would decode as a live handle `{index:0, gen:0}`). + // `Entity` fields take no literal default; an assignment in a scene is + // an entity-name reference resolved at cook (the cross-reference pass), + // never a value encoded here. Component-only (validator/gate), so this + // is reached only for components. + @memset(slot, 0xFF); + continue; + } if (f.default_value.isNone()) continue; const v = evalConst(ast, f.default_value) catch continue; try bridge_mod.writeValueAsBytes(fd.kind, slot, v); @@ -3497,6 +3507,12 @@ fn fieldKindFromTypeName(name: []const u8, reg_kind: RegKind) ?FieldKind { // field types are not builtin names, so they resolve in `compileTypeDecl` // against the AST enum slab (see `findEnumDecl`), not here. if (reg_kind == .resource and std.mem.eql(u8, name, "string")) return .string_; + // `Entity` is component-only (M1.0.6 D-A): the exact mirror of the `string` + // gate above, opposite origin. A `.entity_` field is a POD 8-byte slot; an + // unassigned/dangling value is `EntityId.dead`, and an entity→entity reference + // is resolved by the scene loader's cross-reference pass. Resource→entity refs + // are a future additive milestone, so this is gated out of resources. + if (reg_kind == .component and std.mem.eql(u8, name, "Entity")) return .entity_; return null; } diff --git a/src/etch/scene_cook.zig b/src/etch/scene_cook.zig index 49e64ce..9bdd215 100644 --- a/src/etch/scene_cook.zig +++ b/src/etch/scene_cook.zig @@ -118,6 +118,9 @@ pub const CookError = error{ /// instance does not carry (neither inherited from the prefab nor added by an /// earlier `Comp { … }` member of the same instance body). OverrideTargetMissing, + /// An `Entity` field references an entity name absent from the scene (M1.0.6 + /// E4 — intra-scene only; cross-scene references are a future milestone). + UnresolvedCrossRef, OutOfMemory, }; @@ -250,6 +253,19 @@ const EntityBuild = struct { comp_blobs: [][]u8, }; +/// An unresolved entity→entity reference recorded during the scene build phase +/// (M1.0.6 E4). The target is kept as a NAME (not yet resolved): a forward +/// reference may name an entity declared later, so `name → uuid` resolution waits +/// until `name_to_uuid_idx` is complete (the two-phase crossref pass). The source +/// entity's `uuid` ordinal is already known (its identity was interned before its +/// components were built). +const CrossRefPending = struct { + source_uuid_idx: u32, + component_id: ComponentId, + field_offset: u32, + target_name: []const u8, +}; + const Builder = struct { gpa: std.mem.Allocator, ast: *const AstArena, @@ -267,6 +283,13 @@ const Builder = struct { strings: std.ArrayListUnmanaged([]const u8) = .empty, uuids: std.ArrayListUnmanaged([16]u8) = .empty, + // Cross-references (M1.0.6 E4): collected only for a scene cook + // (`collect_crossrefs`), resolved name→uuid after all entities are built. + // A prefab cook leaves `collect_crossrefs` false — its Entity slots stay + // `dead` and emit no cross-ref entry. + collect_crossrefs: bool = false, + pendings: std.ArrayListUnmanaged(CrossRefPending) = .empty, + fn init(gpa: std.mem.Allocator, ast: *const AstArena, registry: *Registry) Builder { return .{ .gpa = gpa, @@ -289,6 +312,7 @@ const Builder = struct { self.name_to_uuid_idx.deinit(self.gpa); self.strings.deinit(self.gpa); self.uuids.deinit(self.gpa); + self.pendings.deinit(self.gpa); } fn a(self: *Builder) std.mem.Allocator { @@ -371,10 +395,14 @@ const Builder = struct { /// the prefab's components are inherited from `P.prefab.bin` (via the resolver) /// and the instance's overrides applied (M1.0.6 E3). fn build(self: *Builder, scene_decl: ast_mod.SceneDecl, base_resolver: ?BaseResolver, diag_out: ?*[]const u8) CookError!format.CookModel { + // Scene cook collects entity→entity cross-references (an `Entity` field's + // slot is left `dead` and a pending reference recorded); a prefab cook does + // not (it leaves `collect_crossrefs` false). + self.collect_crossrefs = true; const children = self.ast.scene_children.items[scene_decl.children_start .. scene_decl.children_start + scene_decl.children_len]; - // Per-entity build, accumulating the name→uuid-index map for parent - // resolution in a second pass. + // Per-entity build, accumulating the name→uuid-index map for parent + + // cross-reference resolution in a second pass. var entities: std.ArrayListUnmanaged(EntityBuild) = .empty; defer entities.deinit(self.gpa); for (children) |child| { @@ -396,6 +424,7 @@ const Builder = struct { const archetypes = try self.groupArchetypes(entities.items); const resources = try self.buildResources(scene_decl, diag_out); const content_version = try self.sceneContentVersion(scene_decl, diag_out); + const cross_refs = try self.resolveCrossRefs(diag_out); return .{ .strings = try self.a().dupe([]const u8, self.strings.items), @@ -403,10 +432,48 @@ const Builder = struct { .resources = resources, .archetypes = archetypes, .content_version = content_version, + .cross_refs = cross_refs, .arena = self.arena, }; } + /// Phase 2 of the crossref pass: resolve every pending `target_name` against + /// the now-complete `name_to_uuid_idx` (a reference can name an entity declared + /// later in the scene), producing the model's `CrossRef` slice. A target that + /// is not an entity of the scene → `error.UnresolvedCrossRef` (intra-scene + /// only; cross-scene references are a future milestone). + fn resolveCrossRefs(self: *Builder, diag_out: ?*[]const u8) CookError![]format.CrossRef { + const out = try self.a().alloc(format.CrossRef, self.pendings.items.len); + for (self.pendings.items, 0..) |p, i| { + const target = self.name_to_uuid_idx.get(p.target_name) orelse + return fail(diag_out, error.UnresolvedCrossRef, "Entity field references an entity name absent from the scene"); + out[i] = .{ + .source_uuid = p.source_uuid_idx, + .component_id = p.component_id, + .field_offset = p.field_offset, + .target_uuid = target, + }; + } + return out; + } + + /// Record a pending entity→entity reference for a `.entity_` field set to an + /// entity-name string literal (the D-B by-name form; there is no `uuid "…"` + /// form). The slot itself stays `EntityId.dead` — the side-table entry carries + /// the target, resolved at load. No-op for a prefab cook (`!collect_crossrefs`): + /// a prefab's Entity slots stay `dead` and emit no entry. + fn recordCrossRefPending(self: *Builder, source_uuid_idx: u32, component_id: ComponentId, field_offset: u16, value_node: NodeId, diag_out: ?*[]const u8) CookError!void { + if (!self.collect_crossrefs) return; + if (self.ast.exprKind(value_node) != .string_lit) + return fail(diag_out, error.TypeMismatch, "an Entity field value must be the target entity's name (a string literal)"); + try self.pendings.append(self.gpa, .{ + .source_uuid_idx = source_uuid_idx, + .component_id = component_id, + .field_offset = field_offset, + .target_name = self.ast.strings.slice(self.ast.exprData(value_node)), + }); + } + /// Build the neutral model from a `prefab` construct (the M1.0.6 E2 twin of /// `build`). Standalone prefabs cook their entities directly; an `of` variant /// inherits the base prefab's flattened components (read from its cooked @@ -559,10 +626,12 @@ const Builder = struct { const type_name = self.ast.strings.slice(ci.type_name); const id = self.registry.idOf(type_name) orelse return fail(diag_out, error.UndeclaredType, "variant entity references an undeclared component type"); if (indexOfId(ids.items, id)) |ci_idx| { - blobs.items[ci_idx] = try self.mergeComponentBlob(blobs.items[ci_idx], id, ci, diag_out); + // Prefab cook (`collect_crossrefs` false) → `source_uuid_idx` is + // unused (Entity slots stay `dead`, no pending recorded). + blobs.items[ci_idx] = try self.mergeComponentBlob(blobs.items[ci_idx], id, ci, eb.uuid_idx, diag_out); } else { try ids.append(self.gpa, id); - try blobs.append(self.gpa, try self.buildComponentBlob(id, ci, diag_out)); + try blobs.append(self.gpa, try self.buildComponentBlob(id, ci, eb.uuid_idx, diag_out)); } } @@ -577,7 +646,7 @@ const Builder = struct { /// variant override form is `Component { field: value }`; the /// `Component.field =` per-field form is scene-instance-only per the grammar). /// Components are POD scalar, so every field is a scalar encode. - fn mergeComponentBlob(self: *Builder, base_blob: []const u8, id: ComponentId, ci: ast_mod.ComponentInstance, diag_out: ?*[]const u8) CookError![]u8 { + fn mergeComponentBlob(self: *Builder, base_blob: []const u8, id: ComponentId, ci: ast_mod.ComponentInstance, source_uuid_idx: u32, diag_out: ?*[]const u8) CookError![]u8 { const size = self.registry.componentSize(id); const blob = try self.a().alloc(u8, size); @memcpy(blob, base_blob); @@ -586,7 +655,14 @@ const Builder = struct { if (slf.name == 0) return fail(diag_out, error.SpreadUnsupported, "`..spread` is not supported in a component instance"); const fname = self.ast.strings.slice(slf.name); const fd = self.registry.findField(id, fname) orelse return fail(diag_out, error.UnknownField, "variant component instance sets a field the component does not declare"); - try self.encodeScalar(blob, fd, slf.value, diag_out); + if (fd.kind == .entity_) { + // Base slot is already `dead` (copied); record the reference. The + // `Component.field =` per-field form is scene-instance-only, so a + // variant body reaches Entity fields only via this `Comp { … }` form. + try self.recordCrossRefPending(source_uuid_idx, id, fd.offset, slf.value, diag_out); + } else { + try self.encodeScalar(blob, fd, slf.value, diag_out); + } } return blob; } @@ -631,7 +707,7 @@ const Builder = struct { const type_name = self.ast.strings.slice(ci.type_name); const id = self.registry.idOf(type_name) orelse return fail(diag_out, error.UndeclaredType, "entity references an undeclared component type"); ids[k] = id; - blobs[k] = try self.buildComponentBlob(id, ci, diag_out); + blobs[k] = try self.buildComponentBlob(id, ci, uuid_idx, diag_out); } // Sort (ids, blobs) together by id ascending (insertion sort — component // counts per entity are tiny). `archetype.sortComponentIds` sorts ids @@ -656,12 +732,20 @@ const Builder = struct { /// entity's name + uuid (the prefab's template uuid is discarded); instances /// are roots (the grammar's `instance_decl` has no `parent:`). fn buildInstanceEntity(self: *Builder, inst: ast_mod.SceneInstance, base_resolver: ?BaseResolver, diag_out: ?*[]const u8) CookError!EntityBuild { + // Resolve + open the prefab first (structural availability before the + // instance's own field-level validity). const prefab_name = self.ast.strings.slice(inst.prefab_name); const resolver = base_resolver orelse return fail(diag_out, error.BasePrefabMissing, "scene `instance of` cooked without a prefab resolver (no --prefab-dir?)"); const base_bytes = resolver.resolve(prefab_name) orelse return fail(diag_out, error.BasePrefabMissing, "`instance of` references a prefab the resolver does not know"); var acc = accessor.Accessor.open(base_bytes) catch return fail(diag_out, error.BasePrefabCorrupt, "instanced prefab .prefab.bin failed to open (magic/version)"); if (!acc.verifyHash()) return fail(diag_out, error.BasePrefabCorrupt, "instanced prefab .prefab.bin content hash mismatch"); + // Identity next — cross-ref pendings recorded while applying the body need + // the source entity's uuid ordinal (the instance's, not the prefab's). + const name_idx = try self.internString(self.ast.strings.slice(inst.instance_name)); + const uuid_idx = try self.internUuid(try self.parseEntityUuid(inst.uuid, diag_out)); + try self.name_to_uuid_idx.put(self.gpa, self.strings.items[name_idx], uuid_idx); + // Inherit the prefab's single entity's components (variant-registry ids). var ids: std.ArrayListUnmanaged(ComponentId) = .empty; defer ids.deinit(self.gpa); @@ -676,10 +760,10 @@ const Builder = struct { const ci = self.ast.component_instances.items[m.index]; const id = self.registry.idOf(self.ast.strings.slice(ci.type_name)) orelse return fail(diag_out, error.UndeclaredType, "instance component references an undeclared component type"); if (indexOfId(ids.items, id)) |idx| { - blobs.items[idx] = try self.mergeComponentBlob(blobs.items[idx], id, ci, diag_out); + blobs.items[idx] = try self.mergeComponentBlob(blobs.items[idx], id, ci, uuid_idx, diag_out); } else { try ids.append(self.gpa, id); - try blobs.append(self.gpa, try self.buildComponentBlob(id, ci, diag_out)); + try blobs.append(self.gpa, try self.buildComponentBlob(id, ci, uuid_idx, diag_out)); } }, .field_override => { @@ -688,7 +772,13 @@ const Builder = struct { const idx = indexOfId(ids.items, id) orelse return fail(diag_out, error.OverrideTargetMissing, "per-field override targets a component the instance does not carry"); const fname = self.ast.strings.slice(fo.field); const fd = self.registry.findField(id, fname) orelse return fail(diag_out, error.UnknownField, "per-field override sets a field the component does not declare"); - try self.encodeScalar(blobs.items[idx], fd, fo.value, diag_out); + if (fd.kind == .entity_) { + // `Targeting.target = "Boss"` on an inherited Entity field → + // slot stays `dead`, record the reference (not `encodeScalar`). + try self.recordCrossRefPending(uuid_idx, id, fd.offset, fo.value, diag_out); + } else { + try self.encodeScalar(blobs.items[idx], fd, fo.value, diag_out); + } }, }; @@ -696,10 +786,6 @@ const Builder = struct { const comp_blobs = try self.a().dupe([]u8, blobs.items); sortIdsBlobs(comp_ids, comp_blobs); - const name_idx = try self.internString(self.ast.strings.slice(inst.instance_name)); - const uuid_idx = try self.internUuid(try self.parseEntityUuid(inst.uuid, diag_out)); - try self.name_to_uuid_idx.put(self.gpa, self.strings.items[name_idx], uuid_idx); - return .{ .name_idx = name_idx, .uuid_idx = uuid_idx, @@ -736,9 +822,12 @@ const Builder = struct { } /// Build one component blob (`componentSize` bytes) from the type defaults - /// overridden by the instance's fields. Components are POD-strict, so every - /// field is a scalar — no `string_`/`enum_` slots to special-case here. - fn buildComponentBlob(self: *Builder, id: ComponentId, ci: ast_mod.ComponentInstance, diag_out: ?*[]const u8) CookError![]u8 { + /// overridden by the instance's fields. Scalar fields encode in place; an + /// `.entity_` field is NOT encoded — its slot keeps the default `EntityId.dead` + /// and the reference is recorded as a pending cross-ref (`source_uuid_idx` = + /// the owning entity's uuid ordinal). `string_`/`enum_` are resource-only, so + /// components never reach those kinds here. + fn buildComponentBlob(self: *Builder, id: ComponentId, ci: ast_mod.ComponentInstance, source_uuid_idx: u32, diag_out: ?*[]const u8) CookError![]u8 { const size = self.registry.componentSize(id); const blob = try self.a().alloc(u8, size); @memcpy(blob, self.registry.componentDefaultBytes(id)); @@ -748,7 +837,11 @@ const Builder = struct { if (slf.name == 0) return fail(diag_out, error.SpreadUnsupported, "`..spread` is not supported in a component instance"); const fname = self.ast.strings.slice(slf.name); const fd = self.registry.findField(id, fname) orelse return fail(diag_out, error.UnknownField, "component instance sets a field the component does not declare"); - try self.encodeScalar(blob, fd, slf.value, diag_out); + if (fd.kind == .entity_) { + try self.recordCrossRefPending(source_uuid_idx, id, fd.offset, slf.value, diag_out); + } else { + try self.encodeScalar(blob, fd, slf.value, diag_out); + } } return blob; } diff --git a/tests/etch_interp/diff_runner.zig b/tests/etch_interp/diff_runner.zig index 102d376..2cd7fa4 100644 --- a/tests/etch_interp/diff_runner.zig +++ b/tests/etch_interp/diff_runner.zig @@ -285,9 +285,9 @@ fn writeFieldValue(kind: FieldKind, bytes: []u8, v: FieldValue) void { @memcpy(bytes[0..@sizeOf(f32)], std.mem.asBytes(&x)); }, // The S4 differential corpus is POD-only: `string`/enum resource fields - // (M1.0.3) are exercised by inline interpreter tests, never by this - // corpus, so these kinds never reach the runner — proven invariant. - .string_, .enum_ => unreachable, + // (M1.0.3) and `Entity` component fields (M1.0.6) are exercised elsewhere, + // never by this corpus, so these kinds never reach the runner. + .string_, .enum_, .entity_ => unreachable, } } @@ -319,8 +319,8 @@ fn readFieldValue(kind: FieldKind, bytes: []const u8) FieldValue { @memcpy(std.mem.asBytes(&v), bytes[0..@sizeOf(f32)]); break :blk .{ .float_ = v }; }, - // POD-only corpus — `.string_`/`.enum_` (M1.0.3) never enter it (see - // `writeFieldValue`). - .string_, .enum_ => unreachable, + // POD-only corpus — `.string_`/`.enum_` (M1.0.3) and `.entity_` (M1.0.6) + // never enter it (see `writeFieldValue`). + .string_, .enum_, .entity_ => unreachable, }; } diff --git a/tests/scene/crossref_test.zig b/tests/scene/crossref_test.zig new file mode 100644 index 0000000..283d2fa --- /dev/null +++ b/tests/scene/crossref_test.zig @@ -0,0 +1,130 @@ +//! M1.0.6 E4 — entity→entity cross-references. A component `Entity` field is +//! written `EntityId.dead` in its SoA column at cook and the reference carried in +//! the Cross-references Table (by target entity NAME, the D-B by-name form); +//! the loader patches the slot to the target's runtime handle. Covers: a forward +//! reference (target declared later — exercises the two-phase cook), an unset +//! field staying `dead`, a reference to an absent entity rejected at cook +//! (`UnresolvedCrossRef`), and the cook→load→ECS resolved-handle round-trip. + +const std = @import("std"); +const weld_core = @import("weld_core"); +const weld_etch = @import("weld_etch"); + +const scene = weld_core.scene; +const ecs = weld_core.ecs; +const World = ecs.World; +const EntityId = ecs.EntityId; +const Registry = weld_core.ecs.registry.Registry; +const scene_cook = weld_etch.scene_cook; +const Accessor = scene.accessor.Accessor; + +const dead_u64: u64 = std.math.maxInt(u64); // EntityId.dead bit pattern + +// A references B (declared AFTER A → forward reference, two-phase resolution); +// C leaves its Entity field unset (stays dead). +const src = + \\component Marker { v: i32 = 0 } + \\component Link { target: Entity } + \\scene "S" { + \\ entity "A" { uuid: "00000000-0000-0000-0000-0000000000a1" Link { target: "B" } } + \\ entity "B" { uuid: "00000000-0000-0000-0000-0000000000b2" Marker { v: 7 } } + \\ entity "C" { uuid: "00000000-0000-0000-0000-0000000000c3" Link { } } + \\} +; + +fn uuidBytes(last: u8) [16]u8 { + var u = [_]u8{0} ** 16; + u[15] = last; + return u; +} + +fn linkColumn(acc: Accessor, arch: Accessor.Archetype) ?usize { + var c: usize = 0; + while (c < arch.component_count) : (c += 1) { + if (std.mem.eql(u8, acc.schema(arch.schemaIndex(c)).name, "Link")) return c; + } + return null; +} + +fn slotOf(arch: Accessor.Archetype, name: []const u8) ?usize { + var s: usize = 0; + while (s < arch.entity_count) : (s += 1) { + if (std.mem.eql(u8, arch.entityName(s), name)) return s; + } + return null; +} + +test "Entity field cooks to a dead slot + a cross-ref entry; unset stays dead" { + const gpa = std.testing.allocator; + var cooked = try scene_cook.cook(gpa, src, null); + defer cooked.deinit(gpa); + const bytes = try scene.writer.write(gpa, cooked.model, &cooked.registry); + defer gpa.free(bytes); + + var acc = try Accessor.open(bytes); + try std.testing.expect(acc.verifyHash()); + + // Exactly one cross-ref (A.Link.target → B); C's unset field emits none. + try std.testing.expectEqual(@as(u32, 1), acc.crossrefsCount()); + const e = acc.crossref(0); + // source = A, target = B (by UUID first byte through the ordinal). + try std.testing.expectEqual(@as(u8, 0xa1), acc.uuidAt(e.source_uuid_ordinal)[15]); + try std.testing.expectEqual(@as(u8, 0xb2), acc.uuidAt(e.target_uuid_ordinal)[15]); + try std.testing.expectEqual(@as(u32, 0), e.field_offset); // target @0 in Link + try std.testing.expectEqualStrings("Link", acc.schema(e.schema_index).name); + + // On disk, BOTH A's and C's Link.target slots are dead (the side table carries + // the value; the column slot is a placeholder). + var ai: u32 = 0; + while (ai < acc.archetypeCount()) : (ai += 1) { + const arch = acc.archetype(ai); + const lc = linkColumn(acc, arch) orelse continue; + var s: usize = 0; + while (s < arch.entity_count) : (s += 1) { + const raw = std.mem.readInt(u64, arch.componentSlot(lc, s)[0..8], .little); + try std.testing.expectEqual(dead_u64, raw); + } + } +} + +test "cross-ref resolves to the target handle on load; unset reads dead" { + const gpa = std.testing.allocator; + var cooked = try scene_cook.cook(gpa, src, null); + defer cooked.deinit(gpa); + const bytes = try scene.writer.write(gpa, cooked.model, &cooked.registry); + defer gpa.free(bytes); + + var world = World.init(); + defer world.deinit(gpa); + _ = try world.registry.registerComponentRaw(gpa, .{ .name = "Marker", .size = 4, .alignment = 4, .default_bytes = &[_]u8{0} ** 4, .fields = &.{} }); + _ = try world.registry.registerComponentRaw(gpa, .{ .name = "Link", .size = 8, .alignment = 8, .default_bytes = &[_]u8{0xFF} ** 8, .fields = &.{} }); + + var result = try scene.loader.loadFromBytes(&world, gpa, bytes); + defer result.deinit(gpa); + + const link_id = world.componentId("Link").?; + const a = result.uuid_to_entity.get(uuidBytes(0xa1)).?; + const b = result.uuid_to_entity.get(uuidBytes(0xb2)).?; + const c = result.uuid_to_entity.get(uuidBytes(0xc3)).?; + + // A.Link.target resolved to B's runtime handle. + const a_target = std.mem.readInt(u64, world.componentBytes(a, link_id).?[0..8], .little); + try std.testing.expectEqual(@as(u64, @bitCast(b)), a_target); + + // C.Link.target was never assigned → still dead. + const c_target = std.mem.readInt(u64, world.componentBytes(c, link_id).?[0..8], .little); + try std.testing.expectEqual(dead_u64, c_target); +} + +test "cook rejects a reference to an absent entity" { + const gpa = std.testing.allocator; + const bad = + \\component Link { target: Entity } + \\scene "S" { + \\ entity "A" { uuid: "00000000-0000-0000-0000-0000000000a1" Link { target: "Ghost" } } + \\} + ; + var diag: []const u8 = ""; + try std.testing.expectError(error.UnresolvedCrossRef, scene_cook.cook(gpa, bad, &diag)); + try std.testing.expect(diag.len > 0); +} From a8821b517d059067f174629de17bc6443271ed8f Mon Sep 17 00:00:00 2001 From: Guy Senpai Date: Sun, 28 Jun 2026 13:30:39 +0200 Subject: [PATCH 07/15] docs(brief): amend E5 scope (extensions clause + D-E confirmed) --- briefs/M1.0.6-prefabs-crossrefs-extensions.md | 23 +++++++++++-------- 1 file changed, 14 insertions(+), 9 deletions(-) diff --git a/briefs/M1.0.6-prefabs-crossrefs-extensions.md b/briefs/M1.0.6-prefabs-crossrefs-extensions.md index 3244b56..d351544 100644 --- a/briefs/M1.0.6-prefabs-crossrefs-extensions.md +++ b/briefs/M1.0.6-prefabs-crossrefs-extensions.md @@ -68,7 +68,7 @@ Sparse (entities with no active extension are absent). The `§4` note's `u8` cou **D-D — Parent/child hierarchy: deferred.** No runtime hierarchy component in M1.0.6 (risk of the wrong abstraction vs `AttachedToSocket`, and out of testable). Parent ordinals stay validated-not-applied. Hierarchy is a later milestone that will build on D-A/D-B + a hierarchy component. -**D-E — `on_attach`/`on_detach` execution at load (tier boundary).** `loader.zig` (Tier 0) **must not call the Etch VM** (`src/etch`). Position: mirror the M1.0.2 `dispatchOnSpawned` mechanism — the loader materializes extension components (dynamic add-component, Tier-0, defaults copied from the extension's `.prefab.bin`), then fires `on_attach` through a **Tier-0 callback** (a function pointer registered by the Etch bridge, exactly as observer dispatch is wired). **Confirmed-or-blocker at E5 head:** read the `on_spawned` dispatch wiring + the `activate_extension` entry point in the Etch bridge before committing the callback design; if the seam is not as assumed, this is a Case-2 blocker (return to Claude.ai), not an improvised solution. +**D-E — `on_attach`/`on_detach` execution at load (tier boundary).** `loader.zig` (Tier 0) **must not call the Etch VM** (`src/etch`). Position: mirror the M1.0.2 `dispatchOnSpawned` mechanism — the loader materializes extension components (dynamic add-component, Tier-0, defaults copied from the extension's `.prefab.bin`), then fires `on_attach` through a **Tier-0 callback** (a function pointer registered by the Etch bridge, exactly as observer dispatch is wired). **CONFIRMED (Claude.ai amendment, session-2 head).** The seam exists and is not a blocker: `src/core/ecs/observers.zig` (`ObserverRegistry` + `registerOnSpawned(gpa, world, ctx, &trampoline)`, the Tier-0 callback pattern) + `src/etch/interp.zig:1018` (the Etch bridge registers its observers via `&observerTrampoline` + an opaque context). `on_attach`/`on_detach` are **already rendered as statement-runs at cook** (`src/etch/descriptor.zig:1668` `renderStmtRunAlloc`, `descriptor_types.zig:608`). E6 adds an `on_attach` dispatch analogous to `registerOnSpawned`/`dispatchOnSpawned`: the Etch bridge registers the `on_attach` hooks as Tier-0 callbacks; the loader's `applyExtensions` fires them after the dynamic add-component (never a direct VM call from `loader.zig`). A runtime `activate_extension` entry does not yet exist → E6 creates it, reused by the load path. ## Scope @@ -98,9 +98,11 @@ Sparse (entities with no active extension are absent). The `§4` note's `u8` cou - → **STOP** — push, await review + GO. **— session 1 / session 2 boundary —** **E5 — Extensions: extended design (D-E) + cook + `extends` grammar.** -- Resolve D-E first (read the `on_spawned` dispatch + `activate_extension` wiring; STOP as a Case-2 blocker if the seam differs from D-E's assumption). -- `scene_cook.zig`: cook `prefab "Z" extends "X" requires … { entity {…} on_attach {…} on_detach {…} }` → `.prefab.bin` (extension components + the `on_attach`/`on_detach` hook bytecode + the `requires` list). Validate `requires` (compile-time, against X's `of` chain) and additive conflicts (warning per `etch-reference-part2.md §30.5`: two extensions adding the same component to X). A scene's `extensions: ["Z", …]` clause → Entity Extensions Table + Prefab ID Table. -- `format.zig`/`writer.zig`/`accessor.zig`: `ExtensionEntry` + Prefab ID Table (D-C) — struct, emit, getters. +- D-E is CONFIRMED (see Design decisions) — not a blocker. Still read the seam (`observers.zig` + `interp.zig:1018` + `descriptor.zig:1668`) at E5 head to ground the E6 dispatch. +- **`extensions:` clause (Claude.ai amendment — `etch-grammar.md` re-uploaded, option a).** The clause `extensions: [STRING_LITERAL]` on **both** `entity_decl` and `instance_decl` (after `uuid`/`parent`, before components) is now grammatically defined. It requires: **parser** (`parseSceneEntity`/`parseSceneInstance` accept the clause), **AST** (`SceneEntity`/`SceneInstance` gain `extensions: []StringId`), **descriptors**. The references are **by NAME** (STRING_LITERAL — the extension prefab names), exactly like `parent:` / cross-refs (D-B). An absent clause = no active extension. +- **`.prefab.bin` hooks section (Claude.ai amendment).** An extension's `.prefab.bin` must carry the `on_attach`/`on_detach` hook bytecode (statement-runs) in addition to its components. The prefab/scene `.bin` format gains a **hooks section**; its on-disk SHAPE is the E5-opening micro-decision — read the format (`format.zig`/`writer.zig`/`accessor.zig`) + `descriptor.zig:1668` `renderStmtRunAlloc` (which already renders these statement-runs at cook), decide if trivial, else STOP + return to Claude.ai. +- `scene_cook.zig`: cook `prefab "Z" extends "X" requires … { entity {…} on_attach {…} on_detach {…} }` → `.prefab.bin` (extension components + the `on_attach`/`on_detach` hook statement-runs + the `requires` list). Validate `requires` (compile-time, against X's `of` chain) and additive conflicts (warning per `etch-reference-part2.md §30.5`: two extensions adding the same component to X). A scene's `extensions: ["Z", …]` clause → Entity Extensions Table + Prefab ID Table. +- `format.zig`/`writer.zig`/`accessor.zig`: `ExtensionEntry` + Prefab ID Table (D-C) + the prefab hooks section — structs, emit, getters. - → **STOP** — push, await review + GO. **E6 — Extensions: load (`applyExtensions` + `on_attach` dispatch).** @@ -141,11 +143,14 @@ Sparse (entities with no active extension are absent). The `§4` note's `u8` cou - `src/etch/interp.zig` — **edit** (E4) — `fieldKindFromTypeName` (`Entity` → `.entity_`, `.component` gate) + `compileTypeDecl` wiring. - `src/etch/ecs_bridge.zig` — **edit** (E4) — `.entity_` cases in `writeValueAsBytes`/`readBytesAsValue`. - `src/etch/value.zig` (or wherever the `Value` union lives — locate at E4) — **edit** (E4) — `.entity_` variant carrying `EntityId`. -- `src/core/scene/format.zig` — **edit** (E4/E5) — `CrossRefEntry` (16 B extern), `ExtensionEntry` + Prefab ID Table layout, read helpers. -- `src/core/scene/writer.zig` — **edit** (E4/E5) — emit the cross-references table and the extensions + Prefab ID tables from the `CookModel`. -- `src/core/scene/accessor.zig` — **edit** (E4/E5) — `crossrefsCount`/`crossref(i)`, `extensionsCount`/`extension(i)`, prefab-id getters. +- `src/core/scene/format.zig` — **edit** (E4/E5) — `CrossRefEntry` (16 B extern), `ExtensionEntry` + Prefab ID Table layout + the prefab **hooks section** (E5 shape TBD), read helpers. +- `src/core/scene/writer.zig` — **edit** (E4/E5) — emit the cross-references table, the extensions + Prefab ID tables, and the prefab hooks section from the `CookModel`. +- `src/core/scene/accessor.zig` — **edit** (E4/E5) — `crossrefsCount`/`crossref(i)`, `extensionsCount`/`extension(i)`, prefab-id + hooks getters. - `src/core/scene/loader.zig` — **edit** (E4/E6) — `resolveCrossRefs` and `applyExtensions` passes; replace the two reserved-table asserts; canonical load order. -- `src/etch/scene_cook.zig` — **edit** (E2/E3/E4/E5) — `cookPrefab` + `of` resolution (E2); `instance of` flattening + per-field overrides (E3); crossref pass (E4); `extends` cook + `extensions:` clause + `requires`/conflict validation (E5). +- `src/etch/parser.zig` — **edit** (E5, Claude.ai amendment) — `parseSceneEntity`/`parseSceneInstance` accept the `extensions: [STRING_LITERAL]` clause (after `uuid`/`parent`, before components). +- `src/etch/ast.zig` — **edit** (E5, Claude.ai amendment) — `SceneEntity`/`SceneInstance` gain `extensions: []StringId` (a `(start, len)` run, by-name). +- `src/etch/descriptor.zig` — **edit** (E5, Claude.ai amendment) — scene/prefab descriptors carry the `extensions:` clause; `renderStmtRunAlloc` (`:1668`) already renders the `on_attach`/`on_detach` statement-runs reused by the cook. +- `src/etch/scene_cook.zig` — **edit** (E2/E3/E4/E5) — `cookPrefab` + `of` resolution (E2); `instance of` flattening + per-field overrides (E3); crossref pass (E4); `extends` cook (hooks section) + `extensions:` clause → Entity Extensions/Prefab ID tables + `requires`/conflict validation (E5). - `tools/scene_cook/main.zig` — **edit** (E2) — cook `.prefab.etch` → `.prefab.bin`; prefab-before-scene ordering. - `src/etch/root.zig` — **edit if needed** (E5/E6) — re-export / §13 pin for any new `extends`/dispatch surface. - `tests/scene/prefab_cook_test.zig` — **create** (E2) — standalone + `of` cook round-trip via accessor. @@ -246,7 +251,7 @@ Sparse (entities with no active extension are absent). The `§4` note's `u8` cou ## Accepted deviations -- — +- 2026-06-28 (Claude.ai round-trip, session-2 head) — **E5 scope amended** (FROZEN SECTION edited via Claude.ai, per Guy's relay): (1) the `extensions: [STRING_LITERAL]` clause is now grammatically defined on `entity_decl` + `instance_decl` (`etch-grammar.md` re-uploaded, option a — resolves the E1 parse-gap blocker); E5 adds parser + AST (`SceneEntity`/`SceneInstance` `extensions: []StringId`) + descriptors, by-name. (2) The `.prefab.bin` format gains a **hooks section** carrying an extension's `on_attach`/`on_detach` statement-runs; on-disk shape is the E5-opening micro-decision. (3) **D-E status `confirmed-or-blocker` → CONFIRMED** — the Tier-0 callback seam exists (`observers.zig` + `interp.zig:1018`); hooks already render as statement-runs (`descriptor.zig:1668`); E6 adds an `on_attach` dispatch + a runtime `activate_extension`. Files added to the list: `src/etch/parser.zig`, `src/etch/ast.zig`, `src/etch/descriptor.zig`. NB: the re-uploaded `etch-grammar.md` did not reach the local `Downloads/` copy (still the 07:27 original); implemented against Guy's in-message amendment spec (clause shape + placement + by-name), consistent with the known staleness of that copy. ## Blockers encountered From 0e2f9ceae56f1b7e3607230efda3a6bac5ce0134 Mon Sep 17 00:00:00 2001 From: Guy Senpai Date: Sun, 28 Jun 2026 13:36:03 +0200 Subject: [PATCH 08/15] docs(brief): record E5 hooks-section shape blocker --- briefs/M1.0.6-prefabs-crossrefs-extensions.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/briefs/M1.0.6-prefabs-crossrefs-extensions.md b/briefs/M1.0.6-prefabs-crossrefs-extensions.md index d351544..c569b7f 100644 --- a/briefs/M1.0.6-prefabs-crossrefs-extensions.md +++ b/briefs/M1.0.6-prefabs-crossrefs-extensions.md @@ -247,6 +247,8 @@ Sparse (entities with no active extension are absent). The `§4` note's `u8` cou - 2026-06-28 11:30 — E3 STOP: pushed, awaiting Guy's review + GO before E4. - 2026-06-28 12:45 — E4 implemented (entity cross-references). **registry.zig**: `FieldKind.entity_` (8 B / 8-align, `EntityId` packed u64) + `sizeBytes`/`alignBytes`/`fromZigType` (imports `entity.zig`, acyclic). **interp.zig**: `fieldKindFromTypeName` emits `.entity_` for `Entity` when `reg_kind == .component` (mirror of the `string` gate, opposite origin); `compileTypeDecl` defaults an `.entity_` slot to `0xFF` (`EntityId.dead`, not the zeroed `{0,0}` live handle). **ecs_bridge.zig**: `.entity_` cases in `readBytesAsValue`/`writeValueAsBytes` (`value.zig` `EntityId` = u64, bit pattern of core `EntityId`) — the interp runtime read/write path; the cook never uses it. **format.zig**: `CrossRefEntry` (extern 16 B { source_uuid_ordinal, schema_index, field_offset, target_uuid_ordinal } + `readAt`); neutral-model `CrossRef` (carries `component_id`, not schema index) + `CookModel.cross_refs = &.{}` default. **writer.zig**: emit the Cross-references Table @ `crossrefs_offset` (count + entries), converting `component_id` → file-local schema index (`id_to_index`). **accessor.zig**: `crossrefsCount`/`crossref(i)`. **scene_cook.zig**: two-phase crossref pass — build phase records a `CrossRefPending { source_uuid_idx, component_id, field_offset, target_name }` for each set `.entity_` field (slot left `dead`, routed OUT of `encodeScalar` in `buildComponentBlob`/`mergeComponentBlob`/the instance `field_override`); post-build `resolveCrossRefs` resolves `target_name` → uuid ordinal against the complete `name_to_uuid_idx` (forward refs ok), `UnresolvedCrossRef` for an absent target; gated by `collect_crossrefs` (scene only — prefab cook leaves Entity slots `dead`, no entry). Reference is BY NAME (D-B), no `uuid "…"` form. **loader.zig**: `resolveCrossRefs` pass after `instantiate`, before `loadResources` — patches each cooked slot to the target's runtime handle (`@memcpy` + `markComponentChangedDyn`), bounds/identity validated (`MalformedScene`); the `crossrefs_offset==0` assert removed (extensions assert stays — E6). Tests `tests/scene/crossref_test.zig` (3, step `test-crossref`): cook→dead+entry / forward ref / unset=dead / `UnresolvedCrossRef` / cook→load→ECS resolved handle. Green Debug + ReleaseSafe; full `zig build test` 835/852 (17 skip, 0 fail, exit 0); fmt + lint clean. - 2026-06-28 12:45 — **Out-of-list edit (justified):** `tests/etch_interp/diff_runner.zig` (not in the brief's file list) — its two exhaustive `FieldKind` switches (`writeFieldValue`/`readFieldValue`) gained a `.entity_ => unreachable` arm. Adding a `FieldKind` variant mechanically requires updating every exhaustive switch over it; the S4 differential corpus is POD-only and never carries an `Entity` component field, so `unreachable` is the correct (proven) arm. No behavior change. +- 2026-06-28 13:30 — Session 2 head. Applied the Claude.ai FROZEN amendment (E5 scope: `extensions:` clause grammatically defined + parser/AST/descriptors; `.prefab.bin` hooks section; D-E CONFIRMED) — commit `a8821b5`. Re-read the amended grammar intent (worked from Guy's in-message amendment spec — the re-uploaded `etch-grammar.md` did not reach the local `Downloads/` copy, still 07:27) + `engine-scene-serialization.md` §5/§5.5.2 + `etch-reference-part2.md` §30. Confirmed the D-E seam (`observers.zig`, `interp.zig:1018`, `descriptor.zig:1668`). +- 2026-06-28 13:30 — E5 STOP (Case-2): the `.prefab.bin` hooks-section SHAPE is a frozen-format decision (header full, §4 reserved only two sections) — see Blockers. No E5 production code until the Claude.ai ruling. - 2026-06-28 12:45 — E4 STOP — **session 1 / session 2 boundary.** Pushed, awaiting Guy's review + GO. Per the gate plan, the next message relays the KB amendment lot (grammar `extensions:` + the scene-serialization §5 / §30.4 doc drifts) and the FROZEN brief amendment for E5 before session 2 starts. ## Accepted deviations @@ -255,6 +257,7 @@ Sparse (entities with no active extension are absent). The `§4` note's `u8` cou ## Blockers encountered +- **[E5-opening, Case-2 — `.prefab.bin` hooks-section SHAPE] STOP, return to Claude.ai.** Per the E5 instruction ("tranche si trivial, STOP + retour Claude.ai si non"). Findings: (1) `descriptor.zig:1668 renderStmtRunAlloc` renders `on_attach`/`on_detach` to **canonical Etch TEXT** (the `Prefab.on_attach`/`on_detach` descriptor fields are `[]const u8` text, not bytecode) — so the hooks *content* is trivial: two strings per `extends` prefab, storable as string-table refs. (2) The hooks *location* is the hard part and is a **frozen-format decision**: the 64-byte `SceneHeader` is **full** (`@44 extensions_offset`, `@48 crossrefs_offset`, `@52 _reserved` = hash-alignment padding, `@56 hash`) — there is **no free header offset** for a `hooks_offset`, and §4 (frozen M1.0.4) reserved exactly **two** sections (extensions, crossrefs), not a third. Viable shapes, both touching the frozen format contract: **(A) — recommended** — locate the hooks **inside the `extensions_offset` region**, appended after the Entity Extensions Table + Prefab ID Table (D-C): `extensions_offset` section becomes `[ext_count][entries][prefab_id_count][refs][hook_count][hooks…]` (self-delimiting, count-prefixed; no header change; a scene with no extensions = `[0][0][0]`, an `extends` prefab = `[0][0][n][…]`). **(B)** — repurpose `_reserved @52` as `hooks_offset` (header-level; `0` = absent → backward-compat without a version bump; hash stays @56). Either way, the D-C Entity Extensions Table layout already extends past M1.0.4's bare `[0]` at `extensions_offset`, so a **`format_version` 1 → 2 bump** is likely warranted (old v1 files → `BadVersion`, must be re-cooked — acceptable, the cook is deterministic). **Questions for Claude.ai:** (i) shape A vs B? (ii) `format_version` 1 → 2 acceptable, or must v1 files stay loadable? (iii) confirm hooks stored as rendered Etch **text** (string-table refs), not a new bytecode format (which would be out of scope). No production code written for E5 until the ruling lands. — pending - **[E1 finding, affects E5 only — not E2/E4] The scene-side `extensions: [...]` clause is NOT parsed.** The brief E1 task asks to "confirm `prefab_decl`/`instance of`/`extensions:` are already parsed (M0.8)". Result: `prefab_decl` (relation `of`/`extends`, `requires`, `on_attach`/`on_detach`) ✓ and `instance of` ✓ are parsed, but the `entity { extensions: [...] }` / `instance of … { extensions: [...] }` clause (`engine-scene-serialization.md` §3, §5.5.2) is **not** — neither in the grammar EBNF §15 (`entity_decl`/`instance_decl` carry only `uuid` + `parent`), nor in the M0.8 parser (`parseSceneEntity` `parser.zig:3364`, `parseSceneInstance` `:3407` parse only `uuid`/`parent` then require TYPE_IDENT-led `component_instance`; a leading lowercase `extensions` ident → parse error). E5's scope ("a scene's `extensions: ["Z", …]` clause → Entity Extensions Table") therefore needs a grammar + parser + AST addition for the clause, which is **not** in the brief's "Files to create or modify" list (no `parser.zig`/`ast.zig`/grammar entries). **Design/scope question for Guy (Case-2), to resolve before E5:** (a) extend grammar §15 + `parseSceneEntity`/`parseSceneInstance` + `SceneEntity`/`SceneInstance` AST + descriptors to carry `extensions: [string]` (a bounded, additive grammar change — recommended, it is the only way to author active extensions in a scene); or (b) defer scene-authored extension activation and land only the binary table + loader `applyExtensions` in M1.0.6 (table populated by a test fixture / future authoring path), keeping the parser untouched. This does **not** block E2–E4 (prefab cook, flattening, cross-refs). — pending ## Closing notes From ac21aeaf5eaca616a4a77721a0d88ba877973433 Mon Sep 17 00:00:00 2001 From: Guy Senpai Date: Sun, 28 Jun 2026 13:46:48 +0200 Subject: [PATCH 09/15] feat(etch): parse scene extensions clause (parser, AST, descriptors) --- briefs/M1.0.6-prefabs-crossrefs-extensions.md | 1 + build.zig | 3 + src/etch/ast.zig | 15 +++ src/etch/descriptor.zig | 29 ++++- src/etch/descriptor_types.zig | 10 +- src/etch/parser.zig | 35 ++++++ src/etch/zig_codegen/lower.zig | 18 ++- tests/scene/extensions_test.zig | 116 ++++++++++++++++++ 8 files changed, 221 insertions(+), 6 deletions(-) create mode 100644 tests/scene/extensions_test.zig diff --git a/briefs/M1.0.6-prefabs-crossrefs-extensions.md b/briefs/M1.0.6-prefabs-crossrefs-extensions.md index c569b7f..7cbbdcd 100644 --- a/briefs/M1.0.6-prefabs-crossrefs-extensions.md +++ b/briefs/M1.0.6-prefabs-crossrefs-extensions.md @@ -249,6 +249,7 @@ Sparse (entities with no active extension are absent). The `§4` note's `u8` cou - 2026-06-28 12:45 — **Out-of-list edit (justified):** `tests/etch_interp/diff_runner.zig` (not in the brief's file list) — its two exhaustive `FieldKind` switches (`writeFieldValue`/`readFieldValue`) gained a `.entity_ => unreachable` arm. Adding a `FieldKind` variant mechanically requires updating every exhaustive switch over it; the S4 differential corpus is POD-only and never carries an `Entity` component field, so `unreachable` is the correct (proven) arm. No behavior change. - 2026-06-28 13:30 — Session 2 head. Applied the Claude.ai FROZEN amendment (E5 scope: `extensions:` clause grammatically defined + parser/AST/descriptors; `.prefab.bin` hooks section; D-E CONFIRMED) — commit `a8821b5`. Re-read the amended grammar intent (worked from Guy's in-message amendment spec — the re-uploaded `etch-grammar.md` did not reach the local `Downloads/` copy, still 07:27) + `engine-scene-serialization.md` §5/§5.5.2 + `etch-reference-part2.md` §30. Confirmed the D-E seam (`observers.zig`, `interp.zig:1018`, `descriptor.zig:1668`). - 2026-06-28 13:30 — E5 STOP (Case-2): the `.prefab.bin` hooks-section SHAPE is a frozen-format decision (header full, §4 reserved only two sections) — see Blockers. No E5 production code until the Claude.ai ruling. +- 2026-06-28 14:30 — E5 (partial, unblocked chunk): the **`extensions:` clause** (Claude.ai gave the exact EBNF rule in-message; the re-uploaded grammar did not reach the local `Downloads/` copy). Implemented per that rule: **parser** (`parseExtensionsClause` + wired into `parseSceneEntity` after `parent`, `parseSceneInstance` after `uuid`; optional, by STRING_LITERAL name, trailing comma + empty list); **AST** (`SceneEntity`/`SceneInstance` gain `extensions_start`/`extensions_len` into the new `arena.scene_extensions` `StringId` run); **descriptors** (`SceneEntityDesc`/`SceneInstanceDesc` gain `extensions: []const []const u8`; `buildExtensionsRun` + free); **codegen** (`zig_codegen/lower.zig` `emitExtensionEntries` → the emitted `SceneEntityDesc`/`SceneInstanceDesc` literals carry `.extensions`). Test `tests/scene/extensions_test.zig` (4, step `test-extensions`): entity+instance clause → AST names, empty list + trailing comma, absent = empty run, descriptors carry names. Green Debug + ReleaseSafe; full `zig build test` 839/856 (17 skip, 0 fail, exit 0); fmt + lint clean. **The cook/binary + load portions of E5/E6 remain STOPPED on the hooks-section shape ruling** (Blockers) — they are coupled to the `extensions_offset` section layout (A vs B). - 2026-06-28 12:45 — E4 STOP — **session 1 / session 2 boundary.** Pushed, awaiting Guy's review + GO. Per the gate plan, the next message relays the KB amendment lot (grammar `extensions:` + the scene-serialization §5 / §30.4 doc drifts) and the FROZEN brief amendment for E5 before session 2 starts. ## Accepted deviations diff --git a/build.zig b/build.zig index 6846456..e0da62e 100644 --- a/build.zig +++ b/build.zig @@ -442,6 +442,9 @@ pub fn build(b: *std.Build) void { // side-table entry (by-name, two-phase), loader patches the slot to the // target handle; unset = dead; absent target = UnresolvedCrossRef at cook. .{ .path = "tests/scene/crossref_test.zig", .scene = true, .dedicated_step = "test-crossref" }, + // M1.0.6 / E5 — `extensions:` clause parse + AST + descriptors. The + // cook/binary + load portions land once the hooks-section shape unblocks. + .{ .path = "tests/scene/extensions_test.zig", .scene = true, .dedicated_step = "test-extensions" }, // M1.0.5 / E2 — runtime loader `.scene.bin` → ECS: instantiate every // entity (component bytes verbatim), two-phase `on_spawned` (fires once // per entity, after all entities exist). `weld_core` only (no `.scene` diff --git a/src/etch/ast.zig b/src/etch/ast.zig index c02f9af..b8a0c0a 100644 --- a/src/etch/ast.zig +++ b/src/etch/ast.zig @@ -1243,6 +1243,11 @@ pub const SceneEntity = struct { name: StringId, // STRING_LITERAL content uuid: StringId, // 0 if absent parent: StringId, // 0 if absent + /// `extensions: [...]` clause (M1.0.6 E5) — a `(start, len)` run of + /// `arena.scene_extensions` (active-extension prefab names, by name). Empty + /// (len 0) if the clause is absent. + extensions_start: u32, + extensions_len: u32, components_start: u32, // index into `arena.component_instances` components_len: u32, span: SourceSpan, @@ -1256,6 +1261,11 @@ pub const SceneInstance = struct { prefab_name: StringId, // STRING_LITERAL after `of` instance_name: StringId, // STRING_LITERAL uuid: StringId, // 0 if absent + /// `extensions: [...]` clause (M1.0.6 E5) — a `(start, len)` run of + /// `arena.scene_extensions` (active-extension prefab names, by name). Empty + /// (len 0) if the clause is absent. + extensions_start: u32, + extensions_len: u32, members_start: u32, // index into `arena.scene_instance_members` members_len: u32, span: SourceSpan, @@ -2324,6 +2334,10 @@ pub const AstArena = struct { scene_instance_members: std.ArrayListUnmanaged(InstanceMember) = .empty, component_instances: std.ArrayListUnmanaged(ComponentInstance) = .empty, field_overrides: std.ArrayListUnmanaged(FieldOverride) = .empty, + /// Active-extension name references (M1.0.6 E5) — the `extensions:` clause of + /// an `entity`/`instance`, by NAME (STRING_LITERAL `StringId`s, like `parent`). + /// `SceneEntity`/`SceneInstance` reference a `(start, len)` run here. + scene_extensions: std.ArrayListUnmanaged(StringId) = .empty, prefab_decls: std.ArrayListUnmanaged(PrefabDecl) = .empty, prefab_requires: std.ArrayListUnmanaged(StringId) = .empty, quest_decls: std.ArrayListUnmanaged(QuestDecl) = .empty, @@ -2548,6 +2562,7 @@ pub const AstArena = struct { self.scene_instance_members.deinit(gpa); self.component_instances.deinit(gpa); self.field_overrides.deinit(gpa); + self.scene_extensions.deinit(gpa); self.prefab_decls.deinit(gpa); self.prefab_requires.deinit(gpa); self.rule_params.deinit(gpa); diff --git a/src/etch/descriptor.zig b/src/etch/descriptor.zig index e2f8518..8815611 100644 --- a/src/etch/descriptor.zig +++ b/src/etch/descriptor.zig @@ -1438,10 +1438,16 @@ fn freeComponentInstances(gpa: std.mem.Allocator, cis: []const types.ComponentIn gpa.free(cis); } +fn freeStrList(gpa: std.mem.Allocator, list: []const []const u8) void { + for (list) |s| gpa.free(s); + gpa.free(list); +} + fn freeSceneEntityInner(gpa: std.mem.Allocator, e: types.SceneEntityDesc) void { gpa.free(e.name); gpa.free(e.uuid); gpa.free(e.parent); + freeStrList(gpa, e.extensions); freeComponentInstances(gpa, e.components); } @@ -1454,6 +1460,7 @@ fn freeSceneInstanceInner(gpa: std.mem.Allocator, inst: types.SceneInstanceDesc) gpa.free(inst.prefab); gpa.free(inst.name); gpa.free(inst.uuid); + freeStrList(gpa, inst.extensions); freeComponentInstances(gpa, inst.components); for (inst.overrides) |o| { gpa.free(o.type_name); @@ -1531,6 +1538,20 @@ fn buildComponentInstanceRun(gpa: std.mem.Allocator, arena: *const AstArena, sta return try list.toOwnedSlice(gpa); } +/// Build the `extensions:` clause names (M1.0.6 E5) as an owned `[]const []const u8`. +fn buildExtensionsRun(gpa: std.mem.Allocator, arena: *const AstArena, start: u32, len: u32) BuildError![]const []const u8 { + var list: std.ArrayListUnmanaged([]const u8) = .empty; + errdefer { + for (list.items) |s| gpa.free(s); + list.deinit(gpa); + } + var i: u32 = 0; + while (i < len) : (i += 1) { + try list.append(gpa, try gpa.dupe(u8, arena.strings.slice(arena.scene_extensions.items[start + i]))); + } + return try list.toOwnedSlice(gpa); +} + fn buildSceneEntity(gpa: std.mem.Allocator, arena: *const AstArena, e: ast_mod.SceneEntity) BuildError!types.SceneEntityDesc { const name = try gpa.dupe(u8, arena.strings.slice(e.name)); errdefer gpa.free(name); @@ -1538,8 +1559,10 @@ fn buildSceneEntity(gpa: std.mem.Allocator, arena: *const AstArena, e: ast_mod.S errdefer gpa.free(uuid); const parent = if (e.parent == 0) try gpa.dupe(u8, "") else try gpa.dupe(u8, arena.strings.slice(e.parent)); errdefer gpa.free(parent); + const extensions = try buildExtensionsRun(gpa, arena, e.extensions_start, e.extensions_len); + errdefer freeStrList(gpa, extensions); const components = try buildComponentInstanceRun(gpa, arena, e.components_start, e.components_len); - return .{ .name = name, .uuid = uuid, .parent = parent, .components = components }; + return .{ .name = name, .uuid = uuid, .parent = parent, .extensions = extensions, .components = components }; } fn buildSceneEntityRun(gpa: std.mem.Allocator, arena: *const AstArena, start: u32, len: u32) BuildError![]types.SceneEntityDesc { @@ -1562,6 +1585,8 @@ fn buildSceneInstance(gpa: std.mem.Allocator, arena: *const AstArena, inst: ast_ errdefer gpa.free(name); const uuid = if (inst.uuid == 0) try gpa.dupe(u8, "") else try gpa.dupe(u8, arena.strings.slice(inst.uuid)); errdefer gpa.free(uuid); + const extensions = try buildExtensionsRun(gpa, arena, inst.extensions_start, inst.extensions_len); + errdefer freeStrList(gpa, extensions); var comps: std.ArrayListUnmanaged(types.ComponentInstanceDesc) = .empty; errdefer { for (comps.items) |ci| { @@ -1596,7 +1621,7 @@ fn buildSceneInstance(gpa: std.mem.Allocator, arena: *const AstArena, inst: ast_ }, } } - return .{ .prefab = prefab, .name = name, .uuid = uuid, .components = try comps.toOwnedSlice(gpa), .overrides = try overs.toOwnedSlice(gpa) }; + return .{ .prefab = prefab, .name = name, .uuid = uuid, .extensions = extensions, .components = try comps.toOwnedSlice(gpa), .overrides = try overs.toOwnedSlice(gpa) }; } /// Build a `scene` descriptor (M0.8 E7 Level C): version, metadata, resources, diff --git a/src/etch/descriptor_types.zig b/src/etch/descriptor_types.zig index f9bd012..e260f01 100644 --- a/src/etch/descriptor_types.zig +++ b/src/etch/descriptor_types.zig @@ -562,19 +562,25 @@ pub const FieldOverrideDesc = struct { value: []const u8, }; -/// One scene/prefab `entity "Name" { [uuid] [parent] component* }`. +/// One scene/prefab `entity "Name" { [uuid] [parent] [extensions:] component* }`. pub const SceneEntityDesc = struct { name: []const u8, uuid: []const u8, // "" if absent parent: []const u8, // "" if absent + /// Active-extension prefab names (M1.0.6 E5 `extensions:` clause), by name. + /// Empty if the clause is absent. + extensions: []const []const u8, components: []const ComponentInstanceDesc, }; -/// One scene `instance of "Type" "Name" { [uuid] (component | override)* }`. +/// One scene `instance of "Type" "Name" { [uuid] [extensions:] (component | override)* }`. pub const SceneInstanceDesc = struct { prefab: []const u8, name: []const u8, uuid: []const u8, // "" if absent + /// Active-extension prefab names (M1.0.6 E5 `extensions:` clause), by name. + /// Empty if the clause is absent. + extensions: []const []const u8, components: []const ComponentInstanceDesc, overrides: []const FieldOverrideDesc, }; diff --git a/src/etch/parser.zig b/src/etch/parser.zig index 0c09076..1c30fa3 100644 --- a/src/etch/parser.zig +++ b/src/etch/parser.zig @@ -3361,6 +3361,33 @@ pub const Parser = struct { /// {component_instance} "}"` (§15 l.1598). Shared by scene + prefab bodies. /// Appends one `SceneEntity`, returns its index. Components append directly /// (the body closes before the next sibling — contiguous run). + /// Parse the optional `extensions: [ STRING_LITERAL {, STRING_LITERAL} [,] ]` + /// clause (M1.0.6 E5, Claude.ai amendment). Active-extension prefab names, by + /// name; trailing comma tolerated; empty list allowed. Appends each name to + /// `arena.scene_extensions` and returns the `(start, len)` run. + fn parseExtensionsClause(self: *Parser) ParseError!struct { start: u32, len: u32 } { + const start: u32 = @intCast(self.arena.scene_extensions.items.len); + var len: u32 = 0; + if (self.peek() == .ident and std.mem.eql(u8, self.sliceOf(self.peekSpan()), "extensions") and self.peekNext() == .colon) { + _ = try self.advance(); // 'extensions' + _ = try self.advance(); // ':' + _ = try self.expect(.lbracket, "expected '[' to start the extensions list"); + if (self.peek() != .rbracket) { + const first = try self.expect(.string_literal, "expected an extension prefab name (string literal)"); + try self.arena.scene_extensions.append(self.gpa, try self.internStringLiteral(first.span)); + len += 1; + while (try self.match(.comma)) { + if (self.peek() == .rbracket) break; // trailing comma + const t = try self.expect(.string_literal, "expected an extension prefab name (string literal) after ','"); + try self.arena.scene_extensions.append(self.gpa, try self.internStringLiteral(t.span)); + len += 1; + } + } + _ = try self.expect(.rbracket, "expected ']' to close the extensions list"); + } + return .{ .start = start, .len = len }; + } + fn parseSceneEntity(self: *Parser) ParseError!u32 { const kw_span = self.current.span; _ = try self.advance(); // 'entity' @@ -3380,6 +3407,8 @@ pub const Parser = struct { const p = try self.expect(.string_literal, "expected a parent string literal"); parent = try self.internStringLiteral(p.span); } + // `extensions: [...]` — after `parent`, before the components (§15 amend). + const ext = try self.parseExtensionsClause(); const components_start: u32 = @intCast(self.arena.component_instances.items.len); var components_len: u32 = 0; while (self.peek() != .rbrace and self.peek() != .eof) { @@ -3393,6 +3422,8 @@ pub const Parser = struct { .name = try self.internStringLiteral(name_tok.span), .uuid = uuid, .parent = parent, + .extensions_start = ext.start, + .extensions_len = ext.len, .components_start = components_start, .components_len = components_len, .span = .{ .byte_start = kw_span.byte_start, .byte_end = closing.span.byte_end }, @@ -3421,6 +3452,8 @@ pub const Parser = struct { const u = try self.expect(.string_literal, "expected a uuid string literal"); uuid = try self.internStringLiteral(u.span); } + // `extensions: [...]` — after `uuid`, before the members (§15 amend). + const ext = try self.parseExtensionsClause(); var members: std.ArrayListUnmanaged(ast_mod.InstanceMember) = .empty; defer members.deinit(self.gpa); while (self.peek() != .rbrace and self.peek() != .eof) { @@ -3445,6 +3478,8 @@ pub const Parser = struct { .prefab_name = try self.internStringLiteral(prefab_tok.span), .instance_name = try self.internStringLiteral(inst_name_tok.span), .uuid = uuid, + .extensions_start = ext.start, + .extensions_len = ext.len, .members_start = members_start, .members_len = @intCast(members.items.len), .span = .{ .byte_start = kw_span.byte_start, .byte_end = closing.span.byte_end }, diff --git a/src/etch/zig_codegen/lower.zig b/src/etch/zig_codegen/lower.zig index bdcd7fa..e7e8aff 100644 --- a/src/etch/zig_codegen/lower.zig +++ b/src/etch/zig_codegen/lower.zig @@ -5318,6 +5318,16 @@ fn emitComponentInstanceEntries(w: *Writer, gpa: std.mem.Allocator, ast: *const } } +/// Emit the `extensions:` clause names (M1.0.6 E5) as `&[_][]const u8{…}` entries. +fn emitExtensionEntries(w: *Writer, ast: *const AstArena, start: u32, len: u32) CodegenError!void { + var i: u32 = 0; + while (i < len) : (i += 1) { + try w.print(" ", .{}); + try emitZigStringLiteral(w, ast.strings.slice(ast.scene_extensions.items[start + i])); + try w.line(","); + } +} + /// Emit one `SceneEntityDesc` literal. Mirrors `buildSceneEntity`. fn emitSceneEntity(w: *Writer, gpa: std.mem.Allocator, ast: *const AstArena, e: ast_mod.SceneEntity) CodegenError!void { try w.print(" .{{ .name = ", .{}); @@ -5326,7 +5336,9 @@ fn emitSceneEntity(w: *Writer, gpa: std.mem.Allocator, ast: *const AstArena, e: try emitZigStringLiteral(w, if (e.uuid == 0) "" else ast.strings.slice(e.uuid)); try w.print(", .parent = ", .{}); try emitZigStringLiteral(w, if (e.parent == 0) "" else ast.strings.slice(e.parent)); - try w.line(", .components = &[_]etch_descriptor.ComponentInstanceDesc{"); + try w.line(", .extensions = &[_][]const u8{"); + try emitExtensionEntries(w, ast, e.extensions_start, e.extensions_len); + try w.line(" }, .components = &[_]etch_descriptor.ComponentInstanceDesc{"); try emitComponentInstanceEntries(w, gpa, ast, e.components_start, e.components_len); try w.line(" } },"); } @@ -5340,7 +5352,9 @@ fn emitSceneInstance(w: *Writer, gpa: std.mem.Allocator, ast: *const AstArena, i try emitZigStringLiteral(w, ast.strings.slice(inst.instance_name)); try w.print(", .uuid = ", .{}); try emitZigStringLiteral(w, if (inst.uuid == 0) "" else ast.strings.slice(inst.uuid)); - try w.line(", .components = &[_]etch_descriptor.ComponentInstanceDesc{"); + try w.line(", .extensions = &[_][]const u8{"); + try emitExtensionEntries(w, ast, inst.extensions_start, inst.extensions_len); + try w.line(" }, .components = &[_]etch_descriptor.ComponentInstanceDesc{"); var m: u32 = 0; while (m < inst.members_len) : (m += 1) { const mem = ast.scene_instance_members.items[inst.members_start + m]; diff --git a/tests/scene/extensions_test.zig b/tests/scene/extensions_test.zig new file mode 100644 index 0000000..d554cb8 --- /dev/null +++ b/tests/scene/extensions_test.zig @@ -0,0 +1,116 @@ +//! M1.0.6 E5 — `extensions:` clause: parse + AST + descriptors (Claude.ai +//! amendment). The clause `extensions: [STRING_LITERAL]` on `entity`/`instance` +//! (after `uuid`/`parent`, before components) records active-extension prefab +//! names by name (like `parent:` / cross-refs, D-B). +//! +//! The cook/binary portions of E5/E6 (Entity Extensions Table + Prefab ID Table, +//! the `extends` cook + `.prefab.bin` hooks section, `applyExtensions` + the +//! `on_attach` dispatch at load) land here once the `.prefab.bin` hooks-section +//! shape blocker is resolved — see `briefs/M1.0.6-…` Blockers. + +const std = @import("std"); +const weld_etch = @import("weld_etch"); + +const parser = weld_etch.parser; +const descriptor = weld_etch.descriptor; + +test "entity and instance extensions clauses parse to AST names" { + const gpa = std.testing.allocator; + const src = + \\component Health { current: i32 = 100 } + \\scene "S" { + \\ entity "A" { + \\ uuid: "00000000-0000-0000-0000-000000000001" + \\ extensions: ["CombatModule", "DialogueModule"] + \\ Health { current: 50 } + \\ } + \\ instance of "Base" "I" { + \\ uuid: "00000000-0000-0000-0000-000000000002" + \\ extensions: ["MerchantModule"] + \\ } + \\} + ; + var pr = try parser.parse(gpa, src); + defer pr.deinit(gpa); + try std.testing.expectEqual(@as(usize, 0), pr.diagnostics.len); + const ast = &pr.ast; + + const a = ast.scene_entities.items[0]; + try std.testing.expectEqual(@as(u32, 2), a.extensions_len); + try std.testing.expectEqualStrings("CombatModule", ast.strings.slice(ast.scene_extensions.items[a.extensions_start])); + try std.testing.expectEqualStrings("DialogueModule", ast.strings.slice(ast.scene_extensions.items[a.extensions_start + 1])); + + const inst = ast.scene_instances.items[0]; + try std.testing.expectEqual(@as(u32, 1), inst.extensions_len); + try std.testing.expectEqualStrings("MerchantModule", ast.strings.slice(ast.scene_extensions.items[inst.extensions_start])); +} + +test "empty extensions list and trailing comma parse" { + const gpa = std.testing.allocator; + const src = + \\scene "S" { + \\ entity "A" { uuid: "00000000-0000-0000-0000-000000000001" extensions: [] } + \\ entity "B" { uuid: "00000000-0000-0000-0000-000000000002" extensions: ["X",] } + \\} + ; + var pr = try parser.parse(gpa, src); + defer pr.deinit(gpa); + try std.testing.expectEqual(@as(usize, 0), pr.diagnostics.len); + const ast = &pr.ast; + + try std.testing.expectEqual(@as(u32, 0), ast.scene_entities.items[0].extensions_len); + const b = ast.scene_entities.items[1]; + try std.testing.expectEqual(@as(u32, 1), b.extensions_len); + try std.testing.expectEqualStrings("X", ast.strings.slice(ast.scene_extensions.items[b.extensions_start])); +} + +test "an entity with no extensions clause has an empty run" { + const gpa = std.testing.allocator; + const src = + \\scene "S" { + \\ entity "A" { uuid: "00000000-0000-0000-0000-000000000001" } + \\} + ; + var pr = try parser.parse(gpa, src); + defer pr.deinit(gpa); + try std.testing.expectEqual(@as(usize, 0), pr.diagnostics.len); + try std.testing.expectEqual(@as(u32, 0), pr.ast.scene_entities.items[0].extensions_len); +} + +test "descriptors carry the extensions names" { + const gpa = std.testing.allocator; + const src = + \\scene "S" { + \\ entity "A" { + \\ uuid: "00000000-0000-0000-0000-000000000001" + \\ extensions: ["X", "Y"] + \\ } + \\ instance of "Base" "I" { + \\ uuid: "00000000-0000-0000-0000-000000000002" + \\ extensions: ["Z"] + \\ } + \\} + ; + var pr = try parser.parse(gpa, src); + defer pr.deinit(gpa); + try std.testing.expectEqual(@as(usize, 0), pr.diagnostics.len); + + var descs = try descriptor.build(gpa, &pr.ast); + defer descs.deinit(gpa); + + var saw_scene = false; + for (descs.items) |d| switch (d) { + .scene => |sc| { + saw_scene = true; + try std.testing.expectEqual(@as(usize, 1), sc.entities.len); + try std.testing.expectEqual(@as(usize, 2), sc.entities[0].extensions.len); + try std.testing.expectEqualStrings("X", sc.entities[0].extensions[0]); + try std.testing.expectEqualStrings("Y", sc.entities[0].extensions[1]); + try std.testing.expectEqual(@as(usize, 1), sc.instances.len); + try std.testing.expectEqual(@as(usize, 1), sc.instances[0].extensions.len); + try std.testing.expectEqualStrings("Z", sc.instances[0].extensions[0]); + }, + else => {}, + }; + try std.testing.expect(saw_scene); +} From bd51e3b8722bc147b831839e87cbbf07484cb041 Mon Sep 17 00:00:00 2001 From: Guy Senpai Date: Sun, 28 Jun 2026 14:20:43 +0200 Subject: [PATCH 10/15] feat(scene): cook extends prefabs + scene extensions tables (format v2) --- briefs/M1.0.6-prefabs-crossrefs-extensions.md | 2 + src/core/scene/accessor.zig | 88 +++++++++++ src/core/scene/format.zig | 42 ++++- src/core/scene/writer.zig | 27 +++- src/etch/scene_cook.zig | 127 +++++++++++++++- tests/scene/extensions_test.zig | 143 ++++++++++++++++++ tests/scene/prefab_cook_test.zig | 14 +- 7 files changed, 424 insertions(+), 19 deletions(-) diff --git a/briefs/M1.0.6-prefabs-crossrefs-extensions.md b/briefs/M1.0.6-prefabs-crossrefs-extensions.md index 7cbbdcd..4a39ccb 100644 --- a/briefs/M1.0.6-prefabs-crossrefs-extensions.md +++ b/briefs/M1.0.6-prefabs-crossrefs-extensions.md @@ -249,11 +249,13 @@ Sparse (entities with no active extension are absent). The `§4` note's `u8` cou - 2026-06-28 12:45 — **Out-of-list edit (justified):** `tests/etch_interp/diff_runner.zig` (not in the brief's file list) — its two exhaustive `FieldKind` switches (`writeFieldValue`/`readFieldValue`) gained a `.entity_ => unreachable` arm. Adding a `FieldKind` variant mechanically requires updating every exhaustive switch over it; the S4 differential corpus is POD-only and never carries an `Entity` component field, so `unreachable` is the correct (proven) arm. No behavior change. - 2026-06-28 13:30 — Session 2 head. Applied the Claude.ai FROZEN amendment (E5 scope: `extensions:` clause grammatically defined + parser/AST/descriptors; `.prefab.bin` hooks section; D-E CONFIRMED) — commit `a8821b5`. Re-read the amended grammar intent (worked from Guy's in-message amendment spec — the re-uploaded `etch-grammar.md` did not reach the local `Downloads/` copy, still 07:27) + `engine-scene-serialization.md` §5/§5.5.2 + `etch-reference-part2.md` §30. Confirmed the D-E seam (`observers.zig`, `interp.zig:1018`, `descriptor.zig:1668`). - 2026-06-28 13:30 — E5 STOP (Case-2): the `.prefab.bin` hooks-section SHAPE is a frozen-format decision (header full, §4 reserved only two sections) — see Blockers. No E5 production code until the Claude.ai ruling. +- 2026-06-28 15:30 — E5 (binary + cook) complete, post-ruling (SHAPE A, format_version 2, hooks=text). **format.zig**: `format_version` 1 → 2; `ExtModelEntry`/`HookSet` model types; `CookModel.ext_entries`/`prefab_id_table`/`hooks`. **writer.zig**: `writeExtensionsRegion` emits the SHAPE-A region @ `extensions_offset` (Entity Extensions Table + Prefab ID Table + hooks; component ids → file-local indices; hook refs = string-table offsets, 0=absent). **accessor.zig**: `extensionsCount`/`extension(i)` (variable-length walk), `prefabIdCount`/`prefabName(i)`, `hookCount`/`hook(i)`. **scene_cook.zig**: scene cook records `extensions:` → `ext_entries` + dedup `prefab_id_table` (`recordExtensions`/`prefabIdIndex`); `extends` prefab cook (`buildPrefab` no longer rejects extends) → components (the `entity{}` block) + `buildExtendsHooks` (renders `on_attach`/`on_detach` to text via `descriptor.renderStmtRunAlloc`, interned) + `validateRequires` (each `requires C` must be in X's `.prefab.bin`, `RequiresNotSatisfied`); removed `ExtendsUnsupported`, added `RequiresNotSatisfied`/`HookRenderFailed`. Tests `extensions_test.zig` (+3 cook tests, now 7): extends cooks with components/hooks/requires, requires-not-in-base rejected, scene `extensions:` populates the Entity Extensions + dedup Prefab ID tables. `prefab_cook_test` "rejects extends" → "rejects a scene source" (extends now cooked). Green Debug + ReleaseSafe; full `zig build test` 842/859 (17 skip, 0 fail, exit 0); fmt + lint clean. **Residual (journaled): the §30.5 additive-conflict WARNING** (two extensions adding the same component to X) is deferred — it needs a cook warning channel + cross-extension component-set comparison at scene cook; not tested by the acceptance set. **E6 (load) is next** — see the design note in the report. - 2026-06-28 14:30 — E5 (partial, unblocked chunk): the **`extensions:` clause** (Claude.ai gave the exact EBNF rule in-message; the re-uploaded grammar did not reach the local `Downloads/` copy). Implemented per that rule: **parser** (`parseExtensionsClause` + wired into `parseSceneEntity` after `parent`, `parseSceneInstance` after `uuid`; optional, by STRING_LITERAL name, trailing comma + empty list); **AST** (`SceneEntity`/`SceneInstance` gain `extensions_start`/`extensions_len` into the new `arena.scene_extensions` `StringId` run); **descriptors** (`SceneEntityDesc`/`SceneInstanceDesc` gain `extensions: []const []const u8`; `buildExtensionsRun` + free); **codegen** (`zig_codegen/lower.zig` `emitExtensionEntries` → the emitted `SceneEntityDesc`/`SceneInstanceDesc` literals carry `.extensions`). Test `tests/scene/extensions_test.zig` (4, step `test-extensions`): entity+instance clause → AST names, empty list + trailing comma, absent = empty run, descriptors carry names. Green Debug + ReleaseSafe; full `zig build test` 839/856 (17 skip, 0 fail, exit 0); fmt + lint clean. **The cook/binary + load portions of E5/E6 remain STOPPED on the hooks-section shape ruling** (Blockers) — they are coupled to the `extensions_offset` section layout (A vs B). - 2026-06-28 12:45 — E4 STOP — **session 1 / session 2 boundary.** Pushed, awaiting Guy's review + GO. Per the gate plan, the next message relays the KB amendment lot (grammar `extensions:` + the scene-serialization §5 / §30.4 doc drifts) and the FROZEN brief amendment for E5 before session 2 starts. ## Accepted deviations +- 2026-06-28 (Claude.ai ruling, session-2) — **`.prefab.bin` hooks-section SHAPE resolved** (the E5-opening Case-2 blocker): **(A)** hooks live in the `extensions_offset` region, appended **after** the Entity Extensions Table + Prefab ID Table — no new header field, `_reserved @52` preserved, all extension serialization in one place (hooks only ever exist in an `extends` `.prefab.bin`, never a `.scene.bin` / `of` / standalone, so a top-level `hooks_offset` would be 0 almost everywhere). Region layout @ `extensions_offset` (self-delimiting): `ext_count:u32`, `[ext_count]{uuid_ordinal:u32, extension_count:u32, extension_ids:[…]u32}`, `prefab_id_count:u32`, `prefab_name_refs:[…]u32` (string-table offsets, dedup'd), `hook_count:u32`, `[hook_count]{on_attach_ref:u32, on_detach_ref:u32}` (string-table offsets; `0` = absent). Filling: `.scene.bin` → `ext>0, ids>0, hooks=0`; `.prefab.bin extends` → `ext=0, ids=0, hooks=1`; none → `[0][0][0]`. `hook_count ∈ {0,1}` in M1.0.6. **format_version 1 → 2** (the reserved sections go from bare count-placeholder to full structure — a break; v1 → `BadVersion` → re-cook; no prod files in Phase 1). **Hooks = Etch TEXT** (`renderStmtRunAlloc`, string-table refs) — no bytecode format. E6 runs `on_attach` at load by re-parsing+interpreting that text via the Etch bridge (a conscious debt — no pre-compiled bytecode in M1.0.6); `on_detach` is stored for the runtime `deactivate_extension`, not run at load; dispatch via the Tier-0 callback pattern (D-E), `loader.zig` never calls the VM directly. `engine-scene-serialization.md` §4 (region + hooks + version 2) is Guy's to update at close. D-C stays valid; the hooks section is its acted extension. - 2026-06-28 (Claude.ai round-trip, session-2 head) — **E5 scope amended** (FROZEN SECTION edited via Claude.ai, per Guy's relay): (1) the `extensions: [STRING_LITERAL]` clause is now grammatically defined on `entity_decl` + `instance_decl` (`etch-grammar.md` re-uploaded, option a — resolves the E1 parse-gap blocker); E5 adds parser + AST (`SceneEntity`/`SceneInstance` `extensions: []StringId`) + descriptors, by-name. (2) The `.prefab.bin` format gains a **hooks section** carrying an extension's `on_attach`/`on_detach` statement-runs; on-disk shape is the E5-opening micro-decision. (3) **D-E status `confirmed-or-blocker` → CONFIRMED** — the Tier-0 callback seam exists (`observers.zig` + `interp.zig:1018`); hooks already render as statement-runs (`descriptor.zig:1668`); E6 adds an `on_attach` dispatch + a runtime `activate_extension`. Files added to the list: `src/etch/parser.zig`, `src/etch/ast.zig`, `src/etch/descriptor.zig`. NB: the re-uploaded `etch-grammar.md` did not reach the local `Downloads/` copy (still the 07:27 original); implemented against Guy's in-message amendment spec (clause shape + placement + by-name), consistent with the known staleness of that copy. ## Blockers encountered diff --git a/src/core/scene/accessor.zig b/src/core/scene/accessor.zig index 497591d..be59d4c 100644 --- a/src/core/scene/accessor.zig +++ b/src/core/scene/accessor.zig @@ -134,6 +134,94 @@ pub const Accessor = struct { } }; + // ── Entity Extensions region (M1.0.6 E5, SHAPE A) ── + // + // `@ extensions_offset`, three self-delimiting sub-tables in order: + // Entity Extensions Table — `ext_count:u32` then per entity + // `{ uuid_ordinal:u32, extension_count:u32, extension_ids:[…]u32 }` + // Prefab ID Table — `prefab_id_count:u32` then `[…]u32` string-table offsets + // Hooks — `hook_count:u32` then `[…]{ on_attach_ref:u32, on_detach_ref:u32 }` + // (string-table offsets; 0 = absent). `hook_count ∈ {0,1}` in M1.0.6. + + /// A view over one Entity Extensions Table entry. + pub const ExtEntry = struct { + acc: Accessor, + uuid_ordinal: u32, + extension_count: u32, + ids_off: usize, // file offset of the first `extension_id` u32 + + /// The `j`-th active-extension id (an index into the Prefab ID Table). + pub fn extensionId(self: ExtEntry, j: u32) u32 { + return self.acc.readU32(self.ids_off + @as(usize, j) * 4); + } + }; + + pub fn extensionsCount(self: Accessor) u32 { + return self.readU32(self.header.extensions_offset); + } + + /// The `i`-th Entity Extensions Table entry (walks variable-length entries). + pub fn extension(self: Accessor, i: u32) ExtEntry { + var off: usize = self.header.extensions_offset + 4; // skip ext_count + var k: u32 = 0; + while (k < i) : (k += 1) { + const ecount = self.readU32(off + 4); + off += 8 + @as(usize, ecount) * 4; + } + return .{ + .acc = self, + .uuid_ordinal = self.readU32(off), + .extension_count = self.readU32(off + 4), + .ids_off = off + 8, + }; + } + + /// File offset of the Prefab ID Table's `prefab_id_count` (past the ext table). + fn prefabIdTableStart(self: Accessor) usize { + var off: usize = self.header.extensions_offset + 4; + const count = self.extensionsCount(); + var k: u32 = 0; + while (k < count) : (k += 1) { + const ecount = self.readU32(off + 4); + off += 8 + @as(usize, ecount) * 4; + } + return off; + } + + pub fn prefabIdCount(self: Accessor) u32 { + return self.readU32(self.prefabIdTableStart()); + } + + /// The `i`-th deduplicated extension-prefab name (Prefab ID Table). + pub fn prefabName(self: Accessor, i: u32) []const u8 { + const base = self.prefabIdTableStart() + 4; + return self.stringAt(self.readU32(base + @as(usize, i) * 4)); + } + + /// File offset of the hooks sub-section's `hook_count` (past the Prefab ID Table). + fn hooksStart(self: Accessor) usize { + const pcount = self.prefabIdCount(); + return self.prefabIdTableStart() + 4 + @as(usize, pcount) * 4; + } + + pub fn hookCount(self: Accessor) u32 { + return self.readU32(self.hooksStart()); + } + + /// One hooks entry: `on_attach`/`on_detach` rendered Etch text, or null if the + /// hook is absent (its on-disk string-table ref is 0). + pub const Hook = struct { on_attach: ?[]const u8, on_detach: ?[]const u8 }; + + pub fn hook(self: Accessor, i: u32) Hook { + const base = self.hooksStart() + 4 + @as(usize, i) * 8; + const a_ref = self.readU32(base); + const d_ref = self.readU32(base + 4); + return .{ + .on_attach = if (a_ref == 0) null else self.stringAt(a_ref), + .on_detach = if (d_ref == 0) null else self.stringAt(d_ref), + }; + } + // ── Cross-references Table (M1.0.6 E4) ── /// Number of entity→entity cross-reference entries (`0` for a scene with no diff --git a/src/core/scene/format.zig b/src/core/scene/format.zig index 8d23d85..1012d9c 100644 --- a/src/core/scene/format.zig +++ b/src/core/scene/format.zig @@ -56,7 +56,14 @@ pub const magic = [4]u8{ 'W', 'S', 'C', 'N' }; /// `.scene.bin` binary format version (the codec/layout version — bumped on any /// breaking layout change). Distinct from `content_version` (the authored /// scene's `version:` field, opaque to the codec). -pub const format_version: u16 = 1; +/// +/// **2** (M1.0.6): the reserved sections became real — the cross-references table +/// and the `extensions_offset` region (Entity Extensions Table + Prefab ID Table +/// + hooks) went from bare count-placeholders (`[0]`) to full structures. A break +/// vs v1 (M1.0.4/M1.0.5): a v1 file fails `BadVersion` and must be re-cooked +/// (`.scene.bin`/`.prefab.bin` are deterministic build artifacts, no prod files +/// in Phase 1). +pub const format_version: u16 = 2; /// `SceneHeader` size — the fixed 64-byte (cache-line) prefix every file opens /// with. All section offsets in the header are relative to the file start. @@ -308,6 +315,26 @@ pub const CrossRef = struct { target_uuid: u32, }; +/// One entity's active extensions (M1.0.6 E5) in the neutral model — the +/// `extensions:` clause of a scene entity/instance. `uuid` is a `CookModel.uuids` +/// ordinal (the bearing entity); `prefab_ids` are indices into +/// `CookModel.prefab_id_table` (the dedup'd extension-name table). On-disk these +/// become the Entity Extensions Table entries @ `extensions_offset`. +pub const ExtModelEntry = struct { + uuid: u32, + prefab_ids: []const u32, +}; + +/// An `extends` prefab's hooks (M1.0.6 E5) in the neutral model — `on_attach` / +/// `on_detach` rendered as canonical Etch **text** (`CookModel.strings` indices, +/// `null` = the hook is absent). On-disk these become the hooks sub-section's +/// `{on_attach_ref, on_detach_ref}` (string-table offsets; `0` = absent). Only an +/// `extends` `.prefab.bin` carries one (`hook_count ∈ {0,1}` in M1.0.6). +pub const HookSet = struct { + on_attach: ?u32, + on_detach: ?u32, +}; + /// The neutral, World-free model the cook produces. Owns every slice via an /// internal arena; `deinit` frees the lot. The E2 writer reads it to emit /// `.scene.bin`; the E1 cook test inspects it directly (no serialization). @@ -322,6 +349,17 @@ pub const CookModel = struct { /// `Entity` field references and for every prefab. Serialized to the /// Cross-references Table @ `crossrefs_offset`. cross_refs: []const CrossRef = &.{}, + /// Active-extension entries (M1.0.6 E5) — one per scene entity/instance with a + /// non-empty `extensions:` clause. Empty for a prefab and for an extension-free + /// scene. Serialized to the Entity Extensions Table @ `extensions_offset`. + ext_entries: []const ExtModelEntry = &.{}, + /// Deduplicated extension-prefab names (M1.0.6 E5), as `CookModel.strings` + /// indices; `ExtModelEntry.prefab_ids` index this table. Serialized to the + /// Prefab ID Table (string-table offsets). + prefab_id_table: []const u32 = &.{}, + /// `extends` prefab hooks (M1.0.6 E5) — `hook_count ∈ {0,1}`. Empty for a + /// scene and for `of`/standalone prefabs. Serialized to the hooks sub-section. + hooks: []const HookSet = &.{}, /// The authored scene's `version:` field (0 if absent). Propagated to /// `SceneHeader.content_version` — opaque to the codec, for the game's own /// scene-versioning/migration. @@ -353,7 +391,7 @@ test "CookModel arena round-trips an empty model" { test "format magic + version constants are stable" { try std.testing.expectEqualSlices(u8, "WSCN", &magic); - try std.testing.expectEqual(@as(u16, 1), format_version); + try std.testing.expectEqual(@as(u16, 2), format_version); } test "SceneHeader writeTo/read round-trips little-endian" { diff --git a/src/core/scene/writer.zig b/src/core/scene/writer.zig index 1d52077..1d419a3 100644 --- a/src/core/scene/writer.zig +++ b/src/core/scene/writer.zig @@ -87,7 +87,7 @@ const Writer = struct { hdr.archetypes_offset = try self.sectionOffset(); try self.writeArchetypes(); hdr.extensions_offset = try self.sectionOffset(); - try self.appendU32(0); // reserved — empty (M1.0.6 E5/E6) + try self.writeExtensionsRegion(); hdr.crossrefs_offset = try self.sectionOffset(); try self.writeCrossRefs(); @@ -233,6 +233,31 @@ const Writer = struct { /// `CrossRefEntry` (16 B). The model carries `component_id`; here it is /// converted to the file-local Schema Registry index (`id_to_index`) — the /// on-disk entry never stores a runtime `ComponentId`. + /// Entity Extensions region @ `extensions_offset` (M1.0.6 E5, SHAPE A) — three + /// self-delimiting sub-tables: the Entity Extensions Table (per-entity active + /// extensions), the Prefab ID Table (dedup'd extension names → string-table + /// offsets), and the hooks (`extends` prefab `on_attach`/`on_detach` text refs; + /// `0` = absent — safe because a prefab's entity name is interned before its + /// hooks, so no hook text lands at string-table offset 0). + fn writeExtensionsRegion(self: *Writer) WriteError!void { + // Entity Extensions Table. + try self.appendU32(try u32From(self.model.ext_entries.len)); + for (self.model.ext_entries) |e| { + try self.appendU32(e.uuid); + try self.appendU32(try u32From(e.prefab_ids.len)); + for (e.prefab_ids) |pid| try self.appendU32(pid); + } + // Prefab ID Table (dedup'd extension names as string-table offsets). + try self.appendU32(try u32From(self.model.prefab_id_table.len)); + for (self.model.prefab_id_table) |str_idx| try self.appendU32(self.model_str_ref[str_idx]); + // Hooks (`hook_count ∈ {0,1}`; refs are string-table offsets, 0 = absent). + try self.appendU32(try u32From(self.model.hooks.len)); + for (self.model.hooks) |h| { + try self.appendU32(if (h.on_attach) |idx| self.model_str_ref[idx] else 0); + try self.appendU32(if (h.on_detach) |idx| self.model_str_ref[idx] else 0); + } + } + fn writeCrossRefs(self: *Writer) WriteError!void { try self.appendU32(try u32From(self.model.cross_refs.len)); for (self.model.cross_refs) |cr| { diff --git a/src/etch/scene_cook.zig b/src/etch/scene_cook.zig index 9bdd215..cfc2ab9 100644 --- a/src/etch/scene_cook.zig +++ b/src/etch/scene_cook.zig @@ -33,6 +33,9 @@ const ast_mod = @import("ast.zig"); const interp = @import("interp.zig"); const bridge_mod = @import("ecs_bridge.zig"); const value_mod = @import("value.zig"); +// M1.0.6 E5 — `renderStmtRunAlloc` renders an extends prefab's on_attach/on_detach +// statement-runs to canonical Etch text (stored in the .prefab.bin hooks section). +const descriptor = @import("descriptor.zig"); const weld_core = @import("weld_core"); // M1.0.5 — persistent heap moved to Tier 0 (`src/core/memory`); reach it via weld_core. @@ -94,11 +97,15 @@ pub const CookError = error{ MultiplePrefabs, /// A `scene` construct appeared in a source cooked as a `.prefab.etch`. SceneNotAllowedInPrefab, - /// `prefab "Z" extends "X"` — extension cook is owned by M1.0.6 E5, not E2. - ExtendsUnsupported, /// `on_attach`/`on_detach`/`requires` on a standalone or `of` prefab — those /// clauses are valid only with `extends` (`etch-grammar.md` §15 l.1653, §30.5). PrefabHookNotAllowed, + /// `extends "X" requires C` where the base prefab `X` does not declare `C` + /// (`etch-grammar.md` §15 l.1634 — validated against X's cooked `.prefab.bin`). + RequiresNotSatisfied, + /// An `extends` prefab's `on_attach`/`on_detach` body could not be rendered to + /// canonical Etch text (a construct outside the descriptor renderer's surface). + HookRenderFailed, /// `prefab "Y" of "X"` but the base `X.prefab.bin` could not be resolved /// (no resolver, or the resolver returned null for the base name). BasePrefabMissing, @@ -290,6 +297,14 @@ const Builder = struct { collect_crossrefs: bool = false, pendings: std.ArrayListUnmanaged(CrossRefPending) = .empty, + // Active extensions (M1.0.6 E5, scene cook): per-entity `extensions:` clauses + // → Entity Extensions Table + a deduplicated Prefab ID Table (extension names + // as `strings` indices). `prefab_id_map` dedups a name's `strings` index → its + // Prefab ID Table slot. + ext_entries: std.ArrayListUnmanaged(format.ExtModelEntry) = .empty, + prefab_id_table: std.ArrayListUnmanaged(u32) = .empty, + prefab_id_map: std.AutoHashMapUnmanaged(u32, u32) = .empty, + fn init(gpa: std.mem.Allocator, ast: *const AstArena, registry: *Registry) Builder { return .{ .gpa = gpa, @@ -313,6 +328,9 @@ const Builder = struct { self.strings.deinit(self.gpa); self.uuids.deinit(self.gpa); self.pendings.deinit(self.gpa); + self.ext_entries.deinit(self.gpa); + self.prefab_id_table.deinit(self.gpa); + self.prefab_id_map.deinit(self.gpa); } fn a(self: *Builder) std.mem.Allocator { @@ -406,10 +424,23 @@ const Builder = struct { var entities: std.ArrayListUnmanaged(EntityBuild) = .empty; defer entities.deinit(self.gpa); for (children) |child| { + var ext_start: u32 = 0; + var ext_len: u32 = 0; const eb = switch (child.kind) { - .entity => try self.buildEntity(self.ast.scene_entities.items[child.index], diag_out), - .instance => try self.buildInstanceEntity(self.ast.scene_instances.items[child.index], base_resolver, diag_out), + .entity => blk: { + const e = self.ast.scene_entities.items[child.index]; + ext_start = e.extensions_start; + ext_len = e.extensions_len; + break :blk try self.buildEntity(e, diag_out); + }, + .instance => blk: { + const inst = self.ast.scene_instances.items[child.index]; + ext_start = inst.extensions_start; + ext_len = inst.extensions_len; + break :blk try self.buildInstanceEntity(inst, base_resolver, diag_out); + }, }; + try self.recordExtensions(eb.uuid_idx, ext_start, ext_len); try entities.append(self.gpa, eb); } @@ -433,10 +464,37 @@ const Builder = struct { .archetypes = archetypes, .content_version = content_version, .cross_refs = cross_refs, + .ext_entries = try self.a().dupe(format.ExtModelEntry, self.ext_entries.items), + .prefab_id_table = try self.a().dupe(u32, self.prefab_id_table.items), .arena = self.arena, }; } + /// Record an entity's `extensions:` clause (M1.0.6 E5) into the Entity + /// Extensions Table: intern each extension name (by name, D-B) into the model + /// strings + the deduplicated Prefab ID Table, and append an `ExtModelEntry` + /// keyed by the entity's uuid ordinal. No-op for an empty/absent clause. + fn recordExtensions(self: *Builder, uuid_idx: u32, ext_start: u32, ext_len: u32) CookError!void { + if (ext_len == 0) return; + const ids = try self.a().alloc(u32, ext_len); + var k: u32 = 0; + while (k < ext_len) : (k += 1) { + const name = self.ast.strings.slice(self.ast.scene_extensions.items[ext_start + k]); + ids[k] = try self.prefabIdIndex(try self.internString(name)); + } + try self.ext_entries.append(self.gpa, .{ .uuid = uuid_idx, .prefab_ids = ids }); + } + + /// The Prefab ID Table slot for a model-`strings` index, deduplicated. + fn prefabIdIndex(self: *Builder, str_idx: u32) CookError!u32 { + const gop = try self.prefab_id_map.getOrPut(self.gpa, str_idx); + if (!gop.found_existing) { + gop.value_ptr.* = @intCast(self.prefab_id_table.items.len); + try self.prefab_id_table.append(self.gpa, str_idx); + } + return gop.value_ptr.*; + } + /// Phase 2 of the crossref pass: resolve every pending `target_name` against /// the now-complete `name_to_uuid_idx` (a reference can name an entity declared /// later in the scene), producing the model's `CrossRef` slice. A target that @@ -480,9 +538,8 @@ const Builder = struct { /// `.prefab.bin` via the accessor) then applies per-entity overrides. A prefab /// body is `{ entity_decl }` only — no `resources`/`instance` (§15 l.1624). fn buildPrefab(self: *Builder, pd: ast_mod.PrefabDecl, base_resolver: ?BaseResolver, diag_out: ?*[]const u8) CookError!format.CookModel { - if (pd.relation == .extends) return fail(diag_out, error.ExtendsUnsupported, "`extends` (extension) prefab cook is owned by M1.0.6 E5, not E2"); // `requires`/`on_attach`/`on_detach` are valid only with `extends` (§30.5). - if (pd.requires_len != 0 or pd.has_on_attach or pd.has_on_detach) + if (pd.relation != .extends and (pd.requires_len != 0 or pd.has_on_attach or pd.has_on_detach)) return fail(diag_out, error.PrefabHookNotAllowed, "`requires`/`on_attach`/`on_detach` are valid only on an `extends` prefab"); const prefab_entities = self.ast.scene_entities.items[pd.entities_start .. pd.entities_start + pd.entities_len]; @@ -499,6 +556,9 @@ const Builder = struct { try self.reconstructBase(acc, &entities, diag_out); try self.mergeVariantEntities(prefab_entities, &entities, diag_out); } else { + // Standalone OR extends: the prefab's `entity { components }` block(s) + // cook directly. For an extension, those are the components added on + // activation; its hooks + `requires` are handled below. for (prefab_entities) |e| { const eb = try self.buildEntity(e, diag_out); try entities.append(self.gpa, eb); @@ -513,6 +573,7 @@ const Builder = struct { const archetypes = try self.groupArchetypes(entities.items); const content_version = try self.versionFromNode(pd.version, diag_out); + const hooks = if (pd.relation == .extends) try self.buildExtendsHooks(pd, base_resolver, diag_out) else &[_]format.HookSet{}; return .{ .strings = try self.a().dupe([]const u8, self.strings.items), @@ -520,10 +581,54 @@ const Builder = struct { .resources = &.{}, .archetypes = archetypes, .content_version = content_version, + .hooks = hooks, .arena = self.arena, }; } + /// Cook an `extends` prefab's hooks (M1.0.6 E5): validate `requires` against + /// the base `X` (each required component must be present in `X.prefab.bin`), + /// then render `on_attach`/`on_detach` to canonical Etch **text** (interned + /// into the model strings; `null` if the hook is absent). Returns a one-element + /// `HookSet` slice (`hook_count == 1` for an `extends` `.prefab.bin`). + fn buildExtendsHooks(self: *Builder, pd: ast_mod.PrefabDecl, base_resolver: ?BaseResolver, diag_out: ?*[]const u8) CookError![]format.HookSet { + try self.validateRequires(pd, base_resolver, diag_out); + const hooks = try self.a().alloc(format.HookSet, 1); + hooks[0] = .{ + .on_attach = if (pd.has_on_attach) try self.renderHook(pd.on_attach_start, pd.on_attach_len, diag_out) else null, + .on_detach = if (pd.has_on_detach) try self.renderHook(pd.on_detach_start, pd.on_detach_len, diag_out) else null, + }; + return hooks; + } + + /// Render a hook statement-run to canonical Etch text and intern it into the + /// model strings, returning its index. The loader (E6) re-parses this text. + fn renderHook(self: *Builder, body_start: u32, body_len: u32, diag_out: ?*[]const u8) CookError!u32 { + const text = descriptor.renderStmtRunAlloc(self.gpa, self.ast, body_start, body_len) catch |e| switch (e) { + error.OutOfMemory => return error.OutOfMemory, + else => return fail(diag_out, error.HookRenderFailed, "extension hook body could not be rendered to text"), + }; + defer self.gpa.free(text); + return self.internString(text); + } + + /// Validate an `extends` prefab's `requires C1, C2, …`: each required component + /// must be declared by the base `X` (`X.prefab.bin`, read via the resolver). + /// `error.RequiresNotSatisfied` otherwise (`etch-grammar.md` §15 l.1634). + fn validateRequires(self: *Builder, pd: ast_mod.PrefabDecl, base_resolver: ?BaseResolver, diag_out: ?*[]const u8) CookError!void { + if (pd.requires_len == 0) return; + const base_name = self.ast.strings.slice(pd.relation_target); + const resolver = base_resolver orelse return fail(diag_out, error.BasePrefabMissing, "`extends … requires` needs a base-prefab resolver to validate against X"); + const base_bytes = resolver.resolve(base_name) orelse return fail(diag_out, error.BasePrefabMissing, "`extends` references a base prefab the resolver does not know"); + var acc = accessor.Accessor.open(base_bytes) catch return fail(diag_out, error.BasePrefabCorrupt, "base prefab .prefab.bin failed to open (magic/version)"); + if (!acc.verifyHash()) return fail(diag_out, error.BasePrefabCorrupt, "base prefab .prefab.bin content hash mismatch"); + var ri: u32 = 0; + while (ri < pd.requires_len) : (ri += 1) { + const req = self.ast.strings.slice(self.ast.prefab_requires.items[pd.requires_start + ri]); + if (!baseHasComponent(acc, req)) return fail(diag_out, error.RequiresNotSatisfied, "`extends … requires` a component the base prefab does not declare"); + } + } + /// Reconstruct the base prefab's flattened entities from its cooked /// `.prefab.bin` (read via the accessor) as `EntityBuild`s, interning their /// names/uuids into the variant's model tables. Each on-disk schema **name** @@ -1063,6 +1168,16 @@ const Builder = struct { } }; +/// Whether the cooked base prefab `acc` declares a component named `name` (used +/// by `extends … requires` validation — scans the on-disk Schema Registry). +fn baseHasComponent(acc: accessor.Accessor, name: []const u8) bool { + var i: u32 = 0; + while (i < acc.schemaCount()) : (i += 1) { + if (std.mem.eql(u8, acc.schema(i).name, name)) return true; + } + return false; +} + /// Index of `id` in `ids`, or null. Linear (component counts per entity are tiny). fn indexOfId(ids: []const ComponentId, id: ComponentId) ?usize { for (ids, 0..) |x, i| if (x == id) return i; diff --git a/tests/scene/extensions_test.zig b/tests/scene/extensions_test.zig index d554cb8..d48bb32 100644 --- a/tests/scene/extensions_test.zig +++ b/tests/scene/extensions_test.zig @@ -10,9 +10,37 @@ const std = @import("std"); const weld_etch = @import("weld_etch"); +const weld_core = @import("weld_core"); const parser = weld_etch.parser; const descriptor = weld_etch.descriptor; +const scene_cook = weld_etch.scene_cook; +const scene = weld_core.scene; +const Accessor = scene.accessor.Accessor; + +/// One-entry in-process base-prefab resolver (for `extends`/`of`/`instance`). +const OneResolver = struct { + name: []const u8, + bytes: []const u8, + fn resolve(ctx: *anyopaque, name: []const u8) ?[]const u8 { + const self: *OneResolver = @ptrCast(@alignCast(ctx)); + return if (std.mem.eql(u8, name, self.name)) self.bytes else null; + } + fn base(self: *OneResolver) scene_cook.BaseResolver { + return .{ .ctx = self, .resolveFn = OneResolver.resolve }; + } +}; + +// A mono-entity base prefab carrying `Health` (the `requires` target). +const base_character = + \\component Health { current: i32 = 100, max: i32 = 100 } + \\prefab "BaseCharacter" { + \\ entity "root" { + \\ uuid: "7b3e2f1a-42a3-4f2b-8c9d-a3f2b1c98d4e" + \\ Health { current: 100, max: 100 } + \\ } + \\} +; test "entity and instance extensions clauses parse to AST names" { const gpa = std.testing.allocator; @@ -114,3 +142,118 @@ test "descriptors carry the extensions names" { }; try std.testing.expect(saw_scene); } + +test "extends prefab cooks with components, hooks and requires" { + const gpa = std.testing.allocator; + + // Cook the base (carries Health), wire it as the `extends`/`requires` target. + var base = try scene_cook.cookPrefab(gpa, base_character, null, null); + defer base.deinit(gpa); + const base_bytes = try scene.writer.write(gpa, base.model, &base.registry); + defer gpa.free(base_bytes); + var resolver = OneResolver{ .name = "BaseCharacter", .bytes = base_bytes }; + + const combat = + \\component Health { current: i32 = 100, max: i32 = 100 } + \\component Weapon { damage: i32 = 10 } + \\prefab "CombatModule" extends "BaseCharacter" requires Health { + \\ entity "mod" { + \\ uuid: "9c4f3a2b-1e7d-4a5c-b8e9-f4d2c3a1b5e6" + \\ Weapon { damage: 25 } + \\ } + \\ on_attach { entity.get_mut(Health).max += 50 } + \\ on_detach { entity.get_mut(Health).max -= 50 } + \\} + ; + var cooked = try scene_cook.cookPrefab(gpa, combat, resolver.base(), null); + defer cooked.deinit(gpa); + const bytes = try scene.writer.write(gpa, cooked.model, &cooked.registry); + defer gpa.free(bytes); + + var acc = try Accessor.open(bytes); + try std.testing.expect(acc.verifyHash()); + try std.testing.expectEqual(@as(u16, 2), acc.header.version); // format v2 + + // The added component (Weapon) is in an archetype. + try std.testing.expectEqual(@as(u32, 1), acc.archetypeCount()); + try std.testing.expectEqualStrings("Weapon", acc.schema(acc.archetype(0).schemaIndex(0)).name); + + // Hooks: one set, both present, rendered as Etch text. + try std.testing.expectEqual(@as(u32, 1), acc.hookCount()); + const h = acc.hook(0); + try std.testing.expect(h.on_attach != null); + try std.testing.expect(h.on_detach != null); + try std.testing.expect(std.mem.indexOf(u8, h.on_attach.?, "Health") != null); + + // No entity extensions in a prefab.bin. + try std.testing.expectEqual(@as(u32, 0), acc.extensionsCount()); +} + +test "extends requires a component the base lacks is rejected" { + const gpa = std.testing.allocator; + var base = try scene_cook.cookPrefab(gpa, base_character, null, null); + defer base.deinit(gpa); + const base_bytes = try scene.writer.write(gpa, base.model, &base.registry); + defer gpa.free(base_bytes); + var resolver = OneResolver{ .name = "BaseCharacter", .bytes = base_bytes }; + + const bad = + \\component Mana { current: i32 = 0 } + \\prefab "MagicModule" extends "BaseCharacter" requires Mana { + \\ entity "mod" { uuid: "00000000-0000-0000-0000-0000000000e1" Mana { current: 30 } } + \\} + ; + var diag: []const u8 = ""; + try std.testing.expectError(error.RequiresNotSatisfied, scene_cook.cookPrefab(gpa, bad, resolver.base(), &diag)); +} + +test "scene extensions clause populates the Entity Extensions + Prefab ID tables" { + const gpa = std.testing.allocator; + + // A scene with two entities, each activating extensions by name. No prefab + // resolution is needed for the clause itself (it is by-name, like parent:). + const src = + \\component Health { current: i32 = 100 } + \\scene "S" { + \\ entity "A" { + \\ uuid: "00000000-0000-0000-0000-0000000000a1" + \\ extensions: ["CombatModule", "MerchantModule"] + \\ Health { current: 100 } + \\ } + \\ entity "B" { + \\ uuid: "00000000-0000-0000-0000-0000000000b2" + \\ extensions: ["CombatModule"] + \\ Health { current: 80 } + \\ } + \\} + ; + var cooked = try scene_cook.cook(gpa, src, null); + defer cooked.deinit(gpa); + const bytes = try scene.writer.write(gpa, cooked.model, &cooked.registry); + defer gpa.free(bytes); + + var acc = try Accessor.open(bytes); + try std.testing.expect(acc.verifyHash()); + + // Two entities have active extensions. + try std.testing.expectEqual(@as(u32, 2), acc.extensionsCount()); + + // Prefab ID Table is deduplicated: CombatModule + MerchantModule = 2. + try std.testing.expectEqual(@as(u32, 2), acc.prefabIdCount()); + + // Entity A: two extensions; both names resolve through the Prefab ID Table. + // (Entries are in entity-encounter order; A first.) + const ea = acc.extension(0); + try std.testing.expectEqual(@as(u8, 0xa1), acc.uuidAt(ea.uuid_ordinal)[15]); + try std.testing.expectEqual(@as(u32, 2), ea.extension_count); + try std.testing.expectEqualStrings("CombatModule", acc.prefabName(ea.extensionId(0))); + try std.testing.expectEqualStrings("MerchantModule", acc.prefabName(ea.extensionId(1))); + + // Entity B: one extension, sharing the deduplicated CombatModule id with A. + const eb = acc.extension(1); + try std.testing.expectEqual(@as(u32, 1), eb.extension_count); + try std.testing.expectEqual(ea.extensionId(0), eb.extensionId(0)); // same Prefab ID slot + + // No hooks in a .scene.bin. + try std.testing.expectEqual(@as(u32, 0), acc.hookCount()); +} diff --git a/tests/scene/prefab_cook_test.zig b/tests/scene/prefab_cook_test.zig index c3d4c08..fc19e9e 100644 --- a/tests/scene/prefab_cook_test.zig +++ b/tests/scene/prefab_cook_test.zig @@ -142,21 +142,15 @@ test "prefab re-cook is byte-identical" { try std.testing.expectEqualSlices(u8, b1, b2); } -test "cookPrefab rejects extends (M1.0.6 E5) and a scene source" { +test "cookPrefab rejects a scene source" { + // `extends` is COOKED as of M1.0.6 E5 (see tests/scene/extensions_test.zig); + // here we only assert a `.prefab.etch` holding a `scene` is rejected. const gpa = std.testing.allocator; - const extends_src = - \\component Health { current: i32 = 100, max: i32 = 100 } - \\prefab "CombatModule" extends "BaseCharacter" requires Health { - \\ entity "root" { uuid: "00000000-0000-0000-0000-000000000001" Health { max: 150 } } - \\} - ; - var diag: []const u8 = ""; - try std.testing.expectError(error.ExtendsUnsupported, scene_cook.cookPrefab(gpa, extends_src, null, &diag)); - const scene_src = \\component Health { current: i32 = 100, max: i32 = 100 } \\scene "S" { entity "e" { uuid: "00000000-0000-0000-0000-000000000002" Health {} } } ; + var diag: []const u8 = ""; try std.testing.expectError(error.SceneNotAllowedInPrefab, scene_cook.cookPrefab(gpa, scene_src, null, &diag)); } From d70f897ec307649191f2574574758a655a1a7404 Mon Sep 17 00:00:00 2001 From: Guy Senpai Date: Sun, 28 Jun 2026 14:26:52 +0200 Subject: [PATCH 11/15] docs(brief): record E6 on_attach execution blocker (re-scope signal) --- briefs/M1.0.6-prefabs-crossrefs-extensions.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/briefs/M1.0.6-prefabs-crossrefs-extensions.md b/briefs/M1.0.6-prefabs-crossrefs-extensions.md index 4a39ccb..108af8f 100644 --- a/briefs/M1.0.6-prefabs-crossrefs-extensions.md +++ b/briefs/M1.0.6-prefabs-crossrefs-extensions.md @@ -250,6 +250,7 @@ Sparse (entities with no active extension are absent). The `§4` note's `u8` cou - 2026-06-28 13:30 — Session 2 head. Applied the Claude.ai FROZEN amendment (E5 scope: `extensions:` clause grammatically defined + parser/AST/descriptors; `.prefab.bin` hooks section; D-E CONFIRMED) — commit `a8821b5`. Re-read the amended grammar intent (worked from Guy's in-message amendment spec — the re-uploaded `etch-grammar.md` did not reach the local `Downloads/` copy, still 07:27) + `engine-scene-serialization.md` §5/§5.5.2 + `etch-reference-part2.md` §30. Confirmed the D-E seam (`observers.zig`, `interp.zig:1018`, `descriptor.zig:1668`). - 2026-06-28 13:30 — E5 STOP (Case-2): the `.prefab.bin` hooks-section SHAPE is a frozen-format decision (header full, §4 reserved only two sections) — see Blockers. No E5 production code until the Claude.ai ruling. - 2026-06-28 15:30 — E5 (binary + cook) complete, post-ruling (SHAPE A, format_version 2, hooks=text). **format.zig**: `format_version` 1 → 2; `ExtModelEntry`/`HookSet` model types; `CookModel.ext_entries`/`prefab_id_table`/`hooks`. **writer.zig**: `writeExtensionsRegion` emits the SHAPE-A region @ `extensions_offset` (Entity Extensions Table + Prefab ID Table + hooks; component ids → file-local indices; hook refs = string-table offsets, 0=absent). **accessor.zig**: `extensionsCount`/`extension(i)` (variable-length walk), `prefabIdCount`/`prefabName(i)`, `hookCount`/`hook(i)`. **scene_cook.zig**: scene cook records `extensions:` → `ext_entries` + dedup `prefab_id_table` (`recordExtensions`/`prefabIdIndex`); `extends` prefab cook (`buildPrefab` no longer rejects extends) → components (the `entity{}` block) + `buildExtendsHooks` (renders `on_attach`/`on_detach` to text via `descriptor.renderStmtRunAlloc`, interned) + `validateRequires` (each `requires C` must be in X's `.prefab.bin`, `RequiresNotSatisfied`); removed `ExtendsUnsupported`, added `RequiresNotSatisfied`/`HookRenderFailed`. Tests `extensions_test.zig` (+3 cook tests, now 7): extends cooks with components/hooks/requires, requires-not-in-base rejected, scene `extensions:` populates the Entity Extensions + dedup Prefab ID tables. `prefab_cook_test` "rejects extends" → "rejects a scene source" (extends now cooked). Green Debug + ReleaseSafe; full `zig build test` 842/859 (17 skip, 0 fail, exit 0); fmt + lint clean. **Residual (journaled): the §30.5 additive-conflict WARNING** (two extensions adding the same component to X) is deferred — it needs a cook warning channel + cross-extension component-set comparison at scene cook; not tested by the acceptance set. **E6 (load) is next** — see the design note in the report. +- 2026-06-28 15:45 — E6 STOP (Case-2, 2nd blocker → re-scope signal): the `on_attach` execution mechanism at load assumes interp surface that does not exist (compile-once interp, no runtime snippet/text run, no prefab-on_attach compilation). Seam audit done; Tier-0 dispatch + component-add confirmed available. Options A/B/C in Blockers; recommend C. No E6 production code until the ruling. - 2026-06-28 14:30 — E5 (partial, unblocked chunk): the **`extensions:` clause** (Claude.ai gave the exact EBNF rule in-message; the re-uploaded grammar did not reach the local `Downloads/` copy). Implemented per that rule: **parser** (`parseExtensionsClause` + wired into `parseSceneEntity` after `parent`, `parseSceneInstance` after `uuid`; optional, by STRING_LITERAL name, trailing comma + empty list); **AST** (`SceneEntity`/`SceneInstance` gain `extensions_start`/`extensions_len` into the new `arena.scene_extensions` `StringId` run); **descriptors** (`SceneEntityDesc`/`SceneInstanceDesc` gain `extensions: []const []const u8`; `buildExtensionsRun` + free); **codegen** (`zig_codegen/lower.zig` `emitExtensionEntries` → the emitted `SceneEntityDesc`/`SceneInstanceDesc` literals carry `.extensions`). Test `tests/scene/extensions_test.zig` (4, step `test-extensions`): entity+instance clause → AST names, empty list + trailing comma, absent = empty run, descriptors carry names. Green Debug + ReleaseSafe; full `zig build test` 839/856 (17 skip, 0 fail, exit 0); fmt + lint clean. **The cook/binary + load portions of E5/E6 remain STOPPED on the hooks-section shape ruling** (Blockers) — they are coupled to the `extensions_offset` section layout (A vs B). - 2026-06-28 12:45 — E4 STOP — **session 1 / session 2 boundary.** Pushed, awaiting Guy's review + GO. Per the gate plan, the next message relays the KB amendment lot (grammar `extensions:` + the scene-serialization §5 / §30.4 doc drifts) and the FROZEN brief amendment for E5 before session 2 starts. @@ -260,6 +261,7 @@ Sparse (entities with no active extension are absent). The `§4` note's `u8` cou ## Blockers encountered +- **[E6-opening, Case-2 — `on_attach` execution mechanism at load] STOP, return to Claude.ai. This is the 2nd distinct blocker in the milestone → per § Notes, a re-scope signal.** The ruling's approach ("E6 : exécuter on_attach au load = ré-parser+interpréter le texte via le bridge etch") assumes an Etch-bridge entry that **does not exist**. Seam audit (against HEAD): the interpreter (`src/etch/interp.zig`) is **compile-once** — `compile(gpa, ast, world)` turns a whole parsed `AstArena` into pre-compiled `RuleDesc`s (`:753`); `runObserverBody` (`:1048`) runs a **pre-compiled** body by index; **there is NO entry to parse/compile/run a statement-run from source TEXT at runtime**, and the interp does not compile `prefab extends` `on_attach` declarations into runnable bodies at all. The `.prefab.bin` hooks are canonical Etch **text**; running them at load therefore needs **new interpreter surface** that the ruling/brief did not scope. The Tier-0 dispatch seam to mirror IS confirmed (`observers.zig` `ObserverFn`/`registerOnSpawned`/`dispatchOnSpawned` + `interp.zig:1018` trampoline + `World.addComponentDynamic` for the component add) — only the **hook execution** is blocked. **Options for Claude.ai:** **(A)** add an interp runtime entry `runStatementText(world, entity, text)` — parse the hook text → compile a transient rule body → bind `entity` → run via `execStmt` (the literal "re-parse+interpret" of the ruling; new but bounded interp surface; intricate — transient compile against an already-populated world, idempotent re-registration, deferred buffer). **(B)** compile `prefab extends` `on_attach` at interp **startup** (from the `.etch` AST) + an `activate_extension(world, entity, name)` dispatched **by name** via a Tier-0 callback (mirrors observer dispatch, no runtime text compile; the `.prefab.bin` hook text becomes non-executing self-containedness metadata; needs the interp to process prefab decls + a name→body registry). **(C)** land E6's **Tier-0** parts now — `applyExtensions` (read the Entity Extensions Table, resolve each extension's `.prefab.bin` via a runtime resolver, **add its components** via `addComponentDynamic`) + register the `on_attach` **Tier-0 callback seam** into the `World` (fired by the loader, passing `(world, entity, on_attach_text)`) + the `activate_extension` skeleton — and **defer the real Etch-text execution** (the conscious debt) to a follow-up, with the E6 test exercising the seam via a Tier-0 callback. **Recommendation:** (C) for M1.0.6 scope discipline (ships the table + component-add + dispatch seam, all Tier-0 and tested; the Etch on_attach execution is the acknowledged debt), unless you want (A) built now. No E6 production code until the ruling. — pending - **[E5-opening, Case-2 — `.prefab.bin` hooks-section SHAPE] STOP, return to Claude.ai.** Per the E5 instruction ("tranche si trivial, STOP + retour Claude.ai si non"). Findings: (1) `descriptor.zig:1668 renderStmtRunAlloc` renders `on_attach`/`on_detach` to **canonical Etch TEXT** (the `Prefab.on_attach`/`on_detach` descriptor fields are `[]const u8` text, not bytecode) — so the hooks *content* is trivial: two strings per `extends` prefab, storable as string-table refs. (2) The hooks *location* is the hard part and is a **frozen-format decision**: the 64-byte `SceneHeader` is **full** (`@44 extensions_offset`, `@48 crossrefs_offset`, `@52 _reserved` = hash-alignment padding, `@56 hash`) — there is **no free header offset** for a `hooks_offset`, and §4 (frozen M1.0.4) reserved exactly **two** sections (extensions, crossrefs), not a third. Viable shapes, both touching the frozen format contract: **(A) — recommended** — locate the hooks **inside the `extensions_offset` region**, appended after the Entity Extensions Table + Prefab ID Table (D-C): `extensions_offset` section becomes `[ext_count][entries][prefab_id_count][refs][hook_count][hooks…]` (self-delimiting, count-prefixed; no header change; a scene with no extensions = `[0][0][0]`, an `extends` prefab = `[0][0][n][…]`). **(B)** — repurpose `_reserved @52` as `hooks_offset` (header-level; `0` = absent → backward-compat without a version bump; hash stays @56). Either way, the D-C Entity Extensions Table layout already extends past M1.0.4's bare `[0]` at `extensions_offset`, so a **`format_version` 1 → 2 bump** is likely warranted (old v1 files → `BadVersion`, must be re-cooked — acceptable, the cook is deterministic). **Questions for Claude.ai:** (i) shape A vs B? (ii) `format_version` 1 → 2 acceptable, or must v1 files stay loadable? (iii) confirm hooks stored as rendered Etch **text** (string-table refs), not a new bytecode format (which would be out of scope). No production code written for E5 until the ruling lands. — pending - **[E1 finding, affects E5 only — not E2/E4] The scene-side `extensions: [...]` clause is NOT parsed.** The brief E1 task asks to "confirm `prefab_decl`/`instance of`/`extensions:` are already parsed (M0.8)". Result: `prefab_decl` (relation `of`/`extends`, `requires`, `on_attach`/`on_detach`) ✓ and `instance of` ✓ are parsed, but the `entity { extensions: [...] }` / `instance of … { extensions: [...] }` clause (`engine-scene-serialization.md` §3, §5.5.2) is **not** — neither in the grammar EBNF §15 (`entity_decl`/`instance_decl` carry only `uuid` + `parent`), nor in the M0.8 parser (`parseSceneEntity` `parser.zig:3364`, `parseSceneInstance` `:3407` parse only `uuid`/`parent` then require TYPE_IDENT-led `component_instance`; a leading lowercase `extensions` ident → parse error). E5's scope ("a scene's `extensions: ["Z", …]` clause → Entity Extensions Table") therefore needs a grammar + parser + AST addition for the clause, which is **not** in the brief's "Files to create or modify" list (no `parser.zig`/`ast.zig`/grammar entries). **Design/scope question for Guy (Case-2), to resolve before E5:** (a) extend grammar §15 + `parseSceneEntity`/`parseSceneInstance` + `SceneEntity`/`SceneInstance` AST + descriptors to carry `extensions: [string]` (a bounded, additive grammar change — recommended, it is the only way to author active extensions in a scene); or (b) defer scene-authored extension activation and land only the binary table + loader `applyExtensions` in M1.0.6 (table populated by a test fixture / future authoring path), keeping the parser untouched. This does **not** block E2–E4 (prefab cook, flattening, cross-refs). — pending From 255f0f0d1ddab770321550acdc0b98d052c5666e Mon Sep 17 00:00:00 2001 From: Guy Senpai Date: Sun, 28 Jun 2026 16:29:05 +0200 Subject: [PATCH 12/15] docs(brief): re-scope E6 to activation infra (hook exec -> M1.0.9) --- briefs/M1.0.6-prefabs-crossrefs-extensions.md | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/briefs/M1.0.6-prefabs-crossrefs-extensions.md b/briefs/M1.0.6-prefabs-crossrefs-extensions.md index 108af8f..d8483a6 100644 --- a/briefs/M1.0.6-prefabs-crossrefs-extensions.md +++ b/briefs/M1.0.6-prefabs-crossrefs-extensions.md @@ -105,15 +105,16 @@ Sparse (entities with no active extension are absent). The `§4` note's `u8` cou - `format.zig`/`writer.zig`/`accessor.zig`: `ExtensionEntry` + Prefab ID Table (D-C) + the prefab hooks section — structs, emit, getters. - → **STOP** — push, await review + GO. -**E6 — Extensions: load (`applyExtensions` + `on_attach` dispatch).** -- `loader.zig`: `applyExtensions` pass, inserted **after `loadResources`, before `dispatchSpawnLifecycle`**: per entity with active extensions, in table order → dynamic add-component (defaults from the extension's `.prefab.bin`) then fire `on_attach` via the Tier-0 callback (D-E). Replace `assert(extensions_offset==0)` with the real read. -- Etch side: the `activate_extension` runtime entry (reused by the loader path) + the `on_attach` dispatch callback registered into Tier-0 (mirroring observer dispatch). `on_detach`/`deactivate_extension` runtime entry for symmetry. +**E6 — Extensions: load = activation INFRA (NOT hook execution).** *(Re-scoped by Claude.ai: `on_attach`/`on_detach` **execution** moved to a new milestone **M1.0.9** — the interp is compile-once from the AST with no runtime-execution surface, and the text-vs-bytecode decision is unsettled; see Out-of-scope + Accepted deviations.)* +- `loader.zig`: `applyExtensions` pass, inserted **after `loadResources`, before `dispatchSpawnLifecycle`**: per entity with active extensions (Entity Extensions Table), in table order → resolve each extension by name (Prefab ID Table) → load its `.prefab.bin` (runtime extension resolver) → **add its components** via `addComponentDynamic` (defaults from the extension's `.prefab.bin`) → fire the **`on_attach` Tier-0 dispatch seam** after the add. Replace `assert(extensions_offset==0)` with the real read. +- The `on_attach` dispatch SEAM: a callback registerable in the `World` with signature `(world, entity, hook_ref/text)`, fired by the loader after the component add — `loader.zig` **never calls the VM directly**. Plus an `activate_extension` runtime-entry **skeleton** (same dispatch path). The `on_attach`/`on_detach` TEXT cooked in E5 stays as the metadata **M1.0.9** consumes — E6 does **not** execute it. - Canonical final load order: `instantiate` → `resolveCrossRefs` (E4) → `loadResources` → `applyExtensions` (E6) → `dispatchSpawnLifecycle`. - `CLAUDE.md` §3.4 update on the branch (`docs(claude-md): update for M1.0.6`): État courant row (scene track complete), Tags row `v0.10.6-prefabs-crossrefs-extensions`, the cross-references / extensions binary shapes now frozen, the `FieldKind.entity_` addition, and the Last-updated date. - → **STOP** — push, await review + GO. ## Out-of-scope +- **`on_attach`/`on_detach` HOOK EXECUTION** *(Claude.ai re-scope → new milestone **M1.0.9**)* — actually compiling/running the loaded Etch hook code. The interpreter is compile-once from the AST (`compile(ast, world)` → pre-compiled `RuleDesc`s); no runtime-execution surface exists for source text, and the text-vs-bytecode storage decision for executable hooks is unsettled. M1.0.6 E6 ships only the activation **infra** (component add + the `on_attach` Tier-0 dispatch **seam** + `activate_extension` skeleton); the E5-cooked hook text is the metadata M1.0.9 consumes. - **`.prefab.bin` runtime loading / `spawn("Prefab")` dynamic instantiation** — the scene cook reads the `.prefab.bin` to flatten; spawning a prefab into a live world at runtime is a distinct, additive use. Not here. - **Resource→entity references** — `Entity` fields on resources. `.entity_` is component-only in M1.0.6 (D-A); resource-side is a future additive milestone (its own side table + `getMutResource` patch). - **Cross-scene references** — resolving a `target` UUID not present in the loaded scene. Needs a global identity registry; out of M1.0. Intra-scene only (`error.UnresolvedCrossRef` at cook for an absent target). @@ -168,7 +169,7 @@ Sparse (entities with no active extension are absent). The `§4` note's `u8` cou - `tests/scene/prefab_cook_test.zig` — `test "standalone prefab cooks and reads back"` — a `.prefab.etch` cooks to `.prefab.bin`; the accessor reads its components/entities. `test "variant prefab resolves of-chain"` — `prefab "Y" of "X"` flattens X + Y overrides. - `tests/scene/prefab_flatten_test.zig` — `test "instance of expands to the prefab's components"` — an `instance of "P"` with no override yields the same archetype + bytes as the hand-authored equivalent. `test "per-field overrides apply over the prefab"` — an overridden field reads the instance value, non-overridden fields read the prefab value. `test "N instances of one prefab"` — N instances load as N entities. - `tests/scene/crossref_test.zig` — `test "entity field resolves to the target handle on load"` — a `Comp.ref: Entity` pointing at another scene entity reads back that entity's `EntityId`. `test "unset entity field is EntityId.dead"`. `test "cook rejects a reference to an absent entity"` — `error.UnresolvedCrossRef`. -- `tests/scene/extensions_test.zig` — `test "extends prefab cooks with hooks and requires"`. `test "scene extensions clause populates the table"` — round-trips a non-empty Entity Extensions Table + Prefab ID Table. `test "load applies extension components and fires on_attach"` — after load the entity carries the extension's components and the `on_attach` effect is observable (e.g. a boosted `Health.max`). +- `tests/scene/extensions_test.zig` — `test "extends prefab cooks with hooks and requires"`. `test "scene extensions clause populates the table"` — round-trips a non-empty Entity Extensions Table + Prefab ID Table. `test "load applies extension components and the on_attach seam fires"` *(re-scoped)* — after load the entity carries the extension's components, and the Tier-0 `on_attach` dispatch seam fired (asserted via a **test callback** counter/flag, **not** an imperative Etch effect — the real `on_attach` execution / boosted `Health.max` is **M1.0.9**). - `tests/scene/prefab_integration_test.zig` — `test "scene with prefab instances, a cross-ref and an extension loads end to end"` — entity count, an overridden field value, a resolved reference, and an `on_attach` effect all correct. ### Benchmarks @@ -199,7 +200,7 @@ Sparse (entities with no active extension are absent). The `§4` note's `u8` cou - **Format source of truth = code, not §4 prose.** Re-read `src/core/scene/{format,writer,accessor,loader}.zig` (frozen M1.0.4/M1.0.5) before extending them. The reserved offsets `@44`/`@48` and the loader's two asserts are the exact bascule points. - **`.prefab.bin` is the same format as `.scene.bin`** (a prefab is a mini-scene). Reuse `writer`/`format`/`accessor` — do **not** invent a second binary format. The header's `content_version`/`entity_count` etc. apply unchanged. -- **`of` (variant) flattens at cook; `extends` (extension) materializes at load.** A variant is baked into the instance's archetype (static inheritance). An extension's components are added at load and its `on_attach` runs at load (dynamic composition). Per-field overrides in an instance apply to the **final** entity (base + flattened prefab); overrides targeting extension-added components are out-of-scope (see Out-of-scope). +- **`of` (variant) flattens at cook; `extends` (extension) materializes at load.** A variant is baked into the instance's archetype (static inheritance). An extension's components are added at load and its `on_attach` **dispatch seam** fires at load (the actual hook **execution** is M1.0.9 — see Out-of-scope). Per-field overrides in an instance apply to the **final** entity (base + flattened prefab); overrides targeting extension-added components are out-of-scope (see Out-of-scope). - **`.entity_` is the mirror of `.string_`/`.enum_`.** Those are resource-only (gated out of components); `.entity_` is component-only (gated out of resources). Same gate site (`fieldKindFromTypeName`), opposite `reg_kind`. - **Cross-ref slot is `EntityId.dead` on disk.** The 8-byte `Entity` column slot is a placeholder at cook; the side table (`CrossRefEntry`) carries the target; the loader patches it via `componentBytes` + `markComponentChangedDyn`. This mirrors how resource `string` slots are zeroed on disk with the string side table carrying the value. - **Tier discipline holds.** `loader.zig` (`weld_core`) never imports `weld_etch`; `on_attach` dispatch goes through a Tier-0 callback registered by the Etch bridge (D-E), exactly as `on_spawned` observer dispatch is wired. The crossref pass and the flattening live on the Etch cook side (`scene_cook.zig`, which imports `weld_core.scene`). @@ -256,6 +257,7 @@ Sparse (entities with no active extension are absent). The `§4` note's `u8` cou ## Accepted deviations +- 2026-06-28 (Claude.ai re-scope, session-2) — **E6 re-scoped: `on_attach`/`on_detach` HOOK EXECUTION → new milestone M1.0.9** (resolves the 2nd Case-2 blocker). The interp is compile-once from the AST with no runtime source-execution surface, and the text-vs-bytecode decision for executable hooks is unsettled — so executing the loaded hook code is its own milestone. M1.0.6 E6 = activation **infra** only: `applyExtensions` (read Entity Extensions Table → resolve by name → load extension `.prefab.bin` → `addComponentDynamic` the extension's components) + the `on_attach` **Tier-0 dispatch seam** (a `World`-registerable callback `(world, entity, hook_ref/text)`, fired by the loader after the add; `loader.zig` never calls the VM) + an `activate_extension` skeleton. Acceptance verified via a Tier-0 **test callback** (counter/flag), not an imperative Etch effect. The E5-cooked hook text is the metadata M1.0.9 consumes. KB (`engine-phase-1-plan.md` + `engine-scene-serialization.md`) updated by Guy. §30.5 additive-conflict warning: complete in E6 iff the cook diag channel supports a non-fatal warning, else flag. - 2026-06-28 (Claude.ai ruling, session-2) — **`.prefab.bin` hooks-section SHAPE resolved** (the E5-opening Case-2 blocker): **(A)** hooks live in the `extensions_offset` region, appended **after** the Entity Extensions Table + Prefab ID Table — no new header field, `_reserved @52` preserved, all extension serialization in one place (hooks only ever exist in an `extends` `.prefab.bin`, never a `.scene.bin` / `of` / standalone, so a top-level `hooks_offset` would be 0 almost everywhere). Region layout @ `extensions_offset` (self-delimiting): `ext_count:u32`, `[ext_count]{uuid_ordinal:u32, extension_count:u32, extension_ids:[…]u32}`, `prefab_id_count:u32`, `prefab_name_refs:[…]u32` (string-table offsets, dedup'd), `hook_count:u32`, `[hook_count]{on_attach_ref:u32, on_detach_ref:u32}` (string-table offsets; `0` = absent). Filling: `.scene.bin` → `ext>0, ids>0, hooks=0`; `.prefab.bin extends` → `ext=0, ids=0, hooks=1`; none → `[0][0][0]`. `hook_count ∈ {0,1}` in M1.0.6. **format_version 1 → 2** (the reserved sections go from bare count-placeholder to full structure — a break; v1 → `BadVersion` → re-cook; no prod files in Phase 1). **Hooks = Etch TEXT** (`renderStmtRunAlloc`, string-table refs) — no bytecode format. E6 runs `on_attach` at load by re-parsing+interpreting that text via the Etch bridge (a conscious debt — no pre-compiled bytecode in M1.0.6); `on_detach` is stored for the runtime `deactivate_extension`, not run at load; dispatch via the Tier-0 callback pattern (D-E), `loader.zig` never calls the VM directly. `engine-scene-serialization.md` §4 (region + hooks + version 2) is Guy's to update at close. D-C stays valid; the hooks section is its acted extension. - 2026-06-28 (Claude.ai round-trip, session-2 head) — **E5 scope amended** (FROZEN SECTION edited via Claude.ai, per Guy's relay): (1) the `extensions: [STRING_LITERAL]` clause is now grammatically defined on `entity_decl` + `instance_decl` (`etch-grammar.md` re-uploaded, option a — resolves the E1 parse-gap blocker); E5 adds parser + AST (`SceneEntity`/`SceneInstance` `extensions: []StringId`) + descriptors, by-name. (2) The `.prefab.bin` format gains a **hooks section** carrying an extension's `on_attach`/`on_detach` statement-runs; on-disk shape is the E5-opening micro-decision. (3) **D-E status `confirmed-or-blocker` → CONFIRMED** — the Tier-0 callback seam exists (`observers.zig` + `interp.zig:1018`); hooks already render as statement-runs (`descriptor.zig:1668`); E6 adds an `on_attach` dispatch + a runtime `activate_extension`. Files added to the list: `src/etch/parser.zig`, `src/etch/ast.zig`, `src/etch/descriptor.zig`. NB: the re-uploaded `etch-grammar.md` did not reach the local `Downloads/` copy (still the 07:27 original); implemented against Guy's in-message amendment spec (clause shape + placement + by-name), consistent with the known staleness of that copy. From 5a5198a15f5ca6a7955d445035db51be9be4c463 Mon Sep 17 00:00:00 2001 From: Guy Senpai Date: Sun, 28 Jun 2026 16:38:58 +0200 Subject: [PATCH 13/15] feat(scene): activate extensions at load (components + seam) --- bench/scene_load_bench.zig | 2 +- briefs/M1.0.6-prefabs-crossrefs-extensions.md | 3 + build.zig | 3 + src/core/ecs/world.zig | 39 ++++ src/core/scene/loader.zig | 96 ++++++++-- src/etch/interp.zig | 2 +- tests/scene/crossref_test.zig | 2 +- tests/scene/extensions_test.zig | 105 +++++++++++ tests/scene/load_resources_test.zig | 2 +- tests/scene/load_roundtrip_test.zig | 8 +- tests/scene/prefab_flatten_test.zig | 2 +- tests/scene/prefab_integration_test.zig | 168 ++++++++++++++++++ 12 files changed, 413 insertions(+), 19 deletions(-) create mode 100644 tests/scene/prefab_integration_test.zig diff --git a/bench/scene_load_bench.zig b/bench/scene_load_bench.zig index a6c5311..8775c2a 100644 --- a/bench/scene_load_bench.zig +++ b/bench/scene_load_bench.zig @@ -87,7 +87,7 @@ fn timeOneLoad(gpa: std.mem.Allocator, io: std.Io, bytes: []const u8) !u64 { }); const t0 = std.Io.Clock.now(.awake, io); - var result = try loader.loadFromBytes(&world, gpa, bytes); + var result = try loader.loadFromBytes(&world, gpa, bytes, null); const t1 = std.Io.Clock.now(.awake, io); result.deinit(gpa); const elapsed = t0.durationTo(t1).nanoseconds; diff --git a/briefs/M1.0.6-prefabs-crossrefs-extensions.md b/briefs/M1.0.6-prefabs-crossrefs-extensions.md index d8483a6..49839cd 100644 --- a/briefs/M1.0.6-prefabs-crossrefs-extensions.md +++ b/briefs/M1.0.6-prefabs-crossrefs-extensions.md @@ -251,6 +251,9 @@ Sparse (entities with no active extension are absent). The `§4` note's `u8` cou - 2026-06-28 13:30 — Session 2 head. Applied the Claude.ai FROZEN amendment (E5 scope: `extensions:` clause grammatically defined + parser/AST/descriptors; `.prefab.bin` hooks section; D-E CONFIRMED) — commit `a8821b5`. Re-read the amended grammar intent (worked from Guy's in-message amendment spec — the re-uploaded `etch-grammar.md` did not reach the local `Downloads/` copy, still 07:27) + `engine-scene-serialization.md` §5/§5.5.2 + `etch-reference-part2.md` §30. Confirmed the D-E seam (`observers.zig`, `interp.zig:1018`, `descriptor.zig:1668`). - 2026-06-28 13:30 — E5 STOP (Case-2): the `.prefab.bin` hooks-section SHAPE is a frozen-format decision (header full, §4 reserved only two sections) — see Blockers. No E5 production code until the Claude.ai ruling. - 2026-06-28 15:30 — E5 (binary + cook) complete, post-ruling (SHAPE A, format_version 2, hooks=text). **format.zig**: `format_version` 1 → 2; `ExtModelEntry`/`HookSet` model types; `CookModel.ext_entries`/`prefab_id_table`/`hooks`. **writer.zig**: `writeExtensionsRegion` emits the SHAPE-A region @ `extensions_offset` (Entity Extensions Table + Prefab ID Table + hooks; component ids → file-local indices; hook refs = string-table offsets, 0=absent). **accessor.zig**: `extensionsCount`/`extension(i)` (variable-length walk), `prefabIdCount`/`prefabName(i)`, `hookCount`/`hook(i)`. **scene_cook.zig**: scene cook records `extensions:` → `ext_entries` + dedup `prefab_id_table` (`recordExtensions`/`prefabIdIndex`); `extends` prefab cook (`buildPrefab` no longer rejects extends) → components (the `entity{}` block) + `buildExtendsHooks` (renders `on_attach`/`on_detach` to text via `descriptor.renderStmtRunAlloc`, interned) + `validateRequires` (each `requires C` must be in X's `.prefab.bin`, `RequiresNotSatisfied`); removed `ExtendsUnsupported`, added `RequiresNotSatisfied`/`HookRenderFailed`. Tests `extensions_test.zig` (+3 cook tests, now 7): extends cooks with components/hooks/requires, requires-not-in-base rejected, scene `extensions:` populates the Entity Extensions + dedup Prefab ID tables. `prefab_cook_test` "rejects extends" → "rejects a scene source" (extends now cooked). Green Debug + ReleaseSafe; full `zig build test` 842/859 (17 skip, 0 fail, exit 0); fmt + lint clean. **Residual (journaled): the §30.5 additive-conflict WARNING** (two extensions adding the same component to X) is deferred — it needs a cook warning channel + cross-extension component-set comparison at scene cook; not tested by the acceptance set. **E6 (load) is next** — see the design note in the report. +- 2026-06-28 16:30 — E6 (activation infra, per the re-scope) implemented. **world.zig**: the `on_attach` Tier-0 dispatch seam — `ExtensionAttachFn` (`(ctx, *World, entity, extension_name, on_attach_text)`), `World.attach_hook` field, `registerOnAttach`/`dispatchOnAttach` (fires the registered callback; no-op if none). **loader.zig**: `ExtensionResolver` (runtime name → extension `.prefab.bin` bytes); `loadFromBytes`/`loadScene` gain an `ext_resolver: ?ExtensionResolver` param; `applyExtensions` pass (after `loadResources`, before `dispatchSpawnLifecycle`) reads the Entity Extensions Table, resolves each extension by name (Prefab ID Table), and calls `activateExtension` — which opens the extension `.prefab.bin`, adds its mono-entity components via `addComponentDynamic` (size-validated; a component the entity already has → `ExtensionComponentConflict`, not the dynamic-add assert), then fires the `on_attach` seam with the cooked hook text. `activateExtension` is the shared path the runtime `activate_extension` entry will reuse. The old `assert(extensions_offset==0)` is gone (the region is read). Canonical load order now `instantiate → resolveCrossRefs → loadResources → applyExtensions → dispatchSpawnLifecycle`. All 9 `loadFromBytes`/`loadScene` call sites updated (`, null`). Tests: `extensions_test.zig` +1 (now 8) "load applies extension components and the on_attach seam fires" (Weapon added; seam fired once with name+text; **M1.0.9 boundary asserted: `Health.max` stays 100 — hook NOT executed**); new `prefab_integration_test.zig` capstone (prefab ×2 instances + per-field override + cross-ref + extension, cook→load→ECS). Green Debug + ReleaseSafe; full `zig build test` 844/861 (17 skip, 0 fail, exit 0); fmt + lint clean. +- 2026-06-28 16:30 — **§30.5 additive-conflict warning: SIGNAL (per the re-scope note).** The cook's diagnostic channel is a single `diag_out: ?*[]const u8` set-on-error-then-return — there is **no non-fatal warning channel** (no way to emit a warning and continue). A cross-extension conflict warning (two extensions of X adding the same component) therefore needs **new warning infra** (a warnings list threaded through the cook + a way to surface it) — non-trivial. Per the instruction, flagged for a decision: **E6 (build the warning infra) vs M1.0.9 (with hook execution)**. Note: the loader DOES hard-error a runtime conflict (`ExtensionComponentConflict`) when two activated extensions actually collide on an entity — so the unsafe case is caught at load; only the *compile-time warning* is deferred. **Pending Guy's call.** +- 2026-06-28 16:30 — E6 STOP — pushed, awaiting review + GO. **CLAUDE.md update + milestone close (Notes de fin, Status CLOSED, PR) deferred to post-review** (the CLAUDE.md state/tags rows reference post-merge state, and "scene track complete" needs the M1.0.9 re-scope caveat). - 2026-06-28 15:45 — E6 STOP (Case-2, 2nd blocker → re-scope signal): the `on_attach` execution mechanism at load assumes interp surface that does not exist (compile-once interp, no runtime snippet/text run, no prefab-on_attach compilation). Seam audit done; Tier-0 dispatch + component-add confirmed available. Options A/B/C in Blockers; recommend C. No E6 production code until the ruling. - 2026-06-28 14:30 — E5 (partial, unblocked chunk): the **`extensions:` clause** (Claude.ai gave the exact EBNF rule in-message; the re-uploaded grammar did not reach the local `Downloads/` copy). Implemented per that rule: **parser** (`parseExtensionsClause` + wired into `parseSceneEntity` after `parent`, `parseSceneInstance` after `uuid`; optional, by STRING_LITERAL name, trailing comma + empty list); **AST** (`SceneEntity`/`SceneInstance` gain `extensions_start`/`extensions_len` into the new `arena.scene_extensions` `StringId` run); **descriptors** (`SceneEntityDesc`/`SceneInstanceDesc` gain `extensions: []const []const u8`; `buildExtensionsRun` + free); **codegen** (`zig_codegen/lower.zig` `emitExtensionEntries` → the emitted `SceneEntityDesc`/`SceneInstanceDesc` literals carry `.extensions`). Test `tests/scene/extensions_test.zig` (4, step `test-extensions`): entity+instance clause → AST names, empty list + trailing comma, absent = empty run, descriptors carry names. Green Debug + ReleaseSafe; full `zig build test` 839/856 (17 skip, 0 fail, exit 0); fmt + lint clean. **The cook/binary + load portions of E5/E6 remain STOPPED on the hooks-section shape ruling** (Blockers) — they are coupled to the `extensions_offset` section layout (A vs B). - 2026-06-28 12:45 — E4 STOP — **session 1 / session 2 boundary.** Pushed, awaiting Guy's review + GO. Per the gate plan, the next message relays the KB amendment lot (grammar `extensions:` + the scene-serialization §5 / §30.4 doc drifts) and the FROZEN brief amendment for E5 before session 2 starts. diff --git a/build.zig b/build.zig index e0da62e..f0276a1 100644 --- a/build.zig +++ b/build.zig @@ -445,6 +445,9 @@ pub fn build(b: *std.Build) void { // M1.0.6 / E5 — `extensions:` clause parse + AST + descriptors. The // cook/binary + load portions land once the hooks-section shape unblocks. .{ .path = "tests/scene/extensions_test.zig", .scene = true, .dedicated_step = "test-extensions" }, + // M1.0.6 / E3+E4+E6 — capstone: prefab instances + per-field override + + // cross-ref + active extension in one scene, cook → load → ECS. + .{ .path = "tests/scene/prefab_integration_test.zig", .scene = true, .dedicated_step = "test-prefab-integration" }, // M1.0.5 / E2 — runtime loader `.scene.bin` → ECS: instantiate every // entity (component bytes verbatim), two-phase `on_spawned` (fires once // per entity, after all entities exist). `weld_core` only (no `.scene` diff --git a/src/core/ecs/world.zig b/src/core/ecs/world.zig index 9edc48e..8a71233 100644 --- a/src/core/ecs/world.zig +++ b/src/core/ecs/world.zig @@ -92,6 +92,22 @@ const FieldKind = registry_mod.FieldKind; const ResourceStore = resources_mod.ResourceStore; const EntityIdentityStore = entity_mod.EntityIdentityStore; +/// M1.0.6 E6 — the `on_attach` extension dispatch seam (D-E). A Tier-0 function +/// pointer the Etch bridge registers; the scene loader fires it after adding an +/// extension's components, passing the entity, the extension name, and the cooked +/// `on_attach` Etch source text (`null` if absent). M1.0.6 wires + fires the seam; +/// running the text is M1.0.9. +pub const ExtensionAttachFn = *const fn ( + ctx: ?*anyopaque, + world: *World, + entity: EntityId, + extension_name: []const u8, + on_attach_text: ?[]const u8, +) anyerror!void; + +/// A registered `on_attach` callback + its opaque context. +const AttachHook = struct { ctx: ?*anyopaque, func: ExtensionAttachFn }; + /// Top-level ECS world — single archetype list, shared identity, shared /// registry, shared resources. pub const World = struct { @@ -161,6 +177,13 @@ pub const World = struct { /// that don't exercise observers never pay the alloc cost. observer_registry: observers_mod.ObserverRegistry = .{}, + /// M1.0.6 E6 — the `on_attach` extension dispatch seam (D-E). A Tier-0 + /// callback the Etch bridge registers; the scene loader fires it after adding + /// an extension's components. `loader.zig` never calls the Etch VM directly — + /// it goes through this hook. **M1.0.6 wires + fires the seam only**; the + /// actual execution of `on_attach_text` (Etch code) is **M1.0.9**. + attach_hook: ?AttachHook = null, + pub fn init() World { return .{ .identity = EntityIdentityStore.init(), @@ -264,6 +287,22 @@ pub const World = struct { try self.observer_registry.dispatchOnSpawned(gpa, self, eid); } + /// M1.0.6 E6 — register the `on_attach` extension dispatch callback (the Etch + /// bridge supplies the real one; M1.0.6 tests supply a Tier-0 stand-in). One + /// hook per world (last registration wins). + pub fn registerOnAttach(self: *World, ctx: ?*anyopaque, callback: ExtensionAttachFn) void { + self.attach_hook = .{ .ctx = ctx, .func = callback }; + } + + /// M1.0.6 E6 — fire the `on_attach` seam for `entity`'s newly-activated + /// extension `extension_name`, passing the cooked `on_attach_text` (the Etch + /// hook source; `null` if the extension has no `on_attach`). No-op if no hook + /// is registered. The loader calls this after adding the extension's + /// components. Executing the text is M1.0.9 — here the seam just fires. + pub fn dispatchOnAttach(self: *World, entity: EntityId, extension_name: []const u8, on_attach_text: ?[]const u8) anyerror!void { + if (self.attach_hook) |h| try h.func(h.ctx, self, entity, extension_name, on_attach_text); + } + // ─── Component registration helpers ────────────────────────────────── /// Register a component whose layout is described at runtime. diff --git a/src/core/scene/loader.zig b/src/core/scene/loader.zig index 0136bae..a7ccbe3 100644 --- a/src/core/scene/loader.zig +++ b/src/core/scene/loader.zig @@ -61,6 +61,20 @@ pub const RemapError = error{ UnknownComponent, SchemaMismatch } || std.mem.Allo /// M1.0.4 cook never produces this — it is a defensive guard on external input. pub const StructureError = error{MalformedScene}; +/// Resolves an extension prefab name (from the scene's Prefab ID Table) to its +/// cooked `.prefab.bin` bytes at load (M1.0.6 E6) — the runtime twin of the +/// cook's `BaseResolver`. The bytes must outlive the load. Null = unknown name +/// (the loader errors `UnknownExtension`). Wired to a project/asset registry at +/// runtime; tests wire it to an in-process buffer. +pub const ExtensionResolver = struct { + ctx: *anyopaque, + resolveFn: *const fn (ctx: *anyopaque, name: []const u8) ?[]const u8, + + pub fn resolve(self: ExtensionResolver, name: []const u8) ?[]const u8 { + return self.resolveFn(self.ctx, name); + } +}; + /// Open a `.scene.bin` byte image: validate magic + version (via the accessor, /// read little-endian field-by-field — never a raw `@ptrCast` off an unaligned /// buffer), then verify the content hash. Returns a zero-copy `Accessor` @@ -168,17 +182,12 @@ fn uuidCount(acc: Accessor) u32 { /// (`StructureError`) for a structurally-invalid scene (e.g. an out-of-range /// parent ordinal), allocation failure, plus anything an `on_spawned` observer /// propagates (hence the open error set). -pub fn loadFromBytes(world: *World, gpa: std.mem.Allocator, bytes: []const u8) anyerror!LoadResult { +pub fn loadFromBytes(world: *World, gpa: std.mem.Allocator, bytes: []const u8, ext_resolver: ?ExtensionResolver) anyerror!LoadResult { const acc = try openVerified(bytes); const remap = try buildSchemaRemap(gpa, world, acc); defer gpa.free(remap); - // The extensions table is M1.0.6 E6's; until then a cook writes it empty (a - // zero count prefix). Loading a populated one would mean a newer cook — assert - // the contract. The cross-references table (M1.0.6 E4) IS read below. - std.debug.assert(readU32At(acc, acc.header.extensions_offset) == 0); - var spawned: std.ArrayListUnmanaged(EntityId) = .empty; errdefer spawned.deinit(gpa); var uuid_to_entity: UuidMap = .empty; @@ -193,8 +202,11 @@ pub fn loadFromBytes(world: *World, gpa: std.mem.Allocator, bytes: []const u8) a // Cross-references after every entity exists (a reference can point forward), // before resources + on_spawned so a rule sees fully-linked entities. try resolveCrossRefs(world, acc, remap, uuid_to_entity); - // Resources before phase 2 so an `on_spawned` rule can read scene resources. + // Resources before extensions/on_spawned so a hook/rule can read them. try loadResources(world, gpa, acc, remap, &res_strings); + // Extension activation (M1.0.6 E6): add each active extension's components + + // fire the `on_attach` seam. After resources, before `on_spawned`. + try applyExtensions(world, gpa, acc, uuid_to_entity, ext_resolver); try dispatchSpawnLifecycle(world, gpa, spawned.items); return .{ @@ -209,10 +221,10 @@ pub fn loadFromBytes(world: *World, gpa: std.mem.Allocator, bytes: []const u8) a /// `loadFromBytes` over the borrowed bytes. The returned `LoadResult` owns the /// mapping (`LoadResult.deinit` closes it). Adds `fs.Error` (open/map failure) /// to `loadFromBytes`'s error set. -pub fn loadScene(world: *World, gpa: std.mem.Allocator, path: []const u8) anyerror!LoadResult { +pub fn loadScene(world: *World, gpa: std.mem.Allocator, path: []const u8, ext_resolver: ?ExtensionResolver) anyerror!LoadResult { var mmap = try fs.mmapFile(gpa, path); errdefer mmap.close(); - var result = try loadFromBytes(world, gpa, mmap.bytes); + var result = try loadFromBytes(world, gpa, mmap.bytes, ext_resolver); result.mmap = mmap; return result; } @@ -294,6 +306,70 @@ fn resolveCrossRefs(world: *World, acc: Accessor, remap: []const ComponentId, uu } } +/// Extension activation (M1.0.6 E6) — for each entity in the Entity Extensions +/// Table, in table order, activate each of its extensions: resolve the extension +/// `.prefab.bin` by name (Prefab ID Table → `ExtensionResolver`), add its +/// components, and fire the `on_attach` Tier-0 seam. **No-op when the scene has no +/// active extensions** (so an extension-free scene needs no resolver). The actual +/// `on_attach` hook EXECUTION is M1.0.9 — here `dispatchOnAttach` only fires the +/// registered seam with the cooked hook text. +fn applyExtensions(world: *World, gpa: std.mem.Allocator, acc: Accessor, uuid_to_entity: UuidMap, ext_resolver: ?ExtensionResolver) !void { + const count = acc.extensionsCount(); + if (count == 0) return; + const ucount = uuidCount(acc); + const resolver = ext_resolver orelse return error.MissingExtensionResolver; + const pid_count = acc.prefabIdCount(); + + var i: u32 = 0; + while (i < count) : (i += 1) { + const e = acc.extension(i); + if (e.uuid_ordinal >= ucount) return error.MalformedScene; + const entity = uuid_to_entity.get(acc.uuidAt(e.uuid_ordinal).*) orelse return error.MalformedScene; + var j: u32 = 0; + while (j < e.extension_count) : (j += 1) { + const pid = e.extensionId(j); + if (pid >= pid_count) return error.MalformedScene; + const name = acc.prefabName(pid); + const ext_bytes = resolver.resolve(name) orelse return error.UnknownExtension; + try activateExtension(world, gpa, entity, name, ext_bytes); + } + } +} + +/// Activate one extension on one entity (M1.0.6 E6) — the shared path reused by +/// the runtime `activate_extension` entry: open the extension's `.prefab.bin`, add +/// its single entity's components to `entity`, then fire the `on_attach` seam with +/// the cooked hook text. The extension prefab is mono-entity (cooked as such); a +/// component the entity already carries is a conflict (§30.5) — surfaced as +/// `error.ExtensionComponentConflict` rather than the dynamic-add assert. +fn activateExtension(world: *World, gpa: std.mem.Allocator, entity: EntityId, name: []const u8, ext_bytes: []const u8) !void { + const ext = try openVerified(ext_bytes); + + // Mono-entity: the extension's components live on its single entity. + var total: u32 = 0; + var ai: u32 = 0; + while (ai < ext.archetypeCount()) : (ai += 1) total += ext.archetype(ai).entity_count; + if (total > 1) return error.MultiEntityExtensionUnsupported; + + ai = 0; + while (ai < ext.archetypeCount()) : (ai += 1) { + const arch = ext.archetype(ai); + if (arch.entity_count == 0) continue; + var c: usize = 0; + while (c < arch.component_count) : (c += 1) { + const sch = ext.schema(arch.schemaIndex(c)); + const cid = world.componentId(sch.name) orelse return error.UnknownComponent; + if (sch.size != world.registry.componentSize(cid)) return error.SchemaMismatch; + if (world.componentBytes(entity, cid) != null) return error.ExtensionComponentConflict; + try world.addComponentDynamic(gpa, entity, cid, arch.componentSlot(c, 0)); + } + } + + // Fire the `on_attach` dispatch seam (D-E). Executing the text is M1.0.9. + const on_attach_text: ?[]const u8 = if (ext.hookCount() > 0) ext.hook(0).on_attach else null; + try world.dispatchOnAttach(entity, name, on_attach_text); +} + /// Load the resources block (E3) — the load-side mirror of M1.0.3's non-POD /// resource path. For each resource: resolve its schema index → runtime /// `ComponentId` (the E1 remap, already size/alignment-validated), copy the POD @@ -549,5 +625,5 @@ test "loadFromBytes rejects an out-of-range parent ordinal with MalformedScene" const bytes = try writer.write(gpa, model, &world.registry); defer gpa.free(bytes); - try testing.expectError(error.MalformedScene, loadFromBytes(&world, gpa, bytes)); + try testing.expectError(error.MalformedScene, loadFromBytes(&world, gpa, bytes, null)); } diff --git a/src/etch/interp.zig b/src/etch/interp.zig index da57788..4e27d3d 100644 --- a/src/etch/interp.zig +++ b/src/etch/interp.zig @@ -7422,7 +7422,7 @@ test "cooked Etch scene loads, on_spawned rules emit, resource string round-trip const bytes = try weld_core.scene.writer.write(gpa, cooked.model, &cooked.registry); defer gpa.free(bytes); - var result = try weld_core.scene.loader.loadFromBytes(&world, gpa, bytes); + var result = try weld_core.scene.loader.loadFromBytes(&world, gpa, bytes, null); defer result.deinit(gpa); // (a) every entity instantiated. diff --git a/tests/scene/crossref_test.zig b/tests/scene/crossref_test.zig index 283d2fa..88fdc97 100644 --- a/tests/scene/crossref_test.zig +++ b/tests/scene/crossref_test.zig @@ -99,7 +99,7 @@ test "cross-ref resolves to the target handle on load; unset reads dead" { _ = try world.registry.registerComponentRaw(gpa, .{ .name = "Marker", .size = 4, .alignment = 4, .default_bytes = &[_]u8{0} ** 4, .fields = &.{} }); _ = try world.registry.registerComponentRaw(gpa, .{ .name = "Link", .size = 8, .alignment = 8, .default_bytes = &[_]u8{0xFF} ** 8, .fields = &.{} }); - var result = try scene.loader.loadFromBytes(&world, gpa, bytes); + var result = try scene.loader.loadFromBytes(&world, gpa, bytes, null); defer result.deinit(gpa); const link_id = world.componentId("Link").?; diff --git a/tests/scene/extensions_test.zig b/tests/scene/extensions_test.zig index d48bb32..cac6ad0 100644 --- a/tests/scene/extensions_test.zig +++ b/tests/scene/extensions_test.zig @@ -17,6 +17,8 @@ const descriptor = weld_etch.descriptor; const scene_cook = weld_etch.scene_cook; const scene = weld_core.scene; const Accessor = scene.accessor.Accessor; +const World = weld_core.ecs.World; +const EntityId = weld_core.ecs.EntityId; /// One-entry in-process base-prefab resolver (for `extends`/`of`/`instance`). const OneResolver = struct { @@ -29,6 +31,9 @@ const OneResolver = struct { fn base(self: *OneResolver) scene_cook.BaseResolver { return .{ .ctx = self, .resolveFn = OneResolver.resolve }; } + fn ext(self: *OneResolver) scene.loader.ExtensionResolver { + return .{ .ctx = self, .resolveFn = OneResolver.resolve }; + } }; // A mono-entity base prefab carrying `Health` (the `requires` target). @@ -257,3 +262,103 @@ test "scene extensions clause populates the Entity Extensions + Prefab ID tables // No hooks in a .scene.bin. try std.testing.expectEqual(@as(u32, 0), acc.hookCount()); } + +// ── E6 — load applies extension components + fires the on_attach seam ── + +/// Tier-0 `on_attach` dispatch spy (the M1.0.9 Etch execution is out of scope; +/// E6 only proves the seam fires with the right name + hook text). +const AttachSpy = struct { + var fired: u32 = 0; + var saw_name: bool = false; + var saw_text: bool = false; + fn reset() void { + fired = 0; + saw_name = false; + saw_text = false; + } + fn cb(_: ?*anyopaque, _: *World, _: EntityId, name: []const u8, text: ?[]const u8) anyerror!void { + fired += 1; + if (std.mem.eql(u8, name, "CombatModule")) saw_name = true; + if (text != null and std.mem.indexOf(u8, text.?, "Health") != null) saw_text = true; + } +}; + +test "load applies extension components and the on_attach seam fires" { + const gpa = std.testing.allocator; + + // Cook BaseCharacter (the extends/requires base) + CombatModule (adds Weapon, + // with on_attach), wiring the cook-time base resolver. + var base = try scene_cook.cookPrefab(gpa, base_character, null, null); + defer base.deinit(gpa); + const base_bytes = try scene.writer.write(gpa, base.model, &base.registry); + defer gpa.free(base_bytes); + var base_res = OneResolver{ .name = "BaseCharacter", .bytes = base_bytes }; + + const combat = + \\component Health { current: i32 = 100, max: i32 = 100 } + \\component Weapon { damage: i32 = 10 } + \\prefab "CombatModule" extends "BaseCharacter" requires Health { + \\ entity "mod" { + \\ uuid: "9c4f3a2b-1e7d-4a5c-b8e9-f4d2c3a1b5e6" + \\ Weapon { damage: 25 } + \\ } + \\ on_attach { entity.get_mut(Health).max += 50 } + \\} + ; + var combat_cooked = try scene_cook.cookPrefab(gpa, combat, base_res.base(), null); + defer combat_cooked.deinit(gpa); + const combat_bytes = try scene.writer.write(gpa, combat_cooked.model, &combat_cooked.registry); + defer gpa.free(combat_bytes); + + // A scene with one entity activating CombatModule. + const scene_src = + \\component Health { current: i32 = 100, max: i32 = 100 } + \\scene "S" { + \\ entity "npc" { + \\ uuid: "00000000-0000-0000-0000-0000000000f1" + \\ extensions: ["CombatModule"] + \\ Health { current: 100, max: 100 } + \\ } + \\} + ; + var scene_cooked = try scene_cook.cook(gpa, scene_src, null); + defer scene_cooked.deinit(gpa); + const scene_bytes = try scene.writer.write(gpa, scene_cooked.model, &scene_cooked.registry); + defer gpa.free(scene_bytes); + + // World mirrors the component layout; register the Tier-0 on_attach seam. + var world = World.init(); + defer world.deinit(gpa); + _ = try world.registry.registerComponentRaw(gpa, .{ .name = "Health", .size = 8, .alignment = 4, .default_bytes = &[_]u8{0} ** 8, .fields = &.{} }); + _ = try world.registry.registerComponentRaw(gpa, .{ .name = "Weapon", .size = 4, .alignment = 4, .default_bytes = &[_]u8{0} ** 4, .fields = &.{} }); + AttachSpy.reset(); + world.registerOnAttach(null, &AttachSpy.cb); + + var ext_res = OneResolver{ .name = "CombatModule", .bytes = combat_bytes }; + var result = try scene.loader.loadFromBytes(&world, gpa, scene_bytes, ext_res.ext()); + defer result.deinit(gpa); + + const npc = result.uuid_to_entity.get(uuidBytes(0xf1)).?; + + // The extension's component was added (Weapon.damage == 25). + const weapon_id = world.componentId("Weapon").?; + const wb = world.componentBytes(npc, weapon_id) orelse return error.WeaponNotAdded; + try std.testing.expectEqual(@as(i32, 25), std.mem.readInt(i32, wb[0..4], .little)); + + // The on_attach seam fired once with the extension name + the cooked hook text. + try std.testing.expectEqual(@as(u32, 1), AttachSpy.fired); + try std.testing.expect(AttachSpy.saw_name); + try std.testing.expect(AttachSpy.saw_text); + + // M1.0.9 boundary: the hook is NOT executed at load — Health.max stays 100 + // (the `+= 50` effect is M1.0.9, not E6). + const health_id = world.componentId("Health").?; + const hb = world.componentBytes(npc, health_id).?; + try std.testing.expectEqual(@as(i32, 100), std.mem.readInt(i32, hb[4..8], .little)); // max @4 +} + +fn uuidBytes(last: u8) [16]u8 { + var u = [_]u8{0} ** 16; + u[15] = last; + return u; +} diff --git a/tests/scene/load_resources_test.zig b/tests/scene/load_resources_test.zig index 563ba1a..4869cc0 100644 --- a/tests/scene/load_resources_test.zig +++ b/tests/scene/load_resources_test.zig @@ -59,7 +59,7 @@ test "resource string fields round-trip through the persistent heap" { const bytes = try writer.write(gpa, model, &world.registry); defer gpa.free(bytes); - var result = try loader.loadFromBytes(&world, gpa, bytes); + var result = try loader.loadFromBytes(&world, gpa, bytes, null); defer result.deinit(gpa); // Exactly one interned string block, owned by the result. diff --git a/tests/scene/load_roundtrip_test.zig b/tests/scene/load_roundtrip_test.zig index 38390e5..f007af2 100644 --- a/tests/scene/load_roundtrip_test.zig +++ b/tests/scene/load_roundtrip_test.zig @@ -115,7 +115,7 @@ test "loading a cooked scene instantiates every entity" { const bytes = try cookPosVelScene(gpa, &world); defer gpa.free(bytes); - var result = try loader.loadFromBytes(&world, gpa, bytes); + var result = try loader.loadFromBytes(&world, gpa, bytes, null); defer result.deinit(gpa); try std.testing.expectEqual(@as(usize, n_entities), world.entityCount()); @@ -167,7 +167,7 @@ test "on_spawned fires once per loaded entity" { const bytes = try cookPosVelScene(gpa, &world); defer gpa.free(bytes); - var result = try loader.loadFromBytes(&world, gpa, bytes); + var result = try loader.loadFromBytes(&world, gpa, bytes, null); defer result.deinit(gpa); try std.testing.expectEqual(@as(u32, n_entities), Counter.fired); @@ -205,7 +205,7 @@ test "all entities exist before any on_spawned fires" { const bytes = try cookPosVelScene(gpa, &world); defer gpa.free(bytes); - var result = try loader.loadFromBytes(&world, gpa, bytes); + var result = try loader.loadFromBytes(&world, gpa, bytes, null); defer result.deinit(gpa); // At the very first on_spawned invocation, the full set was already present. @@ -233,7 +233,7 @@ test "loadScene mmaps a cooked file and instantiates every entity" { f.close(io); defer root.deleteFile(io, path) catch {}; - var result = try loader.loadScene(&world, gpa, path); + var result = try loader.loadScene(&world, gpa, path, null); defer result.deinit(gpa); // also closes the mmap try std.testing.expectEqual(@as(usize, n_entities), world.entityCount()); diff --git a/tests/scene/prefab_flatten_test.zig b/tests/scene/prefab_flatten_test.zig index 18cb304..aab4d61 100644 --- a/tests/scene/prefab_flatten_test.zig +++ b/tests/scene/prefab_flatten_test.zig @@ -180,7 +180,7 @@ test "N instances of one prefab load as N entities (cook -> load -> ECS)" { _ = try world.registry.registerComponentRaw(gpa, .{ .name = "Transform", .size = 12, .alignment = 4, .default_bytes = &[_]u8{0} ** 12, .fields = &.{} }); _ = try world.registry.registerComponentRaw(gpa, .{ .name = "Light", .size = 8, .alignment = 4, .default_bytes = &[_]u8{0} ** 8, .fields = &.{} }); - var result = try scene.loader.loadFromBytes(&world, gpa, bytes); + var result = try scene.loader.loadFromBytes(&world, gpa, bytes, null); defer result.deinit(gpa); try std.testing.expectEqual(@as(usize, 3), world.entityCount()); diff --git a/tests/scene/prefab_integration_test.zig b/tests/scene/prefab_integration_test.zig new file mode 100644 index 0000000..2287914 --- /dev/null +++ b/tests/scene/prefab_integration_test.zig @@ -0,0 +1,168 @@ +//! M1.0.6 E3/E4/E6 — cross-module capstone: one scene exercising prefab +//! instancing (with a per-field override), an entity→entity cross-reference, and +//! an active extension, end to end (Etch cook → `.scene.bin` → ECS load). Asserts +//! entity count, an overridden field, the resolved reference handle, the added +//! extension component, and that the `on_attach` Tier-0 seam fired (hook +//! EXECUTION is M1.0.9 — see extensions_test.zig). + +const std = @import("std"); +const weld_etch = @import("weld_etch"); +const weld_core = @import("weld_core"); + +const scene_cook = weld_etch.scene_cook; +const scene = weld_core.scene; +const World = weld_core.ecs.World; +const EntityId = weld_core.ecs.EntityId; + +/// Multi-name in-process resolver (cook base-prefab + load extension). +const MultiResolver = struct { + names: []const []const u8, + blobs: []const []const u8, + fn resolve(ctx: *anyopaque, name: []const u8) ?[]const u8 { + const self: *MultiResolver = @ptrCast(@alignCast(ctx)); + for (self.names, self.blobs) |n, b| if (std.mem.eql(u8, n, name)) return b; + return null; + } + fn base(self: *MultiResolver) scene_cook.BaseResolver { + return .{ .ctx = self, .resolveFn = MultiResolver.resolve }; + } + fn ext(self: *MultiResolver) scene.loader.ExtensionResolver { + return .{ .ctx = self, .resolveFn = MultiResolver.resolve }; + } +}; + +const AttachSpy = struct { + var fired: u32 = 0; + fn reset() void { + fired = 0; + } + fn cb(_: ?*anyopaque, _: *World, _: EntityId, _: []const u8, _: ?[]const u8) anyerror!void { + fired += 1; + } +}; + +fn uuidBytes(last: u8) [16]u8 { + var u = [_]u8{0} ** 16; + u[15] = last; + return u; +} + +const torch_prefab = + \\component Transform { x: f32 = 0.0, y: f32 = 0.0, z: f32 = 0.0 } + \\component Light { intensity: f32 = 2000.0, radius: f32 = 8.0 } + \\prefab "Torch" { + \\ entity "root" { + \\ uuid: "7b3e2f1a-42a3-4f2b-8c9d-a3f2b1c98d4e" + \\ Transform { x: 1.0 } + \\ Light { intensity: 1500.0, radius: 6.0 } + \\ } + \\} +; + +const base_char = + \\component Health { current: i32 = 100, max: i32 = 100 } + \\prefab "BaseChar" { + \\ entity "root" { uuid: "9c4f3a2b-1e7d-4a5c-b8e9-f4d2c3a1b5e6" Health { current: 100, max: 100 } } + \\} +; + +const combat_module = + \\component Health { current: i32 = 100, max: i32 = 100 } + \\component Weapon { damage: i32 = 10 } + \\prefab "CombatModule" extends "BaseChar" requires Health { + \\ entity "mod" { uuid: "a1b2c3d4-e5f6-4a5b-8c9d-0e1f2a3b4c5d" Weapon { damage: 25 } } + \\ on_attach { entity.get_mut(Health).max += 50 } + \\} +; + +const scene_src = + \\component Transform { x: f32 = 0.0, y: f32 = 0.0, z: f32 = 0.0 } + \\component Light { intensity: f32 = 2000.0, radius: f32 = 8.0 } + \\component Health { current: i32 = 100, max: i32 = 100 } + \\component Target { who: Entity } + \\scene "Village" { + \\ instance of "Torch" "T1" { uuid: "00000000-0000-0000-0000-000000000011" Light.intensity = 3000.0 } + \\ instance of "Torch" "T2" { uuid: "00000000-0000-0000-0000-000000000012" } + \\ entity "Boss" { + \\ uuid: "00000000-0000-0000-0000-0000000000b0" + \\ extensions: ["CombatModule"] + \\ Health { current: 100, max: 100 } + \\ } + \\ entity "Targeter" { + \\ uuid: "00000000-0000-0000-0000-000000000002" + \\ Target { who: "Boss" } + \\ } + \\} +; + +fn cookStandalone(gpa: std.mem.Allocator, src: []const u8) ![]u8 { + var c = try scene_cook.cookPrefab(gpa, src, null, null); + defer c.deinit(gpa); + return scene.writer.write(gpa, c.model, &c.registry); +} + +test "scene with prefab instances, a cross-ref and an extension loads end to end" { + const gpa = std.testing.allocator; + + // Cook the three prefabs. + const torch_bytes = try cookStandalone(gpa, torch_prefab); + defer gpa.free(torch_bytes); + const base_bytes = try cookStandalone(gpa, base_char); + defer gpa.free(base_bytes); + + var base_only = MultiResolver{ .names = &.{"BaseChar"}, .blobs = &.{base_bytes} }; + var combat = try scene_cook.cookPrefab(gpa, combat_module, base_only.base(), null); + defer combat.deinit(gpa); + const combat_bytes = try scene.writer.write(gpa, combat.model, &combat.registry); + defer gpa.free(combat_bytes); + + // Cook the scene (instance flattening resolves "Torch"). + var torch_only = MultiResolver{ .names = &.{"Torch"}, .blobs = &.{torch_bytes} }; + var cooked = try scene_cook.cookScene(gpa, scene_src, torch_only.base(), null); + defer cooked.deinit(gpa); + const scene_bytes = try scene.writer.write(gpa, cooked.model, &cooked.registry); + defer gpa.free(scene_bytes); + + // Load into a World mirroring the component layout, with the on_attach seam + + // the extension resolver ("CombatModule"). + var world = World.init(); + defer world.deinit(gpa); + _ = try world.registry.registerComponentRaw(gpa, .{ .name = "Transform", .size = 12, .alignment = 4, .default_bytes = &[_]u8{0} ** 12, .fields = &.{} }); + _ = try world.registry.registerComponentRaw(gpa, .{ .name = "Light", .size = 8, .alignment = 4, .default_bytes = &[_]u8{0} ** 8, .fields = &.{} }); + _ = try world.registry.registerComponentRaw(gpa, .{ .name = "Health", .size = 8, .alignment = 4, .default_bytes = &[_]u8{0} ** 8, .fields = &.{} }); + _ = try world.registry.registerComponentRaw(gpa, .{ .name = "Weapon", .size = 4, .alignment = 4, .default_bytes = &[_]u8{0} ** 4, .fields = &.{} }); + _ = try world.registry.registerComponentRaw(gpa, .{ .name = "Target", .size = 8, .alignment = 8, .default_bytes = &[_]u8{0xFF} ** 8, .fields = &.{} }); + AttachSpy.reset(); + world.registerOnAttach(null, &AttachSpy.cb); + + var ext_res = MultiResolver{ .names = &.{"CombatModule"}, .blobs = &.{combat_bytes} }; + var result = try scene.loader.loadFromBytes(&world, gpa, scene_bytes, ext_res.ext()); + defer result.deinit(gpa); + + // 4 entities: T1, T2, Boss, Targeter. + try std.testing.expectEqual(@as(usize, 4), world.entityCount()); + + const light_id = world.componentId("Light").?; + const t1 = result.uuid_to_entity.get(uuidBytes(0x11)).?; + const t2 = result.uuid_to_entity.get(uuidBytes(0x12)).?; + // E3: per-field override on T1 (intensity 3000), inherited on T2 (1500). + try std.testing.expectApproxEqAbs(@as(f32, 3000.0), @as(f32, @bitCast(std.mem.readInt(u32, world.componentBytes(t1, light_id).?[0..4], .little))), 1e-3); + try std.testing.expectApproxEqAbs(@as(f32, 1500.0), @as(f32, @bitCast(std.mem.readInt(u32, world.componentBytes(t2, light_id).?[0..4], .little))), 1e-3); + + // E4: Targeter.Target.who resolved to Boss's runtime handle. + const boss = result.uuid_to_entity.get(uuidBytes(0xb0)).?; + const targeter = result.uuid_to_entity.get(uuidBytes(0x02)).?; + const target_id = world.componentId("Target").?; + const who = std.mem.readInt(u64, world.componentBytes(targeter, target_id).?[0..8], .little); + try std.testing.expectEqual(@as(u64, @bitCast(boss)), who); + + // E6: Boss got the extension's Weapon, and the on_attach seam fired once. + const weapon_id = world.componentId("Weapon").?; + const wb = world.componentBytes(boss, weapon_id) orelse return error.WeaponNotAdded; + try std.testing.expectEqual(@as(i32, 25), std.mem.readInt(i32, wb[0..4], .little)); + try std.testing.expectEqual(@as(u32, 1), AttachSpy.fired); + + // M1.0.9 boundary: on_attach not executed → Boss.Health.max still 100. + const health_id = world.componentId("Health").?; + try std.testing.expectEqual(@as(i32, 100), std.mem.readInt(i32, world.componentBytes(boss, health_id).?[4..8], .little)); +} From 104285b74f847a043ce652192acfd29a5a885ef6 Mon Sep 17 00:00:00 2001 From: Guy Senpai Date: Sun, 28 Jun 2026 17:18:18 +0200 Subject: [PATCH 14/15] docs(claude-md): update for M1.0.6 --- CLAUDE.md | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 26592c2..fc053df 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -11,9 +11,9 @@ knowledge base — see § Quick links spec. |---|---| | Phase | 1 (Etch ↔ ECS) | | Current milestone | (none — between milestones) | -| Last released tag | `v0.10.5-scene-load` | +| Last released tag | `v0.10.6-prefabs-crossrefs-extensions` | | Active branch | `main` | -| Next planned milestone | M1.0.6 — prefab `.prefab.bin` loading + entity→entity cross-references + Entity Extensions Table (to be scoped) | +| Next planned milestone | M1.0.9 — extension hook (`on_attach`/`on_detach`) execution; starts with a text-vs-bytecode design decision (re-scoped out of M1.0.6) | ## Tags @@ -43,6 +43,7 @@ knowledge base — see § Quick links spec. | `v0.10.3-resource-nonpod-fields` | 2026-06-24 | M1.0.3 — String and enum resource fields | Resource fields reach spec parity for the two scalar non-POD cases; founds the Phase 1 persistent heap (system allocator + atomic refcount + drop-by-`type_id` + immortal-interned sentinel). Resource-only `FieldKind.string_`/`.enum_`; components stay POD-strict (validator-gated). The Option A alignment (the former deferred "tranche 7"). | | `v0.10.4-scene-cook` | 2026-06-27 | M1.0.4 — Cooking `.scene.etch` → `.scene.bin` | Offline, World-free cook of direct-entity scenes. Tier-0 `src/core/scene/` codec — `SceneHeader` (64 B) + §10 Schema Registry + `writer` + zero-copy `accessor` (the read half, reused verbatim by the M1.0.5 loader). Etch driver `src/etch/scene_cook.zig` reuses `compileTypeDecl` (refactored `*World`→`*Registry`) + `evalConst`; groups entities by `ComponentSignature` into flat SoA columns. On-disk component identity is the Schema-Registry index → component **name** (no raw `ComponentId`). Resource `string`/enum materialized; `parent` name→UUID; `instance of` + unsupported field kinds rejected with clear diagnostics. `scene_cook` CLI + re-cook byte-identical determinism. | | `v0.10.5-scene-load` | 2026-06-27 | M1.0.5 — Runtime `.scene.bin` loader → ECS | Runtime loader `src/core/scene/loader.zig` reusing `accessor.zig` verbatim: `openVerified` (magic/version + `verifyHash` → `CorruptScene`) + `buildSchemaRemap` (Schema-Registry index → runtime `ComponentId` via `idOf`, size/alignment-validated → `SchemaMismatch`/`UnknownComponent`) + per-entity `spawnDynamicWithValues` instantiation + UUID(16 B)→handle map + two-phase `on_spawned` (`ObserverRegistry.dispatchOnSpawned` + `World.dispatchOnSpawned`, all-entities-exist-first ordering) + resource loading (POD + `string` fields interned into the Tier-0 persistent heap, owned by `LoadResult`). `loadFromBytes` (byte-level core) + `loadScene(path)` (mmap). New `error.MalformedScene` (structure invalid, distinct from `CorruptScene` = hash mismatch). Persistent heap moved `src/etch/persistent.zig` → `src/core/memory/persistent.zig` (Tier 0). Bench median ~1.05 ms / 10k entities (M4 Pro, ReleaseFast). | +| `v0.10.6-prefabs-crossrefs-extensions` | 2026-06-28 | M1.0.6 — prefabs + entity→entity cross-references + extension activation | `.prefab.bin` cook (standalone + `of` variants) + `instance of` flattening at scene cook (byte-identical to hand-authored). Entity→entity cross-refs via new `FieldKind.entity_` (8 B `EntityId`, component-only, default `dead`=`0xFF`) — by **name** (like `parent:`), resolved at load (`resolveCrossRefs`, bounds-checked → `MalformedScene`). Extension activation: `extensions:` grammar clause (entity + instance) + Entity Extensions Table + dedup Prefab ID Table + hooks sub-section in the `extensions_offset` region (shape A); `extends` cook (components + `requires` + `on_attach`/`on_detach` rendered as **text**). Load `applyExtensions`: resolve by name → `addComponentDynamic` (conflict → `ExtensionComponentConflict`) → fire Tier-0 `on_attach` seam (`registerOnAttach`/`dispatchOnAttach`; loader never touches the VM). **`format_version` 1→2** (region restructured; v1 → `BadVersion`, re-cook). Hook **execution** re-scoped → M1.0.9. | ## Hypotheses validated by spikes @@ -69,6 +70,9 @@ knowledge base — see § Quick links spec. - **M1.0.5 scope boundary (runtime loader)**: the loader instantiates **per-entity** via `world.spawnDynamicWithValues` (no chunk/column bulk copy), builds the UUID→handle map and validates parent ordinals but **applies no parent link** (no runtime hierarchy component yet — hierarchy milestone), reads the cross-refs/extensions tables empty (**M1.0.6**), and rejects an unknown component (`error.UnknownComponent`) rather than auto-registering from the on-disk schema (Phase 2+). `loadFromBytes` (byte-level core) + `loadScene(path)` (mmap, `LoadResult` owns the mmap). Out: bulk-spawn, prefab/`.prefab.bin`, save/load, decompression, streaming. - **Per-entity vs bulk-spawn (decided by M1.0.5 bench)**: per-entity instantiation measured **~1.05 ms / 10k entities** (M4 Pro, ReleaseFast) — far under the ~10–50 ms/10k spec reference → **per-entity confirmed; bulk SoA column-copy is a genuine YAGNI, no bulk-spawn milestone scheduled.** The loader's instantiate step keeps a clean boundary so a bulk path could later swap its body without touching `loadScene`'s signature or call sites. - **Persistent heap is Tier 0 (since M1.0.5)**: the refcounted string/value heap moved `src/etch/persistent.zig` → **`src/core/memory/persistent.zig`** (re-exported as `weld_core.memory.persistent`) so the `weld_core` scene loader can intern resource `string` fields without importing `weld_etch`. Tier-neutral (`runDrop` is a no-op); `StringSlot` 16-byte layout + API unchanged (no format/ABI impact). The Etch importers (`interp`/`ecs_bridge`/`scene_cook`) now reach it through `weld_core`. Loaded resource strings are `allocImmortal` + owned by `LoadResult`. +- **M1.0.6 scope boundary**: prefab `instance of` flattened at scene cook + `.prefab.bin` standalone/`of` cook; entity→entity cross-refs by name via `FieldKind.entity_` (resolved at load); extension **activation** = Entity Extensions Table + Prefab ID Table + add-component at load + Tier-0 `on_attach` dispatch seam. Hook **execution** (compile/run the Etch hook text) is **M1.0.9** — the interpreter is compile-once from the AST with no runtime text-execution surface, and the text-vs-bytecode serialization choice is unsettled. The §30.5 additive-conflict **compile-time** warning is also M1.0.9 (the cook's `diag_out` is mono-message set-on-error; a non-fatal warning channel is needed); the dangerous **runtime** case is already caught (`ExtensionComponentConflict` at load). +- **`format_version` 1→2 (correction to the L68 day-1 prediction)**: the cross-references section was genuinely additive (no bump at E4, count-0 back-compatible), but the Entity Extensions Table's full structure (vs M1.0.4's bare `[0]` count-placeholder) is **not** count-0 back-compatible → required `format_version` 1→2. The `FieldKind` dispatch prediction held (`entity_` added as a new variant). `.scene.bin`/`.prefab.bin` are re-cookable Phase-1 artifacts (deterministic cook) → no v1 back-compat. +- **`FieldKind.entity_` realizes `Entity` (M1.0.6)**: 8 B/8-align `EntityId`, default `dead` (`@memset 0xFF`, not `{0,0}` = a live handle to entity 0), component-only (gated to `reg_kind == .component`, mirror of resource-only `string_`/`enum_`). ## Non-negotiable rules @@ -202,4 +206,4 @@ The `briefs/` directory is the source of truth for milestone state. The brief's --- -Last updated: 2026-06-27 +Last updated: 2026-06-28 From f56fc7a19dc330eb03c431c609117ce0e8666357 Mon Sep 17 00:00:00 2001 From: Guy Senpai Date: Sun, 28 Jun 2026 17:21:04 +0200 Subject: [PATCH 15/15] docs(brief): close M1.0.6 --- briefs/M1.0.6-prefabs-crossrefs-extensions.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/briefs/M1.0.6-prefabs-crossrefs-extensions.md b/briefs/M1.0.6-prefabs-crossrefs-extensions.md index 49839cd..e852003 100644 --- a/briefs/M1.0.6-prefabs-crossrefs-extensions.md +++ b/briefs/M1.0.6-prefabs-crossrefs-extensions.md @@ -1,12 +1,12 @@ # M1.0.6 — Prefabs, entity cross-references & extensions -> **Status:** ACTIVE +> **Status:** CLOSED > **Phase:** 1 > **Branch:** `phase-1/scene/prefabs-crossrefs-extensions` > **Tag (set after merge by Guy):** `v0.10.6-prefabs-crossrefs-extensions` > **Dependencies:** M0.1 (Tier-0 ECS: `World` spawn paths, `componentBytes`/`markComponentChangedDyn` dynamic-by-`ComponentId` access, dynamic add-component archetype migration, `Archetype`, `Registry.idOf`, generational `EntityId`), M0.8 (grammar v0.6: `prefab_decl` with `of`/`extends`/`requires`/`on_attach`/`on_detach`, `scene_decl` with `instance of` + `extensions:`), M1.0.2 (`ObserverRegistry` + `dispatchOnSpawned` two-phase pattern — reused for the `on_attach` dispatch seam), M1.0.3 (`writeValueAsBytes`/`readBytesAsValue`, `FieldDesc` metadata, persistent heap now Tier-0), M1.0.4 (`.scene.bin` codec: `src/core/scene/{format,writer,accessor}.zig`; `tools/scene_cook`; the `instance of` rejection this milestone removes), M1.0.5 (runtime loader: `loadFromBytes`/`loadScene`, `LoadResult.uuid_to_entity` seam, the two reserved-table asserts this milestone replaces) > **Open date:** 2026-06-28 -> **Close date:** — +> **Close date:** 2026-06-28 --- @@ -272,8 +272,8 @@ Sparse (entities with no active extension are absent). The `§4` note's `u8` cou ## Closing notes -- **What worked**: -- **What deviated from the original spec**: -- **What to flag explicitly in review**: -- **Final measurements** (load time vs M1.0.5, binary size, compile time as relevant): -- **Residual risk / tech debt left deliberately**: +- **What worked**: The E1 seam-reconfirmation discipline paid off twice — it caught the `extensions:` parse-gap (the brief assumed it parsed; it didn't) and grounded D-B (by-name, not the doc's `uuid "…"`) before any code. The M1.0.4/M1.0.5 codec was genuinely reusable: `.prefab.bin` is literally `.scene.bin` (the writer/accessor took prefabs with zero format invention), and the cross-ref "side table + dead slot" mirrored the resource-string pass exactly. `FieldKind.entity_` slotted in as the clean mirror of `string_`/`enum_`. The two-phase crossref pass (record-then-resolve) made forward references fall out for free. The gate-split (E1→E6 with STOPs) surfaced both genuinely-frozen-contract decisions (hooks shape, hook execution) to Claude.ai instead of improvising them. +- **What deviated from the original spec**: (1) `extensions:` clause was **not** parsed (E1 finding) → grammar/parser/AST/descriptors added via a Claude.ai grammar amendment (option a). (2) The `.prefab.bin` hooks section needed a frozen-format decision (header full) → SHAPE A (hooks in the `extensions_offset` region) + **`format_version` 1 → 2** (the L68 day-1 "additive sections" prediction held for cross-refs but **not** for the Entity Extensions Table — its full structure isn't count-0 back-compatible vs M1.0.4's bare `[0]`). (3) **E6 hook EXECUTION re-scoped → M1.0.9** (the interp is compile-once, no runtime text-execution surface; text-vs-bytecode unsettled) — E6 ships activation **infra** only (component-add + Tier-0 `on_attach` dispatch seam). (4) `of`-variant override granularity is `Component { field: value }` field-merge (the `Component.field =` per-field form is `instance_decl`-only per the grammar). (5) Prefab/extension instantiation is **mono-entity** (the spec defines no internal-uuid remapping for multi-entity prefab instances → `MultiEntityInstanceUnsupported`). +- **What to flag explicitly in review**: the **`format_version` 1 → 2** break (no v1 back-compat — all `.scene.bin`/`.prefab.bin` must be re-cooked; deterministic, no Phase-1 prod files); the **M1.0.9 boundary** (E6 fires the `on_attach` seam but does NOT execute the hook — asserted: `Health.max` stays 100); the **§30.5 conflict warning** still pending a decision (E6-vs-M1.0.9 — see below); `registry.zig` (FROZEN) gained `FieldKind.entity_` + an `entity.zig` import (sanctioned by the brief's E4 file-list); the out-of-list edits (`tests/etch_interp/diff_runner.zig` exhaustive-switch arms; `src/etch/{parser,ast,descriptor,zig_codegen/lower}.zig` for the `extensions:` clause) are all mechanical consequences of the FieldKind variant + the grammar amendment, journaled. +- **Final measurements**: scene load **median 1.111 ms / 10k entities** (M4 Pro, ReleaseFast, 50 runs; min 0.892 / max 1.830) vs M1.0.5's ~1.05 ms — **+~6%, within noise**; the `resolveCrossRefs`/`applyExtensions` passes are O(entries) and **no-op for an extension-free scene** (`crossrefsCount`/`extensionsCount` == 0 → immediate return). Cook stays offline. `format_version` is now 2. Full suite `zig build test` 844/861 (17 skipped, 0 failed); `zig fmt --check` + `zig build lint` clean; pre-push (Debug + ReleaseSafe ×2) green on every push. +- **Residual risk / tech debt left deliberately**: (1) **`on_attach`/`on_detach` execution** → M1.0.9 (the cooked hook text is the metadata it consumes); the dangerous runtime double-add IS caught (`ExtensionComponentConflict`). (2) **§30.5 additive-conflict compile-time warning** deferred — the cook's `diag_out` is a mono-message set-on-error channel with no non-fatal warning path; building one is non-trivial → flagged for an E6-vs-M1.0.9 decision (**open**). (3) Multi-entity prefab instantiation + hierarchy application (D-D) → hierarchy milestone. (4) `Entity` fields on extension-added components + resource→entity refs + cross-scene refs → future additive milestones (per Out-of-scope).