Phase 1 / Scene / Runtime loader .scene.bin → ECS

CLAUDE.md

@@ -11,9 +11,9 @@ knowledge base — see § Quick links spec.
 |---|---|
 | Phase | 1 (Etch ↔ ECS) |
 | Current milestone | (none — between milestones) |
-| Last released tag | `v0.10.4-scene-cook` |
+| Last released tag | `v0.10.5-scene-load` |
 | Active branch | `main` |
-| Next planned milestone | M1.0.5 — runtime `.scene.bin` loader (to be scoped) |
+| Next planned milestone | M1.0.6 — prefab `.prefab.bin` loading + entity→entity cross-references + Entity Extensions Table (to be scoped) |
 
 ## Tags
 
@@ -42,6 +42,7 @@ knowledge base — see § Quick links spec.
 | `v0.10.2-etch-events-observers` | 2026-06-23 | M1.0.2 — Events + structural observers | `emit`/`@on_event` + five lifecycle-annotation observers (`@on_added`/`@on_removed`/`@on_replaced`/`@on_spawned`/`@on_despawned`), annotation-routed; Tier 0 observer registry completed (`on_replaced`, ctx, old/new). Diagnostics E1208/E1209/E1215. Non-POD (`string`) event-field fix folded in via fix-as-you-go. |
 | `v0.10.3-resource-nonpod-fields` | 2026-06-24 | M1.0.3 — String and enum resource fields | Resource fields reach spec parity for the two scalar non-POD cases; founds the Phase 1 persistent heap (system allocator + atomic refcount + drop-by-`type_id` + immortal-interned sentinel). Resource-only `FieldKind.string_`/`.enum_`; components stay POD-strict (validator-gated). The Option A alignment (the former deferred "tranche 7"). |
 | `v0.10.4-scene-cook` | 2026-06-27 | M1.0.4 — Cooking `.scene.etch` → `.scene.bin` | Offline, World-free cook of direct-entity scenes. Tier-0 `src/core/scene/` codec — `SceneHeader` (64 B) + §10 Schema Registry + `writer` + zero-copy `accessor` (the read half, reused verbatim by the M1.0.5 loader). Etch driver `src/etch/scene_cook.zig` reuses `compileTypeDecl` (refactored `*World`→`*Registry`) + `evalConst`; groups entities by `ComponentSignature` into flat SoA columns. On-disk component identity is the Schema-Registry index → component **name** (no raw `ComponentId`). Resource `string`/enum materialized; `parent` name→UUID; `instance of` + unsupported field kinds rejected with clear diagnostics. `scene_cook` CLI + re-cook byte-identical determinism. |
+| `v0.10.5-scene-load` | 2026-06-27 | M1.0.5 — Runtime `.scene.bin` loader → ECS | Runtime loader `src/core/scene/loader.zig` reusing `accessor.zig` verbatim: `openVerified` (magic/version + `verifyHash` → `CorruptScene`) + `buildSchemaRemap` (Schema-Registry index → runtime `ComponentId` via `idOf`, size/alignment-validated → `SchemaMismatch`/`UnknownComponent`) + per-entity `spawnDynamicWithValues` instantiation + UUID(16 B)→handle map + two-phase `on_spawned` (`ObserverRegistry.dispatchOnSpawned` + `World.dispatchOnSpawned`, all-entities-exist-first ordering) + resource loading (POD + `string` fields interned into the Tier-0 persistent heap, owned by `LoadResult`). `loadFromBytes` (byte-level core) + `loadScene(path)` (mmap). New `error.MalformedScene` (structure invalid, distinct from `CorruptScene` = hash mismatch). Persistent heap moved `src/etch/persistent.zig` → `src/core/memory/persistent.zig` (Tier 0). Bench median ~1.05 ms / 10k entities (M4 Pro, ReleaseFast). |
 
 ## Hypotheses validated by spikes
 
@@ -65,6 +66,9 @@ knowledge base — see § Quick links spec.
 - **M1.0.4 scope boundary (scene cook)**: the cook is offline + **World-free** (no entity instantiation). Runtime `.scene.bin` → ECS `World` (mmap + memcpy-into-chunks + UUID→handle remap + `on_spawned`) is **M1.0.5** — it reuses `src/core/scene/accessor.zig` verbatim. Prefab `instance of` flattening + entity→entity cross-refs + the Entity Extensions Table are **M1.0.6** (M1.0.4 rejects `instance of` and writes both reserved sections empty). On-disk component identity is the §10 Schema-Registry index → component name (Phase-1 identity = name; no comptime schema-hash for Etch components), so M1.0.5 remaps via `idOf(name)`. Registration deviation: `interp.compileTypeDecl` was refactored `*World`→`*Registry` (+ `pub`) so the cook reuses it World-free (traced in `briefs/M1.0.4-scene-cook.md` Accepted deviations).
 - **Dynamic collections on resource fields (`string[]`, `[K:V]`, `Set`)**: unscheduled-**additive** — they fold into the first consuming milestone, **not** M1.0.4. The `M1.0.4` label in `briefs/M1.0.3-resource-nonpod-fields.md` Context is **superseded** (M1.0.4 is the scene cook, not dynamic collections). The persistent heap's `type_id`→drop dispatch + open `TypeId` set already accommodate them.
 - **Additive future format sections (design-at-day-1, not deferral)**: M1.0.4's `.scene.bin` reserves the Extensions + Cross-references sections (written empty) and the writer/accessor dispatch on `FieldKind` + name-based schema identity — so M1.0.6 (cross-refs/extensions) and the first milestone adding a 3D/handle `FieldKind` add columns/entries **without** changing the on-disk format or the loader.
+- **M1.0.5 scope boundary (runtime loader)**: the loader instantiates **per-entity** via `world.spawnDynamicWithValues` (no chunk/column bulk copy), builds the UUID→handle map and validates parent ordinals but **applies no parent link** (no runtime hierarchy component yet — hierarchy milestone), reads the cross-refs/extensions tables empty (**M1.0.6**), and rejects an unknown component (`error.UnknownComponent`) rather than auto-registering from the on-disk schema (Phase 2+). `loadFromBytes` (byte-level core) + `loadScene(path)` (mmap, `LoadResult` owns the mmap). Out: bulk-spawn, prefab/`.prefab.bin`, save/load, decompression, streaming.
+- **Per-entity vs bulk-spawn (decided by M1.0.5 bench)**: per-entity instantiation measured **~1.05 ms / 10k entities** (M4 Pro, ReleaseFast) — far under the ~10–50 ms/10k spec reference → **per-entity confirmed; bulk SoA column-copy is a genuine YAGNI, no bulk-spawn milestone scheduled.** The loader's instantiate step keeps a clean boundary so a bulk path could later swap its body without touching `loadScene`'s signature or call sites.
+- **Persistent heap is Tier 0 (since M1.0.5)**: the refcounted string/value heap moved `src/etch/persistent.zig` → **`src/core/memory/persistent.zig`** (re-exported as `weld_core.memory.persistent`) so the `weld_core` scene loader can intern resource `string` fields without importing `weld_etch`. Tier-neutral (`runDrop` is a no-op); `StringSlot` 16-byte layout + API unchanged (no format/ABI impact). The Etch importers (`interp`/`ecs_bridge`/`scene_cook`) now reach it through `weld_core`. Loaded resource strings are `allocImmortal` + owned by `LoadResult`.
 
 ## Non-negotiable rules
 

bench/scene_load_bench.zig

@@ -0,0 +1,136 @@
+//! M1.0.5 E2 — runtime scene loader benchmark.
+//!
+//! Measures the wall time of `scene.loader.loadFromBytes` on a ~10k-entity
+//! scene. The image is produced in-process by the M1.0.4 `writer` (no
+//! `.scene.etch` authoring, no file I/O) so the number isolates the load work:
+//! the per-entity `spawnDynamicWithValues` (archetype find/create + slot alloc +
+//! component memcpy), the schema-identity remap, and the `on_spawned` pass.
+//!
+//! **Measurement, not a gate.** It prints a median; the milestone's per-entity-
+//! vs-bulk decision is recorded from this number (spec ref ~10–50 ms / 10k).
+//! Run in ReleaseFast for a representative figure:
+//!   `zig build bench-scene-load -Doptimize=ReleaseFast`
+
+const std = @import("std");
+const builtin = @import("builtin");
+const weld_core = @import("weld_core");
+
+const World = weld_core.ecs.World;
+const Registry = weld_core.ecs.registry.Registry;
+const scene = weld_core.scene;
+const format = scene.format;
+const writer = scene.writer;
+const loader = scene.loader;
+
+const num_entities: u32 = 10_000;
+const warmup_runs: u32 = 3;
+const measured_runs: u32 = 50;
+const pos_size: u16 = 12; // [3]f32
+const pos_align: u16 = 4;
+
+/// Cook a single `[Pos]` archetype of `num_entities` entities into `.scene.bin`
+/// bytes (caller-owned). The cook registry is local — the writer copies the
+/// schema name into the image, so the bytes outlive it.
+fn buildSceneBytes(gpa: std.mem.Allocator) ![]u8 {
+    var reg = Registry.init();
+    defer reg.deinit(gpa);
+    const pos = try reg.registerComponentRaw(gpa, .{
+        .name = "Pos",
+        .size = pos_size,
+        .alignment = pos_align,
+        .default_bytes = &[_]u8{0} ** pos_size,
+        .fields = &.{},
+    });
+
+    var arena = std.heap.ArenaAllocator.init(gpa);
+    const a = arena.allocator();
+    const names = try a.dupe([]const u8, &.{try a.dupe(u8, "E")});
+    const uuids = try a.alloc([16]u8, num_entities);
+    for (0..num_entities) |i| {
+        uuids[i] = [_]u8{0} ** 16;
+        std.mem.writeInt(u32, uuids[i][0..4], @intCast(i + 1), .little);
+    }
+    const col = try a.alloc(u8, @as(usize, pos_size) * num_entities);
+    @memset(col, 0);
+    const cols = try a.dupe([]u8, &.{col});
+    const ents = try a.alloc(format.EntityEntry, num_entities);
+    for (0..num_entities) |i| ents[i] = .{ .name = 0, .uuid = @intCast(i), .parent_uuid = format.no_parent };
+    const ids = try a.dupe(format.ComponentId, &.{pos});
+    const blocks = try a.dupe(format.ArchetypeBlock, &.{.{
+        .component_ids = ids,
+        .entity_count = num_entities,
+        .columns = cols,
+        .entities = ents,
+    }});
+    var model: format.CookModel = .{
+        .strings = names,
+        .uuids = uuids,
+        .resources = &.{},
+        .archetypes = blocks,
+        .arena = arena,
+    };
+    defer model.deinit();
+    return try writer.write(gpa, model, &reg);
+}
+
+/// One timed load into a fresh world. Returns the elapsed ns (the load only —
+/// world setup and teardown are outside the measured window).
+fn timeOneLoad(gpa: std.mem.Allocator, io: std.Io, bytes: []const u8) !u64 {
+    var world = World.init();
+    defer world.deinit(gpa);
+    _ = try world.registry.registerComponentRaw(gpa, .{
+        .name = "Pos",
+        .size = pos_size,
+        .alignment = pos_align,
+        .default_bytes = &[_]u8{0} ** pos_size,
+        .fields = &.{},
+    });
+
+    const t0 = std.Io.Clock.now(.awake, io);
+    var result = try loader.loadFromBytes(&world, gpa, bytes);
+    const t1 = std.Io.Clock.now(.awake, io);
+    result.deinit(gpa);
+    const elapsed = t0.durationTo(t1).nanoseconds;
+    return @intCast(@max(@as(i96, 0), elapsed));
+}
+
+fn msFromNs(ns: u64) f64 {
+    return @as(f64, @floatFromInt(ns)) / 1_000_000.0;
+}
+
+pub fn main(init: std.process.Init) !void {
+    const gpa = init.gpa;
+    const io = init.io;
+
+    const bytes = try buildSceneBytes(gpa);
+    defer gpa.free(bytes);
+
+    const samples = try gpa.alloc(u64, measured_runs);
+    defer gpa.free(samples);
+
+    var run: u32 = 0;
+    const total = warmup_runs + measured_runs;
+    while (run < total) : (run += 1) {
+        const ns = try timeOneLoad(gpa, io, bytes);
+        if (run >= warmup_runs) samples[run - warmup_runs] = ns;
+    }
+
+    std.mem.sort(u64, samples, {}, std.sort.asc(u64));
+    const median = samples[samples.len / 2];
+
+    var buf: [512]u8 = undefined;
+    var w = std.Io.File.stdout().writer(io, &buf);
+    try w.interface.print(
+        "scene_load_bench [{s}]: {d} entities, {d} runs\n" ++
+            "  median {d:.3} ms | min {d:.3} ms | max {d:.3} ms\n",
+        .{
+            @tagName(builtin.mode),
+            num_entities,
+            measured_runs,
+            msFromNs(median),
+            msFromNs(samples[0]),
+            msFromNs(samples[samples.len - 1]),
+        },
+    );
+    try w.interface.flush();
+}