diff --git a/briefs/M1.0.3-resource-nonpod-fields.md b/briefs/M1.0.3-resource-nonpod-fields.md new file mode 100644 index 0000000..d123a4b --- /dev/null +++ b/briefs/M1.0.3-resource-nonpod-fields.md @@ -0,0 +1,234 @@ +# M1.0.3 — Non-POD resource fields (`string`, enum) exposed to the Etch interpreter + +> **Status:** CLOSED +> **Phase:** 1 +> **Branch:** `phase-1/etch/resource-nonpod-fields` +> **Tag:** `v0.10.3-resource-nonpod-fields` +> **Dependencies:** M0.1 (Tier-0 ECS: `Registry`, `ResourceStore`), M0.2 (RTTI, resources, events), M0.8 (tree-walking interpreter, grammar v0.6, resource access core — `get`/`get_mut`/`when resource`, `FieldDeclOrigin`), M1.0.2 (the `FieldDeclOrigin.event_` non-POD-field unlock pattern this milestone extends to resources) +> **Open date:** 2026-06-23 +> **Close date:** 2026-06-24 + +--- + +# FROZEN SECTION + +*Produced by Claude.ai. Not modifiable by Claude Code outside a Claude.ai round-trip (cf. § Accepted deviations).* + +## Context + +The spec is unambiguous and consistent across five documents: a `resource` is **not** POD-strict and may carry `string`, enum, dynamic array, and map fields — `etch-reference-part1.md` §5.5 ("For resources and locales: no POD constraint"), `etch-reference-part2.md` §4 (the canonical `resource GameState { current_level: string = "intro", difficulty: Difficulty = .normal, … }` example), `etch-memory-model.md` §4.2 / §7.3 (resource dynamic fields live in the persistent heap, refcount shared with the resource). + +The implementation does not yet honour this. The type-checker (`src/etch/types.zig`, `validateFieldsInDecl`, enum `FieldDeclOrigin { component_like, resource, struct_, event_ }`) unlocks `string` and enum-typed fields **only** for `.struct_` and `.event_`; the `.resource` origin falls through to the rejection `"type 'string' is not in the S3 builtin set"`. A guard test acts this rejection (the `resource Settings { player_name: string }` sub-assertion in `test "type-checker accepts string and enum fields on struct, keeps component/resource rejection"`), with the explicit standing comment *"Resources keep the S3 rejection until the **Option A alignment (tranche 7)**"*. The `FieldDeclOrigin` doc comment names the same deferred work. This milestone **is** that Option A alignment: it brings resource fields to spec parity for the two scalar non-POD cases, `string` and enum. + +This is **not** a one-line gate removal. The resource runtime store is a fixed-width POD byte block: `ResourceStore` holds each resource as bytes, decoded through `Registry`'s `FieldKind` (`{ int_, float_, bool_, i32_, u32_, f32_, f64_ }`) at byte offsets. A `string` is `{ ptr, len }` pointing into the **persistent heap** (`etch-memory-model.md` §5.1), and the persistent heap **does not exist yet** (only an API stub in `plugin_loader/api.zig` and an unrelated path-string arena in `tags.zig`). The only runtime string forms today are `string_id` (immutable AST pool) and `string_run` (rule-arena, reset at the rule-body boundary). Writing `get_mut(R).field = "…"` is the `etch-memory-model.md` §6.7 promotion of a rule-arena string into a persistent slot owned by the resource — which requires founding the Phase-1 persistent heap (`etch-memory-model.md` §4 / §11: system allocator + 8-byte `{ refcount: atomic u32, type_id: u32 }` header + drop-by-`type_id`). That foundation is the keystone of this milestone and is built to spec now (not half-baked) because dynamic collections (M1.0.4) reuse it unchanged. + +## Scope + +Executed in three staged sub-scopes with a STOP-for-review gate between each (cf. Notes › Staging). + +**E1 — Phase-1 persistent heap foundation (`src/etch/`, Tier-0-agnostic).** +- A persistent-heap allocator per `etch-memory-model.md` §4 / §11: an invisible 8-byte header `{ refcount: std.atomic.Value(u32), type_id: u32 }` immediately preceding the exposed payload pointer (§4.3); backed by the system allocator (§11: "Persistent heap = allocateur système direct avec refcount atomique"; no pools, no size-classes, no escape-analysis — those are §11 Phase 2+). +- Operations: `alloc(type_id, size) → ptr` (refcount = 1), `incref(ptr)` = `fetchAdd(1, .monotonic)`, `decref(ptr)` = `fetchSub(1, .release)` then drop + free at the last release (`fence(.acquire)` before drop), per §4.4 / §8.3. +- A `drop_` dispatch: a registry mapping `type_id → drop fn` so `decref`-to-zero can free type-owned resources before freeing the block (§4.3). Phase 1 needs at least the `string` drop (free the byte payload) and a no-op/identity for plain blocks. +- Immortal-interned sentinel (§4.4): a string allocated as immortal carries `refcount = u32.max`; `incref`/`decref` are no-ops on it. Compile-time string literals (resource field defaults) use this path — no per-resource-instance allocation. + +**E2 — `string` resource fields (interpreter + registry + bridge + validator).** +- `Registry.FieldKind` gains a `string_` variant (slot = `{ ptr: u64, len: u32 }`, `sizeBytes`/`alignBytes` accordingly). It is **resource-only**: components never emit it (the validator is the gate, see below); the registry/`ResourceStore` byte-block model is unchanged otherwise. +- `interp.zig` `fieldKindFromTypeName` resolves `"string"` to `.string_` **only** when `reg_kind == .resource`; `compileTypeDecl` materialises a `string` default as an **immortal-interned** slot (`{ ptr → interned literal, len }`, sentinel refcount) and `0`-inits (`{ ptr = null, len = 0 }`, the empty string) when no default is given. +- `ecs_bridge.zig`: `readResourceField` on a `.string_` slot returns a borrowed string `Value` viewing the persistent bytes (Phase-1 simplification: borrowed read, **no incref** — safe because the resource outlives the rule body; scope-bound incref/decref is §11 Phase 2). `writeResourceField` on a `.string_` slot **promotes** the incoming rule-arena/literal string into a fresh persistent allocation (copy + refcount = 1), then `decref`s the previous slot value (no-op when it was immortal) and overwrites the slot. +- Teardown discipline: on resource removal and on world/interpreter teardown, the Etch runtime iterates each resource's `FieldDesc`s and `decref`s every `.string_` slot. The Tier-0 `ResourceStore` stays string-agnostic (it only frees the byte block). +- `types.zig`: `validateFieldsInDecl` admits `string` on the `.resource` origin (the `.struct_`/`.event_` unlock now includes `.resource` for `string`). The resource sub-assertion of the guard test flips from *rejected* to *accepted*; the **component** `string` rejection and the component guard test stay green and unchanged. The standing "until tranche 7" comment is resolved. + +**E3 — enum resource fields (interpreter + registry + bridge + validator).** +- `Registry.FieldKind` gains an `enum_` variant (slot = discriminant `u32`, `sizeBytes = 4`). `FieldDesc` carries the field's declared enum **type name** so a read can reconstruct a typed `enum_value`. enum is POD (no persistent heap), and like `string_` it is resource-only. +- `interp.zig`: `fieldKindFromTypeName` resolves a declared-enum type name to `.enum_` when `reg_kind == .resource`; `compileTypeDecl` materialises an enum default (`.variant` shorthand against the field's declared enum) as its declaration-order discriminant; `0`-inits to the first variant when no default is given. +- `ecs_bridge.zig`: read a `.enum_` slot → `enum_value { type_name (from `FieldDesc`), variant = discriminant }`; write an `enum_value` → store its variant index as the discriminant. +- `types.zig`: `validateFieldsInDecl` admits enum-typed fields on the `.resource` origin (mirroring the `string` unlock); the resource enum sub-assertion (if present in the guard test) flips to accepted; component enum rejection unchanged. + +The end-state target — this exact program must compile **and** mutate end-to-end through the interpreter test harness: + +```etch +@state +resource GameState { + current_level: string = "intro" + difficulty: Difficulty = .normal + player_count: int = 0 +} + +enum Difficulty { easy, normal, hard } + +rule advance(dt: float) when resource GameState { + let mut gs = get_mut(GameState) + gs.current_level = "boss_arena" // rule-arena → persistent promotion; decref of the previous slot + gs.difficulty = .hard + gs.player_count += 1 +} +``` + +## Out-of-scope + +- **Dynamic collections on resource fields** (`string[]`, `[K: V]`, `Set`) — deferred to **M1.0.4**. The blanket `validateFieldsInDecl` rejection "collection / composite field types are not supported in E1 … fields must be scalar POD" stays in force for the `.resource` origin. (Additivity is preserved: the persistent heap is type-generic via `type_id`/`drop`; `FieldKind` is an open enum; the collection unlock extends the same machinery without reworking it.) +- **Nested-struct resource fields** — deferred (same treatment as nested-struct on `.struct_`/`.event_`, which is itself deferred). +- **An Etch `remove_resource` surface.** Tier-0 `ResourceStore.removeResource` already exists; this milestone adds no Etch construct to call it. The Phase-1 resource-string lifetime therefore ends at world/interpreter teardown (and at `removeResource` if a native caller invokes it); the teardown decref discipline must be correct regardless. +- **Persistent-heap optimisations** (a global string-interning dedup table beyond literals, size-class pools, escape-analysis-driven promotion, scope-bound read incref/decref) — `etch-memory-model.md` §11 Phase 2+. +- **Serialization of resource fields to disk** per `@config`/`@state` lifecycle annotations — a separate save/asset surface. Phase 1 here is in-memory resource fields only. +- **Codegen support for non-POD resource fields.** Interpreter-only, mirroring the M1.0.2 observer treatment: the Zig-codegen path keeps its current behaviour and does not gain persistent-heap resource fields here. +- **Component POD-strict surface** — unchanged. Components reject `string`/enum/collections exactly as before; the new `FieldKind`s are resource-only and the validator is the guarantee. +- **Editing the specs in the KB** (`engine-*.md`, `etch-*.md`) — not in the repo; reconciled by Claude.ai. Claude Code does not touch them. + +## Specs to read first + +1. `etch-memory-model.md` — §4 (persistent heap: §4.3 invisible header layout, §4.4 refcount mechanics + immortal sentinel), §5.1 (string layout `{ ptr, len }`), §6.7 / §6 (rule-arena → persistent promotion on store into a resource), §7.3 (resource dynamic fields, refcount shared with the resource), §8.3 (atomic refcount ordering), §11 (Phase 1 = system allocator + atomic refcount, no optimisations). +2. `etch-reference-part1.md` — §5.5 (POD strict is component-only; resources and locales explicitly carry `string`/dynamic arrays). +3. `etch-reference-part2.md` — §4 (`resource` construct: canonical `string` + enum field example, `@config`/`@state`/`@transient` lifecycle, `get`/`get_mut` access). +4. `etch-resolver-types.md` — §11.5 (const-required contexts — resource/component/struct field defaults must be const), §12 (resource access validations). +5. `etch-validation-ecs.md` — §7 (ECS rule/resource validations recap), §23.2 / §28 (`E0303 ResourceFieldUnknown`, ECS-access range). +6. `etch-grammar.md` — §5.5 (`resource_decl`), §2.2 / §5.4 (POD-compat reference for the contrast). +7. `engine-ecs-internals.md` — the Tier-0 resource path (`Registry`, `ResourceStore`) and how field descriptors drive byte-offset storage. +8. `etch-diagnostics.md` — §14 (E12XX) and §24.1 (`E0303`) — confirm no new diagnostic code is required; this milestone removes a rejection and flips guard-test assertions. + +## Files to create or modify + +- `src/etch/persistent.zig` — **create** — Phase-1 persistent heap: header `{ refcount: atomic u32, type_id: u32 }`, `alloc`/`incref`/`decref`, `drop_` dispatch, immortal-interned sentinel. Tier-0-agnostic (no `src/core` dependency). Inline tests. +- `src/core/ecs/registry.zig` — edit — `FieldKind.string_` (slot `{ ptr: u64, len: u32 }`) and `.enum_` (discriminant `u32`) with `sizeBytes`/`alignBytes` arms; `FieldDesc` carries the optional declared enum **type name** for `.enum_`. New kinds are resource-only by construction (the validator gates them out of components); document this in the field comment. +- `src/etch/interp.zig` — edit — `fieldKindFromTypeName` resolves `string`/declared-enum to the new kinds **only** when `reg_kind == .resource`; `compileTypeDecl` default materialisation for `string` (immortal-interned slot / empty when absent) and enum (discriminant / first variant when absent); resource teardown path that `decref`s every `.string_` slot of every resource. +- `src/etch/ecs_bridge.zig` — edit — `readResourceField`/`writeResourceField` + `readBytesAsValue`/`writeValueAsBytes` arms for `.string_` (read = borrowed view; write = persistent promotion + decref of the previous value) and `.enum_` (read = typed `enum_value` via `FieldDesc`; write = discriminant). +- `src/etch/value.zig` — edit (if required) — the minimal `Value` representation for a borrowed persistent-string read (a borrowed-string tag, or reuse of an existing `{ ptr, len }` form). Keep additive; do not disturb `string_id`/`string_run` semantics. +- `src/etch/types.zig` — edit — `validateFieldsInDecl`: `.resource` joins `.struct_`/`.event_` for the `string` and enum unlocks (the Option A alignment); flip the resource sub-assertion(s) of the guard test from *rejected* to *accepted*; keep the component rejection and its guard test green; resolve the "until tranche 7" comment. +- Tests — inline in the touched files (`persistent.zig`, `interp.zig`, and the registry/bridge sites consistent with their existing inline test blocks) + the end-state `.etch` example exercised through the interpreter test harness. + +## Acceptance criteria + +### Tests + +**E1 — persistent heap (`src/etch/persistent.zig` inline):** +- `test "alloc sets refcount 1 and decref to zero frees + drops"` — `alloc` yields refcount 1; one `decref` runs the registered `drop_` and frees; an allocation-tracking allocator confirms no leak. +- `test "incref then decref keeps the block alive until the last release"` — N increfs require N+1 decrefs to drop. +- `test "immortal-interned sentinel: incref/decref are no-ops"` — a `u32.max` block is never freed by `decref` and never double-frees. + +**E2 — `string` resource fields:** +- `src/etch/interp.zig` — `test "resource string field compiles and reads its default"` — `resource S { name: string = "intro" }` type-checks with zero diagnostics; `get(S).name` reads `"intro"` in a rule body. +- `src/etch/interp.zig` — `test "resource string field is mutable and the previous value is released"` — `get_mut(S).name = "boss_arena"` updates the field; a re-read returns the new value; an allocation-tracking allocator confirms the previous (non-immortal) persistent value was decref'd, with no leak across multiple writes. +- `src/etch/interp.zig` — `test "resource string field with no default reads empty"` — `resource S { name: string }` reads `""` before any write. +- `src/etch/types.zig` — the guard test (`test "type-checker accepts string and enum fields on struct, keeps component/resource rejection …"`) is updated: the `resource Settings { player_name: string }` sub-assertion now expects **no** `.undefined_symbol`; the `component Name { value: string }` sub-assertion still expects the rejection. + +**E3 — enum resource fields:** +- `src/etch/interp.zig` — `test "resource enum field compiles, reads its default, and is mutable"` — `resource S { diff: Difficulty = .normal }` (with `enum Difficulty { easy, normal, hard }`) type-checks clean; `get(S).diff` reads `.normal`; `get_mut(S).diff = .hard` then a re-read returns `.hard`. +- `src/etch/interp.zig` — `test "resource enum field with no default reads the first variant"`. + +**Combined (observable):** +- `src/etch/interp.zig` — `test "GameState end-state program mutates string + enum + int end-to-end"` — the § Scope end-state program runs through the harness: after `advance`, `current_level == "boss_arena"`, `difficulty == .hard`, `player_count == 1`, zero diagnostics, no leak. + +### Benchmarks + +- None. This milestone founds the Phase-1 persistent heap (system allocator, no optimisation target per `etch-memory-model.md` §11) and wires two resource field kinds through the interpreter (the dev/hot-reload path, not the shipping path). No steady-state throughput target. + +### Observable behaviour + +- The § Scope end-state `.etch` program type-checks with zero diagnostics and, run through the interpreter test harness, deterministically produces `current_level = "boss_arena"`, `difficulty = .hard`, `player_count = 1`, with the persistent heap leak-free at teardown. + +### CI + +- `zig build` clean, zero warnings, on the configured matrix. +- `zig build test` green (debug + ReleaseSafe). +- `zig fmt --check` green. +- `zig build lint` green. +- `commit-msg` hook green on every commit of the branch. +- Allocation-tracking tests (E1/E2) prove the persistent heap is leak-free across alloc/incref/decref/teardown. +- `tests/support/watchdog.zig`: no new `join()`/blocking site is introduced (the persistent heap is synchronous; refcount is atomic but lock-free) — no watchdog change expected. + +## Conventions + +- **Branch:** `phase-1/etch/resource-nonpod-fields` +- **Final tag:** `v0.10.3-resource-nonpod-fields` +- **PR title:** `Phase 1 / Etch / Non-POD resource fields (string, enum) exposed to the interpreter` +- **Commit convention:** Conventional Commits (cf. `engine-development-workflow.md` §4.3) +- **Merge strategy:** squash-and-merge (cf. `engine-development-workflow.md` §4.6) + +## Notes + +**Staging (STOP-for-review).** Implement E1, then E2, then E3. At the end of each stage, with its tests green, emit `étape E terminée, prêt pour review`, commit + push, and **wait for an explicit GO** before starting the next stage. A context overflow inside a stage is not a stop condition — continue in the same session; only the STOP-for-review gate halts. + +**This is the "Option A alignment (tranche 7)".** The resource `string`/enum rejection in `validateFieldsInDecl` and its guard test are not a bug discovered late — they are deferred work named in the code (the guard-test comment and the `FieldDeclOrigin` doc comment). M1.0.3 closes it. Do not treat the flipped guard assertion as a regression: it is the intended resolution. + +**The new `FieldKind`s are resource-only — the validator is the guarantee.** `string_`/`enum_` live in the shared `Registry`/`compileTypeDecl` path, but `fieldKindFromTypeName` emits them only for `reg_kind == .resource`, and `validateFieldsInDecl` rejects `string`/enum on `component`. The component SoA/POD invariant (`etch-reference-part1.md` §5.5, `engine-spec.md` §4) is untouched: no component can reach a non-POD `FieldKind`. State this explicitly in the registry field comment so a future reader does not mistake the new kinds for a component capability. + +**Default strings are immortal, writes allocate.** A resource `string` field default is a compile-time literal → an immortal-interned slot (sentinel refcount `u32.max`, `etch-memory-model.md` §4.4), so no allocation happens at `addResource`. A runtime write (`get_mut(R).field = expr`) is the §6.7 promotion: copy the incoming rule-arena/literal bytes into a fresh persistent allocation (refcount 1), then `decref` the previous slot value (a no-op when it was the immortal default). This keeps `addResource` allocation-free and makes the only persistent allocations the user-written values. + +**Reads are borrowed in Phase 1 (deliberate, additive).** Reading a resource `string` field yields a borrowed view over the persistent bytes with **no incref**, valid for the rule body. This is sound because the resource cannot be removed mid-body (removal is teardown/native-only here), so the bytes outlive the borrow. Full scope-bound incref-on-read / decref-on-scope-exit is `etch-memory-model.md` §11 Phase 2 and is purely additive (it does not change the slot layout, the write path, or the teardown discipline). + +**Teardown decref belongs to the Etch runtime, not Tier-0.** The `ResourceStore` is Tier-0 and string-agnostic — it frees only the byte block. The persistent strings the slots point to are decref'd by the Etch runtime (which owns the registry/`FieldDesc` knowledge and the persistent heap), iterating each resource's `.string_` slots at world/interpreter teardown and at `removeResource`. Keep this layering: do not push persistent-heap knowledge into `src/core/ecs`. + +**enum needs the declared type name on `FieldDesc`.** A read must produce a typed `enum_value { type_name, variant }`, but `readResourceField` only has the `FieldKind`. The `FieldDesc` therefore carries the field's declared enum type name for `.enum_` slots. This is a registry struct extension; it is additive (existing scalar kinds leave it null) and is required now (deferring it would force re-opening `FieldDesc` and every read site when enum lands — done now per the design rule). + +**No new diagnostic code.** Unlike M1.0.2 (which allocated E1208/E1209/E1215 for observer signatures), M1.0.3 mainly *removes* a rejection. If a positive check gap surfaces (e.g. a resource field default that is not const), reuse the existing field-default const check (`checkFieldDefault`, `etch-resolver-types.md` §11.5) rather than allocating a new code; a genuinely new code is a Cas-2 round-trip. + +**Why string + enum together, and collections separately.** The canonical spec resource (`GameState`) uses both `string` and an enum field; shipping one without the other leaves the flagship example non-compiling, and both share the single `validateFieldsInDecl` branch and the single guard test. Dynamic collections (`string[]`, map, set) are a distinct, larger surface: their runtime values are rule-arena today (`array_ref`/`map_ref`/`set_ref`), making them persistent-heap-capable is a separate value-representation change, and the validator's collection rejection is a blanket all-origins rule (not resource debt). Deferring them to M1.0.4 is additive — E1's persistent heap and E2's `FieldKind`/slot/teardown machinery are exactly what they will reuse. + +--- + +# LIVING SECTION + +*Maintained by Claude Code during the milestone.* + +## Specs read + +- [x] `etch-memory-model.md` (§4, §5.1, §6, §6.7, §7.3, §8.3, §11) — read in full 2026-06-23 22:26 +- [x] `etch-reference-part1.md` (§5.5) — read in full 2026-06-23 22:26 +- [x] `etch-reference-part2.md` (§4) — read in full 2026-06-23 22:26 +- [x] `etch-resolver-types.md` (§11.5, §12) — read in full 2026-06-23 22:26 +- [x] `etch-validation-ecs.md` (§7, §23.2, §28) — read in full 2026-06-23 22:26 +- [x] `etch-grammar.md` (§5.5, §2.2, §5.4) — read in full 2026-06-23 22:26 +- [x] `engine-ecs-internals.md` (resources Tier-0) — read in full 2026-06-23 22:26 +- [x] `etch-diagnostics.md` (§14, §24.1) — read in full 2026-06-23 22:26 + +## Execution log + +- **2026-06-23 — E0/E1 setup.** Branch created, brief copied verbatim, 8 specs read in full (§ Specs read), status → ACTIVE. Static-analysis confirmed every symbol the brief names (`FieldKind`/`FieldDesc`/`registerComponentRaw` in `registry.zig`, `ResourceStore` in `resources.zig`, `fieldKindFromTypeName`/`compileTypeDecl` in `interp.zig`, `readResourceField`/`writeResourceField`/`readBytesAsValue`/`writeValueAsBytes` in `ecs_bridge.zig`, `Value.enum_value`/`EnumValue` in `value.zig`, `validateFieldsInDecl`/`FieldDeclOrigin{component_like,resource,struct_,event_}` in `types.zig`). No ghosts. +- **2026-06-23 — E1 persistent heap.** Created `src/etch/persistent.zig` (Tier-0-agnostic, `std`-only): block = `[size:usize | refcount:atomic u32 | type_id:u32 | payload]`, exposed ptr at `block+16` so `{refcount,type_id}` is the 8 bytes immediately before payload (`etch-memory-model.md` §4.3/§5.1). `alloc`/`allocImmortal`/`incref`/`decref`/`destroy` + `runDrop` dispatch + `sentinel` immortal. Wired into the etch test graph via `root.zig`'s `comptime { _ = @import }` guard. 3 inline tests green (debug + ReleaseSafe); `zig build`, `zig build test` (exit 0), `zig build lint`, `zig fmt --check` all green. +- **2026-06-23 — Implementation note (header layout).** Spec §4.3 mandates an 8-byte `{refcount,type_id}` header *immediately preceding* the payload pointer. Zig's `Allocator.free` needs the allocation length, which a literal 8-byte header cannot carry, so the block carries a hidden leading `size: usize` word (at `p-16`) *before* the spec's 8-byte header. The spec-visible contract is preserved exactly: `{refcount,type_id}` are the 8 bytes at `p-8`/`p-4`. Pinned with comptime offset asserts. Not a scope/spec deviation — `size` is invisible bookkeeping (only `free` reads it). +- **2026-06-23 — Heads-up for E2 (FROZEN files).** `registry.zig` and `resources.zig` both carry the C0.5 (M0.9) `FROZEN` header. The brief explicitly lists `registry.zig` under "Files to create or modify" (additive `FieldKind.string_`/`.enum_` + `FieldDesc` extension; new kinds resource-only by construction, validator-gated) and explicitly keeps `resources.zig` string-agnostic (teardown decref lives in the Etch runtime, not Tier-0). The registry edit is Claude.ai-authored, additive, and brief-sanctioned; it will be implemented in E2 with the resource-only invariant stated in the field comment per the brief Notes. +- **2026-06-24 — E1 reviewed PASS (Claude.ai), GO E2.** Two implementation guards to honor (refine write/teardown, no scope change): **(1) write-promote order** — on a `.string_` slot write: read old `{ptr,len}` → `persistent.alloc(type_string,len)`+copy → write new slot → *then* `decref(old_ptr)` (decref the previous value, never after overwrite). **(2) exhaustive teardown** — resource-removal + world/interp teardown iterates *all* `.string_` slots and decrefs uniformly (no "non-immortal" filter; `decref` is already no-op on a sentinel block). Immortal defaults are not reclaimed by that slot-decref path (they have no slot-owner); they are freed separately by the interpreter's compile-time literal registry at `deinit`. Confirmed reminders: default = `allocImmortal`+memcpy (zero alloc at `addResource`); no default = `{ptr=0,len=0}` (empty); read = borrowed view, no incref; `FieldKind.string_` resource-only (validator is the guarantee); flip the resource sub-assertion of the guard test, component unchanged. +- **2026-06-24 — E2 string resource fields.** `value.zig`: `string_persistent: StrView{ptr,len}` borrowed-read variant (+ `eql` byte-compare, + `stringBytes` arm, + string-method `.len()` dispatch). `persistent.zig`: shared `StringSlot extern struct{ptr:u64,len:u32}` (16/8). `registry.zig` (FROZEN, brief-authorized): `FieldKind.string_` (size 16, align 8) with the resource-only invariant documented. `ecs_bridge.zig`: `readBytesAsValue.string_` → borrowed view; `writeValueAsBytes.string_` → defensive `TypeMismatch` (writes route around it); `promoteResourceString` (read-old → alloc+copy → write-slot → decref-old, guard 1); `comptime` size cross-check vs `StringSlot`. `interp.zig`: stored `world` + `persistent_literals` registry; `deinit` teardown (guard 2: uniform slot decref, then `destroy` immortals); `compile` threads the literal registry; `compileTypeDecl` materialises string defaults as immortal blocks; `fieldKindFromTypeName(name, reg_kind)` resource-gated; `execAssign` `.string_` promote branch. `types.zig`: `.resource` joins the `string` unlock; `FieldDeclOrigin` doc + guard test flipped (resource string → accepted, component → still rejected), "tranche 7" resolved. +- **2026-06-24 — Out-of-list file touched (justified).** `tests/etch_interp/diff_runner.zig` (S4 differential runner) got `.string_ => unreachable` arms in its two `FieldKind` switches — a mechanical exhaustiveness consequence of adding `FieldKind.string_` (a proven invariant: the POD-only S4 corpus never carries string resource fields; those are covered by inline interpreter tests). No behaviour change. +- **2026-06-24 — E2 validation.** 4 named tests green debug + ReleaseSafe (3 interp: default read, empty no-default, mutate+previous-released-no-leak; 1 types guard flip). Full suite `zig build test` 788/805 passed (17 skipped, 0 failed) debug AND ReleaseSafe; `zig build`, `zig build lint`, `zig fmt --check` green. Leak-free: the mutate test runs under `CountingAllocator`/`std.testing.allocator` (no leak across writes = previous decref'd) and the teardown reclaims the immortal default. Known out-of-scope edge: hot-reload (re-compile on the same world) of a string-resource program would need persistent-block lifetime handoff between interpreter generations — deferred; POD-only hot-reload (M0.8 E7) is unaffected (teardown skips non-`.string_` fields). +- **2026-06-24 — E2 reviewed PASS (Claude.ai), GO E3.** Deferred-debt note recorded (§ Blockers). Five E3 guards to honor: (1) discriminant = declaration-order index; (2) enum type name on `FieldDesc` with stable lifetime; (3) enum is POD — zero persistent heap / decref / teardown; (4) default `.variant` → discriminant, no default → first variant (index 0); (5) validator unlocks enum on `.resource`, component still rejected, + guard-test resource-enum assertion. +- **2026-06-24 — E3 enum resource fields.** `registry.zig` (FROZEN, brief-authorized): `FieldKind.enum_` (size 4, align 4, POD) + `FieldDesc.enum_type_name_id: u32` (propagated in `registerComponentRaw`). `ecs_bridge.zig`: `readResourceField` `.enum_` branch builds `enum_value{type_name = FieldDesc.enum_type_name_id, variant = discriminant}`; `writeValueAsBytes.enum_` writes the discriminant (POD, generic path — no promote); `readBytesAsValue.enum_` `unreachable` (resource enum branches before it; components have none). `interp.zig`: `compileTypeDecl` resolves a declared-enum field type (resource-only, via `findEnumDecl` on the AST enum slab in pass A) + sets `enum_type_name_id` + materialises the default `.variant` discriminant (first variant when absent); `execAssign` `.enum_` branch resolves the RHS `.variant` shorthand against the field's enum type (`evalEnumShorthandFor`) then writes via the generic path; free-fn `enumVariantIndex`. `types.zig`: `.resource` joins the enum unlock (`declaredEnumName`); guard test gains resource-enum (accepted) + component-enum (rejected) assertions. `diff_runner.zig`: `.enum_` added to the two `unreachable` arms (POD-only corpus). +- **2026-06-24 — Implementation note (enum type name = interned id, not a slice).** Guy's E3 guard 2 asked to store the enum type name "as a slice like `FieldDesc.name`". Implemented instead as the interned AST `StringId` (a `u32`) on `FieldDesc.enum_type_name_id`, because: (a) `enum_value.type_name` IS a `StringId` (keys `enum_decls`), so the bridge must produce that id — a name slice can't become a `StringId` without the AST string pool, which the bridge lacks; (b) a borrowed AST slice would dangle once the AST is freed while the registry persists in the world (`run`/`runProgram` free the AST after the interpreter), forcing the very dup the guard forbade; (c) a `u32` satisfies the guard's stated constraints exactly — no dup, no local slice, stable lifetime. Flagged for the final review. +- **2026-06-24 — E3 validation.** 7 M1.0.3 named tests green debug + ReleaseSafe (3 E2 string, 2 E3 enum, 1 combined GameState end-state, 1 types guard). The flagship § Scope program compiles and mutates `string` + enum + `int` end-to-end (`current_level == "boss_arena"`, `difficulty == .hard`/idx 2, `player_count == 1`), leak-free. Full suite `zig build test` 791/808 passed (17 skipped, 0 failed) debug AND ReleaseSafe; `zig build`, `zig build lint`, `zig fmt --check` green. + +## Accepted deviations + +- **2026-06-24 — `FieldDesc.enum_type_name_id` stored as a `StringId` (u32), not a string slice (Cas-3, refines E3 review guard 2).** The E3 review guard asked to store the enum type name "as a slice like `FieldDesc.name`". Implemented instead as the interned AST `StringId` (a `u32`), confirmed and preferred by Guy at the final review: `enum_value.type_name` IS a `StringId` (it keys `enum_decls`), so the bridge must produce that id — a name slice can't become a `StringId` without the AST string pool (the bridge has none), and a borrowed AST slice would dangle once the AST is freed while the registry persists in the world (forcing the dup the guard forbade). A `u32` satisfies the guard's constraints exactly (no dup, no local slice, stable lifetime). Trade-off recorded in § Closing notes residual debt (multi-AST reload would resolve against the wrong pool — a logic bug, not a UAF; strictly safer than the slice). + +## Blockers encountered + +- **2026-06-24 — Deferred-debt note (not a blocker; recorded per E2 review).** Ownership des strings persistentes resource lié à l'`Interpreter` (`persistent_literals` + slot-decref dans `deinit`). Correct pour tout chemin actuel (mono-programme). Sous un futur pipeline de hot-reload script — world réutilisé à travers un swap d'interpréteur, aujourd'hui stub `src/runtime/main.zig:297`, hors scope — l'ownership devra passer à la durée de vie world/resource (memory-model §4.2/§7.3). Additif au moment du câblage du reload ; consigné ici pour ce milestone futur. + +## Closing notes + +- **What worked:** + - **E1 persistent heap as a clean keystone.** A `std`-only, Tier-0-agnostic `persistent.zig` (refcount + `type_id`→drop dispatch + immortal sentinel) built once and reused unchanged by E2 — `StringSlot` and the literal registry slotted straight in. The drop dispatch and open `TypeId` set are exactly what M1.0.4 collections will register against. + - **Resource-only by construction, validator as the guarantee.** `string_`/`enum_` live in the shared `Registry`/`compileTypeDecl` path but `fieldKindFromTypeName` emits them only for `.resource`, and `validateFieldsInDecl` rejects them on `component` — so the component SoA/POD invariant was never at risk. The guard test pins all four cases (struct/resource accept, component rejects, for both `string` and enum). + - **Write-promote vs POD-write split.** Strings need an allocator + the old slot (promote path); enums are self-contained discriminants (generic `writeValueAsBytes`). Routing strings around the byte-encoder and enums through it kept each path minimal and correct. + - **Staged E1→E2→E3 with STOP-reviews + Claude.ai round-trips** surfaced and settled the two subtle design points (header-layout bookkeeping; enum type-name representation) without re-scope. + +- **What deviated from the original spec:** (tracked in § Accepted deviations) + - **E3** — `FieldDesc` carries the enum type name as the interned `StringId` (`enum_type_name_id: u32`) rather than a string slice (refines review guard 2; confirmed by Guy). The bridge must produce a `StringId` for `enum_value.type_name`, and a slice would dangle post-AST-free while the registry persists — the id satisfies the guard's constraints exactly. + - **Implementation detail (no scope/contract change):** the persistent block carries a hidden leading `size: usize` word before the spec's 8-byte `{refcount,type_id}` header so Zig's `Allocator.free` can recover the length; the spec-visible header is preserved exactly (pinned by comptime offset asserts). + +- **What to flag explicitly in review:** + - **`registry.zig` (FROZEN, C0.5/M0.9) edited** — additive `FieldKind.string_`/`.enum_` + `FieldDesc.enum_type_name_id`, explicitly listed in the brief's "Files to create or modify". `resources.zig` (also FROZEN) stayed string-agnostic per the brief (teardown decref is the Etch runtime's, not Tier-0's). + - **`tests/etch_interp/diff_runner.zig` (out-of-list) touched** — `.string_`/`.enum_ => unreachable` in its two `FieldKind` switches, a mechanical exhaustiveness consequence (POD-only corpus, proven invariant). No behaviour change. + - **No new diagnostic codes** — M1.0.3 removes rejections / flips guard assertions (the intended Option-A resolution, not regressions); `etch-diagnostics.md` needs no patch. + - **Codegen unchanged** — non-POD resource fields are interpreter-only (per § Out-of-scope); the Zig-codegen path is untouched. + +- **Final measures:** + - **No benchmarks** (brief: none — founds the Phase-1 persistent heap + wires two field kinds through the dev/hot-reload interpreter path; no steady-state target). + - **Tests added**: 3 E1 (persistent heap, inline in `persistent.zig`) + 4 E2 (3 interp string + the flipped/extended types guard) + 3 E3 (2 interp enum + the combined flagship GameState end-to-end). The flagship § Scope program type-checks with zero diagnostics and mutates `string` + enum + `int` end-to-end (`current_level=="boss_arena"`, `difficulty==.hard`/idx 2, `player_count==1`), leak-free. + - `zig build test` **exit 0 in debug AND ReleaseSafe** (791/808 passed, 17 skipped, 0 failed); `zig build`, `zig fmt --check`, `zig build lint`, `commit-msg` all green. Persistent-heap leak-freedom proven under `std.testing.allocator` (+ `CountingAllocator` for the previous-value-released assertion). No new `join()`/blocking site — `tests/support/watchdog.zig` unchanged (the heap is synchronous, refcount lock-free). + +- **Residual risk / technical debt left intentionally:** + - **Borrowed-read aliasing (E2)** — reading a resource `string` field yields a borrowed view with no incref (the brief's deliberate Phase-1 choice; scope-bound incref-on-read is Phase 2, `etch-memory-model.md` §11). Hazardous pattern, NOT triggered by any test or the flagship: `let s = get(R).f` → overwrite the *same* field with a previously-refcounted value → use `s` would dangle (the overwrite's `decref` frees the block `s` views; reading the immortal default then overwriting is safe). Closed additively by Phase-2 incref-on-read. + - **Persistent-string ownership tied to the `Interpreter` (E2)** — `persistent_literals` + slot-decref in `deinit`. Correct for every current path (single-program). Under a future script hot-reload pipeline (world reused across an interpreter swap — today the stub `src/runtime/main.zig:297`, out of scope) ownership must move to world/resource lifetime (`etch-memory-model.md` §4.2/§7.3); additive at reload wire-up. Same future milestone for `FieldDesc.enum_type_name_id`: a `StringId` from one AST stored in the world's registry would resolve against the wrong pool under a multi-AST reload (a logic bug, not a UAF — strictly safer than a slice). + - **M1.0.4 — dynamic collections on resource fields** (`string[]`, `[K: V]`, `Set`) — deferred from M1.0.3. The validator's "collection / composite field types … must be scalar POD" rejection stays blanket all-origins. They reuse E1's type-generic persistent heap and E2's `FieldKind`/slot/teardown machinery unchanged — additive. diff --git a/src/core/ecs/registry.zig b/src/core/ecs/registry.zig index cb12203..8c965d0 100644 --- a/src/core/ecs/registry.zig +++ b/src/core/ecs/registry.zig @@ -39,6 +39,22 @@ pub const FieldKind = enum { u32_, f32_, f64_, + /// A `string` field slot: `{ ptr: u64, len: u32 }` (16 bytes, 8-aligned) + /// pointing into the Etch persistent heap (`src/etch/persistent.zig`, + /// `StringSlot`). **Resource-only by construction** (M1.0.3): the Etch + /// validator rejects `string` on `component` and `fieldKindFromTypeName` + /// only emits this kind for the `.resource` origin, so no component can ever + /// carry it — the component SoA/POD invariant (`engine-spec.md` §4) is + /// untouched. Tier-0 stays string-agnostic: it stores/copies the 16 raw + /// slot bytes; the Etch runtime owns the pointed-to bytes' lifetime. + string_, + /// An enum field slot: the variant's declaration-order index as a `u32` + /// discriminant (4 bytes, 4-aligned). POD — no persistent heap, no decref, + /// no teardown. **Resource-only** like `.string_` (validator-gated out of + /// components). The declared enum type's interned name id rides on + /// `FieldDesc.enum_type_name_id` so the Etch bridge can rebuild a typed + /// `enum_value{ type_name, variant }` on read. + enum_, pub fn sizeBytes(self: FieldKind) usize { return switch (self) { @@ -49,6 +65,10 @@ pub const FieldKind = enum { .u32_ => @sizeOf(u32), .f32_ => @sizeOf(f32), .f64_ => @sizeOf(f64), + // `{ ptr: u64, len: u32 }` padded to 8-alignment — must equal + // `@sizeOf(persistent.StringSlot)` (asserted in `ecs_bridge.zig`). + .string_ => 16, + .enum_ => @sizeOf(u32), // declaration-order discriminant }; } @@ -61,6 +81,8 @@ pub const FieldKind = enum { .u32_ => @alignOf(u32), .f32_ => @alignOf(f32), .f64_ => @alignOf(f64), + .string_ => 8, + .enum_ => @alignOf(u32), }; } @@ -83,6 +105,14 @@ pub const FieldDesc = struct { name: []const u8, offset: u16, kind: FieldKind, + /// For a `.enum_` field (resource-only, M1.0.3 E3): the Etch-interned id of + /// the declared enum type name (an AST `StringId`, kept opaque by Tier-0 — + /// a plain `u32`, never dereferenced here). Lets the Etch bridge rebuild a + /// typed `enum_value{ type_name, variant }` on read with no string pool. + /// Stored as the id (not a string) so it needs no allocation and cannot + /// dangle when the AST outlives nothing while the registry persists in the + /// world. `0` and unused for every non-`.enum_` kind. + enum_type_name_id: u32 = 0, }; /// Full descriptor stored by the registry. `default_bytes` is `size` bytes @@ -166,7 +196,12 @@ pub const Registry = struct { errdefer for (fields_owned[0..dup_count]) |f| gpa.free(f.name); for (desc.fields, 0..) |f, i| { const fname_owned = try gpa.dupe(u8, f.name); - fields_owned[i] = .{ .name = fname_owned, .offset = f.offset, .kind = f.kind }; + fields_owned[i] = .{ + .name = fname_owned, + .offset = f.offset, + .kind = f.kind, + .enum_type_name_id = f.enum_type_name_id, + }; dup_count += 1; } diff --git a/src/etch/ecs_bridge.zig b/src/etch/ecs_bridge.zig index 9db132d..04022ba 100644 --- a/src/etch/ecs_bridge.zig +++ b/src/etch/ecs_bridge.zig @@ -9,6 +9,7 @@ const std = @import("std"); const value_mod = @import("value.zig"); +const persistent = @import("persistent.zig"); const weld_core = @import("weld_core"); const RegistryNS = weld_core.ecs.registry; @@ -34,6 +35,13 @@ const EntityId = value_mod.EntityId; const Value = value_mod.Value; const ComponentRef = value_mod.ComponentRef; +comptime { + // The `.string_` slot stride the registry reports must match the canonical + // `StringSlot` layout (`persistent.zig`) the bridge reads/writes — one + // source of truth across the Tier-0 / Etch boundary. + std.debug.assert(@sizeOf(persistent.StringSlot) == FieldKind.string_.sizeBytes()); +} + /// Surfaced so callers of `Bridge.dispatchEntityGet` / /// `dispatchResourceGet` can map a name-resolution failure into a /// typed E-code without depending on `Registry`'s raw lookup return. @@ -180,6 +188,16 @@ pub const Bridge = struct { const bytes = store.getResource(resource_id) orelse return BridgeError.UnknownResource; const field = registry.findField(resource_id, field_name) orelse return BridgeError.UnknownField; const slice = bytes[field.offset .. field.offset + @as(u16, @intCast(field.kind.sizeBytes()))]; + // Enum read (M1.0.3 E3): rebuild a typed `enum_value` from the slot's + // discriminant + the declared enum type's interned id on `FieldDesc` + // (the byte-only `readBytesAsValue` has no access to the latter). The + // `type_name` id matches the rest of the interpreter's enum machinery + // (`enum_decls` is keyed by it), so the value compares/matches correctly. + if (field.kind == .enum_) { + var disc: u32 = 0; + @memcpy(std.mem.asBytes(&disc), slice[0..@sizeOf(u32)]); + return .{ .enum_value = .{ .type_name = field.enum_type_name_id, .variant = disc } }; + } return readBytesAsValue(field.kind, slice); } @@ -195,6 +213,43 @@ pub const Bridge = struct { const slice = bytes[field.offset .. field.offset + @as(u16, @intCast(field.kind.sizeBytes()))]; try writeValueAsBytes(field.kind, slice, v); } + + /// Promote `bytes` into a fresh persistent allocation and store it in a + /// resource `.string_` slot (`etch-memory-model.md` §6.7 rule-arena → + /// persistent promotion). The interpreter resolves the incoming string's + /// bytes (literal / rule-arena) and hands them here with the allocator. + /// + /// Order is load-bearing (M1.0.3 E2 review guard, anti use-after-free): read + /// the old slot → alloc + copy the new value → write the new slot → only then + /// `decref` the *previous* slot value (never after overwriting it). The + /// previous value's `decref` is a no-op when it was the immortal default. An + /// empty write stores `{ptr=0,len=0}` and allocates nothing. + pub fn promoteResourceString( + gpa: std.mem.Allocator, + registry: *const Registry, + store: *ResourceStore, + resource_id: ComponentId, + field_name: []const u8, + bytes: []const u8, + ) BridgeError!void { + const field = registry.findField(resource_id, field_name) orelse return BridgeError.UnknownField; + std.debug.assert(field.kind == .string_); + const buf = store.getMutResource(resource_id) orelse return BridgeError.UnknownResource; + const slot = buf[field.offset .. field.offset + @sizeOf(persistent.StringSlot)]; + + var old: persistent.StringSlot = undefined; + @memcpy(std.mem.asBytes(&old), slot); + + var new_slot: persistent.StringSlot = .{}; + if (bytes.len > 0) { + const block = persistent.alloc(gpa, persistent.type_string, bytes.len) catch return BridgeError.OutOfMemory; + @memcpy(block[0..bytes.len], bytes); + new_slot = .{ .ptr = @intFromPtr(block), .len = @intCast(bytes.len) }; + } + @memcpy(slot, std.mem.asBytes(&new_slot)); + + if (old.ptr != 0) persistent.decref(gpa, @ptrFromInt(old.ptr)); + } }; // ─── Byte ↔ Value conversion ───────────────────────────────────────────── @@ -236,6 +291,19 @@ pub fn readBytesAsValue(kind: FieldKind, bytes: []const u8) Value { @memcpy(std.mem.asBytes(&v), bytes[0..@sizeOf(f64)]); break :blk .{ .float_ = v }; }, + // Borrowed read (M1.0.3 E2, resource-only): decode the `{ptr,len}` slot + // into a `string_persistent` view without incref'ing the block. `ptr==0` + // ⇔ empty string (the no-default / empty-write representation). + .string_ => blk: { + var ss: persistent.StringSlot = undefined; + @memcpy(std.mem.asBytes(&ss), bytes[0..@sizeOf(persistent.StringSlot)]); + break :blk .{ .string_persistent = .{ .ptr = ss.ptr, .len = ss.len } }; + }, + // Enum reads need the declared type's id (on `FieldDesc`), which this + // byte-only decoder lacks — `readResourceField` handles `.enum_` before + // delegating here, and components never carry `.enum_` (validator-gated). + // Proven invariant: this arm is never reached. + .enum_ => unreachable, }; } @@ -292,6 +360,22 @@ pub fn writeValueAsBytes(kind: FieldKind, bytes: []u8, v: Value) BridgeError!voi }; @memcpy(bytes[0..@sizeOf(f32)], std.mem.asBytes(&x)); }, + // A `.string_` write is a persistent promotion (alloc + copy + decref of + // the previous slot), which needs an allocator and the old slot bytes — + // the POD byte-encoder has neither. Resource string writes route through + // `promoteResourceString`; components never carry `.string_` (validator- + // gated). Reaching here is a bug, surfaced as a typed error, never a panic. + .string_ => return error.TypeMismatch, + // Enum write (M1.0.3 E3): store the variant's declaration-order index as + // the `u32` discriminant. POD — self-contained in the `enum_value`, so + // (unlike `.string_`) it goes through the generic write path. + .enum_ => { + const disc: u32 = switch (v) { + .enum_value => |e| e.variant, + else => return error.TypeMismatch, + }; + @memcpy(bytes[0..@sizeOf(u32)], std.mem.asBytes(&disc)); + }, } } diff --git a/src/etch/interp.zig b/src/etch/interp.zig index ccfcb8f..810e1b2 100644 --- a/src/etch/interp.zig +++ b/src/etch/interp.zig @@ -21,6 +21,7 @@ const value_mod = @import("value.zig"); const bridge_mod = @import("ecs_bridge.zig"); const tags_mod = @import("tags.zig"); const descriptor_mod = @import("descriptor.zig"); +const persistent = @import("persistent.zig"); const weld_core = @import("weld_core"); const Registry = weld_core.ecs.registry.Registry; @@ -651,8 +652,44 @@ pub const Interpreter = struct { /// route here instead of `pending_tags`, so they apply at the NEXT flush — /// never re-entrantly during the current one (the no-recursion contract). observer_deferred: ?*CommandBuffer = null, + /// The world this program was compiled against (`compile`), borrowed for the + /// persistent-string teardown in `deinit` (M1.0.3 E2). The interpreter is + /// already lifecycle-coupled to the world (its observer ctxs are registered + /// into the world's `ObserverRegistry`), so storing it here is consistent; + /// the world MUST outlive the interpreter (the existing contract — `deinit` + /// before `world.deinit`). `null` only before `compile` returns. + world: ?*World = null, + /// Immortal persistent-heap blocks holding compile-time `string` field + /// defaults (M1.0.3 E2). Allocated in `compileTypeDecl` via `allocImmortal` + /// (sentinel refcount, so slot decref never frees them) and `destroy`'d here + /// at `deinit` — they have no slot-owner to reclaim them, so the interpreter + /// (their allocator) does. Freed AFTER the per-slot decref so an un-overwritten + /// default (slot still points at its immortal block) is reclaimed exactly once. + persistent_literals: std.ArrayListUnmanaged([*]u8) = .empty, pub fn deinit(self: *Interpreter) void { + // M1.0.3 E2 — persistent-string teardown, BEFORE `bridge.deinit` (which + // drops the resource-name → id map this enumeration needs) and while the + // world's resource store is still alive (freed later by `world.deinit`). + // Iterate every resource's `.string_` slots and `decref` uniformly: a + // sentinel (immortal default) block no-ops, a refcounted user-written + // block frees. Then `destroy` the immortal defaults via the literal + // registry — no double-free: the slot decref left immortals alive. + if (self.world) |w| { + var it = self.bridge.resources.valueIterator(); + while (it.next()) |id_ptr| { + const rid = id_ptr.*; + const buf = w.resources.getResource(rid) orelse continue; + for (w.registry.componentFields(rid)) |fd| { + if (fd.kind != .string_) continue; + var ss: persistent.StringSlot = undefined; + @memcpy(std.mem.asBytes(&ss), buf[fd.offset .. fd.offset + @sizeOf(persistent.StringSlot)]); + if (ss.ptr != 0) persistent.decref(self.gpa, @ptrFromInt(ss.ptr)); + } + } + } + for (self.persistent_literals.items) |block| persistent.destroy(self.gpa, block); + self.persistent_literals.deinit(self.gpa); for (self.rule_descs) |*r| r.deinit(self.gpa); self.gpa.free(self.rule_descs); self.bridge.deinit(self.gpa); @@ -728,14 +765,24 @@ pub const Interpreter = struct { var tag_table = try tags_mod.TagTable.build(gpa, ast, &tag_diags, tags_mod.default_max_tags); errdefer tag_table.deinit(gpa); + // Immortal blocks backing compile-time `string` field defaults (M1.0.3 + // E2). Filled by `compileTypeDecl`; moved into the returned interpreter, + // which `destroy`s them at `deinit`. On a compile error path they are + // reclaimed here so no default literal leaks. + var persistent_literals: std.ArrayListUnmanaged([*]u8) = .empty; + errdefer { + for (persistent_literals.items) |block| persistent.destroy(gpa, block); + persistent_literals.deinit(gpa); + } + // Pass A — register components and resources with the world. var i: u28 = 0; while (i < ast.items.len) : (i += 1) { const kind = ast.items.items(.kind)[i]; const data = ast.items.items(.data)[i]; switch (kind) { - .component_decl => try compileComponent(gpa, ast, world, &bridge, ast.component_decls.items[data]), - .resource_decl => try compileResource(gpa, ast, world, &bridge, ast.resource_decls.items[data]), + .component_decl => try compileComponent(gpa, ast, world, &bridge, ast.component_decls.items[data], &persistent_literals), + .resource_decl => try compileResource(gpa, ast, world, &bridge, ast.resource_decls.items[data], &persistent_literals), else => {}, } } @@ -919,6 +966,8 @@ pub const Interpreter = struct { .has_async = any_async, .async_slots = async_slots, .descriptors = descriptors, + .world = world, + .persistent_literals = persistent_literals, }; } @@ -1999,6 +2048,34 @@ pub const Interpreter = struct { }, .resource_ref => |rref| { if (!rref.mutable) return error.RuntimeFailure; + // A `.string_` slot write is a persistent promotion (M1.0.3 + // E2), not a byte-block overwrite. Resolve the incoming + // string's bytes (literal / rule-arena) here, then hand them + // to `promoteResourceString`, which allocs the fresh block, + // writes the new slot, and decrefs the previous value (order + // enforced there). Only plain `=` is in the M1.0.3 surface; + // a compound op on a string slot is a runtime failure. + if (world.registry.findField(rref.resource_id, field_name)) |field| { + if (field.kind == .string_) { + if (assign.op != .assign) return error.RuntimeFailure; + const rhs = try self.evalExpr(world, locals, assign.value); + const bytes = self.stringBytes(rhs) orelse return error.RuntimeFailure; + Bridge.promoteResourceString(self.gpa, &world.registry, &world.resources, rref.resource_id, field_name, bytes) catch |e| + return self.fail(bridgeFailureKind(e), self.ast.exprSpan(assign.target)); + return; + } + if (field.kind == .enum_) { + // Enum slot write (M1.0.3 E3): resolve the RHS `.variant` + // shorthand against the field's declared enum type (the + // assignment position carries no expected-type context to + // `evalExpr`), then store its discriminant. Only `=`. + if (assign.op != .assign) return error.RuntimeFailure; + const ev = try self.evalEnumShorthandFor(world, locals, assign.value, field.enum_type_name_id); + Bridge.writeResourceField(&world.registry, &world.resources, rref.resource_id, field_name, ev) catch |e| + return self.fail(bridgeFailureKind(e), self.ast.exprSpan(assign.target)); + return; + } + } const cur = Bridge.readResourceField(&world.registry, &world.resources, rref.resource_id, field_name) catch |e| return self.fail(bridgeFailureKind(e), self.ast.exprSpan(assign.target)); const rhs = try self.evalExpr(world, locals, assign.value); @@ -2110,6 +2187,22 @@ pub const Interpreter = struct { return Value{ .enum_value = .{ .type_name = ename, .variant = vidx } }; } + /// Resolve an enum assignment RHS against a known enum type id (M1.0.3 E3, + /// resource enum write). A bare `.variant` shorthand resolves to a typed + /// `enum_value` against `enum_type_name_id`; any other form (a variable, a + /// qualified `Type.variant`) is evaluated normally — its tag already carries + /// the type. The `type_name` id matches `enum_decls`'s keying, so the stored + /// discriminant and any later read/compare agree. + fn evalEnumShorthandFor(self: *Interpreter, world: *World, locals: *Locals, value: NodeId, enum_type_name_id: u32) StmtError!Value { + if (self.ast.exprKind(value) == .tag_path) { + const edecl = self.enum_decls.get(enum_type_name_id) orelse return error.RuntimeFailure; + const variant: StringId = self.ast.exprData(value); + const vidx = self.enumVariantIndexOf(edecl, variant) orelse return error.RuntimeFailure; + return Value{ .enum_value = .{ .type_name = enum_type_name_id, .variant = vidx } }; + } + return try self.evalExpr(world, locals, value); + } + /// The declared-struct name of a field's `.named` type node, or null when /// the field is not struct-typed (M0.8 E3-C tranche 8) — the same /// declared-type lookup as `enumFieldShorthand`, for the anonymous @@ -2202,12 +2295,14 @@ pub const Interpreter = struct { const method = self.trait_methods.get(methodKey(entity_name, mc.method_name)) orelse return error.RuntimeFailure; return try self.callMethod(world, locals, method, mc, recv); }, - .string_id, .string_run => { + .string_id, .string_run, .string_persistent => { // Builtin string methods (M0.8 sub-slice C tranche 1 — // minimal faithful subset). `len` → byte length, on a - // literal (`string_id`) or a runtime-produced string - // (`string_run`, tranche 1b); any other §12 method is - // stdlib Phase 1+ → fail loud. + // literal (`string_id`), a runtime-produced string + // (`string_run`, tranche 1b), or a borrowed resource-string + // view (`string_persistent`, M1.0.3 E2); any other §12 method + // is stdlib Phase 1+ → fail loud. `stringBytes` already covers + // all three forms. const mname = self.ast.strings.slice(mc.method_name); if (std.mem.eql(u8, mname, "len")) { const bytes = self.stringBytes(recv) orelse return error.RuntimeFailure; @@ -2405,12 +2500,18 @@ pub const Interpreter = struct { self.run_strings.clearRetainingCapacity(); } - /// The bytes of a string value — an AST-table literal (`string_id`) or a - /// runtime-produced string (`string_run`). Null for any non-string value. + /// The bytes of a string value — an AST-table literal (`string_id`), a + /// runtime-produced string (`string_run`), or a borrowed resource-string + /// view (`string_persistent`, M1.0.3 E2). Null for any non-string value. fn stringBytes(self: *const Interpreter, v: Value) ?[]const u8 { return switch (v) { .string_id => |sid| self.ast.strings.slice(sid), .string_run => |handle| self.run_strings.items[handle], + .string_persistent => |s| blk: { + if (s.len == 0) break :blk &.{}; + const p: [*]const u8 = @ptrFromInt(s.ptr); + break :blk p[0..s.len]; + }, else => null, }; } @@ -3212,9 +3313,10 @@ fn compileComponent( world: *World, bridge: *Bridge, decl: ast_mod.ComponentDecl, + literals: *std.ArrayListUnmanaged([*]u8), ) !void { const name = ast.strings.slice(decl.name); - _ = try compileTypeDecl(gpa, ast, world, bridge, name, decl.fields_start, decl.fields_len, .component); + _ = try compileTypeDecl(gpa, ast, world, bridge, name, decl.fields_start, decl.fields_len, .component, literals); } fn compileResource( @@ -3223,13 +3325,14 @@ fn compileResource( world: *World, bridge: *Bridge, decl: ast_mod.ResourceDecl, + literals: *std.ArrayListUnmanaged([*]u8), ) !void { const name = ast.strings.slice(decl.name); // On a hot-reload re-compile (M0.8 E7) the resource is already registered // AND already lives in the resource store with its current value — adding // it again would reset it to defaults. Seed the store only on first compile. const pre_existing = world.registry.idOf(name) != null; - const id = try compileTypeDecl(gpa, ast, world, bridge, name, decl.fields_start, decl.fields_len, .resource); + const id = try compileTypeDecl(gpa, ast, world, bridge, name, decl.fields_start, decl.fields_len, .resource, literals); if (!pre_existing) { const default_bytes = world.registry.componentDefaultBytes(id); try world.addResource(gpa, id, default_bytes); @@ -3247,6 +3350,7 @@ fn compileTypeDecl( fields_start: u32, fields_len: u32, reg_kind: RegKind, + literals: *std.ArrayListUnmanaged([*]u8), ) !ComponentId { var fields: std.ArrayListUnmanaged(FieldDesc) = .empty; defer fields.deinit(gpa); @@ -3258,8 +3362,20 @@ fn compileTypeDecl( const f = ast.fields.items[fields_start + f_i]; const tnode = ast.named_types.items[ast.typeNodeData(f.type_node)]; // Resolve through any top-level `type` alias chain (M0.8 foundations). - const tname = ast.strings.slice(ast.resolveTypeAliasName(tnode.name)); - const kind = fieldKindFromTypeName(tname) orelse return error.InvalidProgram; + const resolved_name_id = ast.resolveTypeAliasName(tnode.name); + const tname = ast.strings.slice(resolved_name_id); + var enum_type_id: u32 = 0; + const kind = fieldKindFromTypeName(tname, reg_kind) orelse blk: { + // Enum resource field (M1.0.3 E3): a declared enum type, resource-only + // (mirrors the `string` gate; components reject enum at the validator). + // `enum_decls` is not built yet in this pass, so match the AST enum + // slab directly. The declared enum type's id rides on the FieldDesc. + if (reg_kind == .resource and findEnumDecl(ast, resolved_name_id) != null) { + enum_type_id = resolved_name_id; + break :blk FieldKind.enum_; + } + return error.InvalidProgram; + }; const align_b = kind.alignBytes(); if (align_b > max_align) max_align = align_b; const off = std.mem.alignForward(usize, size, align_b); @@ -3268,6 +3384,7 @@ fn compileTypeDecl( .name = ast.strings.slice(f.name), .offset = @intCast(off), .kind = kind, + .enum_type_name_id = enum_type_id, }); } size = std.mem.alignForward(usize, size, max_align); @@ -3278,10 +3395,46 @@ fn compileTypeDecl( f_i = 0; while (f_i < fields_len) : (f_i += 1) { const f = ast.fields.items[fields_start + f_i]; - if (f.default_value.isNone()) continue; - const v = evalConst(ast, f.default_value) catch continue; const fd = fields.items[f_i]; const slot = default_buf[fd.offset .. fd.offset + @as(u16, @intCast(fd.kind.sizeBytes()))]; + if (fd.kind == .string_) { + // Resource `string` default = compile-time literal → an immortal + // interned block (sentinel refcount): `addResource` copies only the + // 16-byte `{ptr,len}` slot, no per-instance allocation. No default ⇒ + // slot stays `{ptr=0,len=0}` (the empty string; `default_buf` is + // zeroed). The block is owned by `literals` and `destroy`'d at the + // interpreter's `deinit`. Non-literal const string defaults are out + // of the M1.0.3 surface; they leave the empty-string slot. + if (!f.default_value.isNone() and ast.exprKind(f.default_value) == .string_lit) { + const lit = ast.strings.slice(ast.exprData(f.default_value)); + const block = try persistent.allocImmortal(gpa, persistent.type_string, lit.len); + literals.append(gpa, block) catch |e| { + persistent.destroy(gpa, block); + return e; + }; + if (lit.len > 0) @memcpy(block[0..lit.len], lit); + const ss = persistent.StringSlot{ .ptr = @intFromPtr(block), .len = @intCast(lit.len) }; + @memcpy(slot, std.mem.asBytes(&ss)); + } + continue; + } + if (fd.kind == .enum_) { + // Enum default = a bare `.variant` shorthand → its declaration-order + // discriminant (consistent with `EnumValue.variant`). No default ⇒ + // discriminant 0, the first variant (`default_buf` is zeroed). + if (!f.default_value.isNone() and ast.exprKind(f.default_value) == .tag_path) { + const variant = ast.exprData(f.default_value); + if (findEnumDecl(ast, fd.enum_type_name_id)) |edecl| { + if (enumVariantIndex(ast, edecl, variant)) |vidx| { + const disc: u32 = vidx; + @memcpy(slot[0..@sizeOf(u32)], std.mem.asBytes(&disc)); + } + } + } + continue; + } + if (f.default_value.isNone()) continue; + const v = evalConst(ast, f.default_value) catch continue; try bridge_mod.writeValueAsBytes(fd.kind, slot, v); } @@ -3314,7 +3467,7 @@ fn compileTypeDecl( return id; } -fn fieldKindFromTypeName(name: []const u8) ?FieldKind { +fn fieldKindFromTypeName(name: []const u8, reg_kind: RegKind) ?FieldKind { if (std.mem.eql(u8, name, "int")) return .int_; if (std.mem.eql(u8, name, "float")) return .float_; if (std.mem.eql(u8, name, "bool")) return .bool_; @@ -3322,6 +3475,33 @@ fn fieldKindFromTypeName(name: []const u8) ?FieldKind { if (std.mem.eql(u8, name, "u32")) return .u32_; if (std.mem.eql(u8, name, "f32")) return .f32_; if (std.mem.eql(u8, name, "f64")) return .f64_; + // `string` is resource-only (M1.0.3 E2): components are POD-strict (the + // validator rejects `string` on them), so this kind is emitted only for the + // `.resource` origin. Components reaching here with `string` fall to `null` + // → `error.InvalidProgram` (a validator-passed program never does). Enum + // field types are not builtin names, so they resolve in `compileTypeDecl` + // against the AST enum slab (see `findEnumDecl`), not here. + if (reg_kind == .resource and std.mem.eql(u8, name, "string")) return .string_; + return null; +} + +/// The declared `enum` with the given (alias-resolved) interned name, or null +/// — scanned against the AST enum slab so it works in `compileTypeDecl` (pass A, +/// before `enum_decls` is indexed). Mirrors the type-checker's `declaredEnumName`. +fn findEnumDecl(ast: *const AstArena, name: StringId) ?ast_mod.EnumDecl { + for (ast.enum_decls.items) |d| { + if (d.name == name) return d; + } + return null; +} + +/// Declaration-order index of `variant` within `edecl`, or null (free-fn twin +/// of `Interpreter.enumVariantIndexOf`, for the allocator-free compile pass). +fn enumVariantIndex(ast: *const AstArena, edecl: ast_mod.EnumDecl, variant: StringId) ?u32 { + var i: u32 = 0; + while (i < edecl.variants_len) : (i += 1) { + if (ast.enum_variants.items[edecl.variants_start + i].name == variant) return i; + } return null; } @@ -3979,6 +4159,284 @@ test "runProgram resource get/get_mut without receiver reads and writes the reso try std.testing.expectEqual(@as(i32, 15), points); } +// ── M1.0.3 E2 — resource `string` fields ────────────────────────────────── + +/// Read a resource `string` field through the bridge and assert its bytes. +/// `string_persistent` with `ptr == 0` is the empty string. +fn expectResourceStringField( + world: *World, + res_name: []const u8, + field_name: []const u8, + expected: []const u8, +) !void { + const rid = world.registry.idOf(res_name).?; + const v = try bridge_mod.Bridge.readResourceField(&world.registry, &world.resources, rid, field_name); + const got: []const u8 = switch (v) { + .string_persistent => |s| if (s.len == 0) "" else @as([*]const u8, @ptrFromInt(s.ptr))[0..s.len], + else => return error.TestExpectedStringValue, + }; + try std.testing.expectEqualStrings(expected, got); +} + +test "resource string field compiles and reads its default (M1.0.3 E2)" { + const gpa = std.testing.allocator; + var world = World.init(); + defer world.deinit(gpa); + + // The string default `"intro"` materializes as an immortal interned block; + // the rule reads it in-body (`get(S).name`) and stores its byte length, so a + // wrong read would surface as a wrong `len`. + const source = + \\resource S { + \\ name: string = "intro" + \\ n: int = 0 + \\} + \\rule touch() + \\ when resource S + \\{ + \\ let s = get(S).name + \\ get_mut(S).n = s.len() + \\} + ; + var pr = try parser_mod.parse(gpa, source); + defer pr.deinit(gpa); + try std.testing.expect(pr.diagnostics.len == 0); + + var diags: std.ArrayListUnmanaged(Diagnostic) = .empty; + defer { + for (diags.items) |*d| d.deinit(gpa); + diags.deinit(gpa); + } + try types_mod.TypeChecker.check(gpa, &pr.ast, &diags); + try std.testing.expectEqual(@as(usize, 0), diags.items.len); + + var interp = try Interpreter.compile(gpa, &pr.ast, &world); + defer interp.deinit(); + const report = try interp.runFor(&world, 1); + try std.testing.expectEqual(@as(u64, 0), report.runtime_errors); + + // The default reads back as "intro", and the in-body read fed `n = 5`. + try expectResourceStringField(&world, "S", "name", "intro"); + const rid = world.registry.idOf("S").?; + const bytes = world.resources.getResource(rid).?; + const n_field = world.registry.findField(rid, "n").?; + var n: i64 = 0; + @memcpy(std.mem.asBytes(&n), bytes[n_field.offset .. n_field.offset + @sizeOf(i64)]); + try std.testing.expectEqual(@as(i64, 5), n); +} + +test "resource string field with no default reads empty (M1.0.3 E2)" { + const gpa = std.testing.allocator; + var world = World.init(); + defer world.deinit(gpa); + + // No default ⇒ the slot stays `{ptr=0,len=0}` — the empty string. + var pr = try parser_mod.parse(gpa, "resource S { name: string }"); + defer pr.deinit(gpa); + try std.testing.expect(pr.diagnostics.len == 0); + + var diags: std.ArrayListUnmanaged(Diagnostic) = .empty; + defer { + for (diags.items) |*d| d.deinit(gpa); + diags.deinit(gpa); + } + try types_mod.TypeChecker.check(gpa, &pr.ast, &diags); + try std.testing.expectEqual(@as(usize, 0), diags.items.len); + + var interp = try Interpreter.compile(gpa, &pr.ast, &world); + defer interp.deinit(); + + try expectResourceStringField(&world, "S", "name", ""); +} + +test "resource string field is mutable and the previous value is released (M1.0.3 E2)" { + // An allocation-tracking allocator (backed by the leak-detecting testing + // allocator) confirms two things: (a) every overwrite releases the previous + // non-immortal value — proven by a `free_count` increase across writes 2→3; + // (b) no leak across multiple writes — proven by the backing allocator at + // test end (an un-decref'd previous value would leak and fail the test). + var counting = weld_core.testing.alloc_counting.CountingAllocator.init(std.testing.allocator); + const gpa = counting.allocator(); + var world = World.init(); + defer world.deinit(gpa); + + // Each tick writes a fixed non-default value. Tick 1 overwrites the immortal + // default (decref no-op); ticks 2..N overwrite the previous refcounted block + // (decref → free). Re-reading returns the written value. + const source = + \\resource S { name: string = "intro" } + \\rule advance() + \\ when resource S + \\{ + \\ get_mut(S).name = "boss_arena" + \\} + ; + var pr = try parser_mod.parse(gpa, source); + defer pr.deinit(gpa); + try std.testing.expect(pr.diagnostics.len == 0); + + var diags: std.ArrayListUnmanaged(Diagnostic) = .empty; + defer { + for (diags.items) |*d| d.deinit(gpa); + diags.deinit(gpa); + } + try types_mod.TypeChecker.check(gpa, &pr.ast, &diags); + try std.testing.expectEqual(@as(usize, 0), diags.items.len); + + var interp = try Interpreter.compile(gpa, &pr.ast, &world); + defer interp.deinit(); + + // Tick 1: overwrite immortal default (no string free yet). + _ = try interp.runFor(&world, 1); + try expectResourceStringField(&world, "S", "name", "boss_arena"); + const after_first = counting.snapshot(); + + // Tick 2: overwrite the refcounted value from tick 1 → its block is freed. + _ = try interp.runFor(&world, 1); + const after_second = counting.snapshot(); + try std.testing.expect(after_second.free_count > after_first.free_count); + try expectResourceStringField(&world, "S", "name", "boss_arena"); +} + +// ── M1.0.3 E3 — resource enum fields ────────────────────────────────────── + +/// Read a resource enum field's raw `u32` discriminant and assert its value. +fn expectResourceEnumDiscriminant( + world: *World, + res_name: []const u8, + field_name: []const u8, + expected: u32, +) !void { + const rid = world.registry.idOf(res_name).?; + const fd = world.registry.findField(rid, field_name).?; + const bytes = world.resources.getResource(rid).?; + var disc: u32 = 0; + @memcpy(std.mem.asBytes(&disc), bytes[fd.offset .. fd.offset + @sizeOf(u32)]); + try std.testing.expectEqual(expected, disc); +} + +test "resource enum field compiles, reads its default, and is mutable (M1.0.3 E3)" { + const gpa = std.testing.allocator; + var world = World.init(); + defer world.deinit(gpa); + + // `.normal` is declaration-order index 1; `.hard` is 2. + const source = + \\enum Difficulty { easy, normal, hard } + \\resource S { diff: Difficulty = .normal } + \\rule advance() + \\ when resource S + \\{ + \\ let cur = get(S).diff + \\ get_mut(S).diff = .hard + \\} + ; + var pr = try parser_mod.parse(gpa, source); + defer pr.deinit(gpa); + try std.testing.expect(pr.diagnostics.len == 0); + + var diags: std.ArrayListUnmanaged(Diagnostic) = .empty; + defer { + for (diags.items) |*d| d.deinit(gpa); + diags.deinit(gpa); + } + try types_mod.TypeChecker.check(gpa, &pr.ast, &diags); + try std.testing.expectEqual(@as(usize, 0), diags.items.len); + + var interp = try Interpreter.compile(gpa, &pr.ast, &world); + defer interp.deinit(); + + // Default materialized as `.normal` (index 1) before any rule runs. + try expectResourceEnumDiscriminant(&world, "S", "diff", 1); + + const report = try interp.runFor(&world, 1); + try std.testing.expectEqual(@as(u64, 0), report.runtime_errors); + + // The in-body read (`get(S).diff`) ran, then `.hard` (index 2) was written. + try expectResourceEnumDiscriminant(&world, "S", "diff", 2); +} + +test "resource enum field with no default reads the first variant (M1.0.3 E3)" { + const gpa = std.testing.allocator; + var world = World.init(); + defer world.deinit(gpa); + + // No default ⇒ discriminant 0 ⇒ the first declared variant (`easy`). + const source = + \\enum Difficulty { easy, normal, hard } + \\resource S { diff: Difficulty } + ; + var pr = try parser_mod.parse(gpa, source); + defer pr.deinit(gpa); + try std.testing.expect(pr.diagnostics.len == 0); + + var diags: std.ArrayListUnmanaged(Diagnostic) = .empty; + defer { + for (diags.items) |*d| d.deinit(gpa); + diags.deinit(gpa); + } + try types_mod.TypeChecker.check(gpa, &pr.ast, &diags); + try std.testing.expectEqual(@as(usize, 0), diags.items.len); + + var interp = try Interpreter.compile(gpa, &pr.ast, &world); + defer interp.deinit(); + + try expectResourceEnumDiscriminant(&world, "S", "diff", 0); +} + +test "GameState end-state program mutates string + enum + int end-to-end (M1.0.3)" { + // The brief's flagship resource: `string` + enum + `int` fields mutated in + // one rule body, leak-free. Runs under the leak-detecting allocator. + const gpa = std.testing.allocator; + var world = World.init(); + defer world.deinit(gpa); + + const source = + \\enum Difficulty { easy, normal, hard } + \\@state + \\resource GameState { + \\ current_level: string = "intro" + \\ difficulty: Difficulty = .normal + \\ player_count: int = 0 + \\} + \\rule advance(dt: float) + \\ when resource GameState + \\{ + \\ let mut gs = get_mut(GameState) + \\ gs.current_level = "boss_arena" + \\ gs.difficulty = .hard + \\ gs.player_count += 1 + \\} + ; + var pr = try parser_mod.parse(gpa, source); + defer pr.deinit(gpa); + try std.testing.expect(pr.diagnostics.len == 0); + + var diags: std.ArrayListUnmanaged(Diagnostic) = .empty; + defer { + for (diags.items) |*d| d.deinit(gpa); + diags.deinit(gpa); + } + try types_mod.TypeChecker.check(gpa, &pr.ast, &diags); + try std.testing.expectEqual(@as(usize, 0), diags.items.len); + + var interp = try Interpreter.compile(gpa, &pr.ast, &world); + defer interp.deinit(); + + const report = try interp.runFor(&world, 1); + try std.testing.expectEqual(@as(u64, 0), report.runtime_errors); + + // current_level == "boss_arena", difficulty == .hard (2), player_count == 1. + try expectResourceStringField(&world, "GameState", "current_level", "boss_arena"); + try expectResourceEnumDiscriminant(&world, "GameState", "difficulty", 2); + const rid = world.registry.idOf("GameState").?; + const bytes = world.resources.getResource(rid).?; + const pc = world.registry.findField(rid, "player_count").?; + var n: i64 = 0; + @memcpy(std.mem.asBytes(&n), bytes[pc.offset .. pc.offset + @sizeOf(i64)]); + try std.testing.expectEqual(@as(i64, 1), n); +} + test "runProgram type-alias field resolves to the underlying primitive (M0.8 type alias)" { const gpa = std.testing.allocator; var world = World.init(); diff --git a/src/etch/persistent.zig b/src/etch/persistent.zig new file mode 100644 index 0000000..2500c0a --- /dev/null +++ b/src/etch/persistent.zig @@ -0,0 +1,238 @@ +//! Phase-1 persistent heap for the Etch runtime (`etch-memory-model.md` §4 / +//! §11). The keystone for non-POD resource fields: a refcounted, system- +//! allocator-backed heap whose blocks outlive a rule body. M1.0.3 uses it for +//! resource `string` fields; M1.0.4 reuses it unchanged for dynamic +//! collections (the `type_id` → drop dispatch and the open `TypeId` set are +//! exactly what `string[]` / `[K: V]` / `Set` will register against). +//! +//! Layout (`etch-memory-model.md` §4.3 / §5.1). Each block is one system +//! allocation laid out as: +//! +//! ``` +//! offset 0 8 12 16 (= exposed payload pointer) +//! ┌──────────────┬──────────┬──────────┬───────────────────────────┐ +//! │ size: usize │ refcount │ type_id │ payload (variable) │ +//! │ (for free) │ atomic u32│ u32 │ string bytes, … │ +//! └──────────────┴──────────┴──────────┴───────────────────────────┘ +//! ``` +//! +//! The exposed pointer `p` is `block + 16`. The 8-byte `{ refcount, type_id }` +//! header sits **immediately before** `p` (`p-8` / `p-4`), matching the spec's +//! "invisible 8-byte header preceding the exposed payload pointer". The leading +//! `size` word (at `p-16`) is implementation bookkeeping — Zig's `Allocator` +//! requires the length at `free` time, which a literal 8-byte header cannot +//! carry — and is invisible to every consumer; nobody reads it but `free`. +//! +//! Refcount mechanics (`etch-memory-model.md` §4.4 / §8.3). `alloc` yields +//! refcount 1; `incref` is `fetchAdd(1, .monotonic)`; `decref` is +//! `fetchSub(1, .release)` and, on the last release, an acquire load (the +//! `@fence`-free idiom — `@fence` was removed in Zig 0.16, cf. +//! `src/core/jobs/deque.zig`) followed by the type's drop + the block free. +//! A block allocated immortal carries `refcount == sentinel` (`u32.max`): +//! `incref` / `decref` are no-ops on it — compile-time string literals (resource +//! field defaults) use this path so `addResource` allocates nothing. +//! +//! Tier-0-agnostic: imports only `std` (no `src/core` dependency). The Etch +//! runtime owns this heap; the Tier-0 `ResourceStore` stays string-agnostic. + +const std = @import("std"); + +/// Coarse type tag stored in each block's header, used to dispatch the +/// drop that releases a type's owned sub-resources before the block is +/// freed. Open set: M1.0.4 dynamic collections add their own ids. +pub const TypeId = u32; + +/// A block whose payload owns no sub-resources (the bytes/POD live inline +/// and are reclaimed by the block free). Drop is a no-op. +pub const type_plain: TypeId = 0; + +/// A flat UTF-8 string: the bytes live inside the block (`p[0..len]`) and +/// are reclaimed by the block free, so its drop is a no-op. The distinct +/// id documents intent and lets `typeId` round-trip for debug/inspection. +pub const type_string: TypeId = 1; + +/// Refcount value marking an immortal block. `incref` / `decref` are +/// no-ops on it; only `destroy` reclaims it (heap-owner teardown). Used +/// for compile-time interned string literals (`etch-memory-model.md` §4.4). +pub const sentinel: u32 = std.math.maxInt(u32); + +/// On-storage layout of a resource `string` field slot (`etch-memory-model.md` +/// §5.1): `{ ptr, len }`, 16 bytes, 8-aligned. `ptr` is the address of the +/// persistent payload bytes (a `type_string` block's payload), or `0` for the +/// empty string (no backing block). The single source of truth for the slot +/// layout; `Registry.FieldKind.string_` reports `sizeBytes == 16` / `alignBytes +/// == 8` to match (cross-checked by a `comptime` assert in `ecs_bridge.zig`). +pub const StringSlot = extern struct { + ptr: u64 = 0, + len: u32 = 0, +}; + +comptime { + std.debug.assert(@sizeOf(StringSlot) == 16); + std.debug.assert(@alignOf(StringSlot) == 8); +} + +/// Allocation alignment of every block. ≥ `@alignOf(Header)` (8) and a +/// multiple of it, so the payload at `block + @sizeOf(Header)` is itself +/// 16-aligned — enough for any POD payload a future `TypeId` may carry. +const block_align: usize = 16; + +/// Invisible per-block prefix. Field order is load-bearing: `refcount` and +/// `type_id` must be the 8 bytes immediately preceding the payload pointer +/// (`etch-memory-model.md` §4.3); `size` precedes them so `free` can +/// reconstruct the allocation length. +const Header = extern struct { + size: usize, + refcount: std.atomic.Value(u32), + type_id: TypeId, +}; + +comptime { + // Pin the layout invariant: a refactor that reorders the fields (and so + // moves `{ refcount, type_id }` away from being the 8 bytes immediately + // before the payload) breaks the spec contract — catch it at compile time. + std.debug.assert(@sizeOf(Header) == 16); + std.debug.assert(@offsetOf(Header, "refcount") == 8); + std.debug.assert(@offsetOf(Header, "type_id") == 12); + std.debug.assert(block_align >= @alignOf(Header)); + std.debug.assert(block_align % @alignOf(Header) == 0); +} + +fn headerOf(p: [*]u8) *Header { + const base: [*]u8 = @ptrFromInt(@intFromPtr(p) - @sizeOf(Header)); + return @ptrCast(@alignCast(base)); +} + +fn blockSlice(p: [*]u8, size: usize) []align(block_align) u8 { + const base: [*]align(block_align) u8 = @alignCast(@as([*]u8, @ptrFromInt(@intFromPtr(p) - @sizeOf(Header)))); + return base[0 .. @sizeOf(Header) + size]; +} + +/// Allocate a `size`-byte payload owned by `type_id`, refcount 1. Returns +/// the exposed payload pointer (`block + 16`); `p[0..size]` is the writable +/// payload. The `{ refcount, type_id }` header sits at `p-8`. +pub fn alloc(gpa: std.mem.Allocator, type_id: TypeId, size: usize) std.mem.Allocator.Error![*]u8 { + return allocWithRefcount(gpa, type_id, size, 1); +} + +/// Allocate an immortal `size`-byte payload (refcount = `sentinel`). Used +/// for compile-time string literals (resource field defaults): `incref` / +/// `decref` never touch it, and only `destroy` reclaims it at teardown. +pub fn allocImmortal(gpa: std.mem.Allocator, type_id: TypeId, size: usize) std.mem.Allocator.Error![*]u8 { + return allocWithRefcount(gpa, type_id, size, sentinel); +} + +fn allocWithRefcount(gpa: std.mem.Allocator, type_id: TypeId, size: usize, initial: u32) std.mem.Allocator.Error![*]u8 { + const block = try gpa.alignedAlloc(u8, comptime .fromByteUnits(block_align), @sizeOf(Header) + size); + const h: *Header = @ptrCast(block.ptr); + h.* = .{ .size = size, .refcount = .init(initial), .type_id = type_id }; + return block.ptr + @sizeOf(Header); +} + +/// Increment the refcount on a copied handle. No-op on an immortal block. +pub fn incref(p: [*]u8) void { + const h = headerOf(p); + if (h.refcount.load(.monotonic) == sentinel) return; + _ = h.refcount.fetchAdd(1, .monotonic); +} + +/// Drop a handle. On the last release the block's drop runs and the block +/// is freed. No-op on an immortal block (reclaim those via `destroy`). +pub fn decref(gpa: std.mem.Allocator, p: [*]u8) void { + const h = headerOf(p); + if (h.refcount.load(.monotonic) == sentinel) return; + if (h.refcount.fetchSub(1, .release) == 1) { + // Acquire load stands in for the dropped `@fence(.acquire)` (removed + // in Zig 0.16): it synchronizes-with the prior `.release` decrements + // so the drop observes every writer's stores. Cf. `deque.zig`. + _ = h.refcount.load(.acquire); + runDrop(gpa, h.type_id, p, h.size); + freeBlock(gpa, p); + } +} + +/// Unconditionally release a block regardless of refcount (runs its drop, +/// then frees). For the heap owner's teardown of immortal / interned blocks, +/// which `decref` leaves alive by design. +pub fn destroy(gpa: std.mem.Allocator, p: [*]u8) void { + const h = headerOf(p); + runDrop(gpa, h.type_id, p, h.size); + freeBlock(gpa, p); +} + +/// Current refcount (`sentinel` for immortal blocks). Debug / test helper. +pub fn refcount(p: [*]u8) u32 { + return headerOf(p).refcount.load(.monotonic); +} + +/// The block's owning `TypeId`. +pub fn typeId(p: [*]u8) TypeId { + return headerOf(p).type_id; +} + +/// The payload size in bytes recorded at `alloc` time. +pub fn payloadSize(p: [*]u8) usize { + return headerOf(p).size; +} + +/// Release a type's owned sub-resources before its block is freed +/// (`etch-memory-model.md` §4.3). Phase-1 ids (`type_plain` / `type_string`) +/// own nothing beyond their inline payload, so their drop is a no-op — the +/// block free reclaims the bytes. M1.0.4 collection ids free their element / +/// table buffers in the `else` arm before the block goes. +fn runDrop(gpa: std.mem.Allocator, type_id: TypeId, p: [*]u8, size: usize) void { + switch (type_id) { + type_plain, type_string => {}, + else => {}, + } + _ = gpa; + _ = p; + _ = size; +} + +fn freeBlock(gpa: std.mem.Allocator, p: [*]u8) void { + gpa.free(blockSlice(p, headerOf(p).size)); +} + +// ─── tests ──────────────────────────────────────────────────────────────── + +test "alloc sets refcount 1 and decref to zero frees + drops" { + const gpa = std.testing.allocator; + const p = try alloc(gpa, type_string, 5); + @memcpy(p[0..5], "intro"); + try std.testing.expectEqual(@as(u32, 1), refcount(p)); + try std.testing.expectEqual(type_string, typeId(p)); + try std.testing.expectEqual(@as(usize, 5), payloadSize(p)); + // refcount 1 → 0: runs the drop then frees the block (header + bytes). + // `std.testing.allocator` fails the test on any leak — so a clean exit + // proves the byte payload was reclaimed. + decref(gpa, p); +} + +test "incref then decref keeps the block alive until the last release" { + const gpa = std.testing.allocator; + const p = try alloc(gpa, type_plain, 8); + // 3 increfs → refcount 4 → needs 4 decrefs (N increfs require N+1). + incref(p); + incref(p); + incref(p); + try std.testing.expectEqual(@as(u32, 4), refcount(p)); + decref(gpa, p); + decref(gpa, p); + decref(gpa, p); + try std.testing.expectEqual(@as(u32, 1), refcount(p)); // still alive + decref(gpa, p); // last release → free +} + +test "immortal-interned sentinel: incref/decref are no-ops" { + const gpa = std.testing.allocator; + const p = try allocImmortal(gpa, type_string, 5); + @memcpy(p[0..5], "intro"); + try std.testing.expectEqual(sentinel, refcount(p)); + incref(p); + try std.testing.expectEqual(sentinel, refcount(p)); // unchanged + decref(gpa, p); // no-op: not freed, never double-frees + try std.testing.expectEqual(sentinel, refcount(p)); // still alive + // Immortal blocks are reclaimed only by the heap owner at teardown. + destroy(gpa, p); +} diff --git a/src/etch/root.zig b/src/etch/root.zig index 5ac3068..5e616ad 100644 --- a/src/etch/root.zig +++ b/src/etch/root.zig @@ -46,6 +46,7 @@ comptime { _ = @import("interp.zig"); _ = @import("value.zig"); _ = @import("ecs_bridge.zig"); + _ = @import("persistent.zig"); } /// S5 Zig codegen surface — exposed at the module surface so diff --git a/src/etch/types.zig b/src/etch/types.zig index 167f921..e912cf5 100644 --- a/src/etch/types.zig +++ b/src/etch/types.zig @@ -2945,13 +2945,14 @@ pub const TypeChecker = struct { /// Which declaration kind a validated field range belongs to. `component_like` /// (components) keeps the POD-strict surface (`string`/enum/nested-struct - /// rejected — part1 §5.5, SoA archetype storage); `resource` keeps its own - /// S3 rejection wording (Option A alignment is tranche 7). `struct_` and - /// `event_` share the wider surface: `string` + enum-typed fields accepted - /// (the builtin `Error` forces them on structs; events carry frame-arena - /// non-POD payloads per `etch-memory-model.md` §6.7 / §2.5), nested-struct - /// fields deferred. POD-strict is component-only (M1.0.2 ruling, decision on - /// the E1 test-4 blocker). + /// rejected — part1 §5.5, SoA archetype storage). `resource` shares the wider + /// non-POD surface with `struct_` / `event_` for `string` (M1.0.3 E2 — the + /// Option A alignment, formerly deferred "tranche 7"; part1 §5.5 "no POD + /// constraint for resources"); enum-typed resource fields land in M1.0.3 E3, + /// nested-struct fields stay deferred. `struct_` and `event_` accept `string` + /// + enum-typed fields (the builtin `Error` forces them on structs; events + /// carry frame-arena non-POD payloads per `etch-memory-model.md` §6.7 / §2.5). + /// POD-strict is component-only. const FieldDeclOrigin = enum { component_like, resource, struct_, event_ }; fn validateFieldsInDecl(self: *TypeChecker, fields_start: u32, fields_len: u32, origin: FieldDeclOrigin) !void { @@ -3004,19 +3005,23 @@ pub const TypeChecker = struct { const tname = self.arena.strings.slice(resolved_name); if (BuiltinType.fromName(tname) == null) { - if ((origin == .struct_ or origin == .event_) and std.mem.eql(u8, tname, "string")) { + if ((origin == .struct_ or origin == .event_ or origin == .resource) and std.mem.eql(u8, tname, "string")) { // `string` fields unlock for structs (the Error layer, - // M0.8 E3-C tranche 2 — `Error.message` forces them) and for - // events (frame-arena non-POD payload, `etch-memory-model.md` - // §6.7 / §2.5 — M1.0.2 ruling). part1 §5.5 constrains - // components only; component / resource string fields stay - // rejected below. - } else if ((origin == .struct_ or origin == .event_) and self.declaredEnumName(resolved_name)) { + // M0.8 E3-C tranche 2 — `Error.message` forces them), events + // (frame-arena non-POD payload, `etch-memory-model.md` §6.7 / + // §2.5 — M1.0.2 ruling), and now resources (M1.0.3 E2 — the + // Option A alignment; part1 §5.5 "no POD constraint for + // resources"; the persistent-heap slot is `{ptr,len}`). part1 + // §5.5 constrains components only — component string fields + // stay rejected below. + } else if ((origin == .struct_ or origin == .event_ or origin == .resource) and self.declaredEnumName(resolved_name)) { // Enum-typed fields unlock for structs (`Error.code: - // ErrorCode`, same tranche) and events (M1.0.2). Checked + // ErrorCode`, same tranche), events (M1.0.2), and now resources + // (M1.0.3 E3 — mirrors the `string` unlock; the slot stores the + // variant's declaration-order discriminant, POD). Checked // against the AST enum slab (not the symbol table) so a // later-declared enum is seen — pass 1 registers symbols - // incrementally. + // incrementally. Components stay enum-rejected (POD-strict). } else if ((origin == .struct_ or origin == .event_) and self.declaredStructName(resolved_name)) { // Struct-typed STRUCT / event fields are deferred: the // anonymous `.{ … }` field-value context (M0.8 E3-C tranche 8) @@ -3034,9 +3039,10 @@ pub const TypeChecker = struct { // unsupported. The brief enforces builtin POD only. try self.emit(.undefined_symbol, .error_, tspan, "type '{s}' is not in the S3 POD builtin set", .{tname}); } else if (std.mem.eql(u8, tname, "string")) { - // `string` rejected on components per brief §POD; for - // resources `string` is also out of the S3 builtin set - // (resources POD-enforced via the same builtin table). + // Reached only by `component_like` now: struct / event / + // resource `string` fields are accepted above (M1.0.3 E2 + // unlocked resources). Components stay POD-strict (part1 + // §5.5, SoA archetype storage). The `else` is defensive. if (origin == .component_like) { try self.emit(.undefined_symbol, .error_, tspan, "type 'string' is rejected on components in S3 (POD enforcement)", .{}); } else { @@ -7257,7 +7263,7 @@ test "type-checker validates tag mutations (M0.8 E3)" { try expectAnyCode(bad_recv.diagnostics.items, .tag_invalid_operation); } -test "type-checker accepts string and enum fields on struct, keeps component/resource rejection (M0.8 E3-C tranche 2)" { +test "type-checker accepts string + enum fields on struct + resource, keeps component rejection (M1.0.3)" { const gpa = std.testing.allocator; // `string` + enum-typed struct fields are the tranche-2 unlock (the @@ -7272,19 +7278,38 @@ test "type-checker accepts string and enum fields on struct, keeps component/res defer ok.deinit(gpa); try expectNoCode(ok.diagnostics.items, .undefined_symbol); - // Components stay POD: `string` rejected. + // Components stay POD: `string` rejected (part1 §5.5, SoA archetype storage). var comp = try parseAndCheck(gpa, \\component Name { value: string } ); defer comp.deinit(gpa); try expectAnyCode(comp.diagnostics.items, .undefined_symbol); - // Resources keep the S3 rejection until the Option A alignment (tranche 7). + // Components stay POD: enum rejected too. + var comp_enum = try parseAndCheck(gpa, + \\enum Difficulty { easy, normal, hard } + \\component Mode { diff: Difficulty } + ); + defer comp_enum.deinit(gpa); + try expectAnyCode(comp_enum.diagnostics.items, .undefined_symbol); + + // Resources accept `string` since M1.0.3 E2 (the Option A alignment, + // formerly the deferred "tranche 7"; part1 §5.5 "no POD constraint for + // resources"). FLIPPED from rejected → accepted — the intended resolution, + // not a regression. var res = try parseAndCheck(gpa, \\resource Settings { player_name: string } ); defer res.deinit(gpa); - try expectAnyCode(res.diagnostics.items, .undefined_symbol); + try expectNoCode(res.diagnostics.items, .undefined_symbol); + + // Resources accept enum fields since M1.0.3 E3 (mirrors the `string` unlock). + var res_enum = try parseAndCheck(gpa, + \\enum Difficulty { easy, normal, hard } + \\resource GameMode { diff: Difficulty = .normal } + ); + defer res_enum.deinit(gpa); + try expectNoCode(res_enum.diagnostics.items, .undefined_symbol); } test "type-checker resolves the builtin Error struct end-to-end (M0.8 E3-C tranche 2)" { diff --git a/src/etch/value.zig b/src/etch/value.zig index 45a5a16..2ef1457 100644 --- a/src/etch/value.zig +++ b/src/etch/value.zig @@ -100,6 +100,14 @@ pub const Value = union(enum) { /// name (interned `StringId`) and the variant's declaration-order index. /// Value-typed: compared by `(type_name, variant)` equality. enum_value: EnumValue, + /// A borrowed view over a resource `string` field's persistent-heap bytes + /// (M1.0.3 E2). The read path returns this without incref'ing the block — + /// safe for the rule body because the resource (hence the bytes) outlives it + /// (`etch-memory-model.md` §11 Phase 1; scope-bound incref is Phase 2). Self- + /// contained `{ptr,len}` so `readBytesAsValue` can build it with no allocator + /// and no interpreter store; `ptr == 0` ⇔ the empty string. Additive — does + /// not disturb `string_id` (AST pool) / `string_run` (rule-arena) semantics. + string_persistent: StrView, unit, pub fn fromInt(x: i64) Value { @@ -142,6 +150,14 @@ pub const Value = union(enum) { .struct_ref => false, .optional => false, // optional equality is not exercised in M0.8 (unwrap via if/while let) .enum_value => |a| a.type_name == other.enum_value.type_name and a.variant == other.enum_value.variant, + .string_persistent => |a| blk: { + const b = other.string_persistent; + if (a.len != b.len) break :blk false; + if (a.len == 0) break :blk true; + const ab: [*]const u8 = @ptrFromInt(a.ptr); + const bb: [*]const u8 = @ptrFromInt(b.ptr); + break :blk std.mem.eql(u8, ab[0..a.len], bb[0..b.len]); + }, .unit => true, }; } @@ -153,6 +169,13 @@ pub const EnumValue = struct { variant: u32, }; +/// Borrowed view over persistent-heap string bytes (M1.0.3 E2). `ptr` is the +/// raw address of the bytes (`0` for the empty string); `len` the byte count. +pub const StrView = struct { + ptr: u64 = 0, + len: u32 = 0, +}; + /// Typed sum carrying a `SourceSpan` resolved from the AST `NodeId` that /// triggered the failure. The interpreter never silently masks runtime /// errors — it reports them through this type plus the `RuntimeReport` diff --git a/tests/etch_interp/diff_runner.zig b/tests/etch_interp/diff_runner.zig index 4e6773a..102d376 100644 --- a/tests/etch_interp/diff_runner.zig +++ b/tests/etch_interp/diff_runner.zig @@ -284,6 +284,10 @@ fn writeFieldValue(kind: FieldKind, bytes: []u8, v: FieldValue) void { const x: f32 = @floatCast(v.float_); @memcpy(bytes[0..@sizeOf(f32)], std.mem.asBytes(&x)); }, + // The S4 differential corpus is POD-only: `string`/enum resource fields + // (M1.0.3) are exercised by inline interpreter tests, never by this + // corpus, so these kinds never reach the runner — proven invariant. + .string_, .enum_ => unreachable, } } @@ -315,5 +319,8 @@ fn readFieldValue(kind: FieldKind, bytes: []const u8) FieldValue { @memcpy(std.mem.asBytes(&v), bytes[0..@sizeOf(f32)]); break :blk .{ .float_ = v }; }, + // POD-only corpus — `.string_`/`.enum_` (M1.0.3) never enter it (see + // `writeFieldValue`). + .string_, .enum_ => unreachable, }; }