perf(GC): make per-object layout O(1)-loadable — kill per-operation thread-local layout tracking (umbrella: method_calls/array-downgrade/object-property)

[TheHypnoo]

Summary

Several of Perry's worst benchmark gaps share one systemic root cause, not separate ones: Perry tracks per-object / per-slot GC layout (which slots hold raw-f64 numbers vs NaN-boxed pointers) in thread-local hashmaps (TYPED_LAYOUTS, LAYOUT_SLOT_MASKS) and queries or updates them on every field/element access. On macOS each access is a _tlv_get_addr (TLS accessor) + a hashmap hash+lookup. The all-numeric "unboxed" fast paths bypass it, but any heterogeneous shape, downgraded array, or raw-f64 class field falls into the per-op hashmap path, which dominates.

Benchmark	Gap vs Node	Hot path	Per-op cost
method_calls (#5093)	~290× (3300 ms vs 11 ms)	this.field get/set in class methods	js_typed_feedback_class_field_{get,set}_guard → class_field_fast_contract → layout_typed_raw_f64_slot_for_user (TLS TYPED_LAYOUTS)
bench_numeric_array_downgrade	~781×; ~21× over the same-shape numeric array	arr[i]=… on heterogeneous/any[] arrays	js_array_set_f64_extend → note_array_slot → layout_note_slot (TLS), per write
bench_object_property	~17× after #5084	dynamic property writes	same field-access guard family

Evidence (measured)

• Profile (sample): the downgrade hot loop is a storm of _tlv_get_addr (TLS) calls.
• method_calls: removing the per-access register_site call (perf(codegen): make typed-feedback site registration opt-in (3.6x on dynamic property access) #5084) → ~2%; an MRU cache over layout_typed_raw_f64_slot_for_user made it 2× SLOWER (3300 → 7050 ms). Inlining the guard is blocked because the raw-f64 check is a TLS hashmap lookup, not an O(1) header field.
• numeric_array_downgrade: an obj_type pre-filter that skips the per-element typedarray/buffer/set/map registry lookups gave only ~5%; a "skip the in-bounds RuntimeHandleScope rooting" fast path made it worse. So the cost is note_array_slot → layout_note_slot (the TLS layout write), per element.

There is no cheap shortcut. The fix is to make per-object layout O(1)-loadable from the object/GC header so the layout check/update is an inline bit-test/bit-set instead of a thread-local hashmap op.

Proposed fix — "layout is canonical" header bit + static class mask

GcHeader is 8 bytes with a free _reserved: u16 and spare gc_flags bits. Add a bit meaning "this object's slot layout still matches its declared/canonical shape" (no downgrade yet):

• Class instances: when set, the authoritative raw-f64 mask is the compile-time perry_typed_shape_raw_f64_mask_<class> global codegen already emits — a constant codegen can bit-test inline (no TLS). With the method already inlined (perf(transform): inline small this-using methods on exact receivers (1.6x on method_calls) #5092), LLVM LICM hoists the loop-invariant check out of the hot loop.
• Arrays: when set, skip the per-write layout_note_slot for scalar-over-scalar in-bounds stores.
• On downgrade (a pointer/string written into a canonical slot): clear the bit and fall back to today's per-object TYPED_LAYOUTS/LAYOUT_SLOT_MASKS path (unchanged). The GC scanner consults the bit.

The TLS hashmap exists today only to track downgrades; the common (no-downgrade) case does not need it.

Optional complement: store a compact inline mask in _reserved for small objects/arrays (≤16 slots) so even small downgraded shapes avoid the hashmap.

Correctness invariants (the crux — this is GC-internals, memory-corruption risk)

1. GC scanner sees the truth. Per-slot pointer/raw-f64 determination after the change must equal today's (gc/trace.rs:758, gc/copying.rs:309,579 consult per-slot layout_kind). A wrong mask = trace a number as a pointer (crash) or miss a pointer (use-after-free).
2. Representation. A slot read as raw double must hold raw f64; the canonical bit must be cleared before the first non-number is observable in a canonical slot (publish-order discipline, like descriptors_in_use).
3. Downgrade is monotonic + complete; the fallback path stays byte-for-byte current behavior.
4. GC moves transfer the bit + mask (gc/copying.rs:504 layout_transfer).

Phasing (each independently shippable + verifiable)

1. Spike + microbench harness; prototype the bit read-only and confirm it tracks downgrade under GC stress.
2. Arrays first (lowest blast radius): wire the bit for arrays; skip layout_note_slot + write barrier for scalar-over-scalar in-bounds writes; scanner honors the bit.
3. Class fields: emit the inline guard in codegen (expr/property_get.rs:1551, property_set.rs) using the header bit + static class mask, by-name fallback for the cleared-bit case.
4. Object property path if the mechanism generalizes.

Verification

• Full local parity (./run_parity_tests.sh) — zero NEW regressions vs base (compare combined stdout+stderr per file). cargo test --release --workspace.
• GC correctness: run under PERRY_GC_VERIFY_EVACUATION=1, PERRY_GC_FORCE_EVACUATE=1, PERRY_GC_DIAG=1, and PERRY_GEN_GC=0 (full mark-sweep) — these panic on the corruption modes a wrong mask causes.
• Targeted tests: (a) write a non-number into a number-typed class field via an any alias, then read it back; (b) a numeric array that receives an object slot then is GC-evacuated mid-loop; (c) holey/sparse downgraded arrays.
• Per-benchmark perf regression gate.

Risk: HIGH (memory-corruption class). Effort: L (GC-internals). Maintainer-driven; not for autonomous execution.

Prior groundwork on this line

• perf(codegen): make typed-feedback site registration opt-in (3.6x on dynamic property access) #5084 — typed-feedback register_site opt-in (object_property 3.6×).
• perf(transform): inline small this-using methods on exact receivers (1.6x on method_calls) #5092 — inline small this.field methods (method_calls 1.6×; prerequisite — the guard must be inlined into the loop for the phase-3 hoist to pay off).

Files

gc/types.rs (GcHeader), gc/layout.rs (TYPED_LAYOUTS/LAYOUT_SLOT_MASKS, TypedLayoutDescriptor, layout_note_slot, layout_typed_raw_f64_slot_for_user), gc/trace.rs + gc/copying.rs (scanner + layout_transfer), array/indexing.rs + array/header.rs (note_array_slot), typed_feedback/guards.rs (class_field_fast_contract), codegen expr/property_get.rs:1551, expr/property_set.rs, typed_shape.rs.

[TheHypnoo]

Update — precise localization of bench_numeric_array_downgrade (M1 Pro)

Investigated the array side directly. The cost is not inside the array setter functions, and it is not dominated by a single call — three surgical attempts moved the needle 0–5%:

• skip the per-element typedarray/buffer/set/map registry lookups via an obj_type pre-filter in js_array_get_f64/js_array_set_f64* → ~5% (helps the GET path)
• make note_array_slot scalar-aware (skip the layout-mask update + barrier when neither old nor new value is a pointer) → 0%
• reorder the write barrier so the non-pointer skip runs before incremental_mark_barrier_value's thread-local touch → 0%

Why 0%: for an in-bounds arr[i] = value, codegen does not call the array setter — it inlines the slot store and emits separate per-write runtime calls:

; idxset fast path (per element, ~6.25M times)
store double %v, ptr %slot
call void @js_gc_note_slot_layout(i64 %arr, i32 %i, i64 %vbits)   ; → layout_note_slot (TLS hashmap on SIDE_MASK arrays)
call void @js_write_barrier_slot(i64 %arr, i64 %slotaddr, i64 %vbits)
call void @js_array_note_numeric_write(i64 %arr, i64 %vbits)      ; no-op for numbers, but a call
; + js_shadow_slot rooting

js_array_set_f64_extend only appears on the realloc fallback path. So the setter-internal layout tracking I optimized is dead code for this loop.

Measured ~736 ns per arr[i] = arr[i] + 1 on M1 Pro (vs ~21 ns for a plain read) — split across the inline store + ~4 codegen-emitted non-inlined runtime calls (layout note, write barrier, numeric note, shadow rooting) + the js_array_get_f64 read. No single one dominates; it is death-by-many-per-element-calls.

Implication for the fix

This is the codegen emission side of the same systemic issue. The clean wins are at codegen (crates/perry-codegen/src/expr/write_barrier.rs):

1. For a store whose value is statically non-pointer (or the element type is numeric), don't emit js_write_barrier_slot / js_array_note_numeric_write — both are no-ops for numbers. (Limited for any[] where the value is statically any, but covers typed numeric arrays.)
2. The js_gc_note_slot_layout per-write call is the layout-mask update; making it skip cheaply for non-pointer stores needs the O(1) header-state work described in this issue (the SIDE_MASK clear is a thread-local hashmap touch).
3. Longer-term: inline the element get/set so these collapse and LLVM can hoist/eliminate the loop-invariant parts (parallels the class-field inline-guard plan for method_calls).

A properly symbolicated profile is needed to rank the ~4 calls precisely — the release binary is stripped (-Wl,--strip-debug via auto-optimize), so sample only shows _tlv_get_addr storms without function names.

[TheHypnoo]

Update — first concrete win landed: PR #5098 (bench_numeric_array_downgrade 4482 → 427ms, ~10.5× on M1 Pro).

This is the array side of the umbrella: the per-write js_gc_note_slot_layout (→ layout_note_slot) thread-local hashmap touch on every arr[i] = value. An A/B stub experiment confirmed it was the dominant cost (stubbing it: 11× faster; stubbing the write barrier: ~2%).

The fix is the targeted, low-risk version of the design here: js_gc_note_slot_layout_aware(parent, slot, new, old) skips the hashmap when neither old nor new is a heap pointer (slot pointer-ness unchanged → mask invariant preserved). Codegen loads the previous slot value before an in-place array element overwrite and routes only those through it. GC-stress verified (pointer↔number transitions under VERIFY_EVACUATION/FORCE_EVACUATE/full mark-sweep), full parity zero-new-regression.

Still open under this umbrella:

• method_calls / class fields (perf(method dispatch): method_calls ~290× Node — remaining cost is per-field-access shape-guard calls (plan + standby) #5093): the class_field_*_guard raw-f64 check is a TYPED_LAYOUTS thread-local lookup, not O(1) — needs the codegen inline-guard + the O(1) header raw-f64 mask described above.
• bench_object_property: the dynamic-property-write field-guard family.

The js_gc_note_slot_layout_aware predicate + the GcHeader _reserved layout-state machine (GC_LAYOUT_POINTER_FREE/SIDE_MASK, already present) are the building blocks for those.