Build against Mark Tarver's refreshed S41.2 kernel (2026-07-11)

[pyrex41]

Summary

Mark Tarver re-uploaded a restructured kernel under the same "41.2"
version number (canonical mirror: pyrex41/shen-s41.1 tag
s41.2-pristine-20260711, commit 11fc51b; secondary
https://www.shenlanguage.org/Download/S41.2.zip, Last-Modified 2026-07-11,
sha256 51becbfd60fa8c93c3f8ae5b20b948eaa84c4b1d14ad2f5d2a056002a53ee836).
This is a different lineage from the community shen-41.2 release shen-cl
has built from. This branch adapts the shen-cl harness to build against it.

SBCL, CLISP and ECL are all green (build + 134/134 kernel suite + launcher).
Full provenance and status:
docs/KERNEL-PROVENANCE-tarver-s41.2.md.

Standalone canary (Tarver's own install.lsp)

Independent of shen-cl, Tarver's install.lsp was run under macOS SBCL 2.6.2
(stdlib=yes, concurrency/Tk/THORN/LogicLab=no). It builds and boots:
produced sbcl-shen.exe, (+ 2 3)=5, (version)="41.2", stdlib typechecks
and loads. The kernel is proven sound across the fleet (shen-swift boots all 15
KLambda files, *version*="41.2", 12/12).

Usable as a ratatoskr stage-1 host? Not the stock image. Its saved toplevel
is the interactive shen.shen; piping (load "f")+expr over stdin evaluates
((sq 9)=81), but the stock image has no non-interactive launcher CLI, no
cl.exit, and loops forever on stdin EOF. This shen-cl S41.2 build is the
proper same-lineage host — verified below.

Upstream delta (refresh vs community 41.2)

• Removed: compiler.kl (shen-cl generates its own), dict.kl, init.kl,
  stlib.kl, all extension-*.kl (incl. the launcher ratatoskr needs).
• Renamed/moved: hush->shen.hush, input+->shen.input-h+, REPL entry
  shen.repl->shen.shen, shen.initialise-lambda-forms->
  shen.initialise-lambda-tables; shen.initialise gone (folded into
  declarations.kl load-time forms); property store -> bucketed absvector
  *property-vector* with vector put/get.

What this PR does

Assembles a hybrid kernel via scripts/assemble-tarver-kernel.sh
(make fetch-tarver): Tarver's 14 shared KLambda files + community
launcher/features/expand-dynamic/stlib grafts + community tests/lib. Harness
adaptations:

• src/compiler.shen: disable the (trap-error (get ..) ..)->shen-cl.get/or
  peephole (it assumed a hash-table property store).
• src/overwrite.lsp: remove the hash override (vector store buckets by the
  kernel hash); shim shen.repl->shen.shen, a no-op shen.initialise,
  shen.set-lambda-form-entry, and shen.toplevel-display-exception.
• boot.lsp / scripts/build.shen: drop dict/init; align load order to
  Tarver's install.lsp (t-star before types).
• tests/compiler-tests.shen: update the get/or assertion.

Test status (macOS, arm64)

Implementation	Build	Kernel suite runme.shen	Launcher / stlib
SBCL 2.6.2	pass	134/134	eval -e + eval -q -l file -e expr, REPL, clean EOF
GNU CLISP 2.49.92	pass	134/134	eval -l file -e expr + stlib
ECL 26.5.5	pass	134/134	eval -e, eval -l file -e expr + stlib
CCL	out of scope	—	no native Apple-Silicon build

Notes:

• CCL has no native Apple-Silicon build (documented in the ratatoskr
  README), so it is out of scope here rather than tested.
• clisp -q: clisp intercepts a bare -q in its own runtime arg parser
  (clisp has a native -q), so eval -q ... swallows the result there;
  eval -l file -e expr works. Pre-existing, unrelated to the refresh.
• Compiler golden tests: the 1+/EQUALP mismatches and let-NIL crash
  reproduce identically on the master (community-41.1) binary, so they are
  pre-existing and out of scope; the get/or case was updated.

Verified as ratatoskr's stage-1 host

Using this branch's SBCL binary as $RATATOSKR_HOST,
ratatoskr shake tests/fib.shen produces kernel.kl = 54 defuns / 13.4 KB
(the documented figure) + fib.kl + manifest
(kernel-version=41.2-s41r.20260711). This is the designated same-lineage
stage-1 host of record; ratatoskr can swap to it from Tarver's raw image.

Fleet context

Part of the fleet-wide S41.2 refresh migration; all sibling ports are green
(shen-lua Shen-Language#36, shen-rust #9, shen-swift #1, ratatoskr Shen-Language#10, etc.). shen-julia's
demod finding does not affect shen-cl (SBCL is late-bound). This host is the
fleet's reference and fastest stage-1 host.

Known remaining work

• Port stlib and the launcher to Tarver-native mechanisms to retire the
  community grafts — the same interim compromise every fleet port took; the
  StLib-from-source pass is a coordinated fleet-wide follow-up.
• Decide whether to vendor the assembled kernel (Tarver's upstream is
  re-uploaded in place, unversioned).

Discovery other ports will hit

Any port with native put/get assuming a dict breaks: properties now
live on a plain absvector *property-vector*, and there is no
shen.initialise — initialisation is load-time top-level forms in
declarations.kl, so read install.lsp (not make.shen) for .kl load
order: declarations and t-star must precede types. Also subtle: any host
that overrides hash corrupts the store (buckets set at kernel-populate
time must match get time; the kernel hash never returns 0 -- index 0 holds
the vector length).

🤖 Generated with Claude Code

[pyrex41]

Load-order note for other ports booting the compiled .kl directly: read install.lsp, not make.shen. make.shen is the .shen compile order; it is not a valid .kl load order. The refreshed types.kl runs ~161 top-level (declare ..) forms at load that invoke the typechecker, so both declarations and t-star must precede types (shen.rectify-type moved into t-star.kl). This branch's boot.lsp now follows install.lsp's runtime order (declarations 5th, types last), single-pass with no two-phase boot — independently confirmed green by shen-rust (pyrex41/shen-rust#9) and shen-lua (pyrex41/shen-lua#36). Detail in docs/KERNEL-PROVENANCE-tarver-s41.2.md.