stdlib: load from Lib/StLib sources, retire community stlib.kl#10
Merged
Conversation
Make Mark Tarver's S-lineage Lib/StLib Shen sources the standard-library
source of truth and remove the pre-compiled community stlib.kl overlay.
The community ShenOSKernel-41.2 shipped the stdlib as a pre-compiled
stlib.kl; installing those defuns bypassed the kernel's load/define path,
so functions like filter ended up with the "unknown" arity -1 and bare
(fn filter) / top-level (filter ...) failed ("fn: filter is undefined").
The refresh ships the stdlib as sources loaded via install.shen; going
through the real load path registers correct arities and package
namespacing, fixing that.
- Vendor kernel/stlib/ (byte-identical to mirror pyrex41/shen-upstream tag
s41.2-pristine-20260711, Lib/StLib/ — the 22 files install.shen loads).
- Delete kernel/klambda/stlib.kl, its AOT module aot/kernel/stlib.rs, the
stlib entries in aot/kernel/mod.rs, and the embedded stlib.kl in shenffi.
- boot::load_stlib runs install.shen after the kernel is up. It rewrites
the relative (load ...) paths to absolute into a per-boot temp file (no
process-cwd mutation — cwd is process-global and interpreters boot
concurrently) and loads that, hushed. install.shen is otherwise run
verbatim, tc toggles and all, matching the shen-cl reference. Its closing
(preclude-all-but []) keeps datatype recognisers from persisting.
- klcompile KERNEL_SLOW_DEFUNS is now empty (its only members were the four
stlib.initialise-* defuns in the removed stlib.kl).
- .cargo/config.toml sets RUST_MIN_STACK: the type-checked StLib load
recurses past cargo's 2 MB test-thread stack (the binary already boots on
a >=64 MB thread).
- parse_kernel form-count floor lowered (stlib's ~738 forms moved out).
- Regression test library.rs::stdlib_loaded_from_source_has_real_arities:
(arity filter) >= 0, (fn filter) resolves, filter filters, list.reduce
reachable namespaced.
All gates green: kernel-tests 134/134 across release/debug/debug-gc,
fmt+clippy, workspace tests, shen-check, tcb-audit, kernel-aot-audit,
cedar-equiv.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
stdlib: load from
Lib/StLibsources, retire the communitystlib.kloverlayFollow-up to the S41.2 refresh (#9). Makes Mark Tarver's S-lineage
Lib/StLibShen sources the standard-library source of truth andremoves the pre-compiled community
stlib.kloverlay the port had kept.Why
The community
ShenOSKernel-41.2shipped the stdlib as a pre-compiledstlib.kl. Installing thosedefuns bypassed the kernel'sload/definepath, so stdlib functions ended up with the "unknown"arity
-1: bare(fn filter)and top-level(filter …)failed withfn: filter is undefined. Tarver's refresh ships the stdlib as separatesources loaded via
install.shen; loading through the realloadpathregisters correct arities and package namespacing, fixing that.
What this PR does
kernel/stlib/(byte-identical to mirror
pyrex41/shen-upstreamtags41.2-pristine-20260711,Lib/StLib/; the 22 filesinstall.shenloads). See
kernel/stlib/PROVENANCE.md.kernel/klambda/stlib.kl, its AOT moduleaot/kernel/stlib.rs, thestlibentries inaot/kernel/mod.rs, andthe embedded
stlib.klinshenffi.boot::load_stlib, after the kernel is fullyup) by running upstream's
install.shen. Becauseinstall.shenusesrelative
(load …)and shen-rust resolves file paths against theprocess cwd, and because multiple interpreters can boot concurrently
(cwd is process-global), it rewrites the load paths to absolute into a
per-boot temp file and loads that — no cwd mutation, fully re-entrant.
Runs under
*hush*so the per-file chatter stays off the boot path.KERNEL_SLOW_DEFUNSis now empty (crates/klcompile): its onlymembers were the four
stlib.initialise-*defuns instlib.kl..cargo/config.tomlsetsRUST_MIN_STACK=256MB: the type-checkedStLib load recurses past cargo's 2 MB test-thread stack (the binary
already boots on a ≥64 MB thread). Makes
cargo test/cargo runwork without a hand-set env var.
library.rs::stdlib_loaded_from_source_has_real_arities:(arity filter) >= 0,(fn filter)resolves,(filter …)filters, and thepackage-internal
list.reduceis reachable namespaced.Behavior notes (not regressions)
Package-internal (non-exported) stdlib functions are namespaced:
reducelives in(package list …), reachable aslist.reduce, notbare
reduce(the communitystlib.klhad noreduceat all). Thetest's
maphelper was renamedsq→my-squarebecausesqis now areal StLib function and the kernel refuses to redefine a system fn.
Because the load re-runs
install.shenin full — including its closing(preclude-all-but [])— the StLib datatype recognisers (…#type) donot persist into the global namespace, so they can't shadow a user/test
datatype (e.g. the
print#typevs c-minusprintclash other portshit when baking). Immune by construction; kernel-tests stay 134/134.
Upstream StLib source issues
None. (An earlier
concat* is not a legitimate function namewas adouble-load artifact — loading
install.shena second time over analready-installed stdlib, where
concat*is by then a system function.A clean single boot loads it without error.)
Test status
Follow-up
shen_boot_embedded(shenffi, no-filesystem iOS path) does not yetbundle the StLib sources, so that specific path brings up the kernel
without the stdlib. Embedding the sources for the no-fs boot is future
work (flagged in
shenffi/src/lib.rsandkernel/stlib/PROVENANCE.md).🤖 Generated with Claude Code