Skip to content

stdlib: load from Lib/StLib sources, retire community stlib.kl#10

Merged
pyrex41 merged 1 commit into
mainfrom
stdlib/stlib-from-source
Jul 14, 2026
Merged

stdlib: load from Lib/StLib sources, retire community stlib.kl#10
pyrex41 merged 1 commit into
mainfrom
stdlib/stlib-from-source

Conversation

@pyrex41

@pyrex41 pyrex41 commented Jul 14, 2026

Copy link
Copy Markdown
Owner

stdlib: load from Lib/StLib sources, retire the community stlib.kl overlay

Follow-up to the S41.2 refresh (#9). Makes Mark Tarver's S-lineage
Lib/StLib Shen sources the standard-library source of truth and
removes the pre-compiled community stlib.kl overlay the port had kept.

Why

The community ShenOSKernel-41.2 shipped the stdlib as a pre-compiled
stlib.kl. Installing those defuns bypassed the kernel's
load/define path, so stdlib functions ended up with the "unknown"
arity -1: bare (fn filter) and top-level (filter …) failed with
fn: filter is undefined. Tarver's refresh ships the stdlib as separate
sources loaded via install.shen; loading through the real load path
registers correct arities and package namespacing, fixing that.

What this PR does

  • Vendors the S-lineage StLib sources under kernel/stlib/
    (byte-identical to mirror pyrex41/shen-upstream tag
    s41.2-pristine-20260711, Lib/StLib/; the 22 files install.shen
    loads). See kernel/stlib/PROVENANCE.md.
  • Removes kernel/klambda/stlib.kl, its AOT module
    aot/kernel/stlib.rs, the stlib entries in aot/kernel/mod.rs, and
    the embedded stlib.kl in shenffi.
  • Loads StLib at boot (boot::load_stlib, after the kernel is fully
    up) by running upstream's install.shen. Because install.shen uses
    relative (load …) and shen-rust resolves file paths against the
    process cwd, and because multiple interpreters can boot concurrently
    (cwd is process-global), it rewrites the load paths to absolute into a
    per-boot temp file and loads that — no cwd mutation, fully re-entrant.
    Runs under *hush* so the per-file chatter stays off the boot path.
  • KERNEL_SLOW_DEFUNS is now empty (crates/klcompile): its only
    members were the four stlib.initialise-* defuns in stlib.kl.
  • .cargo/config.toml sets RUST_MIN_STACK=256MB: the type-checked
    StLib load recurses past cargo's 2 MB test-thread stack (the binary
    already boots on a ≥64 MB thread). Makes cargo test / cargo run
    work without a hand-set env var.
  • Regression test
    library.rs::stdlib_loaded_from_source_has_real_arities: (arity filter) >= 0, (fn filter) resolves, (filter …) filters, and the
    package-internal list.reduce is reachable namespaced.

Behavior notes (not regressions)

  • Package-internal (non-exported) stdlib functions are namespaced:
    reduce lives in (package list …), reachable as list.reduce, not
    bare reduce (the community stlib.kl had no reduce at all). The
    test's map helper was renamed sqmy-square because sq is now a
    real StLib function and the kernel refuses to redefine a system fn.

  • Because the load re-runs install.shen in full — including its closing
    (preclude-all-but []) — the StLib datatype recognisers (…#type) do
    not persist into the global namespace, so they can't shadow a user/test
    datatype (e.g. the print#type vs c-minus print clash other ports
    hit when baking). Immune by construction; kernel-tests stay 134/134.

Upstream StLib source issues

None. (An earlier concat* is not a legitimate function name was a
double-load artifact — loading install.shen a second time over an
already-installed stdlib, where concat* is by then a system function.
A clean single boot loads it without error.)

Test status

Gate Result
0 shengen-codegen PASS
1 fmt + clippy PASS
2 build (workspace) PASS
3 test (port-authored) PASS
4 shen-check PASS
5 tcb-audit PASS
6 kernel-aot-audit PASS
7 kernel-tests (release) 134/134
8 kernel-tests (debug) 134/134
9 kernel-tests (debug-gc) 134/134
10 cedar-equiv PASS

Follow-up

shen_boot_embedded (shenffi, no-filesystem iOS path) does not yet
bundle the StLib sources, so that specific path brings up the kernel
without the stdlib. Embedding the sources for the no-fs boot is future
work (flagged in shenffi/src/lib.rs and kernel/stlib/PROVENANCE.md).

🤖 Generated with Claude Code

Make Mark Tarver's S-lineage Lib/StLib Shen sources the standard-library
source of truth and remove the pre-compiled community stlib.kl overlay.

The community ShenOSKernel-41.2 shipped the stdlib as a pre-compiled
stlib.kl; installing those defuns bypassed the kernel's load/define path,
so functions like filter ended up with the "unknown" arity -1 and bare
(fn filter) / top-level (filter ...) failed ("fn: filter is undefined").
The refresh ships the stdlib as sources loaded via install.shen; going
through the real load path registers correct arities and package
namespacing, fixing that.

- Vendor kernel/stlib/ (byte-identical to mirror pyrex41/shen-upstream tag
  s41.2-pristine-20260711, Lib/StLib/ — the 22 files install.shen loads).
- Delete kernel/klambda/stlib.kl, its AOT module aot/kernel/stlib.rs, the
  stlib entries in aot/kernel/mod.rs, and the embedded stlib.kl in shenffi.
- boot::load_stlib runs install.shen after the kernel is up. It rewrites
  the relative (load ...) paths to absolute into a per-boot temp file (no
  process-cwd mutation — cwd is process-global and interpreters boot
  concurrently) and loads that, hushed. install.shen is otherwise run
  verbatim, tc toggles and all, matching the shen-cl reference. Its closing
  (preclude-all-but []) keeps datatype recognisers from persisting.
- klcompile KERNEL_SLOW_DEFUNS is now empty (its only members were the four
  stlib.initialise-* defuns in the removed stlib.kl).
- .cargo/config.toml sets RUST_MIN_STACK: the type-checked StLib load
  recurses past cargo's 2 MB test-thread stack (the binary already boots on
  a >=64 MB thread).
- parse_kernel form-count floor lowered (stlib's ~738 forms moved out).
- Regression test library.rs::stdlib_loaded_from_source_has_real_arities:
  (arity filter) >= 0, (fn filter) resolves, filter filters, list.reduce
  reachable namespaced.

All gates green: kernel-tests 134/134 across release/debug/debug-gc,
fmt+clippy, workspace tests, shen-check, tcb-audit, kernel-aot-audit,
cedar-equiv.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@pyrex41
pyrex41 merged commit a529924 into main Jul 14, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant