diff --git a/BENCHMARKS.md b/BENCHMARKS.md index 94d92bb..c28e230 100644 --- a/BENCHMARKS.md +++ b/BENCHMARKS.md @@ -26,6 +26,56 @@ interpreted-dispatch model, not a single hot spot — each remaining local lever measures ≤ ~8%. The tc-cache row is verdict memoization (off by default), not raw speed. +## Cross-port: the wider field (rust vs cl vs lua) + +`scripts/cross-port-bench-4way.sh` extends the headline harness to the two +Shen-Lua execution modes (LuaJIT and PUC Lua 5.4). Same upstream suite, same +machine, interleaved min-of-N, all four timed identically by external +`/usr/bin/time` wall-clock. + +**These numbers have NOT been taken on a quiet machine — treat them as +indicative, not authoritative, and re-run with `cross-port-bench-4way.sh` on an +idle box before quoting.** What is firm (stable across load): **shen-cl is +fastest** (~1.0 s, a saved SBCL image) and **PUC Lua is slowest** (~12–14 s, a +pure interpreter). What is **not** yet resolved: the **shen-rust vs LuaJIT** +ranking — they land close together in the ~2.5 s range and the order flips with +machine load (rust's release tree-walk vs LuaJIT's trace JIT). An earlier draft +of this section claimed "rust ~2× faster than LuaJIT"; that was an +apples-to-oranges error — it compared rust's *internal eval timer* against a +*contended* LuaJIT wall-time. Under the consistent harness they are roughly +comparable. Do not repeat the 2× claim without a clean measurement. + +| Port | engine | indicative wall (loaded box) | status | +|---|---|---:|---| +| shen-cl | SBCL (saved image) | ≈ 1.0 s | firm: fastest | +| **shen-rust** | release tree-walk | ≈ 2.5 s | ~tied with LuaJIT | +| shen-lua | LuaJIT | ≈ 2.5 s | ~tied with rust | +| shen-lua | PUC Lua 5.4 | ≈ 12–14 s | firm: slowest | + +Fairness note: shen-rust (AOT kernel in the binary) and shen-cl (saved Lisp +image) boot in ~0; shen-lua loads its kernel from a cache first +(`load_kernel` ≈ 0.04 s LuaJIT / ≈ 0.39 s PUC, per `run-kernel-tests.lua`'s +printed split) — small relative to the multi-second *execution*, so total +wall-clock is a fair execution proxy. (The Lua driver prints +`Counters: passed=0 failed=0` at the end — a counter-name readout bug in the +driver, not skipped work; the suite genuinely runs and self-reports +`pass rate ... 100%` per test.) + +The interesting LuaJIT finding is independent of the wall-clock ranking: a big +chunk of the suite never JIT-compiles at all, which is why LuaJIT only ties +shen-rust's non-JIT tree-walker and lands in the low end of its usual range over +PUC Lua (~5×) rather than pulling decisively ahead. The cause: a Shen port +allocates a fresh closure per `lambda` / `freeze` / partial application, which +shen-lua emits as Lua `MKFUN(n, function(...) end)` (`compiler.lua:823/845`). +That compiles to the `FNEW` bytecode, which LuaJIT's tracer **cannot compile** +(NYI). On the suite, `luajit -jv` records **977 trace aborts on `FNEW`** and +**579 on `UCLO`** (upvalue-close), and **456 traces blacklisted** — so the +hottest closure-allocating paths permanently fall back to the interpreter. This +is *not* a warm-up artifact (blacklisted traces never recompile no matter how +long the process lives); the lever is in the port's codegen (extend shen-lua's +existing freeze free-var hoisting — the `BIND(kbody[N], …)` scheme — to +lambdas / hot loops to cut `FNEW`). See the LuaJIT-warm note for shen-lua. + ## Warm / served: VM vs tree-walker `scripts/warm-bench.sh` (+ `benches/warm_typecheck.rs`) measures a load-once / diff --git a/Cargo.lock b/Cargo.lock index e4d6f42..2aeda5b 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -1321,6 +1321,13 @@ dependencies = [ "shen-rust", ] +[[package]] +name = "shenffi" +version = "0.1.0" +dependencies = [ + "shen-rust", +] + [[package]] name = "shengen-rust" version = "0.1.0" diff --git a/Cargo.toml b/Cargo.toml index 788779b..2085a18 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -5,6 +5,7 @@ members = [ "crates/shengen-rust", "crates/klcompile", "crates/ratatoskr-build", + "crates/shenffi", "bin/shen-rust", "examples/shen-cedar-authz", ] diff --git a/PERFORMANCE.md b/PERFORMANCE.md index 0aa6e1f..2d8d963 100644 --- a/PERFORMANCE.md +++ b/PERFORMANCE.md @@ -47,6 +47,23 @@ The living, detailed record is in `design/`: filtered closure-capture caching measured −3.5% — the whole-scope memcpy in `capture_used` beats per-creation lookups even with the free-var walk amortized. +5. **Hot-leaf inline / cold-outline round, 2026-06-20** (~6% less CPU work, + paired user-CPU min-of-13, B ShenResult ShenResult { + // Hot path is a single tag decode; keep it tiny so this folds into the + // generated AOT `if` site. The non-boolean diagnostic is cold and outlined + // so its `format!` doesn't block inlining (it was the bulk of this leaf's + // self-time on the kernel-tests profile). if let Some(b) = v.as_bool() { return Ok(b); } match v.as_sym() { Some(s) if s == interp.well_known.k_true => Ok(true), Some(s) if s == interp.well_known.k_false => Ok(false), - _ => Err(ShenError::new(format!("aot: not a boolean: {v:?}"))), + _ => Err(not_a_boolean(v)), } } +#[cold] +#[inline(never)] +fn not_a_boolean(v: &Value) -> ShenError { + ShenError::new(format!("aot: not a boolean: {v:?}")) +} + /// Wrap a Rust closure as a Shen closure value. Used for AOT-compiled lambdas. /// `arity` is the formal-parameter count; partial application works via the same /// path as any other closure. @@ -117,7 +128,7 @@ pub fn is_truthy(interp: &Interp, v: &Value) -> ShenResult { /// and the move-captures hold copies of the same tagged words — the closure body /// is unchanged. Empty when the lambda captures nothing. pub fn make_aot_closure( - name: &str, + name: &'static str, arity: usize, f: F, captures: Vec, @@ -126,7 +137,10 @@ pub fn make_aot_closure( where F: Fn(&mut Interp, &[Value]) -> ShenResult + 'static, { - let sym = interp.intern(name); + // `name` is always an AOT-emitted literal (`""`/`""`); + // route through the pointer-cached path so hot lambda allocation never + // re-probes the intern HashMap. + let sym = interp.intern_static(name); let closure = Closure { name: Some(sym), arity, @@ -137,8 +151,8 @@ where } /// Look up a global. Used for `(value GLOBAL)` form. -pub fn global_value(interp: &mut Interp, name: &str) -> ShenResult { - let sym = interp.intern(name); +pub fn global_value(interp: &mut Interp, name: &'static str) -> ShenResult { + let sym = interp.intern_static(name); interp .env .get_global(sym) @@ -147,8 +161,8 @@ pub fn global_value(interp: &mut Interp, name: &str) -> ShenResult { } /// Look up a function as a value (`(fn NAME)`). -pub fn fn_value(interp: &mut Interp, name: &str) -> ShenResult { - let sym = interp.intern(name); +pub fn fn_value(interp: &mut Interp, name: &'static str) -> ShenResult { + let sym = interp.intern_static(name); interp .env .get_fn(sym) diff --git a/crates/shen-rust/src/interp/eval.rs b/crates/shen-rust/src/interp/eval.rs index 7eb31bb..bbdf286 100644 --- a/crates/shen-rust/src/interp/eval.rs +++ b/crates/shen-rust/src/interp/eval.rs @@ -530,8 +530,22 @@ impl Interp { /// Exhaustion is sticky — `remaining_steps` parks at `Some(0)` so a /// cancellation that slips past a `trap-error` handler re-fires on the /// handler's first step instead of letting evaluation resume. - #[inline] + #[inline(always)] pub(crate) fn charge_step(&mut self) -> ShenResult<()> { + // Default config sets neither budget — fold to one combined branch the + // caller's hot loop can predict-not-taken. Both Options are `None`, so + // this is a single `is_some` test pair with no body. The cold + // exhaustion/deadline machinery is outlined so it never bloats this + // into a real (un-inlined) call frame. + if self.remaining_steps.is_none() && self.deadline.is_none() { + return Ok(()); + } + self.charge_step_limited() + } + + #[cold] + #[inline(never)] + fn charge_step_limited(&mut self) -> ShenResult<()> { if let Some(n) = self.remaining_steps { if n == 0 { return Err(ShenError::cancelled("cancelled: step budget exhausted")); diff --git a/crates/shen-rust/src/symbol.rs b/crates/shen-rust/src/symbol.rs index 27e8b59..3a86343 100644 --- a/crates/shen-rust/src/symbol.rs +++ b/crates/shen-rust/src/symbol.rs @@ -53,7 +53,7 @@ impl Hasher for FnvHasher { type BuildFnv = BuildHasherDefault; /// Slot count of the direct-mapped `&'static str`-pointer cache. Power of -/// two; 8192 × 16 B = 128 KB per `Interner`. The kernel has a few thousand +/// two; 8192 × 24 B = 192 KB per `Interner`. The kernel has a few thousand /// distinct AOT callee names, so collisions (two literals mapping to one /// slot) are rare and only cost a re-intern through `by_name`. const PTR_CACHE_SLOTS: usize = 8192; @@ -76,8 +76,11 @@ pub struct Interner { /// loser re-interns through `by_name` — correct because `intern` is /// idempotent. Per-`Interner` (never process-global), so it stays /// correct even when several interpreters with different id assignments - /// coexist. - by_ptr: Box<[(usize, SymId)]>, + /// coexist. Keyed by `(addr, len)`, not addr alone: a `&'static str` that + /// is a *prefix slice* of another literal shares its start address, so the + /// length disambiguates them (today's callers pass whole literals, but the + /// helper is `pub`, so this keeps it sound for any static-str caller). + by_ptr: Box<[(usize, usize, SymId)]>, } impl Default for Interner { @@ -85,7 +88,7 @@ impl Default for Interner { Interner { names: Vec::new(), by_name: HashMap::default(), - by_ptr: vec![(0, SymId(0)); PTR_CACHE_SLOTS].into_boxed_slice(), + by_ptr: vec![(0, 0, SymId(0)); PTR_CACHE_SLOTS].into_boxed_slice(), } } } @@ -114,16 +117,17 @@ impl Interner { #[inline] pub fn intern_static(&mut self, name: &'static str) -> SymId { let key = name.as_ptr() as usize; + let len = name.len(); // Literals are ≥1-byte aligned with addresses spread across rodata; // fold a high-bit window in so neighbours in one object file don't // contend for adjacent-slot runs. let idx = ((key >> 3) ^ (key >> 16)) & (PTR_CACHE_SLOTS - 1); - let (k, id) = self.by_ptr[idx]; - if k == key { + let (k, l, id) = self.by_ptr[idx]; + if k == key && l == len { return id; } let id = self.intern(name); - self.by_ptr[idx] = (key, id); + self.by_ptr[idx] = (key, len, id); id } diff --git a/crates/shenffi/Cargo.toml b/crates/shenffi/Cargo.toml new file mode 100644 index 0000000..78462dd --- /dev/null +++ b/crates/shenffi/Cargo.toml @@ -0,0 +1,15 @@ +[package] +name = "shenffi" +version.workspace = true +edition.workspace = true + +# A thin C-ABI surface over shen-rust, so the interpreter can be embedded in +# non-Rust hosts (Swift / iOS, etc.) as a static library or framework. +# - staticlib -> libshenffi.a (link into an Xcode app / .xcframework) +# - cdylib -> libshenffi.dylib (handy for desktop linking / testing) +[lib] +name = "shenffi" +crate-type = ["staticlib", "cdylib", "rlib"] + +[dependencies] +shen-rust = { path = "../shen-rust" } diff --git a/crates/shenffi/README.md b/crates/shenffi/README.md new file mode 100644 index 0000000..313ba0e --- /dev/null +++ b/crates/shenffi/README.md @@ -0,0 +1,79 @@ +# shenffi — Swift / C bindings for shen-rust + +A thin C-ABI wrapper that embeds the [shen-rust](../shen-rust) interpreter in +non-Rust hosts — primarily **Swift on iOS/macOS**. shen-rust runs the Shen +kernel via AOT-compiled native code + a bytecode VM (no JIT in the default +build), so it's App Store-safe: it links into your app as a static library, no +runtime codegen. + +## The C surface (`include/shenffi.h`) + +```c +ShenCtx *shen_boot(const char *kernel_dir); // NULL -> auto-locate kernel +char *shen_eval(ShenCtx *ctx, const char *src); // -> rendered result / "error: …" +void shen_string_free(char *s); +void shen_free(ShenCtx *ctx); +``` + +`shen_eval` runs a **Shen-level** expression through the kernel's own pipeline +(`read-from-string` → `head` → `eval`) and renders the result with `shen.app`, +so output matches the REPL. + +## Swift wrapper (`swift/ShenRust.swift`) + +```swift +let shen = try ShenRust(kernelDir: bundleKernelPath) // nil to auto-locate +print(shen.eval("(map (/. X (* X X)) [1 2 3 4])")) // -> [1 4 9 16] +``` + +## Build + +**macOS (desktop / quick test):** +```sh +cargo build --release -p shenffi # -> target/release/libshenffi.{a,dylib} +# round-trip demo: +cd crates/shenffi +swiftc -O -import-objc-header include/shenffi.h swift/ShenRust.swift swift/main.swift \ + -L ../../target/release -lshenffi -o /tmp/shenrust_demo +DYLD_LIBRARY_PATH=../../target/release /tmp/shenrust_demo +``` + +**iOS (device + simulator XCFramework):** +```sh +crates/shenffi/build-xcframework.sh # -> target/ShenRust.xcframework +``` +Then in Xcode: add `ShenRust.xcframework`, set `shenffi.h` as the bridging +header (or wrap in a module map), and drop in `swift/ShenRust.swift`. + +## Kernel on iOS + +`shen_boot(NULL)` auto-locates the kernel by walking the filesystem — fine on +desktop, not on a sandboxed device. Two device-friendly options: + +1. **Bundle the `.kl` kernel** as an app resource and pass its directory path to + `shen_boot(path)`. +2. **Embed a shaken `kernel.kl`** (produced by [ratatoskr](../../ratatoskr)) and + boot from the in-memory source via shen-rust's `boot_from_kl_source` — no + filesystem dependency. (Exposing that through the C ABI is a small addition; + the current `shen_boot` takes a directory path.) + +## Embedding a tree-shaken Shen program + +`shen_boot_shaken(kernel_kl, prog_kl)` boots an arbitrary Ratatoskr-shaken slice +— a minimal `kernel.kl` (only the kernel functions the program reaches) plus an +optional program `.kl` — straight from in-memory source, no filesystem access. +This is how an app embeds a specific Shen program without the full kernel. + +For a worked end-to-end example (the **shen-cas** computer-algebra system shaken +by [ratatoskr](../../ratatoskr), wrapped as a `CasEngine` + a `shen_cas_*` C ABI, +and driven from both an iced GUI and SwiftUI), see the **`cas-engine` crate in +the [shen-calc](../../../shen-calc) repo** — the app that consumes it. shenffi +itself stays program-agnostic. + +## Status + +Verified: the full shen-rust + shenffi stack **compiles for `aarch64-apple-ios` +and `aarch64-apple-ios-sim`**, packages into an `.xcframework`, and the +Swift→Rust→Shen round-trip runs correctly on macOS. The default shen-rust build +has **no JIT** (the optional Cranelift JIT is off by default), so nothing relies +on runtime code generation. diff --git a/crates/shenffi/build-xcframework.sh b/crates/shenffi/build-xcframework.sh new file mode 100755 index 0000000..6aece83 --- /dev/null +++ b/crates/shenffi/build-xcframework.sh @@ -0,0 +1,27 @@ +#!/usr/bin/env bash +# Build ShenRust.xcframework (iOS device + simulator + macOS) from the shenffi +# crate. Output: /target/ShenRust.xcframework — drag into an Xcode app, +# add the bridging header (shenffi.h) + ShenRust.swift, and you're done. +# +# The macOS slice (aarch64-apple-darwin) lets a native macOS target embed the +# same generic Shen interpreter as the iOS slices. +set -euo pipefail + +cd "$(dirname "$0")/../.." # workspace root (shen-rust/) +HDRS="crates/shenffi/include" + +for tgt in aarch64-apple-ios aarch64-apple-ios-sim aarch64-apple-darwin; do + rustup target add "$tgt" >/dev/null 2>&1 || true + echo "building shenffi for $tgt ..." + cargo build --release -p shenffi --target "$tgt" +done + +echo "packaging ShenRust.xcframework ..." +rm -rf target/ShenRust.xcframework +xcodebuild -create-xcframework \ + -library target/aarch64-apple-ios/release/libshenffi.a -headers "$HDRS" \ + -library target/aarch64-apple-ios-sim/release/libshenffi.a -headers "$HDRS" \ + -library target/aarch64-apple-darwin/release/libshenffi.a -headers "$HDRS" \ + -output target/ShenRust.xcframework + +echo "done -> target/ShenRust.xcframework" diff --git a/crates/shenffi/include/shenffi.h b/crates/shenffi/include/shenffi.h new file mode 100644 index 0000000..40b13ca --- /dev/null +++ b/crates/shenffi/include/shenffi.h @@ -0,0 +1,41 @@ +/* shenffi.h — C ABI for embedding shen-rust (Shen language) in a host app. */ +#ifndef SHENFFI_H +#define SHENFFI_H + +#ifdef __cplusplus +extern "C" { +#endif + +/* Opaque handle to a booted Shen image. */ +typedef struct ShenCtx ShenCtx; + +/* Boot a Shen kernel. Pass NULL for kernel_dir to auto-locate the vendored + * kernel, or an absolute path to a directory of .kl files. Returns NULL on + * failure. Free with shen_free(). */ +ShenCtx *shen_boot(const char *kernel_dir); + +/* Boot the full kernel from sources embedded in the binary — no filesystem + * access. Recommended on iOS. Returns NULL on failure; free with shen_free(). */ +ShenCtx *shen_boot_embedded(void); + +/* Evaluate a Shen expression; returns a heap-allocated C string (the rendered + * result, or "error: "). Release it with shen_string_free(). */ +char *shen_eval(ShenCtx *ctx, const char *src); + +/* Boot any Ratatoskr-shaken program from a shaken kernel.kl slice + optional + * program .kl (pass NULL prog_kl for kernel-only). The program is loaded with + * *hush* set, so its load-time stdout chatter is suppressed. Free with + * shen_free(). */ +ShenCtx *shen_boot_shaken(const char *kernel_kl, const char *prog_kl); + +/* Free a string returned by shen_eval(). */ +void shen_string_free(char *s); + +/* Free a handle returned by shen_boot(). */ +void shen_free(ShenCtx *ctx); + +#ifdef __cplusplus +} +#endif + +#endif /* SHENFFI_H */ diff --git a/crates/shenffi/src/lib.rs b/crates/shenffi/src/lib.rs new file mode 100644 index 0000000..3b6f503 --- /dev/null +++ b/crates/shenffi/src/lib.rs @@ -0,0 +1,249 @@ +//! C-ABI bindings for embedding shen-rust in non-Rust hosts (Swift / iOS). +//! +//! The whole surface is four functions: +//! +//! ```c +//! ShenCtx *shen_boot(const char *kernel_dir); // NULL dir -> auto-locate +//! char *shen_eval(ShenCtx *ctx, const char *src); +//! void shen_string_free(char *s); +//! void shen_free(ShenCtx *ctx); +//! ``` +//! +//! `shen_eval` evaluates a *Shen-level* expression (not raw KLambda) by driving +//! the kernel's own pipeline — `read-from-string` -> `head` -> `eval` — and then +//! rendering the result to a string with `shen.app`. The returned C string is +//! heap-allocated by Rust and must be released with `shen_string_free`. + +use std::ffi::{CStr, CString}; +use std::os::raw::c_char; +use std::path::PathBuf; + +use shen_rust::interp::boot::{boot, boot_from_kl_source, boot_with_kernel, eval_kl_source}; +use shen_rust::interp::eval::Interp; +use shen_rust::value::Value; + +/// Opaque handle to a booted Shen image. +pub struct ShenCtx { + interp: Interp, +} + +/// The ShenOSKernel-41.2 `.kl` sources, embedded into the binary at compile +/// time in shen-rust's load order. Lets `shen_boot_embedded` bring up the full +/// kernel with **no filesystem access** — the iOS-friendly path (no bundled +/// resources, no sandbox path juggling). +const KERNEL_PARTS: &[&str] = &[ + include_str!("../../../kernel/klambda/core.kl"), + include_str!("../../../kernel/klambda/toplevel.kl"), + include_str!("../../../kernel/klambda/sys.kl"), + include_str!("../../../kernel/klambda/reader.kl"), + include_str!("../../../kernel/klambda/prolog.kl"), + include_str!("../../../kernel/klambda/load.kl"), + include_str!("../../../kernel/klambda/writer.kl"), + include_str!("../../../kernel/klambda/macros.kl"), + include_str!("../../../kernel/klambda/declarations.kl"), + include_str!("../../../kernel/klambda/types.kl"), + include_str!("../../../kernel/klambda/t-star.kl"), + include_str!("../../../kernel/klambda/sequent.kl"), + include_str!("../../../kernel/klambda/track.kl"), + include_str!("../../../kernel/klambda/dict.kl"), + include_str!("../../../kernel/klambda/compiler.kl"), + include_str!("../../../kernel/klambda/stlib.kl"), + include_str!("../../../kernel/klambda/init.kl"), + include_str!("../../../kernel/klambda/extension-features.kl"), + include_str!("../../../kernel/klambda/extension-expand-dynamic.kl"), + include_str!("../../../kernel/klambda/extension-launcher.kl"), + include_str!("../../../kernel/klambda/yacc.kl"), +]; + +/// Boots a Shen kernel and returns an owning handle, or NULL on failure. +/// +/// `kernel_dir` is an optional path to a directory of `.kl` kernel files; pass +/// NULL to let shen-rust auto-locate its vendored kernel. +/// +/// # Safety +/// `kernel_dir` must be NULL or a valid NUL-terminated C string. +#[no_mangle] +pub extern "C" fn shen_boot(kernel_dir: *const c_char) -> *mut ShenCtx { + let mut interp = Interp::new(); + let result = if kernel_dir.is_null() { + boot(&mut interp) + } else { + let dir = unsafe { CStr::from_ptr(kernel_dir) } + .to_string_lossy() + .into_owned(); + boot_with_kernel(&mut interp, &PathBuf::from(dir)) + }; + match result { + Ok(()) => Box::into_raw(Box::new(ShenCtx { interp })), + Err(_) => std::ptr::null_mut(), + } +} + +/// Boots the full kernel from the **embedded** `.kl` sources — no filesystem +/// access at all. This is the recommended entry point on iOS. Returns NULL on +/// failure; free with `shen_free`. +#[no_mangle] +pub extern "C" fn shen_boot_embedded() -> *mut ShenCtx { + let mut interp = Interp::new(); + let src = KERNEL_PARTS.join("\n"); + match boot_from_kl_source(&mut interp, &src, Some(shen_rust::aot::kernel::install_all)) { + Ok(()) => Box::into_raw(Box::new(ShenCtx { interp })), + Err(_) => std::ptr::null_mut(), + } +} + +/// Boots any Ratatoskr-shaken program: a shaken `kernel.kl` slice plus an +/// optional program `.kl`. Pass NULL `prog_kl` for kernel-only. +/// +/// Follows the builder contract — boot the kernel and run `(shen.initialise)` +/// first, then load the program (whose top-level forms need the initialised +/// environment). The program load runs with `*hush*` set, so the program's own +/// "…loaded" chatter to stdout is suppressed (file writes still happen). +/// +/// # Safety +/// `kernel_kl` must be a valid C string; `prog_kl` NULL or a valid C string. +#[no_mangle] +pub extern "C" fn shen_boot_shaken( + kernel_kl: *const c_char, + prog_kl: *const c_char, +) -> *mut ShenCtx { + if kernel_kl.is_null() { + return std::ptr::null_mut(); + } + let kernel = unsafe { CStr::from_ptr(kernel_kl) } + .to_string_lossy() + .into_owned(); + let prog = if prog_kl.is_null() { + None + } else { + Some( + unsafe { CStr::from_ptr(prog_kl) } + .to_string_lossy() + .into_owned(), + ) + }; + match boot_shaken_inner(&kernel, prog.as_deref()) { + Ok(interp) => Box::into_raw(Box::new(ShenCtx { interp })), + Err(e) => { + eprintln!("shen_boot_shaken error: {e}"); + std::ptr::null_mut() + } + } +} + +fn boot_shaken_inner(kernel: &str, prog: Option<&str>) -> Result { + let mut interp = Interp::new(); + boot_from_kl_source(&mut interp, kernel, None).map_err(|e| e.to_string())?; + if let Some(p) = prog { + // Silence the program's load-time stdout messages (e.g. shen-cas's + // "…loaded" lines). shen-rust's `pr` honours `*hush*` for stdout only, + // so file writes during load are unaffected. Restore afterward. + let hush = interp.intern("*hush*"); + interp.env.set_global(hush, Value::bool(true)); + let r = eval_kl_source(&mut interp, p, "shaken program", &[]).map_err(|e| e.to_string()); + interp.env.set_global(hush, Value::bool(false)); + r?; + } + Ok(interp) +} + +/// Evaluates a Shen expression and returns its rendered value as a freshly +/// allocated C string (release with `shen_string_free`). On error the string is +/// `"error: "`. Returns NULL only if the arguments are NULL. +/// +/// # Safety +/// `ctx` must be a handle from `shen_boot` (not yet freed) and `src` a valid +/// NUL-terminated C string. +#[no_mangle] +pub extern "C" fn shen_eval(ctx: *mut ShenCtx, src: *const c_char) -> *mut c_char { + if ctx.is_null() || src.is_null() { + return std::ptr::null_mut(); + } + let ctx = unsafe { &mut *ctx }; + let src = unsafe { CStr::from_ptr(src) } + .to_string_lossy() + .into_owned(); + let out = match eval_shen(&mut ctx.interp, &src) { + Ok(s) => s, + Err(e) => format!("error: {e}"), + }; + // CString::new fails only on interior NULs; fall back to empty in that case. + CString::new(out) + .unwrap_or_else(|_| CString::new("").unwrap()) + .into_raw() +} + +/// Releases a string returned by `shen_eval`. +/// +/// # Safety +/// `s` must be NULL or a pointer previously returned by `shen_eval`. +#[no_mangle] +pub extern "C" fn shen_string_free(s: *mut c_char) { + if !s.is_null() { + unsafe { drop(CString::from_raw(s)) }; + } +} + +/// Releases a handle returned by `shen_boot`. +/// +/// # Safety +/// `ctx` must be NULL or a handle from `shen_boot` that has not been freed. +#[no_mangle] +pub extern "C" fn shen_free(ctx: *mut ShenCtx) { + if !ctx.is_null() { + unsafe { drop(Box::from_raw(ctx)) }; + } +} + +/// Evaluate one Shen-level expression and render the result to a string, +/// entirely through kernel functions so the semantics match the REPL. +fn eval_shen(interp: &mut Interp, src: &str) -> Result { + // Intern every symbol up front (mutable borrow), then resolve + apply each + // kernel function in turn (the get_fn borrow ends at each `.cloned()`). + let read_sym = interp.intern("read-from-string"); + let head_sym = interp.intern("head"); + let eval_sym = interp.intern("eval"); + let app_sym = interp.intern("shen.app"); + let mode = Value::sym(interp.intern("shen.s")); + + let read_fn = interp + .env + .get_fn(read_sym) + .cloned() + .ok_or_else(|| "read-from-string is undefined".to_string())?; + let forms = interp + .apply(read_fn, vec![Value::str(src)]) + .map_err(|e| e.to_string())?; + + let head_fn = interp + .env + .get_fn(head_sym) + .cloned() + .ok_or_else(|| "head is undefined".to_string())?; + let first = interp + .apply(head_fn, vec![forms]) + .map_err(|e| e.to_string())?; + + let eval_fn = interp + .env + .get_fn(eval_sym) + .cloned() + .ok_or_else(|| "eval is undefined".to_string())?; + let result = interp + .apply(eval_fn, vec![first]) + .map_err(|e| e.to_string())?; + + let app_fn = interp + .env + .get_fn(app_sym) + .cloned() + .ok_or_else(|| "shen.app is undefined".to_string())?; + let rendered = interp + .apply(app_fn, vec![result, Value::str(""), mode]) + .map_err(|e| e.to_string())?; + + rendered + .as_str() + .map(|s| s.to_string()) + .ok_or_else(|| "result did not render to a string".to_string()) +} diff --git a/crates/shenffi/swift/ShenRust.swift b/crates/shenffi/swift/ShenRust.swift new file mode 100644 index 0000000..a947330 --- /dev/null +++ b/crates/shenffi/swift/ShenRust.swift @@ -0,0 +1,35 @@ +import Foundation + +/// Swift wrapper around the shenffi C ABI: a booted, embedded Shen interpreter. +/// +/// The C functions (`shen_boot`, `shen_eval`, ...) are exposed to Swift via the +/// `shenffi.h` bridging header. +public final class ShenRust { + public enum Failure: Error { case bootFailed } + + private let ctx: OpaquePointer + + /// Boots a Shen image. + /// + /// Pass `nil` (the default) to boot from the kernel sources embedded in the + /// binary — no filesystem access, the recommended path on iOS. Pass a + /// directory of `.kl` files to boot from disk instead. + public init(kernelDir: String? = nil) throws { + let handle: OpaquePointer? = kernelDir.map { dir in + dir.withCString { shen_boot($0) } + } ?? shen_boot_embedded() + guard let handle else { throw Failure.bootFailed } + ctx = handle + } + + /// Evaluates a Shen expression and returns its rendered result (or an + /// `"error: …"` string if evaluation failed). + @discardableResult + public func eval(_ source: String) -> String { + guard let out = source.withCString({ shen_eval(ctx, $0) }) else { return "" } + defer { shen_string_free(out) } + return String(cString: out) + } + + deinit { shen_free(ctx) } +} diff --git a/crates/shenffi/swift/main.swift b/crates/shenffi/swift/main.swift new file mode 100644 index 0000000..8d56d82 --- /dev/null +++ b/crates/shenffi/swift/main.swift @@ -0,0 +1,25 @@ +import Foundation + +// Demonstrates the Swift -> Rust -> Shen round-trip via the ShenRust wrapper. +// Usage: demo [kernel-dir] (kernel-dir optional; nil auto-locates) + +let kernelDir = CommandLine.arguments.count > 1 ? CommandLine.arguments[1] : nil + +do { + let shen = try ShenRust(kernelDir: kernelDir) + let exprs = [ + "(+ 1 2)", + "(* 6 7)", + "(append [1 2 3] [4 5 6])", + "(map (/. X (* X X)) [1 2 3 4])", + "(+ 1 2.5)", + "(reverse [a b c])", + "(if (> 3 2) yes no)", + ] + for e in exprs { + print("\(e) => \(shen.eval(e))") + } +} catch { + FileHandle.standardError.write(Data("boot failed: \(error)\n".utf8)) + exit(1) +} diff --git a/scripts/cross-port-bench-4way.sh b/scripts/cross-port-bench-4way.sh new file mode 100755 index 0000000..0ab25cd --- /dev/null +++ b/scripts/cross-port-bench-4way.sh @@ -0,0 +1,60 @@ +#!/bin/bash +# 4-way cross-port benchmark on the upstream Shen kernel test suite: +# shen-rust (release), shen-cl (SBCL image), shen-lua under LuaJIT, and +# shen-lua under PUC Lua. Reports min-of-N wall-clock, interleaved to share +# thermal state. The machine has run-to-run variance, so trust the ratio and +# the ordering, not any single absolute (and run it on an UNLOADED machine — +# a background build or video call dominates the signal). +# +# Note on fairness: shen-rust (AOT kernel baked into the binary) and shen-cl +# (saved Lisp image) boot in ~0; shen-lua loads its kernel from a cache +# (~0.04s LuaJIT / ~0.39s PUC) then *executes* the suite. The kernel-load +# fraction is small for all ports, so total wall-clock is a fair proxy for +# execution speed here — but see run-kernel-tests.lua's printed +# load_kernel/initialise split if you want execution-only. +set -euo pipefail +cd "$(dirname "$0")/.." + +N="${1:-5}" +RUST_BIN="target/release/shen-rust" +CL_BIN="../shen-cl/bin/sbcl/shen" +LUA_DIR="../shen-lua" + +cat > /tmp/cl-in.shen < the pipe -> awk. (Redirecting the program's +# stderr at the outer level would also swallow time's own output, which writes +# to fd 2 — that was the bug this structure avoids.) +timeit() { + { /usr/bin/time -p sh -c "$1 >/dev/null 2>&1"; } 2>&1 | awk '/^real/{print $2}' +} + +mn() { printf '%s\n' "$@" | sort -n | head -1; } + +echo "== 4-way kernel-tests, min-of-$N (interleaved) ==" +declare -a R C LJ L +for i in $(seq 1 "$N"); do + R[$i]=$(timeit "'$RUST_BIN' --kernel-tests") + C[$i]=$(timeit "'$CL_BIN' < /tmp/cl-in.shen") + if command -v luajit >/dev/null; then + LJ[$i]=$(timeit "cd '$LUA_DIR' && luajit run-kernel-tests.lua") + fi + if command -v lua >/dev/null; then + L[$i]=$(timeit "cd '$LUA_DIR' && lua run-kernel-tests.lua") + fi + printf "round %d: rust=%s cl=%s luajit=%s lua=%s\n" \ + "$i" "${R[$i]:-NA}" "${C[$i]:-NA}" "${LJ[$i]:-NA}" "${L[$i]:-NA}" +done + +rmin=$(mn "${R[@]}"); cmin=$(mn "${C[@]}") +echo "---" +printf "shen-cl (SBCL) : %ss 1.00x (ref)\n" "$cmin" +awk -v r="$rmin" -v c="$cmin" 'BEGIN{printf "shen-rust (release): %ss %.2fx\n", r, r/c}' +if [ -n "${LJ[1]:-}" ]; then ljmin=$(mn "${LJ[@]}"); awk -v x="$ljmin" -v c="$cmin" 'BEGIN{printf "shen-lua (luajit) : %ss %.2fx\n", x, x/c}'; fi +if [ -n "${L[1]:-}" ]; then lmin=$(mn "${L[@]}"); awk -v x="$lmin" -v c="$cmin" 'BEGIN{printf "shen-lua (PUC lua) : %ss %.2fx\n", x, x/c}'; fi