About

wasm-llm brings the llm.rb runtime to the WebAssembly platform.
The project traces its lineage to mruby-llm which traces its lineage back to the Ruby runtime llm.rb.

It brings that same runtime shape into WebAssembly hosts. The goal is not to expose an arbitrary Ruby VM in the browser, but to carry over the parts of llm.rb that map well to WASM: provider calls, contexts, agents, streams, and host-defined tools.

Quick start

LLM.Context

The LLM.Context object is at the heart of the runtime. Almost all other features build on top of it. It is a low-level interface to a model, and requires tool execution to be managed manually. The LLM.Agent class is almost the same as LLM.Context but it manages tool execution for you - we'll cover agents next:

const llm = LLM.deepseek({key: process.env.DEEPSEEK_SECRET})
const ctx = new LLM.Context(llm, {
  model: "deepseek-v4-flash",
  stream: process.stdout
})
await ctx.talk("Hello world")

LLM.Prompt

The LLM.Prompt object composes a single request from multiple role-aware messages. It can be passed to a context anywhere a string prompt can be used:

const llm = LLM.deepseek({key: process.env.DEEPSEEK_SECRET})
const ctx = new LLM.Context(llm, {
  model: "deepseek-v4-flash",
  stream: process.stdout
})
const prompt = LLM.Prompt(llm)
  .system("Your task is to assist the user")
  .user("Hello. Can you assist me?")
await ctx.talk(prompt)

LLM.Agent

The LLM.Agent object is implemented on top of LLM.Context. It provides the same interface, but manages tool execution for you:

const llm = LLM.deepseek({key: process.env.DEEPSEEK_SECRET})
const agent = new LLM.Agent(llm, {
  model: "deepseek-v4-flash",
  stream: process.stdout
})
await agent.talk("Write one short sentence about WebAssembly.")

Tools

The LLM.Tool function defines host callbacks that are passed through the normal Ruby tool loop. parameters can be a Zod schema directly. Host tools are synchronous for now so call() must return a plain JSON-like value, not a Promise:

const Echo = LLM.Tool({
  name: "echo",
  description: "Echo text back to the model",
  parameters: z.object({
    text: z.string().describe("The text to echo")
  }),
  call({text}) {
    return {text}
  }
})

LLM.Stream

The LLM.Stream object lets you observe output and runtime events as they happen. In wasm-llm, you define it with LLM.Stream({...}) and provide the callbacks you need:

const llm = LLM.deepseek({key: process.env.DEEPSEEK_SECRET})
const ctx = new LLM.Context(llm, {
  model: "deepseek-v4-flash",
  stream: LLM.Stream({
    onContent(content) {
      process.stdout.write(content)
    }
  })
})
await ctx.talk("Write a haiku about WebAssembly.")

LLM.Stream with reasoning

This example uses LLM.Stream with a DeepSeek reasoning-capable model so reasoning output is streamed separately from visible assistant output:

const llm = LLM.deepseek({key: process.env.DEEPSEEK_SECRET})
const ctx = new LLM.Context(llm, {
  model: "deepseek-v4-flash",
  stream: LLM.Stream({
    onContent(content) {
      process.stdout.write(content)
    },
    onReasoningContent(content) {
      process.stderr.write(content)
    }
  })
})
await ctx.talk("Explain how WebAssembly differs from JavaScript.")

Subset

We tried to keep the runtime surface intentionally small.

It focuses on provider calls, contexts, agents, streams, and host-defined tools, because those map cleanly to browser and edge environments. Features that depend on a local filesystem, spawned processes, or stdio-managed transports do not carry over to the WebAssembly runtime.

Build

build_config.rb

Add to your mruby build config:

MRuby::CrossBuild.new("wasm-llm") do |conf|
  conf.toolchain :emscripten
  conf.gembox "stdlib"
  conf.gembox "stdlib-ext"
  conf.gem core: "mruby-compiler"
  conf.gem File.expand_path("/path/to/wasm-llm")
end

Install

A fresh package is created each time a commit is pushed to the repository. The package includes llm.{js,wasm}, and it can be used to bootstrap a JavaScript environment similar to the examples from the README.

Once this project is stable and out of beta, expect tagged releases instead.

Dependencies

Declared mrbgem dependencies include:

• mruby-json
• mruby-stringio
• mruby-time
• mruby-struct
• mruby-regexp

See mrbgem.rake.

Resources

• doc site has the API docs.
• llm.rb is the CRuby runtime this is based on.

License

BSD Zero Clause
See LICENSE