Skip to content

HubSpot/boomslang

Repository files navigation

boomslang

boomslang embeds CPython 3.14 in Java by running a WASI build with Chicory. Python execution stays inside the JVM, so callers do not need JNI, subprocess management, or a system Python install.

Bundled runtime

The default artifact includes:

  • CPython 3.14 built for wasm32-wasip1
  • Python stdlib plus NumPy, Pandas, Matplotlib, Pydantic, ijson, and Jinja2
  • python/bin/boomslang.wasm
  • generated Chicory AOT classes for the bundled WASM
  • copy-on-write memory snapshots for fast PythonInstance creation
  • boomslang_host, a small bridge for calling Java functions from Python

Java usage

Use the default artifact when you want the bundled Python runtime:

<dependency>
  <groupId>com.hubspot</groupId>
  <artifactId>boomslang</artifactId>
  <version>${boomslang.version}</version>
</dependency>

Create one factory and reuse it. The stdlib path is a host directory where boomslang extracts packaged Python resources. The instance root is the filesystem visible to Python as /.

import com.hubspot.boomslang.PythonExecutorFactory;
import com.hubspot.boomslang.PythonInstance;
import com.hubspot.boomslang.PythonResult;
import java.nio.file.Files;
import java.nio.file.Path;

Path pythonRoot = Files.createTempDirectory("boomslang-python");
PythonExecutorFactory factory = PythonExecutorFactory
  .builder()
  .withStdlibPath(pythonRoot)
  .build();

PythonResult result = factory.runOnWasmThread(() -> {
  PythonInstance instance = factory.createInstance(pythonRoot);
  return instance.execute("print('hello from Python')");
});

System.out.println(result.stdout());

Run Python work through runOnWasmThread. It uses a larger JVM stack and supports timeouts:

PythonInstance instance = factory.createInstance(pythonRoot);
PythonResult result = factory.runOnWasmThread(
  () -> instance.execute("print(sum(range(10)))"),
  Duration.ofSeconds(5),
  instance
);

If execution times out, the instance is poisoned. Call reset() before reusing it, or discard it.

Supported Python libraries

These imports are expected to work with the bundled runtime:

import ijson
import jinja2
import matplotlib
import numpy as np
import pandas as pd
from pydantic import BaseModel

Reusing compiled Python code

Use compile and loadCode when the same source runs many times:

PythonInstance instance = factory.createInstance(pythonRoot);
byte[] bytecode = instance.compile(sourceCode);

PythonResult first = instance.loadCode(bytecode);
instance.reset();
PythonResult second = instance.loadCode(bytecode);

Calling Java from Python

The stock host exposes boomslang_host.call(name, args) and boomslang_host.log(level, message).

PythonExecutorFactory factory = PythonExecutorFactory
  .builder()
  .withStdlibPath(pythonRoot)
  .addExtension(
    HostBridge
      .builder()
      .withFunction("lookup_user", userId -> userService.findById(userId).toJson())
      .withLogHandler((level, message) -> LOG.info("[Python] {}", message))
      .buildExtension()
  )
  .build();
from boomslang_host import call, log

user_json = call("lookup_user", "12345")
log(2, "loaded user")

Use a custom host if you need typed WASM imports or custom Python modules. See examples/custom-host/.

Adding pure-Python modules

Install small in-memory packages at factory creation:

PythonExecutorFactory factory = PythonExecutorFactory
  .builder()
  .withStdlibPath(pythonRoot)
  .withModule("my_package", "helpers", "def double(x): return x * 2")
  .build();

Build larger packages into the WASM/Python resource pipeline instead.

Python resource overlay

Most packaged Python runtime files under core/src/main/resources/python/usr/ are generated by the WASM/CPython resource pipeline and are intentionally ignored by Git. Small source-controlled Python additions that should ship with the stock runtime live under core/src/main/resources/python-overlay/ instead.

The overlay mirrors the final guest filesystem layout. During PythonExecutorFactory creation, boomslang extracts the generated python/ resources first, then copies python-overlay/ on top of that tree. For example:

core/src/main/resources/python-overlay/usr/local/lib/python3.14/boomslang_host/asyncio.py

is copied to:

<stdlibPath>/usr/local/lib/python3.14/boomslang_host/asyncio.py

Use the overlay for small tracked Python helper modules or patches that should not require rebuilding the generated CPython resource tree. Larger third-party packages should still go through the WASM/Python resource pipeline.

Runtime variants

Default artifact

Use com.hubspot:boomslang for the stock runtime. It includes the Java API, bundled WASM, Python resources, and generated Chicory AOT classes.

no-python-runtime

Use the no-python-runtime classifier when your app or another artifact provides the Python runtime:

<dependency>
  <groupId>com.hubspot</groupId>
  <artifactId>boomslang</artifactId>
  <version>${boomslang.version}</version>
  <classifier>no-python-runtime</classifier>
</dependency>

This classifier excludes python/** and com/hubspot/boomslang/compiled/**. It still includes the Java API.

Your app must provide:

  • a WASM binary, usually at python/bin/boomslang.wasm
  • Python resources under python/usr/local/lib/python3.14
  • an AOT machine factory if you want AOT instead of interpreter fallback

If your WASM is not at the default classpath location, set it with withWasmResource(...).

Custom host builds

Build a custom host when the stock boomslang_host.call(...) bridge is not enough. Custom hosts let you change the Rust host, add extensions, prewarm modules, and statically link additional native libraries into the WASI binary.

WASI does not support dynamic linking in this runtime. Any native code needed by Python extensions must be statically linked into the host build.

Use a custom host for:

  • typed WASM imports instead of string/JSON calls
  • host functions exposed as custom Python modules
  • extra Python modules prewarmed into the Wizer snapshot
  • native libraries required by Python extensions

Start from examples/custom-host/. The flow is:

  1. Define an extension contract in extension.toml.
  2. Use boomslang-hostgen to generate Rust and Java bridge code.
  3. Compose the extension with python-host-core in a custom Rust host.
  4. Add any required native libraries to the WASI build as static libraries.
  5. Build the host to wasm32-wasip1.
  6. Package the custom boomslang.wasm and matching Python resources in your app or artifact.
  7. Depend on com.hubspot:boomslang:no-python-runtime for the Java API.

Minimal build command from the example:

export CPYTHON_WASI_DIR=../../cpython/build/cpython-wasi
cargo build --target wasm32-wasip1 --release

For the stock repo build that produces the bundled runtime, use just wasm and just resources.

Building this repo

Requirements: Java 21, Maven, just, and either Docker or Apple container.

just everything

That builds the native WASM artifacts, Rust host, Python resources, Java AOT classes, and Maven packages. First runs take about an hour because the CPython and library builds are container-heavy.

Use Apple container instead of Docker:

container system start
BOOMSLANG_CONTAINER_CLI=container just everything

Common local workflows:

just build     # Maven package with AOT, skips tests
just test      # tests module
mvn compile -pl core
mvn test -pl tests

After Rust or host changes:

just wasm
just resources
just build
just test

Full pipeline stages:

just build-pydantic-core-wasi
just build-numpy-wasi
just build-pandas-wasi
just build-matplotlib-wasi
just build-ijson-wasi
just build-cpython-wasi
just pip-packages
just wasm
just resources
just build
just test

Repo map

  • core/: Java runtime API and bundled Python resources
  • tests/: integration tests
  • benchmarks/: JMH benchmarks
  • python-host/: stock Rust WASM host
  • python-host-core/: reusable Rust host core
  • extensions/: built-in host extensions
  • boomslang-hostgen/: extension code generator
  • examples/custom-host/: custom host example
  • cpython/: CPython, native library, and container build pipeline

License

Apache 2.0

About

Python, but Java

Resources

License

Security policy

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors