Skip to content

Add a component-model benchmark#310

Merged
fitzgen merged 3 commits into
bytecodealliance:mainfrom
fitzgen:cm-benchmark
Jun 2, 2026
Merged

Add a component-model benchmark#310
fitzgen merged 3 commits into
bytecodealliance:mainfrom
fitzgen:cm-benchmark

Conversation

@fitzgen

@fitzgen fitzgen commented Jun 2, 2026

Copy link
Copy Markdown
Member

This commit introduces a component-model benchmark. It exercises tiny components making many, frequent cross-component calls.

The workload computes the running mean, standard deviation, minimum, and maximum of a stream of pseudo-random f64 samples using Welford's online algorithm.

The benchmark is hand-written in the WebAssembly text format (.wat) as a root component containing three sub-components:

  1. $stats defines and exports an online-stats resource:

    resource online-stats;
    new-online-stats: func() -> own<online-stats>;
    add-sample: func(stats: borrow<online-stats>, sample: f64);
    get-mean: func(stats: borrow<online-stats>) -> f64;
    get-std-dev: func(stats: borrow<online-stats>) -> f64;
    get-min: func(stats: borrow<online-stats>) -> f64;
    get-max: func(stats: borrow<online-stats>) -> f64;

    The resource's representation is a pointer into the sub-component's linear memory, bump-allocated by new-online-stats. Statistics are updated incrementally on each add-sample.

  2. $rng is a global xorshift128+ pseudo-random number generator. Its state lives in module globals (rather than supporting many RNGs as resources):

    seed: func(seed: u64);
    next-f64: func() -> f64;
  3. $runner imports the $stats and $rng instances, the bench timing hooks, and the slice of WASI it needs, and exports a run function compatible with the WASI CLI command world.

    It:

    • reads SEED,ITERS from default.input,
    • seeds the PRNG with SEED,
    • calls bench.start,
    • creates an online-stats resource,
    • feeds it ITERS random f64 samples in [0, 1) drawn from $rng,
    • reads back the mean/std-dev/min/max,
    • calls bench.end,
    • and prints the statistics to stdout.

The root component wires everything together.

This commit introduces a component-model benchmark. It exercises tiny components
making many, frequent cross-component calls.

The workload computes the running mean, standard deviation, minimum, and maximum
of a stream of pseudo-random `f64` samples using [Welford's online
algorithm][welford].

[welford]: https://en.wikipedia.org/wiki/Algorithms_for_calculating_variance#Welford's_online_algorithm

The benchmark is hand-written in the WebAssembly text format (`.wat`) as a root
component containing three sub-components:

1. **`$stats`** defines and exports an `online-stats` resource:

   ```wit
   resource online-stats;
   new-online-stats: func() -> own<online-stats>;
   add-sample: func(stats: borrow<online-stats>, sample: f64);
   get-mean: func(stats: borrow<online-stats>) -> f64;
   get-std-dev: func(stats: borrow<online-stats>) -> f64;
   get-min: func(stats: borrow<online-stats>) -> f64;
   get-max: func(stats: borrow<online-stats>) -> f64;
   ```

   The resource's representation is a pointer into the sub-component's linear
   memory, bump-allocated by `new-online-stats`. Statistics are updated
   incrementally on each `add-sample`.

2. **`$rng`** is a global xorshift128+ pseudo-random number generator. Its state
   lives in module globals (rather than supporting many RNGs as resources):

   ```wit
   seed: func(seed: u64);
   next-f64: func() -> f64;
   ```

3. **`$runner`** imports the `$stats` and `$rng` instances, the `bench` timing
   hooks, and the slice of WASI it needs, and exports a `run` function compatible
   with the WASI CLI `command` world.

   It:
   * reads `SEED,ITERS` from `default.input`,
   * seeds the PRNG with `SEED`,
   * calls `bench.start`,
   * creates an `online-stats` resource,
   * feeds it `ITERS` random `f64` samples in `[0, 1)` drawn from `$rng`,
   * reads back the mean/std-dev/min/max,
   * calls `bench.end`,
   * and prints the statistics to stdout.

The root component wires everything together.
@fitzgen fitzgen requested a review from posborne June 2, 2026 19:16

@posborne posborne left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the addition @fitzgen, this is great. If we end up with more cm/wat I could see factoring out some of the common boilerplate at that point, but it wouldn't make sense to start there anyhow.

At some point I'm sure we'll want cm host interactions, but that is going to be a a different beast and impact the bench fixture in a more involved fashion.

@fitzgen

fitzgen commented Jun 2, 2026

Copy link
Copy Markdown
Member Author

Oops, looks like I need to add support for components to sightglass validate.


If we end up with more cm/wat I could see factoring out some of the common boilerplate at that point, but it wouldn't make sense to start there anyhow.

Yeah, we could use wac or something eventually to reuse sub-components. Will cross that bridge when we get to it.

At some point I'm sure we'll want cm host interactions, but that is going to be a a different beast and impact the bench fixture in a more involved fashion.

I'm curious what kinds of interactions your imagining here. Mostly we've sort of kept Sightglass more focused on compilation/Wasm execution, and then have criterion-based micro-benchmarks for runtime interactions in Wasmtime's repo itself (and the requestes/second benchmark for wasi-http).

@fitzgen fitzgen merged commit 0a6dae3 into bytecodealliance:main Jun 2, 2026
16 checks passed
@fitzgen fitzgen deleted the cm-benchmark branch June 2, 2026 20:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants