Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 0 additions & 2 deletions design/mvp/Binary.md
Original file line number Diff line number Diff line change
Expand Up @@ -216,8 +216,6 @@ label' ::= len:<u32> l:<label> => l (if len = |l|)
valtype ::= i:<typeidx> => i
| pvt:<primvaltype> => pvt
resourcetype ::= 0x3f v:<valtype> f?:<core:funcidx>? => (resource (rep v) (dtor f)?)
| 0x3e v:<valtype> f:<core:funcidx>
cb?:<core:funcidx>? => (resource (rep v) (dtor async f (callback cb)?)) 🚝
functype ::= 0x40 ps:<paramlist> rs:<resultlist> => (func ps rs)
| 0x43 ps:<paramlist> rs:<resultlist> => (func async ps rs)
paramlist ::= lt*:vec(<labelvaltype>) => (param lt)*
Expand Down
630 changes: 332 additions & 298 deletions design/mvp/CanonicalABI.md

Large diffs are not rendered by default.

283 changes: 190 additions & 93 deletions design/mvp/Concurrency.md

Large diffs are not rendered by default.

62 changes: 35 additions & 27 deletions design/mvp/Explainer.md
Original file line number Diff line number Diff line change
Expand Up @@ -573,9 +573,7 @@ valtype ::= <typeidx>
| <defvaltype>
keytype ::= bool | s8 | u8 | s16 | u16 | s32 | u32 | s64 | u64 | char | string 🗺️
resourcetype ::= (resource (rep i32) (dtor <core:funcidx>)?)
| (resource (rep i32) (dtor async <core:funcidx> (callback <core:funcidx>)?)?) 🚝
| (resource (rep i64) (dtor <core:funcidx>)?) 🐘
| (resource (rep i64) (dtor async <core:funcidx> (callback <core:funcidx>)?)?) 🚝🐘
functype ::= (func async? (param "<label>" <valtype>)* (result <valtype>)?)
componenttype ::= (component <componentdecl>*)
instancetype ::= (instance <instancedecl>*)
Expand Down Expand Up @@ -828,9 +826,8 @@ is currently fixed to `i32` or `i64`, but will potentially be relaxed to include
other types. When the last handle to a resource is dropped, the resource's
destructor function specified by the `dtor` immediate will be called (if
present), allowing the implementing component to perform clean-up like freeing
linear memory allocations. Destructors can be declared `async`, with the same
meaning for the `async` and `callback` immediates as described below for `canon
lift`. A destructor for a `resource (rep $T)` must have type `($T) -> ()`.
linear memory allocations. A destructor for a `resource (rep $T)` must have type
`($T) -> ()`.

The `instance` type constructor describes a list of named, typed definitions
that can be imported or exported by a component. Informally, instance types
Expand Down Expand Up @@ -1342,13 +1339,10 @@ be deallocated and destructors called. This immediate is always optional but,
if present, is validated to have parameters matching the callee's return type
and empty results.

🔀 The `async` option specifies that the component wants to make (for imports)
or support (for exports) multiple concurrent (asynchronous) calls. This option
can be applied to any component-level function type and changes the derived
Canonical ABI significantly. See the [concurrency explainer] for more details.
When a function signature contains a `future` or `stream`, validation of `canon
lower` requires the `async` option to be set (since a synchronous call to a
function using these types is highly likely to deadlock).
🔀 The `async` option may only be used with `async` function types and specifies
that the component wants to make (for imports) or support (for exports) multiple
concurrent (asynchronous) calls. This option changes the derived Canonical ABI
significantly; see the [concurrency explainer] for more details.

🔀 The `(callback ...)` option may only be present in `canon lift` when the
`async` option has also been set and specifies a core function that is
Expand Down Expand Up @@ -2883,14 +2877,17 @@ start being rejected some time after after [WASI Preview 3] is released.

## Component Invariants

As a consequence of the shared-nothing design described above, all calls into
or out of a component instance necessarily transit through a component function
definition. Thus, component functions form a "membrane" around the collection
of core module instances contained by a component instance, allowing the
Component Model to establish invariants that increase optimizability and
composability in ways not otherwise possible in the shared-everything setting
of Core WebAssembly. The Component Model proposes establishing the following
two runtime invariants:
Component validation rules only allow a component to import and export
component-level functions, not Core WebAssembly functions. Because component-
level functions can only be produced or consumed by Canonical ABI [`lift` and
`lower` definitions](#canonical-definitions), which effectively define
[trampolines] into and out of Core WebAssembly code, the Component Model is able
to define and enforce invariants that component authors and producer toolchains
can depend on. This is analogous to the invariants provided by a traditional
Operating System to user-space code running inside a process.

In particular, the Component Model maintains the following invariants:

1. Components define a "lockdown" state that prevents continued execution
after a trap. This both prevents continued execution with corrupt state and
also allows more-aggressive compiler optimizations (e.g., store reordering).
Expand All @@ -2900,13 +2897,21 @@ two runtime invariants:
implicitly checked at every execution step by component functions. Thus,
after a trap, it's no longer possible to observe the internal state of a
component instance.
2. The Component Model disallows reentrance by trapping if a callee's
component-instance is already on the stack when the call starts.
(For details, see [`call_might_be_recursive`](CanonicalABI.md#component-instances)
in the Canonical ABI explainer.) This default prevents obscure
composition-time bugs and also enables more-efficient non-reentrant
runtime glue code. This rule will be relaxed by an opt-in
function type attribute in the [future](Concurrency.md#todo).

2. Components can only be reentered (via component export or thread resumption)
when they explicitly [block] or call a [donut wrapped] child component. Calls
to non-`async` functions do *not* count as "blocking" nor do non-blocking
(`async`-lowered) calls to `async` functions. Thus, bindings generators and
component authors do not need to always safely handle reentrance at all
import call sites. (In the [future](Concurrency.md#TODO), support for
first-class functions (as parameter and result values) would loosen this
restriction in an explicit opt-in manner.)

3. To ease adoption, unless a component opts in (via "stackful" lift 🚟 or
cooperative threads 🧵), all core wasm execution inside a component instance
is locally serialized (via automatic backpressure applied at export calls) so
that producer toolchains can continue to use a single global linear memory
shadow stack that is pushed and popped in LIFO order.


## JavaScript Embedding
Expand Down Expand Up @@ -3225,6 +3230,7 @@ For some use-case-focused, worked examples, see:
[Universal Types]: https://en.wikipedia.org/wiki/System_F
[Existential Types]: https://en.wikipedia.org/wiki/System_F
[Unit]: https://en.wikipedia.org/wiki/Unit_type
[Trampolines]: https://en.wikipedia.org/wiki/Trampoline_(computing)

[Generative]: https://www.researchgate.net/publication/2426300_A_Syntactic_Theory_of_Type_Generativity_and_Sharing
[Avoidance Problem]: https://counterexamples.org/avoidance.html
Expand All @@ -3247,6 +3253,7 @@ For some use-case-focused, worked examples, see:

[Strongly-unique]: #name-uniqueness

[Donut Wrapped]: Linking.md#higher-order-shared-nothing-linking-aka-donut-wrapping
[Adapter Functions]: FutureFeatures.md#custom-abis-via-adapter-functions
[Canonical ABI explainer]: CanonicalABI.md
[`canon_context_get`]: CanonicalABI.md#-canon-contextget
Expand Down Expand Up @@ -3309,6 +3316,7 @@ For some use-case-focused, worked examples, see:
[Resolved]: Concurrency.md#cancellation
[Cancellation]: Concurrency.md#cancellation
[Cancelled]: Concurrency.md#cancellation
[Block]: Concurrency.md#blocking

[Component Model Documentation]: https://component-model.bytecodealliance.org
[`wizer`]: https://github.com/bytecodealliance/wizer
Expand Down
164 changes: 164 additions & 0 deletions design/mvp/Linking.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,169 @@ content-hash of common modules or components by placing them in separate OCI
the first layer of the OCI Wasm Artifact.


## Higher-order Shared-Nothing Linking (aka "donut wrapping")

When using shared-nothing linking, the Component Model allows a traditional
"first-order" style of linking wherein one component's exports are supplied as
the imports of another component. This kind of linking captures the traditional
developer experience of package managers and package dependencies.

In WAT, a "first-order" dependency from `B` to `A` looks like:
```wat
(component $A
...
(export "foo" (func $foo-internal) (func (result string)))
)
```
```wat
(component $B
(import "A" (instance
(export "foo" (func $foo-internal (result string)))
))
...
)
```

`A` can be linked to `B` either directly by the host (e.g., in browsers, using
`WebAssembly.instantiate` or [ESM-integration]) or by another parent component.
For example, the following parent component `P` links `A` and `B` together:
```wat
(component $P
(import "A" (component $A (export "foo" (func (result string)))))
(import "B" (component $B (import "A" (instance (export "foo" (func (result string)))))))
(instance $a (instantiate $A))
(instance $b (instantiate $B (with "A" (instance $a))))
)
```
Note that `P` is the "parent" of `A` and `B` because `P` `instantiate`s `A` and
`B`. Whether `P` physically contains the bytecode defining `A` and `B` (as
nested `(component ...)` definitions) or `import`s the `component` definitions,
as shown here, is an orthogonal *bundling* choice that does not affect runtime
behavior (as long as the bytecode is the same in the end).

When `P` is instantiated, the resulting 3 component instances can be visualized
as nested boxes:
```
+---------------+
| P |
| +---+ +---+ |
| | A |-->| B | |
| +---+ +---+ |
+---------------+
```
Since `A` and `B` can themselves have child components, boxes can nest and form
a tree. And since `instantiate` can refer to any preceding definition in the
component, the linkage within a single box forms a Directed Acyclic Graph (DAG).

With simpler "first-order" shared-nothing linking, the definitions of parent
components like `P` only contain component-level "linking" definitions
(like `import`, `export`, `alias`, `instance`) and not any Core WebAssembly
"implementation" definitions (like `canon lift` and `canon lower`). Thus `P`
disappears at runtime, with the compiler baking all of `P`'s linkage information
into the generated code and metadata. However, there is nothing to prevent
parent components from including *both* "linking" and "implementation"
definitions.

For example, a parent component `Q` can link a child component `C` to its own
lifted and lowered core wasm modules `M1` and `M2` as follows:
```wat
(component $Q
(import "C" (component $C
(import "foo" (func (result string)))
(export "bar" (func (result string)))
))
(core module $M1
...
(export "foo-impl" (func ...))
)
(core instance $m1 (instantiate $M1))
(canon lift (core func $m1 "foo-impl") (func $foo-impl (result string)))
(instance $c (instantiate $C (with "foo" (func $foo-impl))))
(canon lower (func $c "bar") (core func $bar))
(core module $M2
(import "c" "bar" (func ...))
...
)
(core instance $m2 (instantiate $M2 (with "c" (instance (export "bar" (func $bar))))))
)
```
This new, more complex instance graph can be represented diagrammatically as:
```
+----------------------------------------------------+
| Q |
| +-----------+ +---+ +-----------+ |
| | M1 (in Q) |--lift-->| C |--lower-->| M2 (in Q) | |
| +-----------+ +---+ +-----------+ |
+----------------------------------------------------+
```
The informal term **donut wrapping** is used to describe this more advanced kind
of linking where `Q` is the "donut" with a `C`-shaped donut hole in the middle
and with `M1` and `M2` serving as the toroidal dough. (In general, parent
components can have many child instances, arbitrarily linked together and to the
internal `lift` and `lower` definitions of the parent, so perhaps a different
metaphor than "donut" would be appropriate.)

Because parent components control all linkage of their children's imports and
exports, donut wrapping allows a parent component to run its own Core
WebAssembly code on all paths into and out of all child components, allowing the
parent to arbitrarily *virtualize* the execution environment of its child
components. This is analogous to how a traditional operating system kernel can
control how and when its user-space processes run and what happens when they
make syscalls.

What is particularly powerful about donut wrapping is that, since `M1` and `M2`
are both inside the same component instance, they can be linked together
directly (without intervening `lift` and `lower` definitions) which allows them
to share arbitrary Core WebAssembly definitions (like functions, linear memory,
tables and globals). For example, extending the above definition of `$Q`, `$M1`
could export its `memory` and `funcref` `table` directly to `$M2`:
```wat
(component $Q
...
(core module $M1
...
(memory $mem 0)
(table $ftbl 0 funcref)
(export "mem" (memory $mem))
(export "ftbl" (table $ftbl))
)
(core instance $m1 (instantiate $M1))
...
(core module $M2
(import "m1" "mem" (memory 0))
(import "m1" "ftbl" (table 0 funcref))
...
)
(core instance $m2 (instantiate $M2 (with "m1" (instance $m1))))
...
)
```

Once `M1` and `M2` share linear memory and table state, `M2` can import the
`canon lower`ed exports of the child component `C` and store them into `ftbl`,
so that `M1` can call `C`'s exports via `call_indirect`. This provides `Q` the
flexibility to put *all* its core wasm code in `M1` (using `M2` to only do
`funcref`-plumbing), which is convenient. But this also allows `M1` to attempt
to reenter `C` while `C` is calling an import of `M1`, which would violate
[Component Invariant] #2. To prevent this, the Canonical ABI must place runtime
guards in `lift` that trap if `M1` tries to recursively reenter `C`.

Similarly, donut wrapping allows `Q` to both define resource types that are
imported by `C` and consume resource types that are defined by `C`. This allows
`Q` to create ownership cycles with `C` which may lead to resource leaks that
would normally be prevented in non-donut-wrapping cases by the acyclicity of
component instantiation.

In both of the above problematic cases, the parent is responsible for "closing
the loop" to create the cycle and thus any bugs arising from cycles are, by
default, bugs in the parent. This asymmetry reflects the fact that, when
donut-wrapping, the parent component is taking on part of the role of the "host"
with the child component being the "guest". This is an asymmetric relationship
that gives the host greater power over the guest (e.g., to virtualize the
guest's execution environment), but with this greater power comes greater
responsibility to avoid creating cycles with the guest.


## Fully-runtime dynamic linking

While many use cases for dynamic linking are covered by what is described
Expand Down Expand Up @@ -159,6 +322,7 @@ post-Preview 2 features of WIT and the Component Model.)
[WIT]: WIT.md
[`depname`]: Explainer.md#import-and-export-definitions
[`hashname`]: Explainer.md#import-and-export-definitions
[Component Invariant]: Explainer.md#component-invariants

[WebAssembly/tool-conventions]: https://github.com/WebAssembly/tool-conventions
[WebAssembly Object File]: https://github.com/WebAssembly/tool-conventions/blob/main/Linking.md
Expand Down
Loading
Loading