Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
93ce5ea
Add TinyGo canary tests for base64/hex, io/context, json, wasmimport,…
pflynn-virtru Feb 9, 2026
dfc972b
docs(sdk): add WASM core engine spike plan
pflynn-virtru Feb 9, 2026
6fdeba4
refactor(sdk): remove wasmimport module and consolidate WASM signatur…
pflynn-virtru Feb 9, 2026
f83b6c2
test(sdk): add TDF decrypt benchmarks for throughput baselines
pflynn-virtru Feb 12, 2026
8698ac8
test(sdk): add experimental TDF writer encrypt benchmarks
pflynn-virtru Feb 12, 2026
a734057
feat(sdk): replace encoding/json with tinyjson for TinyGo WASM compat…
pflynn-virtru Feb 12, 2026
9f1a82c
test(sdk): add zipstream TinyGo canary validating TDF ZIP writer unde…
pflynn-virtru Feb 12, 2026
2201192
style(ci): fix shellcheck SC2129 in tinygo-wasm-canary workflow
pflynn-virtru Feb 12, 2026
6092ef9
docs(sdk): add I/O architecture decision to WASM spike plan
pflynn-virtru Feb 13, 2026
2993ea9
feat(sdk): add hostcrypto package with typed WASM host function wrappers
pflynn-virtru Feb 13, 2026
2f39673
feat(sdk): add Wazero host module with crypto and I/O host functions
pflynn-virtru Feb 13, 2026
e819e09
feat(sdk): implement single-segment TDF3 encrypt in WASM module
pflynn-virtru Feb 13, 2026
316db97
test(sdk): add WASM TDF encrypt integration tests with proc_exit fix
pflynn-virtru Feb 13, 2026
651ff53
refactor(lib/ocrypto): split TinyGo-safe utils into cryptoutil sub-pa…
pflynn-virtru Feb 17, 2026
736d65d
feat(sdk): add GMAC integrity support to WASM TDF encrypt
pflynn-virtru Feb 17, 2026
1c9563e
revert: remove unused cryptoutil sub-package
pflynn-virtru Feb 17, 2026
81a6822
docs(sdk): add FIPS compliance constraint to WASM spike doc
pflynn-virtru Feb 17, 2026
cf7e96c
fix(sdk): rename WASM exports malloc/free to tdf_malloc/tdf_free
pflynn-virtru Feb 17, 2026
7dd876e
fix(sdk): write payload sizes in ZIP local file header for single-seg…
pflynn-virtru Feb 17, 2026
204ad50
docs(sdk): add JVM host and cross-platform validation results to WASM…
pflynn-virtru Feb 17, 2026
ce6e5d7
feat(sdk): add single-segment TDF decrypt to WASM module
pflynn-virtru Feb 18, 2026
7eefa58
feat(sdk): fix WASM segment signatures, add multi-segment support, cr…
pflynn-virtru Feb 18, 2026
f5fc048
feat(examples): add cross-SDK benchmark comparing Production, Writer,…
pflynn-virtru Feb 18, 2026
166ffa2
fix(sdk): eliminate WASM decrypt OOM by removing unnecessary memory c…
pflynn-virtru Feb 18, 2026
9beb25c
docs(benchmark): add WASM columns to cross-SDK benchmark results
pflynn-virtru Feb 19, 2026
555cbb1
docs(sdk): update benchmark results with Java/TS WASM encrypt numbers
pflynn-virtru Feb 19, 2026
2d94433
docs(sdk): update benchmark results with full WASM decrypt numbers
pflynn-virtru Feb 19, 2026
826f56f
fix(examples): improve Go benchmark decrypt footnotes and timing
pflynn-virtru Feb 19, 2026
3f1ddf2
docs(sdk): add 10MB and 100MB benchmark results for all WASM hosts
pflynn-virtru Feb 19, 2026
6d922da
docs(sdk): finalize spike doc with benchmark results and GO decision
pflynn-virtru Feb 19, 2026
299847d
docs(sdk): update TS benchmark numbers with KAS-inclusive WASM decrypt
pflynn-virtru Feb 20, 2026
5e29b90
feat(sdk): streaming tdf_encrypt via host I/O callbacks
pflynn-virtru Feb 23, 2026
be5ff40
docs(sdk): update spike doc with streaming I/O and Java SDK benchmark…
pflynn-virtru Feb 23, 2026
ca2bfcb
feat(sdk): switch WASM examples and tests to TinyGo + streaming encrypt
pflynn-virtru Feb 23, 2026
79767d7
docs(sdk): update spike doc with Go/wazero TinyGo benchmark numbers
pflynn-virtru Feb 23, 2026
7ef3911
docs(sdk): update spike doc with browser/V8 TinyGo benchmark numbers
pflynn-virtru Feb 23, 2026
ef37c79
docs(sdk): add TS SDK baseline + 100MB browser WASM to spike doc
pflynn-virtru Feb 23, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
143 changes: 143 additions & 0 deletions .github/workflows/tinygo-wasm-canary.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,143 @@
name: "TinyGo WASM Canary"

# Informational only — NOT in the ci aggregation job.
# Tracks TinyGo compatibility for the WASM core engine spike (SDK-WASM-1).
# Expected to have failures until the spike work is complete.
# See: docs/adr/spike-wasm-core-tinygo-hybrid.md

on:
pull_request:
paths:
- "sdk/experimental/tdf/**"
- "sdk/internal/zipstream/**"
- "sdk/manifest.go"
- "lib/ocrypto/**"
- ".github/workflows/tinygo-wasm-canary.yaml"
push:
branches:
- main
paths:
- "sdk/experimental/tdf/**"
- "sdk/internal/zipstream/**"
- "sdk/manifest.go"
- "lib/ocrypto/**"
- ".github/workflows/tinygo-wasm-canary.yaml"
workflow_dispatch:

permissions:
contents: read

jobs:
tinygo-canary:
runs-on: ubuntu-22.04
strategy:
fail-fast: false
matrix:
canary:
- name: base64hex
path: sdk/experimental/tdf/wasm/base64hex
expected: pass
description: "encoding/base64 + encoding/hex"
- name: zipwrite
path: sdk/experimental/tdf/wasm/zipwrite
expected: pass
description: "encoding/binary + hash/crc32 + bytes + sort + sync"
- name: iocontext
path: sdk/experimental/tdf/wasm/iocontext
expected: fail
description: "io + context + strings + strconv + fmt + errors"
- name: stdjson
path: sdk/experimental/tdf/wasm/stdjson
expected: fail
description: "encoding/json with TDF manifest structs (superseded by tinyjson canary)"
- name: tinyjson
path: sdk/experimental/tdf/wasm/tinyjson
expected: pass
description: "tinyjson codegen manifest + assertion round-trip (replaces encoding/json)"
- name: zipstream
path: sdk/experimental/tdf/wasm/zipstream
expected: pass
description: "production zipstream writer: TDF ZIP creation + CRC32 combine + ZIP64"
- name: wasm
path: sdk/experimental/tdf/wasm
expected: fail
description: "full WASM module — go:wasmimport host ABI + tdf package"
name: "${{ matrix.canary.name }} (${{ matrix.canary.expected }})"
steps:
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
with:
persist-credentials: false

- uses: actions/setup-go@d35c59abb061a4a6fb18e82ac0862c26744d6ab5 # v5.5.0
with:
go-version-file: sdk/go.mod
check-latest: false

- name: Install TinyGo
run: |
wget -q https://github.com/tinygo-org/tinygo/releases/download/v0.37.0/tinygo_0.37.0_amd64.deb
sudo dpkg -i tinygo_0.37.0_amd64.deb
tinygo version

- name: "Build: tinygo compile to WASM (wasip1)"
id: build
continue-on-error: true
run: |
# tinyjson canary has its own go.mod; disable workspace
export GOWORK=off
tinygo build \
-o canary.wasm \
-target=wasip1 \
-no-debug \
-scheduler=none \
-gc=leaking \
./${{ matrix.canary.path }}

- name: "Build: measure binary size"
if: steps.build.outcome == 'success'
run: |
ls -la canary.wasm
SIZE_RAW=$(stat --format=%s canary.wasm)
gzip -k -f canary.wasm
SIZE_GZ=$(stat --format=%s canary.wasm.gz)
{
echo "### ${{ matrix.canary.name }}"
echo "- Raw: ${SIZE_RAW} bytes ($(( SIZE_RAW / 1024 )) KB)"
echo "- Gzipped: ${SIZE_GZ} bytes ($(( SIZE_GZ / 1024 )) KB)"
} >> "$GITHUB_STEP_SUMMARY"

- name: "Run: execute with wasmtime"
id: run
if: steps.build.outcome == 'success'
continue-on-error: true
run: |
# Skip execution for wasm canary (needs host crypto functions)
if [ "${{ matrix.canary.name }}" = "wasm" ]; then
echo "Skipping execution — wasm module needs host crypto functions"
exit 0
fi
# Install wasmtime for execution
curl https://wasmtime.dev/install.sh -sSf | bash
export PATH="$HOME/.wasmtime/bin:$PATH"
wasmtime canary.wasm

- name: "Report result"
if: always()
run: |
BUILD="${{ steps.build.outcome }}"

Check notice

Code scanning / zizmor

code injection via template expansion Note

code injection via template expansion
RUN="${{ steps.run.outcome }}"

Check notice

Code scanning / zizmor

code injection via template expansion Note

code injection via template expansion
EXPECTED="${{ matrix.canary.expected }}"

{
echo "## ${{ matrix.canary.name }}"
echo "**Description:** ${{ matrix.canary.description }}"
echo "**Expected:** $EXPECTED"
echo "**Build:** $BUILD"
echo "**Run:** $RUN"

if [ "$BUILD" = "success" ] && [ "$RUN" != "failure" ]; then
echo "**Status: PASS**"
else
echo "**Status: FAIL**"
fi
} >> "$GITHUB_STEP_SUMMARY"
108 changes: 108 additions & 0 deletions docs/adr/cross-sdk-benchmark-results.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,108 @@
# Cross-SDK TDF Performance Benchmark Report

**Date:** 2026-02-19
**Platform:** localhost:8080 (local Docker Compose)
**Iterations:** 3 per payload size (averaged)
**Machine:** macOS Darwin 25.2.0

## Environment

| SDK | Language | Runtime |
|-----|----------|---------|
| Go Production SDK | Go | Native binary |
| Go Exp. Writer | Go | Native binary (parallel segments) |
| Go WASM | Go/TinyGo | wazero (local RSA unwrap, no KAS) |
| Java SDK | Java 11+ | JVM (HotSpot) |
| Java WASM | Go/TinyGo | Chicory 1.5.3 (pure-Java, local RSA unwrap, no KAS) |
| TypeScript SDK | TypeScript | Node.js |
| TypeScript WASM | Go/TinyGo | Node.js WebAssembly (encrypt: local RSA, decrypt: KAS rewrap) |

## Encrypt Performance (ms)

| Payload | Go SDK | Go Writer | Go WASM | Java SDK | Java WASM† | TS SDK | TS WASM |
|---------|--------|-----------|---------|----------|------------|--------|---------|
| 256 B | 3.4 | 0.2 | 0.4 | 26.0 | 57.9 | 42.8 | 0.7 |
| 1 KB | 0.1 | 0.1 | 0.2 | 5.1 | 29.5 | 25.9 | 0.2 |
| 16 KB | 0.6 | 0.1 | 0.2 | 4.9 | 7.0 | 24.7 | 0.2 |
| 64 KB | 0.1 | 0.1 | 0.3 | 5.0 | 15.6 | 28.9 | 0.3 |
| 256 KB | 0.2 | 0.5 | 1.6 | 7.8 | 51.3 | 35.8 | 0.7 |
| 1 MB | 0.7 | 1.0 | 5.8 | 21.7 | 187.4 | 73.3 | 2.6 |
| 10 MB | 5.8 | 3.0 | 44.1 | 184.3 | 1,728.9 | 519.5 | 21.3 |
| 100 MB | 57.3 | 16.4 | 543.9 | 1,768.4 | OOM | 5,205.3| OOM |

† Chicory is a pure-Java WASM interpreter (no JIT), so WASM encrypt is slower than native Java SDK. A JIT-enabled runtime (e.g., GraalWasm) would be significantly faster.
OOM at 100 MB is due to TinyGo's `gc=leaking` — the WASM linear memory cannot reclaim allocations during encrypt.

## Decrypt Performance (ms)

| Payload | Go SDK* | Go WASM** | Java SDK* | Java WASM** | TS SDK* | TS WASM* |
|---------|---------|-----------|-----------|-------------|---------|----------|
| 256 B | 20.2 | 1.3 | 116.3 | 27.7 | 62.2 | 74.9 |
| 1 KB | 18.7 | 1.2 | 28.0 | 3.3 | 60.7 | 57.6 |
| 16 KB | 18.2 | 1.3 | 24.2 | 3.1 | 46.2 | 59.9 |
| 64 KB | 17.8 | 1.2 | 23.9 | 4.5 | 54.6 | 46.0 |
| 256 KB | 17.6 | 1.6 | 25.1 | 8.8 | 73.6 | 83.6 |
| 1 MB | 17.9 | 2.5 | 39.6 | 26.8 | 71.3 | 76.1 |
| 10 MB | 21.4 | 11.6 | 197.4 | 244.4 | 298.4 | 272.9 |
| 100 MB | 58.9 | 266.1 | 1,747.8 | 2,254.1 | 2,431.5 | 2,525.7 |

\* Includes KAS rewrap network latency (~20-80ms per request)
\*\* Go and Java WASM decrypt uses local RSA-OAEP DEK unwrap (no network); in production the host would call KAS for rewrap

## Key Takeaways

**1. Go SDK is the fastest across the board.**
At 100 MB, Go encrypt (57 ms) is 31x faster than Java and 94x faster than TypeScript. The Go Experimental Writer with parallel segment processing is even faster (16 ms for 100 MB).

**2. Decrypt is dominated by KAS latency at small sizes.**
For payloads up to 1 MB, all three native SDKs show ~18-74 ms, reflecting the network round-trip to the KAS rewrap endpoint. Go and Java WASM decrypt (local RSA unwrap, no network) completes in 1.2-2.5 ms for the same sizes — 7-15x faster. TS WASM decrypt includes KAS rewrap and shows ~46-84 ms, roughly matching the native TS SDK.

**3. WASM decrypt performance varies by host methodology.**
Go and Java WASM use local RSA unwrap (no KAS network call); TS WASM includes KAS rewrap:
- Go/wazero: 1.2-266 ms (local unwrap, JIT-compiled, 100 MB = ~376 MB/s)
- TypeScript/V8: 46-2,526 ms (includes KAS rewrap; roughly matches native TS SDK)
- Java/Chicory: 3.1-2,254 ms (local unwrap, interpreted, 10-20x slower than JIT hosts)

**4. TypeScript has the highest per-operation overhead.**
Even at 256 B, SDK encrypt takes 42 ms in TypeScript vs 5.5 ms in Go and 21.9 ms in Java. This is due to Node.js async/await overhead and the SDK's internal key-fetching flow. But TS WASM bypasses this entirely — 0.7 ms for the same payload.

**5. The WASM approach validates the host-delegation architecture.**
The same `.wasm` binary (150 KB, TinyGo reactor mode) runs on all three hosts with consistent behavior. WASM encrypt+decrypt without KAS is fast enough to be practical for offline TDF operations.

**6. TypeScript WASM encrypt is remarkably fast — near Go WASM speeds.**
V8 JIT-compiles WASM to native code, so TS WASM encrypt (0.2-2.6 ms) matches Go WASM via wazero, bypassing the TS SDK's 25-73 ms overhead entirely. TS WASM decrypt includes KAS rewrap (~50-80 ms round-trip), so it roughly matches native TS SDK rather than local-unwrap Go WASM.

**7. Chicory (pure-Java interpreter) is the slowest WASM host.**
Java WASM encrypt via Chicory (7-1,729 ms) is slower than native Java SDK at all sizes. WASM decrypt is faster than native+KAS for small payloads (3 ms vs 28 ms at 1 KB) but slower at large sizes (2,254 ms vs 1,748 ms at 100 MB). A JIT-enabled WASM runtime (e.g., GraalWasm, Wasmtime-JNI) would likely match Go WASM encrypt performance.

**9. WASM encrypt OOMs at 100 MB due to TinyGo's leaking GC.**
The `gc=leaking` mode (required for stable slice pointers) means WASM linear memory can't reclaim allocations. At 100 MB, encrypt needs ~300 MB of WASM memory (plaintext + ciphertext + ZIP overhead), exceeding the default limit. Decrypt is more memory-efficient (direct-to-output-buffer) and handles 100 MB on all hosts.

**8. Java first-call warmup is visible.**
Java 256 B encrypt (21.9 ms) is 5x slower than 1 KB (4.0 ms), reflecting JIT compilation warmup on the first iteration. Steady-state Java encrypt is roughly 4-5 ms for small payloads.

## Benchmark Sources

| SDK | Benchmark File | WASM Host |
|-----|----------------|-----------|
| Go | `platform/examples/cmd/benchmark_cross_sdk.go` | wazero (built-in) |
| Java | `java-sdk/examples/src/main/java/io/opentdf/platform/BenchmarkCrossSDK.java` | Chicory 1.5.3 (`-w` flag) |
| TypeScript | `web-sdk/cli/src/benchmark.ts` | Node.js WebAssembly (`--wasmBinary` flag) |

### Running WASM benchmarks

All three benchmarks now include WASM encrypt and decrypt columns. The WASM module (`tdfcore.wasm`) is loaded at startup. Go and Java WASM use a local RSA keypair (no KAS needed); TS WASM encrypt uses a cached KAS public key and decrypt calls KAS rewrap over HTTP.

```bash
# Go (WASM compiled automatically from sdk/experimental/tdf/wasm/)
cd platform && go run ./examples benchmark-cross-sdk

# Java (requires tdfcore.wasm from wasm-host test resources)
cd java-sdk && mvn package -DskipTests -pl examples -am
java -cp examples/target/examples-0.12.0.jar io.opentdf.platform.BenchmarkCrossSDK \
-w wasm-host/src/test/resources/tdfcore.wasm

# TypeScript (defaults to ../../wasm-host/tdfcore.wasm relative to dist/)
cd web-sdk/cli && npm run build && node dist/src/benchmark.js \
--wasmBinary ../../wasm-host/tdfcore.wasm
```
Loading
Loading