Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 41 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,10 +19,25 @@ pakt/
│ ├── unmarshal.go # PAKT text → Go struct
│ ├── tags.go # Struct tag parsing (pakt:"name")
│ └── *_test.go # Tests for each component
├── dotnet/ # .NET library (net10.0)
│ ├── src/Pakt/ # Core library
│ │ ├── PaktReader.cs # Tier 0: ref struct token reader (state machine)
│ │ ├── PaktMemoryReader.cs # Tier 1: sync statement reader (memory-backed)
│ │ ├── PaktStreamReader.cs # Tier 1: async statement reader (stream-backed)
│ │ ├── PaktFramedSource.cs # Internal: NUL-aware async buffer for stream reader
│ │ ├── PaktSerializer.cs # Tier 2: convenience Deserialize/DeserializeAsync/Serialize
│ │ ├── PaktUnitMaterializer.cs # Whole-unit binding (sync + async)
│ │ ├── PaktReaderExtensions.cs # Callback-based composite navigation helpers
│ │ ├── PaktWriter.cs # Forward-only PAKT output writer
│ │ └── Serialization/ # Runtime: converters, options, type info, deserialization
│ ├── src/Pakt.Generators/ # Source generator (netstandard2.0)
│ ├── tests/ # xUnit tests
│ └── benchmarks/ # BenchmarkDotNet suites (FS, Fin, Small, Wide, Deep, Collections)
├── main.go # CLI entry point (Kong)
├── cli.go # CLI commands: parse, validate, version
├── cli_test.go # CLI integration tests (build binary, run against testdata)
├── spec/pakt-v0.md # Formal PAKT v0 specification
├── spec/benchmarks-v0.md # Cross-platform benchmark specification
├── docs/guide.md # Human-friendly PAKT guide
├── design/ # Architecture documents
│ └── state-machine-rewrite.md # Decoder state machine design
Expand Down Expand Up @@ -69,6 +84,32 @@ The decoder in `encoding/` uses an explicit-stack state machine rather than recu
- `reader_type.go` — Type annotation parser (recursive descent, bounded depth — stays separate from state machine)
- `design/state-machine-rewrite.md` — Full design doc with state transition narrative, frame payloads, risky areas, and observable behavior contract

## Architecture: .NET Two-Reader Model

The .NET library uses a layered architecture with two distinct Tier 1 readers:

### Tier 0: `PaktReader` (ref struct)

Stack-only, zero-copy tokenizer over `ReadOnlySpan<byte>`. Same state-machine design as the Go decoder but adapted for .NET's ref struct constraints. Source-generated deserializers operate at this level for maximum performance.

### Tier 1: Two readers

- **`PaktMemoryReader`** — Sync-only, `IDisposable`. For `ReadOnlyMemory<byte>` or `IMemoryOwner<byte>` input. No artificial async. The memory-backed fast path.
- **`PaktStreamReader`** — Async-only, `IAsyncDisposable`. Real `Stream.ReadAsync` at I/O refill boundaries. Uses `PaktFramedSource` internally for NUL-delimited unit framing with correct leftover handling. No sync wrappers.

The two readers share no interface or base class. This is intentional — async exists only where the underlying code path is genuinely async.

### Tier 2: `PaktSerializer`

Static convenience API: `Deserialize<T>(ReadOnlyMemory<byte>)` (sync, uses `PaktMemoryReader`), `DeserializeAsync<T>(Stream)` (async, uses `PaktStreamReader`), `Serialize<T>`. Sugar over Tier 1.

### Key design rules

- No fake async adapters. Async only on `PaktStreamReader` where `Stream.ReadAsync` is real.
- `IMemoryOwner<byte>` is the canonical ownership transfer mechanism.
- Source-generated code targets `PaktReader` directly for zero-alloc scalar reads.
- `PaktConvertContext` is a `readonly ref struct` — no heap allocation for converter context.

## Spec Compliance

The specification (`spec/pakt-v0.md`) is the authoritative source for PAKT semantics. When working on the parser or event model:
Expand Down
18 changes: 10 additions & 8 deletions dotnet/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,11 +77,12 @@ byte[] bytes = PaktSerializer.Serialize(server, AppPaktContext.Default);
### Layered Architecture

```
PaktSerializer High-level convenience (whole-unit materialization / serialization)
PaktSerializer High-level convenience (Deserialize / DeserializeAsync / Serialize)
PaktMemoryReader Statement-level iteration (assigns, packs)
PaktMemoryReader Sync statement-level iteration (memory-backed)
PaktStreamReader Async statement-level iteration (stream-backed)
PaktReader / PaktWriter Low-level token-by-token I/O
PaktReader / PaktWriter Low-level token-by-token I/O
Source Generator Compile-time (de)serialization code via [PaktSerializable]
```
Expand All @@ -92,8 +93,9 @@ byte[] bytes = PaktSerializer.Serialize(server, AppPaktContext.Default);
|---|---|
| `PaktReader` | Forward-only, zero-copy tokenizer over `ReadOnlySpan<byte>`. Ref struct — stack-only, pooled allocations. |
| `PaktWriter` | Forward-only PAKT output writer to `IBufferWriter<byte>`. |
| `PaktMemoryReader` | Statement-level reader. Iterates top-level assigns and packs. Supports `ReadValue<T>()`, `ReadPack<T>()`, and `ReadMapPack<TKey, TValue>()`. |
| `PaktSerializer` | Static convenience API for whole-unit deserialize/serialize over generated metadata. |
| `PaktMemoryReader` | Sync statement-level reader for `ReadOnlyMemory<byte>` / `IMemoryOwner<byte>`. `ReadValue<T>()`, `ReadPack<T>()`, `ReadMapPack<TKey, TValue>()`. |
| `PaktStreamReader` | Async statement-level reader for `Stream`. Real `Stream.ReadAsync` at I/O boundaries. `ReadValueAsync<T>()`, `ReadPackAsync<T>()` (`IAsyncEnumerable`). |
| `PaktSerializer` | Static convenience API. `Deserialize<T>` (sync memory), `DeserializeAsync<T>` (async stream), `Serialize<T>`. |
| `PaktSerializerContext` | Base class for source-generated serialization contexts. Provides `GetTypeInfo<T>()` for type resolution. |
| `PaktType` | Immutable PAKT type descriptor (scalars, structs, tuples, lists, maps, atom sets). |
| `PaktException` | Parse/validation error with `PaktPosition` and `PaktErrorCode`. |
Expand Down Expand Up @@ -178,14 +180,14 @@ Consumer projects must reference the generator as an analyzer — not a regular

- ✅ Streaming tokenizer (`PaktReader`) — all PAKT scalar and composite types
- ✅ Forward-only writer (`PaktWriter`)
- ✅ Statement-level reader (`PaktMemoryReader`) — assigns and packs
- ✅ Statement-level reader (`PaktMemoryReader`) — sync, memory-backed
- ✅ Async stream reader (`PaktStreamReader`) — real `Stream.ReadAsync` at I/O boundaries
- ✅ Source-generated (de)serialization — structs, lists, maps, nullable types, nested types
- ✅ Convenience API (`PaktSerializer`) — whole-unit deserialize/serialize
- ✅ Convenience API (`PaktSerializer`) — `Deserialize<T>` (sync) + `DeserializeAsync<T>` (stream)

**Deferred:**

- Tuple deserialization (tuples are tokenized but not mapped to C# types)
- Source-generated tuple serialization/deserialization coverage
- Spec projection (`.spec.pakt` filtering)
- NuGet packaging
- A dedicated stream-native `PaktStreamReader` with real async I/O (planned)
4 changes: 2 additions & 2 deletions site/content/docs/examples.md
Original file line number Diff line number Diff line change
Expand Up @@ -292,9 +292,9 @@ rollback-version:(int, int, int)? = nil # not set

# Service metadata map
meta:<str ; str> = <
'owner' = 'platform-team'
'owner' ; 'platform-team'
'region' ; 'us-east-1'
'tier' = 'critical'
'tier' ; 'critical'
>

# Health check configuration
Expand Down
20 changes: 20 additions & 0 deletions site/content/docs/install.md
Original file line number Diff line number Diff line change
Expand Up @@ -186,6 +186,26 @@ var server = PaktSerializer.Deserialize<Server>(paktBytes, AppPaktContext.Defaul
byte[] bytes = PaktSerializer.Serialize(server, AppPaktContext.Default);
```

### Read from a stream (async)

```csharp
await using var reader = PaktStreamReader.Create(networkStream, AppPaktContext.Default);

while (await reader.ReadStatementAsync(ct))
{
if (reader.IsPack && reader.StatementType.IsList)
{
await foreach (var s in reader.ReadPackAsync<Server>(ct))
Console.WriteLine($"{s.Host}:{s.Port}");
}
else
{
var s = await reader.ReadValueAsync<Server>(ct);
Console.WriteLine($"{s.Host}:{s.Port}");
}
}
```

### Requirements

- .NET 10
Loading