Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 43 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,12 @@ A Roslyn-based source generator that converts FIX Simple Binary Encoding (SBE) X
- Composite types (nested composites, `<ref>` elements)
- Enumerations and bit sets (flag enums) with `Has(flag)` / `Is{Flag}()` extension helpers
- Repeating groups with nested groups (unlimited depth, ancestor context callbacks with `in`)
- `foreach`-style zero-allocation enumerator on simple top-level groups (v1.5.0)
- Variable-length data (varData) with configurable length prefix (uint8/uint16/uint32)
- Constant fields in messages, composites, and groups
- Derived numeric constants (`MinValue`/`MaxValue` on type wrappers; `Decimals`/`Multiplier`/`MultiplierDecimal`/`Divisor` on decimal composites)
- Allocation-free formatting via `ISpanFormattable` on char types and decimal composites; `AsTrimmedSpan()` on InlineArray char types
- Static header parsing helpers (`TryReadTemplateId` / `TryReadHeader`) on the message header composite
- Automatic and explicit field offset calculation
- Byte order handling (little-endian native, big-endian with `readonly` property-based conversion)
- Schema versioning (`sinceVersion`, `deprecated` on fields, enums, sets, data) with per-message `{Msg}VersionMap` for `blockLength → version` lookup
Expand All @@ -30,6 +32,37 @@ A Roslyn-based source generator that converts FIX Simple Binary Encoding (SBE) X
- Zero-cost `SbeDispatcher` + `ISbeMessageHandler` for devirtualized message routing
- Comprehensive build-time diagnostics (SBE001–SBE014)

## What's New in v1.5.0

```csharp
// #156: foreach-style enumerator for top-level simple groups — zero alloc, no closure capture
if (OrderBookData.TryParse(buffer, out var reader))
{
foreach (ref readonly var bid in reader.Bids) Process(in bid);
foreach (ref readonly var ask in reader.Asks) Process(in ask);
}
```

Independent per-group views — out-of-order access, early `break`, and repeated iteration are all safe (no shared cursor). Emitted only when all top-level groups are simple (no nested groups, no group-level varData); otherwise `ReadGroups` remains the only API.

## What's New in v1.4.0

```csharp
// #155: AsTrimmedSpan — like AsSpan but strips trailing space/null padding (FIX convention) without allocating.
ReadOnlySpan<byte> sym = msg.Symbol.AsTrimmedSpan();
if (sym.SequenceEqual("PETR4"u8)) { /* ... */ }

// #153: ISpanFormattable on char types and decimal composites — allocation-free formatting
Span<char> dest = stackalloc char[32];
msg.Price.TryFormat(dest, out int written, default, CultureInfo.InvariantCulture);

// #156 (helpers): single source of truth for header layout
if (MessageHeader.TryReadHeader(buffer, out var blockLength, out var templateId, out var schemaId, out var version))
{
// dispatch without hardcoding offsets
}
```

## What's New in v1.2.0

Four additive consumer-ergonomics features — all emit-time only, zero impact on the encode/decode hot path:
Expand Down Expand Up @@ -174,10 +207,16 @@ bool success = OrderBookData.TryEncode(
if (OrderBookData.TryParse(buffer, out var reader))
{
Console.WriteLine($"Instrument: {reader.Data.InstrumentId}");
reader.ReadGroups(
(in BidsData bid) => Console.WriteLine($"Bid: {bid.Price}"),
(in AsksData ask) => Console.WriteLine($"Ask: {ask.Price}")
);

// v1.5.0: foreach for simple top-level groups — no closure alloc, supports break.
foreach (ref readonly var bid in reader.Bids) Console.WriteLine($"Bid: {bid.Price}");
foreach (ref readonly var ask in reader.Asks) Console.WriteLine($"Ask: {ask.Price}");

// ReadGroups still works (and is required for nested groups / group-level varData):
// reader.ReadGroups(
// (in BidsData bid) => Console.WriteLine($"Bid: {bid.Price}"),
// (in AsksData ask) => Console.WriteLine($"Ask: {ask.Price}")
// );
}
```

Expand Down
52 changes: 44 additions & 8 deletions docs/PERFORMANCE_TUNING_GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -246,10 +246,22 @@ ProcessAsks(asks);

### Streaming Group Processing

Process repeating groups without materializing collections:
For **simple top-level groups** (no nested groups, no group-level varData), prefer the v1.5.0 `foreach` enumerator API — it's allocation-free **and** avoids the closure capture problem entirely:

```csharp
// ✅ Good: Streaming processing
// ✅ Best (v1.5.0+): foreach — no closure allocation, supports break, ref readonly access
long totalVolume = 0;
if (MarketDataData.TryParse(buffer, out var reader))
{
foreach (ref readonly var bid in reader.Bids) totalVolume += bid.Quantity;
foreach (ref readonly var ask in reader.Asks) totalVolume += ask.Quantity;
}
```

For messages with **nested groups or group-level varData**, the foreach API is not generated — fall back to `ReadGroups`:

```csharp
// ✅ Good: Streaming processing via callbacks (works for any message)
long totalVolume = 0;
if (MarketDataData.TryParse(buffer, out var reader))
{
Expand All @@ -274,7 +286,22 @@ long totalVolume = allBids.Sum(b => b.Quantity) + allAsks.Sum(a => a.Quantity);

### Early Exit

Note: The `ReadGroups` callbacks use `void`-returning `in` delegates, so early exit must use external state:
For **simple top-level groups** (v1.5.0+), `foreach` supports natural `break` — independent per-group enumerators mean breaking out of one group does NOT corrupt later groups:

```csharp
// ✅ Best: foreach with break — zero alloc, intuitive
if (MarketDataData.TryParse(buffer, out var reader))
{
foreach (ref readonly var bid in reader.Bids)
{
if (bid.Quantity > 1_000_000) break; // Asks group remains intact
Process(in bid);
}
foreach (ref readonly var ask in reader.Asks) Process(in ask);
}
```

For messages requiring `ReadGroups`: callbacks use `void`-returning `in` delegates, so early exit must use external state:

```csharp
// ✅ Good: Early exit via external flag
Expand Down Expand Up @@ -434,20 +461,29 @@ void ProcessTrade(in Trade trade)

### 5. Closure Allocations

**The biggest source of accidental allocations on the read path.** Lambdas passed to `ReadGroups` that capture locals from the enclosing scope force the compiler to emit a heap-allocated display class — one allocation per `ReadGroups` call.

```csharp
// ❌ Bad: Closure allocation
// ❌ Bad: Closure allocation (display class) per call
int threshold = 100;
reader.ReadGroups(
(in BidData bid) => { _ = bid.Quantity > threshold; }, // Closure - allocation
(in BidData bid) => { _ = bid.Quantity > threshold; },
(in AskData ask) => { }
);

// ✅ Good: Use local function or avoid capture
bool ProcessBid(BidData bid, int threshold)
// ✅ Best (v1.5.0+, simple groups): foreach has no closure to capture
int threshold = 100;
foreach (ref readonly var bid in reader.Bids)
{
_ = bid.Quantity > threshold; // closes over local naturally, no alloc
}

// ✅ Good fallback (complex groups): use a local function or non-capturing lambda
static bool ProcessBid(in BidData bid, int threshold)
=> bid.Quantity > threshold;

reader.ReadGroups(
(in BidData bid) => ProcessBid(bid, 100),
(in BidData bid) => ProcessBid(in bid, 100),
(in AskData ask) => { }
);
```
Expand Down
15 changes: 13 additions & 2 deletions docs/SPAN_READER_README.md
Original file line number Diff line number Diff line change
Expand Up @@ -378,9 +378,20 @@ if (reader.TryRead<MessageHeader>(out var header))

This pattern is ideal for reading multiple messages from a single buffer, where each message's total size (block + groups + varData) is tracked by `BytesConsumed`.

### Callback Pattern (v1.0.0+)
### Callback Pattern (v1.0.0+) and Foreach Pattern (v1.5.0+)

Generated code uses custom delegates with `in` parameters to avoid struct copies:
For **simple top-level groups**, prefer the `foreach` enumerator (zero closure allocation, supports `break`):

```csharp
// Via MessageDataReader (v1.5.0+):
if (OrderBookData.TryParse(buffer, out var reader))
{
foreach (ref readonly var bid in reader.Bids) Console.WriteLine($"Bid: {bid.Price}");
foreach (ref readonly var ask in reader.Asks) Console.WriteLine($"Ask: {ask.Price}");
}
```

For messages with **nested groups or group-level varData**, use callbacks (only API generated for those):

```csharp
// Generated delegates:
Expand Down
Loading