Skip to content

implement CEP-22 oversized payload transfer #87

@harsh04044

Description

@harsh04044

Summary

CEP-22 defines a bounded reassembly profile for JSON-RPC payloads too large for a single relay event (~64 KB). Large messages are split into ordered frames carried inside MCP notifications/progress notifications, transmitted as ordinary kind 25910 events, and reassembled after SHA-256 + size validation before being surfaced to the MCP layer.

No new Nostr kinds. Depends on CEP-6 (done). The rmcp fork (progress-aware-request-timeouts) enables opt-in per-chunk idle timeout reset for the S→C response direction - not automatic, requires send_cancellable_request with explicit options (PR 4).

Spec: contextvm-docs/src/content/docs/spec/ceps/cep-22.md
TS SDK reference: sdk/src/transport/oversized-transfer/


PR 1: Core framing engine (transport-agnostic) (#88 )

  • Add sha2 = "0.10" and hex = "0.4" to Cargo.toml
  • frame.rs - OversizedFrame enum (Start/Accept/Chunk/End/Abort) + serde
  • constants.rs - chunk size, threshold, max bytes, timeouts, progress slots
  • errors.rs - OversizedTransferError variants (Abort/Policy/Digest/Reassembly/Sequence)
  • codec.rs - build_oversized_frames(), SHA-256 digest, UTF-8 char-aware byte split
  • receiver.rs - stateful OversizedTransferReceiver (admission control, out-of-order buffering, triple validation at end)
  • validate_and_parse_oversized() - bypass 1 MB cap for reassembled payloads, bound to maxTransferBytes (100 MiB default)
  • Wire OversizedTransferError into src/core/error.rs
  • Declare module in src/transport/mod.rs
  • Unit tests: roundtrip, out-of-order, digest mismatch, byte-length mismatch, duplicate start, chunk count mismatch, abort, over-limit rejection, multibyte split correctness

PR 2: Config + capability advertisement + server gate

  • OversizedTransferConfig struct with builder, attached to both transport configs
  • Server advertise support_oversized_transfer tag via AnnouncementManager::set_internal_common_tags when enabled
  • Replace hardcoded oversized_enabled = false (server/mod.rs:1158) with config flag
  • Client advertise support_oversized_transfer in get_client_capability_tags when enabled
  • Tests: tag emitted iff enabled, server learns client flag, announcement includes tag, defaults disabled

PR 3: Transport integration (outbound split + inbound reassembly + accept)

  • Client send -- threshold check, oversized branch, accept handshake, register end-frame event id in correlation store
  • Client inbound -- intercept cvm frames before 1 MB cap, feed receiver, guard pending cleanup, tx.send on end, use validate_and_parse_oversized
  • Server send_response -- threshold check, split into start/chunk/end frames (no accept wait server-side); serialize-for-digest must run after original request id is restored (server/mod.rs:517-522) or SHA-256 check fails on client
  • Server inbound -- intercept cvm frames, reassemble, emit accept on unknown-support start, dispatch synthetic IncomingRequest on end
  • One OversizedTransferReceiver per transport, .clear() on close()
  • Tests: oversized request roundtrip, oversized response roundtrip, accept handshake known vs unknown, out-of-order delivery, digest mismatch fail, abort, per-frame < 1 MB, reassembled > 1 MB succeeds

PR 4: Progress-aware timeout + e2e + docs + enable default

  • Consumer-facing helper to issue oversized-aware requests via send_cancellable_request with reset_timeout_on_progress + max_total_timeout
  • Verify ProgressNotificationParam tolerates forwarded stripped progress notifications (unknown fields); if not, rely on receiver watchdog only
  • Forward stripped progress notifications to rmcp per chunk to drive timeout reset (gated on above verification)
  • Flip default to enabled = true (matches TS default)
  • E2e tests over MockRelayPool covering all encryption modes (disabled/optional/required)
  • Conformance test asserting wire format matches spec
  • Docs page + README CEP matrix CEP-22 → Done

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type
No fields configured for issues without a type.

Projects

Status
No status

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions