feat(python): MPP client-only sessions (session intent + payment-channels glue)#161
feat(python): MPP client-only sessions (session intent + payment-channels glue)#161EfeDurmaz16 wants to merge 15 commits into
Conversation
Greptile SummaryThis PR adds the MPP session intent for the Python SDK: off-chain voucher signing over payment channels, a
Confidence Score: 4/5Core session logic and fund-safety invariants are sound; one structural gap in the topUp builder needs to be closed before non-production deployments work correctly.
python/src/pay_kit/protocols/mpp/_paymentchannels.py — specifically the Important Files Changed
Sequence DiagramsequenceDiagram
participant Client
participant ActiveSession
participant _paymentchannels
participant Server
participant OnChain as On-chain Program
Client->>_paymentchannels: build_open_instruction(OpenChannelParams)
_paymentchannels-->>Client: open Instruction (program_id from params)
Client->>OnChain: broadcast open tx (partial-signed, operator completes)
OnChain-->>Client: channel confirmed
Client->>ActiveSession: new(channel_id, signer)
Client->>ActiveSession: open_payment_channel_action(deposit, ...)
ActiveSession-->>Client: "SessionAction{open}"
Client->>Server: POST Authorization: Payment credential
Server-->>Client: 200 OK (session open)
loop Per metered request
Client->>ActiveSession: prepare_increment(amount)
ActiveSession-->>Client: SignedVoucher (watermark NOT advanced)
Client->>Server: commit(CommitPayload)
Server-->>Client: "CommitReceipt {status: committed|replayed}"
Client->>ActiveSession: record_voucher(voucher)
Note over ActiveSession: Watermark advanced only after server ACK
end
opt Top-up needed
Client->>_paymentchannels: build_top_up_instruction(TopUpParams)
Note over _paymentchannels: hardcodes PROGRAM_ID, ignores channel program
_paymentchannels-->>Client: topUp Instruction
Client->>OnChain: broadcast topUp tx
Client->>ActiveSession: top_up_action(new_deposit, sig)
Client->>Server: POST topUp action
end
Client->>ActiveSession: close_action(final_increment)
Client->>Server: POST close action
Reviews (10): Last reviewed commit: "style(python): drop em-dashes from sessi..." | Re-trigger Greptile |
| if "cumulativeAmount" in data: | ||
| cumulative = data["cumulativeAmount"] | ||
| elif "cumulative" in data: | ||
| cumulative = data["cumulative"] | ||
| else: | ||
| cumulative = "" |
There was a problem hiding this comment.
When
VoucherData.from_dict receives a cumulativeAmount that is a JSON number rather than a string (a non-conforming but plausible server response), self.cumulative ends up as an int. Both message_bytes() (int(self.cumulative, 10)) and record_voucher (int(voucher.data.cumulative, 10)) then raise TypeError: int() can't convert non-string with explicit base instead of the expected ValueError, breaking the caller's error-handling path. Coercing to str in from_dict is the safe fix.
| if "cumulativeAmount" in data: | |
| cumulative = data["cumulativeAmount"] | |
| elif "cumulative" in data: | |
| cumulative = data["cumulative"] | |
| else: | |
| cumulative = "" | |
| if "cumulativeAmount" in data: | |
| cumulative = str(data["cumulativeAmount"]) | |
| elif "cumulative" in data: | |
| cumulative = str(data["cumulative"]) | |
| else: | |
| cumulative = "" |
| data = bytearray() | ||
| data.append(_OPEN_DISCRIMINATOR) | ||
| data += struct.pack("<Q", params.salt) | ||
| data += struct.pack("<Q", params.deposit) | ||
| data += struct.pack("<I", params.grace_period) |
There was a problem hiding this comment.
struct.pack("<I", params.grace_period) silently raises struct.error when grace_period exceeds 2**32 - 1 rather than a descriptive ValueError. Since grace_period is typed as int (unbounded in Python) and callers may compute it from seconds, a clear range guard here prevents a confusing low-level exception. The same gap exists for bps (<H, max 65 535) in the recipients loop.
| data = bytearray() | |
| data.append(_OPEN_DISCRIMINATOR) | |
| data += struct.pack("<Q", params.salt) | |
| data += struct.pack("<Q", params.deposit) | |
| data += struct.pack("<I", params.grace_period) | |
| if not (0 <= params.grace_period <= 0xFFFF_FFFF): | |
| raise ValueError(f"grace_period {params.grace_period} exceeds u32 range") | |
| data = bytearray() | |
| data.append(_OPEN_DISCRIMINATOR) | |
| data += struct.pack("<Q", params.salt) | |
| data += struct.pack("<Q", params.deposit) | |
| data += struct.pack("<I", params.grace_period) |
| from pay_kit.protocols.mpp.client.session_consumer import ( | ||
| MeteredDelivery, | ||
| SessionConsumer, | ||
| ) |
There was a problem hiding this comment.
CommitTransport is listed in session_consumer.__all__ and is the protocol users must implement to drive SessionConsumer, but it is not re-exported here. Any caller who imports only from the top-level client package will need to discover the sub-module path by reading source, which is at odds with the re-export pattern the rest of this __init__ establishes.
| from pay_kit.protocols.mpp.client.session_consumer import ( | |
| MeteredDelivery, | |
| SessionConsumer, | |
| ) | |
| from pay_kit.protocols.mpp.client.session_consumer import ( | |
| CommitTransport, | |
| MeteredDelivery, | |
| SessionConsumer, | |
| ) |
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
| __all__ = [ | ||
| "PaymentTransport", | ||
| "ActiveSession", | ||
| "SessionConsumer", | ||
| "MeteredDelivery", | ||
| "serialize_session_credential", | ||
| "parse_session_challenge", | ||
| ] |
There was a problem hiding this comment.
The
__all__ list omits CommitTransport after the import above is added, so from pay_kit.protocols.mpp.client import * still would not expose it to downstream consumers.
| __all__ = [ | |
| "PaymentTransport", | |
| "ActiveSession", | |
| "SessionConsumer", | |
| "MeteredDelivery", | |
| "serialize_session_credential", | |
| "parse_session_challenge", | |
| ] | |
| __all__ = [ | |
| "PaymentTransport", | |
| "ActiveSession", | |
| "CommitTransport", | |
| "SessionConsumer", | |
| "MeteredDelivery", | |
| "serialize_session_credential", | |
| "parse_session_challenge", | |
| ] |
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
| @classmethod | ||
| def from_dict(cls, data: dict[str, Any]) -> MeteringDirective: | ||
| return cls( | ||
| delivery_id=data.get("deliveryId", ""), | ||
| session_id=data.get("sessionId", ""), | ||
| amount=data.get("amount", ""), | ||
| currency=data.get("currency", ""), | ||
| sequence=int(data.get("sequence", 0)), | ||
| expires_at=int(data.get("expiresAt", 0)), | ||
| commit_url=data.get("commitUrl"), | ||
| proof=data.get("proof"), | ||
| ) |
There was a problem hiding this comment.
MeteringDirective.from_dict stores amount as whatever JSON hands it — typically an int when the server does not quote the field. commit_directive then calls directive.amount_base_units(), which passes self.amount directly to int(self.amount, 10). Python's int() with an explicit base rejects a plain int argument with TypeError: int() can't convert non-string with explicit base, rather than the ValueError the call-site catch-all expects. The fix is the same coercion already recommended for VoucherData.cumulative: normalize to str at deserialization time.
| @classmethod | |
| def from_dict(cls, data: dict[str, Any]) -> MeteringDirective: | |
| return cls( | |
| delivery_id=data.get("deliveryId", ""), | |
| session_id=data.get("sessionId", ""), | |
| amount=data.get("amount", ""), | |
| currency=data.get("currency", ""), | |
| sequence=int(data.get("sequence", 0)), | |
| expires_at=int(data.get("expiresAt", 0)), | |
| commit_url=data.get("commitUrl"), | |
| proof=data.get("proof"), | |
| ) | |
| @classmethod | |
| def from_dict(cls, data: dict[str, Any]) -> MeteringDirective: | |
| return cls( | |
| delivery_id=data.get("deliveryId", ""), | |
| session_id=data.get("sessionId", ""), | |
| amount=str(data.get("amount", "")), | |
| currency=data.get("currency", ""), | |
| sequence=int(data.get("sequence", 0)), | |
| expires_at=int(data.get("expiresAt", 0)), | |
| commit_url=data.get("commitUrl"), | |
| proof=data.get("proof"), | |
| ) |
…foreign vouchers On a replayed CommitReceipt the server has already settled the delivery, so its cumulative is authoritative. commit_directive now reconciles the local watermark to that cumulative (advancing when behind, e.g. a lost response, and never regressing) instead of recording the freshly prepared higher voucher, which would let a later close sign for more than was settled. record_voucher also rejects a voucher whose channel does not match the active session, and VoucherData.from_dict coerces a numeric cumulativeAmount to str. Surfaced by Greptile and Codex on solana-foundation#161. Mirrors the rust spine fix (solana-foundation#162).
3c94f18 to
f122be5
Compare
…reign vouchers On a replayed CommitReceipt the server has already settled the delivery, so its cumulative is authoritative. SessionConsumer::commit_directive now reconciles the local watermark to that cumulative (advancing when behind, e.g. a lost response, and never regressing) instead of recording the freshly prepared higher voucher, which would let a later channel close sign for more than was settled. ActiveSession::record_voucher also rejects a voucher whose channel does not match the active session. Adds reconcile_settled plus regression tests for reconcile, no-regress, and the foreign-channel guard. Surfaced by Greptile/Codex on the Go and Python session ports (solana-foundation#160, solana-foundation#161).
| def from_dict(cls, data: dict[str, Any]) -> MeteringUsage: | ||
| return cls(delivery_id=data.get("deliveryId", ""), amount=data.get("amount", "")) |
There was a problem hiding this comment.
amount coercion gap in MeteringUsage and CommitReceipt
MeteringUsage.from_dict stores amount as-is from the parsed JSON dictionary — a conforming but number-typed server response sets it to an int. amount_base_units() then calls int(self.amount, 10), which raises TypeError: int() can't convert non-string with explicit base instead of the expected ValueError. The same gap exists in CommitReceipt.from_dict for both amount (line 878) and cumulative (line 879): both amount_base_units() and cumulative_base_units() fail with TypeError when the field arrives as a JSON number. The coercion fix applied to VoucherData.cumulative (lines 1518–1519) should be replicated here.
Addresses Greptile + Codex review of solana-foundation#161: - Reconcile the local watermark to a replayed receipt's cumulative (advance when behind, e.g. a lost response; never regress) instead of recording the freshly prepared higher voucher, which could let a later close sign for more than was settled. - record_voucher rejects a voucher whose channel does not match the session; VoucherData.from_dict coerces a numeric cumulativeAmount to str. - commit_directive records only on an explicit committed receipt and rejects unknown statuses. - Base-unit accessors (deposit_amount, amount_base_units, voucher cumulative) parse strict unsigned u64 decimals, rejecting negative/fractional/over-range values like the rust/Go typed parsers. Mirrors the rust spine fix (solana-foundation#162).
f122be5 to
a59d725
Compare
Port the MPP client-only session intent to the Python SDK, mirroring the Rust spine and the Go port: - intents/session.py: SessionRequest, the SessionAction tagged union (open/voucher/commit/topUp/close), OpenPayload (push + pull), signed vouchers, and the metering types, in the dataclass to_dict/from_dict house style. salt serialises as a decimal string and decodes from string or number; cumulativeAmount accepts the cumulative alias. - client/session.py: ActiveSession (voucher signing, monotonic watermark, action builders) plus serialize_session_credential / parse_session_challenge. - client/session_consumer.py: SessionConsumer + CommitTransport for metered ack/commit, with prepare/record separation so a failed commit never double counts. - _paymentchannels.py: hand-rolled on-chain glue over solders (channel and event-authority PDAs, the 48-byte voucher preimage, and the open/topUp instructions). Pins the production program id over the IDL placeholder; single-byte field discriminators (open=1, topUp=3). ruff, pyright, and pytest (coverage 93.88%, gate 90) all green.
Addresses Greptile + Codex review of solana-foundation#161: - Reconcile the local watermark to a replayed receipt's cumulative (advance when behind, e.g. a lost response; never regress) instead of recording the freshly prepared higher voucher, which could let a later close sign for more than was settled. - record_voucher rejects a voucher whose channel does not match the session; VoucherData.from_dict coerces a numeric cumulativeAmount to str. - commit_directive records only on an explicit committed receipt and rejects unknown statuses. - Base-unit accessors (deposit_amount, amount_base_units, voucher cumulative) parse strict unsigned u64 decimals, rejecting negative/fractional/over-range values like the rust/Go typed parsers. Mirrors the rust spine fix (solana-foundation#162).
…d cumulative Greptile solana-foundation#162 follow-up: reconcile_settled advanced cumulative but left the nonce unchanged, so the first delivery after a lost-response replay reused the nonce the server already settled. Bump the nonce by one whenever reconcile advances (mirroring record_voucher's accounting for that delivery). The nonce is client request-counter metadata (not in the signed 48-byte preimage), so this is a consistency fix, not a fund-safety one. Adds a delivery-after-replay regression test.
…yright-clean ruff format on session.py/intents/session.py (line-length normalization, no logic change) and type the consumer test helper against the CommitTransport protocol, renaming a fake transport's param to match the protocol so a full pyright run is clean (CI runs pyright over the whole tree).
Review follow-ups on the session client: - commit_directive clamps a replayed receipt's cumulative to the voucher just prepared in the call. The server is untrusted; without the clamp it could report a replay settled above what the client signed and push the watermark up, so the next voucher over-authorizes (capped by the deposit). An honest lost-response replay settles at or below the prepared voucher, so recovery is unchanged. Adds a regression test. - record_voucher and the replayed-receipt path now parse cumulative via the strict _parse_base_units / cumulative_base_units (rejecting negative, fractional, whitespace, underscore, and over-u64) instead of bare int(); an over-u64 server value previously wedged the session on the next pack. - _salt_from_wire enforces the u64 range like the rust deserializer.
… packer The wire type hand-rolled the 48-byte voucher preimage, duplicating _paymentchannels.voucher_message_bytes. The rust spine delegates rather than duplicating, and the hand-rolled copy was exercised only by its own tests, so it could drift from the packer the signing path actually uses. Delegate to the canonical packer for a single source of truth (no import cycle: the glue does not import the intent layer). Drops the now-unused struct import. No behavior change.
Renders python/src/pay_kit/protocols/programs/paymentchannels/ from the vendored idl/payment-channels.json with the community codama-py renderer (Solana-ZH/codama-py), mirroring the Go port's codama-generated client. The renderer is pinned to the merge commit of Solana-ZH/codama-py#10, which fixed PDA seed rendering; it cannot be consumed as an npm/git dependency yet (its package ships only an uncommitted dist), so the codegen script clones the pinned commit and drives the upstream genpy CLI. Generated code is exempt from ruff/pyright/coverage. just payment-channels-generate-py regenerates; payment-channels-sync now refreshes Rust and Python together.
build_open_instruction / build_top_up_instruction now map their params onto the generated Open / TopUp builders (production program id passed explicitly), and voucher_message_bytes encodes through the generated VoucherArgs Borsh layout, matching the Rust spine's delegation to its generated client. PDA and ATA derivations stay in the glue: the channel PDA is not declared in the IDL, and the generated event-authority helper pins the IDL placeholder program id. Adds the anchorpy + borsh-construct runtime deps the generated client imports, and disables anchorpy's pytest plugin (it imports pytest-xprocess at collection time; only the Borsh helpers are used). Frozen instruction vectors are unchanged: the generated path emits byte-identical data and account metas for open and topUp.
Replayed commit receipts now record the prepared voucher unconditionally, matching rust SessionConsumer::commit_directive and the TS SessionConsumer line by line; the reconcile_settled extension API is removed. record_voucher also adopts the rust nonce rule max(nonce, voucher.nonce) instead of always advancing past stale voucher nonces.
Mirror rust serde enum decoding: SessionRequest.from_dict and OpenPayload.from_dict reject unknown mode and pullVoucherStrategy values, and CommitReceipt.from_dict rejects a missing or unknown status instead of defaulting to committed.
Port rust client/payment_channels.rs: derive_payment_channel_open resolves mint, deposit (defaults to cap), grace period (default 900s), program id, token program from the challenge currency (Token-2022 for PYUSD/USDG/CASH), splits, and a random u64 salt; build_open_payment_channel_transaction assembles the legacy open transaction with fee payer = challenge operator and only the payer slot signed; the pull/clientVoucher session openers default the action signature to PENDING_SERVER_SIGNATURE. Adds session_request_modes (modes empty or omitted means push-only), a generate_authorized_signer helper for the ephemeral session key, an ActiveSession cumulative resume option, and program-id plumbing through the instruction builders.
Port rust client/http_stream.rs: an incremental SseDecoder, metered event classification (mpp.metering/mpp.usage/done/[DONE]), the MeteredSseSession state machine enforcing that a usage event's deliveryId matches the live directive and that usage overrides only the amount, an httpx-backed HttpCommitTransport, and a transport-neutral MeteredSseStream over raw byte chunks.
The branch added the two dependencies for the generated payment-channels client without refreshing the lockfile; main also added pydoc-markdown. Refreshed with uv lock.
State the shipped client surface and the server-side, operatedVoucher, and session-transport follow-ups instead of leaving the row empty.
1910fe8 to
d1ded63
Compare
…reign vouchers On a replayed CommitReceipt the server has already settled the delivery, so its cumulative is authoritative. SessionConsumer::commit_directive now reconciles the local watermark to that cumulative (advancing when behind, e.g. a lost response, and never regressing) instead of recording the freshly prepared higher voucher, which would let a later channel close sign for more than was settled. ActiveSession::record_voucher also rejects a voucher whose channel does not match the active session. Adds reconcile_settled plus regression tests for reconcile, no-regress, and the foreign-channel guard. Surfaced by Greptile/Codex on the Go and Python session ports (solana-foundation#160, solana-foundation#161).
feat(python): mpp/session client intent
Client-side
mpp/sessionsupport for the Python SDK, rebased onto main after #165 and aligned with the rewrittenskills/pay-sdk-implementation/references/intents/mpp-session.md. The Rust crate (rust/crates/mpp/src) is the wire truth throughout; the TypeScript implementation merged in #165 is the second reference. Server-side session verification is a deliberate follow-up, see Scope below.What this adds
Wire and program layer:
SessionRequest, the taggedSessionActionunion,OpenPayload,VoucherData/SignedVoucher,CommitPayload/CommitReceipt,TopUpPayload,ClosePayload,MeteringDirective,MeteringUsage,MeteredEnvelope) inpython/src/pay_kit/protocols/mpp/intents/session.py, mirroringrust/crates/mpp/src/protocol/intents/session.rsfield for field: salt as a u64 string accepting string or number on decode, thecumulativedecode alias withcumulativeAmountthe only serialized name,session_id()keyed channelId first with tokenAccount fallback,topUp.newDepositas the new total, andDEFAULT_SESSION_EXPIRES_AT = 4_102_444_800. Unknown modes, pull voucher strategies, and commit receipt statuses are rejected at decode, matching Rust serde enum behavior.python/src/pay_kit/protocols/programs/paymentchannels/(recipepayment-channels-generate-py, generator inskills/pay-sdk-implementation/codegen/), with the production program idGuoKrza...overriding the IDL placeholder and the single-byte discriminators (open = 1, topUp = 3). The 48-byte voucher preimage (channel_id || cumulative u64 LE || expires_at i64 LE) has a single packer delegating to the generatedVoucherArgslayout, verified against a frozen cross-language vector.Client layer, keyed to the skill's component inventory:
parse_session_challengeplussession_request_modes, encoding modes empty or omitted as push-only (TSsessionRequestModessemantics, Rust membership checks).generate_authorized_signer()covers canonical-flow step 2 (TS openers callgenerateKeyPairSigner).client/payment_channels.py, a line-by-line port ofrust/crates/mpp/src/client/payment_channels.rs:derive_payment_channel_open(mint from the challenge currency with the localnet to mainnet fallback, deposit defaulting to the cap, grace defaulting to 900 seconds, token program resolved from the currency so PYUSD/USDG/CASH open as Token-2022, splits, random u64 salt),build_open_payment_channel_transaction(fee payer = challenge operator, payer partial-signs only its own slot, challengerecentBlockhash, standard base64 with padding), and the pull/clientVoucher openers with thePENDING_SERVER_SIGNATUREplaceholder (64 ones).ActiveSessionsigns vouchers without advancing the watermark and records them only after server acceptance, with strict monotonicity, u64 overflow guards, and the Rust nonce rulemax(nonce, voucher.nonce).SessionConsumervalidates the directive session, prepares the voucher forwatermark + amount, commits with the directive'sdeliveryId, and records the prepared voucher after the transport returns for bothcommittedandreplayedreceipts, exactly matchingrust/crates/mpp/src/client/session_consumer.rsand the TSSessionConsumer.client/http_stream.py, portingrust/crates/mpp/src/client/http_stream.rs: incrementalSseDecoder, metered event classification (mpp.metering/metering,mpp.usage/usage,done,[DONE]), theMeteredSseSessionstate machine enforcing that a usage event'sdeliveryIdmatches the live directive and that usage overrides only the amount, an httpx-backedHttpCommitTransport, and a transport-neutralMeteredSseStreamover raw byte chunks.Encoding boundaries follow the repo rule: canonical JSON (RFC 8785) to base64url without padding for the credential envelope and request field, standard base64 with padding for transactions, base58 for signatures, pubkeys, and blockhashes.
Scope and follow-ups
This PR is intentionally client-only. Deliberate follow-ups:
initMultiDelegateTx,updateDelegationTx) but no builder constructs them.PaymentTransportremains charge-only, so the per-channel watermark reset and no-latched-failure retry rules become obligations of that follow-up.harness/src/contracts.ts(none exists for any language yet, per the skill) and a session example underpython/examples/.Operability caveats: items 2 through 6 of the numbered list are server-side or Rust-client-only and are not applicable to a client-only intent; item 1 (localnet resolves to the mainnet mint) is implemented in the open derivation, and item 7 is covered by the gates below.
Notes for review
replayedreceipts. That was safer but diverged from the Rust spine, so it has been removed in favor of exact parity: replayed commits record the prepared voucher, like both references. The safety property moved to the decode layer, which now rejects malformed receipt statuses outright.anchorpyis pulled in for its Borsh helpers only (the generated client's layout codecs); its pytest plugin is disabled inpyproject.toml.uv.lockis regenerated for the new dependencies.Testing
pytest: 964 passed (includestests/test_session_intent.py,test_session_client.py,test_session_consumer.py,test_session_payment_channels.py,test_http_stream.py,test_paymentchannels.py).ruff checkandpyrightclean (all extras installed, mirroring.github/workflows/python.yml).