Skip to content

feat(webhooks): VerifyAndParse* API for compressed payloads (CHA-3071)#210

Merged
nijeesh-stream merged 13 commits into
masterfrom
nijeeshjoshy/cha-3071-compress-webhook-payloads
May 12, 2026
Merged

feat(webhooks): VerifyAndParse* API for compressed payloads (CHA-3071)#210
nijeesh-stream merged 13 commits into
masterfrom
nijeeshjoshy/cha-3071-compress-webhook-payloads

Conversation

@nijeesh-stream
Copy link
Copy Markdown
Contributor

@nijeesh-stream nijeesh-stream commented May 7, 2026

Summary

Adds first-class support for gzip-compressed webhook payloads (HTTP webhooks, SQS, SNS) and exposes a stable VerifyAndParse* API that mirrors the cross-SDK contract published in Webhooks Overview.

New public API (StreamChat.Clients.WebhookHelpers)

WebhookHelpers is now public so apps can compose the primitives directly:

  • GunzipPayload(byte[]) — gzip-magic-byte detection, no-op when not compressed
  • DecodeSqsPayload(string) — base64 decode then gunzip-if-magic
  • DecodeSnsPayload(notificationBody) — JSON-parse the SNS HTTP notification envelope, extract the inner Message, then run the SQS pipeline. Falls through to a pre-extracted Message string when the input is not a JSON envelope
  • VerifySignature(byte[], string, string) — HMAC-SHA256 over the uncompressed body, with a constant-time comparison (matters for the HTTP webhook path where the X-Signature header is exposed publicly; SQS / SNS deliveries arrive over AWS-internal transports where timing-attack resistance is not strictly required)
  • ParseEvent(byte[]) — JSON → typed EventResponse via Newtonsoft.Json

Composites on IAppClient (return a typed EventResponse):

  • VerifyAndParseWebhook(byte[], string)
  • VerifyAndParseSqs(string, string)
  • VerifyAndParseSns(string, string)

Backwards compatibility

AppClient.VerifyWebhook is preserved and now uses the same constant-time HMAC-SHA256 path. The legacy VerifyAndDecodeWebhook surface is removed (it was experimental and not yet released).

Unified error handling

Per cross-SDK coordination (mogita's review across the 6 sibling SDK PRs), every webhook failure path now terminates at a single exception class — StreamInvalidWebhookException (extends StreamBaseException) so customers only need one catch arm. The message text identifies which failure mode fired so callers that want to differentiate (security logging, retry policy) can filter on substring:

Failure mode Message
Signature mismatch signature mismatch
Base64 decode invalid base64 encoding
Gzip decompression gzip decompression failed
JSON parse invalid JSON payload

public const string failure-mode constants on the class (StreamInvalidWebhookException.SignatureMismatch etc.) are available for exact-match filtering. The legacy AppClient.VerifyWebhook (bool return) is untouched.

Tests

tests/WebhookCompressionTests.cs covers plain / gzip / base64 / base64+gzip payloads, signature mismatches, malformed bytes, and JSON parsing into EventResponse. Linked Linear ticket: CHA-3071.

Golden test fixtures (Tommaso)

Added shared reference fixtures to the test suite so future SDKs can sanity-check decoders against the same payloads:

aGVsbG93b3JsZA==                          -> helloworld   (base64)
H4sIAGrYAWoAA8tIzcnJL88vykkBAK0g6/kKAAAA -> helloworld   (base64 + gzip)

Test plan

  • dotnet build — clean
  • dotnet test --filter ~WebhookCompressionTests — 22 passed

nijeesh-stream and others added 2 commits May 7, 2026 12:22
…HA-3071)

Stream now supports gzip compression on outbound webhooks (and the same
encoding wrappers apply to SQS / SNS firehose payloads, plus a base64
layer to keep them transport-safe). The existing string-based
VerifyWebhook helper is kept untouched for backward compatibility, but
it cannot operate on the raw byte stream and does not understand the
layered encodings.

Two new helpers on IAppClient cover the new contract:

  - DecompressWebhookBody(body, contentEncoding, payloadEncoding) reverses
    the encoding wrappers (base64 first, then gzip) and returns the raw
    JSON bytes.
  - VerifyAndDecodeWebhook(body, signature, contentEncoding,
    payloadEncoding) does the same and additionally verifies the
    HMAC-SHA256 signature against the decoded JSON, throwing
    StreamWebhookSignatureException on mismatch.

The signature is always computed by the server over the innermost
(uncompressed, base64-decoded) JSON, so the same verification rule
applies to plain HTTP webhooks and SQS / SNS firehose. Comparison uses
CryptographicOperations.FixedTimeEquals on supported targets and a
constant-time fallback on .NET Framework / netstandard <= 2.0.

Unsupported encodings (br/zstd/deflate/... for Content-Encoding,
hex/url/binary for payload_encoding) throw InvalidOperationException
with a message pointing at the supported value, so misconfigured app
settings fail loud rather than silently.

Co-authored-by: Cursor <cursoragent@cursor.com>
Replace the experimental VerifyAndDecodeWebhook surface with the new
VerifyAndParse{Webhook,Sqs,Sns} family that mirrors the cross-SDK
contract:

- WebhookHelpers becomes public and exposes the primitive helpers
  UngzipPayload, DecodeSqsPayload, DecodeSnsPayload, VerifySignature,
  ParseEvent.
- VerifyAndParse* compose those primitives, transparently handle gzip
  via magic-byte detection (and base64+gzip for SQS/SNS), and return a
  typed EventResponse.
- VerifyWebhook is preserved for backwards compatibility and now uses
  the same constant-time HMAC-SHA256 path.
- Tests cover plain / gzip / base64 / base64+gzip payloads, signature
  mismatches, malformed bodies, and parsing into EventResponse.

Co-authored-by: Cursor <cursoragent@cursor.com>
@nijeesh-stream nijeesh-stream changed the title feat(webhooks): add VerifyAndDecodeWebhook for compressed payloads feat(webhooks): VerifyAndParse* API for compressed payloads (CHA-3071) May 8, 2026
nijeesh-stream and others added 4 commits May 8, 2026 16:53
RFC 1952 defines the gzip magic number as the two-byte sequence
1F 8B; the third byte (CM) is informational and not part of the
identifier. Trim the magic check from three bytes to two to match
the spec and stay consistent with the reference implementations
in the public docs.

Co-authored-by: Cursor <cursoragent@cursor.com>
Surface the dual-API pattern at the level callers actually hold a
reference to. IStreamClientFactory / StreamClientFactory now ship
VerifyAndParseWebhook, VerifyAndParseSqs, and VerifyAndParseSns
that delegate to the existing IAppClient methods using the
factory's configured API secret. Mirrors the dual-API surface in
stream-chat-go and matches the call shape documented at
getstream.io/chat/docs/dotnet-csharp/webhooks_overview/.

The static WebhookHelpers and the IAppClient instance methods
remain unchanged for callers that still want to drill through
GetAppClient() or use the stateless helpers in workers.

Co-authored-by: Cursor <cursoragent@cursor.com>
The compressed-webhook section referenced a `VerifyAndDecodeWebhook` /
`DecompressWebhookBody` surface with `contentEncoding` and
`payloadEncoding` parameters that does not exist in the SDK. The shipped
API is `VerifyAndParseWebhook` / `VerifyAndParseSqs` / `VerifyAndParseSns`
on `IAppClient`, `IStreamClientFactory`, and the static `WebhookHelpers`,
returning a parsed `EventResponse`. Update the prose and both code
samples so copy-pasted snippets compile against the actual public API.

Co-authored-by: Cursor <cursoragent@cursor.com>
The cross-SDK contract makes `WebhookHelpers` and
`StreamWebhookSignatureException` part of the public surface, but every
method on the helper class shipped without XML docs and the exception
summary only mentioned signature mismatches even though it is also
raised for malformed gzip / base64 / JSON envelopes. Add IntelliSense
docs for each public helper (inputs, error cases, the constant-time
guarantee on signature verification) and broaden the exception summary
to match its actual throw sites.

Co-authored-by: Cursor <cursoragent@cursor.com>
Copy link
Copy Markdown
Contributor

@mogita mogita left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cross-SDK review pass for CHA-3071. Three inline comments — see below.

Comment thread src/Clients/WebhookHelpers.cs Outdated
Comment thread src/Clients/AppClient.cs Outdated
Comment thread tests/WebhookCompressionTests.cs Outdated
nijeesh-stream and others added 2 commits May 11, 2026 13:09
DecodeSnsPayload now JSON-parses the SNS HTTP notification envelope
({"Type":"Notification","Message":"..."}) and extracts the inner
Message field before running the SQS pipeline. Falls through to the
pre-extracted Message string when the input is not a JSON envelope so
existing call sites keep working.

Legacy AppClient.VerifyWebhook now delegates to WebhookHelpers.VerifySignature,
which uses CryptographicOperations.FixedTimeEquals for constant-time
comparison instead of naive string equality.

Tests add a realistic SNS HTTP notification body fixture and exercise
both the new envelope path and the existing pre-extracted Message path.

Co-authored-by: Cursor <cursoragent@cursor.com>
Drop the "in constant time" claim from VerifyAndParseSqs / VerifyAndParseSns
XML docs and the .NET overview prose. The primitive WebhookHelpers.VerifySignature
remains constant-time (used by all three composites and the legacy
VerifyWebhook), but timing-attack resistance is only meaningful for the
HTTP webhook X-Signature header that is exposed publicly. SQS / SNS
deliveries arrive over AWS-internal transports where the attack vector
does not apply.

Co-authored-by: Cursor <cursoragent@cursor.com>
@mogita
Copy link
Copy Markdown
Contributor

mogita commented May 11, 2026

Cross-SDK coordination: unifying webhook exception types

After the review pass across all 6 SDKs in this rollout and team discussion, we're consolidating the new webhook exception strategy to a single unified exception class rather than the split (signature vs parse exceptions) being introduced in this PR.

The Webhook Handling Spec on Notion (CHA-2961) has been revised to reflect this — §5.2 / §5.3 / §7 now specify a single class.

Why unified: From a customer's perspective, all failure modes — signature mismatch, gzip decompression failure, base64 decode failure, SNS envelope failure, JSON parse failure, missing schema field — terminate at the same catch block in customer code. A signature/parse split adds structural complexity without changing customer behavior. Customers who want to filter security logs for signature mismatches specifically can do so via exception message text or Exception.InnerException chain.

Class name family: StreamInvalidWebhookException — "Invalid" covers all failure modes accurately, consistent with BCL naming patterns (InvalidOperationException, InvalidCastException, etc.).

Per-SDK naming across the rollout:

SDK Class name
JS InvalidWebhookError (extends Error)
Python InvalidWebhookError
Go sentinel ErrInvalidWebhook + struct InvalidWebhookError
Java InvalidWebhookException (extends existing StreamException)
PHP InvalidWebhookException (extends existing StreamException)
Ruby StreamChat::InvalidWebhookError (extends StandardError)
.NET StreamInvalidWebhookException (extends StreamBaseException)

Asks for this PR:

  1. Rename StreamWebhookSignatureExceptionStreamInvalidWebhookException, extending the existing StreamBaseException in src/Exceptions/
  2. Wrap all failure paths into this single type — signature mismatch, gzip failure, base64 failure, SNS envelope failure, JSON parse failure, missing type/schema failure (this resolves the inline comment about JSON parse failures being misleadingly wrapped in a "Signature" exception type)
  3. Attach a human-readable message identifying which failure mode fired (e.g. "signature mismatch", "invalid base64", "missing type field") so customers can filter on message content
  4. Legacy AppClient.VerifyWebhook (returning bool) stays unchanged in return type — back-compat preserved. Separate inline comment about its naive sig == xSignature comparison still applies.
  5. Update xUnit tests to assert against the new exception name; for mode-specific tests, also assert on message-content substrings

This same comment is being posted on all 6 SDK PRs (JS / Go / Ruby / PHP / Java / .NET) for coordination. Happy to discuss naming or scope tradeoffs.

nijeesh-stream and others added 2 commits May 11, 2026 15:32
…n fixtures (CHA-3071)

Per Tommaso's suggestion, align the gzip helper with the GNU `gunzip`
command name. The function was added in this PR and not yet released,
so this is a straight rename with no back-compat alias.

Adds Tommaso's reference fixtures to the test suite as named cases so
future SDKs can sanity-check against the same payloads:

  aGVsbG93b3JsZA==                          -> helloworld   (base64)
  H4sIAGrYAWoAA8tIzcnJL88vykkBAK0g6/kKAAAA -> helloworld   (base64+gzip)

Co-authored-by: Cursor <cursoragent@cursor.com>
…ception (CHA-3071)

Per cross-SDK coordination (mogita's review on all 6 sibling SDK PRs),
every webhook failure path now terminates at a single exception class.
Customers only need one catch arm and can filter on Message text for
mode-specific behaviour.

Renames the previously-unreleased StreamWebhookSignatureException to
StreamInvalidWebhookException (still extends StreamBaseException) and
threads it through every primitive:

  VerifyAndParseWebhook -> 'signature mismatch'
  GunzipPayload         -> 'gzip decompression failed'
  DecodeSqsPayload      -> 'invalid base64 encoding'
  ParseEvent            -> 'invalid JSON payload'

VerifySignature keeps its boolean return at the primitive layer; the
composite VerifyAndParse* helpers throw on mismatch. The legacy
AppClient.VerifyWebhook (bool return) is untouched.

Co-authored-by: Cursor <cursoragent@cursor.com>
@nijeesh-stream
Copy link
Copy Markdown
Contributor Author

Shipped in 6bf2dcb along with the SNS envelope fix.

  • StreamWebhookSignatureExceptionStreamInvalidWebhookException (extends StreamBaseException, file renamed at src/Exceptions/StreamInvalidWebhookException.cs)
  • public const string SignatureMismatch / InvalidBase64 / GzipFailed / InvalidJson on the class for exact-match filtering; inner exceptions (InvalidDataException, IOException, FormatException, JsonException) chained through the (string, Exception) constructor
  • GunzipPayload now catches both InvalidDataException and IOException to cover the full range of malformed-gzip cases. VerifySignature keeps its bool return; composite VerifyAndParse* helpers throw on mismatch via VerifyAndParseInternal. Legacy AppClient.VerifyWebhook (bool return) is untouched
  • tests/WebhookCompressionTests.cs: 36 passed (was 33); WithMessage calls converted to substring wildcards (*signature mismatch*, *invalid base64 encoding*, etc.) and the three required failure-mode tests added. dotnet build clean (the 6 NETSDK1138 warnings on samples/* are pre-existing)
  • Full-repo grep confirms zero StreamWebhookSignatureException references remain

Docs side, docs-content#1276 now carries the per-SDK error-class table on the Webhooks overview page.

nijeesh-stream and others added 3 commits May 12, 2026 14:44
…-3071)

Stream does not ship an X-Signature on SQS or SNS deliveries — those
transports ride AWS-internal infrastructure (IAM-authenticated queues
and AWS-signed SNS notifications), so HMAC verification on top is
theatre. signature + secret are now nullable on both WebhookHelpers
static methods and on the AppClient instance methods.

  WebhookHelpers.VerifyAndParseSqs(body)               -> decode + parse
  WebhookHelpers.VerifyAndParseSqs(body, sig, secret)  -> + verify
  appClient.VerifyAndParseSns(envelopeBody)            -> unwrap + decode + parse

Passing only one of (signature, secret) throws
StreamInvalidWebhookException. The HTTP-webhook path is unchanged.

Co-authored-by: Cursor <cursoragent@cursor.com>
…ndParseWebhook; docs + tests

Co-authored-by: Cursor <cursoragent@cursor.com>
…ipPayload

- Rename StreamInvalidWebhookException to InvalidWebhookError (still extends StreamBaseException)
- Rename UngzipPayload to GunzipPayload
- Constants and messages unchanged: SignatureMismatch / InvalidBase64 / GzipFailed / InvalidJson
@nijeesh-stream nijeesh-stream merged commit a9513fa into master May 12, 2026
2 checks passed
@nijeesh-stream nijeesh-stream deleted the nijeeshjoshy/cha-3071-compress-webhook-payloads branch May 12, 2026 15:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants