You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Multipart UploadPart flows through the NeedsBody → send_raw path, which buffers the entire part body in WASM linear memory on the Cloudflare Workers runtime:
collect_js_body copies the request stream JS→WASM (array_buffer() + to_vec())
execute_multipart computes SHA256 over the whole part for the outbound signature (hash_payload)
send_raw copies WASM→JS (js_sys::Uint8Array::from) to hand to fetch
So each part incurs two full body copies across the JS/WASM boundary + an O(N) SHA256, and is held in the worker's 128 MB memory heap. Single-object PutObject does not have this problem — it uses the zero-copy forward() path (the ReadableStream is handed straight to fetch, signed with UNSIGNED-PAYLOAD so no hashing).
For typical part sizes (~8 MiB) this is a few ms of CPU and fine. But the 128 MB memory limit caps the maximum multipart_chunksize a client can use before the worker OOMs.
Proposed fix
Route UploadPart through the streaming forward() path instead of send_raw:
Generate a presigned URL for PUT /{key}?partNumber=N&uploadId=X. object_store's Signer::signed_url only takes a Path (no query params), so presign manually via S3RequestSigner — it already builds SigV4 canonical query strings (incl. value-less params, fixed in feat: data edit operations — batch delete, copy rejection, wider write headers #88).
Return HandlerAction::Forward so the ReadableStream streams straight through fetch: no WASM buffering, no payload hash.
CompleteMultipartUpload stays on send_raw (its small XML body must be signed/forwarded), but that body is tiny. CreateMultipartUpload/AbortMultipartUpload are body-less and could also move to the forward path opportunistically.
Interaction with Cloudflare's request-body size limit
Even with streaming, every request through a Worker is bounded by the plan's request-body limit (100 MB Free/Pro, 200 MB Business, 500 MB Enterprise default). Streaming changes which limit binds:
Today (buffered): a part is capped by the smaller of the 128 MB memory limit and the plan body limit.
After streaming: a part is capped only by the plan body limit (no WASM buffering).
So streaming lets clients use the full plan body limit for part sizes. It does not lift the body limit itself — that's a hard platform constraint (see separate doc note). Large objects must still be uploaded as multipart with part sizes ≤ the plan limit.
Impact
Removes the 128 MB part-size ceiling on Workers (raises it to the plan body limit).
Eliminates 2 body copies + 1 SHA256 per part → lower CPU.
Brings UploadPart to parity with the zero-copy PutObject path.
References
crates/cf-workers/src/backend.rs — send_raw (buffers); forward is the zero-copy template
crates/core/src/proxy.rs — execute_multipart, dispatch_operation (the NeedsBody arm)
Problem
Multipart
UploadPartflows through theNeedsBody→send_rawpath, which buffers the entire part body in WASM linear memory on the Cloudflare Workers runtime:collect_js_bodycopies the request stream JS→WASM (array_buffer()+to_vec())execute_multipartcomputes SHA256 over the whole part for the outbound signature (hash_payload)send_rawcopies WASM→JS (js_sys::Uint8Array::from) to hand tofetchSo each part incurs two full body copies across the JS/WASM boundary + an O(N) SHA256, and is held in the worker's 128 MB memory heap. Single-object
PutObjectdoes not have this problem — it uses the zero-copyforward()path (theReadableStreamis handed straight tofetch, signed withUNSIGNED-PAYLOADso no hashing).For typical part sizes (~8 MiB) this is a few ms of CPU and fine. But the 128 MB memory limit caps the maximum
multipart_chunksizea client can use before the worker OOMs.Proposed fix
Route
UploadPartthrough the streamingforward()path instead ofsend_raw:PUT /{key}?partNumber=N&uploadId=X.object_store'sSigner::signed_urlonly takes aPath(no query params), so presign manually viaS3RequestSigner— it already builds SigV4 canonical query strings (incl. value-less params, fixed in feat: data edit operations — batch delete, copy rejection, wider write headers #88).HandlerAction::Forwardso theReadableStreamstreams straight throughfetch: no WASM buffering, no payload hash.CompleteMultipartUploadstays onsend_raw(its small XML body must be signed/forwarded), but that body is tiny.CreateMultipartUpload/AbortMultipartUploadare body-less and could also move to the forward path opportunistically.Interaction with Cloudflare's request-body size limit
Even with streaming, every request through a Worker is bounded by the plan's request-body limit (100 MB Free/Pro, 200 MB Business, 500 MB Enterprise default). Streaming changes which limit binds:
So streaming lets clients use the full plan body limit for part sizes. It does not lift the body limit itself — that's a hard platform constraint (see separate doc note). Large objects must still be uploaded as multipart with part sizes ≤ the plan limit.
Impact
UploadPartto parity with the zero-copyPutObjectpath.References
crates/cf-workers/src/backend.rs—send_raw(buffers);forwardis the zero-copy templatecrates/core/src/proxy.rs—execute_multipart,dispatch_operation(theNeedsBodyarm)