Skip to content

Add Parallel Multipart Download support for Pre-signed URL GetObject#6937

Open
jencymaryjoseph wants to merge 2 commits intofeature/master/pre-signed-url-getobjectfrom
jencyjos/presigned-url/multipart-fixes
Open

Add Parallel Multipart Download support for Pre-signed URL GetObject#6937
jencymaryjoseph wants to merge 2 commits intofeature/master/pre-signed-url-getobjectfrom
jencyjos/presigned-url/multipart-fixes

Conversation

@jencymaryjoseph
Copy link
Copy Markdown
Contributor

@jencymaryjoseph jencymaryjoseph commented May 5, 2026

Motivation and Context

Adding support for FileAsyncResponseTransformerPublisher (toFile path) in pre-signed URL multipart downloads.
This introduces a parallel subscriber ParallelPresignedUrlMultipartDownloaderSubscriber that handles concurrent range requests and explicit resultFuture completion, similar to ParallelMultipartDownloaderSubscriber for regular multipart downloads.

Also fixes error propagation in the serial subscriber (toBytes path) where resultFuture
was being completed after subscription cancel, allowing ByteArraySplittingTransformer to
race and assemble invalid data.

Also fixes small object downloads in the serial subscriber (toBytes path) by reordering
Content-Range parsing before validation, so S3's actual object range is correctly validated
instead of being compared against the larger requested range.

Modifications

  • ParallelPresignedUrlMultipartDownloaderSubscriber (NEW): Parallel subscriber for toFile().
    Discovers object size from first part, fires remaining parts concurrently (up to
    maxInFlightParts=10). Completes resultFuture explicitly when all parts finish. Handles
    single-part objects by completing immediately after first response.

  • PresignedUrlDownloadHelper: Routes based on split.parallelSplitSupported() — parallel
    subscriber for toFile, serial subscriber for toBytes.

  • PresignedUrlMultipartDownloaderSubscriber: Added resultFuture field.
    Reordered validatePart to parse Content-Range before validation, fixing small object downloads where S3 returns the actual object range instead of the requested range.
    handleError() completes resultFuture before cancel to prevent data corruption in toBytes() path.

Testing

  • PresignedUrlMultipartDownloaderSubscriberWiremockTest: Parameterized for toBytes/toFile,
    added small object test case
  • ParallelPresignedUrlMultipartDownloaderSubscriberTest (NEW): Single-part detection,
    multi-part download, error propagation, Content-Range validation failures
  • ParallelPresignedUrlMultipartDownloaderSubscriberTckTest (NEW): Reactive Streams TCK
    compliance
  • PresignedUrlMultipartDownloaderSubscriberTckTest: Updated constructor

Screenshots (if appropriate)

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)

Checklist

  • I have read the CONTRIBUTING document
  • Local run of mvn install succeeds
  • My code follows the code style of this project
  • My change requires a change to the Javadoc documentation
  • I have updated the Javadoc documentation accordingly
  • I have added tests to cover my changes
  • All new and existing tests passed
  • I have added a changelog entry. Adding a new entry must be accomplished by running the scripts/new-change script and following the instructions. Commit the new file created by the script in .changes/next-release with your changes.
  • My change is to implement 1.11 parity feature and I have updated LaunchChangelog

License

  • I confirm that this pull request can be released under the Apache 2 license

@jencymaryjoseph jencymaryjoseph requested a review from a team as a code owner May 5, 2026 18:19
@jencymaryjoseph jencymaryjoseph force-pushed the jencyjos/presigned-url/multipart-fixes branch from 7f0be01 to b19ed2e Compare May 5, 2026 18:22
@jencymaryjoseph jencymaryjoseph force-pushed the feature/master/pre-signed-url-getobject branch from ac6b49f to 78e9cba Compare May 5, 2026 18:25
@jencymaryjoseph jencymaryjoseph force-pushed the jencyjos/presigned-url/multipart-fixes branch from b19ed2e to c928799 Compare May 5, 2026 18:26
@jencymaryjoseph jencymaryjoseph added the no-api-surface-area-change Indicate there is no API surface area change and thus API surface area review is not required label May 5, 2026
joviegas
joviegas previously approved these changes May 5, 2026
@joviegas joviegas self-requested a review May 5, 2026 21:29
@joviegas joviegas dismissed their stale review May 5, 2026 21:31

sorry was approving other PR accidentally clicked on this one

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

no-api-surface-area-change Indicate there is no API surface area change and thus API surface area review is not required

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants