refactor: migrate to AWS SDK v3 (S3)#2293
Draft
aalemayhu wants to merge 1 commit into
Draft
Conversation
aws-sdk v2 has reached end-of-support (the prod box's pm2 logs show the EOL warning on every restart). Replace it with the modular v3 client so security updates resume and the warning clears. Why now: AWS announced v2 end-of-support on 2024-09-08; security patches have stopped. https://aws.amazon.com/blogs/developer/announcing-end-of-support-for-aws-sdk-for-javascript-v2/ Changes: - StorageHandler: rebuilt on @aws-sdk/client-s3 S3Client; getFileContents now eagerly drains the v3 Readable body into a Buffer so existing consumers that read `.Body` keep working unchanged. - getContents: paginates via ContinuationToken (the v2 loop never advanced — IsTruncated alone could not move past page 1). - getPresignedUrl: uses @aws-sdk/s3-request-presigner. - DownloadService, convertPDFToImages, ConvertPPTToPDF, getPackagesFromZip, zip.tsx: replace `S3.Body`/`Body` type with `Buffer | Uint8Array | string` (the body shapes the conversion paths actually receive). - DownloadService.isMissingDownloadError: switch from the removed `AWS.AWSError` ambient type to a structural check on `error.name`. Deps: - Drop `aws-sdk` ^2.1502.0. - Add `@aws-sdk/client-s3` ^3.1048.0 and `@aws-sdk/s3-request-presigner` ^3.1048.0 (pinned to the same version to keep `@smithy/types` deduped; mismatched minors fail tsc with HandlerExecutionContext incompatibility). No user-visible behavior change; no changelog entry. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.



Summary
Migrate the S3 surface from
aws-sdkv2 (end-of-support 2024-09-08) to the modular@aws-sdk/client-s3v3. The prod box's pm2 logs print the EOL warning on every restart; this clears it and re-enables security updates.Files migrated
src/lib/storage/StorageHandler.ts— centralS3Client;getFileContentsnow drains the v3 stream into aBufferso consumers that read.Bodykeep working unchanged.getContentsproperly paginates viaContinuationToken.getPresignedUrluses@aws-sdk/s3-request-presigner.src/services/DownloadService.ts— return typeS3.Body→Buffer;isMissingDownloadErrorstructural check.src/infrastracture/adapters/fileConversion/convertPDFToImages.ts— param typeS3.Body→Buffer | Uint8Array | string.src/infrastracture/adapters/fileConversion/ConvertPPTToPDF.ts— same.src/usecases/uploads/getPackagesFromZip.ts— same.src/lib/zip/zip.tsx—File.contentstyped againstBuffer | Uint8Array | string. Outside the original migration plan — flagged because it was a touch-up the worker did to satisfy types, not part of the file list from the research phase. Worth a careful read.package.json— dropaws-sdk ^2.1502.0; add@aws-sdk/client-s3 ^3.1048.0and@aws-sdk/s3-request-presigner ^3.1048.0(both pinned to the same minor — mismatched versions fail tsc on@smithy/types).Static e2e checks (all 7 passed locally)
pnpm test— 169/169 suites green, 1179 tests passpnpm typecheck— cleanpnpm build— cleangrep -rE \"from 'aws-sdk'\" src/— 0 hitsgrep -rE \"require\\('aws-sdk'\\)\" src/— 0 hitsgrep '\"aws-sdk\":' package.json— 0 hitsgrep '@aws-sdk/client-s3' package.json— 1 hitWhat the static checks DON'T cover (and why this PR is held in draft)
The Jest suite mocks the SDK at the module boundary, so no test in this PR actually exercises the v3 client against real S3 stream-handling behavior. The greps confirm "no v2 left," not "v3 works correctly." The risky paths are dynamic and need manual e2e:
Manual e2e recipe (run in a fresh session before flipping ready)
pnpm dev— start server + web..pptxthat has at least one embedded image. Exercises:ConvertPPTToPDF→ S3 putObject →convertPDFToImages→ S3 getObject → image array. Confirm: the.apkgdownloads, imports into Anki, cards render with the right media..zip. Exercises:getPackagesFromZip→ S3 list/get. Confirm: each deck inside the zip converts and downloads..apkgor PDF. Watchpnpm dev's memory footprint. The newgetFileContentsdrains the v3 stream into a Buffer in one shot viatransformToByteArray()— large files now occupy full memory before any consumer sees them, vs. v2 which also returned a Buffer but with different backpressure characteristics. Mostly a yellow flag, not red — but worth watching once.StorageHandler.getFileContentsin the read-only path.pm2 logson the dev terminal shows no AWS SDK v2 EOL warning at startup.If any of the above breaks, the fix is local to the worker's diff — see "Notes for reviewer" below for the most likely culprits.
Notes for reviewer
getContentscalledlistObjectswith the same params every iteration (no Marker advancement), so it could only ever return page 1 — capped at 1000 objects. v3 usesContinuationTokencorrectly. If any downstream code assumed "showcase will never have more than 1000 objects" or "this list is always exhaustive on first call," it now sees more results. Worth a grep for callers ofgetContents()and a sanity check.getFileContentsusestransformToByteArray()then wraps inBuffer.from(...). This loads the entire object into memory before returning. Fine for typical decks (single-digit MB) but a regression risk for very large.apkgs. If the deploy reveals memory pressure, the fix is to stream directly to the consumer rather than buffer first — but that's a follow-up, not a blocker.zip.tsxchange is wider than the file list. The original research plan listed 5 source files; the worker addedsrc/lib/zip/zip.tsxto satisfy types. Quick read recommended.Sonar
sonar-scannerwas not installed in the worker environment; expect a possible Sonar bounce on the first CI run after flipping ready. Happy to address any cognitive-complexity smells in a follow-up commit.Pre-merge checklist
Per CLAUDE.md ("Touching auth, payments, or external-API integration? Run /security-review before merge."):
/security-reviewrun against the diff (S3 = external-API integration)statusCheckRollupentries non-FAILUREpm2 logson prod confirmed no v2 EOL warning post-deploy🤖 Generated with Claude Code
<a href="https://app.blacksmith.sh/2anki/codesmith/server/pr/2293\"><source media="(prefers-color-scheme: dark)" srcset="https://pr-comments-assets.blacksmith.sh/codesmith/view-in-codesmith-dark.svg\"><source media="(prefers-color-scheme: light)" srcset="https://pr-comments-assets.blacksmith.sh/codesmith/view-in-codesmith-light.svg\"><img alt="View in Codesmith" src="https://pr-comments-assets.blacksmith.sh/codesmith/view-in-codesmith-dark.svg\">
Need help on this PR? Tag
@codesmithwith what you need.