Skip to content

fix(sns): pass messageDeduplicationId from FIFO topics to SQS FIFO queues#171

Merged
hectorvent merged 2 commits intohectorvent:mainfrom
BSchneppe:fix/sns-fifo-dedup-passthrough
Apr 2, 2026
Merged

fix(sns): pass messageDeduplicationId from FIFO topics to SQS FIFO queues#171
hectorvent merged 2 commits intohectorvent:mainfrom
BSchneppe:fix/sns-fifo-dedup-passthrough

Conversation

@BSchneppe
Copy link
Copy Markdown
Contributor

@BSchneppe BSchneppe commented Apr 2, 2026

SnsService.deliverMessage was passing null as the messageDeduplicationId to sqsService.sendMessage when delivering to FIFO SQS subscriptions. This caused the SQS service to throw an InvalidParameterValue exception for queues without ContentBasedDeduplication enabled. The exception was silently swallowed by the catch block, resulting in silent message loss.

Thread the resolved deduplication ID through deliverMessage for both publish() and publishBatch() call sites.

Summary

Type of change

  • Bug fix (fix:)
  • New feature (feat:)
  • Breaking change (feat!: or fix!:)
  • Docs / chore

AWS Compatibility

Tested against real AWS (eu-central-1) with aws-cli/2.27.50
hectorvent/floci-compatibility-tests#31

Checklist

unrelated tests fail locally:

[INFO] 
[ERROR] Failures: 
[ERROR]   S3MultipartIntegrationTest.abortMultipartUpload:152 1 expectation failed.
Expected status code <200> but was <204>.

[ERROR]   S3MultipartIntegrationTest.uploadPartCopy:195 1 expectation failed.
Expected status code <200> but was <204>.

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes SNS→SQS FIFO fanout by ensuring the resolved MessageDeduplicationId is forwarded when SNS delivers messages to SQS FIFO subscriptions, preventing InvalidParameterValue failures (and resulting message loss) when the SQS queue does not have content-based dedup enabled.

Changes:

  • Thread resolved FIFO deduplication ID through SnsService.deliverMessage() for publish().
  • Pass MessageDeduplicationId through deliverMessage() from publishBatch() call sites.
  • Add focused unit tests covering SNS FIFO delivery into SQS FIFO queues, plus a small SQS test factory helper.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File Description
src/main/java/io/github/hectorvent/floci/services/sns/SnsService.java Threads messageDeduplicationId into SQS sendMessage() during SNS deliveries.
src/test/java/io/github/hectorvent/floci/services/sns/SnsSqsFanoutFifoDeliveryTest.java Adds tests validating FIFO topic → FIFO queue delivery for explicit and topic-CBD dedup cases.
src/test/java/io/github/hectorvent/floci/services/sqs/SqsServiceFactory.java Adds a small test helper to instantiate SqsService with in-memory backends.
Comments suppressed due to low confidence (1)

src/main/java/io/github/hectorvent/floci/services/sns/SnsService.java:286

  • For FIFO topics, AWS requires a MessageDeduplicationId unless the topic has ContentBasedDeduplication enabled. If the topic is FIFO, ContentBasedDeduplication is false, and no MessageDeduplicationId is provided, this method currently proceeds and will later attempt delivery with a null dedup ID (leading to SQS errors / dropped deliveries). Consider throwing an AwsException (InvalidParameter/InvalidParameterValue) when dedupId is still null after the ContentBasedDeduplication check to match AWS behavior and avoid silent message loss.
        boolean isFifo = "true".equals(topic.getAttributes().get("FifoTopic"));
        String dedupId = messageDeduplicationId;
        if (isFifo) {
            if (messageGroupId == null || messageGroupId.isBlank()) {
                throw new AwsException("InvalidParameter",
                        "MessageGroupId is required for FIFO topics.", 400);
            }
            if (dedupId == null && "true".equals(topic.getAttributes().get("ContentBasedDeduplication"))) {
                dedupId = sha256(message);
            }
            if (dedupId != null && isDuplicate(effectiveArn, dedupId)) {

…eues

   SnsService.deliverMessage was passing null as the messageDeduplicationId
   to sqsService.sendMessage when delivering to FIFO SQS subscriptions.
   This caused the SQS service to throw an InvalidParameterValue exception
   for queues without ContentBasedDeduplication enabled. The exception was
   silently swallowed by the catch block, resulting in silent message loss.

   Thread the resolved deduplication ID through deliverMessage for both
   publish() and publishBatch() call sites.
@BSchneppe BSchneppe force-pushed the fix/sns-fifo-dedup-passthrough branch from da51462 to 8e859dd Compare April 2, 2026 11:38
@hectorvent hectorvent added the sqs label Apr 2, 2026
@hectorvent
Copy link
Copy Markdown
Owner

Thanks @BSchneppe for the PR,

Please review and fix if is required copilot comments.

Thanks again!

@BSchneppe
Copy link
Copy Markdown
Contributor Author

BSchneppe commented Apr 2, 2026

Thanks @BSchneppe for the PR,

Please review and fix if is required copilot comments.

Thanks again!

i fixed the issue described, needs another compat test i guess. covered in unit tests

@hectorvent
Copy link
Copy Markdown
Owner

Thanks @BSchneppe,

This is the kind of change that make floci better.

PASS  SNS FIFO explicit dedup - message received
  PASS  SNS FIFO explicit dedup - dedup ID matches
  PASS  SNS FIFO content-based dedup - message received
  PASS  SNS FIFO content-based dedup - dedup ID present

@hectorvent hectorvent merged commit 4529823 into hectorvent:main Apr 2, 2026
6 of 11 checks passed
@BSchneppe BSchneppe deleted the fix/sns-fifo-dedup-passthrough branch April 3, 2026 00:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants