fix(sns): pass messageDeduplicationId from FIFO topics to SQS FIFO queues#171
Conversation
There was a problem hiding this comment.
Pull request overview
Fixes SNS→SQS FIFO fanout by ensuring the resolved MessageDeduplicationId is forwarded when SNS delivers messages to SQS FIFO subscriptions, preventing InvalidParameterValue failures (and resulting message loss) when the SQS queue does not have content-based dedup enabled.
Changes:
- Thread resolved FIFO deduplication ID through
SnsService.deliverMessage()forpublish(). - Pass
MessageDeduplicationIdthroughdeliverMessage()frompublishBatch()call sites. - Add focused unit tests covering SNS FIFO delivery into SQS FIFO queues, plus a small SQS test factory helper.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
src/main/java/io/github/hectorvent/floci/services/sns/SnsService.java |
Threads messageDeduplicationId into SQS sendMessage() during SNS deliveries. |
src/test/java/io/github/hectorvent/floci/services/sns/SnsSqsFanoutFifoDeliveryTest.java |
Adds tests validating FIFO topic → FIFO queue delivery for explicit and topic-CBD dedup cases. |
src/test/java/io/github/hectorvent/floci/services/sqs/SqsServiceFactory.java |
Adds a small test helper to instantiate SqsService with in-memory backends. |
Comments suppressed due to low confidence (1)
src/main/java/io/github/hectorvent/floci/services/sns/SnsService.java:286
- For FIFO topics, AWS requires a MessageDeduplicationId unless the topic has ContentBasedDeduplication enabled. If the topic is FIFO, ContentBasedDeduplication is false, and no MessageDeduplicationId is provided, this method currently proceeds and will later attempt delivery with a null dedup ID (leading to SQS errors / dropped deliveries). Consider throwing an AwsException (InvalidParameter/InvalidParameterValue) when dedupId is still null after the ContentBasedDeduplication check to match AWS behavior and avoid silent message loss.
boolean isFifo = "true".equals(topic.getAttributes().get("FifoTopic"));
String dedupId = messageDeduplicationId;
if (isFifo) {
if (messageGroupId == null || messageGroupId.isBlank()) {
throw new AwsException("InvalidParameter",
"MessageGroupId is required for FIFO topics.", 400);
}
if (dedupId == null && "true".equals(topic.getAttributes().get("ContentBasedDeduplication"))) {
dedupId = sha256(message);
}
if (dedupId != null && isDuplicate(effectiveArn, dedupId)) {
src/test/java/io/github/hectorvent/floci/services/sns/SnsSqsFanoutFifoDeliveryTest.java
Outdated
Show resolved
Hide resolved
…eues SnsService.deliverMessage was passing null as the messageDeduplicationId to sqsService.sendMessage when delivering to FIFO SQS subscriptions. This caused the SQS service to throw an InvalidParameterValue exception for queues without ContentBasedDeduplication enabled. The exception was silently swallowed by the catch block, resulting in silent message loss. Thread the resolved deduplication ID through deliverMessage for both publish() and publishBatch() call sites.
da51462 to
8e859dd
Compare
|
Thanks @BSchneppe for the PR, Please review and fix if is required copilot comments. Thanks again! |
i fixed the issue described, needs another compat test i guess. covered in unit tests |
|
Thanks @BSchneppe, This is the kind of change that make floci better. |
SnsService.deliverMessage was passing null as the messageDeduplicationId to sqsService.sendMessage when delivering to FIFO SQS subscriptions. This caused the SQS service to throw an InvalidParameterValue exception for queues without ContentBasedDeduplication enabled. The exception was silently swallowed by the catch block, resulting in silent message loss.
Thread the resolved deduplication ID through deliverMessage for both publish() and publishBatch() call sites.
Summary
Type of change
fix:)feat:)feat!:orfix!:)AWS Compatibility
Tested against real AWS (eu-central-1) with aws-cli/2.27.50
hectorvent/floci-compatibility-tests#31
Checklist
./mvnw testpasses locallyunrelated tests fail locally: