feat: dead letter mechanism for non-retryable messages#77
Merged
Conversation
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Implement DLQ mechanism for malformed webhook messages that should not be retried (missing headers, invalid JSON, missing required fields). Unsupported event types are now silently ignored via Ack instead of being sent to DLQ. - Add DeadLetterExchange and DeadLetterQueue to QueueConfig - Create setupDeadLetterQueue() to encapsulate DLQ declaration - Add declareQueueWithDeadLetter() with x-dead-letter-exchange arg - Add ErrUnsupportedEvent sentinel error for ignorable events - Add ignoreMessage() method that uses Ack for expected filtering - Update pullMessage() to distinguish DLQ vs ignore paths 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
# Conflicts: # cmd/planner/main.go # cmd/planner/main_test.go # internal/planner/consumer.go # internal/queue/config.go # internal/queue/consumer.go # internal/queue/consumer_test.go # internal/queue/mock_test.go # internal/queue/types.go
cbartz
changed the title
cbartz
commented
Jan 8, 2026
cbartz
requested review from
florentianayuwono,
weiiwang01 and
yanksyoon
as code owners
Contributor
There was a problem hiding this comment.
Pull request overview
This PR implements a dead letter mechanism for handling non-retryable messages in the webhook processing pipeline. Messages indicating data corruption or unexpected formats are routed to a Dead Letter Queue (DLQ) for inspection, while valid but unsupported events are acknowledged and discarded without DLQ routing.
- Adds DLX (dead-letter exchange) and DLQ (dead-letter queue) infrastructure to the queue package
- Introduces
ErrUnsupportedEventsentinel error to distinguish non-retryable from retryable failures - Separates message handling:
ignoreMessage()uses Ack for valid unsupported events,discardMessage()uses Nack to route corrupt messages to DLQ
Reviewed changes
Copilot reviewed 13 out of 13 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| internal/queue/types.go | Updates error messages and adds declareQueueWithDeadLetter() function to support DLX configuration |
| internal/queue/mock_test.go | Enhances mock channel to track DLX/DLQ declarations and queue arguments |
| internal/queue/deadletter_test.go | Adds comprehensive tests for DLX/DLQ setup functionality |
| internal/queue/deadletter.go | Implements setupDeadLetterQueue() to configure dead letter infrastructure |
| internal/queue/consumer_test.go | Adds test verifying DLX/DLQ setup during consumer Pull operation |
| internal/queue/consumer.go | Integrates DLX/DLQ setup into consumer initialization flow |
| internal/queue/config.go | Adds DLX and DLQ configuration fields with default values |
| internal/planner/consumer_test.go | Updates tests to reflect new message handling behavior (Ack vs Nack) |
| internal/planner/consumer.go | Implements ErrUnsupportedEvent sentinel error and ignoreMessage() function; routes unsupported events appropriately |
| cmd/planner/main_test.go | Refactors integration tests to verify DLQ behavior and adds scenario-based testing |
| CONTRIBUTING.md | Updates Postgres version from 18.1 to 16.11 in Docker command |
| .github/workflows/planner_integration.yaml | Updates Postgres service version from 18.1 to 16.11 |
| .github/workflows/internal_tests.yaml | Updates Postgres service version from 18.1 to 16.11 |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
yanksyoon
approved these changes
Jan 9, 2026
yanksyoon
left a comment
yanksyoon
left a comment
Member
There was a problem hiding this comment.
Minor nits + idempotency comment. Thanks!
yhaliaw
approved these changes
Jan 9, 2026
yhaliaw
reviewed
Jan 12, 2026
yanksyoon
approved these changes
Jan 12, 2026
yanksyoon
left a comment
yanksyoon
left a comment
Member
There was a problem hiding this comment.
LGTM! Thanks for the changes 👍
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
Add a dead-letter exchange and queue and put non-retryable messages there:
Only messages that indicate data corruption or unexpected formats go to the DLQ for inspection. Valid messages that simply don't require processing are acknowledged and discarded.
Messages sent to DLQ (via Nack):
Messages NOT sent to DLQ (via Ack):
Rationale
Malformed/corrupt messages that fail parsing could indicate a serious issue (e.g. our code does not react to a format change of webhooks) and should be inspected.
Checklist
CONTRIBUTING.mdhas been updated upon changes to the contribution/development process (e.g. changes to the way tests are run)Minor technical update to CONTRIBUTING.md , no TA required.