Describe the solution you'd like:
I propose changing the Global Secondary Index (GSI) sort key from sent_at to sequence_key in DynamoMQ. This modification aims to improve throughput efficiency. When FIFO is disabled, the sequence_key will hold a random string (e.g., UUID), which will enable partition splitting in DynamoDB and thus enhance throughput. When FIFO is enabled, the sequence_key will continue to hold an ISO 8601 formatted date-time string. Although this retains the throughput limitation inherent to FIFO, it is an acceptable constraint given the requirement for ordered processing.
Describe alternatives you've considered:
An alternative considered was maintaining the current design with queue_type as the partition key and sent_at as the sort key in the GSI. However, this approach limits the available throughput to 1000 WCUs and 3000 RCUs due to all data being stored in the same GSI partition. Using Scan for fetching messages was also considered but discarded due to its inefficiency.
Additional context:
Describe the solution you'd like:
I propose changing the Global Secondary Index (GSI) sort key from
sent_attosequence_keyin DynamoMQ. This modification aims to improve throughput efficiency. When FIFO is disabled, thesequence_keywill hold a random string (e.g., UUID), which will enable partition splitting in DynamoDB and thus enhance throughput. When FIFO is enabled, thesequence_keywill continue to hold an ISO 8601 formatted date-time string. Although this retains the throughput limitation inherent to FIFO, it is an acceptable constraint given the requirement for ordered processing.Describe alternatives you've considered:
An alternative considered was maintaining the current design with
queue_typeas the partition key andsent_atas the sort key in the GSI. However, this approach limits the available throughput to 1000 WCUs and 3000 RCUs due to all data being stored in the same GSI partition. Using Scan for fetching messages was also considered but discarded due to its inefficiency.Additional context: