Skip to content

Comments

Send pending records#12

Open
piequi wants to merge 8 commits intoPayU:masterfrom
piequi:fix/send-pending-records
Open

Send pending records#12
piequi wants to merge 8 commits intoPayU:masterfrom
piequi:fix/send-pending-records

Conversation

@piequi
Copy link

@piequi piequi commented Dec 4, 2023

Fixes #11

This PR adds 2 new parameters to the plugin configuration: SQSEndpoint and flushPendingRecordsString

SQSEndpoint was introduced as a convenience to be able to use Locastack SQS queue to experiment around.

flushPendingRecordsString is a boolean parameter that will allow flushing pending records (as described in the issue) to be sent following fluentbit's Flush period. If the records count provided to the plugin when calling FLBPluginFlushCtx() isn't a multiple of batchSize then the remaining records will be sent in a smaller SQS message batch.

As I'm not very knowledeable regarding GO module packaging, I don't know if the dependencies update are concerning or not.

@piequi piequi changed the title Fix/send pending records Send pending records Dec 4, 2023
@piequi piequi force-pushed the fix/send-pending-records branch from fe58010 to 59b3b54 Compare December 4, 2023 16:24
@piequi piequi marked this pull request as ready for review December 7, 2023 08:54
@mfabricanti
Copy link

@shyimo could you please verify this PR?

We need to use SQSEndpoint feature as well.

@piequi Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

Pending records not sent when exiting

2 participants