cloudwatchlogs: buffer resetTimerCh to prevent lost wakeup in force-flush timer

[musa-asad]

Summary

A one-line concurrency fix that prevents the force-flush timer from permanently stalling on low-volume log streams.

Background

The CloudWatch Agent reads log files and publishes events to CloudWatch Logs. Each log stream has a force-flush timer (default 5 seconds) that sends accumulated events even when the batch is not full. This keeps delivery timely for low-volume streams.

The flush timer is one-shot: after it fires, resetFlushTimer() must be called to start it again. If the reset is dropped, the timer stays dead and events only leave when the batch fills up, which for a quiet stream may take minutes or longer.

Problem

resetFlushTimer() uses a non-blocking send:

select {
case q.resetTimerCh <- struct{}{}:
default:
}

When manageFlushTimer is blocked sending on flushCh (waiting for the main loop to accept a flush signal), it cannot receive from resetTimerCh. With an unbuffered channel, the rearm hits the default branch and is silently dropped. After a size-based send, this can leave the force-flush timer permanently un-rearmed. A low-volume stream's trailing batch then sits unsent with no error or retry logged.

Fix

// Before
resetTimerCh: make(chan struct{}),

// After
resetTimerCh: make(chan struct{}, 1),

A buffer of 1 parks the rearm signal so manageFlushTimer picks it up on its next loop iteration. The signal is idempotent (presence means "please reset"), so a buffer of 1 is sufficient.

Tests

Three new tests exercise the fix:

Test	What it proves
TestResetTimerChIsBuffered	resetTimerCh has buffer capacity 1; cap 0 reintroduces the bug.
TestForceFlushTimerRearmsAfterHeldSend	A trailing event is flushed within 3× flushTimeout even when a size-based send held the flush path.
TestForceFlushTimer_SingleLowVolumeEventFlushes	A single event on a quiet stream is flushed by the timer alone.

$ go test -count=1 -race -run "TestResetTimerCh|TestForceFlush" -timeout 30s ./plugins/outputs/cloudwatchlogs/internal/pusher/
ok  	github.com/aws/amazon-cloudwatch-agent/plugins/outputs/cloudwatchlogs/internal/pusher	4.060s

[okankoAMZ]

why are we no longer checking for last line? why do we check ever line instead?

[musa-asad]

So we can find the drop message regardless of its position.