feat: detect log rotation and reopen the rotated file - #13
Merged
Conversation
source now compares the inode at source_path against the one it has open on every 200ms tick. On a mismatch (standard logrotate rename-then-create behavior), it drains whatever was left in the old file, reopens the new one from the start, and re-registers the notify watcher — otherwise it stays bound to the old, now-orphaned inode. Doesn't yet handle copytruncate-style rotation (same inode, truncated in place).
New tests/stress_test.rs — a separate category from the per-function correctness tests elsewhere in this directory, meant for tests that exercise the full pipeline under load/failure conditions rather than one function in isolation. Sends a burst of logs well beyond channel_capacity while the sink's destination is unreachable for the whole test, and asserts the dead-letter file ends up with exactly what was written — no loss, no duplicates — after every batch exhausts its retries.
Checks off log rotation and the channel-boundedness criterion (the latter proven by construction plus the new stress test rather than direct memory measurement), and explicitly notes config validation and metrics as deferred to v1.1 rather than leaving them as unexplained open checkboxes. Also documents the tests/ layout now that stress_test.rs exists as its own category.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request significantly improves the application's robustness in handling log rotation and adds comprehensive stress tests to ensure reliability under failure conditions. The main changes include enhancing the log source to detect and handle log file rotation (such as that performed by
logrotate), updating documentation and acceptance criteria, and introducing new tests to verify these behaviors.Log rotation handling and robustness:
src/source.rs) now detects when the log file has been rotated (renamed and replaced) by monitoring the file's inode. When a rotation is detected, it flushes any remaining lines from the old file and seamlessly switches to the new file, ensuring no logs are lost during rotation. [1] [2]Testing improvements:
test_run_source_detects_log_rotation) to verify that the source correctly handles log rotation and continues processing new lines from the newly created file.tests/stress_test.rs) that simulates a destination outage and verifies that no logs are lost, even under backpressure and repeated send failures. All logs are expected to be written to the dead-letter file.Documentation and acceptance criteria:
README.mdto clarify test organization, document the guarantees around log rotation and channel bounds, and note which acceptance criteria are now met or deferred.These changes collectively make log processing more reliable in production scenarios, especially when dealing with log rotation and temporary sink outages.