docs: outputs: file: add log rotation support and configuration examples#2572
docs: outputs: file: add log rotation support and configuration examples#2572SagiROosto wants to merge 1 commit into
Conversation
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
✅ Files skipped from review due to trivial changes (1)
📝 WalkthroughWalkthroughDocumentation for the File output plugin adds log rotation details: new parameters ( ChangesFile Output Plugin Log Rotation Documentation
Estimated code review effort🎯 1 (Trivial) | ⏱️ ~3 minutes Suggested labels
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Tip 💬 Introducing Slack Agent: The best way for teams to turn conversations into code.Slack Agent is built on CodeRabbit's deep understanding of your code, so your team can collaborate across the entire SDLC without losing context.
Built for teams:
One agent for your entire SDLC. Right inside Slack. Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
SagiROosto
force-pushed
the
aws-iot-creds-docs
branch
from
713685c to
2fc7093
Compare
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@pipeline/outputs/file.md`:
- Line 123: Update the documentation string that describes the rotation filename
pattern used when files_rotation is enabled: replace the incorrect timestamp
token `YYYMMDD_HHMMSS` with the correct `YYYYMMDD_HHMMSS` in the sentence that
explains how files are renamed (the block referencing `files_rotation` and
`max_size` and the pattern `<filename>.<YYYYMMDD_HHMMSS_XXXXXXXX>`).
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
SagiROosto
force-pushed
the
aws-iot-creds-docs
branch
from
2fc7093 to
3f1a696
Compare
|
@coderabbitai review |
✅ Actions performedReview triggered.
|
eschabell
left a comment
eschabell
left a comment
There was a problem hiding this comment.
@SagiROosto see changes to be made first before review passes.
|
|
||
| The plugin retains up to `max_files` rotated files per output file. When the limit is reached, the oldest rotated files are deleted automatically. | ||
|
|
||
| Log rotation works with all supported output [formats](#format), including `out_file`, plain, CSV, LTSV, template, and msgpack. File operations are thread-safe, so rotation can be used alongside multiple [workers](../../administration/multithreading.md#outputs). |
There was a problem hiding this comment.
@SagiROosto can you put back-ticks around all the formats listed here:
- plain
- CSV
- LTSV
- template
- msgpack.
There was a problem hiding this comment.
🧹 Nitpick comments (1)
pipeline/outputs/file.md (1)
123-123: ⚡ Quick winImprove grammar and consistency for the timestamp description.
The phrase "The YYYYMMDD_HHMMSS is timestamp of the time when the rotation occurred" is missing an article and should use backticks for consistency with other technical terms in the sentence.
📝 Suggested improvement
-When `files_rotation` is enabled, the plugin monitors the size of each output file. Once a file exceeds `max_size`, the plugin rotates the file by renaming it with a timestamp suffix in the format `<filename>.<YYYYMMDD_HHMMSS_XXXXXXXX>`. The YYYYMMDD_HHMMSS is timestamp of the time when the rotation occurred, and `XXXXXXXX` is a random identifier to guarantee unique filenames if multiple rotations happen within the same second. +When `files_rotation` is enabled, the plugin monitors the size of each output file. Once a file exceeds `max_size`, the plugin rotates the file by renaming it with a timestamp suffix in the format `<filename>.<YYYYMMDD_HHMMSS_XXXXXXXX>`. The `YYYYMMDD_HHMMSS` part is the rotation timestamp, and `XXXXXXXX` is a random identifier to guarantee unique filenames if multiple rotations happen within the same second.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@pipeline/outputs/file.md` at line 123, The sentence describing the rotation timestamp is missing an article and inconsistent formatting; update the text that explains the rotation suffix `<filename>.<YYYYMMDD_HHMMSS_XXXXXXXX>` (referencing `files_rotation` and `max_size`) to say: "The `YYYYMMDD_HHMMSS` is the timestamp of the time when the rotation occurred" (use backticks around `YYYYMMDD_HHMMSS` and `XXXXXXXX` for consistency) and ensure the full format is shown as `<filename>.<YYYYMMDD_HHMMSS_XXXXXXXX>` with `XXXXXXXX` described as a random identifier to guarantee unique filenames if multiple rotations happen within the same second.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@pipeline/outputs/file.md`:
- Line 123: The sentence describing the rotation timestamp is missing an article
and inconsistent formatting; update the text that explains the rotation suffix
`<filename>.<YYYYMMDD_HHMMSS_XXXXXXXX>` (referencing `files_rotation` and
`max_size`) to say: "The `YYYYMMDD_HHMMSS` is the timestamp of the time when the
rotation occurred" (use backticks around `YYYYMMDD_HHMMSS` and `XXXXXXXX` for
consistency) and ensure the full format is shown as
`<filename>.<YYYYMMDD_HHMMSS_XXXXXXXX>` with `XXXXXXXX` described as a random
identifier to guarantee unique filenames if multiple rotations happen within the
same second.
|
|
||
| The File output plugin supports size-based log rotation. | ||
|
|
||
| When `files_rotation` is enabled, the plugin monitors the size of each output file. Once a file exceeds `max_size`, the plugin rotates the file by renaming it with a timestamp suffix in the format `<filename>.<YYYYMMDD_HHMMSS_XXXXXXXX>`. The YYYYMMDD_HHMMSS is timestamp of the time when the rotation occurred, and `XXXXXXXX` is a random identifier to guarantee unique filenames if multiple rotations happen within the same second. |
There was a problem hiding this comment.
@SagiROosto also add back-ticks to the "YYYYMMDD_HHMMSS" for consistency.
eschabell
added
waiting-for-user
Signed-off-by: SagiROosto <sagi.rosenthal@oosto.com>
SagiROosto
force-pushed
the
aws-iot-creds-docs
branch
from
3f1a696 to
63477a5
Compare
patrick-stephens
left a comment
patrick-stephens
left a comment
There was a problem hiding this comment.
Some queries around bad-path behaviours but nothing blocking, it would be nice to resolve them though for the inevitable failures that people will have.
| | Key | Description | Default | | ||
| | :--- | :--- | :--- | | ||
| | `file` | Set filename to store the records. If not set, the filename will be the `tag` associated with the records. | _none_ | | ||
| | `files_rotation` | Enable size-based [log rotation](#log-rotation). When enabled, files that exceed `max_size` are rotated and optionally compressed. | `false` | |
There was a problem hiding this comment.
Just a query on how this is done, is it as soon as they reach that size or a periodic check or something else?
There was a problem hiding this comment.
@SagiROosto can you answer this for @patrick-stephens ?
| | `max_files` | Maximum number of rotated files to retain per output file. Oldest files are deleted first. Must be `1` or greater. Only applies when `files_rotation` is enabled. | `7` | | ||
| | `max_size` | Maximum size of the active output file before rotation is triggered. Supports size suffixes: `k` (kilobytes), `m` (megabytes), `g` (gigabytes). Only applies when `files_rotation` is enabled. | `100m` | |
There was a problem hiding this comment.
What happens if you do not enable rotation? Does it fail to start, report a warning, etc.?
Similarly what if you use invalid values here? Does it fail with a config error, use the default, etc?
There was a problem hiding this comment.
@SagiROosto can you answer this for @patrick-stephens ?
|
|
||
| The File output plugin supports size-based log rotation. | ||
|
|
||
| When `files_rotation` is enabled, the plugin monitors the size of each output file. Once a file exceeds `max_size`, the plugin rotates the file by renaming it with a timestamp suffix in the format `<filename>.<YYYYMMDD_HHMMSS_XXXXXXXX>`. The `YYYYMMDD_HHMMSS` is timestamp of the time when the rotation occurred, and `XXXXXXXX` is a random identifier to guarantee unique filenames if multiple rotations happen within the same second. |
There was a problem hiding this comment.
Where does the timestamp come from? Is it UTC or local timezone based?
Be nice to have a controllable format for the future too.
There was a problem hiding this comment.
@SagiROosto can you answer this for @patrick-stephens ?
eschabell
left a comment
eschabell
left a comment
There was a problem hiding this comment.
@SagiROosto can you answer the tagged questions for @patrick-stephens ?
| | Key | Description | Default | | ||
| | :--- | :--- | :--- | | ||
| | `file` | Set filename to store the records. If not set, the filename will be the `tag` associated with the records. | _none_ | | ||
| | `files_rotation` | Enable size-based [log rotation](#log-rotation). When enabled, files that exceed `max_size` are rotated and optionally compressed. | `false` | |
There was a problem hiding this comment.
@SagiROosto can you answer this for @patrick-stephens ?
| | `max_files` | Maximum number of rotated files to retain per output file. Oldest files are deleted first. Must be `1` or greater. Only applies when `files_rotation` is enabled. | `7` | | ||
| | `max_size` | Maximum size of the active output file before rotation is triggered. Supports size suffixes: `k` (kilobytes), `m` (megabytes), `g` (gigabytes). Only applies when `files_rotation` is enabled. | `100m` | |
There was a problem hiding this comment.
@SagiROosto can you answer this for @patrick-stephens ?
|
|
||
| The File output plugin supports size-based log rotation. | ||
|
|
||
| When `files_rotation` is enabled, the plugin monitors the size of each output file. Once a file exceeds `max_size`, the plugin rotates the file by renaming it with a timestamp suffix in the format `<filename>.<YYYYMMDD_HHMMSS_XXXXXXXX>`. The `YYYYMMDD_HHMMSS` is timestamp of the time when the rotation occurred, and `XXXXXXXX` is a random identifier to guarantee unique filenames if multiple rotations happen within the same second. |
There was a problem hiding this comment.
@SagiROosto can you answer this for @patrick-stephens ?
3 participants
This is the docs PR for fluent/fluent-bit#11352 about
out_filelogrotate featureSummary by CodeRabbit