diff --git a/pipeline/outputs/file.md b/pipeline/outputs/file.md index 4b05cf381..85e36f4e9 100644 --- a/pipeline/outputs/file.md +++ b/pipeline/outputs/file.md @@ -11,7 +11,11 @@ The _File_ output plugin lets you write the data received through the input plug | Key | Description | Default | | :--- | :--- | :--- | | `file` | Set filename to store the records. If not set, the filename will be the `tag` associated with the records. | _none_ | +| `files_rotation` | Enable size-based [log rotation](#log-rotation). When enabled, files that exceed `max_size` are rotated and optionally compressed. | `false` | | `format` | The [format](#format) of the file content. | _none_ | +| `gzip` | Compress rotated files using gzip. Only applies when `files_rotation` is enabled. | `true` | +| `max_files` | Maximum number of rotated files to retain per output file. Oldest files are deleted first. Must be `1` or greater. Only applies when `files_rotation` is enabled. | `7` | +| `max_size` | Maximum size of the active output file before rotation is triggered. Supports size suffixes: `k` (kilobytes), `m` (megabytes), `g` (gigabytes). Only applies when `files_rotation` is enabled. | `100m` | | `mkdir` | Recursively create output directory if it doesn't exist. Permissions set to `0755`. | `false` | | `path` | Directory path to store files. If not set, Fluent Bit will write the files in its own working directory. | _none_ | | `workers` | The number of [workers](../../administration/multithreading.md#outputs) to perform flush operations for this output. | `1` | @@ -112,6 +116,64 @@ You will get the following output: 1564462620.000254 used=1045448 free=31760160 total=32805608 ``` +## Log rotation + +The File output plugin supports size-based log rotation. + +When `files_rotation` is enabled, the plugin monitors the size of each output file. Once a file exceeds `max_size`, the plugin rotates the file by renaming it with a timestamp suffix in the format `.`. The `YYYYMMDD_HHMMSS` is timestamp of the time when the rotation occurred, and `XXXXXXXX` is a random identifier to guarantee unique filenames if multiple rotations happen within the same second. + +If `gzip` is enabled (the default), rotated files are compressed with gzip and stored with an additional `.gz` extension (for example, `cpu.log.20260512_134500_a1b2c3d4.gz`). + +The plugin retains up to `max_files` rotated files per output file. When the limit is reached, the oldest rotated files are deleted automatically. + +Log rotation works with all supported output [formats](#format): `plain`, `CSV`, `LTSV`, `template`, and `msgpack`. File operations are thread-safe, so rotation can be used alongside multiple [workers](../../administration/multithreading.md#outputs). + +### Log rotation example + +The following configuration writes CPU metrics to file with rotation enabled. Files are rotated at 50 MB and the five most recent rotated files are retained with gzip compression: + +{% tabs %} +{% tab title="fluent-bit.yaml" %} + +```yaml +pipeline: + inputs: + - name: cpu + tag: cpu + + outputs: + - name: file + match: '*' + path: /var/log/fluent-bit + file: cpu.log + files_rotation: true + max_size: 50m + max_files: 5 + gzip: true +``` + +{% endtab %} +{% tab title="fluent-bit.conf" %} + +```text +[INPUT] + Name cpu + Tag cpu + +[OUTPUT] + Name file + Match * + Path /var/log/fluent-bit + File cpu.log + Files_Rotation true + Max_Size 50m + Max_Files 5 + Gzip true +``` + +{% endtab %} +{% endtabs %} + ## Get started You can run the plugin from the command line or through the configuration file.