Skip to content

Update file_storage to write state to subdirectory#2184

Merged
jefchien merged 3 commits into
mainfrom
fix-filestorage-path
Jul 9, 2026
Merged

Update file_storage to write state to subdirectory#2184
jefchien merged 3 commits into
mainfrom
fix-filestorage-path

Conversation

@jefchien

@jefchien jefchien commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Description of the issue

The logfile plugin's cleanupStateFolder() globs the state directory (logs/state/*) and deletes any file it cannot parse as its expected two-line offset format. When the journald receiver uses the file_storage extension to persist state in the same directory, the BoltDB files end up being deleted by the state directory cleanup.

Description of changes

Moves the file_storage directory to logs/state/otel to prevent this conflict. Sets the Compaction.Directory to match the storage directory for consistency.

License

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Tests

Tested on EC2 with both logfile and journald configured. The cleanup is non-deterministic, but is possible if the journald receiver is delayed during startup.

  {
    "agent": {
      "debug": true
    },
    "logs": {
      "logs_collected": {
        "files": {
          "collect_list": [
            {
              "file_path": "/var/log/messages",
              "log_group_name": "/test/v1-logs"
            }
          ]
        },
        "journald": {
          "collect_list": [
            {
              "log_group_name": "/test/journald",
              "units": ["sshd"]
            }
          ]
        }
      }
    }
  }

Before

$ ls -la /opt/aws/amazon-cloudwatch-agent/logs/state/
total 20
drwxr-xr-x. 2 root root    42 Jul  2 19:50 .
drwxr-xr-x. 3 root root    90 Jul  2 19:50 ..
-rw-------. 1 root root 32768 Jul  2 19:50 receiver_journald_journald_0

After

$ sudo find /opt/aws/amazon-cloudwatch-agent/logs/state/ -ls
 41944463   0 drwxr-xr-x   3 root root   18 Jul  2 20:07 /opt/aws/amazon-cloudwatch-agent/logs/state/
   491122   0 drwxr-x---   2 root root   42 Jul  2 20:07 /opt/aws/amazon-cloudwatch-agent/logs/state/otel
   671148  20 -rw-------   1 root root 32768 Jul  2 20:07 /opt/aws/amazon-cloudwatch-agent/logs/state/otel/receiver_journald_journald_0

Requirements

Before commiting your code, please do the following steps.

  1. Run make fmt and make fmt-sh
  2. Run make lint

Integration Tests

To run integration tests against this PR, add the ready for testing label.

@jefchien jefchien requested a review from a team as a code owner July 2, 2026 20:29
@jefchien jefchien added the ready for testing Indicates this PR is ready for integration tests to run label Jul 6, 2026
@Paamicky Paamicky requested review from Paamicky and removed request for Paamicky July 6, 2026 14:40
@jefchien jefchien merged commit efeca8f into main Jul 9, 2026
403 of 407 checks passed
@jefchien jefchien deleted the fix-filestorage-path branch July 9, 2026 17:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready for testing Indicates this PR is ready for integration tests to run

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants