Skip to content

feat: add aws cloud watch logs multi cluster support#110

Open
chalindukodikara wants to merge 5 commits into
openchoreo:mainfrom
chalindukodikara:issue-3149_add_aws_cloud_watch_logs_multi_cluster_support
Open

feat: add aws cloud watch logs multi cluster support#110
chalindukodikara wants to merge 5 commits into
openchoreo:mainfrom
chalindukodikara:issue-3149_add_aws_cloud_watch_logs_multi_cluster_support

Conversation

@chalindukodikara
Copy link
Copy Markdown
Contributor

@chalindukodikara chalindukodikara commented May 15, 2026

Purpose

$subject

Approach

Summarize the solution and implementation details.

Related Issues

Include any related issues that are resolved by this PR.

Checklist

  • Tests added or updated (unit, integration, etc.)
  • Samples updated (if applicable)

Remarks

Add any additional context, known issues, or TODOs related to this PR.

Summary by CodeRabbit

  • New Features

    • Added multi-cluster deployment topology support with topology-specific Helm configurations.
    • Expanded Pod Identity Agent setup with verification steps and resource-specific installation commands.
    • Added EventBridge alerting setup with connection-based configuration.
  • Documentation

    • Updated installation and alerting workflows for improved clarity.
    • Added configuration guidance for EKS Pod Identity vs IRSA vs static credentials.
    • Refreshed troubleshooting and configuration reference sections.
  • Bug Fixes

    • Improved log group configuration scoping and validation logic.

Review Change Stack

Signed-off-by: Chalindu Kodikara <chalindumkodikara@gmail.com>
Signed-off-by: Chalindu Kodikara <chalindumkodikara@gmail.com>
Signed-off-by: Chalindu Kodikara <chalindumkodikara@gmail.com>
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 15, 2026

Warning

Rate limit exceeded

@chalindukodikara has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 25 minutes and 6 seconds before requesting another review.

You’ve run out of usage credits. Purchase more in the billing tab.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 1075f884-b32e-4720-b0eb-23c4b111c88c

📥 Commits

Reviewing files that changed from the base of the PR and between ee27b49 and 239cd9a.

📒 Files selected for processing (3)
  • observability-logs-cloudwatch/README.md
  • observability-logs-cloudwatch/helm/Chart.yaml
  • observability-logs-cloudwatch/internal/config.go
📝 Walkthrough

Walkthrough

This PR consolidates multi-cluster logging by replacing per-cluster naming with a centralized log group naming scheme. ClusterName is removed from configuration, replaced by direct LogGroupName specification. Log group paths are now derived from global.logGroupPrefix (defaulting to /aws/containerinsights) with /application suffix, enabling all clusters to write to a shared application log group. Helm values, Go code, shell scripts, and documentation are updated consistently to support multi-cluster and single-cluster topologies.

Changes

Multi-Cluster Log Group Consolidation

Layer / File(s) Summary
Helm values and helpers: log group naming contract
helm/values.yaml, helm/templates/_helpers.tpl
global.logGroupPrefix introduced and centralized; logs-cloudwatch.logGroupName helper added to derive /application suffix; validation helper refactored with conditional component checks and credential validation; bridgeService disabled by default; adapter alerting enabled by default.
Helm validation template and configuration wiring
helm/templates/validate.yaml
New centralized validation template invokes the updated helper with full chart context, consolidating Helm validation into one place.
Helm template updates: log group and validation
helm/templates/adapter/configmap.yaml, helm/templates/cloudwatch-setup/job.yaml, helm/templates/aws-credentials/secret.yaml, helm/templates/cloudwatch-agent/post-install-hook.yaml
ConfigMap includes LOG_GROUP_NAME from helpers; setup job removes CLUSTER_NAME env var; post-install hook gains credential-injection patching and bridge-service creation; aws-credentials validation moved to centralized template.
Go config: LogGroupName introduction and parsing
internal/config.go, internal/config_test.go
Config struct removes ClusterName, adds LogGroupName; LoadConfig reads LOG_GROUP_PREFIX and LOG_GROUP_NAME, trims trailing slashes, derives default as <prefix>/application; SERVER_PORT validation strengthened to 1..65535 range; comprehensive test coverage for log-group derivation, override, and port validation.
CloudWatch client: direct log group naming
internal/cloudwatch/client.go, internal/cloudwatch/client_test.go
Client and Config structs replace cluster/prefix fields with single LogGroupName; NewClient and NewClientWithAWS populate logGroupName directly; applicationLogGroup helper simplified to return stored value.
Main entrypoint and client initialization
main.go
Logs LogGroupName instead of cluster/prefix; passes LogGroupName to CloudWatch client configuration.
Alert identity parsing: validation and error clarity
internal/cloudwatch/alerts.go, internal/cloudwatch/alerts_test.go, internal/cloudwatch/alerts_crud.go, internal/cloudwatch/alerts_crud_test.go, internal/cloudwatch/webhook.go
ParseAlertIdentityFromAlarmName validation strengthened with explicit part-count and marker-position checks; decoding errors wrapped with context; test coverage extended with stricter alarm name format validation; supporting test setup and formatting adjusted.
Setup script: remove CLUSTER_NAME requirement
init/setup-cloudwatch.sh
CLUSTER_NAME no longer required; log group paths use LOG_GROUP_PREFIX alone for application group; AWS_REGION remains required.
README: multi-cluster topology and deployment guidance
README.md
"Choose a deployment topology" section added for single-cluster, observability-plane, and data-plane topologies; architecture updated to show shared application log group; IAM permissions separated for EKS Pod Identity vs static credentials; installation instructions expanded with Pod Identity Agent verification, topology-specific Helm commands, and multi-cluster association tables; alerting documentation restructured with EventBridge setup and webhook configuration; troubleshooting and configuration reference refreshed; legacy k3d/kind content removed.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Suggested reviewers

  • akila-i
  • nilushancosta
  • LakshanSS

Poem

🐰 A cluster-free path now glows,
Where logs flow to one home, not many—
No names split by host,
Just names shared and true.
Multi-cluster dreams made simple!

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name Status Explanation Resolution
Description check ⚠️ Warning The PR description is largely incomplete, with template placeholders unfilled: Purpose shows '$subject' placeholder, Approach/Related Issues/Remarks sections contain only template text, and Checklist items are unchecked with no context provided. Fill in Purpose with concrete problem statement, Approach with implementation details, Related Issues with issue references, complete Checklist items, and add any relevant Remarks or context about the changes.
Docstring Coverage ⚠️ Warning Docstring coverage is 19.35% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (3 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately describes the main feature: adding AWS CloudWatch Logs multi-cluster support.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented May 15, 2026

Codecov Report

❌ Patch coverage is 92.00000% with 4 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
...lity-logs-cloudwatch/internal/cloudwatch/alerts.go 80.00% 1 Missing and 1 partial ⚠️
observability-logs-cloudwatch/main.go 0.00% 2 Missing ⚠️

📢 Thoughts on this report? Let us know!

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
observability-logs-cloudwatch/README.md (1)

143-163: ⚠️ Potential issue | 🟠 Major | 🏗️ Heavy lift

Document that IAM policy ARNs must be updated if global.logGroupPrefix is customized.

The IAM policy examples hardcode the log group resource as arn:aws:logs:<region>:<account-id>:log-group:/aws/containerinsights/application:*, which assumes the default global.logGroupPrefix value of /aws/containerinsights.

If a user customizes global.logGroupPrefix to a different value, these IAM policy resource ARNs must also be updated to match <custom-prefix>/application, otherwise all adapter operations will fail with permission errors.

📝 Suggested clarification

Add a note after line 143:

Replace:

- `<region>` with your AWS region.
- `<account-id>` with your AWS account ID.
- `/aws/containerinsights/application` with `<your-custom-prefix>/application` if you customize `global.logGroupPrefix`.

Apply the same note to the Setup Job IAM policy section.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@observability-logs-cloudwatch/README.md` around lines 143 - 163, The README's
IAM policy examples assume the default global.logGroupPrefix
(/aws/containerinsights) so update the documentation to warn that if users
customize global.logGroupPrefix they must also update the IAM policy resource
ARNs to match <your-custom-prefix>/application; add a short note (referencing
global.logGroupPrefix and the IAM policy examples shown around the existing
"LogsScoped" statement) that tells readers to replace <region>, <account-id>,
and replace /aws/containerinsights/application with
<your-custom-prefix>/application when customized, and add the same note to the
"Setup Job IAM policy" section.
🧹 Nitpick comments (1)
observability-logs-cloudwatch/README.md (1)

443-450: ⚡ Quick win

Add security guidance for handling static AWS credentials.

The export commands show AWS credentials in plaintext without security warnings. Users copying these examples may inadvertently persist credentials in shell history or commit them to version control.

🔒 Suggested security note

Add a warning before the export commands:

> **Security note:** Avoid hardcoding credentials in scripts or committing them to version control. Use a secure credential manager or read from a secrets vault. These exports will be stored in your shell history unless `HISTCONTROL=ignorespace` is set and commands are prefixed with a space.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@observability-logs-cloudwatch/README.md` around lines 443 - 450, Add a
security note before the export block that warns users not to hardcode or expose
AWS credentials (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_REGION,
WEBHOOK_SHARED_SECRET) in scripts or version control and recommend using a
credential manager, secrets vault, or environment-specific secret injection;
mention shell-history risk (HISTCONTROL=ignorespace and prefixing with a space)
and advise alternatives like AWS CLI credential files, IAM roles, or secrets
managers to securely provide these values.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@observability-logs-cloudwatch/internal/config.go`:
- Around line 78-84: The current atoi-based check accepts signed or
non-canonical forms like "+9098" or "00080"; update validation to first ensure
serverPort is a canonical decimal integer (no leading + or leading zeros) e.g.
by matching serverPort against a regex such as ^[1-9][0-9]{0,4}$ (which also
prevents "0" and limits length), then convert with strconv.Atoi (or
strconv.ParseInt) and keep the existing range check on port; ensure the change
is applied where serverPort is parsed (the strconv.Atoi call and subsequent port
range check) so invalid non-canonical strings are rejected.

In `@observability-logs-cloudwatch/README.md`:
- Line 230: The README currently uses invalid shell syntax `export
AWS_REGION=<your-aws-region>`; update these export lines (the AWS_REGION export
and the other export commands referenced around the README) to use valid shell
syntax by either providing a concrete example (e.g., use a real region string)
or a quoted placeholder (e.g., export AWS_REGION="your-aws-region"), and apply
the same change to the similar export lines mentioned (lines near 445, 448, 449)
so angle brackets are removed and the shell will not treat them as redirection.

---

Outside diff comments:
In `@observability-logs-cloudwatch/README.md`:
- Around line 143-163: The README's IAM policy examples assume the default
global.logGroupPrefix (/aws/containerinsights) so update the documentation to
warn that if users customize global.logGroupPrefix they must also update the IAM
policy resource ARNs to match <your-custom-prefix>/application; add a short note
(referencing global.logGroupPrefix and the IAM policy examples shown around the
existing "LogsScoped" statement) that tells readers to replace <region>,
<account-id>, and replace /aws/containerinsights/application with
<your-custom-prefix>/application when customized, and add the same note to the
"Setup Job IAM policy" section.

---

Nitpick comments:
In `@observability-logs-cloudwatch/README.md`:
- Around line 443-450: Add a security note before the export block that warns
users not to hardcode or expose AWS credentials (AWS_ACCESS_KEY_ID,
AWS_SECRET_ACCESS_KEY, AWS_REGION, WEBHOOK_SHARED_SECRET) in scripts or version
control and recommend using a credential manager, secrets vault, or
environment-specific secret injection; mention shell-history risk
(HISTCONTROL=ignorespace and prefixing with a space) and advise alternatives
like AWS CLI credential files, IAM roles, or secrets managers to securely
provide these values.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: f70cebde-5c72-46a0-9a35-1d46ff6fb5b2

📥 Commits

Reviewing files that changed from the base of the PR and between 541a687 and ee27b49.

⛔ Files ignored due to path filters (8)
  • observability-logs-cloudwatch/images/1-rules-creation.png is excluded by !**/*.png
  • observability-logs-cloudwatch/images/2-rules-creation-config.png is excluded by !**/*.png
  • observability-logs-cloudwatch/images/3-rules-creation-add-api-destination.png is excluded by !**/*.png
  • observability-logs-cloudwatch/images/4-create-api-destination.png is excluded by !**/*.png
  • observability-logs-cloudwatch/images/5-create-api-destination-add-connection.png is excluded by !**/*.png
  • observability-logs-cloudwatch/images/6-create-api-destination-add-api-key.png is excluded by !**/*.png
  • observability-logs-cloudwatch/images/7-rules-creation-select-api-destination.png is excluded by !**/*.png
  • observability-logs-cloudwatch/images/alarm-triggered.png is excluded by !**/*.png
📒 Files selected for processing (19)
  • observability-logs-cloudwatch/README.md
  • observability-logs-cloudwatch/helm/templates/_helpers.tpl
  • observability-logs-cloudwatch/helm/templates/adapter/configmap.yaml
  • observability-logs-cloudwatch/helm/templates/aws-credentials/secret.yaml
  • observability-logs-cloudwatch/helm/templates/cloudwatch-agent/post-install-hook.yaml
  • observability-logs-cloudwatch/helm/templates/cloudwatch-setup/job.yaml
  • observability-logs-cloudwatch/helm/templates/validate.yaml
  • observability-logs-cloudwatch/helm/values.yaml
  • observability-logs-cloudwatch/init/setup-cloudwatch.sh
  • observability-logs-cloudwatch/internal/cloudwatch/alerts.go
  • observability-logs-cloudwatch/internal/cloudwatch/alerts_crud.go
  • observability-logs-cloudwatch/internal/cloudwatch/alerts_crud_test.go
  • observability-logs-cloudwatch/internal/cloudwatch/alerts_test.go
  • observability-logs-cloudwatch/internal/cloudwatch/client.go
  • observability-logs-cloudwatch/internal/cloudwatch/client_test.go
  • observability-logs-cloudwatch/internal/cloudwatch/webhook.go
  • observability-logs-cloudwatch/internal/config.go
  • observability-logs-cloudwatch/internal/config_test.go
  • observability-logs-cloudwatch/main.go
💤 Files with no reviewable changes (2)
  • observability-logs-cloudwatch/helm/templates/cloudwatch-setup/job.yaml
  • observability-logs-cloudwatch/helm/templates/aws-credentials/secret.yaml

Comment thread observability-logs-cloudwatch/internal/config.go
Comment thread observability-logs-cloudwatch/README.md
Signed-off-by: Chalindu Kodikara <chalindumkodikara@gmail.com>
Signed-off-by: Chalindu Kodikara <chalindumkodikara@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants