Skip to content

Log errors returned when sending webhook alerts#1182

Merged
msafwankarim merged 1 commit into
stakater:masterfrom
nikolauspschuetz:fix/alert-swallowed-send-errors-949
Jul 10, 2026
Merged

Log errors returned when sending webhook alerts#1182
msafwankarim merged 1 commit into
stakater:masterfrom
nikolauspschuetz:fix/alert-swallowed-send-errors-949

Conversation

@nikolauspschuetz

Copy link
Copy Markdown
Contributor

Closes #949.

Problem

With ALERT_ON_RELOAD=true and a webhook sink (e.g. ALERT_SINK=teams), a failing webhook produced no output at all — not even at logLevel: trace. The alert simply didn't arrive and there was nothing to debug.

Cause

SendWebhookAlert calls sendSlackAlert / sendTeamsAlert / sendGoogleChatAlert / sendRawWebhookAlert, each of which returns []error, but the return values were discarded. A non-2xx response (or a transport error) was therefore silently swallowed.

Fix

Capture the returned errors and log each with logrus.Errorf("Error sending alert: %s", ...) — exactly as @Felix-Stakater suggested on the issue. A misconfigured or failing webhook now surfaces a clear error log:

level=error msg="Error sending alert: error sending msg. status: 500 Internal Server Error"

Tests

Added TestSendWebhookAlert_LogsSendErrors (internal/pkg/alerts/alert_test.go): stands up an httptest server that returns 500, points a Teams sink at it, and asserts the error is logged (via a logrus test hook). It fails without this change and passes with it. go build ./..., gofmt, and the new test are green.

Developed with AI assistance (Claude Code); I directed the change and verified build/test locally.

The alert send functions (Slack/Teams/GChat/raw) return `[]error`, but
`SendWebhookAlert` discarded them, so a failing webhook — e.g. a Teams
alert returning a non-2xx status — produced no output at all, even at
trace level (stakater#949).

Capture the returned errors and log each with `logrus.Errorf`, as
suggested by the maintainer on the issue. Adds a regression test that
drives a failing (500) webhook and asserts the error is logged.

Closes stakater#949

Assisted-by: Claude Code (Anthropic, Opus 4.x)
@msafwankarim
msafwankarim merged commit a910242 into stakater:master Jul 10, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] Teams alert not send

2 participants