Skip to content

Stabilize flaky notification-webhook integration test#15043

Open
Maffooch wants to merge 1 commit into
bugfixfrom
flaky-webhook-integration-test
Open

Stabilize flaky notification-webhook integration test#15043
Maffooch wants to merge 1 commit into
bugfixfrom
flaky-webhook-integration-test

Conversation

@Maffooch

Copy link
Copy Markdown
Contributor

Problem

tests/notification_webhook_test.py has been failing on most PRs but passing on rerun. Root cause: test_add_notification_webhook set the webhook URL to the public https://httpbin.org/post, and DefectDojo synchronously pings the webhook URL on save (AddNotificationWebhooksView.process_form_test_webhooks_notificationrequests.request(..., timeout=10s)). Every run blocked up to 10s on a rate-limited, intermittently-unreachable external service.

The near-tautological assertions (... or is_text_present_on_page("Webhook")) also masked real failures, so only transient network hiccups ever surfaced.

Changes

  • docker-compose.override.integration_tests.yml — add the webhook.endpoint go-httpbin mock (same pinned image already used by the unit-test override).
  • .github/workflows/integration-tests.yml — add webhook.endpoint to the explicit docker compose up --no-deps service list (--no-deps ignores depends_on).
  • tests/notification_webhook_test.py
    • Point the webhook URL at http://webhook.endpoint:8080/post (local mock, no external egress); edit test resets the URL too so its save-time ping succeeds.
    • Replace escape-hatch assertions with honest ones: wait for the alert, assert no error alert, assert the specific success message.
    • Add an explicit wait_for_alert() helper instead of relying on the 1s implicit wait.

Verification

ln -sf docker-compose.override.integration_tests.yml docker-compose.override.yml
docker compose up --no-deps -d postgres nginx celerybeat celeryworker mailhog uwsgi valkey webhook.endpoint
docker compose up --no-deps --exit-code-from initializer initializer
DD_INTEGRATION_TEST_FILENAME=tests/notification_webhook_test.py \
  docker compose up --no-deps --exit-code-from integration-tests integration-tests
docker compose down

The stronger assertions are now load-bearing: pointing the URL at a dead host correctly fails the test (the old version passed).

🤖 Generated with Claude Code

The Selenium test pinged the public httpbin.org on webhook save (a 10s
synchronous validation request), making it fail intermittently in CI.

Route the webhook URL to the local go-httpbin mock (already used by the
unit tests) by wiring webhook.endpoint into the integration-test stack and
the workflow's explicit service startup. Also replace the tautological
"or Webhook" assertions with explicit alert waits and specific success-message
checks so real failures actually fail.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

@mtesauro mtesauro left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants