Skip to content

improvement: ephemeral environment deploy testing for Renovate PRs #1576

Description

@ixxeL2097

Context

Currently, Renovate PRs that update Helm chart versions or image tags are auto-merged (patch/minor) without any functional validation. If a chart upgrade contains a breaking change, it only surfaces after merge and ArgoCD sync — i.e. in production.

Goal

Implement ephemeral environment testing on Renovate PRs before merge to:

  • Deploy the proposed chart/image change to a short-lived namespace or cluster
  • Run smoke tests / health checks against the ephemeral deployment
  • Report pass/fail back to the PR as a required status check
  • Block platformAutomerge until the check passes
  • Tear down the ephemeral environment after the check

Proposed approach

  1. Trigger: GitHub Actions workflow on pull_request targeting main, filtered on paths gitops/manifests/**
  2. Ephemeral namespace: create renovate-test-<pr-number> namespace in the genmachine cluster (via kubectl or ArgoCD CLI)
  3. Deploy: helm install the changed chart with the PR values against the ephemeral namespace
  4. Validate: run a configurable health-check (e.g. kubectl rollout status, HTTP probe, or a dedicated test job)
  5. Cleanup: delete the namespace on workflow completion (success or failure)
  6. Status check: set as required in branch protection to block Renovate automerge

Considerations

  • Needs cluster access from GitHub Actions (kubeconfig secret or OIDC)
  • Renovate automerge should be gated on this check passing (requiredStatusChecks)
  • Some charts (storage, CRDs) may need special handling or exclusion
  • Cost: ephemeral namespaces are cheap but CI time increases

Tasks

  • Design the GHA workflow for ephemeral deploy
  • Provision cluster access for GHA (OIDC or kubeconfig via Vault)
  • Implement namespace lifecycle (create / deploy / health-check / delete)
  • Add required status check to branch protection
  • Configure Renovate automergeStrategy to respect the check
  • Document the flow and exclusion list for non-testable charts

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions