-
Notifications
You must be signed in to change notification settings - Fork 31
test(e2e): e2e k8s environment tests using chainsaw #365
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,100 @@ | ||
| # Copyright 2026 NVIDIA CORPORATION | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
|
|
||
| name: E2E | ||
|
|
||
| on: | ||
| workflow_dispatch: | ||
| inputs: | ||
| chainsaw_version: | ||
| description: "Chainsaw version to install (e.g. v0.2.12)" | ||
| required: false | ||
| default: "latest" | ||
|
|
||
| env: | ||
| KIND_CLUSTER: topograph-e2e | ||
| IMAGE_REPO: ghcr.io/nvidia/topograph | ||
| CHAINSAW_VERSION: ${{ github.event.inputs.chainsaw_version || 'latest' }} | ||
|
|
||
| jobs: | ||
| e2e: | ||
| name: Chainsaw E2E | ||
| runs-on: ubuntu-latest | ||
| permissions: | ||
| contents: read | ||
|
|
||
| steps: | ||
| - uses: actions/checkout@v5 | ||
|
|
||
| - name: Set up Go | ||
| uses: actions/setup-go@v6 | ||
| with: | ||
| go-version: '1.25.11' | ||
|
|
||
| - name: Install kind | ||
| run: go install sigs.k8s.io/kind@latest | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
|
||
|
|
||
| - name: Install Chainsaw | ||
| run: | | ||
| if [ "$CHAINSAW_VERSION" = "latest" ]; then | ||
| TAG=$(curl -s https://api.github.com/repos/kyverno/chainsaw/releases/latest \ | ||
| | grep '"tag_name"' | cut -d'"' -f4) | ||
| else | ||
| TAG="$CHAINSAW_VERSION" | ||
| fi | ||
| echo "Installing Chainsaw $TAG" | ||
| BASE_URL="https://github.com/kyverno/chainsaw/releases/download/${TAG}" | ||
| curl -fsSL "${BASE_URL}/chainsaw_linux_amd64.tar.gz" -o chainsaw.tar.gz | ||
| curl -fsSL "${BASE_URL}/chainsaw_checksums.txt" -o chainsaw_checksums.txt | ||
| grep "chainsaw_linux_amd64.tar.gz" chainsaw_checksums.txt | sha256sum -c - | ||
| tar xz -f chainsaw.tar.gz chainsaw | ||
| sudo mv chainsaw /usr/local/bin/ | ||
| rm -f chainsaw.tar.gz chainsaw_checksums.txt | ||
| chainsaw version | ||
|
|
||
| - name: Create kind cluster | ||
| run: | | ||
| kind create cluster \ | ||
| --name "$KIND_CLUSTER" \ | ||
| --config tests/chainsaw/kind-config.yaml \ | ||
| --wait 120s | ||
|
|
||
| - name: Build Linux/amd64 image | ||
| run: make build-linux-amd64 | ||
|
|
||
| - name: Build container image | ||
| env: | ||
| GOOS: linux | ||
| GOARCH: amd64 | ||
| run: | | ||
| # Use the short commit SHA as the image tag: always a valid Docker tag, | ||
| # works regardless of branch naming conventions. | ||
| IMAGE_TAG=$(git rev-parse --short HEAD) | ||
| make image-build IMAGE_TAG="$IMAGE_TAG" | ||
| echo "IMAGE_TAG=$IMAGE_TAG" >> "$GITHUB_ENV" | ||
|
|
||
| - name: Load image into kind | ||
| run: | | ||
| kind load docker-image "${IMAGE_REPO}:${IMAGE_TAG}" \ | ||
| --name "$KIND_CLUSTER" | ||
|
|
||
| - name: Run E2E tests | ||
| env: | ||
| TOPOGRAPH_IMAGE_REPO: ${{ env.IMAGE_REPO }} | ||
| TOPOGRAPH_IMAGE_PULL_POLICY: Never | ||
| run: | | ||
| make e2e E2E_IMAGE_TAG="$IMAGE_TAG" | ||
|
|
||
| - name: Collect diagnostic logs on failure | ||
| if: failure() | ||
| run: | | ||
| echo "=== kind nodes ===" | ||
| kubectl get nodes -o wide | ||
| echo "=== all pods ===" | ||
| kubectl get pods -A -o wide | ||
| echo "=== recent events ===" | ||
| kubectl get events -A --sort-by='.lastTimestamp' | tail -50 | ||
|
|
||
| - name: Delete kind cluster | ||
| if: always() | ||
| run: kind delete cluster --name "$KIND_CLUSTER" | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -43,6 +43,7 @@ charts/topograph/ # Helm chart for all Kubernetes components; tests/ holds t | |
| CHANGELOG.md # Release history (Keep a Changelog format); update [Unreleased] for user-facing PRs | ||
| docs/ # Public-facing docs — overview.md, architecture.md, api.md + providers/, engines/, reference/ subdirectories | ||
| tests/models/ # YAML simulation fixtures | ||
| tests/chainsaw/ # Chainsaw E2E test suites (label-application, label-truncation, node-observer, slinky) | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The suite list names |
||
| config/ # Sample topograph-config.yaml | ||
| scripts/ # Build scripts (deb, rpm, SSL, clean) | ||
| localdev/ # Developer-local workspace — not tracked; personal scratch files | ||
|
|
@@ -95,6 +96,20 @@ make coverage # human-readable per-package summary | |
|
|
||
| Run `make qualify` before pushing. The individual targets are available if you want to run a single check during iteration. Run `make chart-test` when you change `charts/topograph/`; CI runs it on every workflow trigger. | ||
|
|
||
| ### E2E tests (Chainsaw) | ||
|
|
||
| Chainsaw conformance tests live in `tests/chainsaw/` and exercise the full Helm deploy → generate → assert cycle against a real cluster. | ||
|
|
||
| ```bash | ||
| make e2e-local # build image, create kind cluster, run all suites, delete cluster | ||
| make kind-load KIND_CLUSTER=<name> # load image into an existing kind cluster (run before make e2e) | ||
| make e2e # run suites against current KUBECONFIG context | ||
| ``` | ||
|
|
||
| `make e2e` uses `E2E_IMAGE_TAG` (defaults to the short commit SHA) as the image tag. For a local kind cluster, run `make image-build && make kind-load KIND_CLUSTER=<name>` before each `make e2e` — the tag changes with every commit, so both steps are needed after any new commit. Prerequisites: `chainsaw`, `kind`, `helm`, `kubectl`, `docker`. See `tests/chainsaw/README.md` for details. | ||
|
|
||
| These tests are triggered manually via `.github/workflows/e2e.yml` (`workflow_dispatch`). Run them before merging changes to the Helm chart, Node Observer, or engine output. | ||
|
|
||
| ### Coverage policy | ||
|
|
||
| From `codecov.yml`: | ||
|
|
@@ -109,6 +124,7 @@ Coverage checks run on pull requests. A drop below target with no matching uplif | |
| - `.github/workflows/chart-test.yaml` — Helm chart lint + helm-unittest suites (`make chart-test`) on every push and PR | ||
| - `.github/workflows/docker.yml` — container image build (manual trigger) | ||
| - `.github/workflows/helm-release.yaml` — Helm chart release (manual trigger) | ||
| - `.github/workflows/e2e.yml` — Chainsaw E2E suite against a kind cluster (manual trigger via `workflow_dispatch`) | ||
|
|
||
| ### Deployment surfaces | ||
|
|
||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -23,7 +23,7 @@ OUTPUT_DIR := ./bin | |
|
|
||
| IMAGE_REPO ?=ghcr.io/nvidia/topograph | ||
| GIT_REF ?=$(shell git rev-parse --abbrev-ref HEAD) | ||
| IMAGE_TAG ?=$(GIT_REF) | ||
| IMAGE_TAG ?=$(shell git rev-parse --short HEAD) | ||
|
|
||
| .PHONY: build | ||
| build: | ||
|
|
@@ -102,7 +102,7 @@ coverage: test | |
|
|
||
| .PHONY: image-build | ||
| image-build: | ||
| $(DOCKER_BIN) build --build-arg TARGETOS=$(GOOS) --build-arg TARGETARCH=$(GOARCH) -t $(IMAGE_REPO):$(IMAGE_TAG) -f ./Dockerfile . | ||
| $(DOCKER_BIN) build --build-arg TARGETOS=linux --build-arg TARGETARCH=$(GOARCH) -t $(IMAGE_REPO):$(IMAGE_TAG) -f ./Dockerfile . | ||
|
|
||
| .PHONY: image-push | ||
| image-push: image-build | ||
|
|
@@ -115,6 +115,52 @@ docker-buildx: | |
| $(DOCKER_BIN) buildx build --platform $(PLATFORMS) -t $(IMAGE_REPO):$(IMAGE_TAG) -f ./Dockerfile --push . | ||
| - $(DOCKER_BIN) buildx rm topograph-builder | ||
|
|
||
| CHAINSAW_BIN ?= chainsaw | ||
| KIND_CLUSTER ?= topograph-e2e | ||
| E2E_IMAGE_TAG ?= $(IMAGE_TAG) | ||
|
|
||
| # Check that chainsaw is installed; print install hint if not. | ||
| .PHONY: chainsaw-install | ||
| chainsaw-install: | ||
| @which $(CHAINSAW_BIN) >/dev/null 2>&1 || \ | ||
| (echo "chainsaw not found — install from https://kyverno.github.io/chainsaw/latest/quick-start/install/"; exit 1) | ||
|
|
||
| # Load the locally-built image into an existing kind cluster with the correct | ||
| # E2E_IMAGE_TAG. Use this before running make e2e against a local kind cluster: | ||
| # make kind-load KIND_CLUSTER=topograph-test && make e2e | ||
| .PHONY: kind-load | ||
| kind-load: | ||
| kind load docker-image $(IMAGE_REPO):$(E2E_IMAGE_TAG) --name $(KIND_CLUSTER) | ||
|
|
||
| # Run all Chainsaw E2E suites against the current KUBECONFIG context. | ||
| # For a pre-pushed registry image: set TOPOGRAPH_IMAGE_REPO and TOPOGRAPH_IMAGE_TAG. | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This comment points users at |
||
| # For a local kind cluster: run "make kind-load KIND_CLUSTER=<cluster>" first. | ||
| .PHONY: e2e | ||
| e2e: chainsaw-install | ||
| TOPOGRAPH_IMAGE_REPO=$(IMAGE_REPO) \ | ||
| TOPOGRAPH_IMAGE_TAG=$(E2E_IMAGE_TAG) \ | ||
| $(CHAINSAW_BIN) test --test-dir tests/chainsaw | ||
|
|
||
| # Build the image, create a 4-worker kind cluster, load the image, run all | ||
| # Chainsaw suites, and destroy the cluster. Requires kind and chainsaw. | ||
| .PHONY: e2e-local | ||
| e2e-local: chainsaw-install image-build | ||
| kind create cluster --name $(KIND_CLUSTER) \ | ||
| --config tests/chainsaw/kind-config.yaml --wait 120s \ | ||
| || kind get clusters | grep -q "^$(KIND_CLUSTER)$$" | ||
| kind load docker-image $(IMAGE_REPO):$(E2E_IMAGE_TAG) --name $(KIND_CLUSTER) | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This |
||
| KUBECONFIG_FILE=$$(mktemp) && \ | ||
| kind export kubeconfig --name $(KIND_CLUSTER) --kubeconfig $$KUBECONFIG_FILE && \ | ||
| KUBECONFIG="$$KUBECONFIG_FILE" \ | ||
| TOPOGRAPH_IMAGE_REPO=$(IMAGE_REPO) \ | ||
| TOPOGRAPH_IMAGE_TAG=$(E2E_IMAGE_TAG) \ | ||
| TOPOGRAPH_IMAGE_PULL_POLICY=Never \ | ||
| $(CHAINSAW_BIN) test --test-dir tests/chainsaw; \ | ||
| E2E_STATUS=$$?; \ | ||
| rm -f $$KUBECONFIG_FILE; \ | ||
| kind delete cluster --name $(KIND_CLUSTER); \ | ||
| exit $$E2E_STATUS | ||
|
|
||
| .PHONY: ssl | ||
| ssl: | ||
| SSL_DIR=ssl ./scripts/configure-ssl.sh | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,14 @@ | ||
| apiVersion: chainsaw.kyverno.io/v1alpha1 | ||
| kind: Configuration | ||
| metadata: | ||
| name: chainsaw | ||
| spec: | ||
| timeouts: | ||
| apply: 120s | ||
| assert: 90s | ||
| cleanup: 60s | ||
| delete: 60s | ||
| error: 30s | ||
| exec: 120s | ||
| fullName: true | ||
| parallel: 1 | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Chainsaw only auto-discovers a file named
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Renamed it to .chainsaw.yaml now |
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This pin went stale mid-review: #395 has since bumped the repo to Go 1.26.5 (go.mod and all four pins in go.yml). On runners where toolchain auto-fetch is restricted this fails outright; everywhere else it silently downloads 1.26.5 anyway, so best to align it. My earlier 1.25.11 suggestion predates the #395 bump.