ci(e2e): add DVP-over-DVP matrix testing with parallel execution#1577

Draft

yachmenevas wants to merge 240 commits intomainfrom

feat/ci-e2e-matrix

Commits on Oct 15, 2025

ci(e2e): add DVP-over-DVP matrix testing with parallel execution
yachmenevas
committed
ci: drop artifacts/binaries from VCS; update .gitignore
yachmenevas
committed
ci(e2e): restore workflow, Taskfiles, charts, scripts and docs
yachmenevas
committed
dvcr: make templates nil-safe; remove temporary root values.yaml
yachmenevas
committed
ci: format YAML to satisfy prettier; drop temporary values.yaml
yachmenevas
committed
revert(templates/dvcr): drop E2E-related edits; keep tests via ModuleConfig/Taskfile
yachmenevas
committed
ci(e2e): remove secrets usage in if; check webhook in step script
yachmenevas
committed
ci(e2e): write kubeconfig via printf to avoid heredoc EOF issues
yachmenevas
committed
ci(e2e): resolve merge with main; keep full E2E workflow
yachmenevas
committed
ci(e2e): fix duplicate workflow_dispatch; prettier passes
yachmenevas
committed
ci(e2e): use env.E2E_K8S_URL in kubeconfig steps; drop report kubeconfig env
yachmenevas
committed
ci(e2e): build e2e-runner wrapper via Taskfile (tests/e2e task run:ci)
yachmenevas
committed
ci(e2e): run tests via tests/e2e Taskfile (run:ci) with TIMEOUT
yachmenevas
committed
ci(e2e): pass TEST_TIMEOUT and E2E_PREFIX to tests/e2e Taskfile run
yachmenevas
committed
ci(e2e): pass TIMEOUT (not TEST_TIMEOUT) to tests/e2e Taskfile
yachmenevas
committed
ci(e2e): write kubeconfig token without base64 decoding to avoid UTF-8 issues
yachmenevas
committed
ci(e2e): quote token in kubeconfig; set LABELS/FOCUS_JSON defaults for jq
yachmenevas
committed
ci(e2e): read JWT from secrets.K8S_CLUSTER_SECRET index 8 (kubeconfig)
yachmenevas
committed
ci(e2e): use azure/k8s-set-context@v4 for kubeconfig setup
yachmenevas
committed
ci(e2e): remove KUBECONFIG env var to use default ~/.kube/config
yachmenevas
committed

Commits on Oct 16, 2025

Commits on Oct 17, 2025

Commits on Oct 19, 2025

refactor(ci): remove unused setup-e2e-tools action and improve E2E workflow

Antony
authored and
Antony
committed
fix(ci): remove extra profiles from profiles.json

Antony
authored and
Antony
committed
refactor(ci): simplify cleanup and fix env variable issues

Antony
authored and
Antony
committed
fix(ci): improve profile config parsing to avoid env errors

Antony
authored and
Antony
committed
fix(ci): use correct ServiceAccount and improve cleanup error handling

Antony
authored and
Antony
committed

Commits on Oct 20, 2025

ci(e2e): build kubeconfig from E2E_NESTED_SA_SECRET without TOK var; drop k8s-set-context

Antony
authored and
Antony
committed
ci(e2e): call correct cleanup task (cleanup:cleanup:namespaces)

Antony
authored and
Antony
committed
ci(e2e): drop NFS server from infra and tasks; gate NFS config by flag

Antony
authored and
Antony
committed
ci(e2e): docker login dev-registry before pulling install image

Antony
authored and
Antony
committed
ci(e2e): use REGISTRY_DOCKER_CFG for registry auth; drop explicit docker login

Antony
authored and
Antony
committed
ci(e2e): use deckhouse/modules-actions/setup for registry auth only

Antony
authored and
Antony
committed
ci(e2e): auth via modules-actions; add GHCR login; setup-task with token

Antony
authored and
Antony
committed
ci(e2e): add explicit registry pull check before bootstrap

Antony
authored and
Antony
committed
ci(e2e): replace Taskfile with minimal GH-oriented version; drop unused tasks/* and loop notify scripts; keep JSON profiles + modules-actions/ghcr auth; Ceph/SDS only

Antony
authored and
Antony
committed
ci(e2e): fix cleanup task; docker login to DEV_REGISTRY; setup-task input; lint YAML

Antony
authored and
Antony
committed

Commits on Oct 21, 2025

ci(e2e): remove GHCR login, move registry verification after docker login

Antony
authored and
Antony
committed
ci(e2e): use direct docker login instead of docker/login-action

Antony
authored and
Antony
committed
ci(e2e): use CE installer image instead of dev-registry for verification

Antony
authored and
Antony
committed
ci(e2e): use CE installer image in dhctl-bootstrap task

Antony
authored and
Antony
committed
ci(e2e): fix cleanup namespaces array init and reorder workflow steps for SSH key generation

Antony
authored and
Antony
committed
ci(e2e): fix workflow architecture - add proper SSH key generation and render-infra step

Antony
authored and
Antony
committed
ci(e2e): rely on render-infra ssh-gen dependency; remove explicit ssh:generate step

Antony
authored and
Antony
committed
ci(e2e): run values prepare before infra; remove intermediate base values creation

Antony
authored and
Antony
committed
fix(ci): align Taskfile dependencies with backup branch logic

Antony
authored and
Antony
committed
fix(ci): add sshPublicKey to cluster-config generation

Antony
authored and
Antony
committed
fix(ci): ensure deckhouse.registryDockerCfg is always a string in cluster-config

Antony
authored and
Antony
committed
fix(ci): remove quote from registryDockerCfg in cluster-config template

Antony
authored and
Antony
committed
fix(ci): remove registryDockerCfg field from cluster-config template

Antony
authored and
Antony
committed
fix(ci): restore registryDockerCfg config and fix cleanup namespaces

Antony
authored and
Antony
committed
fix(ci): use VALUES_TEMPLATE_FILE instead of explicit VALUES_FILE in dhctl-bootstrap

Antony
authored and
Antony
committed
fix(ci): use run:values:prepare to create proper values.yaml with dynamic namespace

Antony
authored and
Antony
committed
fix(ci): use VALUES_TEMPLATE_FILE consistently and remove undeploy-finalizers script call

Antony
authored and
Antony
committed
debug(ci): add debugging info for REGISTRY_DOCKER_CFG secret

Antony
authored and
Antony
committed
fix(ci): properly pass REGISTRY_DOCKER_CFG secret to workflow

Antony
authored and
Antony
committed
fix(ci): pass VALUES_FILE to dhctl-bootstrap task

Antony
authored and
Antony
committed
fix(ci): use dynamic namespace consistently across all steps

Antony
authored and
Antony
committed
cleanup: remove accidentally added submodules and docs

Antony
authored and
Antony
committed
fix(ci): synchronize registry for CE and fix submodule error

Antony
authored and
Antony
committed
fix(ci): remove registryDockerCfg for CE registry

Antony
authored and
Antony
committed
fix(ci): correct registry configuration for OSS cluster with CE dhctl

Antony
authored and
Antony
committed
fix(ci): disable upmeter module and fix virtualization tag

Antony
authored and
Antony
committed
fix(ci): remove duplicate upmeter ModuleConfig

Antony
authored and
Antony
committed
fix(ci): consolidate disabled modules and remove submodule

Antony
authored and
Antony
committed
fix(ci): completely remove virtualization-rnd directory

Antony
authored and
Antony
committed
fix(ci): disable checkout caching in GitHub Actions

Antony
authored and
Antony
committed
fix(ci): skip checksum verification in Deckhouse CLI installation

Antony
authored and
Antony
committed
fix(ci): restore disabled-modules.yaml and fix duplication

Antony
authored and
Antony
committed
fix(ci): restore configuration to match reference version

Antony
authored and
Antony
committed

Commits on Oct 22, 2025

fix(ci): switch from CE to OSS registry with new credentials

Antony
authored and
Antony
committed
fix(ci): restore token creation logic for nested kubeconfig

Antony
authored and
Antony
committed
feat(ci): add SDS module configuration for nested cluster

Antony
authored and
Antony
committed
feat(ci): add disk debugging and Ceph cluster configuration

Antony
authored and
Antony
committed
refactor(ci): remove old Ceph bootstrap approach

Antony
authored and
Antony
committed
fix(ci): optimize Git checkout to resolve cache issues

Antony
authored and
Antony
committed
fix(ci): install d8 via direct tarball to avoid checksum mismatch

Antony
authored and
Antony
committed
fix(ci): install d8 via installer with checksum disabled

Antony
authored and
Antony
committed
feat(ci): conditionally render Ceph/SDS by storageProfile

Antony
authored and
Antony
committed
refactor(ci): move storage CRs post-bootstrap and remove empty templates

Antony
authored and
Antony
committed
fix(ci): move disk debug before storage apply; add retries and --validate=false

Antony
authored and
Antony
committed
fix(ci): move disk debug step before storage configuration

Antony
authored and
Antony
committed
fix(ci): use d8 v ssh for reliable VM disk debugging with retries

Antony
authored and
Antony
committed
fix(ci): correct Ceph deviceFilter and add proper operator readiness checks

Antony
authored and
Antony
committed
fix(ci): make GA YAML valid; move operator-ceph ModuleConfig to file\n\n- Replace heredoc in workflow with file apply to avoid YAML parse errors\n- Add ci/dvp-e2e/manifests/storage/operator-ceph.ya…

Antony
authored and
Antony
committed
chore(ci): include Loop report scripts; add run link in profile cell\n\n- Restore loop_notify.py and loop_junit_notify.py from backup\n- Enhance loop_matrix_summary.py to embed GitHub run link on p…

Antony
authored and
Antony
committed
chore(ci): pass GitHub run URL to Loop matrix summary and link profile name

Antony
authored and
Antony
committed
fix(storage): enable SDS modules and wait for CRDs before applying resources\n\n- Workflow: enable sds-node-configurator and sds-replicated-volume, wait for CRDs, then apply sds.yaml\n- Taskfile: m…

Antony
authored and
Antony
committed
fix(ci): resolve YAML syntax error in workflow SDS module config\n\n- Replace heredoc with separate sds-modules.yaml file\n- Fixes 'Unexpected value apiVersion' error in GitHub Actions\n- YAML synt…

Antony
authored and
Antony
committed
fix(storage): use kubectl patch mc global for default StorageClass\n\n- Replace forbidden kubectl patch storageclass with kubectl patch mc global\n- Use JSON patch to set defaultClusterStorageClass…

Antony
authored and
Antony
committed
fix(sds): add retry logic and remove version from ModuleConfig\n\n- Add retry loop for sds-modules.yaml apply to handle 502 errors\n- Remove spec.version from ModuleConfig (no effect without settin…

Antony
authored and
Antony
committed
feat(ci): improve SDS CRD waiting and Ceph operator readiness checks\n\n- Add two-stage SDS CRD waiting: first check existence, then wait for Established\n- Increase Ceph namespace timeout from 30 …

Antony
authored and
Antony
committed

Commits on Oct 23, 2025

fix(storage): correct disk devices and simplify SDS logic

Antony
authored and
Antony
committed
fix(scripts): fix jq syntax error in get_profile_config.sh

Antony
authored and
Antony
committed
fix(taskfile): fix heredoc syntax in infra:attach-worker-disks task

Antony
authored and
Antony
committed
fix(taskfile): avoid 'workers: unbound variable' by safe array build

Antony
authored and
Antony
committed
fix(taskfile): add quotes around SDS_SC_NAME in storageclass check

Antony
authored and
Antony
committed
fix(ci): enable JUnit XML generation and reporting in E2E matrix workflow

Antony
authored and
Antony
committed
fix(taskfile): correct worker VM selector in infra:attach-worker-disks

Antony
authored and
Antony
committed
fix: apply critical storage and disk attachment fixes

Antony
authored and
Antony
committed
hotfix: fix Taskfile YAML syntax and VirtualDisk API structure

Antony
authored and
Antony
committed
feat: move disk attachment before bootstrap and use base SC

Antony
authored and
Antony
committed
fix: use VM name selector instead of node-group label for worker detection

Antony
authored and
Antony
committed
fix: move disk attachment after bootstrap to ensure VMs exist

Antony
authored and
Antony
committed
fix(ci): resolve SDS and Ceph storage configuration issues

Antony
authored and
Antony
committed
fix(ci): add SCSI bus rescan to activate hotplugged storage disks

Antony
authored and
Antony
committed
fix(ci): add sudo to SCSI rescan commands to fix permission denied errors

Antony
authored and
Antony
committed
feat(ci): implement blockDeviceRefs approach for VM disk attachment

Antony
authored and
Antony
committed
fix(ci): improve blockDeviceRefs task reliability

Antony
authored and
Antony
committed
fix(ci): use correct VM selector for worker nodes

Antony
authored and
Antony
committed
fix(ci): add debug info and validation for blockDeviceRefs patch

Antony
authored and
Antony
committed
fix(ci): use inline patch instead of file-based patch for blockDeviceRefs

Antony
authored and
Antony
committed

Commits on Oct 24, 2025

fix(ci): restore correct namespace naming for cleanup

Antony
authored and
Antony
committed
fix(ci): add VM startup verification after blockDeviceRefs modification

Antony
authored and
Antony
committed
revert(ci): switch back to hotplug for disk attachment

Antony
authored and
Antony
committed
fix(ci): fix E2E test issues with bootstrap and disk attachment

Antony
authored and
Antony
committed
fix(ci): remove forbidden dataDisk property from DVPClusterConfiguration

Antony
authored and
Antony
committed
fix: remove nodeGroups from DVPClusterConfiguration

Antony
authored and
Antony
committed
docs: add comment to cluster-config template

Antony
authored and
Antony
committed
fix(ci): add push trigger to E2E Matrix Tests workflow

Antony
authored and
Antony
committed
fix(ci): limit push trigger to feat/ci-e2e-matrix branch only

Antony
authored and
Antony
committed

Commits on Oct 25, 2025

refactor(ci): simplify storage configuration logic

Antony
authored and
Antony
committed
fix(ci): update storage configuration to use Ceph as default

Antony
authored and
Antony
committed
debug(ci): improve disk attachment debugging and reduce retry time

Antony
authored and
Antony
committed

Commits on Oct 26, 2025

debug(ci): improve PVC waiting logic and add debug environment variable

Antony
authored and
Antony
committed
debug(ci): optimize logging and add fallback for disk attachment

Antony
authored and
Antony
committed

Commits on Oct 27, 2025

feat: add ModulePullOverride for Ceph and SDS modules

Antony
authored and
Antony
committed
fix: improve disk attachment fallback logic and use dynamic storage class

Antony
authored and
Antony
committed
refactor: simplify disk attachment logic and remove debug env

Antony
authored and
Antony
committed
refactor: compactify jq query formatting in disk attachment logic

Antony
authored and
Antony
committed
revert: use hardcoded ceph storage class for disk attachment

Antony
authored and
Antony
committed
improve: add Deckhouse readiness check and better error handling

Antony
authored and
Antony
committed
feat(ci): add platform queue diagnostics for module deployment debugging

Antony
authored and
Antony
committed
fix(ci): improve queue diagnostics error handling and output

Antony
authored and
Antony
committed
fix(ci): replace d8 v ssh queue diagnostics with kubectl-based debugging

Antony
authored and
Antony
committed
feat(ci): execute d8 platform queue command via kubectl exec in deckhouse pod

Antony
authored and
Antony
committed
feat(scripts): add automatic d8 CLI installation on master VM for diagnostics

Antony
authored and
Antony
committed
feat(ci): install d8 CLI in deckhouse pod before queue diagnostics

Antony
authored and
Antony
committed

Commits on Oct 28, 2025

fix(ci): install tar and use official d8 CLI installation method

Antony
authored and
Antony
committed
fix(ci): support both Alpine and Ubuntu for d8 CLI installation

Antony
authored and
Antony
committed
ci: use runner d8 for nested (e2e-matrix.yml)

Antony
authored and
Antony
committed
ci: stabilize d8 queue + webhook wait (e2e-matrix.yml)

Antony
authored and
Antony
committed
ci: persist SSH key secret (Taskfile.yaml)

Antony
authored and
Antony
committed
scripts: nested diag: auto-fetch SSH key + ssh opts (nested_diag.sh)

Antony
authored and
Antony
committed
ci: pin d8 v0.13.2 on runner (e2e-matrix.yml)

Antony
authored and
Antony
committed
scripts: nested_diag.sh: ensure queue-capable d8 inside VM

Antony
authored and
Antony
committed
scripts: nested_diag.sh: use admin.conf for d8 queue

Antony
authored and
Antony
committed
scripts: nested_diag.sh: fallback to kubectl config view for kubeconfig

Antony
authored and
Antony
committed
scripts: nested_diag.sh: robust kubeconfig for d8 (avoid 127.0.0.1:6445, use SA token)

Antony
authored and
Antony
committed
ci(e2e-matrix): fix Ceph and storage setup; import GH SSH keys

Antony
authored and
Antony
committed
ci(e2e-matrix): remove accidental embedded repo from index

Antony
authored and
Antony
committed
ci(e2e-matrix): pin TARGET_STORAGE_CLASS to Ceph immediate SC

Antony
authored and
Antony
committed
ci(e2e-matrix): drop accidentally staged embedded repo

Antony
authored and
Antony
committed
ci(e2e-matrix): default storage profile to ceph and guard empty matrix.profile

Antony
authored and
Antony
committed
ci(e2e-matrix): drop accidentally staged embedded repo

Antony
authored and
Antony
committed

Commits on Oct 29, 2025

chore(ci): disable virtualization in e2e matrix

Antony
authored and
Antony
committed

Commits on Oct 30, 2025

ci(e2e): Ceph in nested cluster via deckhouse-prod ModuleSource; enable Ceph modules; wait Ready; add CSI smoke; use parent Ceph SC for hotplug

Antony
authored and
Antony
committed
ci(e2e): re-enable virtualization in nested cluster

Antony
authored and
Antony
committed

Commits on Oct 31, 2025

ci(e2e): use NFS as initial default SC for nested; add in-cluster NFS server; enable csi-nfs; keep Ceph switch post-bootstrap

Antony
authored and
Antony
committed
Revert "ci(e2e): use NFS as initial default SC for nested; add in-cluster NFS server; enable csi-nfs; keep Ceph switch post-bootstrap"

Antony
authored and
Antony
committed

Commits on Nov 1, 2025

Configure Ceph to use CephStorageClass instead of direct StorageClass

Antony
authored and
Antony
committed
Fix sds-replicated-volume configuration

Antony
authored and
Antony
committed
Remove redundant StorageClass check for Ceph in nested:storage:configure

Antony
authored and
Antony
committed
Fix heredoc syntax error in CephClusterConnection creation

Antony
authored and
Antony
committed
Fix yamllint issues: remove trailing spaces and fix colon spacing

Antony
authored and
Antony
committed
Remove incorrect source: deckhouse-prod from virtualization ModuleConfig

Antony
authored and
Antony
committed
fix(ci): add retry for token creation and fix heredoc syntax in workflow

Antony
authored and
Antony
committed
fix(ci): replace heredoc with echo block to fix YAML syntax error

Antony
authored and
Antony
committed

Commits on Nov 5, 2025

chore(ci): remove nested_diag.sh from matrix branch (unused)

Antony
authored and
Antony
committed
ci: tidy e2e matrix workflow

Antony
authored and
Antony
committed
ci: stabilize heredoc in e2e matrix

Antony
authored and
Antony
committed
ci: drop heredocs from e2e matrix

Antony
authored and
Antony
committed

Commits on Nov 6, 2025

ci: generate manifests with yq in e2e matrix

Antony
authored and
Antony
committed
ci: trim redundant waits in e2e matrix

Antony
authored and
Antony
committed
ci: wire prod registry and fix sds crd wait

Antony
authored and
Antony
committed
ci: keep dev registry login for dhctl bootstrap

Antony
authored and
Antony
committed
ci: include dev registry in dockercfg

Antony
authored and
Antony
committed
ci: stabilize nested kubeconfig and sds setup

Antony
authored and
Antony
committed
chore(ci): add ingress publish smoke to nested e2e matrix (ingress-nginx HostPort, in-cluster curl)

Antony
authored and
Antony
committed
ci: harden module apply and nested ssh retries

Antony
authored and
Antony
committed
ci: simplify cleanup workflow

Antony
authored and
Antony
committed
chore(ci): remove leftover comment from nested e2e matrix flow

Antony
authored and
Antony
committed
ci: fix sds module heredoc

Antony
authored and
Antony
committed

Commits on Nov 7, 2025

chore(ci): split e2e preparation and test jobs

Antony
authored and
Antony
committed

Commits on Nov 10, 2025

ci(e2e): fix heredoc EOFs; route Ceph setup via Taskfile and add waits for CRDs/cluster connection

Antony
authored and
Antony
committed
ci: fix YAML by reindenting heredoc EOF markers

Antony
authored and
Antony
committed
ci(e2e): route storage selection via shell case; drop ingress smoke steps

Antony
authored and
Antony
committed
ci(e2e): fix SDS ModuleConfig apply to avoid heredoc; generate YAML via yq

Antony
authored and
Antony
committed
ci(e2e): fix relative paths for ceph manifests; add cwd debug; remove early Go setup in prepare

Antony
authored and
Antony
committed
ci(e2e): stop polling Deckhouse Module states; wait for CRDs/operator instead (SDS/Ceph)

Antony
authored and
Antony
committed
ci(e2e): deflake SA token retrieval — wait for SA + fallback to token Secret if TokenRequest fails

Antony
authored and
Antony
committed
ci(e2e): use parent token from GitHub secret for bootstrap; prefer kubeconfig token; keep TokenRequest as last resort

Antony
authored and
Antony
committed
ci(e2e): revert kubeconfig token flow to TokenRequest to avoid breaking bootstrap; remove PARENT_SA_TOKEN wiring

Antony
authored and
Antony
committed
ci(e2e): ensure ModuleSource for SDS; set ModuleConfig.source=deckhouse-prod; apply SDS manifests; use profile storage class

Antony
authored and
Antony
committed
ci(sds): fix LVMVolumeGroup apiVersion to storage.deckhouse.io/v1alpha1

Antony
authored and
Antony
committed
ci: trim noisy debug in Ceph task; fix SDS LVMVolumeGroup spec to Local with selector

Antony
authored and
Antony
committed
ci(sds): fix LVMVolumeGroup schema (spec.local) and use LVM pool; ci(ceph): wait for operator-ceph OR rook-ceph-operator

Antony
authored and
Antony
committed
ci(ceph): wait only for deploy/operator-ceph (drop rook-ceph-operator fallback)

Antony
authored and
Antony
committed
ci: normalize waits to 50 iterations (was 36/60) and update log counters

Antony
authored and
Antony
committed
ci(sds): fix module source conflict (use Embedded) and yq env var for per-node LVMVG generation

Antony
authored and
Antony
committed

Commits on Nov 11, 2025

ci(sds): use deckhouse-prod (io) via REGISTRY_DOCKER_CFG; fix yq env(NODE) in per-node LVMVG generation

Antony
authored and
Antony
committed
ci(e2e-matrix): use profile SC; drop aliases; fix summary; stable RUN_ID (no SHA); no docker prune

Antony
authored and
Antony
committed

Commits on Nov 12, 2025

ci(e2e-matrix): keep Ceph SC for bootstrap and worker hotplug as base fast storage

Antony
authored and
Antony
committed