[chore] Adding nightly tests against next RHDH tag

[OpinionatedHeron]

Description

Added a nightly workflow to RHDH Local in order to run tests against the next RHDH tag. Tried to ensure that it would test all release branches, even if new ones are created.

Which issue(s) does this PR fix or relate to

RHIDP-12133

PR acceptance criteria

• Tests updated and passing
• Documentation updated
• Built-in TechDocs updated if needed. Note that TechDocs changes may need to be reviewed by a Product Manager and/or Architect to ensure content accuracy, clarity, and alignment with user needs.

How to test changes / Special notes to the reviewer

Tested workflow in my own fork, all tests seem to have run as expected but there were some failures for 1.8 and 1.7 - mostly related to arm, so I think they are to be expected.

[rhdh-qodo-merge]

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

🎫 Ticket compliance analysis 🔶 RHIDP-12133 - Partially compliant Compliant requirements: Add a nightly GitHub Actions workflow for RHDH Local. Run tests against the unstable/next RHDH tag (to catch breakages before users hit them). Ensure the workflow covers all release branches (including newly created ones) without needing manual workflow updates. Non-compliant requirements: Requires further human verification: Confirm the scheduled cron time is correct for the intended timezone/maintenance window and that the workflow reliably triggers nightly in the target repo.

⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪

🔒 Security concerns Supply chain: The workflow executes npx --yes dclint ., which can fetch and run unpinned code during CI. Consider pinning the dclint version (or installing via the repository lockfile) to reduce supply-chain exposure and improve reproducibility.

⚡ Recommended focus areas for review Action Version The workflow uses actions/checkout@v6, which is likely an invalid/nonexistent major version and would cause the workflow to fail at runtime. Consider switching to a known supported major version (and ideally pin to a SHA for supply-chain hardening). - uses: actions/checkout@v6 - name: Lint Supply Chain The lint step runs npx --yes dclint ., which may download and execute the latest package at runtime. This can lead to non-reproducible runs and increases supply-chain risk. Consider pinning the package/version (or using a lockfile-based install) to keep the workflow deterministic. - name: Lint run: | npx --yes dclint .

📚 Focus areas based on broader codebase context Reproducibility The workflow pins several tool versions, but CORPORATE_PROXY_IMAGE is set to docker.io/ubuntu/squid:latest (line 73), which can change over time and make nightly results non-deterministic. Consider pinning this image to a specific version/digest (and optionally documenting the update policy) to keep failures attributable to code changes rather than drifting dependencies. (Ref 6, Ref 1) env: # Using the next tag from the release branch RHDH_TAG: ${{ github.event.inputs.rhdh_tag || vars.NEXT_RHDH_TAG }} RHDH_IMAGE: quay.io/rhdh-community/rhdh:next DOCKER_COMPOSE_VERSION: v5.0.1 PODMAN_IMAGE: quay.io/podman/stable:v5 CORPORATE_PROXY_IMAGE: docker.io/ubuntu/squid:latest Reference reasoning: Existing YAML in the codebase commonly uses explicit, stable image tags for test tooling and components (e.g., a fixed scorecard-test version and explicitly tagged component images), which avoids unexpected behavior changes from floating tags like latest.

📄 References redhat-developer/rhdh-operator/pkg/model/testdata/rhdh-deployment.yaml [36-58] redhat-developer/rhdh-chart/charts/orchestrator-infra/values.yaml [37-41] redhat-developer/rhdh-chart/charts/orchestrator-software-templates-infra/values.yaml [141-163] redhat-developer/rhdh-chart/charts/orchestrator-infra/templates/tests/infra-test.yaml [0-2] redhat-developer/rhdh-chart/charts/orchestrator-software-templates-infra/templates/tests/infra-test.yaml [0-2] redhat-developer/rhdh-operator/bundle/rhdh/tests/scorecard/config.yaml [5-47] redhat-developer/rhdh-operator/bundle/backstage.io/tests/scorecard/config.yaml [5-47] redhat-developer/rhdh-operator/config/scorecard/patches/olm.config.yaml [1-40]

[rhdh-qodo-merge]

PR Type

Tests, Enhancement

---

Description

• Add nightly workflow to test RHDH Local against next RHDH tag

• Dynamically discover and test all release branches at runtime

• Test matrix includes multiple OS, container tools, and configurations

• Lint checks and comprehensive health verification included

---

File Walkthrough

Relevant files

Tests

nightly.yaml Create nightly test workflow for RHDH releases                      .github/workflows/nightly.yaml New nightly workflow scheduled daily and manually triggerable via workflow_dispatch Lint job validates code quality using dclint Dynamic branch discovery job finds main and release-* branches at runtime Comprehensive test matrix covering ubuntu-24.04/arm, docker/podman, 6 compose configurations, and user config variations Health checks include container engine version display, compose config validation, app startup verification, and detailed logging on failure +244/-0

---

[rhdh-qodo-merge]

PR Code Suggestions ✨

Explore these optional code suggestions:

Category	Suggestion	Impact
High-level	Reduce the nightly test matrix The test matrix is too large for a nightly run, leading to high costs and long execution times. It should be reduced for scheduled runs, with the full matrix reserved for manual triggers. Examples: .github/workflows/nightly.yaml [38-63] strategy: fail-fast: false matrix: branch: ${{ fromJson(needs.discover-branches.outputs.branches) }} os: - ubuntu-24.04 - ubuntu-24.04-arm tool: - docker - podman ... (clipped 16 lines) Solution Walkthrough: Before: jobs: test: strategy: fail-fast: false matrix: branch: ${{ fromJson(needs.discover-branches.outputs.branches) }} os: - ubuntu-24.04 - ubuntu-24.04-arm tool: - docker - podman userConfig: - "false" - "true" composeConfig: - name: "default" - ... (5 more configurations) After: jobs: test: strategy: fail-fast: false matrix: # For nightly schedule, run a smaller, representative set of tests branch: ${{ github.event_name == 'schedule' && '["main"]' || fromJson(needs.discover-branches.outputs.branches) }} os: ${{ github.event_name == 'schedule' && '["ubuntu-24.04"]' || '["ubuntu-24.04", "ubuntu-24.04-arm"]' }} tool: ${{ github.event_name == 'schedule' && '["docker"]' || '["docker", "podman"]' }} userConfig: ${{ github.event_name == 'schedule' && '["false"]' || '["false", "true"]' }} composeConfig: ${{ github.event_name == 'schedule' && fromJson(''[{"name": "default", "cliArgs": ""}]'') || fromJson(needs.discover-branches.outputs.composeConfigs) }} ... Suggestion importance[1-10]: 9 __ Why: The suggestion correctly identifies a massive test matrix that would be excessively costly and slow for a nightly run, proposing a practical solution to run a smaller, representative set on schedule.	High
Possible issue	Fix invalid docker-compose version Update the DOCKER_COMPOSE_VERSION from the invalid v5.0.1 to a valid version like v2.27.0 to prevent the download step from failing. .github/workflows/nightly.yaml [71] -DOCKER_COMPOSE_VERSION: v5.0.1 +DOCKER_COMPOSE_VERSION: v2.27.0 Apply / Chat Suggestion importance[1-10]: 9 __ Why: The suggestion correctly identifies that v5.0.1 is an invalid version for docker/compose, which would cause the workflow to fail. This is a critical bug fix.	High
	✅ Add non-interactive install flag Suggestion Impact: The workflow no longer runs the `sudo apt-get install ca-certificates curl` command at all—the entire "Update Docker version" step containing it was removed/refactored into a reusable action, eliminating the interactive install risk the suggestion targeted. code diff: - - name: Update Docker version - run: | - # Add Docker's official GPG key: - sudo apt-get update - sudo apt-get install ca-certificates curl - sudo install -m 0755 -d /etc/apt/keyrings - sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc - sudo chmod a+r /etc/apt/keyrings/docker.asc - - # Add the repository to Apt sources: - echo \ - "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \ - $(. /etc/os-release && echo "${UBUNTU_CODENAME:-$VERSION_CODENAME}") stable" | \ - sudo tee /etc/apt/sources.list.d/docker.list > /dev/null - sudo apt-get update - sudo apt-get install -y docker-ce \ Add the -y flag to the apt-get install command to ensure it runs non-interactively. .github/workflows/nightly.yaml [100] -sudo apt-get install ca-certificates curl +sudo apt-get install -y ca-certificates curl [Suggestion processed] Suggestion importance[1-10]: 7 __ Why: The suggestion correctly points out the missing -y flag, which is crucial for non-interactive execution in a CI environment to prevent the job from hanging.	Medium
General	Add workflow_dispatch input Define the rhdh_tag input under the workflow_dispatch trigger to make github.event.inputs.rhdh_tag available for manual runs. .github/workflows/nightly.yaml [3-4] on: workflow_dispatch: + inputs: + rhdh_tag: + description: 'Override RHDH tag' + required: false Apply / Chat Suggestion importance[1-10]: 8 __ Why: This suggestion correctly identifies that for github.event.inputs.rhdh_tag to be used, it must be defined as an input for the workflow_dispatch trigger, fixing a potential runtime error.	Medium
Update

[OpinionatedHeron]

/cherry-pick release-1.7

[openshift-cherrypick-robot]

@OpinionatedHeron: once the present PR merges, I will cherry-pick it on top of release-1.7 in a new PR and assign it to you.

Details

In response to this:

/cherry-pick release-1.7

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[rm3l]

/agentic_review

[rhdh-qodo-merge]

Code Review by Qodo

🐞 Bugs (3) 📘 Rule violations (0) 📎 Requirement gaps (0)

1. JS syntax error comment 🐞 Bug ≡ Correctness

Description

The discover-branches job’s actions/github-script block contains a # ... line, which is not
valid JavaScript syntax and will cause the job to fail. Since test depends on discover-branches,
the entire nightly matrix will be blocked.

Code

.github/workflows/nightly.yaml[R30-37]

+          script: |
+            const branches = await github.paginate(github.rest.repos.listBranches, {
+              owner: context.repo.owner,
+              repo: context.repo.repo,
+            });
+
+            # Keep main and the latest 3 release branches (e.g. release-1.2, release-1.1, release-1.0) - to cover EOL branches and period between FF and GA of new releases
+            const topReleaseBranches = branches

Relevance

⭐⭐⭐ High

actions/github-script runs JS; # comment will syntax-error and block dependent jobs; team
accepts workflow parsing fixes.

PR-#169
PR-#48
PR-#58

ⓘ Recommendations generated based on similar findings in past PRs

Evidence

The script: section is executed as JavaScript by actions/github-script, but it contains a #
comment line. This will throw a parse error before core.setOutput(...) runs, and the test job
requires that output.

.github/workflows/nightly.yaml[27-50]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
The `actions/github-script` step in `.github/workflows/nightly.yaml` uses `#` to comment a line inside JavaScript, which will throw a syntax error and prevent `discover-branches` from producing the matrix output.

### Issue Context
`actions/github-script` executes the provided `script:` as JavaScript. JavaScript comments must use `//` or `/* ... */`.

### Fix Focus Areas
- .github/workflows/nightly.yaml[30-43]

### Suggested change
Replace the `# Keep main...` line with `// Keep main...` (or remove the comment entirely).

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

2. Unsupported image tag computed 🐞 Bug ≡ Correctness

Description

The composite action derives RHDH_IMAGE from inputs.git_ref for any non-main branch via
next-${BRANCH#release-}, but CI passes github.head_ref for PRs, which is not constrained to
release-x.y. This can set RHDH_IMAGE to an unsupported/nonexistent :next-<branch> tag and
cause docker/podman compose to fail pulling the image.

Code

.github/actions/rhdh-local-compose-test/action.yaml[R49-55]

+      run: |
+        if [ -n "$INPUT_TAG" ]; then TAG="$INPUT_TAG"
+        elif [ "$BRANCH" = "main" ]; then TAG="next"
+        else TAG="next-${BRANCH#release-}"; fi
+
+        echo "RHDH_TAG=$TAG" >> "$GITHUB_ENV"
+        echo "RHDH_IMAGE=quay.io/rhdh-community/rhdh:$TAG" >> "$GITHUB_ENV"

Relevance

⭐⭐⭐ High

Tag derivation assumes release-*; PR head_ref can be feature branch, likely causing CI pull
failures; team fixes CI breakages.

PR-#75
PR-#164
PR-#169

ⓘ Recommendations generated based on similar findings in past PRs

Evidence

The composite action always exports RHDH_IMAGE=...:$TAG based on the checked-out git_ref. The
main compose file consumes RHDH_IMAGE for the rhdh and install-dynamic-plugins images, so a
bad tag breaks the compose run. The repo docs only describe :next and :next-<version> for
release-1.y branches, but the CI workflow passes github.head_ref for PRs, which can be a feature
branch name (not release-*) and therefore will produce a tag that is not documented/supported.

.github/actions/rhdh-local-compose-test/action.yaml[44-56]
.github/workflows/test.yml[65-81]
compose.yaml[24-57]
docs/rhdh-local-guide/container-image-guide.md[21-33]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
The composite action’s tag derivation sets `RHDH_IMAGE` to `quay.io/rhdh-community/rhdh:next-${BRANCH#release-}` for any non-`main` branch. In CI (`.github/workflows/test.yml`), `git_ref` is set to `github.head_ref` for PRs, which is not guaranteed to be a `release-x.y` branch; this can produce unsupported/nonexistent image tags and break `compose up`.

### Issue Context
- `compose.yaml` uses `${RHDH_IMAGE}` for both `rhdh` and `install-dynamic-plugins`, so an invalid/nonexistent tag fails the whole test.
- The repo’s container image guide documents `:next` and `:next-<version>` (derived from `release-1.y`), not arbitrary branch-derived tags.

### Fix Focus Areas
- .github/actions/rhdh-local-compose-test/action.yaml[44-56]
- .github/workflows/test.yml[65-75]

### Suggested fix
Implement one of these safe strategies:
1) **Guard + fallback in the composite action**:
  - If `BRANCH == main` => `TAG=next`
  - Else if `BRANCH` matches `^release-\d+\.\d+$` => `TAG=next-${BRANCH#release-}`
  - Else **do not set** `RHDH_IMAGE` (let `compose.yaml` default apply) or set a known-good default.

2) **Decouple checkout ref from tag selection**:
  - Keep `git_ref` as the ref to checkout for testing.
  - Add a new input (e.g. `image_ref` or `rhdh_tag_override`) and in `test.yml` pass the base branch (or `main`) so PRs don’t generate branch-derived tags.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

3. Apt install missing -y 🐞 Bug ☼ Reliability

Description

The composite action runs sudo apt-get install ca-certificates curl without -y, which is
inconsistent with other apt-get install -y ... commands in the same step. This can make behavior
depend on runner apt configuration and is less robust for non-interactive execution.

Code

.github/actions/rhdh-local-compose-test/action.yaml[R85-87]

+        sudo apt-get update
+        sudo apt-get install ca-certificates curl
+        sudo install -m 0755 -d /etc/apt/keyrings

Relevance

⭐⭐⭐ High

Non-interactive CI apt installs typically use -y here; consistency/reliability tweak likely
welcomed.

PR-#118
PR-#169
PR-#48

ⓘ Recommendations generated based on similar findings in past PRs

Evidence

The Docker setup step installs some packages with apt-get install -y ... but installs
ca-certificates curl without -y, creating inconsistency and potentially relying on implicit
defaults.

.github/actions/rhdh-local-compose-test/action.yaml[84-100]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
`apt-get install ca-certificates curl` is invoked without `-y`, unlike other installs in the same script.

### Issue Context
This is a robustness/consistency improvement to avoid relying on runner-specific apt non-interactive defaults.

### Fix Focus Areas
- .github/actions/rhdh-local-compose-test/action.yaml[84-88]

### Suggested change
Change:
- `sudo apt-get install ca-certificates curl`
To:
- `sudo apt-get install -y ca-certificates curl`

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

[rm3l]

@OpinionatedHeron Was this the latest run in your fork?
https://github.com/OpinionatedHeron/rhdh-local/actions/runs/24865263077
If so, it looks like there might be some syntax errors..
To make it easier for reviewers, would you mind sharing a link to a successful run (only the nightly workflow) from your fork?

[rm3l]

@OpinionatedHeron Was this the latest run in your fork? OpinionatedHeron/rhdh-local/actions/runs/24865263077 If so, it looks like there might be some syntax errors.. To make it easier for reviewers, would you mind sharing a link to a successful run (only the nightly workflow) from your fork?

Ah actually, looks like Qodo captured this syntax error ;) See #180 (comment)

[OpinionatedHeron]

Sorry completely missed this, making fixes now.

[OpinionatedHeron]

Jobs are passing for main, but failing for 1.9, 1.8, and 1.7 because it cannot find the action file without cherry-picking:

https://github.com/OpinionatedHeron/rhdh-local/actions/runs/25003559224

[OpinionatedHeron]

Test Fail reason - Can't find 'action.yml', 'action.yaml' or 'Dockerfile' under '/home/runner/work/rhdh-local/rhdh-local/.github/actions/rhdh-local-compose-test'. Did you forget to run actions/checkout before running your local action?

[OpinionatedHeron]

/agentic_review

[OpinionatedHeron]

Made recommended changes, and removed developer-lightspeed/compose-with-ollama.yaml as tests were failing because this file was removed in a previous commit.

Nightly tests passing on main:
https://github.com/OpinionatedHeron/rhdh-local/actions/runs/25174317550

[rm3l]

I think we should also set the appropriate tag for the catalog index image depending on the branch:

rhdh-local/default.env

Lines 14 to 16 in 099a563

	# Default plugin catalog index image
	# Requires RHDH 1.9+ to be handled.
	CATALOG_INDEX_IMAGE=quay.io/rhdh/plugin-catalog-index:1.9

See https://quay.io/repository/rhdh/plugin-catalog-index?tab=tags

[OpinionatedHeron]

Okie dokie - added handling to check for the catalog index in the default.env, if there is not set catalog index it skips as no catalog is present. This should cover for future branches that include the catalog index, as well as 1.7/1.8 where it was not included.

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud