Skip to content

Added Cross Platform Tests#39

Open
adutt-mw wants to merge 17 commits intomasterfrom
test/ci
Open

Added Cross Platform Tests#39
adutt-mw wants to merge 17 commits intomasterfrom
test/ci

Conversation

@adutt-mw
Copy link
Copy Markdown
Collaborator

@adutt-mw adutt-mw commented Jan 12, 2026

Description

This PR implements an automated verification workflow for cross-platform tests on Jenkins. It triggers builds, verifies artifacts, and clean up resources on the machine.

Changes

  • Added src/test/cross-platform-tests/jenkins_verifier.py:
    • Connects to the Jenkins server to trigger builds.
    • Verifies build artifacts (JUnit, PDF, TAP, Cobertura).
  • Added .github/workflows/cross-platform-tests.yml:
    • Defines a new workflow triggered manually (workflow_dispatch).
    • Sets up the environment, installs the plugin, and runs the verification script on a self-hosted runner.

Test Run: https://github.com/jenkinsci/matlab-plugin/actions/runs/22570260111/job/65376040065

@adutt-mw adutt-mw changed the title Updated main.yml Added Cross Platform Tests Feb 2, 2026
@adutt-mw adutt-mw marked this pull request as ready for review February 2, 2026 05:35
Comment thread src/test/cross-platform-tests/config.json Outdated
Comment thread .github/workflows/main.yml Outdated
Comment thread src/test/cross-platform-tests/jenkins_verifier.py Outdated
Comment thread src/test/cross-platform-tests/jenkins_verifier.py Outdated
Comment thread src/test/cross-platform-tests/jenkins_verifier.py Outdated
Comment thread src/test/cross-platform-tests/jenkins_verifier.py Outdated
@Vahila
Copy link
Copy Markdown

Vahila commented Feb 13, 2026

Thanks Abhishek for working on this! I added few minor comments.

Comment thread src/test/cross-platform-tests/utils.py Outdated
Copy link
Copy Markdown

@sameagen-MW sameagen-MW left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall, this looks really clean to me, thanks for putting it together! Could you ping me a link to the Jenkins job configuration as well? It's somewhat unfortunate that we can't have that under source control as well.

I'll also follow up because I need to do security-oriented pass.

Comment thread .github/workflows/cross-platform-tests.yml
Comment thread .github/workflows/cross-platform-tests.yml Outdated
Comment thread .github/workflows/cross-platform-tests.yml Outdated
Comment thread src/test/cross-platform-tests/jenkins_verifier.py
Comment thread src/test/cross-platform-tests/jenkins_verifier.py Outdated
Comment thread src/test/cross-platform-tests/utils.py Outdated
Comment thread .github/workflows/cross-platform-tests.yml Outdated
Comment thread .github/workflows/cross-platform-tests.yml Outdated
Comment thread .github/workflows/cross-platform-tests.yml Outdated
Comment thread .github/workflows/cross-platform-tests.yml Outdated
Comment thread .github/workflows/cross-platform-tests.yml Outdated
Comment thread .github/workflows/cross-platform-tests.yml Outdated
Comment thread src/test/cross-platform-tests/utils.py Outdated
Comment thread src/test/cross-platform-tests/utils.py Outdated
Copy link
Copy Markdown

@mw-kapilg mw-kapilg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall LGTM! Thanks for adding these

Comment thread src/test/cross-platform-tests/jenkins_verifier.py Outdated
Copy link
Copy Markdown

@sameagen-MW sameagen-MW left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me overall! Can you confirm that this workflow is protected so that others can't trigger workflow runs? It would present a security vulnerability since an attacker could change the python script and use the Jenkins credentials to execute arbitrary code on the self-hosted runner if so.

I think this is ready for submission after confirming that, thanks for working on this and sorry for the delay in getting my review out.

Comment thread src/test/cross-platform-tests/utils.py Outdated
time.sleep(10)

print(f" [ERROR] Timeout: Build #{build_number} did not complete within {timeout_mins} minutes.")
return "TIMEOUT"
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we probably want to throw / catch an exception from here rather than returning a sentinel value. Sentinel values have some significant drawbacks when it comes to maintainability.

@adutt-mw
Copy link
Copy Markdown
Collaborator Author

Looks good to me overall! Can you confirm that this workflow is protected so that others can't trigger workflow runs? It would present a security vulnerability since an attacker could change the python script and use the Jenkins credentials to execute arbitrary code on the self-hosted runner if so.

Hi, I checked regarding this, GitHub provides a specific toggle to "Require approval for all outside contributors". This ensures that no workflow from a fork can execute on our hardware or access repository secrets until a maintainer manually reviews the code and clicks "Approve and run".

Since I don't have admin access to the repository, someone else has to toggle it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants