ci: automate GitHub Actions freshness audits#66
Conversation
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
Reviewer's GuideAutomates daily GitHub Actions freshness auditing by adding a Node-based audit script, a scheduled workflow that runs it and manages a tracker issue, increasing Dependabot GitHub Actions update cadence, and wiring in tests and an npm script for local/CI use. Sequence diagram for the scheduled Actions freshness auditsequenceDiagram
actor GitHubScheduler
participant ActionsFreshnessWorkflow
participant CheckoutStep
participant SetupNodeStep
participant AuditStep
participant AuditScript
participant GitHubAPI
participant GitHubIssues
participant UploadArtifactStep
GitHubScheduler ->> ActionsFreshnessWorkflow: Trigger on cron or workflow_dispatch
ActionsFreshnessWorkflow ->> CheckoutStep: Run actions/checkout@<pinned_sha>
CheckoutStep -->> ActionsFreshnessWorkflow: Repository checked out
ActionsFreshnessWorkflow ->> SetupNodeStep: Run actions/setup-node@<pinned_sha>
SetupNodeStep -->> ActionsFreshnessWorkflow: Node.js 20 available
ActionsFreshnessWorkflow ->> AuditStep: Run node scripts/audit-actions-freshness.js --report ... --json ... --manage-issue
AuditStep ->> AuditScript: Start script with CLI options
AuditScript ->> AuditScript: parseArguments
AuditScript ->> AuditScript: readWorkflowFiles(.github/workflows)
AuditScript ->> AuditScript: collectWorkflowReferences / splitReferencesByPinning
AuditScript ->> GitHubAPI: Resolve latest versions for pinned action repositories
GitHubAPI -->> AuditScript: Latest tags and SHAs or errors
AuditScript ->> AuditScript: collectStaleReferences
AuditScript ->> AuditScript: buildJsonReport / buildMarkdownReport
AuditScript ->> AuditScript: writeReportFiles (md + json)
AuditScript ->> AuditScript: writeStepSummary (GITHUB_STEP_SUMMARY)
alt manage-issue enabled and token available
AuditScript ->> GitHubIssues: Find existing tracking issue via GitHub API
opt staleCount > 0
AuditScript ->> GitHubIssues: Create or update tracking issue with report
end
opt staleCount == 0 and tracking issue exists
AuditScript ->> GitHubIssues: Comment and close tracking issue
end
end
AuditScript -->> AuditStep: Exit (possibly with nonzero code if stale and failOnStale)
AuditStep -->> ActionsFreshnessWorkflow: Script completed
ActionsFreshnessWorkflow ->> UploadArtifactStep: Upload actions-freshness-report.md/json
UploadArtifactStep -->> ActionsFreshnessWorkflow: Artifact stored
File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
Review Summary by QodoAutomate GitHub Actions freshness audits with daily scanning
WalkthroughsDescription• Add automated GitHub Actions freshness audit script with SHA pinning detection • Create scheduled workflow to scan pinned action references daily • Implement GitHub issue tracking for stale action references • Add comprehensive unit tests for action parsing and report generation • Switch Dependabot GitHub Actions updates from weekly to daily cadence Diagramflowchart LR
A["Workflow Files"] -->|"Parse uses statements"| B["Action References"]
B -->|"Classify by pinning"| C["Pinned vs Unpinned"]
C -->|"Resolve latest versions"| D["GitHub API"]
D -->|"Compare SHAs"| E["Stale References"]
E -->|"Generate reports"| F["Markdown & JSON"]
F -->|"Manage tracking"| G["GitHub Issues"]
File Changes1. scripts/audit-actions-freshness.js
|
Summary of ChangesHello @Mehdi-Bl, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request significantly enhances the repository's continuous integration security and maintenance posture. By increasing the frequency of Dependabot updates for GitHub Actions and introducing an automated freshness audit, it ensures that all pinned actions are regularly checked against their latest available versions. This proactive approach helps prevent the use of outdated or vulnerable action versions, streamlining the process of keeping CI workflows secure and efficient. Highlights
🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Changelog
Ignored Files
Activity
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
📝 WalkthroughWalkthroughAdds an actions-freshness auditing system: new scheduled workflow and CLI script to scan workflows for action references, resolve latest versions via GitHub API, generate reports, manage a tracking issue, plus parsing utilities, tests, and a Dependabot schedule change to daily. Changes
Sequence DiagramsequenceDiagram
participant GHA as GitHub Actions
participant Audit as Audit CLI\n(scripts/audit-actions-freshness.js)
participant Parser as Parser Lib\n(scripts/lib/actions-freshness.js)
participant GHAPI as GitHub REST API
participant Artifacts as Artifacts & Issue
GHA->>Audit: trigger (daily or manual)
Audit->>Parser: read & parse workflows
Parser-->>Audit: list of action references (pinned/unpinned)
Audit->>GHAPI: request latest release/tag/commit for each repo
GHAPI-->>Audit: latest SHAs/tags or errors
Audit->>Parser: generate Markdown + JSON report
Parser-->>Audit: formatted report content
Audit->>Artifacts: upload reports & update/create/close tracking issue
Artifacts-->>GHA: artifacts uploaded / issue updated
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing touches
🧪 Generate unit tests (beta)
No actionable comments were generated in the recent review. 🎉 🧹 Recent nitpick comments
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
There was a problem hiding this comment.
Hey - I've left some high level feedback:
- The
readWorkflowFileshelper only scans files directly under.github/workflowsand will miss any workflows placed in subdirectories there; consider adding a recursive scan if nested workflows are or might be used. resolveRepositoryVersionsperforms GitHub API calls sequentially per repository, which will be slow and more prone to rate limits as the number of pinned actions grows; consider resolving in parallel with a small concurrency limit.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- The `readWorkflowFiles` helper only scans files directly under `.github/workflows` and will miss any workflows placed in subdirectories there; consider adding a recursive scan if nested workflows are or might be used.
- `resolveRepositoryVersions` performs GitHub API calls sequentially per repository, which will be slow and more prone to rate limits as the number of pinned actions grows; consider resolving in parallel with a small concurrency limit.Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
Dependency Review✅ No vulnerabilities or license issues found.Scanned FilesNone |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a robust mechanism for auditing GitHub Actions freshness, including a new script to scan for stale pinned actions, generate reports, and automatically manage a tracking issue, alongside increasing the Dependabot check frequency for actions to daily. However, the new script contains path traversal vulnerabilities in how it handles command-line arguments for directories and file paths, which could allow an attacker to read or write arbitrary files in the CI environment. It is recommended to add path validation to restrict file operations to the intended workspace. Additionally, consider refactoring the argument parsing logic for improved readability.
| } | ||
|
|
||
| function readWorkflowFiles(workflowDirectory) { | ||
| const directoryPath = path.resolve(process.cwd(), workflowDirectory); |
There was a problem hiding this comment.
The workflowDirectory parameter is taken directly from command-line arguments and used in path.resolve(process.cwd(), workflowDirectory) without any validation or sanitization. This allows an attacker who can influence the script's arguments to read arbitrary directories and files on the system by using path traversal sequences like ... In a CI environment, this could be used to exfiltrate sensitive files or environment configurations.
const directoryPath = path.resolve(process.cwd(), workflowDirectory);
if (!directoryPath.startsWith(process.cwd())) {
throw new Error(`Invalid workflow directory: ${workflowDirectory}`);
}| } | ||
|
|
||
| function ensureParentDirectory(filePath) { | ||
| const absolutePath = path.resolve(process.cwd(), filePath); |
There was a problem hiding this comment.
The filePath parameter (derived from --report and --json arguments) is used in path.resolve(process.cwd(), filePath) without validation. This allows an attacker to specify arbitrary file paths for writing the audit reports, potentially overwriting sensitive files or creating files in unintended locations.
const absolutePath = path.resolve(process.cwd(), filePath);
if (!absolutePath.startsWith(process.cwd())) {
throw new Error(`Invalid file path: ${filePath}`);
}| const argument = argv[index]; | ||
|
|
||
| if (argument === '--report') { | ||
| options.reportPath = argv[index + 1]; | ||
| index += 1; | ||
| continue; | ||
| } | ||
|
|
||
| if (argument === '--json') { | ||
| options.jsonPath = argv[index + 1]; | ||
| index += 1; | ||
| continue; | ||
| } | ||
|
|
||
| if (argument === '--workflow-dir') { | ||
| options.workflowDirectory = argv[index + 1]; | ||
| index += 1; | ||
| continue; | ||
| } | ||
|
|
||
| if (argument === '--issue-title') { | ||
| options.issueTitle = argv[index + 1]; | ||
| index += 1; | ||
| continue; | ||
| } | ||
|
|
||
| if (argument === '--fail-on-stale') { | ||
| options.failOnStale = true; | ||
| continue; | ||
| } | ||
|
|
||
| if (argument === '--manage-issue') { | ||
| options.manageIssue = true; | ||
| continue; | ||
| } | ||
|
|
||
| throw new Error(`Unsupported argument: ${argument}`); |
There was a problem hiding this comment.
For better readability and maintainability, you could refactor this series of if statements into a switch statement. This would make the argument parsing logic cleaner and easier to extend in the future.
const argument = argv[index];
switch (argument) {
case '--report':
options.reportPath = argv[index + 1];
index += 1;
break;
case '--json':
options.jsonPath = argv[index + 1];
index += 1;
break;
case '--workflow-dir':
options.workflowDirectory = argv[index + 1];
index += 1;
break;
case '--issue-title':
options.issueTitle = argv[index + 1];
index += 1;
break;
case '--fail-on-stale':
options.failOnStale = true;
break;
case '--manage-issue':
options.manageIssue = true;
break;
default:
throw new Error(`Unsupported argument: ${argument}`);
}![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Fix all issues with AI agents
In @.github/workflows/actions-freshness.yml:
- Around line 8-10: Move the top-level permissions block (permissions: contents:
read, issues: write) down into the specific job that needs issue-write access
(the audit job); remove the workflow-level permissions and add a job-level
permissions block inside the audit job with contents: read and issues: write so
only the audit job has write access to issues, keeping least-privilege.
In `@scripts/audit-actions-freshness.js`:
- Around line 143-154: The githubRequest function currently always includes an
Authorization header even when token is an empty string; change headers
construction in githubRequest so Authorization is added only when token is
truthy (e.g., conditionally set or spread a mini-object when token exists) to
avoid sending Authorization: '' during unauthenticated runs; locate the
githubRequest function and update its headers logic to omit the Authorization
key when token is falsy.
🧹 Nitpick comments (4)
.github/workflows/actions-freshness.yml (1)
41-50: Considerwarninstead oferrorforif-no-files-found.With
if: always(), the upload step runs even if the audit step fails before writing any report files. In that scenario,if-no-files-found: errorwill cause the workflow to fail with a confusing artifact-upload error, masking the real failure. Usingwarnwould let the actual script error surface clearly.Proposed change
- if-no-files-found: error + if-no-files-found: warnscripts/audit-actions-freshness.js (2)
31-68: No bounds check when consuming flag values.If a flag like
--reportis the last argument,argv[index + 1]isundefined. The downstream validation (lines 71–85) catches this becauseundefinedis falsy, but the error message ("The --report option requires a value") is misleading if the user actually passed--reportwithout a value vs. not passing it at all. Minor UX nit — not a blocker.
247-282: Sequential API resolution — acceptable but worth noting.Each repository's version is resolved sequentially. For a handful of distinct action repos this is fine given the 15-minute timeout, but if the set grows, consider parallelizing with
Promise.all(or a bounded concurrency helper) to reduce wall-clock time.scripts/lib/actions-freshness.js (1)
164-169: Pipe characters in error messages could break the markdown table.If an
error.messageever contains a|character, the markdown table row will be malformed. Consider escaping pipe characters in the message.Proposed fix
+ function escapeMarkdownPipe(value) { + return typeof value === 'string' ? value.replace(/\|/g, '\\|') : value; + } + for (const error of errorsSorted) { - lines.push(`| ${error.repository} | ${error.message} |`); + lines.push(`| ${error.repository} | ${escapeMarkdownPipe(error.message)} |`); }
Code Review by Qodo
✅ 1.
|
![qodo-free-for-open-source-projects[bot]](https://avatars.githubusercontent.com/in/1068000?s=60&v=4){kind=small}
| if (staleCount === 0) { | ||
| if (!trackingIssue) { | ||
| return; | ||
| } | ||
|
|
||
| await closeTrackingIssue({ | ||
| owner, | ||
| repository, | ||
| issueNumber: trackingIssue.number, | ||
| token, | ||
| }); | ||
| console.log(`[actions-freshness] Closed issue #${trackingIssue.number}`); | ||
| return; | ||
| } |
There was a problem hiding this comment.
1. resolutionerrors ignored for closing 📘 Rule violation ⛯ Reliability
The tracker issue is closed whenever staleCount === 0, even if there were GitHub API resolution errors, which can incorrectly report a clean audit and suppress follow-up. Resolution errors are also not used to influence the script outcome (exit code), weakening robustness for failure/edge cases.
Agent Prompt
## Issue description
The audit script can report a "clean" state and close the tracker issue when `staleCount === 0` even if GitHub API resolution failed for one or more action repositories (`resolutionErrors` is non-empty). The run also only sets a failure exit code based on staleness, not on resolution failures.
## Issue Context
`resolutionErrors` indicates the audit could not verify freshness for some pinned actions. Treating this as success can hide failures and break the expectation of robust error/edge-case handling.
## Fix Focus Areas
- scripts/audit-actions-freshness.js[391-531]
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
| const { | ||
| buildMarkdownReport, | ||
| collectWorkflowReferences, | ||
| parseWorkflowContent, | ||
| splitReferencesByPinning, | ||
| } = require('../../../scripts/lib/actions-freshness'); |
There was a problem hiding this comment.
2. tests/catalog.md missing actions-freshness 📘 Rule violation ⛯ Reliability
A new unit test suite was added for the actions freshness audit helpers, but tests/catalog.md was not updated to include this new test target/use case mapping. This leaves the test catalog stale relative to the changes.
Agent Prompt
## Issue description
`tests/unit/scripts/actions-freshness.test.js` was added, but the Unit Tests section of `tests/catalog.md` was not updated to reflect this new test target/use case mapping.
## Issue Context
The test catalog is the authoritative reference for what should be tested; adding new test targets should be reflected in the catalog to avoid staleness.
## Fix Focus Areas
- tests/catalog.md[19-40]
- tests/unit/scripts/actions-freshness.test.js[1-106]
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
|
Quality gate triage summary:
Important: this PR is still blocked by a separate valid failure in |
|
1 participant
Summary
Validation
Summary by Sourcery
Automate auditing of pinned GitHub Actions references and tighten CI dependency freshness.
New Features:
Enhancements:
Tests:
Summary by CodeRabbit
Chores
New Features
Tests