Skip to content

Provision Copilot CLI for code-review judge in Claude workflow #701

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
gggdttt merged 4 commits into from
Jun 26, 2026
+20 −4
Merged

Provision Copilot CLI for code-review judge in Claude workflow #701

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
66265ff
ci: provision Copilot CLI for code-review judge in Claude workflow
Jun 26, 2026
c2ffb75
ci: extract eval CLI install into shared composite action; install bo…
Jun 26, 2026
1886bbb
Update .github/actions/install-eval-clis/action.yml
gggdttt Jun 26, 2026
efe69b6
Update .github/workflows/claude-evaluation.yml
gggdttt Jun 26, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions .github/actions/install-eval-clis/action.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
name: Install Agent Harnesses
description: Install the Claude Code and GitHub Copilot CLIs used by the evaluation workflows

runs:
using: composite
steps:
- name: Install Claude Code
run: npm install -g @anthropic-ai/claude-code@2.1.160
shell: pwsh

- name: Install GitHub Copilot CLI
run: npm install -g @github/copilot@1.0.57
shell: pwsh
7 changes: 5 additions & 2 deletions .github/workflows/claude-evaluation.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,7 @@ jobs:
permissions:
contents: read
id-token: write
copilot-requests: write # code-review semantic judge runs via Copilot CLI
name: ${{ matrix.entry }}
strategy:
fail-fast: false
Expand Down Expand Up @@ -113,16 +114,18 @@ jobs:
dotnet tool install -g Microsoft.Dynamics.BusinessCentral.Development.Tools --version 18.0.37.11445-beta
echo "$HOME\.dotnet\tools" >> $env:GITHUB_PATH

- name: Install Claude Code
run: npm install -g @anthropic-ai/claude-code@2.1.160
- name: Install Agent Harnesses
uses: ./.github/actions/install-eval-clis

- name: Run Claude Code for entry ${{ matrix.entry }}
timeout-minutes: 120
shell: pwsh
env:
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
COPILOT_GITHUB_TOKEN: ${{ github.token }} # used by the code-review semantic judge
run: |
Write-Output "::add-mask::$env:ANTHROPIC_API_KEY"
Write-Output "::add-mask::$env:COPILOT_GITHUB_TOKEN"

uv run bcbench evaluate claude "${{ matrix.entry }}" `
--model "${{ inputs.model }}" `
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/copilot-evaluation.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -121,8 +121,8 @@ jobs:
dotnet tool install -g Microsoft.Dynamics.BusinessCentral.Development.Tools --version 18.0.37.11445-beta
echo "$HOME\.dotnet\tools" >> $env:GITHUB_PATH

- name: Install GitHub Copilot CLI
run: npm install -g @github/copilot@1.0.57
- name: Install evaluation CLIs
uses: ./.github/actions/install-eval-clis

- name: Run GitHub Copilot CLI for entry ${{ matrix.entry }}
timeout-minutes: 120
Expand Down
Loading