Skip to content

Add independent peer review validation workflow#64

Open
AndrewGable wants to merge 17 commits into
mainfrom
andrew-validate-independent-peer-review
Open

Add independent peer review validation workflow#64
AndrewGable wants to merge 17 commits into
mainfrom
andrew-validate-independent-peer-review

Conversation

@AndrewGable
Copy link
Copy Markdown
Contributor

Details

Adds a ruleset-compatible workflow that validates pull requests have enough independent Expensify employee approvals before merge. The validator checks PR commits/co-authors, latest approving reviews, Expensify org membership, and repository write permission using Octokit REST APIs.

Related Issues

Prevent merging PRs that aren't peer reviewed

Manual Tests

Ran TypeScript and workflow validation locally:

npm exec tsc -- --noEmit --module NodeNext --moduleResolution NodeNext --target ES2022 --types node scripts/validateIndependentPeerReview.ts
npm exec --yes --package ajv-cli@5.0.0 -- ./scripts/validateWorkflowSchemas.sh
./scripts/actionlint.sh
./scripts/validateImmutableActionRefs.sh

Smoke tested the validator against real PRs using synthetic pull_request payloads. Note: my local token could not read branch protection settings, so these runs exercised the fallback path requiring one independent approval.

===== Expensify/App#86420 (known-self-review) =====
GET /repos/Expensify/App/branches/main/protection/required_pull_request_reviews - 404 with id EE1A:CC534:209938B:20E07F3:6A04E67D in 405ms
Expensify/App@main did not return a branch protection review count; requiring 1 independent approval(s).
::error::Expensify/App#86420 does not have enough independent Expensify employee approvals.%0ARequired independent approvals: 1%0ACommit authors/co-authors: grgia, MelvinBot%0AApprovers: grgia, sobitneupane%0AIndependent employee approvers: (none)
exit_code=1
===== Expensify/App#90237 (known-self-review) =====
GET /repos/Expensify/App/branches/main/protection/required_pull_request_reviews - 404 with id 7ED3:BC6DC:20F0712:2137A8F:6A04E67F in 589ms
Expensify/App@main did not return a branch protection review count; requiring 1 independent approval(s).
::error::Expensify/App#90237 does not have enough independent Expensify employee approvals.%0ARequired independent approvals: 1%0ACommit authors/co-authors: Beamanator, MelvinBot%0AApprovers: Beamanator, Pujan92%0AIndependent employee approvers: (none)
exit_code=1
===== Expensify/Auth#21136 (known-self-review) =====
GET /repos/Expensify/Auth/branches/main/protection/required_pull_request_reviews - 404 with id FDC4:29FBCC:213EBBA:2185FA8:6A04E681 in 469ms
Expensify/Auth@main did not return a branch protection review count; requiring 1 independent approval(s).
::error::Expensify/Auth#21136 does not have enough independent Expensify employee approvals.%0ARequired independent approvals: 1%0ACommit authors/co-authors: AndrewGable, MelvinBot%0AApprovers: AndrewGable%0AIndependent employee approvers: (none)
exit_code=1
===== Expensify/App#90528 (recent-app) =====
GET /repos/Expensify/App/branches/main/protection/required_pull_request_reviews - 404 with id D566:7C9BC:1F566BA:1F9CFF1:6A04E682 in 439ms
Expensify/App@main did not return a branch protection review count; requiring 1 independent approval(s).
Expensify/App#90528 has 1 independent Expensify employee approval(s).
exit_code=0
===== Expensify/Auth#21591 (recent-auth) =====
GET /repos/Expensify/Auth/branches/main/protection/required_pull_request_reviews - 404 with id E95D:7C9BC:1F57666:1F9DF9C:6A04E683 in 435ms
Expensify/Auth@main did not return a branch protection review count; requiring 1 independent approval(s).
Expensify/Auth#21591 has 1 independent Expensify employee approval(s).
exit_code=0
===== Expensify/GitHub-Actions#63 (recent-actions) =====
GET /repos/Expensify/GitHub-Actions/branches/main/protection/required_pull_request_reviews - 404 with id EF12:29906A:1F91F48:1FD97DD:6A04E685 in 408ms
Expensify/GitHub-Actions@main did not return a branch protection review count; requiring 1 independent approval(s).
Expensify/GitHub-Actions#63 has 1 independent Expensify employee approval(s).
exit_code=0

Linked PRs

N/A

Comment thread scripts/validateIndependentPeerReview.ts Outdated
@AndrewGable AndrewGable marked this pull request as ready for review May 15, 2026 23:16
Comment thread scripts/verifyPeerReview.ts
Comment on lines +274 to +281
if (requiredApprovingReviewCount === 0) {
console.log(`${owner}/${repo}#${number} targets ${context.baseRef}, which does not require approving reviews.`);
return;
}
if (approvers.length === 0) {
console.log(`${owner}/${repo}#${number} has no approving reviews from writers; regular branch protection will block merge until an approval exists.`);
return;
}
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given this will cause the check to pass when they are are no reviews, we might need to run this on pull_request_review events too

Comment on lines +31 to +34
permission-contents: read
permission-members: read
permission-metadata: read
permission-pull-requests: read
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you also need permission-administration: read in order to read branch protection rules.

I noticed before this was switched to graphql that it is passing even with incorrect permissions. Ideally I think it should fail in this case

Image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants