Investigate pocket vetoes (open PRs as implicit rejections) in merge rate #51
Description
Problem
The v3 scoring model computes merge rate as merged / (merged + closed). This excludes PRs that are still open, which means pocket vetoes (PRs that maintainers ignore rather than explicitly closing) are absent from both the numerator and denominator. A contributor whose PRs are routinely left to rot rather than closed would have an artificially high merge rate.
Pocket vetoes are common. Many maintainers use them as a softer alternative to explicit rejection, especially for PRs that aren't harmful but aren't wanted.
Prior work
The rejection awareness study (on experiments/ge-validation) and Bad Egg experiments (bad-egg branch) only considered MERGED and CLOSED states. Open PRs were never queried or analyzed. No prior experiment in this repo has addressed pocket vetoes.
Complexity
Defining a pocket veto requires a staleness threshold (e.g., open for 90+ days with no activity). But the right threshold varies by repo. A PR open for 6 months in a fast-moving repo is dead; the same in a slow-moving repo might just be waiting for a release cycle. This likely requires per-repo calibration based on each repo's typical PR lifecycle, which is infrastructurally more complicated than the current approach of pulling a single totalCount from the GraphQL API.
Proposed investigation
- Query open PR counts and ages for the existing author population
- Characterize the distribution of open-PR staleness across repos
- Test whether a repo-calibrated staleness threshold (e.g., open longer than 2x the repo's median time-to-close) changes merge rate distributions meaningfully
- Evaluate impact on v3 scoring accuracy if stale open PRs are counted as rejections
- Determine whether a universal threshold is viable or per-repo calibration is required