Skip to content

docs(parity): regenerate runtime-parity-gaps from a committed generator#5204

Open
TheHypnoo wants to merge 2 commits into
mainfrom
chore/parity-gaps-generator
Open

docs(parity): regenerate runtime-parity-gaps from a committed generator#5204
TheHypnoo wants to merge 2 commits into
mainfrom
chore/parity-gaps-generator

Conversation

@TheHypnoo

@TheHypnoo TheHypnoo commented Jun 15, 2026

Copy link
Copy Markdown
Member

What

Replaces the hand-rolled, badly inaccurate docs/runtime-parity-gaps.md with a doc regenerated from a committed, reproducible generator (scripts/gen_parity_gaps.py).

Why

The old doc's generator was never committed and never parsed the API manifest (crates/perry-api-manifest/src/entries.rs). Since that manifest is the authoritative set of compile-time-dispatched APIs (a CI test asserts it mirrors NATIVE_MODULE_TABLE), missing it meant counting almost everything as a gap.

  • Claimed total: ~1922 gaps. Actual: ~296 across 49 node:* modules (2222 covered / 296 gap) — a ~6x over-count.
  • Whole modules were falsely listed as "zero coverage" (node:dns, node:dns/promises, node:v8, node:inspector, node:string_decoder), though all are implemented.
  • node:crypto: doc said 14 covered / 124 gap; reality is 133 / 5 (the 5 match test-parity/node-suite/crypto/README.md: X509Certificate, KeyObject.from/CryptoKey identity, setEngine, fips).

How

gen_parity_gaps.py derives coverage from four sources: manifest method/property rows, compound Expr::* HIR variants, js_* FFI exports, and module-gated dispatch literals. Re-run with python3 scripts/gen_parity_gaps.py --emit.

The matcher is conservative and self-documents its caveats (module-gated dispatch to avoid stub false-positives; an audited MANUAL_COVERAGE table for APIs dispatched in generic, non-module-named files such as KeyObject access in perry-runtime/src/object/field_get_set.rs).

Summary by CodeRabbit

  • Chores
    • Added internal tooling to automatically regenerate the runtime parity gap report from the project’s authoritative sources. It analyzes module/API coverage and highlights missing or partially supported areas.
  • Documentation
    • Synchronizes the published parity-gap documentation by generating an updated runtime-parity-gaps markdown report in one step.

The previous gap doc was produced by an ad-hoc, uncommitted script whose
matcher never parsed crates/perry-api-manifest/src/entries.rs. As a result
nearly every manifest-covered API was counted as a gap: the doc claimed
~1922 gaps when the real figure is ~296 across node:* modules.

Add scripts/gen_parity_gaps.py, which reconciles docs/runtime-parity.md
(the API inventory) against four coverage sources - manifest method/property
rows, compound Expr::* HIR variants, js_* FFI exports, and module-gated
dispatch literals - and regenerate docs/runtime-parity-gaps.md from it.

Examples of corrected counts:
- node:crypto: 124 -> 5 real gaps (X509Certificate, KeyObject.from, setEngine, fips)
- node:dns / dns/promises / v8 / inspector / string_decoder: were listed as
  "zero coverage" but are implemented
- node:http: 89 -> 1; node:zlib: ~75 -> 1
@coderabbitai

coderabbitai Bot commented Jun 15, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: bdcb9f4d-7894-46a7-81e5-8bc666431842

📥 Commits

Reviewing files that changed from the base of the PR and between 722cf75 and 9aa9512.

📒 Files selected for processing (1)
  • scripts/gen_parity_gaps.py
🚧 Files skipped from review as they are similar to previous changes (1)
  • scripts/gen_parity_gaps.py

📝 Walkthrough

Walkthrough

Adds scripts/gen_parity_gaps.py, a new 330-line CLI script that regenerates Perry's runtime parity gap report. It parses docs/runtime-parity.md inventory tables, entries.rs manifest coverage, HIR Expr::* variants, and Rust FFI/dispatch sources, classifies each API member as covered or a gap, and writes docs/runtime-parity-gaps.md.

Changes

Parity Gap Report Generator

Layer / File(s) Summary
Paths, constants, regex helpers, and casing utilities
scripts/gen_parity_gaps.py
Defines repo-relative source paths, MANUAL_COVERAGE overrides dict, compiled regex patterns for manifest/HIR/FFI scanning, and snake/pascal name conversion helpers.
Inventory, manifest, and HIR/FFI source parsing
scripts/gen_parity_gaps.py
parse_inventory() reads docs/runtime-parity.md tables; extract_member() normalizes member names; parse_manifest() builds coverage sets from entries.rs; parse_expr_variants() collects HIR Expr::* variant identifiers; scan_ffi_and_dispatch() extracts js_* FFI names and dispatch string literals from Rust source trees.
Coverage classifier and compute aggregation
scripts/gen_parity_gaps.py
covered() applies a priority chain: manual override → constants/events → Expr::* membership → FFI name match → dispatch-literal match gated by module_has_impl(). compute() aggregates per-module covered/gap totals and missing signatures.
Markdown report generation and CLI
scripts/gen_parity_gaps.py
emit_doc() formats per-module gap sections with summary totals, methodology, and BEHAVIORAL_NOTE caveats. main() handles --module and --emit flags, printing a summary table or writing docs/runtime-parity-gaps.md.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

  • PerryTS/perry#5148: Adds the scripts/gen_parity_gaps.py generator that produces the parity gap report, while that PR updates docs/runtime-parity-gaps.md to reflect current Node.js API coverage status.

Poem

🐇 Hop through the modules, one by one,
With regex and rust-scans nearly done,
The manifest checked, the HIR Expr found,
Each gap now tracked and coverage bound.
The parity gaps doc writes itself — hooray!
No more manual tallies to dismay. 🌿

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 25.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately summarizes the main change: regenerating the runtime-parity-gaps documentation from a committed, reproducible generator script.
Description check ✅ Passed The description comprehensively covers the what, why, and how of the changes, with specific metrics and examples of accuracy improvements.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch chore/parity-gaps-generator

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (2)
scripts/gen_parity_gaps.py (2)

37-40: 💤 Low value

Consider iterable unpacking for cleaner list construction.

Using [*static, *dynamic] unpacking is more idiomatic than list concatenation.

♻️ Suggested refactor
-SRC_DIRS = [
-    ROOT / "crates" / "perry-runtime" / "src",
-    ROOT / "crates" / "perry-stdlib" / "src",
-] + sorted((ROOT / "crates").glob("perry-ext-*/src"))
+SRC_DIRS = [
+    ROOT / "crates" / "perry-runtime" / "src",
+    ROOT / "crates" / "perry-stdlib" / "src",
+    *sorted((ROOT / "crates").glob("perry-ext-*/src")),
+]
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@scripts/gen_parity_gaps.py` around lines 37 - 40, The SRC_DIRS list
construction uses the concatenation operator (+) to combine a static list with
dynamically sorted glob results. Refactor this to use iterable unpacking syntax
instead by moving the sorted((ROOT / "crates").glob("perry-ext-*/src"))
expression inside the list literal and unpacking it with the * operator, making
the code more idiomatic and readable.

Source: Linters/SAST tools


136-152: 💤 Low value

Unused literals collection.

scan_ffi_and_dispatch() collects dispatch literals (line 150-151), but this return value is discarded in compute() (assigned to _literals and never used). The covered() function re-scans files with its own patterns instead.

Either remove the unused collection or, if it was intended to optimize the dispatch-literal check in covered(), consider passing literals to covered() to avoid repeated regex searches over the same files.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@scripts/gen_parity_gaps.py` around lines 136 - 152, The
`scan_ffi_and_dispatch()` function collects dispatch literals via regex pattern
matching, but the returned literals are discarded (assigned to `_literals`) in
the `compute()` function and never used. Meanwhile, the `covered()` function
performs its own regex searches instead of reusing the pre-collected literals.
Either remove the literal collection logic from `scan_ffi_and_dispatch()` and
its return value, or refactor to pass the literals set to the `covered()`
function and update it to use the pre-collected literals instead of rescanning
files with its own regex patterns.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@scripts/gen_parity_gaps.py`:
- Around line 79-82: In the node module path extraction logic where the regex
search result is processed, line 81 contains a conditional assignment to cur
that is immediately overwritten by line 82. The conditional logic on line 81
checking whether "/" is in title and conditionally splitting the path has no
effect since line 82 unconditionally assigns nm.group(1) to cur. Remove the dead
code on line 81 entirely, keeping only line 82 which correctly assigns the full
path (e.g., fs/promises) as indicated by the comment.

---

Nitpick comments:
In `@scripts/gen_parity_gaps.py`:
- Around line 37-40: The SRC_DIRS list construction uses the concatenation
operator (+) to combine a static list with dynamically sorted glob results.
Refactor this to use iterable unpacking syntax instead by moving the
sorted((ROOT / "crates").glob("perry-ext-*/src")) expression inside the list
literal and unpacking it with the * operator, making the code more idiomatic and
readable.
- Around line 136-152: The `scan_ffi_and_dispatch()` function collects dispatch
literals via regex pattern matching, but the returned literals are discarded
(assigned to `_literals`) in the `compute()` function and never used. Meanwhile,
the `covered()` function performs its own regex searches instead of reusing the
pre-collected literals. Either remove the literal collection logic from
`scan_ffi_and_dispatch()` and its return value, or refactor to pass the literals
set to the `covered()` function and update it to use the pre-collected literals
instead of rescanning files with its own regex patterns.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: b18b4e46-9f13-4404-b703-5940d1320043

📥 Commits

Reviewing files that changed from the base of the PR and between a1d9ce6 and 722cf75.

📒 Files selected for processing (2)
  • docs/runtime-parity-gaps.md
  • scripts/gen_parity_gaps.py

Comment thread scripts/gen_parity_gaps.py Outdated
Line was immediately overwritten by the unconditional assignment on the
next line; the split-on-slash branch never took effect. Behavior-neutral
(regenerated doc is byte-identical).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant