docs: post-classical metrics research foundation#175
Conversation
Survey of heuristic source-code metrics beyond mehen's current classical suite, produced from a read-only inventory of the existing metrics plus a deep-research pass (26 primary sources; top claims adversarially verified). Covers four categories with per-metric fit cards (definition, primary citation, single-file/deterministic, dependency/trained-artifact needs, effort): - Static structural heuristics: DepDegree, Shannon token/AST-edge entropy, indentation moments. - Learned readability: Posnett (portable coefficients) vs Buse-Weimer / Scalabrino (require shipped trained models). - Code naturalness: Hindle cross-entropy (needs a bundled per-language LM; poor fit for the dependency-light contract). - Git/history process metrics: churn, code age, ownership, change coupling, hotspots, Time-Weighted Risk -- the biggest gap, since mehen-git is diff-only today. Includes a prioritization table and a dedicated section (§9) recommending how to change the GitHub Action PR-comment default metric set, grounded in the current DEFAULT_METRICS in mehen-engine. Design doc only; no code or behavior changes.
|
Warning Review limit reached
Next review available in: 56 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Run ID: 📒 Files selected for processing (2)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request adds a comprehensive research foundation and design proposal document for post-classical heuristic source-code metrics in the mehen project. The document details static structural heuristics, learned readability models, code naturalness, and Git/history process metrics, and suggests updating the default metrics shown in GitHub Action comments. The reviewer correctly pointed out a discrepancy: the document lists halstead.effort as an existing metric, but it is not currently defined in KNOWN_METRICS in the engine crate, meaning code changes are required before it can be used as proposed.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| |---|---|---|---| | ||
| | Cognitive | `cognitive` | control-flow *understandability* (keep the better of the two flow metrics) | ✅ exists | | ||
| | ABC | `abc` | computational volume (assignments/branches/conditions) | ✅ exists | | ||
| | Halstead effort | `halstead.effort` | vocabulary/operator burden | ✅ exists | |
There was a problem hiding this comment.
The table lists halstead.effort as an existing selector (✅ exists), and section 9.3 describes this as a 'zero-new-metric change'. However, in crates/mehen-engine/src/metric_selector.rs, halstead.effort is not currently defined in KNOWN_METRICS or mapped in metric_set_key_for (only halstead.volume is present).
To make halstead.effort selectable, it will need to be added to KNOWN_METRICS and metric_set_key_for in crates/mehen-engine/src/metric_selector.rs first.
There was a problem hiding this comment.
✅ Verified against the code and corrected in a0a4a7f — you're right. The walker publishes the halstead.effort value (state.rs), but only halstead.volume is registered as a selector (KNOWN_METRICS + metric_set_key_for), and halstead.effort isn't namespaced, so it would hit the "Unknown metric, skipping" path rather than render. §9.3 now marks that row "computed but not yet selectable", documents the two one-line registrations needed, and is reframed as "no new metric formula" rather than "zero code". The other four columns (cognitive, abc, loc.lloc, mi.visual_studio) are genuinely selectable, so they stay a pure-config change. Thanks @gemini-code-assist!
There was a problem hiding this comment.
Thank you for the correction, @tinovyatkin. I have reviewed the updated document (§9.3) and the clarification regarding the halstead.effort selector registration is accurate and well-documented. The distinction between a 'no new metric formula' change and a 'zero code' change is a helpful clarification for implementation planning. I have no further comments on this thread.
Binary Size ReportMeasured artifact: release Linux
Size historyxychart-beta
title "Binary size (bytes)"
x-axis ["build(deps): bump gix from 0.8", "build(deps): bump log from 0.4", "build(deps): bump the ruff gro", "build(deps): bump the mago gro", "build: sync workspace crate ve", "build(deps): bump the oxc grou", "build(deps): bump tree-sitter ", "build(deps): bump camino from ", "feat: add Java analyzer and up", "build(deps): bump sqruff-lib-d", "build(deps): bump the mago gro", "build(deps): bump ra_ap_syntax", "build(deps): bump the oxc grou", "build(deps): bump mago-syntax-", "chore(main): release 1.3.0 (#1", "docs: surface SQL + ANTLR acro", "refactor(markdown): replace tr", "This PR"]
y-axis "Bytes"
bar [29154336, 29184128, 29154616, 29184664, 29184664, 29194704, 29194696, 29194704, 30767472, 30833880, 30833888, 30834016, 30811952, 30811944, 30811984, 31135680, 31113224, 31113224]
|
Correct §9.3: halstead.effort is computed/published by the walker but not registered as a selector (only halstead.volume is in KNOWN_METRICS / metric_set_key_for), and it is not a namespaced key, so it would be skipped as "Unknown metric" if added to DEFAULT_METRICS. Mark the row "computed but not yet selectable", document the two one-line registrations required, and reframe §9.3 as "no new metric formula" rather than "zero code". The other four columns (cognitive, abc, loc.lloc, mi.visual_studio) are genuinely selectable, so remain a pure-config change. Addresses: #175 (comment)
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a0a4a7f4ef
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| **code-maat is GPL-v3** and its analyses evolved into the **proprietary CodeScene** product. mehen | ||
| may **reimplement the open, reproducible formulas** (all quoted above are published in Tornhill's | ||
| books and the code-maat README), but must **not copy code-maat's Clojure source** into a | ||
| permissively-licensed Rust CLI. CodeScene's *prioritization/ranking algorithms* are proprietary and |
There was a problem hiding this comment.
This sentence says mehen is a permissively licensed CLI, but the workspace declares AGPL-3.0-only in Cargo.toml and the Rust source headers. If the history layer is implemented from §6.8, the GPL-v3/code-maat reuse guidance starts from the wrong compatibility premise; replace this with AGPL-specific guidance or a legal-review warning rather than describing the project as permissive.
Useful? React with 👍 / 👎.
|
|
||
| | Column | Selector | Dimension it uniquely covers | Status | | ||
| |---|---|---|---| | ||
| | Cognitive | `cognitive` | control-flow *understandability* (keep the better of the two flow metrics) | ✅ selectable | |
There was a problem hiding this comment.
When §9.3 is used as the blueprint for the GitHub Action default columns, this table would label the Cognitive column as covering control-flow understandability. That contradicts the document's own caveat that metrics must not be branded as understandability/comprehensibility signals, and would bake misleading product copy into the default PR comment; call this control-flow complexity or review effort instead.
Useful? React with 👍 / 👎.
typos flagged "IST" (Information and Software Technology, the venue for the Radjenović et al. 2013 citation) as a misspelling of IS/IT/ITS/SIT/LIST. Add it to [default.extend-words] alongside the other proper-noun/acronym entries so the CI typos check passes without altering the citation.
📊 Source Code Metrics (this PR vs
|
1 participant
What
Adds
design-docs/mehen_post_classical_metrics_research_foundation.md— a research-backed survey of heuristic source-code metrics beyond mehen's current classical suite (cyclomatic, cognitive, LOC family, Halstead, MI, ABC, NARGS/NOM/NEXIT/NPA/NPM/WMC), with a concrete proposal for candidate additions.Design doc only — no code or behavior changes.
How it was produced
sql.*,markdown.*, and the diff/report defaults) to pin down the real gaps.What's in it
Four categories, each candidate carrying a fit card (what it measures · primary citation · single-file/deterministic? · dependency/trained-artifact needs · effort S/M/L/XL):
mehen-gitis diff-only today. Includes a GPL/licensing note on code-maat.Plus:
DEFAULT_METRICSinmehen-engine: shows the current 5 columns collapse to ~2.5 dimensions, and proposes both a one-column add (abc, zero new code) and a redundancy-removing replacement, with a history-backed end-state.Reviewer notes / decisions to weigh
cyclomaticfrom the default comment columns (keep cognitive; they correlate strongly). This trades familiarity for reduced redundancy — a product call, flagged as such in the doc.No metric is implemented here — this is the foundation doc to decide which to build next.