Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 39 additions & 1 deletion .github/workflows/build-and-release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,44 @@ env:
DOTNET_NOWARN: CS1570%3BCS1571%3BCS1572%3BCS1573%3BCS1574%3BCS1587%3BCS1591%3BCS1711%3BCS1734%3BCS8981%3BNU5048

jobs:
api-coverage:
name: NumPy ↔ NumSharp API coverage
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v4

- name: Setup .NET
uses: actions/setup-dotnet@v4
with:
dotnet-version: 8.0.x

- name: Setup Python
uses: actions/setup-python@v5
with:
python-version: '3.12'

- name: Install pinned NumPy reference
run: python -m pip install numpy==2.4.2

- name: Generate coverage artifact
run: python coverage/generate_coverage.py --output artifacts/numpy-numsharp-coverage

- name: Audit latest NumPy documentation links
run: python coverage/audit_documentation.py artifacts/numpy-numsharp-coverage/coverage.json

- name: Verify checked-in dashboard data
run: diff --recursive --unified coverage/generated artifacts/numpy-numsharp-coverage

- name: Upload coverage artifact
uses: actions/upload-artifact@v4
if: always()
with:
name: numpy-numsharp-api-coverage
path: artifacts/numpy-numsharp-coverage/
if-no-files-found: error
retention-days: 14

test:
strategy:
fail-fast: false
Expand Down Expand Up @@ -90,7 +128,7 @@ jobs:
retention-days: 5

validate-release:
needs: test
needs: [test, api-coverage]
if: startsWith(github.ref, 'refs/tags/v')
runs-on: ubuntu-latest
outputs:
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ on:
branches: ["master", "main"]
paths:
- 'src/**'
- 'coverage/generated/**'
- 'docs/website-src/**'
workflow_dispatch:

Expand Down
10 changes: 10 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,16 @@ NumSharp focuses on:
raw reports, and subsystem matrices. See the
[benchmark dashboard](https://scisharp.github.io/NumSharp/docs/benchmarks-dashboard.md).

## Features Support and Implementation Map

[NumSharp's Coverage & Support Dashboard](https://scisharp.github.io/NumSharp/docs/coverage-support-dashboard.html) is presenting the full implementation
roadmap to complete 100% NumPy porting with an explorer allowing you to quickly check your favorite functions!

<p align="center">
<a href="https://scisharp.github.io/NumSharp/docs/coverage-support-dashboard.html"><img alt="NumSharp API coverage surface scoreboard" src="docs/website-src/images/coverage-support-surface-scoreboard.png" height="320"></a>
<a href="https://scisharp.github.io/NumSharp/docs/coverage-support-dashboard.html"><img alt="NumSharp API coverage capability map" src="docs/website-src/images/coverage-support-capability-map.png" height="320"></a>
</p>

## Performance

[NumSharp benchmarks](https://scisharp.github.io/NumSharp/docs/benchmarks-dashboard.html) are published as tracked release snapshots, not ad hoc
Expand Down
36 changes: 36 additions & 0 deletions coverage/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
# NumPy ↔ NumSharp API coverage

This directory is the reproducible source for NumSharp's public API coverage artifact. It compares the public exports of pinned NumPy **2.4.2** with the public surface of the compiled NumSharp assembly.

## Generate or verify

```bash
python -m pip install numpy==2.4.2
python coverage/generate_coverage.py
python coverage/generate_coverage.py --check
python coverage/audit_documentation.py
```

Generated, reviewable outputs live in `coverage/generated/`:

- `coverage.json` — complete machine-readable inventory used by the documentation dashboard.
- `coverage.csv` — flat data for spreadsheets and downstream tooling.
- `summary.md` — human-readable totals and the highest-priority gaps.
- `manifest.json` — schema, tool versions, scope, and counting rules.

CI generates a fresh copy under `artifacts/numpy-numsharp-coverage/`, validates every headline-scope link against NumPy's official latest-stable Sphinx inventory, compares the result byte-for-byte with the checked-in dashboard data, and uploads the fresh directory as the `numpy-numsharp-api-coverage` artifact.

## What the numbers mean

The default denominator includes NumPy top-level callables, `ndarray` methods and properties, and callables from `numpy.random`, `numpy.linalg`, and `numpy.fft`. NumPy types, constants, and modules are catalogued but do not affect the headline percentage. NumSharp-only APIs are catalogued separately and also do not affect it.

Platform-conditional extended-precision aliases (`float96`, `float128`, `complex192`, and `complex256`) are excluded so the artifact is byte-identical across Windows and Linux. NumPy's portable `longdouble` and `clongdouble` names remain catalogued.

- **Exact** — the corresponding NumSharp surface has the same public member name.
- **Alias** — a reviewed or mechanically safe C# equivalent exists under another name or surface.
- **Partial** — an API exists, but the reviewed mapping has a known semantic limitation.
- **Unsupported** — a public compatibility symbol exists but does not implement the NumPy capability.
- **Missing** — no NumSharp public API mapping was found.
- **NumSharp-only** — an unmatched public member declared by `np`, `NDArray`, or `NumPyRandom`; these rows link directly to their declaration on GitHub.

API availability is not a blanket behavioral-parity claim. Exact edge-case, dtype, layout, and signature parity still requires differential tests. Record reviewed exceptions and cross-surface aliases in `coverage/overrides.json`; the generator validates every referenced NumSharp target.
78 changes: 78 additions & 0 deletions coverage/audit_documentation.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
#!/usr/bin/env python3
"""Audit dashboard links against NumPy's official latest-stable Sphinx inventory."""

from __future__ import annotations

import argparse
import json
import time
import urllib.request
import zlib
from pathlib import Path


INVENTORY_URL = "https://numpy.org/doc/stable/objects.inv"
DOCS_BASE_URL = "https://numpy.org/doc/stable/"


def parse_args() -> argparse.Namespace:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument(
"coverage_json",
nargs="?",
type=Path,
default=Path(__file__).resolve().parent / "generated" / "coverage.json",
)
return parser.parse_args()


def download_inventory() -> bytes:
request = urllib.request.Request(INVENTORY_URL, headers={"User-Agent": "NumSharp coverage audit/1.0"})
error: Exception | None = None
for attempt in range(3):
try:
with urllib.request.urlopen(request, timeout=30) as response:
return response.read()
except Exception as current_error: # pragma: no cover - network-dependent retry
error = current_error
if attempt < 2:
time.sleep(1 + attempt)
raise SystemExit(f"Unable to download NumPy documentation inventory: {error}")


def documented_pages(raw: bytes) -> set[str]:
position = 0
for _ in range(4):
position = raw.index(b"\n", position) + 1
entries = zlib.decompress(raw[position:]).decode("utf-8").splitlines()
pages: set[str] = set()
for entry in entries:
parts = entry.split(" ", 4)
if len(parts) != 5:
continue
uri = parts[3].replace("$", parts[0]).split("#", 1)[0]
pages.add(DOCS_BASE_URL + uri)
return pages


def main() -> None:
args = parse_args()
payload = json.loads(args.coverage_json.read_text(encoding="utf-8"))
pages = documented_pages(download_inventory())
rows = [
row for row in payload["rows"]
if row["origin"] == "numpy" and row["in_default_scope"]
]
missing = [
(row["id"], row["documentation_url"])
for row in rows
if not row["documentation_url"] or row["documentation_url"].split("#", 1)[0] not in pages
]
if missing:
details = "\n".join(f" - {api_id}: {url or '<empty>'}" for api_id, url in missing)
raise SystemExit(f"{len(missing)} coverage links are absent from NumPy's official inventory:\n{details}")
print(f"Validated {len(rows)} latest-stable NumPy documentation links against objects.inv.")


if __name__ == "__main__":
main()
Loading
Loading