Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -45,3 +45,6 @@ docs/superpowers/
.rp1/context/meta.json
.rp1/settings.toml
# rp1:end

# rp1 workspace artifacts — never commit
.rp1/
22 changes: 14 additions & 8 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,17 @@
# Changelog

## Unreleased

- docs: README credibility pass — CI/downloads badges, production-scale benchmark callout, architecture diagram, maintainer credit; "For organizations" now states that the SR 11-7/SR 26-2, EU AI Act Annex IV, and NIST AI RMF validation profiles ship in the OSS core
- docs: quickstart and all Ledger examples persist to `./ledger.db` (the previous `./inventory.db` name collided with the CLI's legacy Inventory-format default and crashed `model-ledger list`); quickstart `history()` example now shows (and CI-asserts) newest-first ordering
- fix(cli): bare installs get a one-line install hint from `model-ledger --help` instead of a `ModuleNotFoundError` traceback (typer/rich live in the `[cli]` extra)
- fix(cli): inventory commands detect a Ledger event-log database and exit with guidance instead of an sqlite traceback
- fix(cli): `model-ledger validate` with an unknown profile exits with the available profile names instead of a `ValueError` traceback
- refactor!: remove the `scanner` module (`Scanner`, `InventoryScanner`, `ModelCandidate`, `ScanReport`), deprecated since v0.4.0 — use `SourceConnector` + `DataNode` + `Ledger.add()/connect()`
- chore: PyPI metadata — Development Status classifier to Beta; add `mcp`, `model-context-protocol`, `ai-governance`, `eu-ai-act`, `nist-ai-rmf`, `sr-26-2` keywords
- docs: add SECURITY.md (private vulnerability reporting)
- chore: stale-reference sweep — MCP tool count 6→8 in docstrings and CLAUDE.md, `v0.3.0` markers out of SDK docstrings, "Task 11" comments out of rest/app.py; fold two stale Unreleased changelog blocks into the releases that shipped them (v0.7.3, v0.4.8)

## v0.7.7

- feat: `SnowflakeLedgerBackend` accepts a `connection_factory` and self-heals on auth-token expiry (Snowflake errno 390114) — on a detected expiry it obtains a fresh connection, swaps it in, and retries the same statement exactly once. Composes with `client_session_keep_alive` heartbeats as the backstop for residual expiries. Backward compatible: with only `connection`, behavior is unchanged. (#24)
Expand All @@ -19,9 +31,6 @@

- Add `metadata: dict` field to `ModelRef`. Thread through `register()` and `register_group()`. Replaces the unintended per-link broadcast of `register_group(metadata=...)` to member links; metadata now lives on the composite ModelRef itself. Backward compatible: existing data loads with `metadata={}`.
- Add optional `model_types` parameter to `composite_summary()` so callers can include custom composite-shaped types (e.g., `ml_model`, `heuristic`) beyond the default `"composite"`. Backward compatible.

## Unreleased

- feat: `RecordOutput.model_hash` — the `/record` response now carries the server's canonical model hash so HTTP clients can reconcile their local `ModelRef` with authoritative server state
- fix: `HttpLedgerBackend.save_model` adopts the server's canonical hash (reassigns `model.model_hash` on the incoming `ModelRef` and caches only the server hash) so follow-up hash-based flows like `Ledger.tag()` round-trip correctly even from a fresh backend instance
- feat: `POST /tag` and `GET /tags/{model_name}` REST endpoints — create, move, and list tags over HTTP
Expand All @@ -46,14 +55,11 @@
- feat: `github_connector()` — discover models from config files in GitHub repos
- feat: Connector factories return `SourceConnector` instances for composability

## Unreleased

- fix: deduplicate `ModelNotFoundError` — use canonical class from `core.exceptions`
- test: add coverage for `'value' AS model_name` extraction pattern

## v0.4.8

- fix: exclude volatile timestamps from content hash dedup
- fix: deduplicate `ModelNotFoundError` — use canonical class from `core.exceptions`
- test: add coverage for `'value' AS model_name` extraction pattern

## v0.4.7

Expand Down
6 changes: 3 additions & 3 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,8 +24,8 @@ Open-source model inventory and governance framework. Apache-2.0.
- `src/model_ledger/backends/http.py` — HttpLedgerBackend (MCP → REST API pass-through)
- `src/model_ledger/backends/json_files.py` — JsonFileLedgerBackend (git-friendly)
- `src/model_ledger/sdk/ledger.py` — Ledger SDK with last_seen, change_detected/change_occurred
- `src/model_ledger/tools/` — 6 agent tool functions (record, query, investigate, trace, changelog, discover)
- `src/model_ledger/mcp/server.py` — MCP server (FastMCP, 6 tools + 3 resources)
- `src/model_ledger/tools/` — 8 agent tool functions (record, query, investigate, trace, changelog, discover, tag, list_tags)
- `src/model_ledger/mcp/server.py` — MCP server (FastMCP, 8 tools + 3 resources)
- `src/model_ledger/rest/app.py` — REST API (FastAPI, create_app() factory)

### v0.4.x (DataNode graph)
Expand All @@ -40,7 +40,7 @@ Open-source model inventory and governance framework. Apache-2.0.
- `src/model_ledger/adapters/cron.py` — Cron expression translation

### v0.3.0 (event-log paradigm)
- `src/model_ledger/scanner/` — Scanner protocol, ModelCandidate, InventoryScanner, ScannerRegistry
- Scanner protocol removed (deprecated v0.4.0, removed post-v0.7.7) — use SourceConnector + DataNode + Ledger.add()/connect()

### v0.2.0 (legacy — retained for reference)
- `src/model_ledger/core/models.py` — Model, ModelVersion, ComponentNode
Expand Down
51 changes: 46 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,11 @@

**git for models** — know what models you have deployed, where they run, what they depend on, and what changed.

[![CI](https://github.com/block/model-ledger/actions/workflows/ci.yml/badge.svg)](https://github.com/block/model-ledger/actions/workflows/ci.yml)
[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](LICENSE)
[![Python](https://img.shields.io/badge/python-3.10+-blue.svg)](https://python.org)
[![PyPI](https://img.shields.io/pypi/v/model-ledger)](https://pypi.org/project/model-ledger/)
[![Downloads](https://img.shields.io/pypi/dm/model-ledger)](https://pypistats.org/packages/model-ledger)
[![Docs](https://img.shields.io/badge/docs-block.github.io/model--ledger-7a1a1a.svg)](https://block.github.io/model-ledger/)

📖 **[Documentation](https://block.github.io/model-ledger/)** ·
Expand All @@ -21,6 +23,9 @@ Unlike registries tied to a single platform (MLflow, SageMaker, W&B), it spans a
them — as one connected graph — and it's built to be driven by AI agents through a
native MCP server.

Benchmarked at production scale: full inventory reconstruction over a ledger of 28.8k
models and 212k events runs in under a second ([CHANGELOG, v0.7.4](CHANGELOG.md)).

## Install

```bash
Expand Down Expand Up @@ -75,21 +80,57 @@ runs in CI:
- **[Agents (MCP)](https://block.github.io/model-ledger/guides/agents/)** — the eight-tool agent surface, with a worked transcript
- **[Connectors](https://block.github.io/model-ledger/guides/connectors/)** — discover from SQL, REST, GitHub, or your own platform
- **[Backends](https://block.github.io/model-ledger/guides/backends/)** — in-memory, SQLite, JSON, Snowflake, or remote HTTP
- **[Governance](https://block.github.io/model-ledger/governance/)** — how the primitives map to SR 26‑2, the EU AI Act, and NIST
- **[Governance](https://block.github.io/model-ledger/governance/)** — how the primitives map to SR 11‑7/SR 26‑2, the EU AI Act, and NIST AI RMF
- **[API reference](https://block.github.io/model-ledger/reference/)** — generated from the source

## Architecture

```mermaid
flowchart LR
subgraph Sources
C1[SQL / REST / GitHub / Prefect<br/>connectors]
end
subgraph Core
L[Ledger<br/>append-only event log,<br/>point-in-time reconstruction]
G[Dependency graph]
V[Compliance profiles<br/>SR 11-7/SR 26-2 · EU AI Act · NIST AI RMF]
end
subgraph Surfaces
S1[Python SDK]
S2[CLI]
S3[REST API]
S4[MCP server · 8 tools]
end
B1[(in-memory · SQLite · JSON ·<br/>Snowflake · remote HTTP)]
C1 --> L
L --> G
L --> V
L --- B1
S1 --> L
S2 --> L
S3 --> L
S4 --> L
```

## For organizations

The OSS core handles discovery, graph building, change tracking, storage, and the agent
protocol. Your internal package provides the thin layer on top — connector configs,
custom connectors for internal platforms, authentication, and compliance profiles. Thin
config and credentials, not reimplemented logic.
The OSS core handles discovery, graph building, change tracking, storage, the agent
protocol, and compliance validation — the SR 11‑7/SR 26‑2, EU AI Act Annex IV, and
NIST AI RMF profiles ship in `model_ledger.validate`. Your internal package provides
only the thin layer on top: connector configs, custom connectors for internal
platforms, and credentials. Thin config, not reimplemented logic.

## Contributing

See [CONTRIBUTING.md](https://github.com/block/model-ledger/blob/main/CONTRIBUTING.md).
All commits require DCO sign-off.

## Security

See [SECURITY.md](SECURITY.md) for how to report vulnerabilities privately.

## License

Apache-2.0. See [LICENSE](https://github.com/block/model-ledger/blob/main/LICENSE).

Created and maintained by [Vignesh Narayanaswamy](https://github.com/vigneshnarayanaswamy) at Block.
29 changes: 29 additions & 0 deletions SECURITY.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
# Security Policy

## Reporting a vulnerability

Please do not report security vulnerabilities through public GitHub issues.

Instead, report them privately via the repository's **Security tab → Report a
vulnerability** ([GitHub's private vulnerability reporting](https://docs.github.com/en/code-security/security-advisories/guidance-on-reporting-and-writing-information-about-vulnerabilities/privately-reporting-a-security-vulnerability)).

For assistance or escalation, contact the
[Block Open Source Governance Committee](mailto:open-source-governance@block.xyz).

## Scope notes

model-ledger stores model inventory metadata. A few boundaries worth knowing when
assessing impact:

- The ledger trusts its callers: `record()` accepts arbitrary payloads, and the REST
API does not ship authentication — deployments are expected to run it behind their
own auth layer (see the [backends guide](https://block.github.io/model-ledger/guides/backends/)).
- Snapshot hashes provide content addressing and tamper evidence for snapshot
payloads, not a cryptographic chain over the full event history — see
[guarantees](https://block.github.io/model-ledger/concepts/guarantees/) for the
precise integrity model.

## Supported versions

Security fixes land on `main` and ship in the next release; older releases are not
patched retroactively.
2 changes: 1 addition & 1 deletion docs/concepts/composite.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ validation. Composites are the layer no plain registry or catalog models.

```python
from model_ledger import Ledger
ledger = Ledger.from_sqlite("./inventory.db")
ledger = Ledger.from_sqlite("./ledger.db")

group = ledger.register_group(
name="Credit Scorecard",
Expand Down
2 changes: 1 addition & 1 deletion docs/concepts/snapshot.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ A model splits into two things:

```python
from model_ledger import Ledger
ledger = Ledger.from_sqlite("./inventory.db")
ledger = Ledger.from_sqlite("./ledger.db")

ref = ledger.register(
name="fraud_scoring", owner="risk-team",
Expand Down
2 changes: 1 addition & 1 deletion docs/governance.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ true then?" is always reconstructable.**
```python
from model_ledger import Ledger

ledger = Ledger.from_sqlite("./inventory.db")
ledger = Ledger.from_sqlite("./ledger.db")

# Identity + risk tier — the minimum a regulator needs
ledger.register(
Expand Down
2 changes: 1 addition & 1 deletion docs/guides/agents.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ pip install "model-ledger[mcp]"

# Claude Code (one time). Drop --demo to start empty; add a backend to persist.
claude mcp add model-ledger -- model-ledger mcp --demo
claude mcp add model-ledger -- model-ledger mcp --backend sqlite --path ./inventory.db
claude mcp add model-ledger -- model-ledger mcp --backend sqlite --path ./ledger.db
```

The server speaks stdio and works with any MCP client (Claude Desktop, Goose, Cursor).
Expand Down
4 changes: 2 additions & 2 deletions docs/guides/backends.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ from model_ledger.backends.json_files import JsonFileLedgerBackend
from model_ledger.backends.http import HttpLedgerBackend

Ledger() # in-memory
Ledger.from_sqlite("./inventory.db") # SQLite
Ledger.from_sqlite("./ledger.db") # SQLite
Ledger(JsonFileLedgerBackend("./inventory")) # JSON files
Ledger.from_snowflake(conn, schema="DB.MODEL_LEDGER") # Snowflake
Ledger(HttpLedgerBackend("https://model-ledger:8000")) # remote REST
Expand Down Expand Up @@ -50,7 +50,7 @@ inventory/
The CLI launches either agent or HTTP surfaces over any backend:

```bash
model-ledger serve --backend sqlite --path ./inventory.db --port 8000
model-ledger serve --backend sqlite --path ./ledger.db --port 8000
model-ledger mcp --backend snowflake --schema DB.MODEL_LEDGER
```

Expand Down
2 changes: 1 addition & 1 deletion docs/guides/connectors.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ factory connectors ship in core; anything else is a small protocol implementatio
```python
from model_ledger import Ledger, sql_connector

ledger = Ledger.from_sqlite("./inventory.db")
ledger = Ledger.from_sqlite("./ledger.db")

# Simple: read a registry table
models = sql_connector(
Expand Down
11 changes: 7 additions & 4 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,9 @@ dependency graph** automatically, and **records every change as an immutable
event**. Unlike registries tied to one platform (MLflow, SageMaker, W&B), it spans
all of them — and it's built to be driven by AI agents through a native MCP server.

Born at Block, where it runs in production inventorying ML models, heuristic rules,
and pipelines across multiple platforms for a second-line model risk program.

[Get started in 60 seconds :octicons-arrow-right-24:](quickstart.md){ .md-button .md-button--primary }
[Why a ledger, not a registry? :octicons-arrow-right-24:](#why-a-ledger-not-a-registry){ .md-button }

Expand Down Expand Up @@ -104,7 +107,7 @@ dependency edge — no hand-wiring.
```python
from model_ledger import Ledger, DataNode

ledger = Ledger.from_sqlite("./inventory.db")
ledger = Ledger.from_sqlite("./ledger.db")

ledger.add([
DataNode("segmentation", platform="etl", outputs=["customer_segments"]),
Expand All @@ -129,15 +132,15 @@ graph LR

## One operation, every surface

The SDK, the REST API, and the MCP tools are the **same six verbs** — `discover`,
`record`, `investigate`, `query`, `trace`, `changelog` (plus `tag`/`list_tags`).
The SDK, the REST API, and the MCP tools are the **same eight verbs** — `discover`,
`record`, `investigate`, `query`, `trace`, `changelog`, `tag`, `list_tags`.
Registering a model looks like this everywhere:

=== "Python"

```python
from model_ledger import Ledger
ledger = Ledger.from_sqlite("./inventory.db")
ledger = Ledger.from_sqlite("./ledger.db")

ledger.register(
name="fraud_scoring", owner="risk-team",
Expand Down
11 changes: 7 additions & 4 deletions docs/quickstart.md
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ A `DataNode` gives you the graph. [`register()`](reference/index.md) gives a mod

```python
from model_ledger import Ledger
ledger = Ledger.from_sqlite("./inventory.db")
ledger = Ledger.from_sqlite("./ledger.db")

ledger.register(
name="fraud_scoring",
Expand All @@ -96,10 +96,13 @@ ledger.register(
ledger.record("fraud_scoring", event="retrained", actor="ml-pipeline",
payload={"accuracy": 0.94, "features_added": ["velocity_24h"]})

for snap in ledger.history("fraud_scoring"):
history = ledger.history("fraud_scoring") # newest first
for snap in history:
print(snap.timestamp, snap.event_type)
# ... registered
# ... retrained
# ... registered

assert [s.event_type for s in history] == ["retrained", "registered"]
```

Every call appends an immutable [Snapshot](concepts/snapshot.md). Nothing is
Expand All @@ -114,7 +117,7 @@ from model_ledger import Ledger
from model_ledger.backends.json_files import JsonFileLedgerBackend

Ledger() # in-memory — tests & demos
Ledger.from_sqlite("./inventory.db") # zero-infra, single file
Ledger.from_sqlite("./ledger.db") # zero-infra, single file
Ledger(JsonFileLedgerBackend("./inventory")) # git-friendly JSON files
Ledger.from_snowflake(conn, schema="DB.MODEL_LEDGER") # production
```
Expand Down
2 changes: 1 addition & 1 deletion docs/recipes/discover-sql.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ so re-running on a schedule only records genuine changes.
import sqlite3
from model_ledger import Ledger, sql_connector

ledger = Ledger.from_sqlite("./inventory.db")
ledger = Ledger.from_sqlite("./ledger.db")
source = sqlite3.connect("./ml_platform.db")

models = sql_connector(
Expand Down
2 changes: 1 addition & 1 deletion docs/recipes/point-in-time.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ the inventory at any date is just a replay of the log up to that moment.
from datetime import datetime, timezone, timedelta
from model_ledger import Ledger

ledger = Ledger.from_sqlite("./inventory.db")
ledger = Ledger.from_sqlite("./ledger.db")

ledger.register(name="fraud_scoring", owner="risk-team",
model_type="ml_model", tier="high",
Expand Down
17 changes: 15 additions & 2 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,22 @@ authors = [
{ name = "Vignesh Narayanaswamy", email = "vigneshn@block.xyz" },
{ name = "Block", email = "opensource@block.xyz" },
]
keywords = ["mlops", "model-governance", "model-inventory", "model-risk", "sr-11-7", "compliance"]
keywords = [
"mlops",
"model-governance",
"model-inventory",
"model-risk",
"sr-11-7",
"sr-26-2",
"compliance",
"mcp",
"model-context-protocol",
"ai-governance",
"eu-ai-act",
"nist-ai-rmf",
]
classifiers = [
"Development Status :: 3 - Alpha",
"Development Status :: 4 - Beta",
"Intended Audience :: Developers",
"Intended Audience :: Financial and Insurance Industry",
"License :: OSI Approved :: Apache Software License",
Expand Down
3 changes: 0 additions & 3 deletions src/model_ledger/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,6 @@
from model_ledger.core.models import ComponentNode, Model, ModelVersion
from model_ledger.graph.models import DataNode, DataPort
from model_ledger.graph.protocol import SourceConnector
from model_ledger.scanner.protocol import ModelCandidate, Scanner
from model_ledger.sdk.inventory import Inventory
from model_ledger.sdk.ledger import Ledger

Expand Down Expand Up @@ -69,8 +68,6 @@
"ModelRef",
"Snapshot",
"Tag",
"ModelCandidate",
"Scanner",
# v0.4.0 — graph
"DataNode",
"DataPort",
Expand Down
Loading
Loading