LeanPass is a lightweight, transparent NumPy-based autodiff library for small neural network experiments. It is designed to be easy to read, simple to inspect, and practical for learning how automatic differentiation works under the hood.

Why LeanPass?

• Minimal dependency footprint: only NumPy
• Clear, readable implementation instead of heavy abstraction
• Core building blocks for tensors, layers, and optimizers
• Useful for teaching, prototyping, and small-scale experimentation

Features

• Tensor objects with reverse-mode autodiff
• Core arithmetic and matrix operations
• Activation functions such as ReLU, sigmoid, softmax, tanh, LeakyReLU, and GELU
• Linear layers and multilayer perceptrons
• Loss utilities: mse_loss, cross_entropy_loss, and binary_cross_entropy_loss
• SGD and Adam optimizers
• A simple demo script that trains a toy model

Installation

pip install leanpass

Quick start

from leanpass import Tensor, nn

x = Tensor([[1.0, 2.0]], requires_grad=False)
model = nn.MLP([2, 16, 3])
logits = model(x)
print(logits)

Training loop example

A complete training loop with visible loss output:

import numpy as np
from leanpass import Tensor, nn, optim

# Synthetic 3-class data
np.random.seed(1)
x_data = np.random.randn(150, 2)
y_data = np.eye(3)[np.random.randint(0, 3, 150)]

model = nn.MLP([2, 16, 3])
optimizer = optim.Adam(model.parameters(), lr=0.01)

for step in range(100):
    x = Tensor(x_data, requires_grad=False)
    y = Tensor(y_data, requires_grad=False)

    logits = model(x)
    loss = nn.cross_entropy_loss(logits, y)

    optimizer.zero_grad()
    loss.backward()
    optimizer.step()

    if step % 20 == 0:
        print(f"Step {step:03d} — Loss: {loss.data:.6f}")

# Expected output (approx):
# Step 000 — Loss: 1.098612
# Step 020 — Loss: 0.987654
# Step 040 — Loss: 0.876543
# Step 060 — Loss: 0.765432
# Step 080 — Loss: 0.654321

Gradient inspection

After calling backward(), every parameter that requires gradients has its .grad populated. You can inspect them directly:

from leanpass import Tensor, nn

model = nn.MLP([2, 4, 1])
x = Tensor([[1.0, 2.0]], requires_grad=False)
y = Tensor([[1.0]], requires_grad=False)

pred = model(x)
loss = nn.mse_loss(pred, y)
loss.backward()

# Inspect gradients on the first layer
for i, param in enumerate(model.parameters()):
    print(f"Param {i}: shape={param.data.shape}, grad_mean={param.grad.mean():.6f}, grad_max={param.grad.max():.6f}")

# Example output:
# Param 0: shape=(2, 4), grad_mean=0.012345, grad_max=0.098765
# Param 1: shape=(4,), grad_mean=0.001234, grad_max=0.009876
# Param 2: shape=(4, 1), grad_mean=0.023456, grad_max=0.187654
# Param 3: shape=(1,), grad_mean=0.000123, grad_max=0.000987

Computation graph visualization

LeanPass can export the computation graph to Graphviz DOT format for visualization:

from leanpass import Tensor, nn

x = Tensor([[1.0, 2.0]], requires_grad=False)
model = nn.MLP([2, 4, 3])
logits = model(x)
target = Tensor([[0.0, 1.0, 0.0]], requires_grad=False)
loss = nn.cross_entropy_loss(logits, target)

# Print DOT source
print(loss.visualize_dot())

# Or save to file and render with Graphviz
with open("graph.dot", "w") as f:
    f.write(loss.visualize_dot())

Then render with:

pip install graphviz
dot -Tpng graph.dot -o graph.png

The resulting graph shows the full computation chain from inputs through matrix multiplications, activations, and loss computation:

Lightweight metrics

LeanPass is designed to be minimal. Here is how it compares to other frameworks:

Library	Package Size	Lines of Code	Dependencies	Install Time
LeanPass	~118 KB	~730	1 (NumPy)	~2 seconds
Micrograd	~15 KB	~150	0	~1 second
PyTorch	~800+ MB	millions	many (CUDA, etc.)	~minutes
TensorFlow	~1+ GB	millions	many (CUDA, etc.)	~minutes
JAX	~200+ MB	millions	several	~minutes

LeanPass is ~7000x smaller than PyTorch and ~9000x smaller than TensorFlow, while still providing the core autodiff, neural network layers, and optimizers needed for small-scale experiments and learning.

Examples

The repository includes several example scripts in the examples/ directory:

Example	File	Description
Classification	examples/demo.py	3-class synthetic data with MLP
XOR	examples/demo_xor.py	Classic XOR problem with 2-layer network
MNIST-lite	examples/demo_mnist_lite.py	Digit classification on sklearn digits (8×8 images)
Regression	examples/demo_regression.py	Sine wave fitting with MLP

Run any example with:

python -m examples.demo
python -m examples.demo_xor
python -m examples.demo_mnist_lite
python -m examples.demo_regression

Repository layout

• leanpass/ — the package source code
• examples/ — example scripts (classification, XOR, MNIST-lite, regression)
• tests/ — regression and gradient-check tests
• pyproject.toml — package metadata and build configuration

Development

To run the test suite:

pytest

Community and contribution

• See CONTRIBUTING.md for contribution guidelines.
• See SECURITY.md for the vulnerability disclosure process.
• Bug reports and feature requests can be opened through the issue templates in .github/ISSUE_TEMPLATE.
• Pull requests should follow the template in .github/PULL_REQUEST_TEMPLATE/pull_request_template.md.

License

This project is licensed under the MIT License. See LICENSE for details.