DiffLisp

An automatically differentiable variant/subset of Lisp implemented in Common Lisp.

DiffLisp provides a framework for automatic differentiation (AD) in Lisp, enabling the computation of derivatives of functions defined in a Lisp-like language. This is particularly useful for machine learning, optimization, and scientific computing applications.

Features

• Automatic Differentiation: Compute gradients automatically using reverse-mode AD (backpropagation)
• Tape-based Implementation: Efficient computation graph construction and traversal
• Native Lisp Integration: Seamless integration with Common Lisp
• Composable: Build complex differentiable functions from simple primitives
• Extensible: Easy to add new differentiable operators

Installation

Prerequisites

• A Common Lisp implementation (SBCL, CCL, or similar)
• Quicklisp (for dependency management)

Installing Quicklisp

If you don’t have Quicklisp installed:

;; Download and load the Quicklisp installer
(load "http://beta.quicklisp.org/quicklisp.lisp")

;; Install Quicklisp
(quicklisp-quickstart:install)

;; Add to your Lisp init file to load Quicklisp on startup
(ql:add-to-init-file)

Installing DiffLisp

Clone this repository to your local Quicklisp projects directory:

cd ~/quicklisp/local-projects/
git clone <your-repo-url> difflisp

Or add a symbolic link:

ln -s /path/to/DiffLisp ~/quicklisp/local-projects/difflisp

Then load the system:

(ql:quickload :difflisp)

Quick Start

(in-package :difflisp)

;; Define a simple differentiable function
;; Example: f(x) = x^2
(defun square (x)
  (diff-lambda (x)
    (* x x)))

;; Compute the gradient at x = 3
;; df/dx = 2x, so at x=3, gradient = 6
(let ((x (make-variable 3.0)))
  (gradient (square x) x))

Architecture

DiffLisp implements a small (but growing) subset of Common Lisp.

• Data Type: Double-precision floats (double-float) only. No complex numbers or matrices yet.
• Control Flow: Straight-line code only (no if or loops yet; we rely on host Lisp for that).

Differentiable Operators

The following operators are differentiable:

• Arithmetic: +, -, *, /
• Exponential: exp, log
• Trigonometric: sin, cos, tan
• Activation functions: relu, sigmoid, tanh

Usage Examples

Basic Differentiation

(defun square (x)
  (* x x))

;; Usage
(let ((grad-fn (grad #'square))) 
  (funcall grad-fn 3.0)) 
;; Returns: 6.0


;; Partial Derivatives
(defun f (x y)
(+ (* x x) 
   (* x y)))

;; Differentiate wrt argument index 1 (y)
(let ((df-dy (grad #'f :argnums 1)))
;; x=3.0, y=4.0
  (funcall df-dy 3.0 4.0)) 
;; Returns: 3.0 (since df/dy = x)

Development

Running Tests

(ql:quickload :difflisp/tests)
(asdf:test-system :difflisp)

Project Structure

DiffLisp/
├── difflisp.asd          # ASDF system definition
├── README                # This file
├── src/                  # Source code
│   ├── package.lisp      # Package definition
│   ├── tape.lisp         # Tape infrastructure
│   ├── operators.lisp    # Differentiable operators
│   ├── diff.lisp         # AD core
│   └── core.lisp         # API
├── tests/                # Test suite
│   └── main.lisp
└── examples/             # Usage examples

Theory

DiffLisp implements reverse-mode automatic differentiation (also known as backpropagation):

1. Forward Pass: Compute the function value while recording operations on a tape
2. Backward Pass: Traverse the tape in reverse, computing gradients using the chain rule

This approach is efficient for functions with many inputs and few outputs (common in machine learning).

Contributing

Contributions are welcome! Please feel free to submit issues or pull requests.

License

MIT License

References

• Automatic Differentiation
• Backpropagation
• ASDF Manual
• Quicklisp

Acknowledgments

Inspired by various AD frameworks including PyTorch, JAX, and other differentiable programming systems.