PyTorch-FD

PyTorch $f$-divergence estimation via variational bounds optimization.

Implemented estimators

• Donsker-Varadhan
• Nguyen-Wainwright-Jordan
• Nishiyama
• InfoNCE

Mutual Information Neural Estimation

PyTorch-FD can be used to estimate the mutual information between random vectors $X$ and $Y$ via estimating $\mathsf{KL}[\mathbb{P}_{X,Y} \Vert \mathbb{P}_X \otimes \mathbb{P}_Y]$. See mutual_information.py.