fmetrics

Fast image & video fidelity metrics in C & Zig.

Usage

Compilation requires Zig ≥0.16.0 & a macOS, Linux, or Unix-like operating system. To compile, run:

zig build --release=fast

You may add -Dflto=true for FLTO, and -Dstrip=true to strip the binary.

Compilation emits:

zig-out
├── bin
│   ├── fmetrics
├── include
│   └── fmetrics.h
└── lib
    └── libfmetrics.a

fmetrics binary usage:

fmetrics by Halide Compression, LLC | [version]

usage: fmetrics <metric> [options] <reference> <distorted>

compare two images/videos using various perceptual quality metrics

metrics:  iwssim, msssim, ssimu2, butter, cvvdp

run `fmetrics <metric> --help` for metric-specific help

options:
  -h, --help
      show this help message

sRGB PNG, PNM/PAM, QOI, or Y4M input expected

Usage is different per-metric; some metrics support outputting visual error maps via --err-map, and some support additional configuration options. I/O is the same for all metrics, and is provided by simpleimgio.

Library Usage

libfmetrics.a exposes a C API declared in fmetrics.h. To use it as a Zig dependency, add it to your build.zig.zon by running:

zig fetch --save git+https://github.com/halidecx/fmetrics.git

This should show something like this in build.zig.zon:

.dependencies = .{
    .fmetrics = .{
        .url = "git+https://github.com/halidecx/fmetrics.git#<commit>",
        .hash = "fmetrics-<version>-<hash>",
    },
},

Then you can link it from your build.zig:

const fmetrics_dep = b.dependency("fmetrics", .{
    .target = target,
    .optimize = optimize,
});
const fmetrics = fmetrics_dep.artifact("fmetrics");
exe.root_module.linkLibrary(fmetrics);
exe.root_module.addIncludePath(fmetrics.getEmittedIncludeTree());

Zig projects can also import the native Zig API:

const fmetrics_dep = b.dependency("fmetrics", .{
    .target = target,
    .optimize = optimize,
});
const fmetrics = fmetrics_dep.module("fmetrics");
exe.root_module.addImport("fmetrics", fmetrics);
exe.root_module.linkLibrary(fmetrics_dep.artifact("libfmetrics"));

const fmetrics = @import("fmetrics");

const reference = try fmetrics.Image.init(reference_rgb, width, height);
const distorted = try fmetrics.Image.init(distorted_rgb, width, height);
const score = try fmetrics.msssim(reference, distorted);

See src/fmetrics.zig for the full Zig API. C projects may use fmetrics.h.

Reference Comparison

Reference metric implementations tested include:

• Butteraugli: libjxl's butteraugli_main
• CVVDP: Our fcvvdp
• IW-SSIM: A fork of Python IW-SSIM
• MS-SSIM: libvmaf's MS-SSIM filter via ffmpeg.
• SSIMULACRA2: Cloudinary's ssimulacra2

MOS Correlation

MOS correlation is how closely a metric correlates with subjective human ratings.

Tested using mos.py via mos-correlation, on CID22. For our purposes, these tests don't determine which metrics we think are better than others, but rather how effective our implementations are relative to their references. Here, we just report the Spearman Rank Correlation Coefficient (SRCC), where higher is better.

metric	srcc (reference)	srcc (fmetrics)	difference (%)
butteraugli (p3 i203)*	0.7929	0.7863	-0.83%
fcvvdp**	0.8274	0.8286	+0.15%
iw_ssim	n/a	0.7925	+0.00%
ms_ssim	0.7845	0.8048	+2.59%
ssimulacra2	0.8916	0.8910	-0.07%

*Note: Because Butteraugli is a smaller-is-better metric, the signs are flipped for the SRCCs reported above.

**Note: fmetrics uses the fcvvdp library (as a Zig module) with different I/O, so the underlying metric implementation is the same.

Speed & Memory Usage

Testing was done on a stock Core i7-13700k with 3840x2160 source & distorted PAM images (Drive link, lossless JPEG-XL sources; run djxl <*.pam.jxl> <*.pam> to decompress).

Speed (ms)

metric	ms (reference)	ms (fmetrics)	difference (%)
butteraugli (p3 i203)	4110	2480	65.73% faster
fcvvdp*	1060	1060	0.00%
iw_ssim	3020	228	1224.6% faster
ms_ssim**	1110	106	947.2% faster
ssimulacra2	722	232	211.2% faster

RAM Usage (MB)

metric	MB (reference)	MB (fmetrics)	difference (%)
butteraugli (p3 i203)	2440	1670	-31.56%
fcvvdp*	1600	1600	0.00%
iw_ssim	2660	551	-79.29%
ms_ssim**	841	376	-55.29%
ssimulacra2	1370	741	-45.91%

*Note: fmetrics uses the fcvvdp library (as a Zig module) with different I/O, so the underlying metric implementation is the same.

**Note: MS-SSIM comparison isn't fair, as libvmaf has to compute other metrics in the filterchain alongside MS-SSIM.

Credits

fmetrics is under the Apache 2.0 License. fmetrics is developed by Halide Compression.

Special thanks to Vship, which has inspired parts of fmetrics. Vship is under the MIT NON-AI license.