Skip to content

NJUVISION/TinyLIC

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

54 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Awesome End-to-End Neural Image Coding Awesome

Survey TinyLIC PRs Welcome Website

This repository accompanies the survey End-to-End Neural Image Coding: Foundation, Evolution, Deployment, and Outlook.

The page first highlights the TinyLIC benchmark, then lists papers by survey section.

Core technique taxonomy for end-to-end learned image coding
Figure 2. Core technique taxonomy for end-to-end learned image coding
TinyLIC benchmark overview
Figure 8. TinyLIC benchmark overview

00 | Table of Contents


01 | TinyLIC and Benchmark

Code links point to confirmed public repositories when available; otherwise - indicates that no reliable open-source implementation was found during this pass.

TinyLIC is the open benchmark centerpiece of this survey: a transparent, efficient, and extensible reference codec for studying practical learned image compression.

1.1 | TinyLIC Design

Component Description
Backbone Lightweight VAE-based transform coding framework
Main block Efficient depth-wise ConvNeXt-style EDN block
Entropy model Lightweight 4-step channel autoregressive Gaussian model
Complexity levels Nano, Small, and Base at roughly 10, 20, and 50 KMACs/pixel
Optimization Objective Fidelity-oriented and Perceptual Realism-oriented variants
Practical tools Variable-rate control, INT8 consistent decoding, attack-resilient training, progressive coding, and loss-resilient coding

1.2 | TinyLIC Results

TinyLIC Variant Dec. KMACs/pixel Params Kodak PSNR BD-rate vs. VTM CLIC PSNR BD-rate vs. VTM Tecnick PSNR BD-rate vs. VTM
Nano 10 3.16M 14.78 14.68 12.76
Small 20 4.76M 5.43 4.63 2.73
Base 50 10.57M -0.79 -1.76 -4.21
+ VR 50 - -1.50 -2.75 -5.02
+ VR + INT8 50 - -0.41 -1.45 -3.68
+ VR + INT8 + Progressive 50 - 3.15 -1.04 -1.71
+ VR + INT8 + Attack-Resilient 50 - -0.20 -1.36 -3.36
+ VR + INT8 + Loss-Resilient 50 - 9.69 8.54 6.01

Lower BD-rate is better. VTM 22.0 is used as the anchor for distortion-oriented evaluation.

1.3 | Benchmark and Evaluation

Method Pub. Arch. Context Params KMACs/pixel BD-rate vs. VTM Article Code
Factorized ICLR 2017 CNN Factorized 7.03M 204.00 67.80 Article GitHub
Hyperprior ICLR 2018 CNN Hyperprior 11.81M 208.97 30.14 Article GitHub
JointAR NeurIPS 2018 CNN SAR 25.50M 224.80 10.61 Article GitHub
ChannelAR ICIP 2020 CNN CAR 28.72M 243.00 7.22 Article GitHub
NLAIC TIP 2021 Attention SCAR 65.50M 823.54 5.92 Article GitHub
GMM CVPR 2020 CNN SAR 29.63M 512.80 4.55 Article GitHub
ELIC CVPR 2022 CNN SCAR 36.93M 573.88 -3.22 Article Github
TIC DCC 2022 Attention SCAR 28.40M 506.72 -4.58 Article GitHub
TCM CVPR 2023 Attention CAR 76.57M 1823.58 -10.70 Article GitHub
QARV TPAMI 2024 CNN HAR 93.40M 718.96 -5.81 Article GitHub
FLIC ICLR 2024 Attention CAR 70.97M 1096.04 -12.97 Article GitHub
MambaIC CVPR 2025 Mamba SCAR 75.78M 1284.86 -15.72 Article GitHub
HPCM ICCV 2025 CNN HAR 89.71M 1261.29 -19.19 Article GitHub
DHIC ICLR 2026 CNN HAR 106.93M 977.73 -19.73 Article GitHub

02 | Core Framework

2.1 | Transform

2.1.1 | Convolutional Transforms

Paper Venue Review Note Article Code
End-to-end optimization of nonlinear transform codes for perceptual quality PCS 2016 Early nonlinear transform coding framework Article GitHub
End-to-end optimized image compression ICLR 2017 Foundational variational transform coding with differentiable quantization surrogate Article GitHub
Lossy Image Compression with Compressive Autoencoders ICLR 2017 Learned autoencoder-based lossy compression Article Github
Variational image compression with a scale hyperprior ICLR 2018 Hyperprior becomes a standard side-information design Article GitHub
An End-to-End Compression Framework Based on Convolutional Neural Networks TCSVT 2018 CNN codec with end-to-end optimization IEEE -
Learned Image Compression With Discretized Gaussian Mixture Likelihoods and Attention Modules CVPR 2020 GMM likelihood and attention-enhanced CNN transform Article GitHub
End-to-End Learnt Image Compression via Non-Local Attention Optimization and Improved Context Modeling TIP 2021 Non-local attention as a bridge from CNNs to global-context codecs Article GitHub

2.1.2 | Transformer-Based Transforms

Paper Venue Review Note Article Code
Transformer-based transform coding ICLR 2022 Early Transformer transform for image compression Article Github
Transformer-based Image Compression DCC 2022 Transformer image compression Article GitHub
Entroformer: A Transformer-based Entropy Model for Learned Image Compression ICLR 2022 Transformer-based entropy modeling Article GitHub
The Devil Is in the Details: Window-based Attention for Image Compression CVPR 2022 Window attention improves practicality over full attention Article GitHub
Contextformer: A Transformer With Spatio-Channel Attention for Context Modeling ECCV 2022 Spatio-channel attention for context modeling Article -
Learned Image Compression With Mixed Transformer-CNN Architectures CVPR 2023 Hybrid CNN-Transformer design balances local and global modeling Article GitHub
Frequency-Aware Transformer for Learned Image Compression ICLR 2024 Frequency-aware attention and channel-wise autoregressive modeling Article GitHub
Linear Attention Modeling for Learned Image Compression CVPR 2025 Linear-complexity attention for LIC Article GitHub

2.1.3 | Mamba-Based Transforms

Paper Venue Review Note Article Code
MambaVC: Learned Visual Compression with Selective State Spaces arXiv 2024 Selective state-space modeling for visual compression Article GitHub
MambaIC: State Space Models for High-Performance Learned Image Compression CVPR 2025 SSM-based transform/context modeling for efficient compression Article GitHub
CASSIC: Towards Content-Adaptive State-Space Models for Learned Image Compression ICCV 2025 Content-adaptive state-space codec Article -
Content-Aware Mamba for Learned Image Compression ICLR 2026 Content-aware Mamba transform for LIC Article GitHub

2.1.4 | Hierarchical and Alternative Transforms

Paper Venue Review Note Article Code
Neural Multi-Scale Image Compression ACCV 2018 Multi-scale learned compression Article GitHub
Efficient Variable Rate Image Compression with Multi-Scale Decomposition Network TCSVT 2018 Multi-scale decomposition for rate scalability Article -
Coarse-to-Fine Hyper-Prior Modeling for Learned Image Compression AAAI 2020 Coarse-to-fine hyperprior modeling Article GitHub
End-to-End Optimized Versatile Image Compression With Wavelet-Like Transform TPAMI 2020 Learned wavelet-like transform Article GitHub
Graph-Convolution Network for Image Compression ICIP 2021 Graph-based transform design Article -
Lossy Image Compression with Quantized Hierarchical VAEs WACV 2023 Hierarchical VAE and quantized residual layers Article GitHub
QARV: Quantization-Aware ResNet VAE for Lossy Image Compression TPAMI 2024 Hierarchical residual VAE with quantization-aware design Article GitHub
WeConvene: Learned Image Compression with Wavelet-Domain Convolution and Entropy Model ECCV 2024 Wavelet-domain convolution and entropy modeling Article GitHub
FDNet: Frequency Decomposition Network for Learned Image Compression TCSVT 2024 Frequency decomposition network Article -
Taming Hierarchical Image Coding Optimization: A Spectral Regularization Perspective ICLR 2026 Spectral regularization for hierarchical coding Article GitHub
Adaptive Learned Image Compression with Graph Neural Networks CVPR 2026 Content-adaptive image compression framework based on Graph Neural Networks Article GitHub

2.2 | Quantization

Function Paper Venue Review Note Article Code
Additive noise surrogate End-to-end optimized image compression ICLR 2017 Uniform noise approximates scalar quantization during training Article GitHub
Universal quantization Variable Rate Deep Image Compression With a Conditional Autoencoder ICCV 2019 Subtractive dithering and conditional autoencoder for rate variation CVF -
Soft-to-hard quantization Soft-to-Hard Vector Quantization for End-to-End Learning Compressible Representations NeurIPS 2017 Temperature-controlled relaxation of hard assignments Proceedings -
Soft-then-hard quantization Soft then Hard: Rethinking the Quantization in Neural Image Compression ICML 2021 Soft training followed by hard quantization finetuning PMLR -
Product quantization NVTC: Nonlinear Vector Transform Coding CVPR 2023 Hierarchical/product quantization for learned compression Article GitHub
Lattice VQ LVQAC: Lattice Vector Quantization Coupled with Spatially Adaptive Companding CVPR 2023 Lattice VQ with adaptive companding CVF -
Adaptive lattice VQ Multirate Neural Image Compression with Adaptive Lattice Vector Quantization CVPR 2025 Multirate adaptive lattice VQ Article -
Dead-zone quantization Learned Progressive Image Compression with Dead-zone Quantizers TCSVT 2022 Sparse-latent quantization with larger zero interval Article -
Contextual sequential quantization NLIC: Non-uniform Quantization-based Learned Image Compression 2024 Context-adaptive quantization levels Article -
RD-aware VQ Differentiable Vector Quantization for Rate-Distortion Optimization of Generative Image Compression CVPR 2026 Differentiable relaxation couples VQ with entropy/R-D loss Article GitHub

2.3 | Entropy Modeling

Model Family Paper Venue Review Note Article Code
Factorized prior End-to-end optimized image compression ICLR 2017 Independent latent prior baseline Article GitHub
Hyperprior Variational image compression with a scale hyperprior ICLR 2018 Side information predicts latent scale Article GitHub
Spatial autoregression Joint Autoregressive and Hierarchical Priors for Learned Image Compression NeurIPS 2018 Combines masked spatial context and hyperprior Article GitHub
Context-adaptive entropy model Context-adaptive Entropy Model for End-to-end Optimized Image Compression ICLR 2019 Adaptive entropy estimation Article GitHub
Channel autoregression Channel-wise Autoregressive Entropy Models for Learned Image Compression ICIP 2020 Channel dependencies reduce spatial sequential cost Article GitHub
Spatial-channel context Spatial-Channel Context-Based Entropy Modeling for End-to-end Optimized Image Compression VCIP 2020 Joint spatial and channel context Article -
Checkerboard context Checkerboard Context Model for Efficient Learned Image Compression CVPR 2021 Parallel-friendly context modeling Article GitHub
Uneven space-channel context ELIC CVPR 2022 Unevenly grouped SC context for strong R-D performance Article Github
Multi-reference context MLIC / MLIC++ MM 2023 / TOMM 2025 Multi-reference entropy modeling and linear-complexity extension Article GitHub
Hierarchical progressive context HPCM ICCV 2025 Multi-scale progressive context schedule Article GitHub
Hierarchical VAE Taming Hierarchical Image Coding Optimization: A Spectral Regularization Perspective ICLR 2026 Hierarchical modeling Article GitHub

2.4 | Rate-Distortion Optimization

Direction Paper Venue Review Note Article Code
Perception-distortion theory The Perception-Distortion Tradeoff CVPR 2018 Formalizes the perception-distortion frontier Article -
Rate-distortion-perception Rethinking Lossy Compression: The Rate-Distortion-Perception Tradeoff ICML 2019 Theoretical R-D-P trade-off Article -
Learned perceptual metric The Unreasonable Effectiveness of Deep Features as a Perceptual Metric CVPR 2018 LPIPS-style perceptual losses Article -
Perceptual/generative compression High-Fidelity Generative Image Compression NeurIPS 2020 GAN-based high-fidelity compression Article GitHub
Statistical fidelity Improving Statistical Fidelity for Neural Image Compression with Implicit Local Likelihood Models ICML 2023 Implicit local likelihood models improve perceptual statistics Article GitHub
Human-machine coding Icmh-Net: Neural Image Compression towards Both Machine Vision and Human Vision ACM MM 2023 Neural image compression for human and machine vision Article -
Multi-machine coding All-in-One Image Coding for Joint Human-Machine Vision with Multi-Path Aggregation NeurIPS 2024 Neural image compression for multi-tasks Article GitHub

03 | Practical Extensions

3.1 | Variable-Rate Coding and Rate Control

3.1.1 | Variable-Rate Coding

Function Paper Venue Review Note Article Code
Recurrent refinement Variable Rate Image Compression with Recurrent Neural Networks ICLR 2016 Bitrate controlled by recurrent iteration count Article -
Recurrent full-resolution codec Full Resolution Image Compression with Recurrent Neural Networks CVPR 2017 RNN residual refinement at full resolution Article GitHub
Improved recurrent codec Improved Lossy Image Compression with Priming and Spatially Adaptive Bit Rates CVPR 2018 Better recurrent training and adaptive bit allocation Article -
Slimmable codec Slimmable Compressive Autoencoders for Practical Neural Image Compression CVPR 2021 Subnetworks represent different rate/complexity levels Article GitHub
Quality scaling factor Variable Bitrate Image Compression with Quality Scaling Factors ICASSP 2020 Latent scaling controls bitrate in a single model Article GitHub
Modulated autoencoder Variable Rate Deep Image Compression with Modulated Autoencoder SPL 2020 Modulation controls rate without multiple models Article GitHub
Conditional autoencoder Variable Rate Deep Image Compression with a Conditional Autoencoder ICCV 2019 Conditional rate control Article -
Interpolation VRC Interpolation Variable Rate Image Compression ACM MM 2021 Fine-grained rate interpolation Article -
Asymmetric gained VRC Asymmetric Gained Deep Image Compression with Continuous Rate Adaptation CVPR 2021 Continuous rate adaptation with gained units Article GitHub
High-fidelity VRC High-Fidelity Variable-Rate Image Compression via Invertible Activation Transformation ACM MM 2022 Invertible activation transformation for perceptual VRC Article GitHub
Spatial importance guidance SIGVIC: Spatial Importance Guided Variable-Rate Image Compression ICASSP 2023 Spatially adaptive bit allocation Article GitHub
Transformer ROI control Transformer-Based Variable-Rate Image Compression with Region-of-Interest Control ICIP 2023 Transformer VRC with ROI control Article GitHub
Adaptive quantization Stanh: Parametric Quantization for Variable Rate Learned Image Compression TIP 2025 Parametric quantizer for variable-rate LIC Article GitHub

3.1.2 | Progressive Coding

Function Paper Venue Review Note Article Code
Recurrent progressive coding Learning to Inpaint for Image Compression NeurIPS 2017 Progressive reconstruction with recurrent refinement Article -
Selective latent compression Selective Compression Learning of Latent Representations for Variable-Rate Image Compression NeurIPS 2022 Selectively transmits latent subsets Article GitHub
Trit-plane coding DPICT: Deep progressive image compression using trit-planes CVPR 2022 Ordered ternary-plane transmission Article GitHub
Intrinsic importance ordering ProgDTD: Progressive Learned Image Compression With Double-Tail-Drop Training CVPRW 2023 Training objective encourages latent importance order Article GitHub
Efficient progressive coding Efficient progressive image compression with variance-aware masking WACV 2025 Element-wise importance ranking and masking Article GitHub

3.1.3 | Rate Control

Function Paper Venue Review Note Article Code
JPEG AI VRC analysis Overview of Variable Rate Coding in JPEG AI TCSVT 2025 Reviews 3D quality control and rate matching IEEE -
Rate-feature-level prediction Rate Controllable Learned Image Compression Based on RFL Model VCIP 2022 Predicts coding parameters from target rate and image features Article -
Lambda-domain RC Lambda-domain Rate Control for Neural Image Compression MMAsia 2023 Predicts lambda from target bitrate Article -
Neural RC for JPEG AI An Efficient Neural Rate Control for JPEG-AI TCSVT 2025 Neural predictor selects discrete model/parameter indices IEEE -
Block-level RC Block-Level Rate Control for Learnt Image Coding PCS 2022 Block-wise bitrate-parameter curves and greedy allocation IEEE -
Accelerated block-level RC Accelerating Block-level Rate Control for Learned Image Compression DCC 2024 Predicts block curves from sampled blocks IEEE -

3.2 | Model Quantization and Cross-Platform Consistence

3.2.1 | QAT and PTQ

Function Paper Venue Review Note Article Code
Integer-only codec Integer Networks for Data Compression with Latent-Variable Models ICLR 2019 Eliminates floating-point mismatch through integer-domain design OpenReview -
Fixed-point decoding Efficient Neural Image Decoding via Fixed-Point Inference TCSVT 2020 Fixed-point inference for deterministic decoding Article GitHub
Fixed-point arithmetic Learned Image Compression with Fixed-Point Arithmetic PCS 2021 Fixed-point arithmetic for LIC Article -
QAT with channel splitting Q-LIC: Quantizing Learned Image Compression with Channel Splitting TCSVT 2022 Splits high-error channels to reduce quantization distortion Article -
PTQ consistency Post-Training Quantization for Cross-Platform Learned Image Compression arXiv 2022 PTQ plus deterministic entropy coding arXiv -
R-D optimized PTQ Rate-Distortion Optimized Post-Training Quantization for Learned Image Compression TCSVT 2023 Optimizes quantization with compression-aware losses Article GitHub

3.2.2 | Mixed-Precision Quantization and Joint Compression

Function Paper Venue Review Note Article Code
Mixed precision Flexible Mixed Precision Quantization for Learned Image Compression ICME 2024 Assigns different precision to sensitive modules Article GitLab
Mixed-precision PTQ Mixed-Precision Post-Training Quantization for Learned Image Compression IoT-J 2025 Post-training bit-width allocation Article -
Pruning + QAT Structured Pruning and Quantization for Learned Image Compression ICIP 2024 Joint model compression for deployment Article -

3.3 | Model Robustness

3.3.1 | Attacks

Function Paper Venue Review Note Article Code
Manipulation attacks MALICE: Manipulation Attacks on Learned Image Compression arXiv 2022 Early attack formulation for learned codecs arXiv -
White-box evasion Toward Robust Neural Image Compression: Adversarial Attack and Model Finetuning TCSVT 2023 FTDA attack and adversarial finetuning defense Article GitHub
Backdoor attack Backdoor Attacks Against Deep Image Compression via Adaptive Frequency Trigger CVPR 2023 Poisoned trigger attacks in compression training Article -
Robust backdoor trigger Robust and Transferable Backdoor Attacks Against Deep Image Compression With Selective Frequency Prior TPAMI 2025 Frequency-domain/robust triggers Article -
JPEG AI robustness Exploring adversarial robustness of JPEG AI: methodology, comparison and new methods arXiv 2024 Large-scale JPEG AI perturbation/attack evaluation arXiv -

3.3.2 | Defenses and Practical Robustness

Function Paper Venue Review Note Article Code
Architecture-level simplification MALICE / manipulation attack follow-ups 2022-2023 Simpler entropy models can reduce attack sensitivity arXiv -
Adversarial training Toward Robust Neural Image Compression TCSVT 2023 Robust finetuning improves adversarial stability Article GitHub
Latent consistency Successive Learned Image Compression: Comprehensive Analysis of Instability Neurocomputing 2022 Repeated compression and latent consistency Article -
Transmission error robustness ResiComp: Loss-Resilient Image Compression via Dual-Functional Masked Visual Token Modeling TCSVT 2025 Joint source-channel style robustness to packet loss Article GitHub
Loss-resilient learned compression Towards Loss-Resilient Image Coding for Unstable Satellite Networks AAAI 2025 Network-aware training for unreliable channels Article GitHub
Compression as defense ComDefend: An Efficient Image Compression Model to Defend Adversarial Examples CVPR 2019 Compression model as adversarial defense for vision tasks Article GitHub

3.4 | Lightweight Deployment

3.4.1 | Knowledge Distillation

Function Paper Venue Review Note Article Code
Output/entropy distillation Fast and High-Performance Learned Image Compression with Improved Checkerboard Context Model, Deformable Residual Module, and Knowledge Distillation TIP 2024 Distillation for faster learned codec Article -
Efficient LIC distillation Efficient Learned Image Compression Through Knowledge Distillation arXiv 2025 Intermediate/latent distillation for smaller students Article GitHub
KDIC Knowledge Distillation for Learned Image Compression ICCV 2025 Multi-level distillation across codec blocks Article -
EVC EVC: Towards Practical Neural Image Compression CVPR/TCSVT-era Mask-decay model slimming from large to small codec Article GitHub
CSLIC Distilling Complexity-Scalable Learned Image Compression Models via Neural Architecture Search TCSVT 2026 NAS + KD for scalable complexity levels Article -

3.4.2 | Architecture-Level Lightweight Design

Function Paper Venue Review Note Article Code
Non-autoregressive codec Towards Efficient Image Compression Without Autoregressive Models NeurIPS 2023 Removes sequential AR bottleneck Article -
Asymmetric codec AsymLLIC: Asymmetric Lightweight Learned Image Compression VCIP 2024 Shifts computation to encoder, keeps decoder light Article GitHub
Shift operations ShiftLIC: Lightweight Learned Image Compression with Spatial-Channel Shift Operations TCSVT 2025 Shift operators replace heavier convolutions Article GitHub
Lightweight attention Towards Real-Time Practical Image Compression with Lightweight Attention ESWA 2024 Lightweight attention for practical decoding Article GitHub
Shallow decoder Computationally-Efficient Neural Image Compression with Shallow Decoders ICCV 2023 Nearly linear decoder for low KMAC deployment Article GitHub
Switchable priors Learning Switchable Priors for Neural Image Compression TCSVT 2025 Low-complexity FastNIC with switchable priors Article -
Dynamic routing AdaNIC: Towards Practical Neural Image Compression via Dynamic Transform Routing ICCV 2023 Content-aware computation allocation CVF GitHub

3.4.3 | Test-Time Adaptation

Function Paper Venue Review Note Article Code
Latent refinement Content Adaptive Optimization for Neural Image Compression CVPRW 2019 Optimizes latents per image at encoding time Article -
Improved latent inference Improving Inference for Neural Image Compression NeurIPS 2020 Refines latents and side information Article GitHub
Instance-adaptive compression Overfitting for Fun and Profit: Instance-Adaptive Data Compression ICLR 2021 Per-instance decoder/entropy optimization Article -
Adapter-based adaptation Dec-Adapter: Exploring Efficient Decoder-Side Adapter for Bridging Screen Content and Natural Image Compression ICCV 2023 Decoder-side adapters for domain shift CVF -
Universal adapters Universal Deep Image Compression via Content-Adaptive Optimization with Adapters WACV 2023 Adapter-based content adaptation CVF GitHub
Dynamic low-rank adaptation Dynamic Low-Rank Instance Adaptation for Universal Neural Image Compression ACM MM 2023 Low-rank adaptation for per-image/domain shifts Article GitHub
Cross-domain TTA Test-Time Adaptation for Image Compression with Distribution Regularization ICLR 2025 Plug-and-play Bayesian regularization under shifts Article GitHub

3.5 | Standardization Progress

Standard / Topic Paper or Document Year Review Note Article Code
JPEG AI standard overview The JPEG AI Standard: Providing Efficient Human and Machine Visual Data Consumption 2023 Early overview of JPEG AI objectives Article -
IEEE 1857.11 IEEE Standard for Neural Network-Based Image Coding 2024 Neural network-based image coding standard Article -


04 | Contributing

Pull requests are welcome. Please add new entries under the matching review subsection.

| Function | Paper | Venue | Review Note | Link |

If a paper spans multiple functions, list it in each relevant subsection, following the structure above.

About

End-to-End Neural Image Coding

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors

Languages