From 2f5daedaa01913399065d5eb52cf858543ea90f4 Mon Sep 17 00:00:00 2001 From: Baris Demir Date: Mon, 20 Jul 2026 12:33:03 +0100 Subject: [PATCH] Arm backend: Document RIFE accuracy handover This patch was authored with Claude. Signed-off-by: Baris Demir Change-Id: I388708d60ea2da8d5d9273e150708a9a8b709fdb --- examples/arm/QAT_example/README.md | 88 ++++++++++++++++++++++++++++ examples/arm/QAT_example/qat_loop.py | 6 +- 2 files changed, 91 insertions(+), 3 deletions(-) create mode 100644 examples/arm/QAT_example/README.md diff --git a/examples/arm/QAT_example/README.md b/examples/arm/QAT_example/README.md new file mode 100644 index 00000000000..06fa6d0a9c3 --- /dev/null +++ b/examples/arm/QAT_example/README.md @@ -0,0 +1,88 @@ +# Practical-RIFE Accuracy Flow + +`qat_loop.py` measures Practical-RIFE eager, PTQ, and QAT accuracy on real +frame triples. It is intended as the accuracy companion to VGF performance +measurements, so each reported artifact can be tied back to a reproducible +model, ExecuTorch revision, and export command. + +## Inputs + +Use a standard Practical-RIFE checkout with this layout: + +```text +/train_log/IFNet_HDv3.py +/train_log/flownet.pkl +``` + +The triples file must identify frame interpolation samples in this order: + +```text +name,input0,input1,target +sample_00000,/path/frame_0.png,/path/frame_2.png,/path/frame_1.png +``` + +The script treats `input0` and `input1` as the two model inputs and `target` +as the expected middle frame. + +## Real Accuracy Run + +Use real frame triples for accuracy reporting. This example compares eager, +PTQ, and QAT at the default 768x384 shape while preserving uint8 image IO: + +```bash +python examples/arm/QAT_example/qat_loop.py \ + --triples-list \ + --model-root \ + --height 768 \ + --width 384 \ + --max-triples 10 \ + --calibration-samples 8 \ + --mode all \ + --io-quantization uint8 \ + --qat-samples 8 \ + --qat-steps 50 \ + --qat-lr 1e-5 \ + --output-dir out/rife_accuracy_10_real_triples_qat50 +``` + +The output directory contains: + +- `report.md`: human-readable aggregate metrics. +- `metrics.json`: aggregate and per-sample metrics. +- `metrics.csv`: tabular per-sample metrics. +- `quantization_coverage/`: PTQ and QAT graph coverage details. + +## Smoke Runs + +Small random or short QAT runs are useful only for checking that the flow runs. +Do not use them as model-quality evidence. For example, two samples and three +QAT steps can produce very low PSNR while still proving that eager, PTQ, and +QAT execution is wired correctly. + +## Reference Samples + +Keep reference images outside this repository and pass them through +`--triples-list`. For internal comparisons, use a fixed triples list and keep +the image bundle unchanged across graph and quantization experiments. + +The triples list is the reproducibility contract. It should use repository- or +bundle-relative paths where possible, so another user can unpack the same image +bundle and run the same command without editing absolute paths. + +## Reporting Checklist + +When sharing accuracy or performance data, include: + +- Practical-RIFE commit hash. +- ExecuTorch commit hash. +- `qat_loop.py` command line and flags. +- Dataset or triples-list identity. +- Input shape and preprocessing mode. +- IO quantization mode. +- PTQ calibration sample count. +- QAT sample count, step count, and learning rate. +- Reference image bundle identity. +- Whether a VGF or PTE was exported from the same run. + +This keeps accuracy numbers comparable with board performance results and +avoids mixing smoke-test numbers with real-data validation. diff --git a/examples/arm/QAT_example/qat_loop.py b/examples/arm/QAT_example/qat_loop.py index 83ad484a4fe..b7ccf0ec97b 100644 --- a/examples/arm/QAT_example/qat_loop.py +++ b/examples/arm/QAT_example/qat_loop.py @@ -7,14 +7,14 @@ """Measure Practical-RIFE PTQ and QAT accuracy on real frame triples. The script is intended as a small handover flow for comparing eager FP32, -PTQ, and QAT PT2E models before generating customer VGF artifacts. The -expected triple order is: +PTQ, and QAT PT2E models before generating VGF artifacts. The expected +triple order is: * first frame: model input 0 * third frame: model input 1 * middle frame: quality target -By default the fixed input shape is 768x384, matching the OPPO request. +By default the fixed input shape is 768x384. """ from __future__ import annotations