vsr

vsr is a Rust CLI for removing hardcoded subtitles from videos.

The project is still in early development, but the STTN-based subtitle removal pipeline is wired up end-to-end for the ONNX runtime.

Current Features

• remove-subtitles subcommand with validated arguments
• manual and auto detection modes
• rectangle parsing for --subtitle-rect x,y,width,height
• STTN ONNX pipeline for frame extraction, inpainting, and video re-encoding
• integration tests for CLI and pipeline behavior

Model Setup

The binary currently looks for a compatible STTN ONNX model in one of these locations:

• models/sttn/infer_model.onnx
• VSR_STTN_MODEL=/absolute/path/to/infer_model.onnx
• Hugging Face cache via VSR_STTN_HF_REPO=<repo> and VSR_STTN_HF_FILE=<path/to/model.onnx>

The latest public STTN checkpoint that I could verify on Hugging Face on 2026-04-02 is:

• Repo: https://huggingface.co/spaces/paulpang/video-subtitle-remover
• Direct download: https://huggingface.co/spaces/paulpang/video-subtitle-remover/resolve/main/backend/models/sttn/infer_model.pth

You can download that original checkpoint with:

mkdir -p models/sttn
curl -L https://huggingface.co/spaces/paulpang/video-subtitle-remover/resolve/main/backend/models/sttn/infer_model.pth -o models/sttn/infer_model.pth

Important:

• Hugging Face currently exposes a PyTorch checkpoint named infer_model.pth.
• This Rust CLI currently expects an ONNX artifact named infer_model.onnx.
• Downloading the .pth file as-is will not make vsr runnable yet.
• After you export or convert a compatible ONNX model, place it at models/sttn/infer_model.onnx or point VSR_STTN_MODEL at the converted file.
• If you host a converted ONNX artifact on Hugging Face, set both VSR_STTN_HF_REPO and VSR_STTN_HF_FILE; the binary will download it through hf-hub when no local model is present.

For the conversion flow, see docs/sttn-model-conversion.md.

Project Layout

• src/main.rs: binary entrypoint
• src/lib.rs: shared crate exports
• src/cli/: CLI parsing and validation
• src/app/: top-level application runner
• src/types/: shared domain types
• tests/: integration tests
• docs/superpowers/: plans and design docs

Development

Build the project:

cargo build

Run the CLI help:

cargo run -- --help

Try the current command surface:

cargo run -- remove-subtitles --input in.mp4 --output out.mp4 --detect manual --subtitle-rect 10,20,300,60

Run the full pipeline with a configured ONNX model:

VSR_STTN_MODEL=/absolute/path/to/infer_model.onnx \
cargo run -- remove-subtitles \
  --input in.mp4 \
  --output out.mp4 \
  --detect manual \
  --subtitle-rect 10,20,300,60 \
  --model sttn \
  --runtime onnx \
  --device cpu

Or let the binary fetch a hosted ONNX artifact from Hugging Face:

VSR_STTN_HF_REPO=your-org/your-model-repo \
VSR_STTN_HF_FILE=models/sttn/infer_model.onnx \
cargo run -- remove-subtitles \
  --input in.mp4 \
  --output out.mp4 \
  --detect manual \
  --subtitle-rect 10,20,300,60 \
  --model sttn \
  --runtime onnx \
  --device cpu

Run tests:

cargo test