perf: VRAM & inference optimizations with CLI feature flags

[cmoyates]

Summary

Phased performance optimization of the inference pipeline, reducing VRAM usage and improving throughput. All optimizations are behind CLI flags — defaults match pre-optimization behavior.

• FP16 weight casting (--fp16) — 7.2 GB VRAM savings, 27% faster
• GPU-side color math (--gpu-color) — eliminates CPU↔GPU transfers
• Decoupled backbone/refiner resolutions (--refiner-res) — run refiner at lower res
• Tiled CNN refiner with tent blending (--tile-size, --tile-overlap) — 52% VRAM reduction
• Benchmark infrastructure + quality gate tests

Why

Large-resolution frames (2K+) exhaust GPU memory on consumer cards. These changes make CorridorKey usable on 8–12 GB GPUs without sacrificing output quality at default settings.

Test plan

• uv run pytest -m "not gpu" passes
• uv run ruff check && uv run ruff format --check clean
• Default flags produce identical output to main
• --fp16 --gpu-color --tile-size 512 reduces VRAM significantly

🤖 Generated with Claude Code

[nikopueringer]

Cool stuff. Some questions-

I thought I was already casting the model to fp16. Perhaps i was doing so incorrectly? Or is this casting a different aspect of the process to fp16? The model is almost 400MB at fp32, and fp16 knocked it down to ~300MB I believe, based on my tests. Since the weights are so small, I came to the conclusion that trying to further quantization on the weights led to diminishing returns.

Likewise, is the CNN taking up that much memory? From what I've seen, it's pretty light. I am dubious of claude's estimations here for vram savings. Have you run tests while observing nvidia-smi?

Thanks for taking a crack at optimization! Let me know your thoughts!

[cmoyates]

Cool stuff. Some questions-

I thought I was already casting the model to fp16. Perhaps i was doing so incorrectly? Or is this casting a different aspect of the process to fp16? The model is almost 400MB at fp32, and fp16 knocked it down to ~300MB I believe, based on my tests. Since the weights are so small, I came to the conclusion that trying to further quantization on the weights led to diminishing returns.

Likewise, is the CNN taking up that much memory? From what I've seen, it's pretty light. I am dubious of claude's estimations here for vram savings. Have you run tests while observing nvidia-smi?

Thanks for taking a crack at optimization! Let me know your thoughts!

Neat thing about what you can see there: those numbers aren't estimations; those are benchmarks that I ran here on my MacBook. It is running the MPS compelled compatibility thing on the main model, but these are real numbers.

As far as casting the model to fp16, I'm not sure. I'll be completely honest with you: I have just been asking various AI things, "What potential paths of optimization do they see in this?" and doing what they suggest and benchmarking the results.

[cmoyates]

Here are the exact numbers from the benchmarks: https://github.com/cmoyates/CorridorKey/blob/feature/misc-optimizations/benchmarks/RESULTS.md. I'm assuming claude got the numbers in the PR description from there.

Also here's the exact plan I followed for those first few optimizations: https://github.com/cmoyates/CorridorKey/blob/feature/misc-optimizations/docs/plans/2026-03-07-feat-vram-performance-optimizations-plan.md

[nikopueringer]

Mm, interesting. Sounds like this is Mac focused at the moment. I’m hesitant to dive in too deep on to Mac optimization a day before public launch, but something like this could be extremely useful to people.

have you verified the results with your eyeballs? Compared frames between outputs?

if there are optimization gains to be made, it’s great that you’re highlighting how they could be achieved. But since I don’t have a Mac platform to test on, I’m gonna take this a bit slow and really rely on others to test and confirm this stuff. I’m still pondering if we should split off a Mac-focused version since this goes pretty deep into Mac architecture.

what are your thoughts?

[cmoyates]

It should be cross platform (I am working on an equivalent set of changes for the MLX repo atm). Only reason it appears to be mac focused is because that's what I'm running it on. I'll send out some messages in the discord for have some people with Nvidia GPUs run it to double check once I've cleaned it up a bit.

[nikopueringer]

Okay, awesome. If this is cross platform, that’s great.

[nikopueringer]

going through PRs, just wanted to let you know that I'm still aware this is open. There have been a few other optimizations that I'm going through. Let me know if there's an update here I should merge

[cmoyates]

I'm 99% sure the stuff the other guys have been doing includes all of this but I'll ask one of them to take a look once things start to slow down

[HYP3R00T]

Who will go through 3000+ lines to review this PR? Using AI to write codes are fine, but the PR needs to be of reasonable size so that others can review it. And using AI to review PRs is a bad precedence. Perhaps ask Claude (or whichever AI you are using) to be smarter and suggest simple and effective changes. And if you still need all these, then split those into multiple PRs so that we can review it independently.

[tonioss22]

Who will go through 3000+ lines to review this PR? Using AI to write codes are fine, but the PR needs to be of reasonable size so that others can review it. And using AI to review PRs is a bad precedence. Perhaps ask Claude (or whichever AI you are using) to be smarter and suggest simple and effective changes. And if you still need all these, then split those into multiple PRs so that we can review it independently.

Have you taken a look at the P/R? 3000 lines of code change in a P/R can be a reasonable amount and it seems like you did not even bother to look into what's in the P/R before writting this comment. I assume you saw that this user wrote that they were using a LLM to help them out and you assumed it was a bad P/R because of it.

After skimming through it briefly you can quickly see that there is not in fact 3000 lines of code since a lot of the new lines in here are documentation, only about 1500 lines remain of actual code change which i see quite often in P/Rs that used no AI. This is not my expertise so i will not be reviewing it but this comment seems to be bad faith AI hate.