Can we quantize the df11 model ourselves?

[win10ogod]

Can we quantize the df11 model ourselves? Or develop new methods based on df11? Are there any licensing issues?
I hope the team will release tutorials on quantization and adding support for new models.
@tonyzhang617

[tonyzhang617]

We'll release the compression script and a tutorial on supporting new models this month.

Thank you for supporting this project!

[tonyzhang617]

The compression code is released!

Check examples/compress_flux1 for an example on compressing FLUX.1.

Let me know if you have any questions.

[ConTest1020]

examples/compress_flux1，Can examples/compress_flux1 compress LLM - type models? For instance, Qwen/Qwen2.5-7B-Instruct. I tried to compress it but failed.

[ConTest1020]

Is there an issue with this script as it's throwing an error?
python3 compress_flux.py
--model_name_or_path /shared/DFloat11/Qwen3-8b
--save_path ./Qwen3-DF11
--save_single_file
--check_correctness

[mingyi456]

@ConTest1020 At the very minimum, you need edit the script to modify the pattern_dict yourself, which I found to be highly nontrivial for me. Otherwise, the provided script only works with Flux-based models, of which I already uploaded a few finetunes.

I also managed to use it to compress 2 other DiT (diffusion transformer) architectures, although one of them, Chroma, is basically just pruned Flux.

As for LLM type models, those turned out to be a major pain, and I only got them working somewhat jankily after a few days of trying. You can check out more details about my struggles with T5XXL and Gemma-2-2b here