lora parameters have no attributes named weight or bias

[pnaclcu]

Hello, and thank you for your excellent work—it has been very helpful.

    When I tried to migrate LoRA parameters, I ran into an issue: the DFloat11 Linear layer has no weight attribute; only bias is kept in _parameters. As a result, loading LoRA fails. Inside dfloat11.py, the line delattr(module, 'weight') removes the weight after quantization.

How should I modify the code so that the weight attribute is preserved (or re-added) and LoRA can be loaded successfully?

best regards~

[tonyzhang617]

You could try loading the original bfloat16 model and then load the LoRA adapters. Then, use DFloat11Model.from_pretrained to convert the bfloat16 model with LoRA into a DFloat11 model with LoRA.

Let me know if that works.

[pnaclcu]

Thank you for your suggestions. My work is based on the FLUX.1-Fill-dev model you quantized.
However, I have encountered a new problem.
In the original pipeline (transformer → dfloat11 -.lora) the error was
"'Linear' object has no attribute 'weight'".
In the new pipeline (transformer → LoRA → dfloat11) the error is
KeyError: "attribute 'bias' already exists".
It seems that the DFloat11 transformer contains an attribute named _parameters, and the bias is stored there. Does this matter?
I tried deleting the _parameters attribute, but it did not work. I also tried setting the weight or bias to None, yet the subsequent operations in dfloat11.py still require them.
That is everything I have done so far.
Best regards.

[mingyi456]

@pnaclcu Could you show the code you used? Sadly, I have not tried using LoRAs much myself.

But what I think @tonyzhang617 meant was not to use DFloat11Model.from_pretrained() with the BF16 weights, I think he meant you should first load the LoRA and fuse it with the BF16 model first, then use the compress_flux.py script to create your own DF11 model with the LoRA fused inside.

In your original question, the weight attributes actually point to the full BF16 weights. The whole point of deleting the weight attribute is to save VRAM, by only generating it just in time (using the DF11-specific tensors) before it is actually required for inference. If you take a look at the state_dict of the model, the weight tensors are usually large 2d tensors (i.e. matrices), of maybe 3072x3072 elements. The bias tensors are only 1d vectors, of 3072 elements.

So the reason why the bias tensors are not deleted, and still exist is most likely because they are much smaller relative to the weight tensors, so they might as well be left in BF16. (or maybe they are actually not as compressible as the weight tensors, but I have not tried testing them myself)

[pnaclcu]

Hi @mingyi456 , it works on my end! thanks
Here’s the final Flux-Fill pipeline I put together:
Load the transformer:
transformer = FluxTransformer2DModel.from_pretrained(...)
Build the pipeline:
pipe = FluxFillPipeline.from_pretrained(...)
Load the merged weights:
transformer_state_dict = load_state(*.safetensors)
Inject LoRA:
pipe.load_lora()
Export the fused weights:
pipe.transformer.save_dict(merged_safetensor)
Run compress_flux() (tweak args as needed) to get the DFloat11 model.
Follow the official example for inference.
The key is to quantize the FLUX model yourself.
Thanks again for all your help — and happy 1024 (2¹⁰) day!