Any chance of adding fp8 to the quantization? I see bf16 fp16 the int one and nv4 i think, i would love fp8 though.
Any chance of adding fp8 to the quantization? I see bf16 fp16 the int one and nv4 i think, i would love fp8 though.