Quantitative Model

[Betsy9999]

Could you please provide the AWQ quantized model for WeDetect-Ref?

[fushh]

You can follow these guidelines to convert the model.
https://medium.com/@bhansalichayan/quantize-hugging-face-model-to-awq-int4-a-step-by-step-guide-with-benchmarks-c9d7ccd024a9
https://docs.vllm.ai/projects/llm-compressor/en/latest/examples/awq/