HF backend support for Qwen3.5 model family

[Seas0]

Using the benchmarking code and requirements specification from the official Github repo will cause a failure claiming Qwen3.5 is unsupported on the transformers v4.57.3. And by forcing the mainline transformers to be installed as stated by the Qwen 3.5 codebase, some random code error complaining about DynamicCache not supporting a specific method will be raised:

File ".venv/lib/python3.12/site-packages/transformers/models/qwen3_5/modeling_qwen3_5.py", line 1387, in _update_linear_attn_mask
 if (past_key_values is not None and past_key_values.has_previous_state) or (
                                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^              
 AttributeError: 'DynamicCache' object has no attribute 'has_previous_state'

Is it planed to add transformers support to this model, at least when transformers release a stable version with Qwen3.5 MoE model family support?

[jianc99]

For Qwen3.5 models, you need to use a later Transformers version, such as 5.2 or 5.3. However, because these models use linear attention in some layers, the verification path needs to be reimplemented. In particular, we need to track the Mamba state and roll it back after verification. We will look into this. At the moment, both SGLang and vLLM support Qwen3.5 inference, and DFlash has been merged into the main branch of vLLM.

[Seas0]

For Qwen3.5 models, you need to use a later Transformers version, such as 5.2 or 5.3. However, because these models use linear attention in some layers, the verification path needs to be reimplemented. In particular, we need to track the Mamba state and roll it back after verification. We will look into this. At the moment, both SGLang and vLLM support Qwen3.5 inference, and DFlash has been merged into the main branch of vLLM.

I see, the dedicate Qwen3_5MoeDynamicCache and Qwen3_5DynamicCache classes have previous states for GDN in conv_states & recurrent_states and can't be cropped to revert to last state. Is a partial deepcopy (only shadowing the SSM style previous states) of these caches suffice to make a backup and revert to it latter?

[Sylvan820]

We looked into the implementation of Qwen3.5 on Hugging Face. Because linear attention seems to require modifications to many parts of the pipeline, and it's quite troublesome, maybe we can see when the official version updates.

[jianc99]

If you have an implementation of Transformers backend inference for Qwen3.5 models, feel free to submit a PR.