Fix compatibility with transformers >= 4.45

[Mr-Neutr0n]

Summary

• Fixes AttributeError: 'CogVLMForCausalLM' object has no attribute '_extract_past_from_model_output' when using transformers >= 4.45
• Adds compatibility patch module that works with both old and new transformers versions

Problem

In transformers 4.45+, the _extract_past_from_model_output method was removed from GenerationMixin. The model's modeling_cogvlm.py (on HuggingFace Hub) calls this method in _update_model_kwargs_for_generation, causing the error:

AttributeError: 'CogVLMForCausalLM' object has no attribute '_extract_past_from_model_output'

This breaks model.generate() when using the model with streaming or any generation that updates model kwargs.

Solution

Since the modeling_cogvlm.py is hosted on HuggingFace Hub rather than in this repo, we add a runtime compatibility patch in the demo scripts:

1. New transformers_compat.py module: Contains patch_model_for_transformers_compat() that replaces _update_model_kwargs_for_generation with a version that directly accesses outputs.past_key_values

2. Updated demo scripts: cli_demo.py and web_demo.py now call the patch after model loading

The patch is backward compatible:

• On transformers < 4.45: Patch is skipped (original method still exists)
• On transformers >= 4.45: Patch is applied automatically

Usage

For users with custom scripts, they can apply the patch:

from basic_demo.transformers_compat import patch_model_for_transformers_compat

model = AutoModelForCausalLM.from_pretrained(...)
patch_model_for_transformers_compat(model)

Long-term fix

The proper fix would be to update modeling_cogvlm.py on HuggingFace Hub to use:

model_kwargs["past_key_values"] = getattr(outputs, "past_key_values", None)

instead of:

model_kwargs["past_key_values"] = self._extract_past_from_model_output(...)

Test plan

• Run cli_demo.py with transformers >= 4.45
• Run web_demo.py with transformers >= 4.45
• Verify generation/streaming works correctly

Fixes #218

[Mr-Neutr0n]

following up — this fixes compatibility with transformers >= 4.45 where some internal APIs changed. lmk if you'd want a different approach

[Mr-Neutr0n]

Friendly bump! Let me know if there's anything I should update or improve to help move this forward.