Skip to content

[Audio support] merge into main#9

Open
alex-t-hu wants to merge 234 commits intomainfrom
audio_support
Open

[Audio support] merge into main#9
alex-t-hu wants to merge 234 commits intomainfrom
audio_support

Conversation

@alex-t-hu
Copy link
Copy Markdown
Collaborator

merge audio support in. i assume Qwen2_5OmniThinkerForConditionalGeneration is in HF so no need for custom logic; keep some debugging stuff here and there (can remove if needed); there is this video budget change in vision_utils.py , that should be a setting?
in rl_dataset , there's a ton of changes, and i try to respect the new audio support which uses self.modalities . there were some intriguing areas like here where self.apply_chat_template_kwargs gets removed
raw_prompt = self.processor.apply_chat_template(
messages, add_generation_prompt=True, tokenize=False, **self.apply_chat_template_kwargs
)

@alex-t-hu alex-t-hu requested a review from DDVD233 September 9, 2025 22:46
raw_prompt = self.processor.apply_chat_template(
messages, add_generation_prompt=True, tokenize=False, **self.apply_chat_template_kwargs
)
multi_modal_data = {}
Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think self.apply_chat_template_kwargs is passed here. Is there a place where it's not passed?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants