Remove custom get_train/eval_dataloader from OnlineDPO

[albertvillanova]

Remove custom get_train/eval_dataloader from OnlineDPO,

This PR updates the configuration and trainer logic for Online DPO implementation by adding a new configuration option default (remove_unused_columns=False), and removing custom dataloader methods to simplify the codebase.

Note that transformers.Trainer.get_train_dataloader (and get_eval_dataloader) support skipping the removal of unused columns by setting args.remove_unused_columns=False since v4.54.1 and we support >=4.56.2:

• Remove unnecessary columns for all dataset types in Trainer transformers#17166

Related to:

• Simplify get_train_dataloader in GRPO and RLOO #5276

Changes

Configuration improvements:

• Added the remove_unused_columns option to OnlineDPOConfig, defaulting to False instead of the previous default of True, and updated the documentation to reflect this change.

Codebase simplification:

• Removed custom implementations of get_train_dataloader and get_eval_dataloader from OnlineDPOTrainer, which previously skipped the remove_unused_columns logic, reverting to the standard Trainer behavior.
• Cleaned up imports in online_dpo_trainer.py by removing unused or redundant imports, including wraps, Trainer, seed_worker, and DataLoader.

---

Note

Medium Risk
Changes Online DPO’s dataloader behavior by relying on transformers.Trainer with remove_unused_columns=False, which may affect training/eval if downstream datasets previously depended on column stripping or the prior custom loader behavior.

Overview
Online DPO now defaults to keeping all dataset columns by adding remove_unused_columns=False to OnlineDPOConfig (and documenting the overridden default).

OnlineDPOTrainer drops its custom get_train_dataloader/get_eval_dataloader overrides that existed solely to bypass unused-column removal, and cleans up the related now-unused imports—delegating dataloader construction back to transformers.Trainer.

^{Written by Cursor Bugbot for commit d9bce4c. This will update automatically on new commits. Configure here.}

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[qgallouedec]

Thanks, I think it makes sense. I'm not even sure that online dpo works with remove_unused_columns=True. If we decide to refactor it in the future we should aim to something closer to GRPO where this feature is properly supported