Skip to content

Conversation

@kmsgnnew
Copy link

Resolves #977

#977

Added implementation of GDPO. Shared below reference link and implementation. This paper improves upon GRPO.

Reference
Reference paper
https://arxiv.org/abs/2601.05242
Reference Implementation
https://github.com/NVlabs/GDPO/blob/b080e63d0126870ad08acc8ebc3f04b728175a9e/trl-GDPO/trl-0.18.0-gdpo/trl/trainer/grpo_trainer.py#L1222

Colab Notebook
NiL

Checklist

  • I have added all the necessary unit tests for my change.
  • I have verified that my change does not break existing code and all unit tests pass.
  • I have added all appropriate doc-strings/documentation.
  • My PR is based on the latest changes of the main branch (if unsure, rebase the code).
  • I have signed the Contributor License Agreement.
  • I have followed Contribution Guidelines.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add GDPO Support

1 participant