Skip to content

Upgrade transformers to 5.x and other dependencies#1285

Draft
dxqb wants to merge 8 commits intoNerogar:masterfrom
dxqb:upgrade_all
Draft

Upgrade transformers to 5.x and other dependencies#1285
dxqb wants to merge 8 commits intoNerogar:masterfrom
dxqb:upgrade_all

Conversation

@dxqb
Copy link
Collaborator

@dxqb dxqb commented Feb 4, 2026

There are some major upgrades of dependencies:

  • transformers from 4.x to 5.0
  • huggingface_hub from 0.34.x to 1.3.x
  • Torch 2.10
  • python 3.13
  • CUDA 13.0

This PR upgrades all dependencies to their latest version (except ROCM).
huggingface_hub was held back by transformers so far. With transformers 5.0, huggingface_hub gets a major upgrade.

This is why I've enabled XET in this PR. Previous problems with XET might not apply anymore - to be tested.
If XET works well, #770 could be reverted because you don't need to download multiple files anymore to get full speed.
Would close #949

@dxqb
Copy link
Collaborator Author

dxqb commented Feb 4, 2026

updating to CUDA 13 might be too early:

/OneTrainer/venv/lib/python3.12/site-packages/torch/cuda/__init__.py:184: UserWarning: CUDA initialization: The NVIDIA driver on your system is too old (found version 12090). Please update your GPU driver by downloading and installing a new version from the URL: http://www.nvidia.com/Download/index.aspx Alternatively, go to: https://pytorch.org to install a PyTorch version that has been compiled with your version of the CUDA driver. (Triggered internally at /pytorch/c10/cuda/CUDAFunctions.cpp:119.)

@dxqb
Copy link
Collaborator Author

dxqb commented Feb 4, 2026

CUDA 12.8 instead. Even many RTX 5090 clouds are still on drivers not supported by CUDA 13

@dxqb

This comment was marked as resolved.

@O-J1

This comment was marked as resolved.

@O-J1
Copy link
Collaborator

O-J1 commented Feb 6, 2026

Self note: Tested and the scalene upgrade breaks on windows. Need to look into it

@dxqb
Copy link
Collaborator Author

dxqb commented Feb 7, 2026

Self note: Tested and the scalene upgrade breaks on windows. Need to look into it

or downgrade to that version, but leave a comment that scalene must not be upgraded. which version?

@O-J1
Copy link
Collaborator

O-J1 commented Feb 10, 2026

Unfortunately have to revert.

Findings:

  1. This PR is bricked in all versions of python we can support thanks to Transformers v5. Migration Guide here
  2. The chosen numpy and scipy variants drop support for python 3.10
  3. We'd have to downgrade transformers and huggingface-hub an 'old-new' version (still published in 2026 but maitenence versions instead of breaking)
  4. Scalene also had breaking changes, pin at 1.5.55 or upgrade commands.

@dxqb dxqb marked this pull request as draft February 12, 2026 09:29
@dxqb dxqb changed the title Upgrade dependencies Upgrade transformers to 5.x and other dependencies Feb 15, 2026
@dxqb
Copy link
Collaborator Author

dxqb commented Feb 15, 2026

torch 2.10 alone first #1266 because this PR requires some more work

@O-J1 O-J1 added the waiting waiting for something / cannot fix currently label Feb 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

waiting waiting for something / cannot fix currently

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feat]: Renable Xet under hf_hub once they get their bugs sorted out.

2 participants