Skip to content

Opt-in batched block-diagonal attention: 2x pretraining throughput, validated equivalent#5

Open
alex-crlhmmr wants to merge 1 commit into
thuml:mainfrom
alex-crlhmmr:feature/batched-block-diagonal-attention
Open

Opt-in batched block-diagonal attention: 2x pretraining throughput, validated equivalent#5
alex-crlhmmr wants to merge 1 commit into
thuml:mainfrom
alex-crlhmmr:feature/batched-block-diagonal-attention

Add opt-in batched block-diagonal attention (2x pretraining throughput)

550da55
Select commit
Loading
Failed to load commit list.

Workflow runs completed with no jobs