v0.2.20: Fused NN Kernels + Flash Attention 3 SM120 + FP8 Block-Scale MMA#193

Merged

m96-chan merged 23 commits intomainfrom

release/v0.2.20

Jan 26, 2026

Commits on Jan 6, 2026

feat(ops): add native Conv1d CUDA kernel (#180)

m96-chan
and
claude
committed

Commits on Jan 12, 2026

feat(examples): add Llama Guard 3 content safety classifier

m96-chan
and
claude
committed

Commits on Jan 16, 2026

feat(llm): add LLaMA 4 native CUDA kernels

m96-chan
and
claude
committed
chore: add security benchmark scripts to gitignore
m96-chan
committed
chore: update cutlass submodule (alignment fix)
m96-chan
committed
feat(attention): add Flash Attention 3 for SM120 (Blackwell)

m96-chan
and
claude
committed
feat(ops): add TMA utilities for SM90+ kernels

m96-chan
and
claude
committed
wip(fa3): add TMA-enabled Flash Attention 3 kernel

m96-chan
and
claude
committed
feat(attention): integrate TMA FA3 into SDPA dispatch

m96-chan
and
claude
committed
fix(fa3): resolve __syncthreads divergence causing kernel hang at scale

m96-chan
and
claude
committed
refactor(fa3): parallelize softmax and fix consumer warp indexing

m96-chan
and
claude
committed
fix(fa3): resolve non-determinism in TMA FA3 attention kernel

m96-chan
and
claude
committed
wip(fa4): add Flash Attention 4 SM120 Phase 1 BF16 baseline

m96-chan
and
claude
committed
wip(fa4): Phase 2 NVFP4 Q@K^T external validation

m96-chan
and
claude
committed
bench(fa4): add Phase 3 full NVFP4 pipeline validation

m96-chan
and
claude
committed
docs(fa4): add SM120 implementation report

m96-chan
and
claude
committed
feat(fa3): add SM120 tuning configs with version selection

m96-chan
and
claude
committed

Commits on Jan 17, 2026

Commits on Jan 26, 2026