Skip to content

fix: prevent fp8 mma dot from lowering to fp16 on sm120#16

Open
stmoonar wants to merge 2 commits into
ByteDance-Seed:distfrom
stmoonar:dist
Open

fix: prevent fp8 mma dot from lowering to fp16 on sm120#16
stmoonar wants to merge 2 commits into
ByteDance-Seed:distfrom
stmoonar:dist

Conversation

@stmoonar

@stmoonar stmoonar commented Jun 2, 2026

Copy link
Copy Markdown

New contributor declaration

  • I am not making a trivial change, such as fixing a typo in a comment.

  • I have written a PR description following these
    rules.

  • I have run pre-commit run --from-ref origin/main --to-ref HEAD.

  • Select one of the following.

    • I have added tests.
      • /test for lit tests
      • /unittest for C++ tests
      • /python/test for end-to-end tests
    • This PR does not need a test because FILL THIS IN.
  • Select one of the following.

    • I have not added any lit tests.
    • The lit tests I have added follow these best practices,
      including the "tests should be minimal" section. (Usually running Python code
      and using the instructions it generates is not minimal.)

Description

When using triton dist write fp8 gemm kernel on RTX PRO5000(sm120), found that the gemm time in the intra profile is not better than fp16.
Then use ncu profile the gemm kernel, found it used fp16->fp32 mma instead of fp8_>fp32 mma:
image

Solution

Found the pr in triton upstream that solve this: triton-lang#7409

Fixed Result

Using ncu profile again:
image

@stmoonar stmoonar changed the title fix: prevent fp8 mma dot from lowering to fp16 fix: prevent fp8 mma dot from lowering to fp16 on sm120 Jun 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant