Skip to content

add deepseek mhc pre#99

Open
qescccczmr wants to merge 5 commits intoDeepLink-org:mainfrom
qescccczmr:chengqinxiu
Open

add deepseek mhc pre#99
qescccczmr wants to merge 5 commits intoDeepLink-org:mainfrom
qescccczmr:chengqinxiu

Conversation

@qescccczmr
Copy link
Contributor

N=8192 H=4096 M=4 BH=1024 NH=4 n_splits=1 sinkhorn=10
── TileLang vs Ref ── ── Triton-TLX vs Ref ──
Output max_abs mean_abs max_rel max_abs mean_abs max_rel
────────────── ────────── ────────── ──────────
post_mix 5.96e-08 1.35e-10 1.37e-07 1.19e-07 1.02e-08 2.38e-07
comb_mix 8.94e-08 1.23e-08 4.21e-07 1.19e-07 1.88e-08 4.96e-07
layer_input 1.56e-02 2.56e-08 4.45e-01 1.56e-02 3.15e-08 6.60e-01

TileLang vs Triton-TLX:
Output max_abs mean_abs max_rel
────────────── ────────── ────────── ──────────
post_mix 1.19e-07 1.02e-08 2.38e-07
comb_mix 1.04e-07 1.79e-08 4.49e-07
layer_input 1.56e-02 1.78e-08 7.05e-01

@CLAassistant
Copy link

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.


chengqinxiu seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.

"""
pid = tl.program_id(0)

mo = tl.arange(0, M) # [M]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mo变量命名改下,以及下面存在同样问题

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants