Power Consumption in Idle on Strix Halo

[Bushido76]

Dear all,

thank you very much for this great project. It's a pleasure to see, what performance increase DFLASH could bring, when combined with local hardware. (It's even faster on my hardware than inference using integrated MTP heads.)

I'm running lucebox-hub in the following environment:

• Framework Desktop with Ryzen AI Max 395+ and 128GB Unified RAM
• lucebox-hub compiled for HIP ROCm 7.2.4
• Fedora 43 on Kernel 7.0.12
• Plain llama.cpp and ROCmFP4 do not have the same issues

Server-Call:
LD_LIBRARY_PATH=/opt/rocm/lib:$LD_LIBRARY_PATH
DFLASH27B_DRAFT_SWA=2048
DFLASH27B_PREFILL_UBATCH=512
./lucebox-hub/server/build/dflash_server
~/luce-models/Qwen3.6-27B-Q4_K_M.gguf
--draft ~/luce-models/draft/dflash-draft-3.6-q8_0.gguf
--draft-swa 2048
--ddtree
--ddtree-budget 22
--max-ctx 131072
--cache-type-k q8_0
--cache-type-v q8_0
--host 0.0.0.0
--port 4500
--model-name lucebox-strix
--draft-residency request-scoped
--prefill-drafter ~/luce-models/draft/Qwen3-0.6B-BF16.gguf
--prefill-compression auto
--prefill-threshold 4000
--prefill-skip-park

The server is running stable (from time to time it seems to fall into "CPU-mode"), but after the first "inference", it does not go back into "idle" - meaning, that my PC continously consumes 50+ Watts instead of around 12 Watts in idle mode. That's why I need your help to justify, whether this could be a bug or whether there is a parameter to mitigate this issues. If you need more information please let me know. As I cannot see any high CPU load it just seems to be an "idle state" issue.

Edit: Deactivating pflash and/or draft-residency = scoped does not change the situation.

Thank you and regards

[davide221]

Thanks for the detailed report. I reproduced your exact config (same gfx1151 Strix Halo iGPU, HIP build, same models and flags) and watchedthe APU power: it goes to 100W during inference, then drops back to 10W within 5s and stays there. Same result for short and long (>4000 tok, full pflash) prompts. I can't reproduce the stuck 50W.

So this doesn't seem to be a bug on our side: the idle loop blocks on a condition variable, no background thread touches the GPU, and nothing pins a GPU power profile, which is why toggling pflash/draft-residency does nothing. It looks like amdgpu gfxoff not re-engaging after compute on your stack.

My env: Ubuntu 24.04, kernel 6.17, ROCm 7.2.2. Yours: Fedora 43, kernel 7.0.12, ROCm 7.2.4.

Could you check whether changing the ROCm version (e.g. downgrading to 7.2.2 like ours) makes the issue persist or go away? That would help narrow it down.

[Bushido76]

Dear Davide,
thank you very much for your kind response and your effort to analyse the situation. I think you are right - the issue is somehow related to my local software constellation. When I'm running lucebox as the only inference engine I'm not able to reproduce the issue. When other inference engines like llama.cpp and ROCmFP4 are on stand-by in parallel (inference is never done in parallel - it's only the "stand-by") the issue occurs. Could be, that the driver accesses are somehow interfering with each others. Llama.cpp and ROCmFP4 are working flawlessly in parallel (I continously hold different models on stand-by in my machines memory). The issues begin, when I boot up lucebox in parallel.
IMO speculative decoding is one of the best evolutions for local AI in the last months. Normally I work with MTP which gives "a good average". For DFlash the Max-Values are higher - with the downside, of much lower speed when it comes to "creativity". Here I'm really curious, how DFlash V2, Eagle 3.x and DSpark will behave on AMD hardware. It's a great time to be part of this bubble. :) Do you see advantages of DFlash V2 for our local Hardware?
Thanks and regards