Fix constrained-decode latency: bounded top-K, skip unused logSumExp#538

FuJacob merged 1 commit into

mainFuJacob/cotabby:mainfrom

fix/llm-generation-slow-regressionFuJacob/cotabby:fix/llm-generation-slow-regressionCopy head branch name to clipboard