linalg/mmm: naive tile walk when L2 is undetectable

[kali]

The single-thread block budget fell back to a 256 KiB guess when the cache probe found nothing, which over-blocks the very cores that fallback is for: small embedded arm parts whose cache the OS does not expose and whose tiny L1 an L2-sized block blows, making the blocked walk slower than the naive stream the hardware prefetcher handled well. The budget helpers now return None on an unknown cache and the walk takes the naive loop (edge 1) there, so only cores with a detected L2 get the blocking they were tuned for.

[kali]

ping @czoli1976

[github-actions]

🔴 Bench vs main — 12 speed regression(s)

Reference: main nightly, latest 2026-07-13 (0d old) · PR e1cbb4288 · ran on apple-m1-max, cortex-a53, cortex-a55, cortex-a7, i9-11900kb_rtx-4060, jetson-orin-nx · 1382 metrics compared

Speed — evaltime · prefill · decode

Δ	metric	device	main → PR
🔴 +13.5%	mobilenet_v1_1 evaltime · pass	cortex-a55	375 ms → 425 ms
🔴 +11.6%	speaker_id evaltime · pulse8	cortex-a55	1.18 ms → 1.32 ms
🔴 +11.5%	mobilenet_v2_1 evaltime · pass	cortex-a55	519 ms → 578 ms
🔴 +9.2%	en_tdnn_8M evaltime · 2600ms	cortex-a7	1.4 s → 1.52 s
🔴 +8.9%	mdl_en_2019_Q3_librispeech_onnx evaltime · 2600ms	cortex-a7	459 ms → 500 ms
🔴 +8.8%	en_tdnn_15M evaltime · 2600ms	cortex-a7	2.91 s → 3.17 s
🔴 +8.4%	mobilenet_v1_1 evaltime · pass	cortex-a7	1.31 s → 1.42 s
🔴 +6.9%	en_tdnn_15M evaltime · 2600ms	cortex-a55	745 ms → 796 ms
🔴 +6.4%	mobilenet_v2_1 evaltime · pass	cortex-a7	1.79 s → 1.9 s
🔴 +5.7%	mobilenet_v2_1 evaltime · pass	cortex-a53	780 ms → 825 ms
🔴 +5.6%	en_tdnn_8M evaltime · 2600ms	cortex-a55	324 ms → 343 ms
🔴 +5.1%	en_tdnn_15M_nnef evaltime · pulse_240ms	cortex-a7	275 ms → 289 ms

🟢 4 improvement(s)

Δ	metric	device	main → PR
🟢 -10.0%	arm_ml_kws_cnn_m evaltime · pass	cortex-a7	16.3 ms → 14.6 ms
🟢 -9.2%	inceptionv3 evaltime · pass	cortex-a53	2.87 s → 2.61 s
🟢 -6.7%	hey_snips_v1 evaltime · 400ms	cortex-a7	15.3 ms → 14.3 ms
🟢 -5.7%	en_tdnn_15M evaltime · pulse_240ms	cortex-a55	66.9 ms → 63 ms

lower is better except prefill/decode (tok/s) · adaptive thresholds (max(floor, k×noise) vs the series' own history) · single-shot vs nightly reference · full report → run

[kali]

Mmmm tricky one. I was chasing the inception regression, and I broke everything else.

[czoli1976]

At which point did you detect the regression? which PR? is the eval time the concern or wha?

[czoli1976]

The tricky part is that "undetectable L2" isn't one workload — the little cores split on what they want:

• mobilenet_v1/v2, en_tdnn_* regress because they were getting real reuse from the 256 KiB guess — which roughly matches the actual (just-unreported) L2 on a53/a55/a7.
• inception / kws / hey_snips improve because 256 KiB over-blocks their tiny L1.

So 256 KiB and edge 1 are both extremes, and the sweet spot is in between. Your own diagnosis points right at it: the failure mode is "an L2-sized block blows L1", so budget from L1 rather than dropping the tile entirely. CacheInfo::l1_data_or_default() already hands back 64 KiB on arm64 when L1 is hidden:

fn l2_block_budget_bytes() -> Option<usize> {
    let ci = crate::cache::cache_info();
    // detected L2 → tuned budget; hidden → block to L1, not an L2-sized guess
    // that blows L1, and not naive which strips the reuse the GEMM-bound models need.
    Some(tier_budget_bytes(ci.l2, 1, 3).unwrap_or_else(|| ci.l1_data_or_default()))
}

That keeps a real but small tile — shouldn't blow L1, should hand mobilenet/tdnn back their reuse without re-inflating inception. The L1 fraction is a knob to tune. Pure hypothesis on my end though: I've got no a7/a53/a55 to bench on, so it'd need a run through the bot to confirm it lands in the middle instead of just tilting the see-saw the other way.

[czoli1976]

My imaginary friend seems confident here

[kali]

I'm trying to recover performance that slipped on long term time series that i was not monitoring very closely (e.g. inceptionv3 on intel). I think these regressions started before you started working on tract, and I'm trying to address a wider bag of performance waste with #2478 . It looks like a simple linear model can equal the choices of the DNN on the cortex devices, so I'm looking into generalizing the approach. I'll come back to the cache once I'm convinced we're not picking stupid kernels to start with.