audio: crossover: Improve robustness for invalid configuration #23655

Job	Run time
manifest-check	2m 46s
LP64-WIP	2m 58s
build-windows (mnfst, tgl)	3m 22s
build-windows (mnfst, ptl)	3m 17s
build-windows (mnfst, imx8m imx8ulp)	3m 12s
build-linux (mnfst, mtl)	3m 29s
build-windows (mnfst, tgl-h)	3m 9s
build-linux (mnfst, tgl-h)	3m 23s
build-windows (mnfst, mtl)	3m 6s
build-linux (-d, mnfst, lnl)	3m 29s
build-linux (mnfst, ptl)	3m 29s
build-linux (mnfst, imx8m imx8ulp)	3m 13s
build-linux (mnfst, tgl)	3m 30s
build-linux (mnfst, wcl imx95)	3m 22s
build-windows (mnfst, imx8 imx8x)	3m 11s
build-windows (-d, mnfst, lnl)	3m 28s
build-linux (mnfst, lnl)	2m 48s
build-linux (mnfst, imx8 imx8x)	3m 21s
build-windows (mnfst, lnl)	3m 18s
compare-linux-win	25s
	1h 2m 16s

Provide feedback