virtio device reset without multiqueue kills vnic instead

[iximeow]

@leftwo spotted this on a minibar. I'd tested #1033 against Debian 13 because it's what I happened to reproduce the issue on, but that "fix" also made Ubuntu unable to drive the NIC at all (with or without a reboot).

(slightly more subtle than this, see below)

[dancrossnyc]

It's worth checking this against c-bhyve to see whether it fails in a similar way.

It's hard to imagine how the reset logic would prevent the driver from, er, driving the NIC initially, though.

[iximeow]

It's worth checking this against c-bhyve to see whether it fails in a similar way.

if you have a moment from other fires, I'd be curious to check, but with #1047 I think that's a pretty clear "Propolis was wrong, don't do that"

It's hard to imagine how the reset logic would prevent the driver from, er, driving the NIC initially, though.

this is where I'm a bit frustrated, because my 24.04 goes exactly like this. and all my test images are either:

• multiqueue aware, set up 11 queue pairs, reboot resets the whole device which gets all queues, shrinks PAIRS/USEPAIRS back down to 1 until multiqueue is negotiated again after reboot, everything's great
• not multiqueue aware, set up 1 queue pair, reboot resets the whole device which EINVAL's for the last 10 queue pairs but we're resetting the device so none of that actually matters

what happens in Alan's image, it looks like, is something before MQ negotiation pokes the reset bit in the virtio status register. it occurs to me it'd be nice to have DTrace probes around virtio device lifecycle like this, because I'm speculating from ioctls/returns and the fact that the guest does this early in boot. regardless we (incorrectly) try resetting all the rings, get everything after the first pair to VRingState::Fatal, and later multiqueue negotiation fails. it's not as bad as "all Linux X.Y is busted", probably, but.. jeez.

[iximeow]

I've figured out what was different and more specifically what's happening: #1047 (comment)

without a boot order OVMF does Something with the nic, then Linux tries to reset it when it starts operating the nic. that reset is what tickled the issue, and doesn't depend on Ubuntu anything. in Octoberish I'd added boot_order for all my test image configs for unrelated reasons.