abort_repair called without a valid client state

[jmpesp]

The following Pantry core was found during the colo rack upgrade to 19.1:

$ /staff/mdb/mdb core.oxz_crucible_pantry_773c1e58-c0b6-4a3f-902d-bb01eb17eefa.crucible-pantry.9810.1776195718
Loading modules: [ libumem.so.1 libnvpair.so.1 libc.so.1 ld.so.1 ]
> $G
C++ symbol demangling enabled
> $C -S
fffff5ffce5fe890 libc.so.1`_lwp_kill+0xa()
fffff5ffce5fe8c0 libc.so.1`raise+0x22(6)
fffff5ffce5fe910 libc.so.1`abort+0x58()
fffff5ffce5fe920 ~std::sys::pal::unix::abort_internal::h515b31030a784ec5+8()
fffff5ffce5fe930 ~std::process::abort::h263ae6dd5f7f06e2+8()
fffff5ffce5fe940 ~__rustc::__rust_abort+8()
fffff5ffce5fe950 ~__rustc::__rust_start_panic+8()
fffff5ffce5fe9b0 __rustc::rust_panic+0xd()
fffff5ffce5fea70 std::panicking::rust_panic_with_hook::h24c1a837a734501c+0x224()
fffff5ffce5feab0 std::panicking::begin_panic_handler::{{closure}}::hebb980b0c684b90d+0x98()
    4246ef0: abort_repair called without a valid client state: [Connecting { state: WaitConnect, mode: New }, Active, Active]
fffff5ffce5feac0 ~std::sys::backtrace::__rust_end_short_backtrace::h4b651dba38b6cba9+8()
fffff5ffce5feaf0 ~__rustc::rust_begin_unwind+0x1b()
fffff5ffce5feb20 ~core::panicking::panic_fmt::h717aa56ae600c892+0x1e()
fffff5ffce5feba0 ~crucible::downstairs::Downstairs::abort_repair::h881c263d2afea588+0x2b9()
fffff5ffce5ff060 crucible::downstairs::Downstairs::check_and_continue_live_repair::h65542c5e6980b37d+0x504()
    17458f10: not ready to deactivate due to state Activetrol-planeoxideinternalAAAAAA
    2c00280: crucible::clientNegotiationStatewait_for_promoteClientStopReasonfailed_reconcileFailedLiveRepairConnectionFailed
    2c02045: upstairs/src/client.rsWaitConnectStartWaitForPromoteWaitForRegionInfoGetExtentVersionsWaitQuorumReconcileLiveRepairReadywait_connectwait_for_region_infoget_extent_versionswait_quorumreconcilelive_repair_readyvariant identifieradjacently tagged enum NegotiationStateOne-shot sender to ask the client to open its connection
fffff5ffce5ff550 crucible::upstairs::Upstairs::apply::h945642c1c69e7130+0x2008()
fffff5ffce5ff9d0 crucible::up_main::_$u7b$$u7b$closure$u7d$$u7d$::h8afda6b8eb8e79d7 +0x5a2()
fffff5ffce5ffa30 tokio::runtime::task::core::Core<T,S>::poll::h5a8439232d0d479f+0x35()
fffff5ffce5ffa90 tokio::runtime::task::harness::Harness<T,S>::poll::h5e199112751e828d+0x5b()
fffff5ffce5ffb00 tokio::runtime::scheduler::multi_thread::worker::Context::run_task::h5f12fa0ca3d2c658+0x26a()
fffff5ffce5ffbc0 tokio::runtime::scheduler::multi_thread::worker::Context::run::hb0449d62a73027d4+0xe6e()
fffff5ffce5ffc20 tokio::runtime::context::scoped::Scoped<T>::set::ha84d898356867e25+0x2a()
fffff5ffce5ffce0 tokio::runtime::context::runtime::enter_runtime::h06610113404d8907+0xb1()
fffff5ffce5ffd20 tokio::runtime::scheduler::multi_thread::worker::run::h0fa02f8f6142cd7d+0xa0()
fffff5ffce5ffd80 tokio::runtime::task::core::Core<T,S>::poll::h0ac37724106c5817+0x70()
fffff5ffce5ffdd0 tokio::runtime::task::harness::Harness<T,S>::poll::h701a6929d17f7324+0x8a()
fffff5ffce5ffea0 tokio::runtime::blocking::pool::Inner::run::h90f1cbc550c29eea+0xf4()
fffff5ffce5ffee0 std::sys::backtrace::__rust_begin_short_backtrace::h5d8319ae68debfd4+0x3e()
fffff5ffce5fff80 core::ops::function::FnOnce::call_once{{vtable.shim}}::h9f704e8cb6e3364b+0xa8()
fffff5ffce5fffb0 std::sys::pal::unix::thread::Thread::new::thread_start::h128a92fdf19e3e8a+0x2f()
fffff5ffce5fffe0 libc.so.1`_thrp_setup+0x77(fffff5ffee62f240)
fffff5ffce5ffff0 libc.so.1`_lwp_start()

The panic string is:

abort_repair called without a valid client state: [Connecting { state: WaitConnect, mode: New }, Active, Active] is unexpected.