Skip to content

Add a test verifying only one downstairs required to activate and read from a RO volume.#1962

Open
leftwo wants to merge 3 commits into
mainfrom
alan/ro-read-just-one
Open

Add a test verifying only one downstairs required to activate and read from a RO volume.#1962
leftwo wants to merge 3 commits into
mainfrom
alan/ro-read-just-one

Conversation

@leftwo

@leftwo leftwo commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Added a test to verify that a read only volume that can connect to just one downstairs will be able to activate and read from that volume.

Shade was being thrown on Crucible, and this test was written in part to confirm that a single downstairs is all you need to talk to a RO volume.

quotes from the chat:

but we get vcpus going at 19:20:57.697 and then at 19:21:07.691 propolis logs about timeouts two downstairses
so that seems funky

And

"msg":"client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }))"
after some seconds when the upstairs tried again, something was quite sad

And

it does seem odd to have failed to connect to a downstairs though
this doesn't explain why we'd fail to boot the RO volume for sure

And

because if that means that downstairs 0 and 1 were active, but 2 wasn't, and then we lost 0 and 1 .. that sounds like it would be a problem
but i don't know if that's possible

These slights were handled, and Crucible's good standing was returned.

leftwo added 2 commits July 23, 2026 14:53
Add integration_test_just_read_one_downstairs, which creates three
read-only downstairs, stops two of them, and confirms that a read-only
upstairs can still activate and read from the remaining single
downstairs.

Also add a stop() helper to TestDownstairs so a test can bring an
individual downstairs down while keeping the rest of the set running.
Augment integration_test_just_read_one_downstairs to manually send a
flush after the first read, then confirm a second read completes
successfully.  With only one downstairs running, the flush still acks
because the two stopped downstairs have their jobs moved to Skipped, so
the flush is complete on all clients rather than hanging.
@leftwo
leftwo requested a review from jmpesp July 23, 2026 22:31
Comment thread integration_tests/src/lib.rs Outdated
id: Uuid::new_v4(),
block_size: BLOCK_SIZE as u64,
sub_volumes: vec![],
read_only_parent: Some(Box::new(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Flushes won't be sent to read-only parents, you'll have to change the structure of the VCR to put this under sub volumes instead

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants