Storage Framework Implementation by gboutry · Pull Request #609 · canonical/snap-openstack

gboutry · 2025-10-22T08:58:33Z

This pull requests introduces the Sunbeam Storage Framework. This framework goal is to simplify enabling third party backend storage solutions in Sunbeam.
First iteration is only enabling cinder backends, but as the feature evolves, Manila shall also be a target.

For simplicy, a backend is defined by:

A pydantic configuration, basemodel holding no default values except None when the value is not required
A Backend class that will define the strict minimum information about this backend. (Charm, channel...)

Backends can be defined inside the manifest. this will lead, when using sunbeam storage add <type> <name> --accept-defaults, to an automated installation, without user interaction.

Risk enablement, by default, any new storage backend shall be available only under beta and edge channel. After proof of rigorous testing only, can a backend be promoted to "stable-r" risks.

Based on the contribution from @happyhackerhour
#544

fix: tox -e fmt, pep8, mypy reimplement Unit testing to conform to new signatures and changes to use terraform Fix storage backend removal regression and address reviewer comments - Fix partial removal regression: write updated Terraform variables before apply - Remove global registry instance, follow Provider pattern for CLI registration - Add Juju application name validation to prevent invalid backend names - Implement atomic removal operations with proper rollback on failure - Add cleanup-config command to resolve invalid backend configurations - Improve error handling and logging throughout storage backend system - Update deprecated Pydantic __fields__ usage for v2 compatibility - Consolidate imports and reduce emoji usage in console output refactor: clean up Hitachi backend CLI and eliminate code duplication - Move CLI code to dedicated module (hitachi/cli.py) with proper separation - Remove duplicate backend_exists() methods, consolidate in service layer - Eliminate unused service methods (set/reset_backend_config) and tests - Refactor duplicate config filtering logic into reusable internal function - Clean up storage backend __init__.py files to match provider pattern - Update all CLI and backend code to use service layer properly - Fix missing abstract method implementations in test mocks - Update test mocking to reflect architectural changes Signed-off-by: Hugo Vinicius Garcia Razera <root@happyhackerhour.net> - Retry apply and destroy plans on terraform locks - Shows friendly error messege on state locks - Verify juju login status, before add and remove. Signed-off-by: Hugo Vinicius Garcia Razera <root@happyhackerhour.net> - Refactor common Backend Methods into Base Class - Implement PureStorage Backend - separate terraforms-key for each backend - Implementaion of Dell Storage Center Backend - fix show config options for pure storage

Copilot

Pull Request Overview

This pull request introduces the Sunbeam Storage Framework, enabling third-party backend storage solutions for Cinder. The implementation provides a pluggable architecture where backends are defined by Pydantic configuration models and backend classes, with support for automated installation via manifest definitions.

Key Changes:

Implements base storage framework with pluggable backend architecture
Adds three initial backend implementations: Hitachi VSP, Pure Storage FlashArray, and Dell Storage Center
Introduces Terraform-based deployment infrastructure for storage backends
Adds database schema and API endpoints for storage backend management

Reviewed Changes

Copilot reviewed 44 out of 45 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
sunbeam-python/sunbeam/storage/*.py	Core storage framework files including base classes, models, service layer, and CLI
sunbeam-python/sunbeam/storage/backends/*/backend.py	Backend-specific implementations for Hitachi, Pure Storage, and Dell SC
sunbeam-python/tests/unit/sunbeam/storage/*.py	Comprehensive test suite covering framework and backend implementations
sunbeam-microcluster/*.go	Backend storage database schema and API endpoints
cloud/etc/deploy-storage/*.tf	Terraform modules for deploying storage backends
sunbeam-python/sunbeam/core/manifest.py	Manifest extensions for storage configuration
sunbeam-python/sunbeam/main.py	Storage manager registration in main CLI

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

Copilot

Pull Request Overview

Copilot reviewed 44 out of 45 changed files in this pull request and generated 5 comments.

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

hemanthnakkina

Nice work! One minor nit

Based on the assumptions that only a single subordinate charm is required, simplify the backends to only contain: a pydantic config, holding no default values (except none), and a backend instance to host the backend specific values (charm, channel...). Implement the a storage-backends api/database in clusterd to have a single managed places where we store these information. Remove backend_type from all commands except add, we don't support having multiple backends named the same way across providers. Signed-off-by: Guillaume Boutry <guillaume.boutry@canonical.com>

Validate interactive questions from the user using pydantic. Merge generate question functions as they have the same purpose. Signed-off-by: Guillaume Boutry <guillaume.boutry@canonical.com>

wolsen

A few minor comments in-line, but nothing major. I think that this PR is already large enough and would like to see follow up patches to address relevant comments - specifically the drivers that are unsupported in the upstream should have an indicator as such. Until then, this can be handled in documentation.

wolsen · 2025-10-24T08:55:43Z

+            return
+
+        table = Table(title="Storage Backends")
+        table.add_column("Name", style="cyan")


nit: I'm not sure we really need / want the colors in here? We can follow up in a separate PR though as this is already large enough.

wolsen · 2025-10-24T09:09:37Z

+
+
+@pytest.fixture
+def dellsc_backend():


I will note that the dellsc backend FC and iSCSI backends are unsupported in the upstream as they don't have voting CI tests. We should potentially find a way to mark that this is an unsupported driver. Unsupported drivers risk being removed in the upstream.

Signed-off-by: Guillaume Boutry <guillaume.boutry@canonical.com>

gboutry requested a review from Copilot October 22, 2025 10:59

Copilot AI reviewed Oct 22, 2025

View reviewed changes

Comment thread sunbeam-microcluster/database/storage_backend.mapper.go Outdated

Comment thread sunbeam-microcluster/database/storage_backend.mapper.go Outdated

gboutry force-pushed the feat/storage-cli branch from 83654aa to 81ab10d Compare October 22, 2025 11:41

gboutry requested a review from Copilot October 22, 2025 11:50

Copilot AI reviewed Oct 22, 2025

View reviewed changes

gboutry force-pushed the feat/storage-cli branch from 81ab10d to ab0db5e Compare October 22, 2025 12:04

gboutry requested a review from hemanthnakkina October 22, 2025 18:24

hemanthnakkina requested changes Oct 23, 2025

View reviewed changes

Comment thread sunbeam-python/sunbeam/core/manifest.py Outdated

gboutry force-pushed the feat/storage-cli branch from ab0db5e to ec5a7a8 Compare October 23, 2025 07:50

gboutry added 2 commits October 23, 2025 10:05

feat(storage): validate backend questions using pydantic

5903959

Validate interactive questions from the user using pydantic. Merge generate question functions as they have the same purpose. Signed-off-by: Guillaume Boutry <guillaume.boutry@canonical.com>

gboutry force-pushed the feat/storage-cli branch from ec5a7a8 to 7303643 Compare October 23, 2025 08:06

wolsen previously approved these changes Oct 24, 2025

View reviewed changes

refactor(storage): drop unused backend properties

141e58a

Signed-off-by: Guillaume Boutry <guillaume.boutry@canonical.com>

gboutry dismissed wolsen’s stale review via 141e58a November 5, 2025 08:36

gboutry force-pushed the feat/storage-cli branch from 7303643 to 141e58a Compare November 5, 2025 08:36

hemanthnakkina approved these changes Nov 5, 2025

View reviewed changes

hemanthnakkina merged commit bf06431 into canonical:main Nov 5, 2025
9 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Storage Framework Implementation#609

Storage Framework Implementation#609
hemanthnakkina merged 4 commits into
canonical:mainfrom
gboutry:feat/storage-cli

gboutry commented Oct 22, 2025

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

hemanthnakkina left a comment

Uh oh!

Uh oh!

wolsen left a comment

Uh oh!

Uh oh!

wolsen Oct 24, 2025

Uh oh!

wolsen Oct 24, 2025 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants



		@pytest.fixture
		def dellsc_backend():

Conversation

gboutry commented Oct 22, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

hemanthnakkina left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

wolsen left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

wolsen Oct 24, 2025

Choose a reason for hiding this comment

Uh oh!

wolsen Oct 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

wolsen Oct 24, 2025 •

edited

Loading