Storage encryption#3
Open
mpyne1 wants to merge 40 commits into
Open
Conversation
- Add control plane integration (controlplane-client.ts, controlplane-types.ts, controlPlane store) - Add backend type detection (auto-detect from connection name/endpoint) - Add backend type picker to EndpointConnectionModal - Add KMS readiness check in Set Encryption modal (based on CP policies) - Add Verify Encryption context menu item in BucketsView - Improve KMS error messages with deep-link to Encryption Manager - Route minio/rustfs/generic backends to correct CP RPC methods - Add bucket encryption get/set/remove via Python CLI (cmd_object.py) - Add storage class support (cmd_storage_class.py) - Show encryption badge and auto-SSE on uploads in ObjectsView - Add encryption details to ObjectDetailsPanel
…utton shows BucketsView.vue was not calling checkAvailability() during onMounted, causing cpStore.isAvailable to remain false and hiding the Verify Encryption action on buckets.
- ObjectsView: fall through to connection defaultSseKmsKeyId when bucket has aws:kms but no KMSMasterKeyID in bucket policy - BucketsView: verify encryption falls back to direct S3 check when control plane verify fails (e.g. remote RGW without local ceph CLI)
- Pass SSE params (sse, sseKmsKeyId) when creating folders - Send bucketName param in all control plane RGW RPC calls - Fall back to bucket-default encryption in stat-object when HeadObject omits SSE fields - Lower multipart copy/rename threshold from 5GiB to 100MiB for backend compatibility - Fix ISO 8601 parsing in object retention (cmd_lock.py) - Move encryption badge to bucket header, remove per-row encryption column - Context menu opens upward for bottom rows in BucketsView
…pit-S3ObjectBroswer into storageEncryption
- Add controlplane-client setBucketEncryption/getBucketEncryption methods - Route MinIO SSE-KMS through control plane (minio.setBucketEncryption) instead of direct boto3 API which doesn't work with KES - Normalize aws:kms/AES256 algorithm names for MinIO backend - Add S3 Bucket Key description text under checkbox - Add MinIO-specific error messaging for KMS not configured
- Add compatiblePolicies computed that filters policies matching the current connection's backend (minio/rustfs/ceph-rgw/generic) - Use compatiblePolicies in BucketsView template instead of all policies - kmsReady computed based on compatible policy availability
- endpoint_url_from_cfg now uses https:// when useTls is true - make_client uses tlsVerify config to control SSL cert verification - Added tlsVerify toggle to connection modal UI - Added tlsVerify field to types, connection-store, and summary
- Update setBucketEncryptionAction to accept and pass accessKeyId/secretAccessKey - Thread credentials through to cpVerifyRoundtrip call - Update applySetEncryption to pass connConfig credentials when calling setBucketEncryptionAction - Update verifyBucketEncryption signature to accept and pass credentials - Fixes NoSuchBucket error during Apply on RGW buckets without named connections
- Remove fallback to bucket default encryption in stat-object - Report only what HEAD response actually says - If object is unencrypted (ServerSideEncryption=None), show as 'None', not bucket default - Fixes UI showing bucket encryption for objects uploaded before encryption was enabled - RGW/MinIO properly distinguish between unencrypted objects and bucket defaults
…ation - controlplane-client.ts: add endpoint/connectionName params to verifyRoundtrip - controlPlane.ts: pass targetId instead of undefined, forward endpoint/connectionName - BucketsView.vue: pass endpoint and connectionName to deepVerify call
- Catch CERTIFICATE_VERIFY_FAILED in cmd_list.py and return user-friendly message - Show helpful tip in BucketsView when SSL error is detected - Fix version string showing 'undefined' (buildVersion -> build_number)
Instead of guessing the S3 backend type from the connection name or port, probe the endpoint directly using HTTP health/header checks: - GET /health for RustFS detection - GET /minio/health/live for MinIO detection - Server header / x-rgw-request-id for Ceph RGW detection - Falls back to 'generic' when no match Added Python cmd_detect_backend_type + _probe_backend_type in cmd_list.py, CLI dispatch in main.py, TypeScript detectBackendType() in s3Buckets.ts, and integration in BucketsView.vue with auto-detection on refresh.
- Add KesSetup.vue: simplified KES config without server CRUD, editable after setup - Add RustfsKmsSetup.vue: RustFS KMS config with provider/key selection - Update controlplane-client.ts: pass host to all MinIO/KES API calls - Update BucketsView.vue: pass connectionHost to KES and RustFS setup tabs
…elper - RustFS KMS Setup: hide vault addr when provider selected, token optional, SSH deploy key helper when remote host SSH fails - KES Setup: same provider-first pattern, pass providerId to backend, SSH deploy key helper for remote hosts - controlplane-client: add sshFailed/isRemote to KmsPreflightResult and MinIODiscovery types, make minioConfigureKes accept providerId - RgwVaultSetup: add component (already deployed earlier)
- Disable Apply button when SSE-KMS is selected without choosing a key policy - Add yellow warning message when KMS algorithm selected but no key policy chosen - Prevents users from attempting encryption configuration without proper policy selection - Improves UX validation for KMS-based encryption setup
- RustFS: sets up ncat proxy (127.0.0.1:8202 -> vault) to bypass SAN mismatch - MinIO KES: passes skipTlsVerify to backend configureKes handler - RGW Vault: passes skipTlsVerify to backend rgwConfigureVault handler - controlplane-client.ts: added skipTlsVerify param to all three configure functions
There was a problem hiding this comment.
Pull request overview
This PR adds end-to-end “storage encryption” support to the Cockpit S3 Browser: bucket-level encryption management (get/put/delete), connection-level default SSE settings, backend-type detection (MinIO/RGW/RustFS), and UI flows + docs for configuring KMS backends via the control plane.
Changes:
- Added CLI + UI support for bucket encryption configuration (SSE-S3/SSE-KMS) and surfaced encryption status in bucket/object views.
- Introduced control-plane integration components (KES/MinIO, RustFS KMS, RGW Vault) plus backend detection to route encryption tooling.
- Added SSE flag plumbing across several object operations (upload/copy/create-folder/storage-class) and improved error classification/messages.
Reviewed changes
Copilot reviewed 47 out of 113 changed files in this pull request and generated 20 comments.
Show a summary per file
| File | Description |
|---|---|
| system_files/opt/45drives/houston/s3Navigator/scripts/utils.py | Add TLS verify toggle + endpoint scheme selection. |
| system_files/opt/45drives/houston/s3Navigator/scripts/main.py | Add CLI commands for backend detection + bucket encryption + SSE args for create-folder. |
| system_files/opt/45drives/houston/s3Navigator/scripts/cmd_storage_class.py | Add SSE flags to storage-class change multipart copy. |
| system_files/opt/45drives/houston/s3Navigator/scripts/cmd_prefix.py | Improve prefix deletion robustness; add SSE flags to folder marker creation. |
| system_files/opt/45drives/houston/s3Navigator/scripts/cmd_object.py | Add SSE flags to upload/copy; add bucket encryption CRUD; surface object encryption in stat. |
| system_files/opt/45drives/houston/s3Navigator/scripts/cmd_lock.py | Add stricter ISO-8601 parsing regex. |
| system_files/opt/45drives/houston/s3Navigator/scripts/cmd_list.py | Add SSL self-signed error hint + unauthenticated backend probing. |
| packaging/ubuntu-jammy/changelog | Version bump + changelog entries. |
| packaging/ubuntu-focal/changelog | Version bump + changelog entries. |
| packaging/rocky-el9/main.spec.j2 | Version bump + RPM changelog entries. |
| packaging/rocky-el8/main.spec.j2 | Version bump + RPM changelog entries. |
| package.json | Update Yarn packageManager version. |
| manifest.json | Version/build bump and stable flag change. |
| docs/rustfs-kms-guide/transcript.txt | New RustFS KMS video transcript. |
| docs/rustfs-kms-guide/rustfs-kms-guide.dokuwiki | New RustFS KMS written guide. |
| docs/rgw-encryption-guide/transcript.txt | New RGW encryption video transcript. |
| docs/rgw-encryption-guide/rgw-encryption-guide.dokuwiki | New RGW encryption written guide. |
| docs/rgw-encryption-guide/README.md | New RGW encryption Markdown guide. |
| docs/minio-kes-guide/transcript.txt | New MinIO KES video transcript. |
| docs/minio-kes-guide/README.md | New MinIO KES Markdown guide. |
| docs/minio-kes-guide/minio-kes-guide.dokuwiki | New MinIO KES written guide. |
| docs/encryption-guide.md | New general encryption user guide (Markdown). |
| docs/encryption-guide.dokuwiki | New general encryption user guide (DokuWiki). |
| cockpit-s3-browser/vite.config.js | Adjust app version define to use manifest build_number. |
| cockpit-s3-browser/tsconfig.tsbuildinfo | Updated TS build metadata (new files in project graph). |
| cockpit-s3-browser/src/views/ObjectsView.vue | Show bucket encryption badge; auto-inject SSE params into operations; improve error messaging. |
| cockpit-s3-browser/src/views/BucketsView.vue | Add encryption column + actions; backend detection; add setup tabs for KES/RustFS/RGW. |
| cockpit-s3-browser/src/types/index.ts | Extend endpoint config with TLS verify + default SSE + backend type; extend Stat with encryption metadata. |
| cockpit-s3-browser/src/stores/controlPlane.ts | New store for control-plane availability, providers/policies, and encryption actions. |
| cockpit-s3-browser/src/scripts/connection-store.py | Persist TLS verify in stored connection config. |
| cockpit-s3-browser/src/operations/useTransfers.ts | Classify S3 errors for friendlier UI messages; extend notification duration. |
| cockpit-s3-browser/src/operations/useDownloads.ts | Classify S3 errors for friendlier UI messages; extend notification duration. |
| cockpit-s3-browser/src/lib/s3Objects.ts | Add SSE flags to various CLI invocations; extend stat parsing for encryption metadata. |
| cockpit-s3-browser/src/lib/s3Buckets.ts | Add bucket-encryption get/put/delete + backend-type detection bridge. |
| cockpit-s3-browser/src/lib/helpers.ts | Add classifyS3Error helper. |
| cockpit-s3-browser/src/lib/controlplane-types.ts | New lightweight control-plane types + backend type resolver. |
| cockpit-s3-browser/src/lib/controlplane-client.ts | New JSON-RPC bridge to control-plane backend + KMS/KES/RGW/RustFS actions. |
| cockpit-s3-browser/src/components/RustfsKmsSetup.vue | New RustFS KMS configuration UI. |
| cockpit-s3-browser/src/components/RgwVaultSetup.vue | New RGW Vault configuration UI. |
| cockpit-s3-browser/src/components/ObjectDetailsPanel.vue | Display object encryption metadata in details panel. |
| cockpit-s3-browser/src/components/Modals/EndpointConnectionModal.vue | Add TLS verify toggle, backend type, and connection default encryption fields. |
| cockpit-s3-browser/src/components/KesSetup.vue | New MinIO KES setup UI. |
| CHANGELOG.md | Version bump + summary entry. |
| .github/workflows/build-packages.yml | Adjust build branch trigger + add contents:write permission. |
| .github/agents/user-guide.agent.md | Add documentation generation agent instructions. |
| .github/agents/test-plan.agent.md | Add QA test plan agent instructions. |
| .github/agents/release.agent.md | Add release process agent instructions. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+67
to
+70
| use_tls = bool(cfg.get("useTls")) | ||
| tls_verify = cfg.get("tlsVerify", not use_tls) | ||
| if tls_verify is None: | ||
| tls_verify = not use_tls |
Comment on lines
425
to
+429
| try: | ||
| # Delete all objects under the prefix (re-list each iteration to avoid | ||
| # stale continuation tokens after deletions). | ||
| while True: | ||
| req: Dict[str, Any] = {"Bucket": bucket, "Prefix": p, "MaxKeys": 1000} | ||
| if token: | ||
| req["ContinuationToken"] = token | ||
|
|
||
| resp = client.list_objects_v2(**req) | ||
| resp = client.list_objects_v2(Bucket=bucket, Prefix=p, MaxKeys=1000) |
Comment on lines
127
to
135
| "config": { | ||
| "name": cfg.get("name"), | ||
| "endpoint": cfg.get("endpoint"), | ||
| "region": cfg.get("region"), | ||
| "accessKeyId": cfg.get("accessKeyId"), | ||
| "secretAccessKey": cfg.get("secretAccessKey"), | ||
| "useTls": bool(cfg.get("useTls")), | ||
| "tlsVerify": cfg.get("tlsVerify", True) is not False, | ||
| }, |
Comment on lines
416
to
426
| const cfg: EndpointConfig = { | ||
| name: (form.name || "").trim() || form.endpoint, | ||
| endpoint: form.endpoint, | ||
| region: (form.region || "").trim() || undefined, | ||
| accessKeyId: (form.accessKeyId || "").trim(), | ||
| useTls: form.useTls, | ||
| tlsVerify: form.tlsVerify, | ||
| secretAccessKey: form.secretAccessKey, | ||
|
|
||
| defaultSse: form.defaultSse === "none" ? undefined : form.defaultSse, | ||
| defaultSseKmsKeyId: form.defaultSse === "aws:kms" && form.defaultSseKmsKeyId ? form.defaultSseKmsKeyId : undefined, | ||
| }; |
Comment on lines
7
to
9
| const getAppVersionDefine = () => { | ||
| return `${manifest.version}-${manifest.buildVersion}${process.env.OS_PACKAGE_RELEASE ?? "built_from_source"}`; | ||
| return `${manifest.version}-${manifest.build_number ?? ""}${process.env.OS_PACKAGE_RELEASE ?? "built_from_source"}`; | ||
| }; |
| cockpit-s3-browser (1.1.1-7jammy) jammy; urgency=medium | ||
|
|
||
| * fixes | ||
| * building test package fro storage encryption |
| cockpit-s3-browser (1.1.1-7focal) focal; urgency=medium | ||
|
|
||
| * fixes | ||
| * building test package fro storage encryption |
| - test | ||
| * Thu May 14 2026 Rachit Hans <rhans@45drives.com> 1.1.1-7 | ||
| - fixes | ||
| - building test package fro storage encryption |
| - test | ||
| * Thu May 14 2026 Rachit Hans <rhans@45drives.com> 1.1.1-7 | ||
| - fixes | ||
| - building test package fro storage encryption |
- Fix URL validation in KesSetup, RustfsKmsSetup, RgwVaultSetup (manual parsing for Cockpit WebKit) - Add null guards (?? '') for vaultAddr in all setup components - Add backendType to save/load in EndpointConnectionModal (fixes defaults reverting) - Persist defaultSse, defaultSseKmsKeyId, backendType in connection-store.py - Disable AES256 option for RustFS with warning in BucketsView and EndpointConnectionModal - Add KMS error hint in useUploads.ts
…ct fallback for RGW SSE-KMS - Remove SSE-S3 (AES256) option from bucket encryption UI, only SSE-KMS - Remove skipTlsVerify/skipTlsProxy from all components and TS interfaces - Fix RgwVaultSetup: loadConfig() no longer overwrites vaultAddr when provider selected - Fix configureVault(): always re-resolve vault address from selected provider - Add CopyObject NotImplemented fallback (download+reupload) for RGW with SSE-KMS in cmd_object.py (copy and rename) and cmd_prefix.py (folder operations) - Remove overly broad MinIO KMS error message that hid real errors
Without this, the bucket encryption dialog checks the local machine's KMS config instead of the remote RustFS host where KMS is configured.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.