Problem
The uniform-multipart fix (#1 has review context) is only built as an overlay on the PG 18 CNPG operand (ghcr.io/0xsend/cnpg-postgresql:18-r2-*) and is deployed to just the two misfits clusters. Every other CNPG cluster backs up to the same Cloudflare R2 endpoint with stock barman and remains exposed to intermittent InvalidPart failures on CompleteMultipartUpload (EnterpriseDB#954).
Confirmed second victim: pqs-testnet/pqs-postgres (stock ghcr.io/cloudnative-pg/postgresql:17.2) failed its 2026-06-10 18:00 UTC scheduled backup with the exact signature:
ERROR: Upload error: An error occurred (InvalidPart) when calling the
CompleteMultipartUpload operation: One or more of the specified parts could not be found.
It self-recovered on the next run — the failures are racy and intermittent, which is why small clusters rarely trip it (few/no multipart parts) while large ones (misfits, pqs) do.
Exposure inventory (2026-06-11)
| Cluster |
Namespace |
Image |
Status |
| misfits-pg18 |
misfits-postgres-{testnet,mainnet} |
fork 18-r2-* |
✅ fixed |
| pqs-postgres |
pqs-testnet |
stock 17.2 |
❌ confirmed hit 2026-06-10 |
| bridge-pg18 |
bridge-{testnet,mainnet} |
stock 18 |
exposed |
| famo-pg18 |
featured-app-marker-orchestrator-{testnet,mainnet} |
stock 18 |
exposed |
| cos-db |
cos-{dev,prod} |
stock 17.2 |
exposed |
| metabase-postgres |
metabase-mainnet |
stock 17.2 |
exposed |
| analytics-postgres |
pqs-{testnet,mainnet} |
(image not pinned in status) |
exposed |
| scan-data-postgres |
pqs-testnet |
(image not pinned in status) |
exposed |
| pool-archive-postgres |
pool-archive-postgres-testnet |
stock 16 |
exposed |
All 14 clusters share R2 endpoint 649b42...r2.cloudflarestorage.com.
Plan
Interim safety
Backup failures are visible: CNPGBackupFailing (failure newer than last success) plus the freshness alerts CNPGBackupStale/CNPGBackupCritical (24h/72h) cover silent gaps, and backups retry on the next 6-hourly schedule.
Problem
The uniform-multipart fix (#1 has review context) is only built as an overlay on the PG 18 CNPG operand (
ghcr.io/0xsend/cnpg-postgresql:18-r2-*) and is deployed to just the two misfits clusters. Every other CNPG cluster backs up to the same Cloudflare R2 endpoint with stock barman and remains exposed to intermittentInvalidPartfailures onCompleteMultipartUpload(EnterpriseDB#954).Confirmed second victim:
pqs-testnet/pqs-postgres(stockghcr.io/cloudnative-pg/postgresql:17.2) failed its 2026-06-10 18:00 UTC scheduled backup with the exact signature:It self-recovered on the next run — the failures are racy and intermittent, which is why small clusters rarely trip it (few/no multipart parts) while large ones (misfits, pqs) do.
Exposure inventory (2026-06-11)
18-r2-*All 14 clusters share R2 endpoint
649b42...r2.cloudflarestorage.com.Plan
.github/workflows/cnpg-image.yml+ Dockerfile) by base operand tag, producing16-r2-*,17-r2-*(and keeping18-r2-*) variants from the same fork commit.ghcr-credentialsimagePullSecret + namespaceghcr-enabledlabel, same pattern as misfits — canton-monorepo #5452 / #5461).pqs-postgres(confirmed) first, thenanalytics-postgres, then the small ones opportunistically.Interim safety
Backup failures are visible:
CNPGBackupFailing(failure newer than last success) plus the freshness alertsCNPGBackupStale/CNPGBackupCritical(24h/72h) cover silent gaps, and backups retry on the next 6-hourly schedule.