feat(registry): add storage backend registry (KPEP-0001 phase 1)#2
Open
zachsmith1 wants to merge 6 commits into
Open
feat(registry): add storage backend registry (KPEP-0001 phase 1)#2zachsmith1 wants to merge 6 commits into
zachsmith1 wants to merge 6 commits into
Conversation
Adds the registry/ subpackage hosting the Backend interface, Factory
type, and Backends instance-scoped registry. This is the first
implementation step for KPEP-0001 (pluggable storage backends).
The package is intentionally additive — nothing in the rest of
kplane-dev/storage imports it, so existing consumers (the apiserver's
StorageWithClusterIdentity decorator path) continue to work unchanged.
The wiring lands in later PRs:
- kplane-dev/spanner adds a Register() that satisfies registry.Backend.
- kplane-dev/apiserver constructs *Backends in main and dispatches via
--storage-backend (alongside the existing hardcoded if/else for
one release, then the hardcoded branch is removed).
The Factory signature matches upstream storagebackend/factory.Create
exactly so existing backend implementations (the BackendFactory type
that kplane-dev/spanner already exposes) plug in without adaptation.
Tested: go test ./registry/... covers Register/Get/Names/AddFlags
fan-out + duplicate-panic behavior. Existing tests (e2e_test.go,
identity_test.go, keylayout_test.go) unaffected since nothing in those
paths references the new package.
See KPEP-0001 in kplane-dev/enhancements for the design rationale.
3 tasks
Adds the seam the apiserver's RESTOptionsDecorator needs to install a registry-selected backend in place of the upstream etcd3 path. When DecoratorConfig.BackendFactory is non-nil, StorageWithClusterIdentity calls it instead of generic.NewRawStorage; the cacher wrapping above is unchanged. The BackendFactory type is declared at the top level (mirror of registry.Factory) so consumers of DecoratorConfig don't need a transitive import of the registry subpackage. Signature is 1-to-1 with upstream factory.Create — any backend already at that shape (etcd3, Spanner, future postgres) plugs in without adaptation. Nil preserves the pre-registry behavior so callers that haven't switched yet continue to hit etcd3 unchanged.
3 tasks
That fork branch is the consolidated source of two prerequisite features
for KPEP-0001:
- 32f5e9075db: move DecodeCallback to storage package (lets non-etcd
backends like Spanner honor cacher-installed decode callbacks).
- 8744b93de42: per-cluster service allocator support (apiserver's
multi-cluster bootstrap needs this when it threads BackendFactory
through DecoratorConfig).
Both consumers downstream of this PR (kplane-dev/spanner and
kplane-dev/apiserver) pin against the same fork commit so the chain
agrees on which symbols exist. When feat/per-cluster-allocators
eventually merges, every consumer moves to the resulting main SHA in a
follow-up bump.
…P-0001)
Brings the Spanner backend into kplane-dev/storage as the first in-tree
implementation, matching the KPEP-0001 end-state layout:
kplane-dev/storage/
registry/ (added in previous commits)
decorator.go BackendFactory = registry.Factory alias
backends/
register.go RegisterBuiltin(b) aggregator
spanner/
broadcast.go, config.go,
factory.go, register.go,
store.go, watcher.go,
store_test.go, register_test.go
What this collapses:
- Drops the standalone kplane-dev/spanner repo entirely. It existed only
because BackendFactory needed to be declared somewhere without
importing kplane-dev/storage (the 'avoid an import cycle' comment in
the old factory.go). Now that Spanner is a subpackage of
kplane-dev/storage, it references storage.BackendFactory directly.
- BackendFactory is now a type alias to registry.Factory, so the
apiserver can pass registry.Backend.Build()'s return value into
DecoratorConfig without a conversion.
- The aggregator (backends/register.go) is one import + one Register()
call per backend. Adding postgres or kine in the future means editing
one file here, not the apiserver.
Tested:
- go test ./registry/... ./backends/... clean.
- Spanner emulator-backed tests pass.
- decorator.go alias compiles cleanly; the BackendFactory hook path in
StorageWithClusterIdentity is unchanged.
Follow-up: archive kplane-dev/spanner (or leave as a one-release re-export
shim). kplane-dev/spanner#1 will be closed since its work is now here.
See KPEP-0001 in kplane-dev/enhancements for the design.
….0 requirement) Auto-bumped by go mod tidy after pulling in Spanner deps from the backends/spanner migration. Local dev needs Go 1.25.8 (or auto-toolchain download). CI bumps will follow.
Wires EnsureSchema into Options.Build() so a fresh Spanner backend doesn't require an out-of-band schema-apply step. EnsureSchema is now idempotent (gRPC AlreadyExists is treated as success) so the call is safe on every apiserver startup, not just first-run. Adds a 30s timeout context so a misconfigured emulator endpoint can't hang apiserver startup indefinitely. Operator UX is now: ./kplane-apiserver --storage-backend=spanner --spanner-* ... instead of: go run ./cmd/spanner-schema --... # one-time ./kplane-apiserver --storage-backend=spanner --spanner-* ... KPEP-0001 local e2e recipe in kplane-dev/apiserver no longer needs the 'apply the schema manually' caveat — Build() handles it.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
First implementation step for KPEP-0001 (pluggable storage backends). Adds the `registry/` subpackage hosting the `Backend` interface, `Factory` type, and `Backends` instance-scoped registry.
The package is purely additive — nothing else in `kplane-dev/storage` imports it. Existing consumers (the apiserver's `StorageWithClusterIdentity` decorator path) continue to work unchanged.
What's in the package
What lands in later PRs
This shape lets us test locally with `go.mod` `replace` directives without forcing a coordinated merge across three repos.
Test plan
`go.mod` change is just promoting `github.com/spf13/pflag` from indirect to direct.