feat: Operator v2#20
Open
as51340 wants to merge 8 commits into
Open
Conversation
as51340
changed the title
Replace the discarded prior attempt (preserved on archive/pre-operator-mvp) with a clean kubebuilder v4 scaffold: - MemgraphCluster API skeleton: group memgraph.com, version v1alpha1, short name mgc; spec fields land in subsequent slices - Hello-world reconciler wired through the manager - Generated CRD manifests (verified to install on a local kind cluster; kubectl get mgc resolves) - CI skeleton: lint, unit, and envtest suites run on every pull request - specs/operator-mvp/ (PRD + issue slices) carried into the new history Implements specs/operator-mvp/issues/01-repo-reset-scaffold.md
…ain auto-switching go test -cover needs the covdata tool; Go 1.26 builds it on demand, and that build fails when the go command auto-switched toolchains (base Go older than go.mod's version), breaking local runs on distro Go installs. Nothing consumes the coverage profiles yet; pass COVER_FLAGS to opt in.
go install pkg@version ignores go.mod, so under GOTOOLCHAIN=auto golangci-lint gets built with its own older minimum toolchain (go1.25) and then refuses to lint a project targeting go 1.26. Export the project's active toolchain version for the install/custom-build recipe.
Applying a minimal MemgraphCluster now provisions one StatefulSet per role (coordinators, data instances), each backed by a headless Service: - API types grow the walking-skeleton spec: coordinator and data-instance counts, image (repository/tag/pullPolicy), and the chart-compatible secrets block. All fields carry CRD schema defaults, mirrored as Go constants so builders behave on specs that never passed admission. - internal/resources holds pure builders (spec in, desired objects out). Coordinator identity (--coordinator-id, advertised --coordinator-hostname FQDN) is derived from the pod ordinal at startup via a uniform shell wrapper, so all pods within a role share one template. Flags, ports, probes, and the uid 101 / gid 103 restricted security context mirror the HA Helm chart. Storage is emptyDir for now; PVC templates land with the storage slice. - The controller server-side-applies builder output with controller owner references (CR deletion cascades to the workloads) and watches the owned StatefulSets and Services. - Builders are covered by golden-style unit tests; the controller by envtest (objects created, spec propagation, schema defaults and validation, idempotent re-reconcile). Slice 02 of specs/operator-mvp (02-provisioning-walking-skeleton.md).
* Bootstrap registration: HA client, planner, controller wiring Make a freshly provisioned cluster become an actual HA cluster without human action (specs/operator-mvp/issues/03-bootstrap-registration.md): - internal/memgraph: narrow HA client interface (show instances, add coordinator, register instance, set main) over the Bolt driver; the only place the driver is referenced, and the mock seam for tests. - internal/planner: pure diff of declared topology against observed SHOW INSTANCES output into ordered registration commands, empty when converged. SET INSTANCE TO MAIN is planned only when no MAIN exists. - internal/resources: DeclaredTopology derives per-pod registration identity (coordinator IDs, instance names, advertised FQDNs) from pod ordinals, sharing the FQDN derivation with the coordinator start script so advertised and registered addresses cannot drift. - internal/controller: once both role StatefulSets report all pods ready, find the coordinator leader (follower views redirect to the leader they report; a fresh cluster bootstraps against the first reachable coordinator), plan, and execute. All interaction is read-before-write, so operator restarts mid-bootstrap are harmless. Planner and topology derivation are covered by pure unit tests; the controller registration flow by envtest with an in-memory fake cluster that rejects duplicate registrations and second MAIN promotions. * feat: 0-based indexing for data instances
* KinD e2e harness: licensed MemgraphCluster bootstrap proof, PR-gated A true end-to-end suite for the provisioning and bootstrap slices: CI boots a multi-node Kind cluster, builds and deploys the operator image into it, applies a MemgraphCluster running real Memgraph images, and asserts every declared instance appears registered in SHOW INSTANCES with exactly one MAIN (queried through mgconsole inside a coordinator pod, the HA chart CI's established practice). Operator deployment (namespace, CRDs, controller) moves from the Manager container into BeforeSuite/AfterSuite, so scenario containers only exercise MemgraphCluster behavior and later slices (re-registration, storage retention) add test cases, not pipeline. The enterprise license flows from the MEMGRAPH_ENTERPRISE_LICENSE / MEMGRAPH_ORGANIZATION_NAME repository secrets into a Kubernetes Secret applied over stdin, so no secret material reaches logged command lines or the repo. Part of specs/operator-mvp/issues/04-kind-e2e-harness.md. * testing: Fix checking for is main * refactor: Move role main under the constants block
as51340
force-pushed
the
01-repo-reset-scaffold
branch
from
July 24, 2026 13:15
60d1422 to
9cd1163
Compare
* feat: continuous re-registration reconciliation (#24) Extend registration reconciliation from one-shot bootstrap to continuous drift recovery: a converged cluster now reschedules a periodic resync so a registration a pod loses (rescheduled onto a fresh node, wiped storage) is re-issued without human action. A lost registration on a still-running pod produces no watch event, so this resync is the mechanism that detects drift. The planner already computed full-diff semantics; the gap was the controller returning no requeue once converged. Add a resyncInterval and requeue on it from the converged branch. Tests: - planner: multiple lost registrations re-issued without a second MAIN - controller envtest: re-register a lost data instance (MAIN untouched), re-add a lost coordinator, no-op across repeated resyncs - e2e: wipe a data instance's registration on the leader, assert the operator converges the cluster back to fully registered * testing: Add test for removal of coordinators
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements
specs/operator-mvp/issues/01-repo-reset-scaffold.md(parent:specs/operator-mvp/PRD.md).What this does
Replaces the discarded prior operator attempt with a fresh kubebuilder v4 scaffold and a green CI skeleton, so every subsequent MVP slice lands PR-gated.
archive/pre-operator-mvp(already pushed).MemgraphCluster, groupmemgraph.com, versionv1alpha1, short namemgc. Spec is intentionally empty — topology/image/storage fields arrive in later slices.Lint(golangci-lint) andTests(unit + envtest jobs) workflows run on every pull request. The e2e workflow scaffold was dropped — the KinD e2e harness is issue 04.specs/operator-mvp/(PRD + 12 issue slices) committed into the new history.Verification (local)
make test-unit— greenmake test(envtest, real kube-apiserver 1.36.2, placeholder reconciler exercised) — greenmake lint/make lint-config— 0 issuesmake installon a throwaway KinD cluster: CRD installs cleanly,kubectl get mgcresolves, sample CR appliesAcceptance criteria
archive/-prefixed branchmainholds a fresh kubebuilder scaffold with kindMemgraphCluster, groupmemgraph.com, versionv1alpha1, short namemgc(upon merge)specs/operator-mvp/(PRD + issues) committed as part of the new historykubectl get mgcresolves