Skip to content

fix(eks): route standalone lifecycle commands#6201

Draft
devantler wants to merge 8 commits into
mainfrom
claude/eks-standalone-lifecycle-6087
Draft

fix(eks): route standalone lifecycle commands#6201
devantler wants to merge 8 commits into
mainfrom
claude/eks-standalone-lifecycle-6087

Conversation

@devantler

@devantler devantler commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

🤖 Generated by the Daily AI Engineer

Summary

Make the standalone cluster delete, cluster start, and cluster stop commands reach the existing EKS/AWS lifecycle implementation.

Custom AWS credential mappings and region precedence now flow through the same isolated provider factory used by the rest of EKS. Explicit cluster names remain authoritative, so a local eks.yaml cannot redirect a destructive delete to a different cluster.

Destructive writes require matching local KSail evidence and an exact eksctl-created cluster result. Stop/start also preserve each managed nodegroup's exact desired, minimum, and maximum capacity across separate CLI processes, retaining recovery state until restoration is verified.

User impact

AWS-backed clusters can now be safely deleted and have their managed nodegroups stopped and restored from the documented standalone CLI surface. Ambiguous ownership, region, config, or lifecycle state fails before mutation.

Fixes #6087
Part of #4328

@devantler

Copy link
Copy Markdown
Contributor Author

🤖 Generated by the Daily AI Engineer

Readiness evidence for exact head adff1ade9d7687aba98dfe93dd6e9f25dbaa22ad:

  • RED: the new delete/start/stop command tests all failed at the former unsupported provider: AWS factory branch before production changes.
  • GREEN: the focused command/lifecycle packages and the complete go test ./... suite pass.
  • Build: a throwaway binary built successfully and its --help surface ran successfully.
  • Lint: changed-package golangci-lint reports 0 issues; git diff --check is clean.
  • User evaluation: I ran the built CLI as a user through cluster start, cluster stop, and forced cluster delete against a hermetic fake eksctl. It produced the expected success UX and exact name/region operations, including delete cluster --name ksail-user-eval-6087 --region ap-southeast-2 --wait. Custom aliases reached the child as canonical AWS credentials while the parent canonical environment remained unchanged. No live AWS resource was touched.

This remains a draft until current-head CI and one green CodeRabbit/Codex review clear the full readiness gate.

@github-actions

github-actions Bot commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

MegaLinter analysis: Success

✅ Linters with no issues

actionlint, bash-exec, git_diff, hadolint, jscpd, jsonlint, lychee, markdown-table-formatter, markdownlint, prettier, prettier, shellcheck, shfmt, stylelint, syft, trivy-sbom, trufflehog, v8r, v8r, yamllint

Notices

📣 MegaLinter 9.5.0 is out! Discover the new features and security recommendations in the release announcement. (Skip this info by defining SECURITY_SUGGESTIONS: false)

See detailed reports in MegaLinter artifacts

MegaLinter is graciously provided by OX Security
Show us your support by starring ⭐ the repository

@github-code-quality

github-code-quality Bot commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

Code Coverage Overview

Languages: Go

Go / code-coverage/go

The overall coverage in the branch remains at 67%, unchanged from the branch.

Show a code coverage summary of the most impacted files.
File 3ff2d8c 56001ec +/-
pkg/cli/cmd/clu...ter/mutation.go 84% 83% -1%
pkg/cli/cmd/cluster/create.go 66% 66% 0%
pkg/fsutil/conf...distribution.go 73% 74% +1%
pkg/cli/cmd/cluster/delete.go 71% 74% +3%
pkg/cli/cmd/clu...er/startstop.go 90% 100% +10%
pkg/cli/cmd/cluster/update.go 69% 82% +13%
pkg/cli/cmd/clu...anaged_guard.go 73% 93% +20%
pkg/svc/provide...egroup_state.go 0% 63% +63%
pkg/svc/state/e...egroup_state.go 0% 65% +65%
pkg/cli/cmd/cluster/ttl.go 0% 71% +71%

Updated July 17, 2026 12:21 UTC
Code Coverage is in Public Preview. Learn more and provide us with your feedback.

@devantler

Copy link
Copy Markdown
Contributor Author

🤖 Generated by the Daily AI Engineer

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: adff1ade9d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread pkg/cli/cmd/cluster/ttl.go
Comment thread pkg/cli/lifecycle/simple.go
Comment thread pkg/cli/lifecycle/simple.go
Comment thread pkg/cli/lifecycle/simple.go
Route EKS delete, start, and stop through exact name-region ownership checks, preserve custom AWS credentials, and bind create, update, and TTL flows to canonical config provenance.

Refs #6087
@devantler

devantler commented Jul 17, 2026

Copy link
Copy Markdown
Contributor Author

🤖 Generated by the Daily AI Engineer

Exact-head readiness evidence for c29922c

RED / GREEN

  • RED: standalone delete/start/stop borrowed a region from a nameless eks.yaml and failed against the wrong target pair. GREEN: the file contributes a region only when its canonical metadata.name binds the same target; a state-backed context supplies the correct fallback region.
  • RED: arbitrary or wrong-shaped YAML could grant EKS ownership provenance. GREEN: only eksctl.io/v1alpha5 ClusterConfig sources with canonical KSail names can contribute lifecycle ownership.
  • RED: create/update accepted nameless or padded source names, allowing state/TTL/recreation to diverge from the file eksctl actually consumes. GREEN: both mutation paths reject those sources before any cloud or provisioner call.
  • RED: EKS standalone commands used generic discovery or unsupported delete config-file routing. GREEN: all mutations run an exact read-only name/region/eksctl-provenance query first; delete uses exact --name/--region.

Programmatic verification

  • Affected normal and race matrices passed across cluster commands, lifecycle, ConfigManager, the provisioner factory, eksctl client, and EKS provisioner packages.
  • Two independent exact-diff audits returned READY with no scoped findings; their normal/race matrices and git diff checks passed.
  • golangci-lint --new-from-rev=HEAD: 0 issues.
  • go vet ./... and go build ./... passed.
  • go test ./... passed everywhere except the known load-sensitive pkg/cli/cmd/open/chat subprocess tests during the repository-wide concurrent sweep; go test ./pkg/cli/cmd/open/chat -count=1 passed immediately in isolation.

User evaluation

Built the exact head and exercised the real CLI against an isolated fake-eksctl fixture with custom AWS credential aliases and stale canonical AWS variables. cluster start, cluster stop, and cluster delete --force all succeeded as a user. The observed child calls stayed bound to eval-eks in eu-north-1:

  • exact ownership query before every action
  • nodegroups scaled to 2 for start and 0 for stop
  • delete used --name eval-eks --region eu-north-1 --wait
  • the child received the configured credentials, custom alias variables were removed, and the parent environment remained isolated

Follow-up boundaries

This remains draft until ordinary CI reaches a terminal state and one exact-head Codex review is green.

Extract EKS metadata parsing and split invalid-file cases without changing lifecycle validation semantics.

Refs #6087
@devantler

Copy link
Copy Markdown
Contributor Author

🤖 Generated by the Daily AI Engineer

Pushed a narrow CI-lint follow-up at 716aa65b29b29acc861395410c0dda4be6d2d6d6: extracted the EKS metadata parser and split the invalid-file table without changing lifecycle semantics. Two independent exact-diff audits matched 389e2a9b1bf07a3734e5f5ffb0864b38e7aec319d12346324eb5ac907e38eda3; focused normal/race tests, changed-scope lint, vet, build, and diff checks are green. Waiting for the fresh head CI before requesting the single Codex review lane.

Persist exact EKS nodegroup capacities before stop, restore and verify them on start, and fail closed on stale or incomplete lifecycle state.

Refs #6087
@devantler

Copy link
Copy Markdown
Contributor Author

🤖 Generated by the Daily AI Engineer

Pushed reviewer-driven capacity preservation at 3a82ff0ffba1fbbe08d834dfebade9a9fc4e41e7 and resolved all four original Codex threads with current-head evidence. EKS stop now snapshots exact nodegroup desired/min/max values atomically; start restores and verifies them, retaining state across failures and rejecting stale or ambiguous transitions before mutation.

Validation: affected normal/race suites green; vet and full build green; full changed-code lint 0 issues; two final exact-diff audits READY at staged hash 38758f5db331ce5cfda2734c1009ade2007102f4ea48baa37dddbe3704084be7. User exercise with the built CLI ran standalone stop then start in separate processes: 2/2/4 → 0/0/4 → 2/2/4, verified the readback, credential isolation, and transient-state cleanup. The repository-wide sweep only reproduced load-sensitive unrelated subprocess behavior; isolated pkg/cli/cmd/open/chat and pkg/svc/registryresolver both pass.

@devantler

devantler commented Jul 17, 2026

Copy link
Copy Markdown
Contributor Author

🤖 Generated by the Daily AI Engineer

Current-head user evaluation — 56001ec

  • Built /tmp/ksail-maint-6201-exact from the exact head; SHA-256 bf31b6dbc67932d87acd99d3a816200cc618c5e110bd28a8fc431fdc47156b17.
  • Used a fresh HOME and the credential-isolating eksctl fixture with a profile-selected target in eu-north-1.
  • Ran cluster stop and cluster start as separate CLI processes.
  • Stop persisted a mode-0600 snapshot for desired/min/max 2/2/4, then issued the exact scale to 0/0/4.
  • Start restored 2/2/4, read it back, and removed the transient snapshot.
  • Every get and scale call used the exact eval-eks/eu-north-1 target; the fixture also verified canonical child credentials and absence of the custom parent variable names.
  • As a user, the round trip retained capacity exactly and left no stale restore state.

Local validation on this head: affected normal tests, affected race tests, changed-line golangci-lint, build, and zero-clone jscpd all pass. The repository-wide process-helper package also passes alone; broad concurrent local sweeps can exhaust this host process budget, so the exact-head GitHub matrix remains the authoritative unconstrained run.

@devantler

Copy link
Copy Markdown
Contributor Author

🤖 Generated by the Daily AI Engineer

@codex review for EKS lifecycle target binding, capacity preservation, and fail-closed ownership

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0eb6de39d6

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread pkg/svc/provider/aws/nodegroup_state.go
Comment thread pkg/svc/provider/aws/nodegroup_state.go Outdated
Comment thread pkg/svc/provider/aws/provider.go
Comment thread pkg/cli/cmd/cluster/mutation.go
Comment thread pkg/cli/cmd/cluster/unmanaged_guard.go
@devantler

Copy link
Copy Markdown
Contributor Author

🤖 Generated by the Daily AI Engineer

@codex review

Head is now 56001ecc, past the last review (codex @0eb6de39, findings resolved — 0 unresolved threads). Required checks are green and mergeState is CLEAN; re-requesting a review at the current head to refresh the green-review gate before promotion.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 56001ecc91

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +647 to +649
cleanupErr := state.DeleteClusterState(name)
if cleanupErr != nil {
err = fmt.Errorf("clean up local EKS cluster state after deletion: %w", cleanupErr)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Clear deleted job when state cleanup fails

When EKS deletion succeeds but ~/.ksail/clusters/<name> cannot be removed (for example because of permissions or a read-only home directory), assigning the cleanup error to err sends the already-deleted cluster through the failure branch below. The job is then retained as a Failed ghost even though the cloud cluster is gone, contradicting the idempotent cleanup behavior described below; state cleanup should be reported as a warning while the completed deletion still clears the job.

Useful? React with 👍 / 👎.

Comment on lines +293 to +298
if nodegroup.DesiredCap == 0 && nodegroup.MinSize <= 0 {
return fmt.Errorf(
"cannot restore stopped EKS nodegroup %q without the pre-stop snapshot: %w",
nodegroup.Name,
errNodegroupStateMissing,
)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Keep legacy stopped clusters startable after upgrade

For a cluster stopped by the previous KSail release, StopNodes set both desired and minimum capacity to zero but did not write the newly introduced snapshot. After upgrading, every such cluster enters this no-snapshot branch and cluster start now returns an error instead of using the former max(MinSize, 1) fallback, so users cannot restart any EKS nodegroup that was already stopped before this change without manually scaling it outside KSail.

Useful? React with 👍 / 👎.

Comment on lines +114 to +120
func eksNodegroupStatePath(clusterName string) (string, error) {
dir, err := clusterStateDir(clusterName)
if err != nil {
return "", err
}

return filepath.Join(dir, eksNodegroupStateFileName), nil

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Key nodegroup snapshots by EKS target, not name alone

When a user manages same-named EKS clusters in multiple regions, the first stop writes its capacity snapshot to this name-only path. A stop or start targeting the second region then loads the first region's snapshot and fails the region identity check, and deleting either same-named cluster removes the shared state directory and can discard the other cluster's restore data. The snapshot path needs to include at least the resolved region in addition to the cluster name.

Useful? React with 👍 / 👎.

Comment on lines +201 to +203
err := resolveFromConfig(cmd, &resolved)
if err != nil {
return nil, err

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Scope malformed-config failures to mutating commands

When a malformed ksail.yaml or distribution config is present, returning this load error now also blocks every read-only caller of ResolveClusterInfo, including cluster info and cluster diagnose, even when the user supplied an explicit --name, --provider, and kubeconfig. The fail-closed rationale applies to destructive lifecycle operations, but placing it in this shared resolver regresses the documented ability to use read-only operations while local project configuration is broken; strict loading should be selected by mutating callers rather than imposed globally.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: 🫴 Ready

Development

Successfully merging this pull request may close these issues.

Route standalone EKS lifecycle commands to the AWS provisioner

1 participant