diff --git a/CHANGELOG.md b/CHANGELOG.md index c007c46..66f5334 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -11,6 +11,60 @@ keeps reading that version for at least 24 months after a successor lands. ## [Unreleased] +## [1.0.11] — 2026-07-16 + +Twelve operator-inconvenience fixes found by exercising the CLI surface, +each covered by a regression test. + +### Fix: false alarms and silent wrong-target + +- `repo scrub` reported 100% chunk corruption (exit 9) on every + ENCRYPTED repository — the default posture after `init` — because it + built a CAS with no decryptor. It now scrubs manifest-aware (the same + per-manifest CAS `repair scrub` uses), so encrypted chunks decrypt + and verify. Scheduled scrubs no longer page on every run. +- The global `-c`/`--config` flag was advertised everywhere but read + nowhere; the tool always loaded the XDG/FHS default. It is now honored + for both reads and write-back, so `-c staging.yaml` operates on that + file. +- `lint` always returned `{"status":"valid"}` without reading anything. + It now validates the resolved config with the real loader (strict + KnownFields + validation) and fails, with the reason, on a broken one. + +### Fix: dry-run / advisory tools no longer give false confidence + +- `recovery windows` advertised a PITR range straight across a WAL + archive hole; it now caps `latest_restore_lsn` at the first hole and + records the gap. +- `restore --preview` reported "Pre-flight: ✓ ready" for a target past a + WAL hole that the real restore warns will HALT recovery; preview now + surfaces the same `wal_archive_hole` finding. +- `capacity report` extrapolated a seconds-long sampling window into + absurd per-day growth labeled "medium confidence"; confidence now + requires a real observation window (≥1 day for medium, ≥1 week for + high) and a sub-day window carries an explicit caveat. +- `rotate` stamped legally-HELD backups `[del ]` in its per-backup + listing while the summary said `held: N (excluded from delete)`; held + backups now render `[held]`. + +### Fix: wrong error class, hollow stubs, muscle memory + +- `recovery readiness` printed the RTO throughput as a nonsensical + duration (`46603h22m40s`) instead of a byte-rate (`160.0 MiB/s`). +- `--incremental-from` against a PostgreSQL < 17 server was reported as + the generic `internal` (file-a-bug) code; it is now the structured + `backup.incremental_unsupported` usage error with a hint. +- `repo init` accepts the repository URL via `--repo` (matching every + other `repo` verb), not only as a bare positional. +- `explain ` now returns the command's real summary, usage, and + description instead of echoing the argument back. +- `glossary ` now returns the term's definition (an unknown term + is `notfound.term`) instead of dropping the description. + +Also: the renderer integer-fidelity fix (YAML/CSV scientific-notation) +was extended to a shared `jsonshape` helper covering tap/junit/pdf/ +template. + ## [1.0.10] — 2026-07-15 ### Fix: `recovery drill` failed every WAL-streaming backup (#26) diff --git a/docs/compliance/pci-dss.md b/docs/compliance/pci-dss.md index 3ffb064..cf01128 100644 --- a/docs/compliance/pci-dss.md +++ b/docs/compliance/pci-dss.md @@ -143,7 +143,7 @@ audit period plus the cosign signatures on the running binary: ```sh -VERSION=1.0.10 # the release / image tag you're attesting +VERSION=1.0.11 # the release / image tag you're attesting # 1. Audit chain bundle pg_hardstorage audit export-bundle \ diff --git a/docs/compliance/slsa-l3-provenance.md b/docs/compliance/slsa-l3-provenance.md index 6191af7..283c3c2 100644 --- a/docs/compliance/slsa-l3-provenance.md +++ b/docs/compliance/slsa-l3-provenance.md @@ -74,7 +74,7 @@ Verify a downloaded artifact against its provenance with the `slsa-verifier`: ```sh -VERSION=1.0.10 # the release you're verifying +VERSION=1.0.11 # the release you're verifying slsa-verifier verify-artifact \ --provenance-path pg_hardstorage.intoto.jsonl \ --source-uri github.com/cybertec-postgresql/pg_hardstorage \ @@ -109,7 +109,7 @@ When images ship, they are intended to carry a cosign signature plus an image-level SLSA provenance attestation: ```sh -VERSION=1.0.10 # the image tag you're verifying +VERSION=1.0.11 # the image tag you're verifying # Verify the image signature cosign verify "ghcr.io/cybertec-postgresql/pg_hardstorage:v${VERSION}" \ @@ -139,7 +139,7 @@ JSON format, generated by `syft` (configured via the and attached to the GitHub Release: ```sh -VERSION=1.0.10 # the release you're inspecting +VERSION=1.0.11 # the release you're inspecting # The SBOM is a plain file on the Release; inspect it directly jq -r '.packages[].name' \ "pg_hardstorage_${VERSION}_linux_amd64.tar.gz.spdx.sbom.json" diff --git a/docs/reference/cli/pg_hardstorage_repo_init.md b/docs/reference/cli/pg_hardstorage_repo_init.md index 10cf900..a8bfee3 100644 --- a/docs/reference/cli/pg_hardstorage_repo_init.md +++ b/docs/reference/cli/pg_hardstorage_repo_init.md @@ -70,6 +70,7 @@ pg_hardstorage repo init [flags] ``` --compression string zstd encoder level: fast | balanced | max (default: balanced) -h, --help help for init + --repo string repository URL to create (alternative to the positional ) --worm-mode string WORM retention mode: compliance | governance (set with --worm-retention) --worm-retention string WORM retention duration: e.g. 7y, 30d, 8760h (required with --worm-mode) ``` diff --git a/docs/reference/runbooks/control-plane-setup.md b/docs/reference/runbooks/control-plane-setup.md index 8326b0a..d484e55 100644 --- a/docs/reference/runbooks/control-plane-setup.md +++ b/docs/reference/runbooks/control-plane-setup.md @@ -249,7 +249,7 @@ curl --cacert /etc/pg_hardstorage/server/cert.pem \ # { # "id": "db1.example.com", # "host": "db1.example.com", -# "version": "v1.0.10", +# "version": "v1.0.11", # "deployments": ["db1", "db2"], # "registered_at": "...", # "last_heartbeat": "..." diff --git a/docs/tutorials/getting-started.md b/docs/tutorials/getting-started.md index 0f4db32..08ace95 100644 --- a/docs/tutorials/getting-started.md +++ b/docs/tutorials/getting-started.md @@ -35,7 +35,7 @@ tarballs (Windows is CLI-only). Grab the matching one from verify the cosign signature, and drop the binary on your `$PATH`: ```sh -VERSION=1.0.10 # latest release: https://github.com/cybertec-postgresql/pg_hardstorage/releases/latest +VERSION=1.0.11 # latest release: https://github.com/cybertec-postgresql/pg_hardstorage/releases/latest curl -LO "https://github.com/cybertec-postgresql/pg_hardstorage/releases/download/v${VERSION}/pg_hardstorage_${VERSION}_linux_amd64.tar.gz" tar xzf "pg_hardstorage_${VERSION}_linux_amd64.tar.gz" sudo install -m 0755 pg_hardstorage /usr/local/bin/ @@ -45,7 +45,7 @@ pg_hardstorage version ### `.deb` (Debian / Ubuntu) ```sh -VERSION=1.0.10 # the release you downloaded +VERSION=1.0.11 # the release you downloaded sudo dpkg -i "pg-hardstorage_${VERSION}_amd64.deb" ``` @@ -57,7 +57,7 @@ creates `/etc/pg_hardstorage/`, `/var/lib/pg_hardstorage/`, ### `.rpm` (Fedora / RHEL / Rocky / Alma) ```sh -VERSION=1.0.10 # the release you downloaded +VERSION=1.0.11 # the release you downloaded sudo rpm -i "pg-hardstorage-${VERSION}-1.x86_64.rpm" ``` @@ -259,7 +259,7 @@ you are doing — exit 9 means the verifier said no. ```sh $ pg_hardstorage version -pg_hardstorage v1.0.10 (abc1234, built 2026-04-29T12:00:00Z) +pg_hardstorage v1.0.11 (abc1234, built 2026-04-29T12:00:00Z) ``` `doctor` is the single-command "is anything wrong" check. It prints a diff --git a/internal/backup/runner/runner.go b/internal/backup/runner/runner.go index 4542016..4c41599 100644 --- a/internal/backup/runner/runner.go +++ b/internal/backup/runner/runner.go @@ -605,9 +605,16 @@ func Take(ctx context.Context, opts TakeOptions) (*Result, error) { // can chain-walk correctly. backupType := backup.BackupTypeFull if opts.Incremental != nil && len(opts.Incremental.ParentPGManifest) > 0 { - // Refuse incremental against pre-PG-17 servers up front. + // Refuse incremental against pre-PG-17 servers up front. This + // is an environment precondition (the operator's server is too + // old), NOT a tool bug — classify it as a structured usage-class + // error, not the generic "internal" (file-a-bug) code. if pgVersion.Major < 17 { - return nil, fmt.Errorf("backup: incremental backups require PG 17+; source is PG %d", pgVersion.Major) + return nil, output.NewError("backup.incremental_unsupported", + fmt.Sprintf("backup: incremental backups require PostgreSQL 17+; source is PG %d", pgVersion.Major)). + WithSuggestion(&output.Suggestion{ + Human: "upgrade the source server to PostgreSQL 17+ (and enable summarize_wal), or take a full backup by omitting --incremental-from", + }).Wrap(output.ErrUsage) } backupType = backup.BackupTypeIncremental } diff --git a/internal/capacity/capacity.go b/internal/capacity/capacity.go index 34637e5..aab8c9f 100644 --- a/internal/capacity/capacity.go +++ b/internal/capacity/capacity.go @@ -185,7 +185,12 @@ func Project(ctx context.Context, sp storage.StoragePlugin, repoURL string, opts r.SamplesUsed = len(allSamples) r.ProjectedDeltaBytes = int64(float64(r.BytesPerDay) * horizon.Hours() / 24) r.ProjectedBytes = r.CurrentBytes + r.ProjectedDeltaBytes - r.Confidence = confidenceFor(r.RSquared, len(allSamples)) + windowSeconds := sampleWindowSeconds(allSamples) + r.Confidence = confidenceFor(r.RSquared, len(allSamples), windowSeconds) + if windowSeconds < 86400 { + r.Note = fmt.Sprintf("samples span only %s; a per-day projection from a sub-day window is unreliable — collect backups over several days before trusting the growth rate", + time.Duration(windowSeconds*float64(time.Second)).Round(time.Second)) + } } else { r.SamplesUsed = len(allSamples) r.Confidence = "insufficient" @@ -203,20 +208,49 @@ func sumDeploymentCurrent(deps []DeploymentProjection) int64 { return s } -// confidenceFor maps (R², sample count) to a human-friendly bucket. -// R² > 0.85 with ≥10 samples = high; > 0.7 with ≥5 = medium; otherwise -// low. Below 3 samples = insufficient (handled upstream). -func confidenceFor(r2 float64, n int) string { +// confidenceFor maps (R², sample count, observation window) to a +// human-friendly bucket. The observation WINDOW is the load-bearing +// guard: a per-day projection extrapolated from samples spanning only +// seconds/minutes is meaningless no matter how tight the R² — e.g. five +// backups taken within a minute yielded "85 GiB/day, 7.5 TiB projected, +// confidence: medium". A projection can only be high/medium if the +// samples actually span a meaningful fraction of the extrapolation unit. +func confidenceFor(r2 float64, n int, windowSeconds float64) string { + const ( + day = 86400.0 + week = 7 * day + ) switch { - case r2 >= 0.85 && n >= 10: + case r2 >= 0.85 && n >= 10 && windowSeconds >= week: return "high" - case r2 >= 0.70 && n >= 5: + case r2 >= 0.70 && n >= 5 && windowSeconds >= day: return "medium" default: + // Under a day of observation the per-day slope is not + // trustworthy; cap at "low" regardless of fit. return "low" } } +// sampleWindowSeconds returns the wall-clock span covered by the +// samples (last timestamp minus first). Samples are assumed sorted or +// are min/max-scanned here. +func sampleWindowSeconds(samples []sample) float64 { + if len(samples) < 2 { + return 0 + } + min, max := samples[0].at, samples[0].at + for _, s := range samples[1:] { + if s.at.Before(min) { + min = s.at + } + if s.at.After(max) { + max = s.at + } + } + return max.Sub(min).Seconds() +} + // sample is one (timestamp, cumulative-bytes) point. type sample struct { at time.Time diff --git a/internal/capacity/window_guard_test.go b/internal/capacity/window_guard_test.go new file mode 100644 index 0000000..897ce92 --- /dev/null +++ b/internal/capacity/window_guard_test.go @@ -0,0 +1,22 @@ +package capacity + +import "testing" + +// Regression: a sub-day observation window must never yield medium/high +// confidence, no matter how tight R² — a per-day projection from +// seconds of data is meaningless (five backups in a minute previously +// gave "medium"). +func TestConfidenceFor_ShortWindowCappedLow(t *testing.T) { + // Perfect fit, plenty of samples, but only 90 seconds of window. + if got := confidenceFor(0.99, 20, 90); got != "low" { + t.Errorf("confidenceFor(tight fit, 90s window) = %q, want low", got) + } + // A week of samples with a good fit earns high. + if got := confidenceFor(0.90, 12, 8*86400); got != "high" { + t.Errorf("confidenceFor(week window) = %q, want high", got) + } + // A couple of days with a decent fit earns medium. + if got := confidenceFor(0.75, 6, 3*86400); got != "medium" { + t.Errorf("confidenceFor(3-day window) = %q, want medium", got) + } +} diff --git a/internal/cli/configio.go b/internal/cli/configio.go index 2107a37..8085b29 100644 --- a/internal/cli/configio.go +++ b/internal/cli/configio.go @@ -5,6 +5,9 @@ import ( "fmt" "os" "path/filepath" + "strings" + + "github.com/spf13/cobra" "github.com/cybertec-postgresql/pg_hardstorage/internal/config" "github.com/cybertec-postgresql/pg_hardstorage/internal/fsutil" @@ -16,7 +19,27 @@ import ( // Same lookup as the rest of the CLI: paths.Resolve handles the // XDG / FHS / env / --config precedence; we just append the // well-known filename here. +// captureConfigFlag forwards the global -c/--config flag to the paths +// layer via PG_HARDSTORAGE_CONFIG_FILE, so every config.Load call in +// this process (all 40-odd, plus write-back) honours it uniformly. +// Must run BEFORE installDispatcher / resolveDeploymentDefaults, which +// both load config. The flag was previously registered but read +// nowhere — a silent no-op. +func captureConfigFlag(cmd *cobra.Command, _ []string) error { + cfgFile, _ := cmd.Flags().GetString("config") + if strings.TrimSpace(cfgFile) != "" { + _ = os.Setenv("PG_HARDSTORAGE_CONFIG_FILE", cfgFile) + } + return nil +} + func configFilePath(p *paths.Paths) string { + // Honour an explicit -c/--config file for write-back too, so + // `deployment edit -c staging.yaml` mutates staging.yaml rather + // than silently rewriting the XDG/FHS default. + if override := p.ConfigFileOverride; override != "" { + return override + } return filepath.Join(p.Config.Value, "pg_hardstorage.yaml") } diff --git a/internal/cli/onboard_helpers.go b/internal/cli/onboard_helpers.go index 1ca37ea..1a7897a 100644 --- a/internal/cli/onboard_helpers.go +++ b/internal/cli/onboard_helpers.go @@ -1,36 +1,116 @@ -// onboard_helpers.go — 'lint' + 'explain' stub commands surfaced during onboarding. +// onboard_helpers.go — 'lint' + 'explain' + 'glossary' commands surfaced during onboarding. package cli import ( + "fmt" + "io" + "sort" + "strings" + + "github.com/spf13/cobra" + + "github.com/cybertec-postgresql/pg_hardstorage/internal/config" "github.com/cybertec-postgresql/pg_hardstorage/internal/output" + "github.com/cybertec-postgresql/pg_hardstorage/internal/paths" "github.com/cybertec-postgresql/pg_hardstorage/internal/version" - "github.com/spf13/cobra" ) +// newLintCmdImpl validates the resolved pg_hardstorage.yaml (honouring +// -c/--config) using the SAME loader real commands use — strict +// KnownFields + validate() — instead of the previous stub that always +// returned {"status":"valid"} without reading anything. func newLintCmdImpl() *cobra.Command { return &cobra.Command{ Use: "lint", Short: "Validate pg_hardstorage.yaml", Args: cobra.NoArgs, SilenceUsage: true, RunE: func(cmd *cobra.Command, _ []string) error { d := DispatcherFrom(cmd) - return d.Result(output.NewResult(cmd.CommandPath()).WithBody(map[string]any{"status": "valid"})) + p, err := paths.Resolve(paths.DefaultOptions()) + if err != nil { + return output.NewError("internal", err.Error()).Wrap(err) + } + loaded, err := config.Load(p) + if err != nil { + // A parse/validation failure is the whole point of lint: + // report it as invalid with the reason, and exit non-zero + // so CI/pre-flight scripts catch a broken config. + return output.NewError("config.invalid", + fmt.Sprintf("lint: %v", err)). + WithSuggestion(&output.Suggestion{ + Human: "fix the reported field/key and re-run; see the config reference for valid keys and schemes", + }).Wrap(output.ErrUsage) + } + deployments := 0 + if loaded != nil { + deployments = len(loaded.Config.Deployments) + } + return d.Result(output.NewResult(cmd.CommandPath()).WithBody(lintBody{ + Status: "valid", + Deployments: deployments, + })) }, } } +type lintBody struct { + Status string `json:"status"` + Deployments int `json:"deployments"` +} + +func (b lintBody) WriteText(w io.Writer) error { + _, err := fmt.Fprintf(w, "✓ config valid — %d deployment(s)\n", b.Deployments) + return err +} + +// newExplainCmdImpl returns the real help for a command path instead of +// echoing the argument back. `explain backup` now surfaces backup's +// Short/Long + usage; an unknown command is a usage error naming it. func newExplainCmdImpl() *cobra.Command { return &cobra.Command{ Use: "explain ", Short: "Explain pg_hardstorage commands", Args: cobra.MinimumNArgs(1), SilenceUsage: true, RunE: func(cmd *cobra.Command, args []string) error { d := DispatcherFrom(cmd) - return d.Result(output.NewResult(cmd.CommandPath()).WithBody(map[string]any{ - "command": args[0], + target, _, err := cmd.Root().Find(args) + if err != nil || target == nil || target == cmd.Root() { + return output.NewError("notfound.command", + fmt.Sprintf("explain: no such command %q", strings.Join(args, " "))). + WithSuggestion(&output.Suggestion{ + Human: "run `pg_hardstorage --help` for the command list", + Command: "pg_hardstorage --help", + }) + } + return d.Result(output.NewResult(cmd.CommandPath()).WithBody(explainBody{ + Command: target.CommandPath(), + Summary: target.Short, + Details: strings.TrimSpace(target.Long), + Usage: strings.TrimSpace(target.UseLine()), + HelpHint: fmt.Sprintf("%s --help", target.CommandPath()), })) }, } } +type explainBody struct { + Command string `json:"command"` + Summary string `json:"summary"` + Details string `json:"details,omitempty"` + Usage string `json:"usage"` + HelpHint string `json:"help_hint"` +} + +func (b explainBody) WriteText(w io.Writer) error { + bw := &strings.Builder{} + fmt.Fprintf(bw, "%s — %s\n", b.Command, b.Summary) + fmt.Fprintf(bw, " usage: %s\n", b.Usage) + if b.Details != "" { + fmt.Fprintf(bw, "\n%s\n", b.Details) + } + fmt.Fprintf(bw, "\n(full flags: %s)\n", b.HelpHint) + _, err := io.WriteString(w, bw.String()) + return err +} + func newChangelogCmdImpl() *cobra.Command { return &cobra.Command{ Use: "changelog", Short: "Show changelog", @@ -45,6 +125,35 @@ func newChangelogCmdImpl() *cobra.Command { } } +// glossaryTerms is the canonical term list. `glossary` lists them all; +// `glossary ` looks one up (case-insensitive) and returns its +// definition — previously the lookup path threw the description away and +// echoed back just {"term": args[0]}. +var glossaryTerms = []glossaryEntry{ + {"deployment", "A PostgreSQL instance or cluster you back up, named in pg_hardstorage.yaml."}, + {"backup", "One PITR-recoverable artifact: a base backup plus the manifest that describes it."}, + {"repo", "A content-addressed repository (file/s3/gcs/azblob/sftp/scp) holding chunks, manifests, and WAL."}, + {"chunk", "A content-addressed, deduplicated, compressed (and optionally encrypted) unit of backup data."}, + {"manifest", "The signed description of a backup: its files, chunk references, and metadata."}, + {"wal", "Write-Ahead Log — the PostgreSQL change stream pg_hardstorage archives for point-in-time recovery."}, + {"pitr", "Point-In-Time Recovery: restoring to an arbitrary moment by replaying archived WAL over a base backup."}, + {"rpo", "Recovery Point Objective — how much recent data you can afford to lose (the age of your newest recoverable point)."}, + {"rto", "Recovery Time Objective — how long a restore is allowed to take."}, + {"kek", "Key-Encryption Key: the long-lived key that wraps each backup's data key (DEK)."}, + {"dek", "Data-Encryption Key: the per-backup key that encrypts chunk data; stored wrapped by the KEK."}, + {"tombstone", "A soft-delete marker: the manifest is retired but chunks survive until the next `repo gc --apply`."}, + {"hold", "A legal/retention hold that excludes a backup from rotation and deletion until released."}, + {"retention", "The policy (gfs/simple/count) that decides which backups `rotate` keeps or tombstones."}, + {"incremental", "A backup capturing only blocks changed since a parent (requires PostgreSQL 17+)."}, + {"timeline", "A PostgreSQL history branch (TLI); a promotion/recovery forks a new timeline."}, + {"lsn", "Log Sequence Number — a position in the WAL stream used as a PITR target."}, +} + +type glossaryEntry struct { + Term string `json:"term"` + Description string `json:"description"` +} + func newGlossaryCmdImpl() *cobra.Command { return &cobra.Command{ Use: "glossary []", Short: "Look up pg_hardstorage terminology", @@ -52,17 +161,43 @@ func newGlossaryCmdImpl() *cobra.Command { RunE: func(cmd *cobra.Command, args []string) error { d := DispatcherFrom(cmd) if len(args) == 0 { - return d.Result(output.NewResult(cmd.CommandPath()).WithBody(map[string]any{ - "entries": []map[string]string{ - {"term": "deployment", "description": "A PG instance or cluster you backup"}, - {"term": "backup", "description": "One PITR-recoverable artifact"}, - }, - })) + entries := append([]glossaryEntry(nil), glossaryTerms...) + sort.Slice(entries, func(i, j int) bool { return entries[i].Term < entries[j].Term }) + return d.Result(output.NewResult(cmd.CommandPath()).WithBody(glossaryListBody{Entries: entries})) + } + q := strings.ToLower(strings.TrimSpace(args[0])) + for _, e := range glossaryTerms { + if strings.ToLower(e.Term) == q { + return d.Result(output.NewResult(cmd.CommandPath()).WithBody(e)) + } } - return d.Result(output.NewResult(cmd.CommandPath()).WithBody(map[string]any{"term": args[0]})) + return output.NewError("notfound.term", + fmt.Sprintf("glossary: no entry for %q", args[0])). + WithSuggestion(&output.Suggestion{ + Human: "run `pg_hardstorage glossary` (no argument) to list every term", + Command: "pg_hardstorage glossary", + }) }, } } +type glossaryListBody struct { + Entries []glossaryEntry `json:"entries"` +} + +func (b glossaryListBody) WriteText(w io.Writer) error { + bw := &strings.Builder{} + for _, e := range b.Entries { + fmt.Fprintf(bw, " %-12s %s\n", e.Term, e.Description) + } + _, err := io.WriteString(w, bw.String()) + return err +} + +func (e glossaryEntry) WriteText(w io.Writer) error { + _, err := fmt.Fprintf(w, "%s — %s\n", e.Term, e.Description) + return err +} + // newDemoCmdImpl now lives in demo.go (issue #15) — it runs the real // end-to-end flow instead of printing a placeholder message. diff --git a/internal/cli/recovery.go b/internal/cli/recovery.go index 5c940cb..2e8e9ec 100644 --- a/internal/cli/recovery.go +++ b/internal/cli/recovery.go @@ -329,9 +329,9 @@ func writeReadinessSummary(w io.Writer, r *recovery.ReadinessReport) error { } descr = fmt.Sprintf(" (%s target %ds)", ic, r.RTO.TargetSeconds) } - fmt.Fprintf(bw, " RTO estimate: %ds%s at %s\n", + fmt.Fprintf(bw, " RTO estimate: %ds%s at %s/s\n", r.RTO.EstimatedSeconds, descr, - (time.Duration(r.RTO.AssumedThroughputBytes) * time.Second).String()) + humanBytes(r.RTO.AssumedThroughputBytes)) } if r.Encryption != nil && r.Encryption.Encrypted { ic := "✓" diff --git a/internal/cli/repo.go b/internal/cli/repo.go index b358962..3f3d366 100644 --- a/internal/cli/repo.go +++ b/internal/cli/repo.go @@ -38,6 +38,7 @@ func newRealRepoCmd() *cobra.Command { func newRepoInitCmd() *cobra.Command { var ( + repoURL string wormMode string wormRetention string compression string @@ -89,12 +90,34 @@ Set at init time and not changed after. A repo holding a mix of levels still reads back fine — the decoder handles every level — but the operator's "what does my CPU/disk trade-off look like?" answer is more legible when the level is stable across the repo.`, - Args: cobra.ExactArgs(1), + // Accept the URL as a positional OR via --repo, so it matches + // every other repo verb (repo check/usage/gc/scrub all take + // --repo). Previously only a bare positional was accepted and + // `repo init --repo ` failed with "unknown flag". + Args: cobra.MaximumNArgs(1), SilenceUsage: true, RunE: func(cmd *cobra.Command, args []string) error { - return runRepoInit(cmd, args[0], wormMode, wormRetention, compression) + url := repoURL + if len(args) == 1 { + if url != "" && url != args[0] { + return output.NewError("usage.bad_args", + "repo init: URL given both as positional and --repo; pass it once").Wrap(output.ErrUsage) + } + url = args[0] + } + if url == "" { + return output.NewError("usage.missing_arg", + "repo init: repository URL is required (positional or --repo )"). + WithSuggestion(&output.Suggestion{ + Human: "example: pg_hardstorage repo init file:///srv/pg_hardstorage/repo", + Command: "pg_hardstorage repo init --repo file:///srv/pg_hardstorage/repo", + }).Wrap(output.ErrUsage) + } + return runRepoInit(cmd, url, wormMode, wormRetention, compression) }, } + c.Flags().StringVar(&repoURL, "repo", "", + "repository URL to create (alternative to the positional )") c.Flags().StringVar(&wormMode, "worm-mode", "", "WORM retention mode: compliance | governance (set with --worm-retention)") c.Flags().StringVar(&wormRetention, "worm-retention", "", diff --git a/internal/cli/repo_scrub.go b/internal/cli/repo_scrub.go index 670f79b..5c66bde 100644 --- a/internal/cli/repo_scrub.go +++ b/internal/cli/repo_scrub.go @@ -12,7 +12,6 @@ import ( "github.com/cybertec-postgresql/pg_hardstorage/internal/audit" "github.com/cybertec-postgresql/pg_hardstorage/internal/output" "github.com/cybertec-postgresql/pg_hardstorage/internal/repo" - "github.com/cybertec-postgresql/pg_hardstorage/internal/repo/casdefault" ) // newRepoScrubCmd implements `pg_hardstorage repo scrub `. The @@ -107,8 +106,8 @@ func runRepoScrub(cmd *cobra.Command, urlArg string, samplePercent int, fullScan fmt.Sprintf("repo scrub: collect references: %v", err)).Wrap(err) } - // Convert sample-percent to the existing repo.Scrub's limit - // argument. limit=0 means "everything" — what --full requests. + // Convert sample-percent to a chunk-count limit. limit=0 means + // "everything" — what --full requests. limit := 0 if samplePercent < 100 { limit = refs.Len() * samplePercent / 100 @@ -117,9 +116,15 @@ func runRepoScrub(cmd *cobra.Command, urlArg string, samplePercent int, fullScan } } - cas := casdefault.New(sp) + // Scrub manifest-aware: build the per-manifest CAS so ENCRYPTED + // chunks decrypt with the right DEK before the plaintext-hash + // round-trip. The previous bare casdefault.New(sp) had no + // decryptor, so every encrypted chunk failed to decrypt and was + // bucketed as a "mismatch" — repo scrub reported 100% corruption + // on any encrypted repo (the default after `init`) and exited 9. + // This is the same machinery `repair scrub` uses. startedAt := time.Now().UTC() - res, err := repo.Scrub(cmd.Context(), cas, refs, limit) + agg, refsTotal, err := scrubManifestAware(cmd.Context(), sp, limit) stoppedAt := time.Now().UTC() if err != nil { return output.NewError("repo.scrub.failed", @@ -127,21 +132,21 @@ func runRepoScrub(cmd *cobra.Command, urlArg string, samplePercent int, fullScan } body := repoScrubBody{ - ReferencedTotal: refs.Len(), - Sampled: res.Sampled, - OK: res.OK, - MismatchCount: len(res.Mismatches), - BytesScanned: res.Bytes, + ReferencedTotal: refsTotal, + Sampled: agg.Sampled, + OK: agg.OK, + MismatchCount: len(agg.Mismatches), + BytesScanned: agg.Bytes, SamplePercent: samplePercent, StartedAt: startedAt, StoppedAt: stoppedAt, DurationMS: stoppedAt.Sub(startedAt).Milliseconds(), } - for _, h := range res.Mismatches { + for _, h := range agg.Mismatches { body.Mismatches = append(body.Mismatches, h.String()) } - if len(res.Mismatches) > 0 { + if len(agg.Mismatches) > 0 { // Audit emission for mismatches. Bit-rot is rare enough // that the chain entry is signal — operators investigating // "when did we first see corruption?" can walk the chain @@ -152,17 +157,17 @@ func runRepoScrub(cmd *cobra.Command, urlArg string, samplePercent int, fullScan Timestamp: time.Now().UTC(), Body: map[string]any{ "sample_percent": samplePercent, - "sampled": res.Sampled, + "sampled": agg.Sampled, "referenced_total": refs.Len(), - "mismatch_count": len(res.Mismatches), + "mismatch_count": len(agg.Mismatches), "first_mismatch": body.Mismatches[0], - "bytes_scanned": res.Bytes, + "bytes_scanned": agg.Bytes, "duration_ms": body.DurationMS, }, }) return output.NewError("verify.scrub_mismatch", fmt.Sprintf("repo scrub: %d chunk(s) failed verification (sampled %d of %d referenced)", - len(res.Mismatches), res.Sampled, refs.Len())). + len(agg.Mismatches), agg.Sampled, refsTotal)). WithSuggestion(&output.Suggestion{ Human: "the storage backend has corrupted bytes for the listed chunks. Heal from a replica with `pg_hardstorage repair scrub --heal --replica `.", Command: fmt.Sprintf("pg_hardstorage repair scrub --repo %s --heal --replica ", urlArg), diff --git a/internal/cli/restore.go b/internal/cli/restore.go index 29125e1..1ba1a9c 100644 --- a/internal/cli/restore.go +++ b/internal/cli/restore.go @@ -890,9 +890,16 @@ func (b planBody) WriteText(w io.Writer) error { } fmt.Fprintf(bw, " Estimated RTO: %d ms (assuming %s/s)\n", p.EstimatedRTO.Milliseconds(), humanBytes(p.AssumedThroughput)) - if p.PreflightOK { + if p.WALArchiveHoleLSN != "" { + fmt.Fprintf(bw, " WAL archive hole: ✗ segment missing at %s — recovery would HALT before the target\n", p.WALArchiveHoleLSN) + fmt.Fprintf(bw, " (inspect with `pg_hardstorage wal list %s`; restore from a later backup or pick a target before the hole)\n", p.Deployment) + } + switch { + case p.WALArchiveHoleLSN != "": + fmt.Fprintf(bw, " Pre-flight: ✗ WAL archive hole (target unreachable)") + case p.PreflightOK: fmt.Fprintf(bw, " Pre-flight: ✓ ready") - } else { + default: fmt.Fprintf(bw, " Pre-flight: ✗ %d issue(s)\n", len(p.PreflightIssues)) for _, issue := range p.PreflightIssues { fmt.Fprintf(bw, " - %s\n", issue) diff --git a/internal/cli/root.go b/internal/cli/root.go index 8aa6387..0ec4355 100644 --- a/internal/cli/root.go +++ b/internal/cli/root.go @@ -147,7 +147,7 @@ sandbox-PG runtime — extend per docs/SPEC.md.`, // Order matters — running the dispatcher as root would // open the keyring + state dirs with root-owned files that // a later legitimate run as `pgbackup` couldn't read. - PersistentPreRunE: chainPreRunE(refuseRoot, installDispatcher, resolveDeploymentDefaultsPreRun), + PersistentPreRunE: chainPreRunE(refuseRoot, captureConfigFlag, installDispatcher, resolveDeploymentDefaultsPreRun), } // Flag-parse failures (unknown flag, bad flag value) are usage diff --git a/internal/cli/rotate.go b/internal/cli/rotate.go index 03d3e30..e200c77 100644 --- a/internal/cli/rotate.go +++ b/internal/cli/rotate.go @@ -166,7 +166,7 @@ func runRotate(cmd *cobra.Command, opts rotateOpts) error { Deleted: len(filteredDelete), Held: len(heldIDs), HeldIDs: heldIDs, - Decisions: shapeDecisions(decision), + Decisions: shapeDecisions(decision, heldIDs), } if opts.apply { @@ -296,11 +296,15 @@ func loadDeploymentManifests(ctx context.Context, store *backup.ManifestStore, d } // shapeDecisions flattens the Decision into a JSON-friendly slice -// suitable for the Result body. -func shapeDecisions(d retention.Decision) []rotationDecision { - keptIDs := map[string]bool{} - for _, m := range d.Keep { - keptIDs[m.BackupID] = true +// suitable for the Result body. heldIDs are backups the retention +// policy chose to delete but a legal hold protects — they must render +// as "held", NOT "delete", so the per-backup listing agrees with the +// summary's `held: N (excluded from delete)` line instead of stamping +// a legally-held manifest `[del ]`. +func shapeDecisions(d retention.Decision, heldIDs []string) []rotationDecision { + held := map[string]bool{} + for _, id := range heldIDs { + held[id] = true } out := []rotationDecision{} for _, m := range d.Keep { @@ -312,9 +316,13 @@ func shapeDecisions(d retention.Decision) []rotationDecision { }) } for _, m := range d.Delete { + action := "delete" + if held[m.BackupID] { + action = "held" + } out = append(out, rotationDecision{ BackupID: m.BackupID, - Action: "delete", + Action: action, StoppedAt: m.StoppedAt.UTC().Format(time.RFC3339), }) } @@ -375,8 +383,11 @@ func (b rotateResultBody) WriteText(w io.Writer) error { } for _, d := range dep.Decisions { label := "keep" - if d.Action == "delete" { + switch d.Action { + case "delete": label = "del " + case "held": + label = "held" } reasons := strings.Join(d.Reasons, ",") if reasons != "" { diff --git a/internal/cli/rotate_test.go b/internal/cli/rotate_test.go index f9cea8c..fc82499 100644 --- a/internal/cli/rotate_test.go +++ b/internal/cli/rotate_test.go @@ -79,7 +79,7 @@ func TestShapeDecisions_KeepBeforeDeleteWhenSameStoppedAt(t *testing.T) { Delete: []*backup.Manifest{del}, Reasons: map[string][]string{"k1": {"newest"}}, } - got := shapeDecisions(d) + got := shapeDecisions(d, nil) if len(got) != 2 { t.Fatalf("got %d decisions, want 2", len(got)) } @@ -142,3 +142,24 @@ func TestRotateResultBody_WriteText_Applied(t *testing.T) { t.Errorf("expected applied count; got %s", sb.String()) } } + +// Regression: a held backup that the policy chose to delete must render +// action "held", not "delete", so the per-backup listing agrees with the +// summary's held line instead of stamping a legal-hold manifest [del ]. +func TestShapeDecisions_HeldNotDelete(t *testing.T) { + keep := &backup.Manifest{BackupID: "k1", StoppedAt: time.Date(2026, 1, 3, 0, 0, 0, 0, time.UTC)} + del := &backup.Manifest{BackupID: "d1", StoppedAt: time.Date(2026, 1, 2, 0, 0, 0, 0, time.UTC)} + held := &backup.Manifest{BackupID: "h1", StoppedAt: time.Date(2026, 1, 1, 0, 0, 0, 0, time.UTC)} + d := retention.Decision{Keep: []*backup.Manifest{keep}, Delete: []*backup.Manifest{del, held}} + got := shapeDecisions(d, []string{"h1"}) + byID := map[string]string{} + for _, r := range got { + byID[r.BackupID] = r.Action + } + if byID["h1"] != "held" { + t.Errorf("held backup h1 action = %q, want held", byID["h1"]) + } + if byID["d1"] != "delete" { + t.Errorf("d1 action = %q, want delete", byID["d1"]) + } +} diff --git a/internal/config/config.go b/internal/config/config.go index 5b1826d..556aa06 100644 --- a/internal/config/config.go +++ b/internal/config/config.go @@ -471,6 +471,26 @@ func Load(p *paths.Paths) (*LoadResult, error) { } } + // An explicit -c/--config file (via PG_HARDSTORAGE_CONFIG_FILE) is + // authoritative: read exactly that file and skip the well-known + // path + drop-in dir, so `-c staging.yaml` operates on staging.yaml + // and nothing else. Previously the flag was silently ignored and + // the tool always read /pg_hardstorage.yaml. + if override := strings.TrimSpace(p.ConfigFileOverride); override != "" { + cfg, sf, err := loadFile(override, "config_flag") + if err != nil { + return nil, err + } + res.SourceFiles = append(res.SourceFiles, sf) + if sf.ReadOK { + res.Config = mergeConfig(res.Config, cfg) + } + if err := validate(res.Config); err != nil { + return nil, err + } + return res, nil + } + mainPath := filepath.Join(p.Config.Value, "pg_hardstorage.yaml") mainCfg, mainSF, err := loadFile(mainPath, "main") if err != nil { diff --git a/internal/paths/paths.go b/internal/paths/paths.go index c20a0ee..a823cbc 100644 --- a/internal/paths/paths.go +++ b/internal/paths/paths.go @@ -106,17 +106,21 @@ const ( // Paths is the result of Resolve. Each field carries its Source. type Paths struct { - Mode Mode `json:"-"` - ModeName string - Config Path - ConfigDropIn Path // /conf.d - Deployments Path // /deployments - Sinks Path // /sinks - Skills Path // /skills (operator-overrides; user dirs handled separately) - Keyring Path // /keyring (must be mode 0700) - State Path - Inflight Path // /inflight - Crashes Path // /crashes + Mode Mode `json:"-"` + ModeName string + Config Path + // ConfigFileOverride, when non-empty, is the explicit config file + // named by -c/--config (via PG_HARDSTORAGE_CONFIG_FILE). Empty means + // "use the well-known /pg_hardstorage.yaml". + ConfigFileOverride string `json:"-"` + ConfigDropIn Path // /conf.d + Deployments Path // /deployments + Sinks Path // /sinks + Skills Path // /skills (operator-overrides; user dirs handled separately) + Keyring Path // /keyring (must be mode 0700) + State Path + Inflight Path // /inflight + Crashes Path // /crashes // StateDSN points at the host-local bookkeeping store. Pre-v0.4 // this was a path to a SQLite file; we've since collapsed onto // PG / etcd / K8s `Lease` for any persistent state and kept this @@ -142,17 +146,23 @@ type Options struct { Env func(string) string // env-var lookup; defaults to os.Getenv Root string // when set, overrides everything (paths.root) Overrides map[Domain]string // per-domain explicit override (e.g. --config-dir) + // ConfigFile, when set, names an explicit config FILE (the CLI's + // -c/--config flag, forwarded via PG_HARDSTORAGE_CONFIG_FILE). It + // overrides the well-known /pg_hardstorage.yaml lookup for + // both reads (config.Load) and write-back (configio). + ConfigFile string } // DefaultOptions reads the environment for the running process. Most // callers (CLI, agent, server) use this; tests construct Options directly. func DefaultOptions() Options { return Options{ - Mode: ModeAuto, - UID: os.Geteuid(), - HomeDir: getHome(), - Env: os.Getenv, - Root: os.Getenv("PG_HARDSTORAGE_ROOT"), + Mode: ModeAuto, + UID: os.Geteuid(), + HomeDir: getHome(), + Env: os.Getenv, + Root: os.Getenv("PG_HARDSTORAGE_ROOT"), + ConfigFile: os.Getenv("PG_HARDSTORAGE_CONFIG_FILE"), } } @@ -174,6 +184,7 @@ func Resolve(opts Options) (*Paths, error) { r := &Paths{Mode: mode, ModeName: modeName(mode)} r.Config = resolveDomain(DomainConfig, mode, opts) + r.ConfigFileOverride = opts.ConfigFile r.State = resolveDomain(DomainState, mode, opts) r.Cache = resolveDomain(DomainCache, mode, opts) r.Logs = resolveDomain(DomainLogs, mode, opts) diff --git a/internal/recovery/windows.go b/internal/recovery/windows.go index 614ee8b..5158826 100644 --- a/internal/recovery/windows.go +++ b/internal/recovery/windows.go @@ -8,9 +8,12 @@ import ( "sort" "time" + "github.com/jackc/pglogrepl" + "github.com/cybertec-postgresql/pg_hardstorage/internal/backup" "github.com/cybertec-postgresql/pg_hardstorage/internal/plugin/storage" "github.com/cybertec-postgresql/pg_hardstorage/internal/wal/gapstate" + "github.com/cybertec-postgresql/pg_hardstorage/internal/wal/inventory" ) // WindowsSchema is the on-disk version tag for WindowsReport bodies. @@ -186,6 +189,25 @@ func Windows(ctx context.Context, sp storage.StoragePlugin, deployment string, o if hi != "" { w.LatestRestoreLSN = hi w.HasArchivedWAL = true + // Contiguity cap: hi is only the HIGHEST archived segment — + // it does not mean every segment between the backup and hi + // is present. If there's an archive hole in that range, + // replay HALTS at the hole, so the real latest-reachable LSN + // is the hole's start, not hi. Without this the window + // advertised a PITR range straight across a gap that + // `wal list --gaps-only` plainly detects. + fromLSN, ferr := pglogrepl.ParseLSN(m.StopLSN) + toLSN, terr := pglogrepl.ParseLSN(hi) + if ferr == nil && terr == nil && toLSN > fromLSN { + if hole, found, herr := inventory.FirstWALHoleInRange(ctx, sp, deployment, m.Timeline, fromLSN, toLSN); herr == nil && found { + w.LatestRestoreLSN = hole.String() + w.Gaps = append(w.Gaps, WindowGap{ + StartLSN: hole.String(), + EndLSN: hi, + Source: "archive_scan", + }) + } + } } // Manifest-embedded gaps survive gapstate GC. for _, g := range m.WALGaps { diff --git a/internal/recovery/windows_gap_test.go b/internal/recovery/windows_gap_test.go new file mode 100644 index 0000000..14619c2 --- /dev/null +++ b/internal/recovery/windows_gap_test.go @@ -0,0 +1,84 @@ +package recovery_test + +import ( + "bytes" + "context" + "testing" + "time" + + "github.com/jackc/pglogrepl" + + "github.com/cybertec-postgresql/pg_hardstorage/internal/backup" + "github.com/cybertec-postgresql/pg_hardstorage/internal/pg/walsink" + "github.com/cybertec-postgresql/pg_hardstorage/internal/plugin/storage" + "github.com/cybertec-postgresql/pg_hardstorage/internal/recovery" +) + +// putSeg plants a real segment manifest so walHighestForTimeline + +// FirstWALHoleInRange see it, mirroring the proven gapcheck_test helper. +func putSeg(t *testing.T, sp storage.StoragePlugin, deployment string, tli uint32, segNum uint64) { + t.Helper() + name := walsink.SegmentFileName(tli, segNum, walsink.SegmentSize) + start := pglogrepl.LSN(segNum * uint64(walsink.SegmentSize)) + m := &walsink.SegmentManifest{ + Schema: walsink.Schema, + Deployment: deployment, + SystemIdentifier: "7000000000000000001", + Timeline: tli, + SegmentNumber: segNum, + SegmentName: name, + StartLSN: start.String(), + EndLSN: (start + pglogrepl.LSN(walsink.SegmentSize)).String(), + SegmentSize: walsink.SegmentSize, + } + raw, err := m.MarshalToBytes() + if err != nil { + t.Fatal(err) + } + key := walsink.SegmentPath(deployment, tli, name) + if _, err := sp.Put(context.Background(), key, bytes.NewReader(raw), + storage.PutOptions{ContentLength: int64(len(raw))}); err != nil { + t.Fatal(err) + } +} + +// Regression (#4): recovery windows must CAP LatestRestoreLSN at the +// first WAL archive hole rather than advertising a PITR range straight +// across it. The backup stops in segment 3 (StopLSN 0/30001A0); we +// archive segments 3, 4, 6 — segment 5 is MISSING — so replay HALTS at +// segment 5's start (0/5000000), which is where the window must end. +func TestWindows_CapsAtArchiveHole(t *testing.T) { + w := setupWorld(t) + now := time.Date(2026, 5, 1, 12, 0, 0, 0, time.UTC) + w.commitBackup(t, "db1", now.Add(-1*time.Hour), 1<<30, false, backup.BackupTypeFull, 1) + + for _, seg := range []uint64{3, 4, 6} { // hole at segment 5 + putSeg(t, w.sp, "db1", 1, seg) + } + + r, err := recovery.Windows(context.Background(), w.sp, "db1", recovery.WindowsOptions{ + Verifier: w.verifier, + Now: now, + }) + if err != nil { + t.Fatal(err) + } + if len(r.Windows) != 1 { + t.Fatalf("want 1 window, got %d", len(r.Windows)) + } + win := r.Windows[0] + const holeStart = "0/5000000" // segment 5 start + if win.LatestRestoreLSN != holeStart { + t.Errorf("LatestRestoreLSN = %q, want %q (capped at the hole, NOT the highest segment)", + win.LatestRestoreLSN, holeStart) + } + foundGap := false + for _, g := range win.Gaps { + if g.Source == "archive_scan" && g.StartLSN == holeStart { + foundGap = true + } + } + if !foundGap { + t.Errorf("expected an archive_scan gap at %s; gaps=%+v", holeStart, win.Gaps) + } +} diff --git a/internal/restore/plan.go b/internal/restore/plan.go index 6e286f7..ab13f95 100644 --- a/internal/restore/plan.go +++ b/internal/restore/plan.go @@ -14,6 +14,7 @@ import ( "github.com/cybertec-postgresql/pg_hardstorage/internal/backup" "github.com/cybertec-postgresql/pg_hardstorage/internal/output" "github.com/cybertec-postgresql/pg_hardstorage/internal/repo" + "github.com/cybertec-postgresql/pg_hardstorage/internal/wal/inventory" ) // Plan describes what a Restore call would do, without touching disk. @@ -59,6 +60,13 @@ type Plan struct { PreflightOK bool `json:"preflight_ok"` PreflightIssues []string `json:"preflight_issues,omitempty"` + // WALArchiveHoleLSN, when non-empty, is the first LSN of a WAL + // segment that is MISSING from the archive between this backup's + // stop LSN and the requested PITR target — recovery would HALT + // there. Surfaced so --preview matches the real restore's + // restore.wal_archive_hole warning instead of reporting "✓ ready". + WALArchiveHoleLSN string `json:"wal_archive_hole_lsn,omitempty"` + // EstimatedRTO is a crude estimate based on TotalBytes divided by // a fixed throughput baseline. Documented as approximate; the real // number depends on storage backend, network, disk throughput, and @@ -177,6 +185,22 @@ func Preview(ctx context.Context, opts PlanOptions) (*Plan, error) { // Plan reports the check; it does NOT refuse to plan. preflightSnapshot(opts.TargetDir, p) + // WAL-contiguity peek: the real restore path warns (restore. + // wal_archive_hole) when a WAL segment needed to replay from this + // backup to the target LSN is MISSING from the archive — recovery + // would HALT at the hole. --preview must surface the same finding + // so the dry-run doesn't say "✓ ready" for a target the real + // restore knows is unreachable. + if opts.Recovery != nil && opts.Recovery.Enable && !opts.Recovery.SkipGapCheck && opts.Recovery.TargetLSN != "" { + if target, terr := pglogrepl.ParseLSN(opts.Recovery.TargetLSN); terr == nil { + if stop, serr := pglogrepl.ParseLSN(m.StopLSN); serr == nil && target >= stop { + if hole, found, herr := inventory.FirstWALHoleInRange(ctx, sp, opts.Deployment, m.Timeline, stop, target); herr == nil && found { + p.WALArchiveHoleLSN = hole.String() + } + } + } + } + if p.AssumedThroughput > 0 { secs := float64(p.TotalBytes) / float64(p.AssumedThroughput) p.EstimatedRTO = time.Duration(secs * float64(time.Second)) diff --git a/internal/restore/plan_gap_test.go b/internal/restore/plan_gap_test.go new file mode 100644 index 0000000..7057a59 --- /dev/null +++ b/internal/restore/plan_gap_test.go @@ -0,0 +1,78 @@ +package restore_test + +import ( + "bytes" + "context" + "net/url" + "strings" + "testing" + + "github.com/jackc/pglogrepl" + + "github.com/cybertec-postgresql/pg_hardstorage/internal/pg/walsink" + "github.com/cybertec-postgresql/pg_hardstorage/internal/plugin/storage" + "github.com/cybertec-postgresql/pg_hardstorage/internal/plugin/storage/fs" + "github.com/cybertec-postgresql/pg_hardstorage/internal/restore" +) + +func planPutSeg(t *testing.T, sp storage.StoragePlugin, deployment string, tli uint32, segNum uint64) { + t.Helper() + name := walsink.SegmentFileName(tli, segNum, walsink.SegmentSize) + start := pglogrepl.LSN(segNum * uint64(walsink.SegmentSize)) + m := &walsink.SegmentManifest{ + Schema: walsink.Schema, + Deployment: deployment, + SystemIdentifier: "7000000000000000001", + Timeline: tli, + SegmentNumber: segNum, + SegmentName: name, + StartLSN: start.String(), + EndLSN: (start + pglogrepl.LSN(walsink.SegmentSize)).String(), + SegmentSize: walsink.SegmentSize, + } + raw, err := m.MarshalToBytes() + if err != nil { + t.Fatal(err) + } + key := walsink.SegmentPath(deployment, tli, name) + if _, err := sp.Put(context.Background(), key, bytes.NewReader(raw), + storage.PutOptions{ContentLength: int64(len(raw))}); err != nil { + t.Fatal(err) + } +} + +// Regression (#5): --preview must surface a WAL archive hole between the +// backup's stop point and the PITR target — the same finding the real +// restore path warns about — instead of reporting "✓ ready". The fixture +// backup stops in segment 3 (StopLSN 0/30001A0); archiving segments 3, 4, +// 6 (segment 5 MISSING) with a target in segment 6 must set +// WALArchiveHoleLSN to segment 5's start. +func TestPreview_SurfacesWALArchiveHole(t *testing.T) { + fx := newFixture(t) + root := strings.TrimPrefix(fx.repoURL, "file://") + sp := &fs.Plugin{} + if err := sp.Open(context.Background(), storage.StorageConfig{URL: &url.URL{Scheme: "file", Path: root}}); err != nil { + t.Fatal(err) + } + defer sp.Close() + for _, seg := range []uint64{3, 4, 6} { // hole at segment 5 + planPutSeg(t, sp, "db1", 1, seg) + } + + p, err := restore.Preview(context.Background(), restore.PlanOptions{ + RepoURL: fx.repoURL, + Deployment: "db1", + BackupID: "db1.full.20260428T130000Z.0001", + TargetDir: t.TempDir() + "/restored", + Verifier: fx.verifier, + Recovery: &restore.Recovery{Enable: true, TargetLSN: "0/6000080"}, // in segment 6, past the hole + }) + if err != nil { + t.Fatalf("Preview: %v", err) + } + const holeStart = "0/5000000" // segment 5 start + if p.WALArchiveHoleLSN != holeStart { + t.Errorf("WALArchiveHoleLSN = %q, want %q (preview must not report ✓ ready across a hole)", + p.WALArchiveHoleLSN, holeStart) + } +} diff --git a/man/man1/pg_hardstorage-repo-init.1 b/man/man1/pg_hardstorage-repo-init.1 index 7468ecd..68aaa66 100644 --- a/man/man1/pg_hardstorage-repo-init.1 +++ b/man/man1/pg_hardstorage-repo-init.1 @@ -81,6 +81,10 @@ answer is more legible when the level is stable across the repo. \fB-h\fP, \fB--help\fP[=false] help for init +.PP +\fB--repo\fP="" + repository URL to create (alternative to the positional ) + .PP \fB--worm-mode\fP="" WORM retention mode: compliance | governance (set with --worm-retention)