Skip to content

feat(miner): add backup and restore commands for the local-state directory (#4872)#5551

Closed
carlh7777 wants to merge 1 commit into
JSONbored:mainfrom
carlh7777:feat/miner-backup-restore-cli
Closed

feat(miner): add backup and restore commands for the local-state directory (#4872)#5551
carlh7777 wants to merge 1 commit into
JSONbored:mainfrom
carlh7777:feat/miner-backup-restore-cli

Conversation

@carlh7777

Copy link
Copy Markdown
Contributor

Summary

There was no backup/restore tooling for the miner's local SQLite state. This adds two offline CLI commands:

  • gittensory-miner backup <target-dir> — copies the entire local-state directory (every SQLite store plus its WAL/-shm sidecars) to <target-dir>. Refuses to overwrite an existing directory, so a backup never silently clobbers a previous one.
  • gittensory-miner restore <source-dir> — copies a backup back over the live state directory.

Both resolve the same directory the rest of the CLI uses (GITTENSORY_MINER_CONFIG_DIR, else the XDG default), are purely local file operations (dispatched before the update check, like status/doctor/migrate), and support --json (reusing the shared cli-error.js failure helper, #4836) for a monitored backup job.

Whole-directory, not per-file. The miner has no central store registry — each store module resolves its own path — so a hand-maintained per-file list would go stale as stores are added. Copying the directory as a unit is complete by construction and future-proof, and it brings the WAL sidecars along so an un-checkpointed write isn't lost. Per the issue, this is the "straightforward file-copy-based" approach; the consistency caveat (a copy taken mid-write can capture a torn page; a restore overwrites open files) is surfaced both in the command's own output and in the docs: stop the miner first.

Documentation: a Backup & restore section in docs/operations-runbook.md.

Closes #4872

Scope

  • The PR title follows type(scope): short summary Conventional Commit format.
  • This PR is focused and does not mix unrelated backend, UI, MCP, docs, dependency, and deploy changes.
  • This follows CONTRIBUTING.md and does not reintroduce GitHub Pages, VitePress, site/, or CNAME.
  • I linked a currently open issue this PR resolves (Closes Add backup/restore tooling and guidance #4872).

Validation

  • git diff --check
  • npm run typecheck
  • node --check on the changed lib/bin files (the miner build gate)
  • npm run docs:drift-check
  • 12 unit tests in miner-backup-cli.test.ts exercise real temp-directory file ops end-to-end — backup (files + sidecars, refuse-overwrite, no-state-dir, --json), restore (data round-trip, no-backup-dir, --json), parse errors, both fail-safe catch paths, empty-arg, and the process.env fallback. 100% statement + branch coverage on backup-cli.js.
  • npm run test:workers · build:mcp · ui:* — N/A (miner-package + test change only; no worker/MCP/UI/OpenAPI surface).

If any required check was skipped, explain why:

Change is confined to the miner package (lib authored JS validated by node --check + its .d.ts, a new bin dispatch case, a help line, an ops-runbook doc section) and a unit test. No worker src/**, workflow, or scripts/** guarded paths are touched.

Safety

  • No secrets, wallet details, hotkeys, coldkeys, PATs, private keys, raw trust scores, private rankings, or private maintainer evidence are exposed. The commands copy files and print only the directory paths + file names, never file contents.
  • Public GitHub text stays sanitized and low-noise.
  • Auth/CORS/session negative-path tests — N/A (local file operations).
  • API/OpenAPI/MCP behavior updated/tested — N/A.
  • UI changes use real states — N/A (no UI).
  • Visible UI changes include UI Evidence — N/A.
  • Public docs/changelogs updated where needed — ops-runbook section added; no changelog edit.

UI Evidence

N/A — CLI + docs only.

Notes

  • backup intentionally errors rather than overwriting an existing target directory; use a fresh (e.g. date-stamped) path per backup.
  • The docs give only single-line command invocations plus the stop-the-miner-first caveat — no wrapper-script logic — so there's nothing shell-fragile to get wrong.
  • To relocate the whole store set instead of copying it, GITTENSORY_MINER_CONFIG_DIR already does that; backup/restore are for point-in-time snapshots.

This update introduces two new commands, `gittensory-miner backup` and `gittensory-miner restore`, allowing users to create and restore backups of the entire local-state directory. The backup command ensures that existing directories are not overwritten, while the restore command requires the miner to be stopped to prevent data corruption. Documentation has been updated to reflect these changes, and unit tests have been added to ensure functionality and error handling.

Closes JSONbored#4872.
@carlh7777 carlh7777 requested a review from JSONbored as a code owner July 13, 2026 04:40
@superagent-security superagent-security Bot added the contributor:flagged Contributor flagged for review by trust analysis. label Jul 13, 2026
@superagent-security

Copy link
Copy Markdown
Contributor

🚨 Contributor flagged. Click here for more info: Superagent Dashboard

@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@codecov

codecov Bot commented Jul 13, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.88%. Comparing base (7cffca2) to head (8c48fac).
⚠️ Report is 5 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #5551   +/-   ##
=======================================
  Coverage   94.87%   94.88%           
=======================================
  Files         568      569    +1     
  Lines       45055    45104   +49     
  Branches    14675    14675           
=======================================
+ Hits        42746    42795   +49     
  Misses       1575     1575           
  Partials      734      734           
Flag Coverage Δ
shard-1 43.73% <0.00%> (-0.48%) ⬇️
shard-2 35.41% <0.00%> (-0.08%) ⬇️
shard-3 32.10% <0.00%> (+0.10%) ⬆️
shard-4 31.32% <0.00%> (-0.72%) ⬇️
shard-5 33.48% <0.00%> (+0.58%) ⬆️
shard-6 44.44% <100.00%> (+0.14%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
packages/gittensory-miner/lib/backup-cli.js 100.00% <100.00%> (ø)
packages/gittensory-miner/lib/cli.js 100.00% <ø> (ø)
🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@gittensory-orb gittensory-orb Bot added the gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier. label Jul 13, 2026
@gittensory-orb

gittensory-orb Bot commented Jul 13, 2026

Copy link
Copy Markdown

Caution

🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥

🛑 Gittensory review result - reject/close recommended

Review updated: 2026-07-13 04:47:31 UTC

6 files · 2 AI reviewers · 1 blocker · readiness 100/100 · CI green · unstable

🛑 Suggested Action - Reject/Close

Review summary
This PR adds `backup`/`restore` CLI commands that copy the miner's local-state directory as a whole unit via `cpSync`, with `--json` support, refusal-to-overwrite semantics on backup, and appropriate documentation warning about stopping the miner first for consistency. The implementation is straightforward, correctly wired into the CLI dispatch table before the update check (matching the pattern of `status`/`doctor`/`migrate`), and the test suite exercises the real code paths (round-trip backup/restore, JSON output, ENOTDIR failure, env fallback) rather than fabricated scenarios. The `console.log`/`console.error` calls flagged as 'debug leftovers' are the command's actual user-facing output contract (also asserted by tests), not debugging leftovers, and the depth-5 nesting in `parseDirArg` is a simple for-loop with two early returns, not a real complexity concern.

Blockers

  • packages/gittensory-miner/lib/backup-cli.js:44 copies only `resolveMinerStateDir(env)`, so a user with documented per-store overrides like `GITTENSORY_MINER_PORTFOLIO_QUEUE_DB=/data/queue.sqlite3` gets a successful backup that silently omits live state; either enumerate/include overridden store paths or change the command/docs/output contract so it does not claim a complete local-state backup.
Nits — 5 non-blocking
  • packages/gittensory-miner/lib/backup-cli.js:59-61 and 85-87 — the plain-text success messages differ slightly in phrasing ('stop the miner first for a consistent copy' vs 'run with the miner stopped'); consider aligning wording for consistency.
  • packages/gittensory-miner/lib/backup-cli.js — `restore` overwrites the live state directory with no confirmation prompt or `--force` flag; given backup already refuses to clobber, consider whether restore should have similar friction (e.g., require the target dir to already exist, or add a safety check), though this may be acceptable given the intended offline/manual-invocation workflow.
  • test/unit/miner-backup-cli.test.ts — solid coverage of both success and failure paths for both commands including JSON mode and env fallback; no substantive gap found.
  • Consider documenting in the CLI help/usage text (cli.js) that `restore` overwrites existing files without confirmation, mirroring the runbook's 'stop the miner first' warning inline in `--help` output.
  • packages/gittensory-miner/lib/backup-cli.js: the WAL/`-shm` sidecar copy is a nice touch — worth calling out explicitly in the doc's file table if a future PR adds a central store registry, to keep this whole-directory approach the source of truth.

Why this is blocked

  • packages/gittensory-miner/lib/backup-cli.js:44 copies only `resolveMinerStateDir(env)`, so a user with documented per-store overrides like `GITTENSORY_MINER_PORTFOLIO_QUEUE_DB=/data/queue.sqlite3` gets a successful backup that silently omits live state; either enumerate/include overridden store paths or change the command/docs/output contract so it does not claim a complete local-state backup.
📋 Copy for AI agents — paste into your coding agent
Fix the following blocker(s) from this PR review:

1. packages/gittensory-miner/lib/backup-cli.js:44 copies only \`resolveMinerStateDir\(env\)\`, so a user with documented per-store overrides like \`GITTENSORY\_MINER\_PORTFOLIO\_QUEUE\_DB=/data/queue.sqlite3\` gets a successful backup that silently omits live state; either enumerate/include overridden store paths or change the command/docs/output contract so it does not claim a complete local-state backup.
Flagged checks (non-blocking)
  • Contributor trust — Contributor flagged for review
Signal Result Evidence
Code review ❌ 1 blocker 2 reviewers, synthesized
Linked issue ✅ Linked #4872
Related work ✅ No active overlap found No same-issue or scoped active PR overlap found.
Change scope ✅ 20/20 Low review scope from cached public metadata (1 linked issue).
Validation posture ✅ 25/25 PR body includes validation/test evidence.
Contributor workload ✅ 10/10 Author activity: 190 registered-repo PR(s), 120 merged, 17 issue(s).
Contributor context ✅ Confirmed Gittensor contributor carlh7777; Gittensor profile; 190 PR(s), 17 issue(s).
Gate result ❌ Blocking Repo-configured hard blocker found.
Improvement ✅ Minor risk: clean · value: minor — Code changes are accompanied by test evidence. LLM value judgment: moderate — The PR closes a genuine operational gap (no backup/restore tooling for local SQLite state) with a simple, well-tested, directory-level copy approach that avoids the staleness risk of a per-file list and is documented in the operations runbook.
Linked issue satisfaction

Addressed
The PR adds working `backup` and `restore` CLI commands that copy the entire local-state SQLite directory (including WAL sidecars), with tests covering success and failure paths, and documents the procedure in a new Backup & restore section of the operations runbook.

Review context
  • Author: carlh7777
  • Role context: outside_contributor
  • Public audience mode: oss maintainer
  • Lane context: Repository is configured for direct PR review.
  • Public profile languages: not available
  • Official Gittensor activity: 190 PR(s), 17 issue(s).
  • PR-specific overlap: none found.
Contributor next steps
  • Keep the PR focused and include validation evidence before maintainer review.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.
[BETA] Chat with Gittensory

Ask Gittensory a question about this PR directly in a comment — grounded only in the same cached, public-safe facts shown above, never a new claim.

  • @gittensory ask &lt;question&gt; answers contribution-quality Q&A with source citations and freshness.
  • @gittensory chat &lt;question&gt; answers in natural prose from cached decision-pack facts via local inference (maintainer/collaborator; read-only).
  • A plain-language @gittensory mention with a real question is routed to the closest matching read-only command automatically -- no exact syntax required.

Full command reference: https://gittensory.aethereal.dev/docs/gittensory-commands

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.

  • Re-run Gittensory review

@gittensory-orb gittensory-orb Bot added the manual-review Gittensor contributor context label Jul 13, 2026
@gittensory-orb

Copy link
Copy Markdown

Gittensory is closing this pull request on the maintainer's behalf (AI reviewers agree on a likely critical defect: packages/gittensory-miner/lib/backup-cli.js:44 copies only `resolveMinerStateDir(env)`, so a user with documented per-store overrides like `GITTENSORY_MINER_PORTFOLIO_QUEUE_DB=/data/queue.sqlite3` gets a successful backup that silently omits live state; either enumerate/include overridden store paths or change the command/docs/output contract so it does not claim a complete local-state backup.). This is an automated maintenance action — to pursue this change, please open a new pull request with the issues resolved. Closed PRs may be analyzed later to improve review accuracy, but they are not automatically reopened or re-reviewed.

@gittensory-orb gittensory-orb Bot closed this Jul 13, 2026
@carlh7777 carlh7777 deleted the feat/miner-backup-restore-cli branch July 13, 2026 04:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

contributor:flagged Contributor flagged for review by trust analysis. gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier. manual-review Gittensor contributor context

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add backup/restore tooling and guidance

1 participant