From 8e390a75abbdbb9acedde8ce707dd886d4b0d084 Mon Sep 17 00:00:00 2001 From: Jaap de Haan <261428+jdehaan@users.noreply.github.com> Date: Tue, 24 Mar 2026 07:08:33 +0000 Subject: [PATCH 1/2] docs: Shift docs into subfolder --- README.md | 2 -- MOCKERY_INTEGRATION.md => docs/MOCKERY_INTEGRATION.md | 0 TESTING_GUIDE.md => docs/TESTING_GUIDE.md | 0 3 files changed, 2 deletions(-) rename MOCKERY_INTEGRATION.md => docs/MOCKERY_INTEGRATION.md (100%) rename TESTING_GUIDE.md => docs/TESTING_GUIDE.md (100%) diff --git a/README.md b/README.md index fcf9307..0c5c13e 100644 --- a/README.md +++ b/README.md @@ -161,6 +161,4 @@ Each job writes its rsync output to a dedicated log file, typically named `job-< The log files contain the full rsync output, including the itemized changes and statistics. A `summary.log` file records the status (SUCCESS, FAILURE, SKIPPED) for each job in the run. -**Empty log files** may indicate that no changes were made during the backup for that job. - You can review these logs to audit what was copied, changed, or deleted during each backup run. diff --git a/MOCKERY_INTEGRATION.md b/docs/MOCKERY_INTEGRATION.md similarity index 100% rename from MOCKERY_INTEGRATION.md rename to docs/MOCKERY_INTEGRATION.md diff --git a/TESTING_GUIDE.md b/docs/TESTING_GUIDE.md similarity index 100% rename from TESTING_GUIDE.md rename to docs/TESTING_GUIDE.md From b82428155525c974b1df4ab351ef312d7b99a21b Mon Sep 17 00:00:00 2001 From: Jaap de Haan <261428+jdehaan@users.noreply.github.com> Date: Tue, 24 Mar 2026 11:26:55 +0000 Subject: [PATCH 2/2] docs: Cleanup documentation --- AGENTS.md | 6 +- CONTRIBUTING.md | 2 +- README.md | 153 ++---------------- docs/configuration.md | 90 +++++++++++ ..._INTEGRATION.md => mockery-integration.md} | 0 docs/rsync.md | 56 +++++++ docs/{TESTING_GUIDE.md => testing-guide.md} | 2 +- 7 files changed, 162 insertions(+), 147 deletions(-) create mode 100644 docs/configuration.md rename docs/{MOCKERY_INTEGRATION.md => mockery-integration.md} (100%) create mode 100644 docs/rsync.md rename docs/{TESTING_GUIDE.md => testing-guide.md} (98%) diff --git a/AGENTS.md b/AGENTS.md index ecc78a3..c61d45e 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -7,7 +7,7 @@ Built in Go with `cobra` for CLI, `afero` for filesystem abstraction, and YAML f ## Code Style -- Go 1.25; follow idiomatic Go conventions +- Follow idiomatic Go conventions (see `go.mod` for the required Go version) - Format with `go fmt`; lint with `golangci-lint` (config in `.golangci.yml`) - All linters enabled by default — check `.golangci.yml` for disabled ones - Keep packages focused: `cmd/` for CLI wiring, `internal/` for core logic @@ -61,12 +61,12 @@ make report-coverage # Generate HTML coverage report ## Testing Conventions -- See `TESTING_GUIDE.md` for patterns and examples +- See `docs/testing-guide.md` for patterns and examples - Use **dependency injection** — inject interfaces, not concrete types - **Mocks**: Generated with [mockery](https://github.com/vektra/mockery) (config: `.mockery.yml`) - Mock files live in `internal/test/` as `mock__test.go` - Mock structs named `Mock` (e.g., `MockExec`, `MockJobCommand`) - - See `MOCKERY_INTEGRATION.md` for setup details + - See `docs/mockery-integration.md` for setup details - Use `testify` for assertions (`require` / `assert`) - Test files live in `/test/` subdirectories - Prefer table-driven tests for multiple input scenarios diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 5c4ba03..6ffc553 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -5,7 +5,7 @@ Thanks for your interest in contributing to backup-rsync! ## Getting Started 1. Fork the repo and clone it -2. Install Go 1.25+ and `rsync` +2. Install Go (see `go.mod` for minimum version) and `rsync` 3. Run `make build` to verify the setup ## Development Workflow diff --git a/README.md b/README.md index 0c5c13e..620e64c 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # backup-rsync -Backup using `rsync` as an engine. +Backup using `rsync` as an engine. NOTE: Using rsync in remote mode is not a use case considered for this tool. Both the source and destination are local mounted drives, ensuring efficient and direct data transfer. @@ -17,148 +17,17 @@ Go tool used for my own private purposes. - All backup operations are extensively logged, including detailed rsync output and job summaries. - A dry run mode is available to preview actions without making changes. -## Configuration File Format (`sync.yaml`) - -The backup tool is configured using a YAML file, typically named `config.yaml`. This file defines the sources, targets, variables, and backup jobs. Below is a description of the structure, settings, and an example configuration. - -### Top-Level Structure - -```yaml -sources: # List of source paths to back up -targets: # List of target paths for backups -variables: # Key-value pairs for variable substitution -jobs: # List of backup jobs -``` - -### Sources and Targets - -Each source and target is defined as a `Path` object: - -```yaml -- path: "/path/to/source/or/target/" - exclusions: - - "/path/to/exclude/" -``` - -- `path`: The directory path to include as a source or target. -- `exclusions` (optional): List of subpaths to exclude from backup. - -### Variables - -Variables are key-value pairs that can be referenced in job definitions using `${varname}` syntax. - -```yaml -variables: - target_base: "/mnt/backup1" -``` - -### Jobs - -Each job defines a backup operation: - -```yaml -- name: "job_name" # Unique name for the job - source: "/path/to/source/" # Source directory - target: "/path/to/target/" # Target directory (can use variables) - delete: true # (Optional) Delete files in target not in source (default: true) - enabled: true # (Optional) Enable/disable the job (default: true) - exclusions: # (Optional) List of subpaths to exclude - - "/subpath/to/exclude/" -``` - -#### Job Fields - -- `name`: Unique identifier for the job. -- `source`: Path to the source directory. -- `target`: Path to the target directory. Variables can be used (e.g., `${target_base}/user/home`). -- `delete`: (Optional) If `true`, files deleted from the source are also deleted from the target. Defaults to `true` if omitted. -- `enabled`: (Optional) If `false`, the job is skipped. Defaults to `true` if omitted. -- `exclusions`: (Optional) List of subpaths to exclude from this job. - -### Example Configuration - -```yaml -sources: - - path: "/home/user/" - exclusions: - - "/Downloads/" - - "/.cache/" -targets: - - path: "/mnt/backup1/" -variables: - target_base: "/mnt/backup1" -jobs: - - name: "user_home" - source: "/home/user/" - target: "${target_base}/user/home" - exclusions: - - "/Downloads/" - - "/.cache/" - delete: true - enabled: true - - name: "user_documents" - source: "/home/user/Documents/" - target: "${target_base}/user/documents" -``` - -### Notes - -- All paths should be absolute. -- Exclusions are relative to the specified source or target path. -- Jobs with `enabled: false` are ignored. -- If `delete` is omitted, it defaults to `true` (target files not present in source will be deleted from the destination). - -## rsync and Logging - -This tool uses `rsync` with the following key options: - -- `-a` : Archive mode (preserves permissions, times, symbolic links, etc.) -- `-i` : Itemize changes (shows a change summary for each file) -- `-v` : Verbose output -- `--stats` : Print a detailed set of statistics on the file transfer -- `--delete` : Delete extraneous files from the destination dirs (if enabled in the job) -- `--exclude=PATTERN` : Exclude files matching PATTERN (from job or source/target exclusions) -- `--log-file=FILE` : Write rsync output to the specified log file -- `--dry-run` : Show what would be done, but make no changes (for simulation/dry-run mode) - -### Understanding the `-i` (itemize changes) Output - -The `-i` flag produces a change summary for each file, with a string of characters indicating what changed. For example: +## Quick Start +```sh +make build # Build to dist/backup +./dist/backup --help # Show available commands ``` ->f.st...... somefile.txt -cd+++++++++ newdir/ -``` - -The first character indicates the file type and action: - -- `>` : File sent to the receiver -- `<` : File received from the sender -- `c` : Local change/creation of a directory - -The next characters indicate what changed: - -- `f` : File -- `d` : Directory -- `L` : Symlink -- `D` : Device -- `S` : Special file - -The remaining characters show what changed (see `man rsync` for full details): - -- `s` : Size -- `t` : Modification time -- `p` : Permissions -- `o` : Owner -- `g` : Group -- `a` : ACL -- `x` : Extended attributes -- `+` : Creation (for directories) - -### Logging - -Each job writes its rsync output to a dedicated log file, typically named `job-.log` in a timestamped log directory (e.g., `logs/sync-YYYY-MM-DDTHH-MM-SS/`). -The log files contain the full rsync output, including the itemized changes and statistics. A `summary.log` file records the status (SUCCESS, FAILURE, SKIPPED) for each job in the run. +## Documentation -You can review these logs to audit what was copied, changed, or deleted during each backup run. +- [Configuration File Format](docs/configuration.md) — YAML structure, job definitions, variables, and examples +- [rsync Options and Logging](docs/rsync.md) — rsync flags, itemize-changes output, and log file layout +- [Testing Guide](docs/testing-guide.md) — testing patterns, dependency injection, mocks, and integration tests +- [Mockery Integration](docs/mockery-integration.md) — mock generation setup and usage examples +- [Contributing](CONTRIBUTING.md) — how to set up, develop, and submit changes diff --git a/docs/configuration.md b/docs/configuration.md new file mode 100644 index 0000000..bc8a982 --- /dev/null +++ b/docs/configuration.md @@ -0,0 +1,90 @@ +# Configuration File Format (`sync.yaml`) + +The backup tool is configured using a YAML file, typically named `sync.yaml`. This file defines the sources, targets, variables, and backup jobs. Below is a description of the structure, settings, and an example configuration. + +## Top-Level Structure + +```yaml +sources: # List of source paths to back up +targets: # List of target paths for backups +variables: # Key-value pairs for variable substitution +jobs: # List of backup jobs +``` + +## Sources and Targets + +Each source and target is defined as a `Path` object: + +```yaml +- path: "/path/to/source/or/target/" + exclusions: + - "/path/to/exclude/" +``` + +- `path`: The directory path to include as a source or target. +- `exclusions` (optional): List of subpaths to exclude from backup. + +## Variables + +Variables are key-value pairs that can be referenced in job definitions using `${varname}` syntax. + +```yaml +variables: + target_base: "/mnt/backup1" +``` + +## Jobs + +Each job defines a backup operation: + +```yaml +- name: "job_name" # Unique name for the job + source: "/path/to/source/" # Source directory + target: "/path/to/target/" # Target directory (can use variables) + delete: true # (Optional) Delete files in target not in source (default: true) + enabled: true # (Optional) Enable/disable the job (default: true) + exclusions: # (Optional) List of subpaths to exclude + - "/subpath/to/exclude/" +``` + +### Job Fields + +- `name`: Unique identifier for the job. +- `source`: Path to the source directory. +- `target`: Path to the target directory. Variables can be used (e.g., `${target_base}/user/home`). +- `delete`: (Optional) If `true`, files deleted from the source are also deleted from the target. Defaults to `true` if omitted. +- `enabled`: (Optional) If `false`, the job is skipped. Defaults to `true` if omitted. +- `exclusions`: (Optional) List of subpaths to exclude from this job. + +## Example Configuration + +```yaml +sources: + - path: "/home/user/" + exclusions: + - "/Downloads/" + - "/.cache/" +targets: + - path: "/mnt/backup1/" +variables: + target_base: "/mnt/backup1" +jobs: + - name: "user_home" + source: "/home/user/" + target: "${target_base}/user/home" + exclusions: + - "/Downloads/" + - "/.cache/" + delete: true + enabled: true + - name: "user_documents" + source: "/home/user/Documents/" + target: "${target_base}/user/documents" +``` + +## Notes + +- All paths should be absolute. +- Exclusions are relative to the specified source or target path. +- Jobs with `enabled: false` are ignored. +- If `delete` is omitted, it defaults to `true` (target files not present in source will be deleted from the destination). diff --git a/docs/MOCKERY_INTEGRATION.md b/docs/mockery-integration.md similarity index 100% rename from docs/MOCKERY_INTEGRATION.md rename to docs/mockery-integration.md diff --git a/docs/rsync.md b/docs/rsync.md new file mode 100644 index 0000000..88035e5 --- /dev/null +++ b/docs/rsync.md @@ -0,0 +1,56 @@ +# rsync and Logging + +## rsync Options + +This tool uses `rsync` with the following key options: + +- `-a` : Archive mode (preserves permissions, times, symbolic links, etc.) +- `-i` : Itemize changes (shows a change summary for each file) +- `-v` : Verbose output +- `--stats` : Print a detailed set of statistics on the file transfer +- `--delete` : Delete extraneous files from the destination dirs (if enabled in the job) +- `--exclude=PATTERN` : Exclude files matching PATTERN (from job or source/target exclusions) +- `--log-file=FILE` : Write rsync output to the specified log file +- `--dry-run` : Show what would be done, but make no changes (for simulation/dry-run mode) + +## Understanding the `-i` (itemize changes) Output + +The `-i` flag produces a change summary for each file, with a string of characters indicating what changed. For example: + +``` +>f.st...... somefile.txt +cd+++++++++ newdir/ +``` + +The first character indicates the file type and action: + +- `>` : File sent to the receiver +- `<` : File received from the sender +- `c` : Local change/creation of a directory + +The next characters indicate what changed: + +- `f` : File +- `d` : Directory +- `L` : Symlink +- `D` : Device +- `S` : Special file + +The remaining characters show what changed (see `man rsync` for full details): + +- `s` : Size +- `t` : Modification time +- `p` : Permissions +- `o` : Owner +- `g` : Group +- `a` : ACL +- `x` : Extended attributes +- `+` : Creation (for directories) + +## Logging + +Each job writes its rsync output to a dedicated log file, typically named `job-.log` in a timestamped log directory (e.g., `logs/sync-YYYY-MM-DDTHH-MM-SS/`). + +The log files contain the full rsync output, including the itemized changes and statistics. A `summary.log` file records the status (SUCCESS, FAILURE, SKIPPED) for each job in the run. + +You can review these logs to audit what was copied, changed, or deleted during each backup run. diff --git a/docs/TESTING_GUIDE.md b/docs/testing-guide.md similarity index 98% rename from docs/TESTING_GUIDE.md rename to docs/testing-guide.md index 17762f3..cb598c3 100644 --- a/docs/TESTING_GUIDE.md +++ b/docs/testing-guide.md @@ -53,7 +53,7 @@ A lightweight exec stub (implementing the `Exec` interface inline) is used inste Generated mocks (via `mockery`) use the `.EXPECT()` pattern for setting expectations. Each test creates its own mock instance — no shared state between tests. -Mock configuration: `.mockery.yml`. See `MOCKERY_INTEGRATION.md` for regeneration instructions. +Mock configuration: `.mockery.yml`. See `mockery-integration.md` for regeneration instructions. ## Integration Tests