Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ Built in Go with `cobra` for CLI, `afero` for filesystem abstraction, and YAML f

## Code Style

- Go 1.25; follow idiomatic Go conventions
- Follow idiomatic Go conventions (see `go.mod` for the required Go version)
- Format with `go fmt`; lint with `golangci-lint` (config in `.golangci.yml`)
- All linters enabled by default — check `.golangci.yml` for disabled ones
- Keep packages focused: `cmd/` for CLI wiring, `internal/` for core logic
Expand Down Expand Up @@ -61,12 +61,12 @@ make report-coverage # Generate HTML coverage report

## Testing Conventions

- See `TESTING_GUIDE.md` for patterns and examples
- See `docs/testing-guide.md` for patterns and examples
- Use **dependency injection** — inject interfaces, not concrete types
- **Mocks**: Generated with [mockery](https://github.com/vektra/mockery) (config: `.mockery.yml`)
- Mock files live in `internal/test/` as `mock_<interface>_test.go`
- Mock structs named `Mock<Interface>` (e.g., `MockExec`, `MockJobCommand`)
- See `MOCKERY_INTEGRATION.md` for setup details
- See `docs/mockery-integration.md` for setup details
- Use `testify` for assertions (`require` / `assert`)
- Test files live in `<package>/test/` subdirectories
- Prefer table-driven tests for multiple input scenarios
Expand Down
2 changes: 1 addition & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ Thanks for your interest in contributing to backup-rsync!
## Getting Started

1. Fork the repo and clone it
2. Install Go 1.25+ and `rsync`
2. Install Go (see `go.mod` for minimum version) and `rsync`
3. Run `make build` to verify the setup

## Development Workflow
Expand Down
155 changes: 11 additions & 144 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# backup-rsync

Backup using `rsync` as an engine.
Backup using `rsync` as an engine.

NOTE: Using rsync in remote mode is not a use case considered for this tool.
Both the source and destination are local mounted drives, ensuring efficient and direct data transfer.
Expand All @@ -17,150 +17,17 @@ Go tool used for my own private purposes.
- All backup operations are extensively logged, including detailed rsync output and job summaries.
- A dry run mode is available to preview actions without making changes.

## Configuration File Format (`sync.yaml`)
## Quick Start

The backup tool is configured using a YAML file, typically named `config.yaml`. This file defines the sources, targets, variables, and backup jobs. Below is a description of the structure, settings, and an example configuration.

### Top-Level Structure

```yaml
sources: # List of source paths to back up
targets: # List of target paths for backups
variables: # Key-value pairs for variable substitution
jobs: # List of backup jobs
```

### Sources and Targets

Each source and target is defined as a `Path` object:

```yaml
- path: "/path/to/source/or/target/"
exclusions:
- "/path/to/exclude/"
```

- `path`: The directory path to include as a source or target.
- `exclusions` (optional): List of subpaths to exclude from backup.

### Variables

Variables are key-value pairs that can be referenced in job definitions using `${varname}` syntax.

```yaml
variables:
target_base: "/mnt/backup1"
```

### Jobs

Each job defines a backup operation:

```yaml
- name: "job_name" # Unique name for the job
source: "/path/to/source/" # Source directory
target: "/path/to/target/" # Target directory (can use variables)
delete: true # (Optional) Delete files in target not in source (default: true)
enabled: true # (Optional) Enable/disable the job (default: true)
exclusions: # (Optional) List of subpaths to exclude
- "/subpath/to/exclude/"
```

#### Job Fields

- `name`: Unique identifier for the job.
- `source`: Path to the source directory.
- `target`: Path to the target directory. Variables can be used (e.g., `${target_base}/user/home`).
- `delete`: (Optional) If `true`, files deleted from the source are also deleted from the target. Defaults to `true` if omitted.
- `enabled`: (Optional) If `false`, the job is skipped. Defaults to `true` if omitted.
- `exclusions`: (Optional) List of subpaths to exclude from this job.

### Example Configuration

```yaml
sources:
- path: "/home/user/"
exclusions:
- "/Downloads/"
- "/.cache/"
targets:
- path: "/mnt/backup1/"
variables:
target_base: "/mnt/backup1"
jobs:
- name: "user_home"
source: "/home/user/"
target: "${target_base}/user/home"
exclusions:
- "/Downloads/"
- "/.cache/"
delete: true
enabled: true
- name: "user_documents"
source: "/home/user/Documents/"
target: "${target_base}/user/documents"
```sh
make build # Build to dist/backup
./dist/backup --help # Show available commands
```

### Notes

- All paths should be absolute.
- Exclusions are relative to the specified source or target path.
- Jobs with `enabled: false` are ignored.
- If `delete` is omitted, it defaults to `true` (target files not present in source will be deleted from the destination).

## rsync and Logging

This tool uses `rsync` with the following key options:

- `-a` : Archive mode (preserves permissions, times, symbolic links, etc.)
- `-i` : Itemize changes (shows a change summary for each file)
- `-v` : Verbose output
- `--stats` : Print a detailed set of statistics on the file transfer
- `--delete` : Delete extraneous files from the destination dirs (if enabled in the job)
- `--exclude=PATTERN` : Exclude files matching PATTERN (from job or source/target exclusions)
- `--log-file=FILE` : Write rsync output to the specified log file
- `--dry-run` : Show what would be done, but make no changes (for simulation/dry-run mode)

### Understanding the `-i` (itemize changes) Output

The `-i` flag produces a change summary for each file, with a string of characters indicating what changed. For example:

```
>f.st...... somefile.txt
cd+++++++++ newdir/
```

The first character indicates the file type and action:

- `>` : File sent to the receiver
- `<` : File received from the sender
- `c` : Local change/creation of a directory

The next characters indicate what changed:

- `f` : File
- `d` : Directory
- `L` : Symlink
- `D` : Device
- `S` : Special file

The remaining characters show what changed (see `man rsync` for full details):

- `s` : Size
- `t` : Modification time
- `p` : Permissions
- `o` : Owner
- `g` : Group
- `a` : ACL
- `x` : Extended attributes
- `+` : Creation (for directories)

### Logging

Each job writes its rsync output to a dedicated log file, typically named `job-<jobname>.log` in a timestamped log directory (e.g., `logs/sync-YYYY-MM-DDTHH-MM-SS/`).

The log files contain the full rsync output, including the itemized changes and statistics. A `summary.log` file records the status (SUCCESS, FAILURE, SKIPPED) for each job in the run.

**Empty log files** may indicate that no changes were made during the backup for that job.
## Documentation

You can review these logs to audit what was copied, changed, or deleted during each backup run.
- [Configuration File Format](docs/configuration.md) — YAML structure, job definitions, variables, and examples
- [rsync Options and Logging](docs/rsync.md) — rsync flags, itemize-changes output, and log file layout
- [Testing Guide](docs/testing-guide.md) — testing patterns, dependency injection, mocks, and integration tests
- [Mockery Integration](docs/mockery-integration.md) — mock generation setup and usage examples
- [Contributing](CONTRIBUTING.md) — how to set up, develop, and submit changes
90 changes: 90 additions & 0 deletions docs/configuration.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
# Configuration File Format (`sync.yaml`)

The backup tool is configured using a YAML file, typically named `sync.yaml`. This file defines the sources, targets, variables, and backup jobs. Below is a description of the structure, settings, and an example configuration.

## Top-Level Structure

```yaml
sources: # List of source paths to back up
targets: # List of target paths for backups
variables: # Key-value pairs for variable substitution
jobs: # List of backup jobs
```

## Sources and Targets

Each source and target is defined as a `Path` object:

```yaml
- path: "/path/to/source/or/target/"
exclusions:
- "/path/to/exclude/"
```

- `path`: The directory path to include as a source or target.
- `exclusions` (optional): List of subpaths to exclude from backup.

## Variables

Variables are key-value pairs that can be referenced in job definitions using `${varname}` syntax.

```yaml
variables:
target_base: "/mnt/backup1"
```

## Jobs

Each job defines a backup operation:

```yaml
- name: "job_name" # Unique name for the job
source: "/path/to/source/" # Source directory
target: "/path/to/target/" # Target directory (can use variables)
delete: true # (Optional) Delete files in target not in source (default: true)
enabled: true # (Optional) Enable/disable the job (default: true)
exclusions: # (Optional) List of subpaths to exclude
- "/subpath/to/exclude/"
```

### Job Fields

- `name`: Unique identifier for the job.
- `source`: Path to the source directory.
- `target`: Path to the target directory. Variables can be used (e.g., `${target_base}/user/home`).
- `delete`: (Optional) If `true`, files deleted from the source are also deleted from the target. Defaults to `true` if omitted.
- `enabled`: (Optional) If `false`, the job is skipped. Defaults to `true` if omitted.
- `exclusions`: (Optional) List of subpaths to exclude from this job.

## Example Configuration

```yaml
sources:
- path: "/home/user/"
exclusions:
- "/Downloads/"
- "/.cache/"
targets:
- path: "/mnt/backup1/"
variables:
target_base: "/mnt/backup1"
jobs:
- name: "user_home"
source: "/home/user/"
target: "${target_base}/user/home"
exclusions:
- "/Downloads/"
- "/.cache/"
delete: true
enabled: true
- name: "user_documents"
source: "/home/user/Documents/"
target: "${target_base}/user/documents"
```

## Notes

- All paths should be absolute.
- Exclusions are relative to the specified source or target path.
- Jobs with `enabled: false` are ignored.
- If `delete` is omitted, it defaults to `true` (target files not present in source will be deleted from the destination).
File renamed without changes.
56 changes: 56 additions & 0 deletions docs/rsync.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
# rsync and Logging

## rsync Options

This tool uses `rsync` with the following key options:

- `-a` : Archive mode (preserves permissions, times, symbolic links, etc.)
- `-i` : Itemize changes (shows a change summary for each file)
- `-v` : Verbose output
- `--stats` : Print a detailed set of statistics on the file transfer
- `--delete` : Delete extraneous files from the destination dirs (if enabled in the job)
- `--exclude=PATTERN` : Exclude files matching PATTERN (from job or source/target exclusions)
- `--log-file=FILE` : Write rsync output to the specified log file
- `--dry-run` : Show what would be done, but make no changes (for simulation/dry-run mode)

## Understanding the `-i` (itemize changes) Output

The `-i` flag produces a change summary for each file, with a string of characters indicating what changed. For example:

```
>f.st...... somefile.txt
cd+++++++++ newdir/
```

The first character indicates the file type and action:

- `>` : File sent to the receiver
- `<` : File received from the sender
- `c` : Local change/creation of a directory

The next characters indicate what changed:

- `f` : File
- `d` : Directory
- `L` : Symlink
- `D` : Device
- `S` : Special file

The remaining characters show what changed (see `man rsync` for full details):

- `s` : Size
- `t` : Modification time
- `p` : Permissions
- `o` : Owner
- `g` : Group
- `a` : ACL
- `x` : Extended attributes
- `+` : Creation (for directories)

## Logging

Each job writes its rsync output to a dedicated log file, typically named `job-<jobname>.log` in a timestamped log directory (e.g., `logs/sync-YYYY-MM-DDTHH-MM-SS/`).

The log files contain the full rsync output, including the itemized changes and statistics. A `summary.log` file records the status (SUCCESS, FAILURE, SKIPPED) for each job in the run.

You can review these logs to audit what was copied, changed, or deleted during each backup run.
2 changes: 1 addition & 1 deletion TESTING_GUIDE.md → docs/testing-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ A lightweight exec stub (implementing the `Exec` interface inline) is used inste

Generated mocks (via `mockery`) use the `.EXPECT()` pattern for setting expectations. Each test creates its own mock instance — no shared state between tests.

Mock configuration: `.mockery.yml`. See `MOCKERY_INTEGRATION.md` for regeneration instructions.
Mock configuration: `.mockery.yml`. See `mockery-integration.md` for regeneration instructions.

## Integration Tests

Expand Down
Loading