diff --git a/Makefile b/Makefile index f12b3d489..8080ca9cb 100644 --- a/Makefile +++ b/Makefile @@ -291,6 +291,7 @@ generate-config-reference: build ## Generate the configuration reference documen .PHONY: documentation documentation: generate-jsonschema generate-config-reference $(GOBIN)/hugo ## Generate the documentation site @echo "[*] $@" + git submodule init && git submodule update --depth=1 cd docs && hugo --minify .PHONY: syslog-ng diff --git a/docs/content/_index.md b/docs/content/_index.md index aabf7c25d..38b18aaa2 100644 --- a/docs/content/_index.md +++ b/docs/content/_index.md @@ -4,40 +4,7 @@ title: resticprofile description: Configuration profiles manager for restic backup --- -Configuration profiles manager for [restic backup](https://restic.net/) +**resticprofile** is a wrapper for [restic backup](https://restic.net/). It bridges the gap between a configuration file and a runner that generates all necessary calls to **restic**. -**resticprofile** bridges the gap between a configuration file and restic backup. Although creating a configuration file for restic has been [discussed](https://github.com/restic/restic/issues/16), it remains a low priority. +Although creating a configuration file for restic has been [discussed](https://github.com/restic/restic/issues/16), it remains a low priority task. -With resticprofile: - -* No need to remember command parameters and environment variables -* Create multiple profiles in one configuration file -* Profiles can inherit options from other profiles -* Run the forget command before or after a backup (in a section called *retention*) -* Check a repository before or after a backup -* Create groups of profiles to run sequentially -* Run [shell commands]({{% relref "/configuration/run_hooks" %}}) before or after running a profile, useful for mounting and unmounting backup disks -* Run a [shell command]({{% relref "/configuration/run_hooks" %}}) if an error occurs -* Send a backup stream via _stdin_ -* Start restic at different [priorities]({{% relref "/configuration/priority" %}}) (Priority Class in Windows, *nice* in Unix, and/or _ionice_ in Linux) -* Check for [enough memory]({{% relref "/usage/memory" %}}) before starting a backup -* Generate cryptographically secure random keys for a restic [key file]({{% relref "/usage/keyfile" %}}) -* Easily [schedule]({{% relref "/schedules" %}}) backups, retentions, and checks (supports *systemd*, *crond*, *launchd*, and *Windows Task Scheduler*) -* Generate a simple [status file]({{% relref "/monitoring/status" %}}) for monitoring software to ensure backups are running smoothly -* Use [template syntax]({{% relref "/configuration/templates" %}}) in your configuration file -* Automatically clear [stale locks]({{% relref "/usage/locks" %}}) -* Export a [prometheus]({{% relref "/monitoring/prometheus" %}}) file after a backup or send the report to a push gateway -* Run shell commands in the background when non-fatal errors are detected -* Send messages to [HTTP hooks]({{% relref "/configuration/http_hooks" %}}) before, after a successful or failed job (backup, forget, check, prune, copy) -* Automatically [initialize the secondary repository]({{% relref "/configuration/copy" %}}) using the `copy-chunker-params` flag -* Send resticprofile [logs]({{% relref "/configuration/logs" %}}) to a syslog server -* Prevent the system from [idle sleeping]({{% relref "/configuration/sleep" %}}) -* View help for both restic and resticprofile via the `help` command or `-h` flag -* Avoid scheduling a job when the system is on battery -* **[new for v0.29.0]** Schedule a group of profiles (configuration `v2` only) - -The configuration file supports various formats: -* [TOML](https://github.com/toml-lang/toml): files with extensions *.toml* and *.conf* (for compatibility with versions before 0.6.0) -* [JSON](https://en.wikipedia.org/wiki/JSON): files with extension *.json* -* [YAML](https://en.wikipedia.org/wiki/YAML): files with extension *.yaml* -* [HCL](https://github.com/hashicorp/hcl): files with extension *.hcl* \ No newline at end of file diff --git a/docs/content/changelog.md b/docs/content/changelog.md index 4ccf5fdd2..d79ba5798 100644 --- a/docs/content/changelog.md +++ b/docs/content/changelog.md @@ -1,6 +1,6 @@ --- archetype: chapter -pre: 8. +pre: 7. title: Release Notes weight: 8 --- diff --git a/docs/content/configuration/_index.md b/docs/content/configuration/_index.md index fa6604dfd..dbd8de4d8 100644 --- a/docs/content/configuration/_index.md +++ b/docs/content/configuration/_index.md @@ -1,12 +1,17 @@ +++ archetype = "chapter" -pre = "2. " +pre = "3. " title = "Configuration file" -weight = 2 +weight = 3 +++ +The configuration file supports various formats: +* [TOML](https://github.com/toml-lang/toml): files with extensions *.toml* and *.conf* (for compatibility with versions before 0.6.0) +* [JSON](https://en.wikipedia.org/wiki/JSON): files with extension *.json* +* [YAML](https://en.wikipedia.org/wiki/YAML): files with extension *.yaml* +* [HCL](https://github.com/hashicorp/hcl): files with extension *.hcl* -* A configuration is a set of _profiles_. +* A configuration is a set of [_profiles_]({{% relref "/configuration/profiles" %}}). * Each profile is in a new section that has the name of the profile. * Inside each profile, you can specify different flags for each command. * A command definition is in a subsection of the name of the command. @@ -65,106 +70,6 @@ To set the flag `--password-file`, the name of the parameter is simply `password There's **one exception**: the flag `--repo` is named `repository` in the configuration -## Example - -So let's say you normally use this simple command: - -```shell -restic --repo "local:/backup" --password-file "password.txt" --verbose backup /home -``` - -For resticprofile to generate this command automatically for you, here's the configuration file: - -{{< tabs groupid="config-with-json" >}} -{{% tab title="toml" %}} - -```toml -# indentation is not needed but it makes it easier to read ;) -# -version = "1" - -[default] - repository = "local:/backup" - password-file = "password.txt" - - [default.backup] - verbose = true - source = [ "/home" ] -``` - -{{% /tab %}} -{{% tab title="yaml" %}} - -```yaml -version: "1" - -default: - repository: "local:/backup" - password-file: "password.txt" - - backup: - verbose: true - source: - - "/home" -``` - -{{% /tab %}} -{{% tab title="hcl" %}} - -```hcl -default { - repository = "local:/backup" - password-file = "password.txt" - - backup = { - verbose = true - source = [ "/home" ] - } -} -``` - -{{% /tab %}} -{{% tab title="json" %}} - -```json -{ - "version": "1", - "default": { - "repository": "local:/backup", - "password-file": "password.txt", - "backup": { - "verbose": true, - "source": [ - "/home" - ] - } - } -} -``` - -{{% /tab %}} -{{< /tabs >}} - - -You may have noticed the `source` flag is accepting an array of values (inside brackets in TOML, list of values in YAML) - -Now, assuming this configuration file is named `profiles.conf` in the current folder (it's the default config file name), you can simply run - -```shell -resticprofile backup -``` - -and resticprofile will do its magic and generate the command line for you. - -If you have any doubt on what it's running, you can try a `--dry-run`: - -```shell -resticprofile --dry-run backup -2022/05/18 17:14:07 profile 'default': starting 'backup' -2022/05/18 17:14:07 dry-run: /usr/bin/restic backup --password-file password.txt --repo local:/backup --verbose /home -2022/05/18 17:14:07 profile 'default': finished 'backup' -``` - ## More information {{% children %}} diff --git a/docs/content/configuration/hooks/_index.md b/docs/content/configuration/hooks/_index.md new file mode 100644 index 000000000..6c0ad0a46 --- /dev/null +++ b/docs/content/configuration/hooks/_index.md @@ -0,0 +1,7 @@ +--- +title: Hooks +weight: 60 +alwaysopen: false +--- + +{{% children sort="weight" depth="2" %}} diff --git a/docs/content/configuration/copy.md b/docs/content/configuration/hooks/copy.md similarity index 99% rename from docs/content/configuration/copy.md rename to docs/content/configuration/hooks/copy.md index 03903f6ea..0b328ec9d 100644 --- a/docs/content/configuration/copy.md +++ b/docs/content/configuration/hooks/copy.md @@ -3,8 +3,6 @@ title: "Copy command" weight: 17 --- - - ## Special case for the **copy** command section The copy command needs two repositories (and quite likely 2 different set of keys). You can configure a `copy` section like this: @@ -190,4 +188,4 @@ profile { ``` {{% /tab %}} -{{< /tabs >}} \ No newline at end of file +{{< /tabs >}} diff --git a/docs/content/configuration/hooks/http_hooks.md b/docs/content/configuration/hooks/http_hooks.md new file mode 100644 index 000000000..4448d2fd7 --- /dev/null +++ b/docs/content/configuration/hooks/http_hooks.md @@ -0,0 +1,169 @@ +--- +title: "Hooks - HTTP" +weight: 22 +tags: [ "monitoring", "healthchecks.io" ] +--- + +{{< toc >}} + +## Send HTTP messages before and after a job + +As well as being able to run [shell commands]({{% relref "run_hooks" %}}), you can now send HTTP messages before, after (success or failure) running a restic command. + +The sections that allow sending HTTP hooks are: +- backup +- copy +- check +- forget +- prune + +{{% notice tip %}} +You might notice that's the same sections that can also be scheduled +{{% /notice %}} + +Each of these commands can send 4 different types of hooks: + +- send-before +- send-after +- send-after-fail +- send-finally + +The configuration is the same for each of these 4 types of hooks: + +| Name | Required | Default | Notes | +|:-----|:---------|:--------|:------| +| url | Yes | None | URL of your Webhook | +| method | No | GET | This is the HTTP method (GET, POST, HEAD, etc.) | +| skip-tls-verification | No | False | **This is not recommended**: Use only if you're using your own server with a self-signed certificate | +| headers | No | User-Agent set to resticprofile | This is a subsection with a list of `name` and `value` | +| body | No | Empty | Used to send data to the Webhook (POST, PUT, PATCH) | +| body-template | No | None | Template file to generate the body (in go template format) | + + +A few environment variables will be available to construct the url and the body: +- `PROFILE_NAME` +- `PROFILE_COMMAND`: backup, check, forget, etc. + +Additionally, for the `send-after-fail` hooks, these environment variables will be available: +- `ERROR` containing the latest error message +- `ERROR_COMMANDLINE` containing the command line that failed +- `ERROR_EXIT_CODE` containing the exit code of the command line that failed +- `ERROR_STDERR` containing any message that the failed command sent to the standard error (stderr) + +URL encoding is applayed for variables `ERROR`, `ERROR_COMMANDLINE` and `ERROR_STDERR` if they are used in URL. + +The `send-finally` hooks are also getting the environment of `send-after-fail` when any previous operation has failed (except any `send` operation). + +Failures in any `send-*` are logged but do not influence environment or return code. + +## body-template + +Templates pull from the standard go template to build the webhook body. The object passed as an argument to the template is: + +- `ProfileName` **string** +- `ProfileCommand` **string** +- `Error` **ErrorContext** +- `Stdout` **string** + +The type **ErrorContext** is available after an error occurred (otherwise all fields are blank): +- `Message` **string** +- `CommandLine` **string** +- `ExitCode` **string** +- `Stderr` **string** + +## Self-signed certificates + +If your monitoring system is using self-signed certificates, you can import them in resticprofile (and you don't need to rely on the `skip-tls-verification` flag) + +The parameter is in the `global` section and is called `ca-certificates`: it contains a list of certificate files (PEM). + +### timeout + +The default timeout for all HTTP requests is 30 seconds. + +You can change the default timeout in the `global` section with a parameter called `send-timeout`. + +The format is like: +- 30s +- 2m +- 1m20s + +### global configuration example + + +{{< tabs groupid="config-with-json" >}} +{{% tab title="toml" %}} + +```toml +version = "1" + +[global] + send-timeout = "10s" + ca-certificates = [ "ca-chain.cert.pem" ] +``` + + +{{% /tab %}} +{{% tab title="yaml" %}} + +```yaml +version: "1" + +global: + send-timeout: 10s + ca-certificates: + - ca-chain.cert.pem + +``` + +{{% /tab %}} +{{% tab title="hcl" %}} + +```hcl + +global { + send-timeout = "10s" + ca-certificates = "ca-chain.cert.pem" +} +``` + +{{% /tab %}} +{{% tab title="json" %}} + +```json +{ + "version": "1", + "global": { + "send-timeout": "10s", + "ca-certificates": [ + "ca-chain.cert.pem" + ] + } +} +``` + +{{% /tab %}} +{{< /tabs >}} + +## order of `send-*` + +Here's the flow of HTTP hooks: + +```mermaid +graph TD + LOCK(set resticprofile lock) + UNLOCK(delete resticprofile lock) + LOCK --> SB + SB('send-before') --> RUN + RUN(run restic command, or group of commands) + RUN -->|Success| SA + RUN -->|Error| SAF + SA('send-after') --> SF + SAF('send-after-fail') --> SF + SF('send-finally') + SF --> UNLOCK +``` + +{{% notice style="warning" title="resticprofile lock" %}} +The local resticprofile lock is surrounding the whole process. It means that the `run-after-fail` target is not called if the lock cannot be obtained. This is a limitation of the current implementation. +{{% /notice %}} diff --git a/docs/content/configuration/run_hooks.md b/docs/content/configuration/hooks/run_hooks.md similarity index 99% rename from docs/content/configuration/run_hooks.md rename to docs/content/configuration/hooks/run_hooks.md index 67c5de3b9..c9c969df1 100644 --- a/docs/content/configuration/run_hooks.md +++ b/docs/content/configuration/hooks/run_hooks.md @@ -1,9 +1,10 @@ --- -title: "Command Hooks" +title: "Hooks - Commands" weight: 20 tags: [ "monitoring" ] --- +{{< toc >}} ## Run commands before, after success or after failure @@ -134,7 +135,7 @@ snapshot 3949d2fb saved == run-finally from profile documents after command backup ``` -### Order of `run-*` during a backup +## Order of `run-*` during a backup The commands will be running in this order **during a backup**: - `run-before` from the profile - if error, go to `run-after-fail` @@ -205,7 +206,7 @@ flowchart TB The local resticprofile lock is surrounding the whole process. It means that the `run-after-fail` target is not called if the lock cannot be obtained. This is a limitation of the current implementation. {{% /notice %}} -### Passing environment variables +## Passing environment variables Environment variables can be adjusted and passed between shell commands & restic by writing one or more `VARIABLE=VALUE` into an `env-file` that is configured within the current profile. To simplify this for shell commands, the [template]({{% relref "templates" %}}) function `{{env}}` can be used to get a temporary dotenv file that is automatically appended to the `env-file` list of the current profile and reloaded whenever modified. diff --git a/docs/content/configuration/http_hooks.md b/docs/content/configuration/http_hooks.md deleted file mode 100644 index acd1c4646..000000000 --- a/docs/content/configuration/http_hooks.md +++ /dev/null @@ -1,532 +0,0 @@ ---- -title: "HTTP Hooks" -weight: 22 -tags: [ "monitoring", "healthchecks.io" ] ---- - - -## Send HTTP messages before and after a job - -As well as being able to run [shell commands]({{% relref "run_hooks" %}}), you can now send HTTP messages before, after (success or failure) running a restic command. - -The sections that allow sending HTTP hooks are: -- backup -- copy -- check -- forget -- prune - -{{% notice tip %}} -You might notice that's the same sections that can also be scheduled -{{% /notice %}} - -Each of these commands can send 4 different types of hooks: - -- send-before -- send-after -- send-after-fail -- send-finally - -The configuration is the same for each of these 4 types of hooks: - -| Name | Required | Default | Notes | -|:-----|:---------|:--------|:------| -| url | Yes | None | URL of your Webhook | -| method | No | GET | This is the HTTP method (GET, POST, HEAD, etc.) | -| skip-tls-verification | No | False | **This is not recommended**: Use only if you're using your own server with a self-signed certificate | -| headers | No | User-Agent set to resticprofile | This is a subsection with a list of `name` and `value` | -| body | No | Empty | Used to send data to the Webhook (POST, PUT, PATCH) | -| body-template | No | None | Template file to generate the body (in go template format) | - - -### Example sending monitoring information to healthchecks.io: - -{{< tabs groupid="config-with-json" >}} -{{% tab title="toml" %}} - -```toml -version = "1" - -[profile] - - [profile.backup] - source = "/source" - exclude = [ "/**/.git/" ] - schedule = [ "*:00,30" ] - schedule-permission = "user" - - # you can have more than one target - - [[profile.backup.send-before]] - method = "HEAD" - url = "https://hc-ping.com/831e288e-1293-46f8-ac31-70ea7f875650/start" - - [[profile.backup.send-before]] - method = "HEAD" - url = "https://httpstat.us/400" - - # you can have more than one target - - [[profile.backup.send-after]] - method = "HEAD" - url = "https://hc-ping.com/831e288e-1293-46f8-ac31-70ea7f875650" - - [[profile.backup.send-after]] - method = "HEAD" - url = "https://httpstat.us/500" - - [profile.backup.send-after-fail] - method = "POST" - url = "https://hc-ping.com/831e288e-1293-46f8-ac31-70ea7f875650/fail" - body = "${ERROR}\n\n${ERROR_STDERR}" - - [[profile.backup.send-after-fail.headers]] - name = "Content-Type" - value = "text/plain; charset=UTF-8" - - [profile.check] - schedule = [ "*:15" ] - - [profile.check.send-before] - method = "HEAD" - url = "https://hc-ping.com/e0f62e41-b75f-450f-8cdd-7f25e466d2dc/start" - - [profile.check.send-after] - method = "HEAD" - url = "https://hc-ping.com/e0f62e41-b75f-450f-8cdd-7f25e466d2dc" - - [profile.retention] - after-backup = true - - -``` - -{{% /tab %}} -{{% tab title="yaml" %}} - -```yaml -version: "1" - -profile: - - backup: - source: "/source" - exclude: - - "/**/.git/" - schedule: - - "*:00,30" - schedule-permission: user - - # you can have more than one target - send-before: - - method: HEAD - url: https://hc-ping.com/831e288e-1293-46f8-ac31-70ea7f875650/start - - method: HEAD - url: https://httpstat.us/400 - - # you can have more than one target - send-after: - - method: HEAD - url: https://hc-ping.com/831e288e-1293-46f8-ac31-70ea7f875650 - - method: HEAD - url: https://httpstat.us/500 - - send-after-fail: - method: POST - url: https://hc-ping.com/831e288e-1293-46f8-ac31-70ea7f875650/fail - body: "${ERROR}\n\n${ERROR_STDERR}" - headers: - - name: Content-Type - value: "text/plain; charset=UTF-8" - check: - schedule: - - "*:15" - - send-before: - method: HEAD - url: https://hc-ping.com/e0f62e41-b75f-450f-8cdd-7f25e466d2dc/start - - send-after: - method: HEAD - url: https://hc-ping.com/e0f62e41-b75f-450f-8cdd-7f25e466d2dc - - retention: - after-backup: true - -``` - -{{% /tab %}} -{{% tab title="hcl" %}} - -```hcl -"profile" { - - "backup" = { - "source" = "/source" - "exclude" = ["/**/.git/"] - "schedule" = ["*:00,30"] - "schedule-permission" = "user" - - "send-before" = { - "method" = "HEAD" - "url" = "https://hc-ping.com/831e288e-1293-46f8-ac31-70ea7f875650/start" - } - - "send-before" = { - "method" = "HEAD" - "url" = "https://httpstat.us/400" - } - - "send-after" = { - "method" = "HEAD" - "url" = "https://hc-ping.com/831e288e-1293-46f8-ac31-70ea7f875650" - } - - "send-after" = { - "method" = "HEAD" - "url" = "https://httpstat.us/500" - } - - "send-after-fail" = { - "method" = "POST" - "url" = "https://hc-ping.com/831e288e-1293-46f8-ac31-70ea7f875650/fail" - "body" = "${ERROR}\n\n${ERROR_STDERR}" - "headers" = { - "name" = "Content-Type" - "value" = "text/plain; charset=UTF-8" - } - } - } - - "check" = { - "schedule" = ["*:15"] - - "send-before" = { - "method" = "HEAD" - "url" = "https://hc-ping.com/e0f62e41-b75f-450f-8cdd-7f25e466d2dc/start" - } - - "send-after" = { - "method" = "HEAD" - "url" = "https://hc-ping.com/e0f62e41-b75f-450f-8cdd-7f25e466d2dc" - } - } - - "retention" = { - "after-backup" = true - } -} -``` - -{{% /tab %}} -{{% tab title="json" %}} - -```json -{ - "version": "1", - "profile": { - "backup": { - "source": "/source", - "exclude": [ - "/**/.git/" - ], - "schedule": [ - "*:00,30" - ], - "schedule-permission": "user", - "send-before": [ - { - "method": "HEAD", - "url": "https://hc-ping.com/831e288e-1293-46f8-ac31-70ea7f875650/start" - }, - { - "method": "HEAD", - "url": "https://httpstat.us/400" - } - ], - "send-after": [ - { - "method": "HEAD", - "url": "https://hc-ping.com/831e288e-1293-46f8-ac31-70ea7f875650" - }, - { - "method": "HEAD", - "url": "https://httpstat.us/500" - } - ], - "send-after-fail": { - "method": "POST", - "url": "https://hc-ping.com/831e288e-1293-46f8-ac31-70ea7f875650/fail", - "body": "${ERROR}\n\n${ERROR_STDERR}", - "headers": [ - { - "name": "Content-Type", - "value": "text/plain; charset=UTF-8" - } - ] - } - }, - "check": { - "schedule": [ - "*:15" - ], - "send-before": { - "method": "HEAD", - "url": "https://hc-ping.com/e0f62e41-b75f-450f-8cdd-7f25e466d2dc/start" - }, - "send-after": { - "method": "HEAD", - "url": "https://hc-ping.com/e0f62e41-b75f-450f-8cdd-7f25e466d2dc" - } - }, - "retention": { - "after-backup": true - } - } -} -``` - -{{% /tab %}} -{{< /tabs >}} - - -A few environment variables will be available to construct the url and the body: -- `PROFILE_NAME` -- `PROFILE_COMMAND`: backup, check, forget, etc. - -Additionally, for the `send-after-fail` hooks, these environment variables will be available: -- `ERROR` containing the latest error message -- `ERROR_COMMANDLINE` containing the command line that failed -- `ERROR_EXIT_CODE` containing the exit code of the command line that failed -- `ERROR_STDERR` containing any message that the failed command sent to the standard error (stderr) - -URL encoding is applayed for variables `ERROR`, `ERROR_COMMANDLINE` and `ERROR_STDERR` if they are used in URL. - -The `send-finally` hooks are also getting the environment of `send-after-fail` when any previous operation has failed (except any `send` operation). - -Failures in any `send-*` are logged but do not influence environment or return code. - -### order of `send-*` - -Here's the flow of HTTP hooks: - -```mermaid -graph TD - LOCK(set resticprofile lock) - UNLOCK(delete resticprofile lock) - LOCK --> SB - SB('send-before') --> RUN - RUN(run restic command, or group of commands) - RUN -->|Success| SA - RUN -->|Error| SAF - SA('send-after') --> SF - SAF('send-after-fail') --> SF - SF('send-finally') - SF --> UNLOCK -``` - -{{% notice style="warning" title="resticprofile lock" %}} -The local resticprofile lock is surrounding the whole process. It means that the `run-after-fail` target is not called if the lock cannot be obtained. This is a limitation of the current implementation. -{{% /notice %}} - - -### body-template - -You can use a standard go template to build the webhook body. It has to be defined in a separate file (otherwise it would clash with the configuration as a go template itself). - -The object passed as an argument to the template is: - -- `ProfileName` **string** -- `ProfileCommand` **string** -- `Error` **ErrorContext** -- `Stdout` **string** - -The type **ErrorContext** is available after an error occurred (otherwise all fields are blank): -- `Message` **string** -- `CommandLine` **string** -- `ExitCode` **string** -- `Stderr` **string** - -Here's an example of a body file: - - -```json -{ - "profileName": "{{ .ProfileName }}", - "profileCommand": "{{ .ProfileCommand }}", - "exitCode": "{{ .Error.ExitCode }}" -} -``` - -The field `exitCode` will be blank if no error occured. - -And here's an example of a configuration using a body template: - - -{{< tabs groupid="config-with-json" >}} -{{% tab title="toml" %}} - -```toml -version = "1" - -[profile] - - [profile.backup] - source = "/source" - - [profile.backup.send-finally] - method = "POST" - url = "https://my/monitoring.example.com/" - body-template = "body-template.json" - - [[profile.backup.send-finally.headers]] - name = "Content-Type" - value = "application/json" - -``` - -{{% /tab %}} -{{% tab title="yaml" %}} - -```yaml -version: "1" - -profile: - - backup: - source: "/source" - - send-finally: - method: POST - url: https://my/monitoring.example.com/ - body-template: body-template.json - headers: - - name: Content-Type - value: "application/json" - - -``` - -{{% /tab %}} -{{% tab title="hcl" %}} - -```hcl -"profile" { - - "backup" = { - "source" = "/source" - - "send-finally" = { - "method" = "POST" - "url" = "https://my/monitoring.example.com/" - "body-template" = "body-template.json" - "headers" = { - "name" = "Content-Type" - "value" = "application/json" - } - } - } -} -``` - -{{% /tab %}} -{{% tab title="json" %}} - -```json -{ - "version": "1", - "profile": { - "backup": { - "source": "/source", - "send-finally": { - "method": "POST", - "url": "https://my/monitoring.example.com/", - "body-template": "body-template.json", - "headers": [ - { - "name": "Content-Type", - "value": "application/json" - } - ] - } - } - } -} -``` - -{{% /tab %}} -{{< /tabs >}} - -### Self-signed certificates - -If your monitoring system is using self-signed certificates, you can import them in resticprofile (and you don't need to rely on the `skip-tls-verification` flag) - -The parameter is in the `global` section and is called `ca-certificates`: it contains a list of certificate files (PEM). - -### timeout - -The default timeout for all HTTP requests is 30 seconds. - -You can change the default timeout in the `global` section with a parameter called `send-timeout`. - -The format is like: -- 30s -- 2m -- 1m20s - -### global configuration example - - -{{< tabs groupid="config-with-json" >}} -{{% tab title="toml" %}} - -```toml -version = "1" - -[global] - send-timeout = "10s" - ca-certificates = [ "ca-chain.cert.pem" ] -``` - - -{{% /tab %}} -{{% tab title="yaml" %}} - -```yaml -version: "1" - -global: - send-timeout: 10s - ca-certificates: - - ca-chain.cert.pem - -``` - -{{% /tab %}} -{{% tab title="hcl" %}} - -```hcl - -global { - send-timeout = "10s" - ca-certificates = "ca-chain.cert.pem" -} -``` - -{{% /tab %}} -{{% tab title="json" %}} - -```json -{ - "version": "1", - "global": { - "send-timeout": "10s", - "ca-certificates": [ - "ca-chain.cert.pem" - ] - } -} -``` - -{{% /tab %}} -{{< /tabs >}} diff --git a/docs/content/configuration/jsonschema/index.md b/docs/content/configuration/jsonschema/index.md deleted file mode 100644 index 94bf722e3..000000000 --- a/docs/content/configuration/jsonschema/index.md +++ /dev/null @@ -1,85 +0,0 @@ ---- -title: "JSON schema" -weight: 55 ---- - -JSON, YAML and TOML configuration files can benefit from a JSON schema that describes the -config structure depending on the selected *restic* and configuration file version. - -## Schema URL - -**{{< absolute "jsonschema/config.json" >}}** - -This schema detects config and restic version and redirects to the corresponding [versioned URL](#versioned-urls). -Detection requires full support for JSON Schema Draft 7. Use a versioned URL where not supported (e.g. TOML). - -## Usage (vscode) - -To use a schema with **vscode**, begin your config files with: - -{{< tabs groupid="config-with-json" >}} -{{% tab title="toml" %}} -``````toml -#:schema {{< absolute "jsonschema/config-2.json" nohtml >}} - -version = "2" - -`````` -{{% /tab %}} -{{% tab title="yaml" %}} -``````yaml -# yaml-language-server: $schema={{< absolute "jsonschema/config.json" nohtml >}} - -version: "2" - -`````` -{{% /tab %}} -{{% tab title="json" %}} -``````json -{ - "$schema": "{{< absolute "jsonschema/config.json" nohtml >}}", - "version": "2" -} -`````` -{{% /tab %}} -{{< /tabs >}} - -{{% notice style="info" %}} -YAML & TOML validation with JSON schema is not supported out of the box. Additional extensions are required, such as 'redhat.vscode-yaml' for YAML and 'tamasfe.even-better-toml' for TOML in Visual Studio Code. -{{% /notice %}} - -**Example** - -{{< figure src="/configuration/jsonschema/jsonschema-vsc.gif" >}} - -**Extension**: `redhat.vscode-yaml` - - -## Versioned URLs - -The following versioned schemas are referenced from `config.json`. Which URL is selected depends -on config and restic version. You may use the URLs directly if you need a self-contained schema -file or to enforce a certain version of configuration format and/or restic flags - -JSON schema URLs for **any** *restic* version: - -* Config V1: {{< absolute "jsonschema/config-1.json" >}} -* Config V2: {{< absolute "jsonschema/config-2.json" >}} - -These universal schemas contain all flags and commands of all known *restic* versions and -may allow to set flags that are not supported by a particular *restic* version. Descriptions -and deprecation markers indicate what is supported and what should no longer be used. - -JSON schema URLs for a **specific** *restic* version (list of [available URLs]({{% relref "reference/json-schema" %}})): - -* `.../config-1-restic-{MAJOR}-{MINOR}.json` -* `.../config-2-restic-{MAJOR}-{MINOR}.json` - -These schemas contain only flags and commands of a specific *restic* version. The schema will -validate a config only when flags are used that the selected *restic* version supports -according to its manual pages. - -{{% notice style="tip" %}} -As an alternative to URLs, schemas can be generated locally with: -`resticprofile generate --json-schema [--version RESTIC_VERSION] [global|v1|v2]` -{{% /notice %}} diff --git a/docs/content/configuration/logs.md b/docs/content/configuration/monitoring/_index.md similarity index 63% rename from docs/content/configuration/logs.md rename to docs/content/configuration/monitoring/_index.md index 1a5cc7ec9..b03117a68 100644 --- a/docs/content/configuration/logs.md +++ b/docs/content/configuration/monitoring/_index.md @@ -1,11 +1,14 @@ --- title: "Logs" -weight: 45 +weight: 50 +alwaysopen: false --- By default **resticprofile** will display all logs (from itself and **restic**) to the console. -You can redirect the logs to a local file, a temporary file or a syslog server. +You can redirect the logs to a local file, [a temporary file]({{% relref "/configuration/monitoring/tempfilelogging" %}}) or a syslog server. + +{{< toc >}} ## Destination @@ -157,78 +160,3 @@ If specified in different places, here's the priority order for the log destinat 2. `schedule-log` in the `profile` section 3. `log` in the `global` section 4. default to the console - -## Send logs to a temporary file - -This can be done by using the [template]({{% relref "/configuration/templates" %}}) function `tempFile`. - -This is to cover a special case when you want to upload the logs one by one to a remote location in a `run-finally` or a `run-after-fail` target. - -{{< tabs groupid="config-with-json" >}} -{{% tab title="toml" %}} - -```toml -version = "1" - -[backup_current] - [backup_current.backup] - verbose = true - no-error-on-warning = true - source = "{{ .CurrentDir }}" - schedule = "*:44" - schedule-log = '{{ tempFile "backup.log" }}' - run-finally = 'cp {{ tempFile "backup.log" }} /logs/backup{{ .Now.Format "2006-01-02T15-04-05" }}.log' -``` - -{{% /tab %}} -{{% tab title="yaml" %}} - -```yaml -version: "1" - -backup_current: - backup: - verbose: true - no-error-on-warning: true - source: "{{ .CurrentDir }}" - schedule: - - "*:44" - schedule-log: '{{ tempFile "backup.log" }}' - run-finally: 'cp {{ tempFile "backup.log" }} /logs/backup{{ .Now.Format "2006-01-02T15-04-05" }}.log' -``` - -{{% /tab %}} -{{% tab title="hcl" %}} - -```hcl -"profile" "backup" { - "verbose" = true - "no-error-on-warning" = true - "source" = "{{ .CurrentDir }}" - "schedule" = "*:44" - "schedule-log" = "{{ tempFile "backup.log" }}" - "run-finally" = "cp {{ tempFile "backup.log" }} /logs/backup{{ .Now.Format "2006-01-02T15-04-05" }}.log" -} -``` - -{{% /tab %}} -{{% tab title="json" %}} - -```json -{ - "version": "1", - "profile": { - "backup": { - "verbose": true, - "no-error-on-warning": true, - "source": "{{ .CurrentDir }}", - "schedule": "*:44", - "schedule-log": "{{ tempFile "backup.log" }}", - "run-finally": "cp {{ tempFile "backup.log" }} /logs/backup{{ .Now.Format "2006-01-02T15-04-05" }}.log" - } - } -} -``` - -{{% /tab %}} -{{< /tabs >}} diff --git a/docs/content/monitoring/prometheus.md b/docs/content/configuration/monitoring/prometheus.md similarity index 96% rename from docs/content/monitoring/prometheus.md rename to docs/content/configuration/monitoring/prometheus.md index bde4cf03f..bbb8e68ab 100644 --- a/docs/content/monitoring/prometheus.md +++ b/docs/content/configuration/monitoring/prometheus.md @@ -1,7 +1,7 @@ --- -title: "Prometheus" +title: "Monitoring - Prometheus" slug: prometheus -weight: 10 +weight: 51 tags: [ "monitoring" ] --- @@ -79,7 +79,7 @@ root: {{< /tabs >}} {{% notice style="note" %}} -Set `extended-status` to `true` to access all available metrics. For details, see [Extended status]({{% relref "/monitoring/status/index.html#-extended-status" %}}). +Set `extended-status` to `true` to access all available metrics. For details, see [Extended status]({{% relref "/configuration/monitoring/status#-extended-status" %}}). {{% /notice %}} Here's an example of a generated prometheus file: @@ -229,4 +229,4 @@ root: {{< /tabs >}} -This adds the `host` label to all your metrics. \ No newline at end of file +This adds the `host` label to all your metrics. diff --git a/docs/content/monitoring/status.md b/docs/content/configuration/monitoring/status.md similarity index 98% rename from docs/content/monitoring/status.md rename to docs/content/configuration/monitoring/status.md index de54d7145..42ad8d7df 100644 --- a/docs/content/monitoring/status.md +++ b/docs/content/configuration/monitoring/status.md @@ -1,7 +1,7 @@ --- -title: "Status file" +title: "Monitoring - Status file" slug: status -weight: 5 +weight: 52 tags: [ "monitoring" ] aliases: - /status diff --git a/docs/content/configuration/monitoring/tempfilelogging.md b/docs/content/configuration/monitoring/tempfilelogging.md new file mode 100644 index 000000000..51db561f5 --- /dev/null +++ b/docs/content/configuration/monitoring/tempfilelogging.md @@ -0,0 +1,77 @@ +--- +title: "Monitoring - Temporary File" +weight: 53 +--- + +This can be done by using the [template]({{% relref "/configuration/variables/templates" %}}) function `tempFile`. + +This is to cover a special case when you want to upload the logs one by one to a remote location in a `run-finally` or a `run-after-fail` target. + +{{< tabs groupid="config-with-json" >}} +{{% tab title="toml" %}} + +```toml +version = "1" + +[backup_current] + [backup_current.backup] + verbose = true + no-error-on-warning = true + source = "{{ .CurrentDir }}" + schedule = "*:44" + schedule-log = '{{ tempFile "backup.log" }}' + run-finally = 'cp {{ tempFile "backup.log" }} /logs/backup{{ .Now.Format "2006-01-02T15-04-05" }}.log' +``` + +{{% /tab %}} +{{% tab title="yaml" %}} + +```yaml +version: "1" + +backup_current: + backup: + verbose: true + no-error-on-warning: true + source: "{{ .CurrentDir }}" + schedule: + - "*:44" + schedule-log: '{{ tempFile "backup.log" }}' + run-finally: 'cp {{ tempFile "backup.log" }} /logs/backup{{ .Now.Format "2006-01-02T15-04-05" }}.log' +``` + +{{% /tab %}} +{{% tab title="hcl" %}} + +```hcl +"profile" "backup" { + "verbose" = true + "no-error-on-warning" = true + "source" = "{{ .CurrentDir }}" + "schedule" = "*:44" + "schedule-log" = "{{ tempFile "backup.log" }}" + "run-finally" = "cp {{ tempFile "backup.log" }} /logs/backup{{ .Now.Format "2006-01-02T15-04-05" }}.log" +} +``` + +{{% /tab %}} +{{% tab title="json" %}} + +```json +{ + "version": "1", + "profile": { + "backup": { + "verbose": true, + "no-error-on-warning": true, + "source": "{{ .CurrentDir }}", + "schedule": "*:44", + "schedule-log": "{{ tempFile "backup.log" }}", + "run-finally": "cp {{ tempFile "backup.log" }} /logs/backup{{ .Now.Format "2006-01-02T15-04-05" }}.log" + } + } +} +``` + +{{% /tab %}} +{{< /tabs >}} diff --git a/docs/content/configuration/warnings.md b/docs/content/configuration/monitoring/warnings.md similarity index 97% rename from docs/content/configuration/warnings.md rename to docs/content/configuration/monitoring/warnings.md index 8bfe1675f..4a5ab4b2f 100644 --- a/docs/content/configuration/warnings.md +++ b/docs/content/configuration/monitoring/warnings.md @@ -1,6 +1,6 @@ --- title: "Warnings" -weight: 30 +weight: 54 --- ## Warnings from restic @@ -64,4 +64,4 @@ profile: ``` {{% /tab %}} -{{< /tabs >}} \ No newline at end of file +{{< /tabs >}} diff --git a/docs/content/configuration/getting_started/TOML.png b/docs/content/configuration/profiles/TOML.png similarity index 100% rename from docs/content/configuration/getting_started/TOML.png rename to docs/content/configuration/profiles/TOML.png diff --git a/docs/content/configuration/getting_started/YAML.png b/docs/content/configuration/profiles/YAML.png similarity index 100% rename from docs/content/configuration/getting_started/YAML.png rename to docs/content/configuration/profiles/YAML.png diff --git a/docs/content/configuration/profiles/_index.md b/docs/content/configuration/profiles/_index.md new file mode 100644 index 000000000..6c30ef5d2 --- /dev/null +++ b/docs/content/configuration/profiles/_index.md @@ -0,0 +1,27 @@ +--- +title: Profiles +weight: 30 +alwaysopen: false +--- + +## Choose Your Favorite Format + +The **resticprofile** configuration file can be written in: +* [TOML](https://github.com/toml-lang/toml): configuration file with extension *.toml* or *.conf* +* [YAML](https://en.wikipedia.org/wiki/YAML): configuration file with extension *.yaml* +* [JSON](https://en.wikipedia.org/wiki/JSON): configuration file with extension *.json* +* [HCL](https://github.com/hashicorp/hcl): configuration file with extension *.hcl* + +We recommend using either TOML or YAML. + +JSON is suitable for auto-generated configurations but is not the easiest format for humans to read and write. + +HCL can be useful if you already use a tool from the Hashicorp stack; otherwise, it's another format to learn. + +## Debugging your template and variable expansion + +If for some reason you don't understand why resticprofile is not loading your configuration file, you can display the generated configuration after executing the template (and replacing the variables and everything) using the `--trace` flag. + +## More Information + +{{% children %}} diff --git a/docs/content/configuration/profiles/editorConfig.md b/docs/content/configuration/profiles/editorConfig.md new file mode 100644 index 000000000..36defc6b3 --- /dev/null +++ b/docs/content/configuration/profiles/editorConfig.md @@ -0,0 +1,21 @@ +--- +title: "Configuring your Editor" +--- + +We'll show you how to get documentation and auto-completion for the **resticprofile** configuration using [Visual Studio Code](https://code.visualstudio.com/). + +You can use any other editor that recognizes the [JSON schema]({{% relref "/reference/json-schema" %}}). The same JSON schema can be used for JSON, TOML, and YAML file formats. + +### TOML + +In Visual Studio Code, install an extension that supports completion and syntax validation using a JSON schema. + +For example, install the [Even Better TOML](https://marketplace.visualstudio.com/items?itemName=tamasfe.even-better-toml) extension: + +![TOML extension]({{< absolute "configuration/profiles/TOML.png" nohtml >}}) + +### YAML + +For YAML, install an extension like the one [provided by Red Hat](https://marketplace.visualstudio.com/items?itemName=redhat.vscode-yaml) in Visual Studio Code to understand the shape of your configuration file. + +![YAML extension]({{< absolute "configuration/profiles/YAML.png" nohtml >}}) diff --git a/docs/content/configuration/include.md b/docs/content/configuration/profiles/include.md similarity index 85% rename from docs/content/configuration/include.md rename to docs/content/configuration/profiles/include.md index b5cfaf4d1..a1ba80c79 100644 --- a/docs/content/configuration/include.md +++ b/docs/content/configuration/profiles/include.md @@ -1,6 +1,6 @@ --- title: "Includes" -weight: 15 +weight: 32 --- The configuration may be split into multiple files by adding `includes = "glob-pattern"` to the main configuration file. @@ -73,7 +73,7 @@ global { Included configuration files may use any supported format and settings are merged so that multiple files can extend the same profiles. The HCL format is special in that it cannot be mixed with other formats. -Within included files, the current [configuration path]({{% relref "path/#how-paths-inside-the-configuration-are-resolved" %}}) is not changed. Path resolution remains relative to the path of the main configuration file. +Within included files, the current [configuration path]({{% relref "/configuration/profiles/path/#how-paths-inside-the-configuration-are-resolved" %}}) is not changed. Path resolution remains relative to the path of the main configuration file. {{% notice style="note" %}} Included files cannot include nested files. Specifying `includes` inside an included file has no effect. @@ -81,7 +81,7 @@ Included files cannot include nested files. Specifying `includes` inside an incl ## Configuration Merging -Loading a configuration file involves loading the physical file from disk and applying all [variables]({{% relref "/configuration/variables" %}}) and [templates]({{% relref "/configuration/templates" %}}) prior to parsing the file in a supported format `hcl`, `json`, `toml` and `yaml`. This means [variables]({{% relref "variables" %}}) and [templates]({{% relref "/configuration/templates" %}}) must create valid configuration markup that can be parsed or loading will fail. +Loading a configuration file involves loading the physical file from disk and applying all [variables]({{% relref "/configuration/variables" %}}) and [templates]({{% relref "/configuration/variables/templates" %}}) prior to parsing the file in a supported format `hcl`, `json`, `toml` and `yaml`. This means [variables]({{% relref "/configuration/variables" %}}) and [templates]({{% relref "/configuration/variables/templates" %}}) must create valid configuration markup that can be parsed or loading will fail. Configuration files are loaded and applied in a fixed order: @@ -89,7 +89,7 @@ Configuration files are loaded and applied in a fixed order: 2. `includes` are iterated in declaration order: * Every item may be a single file path or glob expression * Glob expressions are resolved and iterated in alphabetical order - * All paths are resolved relative to [configuration path]({{% relref "/configuration/path/index.html#how-paths-inside-the-configuration-are-resolved" %}}) + * All paths are resolved relative to [configuration path]({{% relref "/configuration/profiles/path/#how-paths-inside-the-configuration-are-resolved" %}}) Configuration files are loaded in the following order when assuming `/etc/resticprofile/profiles.conf` with `includes = ["first.conf", "conf.d/*.conf", "last.conf"]`: ``` diff --git a/docs/content/configuration/inheritance.md b/docs/content/configuration/profiles/inheritance.md similarity index 90% rename from docs/content/configuration/inheritance.md rename to docs/content/configuration/profiles/inheritance.md index aef205326..ef1bb5a1f 100644 --- a/docs/content/configuration/inheritance.md +++ b/docs/content/configuration/profiles/inheritance.md @@ -1,6 +1,6 @@ --- title: "Inheritance" -weight: 16 +weight: 33 --- {{% notice style="tip" %}} @@ -9,11 +9,11 @@ You can use `resticprofile [.]show` (or `resticprofile [--name }} {{% tab title="yaml" %}} @@ -297,7 +297,7 @@ Every use object within the `use` list has the following structure: | `vars`: `` | Set mixin variable `$` | | `` | Set mixin variable `$` (short syntax) | -Mixins are applied to the configuration after processing all [includes]({{% relref "/configuration/include/index.html" %}}) but prior to [profile inheritance](#profile-inheritance) which means the `use` properties are not inherited but the result of applying `use` is inherited instead. What is defined by a mixin in a parent profile can still be overridden by a definition in a derived profile, but derived profiles can not change which mixins apply to their parent. +Mixins are applied to the configuration after processing all [includes]({{% relref "/configuration/profiles/include/index.html" %}}) but prior to [profile inheritance](#profile-inheritance) which means the `use` properties are not inherited but the result of applying `use` is inherited instead. What is defined by a mixin in a parent profile can still be overridden by a definition in a derived profile, but derived profiles can not change which mixins apply to their parent. List properties that have been inherited from a parent can be modified (append/prepend) and replaced by a mixin. diff --git a/docs/content/usage/locks.md b/docs/content/configuration/profiles/locks.md similarity index 99% rename from docs/content/usage/locks.md rename to docs/content/configuration/profiles/locks.md index 9f091f58e..25db603b7 100644 --- a/docs/content/usage/locks.md +++ b/docs/content/configuration/profiles/locks.md @@ -1,10 +1,8 @@ --- title: "Locks" -weight: 20 +weight: 34 --- - - restic is already using a lock to avoid running some operations at the same time. Since resticprofile can run several commands in a profile, it could be better to run the whole batch in a lock so nobody can interfere in the meantime. diff --git a/docs/content/configuration/path.md b/docs/content/configuration/profiles/path.md similarity index 99% rename from docs/content/configuration/path.md rename to docs/content/configuration/profiles/path.md index e823d7eac..16db4f485 100644 --- a/docs/content/configuration/path.md +++ b/docs/content/configuration/profiles/path.md @@ -1,6 +1,6 @@ --- title: "Path" -weight: 10 +weight: 31 --- ## How paths inside the configuration are resolved diff --git a/docs/content/configuration/priority.md b/docs/content/configuration/profiles/priority.md similarity index 100% rename from docs/content/configuration/priority.md rename to docs/content/configuration/profiles/priority.md diff --git a/docs/content/schedules/_index.md b/docs/content/configuration/schedules/_index.md similarity index 75% rename from docs/content/schedules/_index.md rename to docs/content/configuration/schedules/_index.md index d445e1ebe..361074e4e 100644 --- a/docs/content/schedules/_index.md +++ b/docs/content/configuration/schedules/_index.md @@ -1,19 +1,19 @@ +++ archetype = "chapter" -pre = "4. " title = "Schedules" -weight = 4 +weight = 40 +alwaysopen = false +++ ## Scheduler resticprofile manages scheduled backups using: -- **[launchd]({{% relref "/schedules/launchd" %}})** on macOS -- **[Task Scheduler]({{% relref "/schedules/task_scheduler" %}})** on Windows -- **[systemd]({{% relref "/schedules/systemd" %}})** on Linux and other BSDs -- **[crond]({{% relref "/schedules/cron" %}})** as a fallback (requires `crontab` binary) -- **[crontab]({{% relref "/schedules/cron" %}})** files (with or without a user column) +- **[launchd]({{% relref "./scheduling_methods/launchd" %}})** on macOS +- **[Task Scheduler]({{% relref "./scheduling_methods/task_scheduler" %}})** on Windows +- **[systemd]({{% relref "./scheduling_methods/systemd" %}})** on Linux and other BSDs +- **[crond]({{% relref "./scheduling_methods/cron" %}})** as a fallback (requires `crontab` binary) +- **[crontab]({{% relref "./scheduling_methods/cron" %}})** files (with or without a user column) On Unix systems (excluding macOS), resticprofile uses **systemd** if available, otherwise it falls back to **crond**. diff --git a/docs/content/schedules/commands.md b/docs/content/configuration/schedules/commands.md similarity index 98% rename from docs/content/schedules/commands.md rename to docs/content/configuration/schedules/commands.md index 90c387a62..3a0930e81 100644 --- a/docs/content/schedules/commands.md +++ b/docs/content/configuration/schedules/commands.md @@ -3,6 +3,8 @@ title: "Schedule Commands" weight: 20 --- +{{< toc >}} + resticprofile supports the following commands: - **schedule** - **unschedule** @@ -57,7 +59,7 @@ Starting with `v0.30.0`, the behavior of the `unschedule` command changed: Print the status of all installed schedules for the selected profiles. -The `status` command output varies by OS. See the [examples]({{% relref "/schedules/examples" %}}) for details. +The `status` command output varies by OS. See the [examples]({{% relref "/usage/scheduling" %}}) for details. ### run-schedule command diff --git a/docs/content/schedules/configuration.md b/docs/content/configuration/schedules/configuration.md similarity index 73% rename from docs/content/schedules/configuration.md rename to docs/content/configuration/schedules/configuration.md index 04af2472e..3e4725064 100644 --- a/docs/content/schedules/configuration.md +++ b/docs/content/configuration/schedules/configuration.md @@ -3,6 +3,7 @@ title: "Schedule Configuration" weight: 10 --- +{{< toc >}} The schedule configuration includes several parameters that can be added to each profile: @@ -81,15 +82,9 @@ profile: * *empty*: resticprofile will guess based on how it was started (with sudo or as a normal user). The fallback is `system` on Windows and `user_logged_on` on other platforms. -### Changing schedule-permission - -To change the permission of a schedule, unschedule the profile first. - -Follow this order: - -- Unschedule the job first (before updating the permission in the configuration) -- Change your permission (user to system, or system to user). -- Schedule your updated profile. +{{% notice style="warning" %}} +After changing this setting, you must update the schedule. More information can be found under [Changing Schedule Permissions]({{% relref "/usage/scheduling/change-schedule" %}}) +{{% /notice %}} ## schedule-run-level @@ -201,128 +196,16 @@ Note: It works only on Windows and makes sense only with `user_logged_on` permis Note: The behavior of `conhost.exe` varies between Windows versions. It has been confirmed to work on Windows 11 (24H2) but not on Windows 10 (1607). -## Example - -Here's an example of a scheduling configuration: - -{{< tabs groupid="config-with-json" >}} -{{% tab title="toml" %}} - -```toml -[default] - repository = "d:\\backup" - password-file = "key" - -[self] - inherit = "default" - - [self.retention] - after-backup = true - keep-within = "14d" - - [self.backup] - source = "." - schedule = [ "Mon..Fri *:00,15,30,45", "Sat,Sun 0,12:00" ] - schedule-permission = "user" - schedule-lock-wait = "10m" - - [self.prune] - schedule = "sun 3:30" - schedule-permission = "user" - schedule-lock-wait = "1h" -``` - -{{% /tab %}} -{{% tab title="yaml" %}} - -```yaml -default: - repository: "d:\\backup" - password-file: key - -self: - inherit: default - retention: - after-backup: true - keep-within: 14d - backup: - source: "." - schedule: - - "Mon..Fri *:00,15,30,45" # every 15 minutes on weekdays - - "Sat,Sun 0,12:00" # twice a day on week-ends - schedule-permission: user - schedule-lock-wait: 10m - prune: - schedule: "sun 3:30" - schedule-permission: user - schedule-lock-wait: 1h -``` +## Preventing System Sleep -{{% /tab %}} -{{% tab title="hcl" %}} - -```hcl -"default" = { - "repository" = "d:\\backup" - "password-file" = "key" -} +This feature is available for: +- macOS +- Windows +- Linux with systemd ([logind](https://www.freedesktop.org/software/systemd/man/systemd-logind.service.html)) -"self" = { - "inherit" = "default" +There's a `global` parameter called `prevent-sleep` that you can set to `true`, and resticprofile will prevent your system from idle sleeping. - "retention" = { - "after-backup" = true - "keep-within" = "14d" - } - - "backup" = { - "source" = "." - "schedule" = ["Mon..Fri *:00,15,30,45", "Sat,Sun 0,12:00"] - "schedule-permission" = "user" - "schedule-lock-wait" = "10m" - } - - "prune" = { - "schedule" = "sun 3:30" - "schedule-permission" = "user" - "schedule-lock-wait" = "1h" - } -} -``` - -{{% /tab %}} -{{% tab title="json" %}} - -```json -{ - "default": { - "repository": "d:\\backup", - "password-file": "key" - }, - "self": { - "inherit": "default", - "retention": { - "after-backup": true, - "keep-within": "14d" - }, - "backup": { - "source": ".", - "schedule": [ - "Mon..Fri *:00,15,30,45", - "Sat,Sun 0,12:00" - ], - "schedule-permission": "user", - "schedule-lock-wait": "10m" - }, - "prune": { - "schedule": "sun 3:30", - "schedule-permission": "user", - "schedule-lock-wait": "1h" - } - } -} -``` - -{{% /tab %}} -{{< /tabs >}} +Please note: +- it will not prevent a sleep if the system is running on batteries +- it will not prevent a sleep triggered by a user action: using the sleep button, closing the laptop lid, etc. diff --git a/docs/content/schedules/removable_volume.png b/docs/content/configuration/schedules/removable_volume.png similarity index 100% rename from docs/content/schedules/removable_volume.png rename to docs/content/configuration/schedules/removable_volume.png diff --git a/docs/content/configuration/schedules/scheduling_methods/_index.md b/docs/content/configuration/schedules/scheduling_methods/_index.md new file mode 100644 index 000000000..8d7e88ddd --- /dev/null +++ b/docs/content/configuration/schedules/scheduling_methods/_index.md @@ -0,0 +1,5 @@ +--- +title: "Scheduling Methods" +--- + +{{% children sort="weight" depth="2" %}} diff --git a/docs/content/schedules/cron.md b/docs/content/configuration/schedules/scheduling_methods/cron.md similarity index 100% rename from docs/content/schedules/cron.md rename to docs/content/configuration/schedules/scheduling_methods/cron.md diff --git a/docs/content/schedules/launchd.md b/docs/content/configuration/schedules/scheduling_methods/launchd.md similarity index 88% rename from docs/content/schedules/launchd.md rename to docs/content/configuration/schedules/scheduling_methods/launchd.md index e2c5dde17..c139bbfd1 100644 --- a/docs/content/schedules/launchd.md +++ b/docs/content/configuration/schedules/scheduling_methods/launchd.md @@ -19,10 +19,10 @@ You can wait for the profile to start or start it manually. To start a backup pr Once you grant permission, the profile will run normally until you update resticprofile or restic. This is a macOS limitation. -![access files popup window]({{< absolute "schedules/removable_volume.png" nohtml >}}) +![access files popup window]({{< absolute "configuration/schedules/removable_volume.png" nohtml >}}) ## System permission A launchd daemon is generated when you set `schedule-permission` to `system`. It consists of a `plist` file in `/Library/LaunchDaemons`. -Run resticprofile with sudo to `schedule` and `unschedule` the profile. You can schedule and unschedule system and user profiles simultaneously using the `schedule --all` command. \ No newline at end of file +Run resticprofile with sudo to `schedule` and `unschedule` the profile. You can schedule and unschedule system and user profiles simultaneously using the `schedule --all` command. diff --git a/docs/content/schedules/non-root-schedule-in-container.md b/docs/content/configuration/schedules/scheduling_methods/non-root-schedule-in-container.md similarity index 100% rename from docs/content/schedules/non-root-schedule-in-container.md rename to docs/content/configuration/schedules/scheduling_methods/non-root-schedule-in-container.md diff --git a/docs/content/schedules/systemd.md b/docs/content/configuration/schedules/scheduling_methods/systemd.md similarity index 98% rename from docs/content/schedules/systemd.md rename to docs/content/configuration/schedules/scheduling_methods/systemd.md index 352c6db22..a95515d7d 100644 --- a/docs/content/schedules/systemd.md +++ b/docs/content/configuration/schedules/scheduling_methods/systemd.md @@ -72,7 +72,7 @@ Setting the profile option `schedule-after-network-online: true` ensures schedul ## systemd drop-in files -You can automatically populate `*.conf.d` [drop-in files](https://www.freedesktop.org/software/systemd/man/latest/systemd-system.conf.html#main-conf) for profiles, allowing easy overrides of generated services without [modifying service templates]({{% relref "/schedules/systemd/#how-to-change-the-default-systemd-unit-and-timer-file-using-a-template" %}}). For example: +You can automatically populate `*.conf.d` [drop-in files](https://www.freedesktop.org/software/systemd/man/latest/systemd-system.conf.html#main-conf) for profiles, allowing easy overrides of generated services without [modifying service templates]({{% relref "#how-to-change-the-default-systemd-unit-and-timer-file-using-a-template" %}}). For example: {{< tabs groupid="config-with-json" >}} {{% tab title="toml" %}} diff --git a/docs/content/schedules/task_scheduler/administrator.png b/docs/content/configuration/schedules/scheduling_methods/task_scheduler/administrator.png similarity index 100% rename from docs/content/schedules/task_scheduler/administrator.png rename to docs/content/configuration/schedules/scheduling_methods/task_scheduler/administrator.png diff --git a/docs/content/schedules/task_scheduler/index.md b/docs/content/configuration/schedules/scheduling_methods/task_scheduler/index.md similarity index 79% rename from docs/content/schedules/task_scheduler/index.md rename to docs/content/configuration/schedules/scheduling_methods/task_scheduler/index.md index 597a8e526..c4aa022b6 100644 --- a/docs/content/schedules/task_scheduler/index.md +++ b/docs/content/configuration/schedules/scheduling_methods/task_scheduler/index.md @@ -15,7 +15,7 @@ Generally, you don't need to worry about this: resticprofile will restart itself ### resticprofile is blocked from restarting in elevated mode -![Unwanted software]({{< absolute "schedules/task_scheduler/unwanted_my_ass.png" nohtml >}}) +![Unwanted software](./unwanted_my_ass.png) I can't prevent this without buying a developer certificate. If you know any free or cheap certificate for open-source software, please let me know. @@ -26,11 +26,11 @@ You'll need to start an elevated shell yourself. - Right-click on `Command Prompt`, `Windows Terminal`, or `Windows Powershell` (choose your favorite) - Click on `Run as administrator` -![Start elevated command prompt]({{< absolute "schedules/task_scheduler/runas.png" nohtml >}}) +![Start elevated command prompt](./runas.png) It's easy to spot a terminal window opened with Administrator privileges: -![Administrator prefix]({{< absolute "schedules/task_scheduler/administrator.png" nohtml >}}) +![Administrator prefix](./administrator.png) > [!IMPORTANT] > Running the schedule command might cause Windows to delete _resticprofile.exe_, treating it as a threat. diff --git a/docs/content/schedules/task_scheduler/runas.png b/docs/content/configuration/schedules/scheduling_methods/task_scheduler/runas.png similarity index 100% rename from docs/content/schedules/task_scheduler/runas.png rename to docs/content/configuration/schedules/scheduling_methods/task_scheduler/runas.png diff --git a/docs/content/schedules/task_scheduler/unwanted_my_ass.png b/docs/content/configuration/schedules/scheduling_methods/task_scheduler/unwanted_my_ass.png similarity index 100% rename from docs/content/schedules/task_scheduler/unwanted_my_ass.png rename to docs/content/configuration/schedules/scheduling_methods/task_scheduler/unwanted_my_ass.png diff --git a/docs/content/configuration/sleep.md b/docs/content/configuration/sleep.md deleted file mode 100644 index 1bcc114f6..000000000 --- a/docs/content/configuration/sleep.md +++ /dev/null @@ -1,15 +0,0 @@ ---- -title: "Preventing system sleep" -weight: 40 ---- - -This feature is available for: -- macOS -- Windows -- Linux with systemd ([logind](https://www.freedesktop.org/software/systemd/man/systemd-logind.service.html)) - -There's a `global` parameter called `prevent-sleep` that you can set to `true`, and resticprofile will prevent your system from idle sleeping. - -Please note: -- it will not prevent a sleep if the system is running on batteries -- it will not prevent a sleep triggered by a user action: using the sleep button, closing the laptop lid, etc. diff --git a/docs/content/configuration/v2.md b/docs/content/configuration/v2.md deleted file mode 100644 index 933a90031..000000000 --- a/docs/content/configuration/v2.md +++ /dev/null @@ -1,109 +0,0 @@ ---- -title: "Configuration v2 proposal" -weight: 60 ---- - -{{% notice style="note" %}} - -The configuration file format `v2` is in preview right now. You can try to use it since `v0.17.0` but it's for testing and feedback only. - -{{% /notice %}} - - -## Introduction - -The current file format was decided at the time resticprofile was only using the `toml` format. Nesting pieces of configuration in blocks is not the easiest as you have to specify the whole path in the block: - -```toml -[profile] - -[profile.backup] -source = "some path" - -``` - -Since then, I believe the `yaml` format is preferred over `toml`. - -My proposal is to make a "version 2" of the configuration file, the current file format is "version 1". - -**Both formats will continue to be valid**: - -- if no version is specified, the "version 1" is used. This is the current format -- if a version is specified (`2`) the new format will be expected - -## New format availability - -The new format "version 2" will be available for: -- TOML -- YAML -- JSON - -**It won't be available for HCL**. This may not be definitive, but it's not widely used and it's becoming more and more difficult to support HCL. - -**HCL can still be used as is, "version = 1"** - -## New format specifications - -I will show the specification using the `yaml` as examples, because it's probably the most readable format. - -### version - -```yaml -version: 2 -``` - -### global - -The global section does not change. We'll keep all the global configuration in there. - -```yaml -global: - default-command: snapshots - initialize: false - priority: low - -``` - -### profiles - -All your profiles will be nested under a `profiles` section. Please note the schedules are no longer described inside the profile, but in a separate section `schedules` (see the following example). - -```yaml -version: "2" - -profiles: - default: - env: - tmp: /tmp - password-file: key - repository: /backup - - documents: - inherit: default - backup: - source: ~/Documents - snapshots: - tag: - - documents -``` - -### groups - -The list of profiles will be nested under a `profiles` section, so we can add more configuration to groups later. - -```yaml -version: "2" - -groups: - full: # name of your group - profiles: - - root - - documents - - mysql -``` - -This format leaves more space for improvements later (like a `repos` section maybe?) - -{{% notice style="tip" %}} -You can participate in designing the "version 2" [here](https://github.com/creativeprojects/resticprofile/issues/80) -{{% /notice %}} \ No newline at end of file diff --git a/docs/content/configuration/variables.md b/docs/content/configuration/variables.md deleted file mode 100644 index aaa0a66a5..000000000 --- a/docs/content/configuration/variables.md +++ /dev/null @@ -1,596 +0,0 @@ ---- -title: "Variables" -weight: 26 ---- - -## Variable expansion in configuration file - -You might want to reuse the same configuration (or bits of it) on different environments. One way of doing it is to create a generic configuration where specific bits can be replaced by a variable. - -### There are two kinds of variables: -- **template variables**: These variables are fixed once the full configuration file is loaded: [includes]({{% relref "configuration/include" %}}) are loaded, and [inheritance]({{% relref "/configuration/inheritance" %}}) is resolved. These variables are replaced by their value **before** the configuration is parsed. -- **runtime variables**: These variables are replaced by their value **after** the configuration is parsed. In other words: these variables are replaced by their value just before the command is executed. - -## Template variables -### Pre-defined variables - -The syntax for using a pre-defined variable is: - -``` -{{ .VariableName }} -``` - -The list of pre-defined variables is: - -| Variable | Type | Description | -|-------------------|--------------------------------------------------|------------------------------------------------------------------| -| **.Profile.Name** | string | Profile name | -| **.Now** | [time.Time](https://golang.org/pkg/time/) object | Now object: see explanation bellow | -| **.StartupDir** | string | Current directory at the time resticprofile was started | -| **.CurrentDir** | string | Current directory at the time a profile is executed | -| **.ConfigDir** | string | Directory where the configuration was loaded from | -| **.TempDir** | string | OS temporary directory (might not exist) | -| **.BinaryDir** | string | Directory where resticprofile was started from (since `v0.18.0`) | -| **.OS** | string | GOOS name: "windows", "linux", "darwin", etc. (since `v0.21.0`) | -| **.Arch** | string | GOARCH name: "386", "amd64", "arm64", etc. (since `v0.21.0`) | -| **.Hostname** | string | Host name | -| **.Env.{NAME}** | string | Environment variable `${NAME}` | - -Environment variables are accessible using `.Env.` followed by the (upper case) name of the environment variable. - -Example: `{{ .Env.HOME }}` will be replaced by your home directory (on unixes). The equivalent on Windows would be `{{ .Env.USERPROFILE }}`. - -Default and fallback values for an empty or unset variable can be declared with `{{ ... | or ... }}`. -For example `{{ .Env.HOME | or .Env.USERPROFILE | or "/fallback-homedir" }}` will try to resolve `$HOME`, if empty try to resolve `$USERPROFILE` -and finally default to `/fallback-homedir` if none of the env variables are defined. - -The variables `.OS` and `.Arch` are filled with the target platform that `resticprofile` was compiled for (see -[releases](https://github.com/creativeprojects/resticprofile/releases) for more information on existing precompiled platform binaries). - -For variables that are objects, you can call all public fields or methods on it. -For example, for the variable `.Now` ([time.Time](https://golang.org/pkg/time/)) you can use: - -- `(.Now.AddDate years months days)` -- `.Now.Day` -- `.Now.Format layout` -- `.Now.Hour` -- `.Now.Minute` -- `.Now.Month` -- `.Now.Second` -- `.Now.UTC` -- `.Now.Unix` -- `.Now.Weekday` -- `.Now.Year` -- `.Now.YearDay` - -Time can be formatted with `.Now.Format layout`, for example `{{ .Now.Format "2006-01-02T15:04:05Z07:00" }}` formats the current time as RFC3339 timestamp. -Check [time.Time#constants](https://pkg.go.dev/time#pkg-constants) for more layout examples. - -The variable `.Now` also allows to derive a relative `Time`. For example `{{ (.Now.AddDate 0 -6 -14).Format "2006-01-02" }}` formats a date that -is 6 months and 14 days before now. - - -#### Example - -You can use a combination of inheritance and variables in the resticprofile configuration file like so: - -{{< tabs groupid="config-with-json" >}} -{{% tab title="toml" %}} - -```toml -version = "1" - -[generic] - password-file = "{{ .ConfigDir }}/{{ .Profile.Name }}-key" - repository = "/backup/{{ .Now.Weekday }}" - lock = "$HOME/resticprofile-profile-{{ .Profile.Name }}.lock" - initialize = true - - [generic.backup] - check-before = true - exclude = [ "/**/.git" ] - exclude-caches = true - one-file-system = false - run-after = "echo All Done!" - run-before = [ - "echo Hello {{ .Env.LOGNAME }}", - "echo current dir: {{ .CurrentDir }}", - "echo config dir: {{ .ConfigDir }}", - "echo profile started at {{ .Now.Format "02 Jan 06 15:04 MST" }}" - ] - tag = [ "{{ .Profile.Name }}", "dev" ] - - [generic.retention] - after-backup = true - before-backup = false - compact = false - keep-within = "30d" - prune = true - tag = [ "{{ .Profile.Name }}", "dev" ] - - [generic.snapshots] - tag = [ "{{ .Profile.Name }}", "dev" ] - -[src] - inherit = "generic" - - [src.backup] - source = [ "{{ .Env.HOME }}/go/src" ] - - [src.check] - # Weekday is an integer from 0 to 6 (starting from Sunday) - # Nice trick to add 1 to an integer: https://stackoverflow.com/a/72465098 - read-data-subset = "{{ len (printf "a%*s" .Now.Weekday "") }}/7" - -``` - -{{% /tab %}} -{{% tab title="yaml" %}} - -```yaml ---- -version: "1" - -generic: - password-file: "{{ .ConfigDir }}/{{ .Profile.Name }}-key" - repository: "/backup/{{ .Now.Weekday }}" - lock: "$HOME/resticprofile-profile-{{ .Profile.Name }}.lock" - initialize: true - - backup: - check-before: true - exclude: - - /**/.git - exclude-caches: true - one-file-system: false - run-after: echo All Done! - run-before: - - "echo Hello {{ .Env.LOGNAME }}" - - "echo current dir: {{ .CurrentDir }}" - - "echo config dir: {{ .ConfigDir }}" - - "echo profile started at {{ .Now.Format "02 Jan 06 15:04 MST" }}" - tag: - - "{{ .Profile.Name }}" - - dev - - retention: - after-backup: true - before-backup: false - compact: false - keep-within: 30d - prune: true - tag: - - "{{ .Profile.Name }}" - - dev - - snapshots: - tag: - - "{{ .Profile.Name }}" - - dev - -src: - inherit: generic - - backup: - source: - - "{{ .Env.HOME }}/go/src" - - check: - # Weekday is an integer from 0 to 6 (starting from Sunday) - # Nice trick to add 1 to an integer: https://stackoverflow.com/a/72465098 - read-data-subset: "{{ len (printf "a%*s" .Now.Weekday "") }}/7" - -``` - -{{% /tab %}} -{{% tab title="hcl" %}} - -```hcl -"generic" = { - "password-file" = "{{ .ConfigDir }}/{{ .Profile.Name }}-key" - "repository" = "/backup/{{ .Now.Weekday }}" - "lock" = "$HOME/resticprofile-profile-{{ .Profile.Name }}.lock" - "initialize" = true - - "backup" = { - "check-before" = true - "exclude" = ["/**/.git"] - "exclude-caches" = true - "one-file-system" = false - "run-after" = "echo All Done!" - "run-before" = ["echo Hello {{ .Env.LOGNAME }}", "echo current dir: {{ .CurrentDir }}", "echo config dir: {{ .ConfigDir }}", "echo profile started at {{ .Now.Format "02 Jan 06 15:04 MST" }}"] - "tag" = ["{{ .Profile.Name }}", "dev"] - } - - "retention" = { - "after-backup" = true - "before-backup" = false - "compact" = false - "keep-within" = "30d" - "prune" = true - "tag" = ["{{ .Profile.Name }}", "dev"] - } - - "snapshots" = { - "tag" = ["{{ .Profile.Name }}", "dev"] - } -} - -"src" = { - "inherit" = "generic" - - "backup" = { - "source" = ["{{ .Env.HOME }}/go/src"] - } - - "check" = { - # Weekday is an integer from 0 to 6 (starting from Sunday) - # Nice trick to add 1 to an integer: https://stackoverflow.com/a/72465098 - "read-data-subset" = "{{ len (printf "a%*s" .Now.Weekday "") }}/7" - } -} -``` - -{{% /tab %}} -{{% tab title="json" %}} - -```json -{ - "version": "1", - "generic": { - "password-file": "{{ .ConfigDir }}/{{ .Profile.Name }}-key", - "repository": "/backup/{{ .Now.Weekday }}", - "lock": "$HOME/resticprofile-profile-{{ .Profile.Name }}.lock", - "initialize": true, - "backup": { - "check-before": true, - "exclude": [ - "/**/.git" - ], - "exclude-caches": true, - "one-file-system": false, - "run-after": "echo All Done!", - "run-before": [ - "echo Hello {{ .Env.LOGNAME }}", - "echo current dir: {{ .CurrentDir }}", - "echo config dir: {{ .ConfigDir }}", - "echo profile started at {{ .Now.Format "02 Jan 06 15:04 MST" }}" - ], - "tag": [ - "{{ .Profile.Name }}", - "dev" - ] - }, - "retention": { - "after-backup": true, - "before-backup": false, - "compact": false, - "keep-within": "30d", - "prune": true, - "tag": [ - "{{ .Profile.Name }}", - "dev" - ] - }, - "snapshots": { - "tag": [ - "{{ .Profile.Name }}", - "dev" - ] - } - }, - "src": { - "inherit": "generic", - "backup": { - "source": [ - "{{ .Env.HOME }}/go/src" - ] - }, - "check": { - "read-data-subset": "{{ len (printf "a%*s" .Now.Weekday "") }}/7" - } - } -} -``` - -{{% /tab %}} -{{< /tabs >}} - -This is obviously not a real world example, but it shows many of the possibilities you can do with variable expansion. - -To check the generated configuration, you can use the resticprofile `show` command: - -```shell -% resticprofile -c examples/template.yaml -n src show - -global: - default-command: snapshots - restic-lock-retry-after: 1m0s - restic-stale-lock-age: 2h0m0s - min-memory: 100 - send-timeout: 30s - -profile src: - repository: /backup/Monday - password-file: /Users/CP/go/src/resticprofile/examples/src-key - initialize: true - lock: /Users/CP/resticprofile-profile-src.lock - - backup: - check-before: true - run-before: echo Hello CP - echo current dir: /Users/CP/go/src/resticprofile - echo config dir: /Users/CP/go/src/resticprofile/examples - echo profile started at 05 Sep 22 17:39 BST - run-after: echo All Done! - source: /Users/CP/go/src - exclude: /**/.git - exclude-caches: true - tag: src - dev - - retention: - after-backup: true - keep-within: 30d - path: /Users/CP/go/src - prune: true - tag: src - dev - - check: - read-data-subset: 2/7 - - snapshots: - tag: src - dev -``` - -As you can see, the `src` profile inherited from the `generic` profile. The tags `{{ .Profile.Name }}` got replaced by the name of the current profile `src`. -Now you can reuse the same generic configuration in another profile. - -You might have noticed the `read-data-subset` in the `check` section which will read a seventh of the data every day, meaning the whole repository data will be checked over a week. You can find [more information about this trick](https://stackoverflow.com/a/72465098). - -### Hand-made variables - -You can also define variables yourself. Hand-made variables starts with a `$` ([PHP](https://en.wikipedia.org/wiki/PHP) anyone?) and get declared and assigned with the `:=` operator ([Pascal](https://en.wikipedia.org/wiki/Pascal_(programming_language)) anyone?). - -{{% notice style="info" %}} -You can only use double quotes `"` to declare the string, single quotes `'` are not allowed. You can also use backticks to declare the string. -{{% /notice %}} - -Here's an example: - -```yaml -# declare and assign a value to the variable -{{ $name := "something" }} - -profile: - # put the content of the variable here - tag: "{{ $name }}" -``` -{{% notice style="note" %}} -Variables are only valid in the file they are declared in. They cannot be shared in files loaded via `include`. -{{% /notice %}} - -Variables can be redefined using the `=` operator. The new value will be used from the point of redefinition to the end of the file. - -```yaml -# declare and assign a value to the variable -{{ $name := "something" }} - -# reassign a new value to the variable -{{ $name = "something else" }} - -``` - -#### Windows path inside a variable - -Windows path are using backslashes `\` and are interpreted as escape characters in the configuration file. To use a Windows path inside a variable, you have a few options: -- you can escape the backslashes with another backslash. -- you can use forward slashes `/` instead of backslashes. Windows is able to use forward slashes in paths. -- you can use the backtick to declare the string instead of a double quote. - -For example: -```yaml -# double backslash -{{ $path := "C:\\Users\\CP\\Documents" }} -# forward slash -{{ $path := "C:/Users/CP/Documents" }} -# backticks -{{ $path := `C:\Users\CP\Documents` }} -``` - -#### Example - -Here's an example of a configuration on Linux where I use a variable `$mountpoint` set to a USB drive mount point: - -{{< tabs groupid="config-with-json" >}} -{{% tab title="toml" %}} - -```toml -version = "1" - -[global] - priority = "low" - -{{ $mountpoint := "/mnt/external" }} - -[default] - repository = "local:{{ $mountpoint }}/backup" - password-file = "key" - run-before = "mount {{ $mountpoint }}" - run-after = "umount {{ $mountpoint }}" - run-after-fail = "umount {{ $mountpoint }}" - - [default.backup] - exclude-caches = true - source = [ "/etc", "/var/lib/libvirt" ] - check-after = true -``` - -{{% /tab %}} -{{% tab title="yaml" %}} - -```yaml -version: "1" - -global: - priority: low - -{{ $mountpoint := "/mnt/external" }} - -default: - repository: 'local:{{ $mountpoint }}/backup' - password-file: key - run-before: 'mount {{ $mountpoint }}' - run-after: 'umount {{ $mountpoint }}' - run-after-fail: 'umount {{ $mountpoint }}' - - backup: - exclude-caches: true - source: - - /etc - - /var/lib/libvirt - check-after: true -``` - -{{% /tab %}} -{{% tab title="hcl" %}} - -```hcl -global { - priority = "low" -} - -{{ $mountpoint := "/mnt/external" }} - -default { - repository = "local:{{ $mountpoint }}/backup" - password-file = "key" - run-before = "mount {{ $mountpoint }}" - run-after = "umount {{ $mountpoint }}" - run-after-fail = "umount {{ $mountpoint }}" - - backup { - exclude-caches = true - source = [ "/etc", "/var/lib/libvirt" ] - check-after = true - } -} - -``` - -{{% /tab %}} -{{% tab title="json" %}} - -```json -{{ $mountpoint := "/mnt/external" }} -{ - "version": "1", - "global": { - "priority": "low" - }, - "default": { - "repository": "local:{{ $mountpoint }}/backup", - "password-file": "key", - "run-before": "mount {{ $mountpoint }}", - "run-after": "umount {{ $mountpoint }}", - "run-after-fail": "umount {{ $mountpoint }}", - "backup": { - "exclude-caches": true, - "source": [ - "/etc", - "/var/lib/libvirt" - ], - "check-after": true - } - } -} -``` - -{{% /tab %}} -{{< /tabs >}} - - -## Runtime variable expansion - -Variable expansion as described in the previous section using the `{{ .Var }}` syntax refers to [template variables]({{% relref "/configuration/templates" %}}) that are expanded prior to parsing the configuration file. -This means they must be used carefully to create correct config markup, but they are also very flexible. - -There is also unix style variable expansion using the `${variable}` or `$variable` syntax on configuration **values** that expand after the config file was parsed. Values that take a file path or path expression and a few others support this expansion. - -If not specified differently, these variables resolve to the corresponding environment variable or to an empty value if no such environment variable exists. Exceptions are [mixins]({{% relref "/configuration/inheritance#mixins" %}}) where `$variable` style is used for parametrisation and the profile [config flag]({{% relref "reference/profile" %}}) `prometheus-push-job`. - -### Example - -Backup current dir (`$PWD`) but prevent backup of `$HOME` where the repository is located: - -{{< tabs groupid="config-with-json" >}} -{{% tab title="toml" %}} - -```toml -version = "1" - -[default] - repository = "local:${HOME}/backup" - password-file = "${HOME}/backup.key" - - [default.backup] - source = "$PWD" - exclude = ["$HOME/**", ".*", "~*"] - -``` - -{{% /tab %}} -{{% tab title="yaml" %}} - -```yaml -version: "1" - -default: - repository: 'local:${HOME}/backup' - password-file: '${HOME}/backup.key' - - backup: - source: '$PWD' - exclude: ['$HOME/**', '.*', '~*'] - -``` - -{{% /tab %}} -{{% tab title="hcl" %}} - -```hcl -default { - repository = "local:${HOME}/backup" - password-file = "${HOME}/backup.key" - - backup { - source = [ "$PWD" ] - exclude = [ "$HOME/**", ".*", "~*" ] - } -} -``` - -{{% /tab %}} -{{% tab title="json" %}} - -```json -{ - "default": { - "repository": "local:${HOME}/backup", - "password-file": "${HOME}/backup.key", - "backup": { - "source": [ "$PWD" ], - "exclude": [ "$HOME/**", ".*", "~*" ] - } - } -} -``` - -{{% /tab %}} -{{< /tabs >}} - -{{% notice style="tip" %}} -Use `$$` to escape a single `$` in configuration values that support variable expansion. E.g. on Windows you might want to exclude `$RECYCLE.BIN`. Specify it as: `exclude = ["$$RECYCLE.BIN"]`. -{{% /notice %}} diff --git a/docs/content/configuration/variables/_index.md b/docs/content/configuration/variables/_index.md new file mode 100644 index 000000000..2d326e71b --- /dev/null +++ b/docs/content/configuration/variables/_index.md @@ -0,0 +1,119 @@ +--- +title: "Variables" +weight: 31 +alwaysopen: false +--- + +{{< toc >}} + +## Variable expansion in configuration file + +You might want to reuse the same configuration (or bits of it) on different environments. One way of doing it is to create a generic configuration where specific bits can be replaced by a variable. + +There are two kinds of variables: +- **template variables**: These variables are fixed once the full configuration file is loaded: [includes]({{% relref "/configuration/profiles/include" %}}) are loaded, and [inheritance]({{% relref "/configuration/profiles/inheritance" %}}) is resolved. These variables are replaced by their value **before** the configuration is parsed. +- **runtime variables**: These variables are replaced by their value **after** the configuration is parsed. In other words: these variables are replaced by their value just before the command is executed. + +## Template variables + +The syntax for using a pre-defined variable is: + +``` +{{ .VariableName }} +``` + +The list of pre-defined variables and environment variables can be found in the [reference]({{% relref "/configuration/variables/env_vars" %}}) + +Environment variables are accessible using `.Env.` followed by the (upper case) name of the environment variable. + +Default and fallback values for an empty or unset variable can be declared with `{{ ... | or ... }}`. +For example `{{ .Env.HOME | or .Env.USERPROFILE | or "/fallback-homedir" }}` will try to resolve `$HOME`, if empty try to resolve `$USERPROFILE` +and finally default to `/fallback-homedir` if none of the env variables are defined. + +The variables `.OS` and `.Arch` are filled with the target platform that `resticprofile` was compiled for (see +[releases](https://github.com/creativeprojects/resticprofile/releases) for more information on existing precompiled platform binaries). + +For variables that are objects, you can call all public fields or methods on it. +For example, for the variable `.Now` ([time.Time](https://golang.org/pkg/time/)) you can use: + +- `(.Now.AddDate years months days)` +- `.Now.Day` +- `.Now.Format layout` +- `.Now.Hour` +- `.Now.Minute` +- `.Now.Month` +- `.Now.Second` +- `.Now.UTC` +- `.Now.Unix` +- `.Now.Weekday` +- `.Now.Year` +- `.Now.YearDay` + +Time can be formatted with `.Now.Format layout`, for example `{{ .Now.Format "2006-01-02T15:04:05Z07:00" }}` formats the current time as RFC3339 timestamp. +Check [time.Time#constants](https://pkg.go.dev/time#pkg-constants) for more layout examples. + +The variable `.Now` also allows to derive a relative `Time`. For example `{{ (.Now.AddDate 0 -6 -14).Format "2006-01-02" }}` formats a date that +is 6 months and 14 days before now. + +## Hand-made variables + +You can also define variables yourself. Hand-made variables starts with a `$` ([PHP](https://en.wikipedia.org/wiki/PHP) anyone?) and get declared and assigned with the `:=` operator ([Pascal](https://en.wikipedia.org/wiki/Pascal_(programming_language)) anyone?). + +{{% notice style="info" %}} +You can only use double quotes `"` to declare the string, single quotes `'` are not allowed. You can also use backticks to declare the string. +{{% /notice %}} + +Here's an example: + +```yaml +# declare and assign a value to the variable +{{ $name := "something" }} + +profile: + # put the content of the variable here + tag: "{{ $name }}" +``` +{{% notice style="note" %}} +Variables are only valid in the file they are declared in. They cannot be shared in files loaded via `include`. +{{% /notice %}} + +Variables can be redefined using the `=` operator. The new value will be used from the point of redefinition to the end of the file. + +```yaml +# declare and assign a value to the variable +{{ $name := "something" }} + +# reassign a new value to the variable +{{ $name = "something else" }} + +``` + +### Windows path inside a variable + +Windows path are using backslashes `\` and are interpreted as escape characters in the configuration file. To use a Windows path inside a variable, you have a few options: +- you can escape the backslashes with another backslash. +- you can use forward slashes `/` instead of backslashes. Windows is able to use forward slashes in paths. +- you can use the backtick to declare the string instead of a double quote. + +For example: +```yaml +# double backslash +{{ $path := "C:\\Users\\CP\\Documents" }} +# forward slash +{{ $path := "C:/Users/CP/Documents" }} +# backticks +{{ $path := `C:\Users\CP\Documents` }} +``` + +## Runtime variable expansion + +Variable expansion as described in the previous section using the `{{ .Var }}` syntax refers to [template variables]({{% relref "/configuration/variables/templates" %}}) that are expanded prior to parsing the configuration file. +This means they must be used carefully to create correct config markup, but they are also very flexible. + +There is also unix style variable expansion using the `${variable}` or `$variable` syntax on configuration **values** that expand after the config file was parsed. Values that take a file path or path expression and a few others support this expansion. + +If not specified differently, these variables resolve to the corresponding environment variable or to an empty value if no such environment variable exists. Exceptions are [mixins]({{% relref "/configuration/profiles/inheritance#mixins" %}}) where `$variable` style is used for parametrisation and the profile [config flag]({{% relref "reference/profile" %}}) `prometheus-push-job`. + +{{% notice style="tip" %}} +Use `$$` to escape a single `$` in configuration values that support variable expansion. E.g. on Windows you might want to exclude `$RECYCLE.BIN`. Specify it as: `exclude = ["$$RECYCLE.BIN"]`. +{{% /notice %}} diff --git a/docs/content/configuration/variables/env_vars.md b/docs/content/configuration/variables/env_vars.md new file mode 100644 index 000000000..3415fea5f --- /dev/null +++ b/docs/content/configuration/variables/env_vars.md @@ -0,0 +1,60 @@ +--- +title: Variable Reference +weight: 7 +--- + +## Pre-defined Variables + + +| Variable | Type | Description | +|-------------------|--------------------------------------------------|------------------------------------------------------------------| +| **.Profile.Name** | string | Profile name | +| **.Now** | [time.Time](https://golang.org/pkg/time/) object | Now object: see explanation bellow | +| **.StartupDir** | string | Current directory at the time resticprofile was started | +| **.CurrentDir** | string | Current directory at the time a profile is executed | +| **.ConfigDir** | string | Directory where the configuration was loaded from | +| **.TempDir** | string | OS temporary directory (might not exist) | +| **.BinaryDir** | string | Directory where resticprofile was started from (since `v0.18.0`) | +| **.OS** | string | GOOS name: "windows", "linux", "darwin", etc. (since `v0.21.0`) | +| **.Arch** | string | GOARCH name: "386", "amd64", "arm64", etc. (since `v0.21.0`) | +| **.Hostname** | string | Host name | +| **.Env.{NAME}** | string | Environment variable `${NAME}` (see below) | + + +## Environment Variables + +Example: `{{ .Env.HOME }}` will be replaced by your home directory (on unixes). The equivalent on Windows would be `{{ .Env.USERPROFILE }}`. + +Most flags for resticprofile can be set using environment variables. If both are specified, command line flags take the precedence. + +| Flag | Environment variable | Built-In default | +|-----------------------|-----------------------------------|------------------| +| `--quiet` | `RESTICPROFILE_QUIET` | `false` | +| `--verbose` | `RESTICPROFILE_VERBOSE` | `false` | +| `--trace` | `RESTICPROFILE_TRACE` | `false` | +| `--config` | `RESTICPROFILE_CONFIG` | `"profiles"` | +| `--format` | `RESTICPROFILE_FORMAT` | `""` | +| `--name` | `RESTICPROFILE_NAME` | `"default"` | +| `--log` | `RESTICPROFILE_LOG` | `""` | +| `--command-output` | `RESTICPROFILE_COMMAND_OUTPUT` | `"auto"` | +| `--dry-run` | `RESTICPROFILE_DRY_RUN` | `false` | +| `--no-lock` | `RESTICPROFILE_NO_LOCK` | `false` | +| `--lock-wait` | `RESTICPROFILE_LOCK_WAIT` | `0` | +| `--stderr` | `RESTICPROFILE_STDERR` | `false` | +| `--no-ansi` | `RESTICPROFILE_NO_ANSI` | `false` | +| `--theme` | `RESTICPROFILE_THEME` | `"light"` | +| `--no-priority` | `RESTICPROFILE_NO_PRIORITY` | `false` | +| `--wait` | `RESTICPROFILE_WAIT` | `false` | +| `--ignore-on-battery` | `RESTICPROFILE_IGNORE_ON_BATTERY` | `0` | + +### Other environment variables + +| Environment Variable | Default | Purpose | +|---------------------------------|---------|--------------------------------------------------------------------------------------| +| `RESTICPROFILE_PWSH_NO_AUTOENV` | _empty_ | Disables powershell script pre-processing that converts unset `$VAR` into `$Env:VAR` | + +### Environment variables set by resticprofile + +| Environment Variable | Example | When | +|-----------------------------|--------------------------------|-------------------------------------| +| `RESTICPROFILE_SCHEDULE_ID` | `profiles.yaml:backup@profile` | Set when running scheduled commands | diff --git a/docs/content/configuration/templates.md b/docs/content/configuration/variables/templates.md similarity index 98% rename from docs/content/configuration/templates.md rename to docs/content/configuration/variables/templates.md index 5b1164f42..7cd5bcfc0 100644 --- a/docs/content/configuration/templates.md +++ b/docs/content/configuration/variables/templates.md @@ -410,10 +410,6 @@ azure-mysql: {{< /tabs >}} -## Debugging your template and variable expansion - -If for some reason you don't understand why resticprofile is not loading your configuration file, you can display the generated configuration after executing the template (and replacing the variables and everything) using the `--trace` flag. We will see it in action in a moment. - ## Limitations of using templates There's something to be aware of when dealing with templates: at the time the template is compiled, it has no knowledge of what the end configuration should look like: it has no knowledge of profiles for example. Here is a **non-working** example of what I mean: diff --git a/docs/content/contributions/_index.md b/docs/content/contributions/_index.md index 438b951a0..5aec20fca 100644 --- a/docs/content/contributions/_index.md +++ b/docs/content/contributions/_index.md @@ -1,6 +1,6 @@ +++ archetype = "chapter" -pre = "7. " +pre = "6. " title = "Contributions" weight = 7 +++ diff --git a/docs/content/faq.md b/docs/content/faq.md new file mode 100644 index 000000000..c63787d7c --- /dev/null +++ b/docs/content/faq.md @@ -0,0 +1,38 @@ +--- +title: FAQ +pre: "4. " +weight: 4 +--- + +{{< toc >}} + + +## Installation +### Shell Completion + +To generate the shell completion script, use: + +```shell +# bash +eval "$(resticprofile generate --bash-completion)" + +# zsh +eval "$(resticprofile generate --zsh-completion)" +``` + +### My homebrew install failed! + +Homebrew appears to need access to a compiler. If the installation fails, you may need to install gcc. + +## Docker + +### How do I schedule using Docker Compose? + +There's an example in the contribution section how to schedule backups in a long running container. +The configuration needs to specify `crond` as a scheduler. + +## Usage + +### How do I prevent my system from going into sleep mode? + +Please refer to [preventing system sleep]({{% relref "/configuration/schedules/configuration/#preventing-system-sleep" %}}). diff --git a/docs/content/features/_index.md b/docs/content/features/_index.md new file mode 100644 index 000000000..d2ebb1e81 --- /dev/null +++ b/docs/content/features/_index.md @@ -0,0 +1,38 @@ +--- +title: Features +weight: 1 +description: Features of resticprofile +--- + +Here is a non-exhaustive list of what resticprofile offers: + +* **Profiles** + * No need to remember command parameters and environment variables + * Create multiple profiles in one configuration file + * Profiles can inherit options from other profiles + * Create groups of profiles to run sequentially + * Easily [schedule]({{% relref "/configuration/schedules" %}}) backups, retentions, and checks (supports *systemd*, *crond*, *launchd*, and *Windows Task Scheduler*) + * Use [template syntax]({{% relref "/configuration/variables/templates" %}}) in your configuration file + * **[new for v0.29.0]** Schedule a group of profiles (configuration `v2` only) +* **Automation** + * Run the forget command before or after a backup (in a section called *retention*) + * Check a repository before or after a backup + * Run [shell commands]({{% relref "/configuration/hooks/run_hooks" %}}) before or after running a profile, useful for mounting and unmounting backup disks + * Run a [shell command]({{% relref "/configuration/hooks/run_hooks" %}}) if an error occurs + * Send a backup stream via _stdin_ + * Start restic at different [priorities]({{% relref "/configuration/profiles/priority" %}}) (Priority Class in Windows, *nice* in Unix, and/or _ionice_ in Linux) + * Automatically clear [stale locks]({{% relref "/configuration/profiles/locks" %}}) +* **Monitoring** + * Generate a simple [status file]({{% relref "/configuration/monitoring/status" %}}) for monitoring software to ensure backups are running smoothly + * Export a [prometheus]({{% relref "/configuration/monitoring/prometheus" %}}) file after a backup or send the report to a push gateway + * Run shell commands in the background when non-fatal errors are detected + * Send messages to [HTTP hooks]({{% relref "/configuration/hooks/http_hooks" %}}) before, after a successful or failed job (backup, forget, check, prune, copy) + * Send resticprofile [logs]({{% relref "/configuration/monitoring/index" %}}) to a syslog server +* **Checks** + * Check for [enough memory]({{% relref "/installation/#note-about-memory" %}}) before starting a backup + * Avoid scheduling a job when the system is on battery +* **Misc** + * Generate cryptographically secure random keys for a restic [key file]({{% relref "/usage/keyfile" %}}) + * Automatically [initialize the secondary repository]({{% relref "/configuration/hooks/copy" %}}) using the `copy-chunker-params` flag + * Prevent the system from [idle sleeping]({{% relref "/configuration/schedules/configuration/#preventing-system-sleep" %}}) + * View help for both restic and resticprofile via the `help` command or `-h` flag diff --git a/docs/content/installation/_index.md b/docs/content/installation/_index.md index 7eacf4617..2567e7420 100644 --- a/docs/content/installation/_index.md +++ b/docs/content/installation/_index.md @@ -5,4 +5,16 @@ title = "Installation" weight = 1 +++ +Here are the multiple ways you can install resticprofile: {{% children %}} + +## Upgrading +After installation, you can run `resticprofile self-update` to update resticprofile to the latest release. The `-q` or `--quiet` flag updates the application without prompting. + +## Note about Memory + +restic can be memory hungry. I'm running a few servers with no swap and I managed to kill some of them during a backup. + +For that matter I've introduced a parameter in the `global` section called `min-memory`. The **default value is 100MB**. You can disable it by using a value of `0`. + +It compares against `(total - used)` which is probably the best way to know how much memory is available (that is including the memory used for disk buffers/cache). diff --git a/docs/content/installation/ansible/index.md b/docs/content/installation/ansible/index.md index b8f8fb13d..b232e2e6a 100644 --- a/docs/content/installation/ansible/index.md +++ b/docs/content/installation/ansible/index.md @@ -1,14 +1,16 @@ --- title: "Ansible" -weight: 15 +weight: 16 --- +{{% notice style="caution" %}} +This is a work in progress documentation. +{{% /notice %}} + ## Installation using Ansible Installation using Ansible is not streamlined, but here's the playbook I'm using on my servers: - - {{% notice style="secondary" title="Playbooks" icon="paperclip" %}} * [resticprofile.yml]({{< absolute "installation/ansible/resticprofile.yml" nohtml >}}) {{% /notice %}} diff --git a/docs/content/installation/binaries.md b/docs/content/installation/binaries.md index f4a6cad72..bcf468af2 100644 --- a/docs/content/installation/binaries.md +++ b/docs/content/installation/binaries.md @@ -7,29 +7,34 @@ Pre-built binaries for resticprofile are available for various operating system You can download the latest versions from the [releases page](https://github.com/creativeprojects/resticprofile/releases). ## Pre-built binaries available: -- darwin/amd64 -- darwin/arm64 -- freebsd/386 -- freebsd/amd64 -- freebsd/armv6 -- freebsd/armv7 -- linux/386 -- linux/amd64 -- linux/arm64 -- linux/armv6 -- linux/armv7 -- linux/mips64 hardfloat -- linux/mips64le hardfloat -- linux/mips hardfloat -- linux/mipsle hardfloat -- linux/ppc64 -- linux/ppc64le -- linux/riscv64 -- linux/s390x +- windows + - 386 + - amd64 + - arm64 +- linux + - 386 + - amd64 + - arm64 + - armv6 + - armv7 + - mips64 hardfloat + - mips64le hardfloat + - mips hardfloat + - mipsle hardfloat + - ppc64 + - ppc64le + - riscv64 + - s390x +- darwin + - amd64 + - arm64 +- freebsd + - 386 + - amd64 + - armv6 + - armv7 +- openbsd + - 386 + - amd64 - netbsd/amd64 -- openbsd/386 -- openbsd/amd64 - solaris/amd64 -- windows/386 -- windows/amd64 -- windows/arm64 🆕 diff --git a/docs/content/installation/docker.md b/docs/content/installation/docker.md index 23c02e3d0..c9381f345 100644 --- a/docs/content/installation/docker.md +++ b/docs/content/installation/docker.md @@ -1,28 +1,25 @@ --- title: "Docker" -weight: 30 +weight: 15 --- +Docker images are available at `creativeprojects/resticprofile:latest` and `ghcr.io/creativeprojects/resticprofile:latest` +To run an instance of resticprofile, run: +```shell +docker run -it --rm -v $PWD/examples:/resticprofile ghcr.io/creativeprojects/resticprofile +``` -## Using resticprofile from a docker image ## - -You can run resticprofile in a Docker container, which is the easiest way to install and update resticprofile and restic simultaneously. - -However, you must mount your backup source (and local destination, if applicable) as a Docker volume. On macOS, backups may be slower due to the known performance issues with mounted volumes. +{{% notice style="warning" %}} +- You must mount your backup source (and local destination, if applicable) as a Docker volume. +- On macOS, backups may be slower due to the known performance issues with mounted volumes. +{{% /notice %}} -### Registries +Resticprofile's docker images also include restic and rclone. -The official image is available on docker hub -```shell -docker pull creativeprojects/resticprofile:latest -``` -as well as on Github Container Registry -```shell -docker pull ghcr.io/creativeprojects/resticprofile:latest -``` +{{< toc >}} -### Configuration +## Configuration By default, the resticprofile container starts at `/resticprofile`. So you can feed a configuration this way: @@ -30,45 +27,16 @@ By default, the resticprofile container starts at `/resticprofile`. So you can f docker run -it --rm -v $PWD/examples:/resticprofile ghcr.io/creativeprojects/resticprofile ``` -You can list your profiles: +## List profiles + ```shell docker run -it --rm -v $PWD/examples:/resticprofile ghcr.io/creativeprojects/resticprofile profiles ``` -### Container host name - -Each time a container starts, it is assigned a random name. +## Container host name To set a specific hostname, use the `-h` or `--hostname` flag with `docker run`: ```shell docker run -it --rm -v $PWD:/resticprofile -h my-hostname ghcr.io/creativeprojects/resticprofile -n profile backup ``` - -### Platforms - -The resticprofile docker image is available in these 2 platforms: -- linux/amd64 -- linux/arm64/v8 (compatible with raspberry pi 64bits) - -### rclone - -The resticprofile docker image also includes the latest version of [rclone][1]. - -### Container imager release cycle - -The Docker image is automatically uploaded to both registries when a new release is published on GitHub. The `latest` tag is updated to match the release. - -Each week, the `latest` image is rebuilt to include updates from Restic, Rclone, and the Alpine base image. - -After every commit to the main branch, another image is updated and tagged as `nightly`. This image may be unstable and is not recommended for production use. - -## Scheduling with docker compose - -There's an example in the contribution section how to schedule backups in a long running container. -The configuration needs to specify `crond` as a scheduler. - -See [contrib][2] - -[1]: https://rclone.org/ -[2]: https://github.com/creativeprojects/resticprofile/tree/master/contrib/schedule-in-docker diff --git a/docs/content/installation/linux.md b/docs/content/installation/linux.md index 1eb279524..ffb1e9562 100644 --- a/docs/content/installation/linux.md +++ b/docs/content/installation/linux.md @@ -1,19 +1,20 @@ --- -title: "Linux" +title: "Linux and MacOS" weight: 10 --- +{{< toc >}} ## Installation via a script +Works for: MacOS, Linux -Here's a simple script to download the binary automatically. It works on macOS, FreeBSD and Linux: + +Here's a simple script to download the binary in a `bin` directory under your current directory. ```shell curl -sfL https://raw.githubusercontent.com/creativeprojects/resticprofile/master/install.sh | sh ``` -It should copy resticprofile in a `bin` directory under your current directory. - If you need more control, you can save the shell script and run it manually: ```shell @@ -24,51 +25,23 @@ sudo ./install.sh -b /usr/local/bin It will install resticprofile in `/usr/local/bin/` -## Debian package - -WakeMeOps publishes packages for [restic](https://docs.wakemeops.com/packages/restic/) and [resticprofile](https://docs.wakemeops.com/packages/resticprofile/). - -Learn more: https://docs.wakemeops.com/packages/resticprofile/ - -These packages are managed by a third party. -## Installation with homebrew for Linux +## Installation With Homebrew +Works for: MacOS, Linux -There's a Linux [homebrew](https://brew.sh/) tap for resticprofile: +There's a [homebrew](https://brew.sh/) tap for resticprofile: ```shell brew tap creativeprojects/tap brew install resticprofile -``` - -You can also install `restic` at the same time with `--with-restic` flag: - -```shell +# You can also install `restic` at the same time with `--with-restic` flag: brew install resticprofile --with-restic ``` -You can test that resticprofile is properly installed (make sure you have restic installed or the test will fail): - -```shell -brew test resticprofile -``` - -Upgrading resticprofile installed via homebrew is very easy: - -```shell -brew update -brew upgrade resticprofile -``` - {{% notice style="note" %}} The resticprofile command `self-update` is not available when installed via homebrew. {{% /notice %}} -### Note on installing on Linux via Homebrew - -When testing homebrew after spinning new Linux virtual machines, I noticed resticprofile wouldn't install without a compiler installed on the machine. -Even though resticprofile is distributed as a **binary**, it looks like homebrew needs access to a compiler. - -Depending on your distribution you will need to install gcc: -* `sudo yum install gcc` -* `sudo apt install gcc` +{{% notice style="tip" title="Debian package" %}} +WakeMeOps (a third party) publishes packages for [restic](https://docs.wakemeops.com/packages/restic/) and [resticprofile](https://docs.wakemeops.com/packages/resticprofile/). +{{% /notice %}} diff --git a/docs/content/installation/macosx.md b/docs/content/installation/macosx.md deleted file mode 100644 index 1c6529f09..000000000 --- a/docs/content/installation/macosx.md +++ /dev/null @@ -1,55 +0,0 @@ ---- -title: "mac OS" -weight: 11 ---- - -## Installation with Homebrew - -There's a [homebrew](https://brew.sh/) tap for resticprofile: - -```shell -brew tap creativeprojects/tap -brew install resticprofile -``` - -You can also install `restic` at the same time with `--with-restic` flag: - -```shell -brew install resticprofile --with-restic -``` - -You can test that resticprofile is properly installed (make sure you have restic installed or the test will fail): - -```shell -brew test resticprofile -``` - -Upgrading resticprofile installed via homebrew is very easy: - -```shell -brew update -brew upgrade resticprofile -``` -{{% notice style="note" %}} -The resticprofile command `self-update` is not available when installed via homebrew. -{{% /notice %}} - -## Installation via a script - -Here's a simple script to download the binary automatically. It works on mac OS X, FreeBSD and Linux: - -```shell -curl -sfL https://raw.githubusercontent.com/creativeprojects/resticprofile/master/install.sh | sh -``` - -It should copy resticprofile in a `bin` directory under your current directory. - -If you need more control, you can save the shell script and run it manually: - -```shell -curl -LO https://raw.githubusercontent.com/creativeprojects/resticprofile/master/install.sh -chmod +x install.sh -sudo ./install.sh -b /usr/local/bin -``` - -It will install resticprofile in `/usr/local/bin/` diff --git a/docs/content/installation/shell.md b/docs/content/installation/shell.md deleted file mode 100644 index 747062e3c..000000000 --- a/docs/content/installation/shell.md +++ /dev/null @@ -1,24 +0,0 @@ ---- -title: "Shell Completion" -weight: 100 ---- - - -Shell command line completions are provided for `bash` and `zsh`. - -To load the command completions in shell, use: - -```shell -# bash -eval "$(resticprofile generate --bash-completion)" - -# zsh -eval "$(resticprofile generate --zsh-completion)" -``` - -To install them permanently: - -```shell -resticprofile generate --bash-completion > /etc/bash_completion.d/resticprofile -chmod +x /etc/bash_completion.d/resticprofile -``` diff --git a/docs/content/installation/source.md b/docs/content/installation/source.md index 4ebe508fc..6f4d1ff99 100644 --- a/docs/content/installation/source.md +++ b/docs/content/installation/source.md @@ -1,42 +1,26 @@ --- -title: "Source" -weight: 16 +title: "From Source" +weight: 13 --- ## Installation from source +Requirements: `git`, [Go compiler](https://golang.org/dl/), `make` -It's very easy to compile from the source code. - -Ensure your machine has the following: -- `git` (use `git-bash` on Windows) -- [Go compiler](https://golang.org/dl/) -- `GNU Make` (preinstalled on many Unix systems). On Debian-based distributions (e.g., Ubuntu), install the `build-essential` package. - -Compilation: +Clone the repository by running: ```shell git clone https://github.com/creativeprojects/resticprofile.git cd resticprofile -make build ``` -Your compiled binary (`resticprofile` or `resticprofile.exe`) is available in the current folder. - -To install the binary in your user path: +To compile the binary, run: `make build` -```shell -make install -``` +To install the binary in your user path, run `make install` -To build all common platforms (`build-mac`, `build-linux`, `build-pi` & `build-windows`): - -```shell -make build-all -``` +To build the binary for all common platforms (`build-mac`, `build-linux`, `build-pi` & `build-windows`), run: `make build-all` +{{% notice style="tip" %}} Alternatively, a **go-only** build (without `GNU Make`) is accomplished with: - ```shell -git clone https://github.com/creativeprojects/resticprofile.git -cd resticprofile go build -v -o resticprofile . ``` +{{% /notice %}} diff --git a/docs/content/installation/upgrade.md b/docs/content/installation/upgrade.md deleted file mode 100644 index 634092110..000000000 --- a/docs/content/installation/upgrade.md +++ /dev/null @@ -1,26 +0,0 @@ ---- -title: "Upgrade" -weight: 20 ---- - -After installation, upgrade resticprofile to the latest release with this command: - -```shell -resticprofile self-update -``` - -{{% notice style="note" title="Package Managers" %}} -The `self-update` command is generally unavailable when installed through a package manager like Homebrew or Scoop. Use the package manager's upgrade feature instead. -{{% /notice %}} - -Resticprofile checks for new versions from GitHub releases and prompts you to update. Use the `-q` or `--quiet` flag to update automatically without prompting. - -```shell -resticprofile --quiet self-update -``` - -or - -```shell -resticprofile self-update --quiet -``` diff --git a/docs/content/installation/windows.md b/docs/content/installation/windows.md index 2c5fbbd24..b820b6b7d 100644 --- a/docs/content/installation/windows.md +++ b/docs/content/installation/windows.md @@ -3,7 +3,7 @@ title: "Windows" weight: 12 --- - +{{< toc >}} ## Installation using bash diff --git a/docs/content/monitoring/_index.md b/docs/content/monitoring/_index.md deleted file mode 100644 index 667a68c60..000000000 --- a/docs/content/monitoring/_index.md +++ /dev/null @@ -1,8 +0,0 @@ ---- -archetype: "chapter" -pre: "5. " -title: "Monitoring" -weight: 5 ---- - -{{% children %}} diff --git a/docs/content/configuration/jsonschema/jsonschema-vsc.gif b/docs/content/reference/jsonschema/jsonschema-vsc.gif similarity index 100% rename from docs/content/configuration/jsonschema/jsonschema-vsc.gif rename to docs/content/reference/jsonschema/jsonschema-vsc.gif diff --git a/docs/content/schedules/examples.md b/docs/content/schedules/examples.md deleted file mode 100644 index ede7727d7..000000000 --- a/docs/content/schedules/examples.md +++ /dev/null @@ -1,407 +0,0 @@ ---- -title: "Schedule Examples" -weight: 30 ---- - - -### Examples of scheduling commands under Windows - -{{% notice note %}} -If you create a task with `user` permission under Windows, you will need to enter your password to validate the task. - -It's a requirement of the task scheduler. I'm inviting you to review the code to make sure I'm not emailing your password to myself. Seriously you shouldn't trust anyone. -{{% /notice %}} - -Example of the `schedule` command under Windows (with git bash): - -```shell -$ resticprofile -c examples/windows.yaml -n self schedule - -Analyzing backup schedule 1/2 -================================= - Original form: Mon..Fri *:00,15,30,45 -Normalized form: Mon..Fri *-*-* *:00,15,30,45:00 - Next elapse: Wed Jul 22 21:30:00 BST 2020 - (in UTC): Wed Jul 22 20:30:00 UTC 2020 - From now: 1m52s left - -Analyzing backup schedule 2/2 -================================= - Original form: Sat,Sun 0,12:00 -Normalized form: Sat,Sun *-*-* 00,12:00:00 - Next elapse: Sat Jul 25 00:00:00 BST 2020 - (in UTC): Fri Jul 24 23:00:00 UTC 2020 - From now: 50h31m52s left - -Creating task for user Creative Projects -Task Scheduler requires your Windows password to validate the task: - -2020/07/22 21:28:15 scheduled job self/backup created - -Analyzing retention schedule 1/1 -================================= - Original form: sun 3:30 -Normalized form: Sun *-*-* 03:30:00 - Next elapse: Sun Jul 26 03:30:00 BST 2020 - (in UTC): Sun Jul 26 02:30:00 UTC 2020 - From now: 78h1m44s left - -2020/07/22 21:28:22 scheduled job self/retention created -``` - -To see the status of the triggers, you can use the `status` command: - -```shell -$ resticprofile -c examples/windows.yaml -n self status - -Analyzing backup schedule 1/2 -================================= - Original form: Mon..Fri *:00,15,30,45 -Normalized form: Mon..Fri *-*-* *:00,15,30,45:00 - Next elapse: Wed Jul 22 21:30:00 BST 2020 - (in UTC): Wed Jul 22 20:30:00 UTC 2020 - From now: 14s left - -Analyzing backup schedule 2/2 -================================= - Original form: Sat,Sun 0,12:* -Normalized form: Sat,Sun *-*-* 00,12:*:00 - Next elapse: Sat Jul 25 00:00:00 BST 2020 - (in UTC): Fri Jul 24 23:00:00 UTC 2020 - From now: 50h29m46s left - - Task: \resticprofile backup\self backup - User: Creative Projects - Working Dir: D:\Source\resticprofile - Exec: D:\Source\resticprofile\resticprofile.exe --no-ansi --config examples/windows.yaml --name self backup - Enabled: true - State: ready - Missed runs: 0 - Last Run Time: 2020-07-22 21:30:00 +0000 UTC - Last Result: 0 - Next Run Time: 2020-07-22 21:45:00 +0000 UTC - -Analyzing retention schedule 1/1 -================================= - Original form: sun 3:30 -Normalized form: Sun *-*-* 03:30:00 - Next elapse: Sun Jul 26 03:30:00 BST 2020 - (in UTC): Sun Jul 26 02:30:00 UTC 2020 - From now: 77h59m46s left - - Task: \resticprofile backup\self retention - User: Creative Projects - Working Dir: D:\Source\resticprofile - Exec: D:\Source\resticprofile\resticprofile.exe --no-ansi --config examples/windows.yaml --name self forget - Enabled: true - State: ready - Missed runs: 0 - Last Run Time: 1999-11-30 00:00:00 +0000 UTC - Last Result: 267011 - Next Run Time: 2020-07-26 03:30:00 +0000 UTC - -``` - -To remove the schedule, use the `unschedule` command: - -```shell -$ resticprofile -c examples/windows.yaml -n self unschedule -2020/07/22 21:34:51 scheduled job self/backup removed -2020/07/22 21:34:51 scheduled job self/retention removed -``` - - -### Examples of scheduling commands under Linux - -With this example of configuration for Linux: - -{{< tabs groupid="config-with-json" >}} -{{% tab title="toml" %}} - -```toml -version = "1" - -[default] - password-file = "key" - repository = "/tmp/backup" - -[test1] - inherit = "default" - - [test1.backup] - source = "./" - schedule = "*:00,15,30,45" - schedule-permission = "user" - schedule-lock-wait = "15m" - - [test1.check] - schedule = "*-*-1" - schedule-permission = "user" - schedule-lock-wait = "15m" -``` - -{{% /tab %}} -{{% tab title="yaml" %}} - -```yaml -version: "1" - -default: - password-file: key - repository: /tmp/backup - -test1: - inherit: default - backup: - source: ./ - schedule: "*:00,15,30,45" - schedule-permission: user - schedule-lock-wait: 15m - check: - schedule: "*-*-1" - schedule-permission: user - schedule-lock-wait: 15m - -``` - -{{% /tab %}} -{{% tab title="hcl" %}} - -```hcl -"default" = { - "password-file" = "key" - "repository" = "/tmp/backup" -} - -"test1" = { - "inherit" = "default" - - "backup" = { - "source" = "./" - "schedule" = "*:00,15,30,45" - "schedule-permission" = "user" - "schedule-lock-wait" = "15m" - } - - "check" = { - "schedule" = "*-*-1" - "schedule-permission" = "user" - "schedule-lock-wait" = "15m" - } -} -``` - -{{% /tab %}} -{{% tab title="json" %}} - -```json -{ - "version": "1", - "default": { - "password-file": "key", - "repository": "/tmp/backup" - }, - "test1": { - "inherit": "default", - "backup": { - "source": "./", - "schedule": "*:00,15,30,45", - "schedule-permission": "user", - "schedule-lock-wait": "15m" - }, - "check": { - "schedule": "*-*-1", - "schedule-permission": "user", - "schedule-lock-wait": "15m" - } - } -} -``` - -{{% /tab %}} -{{< /tabs >}} - - -```shell -$ resticprofile -c examples/linux.yaml -n test1 schedule - -Analyzing backup schedule 1/1 -================================= - Original form: *:00,15,30,45 -Normalized form: *-*-* *:00,15,30,45:00 - Next elapse: Thu 2020-07-23 17:15:00 BST - (in UTC): Thu 2020-07-23 16:15:00 UTC - From now: 6min left - -2020/07/23 17:08:51 writing /home/user/.config/systemd/user/resticprofile-backup@profile-test1.service -2020/07/23 17:08:51 writing /home/user/.config/systemd/user/resticprofile-backup@profile-test1.timer -Created symlink /home/user/.config/systemd/user/timers.target.wants/resticprofile-backup@profile-test1.timer → /home/user/.config/systemd/user/resticprofile-backup@profile-test1.timer. -2020/07/23 17:08:51 scheduled job test1/backup created - -Analyzing check schedule 1/1 -================================= - Original form: *-*-1 -Normalized form: *-*-01 00:00:00 - Next elapse: Sat 2020-08-01 00:00:00 BST - (in UTC): Fri 2020-07-31 23:00:00 UTC - From now: 1 weeks 1 days left - -2020/07/23 17:08:51 writing /home/user/.config/systemd/user/resticprofile-check@profile-test1.service -2020/07/23 17:08:51 writing /home/user/.config/systemd/user/resticprofile-check@profile-test1.timer -Created symlink /home/user/.config/systemd/user/timers.target.wants/resticprofile-check@profile-test1.timer → /home/user/.config/systemd/user/resticprofile-check@profile-test1.timer. -2020/07/23 17:08:51 scheduled job test1/check created -``` - -The `status` command shows a combination of `journalctl` displaying errors (only) in the last month and `systemctl status`: - -```shell -$ resticprofile -c examples/linux.yaml -n test1 status - -Analyzing backup schedule 1/1 -================================= - Original form: *:00,15,30,45 -Normalized form: *-*-* *:00,15,30,45:00 - Next elapse: Tue 2020-07-28 15:15:00 BST - (in UTC): Tue 2020-07-28 14:15:00 UTC - From now: 4min 44s left - -Recent log (>= warning in the last month) -========================================== --- Logs begin at Wed 2020-06-17 11:09:19 BST, end at Tue 2020-07-28 15:10:10 BST. -- -Jul 27 20:48:01 Desktop76 systemd[2986]: Failed to start resticprofile backup for profile test1 in examples/linux.yaml. -Jul 27 21:00:55 Desktop76 systemd[2986]: Failed to start resticprofile backup for profile test1 in examples/linux.yaml. -Jul 27 21:15:34 Desktop76 systemd[2986]: Failed to start resticprofile backup for profile test1 in examples/linux.yaml. - -Systemd timer status -===================== -● resticprofile-backup@profile-test1.timer - backup timer for profile test1 in examples/linux.yaml - Loaded: loaded (/home/user/.config/systemd/user/resticprofile-backup@profile-test1.timer; enabled; vendor preset: enabled) - Active: active (waiting) since Tue 2020-07-28 15:10:06 BST; 8s ago - Trigger: Tue 2020-07-28 15:15:00 BST; 4min 44s left - -Jul 28 15:10:06 Desktop76 systemd[2951]: Started backup timer for profile test1 in examples/linux.yaml. - - -Analyzing check schedule 1/1 -================================= - Original form: *-*-1 -Normalized form: *-*-01 00:00:00 - Next elapse: Sat 2020-08-01 00:00:00 BST - (in UTC): Fri 2020-07-31 23:00:00 UTC - From now: 3 days left - -Recent log (>= warning in the last month) -========================================== --- Logs begin at Wed 2020-06-17 11:09:19 BST, end at Tue 2020-07-28 15:10:10 BST. -- -Jul 27 19:39:59 Desktop76 systemd[2986]: Failed to start resticprofile check for profile test1 in examples/linux.yaml. - -Systemd timer status -===================== -● resticprofile-check@profile-test1.timer - check timer for profile test1 in examples/linux.yaml - Loaded: loaded (/home/user/.config/systemd/user/resticprofile-check@profile-test1.timer; enabled; vendor preset: enabled) - Active: active (waiting) since Tue 2020-07-28 15:10:07 BST; 7s ago - Trigger: Sat 2020-08-01 00:00:00 BST; 3 days left - -Jul 28 15:10:07 Desktop76 systemd[2951]: Started check timer for profile test1 in examples/linux.yaml. - - -``` - -And `unschedule`: - -```shell -$ resticprofile -c examples/linux.yaml -n test1 unschedule -Removed /home/user/.config/systemd/user/timers.target.wants/resticprofile-backup@profile-test1.timer. -2020/07/23 17:13:42 scheduled job test1/backup removed -Removed /home/user/.config/systemd/user/timers.target.wants/resticprofile-check@profile-test1.timer. -2020/07/23 17:13:42 scheduled job test1/check removed -``` - -### Examples of scheduling commands under macOS - -Here's an example of scheduling a backup profile named `azure-dev`: - -```shell -% resticprofile -n azure-dev schedule -2025/03/30 18:40:13 using configuration file: profiles.yaml - -Profile (or Group) azure-dev: backup schedule -============================================= - Original form: 22:22 -Normalized form: *-*-* 22:22:00 - Next elapse: Sun Mar 30 22:22:00 BST 2025 - (in UTC): Sun Mar 30 21:22:00 UTC 2025 - From now: 3h41m46s left - -2025/03/30 18:40:13 scheduled job azure-dev/backup created - -``` - -{{% notice info %}} - -In some cases, macOS may request permission to access the network, an external volume (like a USB drive), or a protected directory. A message window will appear while the backup runs in the background. -{{% /notice %}} - -![access files popup window]({{< absolute "schedules/removable_volume.png" nohtml >}}) - -You may want to start the task now so you can grant special permissions: - -1. Retrieve the task name using the status command: - -```shell -% resticprofile -n azure-dev status -2025/03/30 18:40:21 using configuration file: profiles.yaml - -Profile (or Group) azure-dev: backup schedule -============================================= - Original form: *-*-* 22:22:00 -Normalized form: *-*-* 22:22:00 - Next elapse: Sun Mar 30 22:22:00 BST 2025 - (in UTC): Sun Mar 30 21:22:00 UTC 2025 - From now: 3h41m38s left - - service: local.resticprofile.azure-dev.backup - permission: user - program: /usr/local/bin/resticprofile - working directory: /Users/cp/resticprofile - stdout path: local.resticprofile.azure-dev.backup.log - stderr path: local.resticprofile.azure-dev.backup.log - state: not running - runs (*): 0 - last exit code (*): (never exited) - * : since last (re)schedule or last reboot -``` - -The name of the task can be seen on the line `service: ...` - -2. start the task manually - -```shell -% launchctl start local.resticprofile.azure-dev.backup -``` - -You can check the task is currently running: - -```shell -2025/03/30 18:42:07 using configuration file: profiles.yaml - -Profile (or Group) azure-dev: backup schedule -============================================= - Original form: *-*-* 22:22:00 -Normalized form: *-*-* 22:22:00 - Next elapse: Sun Mar 30 22:22:00 BST 2025 - (in UTC): Sun Mar 30 21:22:00 UTC 2025 - From now: 3h39m52s left - - service: local.resticprofile.azure-dev.backup - permission: user - program: /usr/local/bin/resticprofile - working directory: /Users/cp/resticprofile - stdout path: local.resticprofile.azure-dev.backup.log - stderr path: local.resticprofile.azure-dev.backup.log - state: running - runs (*): 1 - last exit code (*): (never exited) - * : since last (re)schedule or last reboot - -``` \ No newline at end of file diff --git a/docs/content/usage/_index.md b/docs/content/usage/_index.md index a63a43abf..ddbfec5f4 100644 --- a/docs/content/usage/_index.md +++ b/docs/content/usage/_index.md @@ -1,133 +1,11 @@ --- archetype: "chapter" -pre: "3. " +pre: "2. " title: "Using resticprofile" -weight: 3 +weight: 2 --- -Here are a few examples how to run resticprofile (using the main example configuration file) - -See all snapshots of your `default` profile: - -```shell -resticprofile -``` - -See all available profiles in your configuration file (and the restic commands where some flags are defined): - -```shell -resticprofile profiles - -Profiles available (name, sections, description): - root: (backup, copy, forget, retention) - self: (backup, check, copy, forget, retention) - src: (backup, copy, retention, snapshots) - -Groups available (name, profiles, description): - full-backup: [root, src] - -``` - -Backup root & src profiles (using _full-backup_ group shown earlier) - -```shell -resticprofile --name "full-backup" backup -``` -or using the alternative syntax: - -```shell -resticprofile full-backup.backup -``` - -Assuming the _stdin_ profile from the configuration file shown before, the command to send a mysqldump to the backup is as simple as: - -```shell -mysqldump --all-databases --order-by-primary | resticprofile --name stdin backup -``` -or using the alternative syntax: - -```shell -mysqldump --all-databases --order-by-primary | resticprofile stdin.backup -``` - -Mount the default profile (_default_) in /mnt/restic: - -```shell -resticprofile mount /mnt/restic -``` - -Display quick help - -```shell -resticprofile --help -``` - A command is either a restic command or a resticprofile own command. +{{% children sort="weight" depth="2" %}} -## Command line reference - -There are not many options on the command line, most of the options are in the configuration file. - -* **[-h | --help]**: Display quick help -* **[-c | --config] configuration_file**: Specify a configuration file other than the default ("profiles") -* **[-f | --format] configuration_format**: Specify the configuration file format: `toml`, `yaml`, `json` or `hcl` -* **[-n | --name] profile_name**: Profile section to use from the configuration file. - You can also use `[profile_name].[command]` syntax instead, this will only work if `-n` is not set. - Using `-n [profile_name] [command]` or `[profile_name].[command]` both select profile and command and are technically equivalent. -* **[--dry-run]**: Doesn't run the restic commands but display the command lines instead -* **[-q | --quiet]**: Force resticprofile and restic to be quiet (override any configuration from the profile) -* **[-v | --verbose]**: Force resticprofile and restic to be verbose (override any configuration from the profile) -* **[--trace]**: Display even more debugging information -* **[--no-ansi]**: Disable console colouring (to save output into a log file) -* **[--stderr]**: Send console output from resticprofile to stderr (is enabled for commands `cat` and `dump`) -* **[--no-lock]**: Disable resticprofile locks, neither create nor fail on a lock. restic locks are unaffected by this option. -* **[--theme]**: Can be `light`, `dark` or `none`. The colours will adjust to a -light or dark terminal (none to disable colouring) -* **[--lock-wait] duration**: Retry to acquire resticprofile and restic locks for up to the specified amount of time before failing on a lock failure. -* **[-l | --log] file path or url**: To write the logs to a file or a syslog server instead of displaying on the console. -The format of the syslog server url is `syslog-tcp://192.168.0.1:514`, `syslog://udp-server:514` or `syslog:`. -For custom log forwarding, the prefix `temp:` can be used (e.g. `temp:/t/msg.log`) to create unique log output that can be fed -into a command or http hook by referencing it with `"{{ tempFile "msg.log" }}"` in the configuration file. -* **[--command-output]**: Sets how to redirect command output when a log target is specified. Can be `auto`, `log`, `console` or `all`. -* **[-w | --wait]**: Wait at the very end of the execution for the user to press enter. -This is only useful in Windows when resticprofile is started from explorer and the console window closes automatically at the end. -* **[--ignore-on-battery]**: Don't start the profile when the computer is running on battery. You can specify a value to ignore only when the % charge left is less or equal than the value. -* **[resticprofile OR restic command]**: Like snapshots, backup, check, prune, forget, mount, etc. -* **[additional flags]**: Any additional flags to pass to the restic command line - -## Environment variables - -Most flags for resticprofile can be set using environment variables. If both are specified, command line flags take the precedence. - -| Flag | Environment variable | Built-In default | -|-----------------------|-----------------------------------|------------------| -| `--quiet` | `RESTICPROFILE_QUIET` | `false` | -| `--verbose` | `RESTICPROFILE_VERBOSE` | `false` | -| `--trace` | `RESTICPROFILE_TRACE` | `false` | -| `--config` | `RESTICPROFILE_CONFIG` | `"profiles"` | -| `--format` | `RESTICPROFILE_FORMAT` | `""` | -| `--name` | `RESTICPROFILE_NAME` | `"default"` | -| `--log` | `RESTICPROFILE_LOG` | `""` | -| `--command-output` | `RESTICPROFILE_COMMAND_OUTPUT` | `"auto"` | -| `--dry-run` | `RESTICPROFILE_DRY_RUN` | `false` | -| `--no-lock` | `RESTICPROFILE_NO_LOCK` | `false` | -| `--lock-wait` | `RESTICPROFILE_LOCK_WAIT` | `0` | -| `--stderr` | `RESTICPROFILE_STDERR` | `false` | -| `--no-ansi` | `RESTICPROFILE_NO_ANSI` | `false` | -| `--theme` | `RESTICPROFILE_THEME` | `"light"` | -| `--no-priority` | `RESTICPROFILE_NO_PRIORITY` | `false` | -| `--wait` | `RESTICPROFILE_WAIT` | `false` | -| `--ignore-on-battery` | `RESTICPROFILE_IGNORE_ON_BATTERY` | `0` | - -### Other environment variables - -| Environment Variable | Default | Purpose | -|---------------------------------|---------|--------------------------------------------------------------------------------------| -| `RESTICPROFILE_PWSH_NO_AUTOENV` | _empty_ | Disables powershell script pre-processing that converts unset `$VAR` into `$Env:VAR` | - -### Environment variables set by resticprofile - -| Environment Variable | Example | When | -|-----------------------------|--------------------------------|-------------------------------------| -| `RESTICPROFILE_SCHEDULE_ID` | `profiles.yaml:backup@profile` | Set when running scheduled commands | diff --git a/docs/content/usage/configuration/_index.md b/docs/content/usage/configuration/_index.md new file mode 100644 index 000000000..008dc9097 --- /dev/null +++ b/docs/content/usage/configuration/_index.md @@ -0,0 +1,103 @@ +--- +title: "Configuration Files" +weight: 20 +alwaysopen: false +--- + +So let's say you normally use this simple command: + +```shell +restic --repo "local:/backup" --password-file "password.txt" --verbose backup /home +``` + +For resticprofile to generate this command automatically for you, here's the configuration file: + +{{< tabs groupid="config-with-json" >}} +{{% tab title="toml" %}} + +```toml +# indentation is not needed but it makes it easier to read ;) +# +version = "1" + +[default] + repository = "local:/backup" + password-file = "password.txt" + + [default.backup] + verbose = true + source = [ "/home" ] +``` + +{{% /tab %}} +{{% tab title="yaml" %}} + +```yaml +version: "1" + +default: + repository: "local:/backup" + password-file: "password.txt" + + backup: + verbose: true + source: + - "/home" +``` + +{{% /tab %}} +{{% tab title="hcl" %}} + +```hcl +default { + repository = "local:/backup" + password-file = "password.txt" + + backup = { + verbose = true + source = [ "/home" ] + } +} +``` + +{{% /tab %}} +{{% tab title="json" %}} + +```json +{ + "version": "1", + "default": { + "repository": "local:/backup", + "password-file": "password.txt", + "backup": { + "verbose": true, + "source": [ + "/home" + ] + } + } +} +``` + +{{% /tab %}} +{{< /tabs >}} + + +You may have noticed the `source` flag is accepting an array of values (inside brackets in TOML, list of values in YAML) + +Now, assuming this configuration file is named `profiles.conf` in the current folder (it's the default config file name), you can simply run + +```shell +resticprofile backup +``` + +and resticprofile will do its magic and generate the command line for you. + +If you have any doubt on what it's running, you can try a `--dry-run`: + +```shell +resticprofile --dry-run backup +2022/05/18 17:14:07 profile 'default': starting 'backup' +2022/05/18 17:14:07 dry-run: /usr/bin/restic backup --password-file password.txt --repo local:/backup --verbose /home +2022/05/18 17:14:07 profile 'default': finished 'backup' +``` diff --git a/docs/content/usage/configuration/azure-storage.md b/docs/content/usage/configuration/azure-storage.md new file mode 100644 index 000000000..fe8d00061 --- /dev/null +++ b/docs/content/usage/configuration/azure-storage.md @@ -0,0 +1,85 @@ +--- +title: "Example: Azure Storage" +weight: 10 +--- + +Here's a simple configuration file using a Microsoft Azure backend. You will notice that the `env` section lets you define environment variables: + +{{< tabs groupid="config-with-hcl" >}} +{{% tab title="toml" %}} + +```toml +version = "1" + +[default] + repository = "azure:restic:/" + password-file = "key" + option = "azure.connections=3" + + [default.env] + AZURE_ACCOUNT_NAME = "my_storage_account" + AZURE_ACCOUNT_KEY = "my_super_secret_key" + + [default.backup] + exclude-file = "excludes" + exclude-caches = true + one-file-system = true + tag = [ "root" ] + source = [ "/", "/var" ] + schedule = "daily" + schedule-after-network-online = true +``` +{{% /tab %}} +{{% tab title="yaml" %}} + +```yaml +version: "1" + +default: + repository: "azure:restic:/" + password-file: "key" + option: "azure.connections=3" + + env: + AZURE_ACCOUNT_NAME: "my_storage_account" + AZURE_ACCOUNT_KEY: "my_super_secret_key" + + backup: + exclude-file: "excludes" + exclude-caches: true + one-file-system: true + tag: + - "root" + source: + - "/" + - "/var" + schedule: "daily" + schedule-after-network-online: true +``` + +{{% /tab %}} +{{% tab title="hcl" %}} + +```hcl +default { + repository = "azure:restic:/" + password-file = "key" + options = "azure.connections=3" + + env { + AZURE_ACCOUNT_NAME = "my_storage_account" + AZURE_ACCOUNT_KEY = "my_super_secret_key" + } + + backup = { + exclude-file = "excludes" + exclude-caches = true + one-file-system = true + tag = [ "root" ] + source = [ "/", "/var" ] + } +} +``` + +{{% /tab %}} +{{< /tabs >}} diff --git a/docs/content/usage/configuration/configWindows.md b/docs/content/usage/configuration/configWindows.md new file mode 100644 index 000000000..063eaef65 --- /dev/null +++ b/docs/content/usage/configuration/configWindows.md @@ -0,0 +1,107 @@ +--- +title: "Example: Windows" +weight: 30 +--- + +{{< tabs groupid="config-with-hcl" >}} +{{% tab title="toml" %}} + +```toml +version = "1" + +[global] + restic-binary = "c:\\ProgramData\\chocolatey\\bin\\restic.exe" + +# Default profile when not specified on the command line +# There's no default inheritance from the 'default' profile, +# but you can use the 'inherit' flag if needed +[default] + repository = "local:r:/" + password-file = "key" + initialize = false + +# New profile named 'test' +[test] + inherit = "default" + initialize = true + + # 'backup' command of profile 'test' + [test.backup] + tag = [ "windows" ] + source = [ "c:\\" ] + check-after = true + run-before = "dir /l" + run-after = "echo All Done!" + # ignore restic warnings + # without it the backup is considered failed when restic can't read some files + no-error-on-warning = true +``` + +{{% /tab %}} +{{% tab title="yaml" %}} + +```yaml +version: "1" + +global: + restic-binary: c:\ProgramData\chocolatey\bin\restic.exe + +# Default profile when not specified on the command line +# There's no default inheritance from the 'default' profile, +# but you can use the 'inherit' flag if needed +default: + repository: local:r:/ + password-file: key + initialize: false + +# New profile named 'test' +test: + inherit: default + initialize: true + + # 'backup' command of profile 'test' + backup: + tag: + - windows + source: + - c:\ + check-after: true + run-before: dir /l + run-after: echo All Done! + # ignore restic warnings + # without it the backup is considered failed when restic can't read some files + no-error-on-warning: true +``` + +{{% /tab %}} +{{% tab title="hcl" %}} + +```hcl +global { + restic-binary = "c:\\ProgramData\\chocolatey\\bin\\restic.exe" +} + +default { + repository = "local:r:/" + password-file = "key" + initialize = false +} + +test { + inherit = "default" + initialize = true + + backup = { + tag = [ "windows" ] + source = [ "c:\\" ] + check-after = true + run-before = "dir /l" + run-after = "echo All Done!" + no-error-on-warning = true + } +} + +``` + +{{% /tab %}} +{{< /tabs >}} diff --git a/docs/content/configuration/examples.md b/docs/content/usage/configuration/configWinheritence.md similarity index 63% rename from docs/content/configuration/examples.md rename to docs/content/usage/configuration/configWinheritence.md index fe22b1518..b793a35ed 100644 --- a/docs/content/configuration/examples.md +++ b/docs/content/usage/configuration/configWinheritence.md @@ -1,93 +1,8 @@ --- -title: "Examples" -weight: 5 +title: "Example: Using Inheritance" +weight: 20 --- -## Simple configuration using Azure storage - -Here's a simple configuration file using a Microsoft Azure backend. You will notice that the `env` section lets you define environment variables: - -{{< tabs groupid="config-with-hcl" >}} -{{% tab title="toml" %}} - -```toml -version = "1" - -[default] - repository = "azure:restic:/" - password-file = "key" - option = "azure.connections=3" - - [default.env] - AZURE_ACCOUNT_NAME = "my_storage_account" - AZURE_ACCOUNT_KEY = "my_super_secret_key" - - [default.backup] - exclude-file = "excludes" - exclude-caches = true - one-file-system = true - tag = [ "root" ] - source = [ "/", "/var" ] - schedule = "daily" - schedule-after-network-online = true -``` -{{% /tab %}} -{{% tab title="yaml" %}} - -```yaml -version: "1" - -default: - repository: "azure:restic:/" - password-file: "key" - option: "azure.connections=3" - - env: - AZURE_ACCOUNT_NAME: "my_storage_account" - AZURE_ACCOUNT_KEY: "my_super_secret_key" - - backup: - exclude-file: "excludes" - exclude-caches: true - one-file-system: true - tag: - - "root" - source: - - "/" - - "/var" - schedule: "daily" - schedule-after-network-online: true -``` - -{{% /tab %}} -{{% tab title="hcl" %}} - -```hcl -default { - repository = "azure:restic:/" - password-file = "key" - options = "azure.connections=3" - - env { - AZURE_ACCOUNT_NAME = "my_storage_account" - AZURE_ACCOUNT_KEY = "my_super_secret_key" - } - - backup = { - exclude-file = "excludes" - exclude-caches = true - one-file-system = true - tag = [ "root" ] - source = [ "/", "/var" ] - } -} -``` - -{{% /tab %}} -{{< /tabs >}} - -## Configuration with inheritance - Here's a more complex configuration file showing profile inheritance and two backup profiles using the same repository: {{< tabs groupid="config-with-hcl" >}} @@ -427,194 +342,3 @@ self { {{% /tab %}} {{< /tabs >}} - -## Configuration example for Windows - - -{{< tabs groupid="config-with-hcl" >}} -{{% tab title="toml" %}} - -```toml -version = "1" - -[global] - restic-binary = "c:\\ProgramData\\chocolatey\\bin\\restic.exe" - -# Default profile when not specified on the command line -# There's no default inheritance from the 'default' profile, -# but you can use the 'inherit' flag if needed -[default] - repository = "local:r:/" - password-file = "key" - initialize = false - -# New profile named 'test' -[test] - inherit = "default" - initialize = true - - # 'backup' command of profile 'test' - [test.backup] - tag = [ "windows" ] - source = [ "c:\\" ] - check-after = true - run-before = "dir /l" - run-after = "echo All Done!" - # ignore restic warnings - # without it the backup is considered failed when restic can't read some files - no-error-on-warning = true -``` - -{{% /tab %}} -{{% tab title="yaml" %}} - -```yaml -version: "1" - -global: - restic-binary: c:\ProgramData\chocolatey\bin\restic.exe - -# Default profile when not specified on the command line -# There's no default inheritance from the 'default' profile, -# but you can use the 'inherit' flag if needed -default: - repository: local:r:/ - password-file: key - initialize: false - -# New profile named 'test' -test: - inherit: default - initialize: true - - # 'backup' command of profile 'test' - backup: - tag: - - windows - source: - - c:\ - check-after: true - run-before: dir /l - run-after: echo All Done! - # ignore restic warnings - # without it the backup is considered failed when restic can't read some files - no-error-on-warning: true -``` - -{{% /tab %}} -{{% tab title="hcl" %}} - -```hcl -global { - restic-binary = "c:\\ProgramData\\chocolatey\\bin\\restic.exe" -} - -default { - repository = "local:r:/" - password-file = "key" - initialize = false -} - -test { - inherit = "default" - initialize = true - - backup = { - tag = [ "windows" ] - source = [ "c:\\" ] - check-after = true - run-before = "dir /l" - run-after = "echo All Done!" - no-error-on-warning = true - } -} - -``` - -{{% /tab %}} -{{< /tabs >}} - -## Use stdin in configuration - -Simple example sending a file via stdin - -{{< tabs groupid="config-with-hcl" >}} -{{% tab title="toml" %}} - -```toml -version = "1" - -[stdin] - repository = "local:/backup/restic" - password-file = "key" - - [stdin.backup] - stdin = true - stdin-filename = "stdin-test" - tag = [ 'stdin' ] - -[mysql] - inherit = "stdin" - - [mysql.backup] - stdin-command = [ 'mysqldump --all-databases --order-by-primary' ] - stdin-filename = "dump.sql" - tag = [ 'mysql' ] - -``` - -{{% /tab %}} -{{% tab title="yaml" %}} - -```yaml -version: "1" - -stdin: - repository: "local:/backup/restic" - password-file: key - backup: - stdin: true - stdin-filename: stdin-test - tag: - - stdin - -mysql: - inherit: stdin - backup: - stdin-command: "mysqldump --all-databases --order-by-primary" - stdin-filename: "dump.sql" - tag: - - mysql - -``` - -{{% /tab %}} -{{% tab title="hcl" %}} - -```hcl -# sending stream through stdin -stdin { - repository = "local:/backup/restic" - password-file = "key" - - backup = { - stdin = true - stdin-filename = "stdin-test" - tag = [ "stdin" ] - } -} - -mysql { - inherit = "stdin" - - backup = { - stdin-command = [ "mysqldump --all-databases --order-by-primary" ] - stdin-filename = "dump.sql" - tag = [ "mysql" ] - } -} -``` - -{{% /tab %}} -{{< /tabs >}} - diff --git a/docs/content/usage/configuration/stdin.md b/docs/content/usage/configuration/stdin.md new file mode 100644 index 000000000..6b9ffd317 --- /dev/null +++ b/docs/content/usage/configuration/stdin.md @@ -0,0 +1,86 @@ +--- +title: "Example: stdin" +weight: 40 +--- + +Simple example sending a file via stdin + +{{< tabs groupid="config-with-hcl" >}} +{{% tab title="toml" %}} + +```toml +version = "1" + +[stdin] + repository = "local:/backup/restic" + password-file = "key" + + [stdin.backup] + stdin = true + stdin-filename = "stdin-test" + tag = [ 'stdin' ] + +[mysql] + inherit = "stdin" + + [mysql.backup] + stdin-command = [ 'mysqldump --all-databases --order-by-primary' ] + stdin-filename = "dump.sql" + tag = [ 'mysql' ] + +``` + +{{% /tab %}} +{{% tab title="yaml" %}} + +```yaml +version: "1" + +stdin: + repository: "local:/backup/restic" + password-file: key + backup: + stdin: true + stdin-filename: stdin-test + tag: + - stdin + +mysql: + inherit: stdin + backup: + stdin-command: "mysqldump --all-databases --order-by-primary" + stdin-filename: "dump.sql" + tag: + - mysql + +``` + +{{% /tab %}} +{{% tab title="hcl" %}} + +```hcl +# sending stream through stdin +stdin { + repository = "local:/backup/restic" + password-file = "key" + + backup = { + stdin = true + stdin-filename = "stdin-test" + tag = [ "stdin" ] + } +} + +mysql { + inherit = "stdin" + + backup = { + stdin-command = [ "mysqldump --all-databases --order-by-primary" ] + stdin-filename = "dump.sql" + tag = [ "mysql" ] + } +} +``` + +{{% /tab %}} +{{< /tabs >}} diff --git a/docs/content/configuration/getting_started/index.md b/docs/content/usage/getting_started.md similarity index 78% rename from docs/content/configuration/getting_started/index.md rename to docs/content/usage/getting_started.md index ba72b485d..d515d8de0 100644 --- a/docs/content/configuration/getting_started/index.md +++ b/docs/content/usage/getting_started.md @@ -1,50 +1,15 @@ --- title: "Getting Started" -weight: 2 +weight: 10 +alwaysopen: false --- -## Prerequisite - -**resticprofile** is an automation tool for restic, also known as a *wrapper*. +{{< toc >}} -In short, **resticprofile** provides a configuration file and a runner that generates all necessary calls to **restic**. +## Prerequisite Unless you're using the **resticprofile** [Docker image]({{% relref "/installation/docker/index.html" %}}), you need to have **restic** [installed on your machine](https://restic.readthedocs.io/en/stable/). -## Choose Your Favorite Format - -The **resticprofile** configuration file can be written in: -* [TOML](https://github.com/toml-lang/toml): configuration file with extension *.toml* or *.conf* -* [YAML](https://en.wikipedia.org/wiki/YAML): configuration file with extension *.yaml* -* [JSON](https://en.wikipedia.org/wiki/JSON): configuration file with extension *.json* -* [HCL](https://github.com/hashicorp/hcl): configuration file with extension *.hcl* - -We recommend using either TOML or YAML. - -JSON is suitable for auto-generated configurations but is not the easiest format for humans to read and write. - -HCL can be useful if you already use a tool from the Hashicorp stack; otherwise, it's another format to learn. - -## Configure Your Text Editor - -We'll show you how to get documentation and auto-completion for the **resticprofile** configuration using [Visual Studio Code](https://code.visualstudio.com/). - -You can use any other editor that recognizes the [JSON schema]({{% relref "/configuration/jsonschema" %}}). The same JSON schema can be used for JSON, TOML, and YAML file formats. - -### TOML - -In Visual Studio Code, install an extension that supports completion and syntax validation using a JSON schema. - -For example, install the [Even Better TOML](https://marketplace.visualstudio.com/items?itemName=tamasfe.even-better-toml) extension: - -![TOML extension]({{< absolute "configuration/getting_started/TOML.png" nohtml >}}) - -### YAML - -For YAML, install an extension like the one [provided by Red Hat](https://marketplace.visualstudio.com/items?itemName=redhat.vscode-yaml) in Visual Studio Code to understand the shape of your configuration file. - -![YAML extension]({{< absolute "configuration/getting_started/YAML.png" nohtml >}}) - ## Write your first configuration file The configuration file lists profiles containing commands and flags. @@ -308,7 +273,7 @@ default { {{% /tab %}} {{< /tabs >}} -**resticprofile** can [schedule]({{% relref "/schedules" %}}) work on macOS, Windows, most Unixes and Linux distributions: it is simply adding an entry in the default scheduler of your platform. +**resticprofile** can [schedule]({{% relref "/configuration/schedules" %}}) work on macOS, Windows, most Unixes and Linux distributions: it is simply adding an entry in the default scheduler of your platform. To schedule the backup of the default profile, simply type the command: @@ -368,7 +333,3 @@ Flags: [...] ``` - -## Next - -Let's have a look at some [examples]({{% relref "/configuration/examples" %}}) diff --git a/docs/content/usage/hooks/_index.md b/docs/content/usage/hooks/_index.md new file mode 100644 index 000000000..ac13a8d52 --- /dev/null +++ b/docs/content/usage/hooks/_index.md @@ -0,0 +1,7 @@ +--- +title: "Hooks" +weight: 50 +alwaysopen: false +--- + +{{% children sort="weight" %}} diff --git a/docs/content/usage/hooks/body-template.md b/docs/content/usage/hooks/body-template.md new file mode 100644 index 000000000..8df72f6a0 --- /dev/null +++ b/docs/content/usage/hooks/body-template.md @@ -0,0 +1,116 @@ +--- +title: "Message Body Template" +weight: 30 +--- + +You can use a standard go template to build the webhook body. It has to be defined in a separate file (otherwise it would clash with the configuration as a go template itself). + +More information can be found under [Hooks - HTTP Hooks]({{% relref "configuration/hooks/http_hooks/#body-template" %}}) + + +```json +{ + "profileName": "{{ .ProfileName }}", + "profileCommand": "{{ .ProfileCommand }}", + "exitCode": "{{ .Error.ExitCode }}" +} +``` + +The field `exitCode` will be blank if no error occured. + +And here's an example of a configuration using a body template: + + +{{< tabs groupid="config-with-json" >}} +{{% tab title="toml" %}} + +```toml +version = "1" + +[profile] + + [profile.backup] + source = "/source" + + [profile.backup.send-finally] + method = "POST" + url = "https://my/monitoring.example.com/" + body-template = "body-template.json" + + [[profile.backup.send-finally.headers]] + name = "Content-Type" + value = "application/json" + +``` + +{{% /tab %}} +{{% tab title="yaml" %}} + +```yaml +version: "1" + +profile: + + backup: + source: "/source" + + send-finally: + method: POST + url: https://my/monitoring.example.com/ + body-template: body-template.json + headers: + - name: Content-Type + value: "application/json" + + +``` + +{{% /tab %}} +{{% tab title="hcl" %}} + +```hcl +"profile" { + + "backup" = { + "source" = "/source" + + "send-finally" = { + "method" = "POST" + "url" = "https://my/monitoring.example.com/" + "body-template" = "body-template.json" + "headers" = { + "name" = "Content-Type" + "value" = "application/json" + } + } + } +} +``` + +{{% /tab %}} +{{% tab title="json" %}} + +```json +{ + "version": "1", + "profile": { + "backup": { + "source": "/source", + "send-finally": { + "method": "POST", + "url": "https://my/monitoring.example.com/", + "body-template": "body-template.json", + "headers": [ + { + "name": "Content-Type", + "value": "application/json" + } + ] + } + } + } +} +``` + +{{% /tab %}} +{{< /tabs >}} diff --git a/docs/content/usage/hooks/healthchecks.md b/docs/content/usage/hooks/healthchecks.md new file mode 100644 index 000000000..5fe2966de --- /dev/null +++ b/docs/content/usage/hooks/healthchecks.md @@ -0,0 +1,255 @@ +--- +title: "Sending to healthchecks.io" +weight: 20 +--- + +For more information, refer to [Hooks - HTTP]({{% relref "configuration/hooks/http_hooks/#send-http-messages-before-and-after-a-job" %}}) + +{{< tabs groupid="config-with-json" >}} +{{% tab title="toml" %}} + +```toml +version = "1" + +[profile] + + [profile.backup] + source = "/source" + exclude = [ "/**/.git/" ] + schedule = [ "*:00,30" ] + schedule-permission = "user" + + # you can have more than one target + + [[profile.backup.send-before]] + method = "HEAD" + url = "https://hc-ping.com/831e288e-1293-46f8-ac31-70ea7f875650/start" + + [[profile.backup.send-before]] + method = "HEAD" + url = "https://httpstat.us/400" + + # you can have more than one target + + [[profile.backup.send-after]] + method = "HEAD" + url = "https://hc-ping.com/831e288e-1293-46f8-ac31-70ea7f875650" + + [[profile.backup.send-after]] + method = "HEAD" + url = "https://httpstat.us/500" + + [profile.backup.send-after-fail] + method = "POST" + url = "https://hc-ping.com/831e288e-1293-46f8-ac31-70ea7f875650/fail" + body = "${ERROR}\n\n${ERROR_STDERR}" + + [[profile.backup.send-after-fail.headers]] + name = "Content-Type" + value = "text/plain; charset=UTF-8" + + [profile.check] + schedule = [ "*:15" ] + + [profile.check.send-before] + method = "HEAD" + url = "https://hc-ping.com/e0f62e41-b75f-450f-8cdd-7f25e466d2dc/start" + + [profile.check.send-after] + method = "HEAD" + url = "https://hc-ping.com/e0f62e41-b75f-450f-8cdd-7f25e466d2dc" + + [profile.retention] + after-backup = true + + +``` + +{{% /tab %}} +{{% tab title="yaml" %}} + +```yaml +version: "1" + +profile: + + backup: + source: "/source" + exclude: + - "/**/.git/" + schedule: + - "*:00,30" + schedule-permission: user + + # you can have more than one target + send-before: + - method: HEAD + url: https://hc-ping.com/831e288e-1293-46f8-ac31-70ea7f875650/start + - method: HEAD + url: https://httpstat.us/400 + + # you can have more than one target + send-after: + - method: HEAD + url: https://hc-ping.com/831e288e-1293-46f8-ac31-70ea7f875650 + - method: HEAD + url: https://httpstat.us/500 + + send-after-fail: + method: POST + url: https://hc-ping.com/831e288e-1293-46f8-ac31-70ea7f875650/fail + body: "${ERROR}\n\n${ERROR_STDERR}" + headers: + - name: Content-Type + value: "text/plain; charset=UTF-8" + check: + schedule: + - "*:15" + + send-before: + method: HEAD + url: https://hc-ping.com/e0f62e41-b75f-450f-8cdd-7f25e466d2dc/start + + send-after: + method: HEAD + url: https://hc-ping.com/e0f62e41-b75f-450f-8cdd-7f25e466d2dc + + retention: + after-backup: true + +``` + +{{% /tab %}} +{{% tab title="hcl" %}} + +```hcl +"profile" { + + "backup" = { + "source" = "/source" + "exclude" = ["/**/.git/"] + "schedule" = ["*:00,30"] + "schedule-permission" = "user" + + "send-before" = { + "method" = "HEAD" + "url" = "https://hc-ping.com/831e288e-1293-46f8-ac31-70ea7f875650/start" + } + + "send-before" = { + "method" = "HEAD" + "url" = "https://httpstat.us/400" + } + + "send-after" = { + "method" = "HEAD" + "url" = "https://hc-ping.com/831e288e-1293-46f8-ac31-70ea7f875650" + } + + "send-after" = { + "method" = "HEAD" + "url" = "https://httpstat.us/500" + } + + "send-after-fail" = { + "method" = "POST" + "url" = "https://hc-ping.com/831e288e-1293-46f8-ac31-70ea7f875650/fail" + "body" = "${ERROR}\n\n${ERROR_STDERR}" + "headers" = { + "name" = "Content-Type" + "value" = "text/plain; charset=UTF-8" + } + } + } + + "check" = { + "schedule" = ["*:15"] + + "send-before" = { + "method" = "HEAD" + "url" = "https://hc-ping.com/e0f62e41-b75f-450f-8cdd-7f25e466d2dc/start" + } + + "send-after" = { + "method" = "HEAD" + "url" = "https://hc-ping.com/e0f62e41-b75f-450f-8cdd-7f25e466d2dc" + } + } + + "retention" = { + "after-backup" = true + } +} +``` + +{{% /tab %}} +{{% tab title="json" %}} + +```json +{ + "version": "1", + "profile": { + "backup": { + "source": "/source", + "exclude": [ + "/**/.git/" + ], + "schedule": [ + "*:00,30" + ], + "schedule-permission": "user", + "send-before": [ + { + "method": "HEAD", + "url": "https://hc-ping.com/831e288e-1293-46f8-ac31-70ea7f875650/start" + }, + { + "method": "HEAD", + "url": "https://httpstat.us/400" + } + ], + "send-after": [ + { + "method": "HEAD", + "url": "https://hc-ping.com/831e288e-1293-46f8-ac31-70ea7f875650" + }, + { + "method": "HEAD", + "url": "https://httpstat.us/500" + } + ], + "send-after-fail": { + "method": "POST", + "url": "https://hc-ping.com/831e288e-1293-46f8-ac31-70ea7f875650/fail", + "body": "${ERROR}\n\n${ERROR_STDERR}", + "headers": [ + { + "name": "Content-Type", + "value": "text/plain; charset=UTF-8" + } + ] + } + }, + "check": { + "schedule": [ + "*:15" + ], + "send-before": { + "method": "HEAD", + "url": "https://hc-ping.com/e0f62e41-b75f-450f-8cdd-7f25e466d2dc/start" + }, + "send-after": { + "method": "HEAD", + "url": "https://hc-ping.com/e0f62e41-b75f-450f-8cdd-7f25e466d2dc" + } + }, + "retention": { + "after-backup": true + } + } +} +``` + +{{% /tab %}} +{{< /tabs >}} + diff --git a/docs/content/usage/hooks/ntfy.md b/docs/content/usage/hooks/ntfy.md new file mode 100644 index 000000000..980f1844a --- /dev/null +++ b/docs/content/usage/hooks/ntfy.md @@ -0,0 +1,49 @@ +--- +title: "Using ntfy" +weight: 10 +--- + +This configuration executes the following: +- Makes a template profile called "default" [(learn more)]({{% relref "/configuration/variables/templates/" %}}) +- Creates a `backup` on a schedule every day at 12am [(learn more)]({{% relref "/configuration/schedules/configuration/" %}}) + - After a fail or success, sends a message through the `ntfy` service using a POST request [(learn more)]({{% relref "/configuration/hooks/http_hooks/" %}}) +- Creates a `forget` schedule that runs every day at 12:05am + - Prunes all non-matching entries [(learn more)]({{% relref "/reference/profile/forget/" %}}) + - Keeps every daily backup for the past week (7 backups), every weekly backup for the past month (4 backups), and a monthly backup for 75 years. + +{{< tabs groupid="example-ntfy" >}} +{{% tab title="toml" %}} +```toml +default: + insecure-no-password: true + backup: + schedule: "*-*-* 00:00:00" + schedule-permission: user_logged_on + send-after: + method: POST + url: [[YOUR_URL_HERE]] + headers: + - name: Title + value: "{{.Profile.Name}} ran successfully!" + - name: Priority + value: "low" + send-after-fail: + method: POST + url: [[YOUR_URL_HERE]] + headers: + - name: Title + value: "{{.Profile.Name}} failed!" + - name: Tags + value: "warning" + - name: Priority + value: "high" + forget: + schedule: "*-*-* 00:05:00" + schedule-permission: user_logged_on + keep-within-daily: '7d' + keep-within-weekly: '1m' + keep-within-monthly: '75y' + prune: true +``` +{{% /tab %}} +{{< /tabs >}} diff --git a/docs/content/usage/keyfile.md b/docs/content/usage/keyfile.md index 7cce3f509..8e078a96a 100644 --- a/docs/content/usage/keyfile.md +++ b/docs/content/usage/keyfile.md @@ -1,10 +1,8 @@ --- -title: "Generate a Keyfile" -weight: 10 +title: Generate Random Keys +weight: 60 --- -## Generating random keys - Resticprofile includes a tool to generate cryptographically secure, base64-encoded random keys. Save the key to a file and use it as a strong key for Restic. - On Linux, FreeBSD, Dragonfly, and Solaris, Reader uses `getrandom(2)`. diff --git a/docs/content/usage/memory.md b/docs/content/usage/memory.md deleted file mode 100644 index bc1a86d95..000000000 --- a/docs/content/usage/memory.md +++ /dev/null @@ -1,16 +0,0 @@ ---- -title: "Memory" -weight: 15 ---- - - -## Minimum memory required - -restic can be memory hungry. I'm running a few servers with no swap and I managed to kill some of them during a backup. - -For that matter I've introduced a parameter in the `global` section called `min-memory`. The **default value is 100MB**. You can disable it by using a value of `0`. - -It compares against `(total - used)` which is probably the best way to know how much memory is available (that is including the memory used for disk buffers/cache). - - - diff --git a/docs/content/usage/scheduling/_index.md b/docs/content/usage/scheduling/_index.md new file mode 100644 index 000000000..4672325e3 --- /dev/null +++ b/docs/content/usage/scheduling/_index.md @@ -0,0 +1,131 @@ +--- +title: "Scheduling" +weight: 20 +alwaysopen: false +--- + +Here's an example of a scheduling configuration: + +{{< tabs groupid="config-with-json" >}} +{{% tab title="toml" %}} + +```toml +[default] + repository = "d:\\backup" + password-file = "key" + +[self] + inherit = "default" + + [self.retention] + after-backup = true + keep-within = "14d" + + [self.backup] + source = "." + schedule = [ "Mon..Fri *:00,15,30,45", "Sat,Sun 0,12:00" ] + schedule-permission = "user" + schedule-lock-wait = "10m" + + [self.prune] + schedule = "sun 3:30" + schedule-permission = "user" + schedule-lock-wait = "1h" +``` + +{{% /tab %}} +{{% tab title="yaml" %}} + +```yaml +default: + repository: "d:\\backup" + password-file: key + +self: + inherit: default + retention: + after-backup: true + keep-within: 14d + backup: + source: "." + schedule: + - "Mon..Fri *:00,15,30,45" # every 15 minutes on weekdays + - "Sat,Sun 0,12:00" # twice a day on week-ends + schedule-permission: user + schedule-lock-wait: 10m + prune: + schedule: "sun 3:30" + schedule-permission: user + schedule-lock-wait: 1h +``` + +{{% /tab %}} +{{% tab title="hcl" %}} + +```hcl +"default" = { + "repository" = "d:\\backup" + "password-file" = "key" +} + +"self" = { + "inherit" = "default" + + "retention" = { + "after-backup" = true + "keep-within" = "14d" + } + + "backup" = { + "source" = "." + "schedule" = ["Mon..Fri *:00,15,30,45", "Sat,Sun 0,12:00"] + "schedule-permission" = "user" + "schedule-lock-wait" = "10m" + } + + "prune" = { + "schedule" = "sun 3:30" + "schedule-permission" = "user" + "schedule-lock-wait" = "1h" + } +} +``` + +{{% /tab %}} +{{% tab title="json" %}} + +```json +{ + "default": { + "repository": "d:\\backup", + "password-file": "key" + }, + "self": { + "inherit": "default", + "retention": { + "after-backup": true, + "keep-within": "14d" + }, + "backup": { + "source": ".", + "schedule": [ + "Mon..Fri *:00,15,30,45", + "Sat,Sun 0,12:00" + ], + "schedule-permission": "user", + "schedule-lock-wait": "10m" + }, + "prune": { + "schedule": "sun 3:30", + "schedule-permission": "user", + "schedule-lock-wait": "1h" + } + } +} +``` + +{{% /tab %}} +{{< /tabs >}} + +For more examples, refer to: +{{% children sort="weight" %}} diff --git a/docs/content/usage/scheduling/change-schedule.md b/docs/content/usage/scheduling/change-schedule.md new file mode 100644 index 000000000..f41c764a7 --- /dev/null +++ b/docs/content/usage/scheduling/change-schedule.md @@ -0,0 +1,12 @@ +--- +title: Changing Schedule Permission +weight: 30 +--- + +When you update the `schedule-permission` field, you **must** unschedule and reschedule the profile to implement the change. + +To make this change, follow these steps: + +- Unschedule the job first (before updating the permission in the configuration) +- Change your permission (user to system, or system to user). +- Schedule your updated profile. diff --git a/docs/content/usage/scheduling/linux.md b/docs/content/usage/scheduling/linux.md new file mode 100644 index 000000000..b825393b8 --- /dev/null +++ b/docs/content/usage/scheduling/linux.md @@ -0,0 +1,208 @@ +--- +title: "Example: Linux" +weight: 32 +--- + +With this example of configuration for Linux: + +{{< tabs groupid="config-with-json" >}} +{{% tab title="toml" %}} + +```toml +version = "1" + +[default] + password-file = "key" + repository = "/tmp/backup" + +[test1] + inherit = "default" + + [test1.backup] + source = "./" + schedule = "*:00,15,30,45" + schedule-permission = "user" + schedule-lock-wait = "15m" + + [test1.check] + schedule = "*-*-1" + schedule-permission = "user" + schedule-lock-wait = "15m" +``` + +{{% /tab %}} +{{% tab title="yaml" %}} + +```yaml +version: "1" + +default: + password-file: key + repository: /tmp/backup + +test1: + inherit: default + backup: + source: ./ + schedule: "*:00,15,30,45" + schedule-permission: user + schedule-lock-wait: 15m + check: + schedule: "*-*-1" + schedule-permission: user + schedule-lock-wait: 15m + +``` + +{{% /tab %}} +{{% tab title="hcl" %}} + +```hcl +"default" = { + "password-file" = "key" + "repository" = "/tmp/backup" +} + +"test1" = { + "inherit" = "default" + + "backup" = { + "source" = "./" + "schedule" = "*:00,15,30,45" + "schedule-permission" = "user" + "schedule-lock-wait" = "15m" + } + + "check" = { + "schedule" = "*-*-1" + "schedule-permission" = "user" + "schedule-lock-wait" = "15m" + } +} +``` + +{{% /tab %}} +{{% tab title="json" %}} + +```json +{ + "version": "1", + "default": { + "password-file": "key", + "repository": "/tmp/backup" + }, + "test1": { + "inherit": "default", + "backup": { + "source": "./", + "schedule": "*:00,15,30,45", + "schedule-permission": "user", + "schedule-lock-wait": "15m" + }, + "check": { + "schedule": "*-*-1", + "schedule-permission": "user", + "schedule-lock-wait": "15m" + } + } +} +``` + +{{% /tab %}} +{{< /tabs >}} + + +```shell +$ resticprofile -c examples/linux.yaml -n test1 schedule + +Analyzing backup schedule 1/1 +================================= + Original form: *:00,15,30,45 +Normalized form: *-*-* *:00,15,30,45:00 + Next elapse: Thu 2020-07-23 17:15:00 BST + (in UTC): Thu 2020-07-23 16:15:00 UTC + From now: 6min left + +2020/07/23 17:08:51 writing /home/user/.config/systemd/user/resticprofile-backup@profile-test1.service +2020/07/23 17:08:51 writing /home/user/.config/systemd/user/resticprofile-backup@profile-test1.timer +Created symlink /home/user/.config/systemd/user/timers.target.wants/resticprofile-backup@profile-test1.timer → /home/user/.config/systemd/user/resticprofile-backup@profile-test1.timer. +2020/07/23 17:08:51 scheduled job test1/backup created + +Analyzing check schedule 1/1 +================================= + Original form: *-*-1 +Normalized form: *-*-01 00:00:00 + Next elapse: Sat 2020-08-01 00:00:00 BST + (in UTC): Fri 2020-07-31 23:00:00 UTC + From now: 1 weeks 1 days left + +2020/07/23 17:08:51 writing /home/user/.config/systemd/user/resticprofile-check@profile-test1.service +2020/07/23 17:08:51 writing /home/user/.config/systemd/user/resticprofile-check@profile-test1.timer +Created symlink /home/user/.config/systemd/user/timers.target.wants/resticprofile-check@profile-test1.timer → /home/user/.config/systemd/user/resticprofile-check@profile-test1.timer. +2020/07/23 17:08:51 scheduled job test1/check created +``` + +The `status` command shows a combination of `journalctl` displaying errors (only) in the last month and `systemctl status`: + +```shell +$ resticprofile -c examples/linux.yaml -n test1 status + +Analyzing backup schedule 1/1 +================================= + Original form: *:00,15,30,45 +Normalized form: *-*-* *:00,15,30,45:00 + Next elapse: Tue 2020-07-28 15:15:00 BST + (in UTC): Tue 2020-07-28 14:15:00 UTC + From now: 4min 44s left + +Recent log (>= warning in the last month) +========================================== +-- Logs begin at Wed 2020-06-17 11:09:19 BST, end at Tue 2020-07-28 15:10:10 BST. -- +Jul 27 20:48:01 Desktop76 systemd[2986]: Failed to start resticprofile backup for profile test1 in examples/linux.yaml. +Jul 27 21:00:55 Desktop76 systemd[2986]: Failed to start resticprofile backup for profile test1 in examples/linux.yaml. +Jul 27 21:15:34 Desktop76 systemd[2986]: Failed to start resticprofile backup for profile test1 in examples/linux.yaml. + +Systemd timer status +===================== +● resticprofile-backup@profile-test1.timer - backup timer for profile test1 in examples/linux.yaml + Loaded: loaded (/home/user/.config/systemd/user/resticprofile-backup@profile-test1.timer; enabled; vendor preset: enabled) + Active: active (waiting) since Tue 2020-07-28 15:10:06 BST; 8s ago + Trigger: Tue 2020-07-28 15:15:00 BST; 4min 44s left + +Jul 28 15:10:06 Desktop76 systemd[2951]: Started backup timer for profile test1 in examples/linux.yaml. + + +Analyzing check schedule 1/1 +================================= + Original form: *-*-1 +Normalized form: *-*-01 00:00:00 + Next elapse: Sat 2020-08-01 00:00:00 BST + (in UTC): Fri 2020-07-31 23:00:00 UTC + From now: 3 days left + +Recent log (>= warning in the last month) +========================================== +-- Logs begin at Wed 2020-06-17 11:09:19 BST, end at Tue 2020-07-28 15:10:10 BST. -- +Jul 27 19:39:59 Desktop76 systemd[2986]: Failed to start resticprofile check for profile test1 in examples/linux.yaml. + +Systemd timer status +===================== +● resticprofile-check@profile-test1.timer - check timer for profile test1 in examples/linux.yaml + Loaded: loaded (/home/user/.config/systemd/user/resticprofile-check@profile-test1.timer; enabled; vendor preset: enabled) + Active: active (waiting) since Tue 2020-07-28 15:10:07 BST; 7s ago + Trigger: Sat 2020-08-01 00:00:00 BST; 3 days left + +Jul 28 15:10:07 Desktop76 systemd[2951]: Started check timer for profile test1 in examples/linux.yaml. + + +``` + +And `unschedule`: + +```shell +$ resticprofile -c examples/linux.yaml -n test1 unschedule +Removed /home/user/.config/systemd/user/timers.target.wants/resticprofile-backup@profile-test1.timer. +2020/07/23 17:13:42 scheduled job test1/backup removed +Removed /home/user/.config/systemd/user/timers.target.wants/resticprofile-check@profile-test1.timer. +2020/07/23 17:13:42 scheduled job test1/check removed +``` diff --git a/docs/content/usage/scheduling/macos.md b/docs/content/usage/scheduling/macos.md new file mode 100644 index 000000000..339c6e94a --- /dev/null +++ b/docs/content/usage/scheduling/macos.md @@ -0,0 +1,91 @@ +--- +title: "Example: MacOS" +weight: 33 +--- + +Here's an example of scheduling a backup profile named `azure-dev`: + +```shell +% resticprofile -n azure-dev schedule +2025/03/30 18:40:13 using configuration file: profiles.yaml + +Profile (or Group) azure-dev: backup schedule +============================================= + Original form: 22:22 +Normalized form: *-*-* 22:22:00 + Next elapse: Sun Mar 30 22:22:00 BST 2025 + (in UTC): Sun Mar 30 21:22:00 UTC 2025 + From now: 3h41m46s left + +2025/03/30 18:40:13 scheduled job azure-dev/backup created + +``` + +{{% notice info %}} + +In some cases, macOS may request permission to access the network, an external volume (like a USB drive), or a protected directory. A message window will appear while the backup runs in the background. +{{% /notice %}} + +![access files popup window]({{< absolute "configuration/schedules/removable_volume.png" nohtml >}}) + +You may want to start the task now so you can grant special permissions: + +1. Retrieve the task name using the status command: + +```shell +% resticprofile -n azure-dev status +2025/03/30 18:40:21 using configuration file: profiles.yaml + +Profile (or Group) azure-dev: backup schedule +============================================= + Original form: *-*-* 22:22:00 +Normalized form: *-*-* 22:22:00 + Next elapse: Sun Mar 30 22:22:00 BST 2025 + (in UTC): Sun Mar 30 21:22:00 UTC 2025 + From now: 3h41m38s left + + service: local.resticprofile.azure-dev.backup + permission: user + program: /usr/local/bin/resticprofile + working directory: /Users/cp/resticprofile + stdout path: local.resticprofile.azure-dev.backup.log + stderr path: local.resticprofile.azure-dev.backup.log + state: not running + runs (*): 0 + last exit code (*): (never exited) + * : since last (re)schedule or last reboot +``` + +The name of the task can be seen on the line `service: ...` + +2. start the task manually + +```shell +% launchctl start local.resticprofile.azure-dev.backup +``` + +You can check the task is currently running: + +```shell +2025/03/30 18:42:07 using configuration file: profiles.yaml + +Profile (or Group) azure-dev: backup schedule +============================================= + Original form: *-*-* 22:22:00 +Normalized form: *-*-* 22:22:00 + Next elapse: Sun Mar 30 22:22:00 BST 2025 + (in UTC): Sun Mar 30 21:22:00 UTC 2025 + From now: 3h39m52s left + + service: local.resticprofile.azure-dev.backup + permission: user + program: /usr/local/bin/resticprofile + working directory: /Users/cp/resticprofile + stdout path: local.resticprofile.azure-dev.backup.log + stderr path: local.resticprofile.azure-dev.backup.log + state: running + runs (*): 1 + last exit code (*): (never exited) + * : since last (re)schedule or last reboot + +``` diff --git a/docs/content/usage/scheduling/windows.md b/docs/content/usage/scheduling/windows.md new file mode 100644 index 000000000..cafd502e1 --- /dev/null +++ b/docs/content/usage/scheduling/windows.md @@ -0,0 +1,109 @@ +--- +title: "Example: Windows" +weight: 31 +--- + +{{% notice note %}} +If you create a task with `user` permission under Windows, you will need to enter your password to validate the task. + +It's a requirement of the task scheduler. I'm inviting you to review the code to make sure I'm not emailing your password to myself. Seriously you shouldn't trust anyone. +{{% /notice %}} + +Example of the `schedule` command under Windows (with git bash): + +```shell +$ resticprofile -c examples/windows.yaml -n self schedule + +Analyzing backup schedule 1/2 +================================= + Original form: Mon..Fri *:00,15,30,45 +Normalized form: Mon..Fri *-*-* *:00,15,30,45:00 + Next elapse: Wed Jul 22 21:30:00 BST 2020 + (in UTC): Wed Jul 22 20:30:00 UTC 2020 + From now: 1m52s left + +Analyzing backup schedule 2/2 +================================= + Original form: Sat,Sun 0,12:00 +Normalized form: Sat,Sun *-*-* 00,12:00:00 + Next elapse: Sat Jul 25 00:00:00 BST 2020 + (in UTC): Fri Jul 24 23:00:00 UTC 2020 + From now: 50h31m52s left + +Creating task for user Creative Projects +Task Scheduler requires your Windows password to validate the task: + +2020/07/22 21:28:15 scheduled job self/backup created + +Analyzing retention schedule 1/1 +================================= + Original form: sun 3:30 +Normalized form: Sun *-*-* 03:30:00 + Next elapse: Sun Jul 26 03:30:00 BST 2020 + (in UTC): Sun Jul 26 02:30:00 UTC 2020 + From now: 78h1m44s left + +2020/07/22 21:28:22 scheduled job self/retention created +``` + +To see the status of the triggers, you can use the `status` command: + +```shell +$ resticprofile -c examples/windows.yaml -n self status + +Analyzing backup schedule 1/2 +================================= + Original form: Mon..Fri *:00,15,30,45 +Normalized form: Mon..Fri *-*-* *:00,15,30,45:00 + Next elapse: Wed Jul 22 21:30:00 BST 2020 + (in UTC): Wed Jul 22 20:30:00 UTC 2020 + From now: 14s left + +Analyzing backup schedule 2/2 +================================= + Original form: Sat,Sun 0,12:* +Normalized form: Sat,Sun *-*-* 00,12:*:00 + Next elapse: Sat Jul 25 00:00:00 BST 2020 + (in UTC): Fri Jul 24 23:00:00 UTC 2020 + From now: 50h29m46s left + + Task: \resticprofile backup\self backup + User: Creative Projects + Working Dir: D:\Source\resticprofile + Exec: D:\Source\resticprofile\resticprofile.exe --no-ansi --config examples/windows.yaml --name self backup + Enabled: true + State: ready + Missed runs: 0 + Last Run Time: 2020-07-22 21:30:00 +0000 UTC + Last Result: 0 + Next Run Time: 2020-07-22 21:45:00 +0000 UTC + +Analyzing retention schedule 1/1 +================================= + Original form: sun 3:30 +Normalized form: Sun *-*-* 03:30:00 + Next elapse: Sun Jul 26 03:30:00 BST 2020 + (in UTC): Sun Jul 26 02:30:00 UTC 2020 + From now: 77h59m46s left + + Task: \resticprofile backup\self retention + User: Creative Projects + Working Dir: D:\Source\resticprofile + Exec: D:\Source\resticprofile\resticprofile.exe --no-ansi --config examples/windows.yaml --name self forget + Enabled: true + State: ready + Missed runs: 0 + Last Run Time: 1999-11-30 00:00:00 +0000 UTC + Last Result: 267011 + Next Run Time: 2020-07-26 03:30:00 +0000 UTC + +``` + +To remove the schedule, use the `unschedule` command: + +```shell +$ resticprofile -c examples/windows.yaml -n self unschedule +2020/07/22 21:34:51 scheduled job self/backup removed +2020/07/22 21:34:51 scheduled job self/retention removed +``` + diff --git a/docs/content/usage/variables/_index.md b/docs/content/usage/variables/_index.md new file mode 100644 index 000000000..302685e62 --- /dev/null +++ b/docs/content/usage/variables/_index.md @@ -0,0 +1,8 @@ +--- +title: "Variables" +weight: 30 +alwaysopen: false +--- + +{{% children sort="weight" %}} + diff --git a/docs/content/usage/variables/inheritance-and-variables.md b/docs/content/usage/variables/inheritance-and-variables.md new file mode 100644 index 000000000..792c81d9c --- /dev/null +++ b/docs/content/usage/variables/inheritance-and-variables.md @@ -0,0 +1,282 @@ +--- +title: "Using inheritance and variables" +--- +You can use a combination of inheritance and variables in the resticprofile configuration file like so: + +{{< tabs groupid="config-with-json" >}} +{{% tab title="toml" %}} + +```toml +version = "1" + +[generic] + password-file = "{{ .ConfigDir }}/{{ .Profile.Name }}-key" + repository = "/backup/{{ .Now.Weekday }}" + lock = "$HOME/resticprofile-profile-{{ .Profile.Name }}.lock" + initialize = true + + [generic.backup] + check-before = true + exclude = [ "/**/.git" ] + exclude-caches = true + one-file-system = false + run-after = "echo All Done!" + run-before = [ + "echo Hello {{ .Env.LOGNAME }}", + "echo current dir: {{ .CurrentDir }}", + "echo config dir: {{ .ConfigDir }}", + "echo profile started at {{ .Now.Format "02 Jan 06 15:04 MST" }}" + ] + tag = [ "{{ .Profile.Name }}", "dev" ] + + [generic.retention] + after-backup = true + before-backup = false + compact = false + keep-within = "30d" + prune = true + tag = [ "{{ .Profile.Name }}", "dev" ] + + [generic.snapshots] + tag = [ "{{ .Profile.Name }}", "dev" ] + +[src] + inherit = "generic" + + [src.backup] + source = [ "{{ .Env.HOME }}/go/src" ] + + [src.check] + # Weekday is an integer from 0 to 6 (starting from Sunday) + # Nice trick to add 1 to an integer: https://stackoverflow.com/a/72465098 + read-data-subset = "{{ len (printf "a%*s" .Now.Weekday "") }}/7" + +``` + +{{% /tab %}} +{{% tab title="yaml" %}} + +```yaml +--- +version: "1" + +generic: + password-file: "{{ .ConfigDir }}/{{ .Profile.Name }}-key" + repository: "/backup/{{ .Now.Weekday }}" + lock: "$HOME/resticprofile-profile-{{ .Profile.Name }}.lock" + initialize: true + + backup: + check-before: true + exclude: + - /**/.git + exclude-caches: true + one-file-system: false + run-after: echo All Done! + run-before: + - "echo Hello {{ .Env.LOGNAME }}" + - "echo current dir: {{ .CurrentDir }}" + - "echo config dir: {{ .ConfigDir }}" + - "echo profile started at {{ .Now.Format "02 Jan 06 15:04 MST" }}" + tag: + - "{{ .Profile.Name }}" + - dev + + retention: + after-backup: true + before-backup: false + compact: false + keep-within: 30d + prune: true + tag: + - "{{ .Profile.Name }}" + - dev + + snapshots: + tag: + - "{{ .Profile.Name }}" + - dev + +src: + inherit: generic + + backup: + source: + - "{{ .Env.HOME }}/go/src" + + check: + # Weekday is an integer from 0 to 6 (starting from Sunday) + # Nice trick to add 1 to an integer: https://stackoverflow.com/a/72465098 + read-data-subset: "{{ len (printf "a%*s" .Now.Weekday "") }}/7" + +``` + +{{% /tab %}} +{{% tab title="hcl" %}} + +```hcl +"generic" = { + "password-file" = "{{ .ConfigDir }}/{{ .Profile.Name }}-key" + "repository" = "/backup/{{ .Now.Weekday }}" + "lock" = "$HOME/resticprofile-profile-{{ .Profile.Name }}.lock" + "initialize" = true + + "backup" = { + "check-before" = true + "exclude" = ["/**/.git"] + "exclude-caches" = true + "one-file-system" = false + "run-after" = "echo All Done!" + "run-before" = ["echo Hello {{ .Env.LOGNAME }}", "echo current dir: {{ .CurrentDir }}", "echo config dir: {{ .ConfigDir }}", "echo profile started at {{ .Now.Format "02 Jan 06 15:04 MST" }}"] + "tag" = ["{{ .Profile.Name }}", "dev"] + } + + "retention" = { + "after-backup" = true + "before-backup" = false + "compact" = false + "keep-within" = "30d" + "prune" = true + "tag" = ["{{ .Profile.Name }}", "dev"] + } + + "snapshots" = { + "tag" = ["{{ .Profile.Name }}", "dev"] + } +} + +"src" = { + "inherit" = "generic" + + "backup" = { + "source" = ["{{ .Env.HOME }}/go/src"] + } + + "check" = { + # Weekday is an integer from 0 to 6 (starting from Sunday) + # Nice trick to add 1 to an integer: https://stackoverflow.com/a/72465098 + "read-data-subset" = "{{ len (printf "a%*s" .Now.Weekday "") }}/7" + } +} +``` + +{{% /tab %}} +{{% tab title="json" %}} + +```json +{ + "version": "1", + "generic": { + "password-file": "{{ .ConfigDir }}/{{ .Profile.Name }}-key", + "repository": "/backup/{{ .Now.Weekday }}", + "lock": "$HOME/resticprofile-profile-{{ .Profile.Name }}.lock", + "initialize": true, + "backup": { + "check-before": true, + "exclude": [ + "/**/.git" + ], + "exclude-caches": true, + "one-file-system": false, + "run-after": "echo All Done!", + "run-before": [ + "echo Hello {{ .Env.LOGNAME }}", + "echo current dir: {{ .CurrentDir }}", + "echo config dir: {{ .ConfigDir }}", + "echo profile started at {{ .Now.Format "02 Jan 06 15:04 MST" }}" + ], + "tag": [ + "{{ .Profile.Name }}", + "dev" + ] + }, + "retention": { + "after-backup": true, + "before-backup": false, + "compact": false, + "keep-within": "30d", + "prune": true, + "tag": [ + "{{ .Profile.Name }}", + "dev" + ] + }, + "snapshots": { + "tag": [ + "{{ .Profile.Name }}", + "dev" + ] + } + }, + "src": { + "inherit": "generic", + "backup": { + "source": [ + "{{ .Env.HOME }}/go/src" + ] + }, + "check": { + "read-data-subset": "{{ len (printf "a%*s" .Now.Weekday "") }}/7" + } + } +} +``` + +{{% /tab %}} +{{< /tabs >}} + +This is obviously not a real world example, but it shows many of the possibilities you can do with variable expansion. + +To check the generated configuration, you can use the resticprofile `show` command: + +```shell +% resticprofile -c examples/template.yaml -n src show + +global: + default-command: snapshots + restic-lock-retry-after: 1m0s + restic-stale-lock-age: 2h0m0s + min-memory: 100 + send-timeout: 30s + +profile src: + repository: /backup/Monday + password-file: /Users/CP/go/src/resticprofile/examples/src-key + initialize: true + lock: /Users/CP/resticprofile-profile-src.lock + + backup: + check-before: true + run-before: echo Hello CP + echo current dir: /Users/CP/go/src/resticprofile + echo config dir: /Users/CP/go/src/resticprofile/examples + echo profile started at 05 Sep 22 17:39 BST + run-after: echo All Done! + source: /Users/CP/go/src + exclude: /**/.git + exclude-caches: true + tag: src + dev + + retention: + after-backup: true + keep-within: 30d + path: /Users/CP/go/src + prune: true + tag: src + dev + + check: + read-data-subset: 2/7 + + snapshots: + tag: src + dev +``` + +As you can see, the `src` profile inherited from the `generic` profile. The tags `{{ .Profile.Name }}` got replaced by the name of the current profile `src`. +Now you can reuse the same generic configuration in another profile. + +You might have noticed the `read-data-subset` in the `check` section which will read a seventh of the data every day, meaning the whole repository data will be checked over a week. You can find [more information about this trick](https://stackoverflow.com/a/72465098). + +More examples can be found [here]({{% relref "/usage/variables/list-variables" %}}). diff --git a/docs/content/usage/variables/list-variables.md b/docs/content/usage/variables/list-variables.md new file mode 100644 index 000000000..9a90abc4f --- /dev/null +++ b/docs/content/usage/variables/list-variables.md @@ -0,0 +1,189 @@ +--- +title: "More on Variables" +--- + +{{< toc >}} + +## Hand-made variables +Here's an example of a configuration on Linux where I use a variable `$mountpoint` set to a USB drive mount point: + +{{< tabs groupid="config-with-json" >}} +{{% tab title="toml" %}} + +```toml +version = "1" + +[global] + priority = "low" + +{{ $mountpoint := "/mnt/external" }} + +[default] + repository = "local:{{ $mountpoint }}/backup" + password-file = "key" + run-before = "mount {{ $mountpoint }}" + run-after = "umount {{ $mountpoint }}" + run-after-fail = "umount {{ $mountpoint }}" + + [default.backup] + exclude-caches = true + source = [ "/etc", "/var/lib/libvirt" ] + check-after = true +``` + +{{% /tab %}} +{{% tab title="yaml" %}} + +```yaml +version: "1" + +global: + priority: low + +{{ $mountpoint := "/mnt/external" }} + +default: + repository: 'local:{{ $mountpoint }}/backup' + password-file: key + run-before: 'mount {{ $mountpoint }}' + run-after: 'umount {{ $mountpoint }}' + run-after-fail: 'umount {{ $mountpoint }}' + + backup: + exclude-caches: true + source: + - /etc + - /var/lib/libvirt + check-after: true +``` + +{{% /tab %}} +{{% tab title="hcl" %}} + +```hcl +global { + priority = "low" +} + +{{ $mountpoint := "/mnt/external" }} + +default { + repository = "local:{{ $mountpoint }}/backup" + password-file = "key" + run-before = "mount {{ $mountpoint }}" + run-after = "umount {{ $mountpoint }}" + run-after-fail = "umount {{ $mountpoint }}" + + backup { + exclude-caches = true + source = [ "/etc", "/var/lib/libvirt" ] + check-after = true + } +} + +``` + +{{% /tab %}} +{{% tab title="json" %}} + +```json +{{ $mountpoint := "/mnt/external" }} +{ + "version": "1", + "global": { + "priority": "low" + }, + "default": { + "repository": "local:{{ $mountpoint }}/backup", + "password-file": "key", + "run-before": "mount {{ $mountpoint }}", + "run-after": "umount {{ $mountpoint }}", + "run-after-fail": "umount {{ $mountpoint }}", + "backup": { + "exclude-caches": true, + "source": [ + "/etc", + "/var/lib/libvirt" + ], + "check-after": true + } + } +} +``` + +{{% /tab %}} +{{< /tabs >}} + +## Runtime variable expansion + +Backup current dir (`$PWD`) but prevent backup of `$HOME` where the repository is located: + +{{< tabs groupid="config-with-json" >}} +{{% tab title="toml" %}} + +```toml +version = "1" + +[default] + repository = "local:${HOME}/backup" + password-file = "${HOME}/backup.key" + + [default.backup] + source = "$PWD" + exclude = ["$HOME/**", ".*", "~*"] + +``` + +{{% /tab %}} +{{% tab title="yaml" %}} + +```yaml +version: "1" + +default: + repository: 'local:${HOME}/backup' + password-file: '${HOME}/backup.key' + + backup: + source: '$PWD' + exclude: ['$HOME/**', '.*', '~*'] + +``` + +{{% /tab %}} +{{% tab title="hcl" %}} + +```hcl +default { + repository = "local:${HOME}/backup" + password-file = "${HOME}/backup.key" + + backup { + source = [ "$PWD" ] + exclude = [ "$HOME/**", ".*", "~*" ] + } +} +``` + +{{% /tab %}} +{{% tab title="json" %}} + +```json +{ + "default": { + "repository": "local:${HOME}/backup", + "password-file": "${HOME}/backup.key", + "backup": { + "source": [ "$PWD" ], + "exclude": [ "$HOME/**", ".*", "~*" ] + } + } +} +``` + +{{% /tab %}} +{{< /tabs >}} + +{{% notice style="tip" %}} +Use `$$` to escape a single `$` in configuration values that support variable expansion. E.g. on Windows you might want to exclude `$RECYCLE.BIN`. Specify it as: `exclude = ["$$RECYCLE.BIN"]`. +{{% /notice %}} diff --git a/docs/content/usage/version.md b/docs/content/usage/version.md deleted file mode 100644 index d842a2942..000000000 --- a/docs/content/usage/version.md +++ /dev/null @@ -1,10 +0,0 @@ ---- -title: "Version" -weight: 5 ---- - -The `version` command displays resticprofile version. If run in verbose mode (using `--verbose` flag) additional information such as OS version, golang version and modules are displayed as well. - -```shell -resticprofile --verbose version -``` \ No newline at end of file diff --git a/docs/layouts/shortcodes/toc.html b/docs/layouts/shortcodes/toc.html new file mode 100644 index 000000000..ba789ee77 --- /dev/null +++ b/docs/layouts/shortcodes/toc.html @@ -0,0 +1 @@ +{{ .Page.TableOfContents }} diff --git a/docs/themes/hugo-theme-relearn b/docs/themes/hugo-theme-relearn index e812df032..d9ca8e8d8 160000 --- a/docs/themes/hugo-theme-relearn +++ b/docs/themes/hugo-theme-relearn @@ -1 +1 @@ -Subproject commit e812df032eeaf0f4dc8880bce350b9c07be3abf4 +Subproject commit d9ca8e8d8f592909707aef80e9a1a6b5b37b4b78