Skip to content

Schedule: Capture os.Environ on schedule creation#212

Merged
jkellerer merged 3 commits intocreativeprojects:masterfrom
jkellerer:enh-capture-schedule-env
Aug 1, 2023
Merged

Schedule: Capture os.Environ on schedule creation#212
jkellerer merged 3 commits intocreativeprojects:masterfrom
jkellerer:enh-capture-schedule-env

Conversation

@jkellerer
Copy link
Collaborator

Fixes: #211

The PR adds a new config option schedule-capture-environment defaulting to RESTIC_* which enables OS environment capturing for schedules at creation time.

@creativeprojects - what do you think (it is an enhancement to the existing capture of the profile env)?
Does this work with #146 ?

@jkellerer jkellerer added the enhancement New feature or request label May 23, 2023
@codecov
Copy link

codecov bot commented May 23, 2023

Codecov Report

Patch coverage: 96.58% and project coverage change: +0.23% 🎉

Comparison is base (b00c8a0) 77.20% compared to head (f35395d) 77.43%.
Report is 3 commits behind head on master.

❗ Current head f35395d differs from pull request most recent head 68fa169. Consider uploading reports for the commit 68fa169 to get more accurate results

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #212      +/-   ##
==========================================
+ Coverage   77.20%   77.43%   +0.23%     
==========================================
  Files          92       93       +1     
  Lines        9949     9984      +35     
==========================================
+ Hits         7681     7731      +50     
+ Misses       2003     1990      -13     
+ Partials      265      263       -2     
Flag Coverage Δ
unittests 77.43% <96.58%> (+0.23%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Changed Coverage Δ
config/schedule_config.go 59.62% <ø> (ø)
schedule/handler_systemd.go 19.15% <0.00%> (-0.10%) ⬇️
config/profile.go 93.10% <90.62%> (+0.14%) ⬆️
schedule/handler_darwin.go 49.58% <100.00%> (ø)
systemd/generate.go 83.18% <100.00%> (ø)
util/env.go 100.00% <100.00%> (ø)
util/templates/data.go 100.00% <100.00%> (+13.73%) ⬆️
wrapper.go 85.50% <100.00%> (-0.15%) ⬇️

... and 3 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@jkellerer jkellerer mentioned this pull request May 24, 2023
@jkellerer
Copy link
Collaborator Author

@creativeprojects , btw. not all platforms have case insensitive variables. But this PR makes env vars case insensitive in all cases. Should we differentiate it by platform?

(I think yes we should)

@creativeprojects
Copy link
Owner

what do you think (it is an enhancement to the existing capture of the profile env)? Does this work with #146 ?

I don't think it would clash with #146 so we're good 👍🏻

not all platforms have case insensitive variables. But this PR makes env vars case insensitive in all cases. Should we differentiate it by platform?

Yeah probably 😐 I'm not too familiar with powershell but I'm pretty sure that could cause some issues.

@jkellerer jkellerer force-pushed the enh-capture-schedule-env branch 3 times, most recently from 93ae8f9 to f35395d Compare July 29, 2023 20:55
@jkellerer
Copy link
Collaborator Author

Env vars will now preserve the case. Case folding only applies on Windows. Will manually test the changes first.

Trying to preserve the case of environment variables where possible.
Also using schedule config environment in systemd and launchd.
@jkellerer jkellerer force-pushed the enh-capture-schedule-env branch from f35395d to 67ac710 Compare July 29, 2023 21:17
@jkellerer
Copy link
Collaborator Author

jkellerer commented Jul 31, 2023

Works fine in launchd and systemd.

Manual test was performed in user home with:

version: "2"

profiles:
  default:
    initialize: true
    repository: 'local:{{.ConfigDir}}/test-repo'
    password-file: 'profiles.key'

    backup:
      run-before:
        - 'echo "Var is [$RESTIC_VAR]" > "{{.ConfigDir}}/test.txt"'
      source:
        - '{{.ConfigDir}}/test.txt'
      schedule: "hourly"

systemd:

$ resticprofile generate --random-key > profiles.key
$ RESTIC_VAR=1 resticprofile schedule
$ resticprofile backup
$ cat test.txt
Var is []
$ systemctl start --user resticprofile-backup@profile-default.service
$ cat test.txt
Var is [1]

launchd:

$ resticprofile generate --random-key > profiles.key
$ RESTIC_VAR=1 resticprofile schedule
$ resticprofile backup
$ cat test.txt
Var is []
$ launchctl start local.resticprofile.default.backup
$ cat test.txt
Var is [1]

@jkellerer
Copy link
Collaborator Author

jkellerer commented Aug 1, 2023

Think it can be merged. @creativeprojects , what do you think?

Please check the environment variables part. I think it works correct now, given the limitation that the config doesn't allow to specify the case, but all unix OS are case sensitive for env vars.

The implementation was adjusted to this:

  • If the config defines a variable that exists in the environment, the config value overrides the existing var when the name matches. Case is not considered, there is a resolve step for this that adjustes the name.
  • If the config defines a new variable, it will be all upper-cased like it was before.
  • This change does affect templates, now they offer the actual case and the uppercase name. If Var, var, VAR is defined on a unix OS, all 3 variables would have different values.

Copy link
Owner

@creativeprojects creativeprojects left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes it works well, nice one!

I think we're good to go with this one as well 👍🏻

Thank you for the great work 😉

@creativeprojects creativeprojects added this to the v0.23.0 milestone Aug 1, 2023
@jkellerer jkellerer merged commit 347501d into creativeprojects:master Aug 1, 2023
@jkellerer jkellerer deleted the enh-capture-schedule-env branch August 1, 2023 22:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Issue with systemd

2 participants