Skip to content

etc: harden litestream.service systemd unit#1364

Open
Self-Perfection wants to merge 2 commits into
benbjohnson:mainfrom
Self-Perfection:harden-systemd-unit
Open

etc: harden litestream.service systemd unit#1364
Self-Perfection wants to merge 2 commits into
benbjohnson:mainfrom
Self-Perfection:harden-systemd-unit

Conversation

@Self-Perfection

@Self-Perfection Self-Perfection commented Jul 12, 2026

Copy link
Copy Markdown

Summary

Harden the packaged etc/litestream.service systemd unit. It currently runs the
daemon as root with no sandboxing; this adds standard systemd confinement in two
reviewable commits without changing how Litestream reaches its databases.

Problem

The shipped unit applies zero confinement:

[Service]
Restart=always
ExecStart=/usr/bin/litestream replicate

systemd-analyze security litestream.service rates it 9.6 "UNSAFE" — full
capability set, all address families, writable /, kernel tunables/modules
reachable, no syscall filter, etc.

Solution

Two commits, layered by blast radius:

  1. Conservative baseline — directives that need no per-deployment tuning and
    cannot change how the daemon accesses its data: NoNewPrivileges,
    ProtectSystem=full, the Protect*/Restrict* kernel & namespace set,
    ProtectProc=invisible/ProcSubset=pid, PrivateDevices,
    SystemCallArchitectures=native. Safe to ship to every existing install on
    upgrade.

  2. Deployment-aware directives — the ones with a caveat, each documented
    inline: CapabilityBoundingSet (see below), PrivateTmp,
    MemoryDenyWriteExecute, RestrictAddressFamilies, SystemCallFilter,
    UMask, plus commented User=/ProtectSystem=strict/ReadWritePaths=/
    ProtectHome= knobs.

Result: 9.6 UNSAFE → 2.2 OK on the loaded unit (still running as root; the
remaining exposure is User=/PrivateNetwork=/ProtectHome=, which are
deployment choices).

The one real regression we guarded against

An empty CapabilityBoundingSet= (the obvious "drop everything" choice)
breaks the common topology where the app writes its SQLite DB as its own user in
a 0750 directory and Litestream runs as root: dropping CAP_DAC_OVERRIDE
subjects root to normal permission checks, so it can't even stat the DB:

system=store db=events.db error="stat /var/lib/.../events.db: permission denied"

Litestream must also create a .<db>-litestream sidecar dir plus -wal/-shm
in that directory (db.go NewDB / metaPath). We therefore keep
CAP_DAC_OVERRIDE CAP_DAC_READ_SEARCH CAP_CHOWN CAP_FOWNER, and document that
setting User= to the DB owner lets operators drop them entirely.

Scope

In scope:

  • etc/litestream.service hardening only.

Not in scope (left as documented, commented knobs for the operator):

  • User=/DynamicUser= (DB ownership is deployment-specific).
  • ProtectSystem=strict + ReadWritePaths= (needs the DB path).
  • ProtectHome=, PrivateNetwork= (depend on DB/replica location).
  • No Go code changes; no behavior change to the binary.

Test Plan

No Go code is touched, so there is no go test for this change; verification is
against the unit itself. Reproducible commands:

# 1. Unit is valid
systemd-analyze verify etc/litestream.service

# 2. Exposure score (authoritative on the loaded unit)
sudo cp etc/litestream.service /etc/systemd/system/ && sudo systemctl daemon-reload
systemd-analyze security litestream.service

# 3. Regression guard: root + app-owned DB must still be readable under the
#    shipped CapabilityBoundingSet, and demonstrably fails with an empty one.
install -d -o nobody -g nogroup -m 0750 /var/lib/dbdir
sudo -u nobody sh -c 'echo x > /var/lib/dbdir/app.db'
systemd-run --wait --pipe -p NoNewPrivileges=yes \
  -p 'CapabilityBoundingSet=CAP_DAC_OVERRIDE CAP_DAC_READ_SEARCH' \
  /usr/bin/stat /var/lib/dbdir/app.db                 # -> succeeds
systemd-run --wait --pipe -p NoNewPrivileges=yes \
  -p 'CapabilityBoundingSet=' \
  /usr/bin/stat /var/lib/dbdir/app.db                 # -> "permission denied"

# 4. End-to-end: real replication under the unit shows no EPERM/permission errors
sudo systemctl restart litestream && journalctl -u litestream -f

Results (Debian 13, systemd 257, static CGO_ENABLED=0 build)

Measured with systemd-analyze security on the loaded unit and a real
replication run (SQLite WAL DB owned by an unprivileged app user in a 0750
directory, file replica):

Configuration Exposure App can write WAL replication Denials in journal
Original packaged unit 9.6 UNSAFE yes yes
Commit 1 (baseline) 5.8 MEDIUM yes yes none
Commit 2 (full), root + shipped caps 2.2 OK yes (app opens DB first) yes (txids advance) none
Commit 2 + User=<owner>, empty caps 1.8 OK yes yes none
Commit 2 but empty CapabilityBoundingSet= as root fails permission denied

The last row is the regression the shipped CapabilityBoundingSet avoids;
step 3 above reproduces it in isolation. No denied/EPERM/panic appears in
the journal for any working configuration, confirming MemoryDenyWriteExecute
and the syscall filter are compatible with the pure-Go binary.

Note on ordering: if Litestream (as root) opens the database before the app,
it creates root-owned -wal/-shm and the app can then get "readonly
database". That is pre-existing root behavior (the old unit runs as root too),
not introduced here; running as the database owner via User= avoids it
entirely and is the documented recommendation.

Recommended tightened mode (documentation, not the default)

The shipped default runs as root and relies on CAP_DAC_OVERRIDE to reach an
app-owned database — a blunt grant. Operators who want least privilege can
instead run Litestream under a dedicated unprivileged user and grant access to
each database's group via a drop-in. This is documented, not the default,
because it requires per-database filesystem setup and would break existing
installs if forced.

# /etc/systemd/system/litestream.service.d/override.conf
[Service]
User=litestream
Group=litestream
SupplementaryGroups=appgroup     # one per database group; add more as needed
CapabilityBoundingSet=           # no capabilities needed in this mode
UMask=0007

Filesystem requirements (tested end-to-end on Debian 13 / systemd 257):

  • Database directory 0770, owned appuser:appgroup; Litestream joins
    appgroup. Do not set the setgid bit on this directory — Litestream
    mirrors the directory's mode onto its own .<db>-litestream metadata dir by
    design (internal.MkdirAll receives the parent dirInfo), and creating a
    setgid dir is refused (EPERM) by this unit's RestrictSUIDSGID=yes.
  • The database file must be group-writable (chmod 0660): Litestream opens it
    read-write (it creates _litestream_seq and checkpoints). SQLite creates
    files from a 0644 base, which has no group-write bit, so this cannot be
    achieved with umask alone — the application (or an admin) must set 0660.
    -wal/-shm then inherit the database file's mode.

Result: runs with no root and no capabilities; exposure drops to 1.5 OK,
and the database's owning application keeps read/write access throughout.

If you also enable the control socket (off by default), note that its default
path /var/run/litestream.sock lives directly in root-owned /run, which an
unprivileged user cannot write. Give the service its own runtime directory and
point the socket at it (Litestream does not read $RUNTIME_DIRECTORY, so the
path must be set explicitly, and CLI clients need the matching -socket):

# override.conf
[Service]
RuntimeDirectory=litestream          # creates /run/litestream owned by the service user
# litestream.yml
socket:
  enabled: true
  path: /run/litestream/litestream.sock

Related

  • (none)

🤖 Generated with Claude Code

Self-Perfection and others added 2 commits July 12, 2026 21:44
The packaged unit ran the daemon as root with no sandboxing
(`systemd-analyze security` exposure 9.6 "UNSAFE"). This adds the subset
of hardening directives that require no per-deployment configuration and
do not change how the daemon accesses its databases, so it is safe to
ship to every existing installation on upgrade:

- NoNewPrivileges, ProtectSystem=full
- Protect{Clock,KernelTunables,KernelModules,KernelLogs,ControlGroups,Hostname}
- ProtectProc=invisible / ProcSubset=pid, PrivateDevices
- RestrictNamespaces, LockPersonality, RestrictRealtime, RestrictSUIDSGID
- RemoveIPC, SystemCallArchitectures=native

Directives that depend on the database path or on capabilities (User=,
ProtectSystem=strict, ReadWritePaths=, CapabilityBoundingSet=, PrivateTmp,
system-call filtering) are deferred to a follow-up so this baseline stays
trivially reviewable and cannot break existing setups.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Builds on the conservative baseline with the directives that carry a
per-deployment caveat, each documented inline so operators can tune them:

- CapabilityBoundingSet=CAP_DAC_OVERRIDE CAP_DAC_READ_SEARCH CAP_CHOWN
  CAP_FOWNER (+ empty AmbientCapabilities). An *empty* bounding set breaks
  the common "root replicates a database owned by another user" topology:
  without DAC override, root is denied by normal permission checks and
  replication fails with "stat ...: permission denied". Retaining just the
  file-access caps keeps that working; setting User= to the DB owner lets
  you drop them entirely.
- PrivateTmp=true, with a note that a control socket configured under /tmp
  (as shown in some docs) becomes unreachable to external `litestream`
  CLI clients; the default socket lives under /run and is unaffected.
- MemoryDenyWriteExecute=true — safe because the shipped binary is pure Go
  (CGO_ENABLED=0, modernc.org/sqlite); no W+X mappings.
- RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6 (all replica backends
  plus the local IPC socket and name resolution).
- SystemCallFilter=@System-service minus @privileged/@resources,
  SystemCallErrorNumber=EPERM, UMask=0077.
- Commented, path-dependent knobs: User=/Group=, ProtectSystem=strict +
  ReadWritePaths=, ProtectHome=.

Exposure on the live unit (still running as root): 9.6 UNSAFE -> 2.2 OK
per `systemd-analyze security`. Verified working in production.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant