Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .prettierrc.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
---
printWidth: 120
proseWrap: "always"
overrides:
- files: "**/*.yaml"
options:
proseWrap: "never"
6 changes: 3 additions & 3 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
ComposeFlux is a Go application that implements a GitOps reconciliation loop for Docker Compose stacks. It polls a Git repository, detects changes via SHA256 checksums stored as container labels, and deploys/prunes stacks using the native Docker Compose SDK.

**Module**: `github.com/veerendra2/composeflux`
**Go version**: 1.26.1 (CGO enabled — Bitwarden SDK uses cgo FFI into Rust)
**Go version**: 1.26.2 (CGO enabled — Bitwarden SDK uses cgo FFI into Rust)

---

Expand All @@ -16,7 +16,7 @@ cmd/composeflux/ # Main binary entry point (CLI setup via kong)
cmd/prune-playground/ # Dev scratch binary
internal/reconcile/ # Core reconciliation logic (private to module)
pkg/dockercompose/ # Docker Compose SDK wrapper (exported)
pkg/secrets/ # Secrets manager integrations (Bitwarden, Infisical)
pkg/secrets/ # Secrets manager integrations (Bitwarden, Infisical) — optional
pkg/source/ # Git client (go-git)
docs/ # MkDocs documentation
```
Expand Down Expand Up @@ -184,6 +184,6 @@ ctx.FatalIfErrorf(ctx.Run())
## Docker / Build Notes

- CGO is enabled (`CGO_ENABLED=1`) for the Bitwarden SDK (Rust FFI).
- Multi-stage Dockerfile: `golang:1.26.1` builder → `gcr.io/distroless/static-debian13` final image.
- Multi-stage Dockerfile: `golang:1.26.2` builder → `gcr.io/distroless/static-debian13` final image.
- Version info injected at link time via `-ldflags` (git tag, commit SHA, branch, build date).
Comment thread
veerendra2 marked this conversation as resolved.
- Local dev: `task compose` runs the app via `compose-dev.yml`.
2 changes: 1 addition & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM golang:1.26.1 AS builder
FROM golang:1.26.2 AS builder
WORKDIR /app
RUN curl -sL https://taskfile.dev/install.sh | sh \
&& apt update \
Expand Down
16 changes: 11 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,7 @@
![GitHub Repo stars](https://img.shields.io/github/stars/veerendra2/composeflux) ![GitHub forks](https://img.shields.io/github/forks/veerendra2/composeflux) ![GitHub License](https://img.shields.io/github/license/veerendra2/composeflux) ![GitHub go.mod Go version](https://img.shields.io/github/go-mod/go-version/veerendra2/composeflux)
![GitHub Repo stars](https://img.shields.io/github/stars/veerendra2/composeflux)
![GitHub forks](https://img.shields.io/github/forks/veerendra2/composeflux)
![GitHub License](https://img.shields.io/github/license/veerendra2/composeflux)
![GitHub go.mod Go version](https://img.shields.io/github/go-mod/go-version/veerendra2/composeflux)

# ComposeFlux

Expand All @@ -8,16 +11,18 @@ _A GitOps continuous deployment tool for Docker Compose_

![ComposeFlux](./docs/assets/composeflux-banner.png)

ComposeFlux automates Docker Compose deployments using GitOps principles. Monitor your Git repository, detect changes, and automatically deploy your Docker stacks—all without manual intervention.
ComposeFlux automates Docker Compose deployments using GitOps principles. Monitor your Git repository, detect changes,
and automatically deploy your Docker stacks—all without manual intervention.

---

## Features

- **GitOps Driven** - Automatic deployment from Git repository
- **Smart Change Detection** - Hash-based detection deploys only changed stacks
- **Pure Go Implementation** - Native Docker [Compose SDK](https://docs.docker.com/compose/compose-sdk/) without shell execution
- **Secrets Management** - Integrated Bitwarden Secrets Manager and Infisical support
- **Pure Go Implementation** - Native Docker [Compose SDK](https://docs.docker.com/compose/compose-sdk/) without shell
execution
- **Secrets Management** - Optional Bitwarden Secrets Manager and Infisical support
- **Automatic Image Updates** - Scheduled registry checks redeploy stacks when newer images are available
- **Flexible Configuration** - Startup order and shared environment variables
- **Simple & Headless** - No UI, no backend database
Expand All @@ -29,7 +34,8 @@ ComposeFlux automates Docker Compose deployments using GitOps principles. Monito

## Contributing

Contributions are welcome! Please see our [Development Guide](https://veerendra2.github.io/composeflux/Development/) for details.
Contributions are welcome! Please see our [Development Guide](https://veerendra2.github.io/composeflux/Development/) for
details.

## License

Expand Down
4 changes: 1 addition & 3 deletions Taskfile.yml
Original file line number Diff line number Diff line change
Expand Up @@ -119,9 +119,7 @@ tasks:
cmds:
- go install golang.org/x/vuln/cmd/govulncheck@latest
- go install github.com/golangci/golangci-lint/cmd/golangci-lint@latest
status:
- test -f {{.GOPATH_BIN}}/govulncheck
- test -f {{.GOPATH_BIN}}/golangci-lint
- pip install "mkdocs>=1.6,<2.0" mkdocs-material

all:
desc: Run comprehensive checks; format, lint, security and test
Expand Down
40 changes: 20 additions & 20 deletions cmd/composeflux/common.go
Original file line number Diff line number Diff line change
Expand Up @@ -17,29 +17,28 @@ import (
"github.com/veerendra2/gopackages/version"
)

// CommonConfig holds configuration shared between all commands
type CommonConfig struct {
SecretsProvider string `name:"secrets-provider" enum:"bitwarden,infisical" env:"SECRETS_PROVIDER" required:"" help:"Secrets manager provider to use (bitwarden or infisical)"`

Bitwarden secrets.BitwardenConfig `embed:"" prefix:"bitwarden-" envprefix:"BITWARDEN_" group:"Bitwarden Options:"`
Infisical secrets.InfisicalConfig `embed:"" prefix:"infisical-" envprefix:"INFISICAL_" group:"Infisical Options:"`

Secrets secrets.Config `embed:""`
Reconciler reconcile.Config `embed:"" group:"Reconciler Options:"`
Source source.Config `embed:"" group:"Git Source Options:"`
DockerCompose dockercompose.Config `embed:"" group:"Docker Compose Options:"`
}

// Validate checks provider-specific configuration
func (c *CommonConfig) Validate() error {
switch c.SecretsProvider {
switch c.Secrets.Provider {
case "":
if c.Source.DeployKeySecretRef != "" {
return fmt.Errorf("--deploy-key-secret-ref requires a secrets provider (--secrets-provider)")
}
case "bitwarden":
if c.Bitwarden.AccessToken == "" || c.Bitwarden.OrgID == "" || c.Bitwarden.ProjectID == "" {
if c.Secrets.Bitwarden.AccessToken == "" || c.Secrets.Bitwarden.OrgID == "" || c.Secrets.Bitwarden.ProjectID == "" {
return fmt.Errorf("bitwarden provider requires: --bitwarden-access-token, " +
"--bitwarden-organization-id, --bitwarden-project-id")
}
case "infisical":
if c.Infisical.ClientID == "" || c.Infisical.ClientSecret == "" ||
c.Infisical.Environment == "" || c.Infisical.ProjectID == "" {
if c.Secrets.Infisical.ClientID == "" || c.Secrets.Infisical.ClientSecret == "" ||
c.Secrets.Infisical.Environment == "" || c.Secrets.Infisical.ProjectID == "" {
return fmt.Errorf("infisical provider requires: --infisical-client-id, " +
"--infisical-client-secret, --infisical-environment, --infisical-project-id")
}
Expand All @@ -49,24 +48,23 @@ func (c *CommonConfig) Validate() error {

// InitClients initializes all required clients (secrets, git, docker, reconciler)
func (c *CommonConfig) InitClients(ctx context.Context) (*reconcile.Reconciler, func(), error) {
// Create secrets client
secretsCfg := secrets.Config{
BitwardenConfig: c.Bitwarden,
InfisicalConfig: c.Infisical,
}

sClient, err := secrets.New(ctx, c.SecretsProvider, secretsCfg)
sClient, err := secrets.New(ctx, c.Secrets)
if err != nil {
slog.Error("Failed to create secrets manager client", "provider", c.SecretsProvider, "error", err)
slog.Error("Failed to create secrets manager client", "provider", c.Secrets.Provider, "error", err)
return nil, nil, err
}

cleanup := func() {
sClient.Close()
if sClient != nil {
sClient.Close()
}
}

// Fetch SSH deploy key from secrets manager if specified
if c.Source.DeployKeySecretRef != "" {
if sClient == nil {
return nil, cleanup, fmt.Errorf("--deploy-key-secret-ref requires a secrets provider (--secrets-provider)")
}
slog.Debug("Fetching SSH deploy key from secrets manager", "deploy_key_ref", c.Source.DeployKeySecretRef)

keyContent, err := sClient.Get(c.Source.DeployKeySecretRef)
Expand Down Expand Up @@ -128,7 +126,9 @@ func (c *CommonConfig) InitClients(ctx context.Context) (*reconcile.Reconciler,
return nil, cleanup, err
}

slog.Info("Reconciler configured", "stack_path", c.Reconciler.StackPath, "config_file", c.Reconciler.ConfigFile)
slog.Info("Reconciler configured", "stack_path", c.Reconciler.StackPath, "config_file", c.Reconciler.ConfigFile,
"secrets_manager", c.Secrets.Provider, "git_poll_interval", c.Reconciler.GitInterval,
"image_update_cron", c.Reconciler.ImageUpdateSchedule)

return rClient, cleanup, nil
}
Expand Down
7 changes: 5 additions & 2 deletions docs/Development.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,9 @@
- Linux: `musl-gcc`
- macOS: Xcode Command Line Tools

> _CGO must be enabled (`CGO_ENABLED=1`, already set in `Taskfile.yml`) because ComposeFlux uses the Bitwarden Go SDK, which calls into the Bitwarden Rust SDK via FFI using cgo. See [Bitwarden SDK Go instructions](https://github.com/bitwarden/sdk-go/blob/main/INSTRUCTIONS.md)._
> _CGO must be enabled (`CGO_ENABLED=1`, already set in `Taskfile.yml`) because ComposeFlux uses the Bitwarden Go SDK,
> which calls into the Bitwarden Rust SDK via FFI using cgo. See
> [Bitwarden SDK Go instructions](https://github.com/bitwarden/sdk-go/blob/main/INSTRUCTIONS.md)._

## Quick Setup

Expand All @@ -26,7 +28,8 @@ task build

### MkDocs Local Setup

If you use direnv and pyenv, an [.envrc](../.envrc) is already configured. Otherwise, set up a Python virtual environment manually:
If you use direnv and pyenv, an [.envrc](../.envrc) is already configured. Otherwise, set up a Python virtual
environment manually:

```bash
python3 -m venv venv/
Expand Down
64 changes: 39 additions & 25 deletions docs/GettingStarted.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,18 +6,23 @@ Deploy ComposeFlux and manage Docker Compose stacks via GitOps.

- Docker with Compose v2+
- Git repository with Compose stacks
- Secrets manager: **Bitwarden** or **Infisical** (required)
- Secrets manager: **Bitwarden** or **Infisical** (optional, for secrets injection and deploy key fetching)
- SSH key for Git access (store in secrets manager or mount as volume)

## Environment Variables

### Required

| Variable | Description |
| ------------------ | ------------------------------------------------------------- |
| `SECRETS_PROVIDER` | Secrets manager: `bitwarden` or `infisical` (required) |
| `GIT_REPO_URL` | Git repository SSH URL (e.g., `git@github.com:user/repo.git`) |
| `STACK_PATH` | Path to stacks directory in repo (relative to repo root) |
| Variable | Description |
| -------------- | ------------------------------------------------------------- |
| `GIT_REPO_URL` | Git repository SSH URL (e.g., `git@github.com:user/repo.git`) |
| `STACK_PATH` | Path to stacks directory in repo (relative to repo root) |

### Optional - Secrets Provider

| Variable | Description |
| ------------------ | ------------------------------------------------------ |
| `SECRETS_PROVIDER` | Secrets manager: `bitwarden` or `infisical` (optional) |

**Bitwarden (when `SECRETS_PROVIDER=bitwarden`):**

Expand All @@ -44,11 +49,11 @@ Deploy ComposeFlux and manage Docker Compose stacks via GitOps.

| Variable | Description | Default |
| --------------------------- | ----------------------------------------------------------------------------------------------------------------------------- | -------------------------- |
| `GIT_DEPLOY_KEY_SECRET_REF` | Deploy key secret reference (name or ID) in secrets manager (See [Deploy Key Secret Reference](#deploy-key-secret-reference)) | `SSH_PRIVATE_KEY` |
| `GIT_DEPLOY_KEY_SECRET_REF` | Deploy key secret reference (name or ID) in secrets manager (See [Deploy Key Secret Reference](#deploy-key-secret-reference)) | |
| `GIT_SSH_KEY_PATH` | SSH key path inside container | `/.ssh/composeflux_id_rsa` |
| `GIT_CLONE_PATH` | Local clone directory | `/opt/compose-stack` |
| `GIT_INTERVAL` | Git sync interval | `5m` |
| `IMAGE_UPDATE_SCHEDULE` | Cron expression for Docker image update checks, e.g. `0 3 * * *`. Empty = disabled. | `""` |
| `IMAGE_UPDATE_SCHEDULE` | Cron expression for Docker image update checks, e.g. `0 3 * * *`. Empty = disabled. | `""` |
| `GIT_BRANCH` | Git branch to track | `main` |
| `CONFIG_FILE` | Stack config file name (see [Stack Configuration](Introduction.md#stack-configuration)) | `stack.yml` |
| `LOG_LEVEL` | Log level (`debug`/`info`/`warn`/`error`) | `info` |
Expand Down Expand Up @@ -78,8 +83,11 @@ Commands:
Run "composeflux <command> --help" for more information on a command.
```

- **`run`** - Daemon mode with continuous reconciliation (default). Performs an initial sync at startup, then checks the Git repository for changes at configured intervals (default: 5 minutes).
- **`sync`** - One-shot sync and deploy. Manually triggers immediate synchronization. Useful when you update secrets in your secrets manager but haven't made Git changes. See [Hash-Based Change Detection](Introduction.md#hash-based-change-detection).
- **`run`** - Daemon mode with continuous reconciliation (default). Performs an initial sync at startup, then checks the
Git repository for changes at configured intervals (default: 5 minutes).
- **`sync`** - One-shot sync and deploy. Manually triggers immediate synchronization. Useful when you update secrets in
your secrets manager but haven't made Git changes. See
[Hash-Based Change Detection](Introduction.md#hash-based-change-detection).

```bash
# Daemon mode (initial sync at startup, then checks Git every 5 minutes)
Expand All @@ -89,7 +97,10 @@ composeflux run
composeflux sync
```

**Important**: After the initial startup sync, the `run` command fetches secrets and deploys changes only when Git updates are detected. If you update secrets in your secrets manager without changing anything in Git, run `composeflux sync` manually to apply updated secrets. See [Hash-Based Change Detection](Introduction.md#hash-based-change-detection).
**Important**: After the initial startup sync, the `run` command fetches secrets and deploys changes only when Git
updates are detected. If you update secrets in your secrets manager without changing anything in Git, run
`composeflux sync` manually to apply updated secrets. See
[Hash-Based Change Detection](Introduction.md#hash-based-change-detection).

## Deploy ComposeFlux

Expand All @@ -109,7 +120,7 @@ composeflux sync
GIT_REPO_URL=git@github.com:user/stacks-repo.git
STACK_PATH=stacks

# Required - Choose one secrets provider:
# Optional - Choose a secrets provider (omit to run without secrets):

# Option A: Bitwarden
SECRETS_PROVIDER=bitwarden
Expand All @@ -120,7 +131,7 @@ BITWARDEN_PROJECT_ID=your-project-id

# Option B: Infisical
# SECRETS_PROVIDER=infisical
# GIT_DEPLOY_KEY_SECRET_REF=SSH_PRIVATE_KEY (Default)
# GIT_DEPLOY_KEY_SECRET_REF=SSH_PRIVATE_KEY
# INFISICAL_CLIENT_ID=your-client-id
# INFISICAL_CLIENT_SECRET=your-client-secret
# INFISICAL_ENVIRONMENT=prod
Expand All @@ -143,12 +154,12 @@ services:
# GIT_INTERVAL: 5m # Sync interval
# GIT_BRANCH: main

# Secrets Manager - Bitwarden
SECRETS_PROVIDER: ${SECRETS_PROVIDER}
GIT_DEPLOY_KEY_SECRET_REF: ${GIT_DEPLOY_KEY_SECRET_REF}
BITWARDEN_ACCESS_TOKEN: ${BITWARDEN_ACCESS_TOKEN}
BITWARDEN_ORGANIZATION_ID: ${BITWARDEN_ORGANIZATION_ID}
BITWARDEN_PROJECT_ID: ${BITWARDEN_PROJECT_ID}
# Secrets Manager - Bitwarden (optional)
# SECRETS_PROVIDER: ${SECRETS_PROVIDER}
# GIT_DEPLOY_KEY_SECRET_REF: ${GIT_DEPLOY_KEY_SECRET_REF}
# BITWARDEN_ACCESS_TOKEN: ${BITWARDEN_ACCESS_TOKEN}
# BITWARDEN_ORGANIZATION_ID: ${BITWARDEN_ORGANIZATION_ID}
# BITWARDEN_PROJECT_ID: ${BITWARDEN_PROJECT_ID}

# Secrets Manager - Infisical (comment out Bitwarden above if using this)
# SECRETS_PROVIDER: infisical
Expand All @@ -170,28 +181,31 @@ services:
# - ./ssh_known_hosts:/etc/ssh/ssh_known_hosts:ro

# Optional: Mount local SSH key instead of fetching from secrets manager
# Uncomment below and set GIT_DEPLOY_KEY_SECRET_REF="" in environment above
# - ~/.ssh/id_rsa:/.ssh/composeflux_id_rsa:ro
```

### Mount SSH Key

If you prefer to mount your SSH key directly instead of storing it in the secrets manager:

1. Set `GIT_DEPLOY_KEY_SECRET_REF=""` (empty string) in environment variables to disable fetch
1. Leave `SECRETS_PROVIDER` unset (or omit it entirely)
2. Mount your SSH key to the container at `GIT_SSH_KEY_PATH` location (default: `/.ssh/composeflux_id_rsa`)

### Deploy Key Secret Reference

ComposeFlux can fetch your SSH deploy key from the secrets manager during startup, so it can clone private repositories without mounting a local key.
ComposeFlux can fetch your SSH deploy key from the secrets manager during startup, so it can clone private repositories
without mounting a local key.

How `GIT_DEPLOY_KEY_SECRET_REF` works:

- When set to a value (e.g., `SSH_PRIVATE_KEY` or a Bitwarden secret ID), ComposeFlux fetches that secret from your secrets manager
- **Bitwarden**: Uses it as the secret ID to fetch (see [Bitwarden Add Secrets](how-to-guides/Bitwarden.md#2-add-secrets))
- Requires `SECRETS_PROVIDER` to be set
- When set to a value (e.g., `SSH_PRIVATE_KEY` or a Bitwarden secret ID), ComposeFlux fetches that secret from your
secrets manager
- **Bitwarden**: Uses it as the secret ID to fetch (see
[Bitwarden Add Secrets](how-to-guides/Bitwarden.md#2-add-secrets))
- **Infisical**: Uses it as the secret key name to fetch
- The fetched content must be your SSH private key
- When set to `""` (empty string), skips fetch and uses mounted key at `GIT_SSH_KEY_PATH`
- When left empty (default), skips fetch and uses mounted key at `GIT_SSH_KEY_PATH`

Example:

Expand Down
Loading
Loading