Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
db-data/data
s3-data/data
garage-data/meta
garage-data/data
.env
113 changes: 101 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,28 +7,117 @@ If you use or plan to use DSW, please let us know via [info@ds-wizard.org](mailt
- Join our [**Discord** server](https://discord.gg/MW3H9tdMcT), where you can be notified about important updates and releases + we can discuss your issues and ideas.
- Provide us feedback (what is good and bad, [feature requests](https://ideas.ds-wizard.org/), etc.)

This example is intended for **local setup and testing**. For production use there are many more things to do such as authentication, controlling exposed ports (e.g. do not expose ports of `postgres` and `minio`), data backups, or using proxy (with HTTPS and WebSocket enabled). As it is highly dependent on your use case, consult production deployment with your sysadmin or [contact us](https://ds-wizard.org/contact).
This example is intended for **local setup and testing**. For production use there are many more things to do such as authentication, controlling exposed ports (e.g. do not expose ports of `postgres` and `garage`), data backups, or using proxy (with HTTPS and WebSocket enabled). As it is highly dependent on your use case, consult production deployment with your sysadmin or [contact us](https://ds-wizard.org/contact).

## Usage
## Overview

This is an example deployment of the [Data Stewardship Wizard](https://ds-wizard.org) using [docker-compose](https://docs.docker.com/compose/). You can clone the repository, create `.env` file using `example.env` and run it with:
This is an example deployment of the [Data Stewardship Wizard](https://ds-wizard.org) using [Docker Compose](https://docs.docker.com/compose/) and [Garage](https://garagehq.deuxfleurs.fr/) as the S3-compatible object storage.

```
$ docker-compose up -d
```
It is intentionally set up as a **single-node local POC**:

Then visit [localhost:8080/wizard](http://localhost:8080/wizard/) and login as `albert.einstein@example.com` with password `password`.
- Garage runs in Docker on host ports `9000` (S3 API) and `9003` (Admin API)
- DSW points to `http://host.docker.internal:9000` so presigned URLs are reachable from the browser
- `create-bucket.sh` performs the one-time Garage bootstrap for this example

For information on how to use Data Stewardship Wizard, visit our [guide](https://guide.ds-wizard.org).

## Important notes
## Quick Start

1. Create the local environment file:

```bash
cp example.env .env
```

2. Start the stack:

```bash
docker compose up -d
```

3. Bootstrap Garage:

```bash
./create-bucket.sh
```

4. Open DSW:

[http://localhost:8080/wizard](http://localhost:8080/wizard/)

5. Log in with:

- Email: `albert.einstein@example.com`
- Password: `password`

## What The Bootstrap Does

`create-bucket.sh` is intended to be safe to rerun for this local setup. It:

- assigns the single-node Garage layout if the node still has no role
- creates the configured S3 bucket if it does not exist
- imports the configured Garage access key if it does not exist
- grants read, write, and owner permissions on the bucket to that key

## Verification Checklist

After setup, use this checklist to confirm the Garage-backed deployment works.

### Infrastructure checks

```bash
docker compose ps
docker compose logs garage --tail=100
docker compose logs server --tail=100
docker compose logs docworker --tail=100
```

Expected results:

- `garage` is running
- `server` creates the S3 client successfully
- `docworker` starts without S3 errors
- no authentication, signing, or region errors appear in the logs

### Application checks

In the DSW UI, verify:

1. the application opens and login works
2. a project file can be uploaded
3. the uploaded file can be downloaded
4. a document preview can be generated
5. a document template asset URL works, if applicable

If these checks pass, Garage is functioning as a drop-in S3-compatible backend for this example deployment.

## Important Notes

* Use `docker compose pull` to get newest image (hotfixes) before starting
* **Do not expose** PostgreSQL and MinIO to the internet (MinIO should be exposed only via proxy)
* **Do not expose** PostgreSQL and Garage to the internet in a public deployment (Garage should be behind your proxy, firewall, or private network setup)
* When you want to use DSW publicly, **set up HTTPS proxy** (e.g. Nginx) with a certificate for your domain and change default accounts
* Set up volume mounted to PostgreSQL and MinIO containers for persistent data
* You have to create S3 bucket, either using Web UI (for MinIO, you can expose and use `http://localhost:9000`) or via client: https://docs.min.io/docs/minio-client-complete-guide.html#mb, e.g. use `create-bucket.sh` script
* Always use **strong passwords** and never use default values, **change the secrets** in `config/application.yml` (32 character string in `general.secret` and RSA private key in `general.rsaPrivateKey` via `ssh-keygen -t rsa -b 4096 -m PEM -f jwtRS256.key`)
* Set up volume mounted to PostgreSQL and Garage containers for persistent data
* Garage needs a one-time bootstrap after the stack starts. `create-bucket.sh` assigns the single-node layout, creates the bucket, imports the configured S3 key, and grants bucket permissions
* DSW uses `http://host.docker.internal:9000` as the S3 endpoint so both the DSW containers and the browser can reach the same local Garage endpoint
* Always use **strong passwords** and never use default values, **change the secrets** in `config/application.yml` and `.env` (JWT secret, RSA private key, Garage RPC/admin tokens, and S3 credentials)

## Troubleshooting

If something does not work:

```bash
docker compose ps
docker compose logs garage --tail=200
docker compose logs server --tail=200
docker compose logs docworker --tail=200
```

Common local issues:

- Garage was started, but `create-bucket.sh` was not run yet
- `.env` values and the imported Garage key no longer match
- the server is still starting and has not reached a healthy state yet
- an old local data directory contains stale state from a previous attempt

## Security Audit

Expand Down
8 changes: 5 additions & 3 deletions config/application.yml
Original file line number Diff line number Diff line change
Expand Up @@ -59,12 +59,14 @@ general:
database:
connectionString: postgresql://postgres:postgres@postgres:5432/engine-wizard

# (!!) Change default password
# (!!) Change default credentials
s3:
# Use a host-reachable endpoint so browser-facing presigned URLs work in local Docker.
url: http://host.docker.internal:9000
username: minio
password: minioPassword
username: GK0123456789abcdef01234567
password: 0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef
bucket: engine-wizard
region: garage

# (!!) Configure SMTP
mail:
Expand Down
17 changes: 17 additions & 0 deletions config/garage.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
metadata_dir = "/var/lib/garage/meta"
data_dir = "/var/lib/garage/data"
db_engine = "sqlite"

# Single-node local POC settings.
replication_factor = 1
rpc_bind_addr = "[::]:3901"
rpc_public_addr = "127.0.0.1:3901"

[s3_api]
# Path-style requests work without extra DNS; root_domain only enables optional vhost-style access.
s3_region = "garage"
api_bind_addr = "[::]:3900"
root_domain = ".s3.garage.localhost"

[admin]
api_bind_addr = "[::]:3903"
69 changes: 53 additions & 16 deletions create-bucket.sh
Original file line number Diff line number Diff line change
@@ -1,18 +1,55 @@
#!/bin/sh

set -a; source .env; set +a

# (!!) Change default password
MINIO_NET="dsw-deployment-example_default"
MINIO_BUCKET="engine-wizard"
MINIO_USER="minio"
MINIO_PASS="minioPassword"

docker run --rm --net $MINIO_NET \
-e MINIO_BUCKET=$MINIO_BUCKET \
-e MINIO_USER=$MINIO_USER \
-e MINIO_PASS=$MINIO_PASS \
--entrypoint sh minio/mc:$MINIO_VERSION -c "\
mc alias set dswminio http://minio:9000 $MINIO_USER $MINIO_PASS && \
mc mb dswminio/\$MINIO_BUCKET
"
set -eu

if [ -f .env ]; then
set -a
. ./.env
set +a
fi

GARAGE_SERVICE="${GARAGE_SERVICE:-garage}"
GARAGE_ZONE="${GARAGE_ZONE:-dc1}"
GARAGE_CAPACITY="${GARAGE_CAPACITY:-1G}"
GARAGE_KEY_NAME="${GARAGE_KEY_NAME:-dsw-engine-wizard}"
S3_URL="${S3_URL:-http://host.docker.internal:9000}"
S3_BUCKET="${S3_BUCKET:-engine-wizard}"
S3_USERNAME="${S3_USERNAME:-GK0123456789abcdef01234567}"
S3_PASSWORD="${S3_PASSWORD:-0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef}"

garage_exec() {
docker compose exec -T "$GARAGE_SERVICE" /garage "$@"
}

if [ -z "$(docker compose ps -q --status running "$GARAGE_SERVICE" 2>/dev/null)" ]; then
echo "Garage service '$GARAGE_SERVICE' is not running. Start the stack first with: docker compose up -d" >&2
exit 1
fi

STATUS="$(garage_exec status)"
NODE_ID="$(printf '%s\n' "$STATUS" | awk '/==== HEALTHY NODES ====/{flag=1; next} flag && $1 ~ /^[0-9a-f]+$/ {print $1; exit}')"

if [ -z "$NODE_ID" ]; then
echo "Unable to determine the Garage node ID from 'garage status'." >&2
exit 1
fi

if printf '%s\n' "$STATUS" | grep -q "NO ROLE ASSIGNED"; then
garage_exec layout assign -z "$GARAGE_ZONE" -c "$GARAGE_CAPACITY" "$NODE_ID"
garage_exec layout apply --version 1
fi

if ! garage_exec bucket info "$S3_BUCKET" >/dev/null 2>&1; then
garage_exec bucket create "$S3_BUCKET"
fi

if ! garage_exec key info "$S3_USERNAME" >/dev/null 2>&1; then
garage_exec key import "$S3_USERNAME" "$S3_PASSWORD" -n "$GARAGE_KEY_NAME" --yes
fi

garage_exec bucket allow --read --write --owner "$S3_BUCKET" --key "$S3_USERNAME"

echo "Garage bootstrap completed."
echo "S3 endpoint: $S3_URL"
echo "Bucket: $S3_BUCKET"
echo "Access key: $S3_USERNAME"
64 changes: 38 additions & 26 deletions docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,22 +4,29 @@ services:
image: datastewardshipwizard/wizard-server:${DSW_VERSION}
platform: linux/amd64
restart: always
# Local demo only; put an HTTPS proxy in front for shared deployments.
ports:
# (!!) Expose only for local deployment, externally use HTTPS proxy
- 127.0.0.1:3000:3000
depends_on:
- postgres
- minio
- garage
volumes:
- ./config/application.yml:/app/config/application.yml:ro
extra_hosts:
- host.docker.internal:host-gateway
# Keep runtime S3 values in .env so they stay aligned with Garage bootstrap.
environment:
S3_URL: ${S3_URL}
S3_USERNAME: ${S3_USERNAME}
S3_PASSWORD: ${S3_PASSWORD}
S3_BUCKET: ${S3_BUCKET}
S3_REGION: ${S3_REGION}

client:
image: datastewardshipwizard/wizard-client:${DSW_VERSION}
restart: always
# Local demo only; put an HTTPS proxy in front for shared deployments.
ports:
# (!!) Expose only for local deployment, externally use HTTPS proxy
- 127.0.0.1:8080:8080
environment:
API_URL: http://localhost:3000/wizard-api
Expand All @@ -29,12 +36,19 @@ services:
restart: always
depends_on:
- postgres
- minio
- garage
- server
volumes:
- ./config/application.yml:/app/config/application.yml:ro
extra_hosts:
- host.docker.internal:host-gateway
# Keep runtime S3 values in .env so they stay aligned with Garage bootstrap.
environment:
S3_URL: ${S3_URL}
S3_USERNAME: ${S3_USERNAME}
S3_PASSWORD: ${S3_PASSWORD}
S3_BUCKET: ${S3_BUCKET}
S3_REGION: ${S3_REGION}

mailer:
image: datastewardshipwizard/mailer:${DSW_VERSION}
Expand All @@ -48,39 +62,37 @@ services:
postgres:
image: postgres:${POSTGRES_VERSION}
restart: always
# (!!) Expose only for debugging locally
# Local debug access only.
ports:
- 127.0.0.1:5432:5432
# (!!) Change default password
# Change defaults before using this outside a local POC.
environment:
POSTGRES_DB: engine-wizard
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
# (!!) Mount for persistent data
POSTGRES_DB: ${POSTGRES_DB}
POSTGRES_USER: ${POSTGRES_USER}
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
# Uncomment one option below to persist PostgreSQL data locally.
# volumes:
# - db-data:/var/lib/postgresql/data
# OR
# - ./db-data/data:/var/lib/postgresql/data

minio:
image: minio/minio:${MINIO_VERSION}
garage:
image: dxflrs/garage:${GARAGE_VERSION}
restart: always
command: server /data --console-address ":9001"
# (!!) Expose only for debugging locally, externally use HTTPS proxy (see MinIO docs)
# Publish Garage on the host so both containers and the browser can reach the same endpoint.
# Do not expose this directly in a public deployment.
ports:
- 9000:9000
- 9001:9001
# (!!) Change default password
- 9000:3900
- 9003:3903
# Change these defaults before sharing or reusing this setup.
environment:
MINIO_ROOT_USER: minio
MINIO_ROOT_PASSWORD: minioPassword
MINIO_VOLUMES: /data
# (!!) Mount and backup for persistent data
# volumes:
# - s3-data:/data
# OR
# - ./s3-data/data:/data
GARAGE_RPC_SECRET: ${GARAGE_RPC_SECRET}
GARAGE_ADMIN_TOKEN: ${GARAGE_ADMIN_TOKEN}
GARAGE_METRICS_TOKEN: ${GARAGE_METRICS_TOKEN}
volumes:
- ./config/garage.toml:/etc/garage.toml:ro
# Uncomment to persist Garage state between local restarts.
# - ./garage-data:/var/lib/garage

# volumes:
# db-data:
# s3-data:
23 changes: 21 additions & 2 deletions example.env
Original file line number Diff line number Diff line change
@@ -1,6 +1,25 @@
DSW_VERSION=4.29

# Versions of dependencies (see their docs)
# - possibly a migration may be needed when upgrading
# Upgrades may require migrations.
POSTGRES_VERSION=17.5
MINIO_VERSION=RELEASE.2025-05-24T17-08-30Z
GARAGE_VERSION=v2.2.0

# DSW DB configuration.
POSTGRES_USER=postgres
POSTGRES_PASSWORD=postgres
POSTGRES_DB=engine-wizard

# DSW S3 configuration.
# Keep these aligned with the credentials imported by create-bucket.sh.
S3_URL=http://host.docker.internal:9000
S3_USERNAME=GK0123456789abcdef01234567
S3_PASSWORD=0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef
S3_BUCKET=engine-wizard
S3_REGION=garage

# Garage daemon secrets.
# Change these defaults before sharing or reusing this setup.
GARAGE_RPC_SECRET=1111111111111111111111111111111111111111111111111111111111111111
GARAGE_ADMIN_TOKEN=changeMeGarageAdminToken
GARAGE_METRICS_TOKEN=changeMeGarageMetricsToken
1 change: 1 addition & 0 deletions garage-data/.gitkeep
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@

Loading