Skip to content
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
352 changes: 352 additions & 0 deletions src/pages/docs/features/secrets.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,352 @@
---
title: Cloud Secrets
tags:
- secrets
- aws
- gcp
- environment variables
- security
---

import { Callout } from "nextra/components";

## Overview

Lifecycle integrates with cloud secret providers to securely inject secrets into your ephemeral environments. You can reference secrets directly in your `lifecycle.yaml` environment variables using a template syntax, and Lifecycle handles the rest.

Supported providers:

- **AWS Secrets Manager**
- **GCP Secret Manager**

## Configuration

Secret providers are configured in the Lifecycle `global_config` database table under the `secretProviders` key.

### Default Configuration

Both AWS and GCP providers are **enabled by default**. The ClusterSecretStore created by your platform team determines which provider actually works in your cluster.

```json
{
"secretProviders": {
"aws": {
"enabled": true,
"clusterSecretStore": "aws-secretsmanager",
"refreshInterval": "1h",
"allowedPrefixes": []
},
"gcp": {
"enabled": true,
"clusterSecretStore": "gcp-secretmanager",
"refreshInterval": "1h",
"allowedPrefixes": []
}
}
}
```

- **On AWS (EKS):** Use `{{aws:path:key}}` syntax. The `aws-secretsmanager` ClusterSecretStore handles the request.
- **On GCP (GKE):** Use `{{gcp:path:key}}` syntax. The `gcp-secretmanager` ClusterSecretStore handles the request.

If you reference a provider that doesn't have a ClusterSecretStore in your cluster, the secret will fail to sync and a warning will be logged.

### Configuration Fields

| Field | Required | Description |
| -------------------- | -------- | ------------------------------------------------------------------ |
| `enabled` | Yes | Enable this provider |
| `clusterSecretStore` | Yes | Name of the ClusterSecretStore CR configured by your platform team |
| `refreshInterval` | Yes | How often ESO syncs secrets (e.g., `1h`, `30m`) |
| `allowedPrefixes` | No | Restrict which secret paths can be referenced (empty = allow all) |

<Callout type="info">
If you're unsure which `clusterSecretStore` to use, check with your platform
team. They configure the External Secrets Operator and ClusterSecretStore as
part of cluster setup.
</Callout>

## Syntax

Reference secrets using the following pattern:

```
{{<provider>:<secret-path>:<key>}}
```

| Component | Description | Example |
| ------------- | ------------------------------------- | ---------------------------- |
| `provider` | Cloud provider (`aws` or `gcp`) | `aws` |
| `secret-path` | Full path to the secret | `myapp/database-credentials` |
| `key` | JSON key within the secret (optional) | `password` |

<Callout type="info">
If your secret is a plaintext value (not JSON), omit the key portion: `
{"{{aws:myapp/api-key}}"}`.
</Callout>

## Basic Usage

### JSON Secrets

For secrets stored as JSON objects, specify the key to extract:

```yaml
services:
- name: api-server
github:
repository: myorg/api-server
branchName: main
docker:
app:
env:
DB_PASSWORD: "{{aws:myapp/rds-credentials:password}}"
DB_USERNAME: "{{aws:myapp/rds-credentials:username}}"
DB_HOST: "{{aws:myapp/rds-credentials:host}}"
```

If your AWS secret `myapp/rds-credentials` contains:

```json
{
"username": "admin",
"password": "supersecret",
"host": "db.example.com"
}
```

Your pod will receive:

- `DB_USERNAME=admin`
- `DB_PASSWORD=supersecret`
- `DB_HOST=db.example.com`

### Plaintext Secrets

For secrets stored as plain strings, omit the key:

```yaml
env:
STRIPE_API_KEY: "{{aws:myapp/stripe-key}}"
```

### Nested JSON

For nested JSON structures, use dot notation:

```yaml
env:
REDIS_HOST: "{{aws:myapp/cache-config:redis.host}}"
REDIS_PORT: "{{aws:myapp/cache-config:redis.port}}"
```

For a secret containing:

```json
{
"redis": {
"host": "redis.example.com",
"port": "6379"
}
}
```

## Where Secrets Work

The secret syntax works in all `env` blocks:

### GitHub Services

```yaml
services:
- name: my-service
github:
docker:
app:
env:
SECRET_KEY: "{{aws:path:key}}"
init:
env:
INIT_SECRET: "{{aws:path:key}}"
```

### Docker Services

```yaml
services:
- name: my-service
docker:
dockerImage: myorg/image
defaultTag: latest
env:
SECRET_KEY: "{{aws:path:key}}"
```

### Webhooks

```yaml
environment:
webhooks:
- name: run-migrations
state: deployed
type: docker
docker:
image: myorg/migrator:latest
env:
DB_ADMIN_PASSWORD: "{{aws:myapp/rds-credentials:admin_password}}"
```

<Callout type="warning">
Helm and Codefresh deployment types do not currently support cloud secrets.
</Callout>

## Mixing Secrets with Template Variables

You can combine cloud secrets with regular template variables in the same `env` block:

```yaml
env:
# Cloud secret
DB_PASSWORD: "{{aws:myapp/rds-credentials:password}}"

# Template variable (service reference)
BACKEND_URL: "{{{backend_publicUrl}}}"

# Static value
APP_ENV: "ephemeral"

# Build context
BUILD_UUID: "{{{buildUUID}}}"
```

## Multiple Providers

Both providers are enabled by default. If your platform team has configured ClusterSecretStores for both AWS and GCP, you can reference secrets from either in the same deployment:

```yaml
env:
AWS_SECRET: "{{aws:myapp/aws-config:key}}"
GCP_SECRET: "{{gcp:my-project/gcp-config:key}}"
```

<Callout type="info">
Each provider requires its own ClusterSecretStore. Contact your platform team
if you need multi-cloud secret access.
</Callout>

## Error Handling

Lifecycle uses a "warn and continue" approach for secret errors:

| Scenario | Behavior |
| --------------------- | ----------------------------------------------------- |
| Secret not found | Warning logged, env var not set, deployment continues |
| Key not found in JSON | Warning logged, env var not set |
| Provider not enabled | Warning logged, env var not set |
| Invalid syntax | Validation error at deploy time |

<Callout type="warning">
If a required secret is missing, your application may fail to start or behave
unexpectedly. Always verify your secrets exist before deploying.
</Callout>

## Best Practices

### Use Descriptive Secret Paths

Organize secrets with meaningful paths:

```yaml
# Good
DB_PASSWORD: "{{aws:myorg/api-server/production/database:password}}"

# Less clear
DB_PASSWORD: "{{aws:db-creds:password}}"
```

### Group Related Secrets

Store related credentials in a single JSON secret:

```json
// myapp/database-credentials
{
"host": "db.example.com",
"port": "5432",
"username": "app_user",
"password": "secret123",
"database": "myapp"
}
```

Then reference individual keys:

```yaml
env:
DB_HOST: "{{aws:myapp/database-credentials:host}}"
DB_PORT: "{{aws:myapp/database-credentials:port}}"
DB_USER: "{{aws:myapp/database-credentials:username}}"
DB_PASS: "{{aws:myapp/database-credentials:password}}"
DB_NAME: "{{aws:myapp/database-credentials:database}}"
```

### Avoid Secrets in Build Logs

Be cautious when using secrets in build-time environment variables, as they may appear in build logs. Prefer injecting secrets at runtime when possible.

## Troubleshooting

### Secret Not Injected

1. **Check the secret exists** in AWS Secrets Manager / GCP Secret Manager
2. **Verify the path** matches exactly (case-sensitive)
3. **Check the key name** if using JSON secrets
4. **Review Lifecycle logs** for warnings about missing secrets
5. **Verify provider is enabled** in global configuration

### Syntax Errors

Common syntax mistakes:

```yaml
# Wrong - spaces in pattern
DB_PASSWORD: "{{ aws:path:key }}"

# Wrong - trailing colon
DB_PASSWORD: "{{aws:path:}}"

# Wrong - missing provider
DB_PASSWORD: "{{path:key}}"

# Correct
DB_PASSWORD: "{{aws:path:key}}"
```

### Permission Denied

If you see permission errors:

1. Verify the External Secrets Operator has access to the secret path
2. Check IAM policies allow access to the specific secret
3. Contact your platform team if the secret is in a restricted path

### Wrong Provider for Cluster

If you use `{{gcp:...}}` on an AWS cluster (or vice versa), the ExternalSecret will fail to sync because the ClusterSecretStore doesn't exist. Check Lifecycle logs for warnings like "ClusterSecretStore not found".

## Prerequisites

For cloud secrets to work, your platform team must set up:

| Component | Description |
| ----------------------------- | ------------------------------------------------------------------------------ |
| **External Secrets Operator** | Kubernetes operator that syncs secrets from cloud providers |
| **ClusterSecretStore** | CR that configures ESO to connect to AWS Secrets Manager or GCP Secret Manager |
| **IAM/Workload Identity** | Permissions for ESO to read secrets (IRSA for AWS, Workload Identity for GCP) |

The `clusterSecretStore` value in your Lifecycle config must match the ClusterSecretStore name configured by your platform team.

## Related

- [Template Variables](/docs/features/template-variables) - Learn about other template syntax
- [Webhooks](/docs/features/webhooks) - Using secrets in webhook environment variables