From b83988cc3d61623b06f87197d6c88a14a955b1e1 Mon Sep 17 00:00:00 2001 From: vmelikyan Date: Mon, 19 Jan 2026 22:12:09 -0800 Subject: [PATCH] native secrets support related to https://github.com/GoodRxOSS/lifecycle/pull/87 --- src/pages/docs/features/secrets.mdx | 352 ++++++++++++++++++++++++++++ 1 file changed, 352 insertions(+) create mode 100644 src/pages/docs/features/secrets.mdx diff --git a/src/pages/docs/features/secrets.mdx b/src/pages/docs/features/secrets.mdx new file mode 100644 index 0000000..bb5047e --- /dev/null +++ b/src/pages/docs/features/secrets.mdx @@ -0,0 +1,352 @@ +--- +title: Cloud Secrets +tags: + - secrets + - aws + - gcp + - environment variables + - security +--- + +import { Callout } from "nextra/components"; + +## Overview + +Lifecycle integrates with cloud secret providers to securely inject secrets into your ephemeral environments. You can reference secrets directly in your `lifecycle.yaml` environment variables using a template syntax, and Lifecycle handles the rest. + +Supported providers: + +- **AWS Secrets Manager** +- **GCP Secret Manager** + +## Configuration + +Secret providers are configured in the Lifecycle `global_config` database table under the `secretProviders` key. + +### Default Configuration + +Both AWS and GCP providers are **enabled by default**. The ClusterSecretStore created by your platform team determines which provider actually works in your cluster. + +```json +{ + "secretProviders": { + "aws": { + "enabled": true, + "clusterSecretStore": "aws-secretsmanager", + "refreshInterval": "1h", + "allowedPrefixes": [] + }, + "gcp": { + "enabled": true, + "clusterSecretStore": "gcp-secretmanager", + "refreshInterval": "1h", + "allowedPrefixes": [] + } + } +} +``` + +- **On AWS (EKS):** Use `{{aws:path:key}}` syntax. The `aws-secretsmanager` ClusterSecretStore handles the request. +- **On GCP (GKE):** Use `{{gcp:path:key}}` syntax. The `gcp-secretmanager` ClusterSecretStore handles the request. + +If you reference a provider that doesn't have a ClusterSecretStore in your cluster, the secret will fail to sync and a warning will be logged. + +### Configuration Fields + +| Field | Required | Description | +| -------------------- | -------- | ------------------------------------------------------------------ | +| `enabled` | Yes | Enable this provider | +| `clusterSecretStore` | Yes | Name of the ClusterSecretStore CR configured by your platform team | +| `refreshInterval` | Yes | How often ESO syncs secrets (e.g., `1h`, `30m`) | +| `allowedPrefixes` | No | Restrict which secret paths can be referenced (empty = allow all) | + + + If you're unsure which `clusterSecretStore` to use, check with your platform + team. They configure the External Secrets Operator and ClusterSecretStore as + part of cluster setup. + + +## Syntax + +Reference secrets using the following pattern: + +``` +{{::}} +``` + +| Component | Description | Example | +| ------------- | ------------------------------------- | ---------------------------- | +| `provider` | Cloud provider (`aws` or `gcp`) | `aws` | +| `secret-path` | Full path to the secret | `myapp/database-credentials` | +| `key` | JSON key within the secret (optional) | `password` | + + + If your secret is a plaintext value (not JSON), omit the key portion: ` + {"{{aws:myapp/api-key}}"}`. + + +## Basic Usage + +### JSON Secrets + +For secrets stored as JSON objects, specify the key to extract: + +```yaml +services: + - name: api-server + github: + repository: myorg/api-server + branchName: main + docker: + app: + env: + DB_PASSWORD: "{{aws:myapp/rds-credentials:password}}" + DB_USERNAME: "{{aws:myapp/rds-credentials:username}}" + DB_HOST: "{{aws:myapp/rds-credentials:host}}" +``` + +If your AWS secret `myapp/rds-credentials` contains: + +```json +{ + "username": "admin", + "password": "supersecret", + "host": "db.example.com" +} +``` + +Your pod will receive: + +- `DB_USERNAME=admin` +- `DB_PASSWORD=supersecret` +- `DB_HOST=db.example.com` + +### Plaintext Secrets + +For secrets stored as plain strings, omit the key: + +```yaml +env: + STRIPE_API_KEY: "{{aws:myapp/stripe-key}}" +``` + +### Nested JSON + +For nested JSON structures, use dot notation: + +```yaml +env: + REDIS_HOST: "{{aws:myapp/cache-config:redis.host}}" + REDIS_PORT: "{{aws:myapp/cache-config:redis.port}}" +``` + +For a secret containing: + +```json +{ + "redis": { + "host": "redis.example.com", + "port": "6379" + } +} +``` + +## Where Secrets Work + +The secret syntax works in all `env` blocks: + +### GitHub Services + +```yaml +services: + - name: my-service + github: + docker: + app: + env: + SECRET_KEY: "{{aws:path:key}}" + init: + env: + INIT_SECRET: "{{aws:path:key}}" +``` + +### Docker Services + +```yaml +services: + - name: my-service + docker: + dockerImage: myorg/image + defaultTag: latest + env: + SECRET_KEY: "{{aws:path:key}}" +``` + +### Webhooks + +```yaml +environment: + webhooks: + - name: run-migrations + state: deployed + type: docker + docker: + image: myorg/migrator:latest + env: + DB_ADMIN_PASSWORD: "{{aws:myapp/rds-credentials:admin_password}}" +``` + + + Helm and Codefresh deployment types do not currently support cloud secrets. + + +## Mixing Secrets with Template Variables + +You can combine cloud secrets with regular template variables in the same `env` block: + +```yaml +env: + # Cloud secret + DB_PASSWORD: "{{aws:myapp/rds-credentials:password}}" + + # Template variable (service reference) + BACKEND_URL: "{{{backend_publicUrl}}}" + + # Static value + APP_ENV: "ephemeral" + + # Build context + BUILD_UUID: "{{{buildUUID}}}" +``` + +## Multiple Providers + +Both providers are enabled by default. If your platform team has configured ClusterSecretStores for both AWS and GCP, you can reference secrets from either in the same deployment: + +```yaml +env: + AWS_SECRET: "{{aws:myapp/aws-config:key}}" + GCP_SECRET: "{{gcp:my-project/gcp-config:key}}" +``` + + + Each provider requires its own ClusterSecretStore. Contact your platform team + if you need multi-cloud secret access. + + +## Error Handling + +Lifecycle uses a "warn and continue" approach for secret errors: + +| Scenario | Behavior | +| --------------------- | ----------------------------------------------------- | +| Secret not found | Warning logged, env var not set, deployment continues | +| Key not found in JSON | Warning logged, env var not set | +| Provider not enabled | Warning logged, env var not set | +| Invalid syntax | Validation error at deploy time | + + + If a required secret is missing, your application may fail to start or behave + unexpectedly. Always verify your secrets exist before deploying. + + +## Best Practices + +### Use Descriptive Secret Paths + +Organize secrets with meaningful paths: + +```yaml +# Good +DB_PASSWORD: "{{aws:myorg/api-server/production/database:password}}" + +# Less clear +DB_PASSWORD: "{{aws:db-creds:password}}" +``` + +### Group Related Secrets + +Store related credentials in a single JSON secret: + +```json +// myapp/database-credentials +{ + "host": "db.example.com", + "port": "5432", + "username": "app_user", + "password": "secret123", + "database": "myapp" +} +``` + +Then reference individual keys: + +```yaml +env: + DB_HOST: "{{aws:myapp/database-credentials:host}}" + DB_PORT: "{{aws:myapp/database-credentials:port}}" + DB_USER: "{{aws:myapp/database-credentials:username}}" + DB_PASS: "{{aws:myapp/database-credentials:password}}" + DB_NAME: "{{aws:myapp/database-credentials:database}}" +``` + +### Avoid Secrets in Build Logs + +Be cautious when using secrets in build-time environment variables, as they may appear in build logs. Prefer injecting secrets at runtime when possible. + +## Troubleshooting + +### Secret Not Injected + +1. **Check the secret exists** in AWS Secrets Manager / GCP Secret Manager +2. **Verify the path** matches exactly (case-sensitive) +3. **Check the key name** if using JSON secrets +4. **Review Lifecycle logs** for warnings about missing secrets +5. **Verify provider is enabled** in global configuration + +### Syntax Errors + +Common syntax mistakes: + +```yaml +# Wrong - spaces in pattern +DB_PASSWORD: "{{ aws:path:key }}" + +# Wrong - trailing colon +DB_PASSWORD: "{{aws:path:}}" + +# Wrong - missing provider +DB_PASSWORD: "{{path:key}}" + +# Correct +DB_PASSWORD: "{{aws:path:key}}" +``` + +### Permission Denied + +If you see permission errors: + +1. Verify the External Secrets Operator has access to the secret path +2. Check IAM policies allow access to the specific secret +3. Contact your platform team if the secret is in a restricted path + +### Wrong Provider for Cluster + +If you use `{{gcp:...}}` on an AWS cluster (or vice versa), the ExternalSecret will fail to sync because the ClusterSecretStore doesn't exist. Check Lifecycle logs for warnings like "ClusterSecretStore not found". + +## Prerequisites + +For cloud secrets to work, your platform team must set up: + +| Component | Description | +| ----------------------------- | ------------------------------------------------------------------------------ | +| **External Secrets Operator** | Kubernetes operator that syncs secrets from cloud providers | +| **ClusterSecretStore** | CR that configures ESO to connect to AWS Secrets Manager or GCP Secret Manager | +| **IAM/Workload Identity** | Permissions for ESO to read secrets (IRSA for AWS, Workload Identity for GCP) | + +The `clusterSecretStore` value in your Lifecycle config must match the ClusterSecretStore name configured by your platform team. + +## Related + +- [Template Variables](/docs/features/template-variables) - Learn about other template syntax +- [Webhooks](/docs/features/webhooks) - Using secrets in webhook environment variables