diff --git a/content/ci/how-to-guides/custom-images.mdx b/content/ci/how-to-guides/custom-images.mdx index 0b85831..e875185 100644 --- a/content/ci/how-to-guides/custom-images.mdx +++ b/content/ci/how-to-guides/custom-images.mdx @@ -10,7 +10,7 @@ Build a reusable custom image from the Depot base image that includes your tools ## How it works -You build a custom image in Depot CI using a job that runs only your setup steps on the Depot base image. After your setup steps complete, a Depot snapshot action captures the state of the sandbox environment and pushes it to the Depot registry as a reusable image. Any job can then use that snapshot as its starting image, skipping the setup steps entirely. +A custom image in Depot CI is a snapshot of a sandbox environment. You build the custom image using a job that runs only your setup steps on the Depot base image. After your setup steps complete, a Depot snapshot action captures the state of the sandbox environment and pushes it to the Depot registry as a reusable image. Any job can then use that snapshot as its starting image, skipping the setup steps entirely. The snapshot action can only be used in workflows running on Depot CI. It's not compatible with GitHub Actions or @@ -19,7 +19,7 @@ You build a custom image in Depot CI using a job that runs only your setup steps ## Snapshot a sandbox to build a custom image -Create a job that runs on a standard Depot sandbox and installs the tools and dependencies you want to bake in. This is a separate workflow that only creates your custom image and pushes it to the Depot Registry. You'll need to run this workflow initially to create the image and then only when dependencies change. +Build a custom image using a job that runs on a standard Depot sandbox and installs the tools and dependencies you want to bake in. The job creates your custom image and pushes it to the Depot Registry. Use a separate workflow for the build image job, and then use the resulting image in one or more other workflows. This pattern is usually the most efficient because you run the image build workflow once to create the image and then only when dependencies change. If your use case differs, you can also run the build image job in the same workflow as other jobs that subsequently use it. Actions that set or change environment variables (such as actions/setup-node) need to be in both the build image @@ -63,7 +63,9 @@ Any job in any workflow on Depot CI can specify your custom image. The custom im To run a job on a custom image, specify `runs-on` with `size` and `image` keys (both required). - `size`: the size of the sandbox -- `image`: is the Depot registry URL you specified in the build image workflow +- `image`: the Depot registry URL of the image (defined in the build image workflow) + +The `runs-on.image` value must be a Depot registry URL for an image created using the `depot/snapshot-action` as described in [Snapshot a sandbox to build a custom image](#snapshot-a-sandbox-to-build-a-custom-image). You can't use any other images or artifacts, even if they exist in the Depot registry. Example specifying a custom image in a job: diff --git a/content/cli/reference/depot-ci.mdx b/content/cli/reference/depot-ci.mdx index ebd1379..ed978fd 100644 --- a/content/cli/reference/depot-ci.mdx +++ b/content/cli/reference/depot-ci.mdx @@ -391,7 +391,7 @@ The command prints: - Org, repo, run ID with status, workflow ID with status, name, path, ref, SHA, and trigger. - Each execution with its status, start time, finish time, and duration. -- Each job with its status and duration. For each job, the latest attempt's ID, status, sandbox ID, session ID, and a ready-to-run `depot ci logs` command are shown. If the job has more than one attempt, all attempts are listed. +- Each job with its status and duration. For each job, the command shows the latest attempt's ID, status, sandbox ID, session ID, and a ready-to-run `depot ci logs` command. If the job has more than one attempt, all attempts are listed. ##### Show a workflow as JSON: @@ -498,15 +498,65 @@ Replace `` with the run ID returned by `depot ci run` or visible in the - The organization and run ID with the overall run status. - Each workflow in the run, with its status and workflow file path. - Each job within a workflow, with its job ID, key, and status. -- Each attempt within a job, with its attempt ID, attempt number, and status. +- Each attempt within a job, with its attempt ID, attempt number, and status. Plus a ready-to-run `depot ci logs` command, a link to the attempt in the Depot dashboard, and (when applicable) `depot ci ssh` and log download commands. ### Flags | Flag | Description | | ----------------- | -------------------------------------------------------------------------- | +| `--output json` | Output as JSON instead of the hierarchical text view | | `--org ` | Organization ID (required when user is a member of multiple organizations) | | `--token ` | Depot API token | +### Examples + +#### Show the status as JSON + +```bash +depot ci status --output json +``` + +JSON response: + +```json +{ + "org_id": "", + "run_id": "", + "status": "running", + "workflows": [ + { + "workflow_id": "", + "status": "running", + "workflow_path": ".depot/workflows/ci.yml", + "name": "CI", + "jobs": [ + { + "job_id": "", + "job_key": "ci.yml:test", + "status": "running", + "attempts": [ + { + "attempt_id": "", + "attempt": 1, + "status": "running", + "sandbox_id": "", + "session_id": "", + "logs_command": "depot ci logs ", + "download_available": false, + "view_url": "https://depot.dev/orgs//workflows/?job=&attempt=", + "ssh_available": true, + "ssh_command": "depot ci ssh --job ci.yml:test" + } + ] + } + ] + } + ] +} +``` + +`download_available` becomes `true` and `download_command` appears once the attempt reaches `finished` status. `ssh_available` and `ssh_command` appear only while the attempt is running with a sandbox. + --- ## `depot ci cancel` @@ -712,12 +762,16 @@ depot ci logs ### Flags -| Flag | Description | -| ------------------- | -------------------------------------------------------------------------- | -| `--job ` | Job key to select (required when the run has multiple jobs) | -| `--workflow ` | Workflow path to filter jobs (for example, `ci.yml`) | -| `--org ` | Organization ID (required when user is a member of multiple organizations) | -| `--token ` | Depot API token | +| Flag | Description | +| ---------------------- | -------------------------------------------------------------------------- | +| `--job ` | Job key to select (required when the run has multiple jobs) | +| `--workflow ` | Workflow path to filter jobs (for example, `ci.yml`) | +| `--follow`, `-f` | Follow live logs as they're produced | +| `--timestamps` | Prefix plain log lines with UTC timestamps | +| `--output ` | Output format: `text` (default) or `json` (newline-delimited events) | +| `--output-file ` | Write a finite log export to the given file path | +| `--org ` | Organization ID (required when user is a member of multiple organizations) | +| `--token ` | Depot API token | ### Examples @@ -745,6 +799,112 @@ depot ci logs --job test depot ci logs --job build --workflow ci.yml ``` +#### Follow live logs + +```bash +depot ci logs --follow +``` + +If you pass a run or job ID that hasn't started yet, the command waits up to 30 seconds for the run to create jobs and start the latest attempt, then streams logs as they arrive. + +#### Prefix log lines with UTC timestamps + +```bash +depot ci logs --timestamps +``` + +#### Emit logs as newline-delimited JSON events + +```bash +depot ci logs --output json +``` + +The finite form emits one `line` event per log line, with `timestamp`, `timestamp_ms`, `stream` (`stdout` or `stderr`), `step_key`, `step_id`, `step_name`, `line_number`, and `body`. + +Combine with `--follow` to stream the live attempt as JSON. The streaming form emits the same `line` events plus two additional event types: + +```bash +depot ci logs --follow --output json +``` + +- `status`: signals an attempt status change (for example, `running`, `finished`, `failed`). +- `end`: marks the end of the stream, with the final `status` and `line_count`. + +#### Download logs to a file + +```bash +depot ci logs --output-file logs.txt +``` + +Use `--output-file` together with `--output json` to download a JSONL export: + +```bash +depot ci logs --output json --output-file logs.jsonl +``` + +`--output-file` doesn't work with `--follow`. + +--- + +## `depot ci metrics` + +Fetches CPU and memory metrics for a CI job attempt, job, or run. + +```bash +depot ci metrics +``` + +When given an attempt ID, the command prints a summary of CPU and memory utilization for that attempt. To inspect every attempt of a job or every job in a run, pass `--job ` or `--run ` instead. The positional attempt ID and the `--attempt`, `--job`, and `--run` flags are mutually exclusive. + +For full per-sample CPU and memory readings on a single attempt, add `--output json`. Job and run requests return per-attempt summary stats (no per-sample `samples` array) even with `--output json`. + +### Flags + +| Flag | Description | +| ------------------------ | -------------------------------------------------------------------------- | +| `--attempt ` | Job attempt ID (alias for the positional argument) | +| `--job ` | Show metrics for every attempt of the given job | +| `--run ` | Show metrics for every job and attempt in the given run | +| `--output ` | Output format: `text` (default) or `json` | +| `--org ` | Organization ID (required when user is a member of multiple organizations) | +| `--token ` | Depot API token | + +### Examples + +#### Show metrics for an attempt + +```bash +depot ci metrics +``` + +#### Show metrics for every attempt of a job + +```bash +depot ci metrics --job +``` + +#### Show metrics for every job in a run + +```bash +depot ci metrics --run +``` + +#### Show full per-sample metrics as JSON + +```bash +depot ci metrics --output json +``` + +The JSON response describes the attempt's availability, summary stats (peak and average CPU and memory utilization, sample counts, observed start and finish times), downsampling cap metadata, and the time-series `samples` array. + +The shape of the JSON response depends on which form you use: + +- `--attempt` (or the positional form): top-level run, workflow, and job context, plus a single `attempt` object with the full `samples` array. +- `--job`: top-level run, workflow, and job context, plus a flat `attempts` array of per-attempt summaries. +- `--run`: top-level run context, plus nested `workflows`, `jobs`, and `attempts` arrays. + +For `--job` and `--run`, attempts include the same availability, summary stats, and cap metadata as `--attempt`, but not the `samples` array. + --- ## `depot ci ssh` diff --git a/content/container-builds/how-to-guides/continuous-integration.mdx b/content/container-builds/how-to-guides/continuous-integration.mdx index 96ed463..d857cc2 100644 --- a/content/container-builds/how-to-guides/continuous-integration.mdx +++ b/content/container-builds/how-to-guides/continuous-integration.mdx @@ -20,6 +20,7 @@ Running `depot` in a continuous integration environment is a great way to get fa - [Bitbucket Pipelines](/docs/container-builds/integrations/bitbucket-pipelines) - [Buildkite](/docs/container-builds/integrations/buildkite) - [CircleCI](/docs/container-builds/integrations/circleci) +- [Depot CI](/docs/container-builds/integrations/depot-ci) - [GitHub Actions](/docs/container-builds/integrations/github-actions) - [GitLab CI](/docs/container-builds/integrations/gitlab-ci) - [Google Cloud Build](/docs/container-builds/integrations/google-cloud-build) diff --git a/content/container-builds/integrations/depot-ci.mdx b/content/container-builds/integrations/depot-ci.mdx new file mode 100644 index 0000000..7dad27c --- /dev/null +++ b/content/container-builds/integrations/depot-ci.mdx @@ -0,0 +1,462 @@ +--- +title: Container builds in Depot CI +ogTitle: Use Depot for container builds in Depot CI +description: Get faster container builds with persistent caching and zero emulation in Depot CI workflows. +--- + +You can use Depot for your container image builds inside Depot CI workflows. + +If you haven't migrated your GitHub Actions workflows to Depot CI yet, see the [Depot CI quickstart](/docs/ci/quickstart). Depot CI workflows are compatible with GitHub Actions YAML. For details, see the [Depot CI compatibility reference](/docs/ci/compatibility). + +## Configuration + +You can trigger Depot container builds in Depot CI using a dedicated build action, a bake action, or the Depot CLI directly. + +### Depot build-push action + +The [`depot/build-push-action`](https://github.com/depot/build-push-action) implements the same inputs and outputs as `docker/build-push-action` but uses the Depot CLI to run the build. Use [`depot/setup-action`](https://github.com/depot/setup-action) to install the Depot CLI first. + +```yaml +jobs: + build: + runs-on: depot-ubuntu-latest + steps: + - uses: actions/checkout@v4 + - uses: depot/setup-action@v1 + - uses: depot/build-push-action@v1 + with: + project: + context: . +``` + +### Depot bake action + +The [`depot/bake-action`](https://github.com/depot/bake-action) builds all images defined in an HCL, JSON, or Docker Compose file. Use it when you need to build multiple images in a single build request. + +```yaml +jobs: + build: + runs-on: depot-ubuntu-latest + steps: + - uses: actions/checkout@v4 + - uses: depot/setup-action@v1 + - uses: depot/bake-action@v1 + with: + project: + files: docker-bake.hcl +``` + +### Depot CLI + +The [`depot/setup-action`](https://github.com/depot/setup-action) installs the `depot` CLI so you can run builds directly from your existing workflows. + +```yaml +jobs: + build: + runs-on: depot-ubuntu-latest + steps: + - uses: actions/checkout@v4 + - uses: depot/setup-action@v1 + - run: depot build --project --push --tag repo/image:tag . +``` + +## Authentication + +Depot CI automatically injects `DEPOT_TOKEN` into every job as a short-lived CI job token with full access to your organization's projects and registry. You don't need to configure OIDC trust relationships or store any secrets, the build authenticates without additional setup. + +If you set `DEPOT_TOKEN` as a workflow secret yourself, your value overrides the auto-injected token. This commonly happens after migrating from GitHub Actions, where `DEPOT_TOKEN` was a project token. A project token is scoped to a single project's repositories in the Depot Registry, so pushes to repositories that don't begin with that project ID will fail. To restore the auto-injected behavior, remove the `DEPOT_TOKEN` secret from the dashboard or with `depot ci secrets remove DEPOT_TOKEN`. + +## Pulling private base images + +Dockerfiles that start with `FROM` referencing a private registry need credentials when Depot builds the image. You have two options. + +### Pull through Depot Registry (recommended) + +Configure your private registry as an upstream once from [Registry settings](/orgs/_/registry/settings), then create a Depot repository that mirrors the upstream and reference the Depot Registry URL in your Dockerfile: + +```dockerfile +FROM .registry.depot.dev/: +``` + +Depot fetches the image from your upstream registry, caches layers on its global CDN, and serves subsequent pulls from the cache. The auto-injected `DEPOT_TOKEN` handles pull authentication, so no login step is needed in the workflow. See [Pull-through cache](/docs/registry/pull-through-cache) for setup details and supported providers. + +### Authenticate to the upstream registry + +If pull-through isn't an option, log in to the upstream registry before the build step using `docker/login-action`. The Depot remote builder picks up the credentials and uses them for `FROM` pulls. + +```yaml +jobs: + build: + runs-on: depot-ubuntu-latest + steps: + - uses: actions/checkout@v4 + - uses: depot/setup-action@v1 + + - name: Login to private registry + uses: docker/login-action@v3 + with: + registry: private-registry.example.com + username: ${{ secrets.REGISTRY_USERNAME }} + password: ${{ secrets.REGISTRY_PASSWORD }} + + - uses: depot/build-push-action@v1 + with: + project: + context: . +``` + +## Registry examples + +### Depot Registry + +Use the `save` input to store the built image in the [Depot Registry](/docs/registry/overview) without any additional login steps. You can tag the image with `save-tag` or `save-tags` to retrieve it later with `depot pull` or `docker pull`. + +```yaml +name: Build image + +on: + push: + branches: + - main + +jobs: + build: + runs-on: depot-ubuntu-latest + steps: + - uses: actions/checkout@v4 + - uses: depot/setup-action@v1 + + - name: Build and save to Depot Registry + uses: depot/build-push-action@v1 + with: + project: + context: . + save: true + save-tags: | + latest + ${{ github.sha }} +``` + +The saved image can then be pulled from the Depot Registry: + +```shell +# Using the Depot CLI (uses existing CLI credentials) +depot pull --project latest + +# Using Docker (requires docker login first) +docker login registry.depot.dev -u x-token -p $(depot pull-token --project ) +docker pull registry.depot.dev/:latest +``` + +### Amazon ECR + +Use the [`aws-actions/configure-aws-credentials`](https://github.com/aws-actions/configure-aws-credentials) and [`aws-actions/amazon-ecr-login`](https://github.com/aws-actions/amazon-ecr-login) actions to authenticate to your ECR registry, then build and push with `depot/build-push-action`. AWS authentication uses the OIDC token Depot CI issues when you set `id-token: write`. + +```yaml +name: Build image + +on: + push: + branches: + - main + +jobs: + build: + runs-on: depot-ubuntu-latest + permissions: + contents: read + id-token: write + steps: + - uses: actions/checkout@v4 + - uses: depot/setup-action@v1 + + - name: Configure AWS Credentials + uses: aws-actions/configure-aws-credentials@v4 + with: + role-to-assume: arn:aws:iam:::role/ + aws-region: + + - name: Login to Amazon ECR + id: ecr-login + uses: aws-actions/amazon-ecr-login@v2 + + - name: Build and push + uses: depot/build-push-action@v1 + with: + project: + context: . + push: true + tags: ${{ steps.ecr-login.outputs.registry }}/:latest +``` + +For the OIDC trust relationship setup on the AWS side, see [OIDC with Depot CI](/docs/ci/oidc). + +### GCP Artifact Registry + +Use the [`google-github-actions/auth`](https://github.com/google-github-actions/auth) and [`google-github-actions/setup-gcloud`](https://github.com/google-github-actions/setup-gcloud) actions to authenticate to your Artifact Registry, then build and push with `depot/build-push-action`. + +```yaml +name: Build image + +on: + push: + branches: + - main + +jobs: + build: + runs-on: depot-ubuntu-latest + permissions: + contents: read + id-token: write + steps: + - uses: actions/checkout@v4 + - uses: depot/setup-action@v1 + + - uses: google-github-actions/auth@v3 + with: + service_account: '...' + workload_identity_provider: '...' + + - uses: google-github-actions/setup-gcloud@v3 + with: + project_id: + + - name: Configure docker for GCP + run: gcloud auth configure-docker -docker.pkg.dev --quiet + + - name: Build and push + uses: depot/build-push-action@v1 + with: + project: + context: . + push: true + tags: -docker.pkg.dev//:latest + provenance: false +``` + +### Azure Container Registry + +Use the [`azure/login`](https://github.com/azure/login) action to authenticate with Azure, then `az acr login` to obtain a registry token before building and pushing with `depot/build-push-action`. + +```yaml +name: Build image + +on: + push: + branches: + - main + +jobs: + build: + runs-on: depot-ubuntu-latest + permissions: + contents: read + id-token: write + steps: + - uses: actions/checkout@v4 + - uses: depot/setup-action@v1 + + - name: Login to Azure + uses: azure/login@v2 + with: + client-id: ${{ secrets.AZURE_CLIENT_ID }} + tenant-id: ${{ secrets.AZURE_TENANT_ID }} + subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }} + + - name: Login to Azure Container Registry + run: az acr login --name + + - name: Build and push + uses: depot/build-push-action@v1 + with: + project: + context: . + push: true + tags: .azurecr.io/: +``` + +### Docker Hub + +Use the [`docker/login-action`](https://github.com/docker/login-action) to authenticate to Docker Hub, then build and push with `depot/build-push-action`. + +```yaml +name: Build image + +on: + push: + branches: + - main + +jobs: + build: + runs-on: depot-ubuntu-latest + steps: + - uses: actions/checkout@v4 + - uses: depot/setup-action@v1 + + - name: Login to DockerHub + uses: docker/login-action@v3 + with: + username: ${{ secrets.DOCKERHUB_USERNAME }} + password: ${{ secrets.DOCKERHUB_TOKEN }} + + - name: Build and push + uses: depot/build-push-action@v1 + with: + project: + context: . + push: true + tags: user/app:latest +``` + +### Multiple registries + +Log in to each registry individually and pass multiple tags to push the image to all of them. + +```yaml +name: Build image + +on: + push: + branches: + - main + +jobs: + build: + runs-on: depot-ubuntu-latest + permissions: + contents: read + id-token: write + steps: + - uses: actions/checkout@v4 + - uses: depot/setup-action@v1 + + - name: Configure AWS Credentials + uses: aws-actions/configure-aws-credentials@v4 + with: + role-to-assume: arn:aws:iam:::role/ + aws-region: + + - name: Login to DockerHub + uses: docker/login-action@v3 + with: + username: ${{ secrets.DOCKERHUB_USERNAME }} + password: ${{ secrets.DOCKERHUB_TOKEN }} + + - name: Login to Amazon ECR + id: ecr-login + uses: aws-actions/amazon-ecr-login@v2 + + - name: Build and push + uses: depot/build-push-action@v1 + with: + project: + context: . + push: true + tags: | + /:latest + ${{ steps.ecr-login.outputs.registry }}/:latest +``` + +## Other examples + +### Multi-platform images + +Use the `platforms` input to build for Intel and Arm architectures natively without emulation. + +```yaml +name: Build image + +on: + push: + branches: + - main + +jobs: + build: + runs-on: depot-ubuntu-latest + steps: + - uses: actions/checkout@v4 + - uses: depot/setup-action@v1 + + - name: Login to DockerHub + uses: docker/login-action@v3 + with: + username: ${{ secrets.DOCKERHUB_USERNAME }} + password: ${{ secrets.DOCKERHUB_TOKEN }} + + - name: Build and push + uses: depot/build-push-action@v1 + with: + project: + context: . + platforms: linux/amd64,linux/arm64 + push: true + tags: user/app:latest +``` + +### Export an image to Docker + +By default, Depot doesn't return the built image to the client. Pass `load: true` to make the image available in your workflow for subsequent steps like integration tests. + +```yaml +name: Build image + +on: + push: + branches: + - main + +jobs: + build: + runs-on: depot-ubuntu-latest + steps: + - uses: actions/checkout@v4 + - uses: depot/setup-action@v1 + + - name: Build and load + uses: depot/build-push-action@v1 + with: + project: + context: . + load: true + tags: test-container + + - name: Run integration test with built container + run: ... +``` + +### Software Bill of Materials + +Use the `sbom` and `sbom-dir` inputs to generate an SBOM for the image and output it to a directory. You can then upload it as a build artifact with `actions/upload-artifact`. + +```yaml +name: Build image with SBOM + +on: + push: + branches: + - main + +jobs: + build: + runs-on: depot-ubuntu-latest + steps: + - uses: actions/checkout@v4 + - uses: depot/setup-action@v1 + + - name: Build with SBOM + uses: depot/build-push-action@v1 + with: + project: + context: . + sbom: true + sbom-dir: ./sbom-output + + - name: Upload SBOM + uses: actions/upload-artifact@v4 + with: + path: ./sbom-output + name: sbom +``` diff --git a/content/container-builds/integrations/github-actions.mdx b/content/container-builds/integrations/github-actions.mdx index a8611f0..d5e32f7 100644 --- a/content/container-builds/integrations/github-actions.mdx +++ b/content/container-builds/integrations/github-actions.mdx @@ -127,10 +127,11 @@ jobs: The saved image can then be pulled from the Depot Registry: ```shell -# Using the Depot CLI +# Using the Depot CLI (uses existing CLI credentials) depot pull --project latest -# Using Docker +# Using Docker (requires docker login first) +docker login registry.depot.dev -u x-token -p $(depot pull-token --project ) docker pull registry.depot.dev/:latest ``` diff --git a/content/container-builds/quickstart.mdx b/content/container-builds/quickstart.mdx index f5e3b16..b74ab89 100644 --- a/content/container-builds/quickstart.mdx +++ b/content/container-builds/quickstart.mdx @@ -56,6 +56,7 @@ Depot integrates with any CI provider. Use the following guides to help you get - [Bitbucket Pipelines](/docs/container-builds/integrations/bitbucket-pipelines) - [Buildkite](/docs/container-builds/integrations/buildkite) - [CircleCI](/docs/container-builds/integrations/circleci) +- [Depot CI](/docs/container-builds/integrations/depot-ci) - [GitHub Actions](/docs/container-builds/integrations/github-actions) - [GitLab CI](/docs/container-builds/integrations/gitlab-ci) - [Google Cloud Build](/docs/container-builds/integrations/google-cloud-build) diff --git a/content/github-actions/troubleshooting.mdx b/content/github-actions/troubleshooting.mdx index 601cfb8..327a217 100644 --- a/content/github-actions/troubleshooting.mdx +++ b/content/github-actions/troubleshooting.mdx @@ -157,7 +157,7 @@ If you continue to see this error, reach out to support with your workflow run U -## Error: `The operation was canceled.` +## Error: `Step canceled by GitHub` This error indicates that GitHub terminated your job before it completed. Some common reasons for workflow cancellation include manual cancellation, workflow concurrency settings, or memory exhaustion.