Use "docker compose" (v2) as a built-in plugin instead of standalone docker-compose

[max-tet]

Future versions of shards will have a modern Docker installed that includes the compose plugin. Use that instead of the outdated "docker-compose". But fall back to "docker-compose" if it does not exit.

[ClaydeCode]

Implementation Plan

Implementation Plan: Use docker compose (v2 plugin) with fallback to docker-compose

Summary

The codebase currently calls docker-compose (standalone v1/v2 binary) as a single executable. Modern Docker installations ship Compose as a built-in plugin invoked via docker compose (two words, no hyphen). The issue asks us to detect which is available at runtime and prefer the plugin form, falling back to the standalone binary.

Files to Modify

1. shard_core/util/subprocess.py

Add a module-level async helper (or sync cached function) that detects which compose command to use, returning either ["docker", "compose"] or ["docker-compose"].

2. shard_core/service/app_tools.py

Replace every hard-coded "docker-compose" string with the result of the detection helper.

3. Dockerfile

Switch from installing docker-compose (APT package) to installing the Docker Compose plugin (docker-compose-plugin on Debian/Ubuntu) while keeping docker-compose as a fallback.

4. .github/workflows/test.yml

Update the CI setup step to use the compose plugin (or keep the existing standalone action but ensure the version matches what's tested).

---

Implementation Approach

Step 1 — Compose command detection utility

Add a cached async function (or sync function using functools.lru_cache) to detect compose availability once per process lifetime:

# shard_core/util/subprocess.py  (new addition)
import shutil, asyncio, functools

@functools.lru_cache(maxsize=1)
def _detect_compose_command() -> tuple[str, ...]:
    """
    Return the compose command prefix to use.
    Checks 'docker compose version' first (plugin), then falls back to
    the standalone 'docker-compose' binary.
    """
    import subprocess as _sp
    try:
        result = _sp.run(
            ["docker", "compose", "version"],
            capture_output=True, timeout=5
        )
        if result.returncode == 0:
            return ("docker", "compose")
    except (FileNotFoundError, _sp.TimeoutExpired):
        pass
    return ("docker-compose",)

def compose_command() -> tuple[str, ...]:
    return _detect_compose_command()

Key decisions:

• Synchronous detection with lru_cache: The detection runs once, synchronously, on first call. This is safe because it completes in milliseconds and happens before any async compose work.
• docker compose version is the canonical way to test plugin availability per Docker docs.
• Returns a tuple so callers can unpack with *compose_command() into the existing subprocess call.

Step 2 — Update app_tools.py

Replace all four "docker-compose" occurrences:

# Before:
await subprocess("docker-compose", "up", "--no-start", cwd=...)

# After:
from shard_core.util.subprocess import compose_command
await subprocess(*compose_command(), "up", "--no-start", cwd=...)

Affected call sites:

• docker_create_app_containers — up --no-start
• docker_start_app — up -d
• docker_stop_app — stop
• docker_shutdown_app — down

Step 3 — Update Dockerfile

# Before:
RUN apt-get update && apt-get install --no-install-recommends -y \
    curl \
    docker-compose \
    rclone \
    && apt-get clean

# After:
RUN apt-get update && apt-get install --no-install-recommends -y \
    curl \
    docker-compose-plugin \
    rclone \
    && apt-get clean

The docker-compose-plugin package installs the v2 plugin. The runtime detection in Step 1 means that if for any reason the plugin is absent (old base image, different OS), docker-compose standalone will be used automatically.

Step 4 — Update CI (.github/workflows/test.yml)

The current CI step:

- uses: KengoTODA/actions-setup-docker-compose@v1
  with:
    version: '2.14.2'

This installs the standalone binary. Options:

• Keep as-is: The fallback detection will find docker-compose and tests will pass. Lower risk.
• Switch to plugin: Remove this step entirely and rely on the compose plugin that comes pre-installed in the GitHub Actions runner (ubuntu-latest ships Docker with compose plugin). Higher fidelity to production.

Recommended: Remove the KengoTODA/actions-setup-docker-compose step entirely and rely on the runner's built-in compose plugin. This tests the primary code path. If the runner's Docker version is too old, add docker/setup-buildx-action or upgrade the runner.

---

Edge Cases and Risks

Case	Handling
Neither docker compose nor docker-compose available	Detection will fall through to returning ("docker-compose",), which will then fail at runtime with a clear SubprocessError. No silent failure.
Docker installed but compose plugin missing	Falls back to standalone docker-compose if present.
Very old Docker where docker compose version returns non-zero despite plugin existing	Unlikely; docker compose version is stable since v2.0. If needed, can also check docker --help | grep compose.
CI environment	GitHub Actions ubuntu-latest runners have Docker 24+ with compose plugin. The fallback covers older setups.
Detection timing	lru_cache ensures detection runs once. Thread-safe for Python's GIL.
DOCKER_HOST / remote daemons	docker compose version uses the same DOCKER_HOST as actual compose commands, so detection is consistent.

---

Verification

1. Unit test — add a test in tests/ that mocks subprocess.run to simulate both plugin-available and plugin-absent scenarios, asserting that compose_command() returns the correct tuple. Note: must clear lru_cache between test cases.
2. Integration test — run the existing test suite; all docker-compose invocations should work unchanged.
3. Manual smoke test — on the target VM, run docker compose version manually to confirm it works, then start an app through the API and confirm containers come up.
4. Fallback test — temporarily rename or remove the docker-compose binary / plugin on a test machine and confirm the appropriate fallback error is raised.

---

Questions

None — the issue is straightforward. The only judgment call is whether to update CI to use the plugin path directly (recommended) or leave it using the standalone binary (safe fallback). I'll go with removing the CI step to test the production code path unless instructed otherwise.

---

React with 👍 to approve this plan and start implementation.

[ClaydeCode]

Implementation Complete

I've implemented the docker compose (v2 plugin) support with fallback to docker-compose. PR: #87

What changed

shard_core/util/subprocess.py — Added compose_command() that runs docker compose version once (cached via lru_cache) and returns ("docker", "compose") if the plugin is available, or ("docker-compose",) as a fallback.

shard_core/service/app_tools.py — All four "docker-compose" subprocess calls replaced with *compose_command().

Dockerfile — Removed docker-compose from the APT packages; the get.docker.com install already ships the compose plugin on modern Docker.

.github/workflows/test.yml — Removed the KengoTODA/actions-setup-docker-compose step; GitHub's ubuntu-latest runners include Docker 24+ with the compose plugin built in, so this step is no longer needed and tests will exercise the primary code path.

tests/test_compose_command.py — Unit tests covering: plugin present, non-zero exit fallback, FileNotFoundError fallback, TimeoutExpired fallback, and that detection is cached (runs only once).

[ClaydeCode]

Implemented docker compose v2 plugin support with fallback to standalone docker-compose. Added compose_command() detection helper (lru_cache) to shard_core/util/subprocess.py, updated all docker-compose calls in app_tools.py to use *compose_command(), updated Dockerfile to drop standalone docker-compose package, removed KengoTODA CI step from test.yml, and added unit tests. Opened PR #87 and posted completion comment on issue #35.