Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 14 additions & 4 deletions .github/workflows/drupal-integration-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -102,7 +102,7 @@ jobs:
--name ${{ env.WORKSPACE_NAME }} \
--yes \
--variable workspace_image_registry=index.docker.io/ddev/coder-ddev \
--variable cache_path=/tmp/ci-no-cache
--variable cache_path=/home/rfay/cache/drupal-core-seed

- name: Create workspace
run: |
Expand Down Expand Up @@ -239,7 +239,7 @@ jobs:
--name ${{ env.WORKSPACE_NAME }} \
--yes \
--variable workspace_image_registry=index.docker.io/ddev/coder-ddev \
--variable cache_path=/tmp/ci-no-cache
--variable cache_path=/home/rfay/cache/drupal-core-seed

- name: Create workspace
run: |
Expand Down Expand Up @@ -341,7 +341,7 @@ jobs:
--name cd-${{ github.run_number }}-${{ github.run_attempt }} \
--yes \
--variable workspace_image_registry=index.docker.io/ddev/coder-ddev \
--variable cache_path=/tmp/ci-no-cache
--variable cache_path=/home/rfay/cache/drupal-core-seed

- name: Create workspace
run: |
Expand Down Expand Up @@ -489,7 +489,7 @@ jobs:
--name gd-${{ github.run_number }}-${{ github.run_attempt }} \
--yes \
--variable workspace_image_registry=index.docker.io/ddev/coder-ddev \
--variable cache_path=/tmp/ci-no-cache
--variable cache_path=/home/rfay/cache/drupal-core-seed

- name: Create workspace
run: |
Expand Down Expand Up @@ -522,6 +522,16 @@ jobs:
echo "Current branch: $CURRENT Expected: ${{ env.ISSUE_BRANCH }}"
[[ "$CURRENT" == "${{ env.ISSUE_BRANCH }}" ]] || { echo "ERROR: wrong branch" >&2; exit 1; }

- name: Verify workspace — git tree is clean
run: |
DIRTY=$(coder ssh ${{ env.WORKSPACE_NAME }} -- git -C /home/coder/drupal-core status --short | tr -d '\r')
if [[ -n "$DIRTY" ]]; then
echo "ERROR: dirty git tree after issue branch checkout:" >&2
echo "$DIRTY" >&2
exit 1
fi
echo "OK: git tree is clean"

- name: Verify workspace — Drush DB connected
run: coder ssh ${{ env.WORKSPACE_NAME }} -- ddev drush status --fields=db-status | grep -i connected

Expand Down
42 changes: 31 additions & 11 deletions docs/admin/server-setup.md
Original file line number Diff line number Diff line change
Expand Up @@ -857,21 +857,27 @@ This deploys four templates:

## Step 10: Set Up the Drupal Core Seed Cache (optional, highly recommended)

The `drupal-core` template can provision workspaces faster using a **seed cache** on the host. The cache is a plain git clone of drupal/drupal. New workspaces pass it as a `--reference` hint to `git clone`, reusing local git objects and avoiding several hundred MB of network transfer. Composer install still runs fresh inside each workspace.
The `drupal-core` template can provision workspaces faster using a **seed cache** on the host. The cache is a **bare** git clone of drupal/drupal. When a workspace starts, the startup script:

The seed cache is just a simple git checkout — no DDEV project, no database, no vendor directory, no composer files.
1. Creates a fresh git repo in the workspace directory
2. Adds the bare cache as a local remote and fetches from it (fast — reuses local objects, no network)
3. Removes the cache remote
4. Fetches from `git.drupalcode.org` for any commits newer than the cache
5. Checks out `main` (or the issue branch)

This avoids downloading hundreds of MB per workspace and eliminates working-tree issues that a non-bare cache causes.

### One-time initial setup

```bash
git clone https://git.drupalcode.org/project/drupal.git ~/cache/drupal-core-seed
git clone --bare https://git.drupalcode.org/project/drupal.git ~/cache/drupal-core-seed
```

The seed directory IS the git clone — no subdirectory nesting. No DDEV, no vendor, no composer files.
The seed directory IS the bare clone root — no subdirectory nesting. No working tree, no DDEV, no vendor, no composer files.

### Install the hourly update timer

The update script runs `git fetch --all --prune` to keep the cache current. Install it as an hourly systemd timer:
The update script runs `git fetch --all --prune` to keep the bare cache current. Because there is no working tree, no `git merge` is needed. Install it as an hourly systemd timer:

```bash
REPO=~/workspace/coder-ddev # adjust if your repo is elsewhere
Expand Down Expand Up @@ -966,19 +972,33 @@ make push-template-drupal-core DRUPAL_CACHE_PATH=/your/cache/path

When a workspace starts for the first time:

1. The startup script checks for `.git` at `/home/coder-cache-seed` (the read-only bind mount of `cache_path`)
2. **Cache hit:** `git clone --reference` reuses local git objects for the initial clone (fast), then `composer install` runs fresh inside the container
3. **Cache miss** (path absent or no `.git` at root): `git clone` runs without a reference — slower but always works
1. The startup script detects a bare repo at `/home/coder-cache-seed` (the read-only bind mount of `cache_path`)
2. **Bare cache hit:** `git init` + `git fetch drupalcache` (fast local copy of all objects) + `git fetch origin` (only the delta) + checkout
3. **Legacy non-bare cache:** `git clone --reference` (old behaviour, still supported)
4. **Cache miss** (path absent or not a git repo): `git clone` runs without any cache — slower but always works

Check workspace startup logs in the Coder dashboard or at `/tmp/drupal-setup.log` inside the workspace to confirm which path was taken. Look for "bare cache" or "reference from cache seed" in the log.

### Migrating from a non-bare (legacy) cache

If you have an existing non-bare clone at `~/cache/drupal-core-seed`, replace it with a bare clone:

```bash
mv ~/cache/drupal-core-seed ~/cache/drupal-core-seed.bak
git clone --bare https://git.drupalcode.org/project/drupal.git ~/cache/drupal-core-seed
# Once confirmed working:
rm -rf ~/cache/drupal-core-seed.bak
```

Check workspace startup logs in the Coder dashboard or at `/tmp/drupal-setup.log` inside the workspace to confirm which path was taken.
The `update-drupal-cache` script handles both bare and non-bare repos automatically.

### Troubleshooting

**Cache not being used:**

- Verify the seed directory is a git clone: `ls ~/cache/drupal-core-seed/.git`
- Verify the seed directory is a bare git repo: `git -C ~/cache/drupal-core-seed rev-parse --is-bare-repository` (should print `true`)
- Confirm `cache_path` in the deployed template matches your actual seed directory
- Look for "with reference from cache seed" in `/tmp/drupal-setup.log`; absence means the path was missing or `.git` was not at the root
- Look for "bare cache" or "reference from cache seed" in `/tmp/drupal-setup.log`; absence means the path was missing or not a valid git repo

**Update script fails:**

Expand Down
48 changes: 33 additions & 15 deletions drupal-core/scripts/update-drupal-cache
Original file line number Diff line number Diff line change
Expand Up @@ -4,23 +4,29 @@
# Refreshes the drupal-core seed cache used by the drupal-core Coder template.
# Run this script on the Coder host server as the user who owns the seed directory.
#
# The seed cache is a plain git clone of drupal/drupal. New workspaces pass it
# as a --reference hint to git clone, reusing local git objects and avoiding
# several hundred MB of network transfer. Composer install still runs fresh
# inside each workspace.
# The seed cache is a BARE git clone of drupal/drupal. Workspaces clone from it
# locally (very fast — git reuses local objects), then fetch only the delta from
# the real remote, avoiding several hundred MB of network transfer per workspace.
#
# Using a bare repo is intentional: there is no working tree to keep in sync and
# no "Your branch is behind" noise. git fetch --all --prune is all that is needed.
#
# Initial setup (one-time, run as the seed-directory owner):
# git clone --bare https://git.drupalcode.org/project/drupal.git ~/cache/drupal-core-seed
#
# Migration from a non-bare clone:
# mv ~/cache/drupal-core-seed ~/cache/drupal-core-seed.bak
# git clone --bare https://git.drupalcode.org/project/drupal.git ~/cache/drupal-core-seed
# rm -rf ~/cache/drupal-core-seed.bak # once you've confirmed the new one works
#
# Usage:
# ./update-drupal-cache [--seed-dir PATH]
#
# Options:
# --seed-dir PATH Absolute path to the seed directory (the git clone root).
# --seed-dir PATH Absolute path to the bare seed repository.
# Default: ~/cache/drupal-core-seed
# The Coder template's cache_path variable must be set to the same path.
#
# Note: When run via the systemd timer, the seed directory is taken from the default
# or from the ExecStart line in drupal-cache-updater.service — edit that file to pass
# --seed-dir if your seed directory differs from the default.
#
# IMPORTANT: This script must be reinstalled to /usr/local/bin/ after any changes:
# sudo install -m 755 drupal-core/scripts/update-drupal-cache /usr/local/bin/update-drupal-cache

Expand All @@ -42,23 +48,35 @@ while [[ $# -gt 0 ]]; do
esac
done

if [ ! -d "$SEED_DIR/.git" ]; then
echo "Error: Git clone not found at $SEED_DIR" >&2
# Detect bare vs. non-bare clone
IS_BARE=false
if git -C "$SEED_DIR" rev-parse --is-bare-repository 2>/dev/null | grep -q true; then
IS_BARE=true
elif [ ! -d "$SEED_DIR/.git" ]; then
echo "Error: No git repository found at $SEED_DIR" >&2
echo "Run the initial setup first. See docs/admin/server-setup.md." >&2
exit 1
fi

if [ "$IS_BARE" = "false" ]; then
echo "WARNING: $SEED_DIR is not a bare repository." >&2
echo "A bare clone is recommended. See migration instructions in this script's header." >&2
fi

echo "=== Updating drupal-core seed cache ==="
echo "Seed directory: $SEED_DIR"
echo "Started: $(date)"
echo "Bare: $IS_BARE"
echo "Started: $(date)"
echo ""

echo "Fetching Drupal core git objects..."
git -C "$SEED_DIR" fetch --all --prune

echo "Fast-forwarding local main branch..."
git -C "$SEED_DIR" merge --ff-only origin/main || \
echo "Warning: could not fast-forward main (local changes?); objects still updated."
if [ "$IS_BARE" = "false" ]; then
echo "Fast-forwarding local main branch..."
git -C "$SEED_DIR" merge --ff-only origin/main || \
echo "Warning: could not fast-forward main (local changes?); objects still updated."
fi

echo ""
echo "=== Seed cache updated successfully ==="
Expand Down
49 changes: 39 additions & 10 deletions drupal-core/template.tf
Original file line number Diff line number Diff line change
Expand Up @@ -608,13 +608,38 @@ STATUS_HEADER
DRUPAL_SETUP_NEEDED=true
update_status "⏳ Git clone: In progress..."
_t=$SECONDS
if [ -d "$CACHE_SEED/.git" ]; then
DRUPAL_REMOTE="https://git.drupalcode.org/project/drupal.git"
if git -C "$CACHE_SEED" rev-parse --is-bare-repository 2>/dev/null | grep -q true; then
# Bare cache: init repo, add all remotes (cache, origin, issue fork if
# applicable), fetch cache first (fast local copy of all objects), then
# fetch origin and issue fork for only the delta.
log_setup "Initialising Drupal core repo (bare cache + origin delta)..."
mkdir -p "$DRUPAL_DIR"
git -C "$DRUPAL_DIR" init >> "$SETUP_LOG" 2>&1
git -C "$DRUPAL_DIR" remote add drupalcache "$CACHE_SEED"
git -C "$DRUPAL_DIR" remote add origin "$DRUPAL_REMOTE"
if [ "$USING_ISSUE_FORK" = "true" ] && [ -n "$ISSUE_FORK" ]; then
git -C "$DRUPAL_DIR" remote add issue "https://git.drupalcode.org/issue/drupal-$${ISSUE_FORK}.git"
fi
git -C "$DRUPAL_DIR" fetch drupalcache >> "$SETUP_LOG" 2>&1
# Remove the local cache remote now — objects are in the local repo and
# keeping it causes ambiguous-ref errors when checking out branches that
# exist in both drupalcache and origin.
git -C "$DRUPAL_DIR" remote remove drupalcache
git -C "$DRUPAL_DIR" fetch origin >> "$SETUP_LOG" 2>&1 || true
if [ "$USING_ISSUE_FORK" = "true" ] && [ -n "$ISSUE_FORK" ]; then
git -C "$DRUPAL_DIR" fetch issue >> "$SETUP_LOG" 2>&1 || true
fi
git -C "$DRUPAL_DIR" checkout -b main --track origin/main >> "$SETUP_LOG" 2>&1
elif [ -d "$CACHE_SEED/.git" ]; then
# Non-bare (legacy) cache: use as --reference hint to avoid re-downloading
# objects that already exist locally.
log_setup "Cloning Drupal core (with reference from cache seed)..."
git clone --reference "$CACHE_SEED" https://git.drupalcode.org/project/drupal.git "$DRUPAL_DIR" >> "$SETUP_LOG" 2>&1 || \
git clone https://git.drupalcode.org/project/drupal.git "$DRUPAL_DIR" >> "$SETUP_LOG" 2>&1
git clone --reference "$CACHE_SEED" "$DRUPAL_REMOTE" "$DRUPAL_DIR" >> "$SETUP_LOG" 2>&1 || \
git clone "$DRUPAL_REMOTE" "$DRUPAL_DIR" >> "$SETUP_LOG" 2>&1
else
log_setup "Cloning Drupal core..."
git clone https://git.drupalcode.org/project/drupal.git "$DRUPAL_DIR" >> "$SETUP_LOG" 2>&1
git clone "$DRUPAL_REMOTE" "$DRUPAL_DIR" >> "$SETUP_LOG" 2>&1
fi
if [ -d "$DRUPAL_DIR/.git" ]; then
log_setup "✓ Drupal core cloned ($((SECONDS - _t))s)"
Expand All @@ -627,18 +652,22 @@ STATUS_HEADER

# Branch or fork checkout
if [ "$SETUP_FAILED" != "true" ] && [ "$USING_ISSUE_FORK" = "true" ] && [ -n "$ISSUE_FORK" ]; then
log_setup "Adding issue fork remote and fetching: $ISSUE_FORK"
git -C "$DRUPAL_DIR" remote add issue "https://git.drupalcode.org/issue/drupal-$${ISSUE_FORK}.git"
if git -C "$DRUPAL_DIR" fetch issue >> "$SETUP_LOG" 2>&1; then
# Add issue remote and fetch unless already done in the bare-cache setup above.
if ! git -C "$DRUPAL_DIR" remote get-url issue >> "$SETUP_LOG" 2>&1; then
log_setup "Adding issue fork remote and fetching: $ISSUE_FORK"
git -C "$DRUPAL_DIR" remote add issue "https://git.drupalcode.org/issue/drupal-$${ISSUE_FORK}.git"
git -C "$DRUPAL_DIR" fetch issue >> "$SETUP_LOG" 2>&1
fi
if git -C "$DRUPAL_DIR" branch -r | grep -q "^ issue/"; then
log_setup " ✓ Fetched from issue remote"
if [ -n "$ISSUE_BRANCH" ]; then
if git -C "$DRUPAL_DIR" checkout -b "$ISSUE_BRANCH" "issue/$ISSUE_BRANCH" >> "$SETUP_LOG" 2>&1 || \
git -C "$DRUPAL_DIR" checkout "$ISSUE_BRANCH" >> "$SETUP_LOG" 2>&1; then
log_setup " ✓ Checked out branch: $ISSUE_BRANCH"
# Ensure the working tree exactly matches the checked-out branch.
# When cloning with --reference, origin/main may be ahead of the
# issue branch base; git updates the index but can leave working tree
# files from those newer main commits behind as modified/untracked.
# origin/main may be ahead of the issue branch base; git updates
# the index but can leave working tree files from those newer main
# commits behind as modified/untracked.
git -C "$DRUPAL_DIR" reset --hard HEAD >> "$SETUP_LOG" 2>&1 || true
git -C "$DRUPAL_DIR" clean -fd >> "$SETUP_LOG" 2>&1 || true
log_setup " ✓ Working tree reset to match branch"
Expand Down