-
Notifications
You must be signed in to change notification settings - Fork 4
Feat/rep 1379 - additional PG-18 container with switch command to go switch between postgresql 14 and 18 (bidirectional switch) #6
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Large diffs are not rendered by default.
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,145 @@ | ||
| # PostgreSQL 14 ↔ 18 switch (DDEV host commands) | ||
|
|
||
| This guide describes how to use the host commands under `.ddev/commands/host/` to back up PostgreSQL 14, load data into the PostgreSQL 18 sidecar, validate, and point the app at PG14 or PG18 via local config files. | ||
|
|
||
| ## Prerequisites | ||
|
|
||
| - DDEV is installed and the project starts successfully (`ddev start`). | ||
| - The **PostgreSQL 18** service is enabled (for example via `.ddev/docker-compose.postgres18.yaml`) so that PG18 listens on **host port `5433`** (mapped to `5432` in the `postgres18` container). The main DDEV database remains **PostgreSQL 14 on port `5432`** unless you change the project database image. | ||
| - You have created the **versioned local config files** described below (they are not generated automatically). | ||
|
|
||
| ## Configuration files you must create manually | ||
|
|
||
| The command `ddev switch-pg-version` does not edit connection strings inline. It expects **pairs** of version-specific files under **`site/config/`** (paths are relative to the **project root**, parent of `.ddev`). | ||
|
|
||
| For each **base name** below, you need **two** files: one for PG14 and one for PG18. The command copies the chosen file onto the active `*-local.php` name the app already uses. | ||
|
|
||
| | Base name | Active file (used by the app) | PG14 source | PG18 source | | ||
| |-----------|-------------------------------|-------------|-------------| | ||
| | `db_write` | `site/config/db_write-local.php` | `site/config/db_write-local-pg14.php` | `site/config/db_write-local-pg18.php` | | ||
| | `db_read` | `site/config/db_read-local.php` | `site/config/db_read-local-pg14.php` | `site/config/db_read-local-pg18.php` | | ||
| | `db_read_report` | `site/config/db_read_report-local.php` | `site/config/db_read_report-local-pg14.php` | `site/config/db_read_report-local-pg18.php` | | ||
| | `common` | `site/config/common-local.php` | `site/config/common-local-pg14.php` | `site/config/common-local-pg18.php` | | ||
|
|
||
| **What to put in each file** | ||
|
|
||
| - **`db_write-local-pg14.php`**, **`db_read-local-pg14.php`**, **`db_read_report-local-pg14.php`**: same connection settings you use today for the **main** DDEV database (**host** `db` or `127.0.0.1` from the host as appropriate for your stack, **port `5432`**, user/password/database as in your current setup). | ||
| - **`db_write-local-pg18.php`**, **`db_read-local-pg18.php`**, **`db_read_report-local-pg18.php`**: point reads/writes to the **PG18 sidecar** from the app’s perspective. Typically from **inside the web container** you use host **`postgres18`** (Docker network alias) and port **`5432`**; from the **host** for CLI tools, use **`127.0.0.1`** and port **`5433`**. Adjust user, password, and database name to match `.ddev/docker-compose.postgres18.yaml` (defaults are often `db` / `db` / `db`). | ||
| - **`common-local-pg14.php`** / **`common-local-pg18.php`**: mirror whatever you keep in `common-local.php` today (cache, queues, etc.), with only the differences required for each database version if any. | ||
|
|
||
| **Minimum checklist:** all **eight** versioned files must exist (`*-local-pg14.php` and `*-local-pg18.php` for the four bases). If either file in a pair is missing, `switch-pg-version` prints a warning and skips that base safely. | ||
|
|
||
| **Tip:** Copy your current working `*-local.php` files to `*-local-pg14.php`, then duplicate and edit copies to `*-local-pg18.php` with PG18 host/port/credentials. | ||
|
|
||
| --- | ||
|
|
||
| ## Suggested sequence (first-time migration PG14 → PG18) | ||
|
|
||
| Follow this order once you are ready to load PG18 and switch the app. | ||
|
|
||
| ### 1. Stay on PostgreSQL 14 in the app | ||
|
|
||
| Ensure `switch-pg-version` has not been run to PG18 yet, or run: | ||
|
|
||
| ```bash | ||
| ddev switch-pg-version 14 | ||
| ``` | ||
|
|
||
| Confirm the app still uses PG14 (`*-local.php` content should match PG14 sources after a successful switch). | ||
|
|
||
| ### 2. Back up the PG14 database | ||
|
|
||
| With DDEV running and PG14 reachable on **`localhost:5432`**: | ||
|
|
||
| ```bash | ||
| ddev backup-pg14 | ||
| ``` | ||
|
|
||
| Optional: pass a path for the dump file: | ||
|
|
||
| ```bash | ||
| ddev backup-pg14 ~/backups/my_project_pg14.dump | ||
| ``` | ||
|
|
||
| This produces a **custom-format** `pg_dump` (`.dump`), not plain SQL. | ||
|
|
||
| ### 3. Ensure PostgreSQL 18 is running | ||
|
|
||
| Start the stack so the `postgres18` service is up and **`127.0.0.1:5433`** accepts connections (see your `docker-compose.postgres18.yaml`). | ||
|
|
||
| ### 4. Restore the backup into PostgreSQL 18 | ||
|
|
||
| Because the backup from step 2 is **custom format**, restore with **`pg_restore`**, not `ddev import-db18`. | ||
|
|
||
| Example from the **host** (adjust path and options to match your dump): | ||
|
|
||
| ```bash | ||
| pg_restore -h 127.0.0.1 -p 5433 -U db -d db --no-owner --no-acl --verbose /path/to/your_backup.dump | ||
| ``` | ||
|
|
||
| You may need to drop/recreate the target database or use `--clean` depending on whether `db` is empty; follow your usual migration practice. | ||
|
|
||
| **`ddev import-db18`** is for **plain SQL** (optionally **`.gz`**). Use it when you have a `.sql` or `.sql.gz` file, for example: | ||
|
|
||
| ```bash | ||
| ddev import-db18 ~/exports/dump.sql | ||
| ddev import-db18 ~/exports/dump.sql.gz | ||
| ``` | ||
|
|
||
| ### 5. Validate PG14 vs PG18 (optional but recommended) | ||
|
|
||
| With **both** PG14 (`5432`) and PG18 (`5433`) running: | ||
|
|
||
| ```bash | ||
| ddev validate-pg18-migration | ||
| ``` | ||
|
|
||
| Review extensions, schema/table counts, and any warnings (for example `pg_trgm` reindex reminders). | ||
|
|
||
| ### 6. Point the application at PostgreSQL 18 | ||
|
|
||
| ```bash | ||
| ddev switch-pg-version 18 | ||
| ``` | ||
|
|
||
| ### 7. Clear cache and restart | ||
|
|
||
| As printed by the switch command: | ||
|
|
||
| ```bash | ||
| ddev exec 'rm -rf site/runtime/cache/*' | ||
| ddev restart | ||
| ``` | ||
|
|
||
| Then verify the application against PG18. | ||
|
|
||
| --- | ||
|
|
||
| ## Switching back to PostgreSQL 14 | ||
|
|
||
| When the eight versioned files exist and PG14 is still available: | ||
|
|
||
| ```bash | ||
| ddev switch-pg-version 14 | ||
| ddev exec 'rm -rf site/runtime/cache/*' | ||
| ddev restart | ||
| ``` | ||
|
|
||
| --- | ||
|
|
||
| ## Command reference | ||
|
|
||
| | Command | Purpose | | ||
| |---------|---------| | ||
| | `ddev backup-pg14 [file]` | Custom-format `pg_dump` of PG14 database `db` on `localhost:5432`. | | ||
| | `ddev import-db18 <file>` | Import **plain SQL** (or **gzip** SQL) into PG18 on `127.0.0.1:5433`. | | ||
| | `ddev validate-pg18-migration` | Compare PG14 and PG18 (connections, versions, extensions, rough counts). | | ||
| | `ddev switch-pg-version 14\|18` | Copy `*-local-pg14.php` or `*-local-pg18.php` into `*-local.php` for `db_write`, `db_read`, `db_read_report`, and `common`. | | ||
|
|
||
| --- | ||
|
|
||
| ## Troubleshooting | ||
|
|
||
| - **`switch-pg-version` warns about missing files:** create the missing `site/config/*-local-pg14.php` or `*-local-pg18.php` files (see table above). | ||
| - **Cannot connect to PG18:** confirm `postgres18` is running, port **`5433`** is not blocked, and PG18 credentials in your `*-local-pg18.php` files match the compose file. | ||
| - **Restore errors after `backup-pg14`:** use `pg_restore` for `.dump` files; `import-db18` does not read custom-format dumps. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,44 @@ | ||
| #!/bin/bash | ||
|
|
||
| ## Description: Backup PostgreSQL 14 database | ||
| ## Usage: backup-pg14 [output_file] | ||
| ## Example: backup-pg14 ~/backups/fdsu_pg14.dump | ||
|
|
||
| OUTPUT_FILE="${1:-$HOME/.ddev/backups/pg14/fdsu_pg14_backup_$(date +%Y%m%d_%H%M%S).dump}" | ||
| LOG_DIR="$HOME/.ddev/logs" | ||
| LOG_FILE="$LOG_DIR/backup-pg14-$(date +%Y%m%d-%H%M%S).log" | ||
|
|
||
| mkdir -p "$(dirname "$OUTPUT_FILE")" | ||
| mkdir -p "$LOG_DIR" | ||
|
|
||
| echo "Backing up PostgreSQL 14 database..." | tee -a "$LOG_FILE" | ||
| echo "Output: $OUTPUT_FILE" | tee -a "$LOG_FILE" | ||
| echo "Log: $LOG_FILE" | tee -a "$LOG_FILE" | ||
|
|
||
| # Get database size first | ||
| DB_SIZE=$(PGPASSWORD=db psql -h localhost -p 5432 -U db -d db -t -c "SELECT pg_size_pretty(pg_database_size('db'));" | xargs) | ||
| echo "Database size: $DB_SIZE" | tee -a "$LOG_FILE" | ||
|
|
||
| # Create backup | ||
| pg_dump -h localhost -p 5432 -U db -d db \ | ||
| --format=custom \ | ||
| --verbose \ | ||
| --no-owner \ | ||
| --no-acl \ | ||
| --file="$OUTPUT_FILE" \ | ||
| 2>&1 | tee -a "$LOG_FILE" | ||
|
|
||
| EXIT_CODE=$? | ||
|
|
||
| if [ $EXIT_CODE -eq 0 ]; then | ||
| BACKUP_SIZE=$(ls -lh "$OUTPUT_FILE" | awk '{print $5}') | ||
| echo "✓ Backup completed successfully" | tee -a "$LOG_FILE" | ||
| echo " File: $OUTPUT_FILE" | tee -a "$LOG_FILE" | ||
| echo " Size: $BACKUP_SIZE" | tee -a "$LOG_FILE" | ||
| ls -lh "$OUTPUT_FILE" | ||
| else | ||
| echo "✗ Backup failed with exit code: $EXIT_CODE" | tee -a "$LOG_FILE" | ||
| echo "Check log file: $LOG_FILE" | tee -a "$LOG_FILE" | ||
| exit 1 | ||
| fi | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,34 @@ | ||
| #!/bin/bash | ||
|
|
||
| ## Description: Import database dump into PostgreSQL 18 | ||
| ## Usage: import-db18 [file] | ||
| ## Example: import-db18 ~/Downloads/backup.sql.gz | ||
|
|
||
| if [ -z "$1" ]; then | ||
| echo "Usage: ddev import-db18 <file>" | ||
| echo "Example: ddev import-db18 ~/Downloads/backup.sql.gz" | ||
| exit 1 | ||
| fi | ||
|
|
||
| FILE="$1" | ||
|
|
||
| if [ ! -f "$FILE" ]; then | ||
| echo "Error: File '$FILE' not found" | ||
| exit 1 | ||
| fi | ||
|
|
||
| echo "Importing $FILE into PostgreSQL 18..." | ||
|
|
||
| if [[ "$FILE" == *.gz ]]; then | ||
| gunzip -c "$FILE" | PGPASSWORD=db psql -h 127.0.0.1 -p 5433 -U db -d db | ||
| else | ||
| PGPASSWORD=db psql -h 127.0.0.1 -p 5433 -U db -d db < "$FILE" | ||
| fi | ||
|
|
||
| if [ $? -eq 0 ]; then | ||
| echo "✓ Successfully imported into PostgreSQL 18" | ||
| else | ||
| echo "✗ Import failed" | ||
| exit 1 | ||
| fi | ||
|
Comment on lines
+22
to
+33
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🧩 Analysis chain🏁 Script executed: #!/bin/bash
# Static verification of import error-handling flags/status capture.
rg -n -C2 'ON_ERROR_STOP|pipefail|PIPESTATUS|if \[ \$\? -eq 0 \]' commands/host/import-db18Repository: sandeepfd/fdsu_ddev Length of output: 162 🏁 Script executed: cat -n commands/host/import-db18Repository: sandeepfd/fdsu_ddev Length of output: 1034 Import can report success on partial/failed SQL execution. The piped Proposed fix #!/bin/bash
+set -o pipefail
## Description: Import database dump into PostgreSQL 18
## Usage: import-db18 [file]
@@ -20,14 +21,18 @@
echo "Importing $FILE into PostgreSQL 18..."
if [[ "$FILE" == *.gz ]]; then
- gunzip -c "$FILE" | PGPASSWORD=db psql -h 127.0.0.1 -p 5433 -U db -d db
+ gunzip -c "$FILE" | PGPASSWORD=db psql -v ON_ERROR_STOP=1 -h 127.0.0.1 -p 5433 -U db -d db
+ IMPORT_EXIT=${PIPESTATUS[1]}
else
- PGPASSWORD=db psql -h 127.0.0.1 -p 5433 -U db -d db < "$FILE"
+ PGPASSWORD=db psql -v ON_ERROR_STOP=1 -h 127.0.0.1 -p 5433 -U db -d db < "$FILE"
+ IMPORT_EXIT=$?
fi
-if [ $? -eq 0 ]; then
+if [ $IMPORT_EXIT -eq 0 ]; then
echo "✓ Successfully imported into PostgreSQL 18"
else
echo "✗ Import failed"
exit 1
fi🤖 Prompt for AI Agents |
||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,82 @@ | ||
| #!/bin/bash | ||
| ## Description: Switch database connection between PostgreSQL 14 and 18 by renaming config files | ||
| ## Usage: switch-pg-version [14|18] | ||
| ## Example: switch-pg-version 18 | ||
|
|
||
| VERSION=${1:-14} | ||
|
|
||
| if [ "$VERSION" != "14" ] && [ "$VERSION" != "18" ]; then | ||
| echo "Error: Version must be 14 or 18" | ||
| echo "Usage: ddev switch-pg-version [14|18]" | ||
| exit 1 | ||
| fi | ||
|
|
||
| CONFIG_DIR="site/config" | ||
|
|
||
| if [ "$VERSION" == "18" ]; then | ||
| echo "Switching to PostgreSQL 18..." | ||
| PG_VERSION="pg18" | ||
| else | ||
| echo "Switching to PostgreSQL 14..." | ||
| PG_VERSION="pg14" | ||
| fi | ||
|
|
||
| # Function to switch a config file | ||
| switch_config() { | ||
| local base_name=$1 | ||
| local active_file="$CONFIG_DIR/${base_name}-local.php" | ||
| local pg14_file="$CONFIG_DIR/${base_name}-local-pg14.php" | ||
| local pg18_file="$CONFIG_DIR/${base_name}-local-pg18.php" | ||
|
|
||
| # Check if versioned files exist | ||
| if [ ! -f "$pg14_file" ] || [ ! -f "$pg18_file" ]; then | ||
| echo "⚠ Warning: Versioned files not found for $base_name" | ||
| echo " Expected: $pg14_file" | ||
| echo " Expected: $pg18_file" | ||
| return 1 | ||
| fi | ||
|
|
||
| # Backup current active file if it exists and is not a versioned file | ||
| if [ -f "$active_file" ] && [ ! -L "$active_file" ]; then | ||
| # Check if it's not already a versioned file | ||
| if [[ "$active_file" != *"-pg14.php" ]] && [[ "$active_file" != *"-pg18.php" ]]; then | ||
| mv "$active_file" "${active_file}.backup.$(date +%Y%m%d_%H%M%S)" | ||
| echo " Backed up existing $base_name-local.php" | ||
| fi | ||
| fi | ||
|
|
||
| # Remove existing active file (if it's a symlink or regular file) | ||
| [ -f "$active_file" ] && rm -f "$active_file" | ||
| [ -L "$active_file" ] && rm -f "$active_file" | ||
|
|
||
| # Copy the appropriate versioned file to the active file | ||
| if [ "$VERSION" == "18" ]; then | ||
| cp "$pg18_file" "$active_file" | ||
| else | ||
| cp "$pg14_file" "$active_file" | ||
| fi | ||
|
Comment on lines
+43
to
+57
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Non-atomic replacement can leave configs missing on copy failure. On Line 43 and Lines 49-57, the current flow removes the active file before confirming the new file is safely staged. If Suggested safer replacement flow- mv "$active_file" "${active_file}.backup.$(date +%Y%m%d_%H%M%S)"
+ cp "$active_file" "${active_file}.backup.$(date +%Y%m%d_%H%M%S)"
echo " Backed up existing $base_name-local.php"
fi
fi
-
- # Remove existing active file (if it's a symlink or regular file)
- [ -f "$active_file" ] && rm -f "$active_file"
- [ -L "$active_file" ] && rm -f "$active_file"
-
- # Copy the appropriate versioned file to the active file
- if [ "$VERSION" == "18" ]; then
- cp "$pg18_file" "$active_file"
- else
- cp "$pg14_file" "$active_file"
- fi
+
+ local source_file tmp_file
+ if [ "$VERSION" == "18" ]; then
+ source_file="$pg18_file"
+ else
+ source_file="$pg14_file"
+ fi
+ tmp_file="${active_file}.tmp.$$"
+
+ if ! cp "$source_file" "$tmp_file"; then
+ echo "✗ Failed to stage $source_file"
+ rm -f "$tmp_file"
+ return 1
+ fi
+ if ! mv -f "$tmp_file" "$active_file"; then
+ echo "✗ Failed to activate $active_file"
+ rm -f "$tmp_file"
+ return 1
+ fi🤖 Prompt for AI Agents |
||
|
|
||
| echo "✓ Switched $base_name-local.php to PostgreSQL $VERSION" | ||
| return 0 | ||
| } | ||
|
|
||
| # Switch all config files | ||
| echo "" | ||
| switch_config "db_write" | ||
| switch_config "db_read" | ||
| switch_config "db_read_report" | ||
| switch_config "common" | ||
|
|
||
| echo "" | ||
| echo "✓ Successfully switched to PostgreSQL $VERSION" | ||
| echo "" | ||
| echo "Current active configuration:" | ||
| echo " - db_write-local.php → PostgreSQL $VERSION" | ||
| echo " - db_read-local.php → PostgreSQL $VERSION" | ||
| echo " - db_read_report-local.php → PostgreSQL $VERSION" | ||
| echo " - common-local.php → PostgreSQL $VERSION" | ||
|
Comment on lines
+65
to
+77
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Final success summary is printed even when one or more switches fail.
Track failures and exit non-zero on partial/total failure-# Switch all config files
-echo ""
-switch_config "db_write"
-switch_config "db_read"
-switch_config "db_read_report"
-switch_config "common"
+echo ""
+failures=0
+for base in db_write db_read db_read_report common; do
+ if ! switch_config "$base"; then
+ failures=$((failures + 1))
+ fi
+done
+
+if [ "$failures" -gt 0 ]; then
+ echo ""
+ echo "✗ Switch completed with $failures failure(s)"
+ exit 1
+fi
echo ""
echo "✓ Successfully switched to PostgreSQL $VERSION"🤖 Prompt for AI Agents |
||
| echo "" | ||
| echo "Next steps:" | ||
| echo " 1. Clear cache: ddev exec 'rm -rf site/runtime/cache/*'" | ||
| echo " 2. Restart DDEV: ddev restart" | ||
| echo " 3. Verify connection by checking your application" | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🧩 Analysis chain
🏁 Script executed:
Repository: sandeepfd/fdsu_ddev
Length of output: 268
🏁 Script executed:
Repository: sandeepfd/fdsu_ddev
Length of output: 1754
Fix pipeline exit-code capture for
pg_dumpcommand.The current code captures
tee's exit code (viaEXIT_CODE=$?), notpg_dump's. A failed backup can be silently misreported as successful while the script exits with code 0.Proposed fix
🤖 Prompt for AI Agents