Only setup temporal schema if it is not yet setup.#123
Conversation
📝 WalkthroughWalkthroughThe Temporal frontend init container now uses an idempotent schema helper that updates existing schemas and creates missing schema state before retrying updates for both Temporal and visibility PostgreSQL databases. ChangesTemporal schema initialization
Estimated code review effort: 2 (Simple) | ~10 minutes Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@deploy/helm/templates/temporal.yaml`:
- Around line 117-125: Update run_schema so setup-schema -v 0.0 runs only when
the database explicitly reports an uninitialized or missing schema. Capture the
update-schema result from the first invocation, propagate any other failure
without setup, and retain the existing successful update path and subsequent
migration behavior.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: d76a806a-d9e1-4684-b3ba-5f586848b761
📒 Files selected for processing (1)
deploy/helm/templates/temporal.yaml
| run_schema() { | ||
| ep="$1"; user="$2"; pass="$3"; db="$4"; dir="$5" | ||
| if temporal-sql-tool --ep "$ep" --user "$user" --password "$pass" --port "$PORT" --db "$db" --plugin postgres12 update-schema -d "$dir"; then | ||
| echo "Schema for '$db' already initialized; applied any pending updates." | ||
| return 0 | ||
| fi | ||
| echo "Schema for '$db' not initialized; performing initial setup..." | ||
| temporal-sql-tool --ep "$ep" --user "$user" --password "$pass" --port "$PORT" --db "$db" --plugin postgres12 setup-schema -v 0.0 | ||
| temporal-sql-tool --ep "$ep" --user "$user" --password "$pass" --port "$PORT" --db "$db" --plugin postgres12 update-schema -d "$dir" |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
helm template temporal deploy/helm \
-f deploy/helm/values-ci.yaml \
>/tmp/temporal-rendered.yaml
helm template temporal deploy/helm \
-f deploy/helm/values-ci.yaml \
-f deploy/helm/values-observability.yaml \
>/tmp/temporal-observability-rendered.yamlRepository: NVIDIA/nv-config-manager
Length of output: 204
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Map the chart template structure first
ast-grep outline deploy/helm/templates/temporal.yaml --view expanded
# Read the relevant range around the schema init logic
sed -n '1,220p' deploy/helm/templates/temporal.yaml | cat -n
# Find related schema/init references in the Helm chart
rg -n "setup-schema|update-schema|run_schema|temporal-sql-tool|visibility" deploy/helm -SRepository: NVIDIA/nv-config-manager
Length of output: 18442
🌐 Web query:
Temporal temporal-sql-tool update-schema setup-schema fresh database schema_version behavior
💡 Result:
Temporal uses the temporal-sql-tool to manage database schema versions, ensuring that the Temporal server is compatible with the underlying database [1][2][3]. The two primary commands are: 1. setup-schema: This command initializes the database by creating the required schema versioning tables (schema_version and schema_update_history) [4]. It is typically run with an initial version (e.g., -v 0.0) to establish these tables before any application tables are applied [1][3]. 2. update-schema: This command applies schema migrations [2]. It reads the current version from the schema_version table, identifies the necessary delta/migrations from the specified schema directory, and applies them sequentially [5][6]. After a successful update, it updates the schema_version table to the new version and adds an entry to the schema_update_history table [5]. The schema_version table acts as the source of truth for the database's current state [4]. When the Temporal server starts, it performs a compatibility check by reading this table; if the table is missing or the version is incompatible, the server will fail to start [4][7]. Key points regarding behavior: - Exclusivity: The schema_version table is shared; currently, Temporal and Visibility schemas generally cannot coexist in the same database because they would conflict over the schema_version table [8]. - Order of Operations: You must perform setup-schema before update-schema [1][9]. The update-schema process relies on the presence of the versioning tables created by the setup process [6][4]. - Automating: While manual usage is common for initial setup, many production environments automate these steps during deployment to ensure the database schema matches the expected server version [10][7].
Citations:
- 1: https://github.com/temporalio/temporal/blob/main/tools/sql/README.md
- 2: https://temporalio-temporal.mintlify.app/reference/database-schema
- 3: https://temporalio-temporal.mintlify.app/deployment/database-setup
- 4: https://wwarfield.com/2024/08/25/temporal-io-schema-management-postgresql-mysql-and-cassandra/
- 5: https://github.com/temporalio/temporal/blob/0d8004bf/tools/common/schema/updatetask.go
- 6: https://community.temporal.io/t/upgrading-schema-on-self-hosted-temporal/17407
- 7: https://community.temporal.io/t/error-sql-schema-version-compatibility-check-failed-unable-to-read-db-schema-version-keyspace-database-temporal-error-pq-relation-schema-version-does-not-exis/7186
- 8: https://community.temporal.io/t/allow-temporal-and-visibility-schemas-in-a-single-database-by-renaming-schema-version-table/17151
- 9: https://github.com/temporalio/temporal/tree/main/tools/sql
- 10: https://temporal.io/blog/auto-setup
🌐 Web query:
Temporal update-schema fails on missing schema_version vs bad credentials connectivity invalid migration
💡 Result:
When Temporal server fails with errors related to "missing schema_version" or "schema version compatibility check failed," it typically indicates that the database has not been correctly initialized or upgraded [1][2][3][4]. These errors are often conflated with connectivity issues (bad credentials), but they are distinct persistence layer problems [4][5]. Common Causes and Troubleshooting Steps: 1. Uninitialized Database: If the error reports that the relation schema_version does not exist, the database has not been initialized [1][4]. Ensure you have run the setup-schema command via temporal-sql-tool before attempting to start the server [5]. 2. Schema Version Mismatch: If the error indicates a version mismatch, the database exists but is at an outdated version [3][5][6]. You must use the update-schema command to bring the schema up to the level required by your Temporal server version [5][6]. 3. Connectivity vs. Authorization: Verify your database credentials (user, password, host, port) independently of Temporal [5][6]. If the server cannot connect, it may report generic errors that mask the underlying failure to read the schema [4]. Ensure the database user has sufficient permissions to create and alter tables [4]. 4. Automation Issues: In environments like Docker, if the database is created automatically (e.g., by a PostgreSQL environment variable) before the Temporal init script runs, the initialization may fail silently or report that the database already exists while the tables remain missing [4]. Ensure the initialization sequence is correctly ordered [4][5]. To resolve, identify the current state by querying the schema version directly: SELECT * FROM schema_version; If this query fails, run the setup-schema command. If it succeeds but shows an older version, run the update-schema command to upgrade [5][6]. Always consult the official Temporal documentation for your specific database plugin (e.g., MySQL, PostgreSQL, Cassandra) and versioning requirements [5][6].
Citations:
- 1: https://community.temporal.io/t/error-sql-schema-version-compatibility-check-failed-unable-to-read-db-schema-version-keyspace-database-temporal-error-pq-relation-schema-version-does-not-exis/7186
- 2: https://community.temporal.io/t/temporal-version-1-22-4-unable-to-start-server-error-sql-schema-version-compatibility-check-failed/13184
- 3: https://community.temporal.io/t/mysql-schema-compatibility-error-when-starting-temporal-service/2117
- 4: Unable to start local server using latest postgres docker-compose file temporalio/temporal#1613
- 5: https://temporalio-temporal.mintlify.app/deployment/database-setup
- 6: https://temporalio-temporal.mintlify.app/operations/persistence
Gate setup-schema on missing schema state, not any update-schema failure
update-schema can fail for reasons other than an uninitialized database. Falling back to setup-schema -v 0.0 on an existing schema can reset schema_version and replay migrations. Detect the missing-schema case explicitly and propagate other errors instead.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@deploy/helm/templates/temporal.yaml` around lines 117 - 125, Update
run_schema so setup-schema -v 0.0 runs only when the database explicitly reports
an uninitialized or missing schema. Capture the update-schema result from the
first invocation, propagate any other failure without setup, and retain the
existing successful update path and subsequent migration behavior.
Source: Coding guidelines
1 participant
Description
This PR fixes an issue in which the Temporal database will is not usable on restore from backup.
The error seen after restore comes from the
schema-setupcontainer:Validation
The kind integration test is manual due to taking ~30 min to complete. When the PR is ready for
review, approve the current commit and start the suite with these PR comments:
As a fallback, run Actions -> Kind Integration -> Run workflow against the copy-pr-bot generated
pull-request/<PR_NUMBER>branch. Use the defaulttest_pathfor the full suite, or narrow itonly while debugging.
The completed workflow updates this PR description with its conclusion and exact run URL.
Passing Kind Integration run, if not automatically reported:
Checklist
CONTRIBUTING.md.docs screenshots, or Helm/rendered outputs.
Summary by CodeRabbit