fix(local-dev): don't abort up on an already-applied sql/updates changeSet - #7069
Closed
Yicong-Huang wants to merge 1 commit into
Closed
fix(local-dev): don't abort up on an already-applied sql/updates changeSet#7069Yicong-Huang wants to merge 1 commit into
up on an already-applied sql/updates changeSet#7069Yicong-Huang wants to merge 1 commit into
Conversation
…angeSet On a fresh docker volume postgres applies sql/texera_ddl.sql itself — compose mounts sql/ into /docker-entrypoint-initdb.d — and that DDL is kept in sync with sql/updates/*, so the changeSets local-dev replays immediately afterwards re-create objects that are already there. 23-27 are incidentally idempotent and pass; 28.sql's dataset_owner_uid_name_key is not, so `up` died before the sbt build ever started. The `seed` path written for exactly this case was unreachable, because the container entrypoint always wins the race. Treat a psql failure whose every ERROR line is "already exists" as already-applied: record the changeSet and carry on. The check is deliberately narrow — a duplicate-key conflict is a data problem, and a failure with no ERROR line at all is never assumed harmless — so an incomplete schema still stops the build instead of reaching jOOQ codegen. psql's stderr is also kept rather than sent to /dev/null, so a real failure no longer requires re-running the file by hand to find out why. Closes #7064
Contributor
Backport auto-label reportThis
|
Contributor
Automated Reviewer SuggestionsBased on the
|
Contributor
Author
|
Superseded by #7076 — same commit, but opened from a fork branch as it should have been. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this PR?
bin/local-dev.sh upagainst a fresh docker volume never reached the sbtbuild: it died on the last
sql/updateschangeSet.Postgres applies
sql/texera_ddl.sqlitself — compose mountssql/into/docker-entrypoint-initdb.d— and that DDL is kept in sync withsql/updates/*, so the changeSets local-dev replays immediately afterwardsre-create objects that are already there. 23–27 are incidentally idempotent
and pass;
28.sql'sdataset_owner_uid_name_keyis not, andtexera_ddl.sql:291(UNIQUE (owner_uid, name)) already created it underexactly that auto-generated name.
infra_ensure_db_schemapicks seed-vs-replay by probing for thefeedbacktable. On a fresh volume the entrypoint has just created it, so the replay
path is taken — and the
seedbranch written for this very case (record everychangeSet as applied without executing it) is unreachable, because the
entrypoint always wins the race.
The fix is in the replay loop rather than the probe: a psql failure whose
every
ERROR:line isalready existsmeans the changeSet's effect isalready in the schema, so record it and carry on. That also covers the next
sql/updates/N.sqlthat isn't accidentally idempotent, instead of fixingjust
28.sql._sql_errors_all_already_existis deliberately narrow — aduplicate keyisa data conflict, not an applied schema change, and a failure with no
ERROR:line at all is never assumed harmless — so an incomplete schema stillstops the build instead of reaching jOOQ codegen with tables that aren't
there.
While in the same lines: psql's stderr is kept instead of redirected to
/dev/null. It holds the one line that explains the abort, and the old codediscarded it and then told the operator to re-run the file by hand to find
out why.
Any related issues, documentation, discussions?
Closes #7064
How was this PR tested?
New unit coverage for the detector, both directions (13 cases), plus two
structural guards on the wiring, in the existing
infra-job suite:That one failure is
test_is_dirty_after_seed_then_edit, and it ispre-existing and unrelated — it reproduces identically on a pristine
upstream/maincheckout on this machine, and this PR touches neithertui.pynor that test. Cause, for the record:_newest_mtime_aftercomparessource mtime to the stamp with a strict
>, and on a filesystem whosetimestamp granularity is coarser than the two consecutive writes the test
performs, both land on the same
st_mtimeand the fast filter reports clean.Filed separately rather than folded in here.
End-to-end on the real stack (Ubuntu 24.04, docker 29.1.3), reproducing the
bug and then confirming the fix:
Before:
After:
The changeSet is recorded, so it is not retried on the next run:
Was this PR authored or co-authored using generative AI tooling?
Generated-by: Claude Code (Opus 5)