Add PostgreSQL 19 (beta1) support by sfc-gh-okalaci · Pull Request #404 · Snowflake-Labs/pg_lake

sfc-gh-okalaci · 2026-06-18T11:03:52Z

This makes pg_lake build and pass tests on PostgreSQL 19beta1, and keeps
16/17/18 working. The majority of the changes are trivial: missing includes,
CI matrix bumps, and small API adjustments behind PG_VERSION_NUM guards. A few
areas needed real thought, noted below.

COPY (JSON): PG19 added a native COPY ... TO (FORMAT json). We don't want to
silently diverge from core, so we give core precedence: for the cases it
handles itself (uncompressed COPY TO a local file or STDOUT) we step aside,
and pg_lake keeps the rest (COPY FROM json, compression, remote/lake
targets). This is the same idea we already use for CSV. The routing sits
behind a hidden, test only GUC pg_lake_copy.json_copy_mode (auto/postgres/
pglake); auto is the production default and the upstream regression suite
forces postgres so core's expected output reproduces exactly. We also map
COPY FORCE_ARRAY to DuckDB's ARRAY writer. On the test side we round-trip
every case (write, then read it back through pg_lake), parametrize the local
cases over both writers, and pin the real divergences (generated columns,
pg_map serialization) so they can't regress unnoticed.
Wire protocol negotiation: PG19 libpq probes whether the server speaks
NegotiateProtocolVersion (it requests protocol 3.9999 and sends a fake
pq.* option). pgduck_server used to drop any connection that was not
exactly 3.0, so every PG19 client failed to attach. It now replies with a
'v' message advertising 3.0 and echoes back the unknown pq.* options, same
as core. Older clients are not affected.
gnu11: PG19 headers use the C11 static_assert keyword (StaticAssertDecl), so
-std=gnu99 stopped compiling. We bump the extension Makefiles to -std=gnu11
unconditionally; it is a superset and still builds on 16/17/18.
TupleDescFinalize: PG19 added a cache field to TupleDesc and asserts it is
populated before a descriptor is blessed. Every place we build a TupleDesc
by hand now calls TupleDescFinalize(), with a no-op shim in pg_compat.h for
<=18 so the call can stay unconditional.
Shared memory init: PG19 replaced the shmem_request_hook/shmem_startup_hook
pair with RegisterShmemCallbacks plus ShmemRequestStruct/ShmemRequestHash.
We use the new path on PG19 and keep the old hooks for <=18, gated by
version.
COPY TO partitioned tables: PG19 lets you COPY a partitioned table directly.
pg_lake rejected this before. We allow it on PG19+ so a partitioned parent
(whose leaves can be pg_lake/iceberg tables) streams out in one statement,
and we set the SELECT inh flag from relkind so the parent descends into its
partitions.
pgaudit: pgaudit has no PG19 SQL/PGQ (graph_table) support yet, and with it
preloaded the backend crashes on the property graph statements in core's own
privileges regression test, cascading into ~100 failures. We drop pgaudit
from the preload list for installcheck-postgres on PG19 only (it stays on for
<=18). It only writes to the server log, so regression output does not
change.
pg_cron: upstream pg_cron has no PG19 release, so for CI we build it from
CyberDem0n's postgresql-19 branch (citusdata/pg_cron PR pg_lake_copy: reject multidimensional arrays in COPY TO before CSV serialization #430) until that
lands.
pull_var_clause recursion: on PG19 aggregate/placeholder nodes can appear
inside baserestrictinfo clauses where earlier majors did not, and
pull_var_clause elog()s on them without a recurse flag, so we pass
PVC_RECURSE_AGGREGATES at those sites. postgres_fdw never needed this because
it does not push down the complex query shapes we do, so it does not reach
those clauses the same way.
GROUP BY ALL: PG19's column-inference GROUP BY ALL also exists in DuckDB, but
the two infer the grouping columns differently and the deparser keeps the
literal "GROUP BY ALL", so pushing it down could let DuckDB pick a different
grouping. We mark such queries as not shippable and run them locally with the
grouping PostgreSQL already resolved. The older "GROUP BY ALL col" modifier
is not affected.
IGNORE NULLS window functions: PG19 added IGNORE NULLS / RESPECT NULLS, and
core deparses it as "lag(v) IGNORE NULLS OVER ...", which DuckDB rejects. We
mark window funcs with ignore_nulls != NO_NULLTREATMENT as not shippable.
RESPECT NULLS is the default and collapses to NO_NULLTREATMENT, so it still
pushes down.
Isolation tests: PG19 reports a serialization conflict from a concurrent
delete with different wording (and our copy-on-write makes an UPDATE look
like a delete to a concurrent reader). Instead of keeping _1.out alternate
expected files, we trap the failure by SQLSTATE (40001) and re-raise one
canonical message, so a single expected file matches every version.

sfc-gh-okalaci · 2026-06-18T15:02:13Z


 # Compile and install pg_cron
-RUN git clone https://github.com/citusdata/pg_cron.git && \
+# Use CyberDem0n's postgresql-19 branch (PR citusdata/pg_cron#430) until the


@sfc-gh-mslot do you have plans to merge that branch :)

Heh, I'd thought the same... 😁

Looks like this is in main now...

i.e., we can revert this hunk.

sfc-gh-dachristensen · 2026-06-18T14:25:15Z

 ARG PGAUDIT16_VERSION=REL_16_STABLE
 ARG PGAUDIT17_VERSION=REL_17_STABLE
 ARG PGAUDIT18_VERSION=REL_18_STABLE
+ARG PGAUDIT19_VERSION=main


FWIW, there is a pgaudit 19 PR that should probably be merged soon.

I tried this: built pgaudit from the maintainer's PG19 PR branch (dwsteele/pgaudit#317, dev-pg19) and re-enabled it on PG19. It compiles fine, but CI showed the backend still segfaults while auditing PG19's SQL/PGQ graph_table statements in PostgreSQL's own privileges regression test, cascading into 109/245 upstream test failures. So the branch isn't there yet.

I reverted to keeping pgaudit out of the PG19 preload (now with the literal "19" check you asked for) and left a comment pointing at the PR so we can flip it back once upstream actually supports PG19.

pgaudit/pgaudit#317 (comment)

This has been fixed now, and there is aREL_19_STABLE branch we can use.

sfc-gh-dachristensen · 2026-06-18T14:29:55Z

+ * the function, so define a no-op shim there and let callers always finalize.
+ */
+#if PG_VERSION_NUM < 190000
+static inline void


why not just an empty #define?

I think we want TupleDescFinalize to be in the C code? So instead of having ifdefs in the code, we have it like this? I don't have any strong opinions, happy to follow if you think strongly about it

It would still appear in the code, just wouldn't be an empty inline function. (I assume it's probably totally possible that the compiler just noops anyway.)

My proposal was just doing something like this instead of static inline void, just feels a little more canonical for this sort of thing:

#if PG_VERSION_NUM < 190000 #define TupleDescFinalize(tupdesc) #endif

with callers unchanged.

This makes pg_lake build and pass tests on PostgreSQL 19beta1, and keeps 16/17/18 working. The majority of the changes are trivial: missing includes, CI matrix bumps, and small API adjustments behind PG_VERSION_NUM guards. A few areas needed real thought, noted below. - COPY (JSON): PG19 added a native COPY ... TO (FORMAT json). We don't want to silently diverge from core, so we give core precedence: for the cases it handles itself (uncompressed COPY TO a local file or STDOUT) we step aside, and pg_lake keeps the rest (COPY FROM json, compression, remote/lake targets). This is the same idea we already use for CSV. The routing sits behind a hidden, test only GUC pg_lake_copy.json_copy_mode (auto/postgres/ pglake); auto is the production default and the upstream regression suite forces postgres so core's expected output reproduces exactly. We also map COPY FORCE_ARRAY to DuckDB's ARRAY writer. On the test side we round-trip every case (write, then read it back through pg_lake), parametrize the local cases over both writers, and pin the real divergences (generated columns, pg_map serialization) so they can't regress unnoticed. - Wire protocol negotiation: PG19 libpq probes whether the server speaks NegotiateProtocolVersion (it requests protocol 3.9999 and sends a fake _pq_.* option). pgduck_server used to drop any connection that was not exactly 3.0, so every PG19 client failed to attach. It now replies with a 'v' message advertising 3.0 and echoes back the unknown _pq_.* options, same as core. Older clients are not affected. - gnu11: PG19 headers use the C11 static_assert keyword (StaticAssertDecl), so -std=gnu99 stopped compiling. We bump the extension Makefiles to -std=gnu11 unconditionally; it is a superset and still builds on 16/17/18. - TupleDescFinalize: PG19 added a cache field to TupleDesc and asserts it is populated before a descriptor is blessed. Every place we build a TupleDesc by hand now calls TupleDescFinalize(), with a no-op shim in pg_compat.h for <=18 so the call can stay unconditional. - Shared memory init: PG19 replaced the shmem_request_hook/shmem_startup_hook pair with RegisterShmemCallbacks plus ShmemRequestStruct/ShmemRequestHash. We use the new path on PG19 and keep the old hooks for <=18, gated by version. - COPY TO partitioned tables: PG19 lets you COPY a partitioned table directly. pg_lake rejected this before. We allow it on PG19+ so a partitioned parent (whose leaves can be pg_lake/iceberg tables) streams out in one statement, and we set the SELECT inh flag from relkind so the parent descends into its partitions. - pgaudit: pgaudit has no PG19 SQL/PGQ (graph_table) support yet, and with it preloaded the backend crashes on the property graph statements in core's own privileges regression test, cascading into ~100 failures. We drop pgaudit from the preload list for installcheck-postgres on PG19 only (it stays on for <=18). It only writes to the server log, so regression output does not change. - pg_cron: upstream pg_cron has no PG19 release, so for CI we build it from CyberDem0n's postgresql-19 branch (citusdata/pg_cron PR #430) until that lands. - pull_var_clause recursion: on PG19 aggregate/placeholder nodes can appear inside baserestrictinfo clauses where earlier majors did not, and pull_var_clause elog()s on them without a recurse flag, so we pass PVC_RECURSE_AGGREGATES at those sites. postgres_fdw never needed this because it does not push down the complex query shapes we do, so it does not reach those clauses the same way. - GROUP BY ALL: PG19's column-inference GROUP BY ALL also exists in DuckDB, but the two infer the grouping columns differently and the deparser keeps the literal "GROUP BY ALL", so pushing it down could let DuckDB pick a different grouping. We mark such queries as not shippable and run them locally with the grouping PostgreSQL already resolved. The older "GROUP BY ALL col" modifier is not affected. - IGNORE NULLS window functions: PG19 added IGNORE NULLS / RESPECT NULLS, and core deparses it as "lag(v) IGNORE NULLS OVER ...", which DuckDB rejects. We mark window funcs with ignore_nulls != NO_NULLTREATMENT as not shippable. RESPECT NULLS is the default and collapses to NO_NULLTREATMENT, so it still pushes down. - Isolation tests: PG19 reports a serialization conflict from a concurrent delete with different wording (and our copy-on-write makes an UPDATE look like a delete to a concurrent reader). Instead of keeping _1.out alternate expected files, we trap the failure by SQLSTATE (40001) and re-raise one canonical message, so a single expected file matches every version. Co-authored-by: Cursor <cursoragent@cursor.com>

The pgaudit-on-PG19 workaround is temporary, so check for the literal major "19" instead of ">= 19". Co-authored-by: Cursor <cursoragent@cursor.com>

pgaudit has no PG19 release yet, so build it from the maintainer's PG19 support branch (dwsteele/pgaudit#317, dev-pg19) instead of pinning main. That lets pgaudit stay preloaded on PG19 like every other version, so we can drop the PG19-only preload/enable workaround in the test action. We can switch to REL_19_STABLE once it is tagged. Co-authored-by: Cursor <cursoragent@cursor.com>

Combine the four per-version pgaudit build steps into a single RUN and remove the extracted source dirs afterwards to cut image layers and size. Co-authored-by: Cursor <cursoragent@cursor.com>

Revert an accidental reordering of the tranche id / name assignments in BaseWorkerSharedMemoryInit so it matches main; the order is immaterial but the swap was unintentional. Co-authored-by: Cursor <cursoragent@cursor.com>

PG19 added PG_SIG_IGN (SIG_IGN cast to pqsigfunc) for pqsignal(). Define the name as plain SIG_IGN on older releases in pg_compat.h so the SIGINT handler setup can use PG_SIG_IGN unconditionally instead of a per-call Co-authored-by: Cursor <cursoragent@cursor.com> #if.

Spread the pg_plan_query() arguments onto their own lines so the PG19-only trailing NULL argument reads cleanly instead of being appended inline. Co-authored-by: Cursor <cursoragent@cursor.com>

Inline the 190000 server-version literal in the PG19 skip helpers instead of defining a one-off PG19 module constant, matching how the rest of the suite spells version checks. Co-authored-by: Cursor <cursoragent@cursor.com>

These COPY-to-partitioned-table tests will outlive PG19, and the version guard already skips them where unsupported, so rename the functions and helper to describe the behavior instead of the release. Co-authored-by: Cursor <cursoragent@cursor.com>

Compare the full, deterministically-ordered table contents before and after REPACK instead of only the row count, so a silent rewrite that preserved cardinality but corrupted rows would be caught. Co-authored-by: Cursor <cursoragent@cursor.com>

Wire protocol 3.2 arrived in PG18 and the GREASE/_pq_ probing that forces pgduck_server to answer NegotiateProtocolVersion is new in PG19; the 'v' message itself predates both. Reword so the versions are attributed correctly. Co-authored-by: Cursor <cursoragent@cursor.com>

The dev-pg19 pgaudit branch builds, but CI showed it still segfaults the backend while auditing PG19's SQL/PGQ graph_table statements in PostgreSQL's own privileges regression test (109/245 upstream tests cascade-failed). Restore the PG19-only pgaudit drop (using a literal "19" check) and build plain pgaudit main in the image until upstream actually supports PG19. Co-authored-by: Cursor <cursoragent@cursor.com>

main #406 split rest_catalog.c into sibling files. The PG19 implicit- include hardening needs access/htup_details.h (GETSTRUCT) in the DDL sibling and pg_extension_base/pg_compat.h (numeric_int4_opt_error shim) in the HTTP sibling, which the old monolithic file pulled in transitively. Co-authored-by: Cursor <cursoragent@cursor.com>

Replace the inline "#if ... ,NULL" spliced argument with two complete pg_plan_query() calls, one per major, per review feedback. Co-authored-by: Cursor <cursoragent@cursor.com>

Give the helper a constant-false definition for pre-19 so the JSON routing in IsPgLakeCopy no longer needs its own version guard; the only PG19 #if now lives at the declaration/definition. Co-authored-by: Cursor <cursoragent@cursor.com>

The include just covers PG19's stricter implicit-include rules for WAIT_EVENT_CLIENT_READ; utils/wait_event.h has existed since PG14, so drop the version guard. Co-authored-by: Cursor <cursoragent@cursor.com>

The header only needs the HTAB typedef for its signatures, not the dynahash internals. Include utils/hsearch.h unconditionally and let callers pull in utils/dynahash.h where they actually use it. Co-authored-by: Cursor <cursoragent@cursor.com>

palloc0 already zeroes the mapping, so the no-matching-column case needs no body. Fold the explanation into the guard and remove the empty else. Co-authored-by: Cursor <cursoragent@cursor.com>

The maintainer fixed the SQL/PGQ graph_table audit crash on the dev-pg19 branch (dwsteele/pgaudit#317, commit ffbae89), so we no longer need to keep pgaudit out of the PG19 preload. Build pgaudit for PG19 from that branch and preload it on PG19 like every other version. Verified locally on PG19beta1: a GRAPH_TABLE query over a property graph is now audited instead of aborting the backend. Co-authored-by: Cursor <cursoragent@cursor.com>

pgguru · 2026-07-02T21:25:27Z

+ARG PGAUDIT19_REPO=dwsteele/pgaudit
+ARG PGAUDIT19_VERSION=dev-pg19


Suggested change

ARG PGAUDIT19_REPO=dwsteele/pgaudit

ARG PGAUDIT19_VERSION=dev-pg19

ARG PGAUDIT19_VERSION=REL_19_STABLE

pgguru · 2026-07-02T21:25:53Z

    wget https://github.com/pgaudit/pgaudit/archive/refs/heads/$PGAUDIT17_VERSION.tar.gz && \
    wget https://github.com/pgaudit/pgaudit/archive/refs/heads/$PGAUDIT18_VERSION.tar.gz && \
-    wget https://github.com/pgaudit/pgaudit/archive/refs/heads/$PGAUDIT19_VERSION.tar.gz && \
+    wget https://github.com/$PGAUDIT19_REPO/archive/refs/heads/$PGAUDIT19_VERSION.tar.gz && \


Suggested change

wget https://github.com/$PGAUDIT19_REPO/archive/refs/heads/$PGAUDIT19_VERSION.tar.gz && \

wget https://github.com/pgaudit/pgaudit/archive/refs/heads/$PGAUDIT19_VERSION.tar.gz && \

sfc-gh-okalaci requested review from sfc-gh-dachristensen and sfc-gh-mslot June 18, 2026 11:06

sfc-gh-dachristensen mentioned this pull request Jun 18, 2026

ci: add PostgreSQL 19beta1 to build/test matrix, bump minor versions #363

Closed

10 tasks

sfc-gh-okalaci mentioned this pull request Jun 18, 2026

Request for postgres 19 support. #399

Open

sfc-gh-okalaci commented Jun 18, 2026

View reviewed changes

sfc-gh-dachristensen reviewed Jun 18, 2026

View reviewed changes

sfc-gh-okalaci requested a review from sfc-gh-dachristensen June 22, 2026 08:37

sfc-gh-okalaci and others added 18 commits June 22, 2026 11:52

ci: match PG19 literally for the temporary pgaudit drop

486e72f

The pgaudit-on-PG19 workaround is temporary, so check for the literal major "19" instead of ">= 19". Co-authored-by: Cursor <cursoragent@cursor.com>

docker: build all pgaudit versions in one layer and drop the sources

c523bf0

Combine the four per-version pgaudit build steps into a single RUN and remove the extracted source dirs afterwards to cut image layers and size. Co-authored-by: Cursor <cursoragent@cursor.com>

pg_extension_base: restore tranche init order

56093e9

Revert an accidental reordering of the tranche id / name assignments in BaseWorkerSharedMemoryInit so it matches main; the order is immaterial but the swap was unintentional. Co-authored-by: Cursor <cursoragent@cursor.com>

pg_lake_engine: format pg_plan_query call one arg per line

ef07252

Spread the pg_plan_query() arguments onto their own lines so the PG19-only trailing NULL argument reads cleanly instead of being appended inline. Co-authored-by: Cursor <cursoragent@cursor.com>

tests: drop the PG19 version constant

7f47dce

Inline the 190000 server-version literal in the PG19 skip helpers instead of defining a one-off PG19 module constant, matching how the rest of the suite spells version checks. Co-authored-by: Cursor <cursoragent@cursor.com>

pg_lake_engine: use version-specific pg_plan_query calls

bf7e3e6

Replace the inline "#if ... ,NULL" spliced argument with two complete pg_plan_query() calls, one per major, per review feedback. Co-authored-by: Cursor <cursoragent@cursor.com>

pg_extension_base: include utils/wait_event.h unconditionally

88c5154

The include just covers PG19's stricter implicit-include rules for WAIT_EVENT_CLIENT_READ; utils/wait_event.h has existed since PG14, so drop the version guard. Co-authored-by: Cursor <cursoragent@cursor.com>

pg_lake_table: drop empty else branch in iceberg column mapping

ab0a534

palloc0 already zeroes the mapping, so the no-matching-column case needs no body. Fold the explanation into the guard and remove the empty else. Co-authored-by: Cursor <cursoragent@cursor.com>

sfc-gh-okalaci force-pushed the okalaci/pg19-support branch from 10002a2 to ab0a534 Compare June 22, 2026 09:00

raphaelauv mentioned this pull request Jul 1, 2026

pg_lake pglayers/pglayers#8

Closed

10 tasks

pgguru reviewed Jul 2, 2026

View reviewed changes

		ARG PGAUDIT19_REPO=dwsteele/pgaudit
		ARG PGAUDIT19_VERSION=dev-pg19

	ARG PGAUDIT19_REPO=dwsteele/pgaudit
	ARG PGAUDIT19_VERSION=dev-pg19
	ARG PGAUDIT19_VERSION=REL_19_STABLE

	wget https://github.com/$PGAUDIT19_REPO/archive/refs/heads/$PGAUDIT19_VERSION.tar.gz && \
	wget https://github.com/pgaudit/pgaudit/archive/refs/heads/$PGAUDIT19_VERSION.tar.gz && \

Uh oh!

Conversation

sfc-gh-okalaci commented Jun 18, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants