Make connection pool options GLOBAL#445
Merged
staticlibs merged 1 commit intoduckdb:mainfrom Apr 20, 2026
Merged
Conversation
This PR changes the default scope from `SESSION` to `GLOBAL` for the following options (see duckdb#430 and duckdb#444 for details): - `pg_pool_acquire_mode` - `pg_pool_max_connections` - `pg_pool_wait_timeout_millis` - `pg_pool_enable_thread_local_cache` - `pg_pool_max_lifetime_millis` - `pg_pool_idle_timeout_millis` - `pg_pool_enable_reaper_thread` - `pg_pool_health_check_query` Additionally it changes the default size of the connection pool to `max(cpu_count * 1.5, 8)`. Fixes: duckdb/ducklake#1031
2 tasks
5 tasks
fuziontech
added a commit
to PostHog/duckgres
that referenced
this pull request
Apr 23, 2026
…#448) Wire DuckLake's Postgres metadata connection through the per-Duckling PgBouncer pooler provisioned by the Crossplane composition in PostHog/charts#10400. Two coupled changes that only take effect when a Duckling opts in via spec.metadataStore.pgbouncer.enabled. 1. Control plane prefers status.metadataStore.pgbouncerEndpoint When the Duckling CR status exposes pgbouncerEndpoint (format "<host>:<port>", e.g. pgbouncer-duckling-foo.ducklings.svc:6543), the activator parses it with net.SplitHostPort and builds the worker DSN against the pooler instead of the Aurora endpoint. When absent, behaviour is unchanged — direct to the Postgres endpoint on 5432. - DucklingStatus.MetadataStore gains PgBouncerEndpoint (controlplane/provisioner/k8s_client.go); parseDucklingStatus reads the new status field. - buildDuckLakeConfigFromDuckling (controlplane/shared_worker_activator.go) parses the endpoint and sets DuckLakeConfig.ViaPgBouncer. 2. Worker disables the in-process pool when behind PgBouncer DuckLakeConfig gains ViaPgBouncer. When true, buildDuckLakePreAttachStatements emits `SET GLOBAL pg_pool_max_connections = 0` before ATTACH so the setting propagates into the __ducklake_metadata_* catalog (per duckdb/duckdb-postgres#445). Rationale, from duckdb/ducklake#1031: behind a network pooler, client-side pooling is redundant and counter-productive. The postgres_scanner thread-local cache pins one server conn per DuckDB worker thread and holds it indefinitely, starving the pool and preventing PgBouncer from reclaiming idle connections. Audited DuckLake and postgres_scanner before wiring this up: no SQL PREPARE, no named extended-protocol prepared statements (unnamed only — no max_prepared_statements needed), no advisory locks, no LISTEN/NOTIFY, no WITH HOLD cursors. 1:1 DuckDB↔Postgres transaction mapping. Safe in both session and transaction pool modes. Test coverage: - TestParseDucklingStatusPgBouncerEndpoint — Duckling CR status round trips the new field. - TestBuildDuckLakePreAttachStatements — two new cases cover the ViaPgBouncer=true emission, alone and combined with the existing thread-local-cache disable. Inline struct literals in org_activation_test.go updated to match the new DucklingStatus.MetadataStore shape (additive only). Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR changes the default scope from
SESSIONtoGLOBALfor the following options (see #430 and #444 for details):pg_pool_acquire_modepg_pool_max_connectionspg_pool_wait_timeout_millispg_pool_enable_thread_local_cachepg_pool_max_lifetime_millispg_pool_idle_timeout_millispg_pool_enable_reaper_threadpg_pool_health_check_queryAdditionally it changes the default size of the connection pool to
max(cpu_count * 1.5, 8).Fixes: duckdb/ducklake#1031