From 4f9d7b01f9b395ea9b7eda2c49740aeda11cc85c Mon Sep 17 00:00:00 2001 From: Lev Kokotov Date: Mon, 11 Aug 2025 11:31:55 -0700 Subject: [PATCH 1/2] Dry run mode docs --- docs/.pages | 2 ++ docs/features/sharding/.pages | 1 + docs/features/sharding/dry-run.md | 36 +++++++++++++++++++++++++++++++ docs/migrating-to-prod.md | 26 ++++++++++++++++++++++ 4 files changed, 65 insertions(+) create mode 100644 docs/features/sharding/dry-run.md create mode 100644 docs/migrating-to-prod.md diff --git a/docs/.pages b/docs/.pages index 42c8f50..92ccb99 100644 --- a/docs/.pages +++ b/docs/.pages @@ -3,5 +3,7 @@ nav: - 'installation.md' - 'features' - 'configuration' + - 'administration' + - 'architecture' - '...' - 'about.md' diff --git a/docs/features/sharding/.pages b/docs/features/sharding/.pages index 5c80d35..5cd2537 100644 --- a/docs/features/sharding/.pages +++ b/docs/features/sharding/.pages @@ -6,4 +6,5 @@ nav: - 'cross-shard.md' - 'sharding-functions.md' - 'copy.md' + - 'resharding' - '...' diff --git a/docs/features/sharding/dry-run.md b/docs/features/sharding/dry-run.md new file mode 100644 index 0000000..999d911 --- /dev/null +++ b/docs/features/sharding/dry-run.md @@ -0,0 +1,36 @@ +# Dry run mode + +In dry run mode, PgDog will parse every single query and record the routing decision in the [admin database](../../administration/index.md). If you're experimenting with sharding, this allows you to test the compatibility of your application, without resharding data in production. + +## How it works + +You can enable dry run in the config: + +```toml +[general] +dry_run = true +``` + +PgDog supports hot reload of its configuration files, so you can toggle this setting without restarting the connection pooler. When enabled, PgDog enables its internal query parser (powered by `pg_query`) and attempts to "shard" every single query it receives. + +The sharding decision is recorded in memory, while the query is sent to the production database unchanged. + +### Querying statistics + +The following information is recorded in the admin database when dry run mode is enabled: + +1. The query (with placeholders for values) +2. How many times it was sent to a **single shard** (direct-to-shard) +3. How many times it was sent to **all shards** (cross-shard or multi-shard) + +You can export these statistics by querying the admin database view: + +``` +SHOW QUERY_CACHE; +``` + +### Performance impact + +Since parsing queries isn't free, there is a small performance impact for enabling dry run mode. If your clients are using prepared statements, `pg_query` outputs will be cached and the same statements from this or other clients will not be parsed again. The cache is quite quick, so the impact of sharding at the pooler is minimal. + +If your client is using the simple protocol, i.e., stores parameter values directly in the SQL, PgDog is unable to take advantage of its cache. Each query will have different parameters, and is, effectively, unique. Therefore, the query has to be "normalized" (parameter values replaced with placeholders, e.g., `$1`) before the query router decision can be recorded. This is done automatically and has an additional performance impact. diff --git a/docs/migrating-to-prod.md b/docs/migrating-to-prod.md new file mode 100644 index 0000000..92412f4 --- /dev/null +++ b/docs/migrating-to-prod.md @@ -0,0 +1,26 @@ +# Migrating to PgDog + +PgDog attempts to make the migration from other connection poolers as smooth as possible. That being said, some changes to your infrastructure may be required to benefit from all our features. + +## Configuration + +PgDog uses the **TOML** configuration language for its [configuration](configuration/index.md) files. If you're coming from PgBouncer, you'll need to rewrite your configs. We separate user [authentication](features/authentication.md) (usernames, passwords) from the main settings, so you'll still be able to encrypt passwords in production. + +## Monitoring + +PgDog has two ways to get real time statistics about its internal operations: + +1. PgBouncer-style [admin database](administration/index.md) +2. Prometheus [metrics](features/metrics.md) + +The admin database isn't 100% compatible with PgBouncer. We have additional fields that reflect our added features. That being said, we try to keep naming conventions as similar as possible, so your existing monitoring tools have a high chance of working without modifications. + +## Sharding + +Sharding is a novel feature not available in other connection poolers. We've added a "dry run" mode to allow you to experiment with it in production, without changing how your databases work. + +In dry run mode, PgDog will parse every single query and record the decision of its query router in an admin view. You can see, in real time, how many queries would go to a single shard (vs. cross-shard), by querying that view: + +``` +SHOW QUERY_CACHE: +``` From e40ab16e680d84a1e24b53b1ba4fe353afd56e01 Mon Sep 17 00:00:00 2001 From: Lev Kokotov Date: Mon, 11 Aug 2025 11:33:50 -0700 Subject: [PATCH 2/2] save --- docs/migrating-to-prod.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/migrating-to-prod.md b/docs/migrating-to-prod.md index 92412f4..3d69361 100644 --- a/docs/migrating-to-prod.md +++ b/docs/migrating-to-prod.md @@ -17,7 +17,7 @@ The admin database isn't 100% compatible with PgBouncer. We have additional fiel ## Sharding -Sharding is a novel feature not available in other connection poolers. We've added a "dry run" mode to allow you to experiment with it in production, without changing how your databases work. +Sharding is a novel feature not available in other connection poolers. We've added a ["dry run" mode](features/sharding/dry-run.md) to allow you to experiment with it in production, without changing how your databases work. In dry run mode, PgDog will parse every single query and record the decision of its query router in an admin view. You can see, in real time, how many queries would go to a single shard (vs. cross-shard), by querying that view: