Commit 49cbbbd
refactor(bigquery-jdbc): migrate metadata thread management to connection-scoped executor (#13556)
b/520400589
This PR optimizes thread management for JDBC metadata methods,
implementing a deadlock-free, bipartite thread pool design and unifying
resource lifecycle management.
### Key Changes
* **Bipartite Thread Pool Architecture (Deadlock Prevention)**:
* Redirected all outer, orchestrating fetcher tasks (parent tasks) to
the connection’s cached query executor
(`connection.getExecutorService()`). Because the cached pool scales
dynamically as needed, this structurally prevents pool-induced deadlocks
(thread starvation).
* Kept all inner, highly concurrent parallel network API calls (child
tasks) on the connection's fixed-size metadata executor
(`connection.getMetadataExecutor()`). This bounds parallel network
traffic to prevent overwhelming the BigQuery backend.
* *Result*: The driver is fully protected against pool-induced
deadlocks, even under high concurrency or if `metadataFetchThreadCount`
is configured to `1`.
* **Connection-Scoped Shared Executor**:
* Migrated all asynchronous metadata operations (`getSchemas`,
`getTables`, `getColumns`, `getProcedures`, `getProcedureColumns`,
`getFunctions`, and `getFunctionColumns`) from spawning raw threads or
short-lived local pools to connection-managed executors.
* Configured the shared metadata pool to be lazily instantiated on
demand and gracefully shut down with the connection lifecycle. Idle
threads are automatically reclaimed after 60 seconds.
* **Lower CPU Overhead (Sequential Row-Building)**:
* Completely removed secondary CPU-bound thread pools (such as
`routineProcessorExecutor`, `processArgsExecutor`,
`tableProcessorExecutor`, and `processParamsExecutor`) previously used
for row-building.
* Refactored row-building and parameter-processing tasks to execute
sequentially on the background fetcher threads, removing
context-switching overhead and preventing nested-pool deadlock risks.
* **Teardown of Legacy Bridges**:
* Deleted the obsolete `wrapThread(Thread)` compatibility adapter in
`BigQueryDatabaseMetaData.java`.
* Removed obsolete `testWrapThread_*` cases from
`BigQueryDatabaseMetaDataTest.java`. Metadata queries now track and
cancel background tasks natively via standard `Future<?>` handles.
* **Test Suite Refactoring**:
* Updated `BigQueryDatabaseMetaDataTest` setup to inject real,
connection-scoped cached executor services, ensuring that background
metadata tasks execute deterministically during testing.
* Added `@AfterEach` teardown logic to prevent test-suite thread leaks.
* Refactored argument-processing tests to verify row-building
synchronously without mock executors.1 parent 6ca6b2f commit 49cbbbd
4 files changed
Lines changed: 101 additions & 491 deletions
File tree
- java-bigquery-jdbc/src
- main/java/com/google/cloud/bigquery/jdbc
- test/java/com/google/cloud/bigquery/jdbc
Lines changed: 3 additions & 5 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
362 | 362 | | |
363 | 363 | | |
364 | 364 | | |
365 | | - | |
366 | | - | |
367 | | - | |
368 | | - | |
369 | | - | |
370 | 365 | | |
371 | 366 | | |
372 | 367 | | |
373 | 368 | | |
| 369 | + | |
| 370 | + | |
| 371 | + | |
374 | 372 | | |
375 | 373 | | |
376 | 374 | | |
| |||
0 commit comments