perf(bigquery-jdbc): optimize metadata introspection performance and parallel project scanning

[keshavdandeva]

b/530193584

This PR optimizes metadata introspection performance in the BigQuery JDBC driver by implementing parallel project scanning and reducing BigQuery REST API response payloads through field masking.

Key Changes

1. Parallel Project Scanning:
   • Refactored fetchMatchingDatasets to scan multiple GCP projects concurrently instead of sequential
   • Retained a synchronous single-project fast path to avoid task scheduling overhead.
   • Handled fail-fast SQLException propagation from child threads.
2. Simplified getSchemas Logic:
   • Simplified the schema fetcher logic by removing its redundant multi-project thread spawning, delegating project resolution entirely to fetchMatchingDatasets.
3. Table Field Masking (getTables & getColumns):
   • Applied TableOption.fields(...) masks to REST API listing calls.
   • getTables now only fetches TABLE_REFERENCE, TYPE, and DESCRIPTION (reducing average latency by ~26%).
   • getColumns now only fetches TABLE_REFERENCE, TYPE, and SCHEMA.
4. Routine Field Masking (getProcedureColumns):
   • Parametrized fetchFullRoutineDetailsForIds to support optional RoutineOption... varargs.
   • Configured getProcedureColumns to only request ROUTINE_REFERENCE, ARGUMENTS, and ROUTINE_TYPE from the backend, avoiding full routine downloads.

[gemini-code-assist]

Code Review

This pull request optimizes metadata retrieval in BigQueryDatabaseMetaData by specifying field options to limit the data fetched for routines and tables. It also refactors dataset fetching in getSchemas and fetchMatchingDatasets to parallelize multi-project scans. The review feedback points out a potential resource leak where outstanding futures are not cancelled if one of the parallel tasks fails, and suggests adding a finally block to clean them up.

[keshavdandeva]

/gemini review

[keshavdandeva]

/gemini review

[gemini-code-assist]

Warning

Gemini encountered an error creating the review. You can try again by commenting /gemini review.

[gemini-code-assist]

Code Review

This pull request optimizes metadata retrieval by introducing field masks (using RoutineOption.fields and TableOption.fields) when fetching routines and tables, and refactors getSchemas to parallelize dataset fetching across projects. The feedback suggests moving the task submission loop inside the try-finally block in fetchMatchingDatasets to prevent resource leaks on submission failure, and updating a log message to accurately reflect that a field mask is being used instead of fetching full table details.

[keshavdandeva]

/gemini review

[gemini-code-assist]

Warning

Gemini encountered an error creating the review. You can try again by commenting /gemini review.