Skip to content

Preserve partitioning on temp FILE_LOADS tables#38833

Draft
PRADDZY wants to merge 1 commit into
apache:masterfrom
PRADDZY:fix/beam-38017-partitioned-file-loads
Draft

Preserve partitioning on temp FILE_LOADS tables#38833
PRADDZY wants to merge 1 commit into
apache:masterfrom
PRADDZY:fix/beam-38017-partitioned-file-loads

Conversation

@PRADDZY
Copy link
Copy Markdown

@PRADDZY PRADDZY commented Jun 5, 2026

Summary

  • preserve destination partitioning metadata when FILE_LOADS writes through temporary tables
  • reuse the destination table lookup for missing schema resolution and only inject partitioning when the caller did not already supply it
  • add regression coverage for inherited time/range partitioning and explicit partitioning precedence

Fixes #38017.

Testing

  • python -m pytest apache_beam/io/gcp/bigquery_file_loads_test.py -k "temporary_table_load_inherits_destination_time_partitioning or temporary_table_load_inherits_destination_range_partitioning or temporary_table_load_keeps_explicit_partitioning_parameters" -q
  • python -m pytest apache_beam/io/gcp/bigquery_file_loads_test.py::TestBigQueryFileLoads::test_wait_for_load_job_completion -q
  • python -m py_compile apache_beam/io/gcp/bigquery_file_loads.py apache_beam/io/gcp/bigquery_file_loads_test.py

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: 'Failed to copy Non partitioned table to Column partitioned table: not supported.'

1 participant