Skip to content

Disallow setting retention on non-partitioned columns#588

Open
jiang95-dev wants to merge 1 commit into
linkedin:mainfrom
jiang95-dev:lejiang/validate-retention-partition-col
Open

Disallow setting retention on non-partitioned columns#588
jiang95-dev wants to merge 1 commit into
linkedin:mainfrom
jiang95-dev:lejiang/validate-retention-partition-col

Conversation

@jiang95-dev
Copy link
Copy Markdown
Collaborator

Summary

Follow up PR on #552. Disallow setting retention on non-partitioned columns because it is not a metadata-only delete.

Changes

  • Client-facing API Changes
  • Internal API Changes
  • Bug Fixes
  • New Features
  • Performance Improvements
  • Code Style
  • Refactoring
  • Documentation
  • Tests

For all the boxes checked, please include additional details of the changes made in this pull request.

Testing Done

  • Manually Tested on local docker setup. Please include commands ran, and their output.
  • Added new tests for the changes made.
  • Updated existing tests to reflect the changes made.
  • No tests added or updated. Please explain why. If unsure, please feel free to ask for help.
  • Some other form of testing like staging or soak time in production. Please explain.

For all the boxes checked, include a detailed description of the testing done for the changes made in this pull request.

Additional Information

  • Breaking Changes
  • Deprecations
  • Large PR broken into smaller PRs, and PR plan linked in the description.

For all the boxes checked, include additional details of the changes made in this pull request.

@jiang95-dev jiang95-dev requested a review from cbb330 May 16, 2026 00:07
Copy link
Copy Markdown
Collaborator

@cbb330 cbb330 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it looks like this change will invoke per INSERT to table, so could suddenly cause failed writer pipelines.

can we:

  • (ideal) make backup feature succeed even for tables with this partition scheme
  • (suitable) prevent alter table only in order to stop the bleed?

// Retention DELETE must be satisfiable by partition pruning. When the table is not
// time-partitioned, the retention column must be one of the clustering columns;
// otherwise the retention job would degrade into a row-level rewrite (or fail outright
// when backup is enabled).
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please add why it fails backup and the impact of a failing backup

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants