Skip to content

Reject bucketed legacy epoch TimeFieldSpec values#18978

Open
Vamsi-klu wants to merge 1 commit into
apache:masterfrom
Vamsi-klu:fix-time-granularity-size-validation
Open

Reject bucketed legacy epoch TimeFieldSpec values#18978
Vamsi-klu wants to merge 1 commit into
apache:masterfrom
Vamsi-klu:fix-time-granularity-size-validation

Conversation

@Vamsi-klu

@Vamsi-klu Vamsi-klu commented Jul 11, 2026

Copy link
Copy Markdown
Contributor

Fixes #6240

What changed

This PR rejects legacy TimeFieldSpec definitions that use epoch time with a bucket size greater than one.

The validation is added in SchemaUtils.validateTimeFieldSpec(...), which is the schema-validation boundary used when Pinot validates table schemas. A TimeFieldSpec with TimeFormat.EPOCH remains valid when the timeUnitSize is 1, but values such as 5:MINUTES or any other bucketed epoch granularity are rejected with guidance to use DateTimeFieldSpec for bucketed epoch values.

The change intentionally stays at validation time rather than changing low-level TimeGranularitySpec construction. That keeps object creation and legacy parsing behavior intact while preventing invalid legacy time-field schemas from being accepted.

Why it matters

TimeFieldSpec is the legacy time-column representation. Allowing bucketed epoch granularities there is ambiguous and can lead to schemas that look valid but do not match the intended time semantics. Pinot already has DateTimeFieldSpec for richer date-time formats and bucketed granularities.

Rejecting bucketed epoch TimeFieldSpec values at schema validation makes the failure explicit and points users toward the correct schema construct.

Impact

Schemas using legacy TimeFieldSpec with epoch unit size 1 continue to validate.

Schemas using legacy TimeFieldSpec with bucketed epoch unit sizes are rejected during schema validation.

Users who need bucketed epoch semantics should model those columns as DateTimeFieldSpec.

No ingestion, indexing, or query execution code paths are changed directly.

Testing

  • ./mvnw -Dmaven.repo.local=/tmp/m2-pinot-6240 -pl pinot-segment-local -am -Dtest=SchemaUtilsTest -Dsurefire.failIfNoSpecifiedTests=false test
  • ./mvnw -Dmaven.repo.local=/tmp/m2-pinot-6240 -pl pinot-segment-local spotless:apply checkstyle:check license:check
  • git diff --check

The tests cover rejection for bucketed incoming and outgoing legacy time specs, and verify that the existing epoch unit-size-one case remains valid.


Drafted-by: Codex (GPT-5); human-reviewed by Vamsi-klu before marking ready for review

@Vamsi-klu
Vamsi-klu marked this pull request as ready for review July 11, 2026 23:38
@Vamsi-klu

Copy link
Copy Markdown
Contributor Author

Review request: @npawar @Jackie-Jiang

GitHub did not allow this account/integration to add requested reviewers through the reviewer section, so I am tagging the relevant reviewers here for visibility.

This PR fixes #6240 by rejecting legacy TimeFieldSpec definitions that use epoch time with a bucket size greater than one, while leaving epoch unit-size-one validation untouched and pointing bucketed epoch users toward DateTimeFieldSpec. Review from the time/schema side would be especially useful to confirm the validation boundary and the compatibility impact for legacy time specs.


Drafted-by: Codex (GPT-5); human-reviewed by Vamsi-klu before posting

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Epoch time format does not work with size other than 1

1 participant