Skip to content

Subscription: Fix DATE serialization for consensus dataset#18227

Open
Caideyipi wants to merge 2 commits into
apache:masterfrom
Caideyipi:fix/consensus-subscription-date-tablet
Open

Subscription: Fix DATE serialization for consensus dataset#18227
Caideyipi wants to merge 2 commits into
apache:masterfrom
Caideyipi:fix/consensus-subscription-date-tablet

Conversation

@Caideyipi

Copy link
Copy Markdown
Collaborator

Description

Problem

For table-model topics with format=dataset and mode=consensus, DATE columns in WAL insert nodes are represented as int[]. ConsensusLogToTabletConverter reused those arrays in a TsFile Tablet, whose DATE serialization requires LocalDate[]. The resulting array type mismatch prevented subscription events from being generated, so pull consumers repeatedly received no messages.

Reproduction: https://gitlab-eco.timecho.com/test/pipe-client/-/commit/1dcc9ccbbb2e6ee1ab8e8c95231dd05e314d08f4

Fix

  • Convert DATE columns from the WAL int[] representation to the Tablet LocalDate[] representation.
  • Preserve null semantics by skipping bitmap-marked positions during conversion.
  • Apply the same representation boundary to row-based consensus conversions while retaining already-normalized LocalDate values.

Tests

  • Added a regression test using the reported 2026-01-31 DATE value.
  • The test serializes and deserializes the converted Tablet to verify the dataset payload path.
  • ConsensusLogToTabletConverterTest: 13 tests passed.
  • Reactor build for iotdb-core/datanode and required upstream modules passed.

This PR has:

  • been self-reviewed.
  • added unit tests or modified existing tests to cover new code paths, ensuring the threshold for code coverage.

Key changed/added classes (or packages if there are too many classes) in this PR
  • ConsensusLogToTabletConverter
  • ConsensusLogToTabletConverterTest

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant