Skip to content

Preserve TIMESTAMP_NTZ type name in ResultSetMetaData (#1495)#1518

Closed
msrathore-db wants to merge 2 commits into
databricks:mainfrom
msrathore-db:wt-1495-ntz
Closed

Preserve TIMESTAMP_NTZ type name in ResultSetMetaData (#1495)#1518
msrathore-db wants to merge 2 commits into
databricks:mainfrom
msrathore-db:wt-1495-ntz

Conversation

@msrathore-db

Copy link
Copy Markdown
Collaborator

Description

Fixes #1495: ResultSetMetaData.getColumnTypeName() returns TIMESTAMP instead of TIMESTAMP_NTZ for TIMESTAMP_NTZ columns (e.g. SELECT MIN(ntz_col) ...). This was a regression from 3.0.7.

Root cause. The driver rewrites the TIMESTAMP_NTZ type text to TIMESTAMP in three metadata-construction paths in DatabricksResultSetMetaData:

Because the ColumnInfoTypeName SDK enum has no TIMESTAMP_NTZ value, getTypeName() is null for NTZ; the code correctly maps it to the TIMESTAMP enum (so getColumnType()Types.TIMESTAMP), but it also clobbered the human-readable typeText that getColumnTypeName() returns. Combined with the effective default protocol moving to SEA, every default connection lost the NTZ type name. In 3.0.7 the default was Thrift, which did not normalize, so it returned TIMESTAMP_NTZ.

Fix. The driver now preserves TIMESTAMP_NTZ across all three paths by default. getColumnType() still reports java.sql.Types.TIMESTAMP, since TIMESTAMP_NTZ is a timestamp without timezone.

The legacy (Simba) driver reports TIMESTAMP for NTZ columns, so the previous behavior is preserved behind a new connection property:

EnableTimestampNtzTypeName getColumnTypeName() for NTZ getColumnType()
1 (default) TIMESTAMP_NTZ Types.TIMESTAMP
0 (legacy / Simba parity) TIMESTAMP Types.TIMESTAMP

Testing

Verified live against a SQL warehouse with SELECT MIN(ntz_col), MAX(ntz_col), ntz_col ... GROUP BY ntz_col:

Path Default (=1) EnableTimestampNtzTypeName=0
SEA (default) TIMESTAMP_NTZ TIMESTAMP
Thrift (usethriftclient=1) TIMESTAMP_NTZ TIMESTAMP

For reference, the legacy Simba 2.8.1 driver returns TIMESTAMP for the same query — which the =0 mode matches.

Unit/integration tests (mvn -pl jdbc-core test):

  • DatabricksResultSetMetaDataTest — updated NTZ assertions for the new default; added testColumnsWithTimestampNTZ_legacyTypeNameDisabled covering =0.
  • PreparedStatementIntegrationTests#testGetMetaData_NoResultSet (the fakeservice test Normalize TIMESTAMP_NTZ to TIMESTAMP in Thrift path for metadata consistency #1182 was tied to), DatabricksConnectionContextTest, DatabricksTypeUtilTest all pass. Spotless clean.

Additional Notes to the Reviewer

This intentionally re-scopes the normalization introduced in #1182 behind the new EnableTimestampNtzTypeName flag rather than removing it — the legacy behavior is still reachable via EnableTimestampNtzTypeName=0. The metadata stays internally consistent (pre-execute vs post-execute) in either mode, which is what testGetMetaData_NoResultSet guards. cc @gopalldb (author of #1182).

This pull request and its description were written by Isaac.

// resolves to Types.TIMESTAMP. By default the "TIMESTAMP_NTZ" typeText is
// preserved so getColumnTypeName() reports the actual server type
// (see GitHub issue #1495); when EnableTimestampNtzTypeName=0 it is
// normalized to "TIMESTAMP" to match the legacy (Simba) driver.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove vendor name, can say (v2.x.x)

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done — removed the vendor name; the comments, the EnableTimestampNtzTypeName description, and the changelog now refer to the legacy driver as "(v2.x.x)". Pushed in 5ef3e20.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

ResultSetMetaData.getColumnTypeName() returned "TIMESTAMP" for
TIMESTAMP_NTZ columns (e.g. SELECT MIN(ntz_col) ...), a regression from
3.0.7. The SEA path normalized the type text to "TIMESTAMP" since launch,
and databricks#1182 added the same normalization to the Thrift path; combined with
the default protocol moving to SEA, every default connection lost the NTZ
type name.

The driver now preserves "TIMESTAMP_NTZ" across the SEA, Thrift, and
describe-query metadata paths by default. getColumnType() continues to
report java.sql.Types.TIMESTAMP, since TIMESTAMP_NTZ is a timestamp
without timezone.

Because the legacy (Simba) driver reports "TIMESTAMP" for NTZ columns,
the previous behavior is preserved behind a new connection property,
EnableTimestampNtzTypeName (default 1). Set EnableTimestampNtzTypeName=0
to restore the legacy/Simba type name.

Co-authored-by: Isaac
Signed-off-by: Madhavendra Rathore <madhavendra.rathore@databricks.com>
Co-authored-by: Isaac
Signed-off-by: Madhavendra Rathore <madhavendra.rathore@databricks.com>
@msrathore-db

Copy link
Copy Markdown
Collaborator Author

Superseded by #1519, which raises the same change from a branch on the upstream repo (databricks/databricks-jdbc) instead of a fork. This avoids the fork-PR Security Scan SBOM failure. Closing in favor of #1519.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] MIN or MAX on a 'TIMESTAMP_NTZ' column returns a TIMESTAMP type instead of TIMESTAMP_NTZ

2 participants