Preserve TIMESTAMP_NTZ type name in ResultSetMetaData (#1495)#1519
Preserve TIMESTAMP_NTZ type name in ResultSetMetaData (#1495)#1519msrathore-db wants to merge 2 commits into
Conversation
ResultSetMetaData.getColumnTypeName() returned "TIMESTAMP" for TIMESTAMP_NTZ columns (e.g. SELECT MIN(ntz_col) ...), a regression from 3.0.7. The SEA path normalized the type text to "TIMESTAMP" since launch, and #1182 added the same normalization to the Thrift path; combined with the default protocol moving to SEA, every default connection lost the NTZ type name. The driver now preserves "TIMESTAMP_NTZ" across the SEA, Thrift, and describe-query metadata paths by default. getColumnType() continues to report java.sql.Types.TIMESTAMP, since TIMESTAMP_NTZ is a timestamp without timezone. Because the legacy (Simba) driver reports "TIMESTAMP" for NTZ columns, the previous behavior is preserved behind a new connection property, EnableTimestampNtzTypeName (default 1). Set EnableTimestampNtzTypeName=0 to restore the legacy/Simba type name. Co-authored-by: Isaac Signed-off-by: Madhavendra Rathore <madhavendra.rathore@databricks.com>
Co-authored-by: Isaac Signed-off-by: Madhavendra Rathore <madhavendra.rathore@databricks.com>
|
📄 src/main/java/com/databricks/jdbc/api/impl/DatabricksResultSet.java 💡 [MINOR] getObject(int, Map) misses type maps keyed on "TIMESTAMP" for NTZ columns Type-map lookup regression for NTZ columns in getObject(int, Map). DatabricksResultSet.java:1514 does map.get(columnTypeText) where columnTypeText = getColumnTypeName(columnIndex). With EnableTimestampNtzTypeName=1 (the new default) this now returns "TIMESTAMP_NTZ" instead of "TIMESTAMP" for NTZ columns. A consumer whose custom type map is keyed on "TIMESTAMP" and previously matched NTZ columns will now miss the lookup, causing getObject(int, Map) to log a warning and return null (lines 1526-1529) instead of applying the mapped conversion. This is a direct behavior consequence of preserving the NTZ type name. |
Description
Fixes #1495:
ResultSetMetaData.getColumnTypeName()returnsTIMESTAMPinstead ofTIMESTAMP_NTZforTIMESTAMP_NTZcolumns (e.g.SELECT MIN(ntz_col) ...). This was a regression from 3.0.7.Root cause. The driver rewrites the
TIMESTAMP_NTZtype text toTIMESTAMPin three metadata-construction paths inDatabricksResultSetMetaData:columnInfo.setTypeText(TIMESTAMP)), to "maintain parity with thrift output".Because the
ColumnInfoTypeNameSDK enum has noTIMESTAMP_NTZvalue,getTypeName()isnullfor NTZ; the code correctly maps it to theTIMESTAMPenum (sogetColumnType()→Types.TIMESTAMP), but it also clobbered the human-readabletypeTextthatgetColumnTypeName()returns. Combined with the effective default protocol moving to SEA, every default connection lost the NTZ type name. In 3.0.7 the default was Thrift, which did not normalize, so it returnedTIMESTAMP_NTZ.Fix. The driver now preserves
TIMESTAMP_NTZacross all three paths by default.getColumnType()still reportsjava.sql.Types.TIMESTAMP, sinceTIMESTAMP_NTZis a timestamp without timezone.The legacy (Simba) driver reports
TIMESTAMPfor NTZ columns, so the previous behavior is preserved behind a new connection property:EnableTimestampNtzTypeNamegetColumnTypeName()for NTZgetColumnType()1(default)TIMESTAMP_NTZTypes.TIMESTAMP0(legacy / Simba parity)TIMESTAMPTypes.TIMESTAMPTesting
Verified live against a SQL warehouse with
SELECT MIN(ntz_col), MAX(ntz_col), ntz_col ... GROUP BY ntz_col:=1)EnableTimestampNtzTypeName=0TIMESTAMP_NTZTIMESTAMPusethriftclient=1)TIMESTAMP_NTZTIMESTAMPFor reference, the legacy Simba 2.8.1 driver returns
TIMESTAMPfor the same query — which the=0mode matches.Unit/integration tests (
mvn -pl jdbc-core test):DatabricksResultSetMetaDataTest— updated NTZ assertions for the new default; addedtestColumnsWithTimestampNTZ_legacyTypeNameDisabledcovering=0.PreparedStatementIntegrationTests#testGetMetaData_NoResultSet(the fakeservice test Normalize TIMESTAMP_NTZ to TIMESTAMP in Thrift path for metadata consistency #1182 was tied to),DatabricksConnectionContextTest,DatabricksTypeUtilTestall pass. Spotless clean.Additional Notes to the Reviewer
This intentionally re-scopes the normalization introduced in #1182 behind the new
EnableTimestampNtzTypeNameflag rather than removing it — the legacy behavior is still reachable viaEnableTimestampNtzTypeName=0. The metadata stays internally consistent (pre-execute vs post-execute) in either mode, which is whattestGetMetaData_NoResultSetguards. cc @gopalldb (author of #1182).This pull request and its description were written by Isaac.