Add specialized negative suites: truncated / resultset / async (PR 6/8)#1530
Merged
Merged
Conversation
The final batch of negative suites — row-limit rejection, ResultSet misuse, and the Databricks async extension API. - NEGATIVE_STATEMENT_SELECT_TRUNCATED: invalid setMaxRows/setLargeMaxRows values (negative, MIN_VALUE) — the set-then-execute sequence is captured since the failure may originate at either point. - NEGATIVE_RESULTSET: ResultSet misuse — next()/getObject() after the ResultSet or Statement is closed, and an out-of-range column index. The originally-planned CloudFetch link-expiry / chunk-download failure is intentionally omitted (not deterministically reproducible from a client test; would need server-side fault injection) — documented in the provider. - NEGATIVE_ASYNC: Databricks async extension via stmt.unwrap(IDatabricksStatement.class) — getExecutionResult() before any async execution, executeAsync() on invalid SQL (polls getExecutionResult so the async failure is actually captured, not the successful submit), and getExecutionResult() after the statement is closed. Validated live against the peco serverless warehouse (Thrift-vs-SEA, shadow): positive STATEMENT_SELECT runs in the same JVM and still PASSes. Each case was probe-verified to hit a real error path (setMaxRows(-1) -> ValidationException, getExecutionResult-before-exec -> "No execution available", after-close -> "ResultSet is closed"); the all-PASS cases are genuine agreement, not vacuous. getObject(999) surfaced a real 3-field mismatch. Also caught and fixed a vacuous first cut of the async invalid-SQL case that captured only the submit. ErrorComparatorTest: 20/20. Isaac Review: 0 findings. NO_CHANGELOG=true (test-only comparator tooling). Co-authored-by: Isaac Signed-off-by: Sreekanth Vadigi <sreekanth.vadigi@databricks.com>
sreekanth-db
added a commit
that referenced
this pull request
Jul 3, 2026
…8) (#1532) ## What Turns deep error comparison **on** for the weekly CI run, in **shadow** mode: every Thrift-vs-SEA error divergence (class / SQLState / code / message / one-sided) is recorded in the report/CSV, but the run stays green. Final PR of the negative-cases series (builds on #1524–#1530). - `run-comparator.sh` and `runJdbcComparator.yml` default `ERROR_COMPARISON_MODE` to `shadow` (previously the deep comparison only ran when a caller set the flag). - `ErrorPolicy` is simplified to two modes: `off` and `shadow`. - README documents the two modes. ## Why no `authoritative` (fail-the-build) mode An enforcement mode was prototyped and **deliberately removed** before this PR. Reason: enforcement over the `DatabaseMetaData` matrix can't be done cleanly at the diff-string level — `CombinationExecutor` collapses one-sided throws *and* infra failures into the same `"Execution error: …"` string, so a gate would either false-fail on an infra flake or silently miss a real one-sided metadata divergence. Rather than ship a gate with that known hole (dormant and unused, since we're not enforcing yet), enforcement is left as a clean future change, driven by the divergence data these shadow runs will accumulate. ## Behavior Shadow produces the **same report/CSV as before** and never fails on a diff — nothing else changes. Exception-class comparison stays strict (exact match); message text is compared raw (no normalization). ## Testing `ErrorComparatorTest` — 20/20; compiles clean; dead-code check clean after removing the prototype gate; Isaac Review (`/review --uncommitted`) — 0 findings. NO_CHANGELOG=true (test-only comparator tooling, no driver behavior change). This pull request and its description were written by Isaac. Signed-off-by: Sreekanth Vadigi <sreekanth.vadigi@databricks.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
The final batch of negative suites — row-limit rejection, ResultSet misuse, and the Databricks async extension API. Builds on #1524–#1529.
NEGATIVE_STATEMENT_SELECT_TRUNCATEDsetMaxRows/setLargeMaxRowsvalues (negative,MIN_VALUE) — set-then-execute captured together since the failure may originate at either pointNEGATIVE_RESULTSETnext()/getObject()after the ResultSet or Statement is closed; out-of-range column indexNEGATIVE_ASYNCgetExecutionResult()before any async execution;executeAsync()on invalid SQL;getExecutionResult()after the statement is closedNotes on scope
NEGATIVE_RESULTSET: the originally-planned CloudFetch link-expiry / chunk-download failure is intentionally omitted — it can't be provoked deterministically from a client test (links are valid for the query's lifetime; expiry is time/infra dependent), and would require server-side fault injection out of scope for this harness. Documented in the provider javadoc.NEGATIVE_ASYNC: usesstmt.unwrap(IDatabricksStatement.class). The invalid-SQL case pollsgetExecutionResultso the async failure is actually captured —executeAsynconly submits; the syntax error surfaces on result retrieval.Live validation
Ran against the peco serverless warehouse (Thrift-vs-SEA,
ERROR_COMPARISON_MODE=shadow). PositiveSTATEMENT_SELECTruns in the same JVM and still PASSes.Each case was probe-verified to hit a real error path (not vacuous PASS):
setMaxRows(-1)→DatabricksValidationException: Invalid input for maxRowsgetExecutionResult()before exec →DatabricksSQLException: No execution availableOperation not allowed - ResultSet is closedgetObject(999)→ real 3-field error mismatch (DIFF)The all-PASS cases are genuine agreement (both endpoints throw identically), confirmed via a throwaway probe. Also caught and fixed a vacuous first cut of the async invalid-SQL case that captured only the successful submit rather than the eventual failure.
Testing
ErrorComparatorTest— 20/20; live shadow runs + probe; clean compile; Isaac Review (/review --uncommitted) — 0 findings.NO_CHANGELOG=true (test-only comparator tooling, no driver behavior change).
This pull request and its description were written by Isaac.