Describe the bug
#4541 (review)
Steps to reproduce
import java.nio.file.Files
import spark.implicits._
spark.conf.set("spark.sql.adaptive.enabled", "false")
val path = Files.createTempDirectory("comet-ym-").resolve("data").toString
Seq((1, 2), (0, -1)).toDF("y", "m").write.parquet(path)
// Case 1: nested YearMonthIntervalType fallback
val nested = spark.sql(
"""SELECT transform(a, x -> x) AS result
|FROM VALUES
| (array(make_ym_interval(1, 2),
| CAST(NULL AS INTERVAL YEAR TO MONTH)))
|AS t(a)""".stripMargin)
nested.explain("extended")
nested.show(false)
// Case 2: native-shuffle fallback
val shuffled = spark.sql(
s"""SELECT y, m, make_ym_interval(y, m) AS i
|FROM parquet.`$path`
|DISTRIBUTE BY y""".stripMargin)
shuffled.explain("extended")
shuffled.show(false)
Expected behavior
No response
Additional context
No response
Describe the bug
#4541 (review)
Steps to reproduce
Expected behavior
No response
Additional context
No response