Update parquet to v59.1.0

[renovate]

This PR contains the following updates:

Package	Type	Update	Change
parquet	dependencies	minor	59.0.0 → 59.1.0
parquet_derive	dependencies	minor	59.0.0 → 59.1.0

---

Release Notes

apache/arrow-rs (parquet)

v59.1.0

Compare Source

Full Changelog

Implemented enhancements:

• Fast path for nested DictionaryArray casting #​10247 [arrow]
• parquet/arrow: reading multiple nested columns fails with "Not all children array length are the same!" when a list continues across DataPageV2 page boundary #​10243 [parquet]
• Add product aggregate kernel to arrow-rs #​10150 [arrow]
• Stricter DataType parsing #​10146 [arrow]
• Support validating CSV headers against Schema #​10143 [arrow]
• arrow-ipc: Supports compression level configuration for arrow-ipc writer #​10132 [arrow]
• [Variant] VariantArray field API naming #​10093
• Add StructArray::field_ APIs symmetric to StructArray::column_ ones #​10092 [arrow]
• arrow-buffer: implement Saturating, CheckedShl, Not num-traits for i256 #​10087 [arrow]
• feat: native concat for MapArray #​10047 [arrow]
• [Variant] Add variant_to_arrow Dictionary/REE type support #​10013

Fixed bugs:

• arrow-row on fixed size binary/list with size 0 and no nulls return wrong length #​10270
• casting list to 0-size fixedsizelist can cause incorrect output length #​10227 [arrow]
• Buffer count mismatched with metadata when encoding records with dictionary of dictionaries #​10213 [arrow]
• Buffer::into_mutable is not consistent regarding sliced data and can lead to panics #​10117 [arrow]
• parquet_derive: cannot read or write columns whose name is a Rust keyword (raw identifiers like r#type become column "r#type") #​10112
• parquet: fix OffsetBuffer panic on corrupt input #​10107 [parquet]
• Parquet geospatial conversion uses metadata key "algorithm" instead of "edges" in geoarrow metadata #​9929 [parquet]

Documentation updates:

• doc: More comments to concat_batches #​10178 (2010YOUY01)
• Minor: improve PageStore docs with a temp-file spilling example #​10074 [parquet] (alamb)

Performance improvements:

• perf: interleave_list for List<Primitive> could be optimized? #​10022 [arrow]
• perf(arrow-ord): Avoid full index materialization for small-limit lexsorts #​9990 [arrow]
• Replace conversion of binary->string in arrow-row from arraydata to direct construction #​10261 [arrow] (Jefffrey)
• [arrow-flight encode path]re-use flatbufferbuilder #​10220 [arrow] (Rich-T-kid)
• [10125] arrow-flight decode path optimizations (add skip_validation to arrow-flight) #​10206 [arrow] [arrow-flight] (Rich-T-kid)
• Improve performance of concat_elements ByteViewArray concatenation #​10161 [arrow] (pepijnve)
• [arrow-flight] Optimize flight, remove some allocations, add dictionary focused benchmarks #​10126 [arrow] [arrow-flight] (Rich-T-kid)
• optimize(concat): concat map implementation #​10048 [arrow] (mapleFU)
• Reduce copies in Arrow IPC writer #​10044 [arrow] (Rich-T-kid)
• perf(interleave): Optimize list interleave_list when child is primitive #​10025 [arrow] (mapleFU)

Closed issues:

• Soundness: Unsound alignment contract in public FromBytes trait and BitReader::get_batch #​10164 [parquet]
• ParquetPushDecoder: expose the next row-group index that try_next_reader will yield #​10148 [parquet]
• arrow-ipc: Extend writer benchmarks to include dictionaries #​10119 [arrow]
• bench(parquet): benchmark for nested list write #​10083 [parquet]
• Support i256 implement From<i128> #​10080 [arrow]

Merged pull requests:

• fix(arrow-row): allow to convert non empty fixed size binary/list array with size length 0 and no nulls #​10271 [arrow] (rluvaton)
• chore: Fix audit CI run by ignore quick-xml audit advisories #​10267 (alamb)
• fix main: parquet test compilation failure #​10266 [parquet] (Jefffrey)
• minor: drive-by refactors for dicts in substring & filter #​10264 [arrow] (Jefffrey)
• Add validated row decode benchmark #​10259 [arrow] (alamb)
• arrow-cast: Add optimized path for unnesting a dict #​10248 [arrow] (brancz)
• feat: support uuid from fixed type of length 16 #​10241 [arrow] (ariel-miculas)
• chore(deps): bump actions/cache from 6.0.0 to 6.1.0 #​10240 (dependabot[bot])
• fix: Rename parquet feature flag 'flate2-rust_backened' to 'flate2-rust_backend' #​10239 [parquet] (dannycjones)
• chore: Make clippy::question_mark happy #​10231 (Tpt)
• fix(ipc): reject dictionary-encoded dictionary values #​10230 [arrow] (goutamadwant)
• Replace ArrayData with direct Array construction in arrow-row #​10229 [arrow] (Jefffrey)
• fix: casting list to fixedsizelist didn't respect input length #​10228 [arrow] (Jefffrey)
• chore: Fix clippy::byte_char_slices (use byte strings instead of explicit arrays) #​10225 [parquet] [arrow] (Tpt)
• nit: arrow-pyarrow: Use string interning #​10224 (Tpt)
• Support concatenation of mixed FixedSizeBinary via concat_elements_dyn #​10222 [arrow] (pepijnve)
• rename Compression struct #​10221 [arrow] [arrow-flight] (Rich-T-kid)
• chore(deps): bump the all-other-cargo-deps group across 1 directory with 16 updates #​10218 (dependabot[bot])
• chore(deps): bump actions/setup-python from 6.2.0 to 6.3.0 #​10210 (dependabot[bot])
• [10125] Introduce mult-batch decode benchmarks #​10207 [arrow] [arrow-flight] (Rich-T-kid)
• chore(deps): bump actions/cache from 5.0.5 to 6.0.0 #​10203 (dependabot[bot])
• introduce decode benchmarks #​10202 [arrow] [arrow-flight] (Rich-T-kid)
• Fix merge_kernels benchmark panic due to not wrapping with Scalar #​10199 [arrow] (Jefffrey)
• Benchmarks and performance improvement for parquet boolean reader #​10196 [parquet] (jhorstmann)
• add stale PR workflow #​10194 (Jefffrey)
• chore: group minor/patch dependabot updates #​10193 (Jefffrey)
• chore(deps): bump http from 1.4.0 to 1.4.2 #​10191 (dependabot[bot])
• chore(deps): bump syn from 2.0.117 to 2.0.118 #​10190 (dependabot[bot])
• chore(deps): bump chrono from 0.4.44 to 0.4.45 #​10188 (dependabot[bot])
• chore(deps): bump uuid from 1.23.1 to 1.23.3 #​10186 (dependabot[bot])
• chore: run cargo update to bump quinn #​10181 (Jefffrey)
• test: cover signed integers and bool in BitReader::get_batch test #​10180 [parquet] (alamb)
• [arrow-select] perf: Replace ArrayData with direct Array construction in take kernels #​10176 [arrow] (liamzwbao)
• Return PyValueError for nullable PyArrow struct imports #​10174 (fallintoplace)
• Fix Variant time microsecond JSON formatting #​10173 (fallintoplace)
• Split traits for plain and bitpacked decoding and fix soundness issue in BitReader::get_batch #​10172 [parquet] (jhorstmann)
• fix: switch generic usages of i128 to IntervalMonthDayNano for MonthDayNano type #​10171 [arrow] (Jefffrey)
• chore: specify --locked when cargo installing cargo-audit #​10170 (Jefffrey)
• chore: Fix clippy::useless_borrows_in_formatting #​10163 [parquet] [arrow] (Tpt)
• fix(arrow-cast): respect cast safety for overflowing temporal casts #​10162 [arrow] (SAY-5)
• chore(deps): bump actions/checkout from 6 to 7 #​10159 (dependabot[bot])
• feat(parquet): add ParquetPushDecoder::peek_next_row_group() #​10158 [parquet] (zhuqi-lucas)
• feat(pyarrow) FromPyArrow on Vec<T>: allow any iterable for input #​10155 (Tpt)
• nit: pyarrow: simplify class validation error creation #​10154 (Tpt)
• [Variant] add doc reference to VariantArrayBuilder #​10152 (sdf-jkl)
• feat: Adds product aggregate compute kernel #​10151 [arrow] (devanbenz)
• Stricter datatype parsing for decimals, fixedsizelists and time32/64 #​10147 [arrow] (Jefffrey)
• feat(arrow_csv): add header validation option #​10144 [arrow] (XiNiHa)
• [Parquet] route dictionary page through the PageStore #​10142 [parquet] (liamzwbao)
• chore: update pyo3 dependency to 0.29 #​10134 (timsaucer)
• feat(ipc): Supports compression level configuration #​10133 [arrow] (wForget)
• fix: write error for dbg output of out of range timestamps #​10130 [arrow] (Jefffrey)
• [Variant] VariantArray field API naming #​10124 (sdf-jkl)
• feat(arrow_array): add helper function to create MapArray from Vec<Option<Vec<(Key, Option<Value>)>>> for tests #​10123 [arrow] [arrow-flight] (rluvaton)
• perf(arrow-ipc): Add writer benchmarks for dictionaries #​10122 [arrow] (JakeDern)
• feat: support MapArray in lengths kernel #​10121 [arrow] (rluvaton)
• feat: add OffsetBuffer::subtract to allow to shift offsets by value #​10120 [arrow] (rluvaton)
• fix: Buffer::into_mutable return error instead of panic for converting owned sliced when not start at 0 and fix returned Mutable length #​10118 [arrow] (rluvaton)
• chore: update Bytes visibility to correctly reflect the actual visibility #​10115 [arrow] (rluvaton)
• fix(parquet_derive): support raw identifiers as column names #​10113 (cbmixx)
• removed clippy ignore statment #​10111 [arrow] (Rich-T-kid)
• Add StructArray::field_ APIs symmetric to StructArray::column_ ones #​10110 [arrow] (sdf-jkl)
• fix(parquet): return error instead of panicking in pad_nulls on corrupt input #​10108 [parquet] (thepenguinco)
• Minor: Add interleave tests for List<Decimal128> and List<Timestamp(tz)> #​10099 [arrow] (alamb)
• Add arrow-flight test coverage for IPC compression #​10097 [arrow] [arrow-flight] (alamb)
• chore(deps): bump pyspark from 3.3.2 to 3.4.4 in /parquet/pytest #​10091 [parquet] (dependabot[bot])
• refactor(parquet): bundle array reader recursion args into ReaderArgs #​10089 [parquet] (HippoBaro)
• arrow-buffer: implement Saturating, Checked num-traits for i256 #​10088 [arrow] (theirix)
• bench(parquet): add nested list writer benchmarks #​10084 [parquet] (mapleFU)
• Implement From<i128> for i256 #​10081 [arrow] (AdamGS)
• test(parquet): drop confusing main reference in page-roundtrip test comment #​10072 [parquet] (adriangb)
• ci: Split miri tests into 4 parallel shards #​10067 (AdamGS)
• Add tests and fix corner cases for Parquet/GeoArrow extension type conversion #​10065 [parquet] (paleolimbot)
• Support writing REE arrays directly to Parquet #​10064 [parquet] (Rich-T-kid)
• test(arrow-select): additional tests for inline-view filter fast path (tests for #​9755) #​10054 [arrow] (alamb)
• test(arrow-select): add take_bytes coverage for sliced values and nullable offset overflow #​10053 [arrow] (alamb)
• Consolidate filter_null_mask into FilterPredicate::filter_nulls #​10049 [arrow] (alamb)
• [Variant] Add VariantBuilder values check #​10016 (sdf-jkl)
• [Variant] Preserve UUID extension type metadata for Parquet writer #​10015 (sdf-jkl)
• feat(parquet-variant): add Dictionary and REE variant_to_arrow support #​10014 (mneetika)
• perf(arrow-ord): Avoid full index materialization for small-limit lexsorts #​9991 [arrow] (pchintar)

* This Changelog was automatically generated by github_changelog_generator

---

Configuration

📅 Schedule: (in timezone America/Detroit)

• Branch creation
  • Only on Monday (* * * * 1)
• Automerge
  • At any time (no schedule defined)

🚦 Automerge: Enabled.

♻ Rebasing: Whenever PR is behind base branch, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about these updates again.

---

• If you want to rebase/retry this PR, check this box

---

This PR was generated by Mend Renovate. View the repository job log.