GH-49114: [C++][Parquet] Fix converting schema failure with deep nested two-level encoding list structure by HuaHuaY · Pull Request #49125 · apache/arrow

HuaHuaY · 2026-02-03T08:29:05Z

Rationale for this change

Fix the failure when converting parquet schema with deep nested two-level encoding list structure to arrow schema.

What changes are included in this PR?

Modified the ListToSchemaField and MapToSchemaField methods.

Are these changes tested?

Yes.

Are there any user-facing changes?

It allows some legal schemas that were once considered illegal.

GitHub Issue: [C++][Parquet] Failed to read deep nested two-level encoding list structure #49114

HuaHuaY · 2026-02-03T08:39:30Z

@pitrou @emkornfield @wgtmac Please take a look.

Copilot

Pull request overview

Fixes Parquet-to-Arrow schema conversion for deep nested two-level encoding LIST structures (and related MAP-in-LIST cases) that previously errored as illegal.

Changes:

Refactors list element resolution into a new ResolveList() helper to better handle nested two-level LIST encodings.
Relaxes repetition constraints to allow LIST/MAP nodes to be REPEATED when they are elements of an enclosing LIST.
Adds/updates unit tests to cover deep nested two-level LISTs and two-level LIST-of-MAP schemas, and adjusts expected error text.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File	Description
`cpp/src/parquet/arrow/schema.cc`	Adjusts LIST/MAP schema conversion logic and introduces `ResolveList()` for nested/two-level handling.
`cpp/src/parquet/arrow/arrow_schema_test.cc`	Adds coverage for deep nested two-level LIST and allows previously-failing LIST-of-MAP case; updates an error expectation.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

wgtmac

Mostly look good to me. Thanks for fixing this!

wgtmac · 2026-02-10T03:39:13Z

-    } else if (list_group.field_count() == 1) {
-      const auto& repeated_field = list_group.field(0);
-      if (repeated_field->is_repeated()) {
+      RETURN_NOT_OK(check_two_level_list_repeated(group));


Do we need to validate that list_group has no logical type annotation?

I think this is not in backward-compatibility rules.

wgtmac · 2026-02-10T03:58:23Z

+    //       optional binary value (STRING);
+    //     }
+    //   }
+    // }


Do we want to add a case for the below which is still produced by parquet-java by default:

// optional group my_list (LIST) { // repeated group array (MAP) { // repeated group key_value (MAP_KEY_VALUE) { // required binary key (STRING); // optional binary value (STRING); // } // } // }

I think it's not in backward-compatibility rules too. We may add it into document first.

…t structure

wgtmac · 2026-02-10T09:37:46Z

Do you want to take a look? @pitrou @emkornfield @mapleFU

conbench-apache-arrow · 2026-02-21T22:49:22Z

After merging your PR, Conbench analyzed the 3 benchmarking runs that have been run so far on merge-commit 0cf32b2.

There were no benchmark performance regressions. 🎉

The full Conbench report has more details. It also includes information about 1 possible false positive for unstable benchmarks that are known to sometimes produce them.

…p nested two-level encoding list structure (apache#49125) ### Rationale for this change Fix the failure when converting parquet schema with deep nested two-level encoding list structure to arrow schema. ### What changes are included in this PR? Modified the `ListToSchemaField` and `MapToSchemaField` methods. ### Are these changes tested? Yes. ### Are there any user-facing changes? It allows some legal schemas that were once considered illegal. * GitHub Issue: apache#49114 Authored-by: Zehua Zou <zehuazou2000@gmail.com> Signed-off-by: Gang Wu <ustcwg@gmail.com>

HuaHuaY requested a review from wgtmac as a code owner February 3, 2026 08:29

github-actions Bot added Component: Parquet Component: C++ awaiting review Awaiting review labels Feb 3, 2026

HuaHuaY force-pushed the fix_issue_49114 branch from b494f79 to 7176212 Compare February 4, 2026 03:29

wgtmac requested a review from Copilot February 4, 2026 10:11

Copilot started reviewing on behalf of wgtmac February 4, 2026 10:12 View session

Copilot AI reviewed Feb 4, 2026

View reviewed changes

Comment thread cpp/src/parquet/arrow/schema.cc

HuaHuaY requested a review from Copilot February 4, 2026 11:12

Copilot started reviewing on behalf of HuaHuaY February 4, 2026 11:12 View session

Copilot AI reviewed Feb 4, 2026

View reviewed changes

Comment thread cpp/src/parquet/arrow/schema.cc Outdated

github-actions Bot added awaiting committer review Awaiting committer review and removed awaiting review Awaiting review labels Feb 4, 2026

HuaHuaY requested a review from Copilot February 4, 2026 11:36

Copilot started reviewing on behalf of HuaHuaY February 4, 2026 11:36 View session

Copilot AI reviewed Feb 4, 2026

View reviewed changes

HuaHuaY force-pushed the fix_issue_49114 branch 2 times, most recently from d9eb0dc to 4f7558f Compare February 9, 2026 02:45

wgtmac reviewed Feb 9, 2026

View reviewed changes

Comment thread cpp/src/parquet/arrow/schema.cc

Comment thread cpp/src/parquet/arrow/schema.cc Outdated

Comment thread cpp/src/parquet/arrow/schema.cc

Comment thread cpp/src/parquet/arrow/schema.cc Outdated

HuaHuaY force-pushed the fix_issue_49114 branch from 4f7558f to 145580f Compare February 10, 2026 03:17

wgtmac reviewed Feb 10, 2026

View reviewed changes

Fix converting schema failure with deep nested two-level encoding lis…

404a85e

…t structure

HuaHuaY force-pushed the fix_issue_49114 branch from efddcaa to 404a85e Compare February 10, 2026 08:06

wgtmac approved these changes Feb 10, 2026

View reviewed changes

wgtmac merged commit 0cf32b2 into apache:main Feb 21, 2026
50 of 51 checks passed

wgtmac removed the awaiting committer review Awaiting committer review label Feb 21, 2026

wgtmac mentioned this pull request Feb 21, 2026

[C++][Parquet] Failed to read deep nested two-level encoding list structure #49114

Closed

HuaHuaY deleted the fix_issue_49114 branch February 25, 2026 03:34

Uh oh!

Conversation

HuaHuaY commented Feb 3, 2026 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

Uh oh!

HuaHuaY commented Feb 3, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

wgtmac left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

wgtmac Feb 10, 2026

Choose a reason for hiding this comment

Uh oh!

HuaHuaY Feb 10, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

wgtmac Feb 10, 2026

Choose a reason for hiding this comment

Uh oh!

HuaHuaY Feb 10, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

wgtmac commented Feb 10, 2026

Uh oh!

Uh oh!

conbench-apache-arrow Bot commented Feb 21, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

HuaHuaY commented Feb 3, 2026 •

edited by github-actions Bot

Loading