Skip to content

GH-50149: [C++][Parquet] Handle OOM as soft failure in parquet encoding fuzzer#50154

Closed
omertt27 wants to merge 1 commit into
apache:mainfrom
omertt27:GH-50149-fix-oom-encoding-fuzzer
Closed

GH-50149: [C++][Parquet] Handle OOM as soft failure in parquet encoding fuzzer#50154
omertt27 wants to merge 1 commit into
apache:mainfrom
omertt27:GH-50149-fix-oom-encoding-fuzzer

Conversation

@omertt27

@omertt27 omertt27 commented Jun 10, 2026

Copy link
Copy Markdown

Rationale for this change

When the CappedMemoryPool (2.2 GB limit, added in GH-48105) triggers an OOM during encoding roundtrip verification, the resulting Status::OutOfMemory propagates back to call sites that used ARROW_CHECK_OK(...) or .ValueOrDie(). Both of these expand to ARROW_LOG(FATAL)std::abort(), which is not an exception — BEGIN_PARQUET_CATCH_EXCEPTIONS cannot intercept it. OSS-Fuzz then sees a process crash instead of a resource-limit event.

What changes are included in this PR?

Added FuzzCheckOk(Status) in the anonymous namespace of fuzz_encoding_internal.cc:

// OOM during fuzzing is an expected soft failure; any other non-OK status
// indicates a real bug and should abort so OSS-Fuzz can report it.
Status FuzzCheckOk(const Status& st) {
  if (st.IsOutOfMemory()) return st;
  ARROW_CHECK_OK(st);
  return Status::OK();
}

Six call sites in TypedFuzzEncoding::Fuzz() replaced with RETURN_NOT_OK(FuzzCheckOk(...)):

  • reference_array_->ValidateFull()
  • DecodeArrow(...).ValueOrDie() (replaced with status check + std::move(*result))
  • array->ValidateFull() (on roundtrip result)
  • CompareAgainstReference(array)
  • RunOnDecodedChunks(...) × 2 (both arrow_supported() and non-Arrow branches)

Three invariant checks intentionally left as hard aborts — they indicate actual decoder/encoder bugs, not resource limits:

  • ARROW_CHECK_LE(values_read, read_size) — decoder returning more values than requested
  • ARROW_CHECK_EQ(acc.chunks.size(), 0) — BinaryBuilder invariant
  • ARROW_CHECK_EQ(offset, total_data_size) — byte count invariant in MakeArrow

Are these changes tested?

The OOM path is exercised by fuzzing_memory_pool() in fuzz_internal.cc whenever the cumulative allocation exceeds the 2.2 GB cap. The existing parquet-encoding-test suite covers the non-OOM code paths.

Closes #50149

…encoding fuzzer

When the CappedMemoryPool (2.2 GB limit) triggers OOM during encoding
roundtrip, the resulting Status::OutOfMemory was passed to ARROW_CHECK_OK
or .ValueOrDie(), both of which call ARROW_LOG(FATAL) -> std::abort().
This is not an exception, so BEGIN_PARQUET_CATCH_EXCEPTIONS cannot intercept
it, and OSS-Fuzz sees a process crash instead of a resource-limit event.

Add FuzzCheckOk(Status) in the anonymous namespace: returns the status if
OutOfMemory (soft failure, propagates up to LogFuzzStatus which already
handles it), calls ARROW_CHECK_OK for any other non-OK status (hard abort
preserved so OSS-Fuzz still catches real decoder/encoder bugs).

Six call sites fixed in TypedFuzzEncoding::Fuzz(); three invariant checks
(decoder value count, BinaryBuilder state, byte count) intentionally left
as hard aborts because they indicate actual implementation bugs.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@omertt27
omertt27 requested a review from wgtmac as a code owner June 10, 2026 21:21
Copilot AI review requested due to automatic review settings June 10, 2026 21:21

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Adds fuzzing-specific status handling to treat OutOfMemory as a non-fatal outcome while ensuring all other non-OK statuses abort, improving OSS-Fuzz signal quality.

Changes:

  • Introduce FuzzCheckOk helper to allow OOM to bubble up while aborting on other failures.
  • Replace several ARROW_CHECK_OK / ValueOrDie() call sites with RETURN_NOT_OK(FuzzCheckOk(...)) to avoid hard aborts on OOM.
  • Propagate OOM through decoding/validation/compare and chunked decoding paths.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread cpp/src/parquet/arrow/fuzz_encoding_internal.cc
Comment thread cpp/src/parquet/arrow/fuzz_encoding_internal.cc
Comment thread cpp/src/parquet/arrow/fuzz_encoding_internal.cc
@pitrou

pitrou commented Jun 11, 2026

Copy link
Copy Markdown
Member

Closing this PR as it's superseded by #50150

@pitrou pitrou closed this Jun 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[C++][Parquet] OOM leads to hard failure in parquet-encoding-fuzz

3 participants