GH-49837: [C++][Parquet] Avoid unbounded temporary std::vector in DELTA_(LENGTH_)BYTE_ARRAY decoder#49838
Conversation
714d562 to
39fc02f
Compare
|
@github-actions crossbow submit -g cpp |
|
Revision: 39fc02f Submitted crossbow builds: ursacomputing/crossbow @ actions-2ca183efe9 |
…in DELTA_(LENGTH_)BYTE_ARRAY decoder
39fc02f to
14d3f53
Compare
|
@github-actions crossbow submit -g cpp |
|
Revision: 14d3f53 Submitted crossbow builds: ursacomputing/crossbow @ actions-3c858e4ad0 |
| #ifndef ARROW_ENABLE_THREADING | ||
| GTEST_SKIP() << "Test requires threading support"; | ||
| #endif |
There was a problem hiding this comment.
This is a quick fix for an unrelated CI regression.
|
Hi @mapleFU and thanks a lot for the review! 👋 |
|
After merging your PR, Conbench analyzed the 3 benchmarking runs that have been run so far on merge-commit 487f148. There were no benchmark performance regressions. 🎉 The full Conbench report has more details. It also includes information about 4 possible false positives for unstable benchmarks that are known to sometimes produce them. |
|
Hi, I also wonder does Maybe when I writing this I think this would be small vector, but in fuzz it could be wrongly large... |
That's a good question. We'll see if the fuzzer flags it, IMHO. |
…in DELTA_(LENGTH_)BYTE_ARRAY decoder (apache#49838) ### Rationale for this change `DeltaLengthByteArrayDecoder::DecodeArrow` and `DeltaByteArrayDecoder::DecodeArrow` both allocate a temporary `std::vector` for the entire range of decoded values. This generates out-of-memory failures in the Parquet encoding fuzzer as this unbounded allocation bypasses the custom fuzzing memory pool. This issue was found by OSS-Fuzz: https://oss-fuzz.com/testcase-detail/6634166832922624 ### What changes are included in this PR? Use the given MemoryPool to allocate a temporary buffer, so that potential memory limits are applied. ### Are these changes tested? Yes, using existing tests. ### Are there any user-facing changes? No. * GitHub Issue: apache#49837 Authored-by: Antoine Pitrou <antoine@python.org> Signed-off-by: Antoine Pitrou <antoine@python.org>
Rationale for this change
DeltaLengthByteArrayDecoder::DecodeArrowandDeltaByteArrayDecoder::DecodeArrowboth allocate a temporarystd::vectorfor the entire range of decoded values.This generates out-of-memory failures in the Parquet encoding fuzzer as this unbounded allocation bypasses the custom fuzzing memory pool.
This issue was found by OSS-Fuzz: https://oss-fuzz.com/testcase-detail/6634166832922624
What changes are included in this PR?
Use the given MemoryPool to allocate a temporary buffer, so that potential memory limits are applied.
Are these changes tested?
Yes, using existing tests.
Are there any user-facing changes?
No.