pd.read_parquet("gs://...") fails with CURL error 56 when HTTPS_PROXY is set, pyarrow 23 C++ GCS client ignores NO_PROXY #49979
dipk-mish7
started this conversation in
General
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
We recently upgraded from pyarrow=12 to pyarrow=23 and started seeing this error when reading parquet files from GCS in environments where HTTPS_PROXY is set:
OSError: google::cloud::Status(UNAVAILABLE: Retry policy exhausted ...
PerformWork() - CURL error [56]=Failure when receiving data from the peer)
After investigating, we found that setting empty storage option was letting me read the file
df = pd.read_parquet("gs://bucket/file.parquet", storage_options={})
Questions:
Beta Was this translation helpful? Give feedback.
All reactions