downloads: don't use Content-Length as progress total when Content-Encoding is set#1816
Open
alliasgher wants to merge 2 commits intohttpie:masterfrom
Open
downloads: don't use Content-Length as progress total when Content-Encoding is set#1816alliasgher wants to merge 2 commits intohttpie:masterfrom
alliasgher wants to merge 2 commits intohttpie:masterfrom
Conversation
…coding is set When a server sends Content-Encoding (e.g. gzip), the `requests` library transparently decompresses the response body before httpie receives it. This means the bytes written to disk represent the *decompressed* size, which is typically much larger than the Content-Length header (which reflects the compressed wire size). Using Content-Length as the expected total in that case causes the downloader's "interrupted" check to fire at the end of an otherwise successful download, reporting a false "Incomplete download" error. Fix: skip Content-Length as the progress total whenever Content-Encoding is present. Progress will be shown without a denominator, which is the same behaviour already used for chunked responses with no Content-Length. Fixes httpie#1642 Signed-off-by: alliasgher <alliasgher123@gmail.com>
Python 3.14 introduced an eager _check_help() call in add_argument() that validates the help string immediately (cpython#65865). This causes LazyChoices.help to invoke the getter at argument-registration time rather than only when --help is printed, breaking the old assertion that getter.assert_not_called() after add_argument(). Expose a _ARGPARSE_CHECKS_HELP_EAGERLY flag and update the test to: - Expect getter and help_formatter to be called once at registration on 3.14+ (and reset their mocks so subsequent assertions stay clean). - Keep the strict lazy-evaluation assertions for 3.13 and earlier. Fixes httpie#1641 Signed-off-by: alliasgher <alliasgher123@gmail.com>
2 tasks
Author
|
The 2 failing tests ( |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #1642.
When a server sends
Content-Encoding: gzip(or any encoding), therequestslibrary transparently decompresses the response body. The bytes written to disk are the decompressed size, which is usually much larger than the compressedContent-Length. UsingContent-Lengthas the expected total causesdownloader.interruptedto returnTrueat the end of an otherwise successful download, producing a false"Incomplete download"error.Before
After
Progress is shown without a denominator (same as chunked responses with no
Content-Length), and no spurious error is emitted.Fix
Skip
Content-Lengthas the progress total wheneverContent-Encodingis present.The existing
# FIXME: some servers still might send Content-Encoding: gzipcomment (which referenced issue #423) is replaced by this explicit guard.Tests
Added
test_download_with_content_encoding_does_not_report_incompletewhich:Content-Length: 5andContent-Encoding: gzipdownloader.interruptedisFalse