Async streaming responses documentation is missing some details

[simonw]

https://www.python-httpx.org/async/#streaming-responses

Current snapshot: https://github.com/encode/httpx/blob/67e9c14952340bfd799a0de41dfaf269f376653d/docs/async.md#streaming-responses

This section in particular:

The async response streaming methods are:

• Response.aread() - For conditionally reading a response inside a stream block.
• Response.aiter_bytes() - For streaming the response content as bytes.
• Response.aiter_text() - For streaming the response content as text.
• Response.aiter_lines() - For streaming the response content as lines of text.
• Response.aiter_raw() - For streaming the raw response bytes, without applying content decoding.
• Response.aclose() - For closing the response. You don't usually need this, since .stream block closes the response automatically on exit.

The missing details I'm talking about include:

• await Response.aread() returns a bytes object
• Response.aiter_bytes() returns an async iterator over bytes - and accepts an optional chunk_size= integer
• Response.aiter_text() returns an async iterator over str - and accepts an optional chunk_size= integer - it decodes text using the the response's encoding, if set, but then defaults to utf-8
• Response.aiter_lines() returns an async iterator over str - but each string is a full line, unlike aiter_text()
• Response.aiter_raw() returns an async iterator over bytes and accepts an optional chunk_size= integer. The difference between this and .aiter_bytes() isn't obvious to me

I figured most of this out by looking at the type annotations in this file:

httpx/httpx/_models.py

Line 890 in 67e9c14

async def aread(self) -> bytes:

[florimondmanca]

Interesting, thanks. These bits of info are documented in the Developer Interface docs. Ideally readers would be directed there if they'd like to see what methods are available for async responses, rather than half-duplicating that info in the guide. There's wider discussion on the quality of the API Reference material in the HTTPX documentation here, #1220.

[simonw]

Huh, thanks - I didn't know about https://www.python-httpx.org/api/#response !

I always hit https://www.python-httpx.org/async/ when I'm looking for documentation.

Having an obvious link from that section in https://www.python-httpx.org/async/ to https://www.python-httpx.org/api/#response would solve this issue for me.

[simonw]

One thing that might have helped me spot that: I think my eyes skipped straight over "Developer Interface", since I assumed it was aimed at maintainers of the library and not at library users for some reason. If that menu item had been called "API Reference" I think I would have been more likely to have explored it.

[florimondmanca]

I think my eyes skipped straight over "Developer Interface",

If that menu item had been called "API Reference"

Right, to be fair the exact same happens to me all the time! I proposed #2308 which groups all "Reference material" stuff under a more usual API Reference section.

Doesn't change the "Developer Interface" yet. I'm thinking seeing the only other items under that section would be "Exceptions" and "Environment Variables", it might become clear enough that this contains a reference of the public API...

[florimondmanca]

Ah, that name might come from us taking inspiration from Requests... https://requests.readthedocs.io/en/latest/api/ is titled "Developer Interface" (but the sidebar item reads "API Reference").

[Tronic]

• Response.aiter_raw() returns an async iterator over bytes and accepts an optional chunk_size= integer. The difference between this and .aiter_bytes() isn't obvious to me

Yet, there doesn't appear to be any explanation for this difference in the aforementioned docs.

Perhaps with content-encoding or transfer-encoding raw returns the compressed content? In any case, this needs to be documented.

[lovelydinosaur]

Perhaps with content-encoding or transfer-encoding raw returns the compressed content?

Correct. iter_raw and aiter_raw return the bytewise content of the response, which may be compressed.
The iter_bytes and aiter_bytes methods return the uncompessed bytewise content.

In any case, this needs to be documented.

Sure.
We're hardly mentioning those methods at all in the docs.

Should we start by making sure the docstrings are clear enough here?

The various accessors onto the response content stack up like this...

• iter_raw - The possibly compressed byte content of the response.
• iter_bytes / .read() / .content - The byte content of the response, with any neccessary decompression applied. (gzip/deflate/brotli)
• iter_text / .text - The text content of the response, decoded from the byte content as determined by the charset parameter in the Content-Type header.
• iter_lines - An iterator over the text content of the response with line-by-line buffering applied.