gh-146311: Reject non-canonical padding bits in base32, 64, & 85 decoding

Doc/library/base64.rst

@@ -76,8 +76,8 @@ POST request.
       Added the *padded* and *wrapcol* parameters.
 
 
-.. function:: b64decode(s, altchars=None, validate=False, *, padded=True)
-              b64decode(s, altchars=None, validate=True, *, ignorechars, padded=True)
+.. function:: b64decode(s, altchars=None, validate=False, *, padded=True, canonical=False)
+              b64decode(s, altchars=None, validate=True, *, ignorechars, padded=True, canonical=False)
 
    Decode the Base64 encoded :term:`bytes-like object` or ASCII string
    *s* and return the decoded :class:`bytes`.
@@ -112,10 +112,13 @@ POST request.
    If *validate* is true, these non-alphabet characters in the input
    result in a :exc:`binascii.Error`.
 
+   If *canonical* is true, non-zero padding bits are rejected.
+   See :func:`binascii.a2b_base64` for details.
+
    For more information about the strict base64 check, see :func:`binascii.a2b_base64`
 
    .. versionchanged:: 3.15
-      Added the *ignorechars* and *padded* parameters.
+      Added the *ignorechars*, *padded*, and *canonical* parameters.
 
    .. deprecated:: 3.15
       Accepting the ``+`` and ``/`` characters with an alternative alphabet
@@ -179,7 +182,7 @@ POST request.
       Added the *padded* and *wrapcol* parameters.
 
 
-.. function:: b32decode(s, casefold=False, map01=None, *, padded=True, ignorechars=b'')
+.. function:: b32decode(s, casefold=False, map01=None, *, padded=True, ignorechars=b'', canonical=False)
 
    Decode the Base32 encoded :term:`bytes-like object` or ASCII string *s* and
    return the decoded :class:`bytes`.
@@ -203,12 +206,15 @@ POST request.
    *ignorechars* should be a :term:`bytes-like object` containing characters
    to ignore from the input.
 
+   If *canonical* is true, non-zero padding bits are rejected.
+   See :func:`binascii.a2b_base32` for details.
+
    A :exc:`binascii.Error` is raised if *s* is
    incorrectly padded or if there are non-alphabet characters present in the
    input.
 
    .. versionchanged:: next
-      Added the *ignorechars* and *padded* parameters.
+      Added the *ignorechars*, *padded*, and *canonical* parameters.
 
 
 .. function:: b32hexencode(s, *, padded=True, wrapcol=0)
@@ -222,7 +228,7 @@ POST request.
       Added the *padded* and *wrapcol* parameters.
 
 
-.. function:: b32hexdecode(s, casefold=False, *, padded=True, ignorechars=b'')
+.. function:: b32hexdecode(s, casefold=False, *, padded=True, ignorechars=b'', canonical=False)
 
    Similar to :func:`b32decode` but uses the Extended Hex Alphabet, as defined in
    :rfc:`4648`.
@@ -235,7 +241,7 @@ POST request.
    .. versionadded:: 3.10
 
    .. versionchanged:: next
-      Added the *ignorechars* and *padded* parameters.
+      Added the *ignorechars*, *padded*, and *canonical* parameters.
 
 
 .. function:: b16encode(s, *, wrapcol=0)
@@ -315,7 +321,7 @@ Refer to the documentation of the individual functions for more information.
    .. versionadded:: 3.4
 
 
-.. function:: a85decode(b, *, foldspaces=False, adobe=False, ignorechars=b' \t\n\r\v')
+.. function:: a85decode(b, *, foldspaces=False, adobe=False, ignorechars=b' \t\n\r\v', canonical=False)
 
    Decode the Ascii85 encoded :term:`bytes-like object` or ASCII string *b* and
    return the decoded :class:`bytes`.
@@ -332,8 +338,16 @@ Refer to the documentation of the individual functions for more information.
    This should only contain whitespace characters, and by
    default contains all whitespace characters in ASCII.
 
+   If *canonical* is true, non-canonical encodings are rejected.
+   See :func:`binascii.a2b_ascii85` for details.
+
    .. versionadded:: 3.4
 
+   .. versionchanged:: next
+      Added the *canonical* parameter.
+      Single-character final groups are now always rejected as encoding
+      violations.
+
 
 .. function:: b85encode(b, pad=False, *, wrapcol=0)
 
@@ -353,7 +367,7 @@ Refer to the documentation of the individual functions for more information.
       Added the *wrapcol* parameter.
 
 
-.. function:: b85decode(b, *, ignorechars=b'')
+.. function:: b85decode(b, *, ignorechars=b'', canonical=False)
 
    Decode the base85-encoded :term:`bytes-like object` or ASCII string *b* and
    return the decoded :class:`bytes`.  Padding is implicitly removed, if
@@ -362,10 +376,15 @@ Refer to the documentation of the individual functions for more information.
    *ignorechars* should be a :term:`bytes-like object` containing characters
    to ignore from the input.
 
+   If *canonical* is true, non-canonical encodings are rejected.
+   See :func:`binascii.a2b_base85` for details.
+
    .. versionadded:: 3.4
 
    .. versionchanged:: next
-      Added the *ignorechars* parameter.
+      Added the *ignorechars* and *canonical* parameters.
+      Single-character final groups are now always rejected as encoding
+      violations.
 
 
 .. function:: z85encode(s, pad=False, *, wrapcol=0)
@@ -390,7 +409,7 @@ Refer to the documentation of the individual functions for more information.
       Added the *wrapcol* parameter.
 
 
-.. function:: z85decode(s, *, ignorechars=b'')
+.. function:: z85decode(s, *, ignorechars=b'', canonical=False)
 
    Decode the Z85-encoded :term:`bytes-like object` or ASCII string *s* and
    return the decoded :class:`bytes`.  See `Z85  specification
@@ -399,10 +418,15 @@ Refer to the documentation of the individual functions for more information.
    *ignorechars* should be a :term:`bytes-like object` containing characters
    to ignore from the input.
 
+   If *canonical* is true, non-canonical encodings are rejected.
+   See :func:`binascii.a2b_base85` for details.
+
    .. versionadded:: 3.13
 
    .. versionchanged:: next
-      Added the *ignorechars* parameter.
+      Added the *ignorechars* and *canonical* parameters.
+      Single-character final groups are now always rejected as encoding
+      violations.
 
 
 .. _base64-legacy:

Doc/library/binascii.rst

@@ -48,8 +48,8 @@ The :mod:`!binascii` module defines the following functions:
       Added the *backtick* parameter.
 
 
-.. function:: a2b_base64(string, /, *, padded=True, alphabet=BASE64_ALPHABET, strict_mode=False)
-              a2b_base64(string, /, *, ignorechars, padded=True, alphabet=BASE64_ALPHABET, strict_mode=True)
+.. function:: a2b_base64(string, /, *, padded=True, alphabet=BASE64_ALPHABET, strict_mode=False, canonical=False)
+              a2b_base64(string, /, *, ignorechars, padded=True, alphabet=BASE64_ALPHABET, strict_mode=True, canonical=False)
 
    Convert a block of base64 data back to binary and return the binary data. More
    than one line may be passed at a time.
@@ -80,11 +80,15 @@ The :mod:`!binascii` module defines the following functions:
    * Contains no excess data after padding (including excess padding, newlines, etc.).
    * Does not start with a padding.
 
+   If *canonical* is true, non-zero padding bits in the last group are rejected
+   with :exc:`binascii.Error`, enforcing canonical encoding as defined in
+   :rfc:`4648` section 3.5.  This check is independent of *strict_mode*.
+
    .. versionchanged:: 3.11
       Added the *strict_mode* parameter.
 
    .. versionchanged:: 3.15
-      Added the *alphabet*, *ignorechars* and *padded* parameters.
+      Added the *alphabet*, *ignorechars*, *padded*, and *canonical* parameters.
 
 
 .. function:: b2a_base64(data, *, padded=True, alphabet=BASE64_ALPHABET, wrapcol=0, newline=True)
@@ -110,16 +114,17 @@ The :mod:`!binascii` module defines the following functions:
       Added the *alphabet*, *padded* and *wrapcol* parameters.
 
 
-.. function:: a2b_ascii85(string, /, *, foldspaces=False, adobe=False, ignorechars=b'')
+.. function:: a2b_ascii85(string, /, *, foldspaces=False, adobe=False, ignorechars=b'', canonical=False)
 
    Convert Ascii85 data back to binary and return the binary data.
 
    Valid Ascii85 data contains characters from the Ascii85 alphabet in groups
-   of five (except for the final group, which may have from two to five
+   of five (except for the final group, which may have from two to four
    characters). Each group encodes 32 bits of binary data in the range from
    ``0`` to ``2 ** 32 - 1``, inclusive. The special character ``z`` is
    accepted as a short form of the group ``!!!!!``, which encodes four
-   consecutive null bytes.
+   consecutive null bytes. A single-character final group is always rejected
+   as an encoding violation.
 
    *foldspaces* is a flag that specifies whether the 'y' short sequence
    should be accepted as shorthand for 4 consecutive spaces (ASCII 0x20).
@@ -132,10 +137,20 @@ The :mod:`!binascii` module defines the following functions:
    to ignore from the input.
    This should only contain whitespace characters.
 
+   If *canonical* is true, non-canonical encodings are rejected with
+   :exc:`binascii.Error`.  This enforces that the ``z`` abbreviation is used
+   for all-zero groups (rather than ``!!!!!``), and that partial final groups
+   use the same padding digits the encoder would produce.
+
    Invalid Ascii85 data will raise :exc:`binascii.Error`.
 
    .. versionadded:: 3.15
 
+   .. versionchanged:: next
+      Single-character final groups are now always rejected as encoding
+      violations. Previously they were silently ignored, producing no output
+      bytes.
+
 
 .. function:: b2a_ascii85(data, /, *, foldspaces=False, wrapcol=0, pad=False, adobe=False)
 
@@ -160,26 +175,36 @@ The :mod:`!binascii` module defines the following functions:
    .. versionadded:: 3.15
 
 
-.. function:: a2b_base85(string, /, *, alphabet=BASE85_ALPHABET, ignorechars=b'')
+.. function:: a2b_base85(string, /, *, alphabet=BASE85_ALPHABET, ignorechars=b'', canonical=False)
 
    Convert Base85 data back to binary and return the binary data.
    More than one line may be passed at a time.
 
    Valid Base85 data contains characters from the Base85 alphabet in groups
-   of five (except for the final group, which may have from two to five
+   of five (except for the final group, which may have from two to four
    characters). Each group encodes 32 bits of binary data in the range from
-   ``0`` to ``2 ** 32 - 1``, inclusive.
+   ``0`` to ``2 ** 32 - 1``, inclusive. A single-character final group is
+   always rejected as an encoding violation.
 
    Optional *alphabet* must be a :class:`bytes` object of length 85 which
    specifies an alternative alphabet.
 
    *ignorechars* should be a :term:`bytes-like object` containing characters
    to ignore from the input.
 
+   If *canonical* is true, non-canonical encodings in partial final groups
+   are rejected with :exc:`binascii.Error`.  This enforces that the padding
+   digits match what the encoder would produce.
+
    Invalid Base85 data will raise :exc:`binascii.Error`.
 
    .. versionadded:: 3.15
 
+   .. versionchanged:: next
+      Single-character final groups are now always rejected as encoding
+      violations. Previously they were silently ignored, producing no output
+      bytes.
+
 
 .. function:: b2a_base85(data, /, *, alphabet=BASE85_ALPHABET, wrapcol=0, pad=False)
 
@@ -199,7 +224,7 @@ The :mod:`!binascii` module defines the following functions:
    .. versionadded:: 3.15
 
 
-.. function:: a2b_base32(string, /, *, padded=True, alphabet=BASE32_ALPHABET, ignorechars=b'')
+.. function:: a2b_base32(string, /, *, padded=True, alphabet=BASE32_ALPHABET, ignorechars=b'', canonical=False)
 
    Convert base32 data back to binary and return the binary data.
 
@@ -228,6 +253,10 @@ The :mod:`!binascii` module defines the following functions:
    presented before the end of the encoded data and the excess pad characters
    will be ignored.
 
+   If *canonical* is true, non-zero padding bits in the last group are rejected
+   with :exc:`binascii.Error`, enforcing canonical encoding as defined in
+   :rfc:`4648` section 3.5.
+
    Invalid base32 data will raise :exc:`binascii.Error`.
 
    .. versionadded:: next

Include/internal/pycore_global_strings.h

@@ -359,6 +359,7 @@ struct _Py_global_strings {
         STRUCT_FOR_ID(callable)
         STRUCT_FOR_ID(callback)
         STRUCT_FOR_ID(cancel)
+        STRUCT_FOR_ID(canonical)
         STRUCT_FOR_ID(capath)
         STRUCT_FOR_ID(capitals)
         STRUCT_FOR_ID(category)