JSONEncoder with ensure_ascii=True incorrectly escapes DEL character (U+007F)

[T90REAL]

Bug report

Bug description:

There is an inconsistency in Python's json module between py_encode_basestring and JSONEncoder.encode() when handling the DEL character (U+007F, \x7f) with ensure_ascii=True.
The DEL character (ASCII code 127) is being incorrectly escaped as \u007f by JSONEncoder.encode() when ensure_ascii=True, while py_encode_basestring() correctly outputs it as a literal character.

from json.encoder import py_encode_basestring, JSONEncoder

s = '\x7f'
encoder = JSONEncoder(ensure_ascii=True)
expected = encoder.encode(s)
actual = py_encode_basestring(s)

assert actual == expected, f"py_encode_basestring({s!r}) = {actual!r}, but JSONEncoder().encode({s!r}) = {expected!r}"

Traceback (most recent call last):
  File "/data/test.py", line 10, in <module>
    assert actual == expected, f"py_encode_basestring({s!r}) = {actual!r}, but JSONEncoder().encode({s!r}) = {expected!r}"
           ^^^^^^^^^^^^^^^^^^
AssertionError: py_encode_basestring('\x7f') = '"\x7f"', but JSONEncoder().encode('\x7f') = '"\\u007f"'

CPython versions tested on:

3.12

Operating systems tested on:

Linux

Linked PRs

• gh-140793: Make \x7f handling consistent across JSON #140794
• gh-140793: Improve documentation for the ensure_ascii option in the json module #140906
• [3.14] gh-140793: Improve documentatation and tests for the ensure_ascii option in the json module (GH-140906) #141227
• [3.13] gh-140793: Improve documentatation and tests for the ensure_ascii option in the json module (GH-140906) #141228

[serhiy-storchaka]

The purpose of the ensure_ascii option is to make the output encoding-independent. It is compatible with any ASCII-compatible encoding -- UTF-8, most of 8-bit encodings and many multibyte encodings.

But there is other purpose. As well as repr(), it escapes control and invisible characters, so nothing can hide from human eyes. The DEL character is invisible on many output devices, it is the only invisible ASCII character which is not required to be escaped in JSON by RFC 8259. So ensure_ascii=True intentionally escapes it too.

This is rather a documentation issue.

[nineteendo]

You're calling the wrong function...

>>> from json.encoder import py_encode_basestring_ascii
>>> print(py_encode_basestring_ascii("\x7f"))
"\u007f"

If you want to test the Python implementation:

>>> import sys
>>> sys.modules["_json"] = None
>>> import json
>>> print(json.dumps("\x7f", ensure_ascii=True))
"\u007f"