-
-
Notifications
You must be signed in to change notification settings - Fork 1.7k
PEP 821: Support for unpacking TypedDicts in Callable type hints #4764
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| @@ -0,0 +1,358 @@ | ||||||||||
| PEP: 821 | ||||||||||
| Title: Support for unpacking TypedDicts in Callable type hints | ||||||||||
| Author: Daniel Sperber <github.blurry@9ox.net> | ||||||||||
| Sponsor: Pending | ||||||||||
| Discussions-To: Pending | ||||||||||
| Status: Draft | ||||||||||
| Type: Standards Track | ||||||||||
| Topic: Typing | ||||||||||
| Created: 30-Dec-2025 | ||||||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Date of PEP number assingment:
Suggested change
|
||||||||||
| Python-Version: 3.15 | ||||||||||
| Post-History: `28-Jun-2025 <https://discuss.python.org/t/pep-idea-extend-spec-of-callable-to-accept-unpacked-typedicts-to-specify-keyword-only-parameters/96975>`__ | ||||||||||
|
|
||||||||||
| Abstract | ||||||||||
| ======== | ||||||||||
|
|
||||||||||
| This PEP proposes allowing ``Unpack[TypedDict]`` as the parameter list inside | ||||||||||
| ``Callable``, enabling concise and type-safe ways to describe keyword-only | ||||||||||
| callable signatures. Currently, ``Callable`` assumes positional-only | ||||||||||
| parameters, and typing keyword-only functions requires verbose callback | ||||||||||
| protocols. With this proposal, the keyword structure defined by a ``TypedDict`` | ||||||||||
| can be reused directly in ``Callable``. | ||||||||||
|
|
||||||||||
|
|
||||||||||
| Motivation | ||||||||||
| ========== | ||||||||||
|
|
||||||||||
| The typing specification states: | ||||||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We can link to the spec using intersphinx, something like:
Suggested change
|
||||||||||
|
|
||||||||||
| "Parameters specified using Callable are assumed to be positional-only. | ||||||||||
| The Callable form provides no way to specify keyword-only parameters, | ||||||||||
| variadic parameters, or default argument values. For these use cases, | ||||||||||
| see the section on Callback protocols." | ||||||||||
|
|
||||||||||
| — https://typing.python.org/en/latest/spec/callables.html#callable | ||||||||||
|
|
||||||||||
|
Comment on lines
+34
to
+35
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||||
| This limitation makes it cumbersome to declare callables meant to be invoked | ||||||||||
| with keyword arguments. The existing solution is to define a Protocol:: | ||||||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||||
|
|
||||||||||
| class Signature(TypedDict, closed=True): | ||||||||||
| a: int | ||||||||||
|
|
||||||||||
| class KwCallable(Protocol): | ||||||||||
| def __call__(self, **kwargs: Unpack[Signature]) -> Any: ... | ||||||||||
|
|
||||||||||
| # or | ||||||||||
|
|
||||||||||
| class KwCallable(Protocol): | ||||||||||
| def __call__(self, *, a: int) -> Any: ... | ||||||||||
|
|
||||||||||
| This works but is verbose. The new syntax allows the equivalent to be written | ||||||||||
| more succinctly:: | ||||||||||
|
|
||||||||||
| type KwCallable = Callable[[Unpack[Signature]], Any] | ||||||||||
|
|
||||||||||
|
|
||||||||||
| Rationale | ||||||||||
| ========= | ||||||||||
|
|
||||||||||
| This proposal extends the existing Callable semantics by reusing a ``TypedDict``'s | ||||||||||
| keyed structure for keyword arguments. It avoids verbose Protocol-based | ||||||||||
|
Comment on lines
+59
to
+60
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||||
| callable definitions while remaining compatible with current typing concepts | ||||||||||
| (:pep:`692` Unpack for ``kwargs``, and :pep:`728` ``extra_items``). It preserves backward | ||||||||||
| compatibility by being purely a typing feature. | ||||||||||
|
|
||||||||||
|
|
||||||||||
| Specification | ||||||||||
| ============= | ||||||||||
|
|
||||||||||
| New allowed form | ||||||||||
| ---------------- | ||||||||||
|
|
||||||||||
| It becomes valid to write:: | ||||||||||
|
|
||||||||||
| Callable[[Unpack[TD]], R] | ||||||||||
|
|
||||||||||
| where ``TD`` is a ``TypedDict``. A shorter form is also allowed:: | ||||||||||
|
|
||||||||||
| Callable[Unpack[TD], R] | ||||||||||
|
|
||||||||||
| Additionally, positional parameters may be combined with an unpacked ``TypedDict``:: | ||||||||||
|
|
||||||||||
| Callable[[int, str, Unpack[TD]], R] | ||||||||||
|
|
||||||||||
| Semantics | ||||||||||
| --------- | ||||||||||
|
|
||||||||||
| * Each key in the ``TypedDict`` must be accepted as a keyword parameter. | ||||||||||
| * TypedDict keys cannot be positional-only; they must be valid keyword parameters. | ||||||||||
| * Positional parameters may appear in ``Callable`` before ``Unpack[TD]`` and follow normal ``Callable`` semantics. | ||||||||||
| * Required keys must be accepted, but may correspond to parameters with a | ||||||||||
| default value. | ||||||||||
| * ``NotRequired`` keys must still be accepted, but may be omitted at call sites. | ||||||||||
| This respectively applies to ``TypedDict`` with ``total=False``. | ||||||||||
| * Functions with ``**kwargs`` are compatible if the annotation of ``**kwargs`` | ||||||||||
| matches or is a supertype of the ``TypedDict`` values. | ||||||||||
| * ``extra_items`` from PEP 728 is respected: functions accepting additional | ||||||||||
| ``**kwargs`` are valid if their annotation is compatible with the declared | ||||||||||
| type. | ||||||||||
| * If neither ``extra_items`` nor ``closed`` (PEP 728) is specified on the | ||||||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||||
| ``TypedDict``, additional keyword arguments are implicitly permitted with | ||||||||||
| type ``object`` (i.e., compatible with ``**kwargs: object``). Setting | ||||||||||
| ``closed=True`` forbids any additional keyword arguments beyond the keys | ||||||||||
| declared in the ``TypedDict``. Setting ``extra_items`` to a specific type | ||||||||||
| requires that any additional keyword arguments match that type. | ||||||||||
| * Only a single ``TypedDict`` may be unpacked inside a ``Callable``. Support | ||||||||||
| for multiple unpacks may be considered in the future. | ||||||||||
|
|
||||||||||
| Examples | ||||||||||
| -------- | ||||||||||
|
|
||||||||||
| The following examples illustrate how unpacking a ``TypedDict`` into a | ||||||||||
| ``Callable`` enforces acceptance of specific keyword parameters. A function is | ||||||||||
| compatible if it can be called with the required keywords (even if they are | ||||||||||
| also accepted positionally); positional-only parameters for those keys are | ||||||||||
| rejected. | ||||||||||
|
|
||||||||||
| .. code-block:: python | ||||||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Nit, you used |
||||||||||
| from typing import TypedDict, Callable, Unpack, Any, NotRequired | ||||||||||
| class Signature(TypedDict): | ||||||||||
| a: int | ||||||||||
| type IntKwCallable = Callable[[Unpack[Signature]], Any] | ||||||||||
| def normal(a: int): ... | ||||||||||
| def kw_only(*, a: int): ... | ||||||||||
| def pos_only(a: int, /): ... | ||||||||||
| def different(bar: int): ... | ||||||||||
| f1: IntKwCallable = normal # Accepted | ||||||||||
| f2: IntKwCallable = kw_only # Accepted | ||||||||||
| f3: IntKwCallable = pos_only # Rejected | ||||||||||
| f4: IntKwCallable = different # Rejected | ||||||||||
| Optional arguments | ||||||||||
| ------------------ | ||||||||||
|
|
||||||||||
| Keys marked ``NotRequired`` in the ``TypedDict`` correspond to optional | ||||||||||
| keyword arguments. | ||||||||||
| Meaning the callable must accept them, but callers may omit them. | ||||||||||
| Functions that accept the keyword argument must also provide a default value that is compatible; | ||||||||||
| functions that omit the parameter entirely are rejected. | ||||||||||
|
|
||||||||||
| .. code-block:: python | ||||||||||
| class OptSig(TypedDict): | ||||||||||
| a: NotRequired[int] | ||||||||||
| type OptCallable = Callable[[Unpack[OptSig]], Any] | ||||||||||
| def defaulted(a: int = 1): ... | ||||||||||
| def kw_default(*, a: int = 1): ... | ||||||||||
| def no_params(): ... | ||||||||||
| def required(a: int): ... | ||||||||||
| g1: OptCallable = defaulted # Accepted | ||||||||||
| g2: OptCallable = kw_default # Accepted | ||||||||||
| g3: OptCallable = no_params # Rejected | ||||||||||
| g4: OptCallable = required # Rejected | ||||||||||
| Additional keyword arguments | ||||||||||
| ---------------------------- | ||||||||||
|
|
||||||||||
| Default Behavior (no ``extra_items`` or ``closed``) | ||||||||||
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||||||||||
|
|
||||||||||
| If the ``TypedDict`` does not specify ``extra_items`` or ``closed``, additional | ||||||||||
| keyword arguments are permitted with type ``object``. This is the default behavior. | ||||||||||
|
|
||||||||||
| .. code-block:: python | ||||||||||
| # implies extra_items=object | ||||||||||
| class DefaultTD(TypedDict): | ||||||||||
| a: int | ||||||||||
| type DefaultCallable = Callable[[Unpack[DefaultTD]], Any] | ||||||||||
| def v_any(**kwargs: object): ... | ||||||||||
| def v_ints(a: int, b: int=2): ... | ||||||||||
| d1: DefaultCallable = v_any # Accepted (implicit object for extras) | ||||||||||
| d1(a=1, c="more") # Accepted (extras allowed) | ||||||||||
| d2: DefaultCallable = v_ints # Rejected (b: int is not a supertype of object) | ||||||||||
| ``closed`` behavior (PEP 728) | ||||||||||
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||||||||||
|
|
||||||||||
| If ``closed=True`` is specified on the ``TypedDict``, no additional keyword | ||||||||||
| arguments beyond those declared are expected. | ||||||||||
|
|
||||||||||
| .. code-block:: python | ||||||||||
| class ClosedTD(TypedDict, closed=True): | ||||||||||
| a: int | ||||||||||
| type ClosedCallable = Callable[[Unpack[ClosedTD]], Any] | ||||||||||
| def v_any(**kwargs: object): ... | ||||||||||
| def v_ints(a: int, b: int=2): ... | ||||||||||
| c1: ClosedCallable = v_any # Accepted | ||||||||||
| c1(a=1, c="more") # Rejected (extra c not allowed) | ||||||||||
| c2: ClosedCallable = v_ints # Accepted | ||||||||||
| c2(a=1, b=2) # Rejected (extra b not allowed) | ||||||||||
| Interaction with ``extra_items`` (PEP 728) | ||||||||||
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||||||||||
|
|
||||||||||
| If a ``TypedDict`` specifies the ``extra_items`` parameter (with the exemption of ``extra_items=Never``), the corresponding ``Callable`` | ||||||||||
| must accept additional keyword arguments of the specified type. | ||||||||||
|
|
||||||||||
| For example: | ||||||||||
|
|
||||||||||
| .. code-block:: python | ||||||||||
| class ExtraTD(TypedDict, extra_items=str): | ||||||||||
| a: int | ||||||||||
| type ExtraCallable = Callable[[Unpack[ExtraTD]], Any] | ||||||||||
| def accepts_str(**kwargs: str): ... | ||||||||||
| def accepts_object(**kwargs: object): ... | ||||||||||
| def accepts_int(**kwargs: int): ... | ||||||||||
| e1: ExtraCallable = accepts_str # Accepted (matches extra_items type) | ||||||||||
| e2: ExtraCallable = accepts_object # Accepted (object is a supertype of str) | ||||||||||
| e3: ExtraCallable = accepts_int # Rejected (int is not a supertype of str) | ||||||||||
| e1(a=1, b="foo") # Accepted | ||||||||||
| e1(a=1, b=2) # Rejected (b must be str) | ||||||||||
| Interaction with ``ParamSpec`` and ``Concatenate`` | ||||||||||
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||||||||||
|
|
||||||||||
| A ``ParamSpec`` can be substituted by ``Unpack[TypedDict]`` to define a | ||||||||||
| parameterized callable alias. Substituting ``Unpack[Signature]`` produces the | ||||||||||
| same effect as writing the callable with an unpacked ``TypedDict`` directly. | ||||||||||
| Using a ``TypedDict`` within ``Concatenate`` is not allowed. | ||||||||||
|
|
||||||||||
| .. code-block:: python | ||||||||||
| from typing import ParamSpec | ||||||||||
| P = ParamSpec("P") | ||||||||||
| type CallableP = Callable[P, Any] | ||||||||||
| # CallableP[Unpack[Signature]] is equivalent to Callable[[Unpack[Signature]], Any] | ||||||||||
| h: CallableP[Unpack[Signature]] = normal # Accepted | ||||||||||
| h2: CallableP[Unpack[Signature]] = kw_only # Accepted | ||||||||||
| h3: CallableP[Unpack[Signature]] = pos_only # Rejected | ||||||||||
| Combined positional parameters and ``Unpack``: | ||||||||||
|
|
||||||||||
| Positional parameters may precede an unpacked ``TypedDict`` inside ``Callable``. | ||||||||||
| Functions that accept the required positional arguments and can be called with | ||||||||||
| the specified keyword(s) are compatible; making the keyword positional-only is | ||||||||||
| rejected. | ||||||||||
|
|
||||||||||
| .. code-block:: python | ||||||||||
| from typing import TypedDict, Callable, Unpack, Any | ||||||||||
| class Signature(TypedDict): | ||||||||||
| a: int | ||||||||||
| type IntKwPosCallable = Callable[[int, str, Unpack[Signature]], Any] | ||||||||||
| def mixed_kwonly(x: int, y: str, *, a: int): ... | ||||||||||
| def mixed_poskw(x: int, y: str, a: int): ... | ||||||||||
| def mixed_posonly(x: int, y: str, a: int, /): ... | ||||||||||
| m1: IntKwPosCallable = mixed_kwonly # Accepted | ||||||||||
| m2: IntKwPosCallable = mixed_poskw # Accepted | ||||||||||
| m3: IntKwPosCallable = mixed_posonly # Rejected | ||||||||||
| Inline TypedDicts (PEP 764): | ||||||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||||
|
|
||||||||||
| Inline ``TypedDict`` forms are supported like any other ``TypedDict``, allowing compact definitions when the | ||||||||||
| structure is used only once. | ||||||||||
|
|
||||||||||
| .. code-block:: python | ||||||||||
| Callable[[Unpack[TypedDict({"a": int})]], Any] | ||||||||||
| Backwards Compatibility | ||||||||||
| ======================= | ||||||||||
|
|
||||||||||
| This feature is additive. Existing code is unaffected. Runtime behavior does | ||||||||||
| not change; this is a typing-only feature. | ||||||||||
|
|
||||||||||
| How to Teach This | ||||||||||
| ================= | ||||||||||
|
|
||||||||||
| This feature is a shorthand for Protocol-based callbacks. Users should be | ||||||||||
| taught that with | ||||||||||
|
|
||||||||||
| .. code-block:: python | ||||||||||
| class Signature(TypedDict): | ||||||||||
| a: int | ||||||||||
| b: NotRequired[str] | ||||||||||
| * ``Callable[[Unpack[Signature]], R]`` is equivalent to defining a Protocol with | ||||||||||
| ``__call__(self, **kwargs: Unpack[Signature]) -> R`` | ||||||||||
| or ``__call__(self, a: int, b: str = ..., **kwargs: object) -> R``. | ||||||||||
| * The implicit addition of ``**kwargs: object`` might come surprising to users, | ||||||||||
| using ``closed=True`` for definitions will create the more intuitive equivalence | ||||||||||
| of ``__call__(self, a: int, b: str = ...) -> R`` | ||||||||||
|
|
||||||||||
|
|
||||||||||
| Reference Implementation | ||||||||||
| ======================== | ||||||||||
|
|
||||||||||
| A prototype exists in mypy: | ||||||||||
| https://github.com/python/mypy/pull/16083 | ||||||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||||
|
|
||||||||||
|
|
||||||||||
| Rejected Ideas | ||||||||||
| ============== | ||||||||||
|
|
||||||||||
| - Combining ``Unpack[TD]`` with ``Concatenate``. With such support, one could write | ||||||||||
| ``Callable[Concatenate[int, Unpack[TD], P], R]`` which in turn would allow a keyword-only parameter between ``*args`` and ``**kwargs``, i.e. | ||||||||||
| ``def func(*args: Any, a: int, **kwargs: Any) -> R: ...`` which is currently not allowed per :pep:`612`. | ||||||||||
| To keep the initial implementation simple, this PEP does not propose such | ||||||||||
| support. | ||||||||||
|
|
||||||||||
| Open Questions | ||||||||||
| ============== | ||||||||||
|
|
||||||||||
| * Should multiple ``TypedDict`` unpacks be allowed to form a union, and if so, how to handle | ||||||||||
| overlapping keys of non-identical types? Which restrictions should apply in such a case? Should the order matter? | ||||||||||
| * Is there a necessity to differentiate between normal and ``ReadOnly`` keys? | ||||||||||
| * Is it necessary to specify generic behavior for ``TypedDict`` and the resulting ``Callable`` when the ``TypedDict`` itself is generic? | ||||||||||
|
|
||||||||||
|
|
||||||||||
| Acknowledgements | ||||||||||
| ================ | ||||||||||
| TODO | ||||||||||
|
|
||||||||||
|
|
||||||||||
|
|
||||||||||
| References | ||||||||||
| ========== | ||||||||||
|
|
||||||||||
| * :pep:`692` - Using ``Unpack`` with ``**kwargs`` | ||||||||||
| * :pep:`728` - ``extra_items`` in TypedDict | ||||||||||
| * :pep:`764` - Inline TypedDict | ||||||||||
|
Comment on lines
+349
to
+350
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||||
| * `mypy PR #16083 - Prototype support <https://github.com/python/mypy/pull/16083>`__ | ||||||||||
| * Revisiting PEP 677 (`discussion thread <https://discuss.python.org/t/pep-677-with-an-easier-to-parse-and-more-expressive-syntax/98408/33>`__) | ||||||||||
|
|
||||||||||
|
|
||||||||||
| Copyright | ||||||||||
| ========= | ||||||||||
|
|
||||||||||
| This document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive. | ||||||||||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
CODEOWNERS can only list those with write permission, so put Jelle here: