Skip to content

Docs and TypedDicts for tool.pdm table and subtables#3784

Open
sneakers-the-rat wants to merge 6 commits into
pdm-project:mainfrom
sneakers-the-rat:docs-tool-pdm
Open

Docs and TypedDicts for tool.pdm table and subtables#3784
sneakers-the-rat wants to merge 6 commits into
pdm-project:mainfrom
sneakers-the-rat:docs-tool-pdm

Conversation

@sneakers-the-rat
Copy link
Copy Markdown
Contributor

@sneakers-the-rat sneakers-the-rat commented May 9, 2026

Pull Request Checklist

  • A news fragment is added in news/ describing what is new.
  • Test cases added for changed code - none needed, as far as i'm aware

Describe what you have changed in this PR.

Fix: #3783

(raising this as a draft but i have to run right now and can't do a full rundown just yet, wanted to open to see if you approved of the general approach)

So i figured out how to get mkdocs to do a summary-table style like what is done in the other refrence docs pages, but it is a bit of a hack. the big thing that is missing here is that functional typeddicts get rendered like garbage, so i think i'll need to write a small extension for that, if you would accept that ( opened a discussion: mkdocstrings/python#329 )

Screenshot 2026-05-08 at 7 07 29 PM

Getting that summary form with the correct TOC headings also required doing manual autodoc invocations for each class, which isn't so bad, but it does risk getting stale if there are a bunch of new tables, but not that bad.

I went through the docs and source to find all mentions of [tool.pdm] entries i could find, and added some docstrings as examples, but if we generally like this approach then i can go through and fill those in more completely (or you can do that, no preference).

Feel free to edit anything, if i've gotten something wrong, if you don't like my working, etc. i'm not precious - these are your docs, just trying to help out!

will be back later for more complete comment and to figure out an approach for the functional typed dicts.

todo

  • python 3.9 compat - use Union
  • extension to document functional typed dicts
  • ... will fill in later

Comment thread src/pdm/project/project_file.py Outdated
"""See [pdm-scripts][pdm-scripts]"""
source: list[SourceTable]
"""Repositories where packages may be found"""
version: VersionTable
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do not recommend adding fields that are not used by this project to the schema, since they may be changed by the consumers.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are you referring to the version table? I am not sure what you mean 'not used by this project' - do you mean that version is interpreted by pdm-backend and not pdm and should go there? or what do you mean?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, if this table is not directly used by PDM itself, it's better to no include it here.

@codecov
Copy link
Copy Markdown

codecov Bot commented May 9, 2026

Codecov Report

❌ Patch coverage is 98.55072% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 86.31%. Comparing base (b059077) to head (b0a0f32).

Files with missing lines Patch % Lines
src/pdm/project/core.py 80.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3784      +/-   ##
==========================================
+ Coverage   86.25%   86.31%   +0.06%     
==========================================
  Files         118      118              
  Lines       12432    12492      +60     
  Branches     2082     2082              
==========================================
+ Hits        10723    10783      +60     
  Misses       1139     1139              
  Partials      570      570              
Flag Coverage Δ
unittests 86.17% <98.55%> (+0.06%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@sneakers-the-rat
Copy link
Copy Markdown
Contributor Author

there we go! look at that, so nice: https://python-pdm--3784.org.readthedocs.build/en/3784/reference/tool/

got the mkdocs transformation of functional TypedDicts -> class TypedDicts working with the dict keys that can't be python names:

Screenshot 2026-05-08 at 11 49 59 PM

will annotate a few choices here but then i think this is ready for review - basically up to you on what fields need docstrings, what should go in them, etc. feel free to edit anything or ask further questions

Comment thread docs/_ext/typeddict.py
Comment on lines +62 to +66
for k, v in mod.members.items()
if isinstance(v, Attribute)
and "module-attribute" in v.labels
and isinstance(v.value, ExprCall)
and v.value.function.name == "TypedDict"
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

intercepting at the module level means that the members are pretty polytyped, so these checks are both to filter to only functional typeddicts and also to avoid attribute errors/satisfy mypy/etc.

Comment thread docs/_ext/typeddict.py
mod.members.update(classes)
return mod

def _extract_comment_docstring(self, key: str, td: Attribute) -> Docstring | None:
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wrote this not to be super general, assuming we are defining things in a standard way that should be enforced by the linter (fields and comments on newlines, etc.)

the failure mode should be missing docstrings rather than failing building if any errors should occur, but it looks like it passes through all the markdown just fine

Comment thread docs/reference/tool.md
Comment on lines +3 to +8
<style>
/* Temporary hack pending guidance to show compact summary tables without double-displaying each attribute */
.doc-class .doc-children {
display: none;
}
</style>
Copy link
Copy Markdown
Contributor Author

@sneakers-the-rat sneakers-the-rat May 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so this is the remaining hack that i'm not sure about - the purpose is to make it so we just get the summary tables but don't get the larger attribute listings. i assumed we would like the former because it's more in the style of the other docs, but reverting this back to just being normal attribute listings wouldn't be hard. one bonus is that the summary tables link out to the types.

both a) linking to types from attribute annotations and b) displaying summary tables without attribute listings are dropped features during the transition into zensical:

I am not sure if it would be possible for me to hack this together with the extension, since the summary table is generated from the members, so while inline CSS is ugly it is scoped to just this one page and it does do the trick.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could use custom mkdocstrings-python templates to avoid rendering the attributes (while still displaying the summary table). This way, no objects inventory pollution and no CSS needed.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a way to control which template is used on a per-object/per-module level? i imagine we might want to control omitting the attrs/members in some cases and not others. it seems like that issue 139 might be the fix that's needed upstream, it does feel like something that should have an intuitive control - "show me summary, attrs, both, or neither" when currently only "show me summary+attrs, attrs, or niether" is possible.

Copy link
Copy Markdown
Contributor

@pawamoy pawamoy May 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a way to control which template is used on a per-object/per-module level?

You can override, say, the base children template, in which you check each attribute metadata:

{% if not attr.extra.your_typeddict_extension_name.dont_render %}
    ...  {# proceed #}
{% endif %}

See https://mkdocstrings.github.io/griffe/guide/users/extending/?h=extra#extra-data.

I don't think the children template uses Jinja blocks though so it could mean a big copy-paste 🤔

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Heh, maybe just override the base attribute template and check the metadata to decide whether to render anything at all. This one has Jinja blocks, much easier.

Copy link
Copy Markdown
Contributor

@pawamoy pawamoy May 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another idea: set attr.extra["mkdocstrings"]["template"] = "empty.html.jinja". This needs either an entry-point (see https://mkdocstrings.github.io/usage/handlers/#handler-extensions), or you can just add this empty template to the folder pointed at by the custom_templates setting (see https://mkdocstrings.github.io/usage/theming/?h=custom_#templates).

Copy link
Copy Markdown
Contributor Author

@sneakers-the-rat sneakers-the-rat May 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for the tips! i'll wait to see what the maintainers here say to see what they think is best.

separately - since this seems like a pretty straightforward need that would probably come up pretty often (this combination of typed dict user-facing documentation and wanting to toggle between an abbreviated form for things where it makes sense vs. an expanded form when more documentation is needed. i mean if mkdocs did this more elegantly than sphinx i would switch over) would you be interested in a PR upstream to handle this? imo it seems like a three way switch: summary is "off/only/both" but i could also do the suggestion in your issue 139 et al of having it be a separate bool flag

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Happy to help!

mkdocstrings and handlers are in maintenance mode so we won't implement new features there.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ig idk where to contribute then! lmk where if anywhere. thanks again.


def get_message(self) -> str:
package_type = self.project.pyproject.settings["package-type"]
package_type = self.project.pyproject.settings["package-type"] # type: ignore[typeddict-item]
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ignored these because this is a deprecated key and it probably doesn't go in the docs

Comment thread src/pdm/project/core.py
result: dict[str, RepositoryConfig] = {}
for source in self.pyproject.settings.get("source", []):
result[source["name"]] = RepositoryConfig(**source, config_prefix="pypi")
for project_source in self.pyproject.settings.get("source", []):
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mypy doesn't handle changing the type of a same-named variable, so changing the name is just for satisfying mypy

Comment thread src/pdm/project/core.py
"""
try:
return reduce(operator.getitem, key.split("."), self.pyproject.settings)
return reduce(operator.getitem, key.split("."), self.pyproject.settings) # type: ignore[arg-type]
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i am sort of a python type system head and i genuinely don't know how one would correct this mypy error:

Argument 1 to "reduce" has incompatible type overloaded function; expected "Callable[[ToolPDMTable, str], ToolPDMTable]"

something with the generic types in reduce but also it's saying that the list[str] part in arg 1 is the problem? no idea. it would probably be cleanest to just remove this method and directly access the keys, but i didn't want to touch the rest of the code outside the scope of the PR.

"""


class OptionsTable(TypedDict, total=False):
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this might not need a typeddict, or there might be some way to generate it dynamically, so this is just for completeness sake but could be removed/replaced with something that doesn't hardcode the command names like this.

Comment thread mkdocs.yml
custom_dir: docs/overrides

plugins:
- autorefs
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

needed this to be able to refer to other parts of the docs from the docstrings like [Display Text][reference-to-anchor] - less fragile than direct URL references because they work locally and also the build process can warn if the link breaks.

@sneakers-the-rat sneakers-the-rat marked this pull request as ready for review May 9, 2026 07:08
@sneakers-the-rat
Copy link
Copy Markdown
Contributor Author

I'll pitch upstream mkdocstrings-python on adding this as a builtin extension so we can take it out of pdm's responsibility

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Document possible values of [tool.pdm] table in one place

3 participants