Skip to content

Deduplicate HTTP route index entries across netapi doc pages (#69724)#69725

Open
ggiesen wants to merge 3 commits into
saltstack:3006.xfrom
ggiesen:fix-docs-dup-http
Open

Deduplicate HTTP route index entries across netapi doc pages (#69724)#69725
ggiesen wants to merge 3 commits into
saltstack:3006.xfrom
ggiesen:fix-docs-dup-http

Conversation

@ggiesen

@ggiesen ggiesen commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

Makes the -W docs builds deterministic by removing duplicate HTTP route registrations across the netapi doc pages. The rest_cherrypy, rest_tornado and rest_wsgi pages document overlapping routes (9 shared between cherrypy and tornado, plus POST / also in wsgi); each shared route therefore registered several httpdomain index entries. sphinxcontrib-httpdomain (2.0.0 as pinned) only detects duplicates in merge_domaindata, which runs when sphinx -j parallel reader chunks are merged, so tools docs man/html (SPHINXOPTS=-W -j auto --keep-going) fail intermittently depending on where the chunk boundary happens to land:

WARNING: duplicate HTTP post method definition / in doc/ref/netapi/all/salt.netapi.rest_tornado.rst, other instance is in doc/ref/netapi/all/salt.netapi.rest_wsgi.rst

When it hits in Prepare Release, the release patch artifact is never produced and every downstream job in the run (Documentation, Source Tarball, all package/onedir builds) dies with "Artifact not found" -- one flaky warning takes out the whole run.

This PR adds :noindex: to the tornado and wsgi copies of the shared routes, keeping rest_cherrypy as the single indexed, canonical instance of each. :noindex: directives never enter the domain data, so the merge collision becomes impossible under any chunking or worker count. As a side benefit the HTTP routing index becomes deterministic; previously shared routes pointed at whichever page the readers processed last.

What issues does this PR fix or reference?

Fixes #69724

Previous Behavior

-W -j auto docs builds failed intermittently with duplicate HTTP method definition warnings, cascading into full-run artifact failures.

New Behavior

Every HTTP route registers exactly once. Verified with the pinned docs.lock (sphinx 7.0.1, httpdomain 2.0.0): the pickled build environment previously showed shared routes registered by rest_tornado/rest_wsgi; with this change every netapi route is registered once, by rest_cherrypy. make man and make html with -W -j auto --keep-going build clean (repeated across several -j values); the tornado and wsgi pages still render their full endpoint documentation, and there are no :http:<method>: cross-references in the tree that could be affected.

Merge requirements satisfied?

  • Changelog

Docs-only change; no runtime code paths affected, so no unit tests apply.

Commits signed with GPG?

No

Notes from validation

  • The :noindex: lines are indented to the directive content column (not the option column). With the shallower indent, docutils dedents the block by the option line and the directive content nested-parses as a block quote, which silently disables the doc-field transformation (Status Codes/Request Headers grouping and links) on those endpoints while -W stays green. Rendering was verified byte-equivalent to the pre-change pages with the content-column indent.
  • :noindex: alone would also drop the HTML anchors/permalinks on the tornado/wsgi signatures, because Sphinx skips add_target_and_index entirely under noindex -- even though httpdomain itself only gates the global registration behind the option and always appends the anchor. Since anchors are per-page HTML ids that cannot collide across pages (only the global domain data can produce the parallel-merge warning), the second commit adds a small doc/_ext/salthttpanchors.py extension that hides the option from Sphinx's outer gate and re-presents it to httpdomain's inner gate: the anchors and permalinks stay (existing deep links into those pages keep working), while the routes stay out of the domain data. Verified in the built HTML: all 10 tornado anchors (#post--login, #get--events, ...) and 28 permalinks present, domain data still registered solely by the cherrypy page.
  • Man pages are byte-identical before/after; html/man build clean under -W with -j auto, -j 2, and -j 4.

@ggiesen ggiesen requested a review from a team as a code owner July 6, 2026 01:23
The rest_cherrypy, rest_tornado and rest_wsgi doc pages document
overlapping HTTP routes, so each shared route registered multiple
httpdomain index entries. sphinxcontrib-httpdomain only detects the
duplicates in merge_domaindata, which runs when sphinx -j parallel
reader chunks are merged, so the -W builds in tools docs (Prepare
Release, Documentation) fail intermittently depending on where the
chunk boundary lands:

    WARNING: duplicate HTTP post method definition / in
    doc/ref/netapi/all/salt.netapi.rest_tornado.rst, other instance
    is in doc/ref/netapi/all/salt.netapi.rest_wsgi.rst

When it hits, the release patch artifact is never produced and every
downstream build job in the run fails with 'Artifact not found'.

Mark the tornado and wsgi copies of the shared routes with :noindex:
so each route is registered exactly once, by the canonical
rest_cherrypy reference. noindex'd directives never enter the domain
data, so the merge collision is impossible under any chunking. Page
content is unchanged and no :http: cross-references exist that could
be affected. This also makes the HTTP routing index deterministic; it
previously pointed at whichever page the readers processed last.

Fixes saltstack#69724
sphinxcontrib-httpdomain's add_target_and_index always appends the
signature anchor and only gates the global route registration behind
:noindex:, but Sphinx's ObjectDescription.run skips the whole method
when noindex is set, so the previous commit's dedup also dropped the
per-page anchors and permalinks from the rest_tornado and rest_wsgi
endpoint signatures. Anchors are per-document HTML ids and cannot
collide across pages; only the global registration can produce the
parallel-merge duplicate warnings.

Add a small extension that hides the noindex option from Sphinx's
outer gate and re-presents it to httpdomain's inner gate, restoring
the anchors and permalinks (existing deep links into those pages keep
working) while the routes stay out of the domain data, so the
duplicate-route collision remains impossible under any chunking.
@ggiesen

ggiesen commented Jul 6, 2026

Copy link
Copy Markdown
Contributor Author

The behaviour the doc/_ext/salthttpanchors.py extension works around has been proposed as a fix upstream (sphinx-contrib/httpdomain#112, issue sphinx-contrib/httpdomain#111). If that lands and the docs.lock pin is eventually bumped past it, the extension can simply be deleted; nothing else depends on it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

test:full Run the full test suite

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants