Skip to content

feat: move to mystmd#792

Merged
henryiii merged 25 commits into
mainfrom
henryiii/feat/mystmd
Jun 9, 2026
Merged

feat: move to mystmd#792
henryiii merged 25 commits into
mainfrom
henryiii/feat/mystmd

Conversation

@henryiii

@henryiii henryiii commented May 27, 2026

Copy link
Copy Markdown
Collaborator

Experimenting with a MystMD version.

Used CopilotCLI:claude-sonnet-4.6.

Moved prettier -> rumdl, works nicely with myst. Fewer ignores.

Some remaining issues:

  • Coverage could use tab sync (mystmd feature) to keep tabs in sync (also documentation page for mkdocs / sphinx)
  • Some {% commands still show up in output.

And I'm not sure these are solvable:

  • No TOML highlighting - we have a lot of TOML. Looks like mystmd uses a really old highlighter
  • I'd like a three-way toggle (auto/light/dark), not just (light/dark)
  • Very plain and empty, maybe we could theme a bit?
  • Repo-review is an iframe, seems you can't easily put in custom html? Probably solvable without an iframe.

Positives:

  • Lots of built-in functionality we can just use (tabs, admonitions, etc)
  • Can adopt some new functionality later, too (glossary, etc)
  • Wide / mid screen layout is good, header stays on top in narrow layout
  • Pretty fast, bun serve from docs folder.

@henryiii henryiii force-pushed the henryiii/feat/mystmd branch 2 times, most recently from 5ebe005 to f94e6ed Compare May 27, 2026 21:19
@henryiii henryiii requested a review from Copilot May 28, 2026 16:26

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR migrates the documentation site from the previous Jekyll/Ruby setup to a MystMD (MyST) build, updating pages to MyST-native directives (tabs/admonitions/includes), replacing the interactive repo-review embed, and switching Markdown formatting from Prettier to rumdl (MyST flavor).

Changes:

  • Replace Jekyll/Ruby docs tooling with MystMD config, Node/Bun scaffolding, and updated RTD build commands.
  • Convert docs content to MyST-native constructs (tab-sets, admonitions, includes, internal links) and add a custom {rr} role plugin.
  • Adopt rumdl for Markdown formatting/linting and update related ignore/disable markers.

Reviewed changes

Copilot reviewed 60 out of 63 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
src/sp_repo_review/checks/ruff.py Adjusts fenced-code language tags in check guidance text.
src/sp_repo_review/checks/pyproject.py Adjusts fenced-code language tags in check guidance text.
src/sp_repo_review/checks/noxfile.py Adjusts fenced-code language tags in check guidance text.
src/sp_repo_review/checks/mypy.py Adjusts fenced-code language tags in check guidance text.
README.md Updates Markdown formatting/ignore markers and minor heading/list formatting.
pyproject.toml Adds [tool.rumdl] configuration for MyST-flavored Markdown lint/format.
package.json Adds MystMD dependency and build/serve scripts for docs.
helpers/fetch_repo_review_app.sh Removes helper used to fetch the legacy repo-review JS bundle.
helpers/cog_helpers.py Updates code-fence helper to use rumdl markers and TOML→INI highlighting workaround.
Gemfile.lock Removes Ruby dependency lockfile (Jekyll toolchain removal).
Gemfile Removes Ruby/Jekyll dependency definitions.
docs/rr-role.mjs Adds Myst plugin for {rr} inline role used in docs.
docs/pages/tutorials/test.md Converts Jekyll frontmatter/TOC and links to MyST-native structure.
docs/pages/tutorials/packaging.md Converts Jekyll frontmatter/TOC and links to MyST-native structure.
docs/pages/tutorials/module.md Converts Jekyll frontmatter/TOC and links to MyST-native structure.
docs/pages/tutorials/index.md Converts Jekyll frontmatter and adds MyST table-of-contents directive.
docs/pages/tutorials/docs.md Converts headings/directives/links to MyST-native forms.
docs/pages/tutorials/dev-environment.md Replaces Liquid tabs/admonitions and removes tabs.js dependency.
docs/pages/principles/testing.md Converts headings, anchors, admonitions, and links to MyST-native forms.
docs/pages/principles/process.md Converts headings/directives to MyST-native forms.
docs/pages/principles/index.md Converts links and adds MyST table-of-contents directive.
docs/pages/principles/design.md Converts admonitions and removes legacy prettier ignore markers.
docs/pages/patterns/index.md Converts links and adds MyST table-of-contents directive.
docs/pages/patterns/exports.md Converts admonitions/headings to MyST-native forms.
docs/pages/patterns/data_files.md Converts details blocks to MyST dropdown directives and updates formatting.
docs/pages/patterns/backports.md Converts headings and fenced-code language tags for highlighting.
docs/pages/index.md Converts home page content/admonitions/links to MyST-native forms.
docs/pages/guides/tasks.md Converts tabs/admonitions/rr markers and replaces legacy Jekyll constructs.
docs/pages/guides/style.md Large MyST conversion: tabs, dropdowns, admonitions, rr markers, and link updates.
docs/pages/guides/repo_review.md Replaces legacy iframe/include with an {anywidget} embed configuration.
docs/pages/guides/pytest.md Converts tabs/admonitions/headings and updates fenced-code language tags.
docs/pages/guides/packaging_simple.md Converts tabs/admonitions and uses MyST {include} for shared partial content.
docs/pages/guides/packaging_compiled.md Converts tab-sets with :sync: keys and updates code-fence markers.
docs/pages/guides/packaging_classic.md Converts admonitions/headings and link references to MyST-native forms.
docs/pages/guides/mypy.md Converts headings/links to MyST-native forms.
docs/pages/guides/index.md Converts links and adds MyST table-of-contents directive.
docs/pages/guides/gha_wheels.md Converts tabs/admonitions/headings and updates code fences and raw blocks.
docs/pages/guides/gha_pure.md Converts tabs/admonitions and updates links/headings for MyST.
docs/pages/guides/gha_basic.md Converts tabs/admonitions and updates code-fence markers.
docs/pages/guides/docs.md Converts extensive tabs/admonitions/includes to MyST-native directives.
docs/pages/guides/coverage.md Converts tabs/headings and updates coverage guidance structure.
docs/myst.yml Adds MystMD project/site configuration and explicit TOC structure.
docs/config/scientific-python.yml Adds shared MystMD theme options/config extension.
docs/assets/js/tabs.js Removes legacy JS used to implement Jekyll-era tabs.
docs/assets/css/site.css Adds custom CSS overrides for the MystMD theme and {rr} badges.
docs/_sass/custom/custom.scss Removes Jekyll-era SCSS customizations.
docs/_sass/color_schemes/skhep.scss Removes Jekyll-era color scheme SCSS.
docs/_plugins/tabs.rb Removes Jekyll Liquid plugin for tabs.
docs/_plugins/repo_review.rb Removes Jekyll Liquid plugin for repo-review badges.
docs/_plugins/details.rb Removes Jekyll Liquid plugin for details/summary blocks.
docs/_partials/pyproject.md Updates shared partial used via MyST {include} and adjusts formatting.
docs/_includes/toc.html Removes Jekyll include for TOC.
docs/_includes/interactive_repo_review.html Removes Jekyll include for interactive repo-review embed.
docs/_includes/head.html Removes Jekyll custom head include.
docs/_includes/head_custom.html Removes Jekyll custom head include for repo-review assets.
bun.lock Adds Bun lockfile for the MystMD dependency.
AGENTS.md Adds repository agent/contributor guide, including docs tooling notes.
action.yml Removes legacy prettier-ignore comment in the action steps.
.ruby-version Removes Ruby version pin (Jekyll toolchain removal).
.readthedocs.yaml Updates RTD build to MystMD/Node-based commands and output copy.
.pre-commit-config.yaml Switches Markdown formatting from Prettier to rumdl and updates cog target paths.
.gitignore Stops ignoring package.json and keeps Node-related ignores relevant to new tooling.
_config.yml Removes Jekyll site configuration.
Comments suppressed due to low confidence (2)

docs/_partials/pyproject.md:1

  • This file is included into other pages via the {include} directive. Starting the partial with an H1 (# ...) can create multiple top-level headings on the rendered page and distort the ToC. Consider using a lower heading level (e.g. ##/###) or removing the heading and letting the parent page introduce the section.
    docs/_partials/pyproject.md:67
  • Within this included partial, ## License jumps heading levels relative to nearby subsections (e.g. ### Extras, ### Command line). This can produce a confusing outline when included. Consider making this heading consistent with the rest of the partial (likely ### License).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread docs/rr-role.mjs
Comment thread docs/pages/guides/coverage.md Outdated
Comment thread docs/pages/guides/coverage.md Outdated
Comment thread docs/assets/css/site.css
Comment thread AGENTS.md
Comment thread AGENTS.md Outdated

@drammock drammock left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did a quick skim of the rendered output: It looks good! My only substantive UI critiques (neither are blockers) are that:

  1. the main content column is quite narrow on big screens (didn't test small screens)
  2. it's a bit confusing that the left sidebar categories are both links to pages and also expand/collapse toggles. The chevron being a separate button helps, but there may be some risk of folks only clicking the chevron and not realizing that there is a "landing" page for each section. This would be somewhat mitigated if there were next/previous links at page bottom (which you've mentioned already would be nice)

@henryiii

henryiii commented Jun 8, 2026

Copy link
Copy Markdown
Collaborator Author
  1. Not sure I can adjust the width, it is responsive though and will add remove the two sidebars as it changes.
  2. The default in mystmd is to not have a main page, but I don't think missing it is that big of an issue; if someone doesn't know it's there, it's mostly an intro and list of links. And if someone doesn't realize the cheveron is separate, then the page does have a list of links, so they can still find the content.

Side note: if you don't have a section page, the UI still highlights the button and the cheveron differently, as if there was a difference clicking them (there's not in that case).

henryiii and others added 20 commits June 8, 2026 13:50
- Replace Jekyll + just-the-docs with MyST using scientific-python-myst-theme
- Add docs/myst.yml with full TOC structure (29 pages across 5 sections)
- Add docs/config/scientific-python.yml for book-theme configuration
- Add docs/rr-role.mjs custom MyST plugin replacing repo_review.rb Liquid tag
- Add docs/assets/css/site.css with SP theme CSS and .rr-btn styles
- Convert all Liquid tags: {%rr%}→{rr} role, tabs→tab-set/tab-item,
  details→{dropdown}, callouts→admonitions, links→relative paths
- Inline interactive_repo_review.html as raw html block with Pyodide loading
- Move _includes/pyproject.md to _partials/ and update include paths
- Update .readthedocs.yaml from Ruby/Jekyll to Node.js/mystmd build
- Update .pre-commit-config.yaml for new _partials/ path
- Remove all Jekyll infrastructure: Gemfile, _config.yml, _plugins/,
  _sass/, _includes/, assets/js/tabs.js

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Assisted-by: copilot-cli:claude-sonnet-4.6
- Replace raw HTML block in repo_review.md with {iframe} directive pointing
  to https://scientific-python.github.io/repo-review/ — the book-theme has
  no renderer for type:"html" blocks, so raw HTML was rendered as escaped
  text in the static output; iframe is natively supported
- Remove html_head_extra frontmatter from repo_review.md (unsupported by
  the book-theme, was generating a build warning)
- Remove fetch_repo_review_app.sh step from .readthedocs.yaml (no longer
  needed with external iframe URL)
- Simplify rr-role.mjs: remove broken rawHtmlTransform (ran after
  htmlTransform so type:"html" nodes were never further processed)
- Restore style.md (was accidentally emptied by shell redirection in prior
  session); re-converted from original Jekyll source
- Fix myst.yml: add .md extensions to all 29 TOC entries to resolve
  build warnings about missing file extensions

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Assisted-by: cli:claude-sonnet-4.6
- Fix {% rr PP007 %} unconverted liquid tag in _partials/pyproject.md
- Remove {% raw %}/{% endraw %} from coverage.md
- Fix split note box title in packaging_simple.md
- Hide right-sidebar outline on homepage (site.hide_outline)
- Fix TOC duplication: use file+children instead of title+children
- Add {tableofcontents} to all section index pages (tutorials, guides, principles, patterns)
- Add sync:coverage-tool to coverage page tab-sets
- Add sync:docs-framework to all Sphinx/MkDocs tab-sets in docs.md
- Remove id attribute from rr-btn spans to fix duplicate identifier warnings

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Assisted-by: copilot-cli:claude-sonnet-4.6
- Add explicit MyST labels to testing.md headings to fix implicit
  target warnings (public-interface-tests, project-level-integration-tests,
  unit-tests, additional-types-of-test-suites, fuzz-tests, end-to-end-tests)
- Add sync:publish-method to Trusted Publishing/Token tabs in gha_pure.md
  and gha_wheels.md (cross-page sync)
- Add sync:compiled-backend to Scikit-build-core/Meson-python/Maturin tabs
  in packaging_compiled.md (4 tab-sets now sync)
- Update AGENTS.md: docs site is now MyST (not Jekyll)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Assisted-by: copilot-cli:claude-sonnet-4.6
Sync keys must match the tab label, not a group name, so that MyST
can identify matching tabs across tab-sets.

- docs.md: sphinx, mkdocs
- gha_pure.md + gha_wheels.md: trusted-publishing, token
- packaging_compiled.md: scikit-build-core, meson-python, maturin
- coverage.md was already correct (coverage, pytest-cov)
- Update AGENTS.md with correct sync key convention

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Assisted-by: copilot-cli:claude-sonnet-4.6
Signed-off-by: Henry Schreiner <henryfs@princeton.edu>
Assisted-by: CopilotCLI:claude-hauku-4.5
Signed-off-by: Henry Schreiner <henryfs@princeton.edu>
Signed-off-by: Henry Schreiner <henryfs@princeton.edu>
Signed-off-by: Henry Schreiner <henryfs@princeton.edu>
Assisted-by: CopilotCLI:claude-haiku-4.5
Signed-off-by: Henry Schreiner <henryfs@princeton.edu>
Signed-off-by: Henry Schreiner <henryfs@princeton.edu>
Signed-off-by: Henry Schreiner <henryfs@princeton.edu>
Assisted-by: CopilotCLI:GPT-5.4-mini
Signed-off-by: Henry Schreiner <henryfs@princeton.edu>
Signed-off-by: Henry Schreiner <henryfs@princeton.edu>
Signed-off-by: Henry Schreiner <henryfs@princeton.edu>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Henry Schreiner <henryfs@princeton.edu>
Assisted-by: OpenCode:glm-5.1
Signed-off-by: Henry Schreiner <henryfs@princeton.edu>
henryiii and others added 4 commits June 8, 2026 13:50
Assisted-by: OpenCode:glm-5.1
Signed-off-by: Henry Schreiner <henryfs@princeton.edu>
Signed-off-by: Henry Schreiner <henryfs@princeton.edu>
Remove unnecessary HTML head meta tag from configuration.
- Revert the toml→ini fence change in src/sp_repo_review/checks/*.py;
  those docstrings render in repo-review itself, so the docs-only
  highlighter workaround (code_fence cog helper) should not leak into
  the published package.
- Restore validate-pyproject-schema-store pin to 2026.04.03 in the
  repo_review anywidget deps (was unintentionally downgraded).
- Add a PR-time MyST build check (reusable-docs.yml) wired into CI via
  change detection on docs/helpers/package.json/bun.lock/.readthedocs.yaml.
- Clarify in AGENTS.md that the ini-for-toml workaround is docs-only.

Assisted-by: ClaudeCode:claude-opus-4.8
@henryiii henryiii force-pushed the henryiii/feat/mystmd branch from 9cb624f to d49e143 Compare June 8, 2026 17:50
@lundybernard

Copy link
Copy Markdown
Contributor

I ran a test build of the learn site using this branch (on scientific-python/learn.scientific-python.org#315) with no major problems. Since we will have to update learn to pick up these changes any way I think its safe to merge.

@lundybernard

Copy link
Copy Markdown
Contributor

Regression: Repo Review badges lost their id anchor (regression vs Jekyll)

The old Jekyll repo_review.rb rendered each badge with an id:

<span class="rr-btn" id="PP007">PP007</span>

The new rr-role.mjs drops it:

<span class="rr-btn">PP007</span>

The plugin's own docstring still documents the id:

*   {rr}`PP007`    <span class="rr-btn" id="PP007">PP007</span>

The run(data) render method omits it. Nothing inside the docs links to these anchors, but #<CODE> deep-links into the guides (e.g. from sp-repo-review check results) would no longer resolve.

One-line possible fix in the run(data) function:

value: `<span class="rr-btn" id="${escapeHtml(code)}">${escapeHtml(code)}</span>`,

@henryiii

henryiii commented Jun 8, 2026

Copy link
Copy Markdown
Collaborator Author

Ah, nice catch! I was trying to fix a warning that shows up when you have multiple IDs across different pages in MystMD. Because the core pyproject.toml part is stored once and rendered on both pages, the badges there show up on two pages. This wasn't the right fix, so let me see if I can silence the warning or fix it another way.

Emit a span with an explicit html_id instead of dropping the id or using
MyST's identifier/label (which lowercases and de-duplicates). This restores
the id="PP007" anchors so repo-review deep-links resolve, preserves their
case for the case-sensitive fragment match, and avoids the duplicate
identifier warning since badges legitimately repeat across pages.

Assisted-by: ClaudeCode:claude-opus-4.8
@henryiii henryiii merged commit d8a8d57 into main Jun 9, 2026
33 checks passed
@henryiii henryiii deleted the henryiii/feat/mystmd branch June 9, 2026 03:47
henryiii added a commit to scientific-python/learn.scientific-python.org that referenced this pull request Jun 9, 2026
Cookie moved its docs from Jekyll to MyST in scientific-python/cookie#792.
Build the development site with `myst build --html` (BASE_URL=/development)
and drop the obsolete ruby/jekyll and repo-review-app fetch steps.

Assisted-by: ClaudeCode:claude-opus-4.8
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants