Fix error-handling, improve README/CI

.github/workflows/ci.yml

@@ -1,5 +1,7 @@
 name: Test, release
 on:
+  pull_request:
+    branches: [main]
   push:
     branches: [main]
     tags: ["v*.*.*"]
@@ -13,9 +15,7 @@ jobs:
         python-version: ["3.10", "3.11", "3.12", "3.13"]
     steps:
     - uses: actions/checkout@v4
-    - uses: astral-sh/setup-uv@v4
-      with:
-        enable-cache: true
+    - uses: astral-sh/setup-uv@v7
     - run: uv python install ${{ matrix.python-version }}
     - name: Install mdcmd globally via uv tool
       run: uv tool install --python python${{ matrix.python-version }} .

0.7.2.md

@@ -0,0 +1,17 @@
+v0.7.2: Error handling improvements
+
+🐛 Bug fixes
+- Fix `mdcmd` to exit with non-zero code when embedded commands fail
+- Commands that fail no longer crash with stack traces
+- Failed commands are reported gracefully with exit code and output
+- Other commands continue to execute in concurrent mode
+
+📦 Dependencies
+- Bump `utz` requirement to `>=0.21.1` (includes fix for `text()` handling `None`)
+
+🧪 Tests
+- Add tests for command failure handling
+- Add tests for mixed success/failure scenarios
+- Add tests to ensure no uncaught exceptions on errors
+
+This release fixes a critical issue where `mdcmd` would exit with code 0 even when commands failed, breaking shell scripts using `set -e`.

README.md

@@ -10,11 +10,7 @@ seq 3
 # 3
 ```
 
-<!-- `python test/print-ci-yml-ref.py mdcmd` -->
-<p>
-
-☝️ This block is updated programmatically by [`mdcmd`] (and verified [in CI](.github/workflows/ci.yml#L28-L31); see [raw README.md](README.md?plain=1#L5-L11)).
-</p>
+☝️ This block is updated programmatically by [`mdcmd`] (and verified [in CI]; see [raw README.md][raw-mdcmd]).
 
 [![](https://img.shields.io/pypi/v/mdcmd?label=mdcmd&color=blue)][mdcmd] (formerly: [`bmdf`][bmdf])
 
@@ -35,11 +31,7 @@ seq 3
 - [`toc`: Markdown Table of Contents](#toc)
 - [Examples](#examples)
 
-<!-- `python test/print-ci-yml-ref.py toc` -->
-<p>
-
-☝️ This TOC is generated programmatically by [`mdcmd`] and [`toc`] (and verified [in CI](.github/workflows/ci.yml#L28-L31); see [raw README.md](README.md?plain=1#L22-L36)).
-</p>
+☝️ This TOC is generated programmatically by [`mdcmd`] and [`toc`] (and verified [in CI]; see [raw README.md][raw-toc]).
 
 ## Overview <a id="overview"></a>
 This package provides 3 CLIs:
@@ -134,7 +126,7 @@ Notes:
   - If there's already output there, it will be replaced with new/current output.
 
 ### HTML example <a id="mdcmd-html-example"></a>
-Scripts that output raw HTML also work, e.g. [print-table.py](test/print-table.py) generates this table:
+Scripts that output raw HTML also work, e.g. [print-table.py] generates this table:
 
 <!-- `python test/print-table.py` -->
 <table>
@@ -435,7 +427,7 @@ A `mktoc` script is also provided, which just wraps `mdcmd -x '^toc$'` (`mktoc`
 
 ## Examples <a id="examples"></a>
 - The examples in this file are all rendered by [`bmdf`] and [`mdcmd`].
-- [The TOC](#toc) is rendered by [`toc`].
+- [The TOC](#toc) above is rendered by [`toc`].
 - The [`ci.yml`] GitHub Action verifies the examples and TOC.
 
 These repos' READMEs also use [`bmdf`] / [`mdcmd`] / [`toc`] to execute example commands (and in some cases also verify them with a GitHub Action):
@@ -449,14 +441,25 @@ These repos' READMEs also use [`bmdf`] / [`mdcmd`] / [`toc`] to execute example
 - [ryan-williams/tdbs-dask]
 - [TileDB-Inc/scverse-ml-workshop-2024]
 
+<!-- `scripts/gh-url.py '`ci.yml`' .github/workflows/ci.yml` -->
 [`ci.yml`]: .github/workflows/ci.yml
 [`bmd`]: #bmd
 [`bmdf`]: #bmdf
 [`bmdff`]: #bmdff
 [`bmdfff`]: #bmdfff
 [`mdcmd`]: #mdcmd
 [`toc`]: #toc
+
+<!-- `scripts/gh-url.py toc.py src/bmdf/toc.py` -->
 [toc.py]: src/bmdf/toc.py
+<!-- `scripts/gh-url.py print-table.py test/print-table.py` -->
+[print-table.py]: test/print-table.py
+<!-- `scripts/gh-url.py 'in CI' .github/workflows/ci.yml#L28-L31` -->
+[in CI]: .github/workflows/ci.yml#L28-L31
+<!-- `scripts/raw-readme-link.py mdcmd` -->
+[raw-mdcmd]: README.md?plain=1#L5-L11
+<!-- `scripts/raw-readme-link.py toc` -->
+[raw-toc]: README.md?plain=1#L18-L32
 
 [runsascoded/utz]: https://github.com/runsascoded/utz?tab=readme-ov-file#utz
 [TileDB-Inc/scverse-ml-workshop-2024]: https://github.com/TileDB-Inc/scverse-ml-workshop-2024?tab=readme-ov-file#training-models-on-atlas-scale-single-cell-datasets

pyproject.toml

@@ -1,15 +1,15 @@
 [project]
 name = "mdcmd"
 description = "Execute commands in Markdown files, embed output, generate TOCs"
-version = "0.7.1"
+version = "0.7.2"
 readme = "README.md"
 authors = [
     { name = "Ryan Williams", email = "ryan@runsascoded.com" }
 ]
 license = { text = "MIT" }
 requires-python = ">=3.10"
 dependencies = [
-    "utz>=0.19.1",
+    "utz>=0.21.1",
     "click>=8",
 ]
 

scripts/gh-url.py

@@ -0,0 +1,40 @@
+#!/usr/bin/env python3
+"""Generate GitHub URLs for links in README.
+
+Usage:
+    # For a file link:
+    python scripts/gh-url.py toc.py src/bmdf/toc.py
+
+    # For a link with line numbers:
+    python scripts/gh-url.py 'in CI' .github/workflows/ci.yml#L28-L31
+
+When README_ABSOLUTE_URLS=1 is set, generates absolute GitHub URLs instead of
+relative paths. This is used during PyPI package builds to ensure links work
+correctly on PyPI's README display. Requires being on a git tag when this
+environment variable is set.
+"""
+
+import sys
+from pathlib import Path
+
+# Add parent directory to path to import gh_url_utils
+sys.path.insert(0, str(Path(__file__).parent))
+from gh_url_utils import get_github_base_url, format_url
+
+
+def main():
+    """Generate the link definition."""
+    if len(sys.argv) < 3:
+        print("Usage: gh-url.py <link_text> <link_path>", file=sys.stderr)
+        sys.exit(1)
+
+    link_text = sys.argv[1]
+    link_path = sys.argv[2]
+
+    base_url = get_github_base_url()
+    url = format_url(link_path, base_url)
+    print(f"[{link_text}]: {url}")
+
+
+if __name__ == "__main__":
+    main()

scripts/gh_url_utils.py

@@ -0,0 +1,32 @@
+#!/usr/bin/env python3
+"""Shared utilities for generating GitHub URLs."""
+
+import os
+import subprocess
+import sys
+
+
+def get_github_base_url():
+    """Get the appropriate GitHub base URL based on environment."""
+    if os.getenv("README_ABSOLUTE_URLS"):
+        try:
+            result = subprocess.run(
+                ["git", "describe", "--exact-match", "--tags", "HEAD"],
+                capture_output=True, text=True, check=True
+            )
+            ref = result.stdout.strip()
+            if not ref:
+                print("ERROR: README_ABSOLUTE_URLS=1 but not on a tag", file=sys.stderr)
+                sys.exit(1)
+            return f"https://github.com/runsascoded/mdcmd/blob/{ref}"
+        except subprocess.CalledProcessError:
+            print("ERROR: README_ABSOLUTE_URLS=1 but not on a tag", file=sys.stderr)
+            sys.exit(1)
+    return None  # Use relative URLs
+
+
+def format_url(path, base_url=None):
+    """Format a URL as either absolute or relative."""
+    if base_url:
+        return f"{base_url}/{path}"
+    return path

scripts/raw-readme-link.py

@@ -0,0 +1,85 @@
+#!/usr/bin/env python3
+"""Generate raw README.md link with dynamic line numbers.
+
+Usage:
+    python scripts/raw-readme-link.py mdcmd
+    python scripts/raw-readme-link.py toc
+
+Finds the relevant section in README.md and generates a link with line numbers.
+When README_ABSOLUTE_URLS=1 is set, generates absolute GitHub URLs.
+"""
+
+import sys
+from pathlib import Path
+
+# Add current directory to path to import gh_url_utils
+sys.path.insert(0, str(Path(__file__).parent))
+from gh_url_utils import get_github_base_url, format_url
+
+
+def find_readme_lines(section):
+    """Find line numbers for the relevant section in README.md."""
+    try:
+        content = Path("README.md").read_text()
+        lines = content.splitlines()
+
+        # Map section names to their command patterns
+        patterns = {
+            "mdcmd": "<!-- `bmdf seq 3` -->",
+            "toc": "<!-- `toc` -->"
+        }
+
+        if section not in patterns:
+            return None
+
+        pattern = patterns[section]
+
+        # Find the pattern and determine the end based on content type
+        for i, line in enumerate(lines, 1):
+            if pattern in line:
+                # For mdcmd example, find until closing ```
+                if section == "mdcmd":
+                    for j in range(i+1, len(lines)+1):
+                        if j > len(lines) or lines[j-1] == "```":
+                            return (i, j)
+                # For toc, find until empty line
+                else:
+                    for j in range(i+1, len(lines)+1):
+                        if j > len(lines):
+                            return (i, j-1)
+                        if not lines[j-1].strip():
+                            return (i, j-1)
+    except:
+        pass
+    return None
+
+
+def main():
+    """Generate the raw README link with line numbers."""
+    if len(sys.argv) < 2:
+        print("Usage: raw-readme-link.py {mdcmd|toc}", file=sys.stderr)
+        sys.exit(1)
+
+    section = sys.argv[1]
+    if section not in ["mdcmd", "toc"]:
+        print(f"Unknown section: {section}", file=sys.stderr)
+        sys.exit(1)
+
+    lines = find_readme_lines(section)
+    base_url = get_github_base_url()
+
+    # Build the path with line numbers
+    if lines:
+        start, end = lines
+        readme_path = f"README.md?plain=1#L{start}-L{end}"
+    else:
+        readme_path = "README.md?plain=1"
+
+    url = format_url(readme_path, base_url)
+
+    # Generate the appropriate link text based on section
+    print(f"[raw-{section}]: {url}")
+
+
+if __name__ == "__main__":
+    main()