chore(draft): setup github action for generating editor's draft .html/ .docx

[awoie]

This PR does the following:

• setup github action to build .html/ .docx working draft
• working draft is currently a skeleton document aligned with the ISO template

Fixes #12

[andrewhughes3000]

lgtm

[jogu]

(feedback from claude/codex, and '1' is missing as it wasn't helpful so I deleted it)

2. HTML draft numbers ISO front matter as clauses; DOCX does not

File: draft/digital-credentials-harmonized-presentation.md:30 ·
Severity: High · Status: Confirmed

{mainmatter} starts before # Foreword, so the HTML output numbers
Foreword = 1, Introduction = 2, Scope = 3. The DOCX filter
(tools/iso-styles.lua) separately makes Foreword/Introduction unnumbered so
that Scope numbers as clause 1 — its stated intent. The two artifacts from the
same source therefore disagree on every clause number (HTML "3. Scope" is DOCX
"1 Scope"). For a document where ISO liaison discussion happens by clause
number, this divergence is already visible and will compound as clauses are
added.

Fix: Make the HTML rendition unnumber the front matter too (e.g. frontmatter
special sections), or number both consistently — a deliberate choice either
way.

3. mmark-to-pandoc line filters are not fenced-code-block aware

File: tools/mmark-to-pandoc.py:70 · Severity: High ·
Status: Confirmed

The filters that drop {:...} IAL lines and {mainmatter}/{backmatter}
markers, and rewrite .# headings, run on every line including lines inside
fenced code blocks. Once the spec gains request/response examples, any example
line matching those patterns is silently deleted or mangled in the Word output
while the HTML stays correct — a silent divergence in a normative example, with
no build error.

Fix: Track fence state (```/~~~) and skip filtering inside
fences. See also the structural recommendation below.

4. Abstract-dropping swallows any lower-level headings after .# Abstract

File: tools/mmark-to-pandoc.py:58 · Severity: High ·
Status: Confirmed

The loop that removes the abstract only stops at part markers or level-1 #
headings, but mmark ends the abstract at the next heading of any level. A
frontmatter subsection added after .# Abstract (e.g. ## Notice) appears in
the HTML draft but is silently missing from the ISO Word document.

Fix: Stop the drop loop at any heading, not only level-1.

5. Reference docx carries the template's section setup through wholesale

File: tools/make-iso-reference.py:139 (with --reference-doc in
Makefile:36) · Severity: Medium · Status: Confirmed (mechanism in
code; observed empirically in a generated DOCX)

transform() only touches [Content_Types].xml, .rels files, styles.xml,
and header/footer parts; word/document.xml — where the template's
multi-section setup (sectPr) lives — passes through untouched. Pandoc emits a
single-section document derived from it: in a generated DOCX the final sectPr
referenced only even/default header/footer parts and used the template's
first-section margins. The template's separate front-matter/body sections
(page-numbering restarts, per-section headers, margins) cannot survive this,
risking wrong page layout/numbering as the draft grows.

Fix: Curate the reference docx's single section deliberately (see
structural recommendation: commit a hand-finished reference docx).

6. TOML title extraction only handles plain double-quoted strings

File: tools/mmark-to-pandoc.py:41 · Severity: Medium ·
Status: Confirmed

Single-quoted TOML literal strings (title = 'Foo') never match, leaving the
Word document with no title block at all; escaped quotes/backslashes are
captured raw and re-escaped by json.dumps, rendering literal \" sequences
on the title page.

Fix: Parse the %%% block with tomllib instead of regexes.

7. Title/subtitle split truncates titles containing " - "

File: tools/iso-styles.lua:70 · Severity: Medium (latent — the
current title has exactly one separator) · Status: Confirmed

The split matches the first " - ". A title like "Digital Credentials -
Harmonized Presentation - Working Group Draft" renders with main title
"Digital Credentials" and the rest demoted to subtitle — a wrong title page
with no error.

Fix: Split on the last " - ", or pass title and status as separate
metadata fields instead of encoding them in one string.

8. Standalone reference-docx output keeps ISO OLE object and dangling image refs

File: tools/make-iso-reference.py:74 · Severity: Medium (standalone
usage only; pandoc consumes only styles) · Status: Confirmed

DROP_PREFIXES omits word/embeddings/, so the template's embedded
oleObject1.bin survives into the output, contradicting the docstring's claim
that no ISO branding travels with the file. The media relationships are
stripped but the r:embed="rId25" references in document.xml remain, so
opening the standalone output in Word shows broken-image placeholders.

Fix: Add word/embeddings/ to DROP_PREFIXES and remove the drawing
elements that reference dropped media (or stop advertising standalone usage).

9. Empty stream in [seriesInfo]; unpinned markdown2rfc image

Files: draft/digital-credentials-harmonized-presentation.md:8,
Makefile:25 · Severity: Low/Medium · Status: Confirmed

make html succeeds but mmark warns Empty 'stream' in [seriesInfo], resulting XML may fail to parse — the block has name/value/status but no stream.
The danielfett/markdown2rfc image is untagged (:latest), so a future
image/toolchain update could turn the warning into a CI break. The
[seriesInfo] block looks carried over from IETF drafts (ipr = "none",
OpenID workgroup); worth deciding what series info this document actually has
rather than just adding a stream value.

Fix: Decide the correct seriesInfo for an OpenID/ISO joint document; pin
the Docker image by tag or digest.

10. No concurrency group on the GitHub Pages deployment

File: .github/workflows/build.yml:68 · Severity: Low/Medium ·
Status: Plausible

Two quick merges to main can run deploy-pages concurrently: one fails with
"a deployment is already in progress" (red CI on main), or the older build
finishes last and the published draft goes stale.

Fix: Add the standard concurrency: { group: "pages", cancel-in-progress: false }
block.

11. Unpinned pandoc install via apt

File: .github/workflows/build.yml:42 · Severity: Low (cleanup) ·
Status: Confirmed

apt-get update && apt-get install pandoc adds 30–60 s to every run (doubled
across push and PR triggers) and pulls whatever version the runner image
resolves, so the ISO .docx rendering can silently change between identical
commits.

Fix: Install a pinned pandoc release .deb — faster and reproducible.

12. "Working Group Draft" label on an automatic per-commit build

Files: README.md:27, Makefile (-wg-draft.html naming), and the
draft's own Foreword text · Severity: Low (process/status question) ·
Status: Plausible

The removed README text distinguished "Editor's Copy" from "Working Group
Drafts" — distinct approval statuses in the OpenID process. The new text and
the output filename label the per-commit build from main as "The current
Working Group Draft", inviting readers to treat unapproved content as an
approved draft. The branch itself is named setup-editors-draft.

Fix: Decide the intended status with the WG and label the artifact
accordingly (e.g. "Editor's Copy" until a draft is approved).

13. Fragile cp glob in the Makefile

File: Makefile:27 · Severity: Low · Status: Plausible

cp $(SRCDIR)/$(DOC)*.html <single file> breaks if the glob ever matches two
files — e.g. after an interrupted run leaves a stale intermediate behind —
failing with "target is not a directory" until manually cleaned up.

Fix: Copy the exact expected filename, or clean before building.

---

Architecture assessment

The goal — one markdown source producing both an OpenID-style HTML draft and an
ISO-styled Word document — is sound, and single-sourcing is the right call.
The fragility concentrates in three stacked regex transformation layers
(line-filtering mmark syntax, regex surgery on the .dotx zip, string-splitting
the title in Lua), each of which fails silently. Nine of the thirteen
findings live in those layers.

Recommended restructuring, without changing the architecture:

1. Commit a hand-finished reference docx instead of regenerating it every
   build. Run make-iso-reference.py once (or do the cleanup by hand in
   Word), fix the section setup, headers/footers, and margins properly, verify
   no ISO branding remains, and commit template/iso-reference.docx. This
   removes an entire fragile script from the build path and fixes finding 5 in
   a way regex-editing OOXML inside a zip never can. (The .dotx is already
   committed, so redistribution of a derived file is no new concern.)

2. Parse, don't pattern-match, in the converter. The %%% TOML block is
   cleanly delimited: split it off, parse with tomllib (stdlib since Python
   3.11), and emit real pandoc metadata. Process the body with fence awareness.
   This kills findings 3, 4, and 6 at the root. Add one golden-output test (a
   sample input with fences, IALs, and multi-heading frontmatter, plus its
   expected output) so future silent corruption becomes a test failure.

3. Pin everything — the Docker image by tag/digest, pandoc by release
   .deb. Reproducible ISO output is the point of the docx path.

The genuinely different alternative is Metanorma (metanorma-iso),
purpose-built for authoring ISO deliverables: clause numbering,
Foreword/Introduction handling, terms-and-definitions structure, and Word
output are native. But it means AsciiDoc instead of mmark and abandons the
xml2rfc-style HTML that OpenID WGs conventionally publish.

Open question to settle with the WG before investing further: what is the
generated docx for? If WG10's project editor will re-flow content into the
official ISO template manually (common JTC1 practice), the docx only needs to
be complete and readable, and most of the styling precision is effort spent on
an intermediate artifact. If the generated file is meant to be the
submission, the pipeline will keep fighting pandoc's single-section model, and
Metanorma deserves a real evaluation before more regexes accumulate.

[awoie]

(feedback from claude/codex, and '1' is missing as it wasn't helpful so I deleted it)

Thanks for the detailed review. Addressed in the latest push (AI-assisted).

Converter (parse, don't pattern-match). mmark-to-pandoc.py now parses the
%%% block with tomllib and processes the body fence-aware:

• finding 6: single/escaped TOML titles now parse.
• finding 7: title/status emitted as separate pandoc fields (split on the last " - ").
• finding 4: abstract drop stops at the next heading of any level.
• finding 3: %%%/{...}/.# inside code fences pass through verbatim.
• Added a golden-output test (tests/) that runs in CI.

Reference docx. Generated once and committed as template/iso-reference.docx;
make-iso-reference.py is out of the build path (finding 5). finding 8: drops
word/embeddings/ and strips the drawing/object refs so it opens clean.

Build/CI. finding 13 clean-before-build; finding 9 markdown2rfc pinned by
digest + stream added; finding 11 pandoc from pinned .deb; finding 10
concurrency: pages on deploy; Python pinned to 3.12 (converter needs 3.11+).

Findings 2 + 12. Foreword/Introduction are now unnumbered front matter (.#)
with {mainmatter} before Scope, so HTML and DOCX agree Scope = clause 1;
relabeled "Working Group Draft" to "Editor's Copy" throughout.

Not going the Metanorma route: it would mean AsciiDoc and dropping the
xml2rfc-style HTML that OpenID WGs publish. The committed reference docx would
still benefit from a one-time manual Word pass to finalize section setup
(finding 5). seriesInfo is the mmark/xml2rfc front-matter block
(name/value/status/stream) that labels the document's series in the HTML header;
it is currently carried over from IETF Internet-Drafts, so its final values for
an OpenID/ISO joint document are a WG decision (finding 9). For now I only added
stream to clear the build warning.

[awoie]

(feedback from claude/codex, and '1' is missing as it wasn't helpful so I deleted it)

Thanks for the detailed review. Addressed in the latest push (AI-assisted).

Converter (parse, don't pattern-match). mmark-to-pandoc.py now parses the %%% block with tomllib and processes the body fence-aware:

• finding 6: single/escaped TOML titles now parse.
• finding 7: title/status emitted as separate pandoc fields (split on the last " - ").
• finding 4: abstract drop stops at the next heading of any level.
• finding 3: %%%/{...}/.# inside code fences pass through verbatim.
• Added a golden-output test (tests/) that runs in CI.

Reference docx. Generated once and committed as template/iso-reference.docx; make-iso-reference.py is out of the build path (finding 5). finding 8: drops word/embeddings/ and strips the drawing/object refs so it opens clean.

Build/CI. finding 13 clean-before-build; finding 9 markdown2rfc pinned by digest + stream added; finding 11 pandoc from pinned .deb; finding 10 concurrency: pages on deploy; Python pinned to 3.12 (converter needs 3.11+).

Findings 2 + 12. Foreword/Introduction are now unnumbered front matter (.#) with {mainmatter} before Scope, so HTML and DOCX agree Scope = clause 1; relabeled "Working Group Draft" to "Editor's Copy" throughout.

Not going the Metanorma route: it would mean AsciiDoc and dropping the xml2rfc-style HTML that OpenID WGs publish. The committed reference docx would still benefit from a one-time manual Word pass to finalize section setup (finding 5). seriesInfo is the mmark/xml2rfc front-matter block (name/value/status/stream) that labels the document's series in the HTML header; it is currently carried over from IETF Internet-Drafts, so its final values for an OpenID/ISO joint document are a WG decision (finding 9). For now I only added stream to clear the build warning.

Follow-up: fixed two build issues from the first pass (AI-assisted).

• make html was failing: seriesInfo stream = "independent" conflicts with
  xml2rfc's default submissionType (IETF). Set stream = "IETF" to clear the
  empty-stream warning without the conflict.
• make docx failed on Python older than 3.11 (no tomllib). The Makefile now
  auto-picks a tomllib-capable interpreter (override with PYTHON=...), and the
  HTML copy is back to a glob since markdown2rfc may add a draft-version suffix.

[awoie]

It seems there is a build error on github. I will fix and let you know once fixed @jogu .

[awoie]

It seems there is a build error on github. I will fix and let you know once fixed @jogu .

I forgot to push the fixes. Fixes are in and build action succeeded. Please review again @jogu.

[c2bo]

Would it make sense to move the tooling to a separate repository in OIDF and provide it ready to use as a container?