A Mediforce workflow for the CDISC AI Innovation Challenge, Use Case 3: AI-Driven Tables, Figures, and Listings (TFL) Generation — now the full protocol-to-TFL pipeline. Given a study's USDM metadata (and its SDTM datasets) it plans the required TFLs, builds the analysis + ADaM specs, derives ADaM, generates the ARD (numbers) and rendered TFLs (display), and assembles an interactive objective → endpoint → SDTM → ADaM → TFL traceability explorer.
Thesis: plan from the objectives, generate from the spec, prove with the numbers. Accuracy is measured cell-by-cell against known-good CSR outputs; every TFL traces back through the ARD to the ADaM, SDTM, endpoint, and objective that justify it. Three human review gates (plan, specs, TFLs) each feed a self-learning skill-refinement loop: reviewer feedback is distilled into durable, per-skill lessons and opened as a PR, so the skills improve over time.
| # | Step | Executor | Skill / script | Output |
|---|---|---|---|---|
| 1 | upload-inputs | human | — | upload the study's USDM JSON + SDTM datasets (→ /data/ for plan-tlfs) |
| 2 | plan-tlfs | agent | tlf-planner |
study-model.json, tlf-plan.json, tlf-index.md (+ persists SDTM → /workspace/sdtm/) |
| 3 | audit-plan | agent | tlf-plan-critic |
coverage-report.md + verdict |
| 4 | review-plan | human review | — | approve → specs · revise → plan |
| 5 | build-specs | agent | tlf-analysis-spec |
analysis-spec.json, adam-spec.json |
| 6 | review-specs | human review | — | approve → ADaM · revise → specs |
| 7 | derive-adam | agent | sdtm-to-adam |
/workspace/adam/* + conformance report |
| 8 | generate-tlfs | agent | tlf-generator |
ARD + rendered TFLs |
| 9 | review-tlfs | human review | — | approve → trace · revise → generate |
| 10 | build-traceability | script | build_traceability.py |
traceability.html, trace_graph.json, manifest.json |
| 11 | propose-skill-update | agent | propose-skill-lesson |
per-skill lesson blocks |
| 12 | open-skill-pr | script | open_skill_pr.py |
PR against main (or clean no-op) |
| 13 | done | human (terminal) | — | — |
All inputs are uploaded up front at a single file-upload step (step 1): the
USDM JSON and the SDTM datasets together. They are made available read-only
under /data/ to the next step (plan-tlfs), which persists the SDTM into
/workspace/sdtm/ for the later ADaM step — so no separate staging step is needed.
Transitions include three revise loops (each review gate can send the run back to its producing agent step).
On each Request Changes, the producing agent step (plan-tlfs,
build-specs, generate-tlfs) appends the reviewer comment to
/workspace/review_feedback.jsonl, tagged with its skill. After approval,
propose-skill-update groups that feedback by skill and distils durable,
skill-general lessons; open-skill-pr appends each to the target skill's
references/lessons-learned.md and opens one PR against main. A human merges;
the next run reads the improved skills. First-pass approvals (no revisions)
produce no PR.
Six skills run as agents: tlf-planner, tlf-plan-critic, tlf-analysis-spec,
sdtm-to-adam, tlf-generator, propose-skill-lesson. The revisable ones
(tlf-planner, tlf-analysis-spec, tlf-generator) each carry a
references/lessons-learned.md that the loop appends to.
traceability-builder is no longer invoked as an agent — the traceability
step is now the deterministic container/build_traceability.py script (it never
recomputes a statistic, so it needs no LLM; this removes its per-run cost and
nondeterminism). The skill directory is retained as the shared design/mirror and
as the human-readable contract the script implements
(references/graph-data-schema.md).
| Name | Secret | Scope | Used by | Meaning | How to set | Example |
|---|---|---|---|---|---|---|
GITHUB_TOKEN |
✅ | workflow | all script/agent steps (Docker build context + skills fetch) and open-skill-pr |
Token with repo read for the pinned build/skills context, plus contents:write + pull-requests:write for the skill-lesson PR |
namespace/workflow secret | ghp_… |
OPENROUTER_API_KEY |
✅ | workflow | every agent step (mapped to ANTHROPIC_AUTH_TOKEN, base URL https://openrouter.ai/api) |
LLM credential for the Claude-Code agents | namespace/workflow secret | sk-or-… |
SKILL_REPO |
— | open-skill-pr step env |
open_skill_pr.py |
<owner>/<repo> the skills live in |
step env |
vedhav/cdisc-case-3 |
Secrets are referenced via {{SECRET_NAME}} templates and are never committed or
baked into the image.
mediforce-agent:cdisc-case-3, built from Dockerfile (FROM mediforce-golden-image). Adds the pinned R stack — admiral/admiraldev/metacore/
metatools (ADaM); cards/cardx/emmeans/mmrm/survival/broom.helpers (ARD + models);
gtsummary/gt/tfrmt/rtables/rlistings/ggsurvfit/ggplot2 (display); dplyr/tidyr/
haven/jsonlite — plus Python pyyaml and the step scripts (container/). Skills
and the local-test fixtures/ are not baked in (inputs are uploaded per run;
fixtures are for local testing only).
Every step references this image by tag only (image: mediforce-agent:cdisc-case-3,
no repo/commit/dockerfile) — same as apps/protocol-to-tfl. The image must
already exist on the host that runs the containers; the platform does not
auto-build it. Build it once on that host:
docker build -t mediforce-agent:cdisc-case-3 .. Rebuild + re-tag whenever the
Dockerfile or container/ scripts change. (The prior per-step auto-build shape
— repo+commit+dockerfile — was removed: a hand-built image carries no
mediforce.build.commit label, so the runtime saw it as "stale" and tried to
rebuild the R stack through execSync(stdio:'pipe'), whose 1 MB maxBuffer
overflows on the chatty source compile and hangs the step outside its timeout.)
study-model.json, tlf-plan.json, tlf-index.md, coverage-report.md,
analysis-spec.json, adam-spec.json, the ARD + rendered TFLs,
traceability.html, trace_graph.json, manifest.json, and each
step's result.json.
A single upload-inputs step (step 1) collects everything up front; the files land
read-only under /data/ for plan-tlfs (which persists the SDTM into /workspace/sdtm/):
- the study's USDM study-definition JSON (required).
- the study's SDTM datasets (required;
.xpt/ Dataset-JSON /.csv/.sas7bdat).
The pipeline runs against any new study's USDM + SDTM. A known-good reference for
smoke-testing is the CDISCPILOT01 (H2Q-MC-LZZT Alzheimer's) USDM + SDTM,
produced/held upstream (Case 1/Case 2 and the protocol-to-tfl source).
Only externalSkillsRepo.commit is pinned — it is the sole runtime source the
platform fetches at run time (the pipeline skills, cloned from this repo at that
commit). Update it on each skill release via the two-step pin dance: push, read
HEAD, rewrite the pin to that SHA, push again. The Docker image is not pinned
by commit here — it is referenced by tag and built out-of-band on the host (see
Docker image above), so keep the image in sync with Dockerfile/container/ by
rebuilding when they change. workspace.remote is unused; source (git-import
provenance) is n/a.
Import/register a new workflow version for every released change
(mediforce workflow validate then register). See src/cdisc-case-3.wd.json.