Skip to content

Proposal: add a "multi-role paper review" reference trace #15

Description

@Joe-rq

Hi maintainers — I would like to gauge interest in contributing a new reference
trace before doing the build work.

Background

I have been building PaperSwarm
on top of Flowtrace — a multi-role academic peer-review workflow for research
drafts. A paper is parsed, a reviewer set is selected by paper type, independent
reviewer roles run in parallel, conflicts are detected, a revision plan is
produced, and a Worker/Verifier loop verifies the revision.

What I would contribute

A new builder under scripts/examples/ (e.g. paper-review/) that walks the
full CLI lifecycle and demonstrates Flowtrace capabilities that the current
six reference traces do not cover well together:

  • fan-out / fan-in — 4 reviewer roles (editor, methodologist, domain
    expert, devil's advocate) run as independent steps, then fan into a single
    conflict step. None of the existing data-analysis traces have a wide
    many-to-one fan-in like this.
  • blocked → resume — the conflict step blocks awaiting author
    adjudication between disagreeing reviewers, then resumes (analogous to
    assumption_check in iris-analysis, but for multi-source disagreement).
  • error → retry — the revise step's Verifier rejects an incomplete
    revision, triggering a retry loop.

Questions before I start

  1. Is a non-data-analysis reference trace welcome? Most existing traces
    are data pipelines; only tailored-resume and spring-demo are not. A
    multi-agent review DAG would extend that non-data coverage.
  2. Fixture domain — I am leaning toward a short, fictional
    social-science / education paper so only the 4 base roles are needed and
    the fixture stays small. Any preference?
  3. Scope — I have a 9-step working version (already validated end-to-end
    with a real run). Does that size fit the "land in an hour or two" spirit
    of the existing examples?

Happy to adjust the theme/scope to whatever fits the project best.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions