Skip to content

Harden debate flow for long-context runs and invalid stage outputs #12

@simonhsze

Description

@simonhsze

What

Make ac debate more robust when prompts are long, context files are heavy, or one role returns invalid JSON.

Motivation

In real use, the current debate path can become brittle under long-context conditions:

  • topic and supporting context are passed into the analyst, then the same combined context is passed into debate stages again
  • challenger / defender / judge outputs are schema-validated strictly
  • if one role drifts out of JSON or partially truncates, the run can fail hard instead of degrading gracefully

In practice this pushes users toward manual recovery patterns such as splitting the flow into analyst / critic / defender / judge steps outside the CLI. That can work as an operator workaround, but it is not yet a first-class product path.

Scope

  • review how much context is repeated across analyst and debate stages
  • reduce unnecessary prompt duplication where possible
  • add a repair / retry path for invalid JSON outputs before failing the whole run
  • consider a CLI option for non-strict or recovery-oriented execution
  • add tests for long-context and malformed-stage behavior
  • document when users should expect a single-pass debate vs a staged recovery path

Why this matters

This is likely to be a common failure mode, not just a one-off operator issue. If the project is meant to support document-driven high-stakes review, it needs a more reliable path for long prompts and imperfect role outputs.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions