Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,13 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

### Added

- **Durable buffered CLI results (#58).** `conclave ask --json-output PATH` atomically
persists the complete `CouncilResult` JSON with user-private temporary-file semantics
while preserving stdout and exit-code behavior. Persistence remains opt-in and is
rejected with streaming; write failures retain normal stdout before exiting 1.

## [1.2.0] - 2026-07-18

### Added
Expand Down
1 change: 1 addition & 0 deletions DOCUMENTATION_INDEX.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@ the canonical authority spec on top of those.
| **H1 Synthetic QA Pack** | [`studies/elite_qa_v1/README.md`](studies/elite_qa_v1/README.md) | Balanced 24-task open-book harness fixture, QA protocol, and confirmatory preregistration template. |
| **H1 Live Runner Design** | [`docs/plans/2026-07-18-h1-live-evaluation-runner-design.md`](docs/plans/2026-07-18-h1-live-evaluation-runner-design.md) | Sequential paid-exploratory execution, hash-bound UTF-8 estimates, USD 10 cap, authenticated checkpoints, and no-repeat resume. |
| **H1 Live Runner Plan** | [`docs/plans/2026-07-18-h1-live-evaluation-runner.md`](docs/plans/2026-07-18-h1-live-evaluation-runner.md) | Exact TDD tasks for the six live conditions, dry-run estimator, replay fixtures, CLI gate, and correctness-only paid smoke. |
| **Durable JSON Output Design** | [`docs/plans/2026-07-21-durable-json-output-design.md`](docs/plans/2026-07-21-durable-json-output-design.md) | Opt-in atomic user-private result persistence for long buffered council runs and detached supervisors. |

---

Expand Down
12 changes: 12 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -132,8 +132,20 @@ conclave ask "Should we adopt a service mesh for 8 services?" \

# Machine-readable output (works for every mode; carries elite phase artifacts too)
conclave ask "..." -c grok,perplexity --mode debate --json

# Durable buffered output for supervisors that may detach before a long run finishes
conclave ask "..." -c grok,gemini,claude --mode adversarial \
--json --json-output /secure/local/run-result.json
```

`--json-output PATH` is opt-in and available for buffered runs. It atomically
persists the complete result with user-private temporary-file semantics (`0600` on
POSIX) while leaving stdout and exit codes unchanged. The parent directory must
already exist, existing files are replaced atomically, and `--stream` cannot be
combined with durable output. Result files contain the prompt and model answers, so
choose a secure local path. A persistence error still renders normal stdout, then
exits 1.

Published v1.2 mode flags are `--mode synthesize|raw|debate|adversarial|vote|elite`. `--rounds N`
(default 2) is the *maximum* round count for `debate`; `--converge-threshold FLOAT`
(or `--converge`/`--no-converge`) optionally stops a debate early once answers
Expand Down
5 changes: 5 additions & 0 deletions docs/PRODUCT_DESIGN_DOCUMENT.md
Original file line number Diff line number Diff line change
Expand Up @@ -111,6 +111,11 @@ markup, no middleman) and a security property.
absent is skipped with a warning and recorded in `CouncilResult.skipped`. Unknown
providers (no static env-var mapping) are *not* pre-emptively skipped — the live call is
attempted and any auth error is captured as a `ModelAnswer.error`.
6. **Result persistence is explicit and local.** Buffered CLI runs persist nothing by
default. `--json-output PATH` opts into an atomic user-private file (`0600` on POSIX)
containing the same secret-free `CouncilResult` JSON; normal stdout remains available
even if persistence fails. The file still contains the prompt and answers, so operators
must select a secure path.

**Residual considerations:** a provider error could in principle echo a key fragment. Since
v0.3 every provider/transport error is passed through `redact()` before it reaches
Expand Down
52 changes: 52 additions & 0 deletions docs/plans/2026-07-21-durable-json-output-design.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
# Durable JSON Output Design

## Problem

`conclave ask --json` writes its result only to stdout after the full council run.
Long adversarial runs can outlive a supervising shell or agent tool's output session,
so the council succeeds but the caller loses the final JSON. Increasing a caller
timeout is brittle, and a wrapper cannot guarantee that every integration handles
process detachment correctly.

## Decision

Add an opt-in `--json-output PATH` option to `conclave ask`. After a council result
is complete, Conclave serializes the same payload used by `--json` and writes it to
the requested path before applying the existing exit-code contract. The option does
not require `--json`: callers may retain human terminal output while polling the
durable file. Existing stdout behavior remains unchanged.

The file is written atomically through a securely created temporary sibling and
replaced only after the complete JSON has been flushed. It uses user-private
temporary-file semantics (`0600` on POSIX). The parent directory must already exist;
Conclave will not create directory trees or silently redirect output. A write failure
is reported without exposing the result body, preserves normal stdout, and exits 1.

Alternatives rejected:

- Raising orchestration timeouts still loses output when the supervisor detaches.
- A repository-specific shell wrapper duplicates lifecycle and security behavior.
- Always persisting results would violate Conclave's current no-storage default.

## Data Flow and Safety

The council runs exactly as it does today. The completed `CouncilResult` is converted
once with `_result_to_dict`, encoded as JSON, optionally written to the owner-only
path, and then rendered to stdout or Rich panels. API keys remain outside the result
model and are never written. Because prompts and model answers may be sensitive,
persistence stays explicit and the documentation warns operators to select a secure
local path.

Streaming remains unchanged: `--json-output` is rejected with `--stream`, because the
streaming branch currently returns before constructing the shared rendering path.
Supporting both would enlarge the fix without solving the detached adversarial-run
failure.

## Verification and Rollback

Regression tests cover successful durable output, user-private permissions, identical
stdout/file payloads under `--json`, persistence-failure stdout, portability, and
pre-call rejection of unsupported destinations or streaming. Existing CLI and full
offline suites must remain green, followed by lint and secret scanning. Rollback is a
single option/helper removal; no persisted format, cache, config, or public Python API
changes.
66 changes: 64 additions & 2 deletions src/conclave/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,9 @@
from __future__ import annotations

import json
import os
import tempfile
from pathlib import Path

import typer
from rich.console import Console
Expand Down Expand Up @@ -41,6 +44,34 @@ def _result_to_dict(result: CouncilResult) -> dict:
return result.model_dump(mode="json")


def _write_json_output(path: Path, payload: dict) -> None:
"""Atomically persist a complete JSON payload with user-private permissions."""
# mkstemp avoids name races and creates a user-private file on supported
# platforms; replacing the destination preserves those permissions.
fd, temp_name = tempfile.mkstemp(
dir=path.parent,
prefix=f".{path.name}.",
suffix=".tmp",
)
temp_path = Path(temp_name)
try:
with os.fdopen(fd, "w", encoding="utf-8") as handle:
json.dump(payload, handle)
handle.write("\n")
handle.flush()
os.fsync(handle.fileno())
os.replace(temp_path, path)
finally:
try:
os.close(fd)
except OSError:
pass
try:
temp_path.unlink(missing_ok=True)
except OSError:
pass


def _answer_panel(ans, *, border: str = "cyan") -> Panel:
"""Build a rich panel for a single ModelAnswer (ok or failed)."""
if ans.ok:
Expand Down Expand Up @@ -493,6 +524,14 @@ def ask(
as_json: bool = typer.Option(
False, "--json", help="Emit the full result as JSON instead of panels."
),
json_output: Path | None = typer.Option( # noqa: B008
None,
"--json-output",
help=(
"Persist the complete result JSON to PATH with user-private permissions. "
"Buffered modes only; stdout rendering is unchanged."
),
),
cache: bool | None = typer.Option(
None,
"--cache/--no-cache",
Expand Down Expand Up @@ -538,6 +577,14 @@ def ask(
)
raise typer.Exit(code=2)

if stream and json_output is not None:
err_console.print("[red]--json-output cannot be combined with --stream.[/red]")
raise typer.Exit(code=2)

if json_output is not None and (not json_output.parent.is_dir() or json_output.is_dir()):
err_console.print("[red]--json-output requires an existing parent directory.[/red]")
raise typer.Exit(code=2)

if mode_lower == "vote" and not choices:
err_console.print(
"[red]--mode vote requires --choices (e.g. --choices 'Yes,No,Abstain').[/red]"
Expand Down Expand Up @@ -587,11 +634,22 @@ def ask(
result.elite is None or result.elite.decision_readiness != "ready"
)

payload = _result_to_dict(result) if as_json or json_output is not None else None
json_output_failed = False
if json_output is not None:
try:
_write_json_output(json_output, payload or {})
except Exception:
# The completed result must remain available on its normal stdout path
# even when the optional persistence side effect fails.
err_console.print("[red]Could not write --json-output.[/red]")
json_output_failed = True

if as_json:
# Always emit valid JSON to stdout so a consumer can parse the payload,
# then signal failure via the exit code if nothing usable came back.
console.print_json(json.dumps(_result_to_dict(result)))
if no_usable_answers or elite_not_ready:
console.print_json(json.dumps(payload))
if json_output_failed or no_usable_answers or elite_not_ready:
raise typer.Exit(code=1)
return

Expand All @@ -611,6 +669,8 @@ def ask(
f"{result.elite.decision_readiness} ({reasons})[/red]"
)
raise typer.Exit(code=1)
if json_output_failed:
raise typer.Exit(code=1)
return

if no_usable_answers:
Expand All @@ -620,6 +680,8 @@ def ask(
raise typer.Exit(code=1)

_RENDERERS[result.mode](result)
if json_output_failed:
raise typer.Exit(code=1)


@app.command()
Expand Down
Loading