feat(report): POST /api/report/<id>/signal — extract structured miro_signal from simulation#321
Open
LoryGlory wants to merge 3 commits into666ghj:mainfrom
Open
feat(report): POST /api/report/<id>/signal — extract structured miro_signal from simulation#321LoryGlory wants to merge 3 commits into666ghj:mainfrom
LoryGlory wants to merge 3 commits into666ghj:mainfrom
Conversation
…r services - chat_json() gains configurable retry (max_attempts), temperature backoff (temperature_step), optional retry delay, and a fallback_parser hook for service-specific rescue logic - _clean_response_text: strip <think> tags and markdown code fences - _fix_truncated_json: use unescaped-quote parity instead of last-char check to avoid spuriously quoting numeric values; fixes broken repair for arrays - _try_fix_json: generic near-valid JSON salvage (newline normalisation, control-character stripping, greedy object extraction) - simulation_config_generator: replace raw OpenAI client with LLMClient, remove duplicated local retry loop and truncated-JSON repair, pass service-specific config salvage as fallback_parser - oasis_profile_generator: same refactor; keep rule-based profile fallback Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
30 tests covering _clean_response_text (think tags, markdown fences), _fix_truncated_json (unclosed braces/brackets/strings), _try_fix_json (near-valid JSON recovery), and chat_json retry behaviour (max_attempts, temperature backoff, fallback_parser, ValueError after all attempts fail, finish_reason==length truncation repair, API exception handling). All tests use mocked OpenAI client — no real API calls. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…from simulation report
Adds a canonical machine-readable probability signal endpoint that distils
a completed simulation report into a structured prediction thesis.
New: backend/app/services/signal_extractor.py
- SignalExtractor.extract() calls chat_json() (3 attempts, temp 0.1, step 0.05)
against the report markdown and returns a validated MiroSignal dataclass
- Validates and normalises all fields: p_yes clamped to [0.01, 0.99],
confidence/action enums enforced, action recomputed from p_yes when invalid
- _trim_report() keeps the tail (conclusions) for long reports to stay within
token limits
- _salvage() fallback_parser recovers a minimal signal from partial LLM output
using regex probability extraction
New endpoint: POST /api/report/<report_id>/signal
- 404 if report not found
- 400 if report not yet completed or content is empty
- 422 if LLM fails after all retry attempts
- Returns canonical signal with thesis.{p_yes, confidence, action, regime,
summary, drivers, invalidators} — schema_version 1.1
New: backend/tests/services/test_signal_extractor.py
- 27 tests covering happy path, field validation/normalisation, report
trimming, _salvage fallback, and LLM failure propagation
- No real API calls — LLMClient fully mocked
Closes 666ghj#277
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
mohamedsorour1998
added a commit
to mohamedsorour1998/MiroFish
that referenced
this pull request
Mar 27, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
MiroFish produces rich simulation reports but no machine-readable output contract. External pipelines that want to consume a prediction probability (e.g. prediction-market bots, calibration trackers) have to parse unstructured markdown — or receive a JSON block that doesn't match any stable schema. This was reported in #277.
Solution
A new
POST /api/report/<report_id>/signalendpoint that distils a completed report into a canonical probability signal with a stable schema.Changes
backend/app/services/signal_extractor.py(new)SignalExtractor.extract()takes the report'smarkdown_contentandsimulation_requirementand useschat_json()to extract a structured thesis:_trim_report()keeps the tail of long reports (conclusions matter most for signal extraction)p_yesclamped to[0.01, 0.99]confidenceenforced tohigh | medium | low, falls back tomediumactionenforced tobuy_yes | buy_no | hold, recomputed fromp_yesif invalidregimedefaults touncertainif missing_salvage()is wired asfallback_parser: if the LLM returns near-valid output, a regex scan extracts any float probability and constructs a minimal signal rather than failingbackend/app/api/report.pyNew route
POST /api/report/<report_id>/signal:404— report not found400— report not yet completed, or empty content422— LLM failed after all retry attempts (surfaced asValueError)200— returns canonical signalbackend/tests/services/test_signal_extractor.py(new)27 tests, LLMClient fully mocked:
to_dict()structure, LLM call parametersp_yesclamping, invalidconfidence/actionfallbacks, missingregime_salvage: probability extraction, action derivation, confidence detectionValueErrorpropagates correctlyCanonical signal schema (v1.1)
{ "signal_id": "<uuid>", "schema_version": "1.1", "report_id": "report_xxxx", "simulation_id": "sim_xxxx", "generated_at": "2026-...", "thesis": { "p_yes": 0.73, "confidence": "high", "action": "buy_yes", "regime": "consensus_forming", "summary": "Strong agent consensus supports a YES outcome.", "drivers": ["70%+ agent agreement", "positive social momentum"], "invalidators": ["marginal counter-narrative", "low information diversity"] } }Dependency
This PR is built on top of #318 (
feat/llm-structured-output-reliability) and uses its retry and JSON repair infrastructure inchat_json(). It can be merged independently if #318 lands first, or merged together with it.Fixes
Closes #277
Testing