Skip to content

Add agent-backed PubMed fact verifier#9

Open
mindbomber wants to merge 1 commit into
sarvanithin:mainfrom
mindbomber:codex/agent-fact-verifier
Open

Add agent-backed PubMed fact verifier#9
mindbomber wants to merge 1 commit into
sarvanithin:mainfrom
mindbomber:codex/agent-fact-verifier

Conversation

@mindbomber
Copy link
Copy Markdown

Summary

  • Adds an opt-in AgentFactVerifier that retrieves PubMed abstracts and asks the configured LLM caller for a structured JSON verdict.
  • Keeps the existing keyword-backed FactVerifier as the fallback when no LLM caller is configured or the agent path fails.
  • Adds FactCheckConfig.use_agent and wires MedGuard so the agent verifier receives the configured LLM caller.
  • Adds regression tests for reasoned evidence, fallback behavior, and MedGuard component wiring.

Fixes #1.

Verification

  • python -m pytest tests\test_fact_check.py tests\test_pipeline.py -q -> 16 passed
  • python -m ruff check medguard\guardrails\fact_check.py medguard\knowledge\pubmed.py medguard\config.py medguard\core.py tests\test_fact_check.py -> passed
  • python -m compileall medguard\guardrails\fact_check.py medguard\knowledge\pubmed.py medguard\config.py medguard\core.py tests\test_fact_check.py -> passed
  • git diff --check -> clean apart from Windows LF/CRLF warnings

AANA gate

  • Intake gate: pass, recommended action accept
  • Code-change guardrail: pass, recommended action accept
  • AIx score: 1.0
  • Violations: none

Scope note

The agent verifier is a source-grounding aid, not a medical or compliance authority. It only scores claims against retrieved PubMed abstracts and preserves fallback behavior when evidence or LLM output is unavailable.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Agent-based fact verification using LLM reasoning over PubMed results

1 participant