Synkro reads your policy documents, auto-generates diverse test scenarios, runs multi-turn conversations against your agent with a simulated user, and verifies every response against the policy rules. No hand-written test cases. No manual review.
pip install synkroimport synkro
def my_agent(messages):
resp = openai.chat.completions.create(model="gpt-4o", messages=messages)
return resp.choices[0].message.content
results = synkro.simulate(
agent=my_agent,
policy="All refunds require a receipt. Max refund is $500.",
scenarios=10,
turns=3,
)
print(results.pass_rate) # 0.85
print(results.passed) # 8 of 10 passed
for r in results:
if not r.passed:
print(r.scenario.description, r.issues)Works with any agent — just pass a callable that takes messages and returns a string. Sync or async.
Use the same pipeline to generate training datasets for fine-tuning:
import synkro
dataset = synkro.generate(
"All refunds require a receipt. Max refund is $500.",
traces=100,
)
dataset.save("training.jsonl")Or use the CLI:
synkro generate policy.pdf --traces 50
# Quick demo with built-in policy
synkro demo- Agent simulation — Test agents against auto-generated, policy-grounded scenarios
- Policy verification — Every conversation verified for rule compliance, contradictions, hallucinations
- Training data generation — Conversation, Instruction, Evaluation, and Tool Calling datasets
- Coverage tracking — Track scenario diversity like code coverage, identify gaps
- Any LLM — OpenAI, Anthropic, Google, Ollama, vLLM
- Any document — PDF, DOCX, TXT, Markdown, URLs
Full documentation at synkro.sh/docs