Skip to content

Add Litmus to AI & LLM Testing#111

Merged
TheJambo merged 1 commit intoTheJambo:masterfrom
rylinjames:add-litmus
Mar 29, 2026
Merged

Add Litmus to AI & LLM Testing#111
TheJambo merged 1 commit intoTheJambo:masterfrom
rylinjames:add-litmus

Conversation

@rylinjames
Copy link
Copy Markdown
Contributor

Adding Litmus — an open-source Python CLI that records and replays AI agent LLM calls deterministically.

What makes it unique: Most LLM testing tools focus on prompt evaluation or observability. Litmus captures the full execution trace (every API call) and replays it deterministically — like VCR/cassette testing but for multi-step AI agents. It also supports fault injection (inject timeouts, errors, refusals) and reliability scoring for CI gating.

  • MIT licensed, 47 tests, supports 14+ LLM providers
  • Zero code changes needed — wraps any Python agent
  • pip install litmus-trace

@TheJambo TheJambo merged commit ebd2bcd into TheJambo:master Mar 29, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants