diff --git a/docs/_reference/ecosystem.md b/docs/_reference/ecosystem.md index 59b3f116a..db2056909 100644 --- a/docs/_reference/ecosystem.md +++ b/docs/_reference/ecosystem.md @@ -31,6 +31,7 @@ After reading this guide, you will know: * How `RubyLLM::RedCandle` enables local model execution from Ruby * How OpenTelemetry instrumentation for RubyLLM provides observability into your LLM applications * How to test application code by stubbing responses with `RubyLLM::Test` +* How `RubyLLM::Contract` adds runtime contracts, model-escalating retries, and regression evals on top of RubyLLM * Where to find community projects and how to contribute your own ## RubyLLM::Schema @@ -283,6 +284,42 @@ For detailed documentation and examples, visit the [RubyLLM::Tribunal repository --- +## RubyLLM::Contract + +**Contracts and Evals for LLM Outputs** + +[`RubyLLM::Contract`](https://github.com/justi/ruby_llm-contract) wraps `RubyLLM::Chat` with input/output contracts, business-rule validation, retry with model escalation, pre-flight cost ceilings, and a regression-eval framework. It catches schema-valid-but-logically-wrong output before it reaches your code. + +### Why Use RubyLLM::Contract? + +LLMs can return JSON that looks correct - valid shape, right types, right fields - while being silently wrong in ways schema validation alone doesn't catch. You often need: + +- Business rules that schema can't express +- Retry with model escalation when a cheap model's output fails the contract +- Regression evals with baselines to block prompt regressions in CI +- Pre-flight cost ceilings so a large input doesn't blow your budget + +### Key Features + +- Class-based DSL: `prompt`, `output_schema`, `validate`, `retry_policy`, `max_cost` +- Schema validation via [`RubyLLM::Schema`](https://github.com/danielfriis/ruby_llm-schema) with client-side verification +- Model escalation on validation failure and pre-flight refusal on cost limits +- LLM-as-judge checks and a regression eval framework with frozen datasets and baselines +- Pipeline composition with fail-fast and per-step models +- RSpec / Minitest matchers (`pass_eval`, `satisfy_contract`, `stub_step`) + +`RubyLLM::Contract` is runtime - it gates the LLM call and retries on failure - while [`RubyLLM::Tribunal`](https://github.com/Alqemist-labs/ruby_llm-tribunal) grades outputs at test time; the two compose well in the same project. + +### Installation + +```bash +gem install ruby_llm-contract +``` + +For detailed documentation and examples, visit the [RubyLLM::Contract repository](https://github.com/justi/ruby_llm-contract). + +--- + ## RubyLLM::TopSecret **Automatically filter sensitive information from RubyLLM conversations using Top Secret.**