Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 37 additions & 0 deletions docs/_reference/ecosystem.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ After reading this guide, you will know:
* How `RubyLLM::RedCandle` enables local model execution from Ruby
* How OpenTelemetry instrumentation for RubyLLM provides observability into your LLM applications
* How to test application code by stubbing responses with `RubyLLM::Test`
* How `RubyLLM::Contract` adds runtime contracts, model-escalating retries, and regression evals on top of RubyLLM
* Where to find community projects and how to contribute your own

## RubyLLM::Schema
Expand Down Expand Up @@ -283,6 +284,42 @@ For detailed documentation and examples, visit the [RubyLLM::Tribunal repository

---

## RubyLLM::Contract

**Contracts and Evals for LLM Outputs**

[`RubyLLM::Contract`](https://github.com/justi/ruby_llm-contract) wraps `RubyLLM::Chat` with input/output contracts, business-rule validation, retry with model escalation, pre-flight cost ceilings, and a regression-eval framework. It catches schema-valid-but-logically-wrong output before it reaches your code.

### Why Use RubyLLM::Contract?

LLMs can return JSON that looks correct - valid shape, right types, right fields - while being silently wrong in ways schema validation alone doesn't catch. You often need:

- Business rules that schema can't express
- Retry with model escalation when a cheap model's output fails the contract
- Regression evals with baselines to block prompt regressions in CI
- Pre-flight cost ceilings so a large input doesn't blow your budget

### Key Features

- Class-based DSL: `prompt`, `output_schema`, `validate`, `retry_policy`, `max_cost`
- Schema validation via [`RubyLLM::Schema`](https://github.com/danielfriis/ruby_llm-schema) with client-side verification
- Model escalation on validation failure and pre-flight refusal on cost limits
- LLM-as-judge checks and a regression eval framework with frozen datasets and baselines
- Pipeline composition with fail-fast and per-step models
- RSpec / Minitest matchers (`pass_eval`, `satisfy_contract`, `stub_step`)

`RubyLLM::Contract` is runtime - it gates the LLM call and retries on failure - while [`RubyLLM::Tribunal`](https://github.com/Alqemist-labs/ruby_llm-tribunal) grades outputs at test time; the two compose well in the same project.

### Installation

```bash
gem install ruby_llm-contract
```

For detailed documentation and examples, visit the [RubyLLM::Contract repository](https://github.com/justi/ruby_llm-contract).

---

## RubyLLM::TopSecret

**Automatically filter sensitive information from RubyLLM conversations using Top Secret.**
Expand Down
Loading