Add retrieval-provider rerank capability (0060a)

[chris-colinsky]

Adds the rerank half of the retrieval-provider capability (proposal 0060), as the first of a two-PR split (0060a here, observability rendering in 0060b), mirroring the 0059a/0059b embedding split.

What's here

• RerankProvider protocol (ready() + rerank(query, documents, *, top_k=None, config=None)), with validate_rerank_input / validate_rerank_response (empty query/documents or top_k <= 0 gives provider_invalid_request; out-of-range/duplicate index or len(results) > top_k gives provider_invalid_response).
• Response types: RerankResponse / ScoredDocument / RerankUsage / RerankRuntimeConfig. usage is record | null from the start (proposal 0093); no fabricated usage records.
• Typed RerankEvent / RerankFailedEvent on the observer event union, mirroring the embedding events (scalar fan_out_index / branch_name; output_results populated on success per 0089).
• CohereRerankProvider, the Cohere-shape reference reranker (POST /v2/rerank): request body {model, query, documents, top_n} (max_tokens_per_doc rides the extras bag; no return_documents, which is a no-op on the Cohere v2 wire), response parse + descending sort + the section 6 invariants.
• Conformance: the calls_rerank harness directive; protocol fixtures 006-012 un-deferred and passing.
• conformance.toml 0060 moves to partial.

Deferred to 0060b

The observer rendering (the OTel openarmature.rerank.complete span, the Langfuse Retriever observation, and rerank metrics; fixtures 099-109 / 138 / 141 / 142). The bundled observers safe-skip the rerank events until then. The Cohere /v2/rerank request wire is implemented but live-untested; a live integration test lands with the Cohere wire mapping (0090), as the OpenAI embed test did with 0079.

Validation

• Protocol fixtures 006-012 pass; full conformance + unit suites green
• ruff + pyright clean

[Copilot]

Pull request overview

Implements the retrieval-provider rerank surface (proposal 0060a) in the Python runtime, including the provider protocol, response/runtime-config types, a Cohere-shaped reference provider, and conformance/unit test coverage. This fits alongside the existing embedding capability by adding rerank call/response normalization, validation, and typed observer events, while deferring observer rendering to a follow-up PR (0060b).

Changes:

• Add RerankProvider protocol plus validate_rerank_input / validate_rerank_response invariants enforcement.
• Introduce rerank response/config types (RerankResponse, ScoredDocument, RerankUsage, RerankRuntimeConfig) and the CohereRerankProvider implementation (POST /v2/rerank).
• Extend conformance harness + tests to run rerank fixtures and to safely skip rerank events in bundled observers until rendering is implemented.

Reviewed changes

Copilot reviewed 14 out of 14 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
tests/unit/test_retrieval_provider.py	Adds unit coverage for rerank validators, Cohere request/response behavior, and typed rerank event dispatch + observer safe-skip behavior.
tests/conformance/test_retrieval_provider.py	Extends conformance runner to execute rerank fixtures (006–012) against CohereRerankProvider and assert rerank invariants/expected state.
tests/conformance/test_observability.py	Updates deferral messaging to reflect rerank rendering deferred to 0060b (not the underlying rerank capability/events).
tests/conformance/test_fixture_parsing.py	Adjusts parser deferrals now that calls_rerank is modeled; keeps only typed-event-collector-shape deferrals here.
tests/conformance/harness/directives.py	Adds CallsRerankSpec and wires it into NodeSpec primary directives.
src/openarmature/retrieval/response.py	Adds rerank response/config/usage models and updates module documentation/exports.
src/openarmature/retrieval/providers/cohere.py	Introduces the Cohere-shaped reference rerank provider with request mapping, response parsing/sorting, validation, and typed-event dispatch.
src/openarmature/retrieval/provider.py	Adds RerankProvider protocol and rerank input/response validators.
src/openarmature/retrieval/init.py	Exports rerank protocols/types and CohereRerankProvider from the retrieval package surface.
src/openarmature/observability/otel/observer.py	Adds type discrimination to safely skip rerank events until rerank span/metrics rendering lands (0060b).
src/openarmature/observability/langfuse/observer.py	Adds type discrimination to safely skip rerank events until Langfuse retriever rendering lands (0060b).
src/openarmature/graph/observer.py	Extends ObserverEvent union to include RerankEvent / RerankFailedEvent.
src/openarmature/graph/events.py	Adds typed RerankEvent / RerankFailedEvent dataclasses to the observer event model.
conformance.toml	Marks proposal 0060 as partial and documents what 0060a ships vs what is deferred to 0060b.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.