Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). The

### Changed

- **Pinned spec advanced from v0.38.0 to v0.45.0.** Submodule + `[tool.openarmature].spec_version` + `conformance.toml` `spec_pin` advance together. Absorbs seven new proposals (0047-0053) into the conformance manifest. Two of them ship as part of the v0.12.0 cycle as textual-only acknowledgments with no code change required: **proposal 0051** (observability §8.4.1 Langfuse `trace.input` / `trace.output` implementation-surface caveat — documents that vendor SDK round-trip is required to project caller-side trace I/O updates onto the wire; the v0.11.0 (proposal 0043) caller-hook shape already matches the documented behavior) and **proposal 0053** (observability §3.4 shared-parent boundary clarification — tightens the structural-shared-parent classification to predicate the invocation span on whether at least one fan-out or parallel-branches dispatch is on the augmenter's call-stack path; behavior already matches via fixtures 034 + 039). The remaining five proposals are marked `not-yet` in the conformance manifest with roadmap targets: 0048 + 0052 (this v0.12.0 cycle), 0047 + 0049 (v0.13.0 LLM provider hardening batch), 0050 (v0.14.0 retry & reliability batch).
- **README and docs homepage refreshed around reasons-to-choose.** Replaced the 10-bullet "Why OpenArmature" feature inventory in `README.md` with 5 differentiating reasons (LLM-infused workflows to agents on one engine; crash-safe resume by contract; destination-pluggable observability with OTel + Langfuse, no SaaS lock-in; compile-time topology checks; spec + conformance). The docs homepage (`docs/index.md`) card grid carries the same five plus a sixth card retained from the previous grid for async-first / LLM-agnostic: workflows-to-agents, crash-safe, pluggable observability, bad-graphs-don't-compile, parallelism (fan-out + parallel-branches + nested correctness), async-first.
- **Docs sweep: stale references and em-dash normalization.** Fixed three definite stale references (`spec_version='0.26.0'` in the Langfuse example output now reads `'0.38.0'`; the dangling `v0.16.1` qualifier dropped from the parallel-branches concept page; `compiled.attach_observer` corrected to `graph.attach_observer` in `non-obvious-shapes.md` for variable-name consistency with the rest of the docs). Swept em dashes out of the user-facing docs (130 instances across 17 files) per the convention set during the patterns expansion. mkdocs strict build clean; no broken intra-docs links.
- **The checkpointing-and-migration example grows a crash-and-resume drama.** The first invoke of the v1 graph now hits a simulated transient failure inside `size_crew` (raises a `RuntimeError` on its first attempt only). The example catches `NodeException` at the `invoke()` boundary, prints what's saved on disk (`define_objective`'s position is already in `completed_positions`), then re-invokes with `resume_invocation=<id>`. The retried `size_crew` succeeds, `draft_timeline` runs, and the pipeline finishes - dramatizing the synchronous-checkpoint-by-contract reliability claim from the README pitch. The existing v1->v2 migration phase rides on top of the crash-survived checkpoint, so both reliability stories compose in one demo. Walk-through doc rewritten to cover both phases.
Expand Down
58 changes: 57 additions & 1 deletion conformance.toml
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@

[manifest]
implementation = "openarmature-python"
spec_pin = "v0.38.0"
spec_pin = "v0.45.0"

# Status values:
# implemented — shipped behavior matches the proposal's contract
Expand Down Expand Up @@ -240,3 +240,59 @@ since = "0.11.0"
[proposals."0046"]
status = "implemented"
since = "0.11.0"

# Spec v0.39.0 (proposal 0047). Implicit prefix-cache wire-byte
# stability. Cross-provider invariant requiring intra-impl byte
# equality across calls with equivalent inputs. Queued for v0.13.0
# alongside 0049 (LLM provider hardening + typed event batch).
[proposals."0047"]
status = "not-yet"

# Spec v0.40.0 (proposal 0048). Read-symmetric invocation metadata
# (``get_invocation_metadata()`` symmetric to ``set_invocation_metadata()``)
# + queryable observer pattern blessing. Queued for v0.12.0 (this
# cycle); will flip to ``implemented`` in the implementation PR.
[proposals."0048"]
status = "not-yet"

# Spec v0.41.0 (proposal 0049). Typed LLM Completion Event — first
# typed event variant on the observer event union. Queued for
# v0.13.0 alongside 0047.
[proposals."0049"]
status = "not-yet"

# Spec v0.42.0 (proposal 0050). Retry & degradation primitives —
# failure-isolation middleware + call-level retry. Queued for
# v0.14.0 (largest single piece in the roadmap).
[proposals."0050"]
status = "not-yet"

# Spec v0.43.0 (proposal 0051). Langfuse trace.input/trace.output
# implementation-surface caveat. Purely textual: documents that the
# Langfuse SDK's ``LangfuseClient`` API requires a trace object
# round-trip to project caller-side ``trace.input`` / ``trace.output``
# updates onto the wire. No module-level change required; the python
# implementation's existing v0.11.0 (proposal 0043) shape already
# matches the documented behavior.
[proposals."0051"]
status = "textual-only"
since = "0.12.0"

# Spec v0.44.0 (proposal 0052). Implementation attribution attributes
# (``openarmature.implementation.name`` + ``openarmature.implementation.version``
# on every invocation span). Queued for v0.12.0 (this cycle); will
# flip to ``implemented`` in the implementation PR.
[proposals."0052"]
status = "not-yet"

# Spec v0.45.0 (proposal 0053). §3.4 shared-parent boundary
# clarification. Purely textual: tightens the structural-shared-parent
# classification to predicate the invocation span on whether at least
# one fan-out or parallel-branches dispatch is on the augmenter's
# call-stack path. Implementation-side: behavior already matches the
# clarified contract via fixtures 034 (outermost-serial augmentation
# reaches the invocation span) and 039 (nested cases do not). No
# module-level change required.
[proposals."0053"]
status = "textual-only"
since = "0.12.0"
2 changes: 1 addition & 1 deletion openarmature-spec
Submodule openarmature-spec updated 94 files
+97 −0 CHANGELOG.md
+30 −0 GOVERNANCE.md
+6 −6 README.md
+147 −0 docs/compatibility.md
+171 −0 docs/open-questions.md
+10 −3 docs/proposals.md
+1 −0 docs/proposals/0047-implicit-prefix-cache-wire-stability.md
+1 −0 docs/proposals/0048-read-symmetric-invocation-metadata-queryable-observer.md
+1 −0 docs/proposals/0049-typed-llm-completion-event.md
+1 −0 docs/proposals/0050-retry-and-degradation-primitives.md
+1 −0 docs/proposals/0051-langfuse-trace-io-deprecation-caveat.md
+1 −0 docs/proposals/0052-implementation-attribution-rows.md
+1 −0 docs/proposals/0053-shared-parent-boundary-clarification.md
+1 −0 mkdocs.yml
+640 −0 proposals/0047-implicit-prefix-cache-wire-stability.md
+534 −0 proposals/0048-read-symmetric-invocation-metadata-queryable-observer.md
+469 −0 proposals/0049-typed-llm-completion-event.md
+659 −0 proposals/0050-retry-and-degradation-primitives.md
+239 −0 proposals/0051-langfuse-trace-io-deprecation-caveat.md
+423 −0 proposals/0052-implementation-attribution-rows.md
+315 −0 proposals/0053-shared-parent-boundary-clarification.md
+49 −0 spec/graph-engine/spec.md
+27 −0 spec/llm-provider/conformance/054-openai-wire-byte-stability.md
+110 −0 spec/llm-provider/conformance/054-openai-wire-byte-stability.yaml
+26 −0 spec/llm-provider/conformance/055-anthropic-wire-byte-stability.md
+96 −0 spec/llm-provider/conformance/055-anthropic-wire-byte-stability.yaml
+43 −0 spec/llm-provider/conformance/056-call-level-retry-transient.md
+49 −0 spec/llm-provider/conformance/056-call-level-retry-transient.yaml
+34 −0 spec/llm-provider/conformance/057-call-level-retry-exhaustion.md
+41 −0 spec/llm-provider/conformance/057-call-level-retry-exhaustion.yaml
+38 −0 spec/llm-provider/conformance/058-call-level-retry-non-transient-no-retry.md
+36 −0 spec/llm-provider/conformance/058-call-level-retry-non-transient-no-retry.yaml
+196 −14 spec/llm-provider/spec.md
+3 −3 spec/observability/conformance/011-otel-determinism.md
+1 −1 spec/observability/conformance/011-otel-determinism.yaml
+6 −0 spec/observability/conformance/028-caller-metadata-namespace-rejection.md
+57 −0 spec/observability/conformance/028-caller-metadata-namespace-rejection.yaml
+35 −0 spec/observability/conformance/040-llm-cache-attribute-emission.md
+60 −0 spec/observability/conformance/040-llm-cache-attribute-emission.yaml
+32 −0 spec/observability/conformance/041-llm-cache-attribute-absence.md
+56 −0 spec/observability/conformance/041-llm-cache-attribute-absence.yaml
+30 −0 spec/observability/conformance/042-llm-cache-attribute-reported-zero.md
+60 −0 spec/observability/conformance/042-llm-cache-attribute-reported-zero.yaml
+33 −0 spec/observability/conformance/043-get-invocation-metadata-roundtrip.md
+38 −0 spec/observability/conformance/043-get-invocation-metadata-roundtrip.yaml
+41 −0 spec/observability/conformance/044-get-invocation-metadata-fan-out-scoping.md
+72 −0 spec/observability/conformance/044-get-invocation-metadata-fan-out-scoping.yaml
+35 −0 spec/observability/conformance/045-get-invocation-metadata-retry-scoping.md
+64 −0 spec/observability/conformance/045-get-invocation-metadata-retry-scoping.yaml
+37 −0 spec/observability/conformance/046-get-invocation-metadata-outside-invocation.md
+26 −0 spec/observability/conformance/046-get-invocation-metadata-outside-invocation.yaml
+55 −0 spec/observability/conformance/047-queryable-observer-pattern.md
+63 −0 spec/observability/conformance/047-queryable-observer-pattern.yaml
+46 −0 spec/observability/conformance/048-queryable-observer-async-safety.md
+70 −0 spec/observability/conformance/048-queryable-observer-async-safety.yaml
+51 −0 spec/observability/conformance/049-queryable-observer-lifecycle-drop.md
+85 −0 spec/observability/conformance/049-queryable-observer-lifecycle-drop.yaml
+44 −0 spec/observability/conformance/050-llm-completion-event-dispatch.md
+70 −0 spec/observability/conformance/050-llm-completion-event-dispatch.yaml
+51 −0 spec/observability/conformance/051-llm-completion-event-type-discrimination.md
+60 −0 spec/observability/conformance/051-llm-completion-event-type-discrimination.yaml
+45 −0 spec/observability/conformance/052-llm-completion-event-caller-metadata-opt-in.md
+101 −0 spec/observability/conformance/052-llm-completion-event-caller-metadata-opt-in.yaml
+42 −0 spec/observability/conformance/053-llm-completion-event-no-event-on-failure.md
+50 −0 spec/observability/conformance/053-llm-completion-event-no-event-on-failure.yaml
+42 −0 spec/observability/conformance/054-llm-completion-event-fan-out-index-population.md
+81 −0 spec/observability/conformance/054-llm-completion-event-fan-out-index-population.yaml
+40 −0 spec/observability/conformance/055-llm-completion-event-branch-name-population.md
+86 −0 spec/observability/conformance/055-llm-completion-event-branch-name-population.yaml
+46 −0 spec/observability/conformance/056-llm-completion-event-strict-serial-ordering.md
+59 −0 spec/observability/conformance/056-llm-completion-event-strict-serial-ordering.yaml
+40 −0 spec/observability/conformance/057-llm-attempt-index-single-attempt-default.md
+57 −0 spec/observability/conformance/057-llm-attempt-index-single-attempt-default.yaml
+52 −0 spec/observability/conformance/058-implementation-attribution-otel.md
+132 −0 spec/observability/conformance/058-implementation-attribution-otel.yaml
+47 −0 spec/observability/conformance/059-implementation-attribution-langfuse.md
+67 −0 spec/observability/conformance/059-implementation-attribution-langfuse.yaml
+340 −26 spec/observability/spec.md
+39 −0 spec/pipeline-utilities/conformance/058-failure-isolation-static-degraded.md
+64 −0 spec/pipeline-utilities/conformance/058-failure-isolation-static-degraded.yaml
+32 −0 spec/pipeline-utilities/conformance/059-failure-isolation-callable-degraded.md
+61 −0 spec/pipeline-utilities/conformance/059-failure-isolation-callable-degraded.yaml
+31 −0 spec/pipeline-utilities/conformance/060-failure-isolation-predicate-filtering.md
+78 −0 spec/pipeline-utilities/conformance/060-failure-isolation-predicate-filtering.yaml
+41 −0 spec/pipeline-utilities/conformance/061-failure-isolation-retry-three-piece-composition.md
+75 −0 spec/pipeline-utilities/conformance/061-failure-isolation-retry-three-piece-composition.yaml
+42 −0 spec/pipeline-utilities/conformance/062-failure-isolation-on-caught-callback.md
+64 −0 spec/pipeline-utilities/conformance/062-failure-isolation-on-caught-callback.yaml
+41 −0 spec/pipeline-utilities/conformance/063-failure-isolation-default-predicate-bare-exception.md
+57 −0 spec/pipeline-utilities/conformance/063-failure-isolation-default-predicate-bare-exception.yaml
+102 −0 spec/pipeline-utilities/spec.md
+41 −0 spec/prompt-management/conformance/032-cross-variable-substring-stability.md
+89 −0 spec/prompt-management/conformance/032-cross-variable-substring-stability.yaml
+58 −1 spec/prompt-management/spec.md
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ Specification = "https://github.com/LunarCommand/openarmature-spec"
openarmature = "openarmature.cli:main"

[tool.openarmature]
spec_version = "0.38.0"
spec_version = "0.45.0"

[dependency-groups]
dev = [
Expand Down
4 changes: 2 additions & 2 deletions src/openarmature/AGENTS.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# OpenArmature — Agent documentation

*This is the agent guide bundled with the openarmature Python package, version 0.11.0 (spec v0.38.0). For the full docs site see [openarmature.ai](https://openarmature.ai). For the canonical spec text see [openarmature.org/capabilities](https://openarmature.org/capabilities/). For project-specific conventions for the code you're editing, see the host project's `AGENTS.md` or `CLAUDE.md`.*
*This is the agent guide bundled with the openarmature Python package, version 0.11.0 (spec v0.45.0). For the full docs site see [openarmature.ai](https://openarmature.ai). For the canonical spec text see [openarmature.org/capabilities](https://openarmature.org/capabilities/). For project-specific conventions for the code you're editing, see the host project's `AGENTS.md` or `CLAUDE.md`.*

## TL;DR

Expand All @@ -10,7 +10,7 @@ OpenArmature is a workflow framework for LLM pipelines and tool-calling agents:

## Capability contracts

_Sourced from openarmature-spec v0.38.0. Each entry below reproduces §1 (Purpose) and §2 (Concepts) of the capability's `spec.md`. For the full spec text (execution model, error semantics, determinism, observer hooks, etc.) see the linked docs site._
_Sourced from openarmature-spec v0.45.0. Each entry below reproduces §1 (Purpose) and §2 (Concepts) of the capability's `spec.md`. For the full spec text (execution model, error semantics, determinism, observer hooks, etc.) see the linked docs site._

### Capability: `graph-engine`

Expand Down
2 changes: 1 addition & 1 deletion src/openarmature/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -25,4 +25,4 @@
"""

__version__ = "0.11.0"
__spec_version__ = "0.38.0"
__spec_version__ = "0.45.0"
39 changes: 39 additions & 0 deletions tests/conformance/_deferral.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
"""Shared deferral plumbing for conformance test files.

Each fixture-level conformance test (``test_fixture_parsing``,
``test_llm_provider``, ``test_prompt_management``) maintains its own
``_DEFERRED_FIXTURES`` dict mapping fixture IDs to human-readable
reasons. Entries come and go as proposals land in their implementing
PRs.

This helper centralizes the skip-call site so the format
(``f"{fixture_id}: {reason}"``) stays consistent across files and so
future tweaks to the message shape land in one place.

The case-level skip inside ``_run_fixture_028`` in
``test_observability.py`` deliberately doesn't use this helper — it
executes ``continue`` inside an async loop iterating sub-cases of a
single fixture rather than skipping the outer pytest test.
"""

from __future__ import annotations

from collections.abc import Mapping

import pytest


def skip_if_deferred(fixture_id: str, deferrals: Mapping[str, str]) -> None:
"""Raise :func:`pytest.skip` if ``fixture_id`` appears in ``deferrals``.

Standard call site at the top of a conformance test that processes
one fixture per pytest test::

@pytest.mark.parametrize(...)
def test_xxx_fixture(fixture_path: Path) -> None:
fixture_id = _fixture_id(fixture_path)
skip_if_deferred(fixture_id, _DEFERRED_FIXTURES)
# ... rest of the test ...
"""
if fixture_id in deferrals:
pytest.skip(f"{fixture_id}: {deferrals[fixture_id]}")
110 changes: 106 additions & 4 deletions tests/conformance/test_fixture_parsing.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@

import pytest

from ._deferral import skip_if_deferred
from .harness import discover_fixtures, load_fixture


Expand Down Expand Up @@ -243,6 +244,109 @@ def _id(case: tuple[str, Path]) -> str:
# Proposal 0046 (chat-prompt rendering, v0.38.0) — fixtures 017-031
# activate in PR 12 against the prompt-management harness's
# extended ``chat_template`` + ``placeholders`` directive shapes.
# ----- v0.12.0 cycle spec-pin bump (v0.38.0 -> v0.45.0) ----------------
# Proposal 0047 (implicit prefix-cache wire-byte stability, v0.39.0)
# — fixtures 054-055 require the wire-byte hashing directive shape.
# Queued for v0.13.0 LLM provider hardening batch.
"llm-provider/054-openai-wire-byte-stability": ("Proposal 0047 wire-byte stability; queued for v0.13.0"),
"llm-provider/055-anthropic-wire-byte-stability": (
"Proposal 0047 wire-byte stability; queued for v0.13.0"
),
# Proposal 0047 also adds cache attribute emission fixtures
# (observability/040-042). Same v0.13.0 batch.
"observability/040-llm-cache-attribute-emission": (
"Proposal 0047 cache attribute emission; queued for v0.13.0"
),
"observability/041-llm-cache-attribute-absence": (
"Proposal 0047 cache attribute emission; queued for v0.13.0"
),
"observability/042-llm-cache-attribute-reported-zero": (
"Proposal 0047 cache attribute emission; queued for v0.13.0"
),
# Proposal 0048 (read-symmetric metadata + queryable observer, v0.40.0)
# — fixtures 043-049 require the ``capture_invocation_metadata_into`` /
# ``augment_metadata`` / queryable-observer directive shapes. Lands
# in PR 2 of the v0.12.0 cycle along with the implementation; this
# PR-1 deferral keeps the parse tests green in the meantime.
"observability/043-get-invocation-metadata-roundtrip": (
"Proposal 0048 read-symmetric metadata; lands in PR 2 of v0.12.0"
),
"observability/044-get-invocation-metadata-fan-out-scoping": (
"Proposal 0048 read-symmetric metadata; lands in PR 2 of v0.12.0"
),
"observability/045-get-invocation-metadata-retry-scoping": (
"Proposal 0048 read-symmetric metadata; lands in PR 2 of v0.12.0"
),
"observability/046-get-invocation-metadata-outside-invocation": (
"Proposal 0048 read-symmetric metadata; lands in PR 2 of v0.12.0"
),
"observability/047-queryable-observer-pattern": (
"Proposal 0048 queryable observer pattern; lands in PR 2 of v0.12.0"
),
"observability/048-queryable-observer-async-safety": (
"Proposal 0048 queryable observer pattern; lands in PR 2 of v0.12.0"
),
"observability/049-queryable-observer-lifecycle-drop": (
"Proposal 0048 queryable observer pattern; lands in PR 2 of v0.12.0"
),
# Proposal 0049 (typed LLM completion event, v0.41.0) — fixtures
# 050-056 require the ``LlmCompletionEvent`` typed-event directive
# shape. Queued for v0.13.0 LLM provider hardening batch.
"observability/050-llm-completion-event-dispatch": (
"Proposal 0049 typed LLM completion event; queued for v0.13.0"
),
"observability/051-llm-completion-event-type-discrimination": (
"Proposal 0049 typed LLM completion event; queued for v0.13.0"
),
"observability/052-llm-completion-event-caller-metadata-opt-in": (
"Proposal 0049 typed LLM completion event; queued for v0.13.0"
),
"observability/053-llm-completion-event-no-event-on-failure": (
"Proposal 0049 typed LLM completion event; queued for v0.13.0"
),
"observability/054-llm-completion-event-fan-out-index-population": (
"Proposal 0049 typed LLM completion event; queued for v0.13.0"
),
"observability/055-llm-completion-event-branch-name-population": (
"Proposal 0049 typed LLM completion event; queued for v0.13.0"
),
"observability/056-llm-completion-event-strict-serial-ordering": (
"Proposal 0049 typed LLM completion event; queued for v0.13.0"
),
# Proposal 0050 (failure-isolation middleware + call-level retry,
# v0.42.0) — llm-provider fixtures 056-058 (call-level retry) and
# pipeline-utilities fixtures 058-063 (failure-isolation
# middleware) require new directive shapes. Queued for v0.14.0
# retry & reliability primitives batch.
"llm-provider/056-call-level-retry-transient": ("Proposal 0050 call-level retry; queued for v0.14.0"),
"llm-provider/057-call-level-retry-exhaustion": ("Proposal 0050 call-level retry; queued for v0.14.0"),
"llm-provider/058-call-level-retry-non-transient-no-retry": (
"Proposal 0050 call-level retry; queued for v0.14.0"
),
"pipeline-utilities/058-failure-isolation-static-degraded": (
"Proposal 0050 failure-isolation middleware; queued for v0.14.0"
),
"pipeline-utilities/059-failure-isolation-callable-degraded": (
"Proposal 0050 failure-isolation middleware; queued for v0.14.0"
),
"pipeline-utilities/060-failure-isolation-predicate-filtering": (
"Proposal 0050 failure-isolation middleware; queued for v0.14.0"
),
"pipeline-utilities/061-failure-isolation-retry-three-piece-composition": (
"Proposal 0050 failure-isolation middleware; queued for v0.14.0"
),
"pipeline-utilities/062-failure-isolation-on-caught-callback": (
"Proposal 0050 failure-isolation middleware; queued for v0.14.0"
),
"pipeline-utilities/063-failure-isolation-default-predicate-bare-exception": (
"Proposal 0050 failure-isolation middleware; queued for v0.14.0"
),
# Proposal 0052 (implementation attribution attributes, v0.44.0)
# — observability/059 is the Langfuse-side mapping fixture. Lands
# in PR 3 of the v0.12.0 cycle along with the implementation.
"observability/059-implementation-attribution-langfuse": (
"Proposal 0052 implementation attribution; lands in PR 3 of v0.12.0"
),
}


Expand All @@ -260,8 +364,7 @@ def test_fixture_parses(case: tuple[str, Path]) -> None:
``GraphFixture`` based on top-level keys; ``extra="forbid"`` rejects
any unknown top-level field."""
case_id = _id(case)
if case_id in _DEFERRED_FIXTURES:
pytest.skip(f"{case_id}: {_DEFERRED_FIXTURES[case_id]}")
skip_if_deferred(case_id, _DEFERRED_FIXTURES)
_, path = case
load_fixture(path)

Expand All @@ -272,8 +375,7 @@ def test_fixture_round_trips(case: tuple[str, Path]) -> None:
Phase 0 per the implementation plan: catches dropped fields the user
intended to use later."""
case_id = _id(case)
if case_id in _DEFERRED_FIXTURES:
pytest.skip(f"{case_id}: {_DEFERRED_FIXTURES[case_id]}")
skip_if_deferred(case_id, _DEFERRED_FIXTURES)
_, path = case
parsed = load_fixture(path)
dumped = parsed.model_dump(exclude_none=True)
Expand Down
18 changes: 16 additions & 2 deletions tests/conformance/test_llm_provider.py
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,7 @@
UserMessage,
)

from ._deferral import skip_if_deferred
from .harness import (
assert_error_carries,
assert_response_format_absent,
Expand Down Expand Up @@ -94,6 +95,20 @@
"051-gemini-structured-output-fallback": "Gemini provider not implemented (0038 not-yet)",
"052-gemini-thought-signature-round-trip": "Gemini provider not implemented (0038 not-yet)",
"053-cross-provider-signature-strip": "Gemini provider not implemented (0038 not-yet)",
# ----- v0.12.0 cycle spec-pin bump (v0.38.0 -> v0.45.0) -------------
# Proposal 0047 (implicit prefix-cache wire-byte stability, v0.39.0)
# — wire-byte hashing across providers. Queued for v0.13.0 LLM
# provider hardening batch.
"054-openai-wire-byte-stability": ("Proposal 0047 wire-byte stability; queued for v0.13.0"),
"055-anthropic-wire-byte-stability": (
"Proposal 0047 wire-byte stability; queued for v0.13.0 (also Anthropic-pending)"
),
# Proposal 0050 (call-level retry, v0.42.0) — three fixtures
# exercise the new ``retry`` kwarg on ``complete()``. Queued for
# v0.14.0 retry & reliability primitives batch.
"056-call-level-retry-transient": ("Proposal 0050 call-level retry; queued for v0.14.0"),
"057-call-level-retry-exhaustion": ("Proposal 0050 call-level retry; queued for v0.14.0"),
"058-call-level-retry-non-transient-no-retry": ("Proposal 0050 call-level retry; queued for v0.14.0"),
}


Expand Down Expand Up @@ -458,8 +473,7 @@ def _assert_raises_matches(
@pytest.mark.parametrize("fixture_path", _fixture_paths(), ids=_fixture_id)
async def test_llm_provider_fixture(fixture_path: Path) -> None:
fixture_id = fixture_path.stem
if fixture_id in _DEFERRED_FIXTURES:
pytest.skip(f"{fixture_id}: {_DEFERRED_FIXTURES[fixture_id]}")
skip_if_deferred(fixture_id, _DEFERRED_FIXTURES)
spec = _load(fixture_path)

if "cases" in spec:
Expand Down
19 changes: 19 additions & 0 deletions tests/conformance/test_observability.py
Original file line number Diff line number Diff line change
Expand Up @@ -892,9 +892,28 @@ async def _run_fixture_028(spec: Mapping[str, Any]) -> None:
LangfuseObserver,
)

# Case-level deferrals for fixture 028. The spec extends the
# fixture with new negative-control cases as new reserved keys
# land in §3.4; impl-side coverage for those cases lands in the
# PR that implements the corresponding key reservation. Stored as
# a set since the skip is a ``continue`` (no ``pytest.skip``
# reason surface); rationale lives in the per-block comment above
# each name.
_deferred_cases: set[str] = {
# Proposal 0052 (implementation attribution, spec v0.44.0)
# extends the reserved-set 24 → 26 names with
# ``implementation_name`` / ``implementation_version``.
# Coverage lands in PR 3 of the v0.12.0 cycle alongside the
# ``openarmature.implementation.*`` invocation-span attribute
# emission.
"rejects_reserved_oa_name_implementation_name",
"rejects_reserved_oa_name_implementation_version",
}
cases = cast("list[dict[str, Any]]", spec["cases"])
for case in cases:
case_name = cast("str", case["name"])
if case_name in _deferred_cases:
continue
try:
# Build a minimal graph from the case's nodes/edges. The
# fixture's node is a noop update — we never expect it to
Expand Down
Loading