Commit 2ecb7b1
authored
feat(llm): structured output (proposal 0016) (#42)
* chore: bump spec to v0.15.0; add jsonschema; skip deferred fixtures
Spec submodule moves from v0.10.0 to v0.15.0 — covers the full
5-proposal batch (0011, 0014, 0015, 0016, 0017) in one bump per
the skip-ahead governance principle. spec_version in pyproject.toml
bumped to match.
Adds jsonschema>=4.0 as a runtime dependency (used by the
forthcoming structured-output validation path on the dict-schema
side; Pydantic-class path uses its own validator).
Adds skip markers to the conformance test files for fixtures whose
runtime support lands in a later PR of the batch:
- llm-provider 009-020 → 0015 multimodal (PR-2)
- llm-provider 021-028 → 0016 structured output (this PR, wired up
in a later commit)
- pipeline-utilities 032-038 → 0011 parallel branches (PR-5)
- pipeline-utilities 039-046 → 0014 state migration (PR-4)
- graph-engine 021-observer-branch-name → 0011 parallel branches
(PR-5)
Skip markers also apply to test_fixture_parsing.py for the same set
— the typed harness models in tests/conformance/harness/ don't yet
know about the new directive shapes (state_migration, parallel
branches state-schema variation, NodeEvent.branch_name); each
deferring PR drops its own skip rows when it lands the harness
work.
* feat(llm): add StructuredOutputInvalid error category
Adds the structured_output_invalid canonical category. Raised when a
complete() call requested a response_schema and the provider's
returned content could not be parsed as JSON OR did not validate
against the schema. The exception carries response_schema,
raw_content, and failure_description attributes for caller
introspection.
Non-transient by default — NOT added to TRANSIENT_CATEGORIES. The
default RetryMiddleware classifier will not retry this category;
callers wanting retry-on-validation-failure can include the category
in a custom classifier's transient set.
* feat(llm): add Response.parsed field
Adds the parsed field to the Response record. Default None, populated
by structured-output calls (response_schema set on complete() and the
model returned structured content). The runtime type is a discriminated
union over dict (when the caller passed a JSON-Schema dict) and
BaseModel instance (when the caller passed a Pydantic class).
Pydantic Response config now allows arbitrary types so a BaseModel
instance can sit in the parsed slot. No public surface change for
free-form callers — parsed defaults to None and remains None when
response_schema is not supplied.
* feat(llm): validate_response_schema + strict_mode_supported helpers
Adds two provider-agnostic helpers in openarmature.llm.provider used
by structured-output Provider implementations:
- validate_response_schema(schema) — pre-send structural check that
the value is a dict and its top-level type is "object". Raises
ProviderInvalidRequest on failure.
- strict_mode_supported(schema) — whether the schema satisfies the
strict-mode constraint set (additionalProperties not true,
properties fully covered by required) across the full schema tree.
Walks anyOf/oneOf/allOf branches and follows $ref targets with
cycle protection. An unresolvable $ref or unknown shape returns
False (conservative fail).
Both are exported from openarmature.llm so OpenAI-compatible
providers and any future Anthropic/Gemini provider share the same
constraint heuristic.
* feat(llm): Provider Protocol gains response_schema parameter
Extends the Provider Protocol's complete() method signature to accept
an optional response_schema parameter. Accepts either a JSON Schema
dict or a Pydantic BaseModel subclass; the implementation converts
the class form to a JSON Schema at the boundary.
Free-form callers (response_schema=None or absent) see no behavior
change — the parameter defaults to None and the v0.4.0 contract is
preserved.
OpenAIProvider's complete() still has the v0.4.0 signature; the next
commit wires the response_schema parameter through it.
* feat(llm/openai): native response_format wire path + Pydantic overload
Threads response_schema through OpenAIProvider.complete() → _do_complete()
→ _parse_response(). Accepts either a JSON Schema dict OR a Pydantic
BaseModel subclass; the latter is converted via model_json_schema()
at the boundary.
Native wire path: when response_schema is supplied, the request body
includes response_format: { type: "json_schema", json_schema: { name,
schema, strict } }. The name field comes from schema.title when
non-empty, otherwise a deterministic sha256 hash of the schema. The
strict flag is set per strict_mode_supported() — true only when the
schema cleanly satisfies the constraints across the full tree.
Post-receive: parses message.content as JSON, then validates against
the schema. Dict-input path validates with jsonschema and returns a
dict. BaseModel-class-input path validates with model.model_validate()
and returns a BaseModel instance. Either way, JSON parse failure or
schema validation failure raises StructuredOutputInvalid carrying the
schema, raw content, and failure description.
parsed is absent on tool-call responses regardless of whether
response_schema was supplied (mutually exclusive paths). Free-form
calls (response_schema=None) see no behavior change — body omits
response_format, parsed stays None.
The prompt-augmentation fallback path is the next commit.
* feat(llm/openai): prompt-augmentation fallback + inspect property
Adds the prompt-augmentation fallback for OpenAI-compatible servers
that don't implement response_format (older vLLM, some LM Studio
releases, llama.cpp variants).
Constructor:
force_prompt_augmentation_fallback: bool = False
When True, structured-output calls build the wire body by augmenting
the message list with a system directive that includes the serialized
JSON Schema, and omit response_format entirely. Native path is the
default (False).
Inspect property:
uses_prompt_augmentation_fallback -> bool
Read-only; lets callers verify which wire path is active without
poking private state.
_augment_messages_with_schema_directive returns a fresh list. When
the first message is system, its content is extended with the schema
directive (preserving caller intent); otherwise a new system message
is prepended. The caller's original messages list is NOT mutated —
Message instances are reused unchanged (immutable Pydantic models).
Response parsing is unchanged from the native path: parse + validate
post-receive raise StructuredOutputInvalid on failure. parsed is
populated identically whether the wire took the native or fallback
route.
* test(conformance): capability-agnostic harness helpers for wire + carries
Adds tests/conformance/harness/wire.py with helpers used by structured-
output and content-block fixtures (and any future capability fixtures
that need the same shapes):
- match_wire_body(actual, expected) — recursive deep-equal with "*"
wildcard support for string slots.
- assert_response_format_absent(body) — asserts the wire body has no
response_format key.
- assert_system_references_schema(body, schema) — asserts the first
message in the body is a system message whose content contains the
canonical-JSON form of the schema as a substring.
- assert_error_carries(exc, carries) — introspects a raised
exception's attributes against an expected_carries block; supports
_present / _mentions / literal-equal forms; handles the
raw_response_content → raw_content fixture-vs-impl naming alias.
Extends test_llm_provider.py to drive these from the existing
fixture loop:
- response_schema is read from call_spec and threaded through
provider.complete().
- expected_wire_request literal compare + expected_wire_request_checks
sibling checks fire after each captured chat-completions request.
- caller_messages_unmodified takes a model_dump snapshot pre-call and
asserts byte-equality post-call.
- expected.response.parsed is compared for equality.
- expected.raises.carries is fed to assert_error_carries.
- retry_middleware: block wraps the call in a default-classifier
retry simulator (transient = TRANSIENT_CATEGORIES membership); the
captured-request count provides provider_call_count.
- mock_provider.capabilities.supports_native_response_format: false
constructs the provider with force_prompt_augmentation_fallback=True.
The 0016 structured-output fixtures (021–028) remain skipped at this
commit. The next commit removes their skip markers.
* test: drive 0016 fixtures 021-028 + add structured-output unit tests
Removes the deferred-fixture skip markers for the 8 structured-output
conformance fixtures (021–028). All pass against the OpenAIProvider
+ harness extensions landed in earlier commits.
Adds tests/unit/test_structured_output.py covering bits the
conformance fixtures don't exercise directly:
- validate_response_schema edge cases: non-dict, non-object top-level,
missing type.
- strict_mode_supported: required-coverage rule, additionalProperties
true, nested-object violation, anyOf branch violation, internal $ref
resolution, unresolvable $ref, $ref cycle (self-referential schema).
- _derive_schema_name: title-when-present, hash-fallback, determinism,
empty-title behavior.
- _augment_messages_with_schema_directive: prepend-when-no-system,
extend-existing-system, caller-list-not-mutated, serialized-schema-
substring.
- Pydantic-class overload: class-in returns validated BaseModel
instance; pydantic ValidationError wraps in StructuredOutputInvalid;
wire body produced from class equals wire body produced from the
equivalent .model_json_schema() dict.
- uses_prompt_augmentation_fallback inspect property: False by
default, True when constructor flag is set.
* docs: changelog entry for proposal 0016 under [Unreleased]
Documents the structured-output surface added in this PR: the
response_schema parameter, Response.parsed field, StructuredOutputInvalid
error category, OpenAIProvider native + fallback wire paths, the
provider-agnostic schema helpers, the capability-agnostic conformance
harness extensions, and the jsonschema runtime dependency.
Also records:
- Spec pin bump 0.10.0 → 0.15.0 (skip-ahead governance) with
per-proposal deferred-skip in the conformance suite until each PR
lands.
- Release gate: do not tag the consolidated release until all five
PRs of the batch (0011, 0014, 0015, 0016, 0017) are merged.
* fix(llm): address CoPilot review on PR #42
Addresses the 8 CoPilot review threads on the structured-output PR:
- strict_mode_supported now requires additionalProperties to be
EXPLICITLY false (not just missing-or-false). Missing implies the
JSON Schema default of permitting extras, which OpenAI's strict
mode rejects. Pydantic's .model_json_schema() omits the key by
default, so the class-input path would have 400ed against OpenAI
even with conformance fixtures passing.
- _normalize_response_schema now raises ProviderInvalidRequest when
the class form is not a BaseModel subclass, instead of letting
AttributeError leak from model_json_schema.
- validate_response_schema now runs jsonschema.Draft202012Validator
.check_schema() at the boundary, wrapping SchemaError as
ProviderInvalidRequest. Malformed schemas now fail at the API
boundary instead of escaping at decode time.
- _derive_schema_name now regex-checks the title against OpenAI's
name constraint (^[a-zA-Z0-9_-]{1,64}$) and falls back to the
hashed name when the title doesn't match. Sanitizing-in-place
would silently mutate user intent; the hash is a more honest
fallback.
- Two comments claiming Message instances are immutable Pydantic
models were updated. The models are not configured with
frozen=True; the safety actually comes from the helpers not
modifying them in place.
- match_wire_body now fails on extra keys in actual. The previous
permissive default defeated the point of expected_wire_request
being a literal compare; partial assertions continue to live in
the sibling expected_wire_request_checks block.
- _iter_calls now propagates expected_wire_request,
expected_wire_request_checks, response_schema, and
retry_middleware from sibling-of-call into the call dict. Only
expected was being copied before. Cases-form fixtures with
case-level wire expectations were silently running without those
assertions.
The _iter_calls fix surfaced two pre-existing gaps in the harness's
handling of cases-shape fixtures, fixed inline:
- The harness was never wiring config from the call spec into
provider.complete(); fixture 005's runtime_config_passthrough
case was effectively a no-op.
- OpenAIProvider was using json.dumps default formatting for
tool_call.function.arguments (with spaces after colons), which
doesn't match the canonical compact form OpenAI emits or the
spec's fixture 005 expectations. Switched to compact form.
New unit tests cover the missing-additionalProperties strict-mode
case, the non-BaseModel class rejection, the malformed JSON Schema
rejection, and the title-falls-back hash cases.
* docs: upgrade hello-world to demo structured output
Replaces the no-LLM hello-world in README.md with a version that
makes a real LLM call via OpenAIProvider and uses a Pydantic class
as the response_schema. The resulting Response.parsed flows through
state as a typed Classification instance and drives the conditional
edge that routes between research and summarize.
Defaults to OpenAI public API (gpt-4o-mini) with env-var config:
LLM_BASE_URL, LLM_MODEL, LLM_API_KEY. A trailing line in the README
calls out OpenRouter, vLLM, LM Studio, llama.cpp as drop-in swaps
via base_url/model.
The example also lands as a runnable file at
examples/00-hello-world/main.py and is added to the smoke test
suite. examples/README.md gets a corresponding entry.
* fix(examples): hello-world base_url default and observer trace
Two bugs surfaced during live validation against OpenAI:
- The default LLM_BASE_URL was https://api.openai.com/v1, but our
OpenAIProvider's wire path posts to /v1/chat/completions itself.
httpx URL join produced https://api.openai.com/v1/v1/chat/completions
→ 404. Convention is base_url = host root; impl adds /v1. Default
now matches; doc-string + README comment make it explicit.
- The observer trace fired on the OpenAIProvider LLM-span event
(sentinel namespace, post_state=None) and crashed accessing .sources.
Added a post_state is not None guard.
* docs(examples): wire research + summarize nodes to real LLM calls
The hello-world's research and summarize nodes were returning
hard-coded source lists. Replaces both with real provider.complete()
calls that emit typed structured output, so the example demonstrates
the value of a structured-output pipeline end-to-end instead of just
the framework's plumbing.
The example now exercises both response_schema forms in one demo:
- classify and summarize use Pydantic classes (Classification,
Summary); Response.parsed comes back as a validated instance.
- research uses a raw JSON Schema dict; Response.parsed comes back
as a plain dict.
State gains two intermediate-artifact fields (research_plan,
summary). Final output prints whichever branch fired, in addition
to the existing sources/metadata. The reducer-policy story stays
intact (last_write_wins on the LLM outputs, append on sources,
merge on metadata).
Live-validated against OpenAI gpt-4o-mini; both branches verified
(structured class instance + structured dict on Response.parsed).
* docs: structured output concepts page + model-providers updates
Adds docs/concepts/llms.md covering how LLM calls fit into the graph
model: LLM calls as async IO inside nodes, structured output (both
response_schema forms + native/fallback wire paths + strict mode),
routing on parsed fields, and errors at the LLM boundary. Nav entry
added to mkdocs.yml's Concepts section; concepts/index.md TOC
extended.
Updates docs/model-providers/index.md: Protocol signature now shows
the response_schema parameter; errors table adds StructuredOutputInvalid;
new Structured output section walks through both response_schema
forms, the native/fallback wire paths, and strict-mode constraints.
Updates docs/model-providers/authoring.md: skeleton's complete()
signature now matches the Protocol (response_schema parameter); a
new "Structured output" entry in Beyond the skeleton points custom-
provider authors at validate_response_schema and strict_mode_supported.
mkdocs builds clean in strict mode; the runnable example in the new
Structured output section is verified by tests/test_docs_examples.py.
* docs: fix Provider.complete docstring Returns rendering
The Returns block on Provider.complete started with "A :class:Response
carrying ...", which mkdocstrings' Google-parser misread as a
name-type pair: it pulled out "A" as the Name column entry and split
the multi-line description across three table rows.
Moving the return-value sentence into the prose summary at the top of
the docstring (matching the pattern OpenAIProvider.complete already
uses) renders cleanly: no spurious Name column entry, single
description block.
* fix(llm): second CoPilot review pass on PR #42
Addresses 19 review threads from the second CoPilot pass; about half
were duplicates of the same underlying issue:
- examples/00-hello-world/main.py + README hello-world: api_key now
uses `os.environ.get("LLM_API_KEY") or None` so an exported-but-
empty env var falls through to no-auth (matters for local servers
that reject an empty bearer header).
- Both examples now close the OpenAIProvider in the finally block
alongside graph.drain(). Long-running consumers that copy the
snippet had been leaking the underlying httpx.AsyncClient.
- errors.py header dropped the hard-coded "seven canonical
categories" count after StructuredOutputInvalid landed.
- strict_mode_supported docstring and the surrounding spec-anchor
comment block both updated to match the implementation:
additionalProperties must be EXPLICITLY false (an omitted key
counts as non-strict, since JSON Schema's default permits extras).
- _resolve_ref now handles ref == "#" as the document root before
rejecting external refs. Root-recursive schemas that use the bare
JSON-Pointer-root form now resolve correctly. Unit test added.
- _strict_mode_check tightened to return False on unrecognized
shapes (empty {}, const-only, enum-only, unknown keywords) instead
of falling through to True. Primitive types (string/integer/
number/boolean/null) classified as terminal-strict-compatible.
Two unit tests added.
- _build_request_body now explicitly strips response_format from the
body when the provider is in fallback mode. RuntimeConfig is
extra="allow", so a caller could have piped response_format
through the extras loop past the include_response_format gate.
- provider.py module docstring's summary signature line updated to
match the Protocol's response_schema parameter.
- validate_response_schema's spec-anchor comment updated to reflect
that JSON Schema validity is now checked at the boundary via
Draft202012Validator.check_schema(), not delegated to parse time.
- test_pydantic_class_wire_body_matches_dict_form: widened the
assertion from response_format-only to full body equality, so any
regression in the class-input wire mapping (not just
response_format) gets caught.
- test_inspect_property_native_default and
test_inspect_property_fallback_when_forced converted to async
with try/finally + aclose() to match the rest of the file's
provider-lifecycle pattern.
* fix: third CoPilot review pass on PR #42
Addresses 5 remaining review threads (3 substantive, 2 stale on
already-fixed code):
- LlmProviderResponseAssertion (the typed assertion model in
harness/expectations.py) now lists `parsed: Any | None`. The
runtime assertion in test_llm_provider.py already handled it, but
the typed parser had it under extra="forbid" and would have
rejected any future case-shape LLM fixture using `parsed`. The
021-028 fixtures slip past today on `calls:` form's permissive
`LlmCallSpec.expected: dict[str, Any]`; this lines the two paths
up.
- docs/model-providers/authoring.md skeleton comment tightened:
removed the "ignore it and return free-form text" option from
the response_schema guidance. A provider that silently drops the
parameter violates the Protocol contract; callers expect either
Response.parsed populated or StructuredOutputInvalid raised. Now
only two valid options surfaced: raise ProviderInvalidRequest
until implemented, or wire it through.
- docs/concepts/llms.md softened the static-typing claim in the
Pydantic-class form section. Response.parsed is
`dict[str, Any] | BaseModel | None`, so a type checker won't
narrow from `response_schema=Classification` alone. The page now
separates the runtime guarantee (validated instance) from static
access (requires cast/isinstance/typed assignment); generic
Response[T] flagged as a follow-up.
The two stale threads (examples/00-hello-world/main.py provider
cleanup, test_structured_output.py provider cleanup) were already
fixed in commit 8ed334c; replies sent + threads resolved without
code changes.
* fix: fourth CoPilot review pass on PR #42
Addresses 6 review threads, several of which surfaced second-order
issues from previous rounds:
- openai.py complete(): the fallback flag was driving
include_response_format=False for every call, including free-form
ones. That triggered the response_format strip on calls that
weren't structured-output at all, clobbering caller-supplied
RuntimeConfig extras. Gating the flag on schema_dict being set so
free-form calls preserve extras. Unit test added.
- src/openarmature/__init__.py + tests/test_smoke.py: bumped
__spec_version__ from "0.10.0" to "0.15.0" to match the
pyproject.toml [tool.openarmature].spec_version bump. AGENTS.md
flags these three values as required to stay in sync; the
submodule-bump commit missed the runtime sources.
- _strict_mode_check array branch: {"type": "array"} without
`items` no longer returns True. Unconstrained array content is
the array analog of an object with no additionalProperties: false:
the walker can't statically verify nested shapes, so strict mode
rejects. Unit test added.
- docs/model-providers/authoring.md: skeleton's complete() now
actually enforces what its comment promised. Added
`if response_schema is not None: raise ProviderInvalidRequest`
to the body and surfaced the exception in the import list, so a
provider copied from the skeleton can't silently violate the
Protocol contract.
- docs/concepts/llms.md Pydantic-class snippet: added
`from typing import Literal` so the example is copy-paste-
runnable (the snippet uses Literal in the class but only imported
BaseModel).
- tests/unit/test_structured_output.py nested-recursion tests:
test_strict_mode_recurses_into_nested_object and
test_strict_mode_anyof_branch_must_satisfy were short-circuiting
at the root because the root schema itself failed strict rules.
Tightened both root schemas so the recursive walk actually fires;
the tests now guard the recursion they claim to.
* docs: cumulative strict-mode constraint list + spec-version-drift test
Captures two follow-ups surfaced by the four CoPilot review rounds:
- docs/concepts/llms.md "Strict mode" section expanded into the
full constraint list. After four rounds of tightening the
strict_mode_supported heuristic, the rule set is stable and the
user-facing surface should list it directly rather than make
callers read provider.py. The page frames the list as the
authoritative set: anything not on it trips to non-strict.
- docs/model-providers/index.md "Strict mode" subsection trimmed
and now links into the concepts page for the full list,
following the established split (concepts/ owns the deep-dive,
model-providers/ stays terse).
- tests/test_smoke.py adds test_spec_version_matches_pyproject:
reads pyproject.toml's [tool.openarmature].spec_version and
asserts it equals openarmature.__spec_version__. AGENTS.md
flags these as required to stay in sync; the previous smoke
test only checked internal consistency between __spec_version__
and its asserted value, so the pyproject side could drift
silently (and did, in the original submodule-bump commit).
* fix: fifth CoPilot review pass on PR #42
Addresses 4 review threads:
- examples/00-hello-world/main.py: provider construction moved
from module level to a lazy _get_provider() helper backed by a
module global. Avoids opening an httpx.AsyncClient when tooling
imports the module without running main() — the smoke test now
doesn't trigger construction across 6 example loads. main()'s
finally only closes when the cached instance is set.
- src/openarmature/llm/provider.py: validate_response_schema now
walks all $ref values via _check_refs_resolvable and raises
ProviderInvalidRequest for any non-internal-resolvable ref.
Draft202012Validator.check_schema doesn't traverse refs, so
previously an external ref slipped past the boundary and
surfaced as a raw referencing-library exception at validate
time. Pre-validation surfaces the clean category at the API
boundary.
- src/openarmature/llm/providers/openai.py: _parse_and_validate
now also catches jsonschema.SchemaError and maps it to
StructuredOutputInvalid. Safety net for any schema-side
exception (including ref-resolution failures) that pre-
validation might miss.
- tests/unit/test_structured_output.py:
- test_strict_mode_unresolvable_ref_fails: root tightened with
additionalProperties: false so the walk reaches the $ref
branch (was short-circuiting at the root).
- Added test_validate_response_schema_rejects_external_ref
covering the new pre-validation path.
- tests/test_smoke.py: added test_spec_version_matches_submodule_pin
shelling to `git -C openarmature-spec describe --tags
--exact-match HEAD` and asserting it equals
v{__spec_version__}. Skips cleanly when the submodule isn't a
git checkout (installed-package CI lanes). Completes the
three-place drift check from AGENTS.md
(__spec_version__ ↔ pyproject ↔ submodule pin).
* fix(test): replace git-describe submodule check with CHANGELOG parse
The git-describe-based submodule check from the previous commit
passed locally but failed in CI because actions/checkout pins the
submodule to its recorded SHA without fetching the spec repo's
tags. `git describe --tags --exact-match` then finds nothing and
the test fails with "submodule HEAD is not at any tag."
Switching to parsing openarmature-spec/CHANGELOG.md: the spec
follows Keep a Changelog, so the first non-[Unreleased]
`## [X.Y.Z]` heading is the version at the pinned commit. This
works regardless of CI tag-fetch state and catches the same drift
class (submodule moved to a different release).
Skips cleanly when CHANGELOG.md isn't present (installed-package
lanes that don't ship the submodule checkout).
* fix(test): satisfy CodeQL on the changelog-parsing test
CodeQL flagged the for/else: pytest.fail() pattern as a
potentially-uninitialized-local-variable warning because it
doesn't model pytest.fail as NoReturn — the analyzer sees a path
where submodule_latest is referenced after the loop without ever
being bound.
Pulling the parse into _read_latest_spec_version_from_changelog
that explicitly returns the version or raises AssertionError.
Eliminates the unreachable-after-fail pattern and reads cleaner.
* fix: seventh CoPilot review pass on PR #42
Six second-order correctness fixes surfaced by the round-7 review,
mostly hardening _resolve_ref, _check_refs_resolvable, and the
Pydantic-class validation path.
- _resolve_ref now distinguishes "unresolvable" (path doesn't exist
/ external ref) from "resolved to non-dict" via a module-level
_UNRESOLVABLE sentinel. Boolean schemas (true/false) are valid
JSON Schema subschemas; a $ref to one was being incorrectly
rejected as ProviderInvalidRequest. Now resolves cleanly and
strict-mode still returns False on bool targets (the correct
conservative answer).
- validate_response_schema's metaschema check now uses
jsonschema.validators.validator_for(schema) instead of the
hard-coded Draft 2020-12. A valid draft-07 schema (e.g. tuple-
form items, common in tooling) was being rejected at the
boundary but accepted at runtime. Boundary and runtime now agree.
- _resolve_ref percent-decodes JSON Pointer tokens before applying
the ~1 / ~0 unescape pair. Per RFC 6901 §6, a JSON Pointer in a
URI fragment is percent-encoded; refs like
#/$defs/Name%20With%20Spaces now resolve correctly.
- _check_refs_resolvable now walks only known subschema-bearing
keywords (properties, patternProperties, additionalProperties,
items, prefixItems, contains, if/then/else, allOf/anyOf/oneOf/not,
$defs/definitions, dependentSchemas, propertyNames,
unevaluatedItems, unevaluatedProperties). A "$ref" key under data
positions (default, const, enum, $comment, x-* extensions) is
data, not a schema reference, and is no longer incorrectly
resolved.
- docs/concepts/llms.md "LLM calls are async IO inside a node"
section reframed: module-level provider construction leaks the
httpx.AsyncClient in tooling/test/docs-build imports. The page
now documents application-startup / lifecycle-managed
construction (lazy on-first-use plus aclose in finally / shutdown
hook), matching the pattern the hello-world example was made
lazy for.
- _parse_and_validate's Pydantic-class path now runs
jsonschema.validate against the generated JSON Schema BEFORE
calling model_validate. Pydantic's default model_validate is
coercive (accepts "30" for an int field), which diverged from
the strict dict-schema path. Both paths now apply the same
jsonschema check first; model_validate then constructs the
typed instance.
- jsonschema.ValidationError's failure description now includes
exc.json_path (e.g. "$.age: '30' is not of type 'integer'"). The
bare exc.message lost the field name, breaking caller diagnostics
for the missing-field / wrong-type-at-path cases.
Five new unit tests cover the bool-ref, draft-07, percent-encoded
ref, ref-under-data, and Pydantic-coercion-rejection cases.1 parent 89df03d commit 2ecb7b1
28 files changed
Lines changed: 2872 additions & 108 deletions
File tree
- docs
- concepts
- model-providers
- examples
- 00-hello-world
- src/openarmature
- llm
- providers
- tests
- conformance
- harness
- unit
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
6 | 6 | | |
7 | 7 | | |
8 | 8 | | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
9 | 27 | | |
10 | 28 | | |
11 | 29 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
55 | 55 | | |
56 | 56 | | |
57 | 57 | | |
58 | | - | |
| 58 | + | |
59 | 59 | | |
60 | 60 | | |
61 | 61 | | |
62 | | - | |
63 | | - | |
64 | | - | |
65 | | - | |
66 | | - | |
67 | | - | |
68 | | - | |
69 | | - | |
70 | | - | |
71 | | - | |
72 | | - | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
73 | 79 | | |
74 | 80 | | |
75 | 81 | | |
76 | 82 | | |
77 | | - | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
78 | 86 | | |
79 | 87 | | |
80 | 88 | | |
| |||
83 | 91 | | |
84 | 92 | | |
85 | 93 | | |
86 | | - | |
87 | | - | |
88 | | - | |
89 | | - | |
90 | | - | |
91 | | - | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
92 | 99 | | |
93 | 100 | | |
94 | | - | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
95 | 122 | | |
| 123 | + | |
96 | 124 | | |
97 | | - | |
| 125 | + | |
98 | 126 | | |
99 | 127 | | |
100 | 128 | | |
101 | | - | |
102 | | - | |
103 | | - | |
104 | | - | |
105 | | - | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
106 | 135 | | |
107 | 136 | | |
108 | 137 | | |
109 | | - | |
| 138 | + | |
| 139 | + | |
110 | 140 | | |
111 | 141 | | |
112 | 142 | | |
113 | | - | |
| 143 | + | |
114 | 144 | | |
115 | 145 | | |
116 | 146 | | |
| |||
127 | 157 | | |
128 | 158 | | |
129 | 159 | | |
| 160 | + | |
130 | 161 | | |
131 | 162 | | |
132 | | - | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
133 | 169 | | |
134 | 170 | | |
| 171 | + | |
135 | 172 | | |
136 | 173 | | |
137 | 174 | | |
138 | | - | |
139 | | - | |
140 | 175 | | |
141 | 176 | | |
142 | | - | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
143 | 180 | | |
144 | | - | |
145 | | - | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
146 | 184 | | |
147 | 185 | | |
148 | 186 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
12 | 12 | | |
13 | 13 | | |
14 | 14 | | |
| 15 | + | |
| 16 | + | |
15 | 17 | | |
16 | 18 | | |
17 | 19 | | |
| |||
0 commit comments