feat: EXPR and WRAP_ACTIONS v0 model parity via Rust bindings

[leongdl]

TL;DR

This PR makes the pure-Python (v0) 2023-09 model speak the same EXPR (RFC 0007) and WRAP_ACTIONS (RFC 0008) language as the Rust reference implementation. Templates can now use typed expressions end-to-end (range: "{{Param.Values}}" becomes a real list, not a string), wrap hooks can forward the wrapped action's timeout and cancelation (mode: "{{WrappedAction.Cancelation.Mode}}" decided at run time), and the expression engine is the Rust one, reached through PyO3 bindings — so Python and Rust can no longer silently disagree. It is the base of the RFC 0008 stack: openjd-sessions-for-python #333 and openjd-cli #230 build on this surface.

Everything new is extension-gated. Three validation tightenings (listed below) are the only behavior changes for existing templates, and each one matches both the spec and openjd-rs.

What's in it

Deferred cancelation mode (FEATURE_BUNDLE_1). A cancelation mode that is a format string becomes a third union shape, CancelationMethodDeferred, mirroring CancelationMode::DeferredMode in openjd-rs. mode is normally the schema selector — the parser needs it to pick an object shape — but a forwarded value like {{WrappedAction.Cancelation.Mode}} only exists at run time, so the shape decision is deferred: the parser accepts the format string (a callable pydantic Discriminator routes on "{{"), static validation still checks the expression, and the runtime resolves it right before the action runs.

Typed whole-field resolution (RFC 0006). FormatString.resolve_value and the typed_resolve_fields job-creation metadata keep the engine's typed value when a field is a single whole-field {{ ... }} expression — lists stay lists, null stays null. Task-parameter ranges use this so a LIST[*] job parameter instantiates to the literal value list; the resolution is computed exactly once per field and shared with the target-model decision so the two can never disagree.

Flow: create_job → range: "{{Param.Values}}" with LIST[INT] param
  1. instantiate_model            extends_symtab: Step.Name + step-level lets seeded
  2. _compute_typed_resolutions   evaluate once  ──BOUNDARY──▶ Rust engine ◀── ExprValue
  3. create_as callable           list result → RangeListTaskParameterDefinition
  4. field instantiation          reuses the same typed value (no re-evaluation)
  5. range-cap validator          §3.4: >1024 values rejected (matches openjd-rs ranges.rs)

Per-hook WRAP_ACTIONS variables. WrappedAction.* (including Cancelation.Mode / Cancelation.NotifyPeriodInSeconds), WrappedEnv.Name, and WrappedStep.Name are injected per wrap hook via new _template_field_inject metadata — visible only inside their hook's action, replacing the first-cut script-wide injection (referencing them in onEnter is now rejected). timeout/cancelation validate at template scope via _template_field_scopes, since they resolve at job creation, while still allowing round-trip forwarding of the wrapped action's values.

EXPR runtime plumbing. Step-level let bindings evaluate at job creation (extends_symtab) and are preserved on Step.let for the runtime; Job.Name is seeded per RFC 0007 §7.3.1; URI-form PATH values (s3://...) are preserved verbatim under EXPR; openjd.model.evaluate_let_bindings is exported (parse-memoized) and consumed by the sessions runtime so the binding-evaluation policy is single-sourced.

Performance. The EXPR engine symbol table and host-context profile are cached on SymbolTable, keyed on a mutation version (a versioned expr_types dict, property setters, __ior__, pickle via __reduce__). A session action resolving ~20 expressions previously rebuilt the full engine table across the Rust boundary per expression; now it builds twice per action (once before let bindings, once after): 8.6 → 1.9 ms per 20-arg action on a 125-symbol table.

Validation behavior changes (not purely additive)

These newly reject previously-accepted templates; all three match openjd-rs and the spec:

1. onEnter is required for an environment script unless WRAP_ACTIONS is declared (Template Schemas §3.5).
2. Task-parameter ranges are capped at 1024 values (§3.4) — at parse time for literals, at job creation for expression-driven ranges.
3. Identifier name fields are capped at 64 characters without FEATURE_BUNDLE_1 (§7.1; 512 with it), and decimals requires a SPIN_BOX control (§2.4).

Commit guide

Commit	What
base	The EXPR/WRAP_ACTIONS parity surface described above
review fixes	CI build (crates.io resolution), §3.4 create-time range cap (parity fix found in review), symbol-table caching, single-evaluation restructure, shared helpers
crate pins	openjd-expr 0.2.1 / openjd-model 0.4.0 / openjd-sessions 0.4.0 (the published releases containing DeferredMode) + regenerated lock and THIRD-PARTY-LICENSES
test port	The deferred-cancelation and None-rendering tests from #313, which this PR supersedes

How was this change tested?

• Full unit suite: 5351 passed, 24 skipped, 3 xfailed (includes 21 new regression tests: SymbolTable cache-invalidation contract incl. |=/reassignment/pickle, single-evaluation invariant, deferred-cancelation round-trip forwarding, None-rendering). mypy clean (106 files); black/ruff clean.
• Rust bindings built against the published crates.io releases with CI's exact commands (build/clippy -D warnings/fmt/doc).
• OpenJD conformance suite (2023-09) through openjd-cli with this branch: 1154/1158; the 4 wrap-cancelation failures need runtime-side eager validation (openjd-sessions follow-up — not a model gap). Four new conformance fixtures (§3.4 range cap at create time, both polarities; §7.3.1 Step.Name in step environments) were added in openjd-specifications#155 and validated against both this branch and the openjd-rs CLI.
• Multi-aspect review (crashes / performance / maintainability) with a post-fix multi-agent clean sweep; call-stack traces and the full findings log are in the review write-up.

Supersedes #313 (all of its production code is contained here, verified hunk-by-hunk; its tests are ported in the final commit).

Review guide

Review sign-off guide — openjd-model-for-python PR #318

PR: #318
Branch: python-rs, tip ca44376 · Merge: squash (auto-merge armed)
Verification at tip: 5437 tests passed / coverage 94% / mypy, black, ruff,
clippy -D warnings all clean · conformance 2023-09: 1159/1162 (the 3
remaining failures need sessions-runtime work outside this PR)

---

1. Problem statement

Open Job Description has two implementations of one specification: the Rust
reference (openjd-rs) and
this pure-Python "v0" model. Before this PR, the Python 2023-09 model had no
support for the EXPR expression language
(RFC 0007)
or environment wrap actions
(RFC 0008),
and its format-string evaluator was a hand-written Python implementation that
could silently drift from the Rust engine.

Concretely, that meant: no typed values ({{ true }} rendered as Python's
"True" instead of the spec's "true"), no LIST[*]/BOOL/RANGE_EXPR
job-parameter types, no let bindings, no wrap-hook template variables, and
no way for a wrap hook to forward the wrapped action's timeout or cancelation.
The downstream RFC 0008 stack (openjd-sessions-for-python
#333,
openjd-cli
#230) needs all
of this surface.

This PR closes that gap by binding the Python model to the same Rust crates
the reference uses (openjd-expr 0.2.1, openjd-model 0.4.0 via PyO3 in
rust-bindings/), so both implementations share one expression engine — and
then aligning every model-level behavior the review process found diverging.

---

2. What we fixed, and what it mirrors in openjd-rs

Every behavioral decision traces to a spec section and a Rust function. The
review process (multi-aspect agent passes + an independent external model
review + the repo's automated PR review) confirmed each divergence with
two-sided repros before it was fixed.

#	Fix	Mirrors (openjd-rs)	Spec
1	§3.4 range caps at create time for expression-driven ranges (typed LIST[*] and RANGE_EXPR), not just parse time	job/create_job/ranges.rs — max_task_param_range_len checked on every resolve path	Template Schemas §3.4
2	§3.4.2 empty-string PATH range items rejected at create time	ranges.rs ~L379 (is_path && s.is_empty())	Template Schemas §3.4.2
3	Deferred (format-string) cancelation mode as a third union shape, decided at run time	template/actions.rs — CancelationMode::DeferredMode serde routing on "{{"	Template Schemas §5.3 (FEATURE_BUNDLE_1)
4	timeout/notifyPeriodInSeconds carried unresolved into instantiated Jobs (run-time resolution) — makes wrap-hook forwarding ({{WrappedAction.Timeout}}) work	job/create_job/instantiate.rs:253-274 clones them unresolved; symtab filter preserves their symbols	RFC 0008 "Cancelation behavior"
5	Per-hook WrappedAction.*/WrappedEnv.Name/WrappedStep.Name injection; hook timeout/cancelation see only WrappedAction.*	template/validate_v2023_09/format_strings.rs:1173-1195	RFC 0008
6	onEnter required without WRAP_ACTIONS; §7.1 name lengths; §2.4 decimals rule (validation tightenings)	validate_v2023_09/structure.rs:383-406; validate_ui	Template Schemas §3.5 / §7.1 / §2.4
7	Typed whole-field range elements: INT accepts only int, FLOAT accepts int/float, STRING/PATH use spec display form (true/false, "[1, 2]")	ranges.rs resolve_int_range / resolve_float_range / resolve_string_range (to_display_string)	RFC 0005 §1.3 typed coercion
8	PATH/LIST[PATH] symbol contracts: processed Param.* is session-scope only; RawParam.* is template-scope string/list[string]	format_strings.rs build_param_symtab/build_template_scope_symtab; parameters.rs build_symbol_table	RFC 0005 "Job Parameter Types"; Template Schemas §2.12/§7.3.1
9	Per-step typed expression validation: Task.Param.*/Task.RawParam.* type-checked per step (raw PATH → string), same-named params across steps independent	format_strings.rs build_task_scope_symtab (~160-230)	RFC 0005/0007
10	Step-level let at job creation + Step.Name/Job.Name seeding; URI-form PATH values preserved	instantiate_step per-step symtab; uri_path::is_uri	RFC 0007 §3.6/§7.3.1; RFC 0006

Conformance backstop: 4 new fixtures were added to the shared suite in
openjd-specifications#155
(§3.4 create-time caps, both polarities; §7.3.1 Step.Name in step
environments), each validated against both implementations, so fixes 1
and the Step.Name contract can never silently regress.

Engineering fixes that rode along (not spec-behavioral): a
mutation-versioned cache for the engine symbol table (8.6 → 1.9 ms per
20-arg action, 4.4x), pickle/deepcopy support for Jobs carrying deferred
FormatStrings (parse-context snapshot in __getnewargs_ex__), single
evaluation of typed range expressions, and the shared parse-memoized
openjd.model.evaluate_let_bindings consumed by the sessions runtime.

---

3. The code changes, with call stacks

Read the diff in this order — each layer builds on the previous one.

3.1 Foundation: the Rust boundary and the cache

Files: rust-bindings/**, Cargo.toml/Cargo.lock,
src/openjd/model/_symbol_table.py,
_format_strings/_expr_support.py, _format_strings/_nodes.py

The bindings pin the published crates (openjd-expr 0.2.1,
openjd-model 0.4.0) — no local path patches. SymbolTable carries a
monotonic _version bumped by every mutation (__setitem__, a versioned
expr_types dict including |=/reassignment, the expr_host_rules setter),
and the expensive engine-table build is cached per version:

Flow: run-time FormatString.resolve / resolve_value
 1. _format_string.py  resolve (per-segment) | resolve_value → whole_field_expression
 2. _nodes.py          ExprNode.evaluate_value → _evaluate_raw
 3. _expr_support.py   symtab_to_expr_values — cache probe keyed
                       (version, path_format, typedness); on miss:
                       flatten + per-type spec lookup
                       ──BOUNDARY──▶ Rust build_symbol_table (cached AFTER success)
 4. _expr_support.py   profile_for_symtab — cached; host rules → HostContext
 5.                    parsed.evaluate ──BOUNDARY──▶ ExprValue; errors →
                       map_eval_error → ExpressionError (typed, catchable)
 6. display            .item() native | str(ExprValue) = spec coercion | None → ""

Key insight: cache entries are written only after a successful Rust build
(exception-safe), and a session action resolving ~20 expressions builds the
engine table twice (before/after let bindings) instead of 20+ times.

3.2 Job creation: typed resolution and deferral

Files: _create_job.py, _internal/_create_job.py, _types.py,
v2023_09/_model.py (task-parameter classes, StepTemplate)

Flow: create_job(template, parameter_values)
 1. _create_job.py     symtab: Param./RawParam. values + expr_types
                       (path types: RawParam only, string/list[string])   [fix 8]
 2.                    EXPR: Job.Name resolved and seeded                 [fix 10]
 3. _internal/_create_job.py  instantiate_model (recursive)
 4.   extends_symtab → StepTemplate._extend_step_symtab: Step.Name +
      step-level lets ──BOUNDARY──▶ evaluate_let_bindings (memoized parse)
 5.   _compute_typed_resolutions: whole-field "{{Param.Values}}" evaluated
      ONCE ──BOUNDARY──▶ raw list ExprValue (element variants preserved)
 6.   create_as callable picks RangeList vs RangeExpression from the same result
 7.   typed_resolve_coerce hook: element/target agreement                 [fix 7]
      INT←int only | FLOAT←int/float | STRING/PATH←display form
      ("Expected int in range, got bool" — Rust message shape)
 8.   target-model validators: §3.4 cap [fix 1] + §3.4.2 empty-PATH [fix 2]
 9.   timeout/cancelation NOT resolved — carried as raw FormatStrings     [fix 4]
10.   capture_validation_errors aggregates everything → DecodeValidationError

Key insight: the typed resolution result feeds both the target-model decision
and the field value, so they cannot disagree; and the deferral in step 9 is
what makes timeout: "{{WrappedAction.Timeout}}" create successfully — the
symbol only exists when the session runs the hook.

3.3 Template validation: scopes and types

Files: _internal/_variable_reference_validation.py, v2023_09/_model.py
(EnvironmentActions, Action, SimpleAction, parameter definitions)

Flow: decode_job_template — EXPR template
 1. _parse.py          parse_model w/ ModelParsingContext(extensions)
 2. _variable_reference_validation.py  recursive model walk:
      · _template_field_inject: WrappedAction.* per wrap hook (TEMPLATE scope)
      · _template_field_inject_session: WrappedEnv.Name/WrappedStep.Name
        (SESSION scope — hook timeout/cancelation cannot see them)   [fix 5]
      · _template_field_scopes: timeout/cancelation VALIDATE at template
        scope (they RESOLVE at run time — scope ≠ timing)
      · ScopedSymtabs.types: per-defining-model symbol types threaded with
        the names (TemplateVariableDef.raw → PATH string semantics)  [fixes 8,9]
 3. _nodes.py          validate_symbol_refs: all prefixes typed →
                       ──BOUNDARY──▶ Rust typecheck (unresolved placeholders);
                       any untyped prefix → name-only fallback (no false rejects)
 4. FormatString parse ──BOUNDARY──▶ Rust parse_expression;
                       symbol-free exprs ──BOUNDARY──▶ static validation
 5. errors → DecodeValidationError

Key insight: types ride the same per-field threading as the symbol names, so
two steps defining a task parameter File as PATH and STRING validate
independently (error localized to the offending step only) — exactly Rust's
per-step symtabs. Conflicting merged types drop to name-only validation,
never last-writer-wins.

3.4 Deferred cancelation round-trip (the RFC 0008 headline)

 1. decode    mode "{{...}}" → callable Discriminator → CancelationMethodDeferred
              (FEATURE_BUNDLE_1-gated; expression statically validated)
 2. create    mode + notifyPeriodInSeconds survive as raw FormatStrings [fix 4]
 3. bindings  template_types.rs PyCancelationMode — "{{" →
              CancelationMode::DeferredMode (map_err → PyValueError; no unwraps)
 4. sessions  at cancel time: WrappedAction.* seeded → mode resolves
              ──BOUNDARY──▶; null drops the cancelation; TERMINATE forbids a
              notify period; errors degrade to Terminate (cancel never blocks)

---

4. Sign-off checklist

• Boundary safety: no unwrap/expect/panic! on user data in
  rust-bindings/**; every Rust error surfaces as
  DecodeValidationError/ExpressionError/FormatStringError
  (audited three times; spot-check template_types.rs DeferredMode arms).
• Parity: the table in §2 — each row names the Rust function; the
  behaviors were verified by running both implementations, and the
  conformance suite passes 1159/1162 (3 known sessions-side gaps).
• Behavior changes acknowledged: the three validation tightenings
  (§2 row 6) and the FB1 deferral (row 4 — instantiated Jobs may carry
  FormatString timeouts; pickle/deepcopy verified) are intentional and
  Rust-matching. They newly reject/change previously-accepted inputs.
• Cache soundness: mutation-version invalidation covers
  __setitem__, expr_types item-set/update/|=/reassignment,
  expr_host_rules assignment; entries written only after successful
  builds; pinned by test_symbol_table_cache.py.
• Tests: ~600 new test lines across cache, pickle, typed ranges,
  symbol contracts, per-step typing, wrap forwarding; coverage holds the
  94% gate.

Known-and-accepted (not blockers): 3 wrap-cancelation conformance
fixtures need sessions-runtime eager validation (tracked for #333's repo);
serialized-Job re-parse requires callers to re-supply extensions (FB1 fields
were ints before); evaluate_let_bindings raises ValueError (not
ExpressionError); CHUNK[INT] task params stay untyped at validation
(Python runtime skips them; Rust types them RANGE_EXPR — noted in code).

Companion documents: deep findings log 2026-07-23-pr318-cr-pass.md;
stack-wide review history 2026-07-22-review.md; call-stack artifacts under
openjd-model-for-python/review/call-stacks/.

[github-actions]

The new §3.4 1024-element range cap is bypassed on the typed whole-field resolve path.

When a range is a whole-field expression that resolves to a list (e.g. range: "{{Param.Values}}" with a LIST[*] job parameter), _get_range_task_param_type returns RangeListTaskParameterDefinition and the field is instantiated to the native list via resolve_whole_field_typed_list. But RangeListTaskParameterDefinition.range is typed TaskRangeList (list[...], no max_length, line 1176/1213), and no length check runs on this path:

• validate_task_param_range_list_len only runs at parse time, and the template value here is a RangeString (format string), so it takes the deferred format-string branch and skips the list-length check.
• The IntRangeExpr length check only applies when the string parses as a range expression, not when it resolves to a literal list.

So an EXPR LIST parameter carrying >1024 values yields a task-parameter range of >1024, while an equivalent literal list-form range is correctly rejected. This is inconsistent with the limit this PR is introducing, and openjd-rs enforces max_task_param_range_len during range instantiation. Consider re-applying validate_task_param_range_list_len to the resolved list in resolve_whole_field_typed_list’s callers (or at instantiation of RangeListTaskParameterDefinition).

[github-actions]

The range-length cap was correctly propagated to the instantiation target RangeListTaskParameterDefinition so that ranges produced by RFC 0006 typed whole-field resolution (range: "{{Param.X}}") are checked at resolve time. But the PATH-specific empty-string check (_validate_no_empty_path_values, §3.4.2) was not propagated here.

A PATH task parameter whose range is a whole-field expression against a LIST[PATH] job parameter (e.g. range: "{{Param.Paths}}") instantiates directly into RangeListTaskParameterDefinition. PathTaskParameterDefinition._validate_no_empty_path_values runs only at decode time, where the value is still a FormatString with expressions and is therefore skipped ("resolve later"). LIST[PATH] element validation (_check_string_item) does not reject empty strings either (no implicit minLength). So an empty path can reach the resolved range, which §3.4.2 says is never a valid path on any OS — the same resolve-time gap the length cap was added here to close, but for empties.

Consider adding an equivalent empty-string check to RangeListTaskParameterDefinition (guarded on type == PATH) to match openjd-rs’ resolve-time checks.

[leongdl]

Already addressed in 53b0fd1.

[jericht]

Still got a Windows test failing:

FAILED test/openjd/model_v0/v2023_09/test_create_job_expr_params.py::TestTypedPathRangeEmptyValues::test_valid_paths_in_typed_range_accepted - AssertionError: assert ['/a', '/b'] == ['\\a', '\\b']
  
  At index 0 diff: '/a' != '\\a'
  
  Full diff:
    [
  -     '\\a',
  ?      ^^
  +     '/a',
  ?      ^
  -     '\\b',
  ?      ^^
  +     '/b',
  ?      ^
    ]

[jericht]

This returns a reference to the _expr_host_rules list so callers can append to it directly, bypassing the bump_version stuff. Should we return a read-only view or copy from this getter instead?

Similar thing for expr_types