feat: Improve Hermes Relay hook injection and event alignment#205
feat: Improve Hermes Relay hook injection and event alignment#205yczhang-nv wants to merge 19 commits into
Conversation
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
WalkthroughAdds Hermes task→session correlation and routing, per-tool ActiveTool tracking for robust end-event matching, Hermes hook backup/merge/restore in launcher runs, LLM span de-duplication and metric merging in ATIF, Node API main-thread Arg0 builders, many tests, and documentation/workflow updates. ChangesHermes Task-Session Correlation and Tool-ID Resilience
Observability: ATIF & OpenInference
Node API: main-thread Arg0 and with_scope
OpenClaw integration & replay tests
Docs, workflows, and scripts
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Possibly related PRs
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
There was a problem hiding this comment.
Actionable comments posted: 5
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@crates/cli/src/alignment/mod.rs`:
- Line 113: hermes_task_sessions is keyed only by task_id, so
record_hermes_task_session and route_hermes_task_event can collide across
sessions; change the storage to be collision-resistant by including session
scope in the key (e.g., use a composite key session_id + task_id or make
hermes_task_sessions a HashMap<session_id, HashMap<task_id, session_id_or_meta>>
/ nested map) and update record_hermes_task_session and route_hermes_task_event
to read/write using that composite/session-scoped key so entries cannot be
overwritten when different sessions reuse the same task_id.
In `@crates/cli/src/launcher.rs`:
- Around line 466-483: prepare_hermes currently mutates the user's Hermes config
during PreparedRun::new which can leave config.yaml patched if execute_live_run
returns early on failed health checks; update the flow so we either defer
calling prepare_hermes (or the write_merged_hermes_hooks step) until after
wait_for_health succeeds in execute_live_run, or ensure PreparedRun::restore is
always invoked on every early-return path from execute_live_run (add
try/finally-like cleanup or explicit restore on error paths). Refer to
prepare_hermes, write_merged_hermes_hooks, execute_live_run, wait_for_health,
and PreparedRun::restore to locate and implement the change.
In `@crates/cli/src/session.rs`:
- Around line 129-134: ActiveTool currently only stores handle, name, and
arguments so identical name+args from different subagents get conflated; add a
resolved owner/subagent identity field (e.g., owner or subagent_id) to the
ActiveTool struct and populate it wherever ActiveTool instances are created (the
synthetic start path and any start_tool constructors), update derives as needed,
and change the fallback/matching logic in end_tool (and the analogous code at
the other referenced range) to require the owner/subagent to match before
considering a name+args fallback so the real handle for a different subagent is
not left open.
In `@crates/cli/tests/coverage/session_tests.rs`:
- Around line 1615-1634: The pre/post tool-call test is vacuous because the
pre_tool_call JSON uses an empty session_id and can be filtered out, so the
subsequent assertion on session.tools.is_empty() doesn't prove mismatched
tool_call_id handling; update the test to make the pre_tool_call and
post_tool_call belong to a real session (non-empty session_id) and split the
test into phases: after injecting the pre_tool_call assert
!session.tools.is_empty() (e.g., session.tools.len() > 0), then inject the
post_tool_call with a different tool_call_id and assert session.tools.is_empty()
to verify the mismatch causes the tool to be removed; apply the same change
pattern to the other occurrence referenced (the block around the second instance
at the later lines).
In `@python/nemo_relay/nat_exporter.py`:
- Around line 40-47: The subscriber() currently reopens the file on every event
causing syscall overhead; instead open a single append handle in __init__ (e.g.
self._output = self._path.open("a")) and reuse it in subscriber() to write
_event_to_json(event) + "\n", calling self._output.flush() after each write for
durability; keep the existing self._lock and self._closed checks in
subscriber(), and close/cleanup the handle in shutdown() (and guard against
double-close) so the file is only opened once and closed on shutdown.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Enterprise
Run ID: 539e1bb4-0a60-4cd1-880a-82c4de95efd0
📒 Files selected for processing (9)
crates/cli/src/alignment/mod.rscrates/cli/src/launcher.rscrates/cli/src/session.rscrates/cli/tests/coverage/launcher_tests.rscrates/cli/tests/coverage/session_tests.rspython/nemo_relay/README.mdpython/nemo_relay/__init__.pypython/nemo_relay/nat_exporter.pypython/tests/test_nat_exporter.py
📜 Review details
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (2)
- GitHub Check: Check / Run
- GitHub Check: Preview docs
🧰 Additional context used
📓 Path-based instructions (29)
**/*.rs
📄 CodeRabbit inference engine (.agents/skills/add-binding-feature/SKILL.md)
Use
snake_casenaming convention for Rust identifiers (e.g.,nemo_relay_tool_call)
**/*.rs: Any Rust change must runjust test-rust
Any Rust change must runcargo fmt --all
Any Rust change must runcargo clippy --workspace --all-targets -- -D warnings
**/*.rs: Runcargo fmt --allfor all FFI work since it is Rust work
Runjust test-rustto validate FFI changes
Runcargo clippy --workspace --all-targets -- -D warningsto enforce strict linting on FFI workWhen Rust files changed as part of Go work, also run
cargo fmt --all,just test-rust, andcargo clippy --workspace --all-targets -- -D warnings
**/*.rs: Runcargo fmt --allwhen Rust files are changed as part of Node work
Runcargo clippy --workspace --all-targets -- -D warningswhen Rust files are changed as part of Node work
Runjust test-rustwhen Rust files are changed as part of Node work
**/*.rs: Runcargo fmt --allto format all Rust code
Runcargo clippy --workspace --all-targets -- -D warningsto enforce all clippy lints as errors
**/*.rs: Runcargo fmt --allwhen Rust files changed as part of WebAssembly work
Runcargo clippy --workspace --all-targets -- -D warningswhen Rust files changed as part of WebAssembly work
**/*.rs: If any Rust code changed, always runjust test-rust
If any Rust code changed, also runcargo fmt --all
If any Rust code changed, also runcargo clippy --workspace --all-targets -- -D warnings
Run Rust formatting withcargo fmt --all
Run Rust linting withcargo clippy --workspace --all-targets -- -D warnings
**/*.rs: Usecargo fmtfor Rust code formatting
Runcargo clippy -- -D warningsto lint Rust code and treat all warnings as errors
Use Rust snake_case naming convention for Rust identifiers
Include SPDX license header in all Rust source files using double-slash comment syntax
Validate Rust code withuv run pre-commit run --all-filesto enforce cargo fmt formatting check, cargo clippy lints, and cargo deny aud...
Files:
crates/cli/tests/coverage/session_tests.rscrates/cli/src/launcher.rscrates/cli/tests/coverage/launcher_tests.rscrates/cli/src/session.rscrates/cli/src/alignment/mod.rs
{crates/adaptive/**/*.rs,**/*test*.{rs,py,go,ts,js},**/*adaptive*test*.{rs,py,go,ts,js},docs/plugins/adaptive/**}
📄 CodeRabbit inference engine (.agents/skills/maintain-optimizer/SKILL.md)
Maintain documented and tested validation and report behavior for adaptive surfaces
Files:
crates/cli/tests/coverage/session_tests.rspython/tests/test_nat_exporter.pycrates/cli/tests/coverage/launcher_tests.rs
**/{Cargo.toml,**/*.rs}
📄 CodeRabbit inference engine (.agents/skills/maintain-packaging/SKILL.md)
Maintain consistency between Rust package names in
Cargo.tomland their actual usage across the codebase
Files:
crates/cli/tests/coverage/session_tests.rscrates/cli/src/launcher.rscrates/cli/tests/coverage/launcher_tests.rscrates/cli/src/session.rscrates/cli/src/alignment/mod.rs
**/*.{h,hpp,c,cpp,rs}
📄 CodeRabbit inference engine (.agents/skills/maintain-packaging/SKILL.md)
Ensure FFI header and library naming follows consistent conventions across platform-specific builds
Files:
crates/cli/tests/coverage/session_tests.rscrates/cli/src/launcher.rscrates/cli/tests/coverage/launcher_tests.rscrates/cli/src/session.rscrates/cli/src/alignment/mod.rs
**/*.{rs,toml}
📄 CodeRabbit inference engine (.agents/skills/rename-surfaces/SKILL.md)
Update Rust crate names and module prefixes during coordinated rename operations
Files:
crates/cli/tests/coverage/session_tests.rscrates/cli/src/launcher.rscrates/cli/tests/coverage/launcher_tests.rscrates/cli/src/session.rscrates/cli/src/alignment/mod.rs
**/*.{rs,py,js,ts,tsx,jsx,go,sh,toml,yaml,yml,md}
📄 CodeRabbit inference engine (AGENTS.md)
Keep SPDX headers on source, docs, scripts, and configuration files. The project is Apache-2.0.
Files:
crates/cli/tests/coverage/session_tests.rspython/nemo_relay/__init__.pypython/nemo_relay/nat_exporter.pypython/tests/test_nat_exporter.pypython/nemo_relay/README.mdcrates/cli/src/launcher.rscrates/cli/tests/coverage/launcher_tests.rscrates/cli/src/session.rscrates/cli/src/alignment/mod.rs
**/*.{rs,py,go,js,ts,tsx}
📄 CodeRabbit inference engine (AGENTS.md)
Follow binding naming conventions: Rust and Python use
snake_case, C FFI exports prefixednemo_relay_, Go usesPascalCasefor public APIs, Node.js usescamelCase.
Files:
crates/cli/tests/coverage/session_tests.rspython/nemo_relay/__init__.pypython/nemo_relay/nat_exporter.pypython/tests/test_nat_exporter.pycrates/cli/src/launcher.rscrates/cli/tests/coverage/launcher_tests.rscrates/cli/src/session.rscrates/cli/src/alignment/mod.rs
crates/**/*.rs
📄 CodeRabbit inference engine (AGENTS.md)
crates/**/*.rs: Keep async behavior on the existing tokio-based model. Bindings should preserve callback and future lifetimes rather than blocking or hiding async work unexpectedly.
UseJson = serde_json::Valuein Rust-facing runtime APIs for JSON payload handling.
Files:
crates/cli/tests/coverage/session_tests.rscrates/cli/src/launcher.rscrates/cli/tests/coverage/launcher_tests.rscrates/cli/src/session.rscrates/cli/src/alignment/mod.rs
{crates/**/tests/**,python/tests/**,go/nemo_relay/**/*_test.go}
⚙️ CodeRabbit configuration file
{crates/**/tests/**,python/tests/**,go/nemo_relay/**/*_test.go}: Tests should cover the behavior promised by the changed API surface, including error paths and cross-request isolation where relevant.
Prefer assertions on lifecycle events, scope stacks, middleware ordering, and binding parity over shallow smoke tests.
Files:
crates/cli/tests/coverage/session_tests.rspython/tests/test_nat_exporter.pycrates/cli/tests/coverage/launcher_tests.rs
{crates/python/src/py_api/**/*.rs,python/nemo_relay/**/*.py,python/nemo_relay/**/*.pyi}
📄 CodeRabbit inference engine (.agents/skills/add-binding-feature/SKILL.md)
Update Python native binding in
crates/python/src/py_api/mod.rswith Python wrapper docstring inpython/nemo_relay/<module>.pyand type stubs inpython/nemo_relay/*.pyimodules
Files:
python/nemo_relay/__init__.pypython/nemo_relay/nat_exporter.py
python/nemo_relay/**/*.py
📄 CodeRabbit inference engine (.agents/skills/add-binding-feature/SKILL.md)
Use
snake_casenaming convention for Python identifiers (e.g.,nemo_relay.tools.call)Format changed Python wrapper and test files with
uv run ruff format pythonPython wrapper modules live under
python/nemo_relay/; the native extension is built fromcrates/pythonwithmaturin.
Files:
python/nemo_relay/__init__.pypython/nemo_relay/nat_exporter.py
{pyproject.toml,**/*.py}
📄 CodeRabbit inference engine (.agents/skills/maintain-packaging/SKILL.md)
Maintain consistency between Python package names in
pyproject.tomland import paths used throughout the codebase
Files:
python/nemo_relay/__init__.pypython/nemo_relay/nat_exporter.pypython/tests/test_nat_exporter.py
**/*.{py,txt,toml,cfg,yaml,yml}
📄 CodeRabbit inference engine (.agents/skills/rename-surfaces/SKILL.md)
Update Python package names and top-level module imports during coordinated rename operations
Files:
python/nemo_relay/__init__.pypython/nemo_relay/nat_exporter.pypython/tests/test_nat_exporter.py
**/*.py
📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)
**/*.py: Run Python formatting withuv run ruff format python
Run Python testing withuv run pytest -k "<pattern>"
**/*.py: Use Ruff with rule sets E, F, W, I for Python linting
Use Ruff formatter with line length 120 and double quotes for Python code formatting
Runtyfor Python type checking
Use Python snake_case naming convention for Python identifiers
Include SPDX license header in all Python source files using hash comment syntax
Validate Python code withuv run pre-commit run --all-filesto enforce Ruff linting and formatting, and ty type checking
Files:
python/nemo_relay/__init__.pypython/nemo_relay/nat_exporter.pypython/tests/test_nat_exporter.py
**/*.{md,mdx,py,sh,yaml,yml,toml,json}
📄 CodeRabbit inference engine (.agents/skills/contribute-docs/SKILL.md)
Keep package names, repo references, and build commands current
Files:
python/nemo_relay/__init__.pypython/nemo_relay/nat_exporter.pypython/tests/test_nat_exporter.pypython/nemo_relay/README.md
python/nemo_relay/**/*
⚙️ CodeRabbit configuration file
python/nemo_relay/**/*: Review Python wrapper changes for typed API consistency, contextvars-based scope isolation, async behavior, and parity with the native extension.
Stubs and runtime implementations should stay aligned.
Files:
python/nemo_relay/__init__.pypython/nemo_relay/nat_exporter.pypython/nemo_relay/README.md
**/test_*.{py,py}
📄 CodeRabbit inference engine (.agents/skills/add-integration/SKILL.md)
Relevant integration tests or smoke coverage must exist for the integration path
Files:
python/tests/test_nat_exporter.py
python/**/*test*.py
📄 CodeRabbit inference engine (.agents/skills/test-python-binding/SKILL.md)
python/**/*test*.py: Do not add@pytest.mark.asyncioto any test in Python test files
Do not add a-> Nonereturn type annotation to test functions
When mocking a class, useunittest.mock.MagicMockorunittest.mock.AsyncMockwith thespecconstructor argument when necessary, rather than defining a new class
Prefix mocked class names withmock, notfake
Prefer pytest fixtures over helper methods in Python tests
Preferpytest.mark.parametrizeover creating individual tests for different input types
Files:
python/tests/test_nat_exporter.py
python/**/{conftest.py,*test*.py}
📄 CodeRabbit inference engine (.agents/skills/test-python-binding/SKILL.md)
When creating a fixture follow the pattern:
@pytest.fixture(name="<fixture_name>"[, scope="<scope>"]) def <fixture_name>_fixture() -> <return_type>:and only specify the scope argument when the value is something other than "function"
Files:
python/tests/test_nat_exporter.py
**/*.{md,rst,html,txt}
📄 CodeRabbit inference engine (.agents/skills/review-doc-style/assets/nvidia-style-brand-terminology.md)
**/*.{md,rst,html,txt}: Always spellNVIDIAin all caps. Do not useNvidia,nvidia,nVidia,nVIDIA, orNV.
Usean NVIDIAbefore a noun because the name starts with an 'en' sound.
Do not add a registered trademark symbol afterNVIDIAwhen referring to the company.
Use trademark symbols with product names only when the document type or legal guidance requires them.
Verify official capitalization, spacing, and hyphenation for product names.
Precede NVIDIA product names withNVIDIAon first mention when it is natural and accurate.
Do not rewrite product names for grammar or title-case rules.
Preserve third-party product names according to the owner's spelling.
Include the company name and full model qualifier on first use when it helps identify the model.
Preserve the official capitalization and punctuation of model names.
Use shorter family names only after the full name is established.
Spell out a term on first use and put the acronym in parentheses unless the acronym is widely understood by the intended audience.
Use the acronym on later mentions after it has been defined.
For long documents, reintroduce the full term if readers might lose context.
Form plurals of acronyms withs, not an apostrophe, such asGPUs.
In headings, common acronyms can remain abbreviated. Spell out the term in the first or second sentence of the body.
Common terms such asCPU,GPU,PC,API, andUIusually do not need to be spelled out for developer audiences.
Files:
python/nemo_relay/README.md
**/*.{md,rst,html}
📄 CodeRabbit inference engine (.agents/skills/review-doc-style/assets/nvidia-style-brand-terminology.md)
Link the first mention of a product name when the destination helps the reader.
Files:
python/nemo_relay/README.md
**/*.md
📄 CodeRabbit inference engine (.agents/skills/contribute-integration/SKILL.md)
Documentation must be updated if activation or usage changed
**/*.md: Use title case consistently in technical documentation headings
Avoid quotation marks, ampersands, and exclamation marks in headings
Keep product, event, research, and whitepaper names in their official title case
Use title case for table headers
Do not force social-media sentence case into technical docs
Format code elements, commands, parameters, package names, and expressions in monospace
Format directories, file names, and paths in monospace using backticks
Use angle brackets inside monospace for variables inside paths, such as/home/<username>/.login
Format error messages and strings in quotation marks, keeping literal code strings in code formatting when clearer
Format UI buttons, menus, fields, and labels in bold
Use angle brackets between UI labels for menu paths, such as File > Save As
Use italics for new terms on first use, sparingly and only when introducing the term
Use italics for publication titles
Format keyboard shortcuts in plain text, such as Press Ctrl+Alt+Delete
Use owner/repo link text for GitHub repositories, preferring[NVIDIA/NeMo](link)over prose references like 'the GitHub repo'
Introduce every code block with a complete sentence
Do not make a code block complete the grammar of the previous sentence
Do not continue a sentence after a code block
Use syntax highlighting when the format supports it for code blocks
Avoid the word 'snippet' unless the surrounding docs already use it as a term of art
Keep inline method, function, and class references consistent with nearby docs, omitting empty parentheses for prose readability when no call is shown
Use descriptive anchor text that matches the destination title when possible for links
Avoid raw URLs in running text
Avoid generic anchor text such as 'here,' 'this page,' and 'read more'
Include acronyms in link text when a linked term includes an acronym
Do not link long sentences or multiple sentences
Avoid links ...
Files:
python/nemo_relay/README.md
**/{docs,examples,**/*.md,*.patch,*.diff,.github,*.sh,*.yaml,*.yml}
📄 CodeRabbit inference engine (.agents/skills/rename-surfaces/SKILL.md)
Update documentation, examples, CI configuration, and patch artifacts when performing rename operations
Files:
python/nemo_relay/README.md
**/*.{md,rst,txt}
📄 CodeRabbit inference engine (.agents/skills/review-doc-style/assets/nvidia-style-guide.md)
Spell
NVIDIAin all caps. Do not useNvidia,nvidia, orNV.
Files:
python/nemo_relay/README.md
**/*.{md,rst}
📄 CodeRabbit inference engine (.agents/skills/review-doc-style/assets/nvidia-style-guide.md)
**/*.{md,rst}: Format commands, code elements, expressions, package names, file names, and paths as inline code.
Use descriptive link text. Avoid raw URLs and weak anchors such as "here" or "read more."
Use title case consistently for technical documentation headings.
Introduce code blocks, lists, tables, and images with complete sentences.
Write procedures as imperative steps. Keep steps parallel and split long procedures into smaller tasks.
Prefer active voice, present tense, short sentences, contractions, and plain English.
Usecanfor possibility and reservemayfor permission.
Useafterfor temporal relationships instead ofonce.
Preferrefer tooverseewhen the wording points readers to another resource.
Avoid culture-specific idioms, unnecessary Latinisms, jokes, and marketing exaggeration in technical docs.
Spell out months in body text, avoid ordinal dates, and use clear time zones.
Spell out whole numbers from zero through nine unless they are technical values, parameters, versions, or UI values.
Use numerals for 10 or greater and include commas in thousands.
Do not add trademark symbols to learning-oriented docs unless the source, platform, or legal guidance explicitly requires them.
Files:
python/nemo_relay/README.md
{docs/**,README.md,CONTRIBUTING.md,**/*.md}
📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)
Run docs link validation with
just docs-linkcheckwhen links change
Files:
python/nemo_relay/README.md
{docs/**,README.md,**/Cargo.toml,**/package.json,**/*.md}
📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)
Ensure renamed public surfaces are reflected consistently in manifests and docs for large or public-facing changes
Files:
python/nemo_relay/README.md
**/*.{html,md,mdx}
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Include SPDX license header in HTML and Markdown files using HTML comment syntax
Files:
python/nemo_relay/README.md
**/README.md
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Update relevant crate or package README when that surface changed
Files:
python/nemo_relay/README.md
🪛 Ruff (0.15.15)
python/nemo_relay/nat_exporter.py
[warning] 33-33: Avoid specifying long messages outside the exception class
(TRY003)
python/tests/test_nat_exporter.py
[warning] 41-41: Do not call getattr with a constant attribute value. It is not any safer than normal property access.
Replace getattr with attribute access
(B009)
🔇 Additional comments (7)
python/nemo_relay/nat_exporter.py (3)
23-33: LGTM!
49-68: LGTM!
71-76: LGTM!python/nemo_relay/__init__.py (1)
27-28: LGTM!Also applies to: 192-201, 431-464
python/nemo_relay/README.md (1)
188-210: LGTM!python/tests/test_nat_exporter.py (2)
40-41: LGTM!
14-37: ⚡ Quick winConfirm
scope_categoryend discriminant — CanonicalScopeEventJSON includes ascope_categoryfield and emits"scope_category": "end"for the pop/end event (e.g.,crates/core/tests/unit/types_tests.rsandcrates/ffi/tests/integration/api_tests.rsassert"scope_category" == "end"), soassert lines[2]["scope_category"] == "end"matches the real serialization.
willkill07
left a comment
There was a problem hiding this comment.
The improvements are more representative than the added exporter. Could the PR title be updated accordingly?
Otherwise, my only feedback is related to the added exporter and what value it adds over the existing AtofExporter.
Conditionally approving.
Temporarily patch and restore Hermes hook configuration during transparent Relay runs so Hermes lifecycle events reach the local gateway without requiring a permanent user config change. Correlate Hermes task and tool hook events back to the owning session, including fallback matching when provider tool IDs differ between pre and post hooks. Add a Python NatTelemetryExporter that writes canonical ATOF JSONL for NAT ingestion, with README and tests. Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com>
#### Overview Fixes a Hermes CLI hook-forward issue where sparse `pre_tool_call` payloads could create a separate task-id session and export a one-step ATIF trajectory containing only `gateway_shutdown`. - [x] I confirm this contribution is my own work, or I have the right to submit it under this project's license. - [x] I searched existing issues and open pull requests, and this does not duplicate existing work. #### Details Public Hermes releases can emit `pre_tool_call` hooks with a task id but without enough stable session/tool-call identity to pair the hook with the eventual `post_tool_call`. NeMo Relay previously treated the task id as a session id, opened a synthetic Hermes session, and later closed it during gateway shutdown. This PR drops uncorrelatable Hermes `pre_tool_call` hooks unless they include both a real session identifier and a stable tool-call id. The `post_tool_call` hook still records the tool result and attaches it to the main trajectory. Added regression coverage for the sparse `pre_tool_call` case and updated adapter coverage for correlatable Hermes tool payloads. #### Where should the reviewer start? Start with `crates/cli/src/adapters/hermes.rs`, especially the `pretoolcall` guard and `hermes_pre_tool_call_is_correlatable` helper. The key regression test is `hermes_uncorrelatable_pre_tool_call_does_not_create_shutdown_trajectory` in `crates/cli/tests/coverage/session_tests.rs`. #### Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to) - Relates to NVIDIA#176 ## Summary by CodeRabbit * **Improvements** * Hermes adapter now filters pre-tool-call events that lack valid session and tool call identifiers, reducing noise in the event stream. * **Tests** * Added test coverage validating proper filtering of uncorrelatable Hermes payloads and session trajectory behavior. [](https://app.coderabbit.ai/change-stack/NVIDIA/NeMo-Relay/pull/182?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack) Authors: - Bryan Bednarski (https://github.com/bbednarski9) Approvers: - Will Killian (https://github.com/willkill07) URL: NVIDIA#182 Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com>
#### Overview Adds ATIF exporter de-duplication for overlapping LLM spans that represent the same physical provider request, such as a hook-observed span and a gateway-observed span. - [x] I confirm this contribution is my own work, or I have the right to submit it under this project's license. - [x] I searched existing issues and open pull requests, and this does not duplicate existing work. #### Details Some harnesses can emit multiple LLM spans for one underlying request. Without de-duplication, ATIF can contain repeated user/agent steps for a single model call. This PR adds an exporter pre-pass that collects complete LLM start/end span candidates, detects overlapping duplicates under the same parent/model, suppresses the lower-fidelity span, and merges metrics from the suppressed span into the canonical step when needed. It also adds tests for overlapping hook/gateway spans, preferring a higher-fidelity gateway span over a non-exact hook summary, and preserving sequential same-content LLM calls as separate steps. #### Where should the reviewer start? Start with `crates/core/src/observability/atif.rs`, specifically `build_llm_dedupe`, `same_physical_llm_request`, and `llm_event_fidelity_score`. The main tests are in `crates/core/tests/unit/atif_tests.rs` around `test_exporter_dedupes_overlapping_hook_and_gateway_llm_spans`. #### Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to) - Relates to NVIDIA#176 ## Summary by CodeRabbit * **New Features** * Enhanced ATIF exporter with improved LLM span deduplication and token-metric consolidation from multiple instrumentation sources. * **Bug Fixes** * Fixed metric handling for overlapping LLM requests to prevent inaccurate data consolidation. * **Tests** * Added comprehensive unit tests validating LLM span deduplication behavior across instrumentation scenarios. [](https://app.coderabbit.ai/change-stack/NVIDIA/NeMo-Relay/pull/183?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack) Authors: - Bryan Bednarski (https://github.com/bbednarski9) Approvers: - Will Killian (https://github.com/willkill07) URL: NVIDIA#183 Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com>
#### Overview
Documentation cleanup that removes stale "NeMo Flow" / "NAT" / "NeMo Agent
Toolkit" naming in favor of the current project name (NeMo Relay), refreshes the
Quick Start landing page with per-binding navigation, and corrects a few stale
integration notes. Docs-only; no runtime or public API behavior changes.
- [x] I confirm this contribution is my own work, or I have the right to submit it under this project's license.
- [x] I searched existing issues and open pull requests, and this does not duplicate existing work.
#### Details
- Quick Start (`docs/getting-started/quick-start/index.mdx`): add a 3-column
`CardGroup` linking to the Python, Node.js, and Rust Quick Start guides.
- Correct stale naming to "NeMo Relay":
- `.github/CODE_OF_CONDUCT.md`: title is now "NVIDIA NeMo Relay Code of Conduct".
- `README.md`, `docs/integrate-into-frameworks/about.mdx`,
`docs/integrate-into-frameworks/non-serializable-data.mdx`,
`docs/nemo-relay-cli/hermes.mdx`, `docs/resources/support-and-faqs.mdx`:
replace leftover "NeMo Flow" references with "NeMo Relay".
- `docs/about-nemo-relay/ecosystem.mdx`: rename the `Flow` node to `Relay` in
the architecture mermaid diagram.
- `AGENTS.md`: refresh the public-API integration notes to list LangGraph and
Deep Agents alongside LangChain, and point to `docs/supported-integrations/`
and `python/tests/integrations/`.
- `integrations/coding-agents/README.md`: remove the stale `hermes/` native
plugin bundle bullet.
#### Where should the reviewer start?
Start with the naming corrections, especially `.github/CODE_OF_CONDUCT.md` (it
was mistakenly set to "NeMo Flow") and the mermaid node rename in
`docs/about-nemo-relay/ecosystem.mdx`. Then review the new `CardGroup` in
`docs/getting-started/quick-start/index.mdx`. Validated with `just docs-linkcheck`
(0 errors; the only warning is the auth-gated redirects check being skipped).
#### Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)
- Relates to: none
## Summary by CodeRabbit
* **New Features**
* Added interactive quick-start cards for Python, Node.js, and Rust linking to language guides.
* **Documentation**
* Broadened public integrations list to include LangGraph and Deep Agents.
* Standardized product name/terminology from “Flow” to “Relay” across docs and diagrams.
* Clarified framework boundary guidance and non-serializable data wording.
* Updated Hermes agent guidance and support/FAQ wording.
* Adjusted Hermes packaging guidance in integrations docs.
[](https://app.coderabbit.ai/change-stack/NVIDIA/NeMo-Relay/pull/188?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack)
Authors:
- Zhongxuan (Daniel) Wang (https://github.com/zhongxuanwang-nv)
Approvers:
- Will Killian (https://github.com/willkill07)
URL: NVIDIA#188
Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com>
#### Overview Register the adaptive plugin component before `nemo-relay doctor` validates plugin configuration, and make Cursor support limitations more explicit in the CLI docs. - [x] I confirm this contribution is my own work, or I have the right to submit it under this project's license. - [x] I searched existing issues and open pull requests, and this does not duplicate existing work. #### Details - Register the built-in adaptive component before doctor plugin validation so adaptive plugin configs no longer report `plugin.unknown_component`. - Add a focused doctor regression test for adaptive validation. - Mark Cursor support as partial/highly experimental in the CLI overview. - Add prominent Cursor cautions for Fern and GitHub docs covering manual proxy/API-key setup, lack of automatic model routing, and subagent model behavior. Validation: - `cargo fmt --check` - `cargo test -p nemo-relay-cli collect_observability_registers_adaptive_before_validation -- --nocapture` - `just docs` - Commit pre-checks: docs linkcheck, `cargo fmt`, `cargo clippy`, and `cargo check` #### Where should the reviewer start? Start with `crates/cli/src/doctor.rs` for the adaptive registration path, then `docs/nemo-relay-cli/cursor.mdx` for the Cursor support warning. #### Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to) - Relates to: none ## Summary by CodeRabbit * **Documentation** * Clarified Cursor integration support as highly experimental with limitations; cannot automatically route model traffic through the gateway * Updated observability support matrix for Cursor from fully supported to partial (highly experimental) * **New Features** * Enhanced doctor command to register adaptive components during observability health checks [](https://app.coderabbit.ai/change-stack/NVIDIA/NeMo-Relay/pull/186?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack) Authors: - Will Killian (https://github.com/willkill07) Approvers: - Bryan Bednarski (https://github.com/bbednarski9) URL: NVIDIA#186 Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com>
#### Overview Fixes the Hermes gateway session fallback and tightens ATIF LLM dedupe so complementary hook/gateway spans are only collapsed when they represent the same physical request. - [x] I confirm this contribution is my own work, or I have the right to submit it under this project's license. - [x] I searched existing issues and open pull requests, and this does not duplicate existing work. #### Details - Uses the OpenAI-compatible request body session_id as a gateway fallback when explicit session headers are absent. - Keeps the existing explicit Claude/Codex session fallbacks ahead of the OpenAI body fallback. - Requires complementary hook/gateway LLM spans to share a request signature or strong request correlation key before ATIF dedupes them. - Adds regression coverage for gateway fallback selection and concurrent overlapping LLM spans that should remain distinct. #### Where should the reviewer start? Start with `crates/cli/src/alignment/mod.rs` for the gateway fallback behavior, then review `crates/core/src/observability/atif.rs` for the strengthened complementary hook/gateway dedupe guard. The focused regression tests are in `crates/cli/tests/coverage/alignment_tests.rs`, `crates/cli/tests/coverage/gateway_tests.rs`, and `crates/core/tests/unit/atif_tests.rs`. #### Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to) - Closes NVIDIA#176 ## Summary by CodeRabbit * **Bug Fixes** * Session ID resolution enhanced to properly support OpenAI-compatible API request formats, including additional fallback to request body identifiers * LLM span correlation and deduplication logic improved with request-level identifier matching, enabling more accurate observability tracking and better event correlation for request tracing [](https://app.coderabbit.ai/change-stack/NVIDIA/NeMo-Relay/pull/189?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack) Authors: - Bryan Bednarski (https://github.com/bbednarski9) Approvers: - Will Killian (https://github.com/willkill07) URL: NVIDIA#189 Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com>
#### Overview Update the 0.3 release documentation so the release notes, README, and integration support pages reflect the current 0.3 feature set and compatibility notes. - [x] I confirm this contribution is my own work, or I have the right to submit it under this project's license. - [x] I searched existing issues and open pull requests, and this does not duplicate existing work. #### Details - Expanded the 0.3 release notes with breaking changes, feature highlights, known issues, fixed issues, and links to the most relevant feature documentation. - Removed the extra release-notes related-topics page and added a Fern redirect for the removed URL. - Updated README and docs for ATIF v1.7, Node.js 24, non-blocking subscriber delivery, CLI harness support, OpenClaw partial security support, and current OpenClaw requirements. - Cleaned stale rename and subscriber wording in support, performance, subscriber, and OpenClaw documentation. #### Where should the reviewer start? Start with `docs/about-nemo-relay/release-notes/index.mdx` and `docs/about-nemo-relay/release-notes/highlights.mdx` for the release note shape, then review `README.md` and `docs/supported-integrations/openclaw-plugin.mdx` for the feature-support reconciliation. #### Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to) - Relates to: none ## Summary by CodeRabbit * **Documentation** * Updated release notes, highlights, and installation docs for NeMo Relay 0.3 (project rename, explicit Apache 2.0 license). * Node.js 24+ requirement, CLI installation instructions, support-matrix and known-issues expanded (agent harness details, OpenClaw notes, non-blocking native subscriber delivery requiring explicit flush). * Removed/redirected an obsolete related-topics page. * **New Features** * Pre-tool guardrails via OpenClaw hook-backed telemetry; OpenClaw minimum version bumped. [](https://app.coderabbit.ai/change-stack/NVIDIA/NeMo-Relay/pull/180?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack) Authors: - Will Killian (https://github.com/willkill07) Approvers: - Bryan Bednarski (https://github.com/bbednarski9) URL: NVIDIA#180 Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com>
#### Overview
Fix Fern release publishing so version snapshots are built from the selected release tag source docs instead of whatever `docs-website` already has in `fern/pages-dev`.
- [x] I confirm this contribution is my own work, or I have the right to submit it under this project's license.
- [x] I searched existing issues and open pull requests, and this does not duplicate existing work.
#### Details
- Check out the selected SemVer tag in `release-version-docs` for both tag pushes and manual dispatches.
- Generate ignored API reference pages from that tag before snapshotting so versioned docs include tag-accurate Python, Node.js, and Rust references.
- Make `sync_fern_docs_branch.py release-version` require `--source-root` and copy source docs directly into the versioned Fern pages, removing the stale `pages-dev` fallback.
Validation:
- `ruby -e 'require "yaml"; Dir[".github/workflows/*.{yml,yaml}"].each { |f| YAML.load_file(f) }; puts "yaml-ok"'`
- `uv run --no-sync python -m py_compile scripts/docs/sync_fern_docs_branch.py`
- Temporary `release-version --source-root ... --tag 0.3.0` snapshot check verified the detailed release highlights land in `fern/pages-v0.3.0`.
- `uv run pre-commit run --files .github/workflows/fern-docs.yml scripts/docs/sync_fern_docs_branch.py`
- `uv run pre-commit run --all-files`
#### Where should the reviewer start?
Start with `.github/workflows/fern-docs.yml`, specifically the `release-version-docs` job, then review `scripts/docs/sync_fern_docs_branch.py` for the simplified source-to-version snapshot path.
#### Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)
- Relates to: none
## Summary by CodeRabbit
* **Chores**
* Improved documentation release workflow: longer job timeout, added workspace env var, and checkout of the computed release tag.
* Release docs now include generated API reference and build snapshots directly from the source docs using a configurable source-root.
* Updated link/path handling for versioned docs and CLI now requires an explicit source-root for release snapshot creation.
[](https://app.coderabbit.ai/change-stack/NVIDIA/NeMo-Relay/pull/194?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack)
Authors:
- Will Killian (https://github.com/willkill07)
Approvers:
- Bryan Bednarski (https://github.com/bbednarski9)
URL: NVIDIA#194
Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com>
#### Overview Fix manual Fern release snapshot reruns for historical tags. The release job should read documentation from the selected tag, but it must use the current workflow helper script so older tags can be repaired after workflow fixes land. - [x] I confirm this contribution is my own work, or I have the right to submit it under this project's license. - [x] I searched existing issues and open pull requests, and this does not duplicate existing work. #### Details - Add a separate `workflow-checkout` in `release-version-docs` for current workflow tooling. - Keep `source-checkout` pinned to the selected release tag so the docs content remains tag-sourced. - Invoke `sync_fern_docs_branch.py` from `workflow-checkout`, fixing manual reruns such as `tag=0.3.0` where the tag contains an older helper that does not support `--source-root`. Validation: - Reproduced the failing shape from https://github.com/NVIDIA/NeMo-Relay/actions/runs/26668928588/job/78608028705: source checkout from `0.3.0`, helper script from current branch, `release-version --source-root ... --tag 0.3.0`. - Verified the snapshot writes `fern/pages-v0.3.0` with the detailed release highlights. - `ruby -e 'require "yaml"; Dir[".github/workflows/*.{yml,yaml}"].each { |f| YAML.load_file(f) }; puts "yaml-ok"'` - `uv run pre-commit run --files .github/workflows/fern-docs.yml` - `uv run pre-commit run --all-files` #### Where should the reviewer start? Start with `.github/workflows/fern-docs.yml` in the `release-version-docs` job. The important detail is that workflow tools and release docs content now come from different checkouts. #### Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to) - Relates to: https://github.com/NVIDIA/NeMo-Relay/actions/runs/26668928588/job/78608028705 ## Summary by CodeRabbit * **Chores** * Updated internal CI/CD workflow configuration for documentation synchronization processes. **Note:** This release contains no end-user visible changes. Updates are limited to workflow infrastructure improvements. [](https://app.coderabbit.ai/change-stack/NVIDIA/NeMo-Relay/pull/195?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack) Signed-off-by: Will Killian <wkillian@nvidia.com> Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com>
* docs: fix Node.js quickstart scope handle and LLM request usage
The Node.js quick start and the nemo-relay-node package README passed the
informational handle from `withScope` straight into `event`, `toolCallExecute`,
and `llmCallExecute`. Those APIs expect either `null` (the current scope) or a
real `ScopeHandle`, so the plain handle object failed with "Failed to recover
`ScopeHandle` type from napi value" on the first call. The rejected promise then
skipped `deregisterSubscriber`, leaving the subscriber's ThreadsafeFunction
ref'd and hanging the process.
The LLM example also built the request with `new LlmRequest(...)`, but
`llmCallExecute` takes a plain `{ headers, content }` JSON object.
Update both examples to pass `null` for the active scope and a plain request
object, matching the behavior covered by the Node binding tests.
Signed-off-by: Zhongxuan Wang <daniewang@nvidia.com>
* fix: pass a real ScopeHandle to Node withScope callbacks
Node's withScope handed its callback a plain JSON object instead of a
real ScopeHandle, so passing it back into event, toolCallExecute, or
llmCallExecute threw "Failed to recover ScopeHandle type from napi
value" on the first call, and the skipped deregisterSubscriber left the
process hung. Materialize a real ScopeHandle on the JS thread inside the
threadsafe-function call (via a new Arg0Builder path in PromiseAwareFn)
so the callback receives a usable handle, matching the Rust, Python, and
WebAssembly bindings.
Keep the Node quick start passing the handle and send the LLM request as
a plain { headers, content } object, and add a Node test that reuses the
callback handle across event, pushScope, toolCallExecute, and
llmCallExecute.
Signed-off-by: Zhongxuan Wang <daniewang@nvidia.com>
* docs: keep Node quick start using LlmRequest
LlmRequest delivers headers and content to llmCallExecute correctly, so
the quick start needs no plain-object request. Restore the quick start to
its original LlmRequest usage; the scope-handle fix alone makes the
example run end to end.
Signed-off-by: Zhongxuan Wang <daniewang@nvidia.com>
---------
Signed-off-by: Zhongxuan Wang <daniewang@nvidia.com>
Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com>
…VIDIA#201) #### Overview The package READMEs' "Getting Started" examples register a subscriber, emit events, then deregister without calling flush. Native subscriber delivery is non-blocking, so an example that exits right after deregistering can lose queued events. This updates all four package READMEs to flush before deregister, matching the quickstarts (which already flush). - [x] I confirm this contribution is my own work, or I have the right to submit it under this project's license. - [x] I searched existing issues and open pull requests, and this does not duplicate existing work. #### Details **Functional fixes** (verified by running — the example dropped events without flush): - `python/nemo_relay/README.md`: printed nothing, or intermittently a single partial line; the `to_dict()` / `to_json()` callback never ran. Adds `nemo_relay.subscribers.flush()` before each `deregister(...)`. - `go/nemo_relay/README.md`: flaky — dropped the trailing tool-end and scope-end events on exit (2 of 3 runs partial). Adds `defer nemo.FlushSubscribers()`, ordered to run after the deferred `scope.Pop` emits the scope-end event and before the deferred deregister. **Parity** (not currently broken; updated so every README models the same pattern): - `crates/node/README.md`: Node keeps the event loop alive via the ref'd ThreadsafeFunction, so it already delivers; adds `flushSubscribers()` for consistency. - `crates/wasm/README.md`: WASM's `flushSubscribers()` is a single-threaded no-op (tested as such); adds it for consistency. #### Where should the reviewer start? The four README diffs — each adds a flush before `deregister(...)` (Node/WASM also import `flushSubscribers`). Verified by running each example with the flush added: events deliver deterministically (Go 5/5, Python and Node 3/3 across repeated runs). WASM's `flushSubscribers` is an exported no-op (`crates/wasm/src/api/mod.rs`; covered by `crates/wasm/tests-js/scope_tests.mjs`). Note: the `crates/node/README.md` example also depends on the Node binding change in NVIDIA#196 (pass a real `ScopeHandle` to `withScope` callbacks). That is orthogonal to this flush change — NVIDIA#196 is code-only and does not touch the README. #### Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to) - Relates to: NVIDIA#196 (Node `withScope` ScopeHandle; orthogonal, touches the same Node README example) ## Summary by CodeRabbit * **Documentation** * Updated Getting Started examples across Python, Node.js, WebAssembly, and Go language bindings to reflect the proper sequence for flushing subscribers before deregistration. Authors: - Zhongxuan (Daniel) Wang (https://github.com/zhongxuanwang-nv) Approvers: - Will Killian (https://github.com/willkill07) URL: NVIDIA#201 Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com>
…VIDIA#200) #### Overview Reorder the "Register Plugin Behavior" guide so a plugin kind is registered before it is activated, and document the `validate()`-warns / `initialize()`-raises behavior. Run on its own, the **Activation APIs** example previously failed with `RuntimeError: ... 'header-plugin' is not registered`, because the kind it activates is only registered later on the page in the **Header Plugin Example**. - [x] I confirm this contribution is my own work, or I have the right to submit it under this project's license. - [x] I searched existing issues and open pull requests, and this does not duplicate existing work. #### Details - `docs/build-plugins/register-behavior.mdx`: move the **Header Plugin Example** (which calls `plugin.register("header-plugin", ...)`) above the **Activation APIs** lifecycle so the page reads and runs top to bottom. Drop the now-redundant "Register the plugin kind" item from the activation list, and note that an unregistered kind is reported by `validate()` only as a warning under the default `unknown_component="warn"` policy, while `initialize()` raises. The reorder moves whole sections only; no code-block content changed (verified by an order-independent line diff). - `docs/build-plugins/validate-configuration.mdx`: add the same clarification to **Validate Before Initialization** — the error-only `has_errors` check does not catch an unknown/unregistered kind; register kinds first, or set `unknown_component="error"`. Heads-up: this edits the same `register-behavior.mdx` **Activation APIs** section as NVIDIA#199 (which adds `plugin.layer(...)` to that example but does not change the registration ordering). The two overlap, so whichever lands second will need a small conflict resolution. #### Where should the reviewer start? `docs/build-plugins/register-behavior.mdx` — confirm the section reorder preserved every code block (an order-independent line diff shows only the numbered list and intro changed) and that the Activation APIs example now follows the Header Plugin Example that registers the kind. Verified locally: running the Header Plugin Example block and then the Activation APIs block in one process registers `header-plugin`, after which `validate()` / `initialize()` / `clear()` succeed with empty diagnostics and no `RuntimeError`. Targeted `pre-commit` (trailing whitespace, end-of-files, docs markdown linkcheck) passes on both files. #### Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to) - Relates to: NVIDIA#199 (edits the same `register-behavior.mdx` section) ## Summary by CodeRabbit * **Documentation** * Clarified that plugin kinds must be registered before calling initialize() * Explained that unknown/unregistered component kinds are reported as warnings by default during validation and may still cause errors at initialization unless policy is changed * Reorganized and updated the activation guide and language across Python/Node/Rust examples for clearer ordering and guidance Authors: - Zhongxuan (Daniel) Wang (https://github.com/zhongxuanwang-nv) Approvers: - Will Killian (https://github.com/willkill07) URL: NVIDIA#200 Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com>
#### Overview This PR covers the OpenClaw-focused ATIF, ATOF, and OpenInference consistency work against the shared observability contract. - [x] I confirm this contribution is my own work, or I have the right to submit it under this project's license. - [x] I searched existing issues and open pull requests, and this does not duplicate existing work. #### Details - Normalizes Anthropic `/v1/messages` content blocks so ATIF promotes `tool_use` blocks into tool calls while keeping readable assistant text and cache usage metrics. - Improves OpenInference display and usage extraction for Anthropic messages, OpenAI Responses, and Chat Completions cache-token shapes. - Preserves raw ATOF JSONL lifecycle payloads for `/v1/messages`, `/v1/responses`, and `/v1/chat/completions`. - Extends OpenClaw hook replay usage normalization for provider-specific token and cache fields. #### Where should the reviewer start? Start with `crates/core/src/observability/atif.rs` and `crates/core/src/observability/openinference.rs`, then review the new fixture coverage in the matching Rust tests and `integrations/openclaw/test/llm-replay.test.ts`. #### Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to) - Relates to OpenClaw observability consistency work. #### Validation - `cargo test -p nemo-relay` - `cargo test -p nemo-relay atif` - `cargo test -p nemo-relay openinference` - `cargo test -p nemo-relay atof` - `cargo fmt --all --check` - `env PYO3_PYTHON=/Users/mnajafian/projects/NeMo-Relay/.venv/bin/python cargo clippy --workspace --all-targets -- -D warnings` - `just test-rust` - `npm test --workspace=nemo-relay-openclaw` - `npm run typecheck --workspace=nemo-relay-openclaw` - `npm run test:live --workspace=nemo-relay-openclaw` - `npm run pack:check --workspace=nemo-relay-openclaw` - `uv run pre-commit run --files crates/core/src/observability/atif.rs crates/core/src/observability/openinference.rs crates/core/tests/unit/atif_tests.rs crates/core/tests/unit/observability/atof_tests.rs crates/core/tests/unit/observability/openinference_tests.rs integrations/openclaw/src/hook-replay/llm.ts integrations/openclaw/test/llm-replay.test.ts` #### Additional local validation - Ran OpenClaw + NeMo Relay examples locally with NVIDIA Inference and Tavily-backed tool use across diverse prompts. - Inspected generated ATIF, ATOF, OpenInference, and Phoenix traces for multi-step agent runs. #### Follow-up Work I’ll keep the remaining items as follow-ups rather than expanding this PR: flattened OpenInference message/tool attrs, OpenClaw ATOF provenance/fidelity audit, and nested-subagent fixture coverage if we have a real OpenClaw scenario. ## Summary by CodeRabbit * **New Features** * Added support for Anthropic Claude message workflows including tool-use and function calling. * Extended OpenAI integration with response format support and cached token tracking. * Improved multi-provider LLM observability and message parsing. * **Tests** * Added test coverage for Anthropic and OpenAI LLM interaction scenarios. Authors: - https://github.com/mnajafian-nv Approvers: - Will Killian (https://github.com/willkill07) URL: NVIDIA#203 Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com>
Signed-off-by: Will Killian <wkillian@nvidia.com> Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com>
Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com>
Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com>
Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com>
Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com>
Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com>
73d3761 to
b1df460
Compare
There was a problem hiding this comment.
Actionable comments posted: 5
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/fern-docs.yml:
- Around line 377-378: Add an explanatory comment above the job-level
permissions block so reviewers know why contents: write is required;
specifically, update the permissions block that contains "permissions:" and
"contents: write" to include a short comment explaining the reason (e.g., token
usage for pushing docs or updating releases) and any security scope constraints.
In `@crates/cli/src/session.rs`:
- Around line 1605-1633: The matcher currently restricts to root-owned tools
when owner resolution returns None, causing a root-biased fallback; modify
matching_active_tool_key so the owner check only applies when owner_subagent_id
is Some: in matching_active_tool_key (used by remove_tool_handle_for_event)
change the filter closure to require active.owner_subagent_id ==
owner_subagent_id only when owner_subagent_id.is_some(), otherwise ignore
owner_subagent_id and match on name and arguments across all owners, then keep
the existing uniqueness check (matches.len() == 1) to return a single global
match.
In `@crates/cli/tests/coverage/adapters_tests.rs`:
- Around line 312-320: The test drops_uncorrelatable_hermes_pre_tool_call
currently omits both session_id and tool_call_id so it can pass when only one
correlatability gate exists; change the test input passed to hermes::adapt to
include a valid tool_call_id (e.g. "tool_call_id": "toolcall-1") while keeping
session_id omitted so the test isolates the missing-session correlatability
check and still expects the adapter to drop the event.
In `@crates/core/src/observability/atif.rs`:
- Around line 1327-1329: The current llm_response_signature function reduces the
signature to only the extracted message text (via extract_llm_response_message),
which causes different tool-call/raw-response structures — including tool-only
Anthropic responses that become empty strings — to be treated as identical;
update llm_response_signature to include the full promoted tool-call/raw
response structure (or a stable request correlation key when available) in the
returned signature so that tool invocations and raw response metadata are part
of the uniqueness check used by same_physical_llm_request; locate
llm_response_signature and include either the promoted tool call fields or a
correlation id from the response (and apply the same change to the other
occurrences noted around the commented ranges) so signature-only suppression no
longer collapses distinct calls.
In `@integrations/openclaw/src/hook-replay/llm.ts`:
- Around line 1197-1202: The cacheRead normalization currently misses top-level
fields named cached_tokens (and camelCase cachedTokens), causing cache-read
metrics to be dropped; update the cacheRead resolution in the llm replay
normalization to check for numberField(usage, 'cached_tokens') and
numberField(usage, 'cachedTokens') (prior to or alongside the other fallbacks)
so that usage objects already using the canonical NeMo Relay name or camelCase
variant are preserved; locate the cacheRead variable assignment and add those
checks to the existing chain that uses numberField(...) and
nestedNumberField(...).
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Enterprise
Run ID: 862a7d2a-a412-4c2a-8780-757a946da69f
📒 Files selected for processing (53)
.github/CODE_OF_CONDUCT.md.github/nightly-alpha-branches.yaml.github/workflows/fern-docs.ymlAGENTS.mdREADME.mdcrates/cli/src/adapters/hermes.rscrates/cli/src/alignment/mod.rscrates/cli/src/doctor.rscrates/cli/src/launcher.rscrates/cli/src/session.rscrates/cli/tests/coverage/adapters_tests.rscrates/cli/tests/coverage/alignment_tests.rscrates/cli/tests/coverage/doctor_tests.rscrates/cli/tests/coverage/gateway_tests.rscrates/cli/tests/coverage/launcher_tests.rscrates/cli/tests/coverage/session_tests.rscrates/core/src/observability/atif.rscrates/core/src/observability/openinference.rscrates/core/tests/unit/atif_tests.rscrates/core/tests/unit/observability/atof_tests.rscrates/core/tests/unit/observability/openinference_tests.rscrates/node/README.mdcrates/node/src/api/mod.rscrates/node/src/promise_call.rscrates/node/tests/scope_tests.mjscrates/wasm/README.mddocs/about-nemo-relay/concepts/subscribers.mdxdocs/about-nemo-relay/ecosystem.mdxdocs/about-nemo-relay/release-notes/highlights.mdxdocs/about-nemo-relay/release-notes/index.mdxdocs/about-nemo-relay/release-notes/known-issues.mdxdocs/about-nemo-relay/release-notes/related-topics.mdxdocs/build-plugins/register-behavior.mdxdocs/build-plugins/validate-configuration.mdxdocs/getting-started/quick-start/index.mdxdocs/integrate-into-frameworks/about.mdxdocs/integrate-into-frameworks/non-serializable-data.mdxdocs/nemo-relay-cli/about.mdxdocs/nemo-relay-cli/cursor.mdxdocs/nemo-relay-cli/hermes.mdxdocs/reference/performance.mdxdocs/resources/support-and-faqs.mdxdocs/supported-integrations/about.mdxdocs/supported-integrations/openclaw-plugin.mdxfern/docs.ymlgo/nemo_relay/README.mdintegrations/coding-agents/README.mdintegrations/coding-agents/cursor/README.mdintegrations/openclaw/README.mdintegrations/openclaw/src/hook-replay/llm.tsintegrations/openclaw/test/llm-replay.test.tspython/nemo_relay/README.mdscripts/docs/sync_fern_docs_branch.py
💤 Files with no reviewable changes (3)
- docs/about-nemo-relay/release-notes/related-topics.mdx
- .github/nightly-alpha-branches.yaml
- integrations/coding-agents/README.md
📜 Review details
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (2)
- GitHub Check: Check / Run
- GitHub Check: Preview docs
🧰 Additional context used
📓 Path-based instructions (54)
{docs/**,README.md,CONTRIBUTING.md}
📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)
{docs/**,README.md,CONTRIBUTING.md}: For docs-only changes, run targeted checks only if commands, package names, or examples changed. Usejust docsfor docs-site builds andjust docs-linkcheckwhen links changed
Run docs site build withjust docs
Files:
docs/build-plugins/validate-configuration.mdxdocs/nemo-relay-cli/about.mdxdocs/supported-integrations/about.mdxdocs/nemo-relay-cli/hermes.mdxdocs/about-nemo-relay/ecosystem.mdxdocs/getting-started/quick-start/index.mdxdocs/reference/performance.mdxdocs/integrate-into-frameworks/non-serializable-data.mdxdocs/about-nemo-relay/release-notes/highlights.mdxdocs/integrate-into-frameworks/about.mdxdocs/resources/support-and-faqs.mdxdocs/nemo-relay-cli/cursor.mdxdocs/about-nemo-relay/concepts/subscribers.mdxREADME.mddocs/supported-integrations/openclaw-plugin.mdxdocs/about-nemo-relay/release-notes/known-issues.mdxdocs/build-plugins/register-behavior.mdxdocs/about-nemo-relay/release-notes/index.mdx
{docs/**,README.md,CONTRIBUTING.md,**/*.md}
📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)
Run docs link validation with
just docs-linkcheckwhen links change
Files:
docs/build-plugins/validate-configuration.mdxdocs/nemo-relay-cli/about.mdxpython/nemo_relay/README.mdgo/nemo_relay/README.mddocs/supported-integrations/about.mdxdocs/nemo-relay-cli/hermes.mdxintegrations/coding-agents/cursor/README.mddocs/about-nemo-relay/ecosystem.mdxdocs/getting-started/quick-start/index.mdxdocs/reference/performance.mdxdocs/integrate-into-frameworks/non-serializable-data.mdxcrates/node/README.mdcrates/wasm/README.mddocs/about-nemo-relay/release-notes/highlights.mdxdocs/integrate-into-frameworks/about.mdxdocs/resources/support-and-faqs.mdxdocs/nemo-relay-cli/cursor.mdxdocs/about-nemo-relay/concepts/subscribers.mdxintegrations/openclaw/README.mdAGENTS.mdREADME.mddocs/supported-integrations/openclaw-plugin.mdxdocs/about-nemo-relay/release-notes/known-issues.mdxdocs/build-plugins/register-behavior.mdxdocs/about-nemo-relay/release-notes/index.mdx
{docs/**,README.md}
📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)
Verify README and docs entry points still match current package names and paths for large or public-facing changes
Files:
docs/build-plugins/validate-configuration.mdxdocs/nemo-relay-cli/about.mdxdocs/supported-integrations/about.mdxdocs/nemo-relay-cli/hermes.mdxdocs/about-nemo-relay/ecosystem.mdxdocs/getting-started/quick-start/index.mdxdocs/reference/performance.mdxdocs/integrate-into-frameworks/non-serializable-data.mdxdocs/about-nemo-relay/release-notes/highlights.mdxdocs/integrate-into-frameworks/about.mdxdocs/resources/support-and-faqs.mdxdocs/nemo-relay-cli/cursor.mdxdocs/about-nemo-relay/concepts/subscribers.mdxREADME.mddocs/supported-integrations/openclaw-plugin.mdxdocs/about-nemo-relay/release-notes/known-issues.mdxdocs/build-plugins/register-behavior.mdxdocs/about-nemo-relay/release-notes/index.mdx
{docs/**,examples/**,README.md}
📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)
Verify examples still run with documented commands for large or public-facing changes
Files:
docs/build-plugins/validate-configuration.mdxdocs/nemo-relay-cli/about.mdxdocs/supported-integrations/about.mdxdocs/nemo-relay-cli/hermes.mdxdocs/about-nemo-relay/ecosystem.mdxdocs/getting-started/quick-start/index.mdxdocs/reference/performance.mdxdocs/integrate-into-frameworks/non-serializable-data.mdxdocs/about-nemo-relay/release-notes/highlights.mdxdocs/integrate-into-frameworks/about.mdxdocs/resources/support-and-faqs.mdxdocs/nemo-relay-cli/cursor.mdxdocs/about-nemo-relay/concepts/subscribers.mdxREADME.mddocs/supported-integrations/openclaw-plugin.mdxdocs/about-nemo-relay/release-notes/known-issues.mdxdocs/build-plugins/register-behavior.mdxdocs/about-nemo-relay/release-notes/index.mdx
{docs/**,README.md,**/Cargo.toml,**/package.json,**/*.md}
📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)
Ensure renamed public surfaces are reflected consistently in manifests and docs for large or public-facing changes
Files:
docs/build-plugins/validate-configuration.mdxdocs/nemo-relay-cli/about.mdxpython/nemo_relay/README.mdgo/nemo_relay/README.mddocs/supported-integrations/about.mdxdocs/nemo-relay-cli/hermes.mdxintegrations/coding-agents/cursor/README.mddocs/about-nemo-relay/ecosystem.mdxdocs/getting-started/quick-start/index.mdxdocs/reference/performance.mdxdocs/integrate-into-frameworks/non-serializable-data.mdxcrates/node/README.mdcrates/wasm/README.mddocs/about-nemo-relay/release-notes/highlights.mdxdocs/integrate-into-frameworks/about.mdxdocs/resources/support-and-faqs.mdxdocs/nemo-relay-cli/cursor.mdxdocs/about-nemo-relay/concepts/subscribers.mdxintegrations/openclaw/README.mdAGENTS.mdREADME.mddocs/supported-integrations/openclaw-plugin.mdxdocs/about-nemo-relay/release-notes/known-issues.mdxdocs/build-plugins/register-behavior.mdxdocs/about-nemo-relay/release-notes/index.mdx
**/*.{md,mdx,py,sh,yaml,yml,toml,json}
📄 CodeRabbit inference engine (.agents/skills/contribute-docs/SKILL.md)
Keep package names, repo references, and build commands current
Files:
docs/build-plugins/validate-configuration.mdxdocs/nemo-relay-cli/about.mdxpython/nemo_relay/README.mdgo/nemo_relay/README.mddocs/supported-integrations/about.mdxdocs/nemo-relay-cli/hermes.mdxintegrations/coding-agents/cursor/README.mddocs/about-nemo-relay/ecosystem.mdxdocs/getting-started/quick-start/index.mdxdocs/reference/performance.mdxdocs/integrate-into-frameworks/non-serializable-data.mdxcrates/node/README.mdcrates/wasm/README.mddocs/about-nemo-relay/release-notes/highlights.mdxdocs/integrate-into-frameworks/about.mdxdocs/resources/support-and-faqs.mdxdocs/nemo-relay-cli/cursor.mdxdocs/about-nemo-relay/concepts/subscribers.mdxintegrations/openclaw/README.mdAGENTS.mdfern/docs.ymlREADME.mddocs/supported-integrations/openclaw-plugin.mdxdocs/about-nemo-relay/release-notes/known-issues.mdxdocs/build-plugins/register-behavior.mdxscripts/docs/sync_fern_docs_branch.pydocs/about-nemo-relay/release-notes/index.mdx
**/*.mdx
📄 CodeRabbit inference engine (.agents/skills/contribute-docs/SKILL.md)
In MDX files, top-of-file comments must use JSX comment delimiters: {/* to open and */} to close. Do not use HTML comments for MDX SPDX headers.
MDX top-of-file SPDX comments must use {/* ... */} delimiters instead of HTML comment delimiters (Must-Fix)
Files:
docs/build-plugins/validate-configuration.mdxdocs/nemo-relay-cli/about.mdxdocs/supported-integrations/about.mdxdocs/nemo-relay-cli/hermes.mdxdocs/about-nemo-relay/ecosystem.mdxdocs/getting-started/quick-start/index.mdxdocs/reference/performance.mdxdocs/integrate-into-frameworks/non-serializable-data.mdxdocs/about-nemo-relay/release-notes/highlights.mdxdocs/integrate-into-frameworks/about.mdxdocs/resources/support-and-faqs.mdxdocs/nemo-relay-cli/cursor.mdxdocs/about-nemo-relay/concepts/subscribers.mdxdocs/supported-integrations/openclaw-plugin.mdxdocs/about-nemo-relay/release-notes/known-issues.mdxdocs/build-plugins/register-behavior.mdxdocs/about-nemo-relay/release-notes/index.mdx
**/*.{html,md,mdx}
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Include SPDX license header in HTML and Markdown files using HTML comment syntax
Files:
docs/build-plugins/validate-configuration.mdxdocs/nemo-relay-cli/about.mdxpython/nemo_relay/README.mdgo/nemo_relay/README.mddocs/supported-integrations/about.mdxdocs/nemo-relay-cli/hermes.mdxintegrations/coding-agents/cursor/README.mddocs/about-nemo-relay/ecosystem.mdxdocs/getting-started/quick-start/index.mdxdocs/reference/performance.mdxdocs/integrate-into-frameworks/non-serializable-data.mdxcrates/node/README.mdcrates/wasm/README.mddocs/about-nemo-relay/release-notes/highlights.mdxdocs/integrate-into-frameworks/about.mdxdocs/resources/support-and-faqs.mdxdocs/nemo-relay-cli/cursor.mdxdocs/about-nemo-relay/concepts/subscribers.mdxintegrations/openclaw/README.mdAGENTS.mdREADME.mddocs/supported-integrations/openclaw-plugin.mdxdocs/about-nemo-relay/release-notes/known-issues.mdxdocs/build-plugins/register-behavior.mdxdocs/about-nemo-relay/release-notes/index.mdx
docs/**/*.{md,mdx}
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Update embedded documentation snippets, patch docs, and binding-support notes if examples or supported bindings changed
Files:
docs/build-plugins/validate-configuration.mdxdocs/nemo-relay-cli/about.mdxdocs/supported-integrations/about.mdxdocs/nemo-relay-cli/hermes.mdxdocs/about-nemo-relay/ecosystem.mdxdocs/getting-started/quick-start/index.mdxdocs/reference/performance.mdxdocs/integrate-into-frameworks/non-serializable-data.mdxdocs/about-nemo-relay/release-notes/highlights.mdxdocs/integrate-into-frameworks/about.mdxdocs/resources/support-and-faqs.mdxdocs/nemo-relay-cli/cursor.mdxdocs/about-nemo-relay/concepts/subscribers.mdxdocs/supported-integrations/openclaw-plugin.mdxdocs/about-nemo-relay/release-notes/known-issues.mdxdocs/build-plugins/register-behavior.mdxdocs/about-nemo-relay/release-notes/index.mdx
docs/**
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Run
just docsor./scripts/build-docs.sh htmlto regenerate ignored Fern API reference pages before validation for documentation site changes
Files:
docs/build-plugins/validate-configuration.mdxdocs/nemo-relay-cli/about.mdxdocs/supported-integrations/about.mdxdocs/nemo-relay-cli/hermes.mdxdocs/about-nemo-relay/ecosystem.mdxdocs/getting-started/quick-start/index.mdxdocs/reference/performance.mdxdocs/integrate-into-frameworks/non-serializable-data.mdxdocs/about-nemo-relay/release-notes/highlights.mdxdocs/integrate-into-frameworks/about.mdxdocs/resources/support-and-faqs.mdxdocs/nemo-relay-cli/cursor.mdxdocs/about-nemo-relay/concepts/subscribers.mdxdocs/supported-integrations/openclaw-plugin.mdxdocs/about-nemo-relay/release-notes/known-issues.mdxdocs/build-plugins/register-behavior.mdxdocs/about-nemo-relay/release-notes/index.mdx
{docs/**,README.md,CONTRIBUTING.md,RELEASING.md,SECURITY.md}
⚙️ CodeRabbit configuration file
{docs/**,README.md,CONTRIBUTING.md,RELEASING.md,SECURITY.md}: Review documentation for technical accuracy against the current API, command correctness, and consistency across language bindings.
Flag stale examples, missing SPDX headers where required, and instructions that no longer match CI or pre-commit behavior.
Files:
docs/build-plugins/validate-configuration.mdxdocs/nemo-relay-cli/about.mdxdocs/supported-integrations/about.mdxdocs/nemo-relay-cli/hermes.mdxdocs/about-nemo-relay/ecosystem.mdxdocs/getting-started/quick-start/index.mdxdocs/reference/performance.mdxdocs/integrate-into-frameworks/non-serializable-data.mdxdocs/about-nemo-relay/release-notes/highlights.mdxdocs/integrate-into-frameworks/about.mdxdocs/resources/support-and-faqs.mdxdocs/nemo-relay-cli/cursor.mdxdocs/about-nemo-relay/concepts/subscribers.mdxREADME.mddocs/supported-integrations/openclaw-plugin.mdxdocs/about-nemo-relay/release-notes/known-issues.mdxdocs/build-plugins/register-behavior.mdxdocs/about-nemo-relay/release-notes/index.mdx
**/*.{md,rst,html,txt}
📄 CodeRabbit inference engine (.agents/skills/review-doc-style/assets/nvidia-style-brand-terminology.md)
**/*.{md,rst,html,txt}: Always spellNVIDIAin all caps. Do not useNvidia,nvidia,nVidia,nVIDIA, orNV.
Usean NVIDIAbefore a noun because the name starts with an 'en' sound.
Do not add a registered trademark symbol afterNVIDIAwhen referring to the company.
Use trademark symbols with product names only when the document type or legal guidance requires them.
Verify official capitalization, spacing, and hyphenation for product names.
Precede NVIDIA product names withNVIDIAon first mention when it is natural and accurate.
Do not rewrite product names for grammar or title-case rules.
Preserve third-party product names according to the owner's spelling.
Include the company name and full model qualifier on first use when it helps identify the model.
Preserve the official capitalization and punctuation of model names.
Use shorter family names only after the full name is established.
Spell out a term on first use and put the acronym in parentheses unless the acronym is widely understood by the intended audience.
Use the acronym on later mentions after it has been defined.
For long documents, reintroduce the full term if readers might lose context.
Form plurals of acronyms withs, not an apostrophe, such asGPUs.
In headings, common acronyms can remain abbreviated. Spell out the term in the first or second sentence of the body.
Common terms such asCPU,GPU,PC,API, andUIusually do not need to be spelled out for developer audiences.
Files:
python/nemo_relay/README.mdgo/nemo_relay/README.mdintegrations/coding-agents/cursor/README.mdcrates/node/README.mdcrates/wasm/README.mdintegrations/openclaw/README.mdAGENTS.mdREADME.md
**/*.{md,rst,html}
📄 CodeRabbit inference engine (.agents/skills/review-doc-style/assets/nvidia-style-brand-terminology.md)
Link the first mention of a product name when the destination helps the reader.
Files:
python/nemo_relay/README.mdgo/nemo_relay/README.mdintegrations/coding-agents/cursor/README.mdcrates/node/README.mdcrates/wasm/README.mdintegrations/openclaw/README.mdAGENTS.mdREADME.md
**/*.md
📄 CodeRabbit inference engine (.agents/skills/contribute-integration/SKILL.md)
Documentation must be updated if activation or usage changed
**/*.md: Use title case consistently in technical documentation headings
Avoid quotation marks, ampersands, and exclamation marks in headings
Keep product, event, research, and whitepaper names in their official title case
Use title case for table headers
Do not force social-media sentence case into technical docs
Format code elements, commands, parameters, package names, and expressions in monospace
Format directories, file names, and paths in monospace using backticks
Use angle brackets inside monospace for variables inside paths, such as/home/<username>/.login
Format error messages and strings in quotation marks, keeping literal code strings in code formatting when clearer
Format UI buttons, menus, fields, and labels in bold
Use angle brackets between UI labels for menu paths, such as File > Save As
Use italics for new terms on first use, sparingly and only when introducing the term
Use italics for publication titles
Format keyboard shortcuts in plain text, such as Press Ctrl+Alt+Delete
Use owner/repo link text for GitHub repositories, preferring[NVIDIA/NeMo](link)over prose references like 'the GitHub repo'
Introduce every code block with a complete sentence
Do not make a code block complete the grammar of the previous sentence
Do not continue a sentence after a code block
Use syntax highlighting when the format supports it for code blocks
Avoid the word 'snippet' unless the surrounding docs already use it as a term of art
Keep inline method, function, and class references consistent with nearby docs, omitting empty parentheses for prose readability when no call is shown
Use descriptive anchor text that matches the destination title when possible for links
Avoid raw URLs in running text
Avoid generic anchor text such as 'here,' 'this page,' and 'read more'
Include acronyms in link text when a linked term includes an acronym
Do not link long sentences or multiple sentences
Avoid links ...
Files:
python/nemo_relay/README.mdgo/nemo_relay/README.mdintegrations/coding-agents/cursor/README.mdcrates/node/README.mdcrates/wasm/README.mdintegrations/openclaw/README.mdAGENTS.mdREADME.md
**/{docs,examples,**/*.md,*.patch,*.diff,.github,*.sh,*.yaml,*.yml}
📄 CodeRabbit inference engine (.agents/skills/rename-surfaces/SKILL.md)
Update documentation, examples, CI configuration, and patch artifacts when performing rename operations
Files:
python/nemo_relay/README.mdgo/nemo_relay/README.mdintegrations/coding-agents/cursor/README.mdcrates/node/README.mdcrates/wasm/README.mdintegrations/openclaw/README.mdAGENTS.mdfern/docs.ymlREADME.md
**/*.{md,rst,txt}
📄 CodeRabbit inference engine (.agents/skills/review-doc-style/assets/nvidia-style-guide.md)
Spell
NVIDIAin all caps. Do not useNvidia,nvidia, orNV.
Files:
python/nemo_relay/README.mdgo/nemo_relay/README.mdintegrations/coding-agents/cursor/README.mdcrates/node/README.mdcrates/wasm/README.mdintegrations/openclaw/README.mdAGENTS.mdREADME.md
**/*.{md,rst}
📄 CodeRabbit inference engine (.agents/skills/review-doc-style/assets/nvidia-style-guide.md)
**/*.{md,rst}: Format commands, code elements, expressions, package names, file names, and paths as inline code.
Use descriptive link text. Avoid raw URLs and weak anchors such as "here" or "read more."
Use title case consistently for technical documentation headings.
Introduce code blocks, lists, tables, and images with complete sentences.
Write procedures as imperative steps. Keep steps parallel and split long procedures into smaller tasks.
Prefer active voice, present tense, short sentences, contractions, and plain English.
Usecanfor possibility and reservemayfor permission.
Useafterfor temporal relationships instead ofonce.
Preferrefer tooverseewhen the wording points readers to another resource.
Avoid culture-specific idioms, unnecessary Latinisms, jokes, and marketing exaggeration in technical docs.
Spell out months in body text, avoid ordinal dates, and use clear time zones.
Spell out whole numbers from zero through nine unless they are technical values, parameters, versions, or UI values.
Use numerals for 10 or greater and include commas in thousands.
Do not add trademark symbols to learning-oriented docs unless the source, platform, or legal guidance explicitly requires them.
Files:
python/nemo_relay/README.mdgo/nemo_relay/README.mdintegrations/coding-agents/cursor/README.mdcrates/node/README.mdcrates/wasm/README.mdintegrations/openclaw/README.mdAGENTS.mdREADME.md
**/README.md
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Update relevant crate or package README when that surface changed
Files:
python/nemo_relay/README.mdgo/nemo_relay/README.mdintegrations/coding-agents/cursor/README.mdcrates/node/README.mdcrates/wasm/README.mdintegrations/openclaw/README.mdREADME.md
**/*.{rs,py,js,ts,tsx,jsx,go,toml,yaml,yml,md,sh,bash,json}
📄 CodeRabbit inference engine (AGENTS.md)
Keep SPDX headers on source, docs, scripts, and configuration files. The project is Apache-2.0.
Files:
python/nemo_relay/README.mdgo/nemo_relay/README.mdintegrations/coding-agents/cursor/README.mdcrates/node/README.mdcrates/wasm/README.mdintegrations/openclaw/README.mdAGENTS.mdcrates/cli/src/doctor.rsfern/docs.ymlcrates/cli/tests/coverage/doctor_tests.rsREADME.mdcrates/cli/src/adapters/hermes.rsintegrations/openclaw/src/hook-replay/llm.tscrates/cli/tests/coverage/adapters_tests.rscrates/cli/tests/coverage/gateway_tests.rscrates/node/src/promise_call.rscrates/node/src/api/mod.rscrates/core/tests/unit/observability/openinference_tests.rscrates/cli/tests/coverage/session_tests.rsintegrations/openclaw/test/llm-replay.test.tscrates/core/tests/unit/observability/atof_tests.rscrates/core/src/observability/openinference.rscrates/core/tests/unit/atif_tests.rscrates/cli/tests/coverage/launcher_tests.rscrates/cli/src/alignment/mod.rscrates/cli/tests/coverage/alignment_tests.rscrates/core/src/observability/atif.rscrates/cli/src/launcher.rsscripts/docs/sync_fern_docs_branch.pycrates/cli/src/session.rs
python/nemo_relay/**/*
⚙️ CodeRabbit configuration file
python/nemo_relay/**/*: Review Python wrapper changes for typed API consistency, contextvars-based scope isolation, async behavior, and parity with the native extension.
Stubs and runtime implementations should stay aligned.
Files:
python/nemo_relay/README.md
{crates/adaptive/**,python/nemo_relay/plugin.py,go/nemo_relay/**,**/node/**,**/wasm/**}
📄 CodeRabbit inference engine (.agents/skills/maintain-optimizer/SKILL.md)
{crates/adaptive/**,python/nemo_relay/plugin.py,go/nemo_relay/**,**/node/**,**/wasm/**}: Maintain consistent plugin lifecycle across all language bindings (Python, Go, Node/WebAssembly, and Rust)
Keep plugin context surfaces aligned across all language implementations
Files:
go/nemo_relay/README.mdcrates/node/README.mdcrates/wasm/README.mdcrates/node/tests/scope_tests.mjscrates/node/src/promise_call.rscrates/node/src/api/mod.rs
go/nemo_relay/**/*
⚙️ CodeRabbit configuration file
go/nemo_relay/**/*: Review Go binding changes for cgo memory ownership, race safety, callback cleanup, idiomatic exported APIs, and parity with Rust/FFI behavior.
Any API change should include focused Go tests and consider race-test behavior.
Files:
go/nemo_relay/README.md
docs/reference/**
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Update relevant reference documentation for any public API changes
Files:
docs/reference/performance.mdx
{crates/adaptive/**,python/nemo_relay/adaptive.py,python/nemo_relay/plugin.py,go/nemo_relay/adaptive/**,go/nemo_relay/!(adaptive)/**,**/node/**,**/wasm/**}
📄 CodeRabbit inference engine (.agents/skills/maintain-optimizer/SKILL.md)
Keep adaptive surface in sync across crates/adaptive, shared plugin behavior in core and bindings, Python adaptive/plugin wrappers in python/nemo_relay/adaptive.py and python/nemo_relay/plugin.py, Go adaptive helpers under go/nemo_relay/adaptive plus shared plugin helpers in go/nemo_relay, and Node/WebAssembly adaptive helpers and plugin wrappers
Files:
crates/node/README.mdcrates/wasm/README.mdcrates/node/tests/scope_tests.mjscrates/node/src/promise_call.rscrates/node/src/api/mod.rs
crates/{python,ffi,node,wasm}/**/*
⚙️ CodeRabbit configuration file
crates/{python,ffi,node,wasm}/**/*: Treat binding changes as public API changes. Check for parity with the other language bindings, FFI ownership/lifetime safety,
callback error propagation, stable type conversion, and consistent async/stream semantics.
Flag changes that update one binding without corresponding tests or documentation for the same surface elsewhere.
Files:
crates/node/README.mdcrates/wasm/README.mdcrates/node/tests/scope_tests.mjscrates/node/src/promise_call.rscrates/node/src/api/mod.rs
AGENTS.md
📄 CodeRabbit inference engine (CLAUDE.md)
Document agent implementations in AGENTS.md with clear descriptions of functionality and usage
Files:
AGENTS.md
{.github/**,.gitlab-ci.yml,.pre-commit-config.yaml,justfile,scripts/**}
⚙️ CodeRabbit configuration file
{.github/**,.gitlab-ci.yml,.pre-commit-config.yaml,justfile,scripts/**}: Review automation changes for reproducibility, pinned versions where appropriate, secret handling, and consistency with the documented validation matrix.
Pay attention to commands that need generated native artifacts, FFI libraries, or platform-specific environment variables.
Files:
.github/CODE_OF_CONDUCT.md.github/workflows/fern-docs.ymlscripts/docs/sync_fern_docs_branch.py
**/{crates/**/*.rs,python/**/*.py,crates/ffi/**/*.{h,c},go/**/*.go,crates/node/**/*.{js,ts,mjs}}
📄 CodeRabbit inference engine (AGENTS.md)
Follow binding naming conventions: Rust and Python
snake_case, C FFI exports prefixednemo_relay_, GoPascalCasefor public APIs, Node.jscamelCase.
Files:
crates/node/tests/scope_tests.mjscrates/cli/src/doctor.rscrates/cli/tests/coverage/doctor_tests.rscrates/cli/src/adapters/hermes.rscrates/cli/tests/coverage/adapters_tests.rscrates/cli/tests/coverage/gateway_tests.rscrates/node/src/promise_call.rscrates/node/src/api/mod.rscrates/core/tests/unit/observability/openinference_tests.rscrates/cli/tests/coverage/session_tests.rscrates/core/tests/unit/observability/atof_tests.rscrates/core/src/observability/openinference.rscrates/core/tests/unit/atif_tests.rscrates/cli/tests/coverage/launcher_tests.rscrates/cli/src/alignment/mod.rscrates/cli/tests/coverage/alignment_tests.rscrates/core/src/observability/atif.rscrates/cli/src/launcher.rscrates/cli/src/session.rs
{crates/**/tests/**,python/tests/**,go/nemo_relay/**/*_test.go}
⚙️ CodeRabbit configuration file
{crates/**/tests/**,python/tests/**,go/nemo_relay/**/*_test.go}: Tests should cover the behavior promised by the changed API surface, including error paths and cross-request isolation where relevant.
Prefer assertions on lifecycle events, scope stacks, middleware ordering, and binding parity over shallow smoke tests.
Files:
crates/node/tests/scope_tests.mjscrates/cli/tests/coverage/doctor_tests.rscrates/cli/tests/coverage/adapters_tests.rscrates/cli/tests/coverage/gateway_tests.rscrates/core/tests/unit/observability/openinference_tests.rscrates/cli/tests/coverage/session_tests.rscrates/core/tests/unit/observability/atof_tests.rscrates/core/tests/unit/atif_tests.rscrates/cli/tests/coverage/launcher_tests.rscrates/cli/tests/coverage/alignment_tests.rs
**/*.rs
📄 CodeRabbit inference engine (.agents/skills/add-binding-feature/SKILL.md)
Use
snake_casenaming convention for Rust identifiers (e.g.,nemo_relay_tool_call)
**/*.rs: Any Rust change must runjust test-rust
Any Rust change must runcargo fmt --all
Any Rust change must runcargo clippy --workspace --all-targets -- -D warnings
**/*.rs: Runcargo fmt --allfor all FFI work since it is Rust work
Runjust test-rustto validate FFI changes
Runcargo clippy --workspace --all-targets -- -D warningsto enforce strict linting on FFI workWhen Rust files changed as part of Go work, also run
cargo fmt --all,just test-rust, andcargo clippy --workspace --all-targets -- -D warnings
**/*.rs: Runcargo fmt --allwhen Rust files are changed as part of Node work
Runcargo clippy --workspace --all-targets -- -D warningswhen Rust files are changed as part of Node work
Runjust test-rustwhen Rust files are changed as part of Node work
**/*.rs: Runcargo fmt --allto format all Rust code
Runcargo clippy --workspace --all-targets -- -D warningsto enforce all clippy lints as errors
**/*.rs: Runcargo fmt --allwhen Rust files changed as part of WebAssembly work
Runcargo clippy --workspace --all-targets -- -D warningswhen Rust files changed as part of WebAssembly work
**/*.rs: If any Rust code changed, always runjust test-rust
If any Rust code changed, also runcargo fmt --all
If any Rust code changed, also runcargo clippy --workspace --all-targets -- -D warnings
Run Rust formatting withcargo fmt --all
Run Rust linting withcargo clippy --workspace --all-targets -- -D warnings
**/*.rs: Usecargo fmtfor Rust code formatting
Runcargo clippy -- -D warningsto lint Rust code and treat all warnings as errors
Use Rust snake_case naming convention for Rust identifiers
Include SPDX license header in all Rust source files using double-slash comment syntax
Validate Rust code withuv run pre-commit run --all-filesto enforce cargo fmt formatting check, cargo clippy lints, and cargo deny aud...
Files:
crates/cli/src/doctor.rscrates/cli/tests/coverage/doctor_tests.rscrates/cli/src/adapters/hermes.rscrates/cli/tests/coverage/adapters_tests.rscrates/cli/tests/coverage/gateway_tests.rscrates/node/src/promise_call.rscrates/node/src/api/mod.rscrates/core/tests/unit/observability/openinference_tests.rscrates/cli/tests/coverage/session_tests.rscrates/core/tests/unit/observability/atof_tests.rscrates/core/src/observability/openinference.rscrates/core/tests/unit/atif_tests.rscrates/cli/tests/coverage/launcher_tests.rscrates/cli/src/alignment/mod.rscrates/cli/tests/coverage/alignment_tests.rscrates/core/src/observability/atif.rscrates/cli/src/launcher.rscrates/cli/src/session.rs
**/{Cargo.toml,**/*.rs}
📄 CodeRabbit inference engine (.agents/skills/maintain-packaging/SKILL.md)
Maintain consistency between Rust package names in
Cargo.tomland their actual usage across the codebase
Files:
crates/cli/src/doctor.rscrates/cli/tests/coverage/doctor_tests.rscrates/cli/src/adapters/hermes.rscrates/cli/tests/coverage/adapters_tests.rscrates/cli/tests/coverage/gateway_tests.rscrates/node/src/promise_call.rscrates/node/src/api/mod.rscrates/core/tests/unit/observability/openinference_tests.rscrates/cli/tests/coverage/session_tests.rscrates/core/tests/unit/observability/atof_tests.rscrates/core/src/observability/openinference.rscrates/core/tests/unit/atif_tests.rscrates/cli/tests/coverage/launcher_tests.rscrates/cli/src/alignment/mod.rscrates/cli/tests/coverage/alignment_tests.rscrates/core/src/observability/atif.rscrates/cli/src/launcher.rscrates/cli/src/session.rs
**/*.{h,hpp,c,cpp,rs}
📄 CodeRabbit inference engine (.agents/skills/maintain-packaging/SKILL.md)
Ensure FFI header and library naming follows consistent conventions across platform-specific builds
Files:
crates/cli/src/doctor.rscrates/cli/tests/coverage/doctor_tests.rscrates/cli/src/adapters/hermes.rscrates/cli/tests/coverage/adapters_tests.rscrates/cli/tests/coverage/gateway_tests.rscrates/node/src/promise_call.rscrates/node/src/api/mod.rscrates/core/tests/unit/observability/openinference_tests.rscrates/cli/tests/coverage/session_tests.rscrates/core/tests/unit/observability/atof_tests.rscrates/core/src/observability/openinference.rscrates/core/tests/unit/atif_tests.rscrates/cli/tests/coverage/launcher_tests.rscrates/cli/src/alignment/mod.rscrates/cli/tests/coverage/alignment_tests.rscrates/core/src/observability/atif.rscrates/cli/src/launcher.rscrates/cli/src/session.rs
**/*.{rs,toml}
📄 CodeRabbit inference engine (.agents/skills/rename-surfaces/SKILL.md)
Update Rust crate names and module prefixes during coordinated rename operations
Files:
crates/cli/src/doctor.rscrates/cli/tests/coverage/doctor_tests.rscrates/cli/src/adapters/hermes.rscrates/cli/tests/coverage/adapters_tests.rscrates/cli/tests/coverage/gateway_tests.rscrates/node/src/promise_call.rscrates/node/src/api/mod.rscrates/core/tests/unit/observability/openinference_tests.rscrates/cli/tests/coverage/session_tests.rscrates/core/tests/unit/observability/atof_tests.rscrates/core/src/observability/openinference.rscrates/core/tests/unit/atif_tests.rscrates/cli/tests/coverage/launcher_tests.rscrates/cli/src/alignment/mod.rscrates/cli/tests/coverage/alignment_tests.rscrates/core/src/observability/atif.rscrates/cli/src/launcher.rscrates/cli/src/session.rs
crates/**/*.rs
📄 CodeRabbit inference engine (AGENTS.md)
Use
Json = serde_json::Valuein Rust-facing runtime APIs where the existing code expects JSON payloads.Use
Result<T>withFlowErrorin core runtime paths. Keep errors explicit and binding-appropriate at the wrapper layer.Keep async behavior on the existing tokio-based model. Bindings should preserve callback and future lifetimes rather than blocking or hiding async work unexpectedly.
Files:
crates/cli/src/doctor.rscrates/cli/tests/coverage/doctor_tests.rscrates/cli/src/adapters/hermes.rscrates/cli/tests/coverage/adapters_tests.rscrates/cli/tests/coverage/gateway_tests.rscrates/node/src/promise_call.rscrates/node/src/api/mod.rscrates/core/tests/unit/observability/openinference_tests.rscrates/cli/tests/coverage/session_tests.rscrates/core/tests/unit/observability/atof_tests.rscrates/core/src/observability/openinference.rscrates/core/tests/unit/atif_tests.rscrates/cli/tests/coverage/launcher_tests.rscrates/cli/src/alignment/mod.rscrates/cli/tests/coverage/alignment_tests.rscrates/core/src/observability/atif.rscrates/cli/src/launcher.rscrates/cli/src/session.rs
**/*.{py,txt,toml,cfg,yaml,yml}
📄 CodeRabbit inference engine (.agents/skills/rename-surfaces/SKILL.md)
Update Python package names and top-level module imports during coordinated rename operations
Files:
fern/docs.ymlscripts/docs/sync_fern_docs_branch.py
{README.md,docs/**/*.{md,rst,txt},fern/**/*}
📄 CodeRabbit inference engine (.agents/skills/prepare-code-freeze/SKILL.md)
Search and update documentation source for references to the old version in
README.md,docs, andferndirectories, updating current-version install commands, package examples, and configuration examples to<next-version>
Files:
fern/docs.ymlREADME.md
{crates/adaptive/**/*.rs,**/*test*.{rs,py,go,ts,js},**/*adaptive*test*.{rs,py,go,ts,js},docs/plugins/adaptive/**}
📄 CodeRabbit inference engine (.agents/skills/maintain-optimizer/SKILL.md)
Maintain documented and tested validation and report behavior for adaptive surfaces
Files:
crates/cli/tests/coverage/doctor_tests.rscrates/cli/tests/coverage/adapters_tests.rscrates/cli/tests/coverage/gateway_tests.rscrates/core/tests/unit/observability/openinference_tests.rscrates/cli/tests/coverage/session_tests.rsintegrations/openclaw/test/llm-replay.test.tscrates/core/tests/unit/observability/atof_tests.rscrates/core/tests/unit/atif_tests.rscrates/cli/tests/coverage/launcher_tests.rscrates/cli/tests/coverage/alignment_tests.rs
{README.md,docs/getting-started/**/*.md}
📄 CodeRabbit inference engine (.agents/skills/add-binding-feature/SKILL.md)
Update
README.md,docs/getting-started/, or binding-level READMEs if behavior differs by language or usage changed
Files:
README.md
README.md
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Update
README.mdto reflect current workspace members and top-level documentation when workspace structure changes
Files:
README.md
**/*.{wasm,js,ts}{,x}
📄 CodeRabbit inference engine (.agents/skills/maintain-packaging/SKILL.md)
Ensure WebAssembly package naming conventions are consistent with generated package expectations and downstream consumption
Files:
integrations/openclaw/src/hook-replay/llm.tsintegrations/openclaw/test/llm-replay.test.ts
**/*.{js,ts,jsx,tsx}
📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)
Run Node.js formatting with
npm run format --workspace=nemo-relay-nodeInclude SPDX license header in all JavaScript and TypeScript source files using double-slash comment syntax
Files:
integrations/openclaw/src/hook-replay/llm.tsintegrations/openclaw/test/llm-replay.test.ts
crates/node/src/**/*.rs
📄 CodeRabbit inference engine (.agents/skills/add-binding-feature/SKILL.md)
Use
camelCasenaming convention for Node.js identifiers (e.g.,toolCall)
Files:
crates/node/src/promise_call.rscrates/node/src/api/mod.rs
crates/node/src/api/**/*.rs
📄 CodeRabbit inference engine (.agents/skills/add-binding-feature/SKILL.md)
Update Node.js binding in
crates/node/src/api/mod.rsfor language-native bindings
Files:
crates/node/src/api/mod.rs
{crates/core,crates/adaptive}/**/*
📄 CodeRabbit inference engine (.agents/skills/prepare-pr/SKILL.md)
Changes to
crates/coreorcrates/adaptivemust run the full language matrix
Files:
crates/core/tests/unit/observability/openinference_tests.rscrates/core/tests/unit/observability/atof_tests.rscrates/core/src/observability/openinference.rscrates/core/tests/unit/atif_tests.rscrates/core/src/observability/atif.rs
crates/core/**/*.rs
📄 CodeRabbit inference engine (.agents/skills/test-go-binding/SKILL.md)
If the change touched
crates/coreor shared runtime semantics, also usevalidate-changefor broader validation
Files:
crates/core/tests/unit/observability/openinference_tests.rscrates/core/tests/unit/observability/atof_tests.rscrates/core/src/observability/openinference.rscrates/core/tests/unit/atif_tests.rscrates/core/src/observability/atif.rs
crates/{core,adaptive}/**
📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)
If
crates/coreorcrates/adaptivechanged, run the full matrix across Rust, Python, Go, Node.js, and WebAssembly
Files:
crates/core/tests/unit/observability/openinference_tests.rscrates/core/tests/unit/observability/atof_tests.rscrates/core/src/observability/openinference.rscrates/core/tests/unit/atif_tests.rscrates/core/src/observability/atif.rs
crates/{core,adaptive}/**/*.rs
⚙️ CodeRabbit configuration file
crates/{core,adaptive}/**/*.rs: Review the Rust runtime for async correctness, scope isolation, middleware ordering, and event lifecycle regressions.
Pay close attention to task-local/thread-local scope propagation, callback lifetimes, stream finalization, and root_uuid isolation.
Public API changes should preserve existing behavior unless tests and docs show the intended migration path.
Files:
crates/core/tests/unit/observability/openinference_tests.rscrates/core/tests/unit/observability/atof_tests.rscrates/core/src/observability/openinference.rscrates/core/tests/unit/atif_tests.rscrates/core/src/observability/atif.rs
**/*{test,spec,smoke}.{js,ts,py}
📄 CodeRabbit inference engine (.agents/skills/contribute-integration/SKILL.md)
Relevant integration tests or smoke path must pass
Files:
integrations/openclaw/test/llm-replay.test.ts
crates/core/src/observability/{atif,otel,openinference}.rs
📄 CodeRabbit inference engine (.agents/skills/maintain-observability/SKILL.md)
When changing event fields in ATIF, OpenTelemetry, or OpenInference observability surfaces, keep the core event model in
crates/core/src/observability/atif.rs,crates/core/src/observability/otel.rs, andcrates/core/src/observability/openinference.rsin sync
Files:
crates/core/src/observability/openinference.rscrates/core/src/observability/atif.rs
.github/workflows/*.{yml,yaml}
📄 CodeRabbit inference engine (.agents/skills/maintain-ci/SKILL.md)
.github/workflows/*.{yml,yaml}: Putpermissions:on each job that needs token access in GitHub Actions workflows
Avoid workflow-level permissions unless the repository intentionally centralizes them and the inheritance tradeoff is documented
Keep third-party actions pinned to full commit SHAs and preserve the readable version comment after the SHA
Prefer action-native or ecosystem-native caching over genericactions/cache
Use lockfiles or dependency manifests to drive cache invalidation in GitHub Actions workflows
Keep deploy and publish permissions isolated to the jobs that need them in GitHub Actions
Read both caller and callee when a workflow usesworkflow_callin GitHub Actions
Put release-tag validation in the earliest practical caller job when the pipeline has tag-based publish behavior
Keep release-tag policy aligned withRELEASING.md: raw SemVer tags only, no leadingv
contents: readis the default minimum permission for checkout-based build, test, docs, and packaging jobs
pull-requests: readis required for PR metadata lookup jobs in GitHub Actions workflows
pages: writeandid-token: writeshould be limited to Pages deployment jobs and callers that invoke them through reusable workflows
For reusable workflows, the caller must grant every permission the called jobs require; the callee cannot elevate beyond what the caller provides
Preferastral-sh/setup-uvcache support withcache-dependency-globanchored touv.lock
PreferSwatinem/rust-cachewith explicitshared-keyandworkspacesinstead of ad hoc target-directory caching
Avoid caching generated outputs that can hide stale behavior unless the repo already relies on them deliberately
Files:
.github/workflows/fern-docs.yml
.{github/workflows/*.{yml,yaml},gitlab-ci.yml}
📄 CodeRabbit inference engine (.agents/skills/maintain-packaging/SKILL.md)
Ensure CI workflows reference the same package names, install commands, and build commands as local development workflows
Files:
.github/workflows/fern-docs.yml
{pyproject.toml,**/*.py}
📄 CodeRabbit inference engine (.agents/skills/maintain-packaging/SKILL.md)
Maintain consistency between Python package names in
pyproject.tomland import paths used throughout the codebase
Files:
scripts/docs/sync_fern_docs_branch.py
{scripts/**,third-party/**}
📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)
{scripts/**,third-party/**}: For third-party integration or patch changes, run patch validation with./scripts/apply-patches.sh --checkand relevant integration tests. Keep root./scripts/*.shwrappers for third-party flows
Run third-party patch bootstrap with./scripts/bootstrap-third-party.sh
Run third-party patch validation with./scripts/apply-patches.sh --check
Files:
scripts/docs/sync_fern_docs_branch.py
**/*.py
📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)
**/*.py: Run Python formatting withuv run ruff format python
Run Python testing withuv run pytest -k "<pattern>"
**/*.py: Use Ruff with rule sets E, F, W, I for Python linting
Use Ruff formatter with line length 120 and double quotes for Python code formatting
Runtyfor Python type checking
Use Python snake_case naming convention for Python identifiers
Include SPDX license header in all Python source files using hash comment syntax
Validate Python code withuv run pre-commit run --all-filesto enforce Ruff linting and formatting, and ty type checking
Files:
scripts/docs/sync_fern_docs_branch.py
🧠 Learnings (3)
📓 Common learnings
Learnt from: CR
Repo: NVIDIA/NeMo-Relay
Timestamp: 2026-06-03T04:11:40.730Z
Learning: Do not hand-edit generated or packaged outputs unless the repository workflow expects them to be checked in. Regenerate through the documented recipe or script.
Learnt from: CR
Repo: NVIDIA/NeMo-Relay
Timestamp: 2026-06-03T04:11:40.730Z
Learning: Prefer the repository `just` recipes over raw tool commands. Use raw `cargo`, `pytest`, `go test`, `npm`, or `wasm-pack` commands only for focused debugging or targeted single-test reruns.
Learnt from: CR
Repo: NVIDIA/NeMo-Relay
Timestamp: 2026-06-03T04:11:40.730Z
Learning: Run tests for every language affected by a change. If you touch the Rust core runtime, middleware semantics, event shape, scope behavior, typed codecs, plugins, or observability, validate every affected binding.
Learnt from: CR
Repo: NVIDIA/NeMo-Relay
Timestamp: 2026-06-03T04:11:40.730Z
Learning: Before review, prefer `uv run pre-commit run --all-files` when the change crosses languages or tooling.
Learnt from: CR
Repo: NVIDIA/NeMo-Relay
Timestamp: 2026-06-03T04:11:40.730Z
Learning: Preserve the shared runtime model across bindings. Do not add behavior to one primary binding without considering Rust, Python, and Node.js parity.
Learnt from: CR
Repo: NVIDIA/NeMo-Relay
Timestamp: 2026-06-03T04:11:40.730Z
Learning: Prefer documented public APIs and stable wrapper commands. Do not rely on internal helpers in examples or user-facing docs.
Learnt from: CR
Repo: NVIDIA/NeMo-Relay
Timestamp: 2026-06-03T04:11:40.730Z
Learning: Keep primary documentation focused on Rust, Python, and Node.js. Treat Go, WebAssembly, and raw FFI as experimental and source-first unless binding-support guidance changes.
Learnt from: CR
Repo: NVIDIA/NeMo-Relay
Timestamp: 2026-06-03T04:11:40.730Z
Learning: Update `README.md`, `fern/`, package READMEs, and binding-support notes when public behavior, package names, examples, or supported bindings change.
Learnt from: CR
Repo: NVIDIA/NeMo-Relay
Timestamp: 2026-06-03T04:11:40.730Z
Learning: Keep release-process details in maintainer docs such as `RELEASING.md`. Do not move release-history policy into user-facing docs or `CHANGELOG.md`.
Learnt from: CR
Repo: NVIDIA/NeMo-Relay
Timestamp: 2026-06-03T04:11:40.730Z
Learning: Keep stable public wrappers at the `scripts/` root in docs and examples. Reference namespaced helper paths only when documenting internal maintenance work.
Learnt from: CR
Repo: NVIDIA/NeMo-Relay
Timestamp: 2026-06-03T04:11:40.730Z
Learning: Use branch prefixes from the contributor docs: `feat/`, `fix/`, `docs/`, `test/`, or `refactor/`.
Learnt from: CR
Repo: NVIDIA/NeMo-Relay
Timestamp: 2026-06-03T04:11:40.730Z
Learning: Use signed-off commits for PR work: `git commit -s`.
Learnt from: CR
Repo: NVIDIA/NeMo-Relay
Timestamp: 2026-06-03T04:11:40.730Z
Learning: Before creating, opening, publishing, or editing a pull request, read `.github/pull_request_template.md` and use it as the PR body skeleton. Preserve its visible headings, checklist items, and related-issue guidance.
Learnt from: CR
Repo: NVIDIA/NeMo-Relay
Timestamp: 2026-06-03T04:11:40.730Z
Learning: PR descriptions should include what changed, why, how it was tested, and any breaking changes within the repository template format.
Learnt from: CR
Repo: NVIDIA/NeMo-Relay
Timestamp: 2026-06-03T04:11:40.730Z
Learning: If repo-local PR guidance conflicts with generic GitHub connector or plugin guidance, follow the repo-local PR guidance for PR body format and review handoff details.
📚 Learning: 2026-05-07T18:04:44.387Z
Learnt from: mnajafian-nv
Repo: NVIDIA/NeMo-Flow PR: 67
File: integrations/openclaw/src/modules.ts:1-2
Timestamp: 2026-05-07T18:04:44.387Z
Learning: In NVIDIA/NeMo-Flow, TypeScript source files should use `//` line comments for SPDX headers (e.g., `// SPDX-FileCopyrightText: ...` and `// SPDX-License-Identifier: ...`) rather than C-style block comments (`/* ... */`). The repo’s copyright checker enforces this mapping, so `//` SPDX headers in `.ts` files should not be flagged as a style violation.
Applied to files:
integrations/openclaw/src/hook-replay/llm.tsintegrations/openclaw/test/llm-replay.test.ts
📚 Learning: 2026-05-03T04:23:07.497Z
Learnt from: willkill07
Repo: NVIDIA/NeMo-Flow PR: 46
File: .github/workflows/ci_rust.yml:31-64
Timestamp: 2026-05-03T04:23:07.497Z
Learning: In GitHub Actions workflow YAML, it’s valid to conditionally disable a service container by setting the service container’s `image` to an empty string (`''`) via a matrix variable (e.g., `redis_service_image: ''`). This intentionally makes the runner skip service initialization for that matrix entry rather than failing the job. When reviewing workflows, don’t flag this as an error if the workflow uses an empty `image` to disable the service on specific matrix entries (e.g., OS-specific setups); verify the `image` is sourced from the matrix variable and that the service is only expected to be available when a non-empty image is provided.
Applied to files:
.github/workflows/fern-docs.yml
🪛 LanguageTool
docs/about-nemo-relay/release-notes/highlights.mdx
[style] ~37-~37: Three successive sentences begin with the same word. Consider rewording the sentence or use a thesaurus to find a synonym.
Context: ... for guardrails plugin configuration. - Added guardrail scopes for conditional guardr...
(ENGLISH_WORD_REPEAT_BEGINNING_RULE)
[style] ~71-~71: Three successive sentences begin with the same word. Consider rewording the sentence or use a thesaurus to find a synonym.
Context: ...t-in guardrails plugin documentation. - Added CI path filters, CLI draft release asse...
(ENGLISH_WORD_REPEAT_BEGINNING_RULE)
🪛 markdownlint-cli2 (0.22.1)
.github/CODE_OF_CONDUCT.md
[warning] 6-6: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
🪛 Ruff (0.15.15)
scripts/docs/sync_fern_docs_branch.py
[warning] 203-203: Avoid specifying long messages outside the exception class
(TRY003)
🪛 zizmor (1.25.2)
.github/workflows/fern-docs.yml
[warning] 378-378: permissions without explanatory comments (undocumented-permissions): needs an explanatory comment
(undocumented-permissions)
[info] 451-451: code injection via template expansion (template-injection): may expand into attacker-controllable code
(template-injection)
[error] 439-439: runtime artifacts potentially vulnerable to a cache poisoning attack (cache-poisoning): enables caching by default
(cache-poisoning)
| fn remove_tool_handle_for_event(&mut self, event: &ToolEvent) -> Option<ToolHandle> { | ||
| if let Some(active) = self.tools.remove(&event.tool_call_id) { | ||
| return Some(active.handle); | ||
| } | ||
| let owner_subagent_id = self.tool_event_owner_subagent_id(event); | ||
| let key = self.matching_active_tool_key(event, owner_subagent_id.as_deref())?; | ||
| self.tools.remove(&key).map(|active| active.handle) | ||
| } | ||
|
|
||
| fn matching_active_tool_key( | ||
| &self, | ||
| event: &ToolEvent, | ||
| owner_subagent_id: Option<&str>, | ||
| ) -> Option<String> { | ||
| if event.arguments.is_null() { | ||
| return None; | ||
| } | ||
| let matches = self | ||
| .tools | ||
| .iter() | ||
| .filter_map(|(key, active)| { | ||
| (active.owner_subagent_id.as_deref() == owner_subagent_id | ||
| && active.name == event.tool_name | ||
| && active.arguments == event.arguments) | ||
| .then_some(key.clone()) | ||
| }) | ||
| .collect::<Vec<_>>(); | ||
| (matches.len() == 1).then(|| matches[0].clone()) | ||
| } |
There was a problem hiding this comment.
Avoid root-biased fallback when tool owner is unresolved.
When owner resolution returns None, the matcher currently filters to root-owned tools only. That can close the wrong root handle if the event owner is actually unknown and a subagent has the same (name, arguments) active. In the unresolved-owner case, match across all owners and require a single global match.
Proposed patch
fn matching_active_tool_key(
&self,
event: &ToolEvent,
owner_subagent_id: Option<&str>,
) -> Option<String> {
if event.arguments.is_null() {
return None;
}
let matches = self
.tools
.iter()
.filter_map(|(key, active)| {
- (active.owner_subagent_id.as_deref() == owner_subagent_id
+ let owner_matches = match owner_subagent_id {
+ Some(owner) => active.owner_subagent_id.as_deref() == Some(owner),
+ None => true,
+ };
+ (owner_matches
&& active.name == event.tool_name
&& active.arguments == event.arguments)
.then_some(key.clone())
})
.collect::<Vec<_>>();
(matches.len() == 1).then(|| matches[0].clone())
}📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| fn remove_tool_handle_for_event(&mut self, event: &ToolEvent) -> Option<ToolHandle> { | |
| if let Some(active) = self.tools.remove(&event.tool_call_id) { | |
| return Some(active.handle); | |
| } | |
| let owner_subagent_id = self.tool_event_owner_subagent_id(event); | |
| let key = self.matching_active_tool_key(event, owner_subagent_id.as_deref())?; | |
| self.tools.remove(&key).map(|active| active.handle) | |
| } | |
| fn matching_active_tool_key( | |
| &self, | |
| event: &ToolEvent, | |
| owner_subagent_id: Option<&str>, | |
| ) -> Option<String> { | |
| if event.arguments.is_null() { | |
| return None; | |
| } | |
| let matches = self | |
| .tools | |
| .iter() | |
| .filter_map(|(key, active)| { | |
| (active.owner_subagent_id.as_deref() == owner_subagent_id | |
| && active.name == event.tool_name | |
| && active.arguments == event.arguments) | |
| .then_some(key.clone()) | |
| }) | |
| .collect::<Vec<_>>(); | |
| (matches.len() == 1).then(|| matches[0].clone()) | |
| } | |
| fn remove_tool_handle_for_event(&mut self, event: &ToolEvent) -> Option<ToolHandle> { | |
| if let Some(active) = self.tools.remove(&event.tool_call_id) { | |
| return Some(active.handle); | |
| } | |
| let owner_subagent_id = self.tool_event_owner_subagent_id(event); | |
| let key = self.matching_active_tool_key(event, owner_subagent_id.as_deref())?; | |
| self.tools.remove(&key).map(|active| active.handle) | |
| } | |
| fn matching_active_tool_key( | |
| &self, | |
| event: &ToolEvent, | |
| owner_subagent_id: Option<&str>, | |
| ) -> Option<String> { | |
| if event.arguments.is_null() { | |
| return None; | |
| } | |
| let matches = self | |
| .tools | |
| .iter() | |
| .filter_map(|(key, active)| { | |
| let owner_matches = match owner_subagent_id { | |
| Some(owner) => active.owner_subagent_id.as_deref() == Some(owner), | |
| None => true, | |
| }; | |
| (owner_matches | |
| && active.name == event.tool_name | |
| && active.arguments == event.arguments) | |
| .then_some(key.clone()) | |
| }) | |
| .collect::<Vec<_>>(); | |
| (matches.len() == 1).then(|| matches[0].clone()) | |
| } |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@crates/cli/src/session.rs` around lines 1605 - 1633, The matcher currently
restricts to root-owned tools when owner resolution returns None, causing a
root-biased fallback; modify matching_active_tool_key so the owner check only
applies when owner_subagent_id is Some: in matching_active_tool_key (used by
remove_tool_handle_for_event) change the filter closure to require
active.owner_subagent_id == owner_subagent_id only when
owner_subagent_id.is_some(), otherwise ignore owner_subagent_id and match on
name and arguments across all owners, then keep the existing uniqueness check
(matches.len() == 1) to return a single global match.
| fn drops_uncorrelatable_hermes_pre_tool_call() { | ||
| let headers = HeaderMap::new(); | ||
| let outcome = hermes::adapt( | ||
| json!({ | ||
| "hook_event_name": "pre_tool_call", | ||
| "task_id": "task-1", | ||
| "tool_name": "terminal", | ||
| "tool_input": { "command": "pwd" } | ||
| }), |
There was a problem hiding this comment.
Isolate the missing-session correlatability check in this test.
This case currently omits both session_id and tool-call id, so it can pass even if only one gate is enforced. Keep a valid tool-call id and omit only session_id to target the intended behavior.
Proposed patch
fn drops_uncorrelatable_hermes_pre_tool_call() {
let headers = HeaderMap::new();
let outcome = hermes::adapt(
json!({
"hook_event_name": "pre_tool_call",
"task_id": "task-1",
+ "tool_call_id": "tool-1",
"tool_name": "terminal",
"tool_input": { "command": "pwd" }
}),
&headers,
);📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| fn drops_uncorrelatable_hermes_pre_tool_call() { | |
| let headers = HeaderMap::new(); | |
| let outcome = hermes::adapt( | |
| json!({ | |
| "hook_event_name": "pre_tool_call", | |
| "task_id": "task-1", | |
| "tool_name": "terminal", | |
| "tool_input": { "command": "pwd" } | |
| }), | |
| fn drops_uncorrelatable_hermes_pre_tool_call() { | |
| let headers = HeaderMap::new(); | |
| let outcome = hermes::adapt( | |
| json!({ | |
| "hook_event_name": "pre_tool_call", | |
| "task_id": "task-1", | |
| "tool_call_id": "tool-1", | |
| "tool_name": "terminal", | |
| "tool_input": { "command": "pwd" } | |
| }), |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@crates/cli/tests/coverage/adapters_tests.rs` around lines 312 - 320, The test
drops_uncorrelatable_hermes_pre_tool_call currently omits both session_id and
tool_call_id so it can pass when only one correlatability gate exists; change
the test input passed to hermes::adapt to include a valid tool_call_id (e.g.
"tool_call_id": "toolcall-1") while keeping session_id omitted so the test
isolates the missing-session correlatability check and still expects the adapter
to drop the event.
| const cacheRead = | ||
| numberField(usage, 'cacheRead') ?? | ||
| numberField(usage, 'cache_read_tokens') ?? | ||
| numberField(usage, 'cache_read_input_tokens') ?? | ||
| nestedNumberField(usage, 'input_tokens_details', 'cached_tokens') ?? | ||
| nestedNumberField(usage, 'prompt_tokens_details', 'cached_tokens'); |
There was a problem hiding this comment.
Preserve top-level cached_tokens during replay normalization.
This branch still drops usage objects that already use the canonical NeMo Relay field name cached_tokens (and the common camelCase variant). That silently loses cache-read metrics on replayed spans.
Suggested fix
const cacheRead =
numberField(usage, 'cacheRead') ??
+ numberField(usage, 'cached_tokens') ??
+ numberField(usage, 'cachedTokens') ??
numberField(usage, 'cache_read_tokens') ??
numberField(usage, 'cache_read_input_tokens') ??
nestedNumberField(usage, 'input_tokens_details', 'cached_tokens') ??
nestedNumberField(usage, 'prompt_tokens_details', 'cached_tokens');📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| const cacheRead = | |
| numberField(usage, 'cacheRead') ?? | |
| numberField(usage, 'cache_read_tokens') ?? | |
| numberField(usage, 'cache_read_input_tokens') ?? | |
| nestedNumberField(usage, 'input_tokens_details', 'cached_tokens') ?? | |
| nestedNumberField(usage, 'prompt_tokens_details', 'cached_tokens'); | |
| const cacheRead = | |
| numberField(usage, 'cacheRead') ?? | |
| numberField(usage, 'cached_tokens') ?? | |
| numberField(usage, 'cachedTokens') ?? | |
| numberField(usage, 'cache_read_tokens') ?? | |
| numberField(usage, 'cache_read_input_tokens') ?? | |
| nestedNumberField(usage, 'input_tokens_details', 'cached_tokens') ?? | |
| nestedNumberField(usage, 'prompt_tokens_details', 'cached_tokens'); |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@integrations/openclaw/src/hook-replay/llm.ts` around lines 1197 - 1202, The
cacheRead normalization currently misses top-level fields named cached_tokens
(and camelCase cachedTokens), causing cache-read metrics to be dropped; update
the cacheRead resolution in the llm replay normalization to check for
numberField(usage, 'cached_tokens') and numberField(usage, 'cachedTokens')
(prior to or alongside the other fallbacks) so that usage objects already using
the canonical NeMo Relay name or camelCase variant are preserved; locate the
cacheRead variable assignment and add those checks to the existing chain that
uses numberField(...) and nestedNumberField(...).
Overview
Adds Relay-side support needed for NVIDIA NeMo Agent Toolkit to consume Hermes Relay telemetry.
This change:
Why
Hermes invokes hooks from its own config and can execute through a different process boundary than the caller. For NeMo Agent Toolkit integration, Relay needs to capture Hermes lifecycle events reliably and make them available as replayable ATOF JSONL so the toolkit can import them into its telemetry stream.
Details
Where should the reviewer start?
Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)
Summary by CodeRabbit
New Features
Bug Fixes
Tests
Documentation