[DO NOT MERGE]feat: add streaming API POC - #104
Conversation
Proof-of-concept for a Fabric streaming API built on raw, Relay-generated
ATOF, surfaced as sugar over Runtime.invoke() (invoke_stream() -> async
iterator of raw ATOF records; RunResult out of band).
Each harness was run for real with Relay and its raw native events + ATOF
captured; the same harness-agnostic prototype was exercised across all four:
- hermes/ in-process; callback scopes; scope-level (no token deltas)
- deepagents/ in-process; nested/delegated sub-agents; scope tree via parent_uuid
- codex/ gateway; OpenAI Responses SSE; token-level deltas
- claude/ gateway; Anthropic Messages SSE; token-level deltas (fixture from a
genuine prior capture; live run credential-blocked)
Each child folder ships native-events.jsonl, events.atof.jsonl, and findings.md
(native->ATOF mapping, delta-vs-terminal, duplicate-rendering risks, per-harness
recommendation). synthesis/ records the cross-harness conclusion (v0.1 = raw
ATOF pass-through; normalization deferred) and the production work breakdown.
common/ holds the prototype: loopback ndjson listener, invoke_stream, runner.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com>
The relay gateway can retarget its upstream to NVIDIA inference (NEMO_RELAY_OPENAI_BASE_URL); the runner now forwards that override plus an optional FABRIC_MODEL via harness.settings.env. Routing verified (Codex's request reached NVIDIA), but NVIDIA's litellm-backed Responses proxy rejects Codex's proprietary namespace/web_search tool types (400) — so Codex still needs a funded OpenAI account. Findings updated in codex/findings.md. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com>
Previously native-events.jsonl was extracted from ATOF, which is not independent evidence. Add a POC-only recorder (common/native_recorder.py) that tees each harness's native SDK stream BEFORE Relay, run once with both recorders active, and base the loss analysis on the real native-vs-ATOF comparison. - Hermes: teed PluginManager.invoke_hook -> 12 real lifecycle hooks (pre/post_tool_call with args+IDs, transform_llm_output, ...). - Deep Agents: teed the astream (namespace, mode, chunk) tuples -> 26 real events incl. two delegated subagent namespaces and values/updates modes. - Instrumentation was applied at the documented seams, captured, then reverted; the adapters are unchanged. - Corrected (comparison-based) loss: ATOF is a faithful projection — it keeps the IDs, tool args, subagent namespaces, usage_metadata and tool_calls; it drops only transform_llm_output + middleware_trace (Hermes) and the values/updates mode + snapshots (Deep Agents). This strengthens the raw-ATOF-passthrough call. - Renamed derive_native_events.py -> extract_relay_payloads.py (ATOF inspection, not native evidence). Claude/Codex remain credential/billing-blocked, so their ATOF extracts are relabeled relay-payloads.jsonl with the native seam identified. Verified: native fixtures cannot be recreated from ATOF; order+ids pair native to ATOF; claimed fields appear in the fixtures. v0.1 API unchanged (raw ATOF). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com>
Reorganize so each folder holds exactly what its ticket requires: - Hermes (FABRIC-102) and Deep Agents (FABRIC-104): native-events.jsonl + events.atof.jsonl + findings.md — the raw native fixtures, the Relay-crossing prototype output, and the mapping/loss/deltas-vs-terminal/dup-risk/recommendation. - Codex and Claude (FABRIC-103): reduced to brief findings.md stubs noting they are pending a usable API key, with the native capture seam identified. - common/: the experimental prototype only (listener, invoke_stream, run_harness, native_recorder). Removed non-essential content: the ATOF-inspection helper (extract_relay_payloads.py) and the Codex/Claude ATOF extracts (relay-payloads.jsonl, events.atof.jsonl); the NVIDIA-upstream investigation cruft in run_harness (FABRIC_MODEL, gateway env forwarding). Synthesis marks the Codex/Claude rows preliminary/pending. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com>
Each findings.md now documents how to run the streaming experiment and produce its two fixtures: the exact run_harness command + prompt, the POC_NATIVE_RECORD / POC_RECORDER_DIR env, the temporary native-tee seam patch, and the revert. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com>
Replace internal tracker IDs with descriptive labels (harness mode / synthesis) so the POC docs are self-describing. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com>
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
WalkthroughAdds a Relay-backed streaming prototype that yields raw ATOF records, captures native and ATOF harness traces, provides a runnable harness CLI, and documents cross-harness findings plus a proposed production runtime contract. ChangesFabric streaming proof of concept
Estimated code review effort: 3 (Moderate) | ~30 minutes Suggested labels: Sequence Diagram(s)sequenceDiagram
participant Caller
participant StreamingRuntime
participant FabricRuntime
participant AtofStreamListener
Caller->>StreamingRuntime: invoke_stream(input)
StreamingRuntime->>FabricRuntime: invoke with loopback ATOF endpoint
FabricRuntime->>AtofStreamListener: emit NDJSON ATOF records
AtofStreamListener-->>Caller: yield parsed records
Caller->>StreamingRuntime: result()
StreamingRuntime-->>Caller: return RunResult
Important Pre-merge checks failedPlease resolve all errors before merging. Addressing warnings is optional. ❌ Failed checks (1 error, 1 warning)
✅ Passed checks (3 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
Add streaming-poc/implementation-spec.md — the end-to-end architecture of the relay-only streaming, centered on the loopback NDJSON listener, with a mermaid flow diagram (start_runtime endpoint injection → invoke_stream → live NDJSON push → yield events → out-of-band RunResult) and the in-process vs gateway Relay modes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com>
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 13
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@streaming-poc/claude/findings.md`:
- Line 1: Add the repository-standard HTML-comment SPDX copyright and Apache-2.0
license header before the existing headings in streaming-poc/claude/findings.md,
streaming-poc/codex/findings.md, streaming-poc/deepagents/findings.md,
streaming-poc/hermes/findings.md, streaming-poc/synthesis/README.md, and
streaming-poc/synthesis/work-breakdown.md.
- Line 1: Apply documentation style fixes across all listed sites: in
streaming-poc/claude/findings.md (line 1) and streaming-poc/codex/findings.md
(line 1), convert the titles to title case; in
streaming-poc/deepagents/findings.md (line 7), title-case headings and add blank
lines around headings, tables, and fenced blocks; in
streaming-poc/hermes/findings.md (line 6), title-case headings, add the missing
data-fence language, and correct reported spacing; in
streaming-poc/synthesis/README.md (line 11), title-case headings and add blank
lines around tables and fences; and in streaming-poc/synthesis/work-breakdown.md
(line 6), title-case section headings.
In `@streaming-poc/common/atof_stream.py`:
- Around line 25-32: Make the default maxsize in the ATOF stream __init__
positive so asyncio.Queue is bounded and applies backpressure, or require
callers to provide a bound; preserve the existing queue behavior for explicitly
supplied values. Update streaming-poc/common/README.md line 8 to document the
actual queue behavior and selected bound, verifying the claim against the
repository.
- Line 19: Add aiohttp to the dependency group or installation configuration
used by the streaming POC harness, ensuring run_harness.py can import it in a
clean environment. Keep the existing import unchanged.
In `@streaming-poc/common/fabric_stream.py`:
- Around line 155-158: Update the runtime startup flow around
fabric.start_runtime to catch startup failures, close the already-started
AtofStreamListener, and then re-raise the original exception. Preserve the
successful path returning StreamingRuntime(runtime, listener).
- Around line 131-132: Update invoke_stream to prevent concurrent active streams
from consuming the shared self._listener.records queue: either reject a second
invocation while one is active, or demultiplex records using the established
invocation identifier. Ensure each InvokeStream receives only its own ATOF
records and preserve normal behavior after the active stream completes.
- Around line 104-110: Update the iterator wait logic around the queue getter
created with asyncio.ensure_future(q.get()) so cancellation of the surrounding
await always cancels and awaits the pending getter in a finally block before
propagating cancellation. Preserve returning getter.result() when the getter
completes successfully, and avoid consuming queue records after iterator
cancellation.
In `@streaming-poc/common/native_recorder.py`:
- Around line 39-59: Update _redact so fallback SDK object handling never
persists unsanitized repr(value). Recursively redact structured data, including
values under sensitive keys, and omit opaque repr output unless it is explicitly
scrubbed; preserve the existing type marker and safe primitive/container
behavior.
In `@streaming-poc/common/README.md`:
- Line 1: Update streaming-poc/common/README.md at lines 1-1 by prepending the
required HTML-comment SPDX copyright and Apache-2.0 headers, then change the H1
to the title-case common shared-prototype title. Update streaming-poc/README.md
at lines 1-1 with the same SPDX headers and set its H1 to the full “NVIDIA NeMo
Fabric Streaming POC” product name.
In `@streaming-poc/deepagents/findings.md`:
- Line 93: Replace the destructive git checkout cleanup instructions in
streaming-poc/deepagents/findings.md:93-93 and
streaming-poc/hermes/findings.md:97-97 with a clean-worktree prerequisite or a
hunk-only restore command, ensuring unrelated uncommitted changes in
adapters/deepagents/src/nemo_fabric_adapters/deepagents/adapter.py are
preserved.
In `@streaming-poc/deepagents/native-events.jsonl`:
- Line 9: Scrub internal trace data from the listed fixtures before committing:
in streaming-poc/deepagents/native-events.jsonl lines 9-9 remove the raw system
prompt, tool catalog, and local paths; in
streaming-poc/deepagents/events.atof.jsonl lines 9-9 remove the raw annotated
request; in streaming-poc/hermes/events.atof.jsonl lines 9-9 remove reasoning,
reasoning_content, and unnecessary conversation data; and in
streaming-poc/hermes/native-events.jsonl lines 3-3 redact the full request
prompt, tool catalog, and local paths.
In `@streaming-poc/README.md`:
- Around line 9-19: Update the v0.1 contract example in the README to use the
runnable POC API, specifically start_streaming_runtime(fabric, config), matching
the harness behavior and wrapper it returns. Alternatively, explicitly label the
current Fabric.start_runtime example as a proposed production SDK contract
rather than a runnable POC API.
- Around line 9-31: Update the README documentation around the contract and
layout sections: add blank lines between headings, prose, and fenced blocks to
satisfy MD022/MD031; introduce each fenced block with a complete sentence; and
change the layout fence language identifier to text for MD040. Use consistent
title case for the affected headings while preserving the documented content.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Enterprise
Run ID: 4450179c-e1c8-41a2-93ff-afabcbb071d8
📒 Files selected for processing (17)
streaming-poc/.gitignorestreaming-poc/README.mdstreaming-poc/claude/findings.mdstreaming-poc/codex/findings.mdstreaming-poc/common/README.mdstreaming-poc/common/atof_stream.pystreaming-poc/common/fabric_stream.pystreaming-poc/common/native_recorder.pystreaming-poc/common/run_harness.pystreaming-poc/deepagents/events.atof.jsonlstreaming-poc/deepagents/findings.mdstreaming-poc/deepagents/native-events.jsonlstreaming-poc/hermes/events.atof.jsonlstreaming-poc/hermes/findings.mdstreaming-poc/hermes/native-events.jsonlstreaming-poc/synthesis/README.mdstreaming-poc/synthesis/work-breakdown.md
📜 Review details
⏰ Context from checks skipped due to timeout. (12)
- GitHub Check: Test (Python 3.13, linux-amd64)
- GitHub Check: Test (Python 3.13, macos-arm64)
- GitHub Check: Test (Python 3.14, linux-amd64)
- GitHub Check: Test (Python 3.12, linux-amd64)
- GitHub Check: Test (Python 3.13, linux-arm64)
- GitHub Check: Test (Python 3.12, macos-arm64)
- GitHub Check: Test (Python 3.11, linux-arm64)
- GitHub Check: Test (Python 3.11, macos-arm64)
- GitHub Check: Test (Python 3.12, linux-arm64)
- GitHub Check: Test (Python 3.14, macos-arm64)
- GitHub Check: Test (Python 3.14, linux-arm64)
- GitHub Check: Pre-commit
🧰 Additional context used
📓 Path-based instructions (15)
**/*.{md,rst}
📄 CodeRabbit inference engine (.agents/skills/contribute-api/SKILL.md)
Update documentation and examples in the same branch as the public API change.
Files:
streaming-poc/common/README.mdstreaming-poc/claude/findings.mdstreaming-poc/synthesis/README.mdstreaming-poc/codex/findings.mdstreaming-poc/README.mdstreaming-poc/deepagents/findings.mdstreaming-poc/hermes/findings.mdstreaming-poc/synthesis/work-breakdown.md
**/*
📄 CodeRabbit inference engine (.agents/skills/karpathy-guidelines/SKILL.md)
**/*: Before implementing, explicitly state assumptions, surface ambiguity and tradeoffs, present multiple interpretations when relevant, and ask for clarification rather than silently deciding or proceeding when requirements are unclear.
Prefer the minimum code needed to solve the requested problem: avoid speculative features, unnecessary abstractions, unrequested flexibility, and handling of impossible scenarios; simplify overcomplicated solutions.
When editing existing code, make surgical changes only: do not modify unrelated code, comments, formatting, or pre-existing dead code; match the existing style, and remove only unused imports, variables, or functions introduced by your changes.
Define verifiable success criteria for each task, such as writing regression tests for bugs and invalid-input tests for validation, then verify the implementation against those criteria. For multi-step work, state a brief plan with a verification check for each step.
**/*: Always spellNVIDIAin all caps; do not useNvidia,nvidia,nVidia,nVIDIA, orNV.
Usean NVIDIAbefore a noun, because the name begins with an “en” sound.
Do not add a registered trademark symbol afterNVIDIAwhen referring to the company; use trademark symbols with product names only when required by the document type or legal guidance.
Verify official capitalization, spacing, hyphenation, and spelling for NVIDIA and third-party product names; do not rewrite official product names for grammar or title-case rules.
Precede NVIDIA product names withNVIDIAon first mention when natural and accurate, and link the first mention when the destination helps the reader.
On first use, include the company name and full model qualifier when it helps identify the model; preserve official capitalization and punctuation, and use shorter family names only after establishing the full name.
For learning-oriented and developer content, do not force trademark symbols unless explicitly required; for press, ...
Files:
streaming-poc/common/README.mdstreaming-poc/claude/findings.mdstreaming-poc/synthesis/README.mdstreaming-poc/deepagents/native-events.jsonlstreaming-poc/hermes/native-events.jsonlstreaming-poc/codex/findings.mdstreaming-poc/README.mdstreaming-poc/deepagents/events.atof.jsonlstreaming-poc/hermes/events.atof.jsonlstreaming-poc/common/native_recorder.pystreaming-poc/deepagents/findings.mdstreaming-poc/hermes/findings.mdstreaming-poc/common/atof_stream.pystreaming-poc/synthesis/work-breakdown.mdstreaming-poc/common/run_harness.pystreaming-poc/common/fabric_stream.py
**/*.{md,mdx,rst}
📄 CodeRabbit inference engine (.agents/skills/review-doc-style/assets/nvidia-style-guide.md)
**/*.{md,mdx,rst}: For NeMo Fabric documentation, verify technical claims against the current repository, public API, or documented command before reviewing style.
Always spellNVIDIAin all caps; do not useNvidia,nvidia, orNV.
Format commands, code elements, expressions, package names, file names, and paths as inline code.
Use descriptive link text; avoid raw URLs and weak anchors such ashereorread more.
Use title case consistently for technical documentation headings.
Introduce code blocks, lists, tables, and images with complete sentences.
Write procedures as imperative, parallel steps; split long procedures into smaller tasks.
Prefer active voice, present tense, short sentences, contractions, and plain English while preserving necessary technical precision.
Usecanfor possibility and reservemayfor permission.
Useafterfor temporal relationships instead ofonce, and preferrefer tooverseewhen directing readers to another resource.
Avoid culture-specific idioms, unnecessary Latinisms, jokes, and marketing exaggeration in technical documentation.
Spell out months in body text, avoid ordinal dates, and use clear time zones.
Spell out whole numbers from zero through nine unless they are technical values, parameters, versions, or UI values; use numerals for 10 or greater and commas in thousands.
Do not add trademark symbols to learning-oriented documentation unless the source, platform, or legal guidance explicitly requires them.
Do not replace precise technical terms with simpler words when doing so would lose precision.
Do not flag passive voice when the actor is unknown or the action is the important part.
Do not rewrite API names, package names, command flags, or code literals for style.
**/*.{md,mdx,rst}: Use consistent title case for technical-document headings and table headers; avoid quotation marks, ampersands, and exclamation marks in headings, while preserving official product, event, research, and whitepaper title ...
Files:
streaming-poc/common/README.mdstreaming-poc/claude/findings.mdstreaming-poc/synthesis/README.mdstreaming-poc/codex/findings.mdstreaming-poc/README.mdstreaming-poc/deepagents/findings.mdstreaming-poc/hermes/findings.mdstreaming-poc/synthesis/work-breakdown.md
**/*.{md,rst,txt,adoc}
📄 CodeRabbit inference engine (.agents/skills/review-doc-style/assets/nvidia-style-language-mechanics.md)
**/*.{md,rst,txt,adoc}: For technical documentation, use professional, active, conversational, engaging, precise, and plain-English prose. Prefer active voice, present tense, short sentences, and scannable paragraphs. Avoid casual or imprecise language, swearing, threats, insults, jokes, puns, culture-specific idioms, marketing exaggeration, and unsupported third-party comparisons.
Usecanfor possibility and reservemayfor permission; useafterfor temporal order; userefer tofor cross-references; prefer short direct sentences and specific verbs; avoid unnecessarypleasein technical documentation.
Prefer active voice when the actor matters. Passive voice is acceptable when the actor is unknown or irrelevant, when the action or result is the focus, or in programmer documentation.
Use natural contractions in conversational technical prose, but do not force them in formal legal copy, API references, or generated text.
Prefer simpler English over Latinisms: usefor exampleorsuch asinstead ofe.g.,and so oninstead ofetc.,that isinstead ofi.e.,compared toinstead ofvs., andby,through, orusinginstead ofvia. Use industry-standard terms such as in silico, in vitro, and in vivo when appropriate, and italicize them in running text.
Usethatwithout commas for essential clauses, andwhichwith commas for nonessential clauses.
Format dates and times clearly: spell out months in body text; use forms such asJune 12, 2025; avoid numeric or ordinal dates; capitalize days; use 12-hour time when appropriate; include a space beforea.m.orp.m.; useETandPTfor needed time zones; avoid24/7; and preferfrom 12:30 to 1:00 p.m.for prose ranges.
Format numbers consistently: spell out zero through nine in body text, use numerals for 10 or greater and for technical values, use commas in thousands, do not begin a sentence with a numeral, spell out ordinals, and use numerals consistently within a category wh...
Files:
streaming-poc/common/README.mdstreaming-poc/claude/findings.mdstreaming-poc/synthesis/README.mdstreaming-poc/codex/findings.mdstreaming-poc/README.mdstreaming-poc/deepagents/findings.mdstreaming-poc/hermes/findings.mdstreaming-poc/synthesis/work-breakdown.md
**/README.md
📄 CodeRabbit inference engine (.agents/skills/review-doc-style/SKILL.md)
Update relevant package, crate, adapter, and integration README files when public behavior or entry-point documentation changes.
Files:
streaming-poc/common/README.mdstreaming-poc/synthesis/README.mdstreaming-poc/README.md
**/*.{md,mdx}
📄 CodeRabbit inference engine (.agents/skills/review-doc-style/SKILL.md)
**/*.{md,mdx}: Use the full product nameNVIDIA NeMo Fabricon its first usage, typically in the title or H1; useNeMo Fabricthereafter.
Usefabricby itself only when referring to the CLI tool, and surround those references with backticks.
CapitalizeNVIDIAcorrectly in public documentation.
Format commands, code elements, expressions, file names, paths, and filenames as inline code where needed.
Use title case consistently for headings in technical documentation.
Introduce code blocks, tables, and lists with complete lead-in sentences.
Use descriptive anchor text instead of raw URLs or generic link text such ashere.
Prefer active voice, present tense, short sentences, and plain English.
Use consistent terminology for the same concept throughout a document.
Write procedures as imperative, parallel, easy-to-scan steps, and split long sequences into smaller tasks.
Useafterinstead ofoncewhen expressing temporal sequence.
Usecaninstead ofmaywhen the intended meaning is possibility rather than permission.
Avoid ambiguous numeric dates and ordinal dates in body text.
For learning-oriented documentation, do not force trademark symbols unless the source document explicitly requires them.
Introduce examples' code blocks with full sentences and ensure examples match current APIs and build commands.For documentation-site changes, run
just docsto regenerate Python and Rust API references and validate Fern configuration.
Files:
streaming-poc/common/README.mdstreaming-poc/claude/findings.mdstreaming-poc/synthesis/README.mdstreaming-poc/codex/findings.mdstreaming-poc/README.mdstreaming-poc/deepagents/findings.mdstreaming-poc/hermes/findings.mdstreaming-poc/synthesis/work-breakdown.md
**/{README.md,*.md,*.mdx}
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Update relevant SDK, API, adapter, example, integration, and embedded documentation when public behavior or the corresponding surface changes.
Files:
streaming-poc/common/README.mdstreaming-poc/claude/findings.mdstreaming-poc/synthesis/README.mdstreaming-poc/codex/findings.mdstreaming-poc/README.mdstreaming-poc/deepagents/findings.mdstreaming-poc/hermes/findings.mdstreaming-poc/synthesis/work-breakdown.md
**/*.{html,md}
📄 CodeRabbit inference engine (CONTRIBUTING.md)
HTML and Markdown files must use the specified HTML comment form for SPDX license headers.
Files:
streaming-poc/common/README.mdstreaming-poc/claude/findings.mdstreaming-poc/synthesis/README.mdstreaming-poc/codex/findings.mdstreaming-poc/README.mdstreaming-poc/deepagents/findings.mdstreaming-poc/hermes/findings.mdstreaming-poc/synthesis/work-breakdown.md
**/*.{rs,py,pyi,json,yaml,yml}
📄 CodeRabbit inference engine (.agents/skills/contribute-api/SKILL.md)
Determine and update every affected public surface, including the CLI, PyO3 bindings, Python SDK, type stubs, schemas, and adapter contract, so they remain in parity.
Files:
streaming-poc/common/native_recorder.pystreaming-poc/common/atof_stream.pystreaming-poc/common/run_harness.pystreaming-poc/common/fabric_stream.py
**/*.{rs,py}
📄 CodeRabbit inference engine (.agents/skills/prepare-pr/SKILL.md)
For native binding changes, run
cargo check -p fabric-python --locked.
Files:
streaming-poc/common/native_recorder.pystreaming-poc/common/atof_stream.pystreaming-poc/common/run_harness.pystreaming-poc/common/fabric_stream.py
**/*.{py,pyi}
📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)
If Python code or a Python-facing adapter changes, run
just test-python.Use type annotations for public Python APIs and keep native Python binding declarations synchronized with their Rust implementations.
Files:
streaming-poc/common/native_recorder.pystreaming-poc/common/atof_stream.pystreaming-poc/common/run_harness.pystreaming-poc/common/fabric_stream.py
**/*.{rs,py,pyi}
📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)
**/*.{rs,py,pyi}: If public configuration types change, confirm schema snapshot tests injust test-rustpass and review generated schema diffs.
For schema or public contract changes, run both language suites and review changes underschemas/and generated API references.
**/*.{rs,py,pyi}: Usesnake_casefor Rust and Python functions and variables; usePascalCasefor Rust types and Python classes.
Keep native Python binding declarations synchronized with their Rust implementations when public contracts change.
Files:
streaming-poc/common/native_recorder.pystreaming-poc/common/atof_stream.pystreaming-poc/common/run_harness.pystreaming-poc/common/fabric_stream.py
**/*.{py,pyi,rs}
📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)
For Python SDK or PyO3 binding changes, use
python-tests, run focused pytest tests first, thenjust test-python; rebuild withjust build-pythonwhen native code or packaging changes.
Files:
streaming-poc/common/native_recorder.pystreaming-poc/common/atof_stream.pystreaming-poc/common/run_harness.pystreaming-poc/common/fabric_stream.py
**/*.{rs,py,toml}
📄 CodeRabbit inference engine (.agents/skills/update-project-version/SKILL.md)
When editing version helpers, verify every
nemo-fabric-*workspace package through Cargo metadata and reject a static version inpython/pyproject.toml.
Files:
streaming-poc/common/native_recorder.pystreaming-poc/common/atof_stream.pystreaming-poc/common/run_harness.pystreaming-poc/common/fabric_stream.py
**/*.{toml,rs,py}
📄 CodeRabbit inference engine (.agents/skills/update-project-version/SKILL.md)
Avoid blind repository-wide replacement of version-like strings; distinguish package-version references from examples and unrelated dependency versions.
Files:
streaming-poc/common/native_recorder.pystreaming-poc/common/atof_stream.pystreaming-poc/common/run_harness.pystreaming-poc/common/fabric_stream.py
🪛 ast-grep (0.44.1)
streaming-poc/common/native_recorder.py
[warning] 77-77: File path is request-/variable-derived; validate and normalize to prevent path traversal.
Context: open(path, "a", encoding="utf-8")
Note: [CWE-22] Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal').
(open-filename-from-request)
[info] 68-76: use jsonify instead of json.dumps for JSON output
Context: json.dumps(
{
"sequence": _seq,
"source": source,
"event_type": str(event_type),
"native": _redact(native),
},
ensure_ascii=False,
)
Note: [CWE-116] Improper Encoding or Escaping of Output.
(use-jsonify)
streaming-poc/common/atof_stream.py
[warning] 39-39: Do not make http calls without encryption
Context: f"http://{self._host}:{self._bound_port}/atof"
Note: [CWE-319] Cleartext Transmission of Sensitive Information.
(requests-http)
streaming-poc/common/run_harness.py
[info] 112-112: use jsonify instead of json.dumps for JSON output
Context: json.dumps(rec)
Note: [CWE-116] Improper Encoding or Escaping of Output.
(use-jsonify)
🪛 LanguageTool
streaming-poc/synthesis/work-breakdown.md
[grammar] ~25-~25: Ensure spelling is correct
Context: ...Bounded queue + backpressure** (default maxsize ~1024); document that a consumer stal...
(QB_NEW_EN_ORTHOGRAPHY_ERROR_IDS_1)
🪛 markdownlint-cli2 (0.23.0)
streaming-poc/synthesis/README.md
[warning] 11-11: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
[warning] 12-12: Tables should be surrounded by blank lines
(MD058, blanks-around-tables)
[warning] 23-23: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
[warning] 34-34: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
[warning] 41-41: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
[warning] 51-51: Fenced code blocks should be surrounded by blank lines
(MD031, blanks-around-fences)
[warning] 56-56: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
[warning] 74-74: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
streaming-poc/README.md
[warning] 9-9: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
[warning] 10-10: Fenced code blocks should be surrounded by blank lines
(MD031, blanks-around-fences)
[warning] 16-16: Fenced code blocks should be surrounded by blank lines
(MD031, blanks-around-fences)
[warning] 21-21: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
[warning] 22-22: Fenced code blocks should be surrounded by blank lines
(MD031, blanks-around-fences)
[warning] 22-22: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
[warning] 31-31: Fenced code blocks should be surrounded by blank lines
(MD031, blanks-around-fences)
[warning] 43-43: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
[warning] 51-51: Fenced code blocks should be surrounded by blank lines
(MD031, blanks-around-fences)
[warning] 57-57: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
streaming-poc/deepagents/findings.md
[warning] 7-7: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
[warning] 13-13: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
[warning] 21-21: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
[warning] 29-29: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
[warning] 36-36: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
[warning] 37-37: Tables should be surrounded by blank lines
(MD058, blanks-around-tables)
[warning] 46-46: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
[warning] 56-56: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
[warning] 63-63: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
[warning] 71-71: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
[warning] 83-83: Fenced code blocks should be surrounded by blank lines
(MD031, blanks-around-fences)
[warning] 89-89: Fenced code blocks should be surrounded by blank lines
(MD031, blanks-around-fences)
streaming-poc/hermes/findings.md
[warning] 6-6: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
[warning] 11-11: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
[warning] 19-19: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
[warning] 20-20: Fenced code blocks should be surrounded by blank lines
(MD031, blanks-around-fences)
[warning] 20-20: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
[warning] 25-25: Fenced code blocks should be surrounded by blank lines
(MD031, blanks-around-fences)
[warning] 31-31: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
[warning] 37-37: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
[warning] 38-38: Tables should be surrounded by blank lines
(MD058, blanks-around-tables)
[warning] 49-49: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
[warning] 61-61: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
[warning] 67-67: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
[warning] 75-75: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
[warning] 87-87: Fenced code blocks should be surrounded by blank lines
(MD031, blanks-around-fences)
[warning] 93-93: Fenced code blocks should be surrounded by blank lines
(MD031, blanks-around-fences)
streaming-poc/synthesis/work-breakdown.md
[warning] 6-6: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
[warning] 13-13: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
[warning] 21-21: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
[warning] 32-32: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
[warning] 37-37: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
[warning] 43-43: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
[warning] 47-47: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
[warning] 52-52: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
[warning] 61-61: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
[warning] 66-66: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
[warning] 70-70: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
[warning] 76-76: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
🪛 Ruff (0.15.21)
streaming-poc/common/native_recorder.py
[warning] 39-39: Too many return statements (7 > 6)
(PLR0911)
[warning] 39-39: Dynamically typed expressions (typing.Any) are disallowed in value
(ANN401)
[warning] 39-39: Dynamically typed expressions (typing.Any) are disallowed in _redact
(ANN401)
[warning] 54-54: Do not catch blind exception: Exception
(BLE001)
[warning] 62-62: Dynamically typed expressions (typing.Any) are disallowed in native
(ANN401)
[warning] 64-64: Using the global statement to update _seq is discouraged
(PLW0603)
streaming-poc/common/atof_stream.py
[warning] 39-39: Avoid specifying long messages outside the exception class
(TRY003)
[warning] 43-43: Remove quotes from type annotation
Remove quotes
(UP037)
[warning] 46-46: Remove quotes from type annotation
Remove quotes
(UP037)
streaming-poc/common/run_harness.py
[warning] 110-110: Async functions should not open files with blocking methods like open
(ASYNC230)
streaming-poc/common/fabric_stream.py
[warning] 55-55: Dynamically typed expressions (typing.Any) are disallowed in value
(ANN401)
[warning] 55-55: Dynamically typed expressions (typing.Any) are disallowed in _coerce
(ANN401)
[warning] 69-71: Consider iterable unpacking instead of concatenation
Replace with iterable unpacking
(RUF005)
[warning] 85-85: Missing return type annotation for special method __init__
Add return type annotation: None
(ANN204)
[warning] 86-86: Dynamically typed expressions (typing.Any) are disallowed in runtime
(ANN401)
[error] 86-86: Function argument input is shadowing a Python builtin
(A002)
[warning] 86-86: Dynamically typed expressions (typing.Any) are disallowed in input
(ANN401)
[warning] 86-86: Dynamically typed expressions (typing.Any) are disallowed in request
(ANN401)
[warning] 91-91: Remove quotes from type annotation
Remove quotes
(UP037)
[warning] 102-102: Replace aliased errors with TimeoutError
Replace asyncio.TimeoutError with builtin TimeoutError
(UP041)
[warning] 103-103: Within an except clause, raise exceptions with raise ... from err or raise ... from None to distinguish them from errors in exception handling
(B904)
[warning] 112-112: Dynamically typed expressions (typing.Any) are disallowed in result
(ANN401)
[warning] 118-121: Use contextlib.suppress(BaseException) instead of try-except-pass
Replace try-except-pass with with contextlib.suppress(BaseException): ...
(SIM105)
[error] 120-121: try-except-pass detected, consider logging the exception
(S110)
[warning] 120-120: Do not catch blind exception: BaseException
(BLE001)
[warning] 127-127: Missing return type annotation for special method __init__
Add return type annotation: None
(ANN204)
[warning] 127-127: Dynamically typed expressions (typing.Any) are disallowed in runtime
(ANN401)
[error] 131-131: Function argument input is shadowing a Python builtin
(A002)
[warning] 131-131: Dynamically typed expressions (typing.Any) are disallowed in input
(ANN401)
[warning] 131-131: Dynamically typed expressions (typing.Any) are disallowed in request
(ANN401)
[warning] 140-140: Remove quotes from type annotation
Remove quotes
(UP037)
[warning] 143-143: Dynamically typed expressions (typing.Any) are disallowed in *exc
(ANN401)
[warning] 148-148: Dynamically typed expressions (typing.Any) are disallowed in fabric
(ANN401)
[warning] 152-154: Avoid specifying long messages outside the exception class
(TRY003)
🔇 Additional comments (5)
streaming-poc/synthesis/work-breakdown.md (1)
22-30: 🚀 Performance & ScalabilityVerify the listener limits before making them production contract.
The 512 KB readline limit, approximately 3-second Relay timeout, and
aiohttpdependency status are concrete runtime claims, but the supplied implementation context does not establish them. Verify these values against the actual listener and Relay versions, or label them as provisional.streaming-poc/synthesis/README.md (1)
45-50: 🎯 Functional Correctness
Fabric.start_runtime(...)is the documented entry point here; this section is already framed as a v0.1 recommendation.> Likely an incorrect or invalid review comment.streaming-poc/common/fabric_stream.py (1)
147-158: 📐 Maintainability & Code QualityRun the required Python and documentation validation.
The PR summary reports that repository suites were not run. Run
just test-pythonandjust docs, and record any blocked validation.As per coding guidelines, “If Python code or a Python-facing adapter changes, run
just test-python,” and documentation changes must runjust docswhen practical.Source: Coding guidelines
streaming-poc/.gitignore (1)
1-5: LGTM!streaming-poc/common/run_harness.py (1)
1-136: LGTM!
| import json | ||
| from typing import Any | ||
|
|
||
| from aiohttp import web |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
Declare aiohttp in the POC install path. run_harness.py imports this module directly, but aiohttp is not in the declared dependency groups, so a clean environment will fail on import.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@streaming-poc/common/atof_stream.py` at line 19, Add aiohttp to the
dependency group or installation configuration used by the streaming POC
harness, ensuring run_harness.py can import it in a clean environment. Keep the
existing import unchanged.
| {"sequence": 6, "source": "langgraph", "event_type": "updates", "native": {"namespace": [], "chunk": {"TodoListMiddleware.after_model": null}}} | ||
| {"sequence": 7, "source": "langgraph", "event_type": "values", "native": {"namespace": ["tools:e55b5b5f-078f-28c8-d4dc-639e3a13fbf8"], "chunk": {"messages": [{"content": "You are a subagent tasked with computing the product of 15 and 23. Provide only the numeric result.", "additional_kwargs": {}, "response_metadata": {}, "type": "human", "name": null, "id": "20ebb2a1-9749-4f09-8994-65dabe1527fb"}], "files": {}}}} | ||
| {"sequence": 8, "source": "langgraph", "event_type": "updates", "native": {"namespace": ["tools:e55b5b5f-078f-28c8-d4dc-639e3a13fbf8"], "chunk": {"PatchToolCallsMiddleware.before_agent": null}}} | ||
| {"sequence": 9, "source": "langgraph", "event_type": "updates", "native": {"namespace": ["tools:e55b5b5f-078f-28c8-d4dc-639e3a13fbf8"], "chunk": {"model": {"messages": [{"content": "345", "additional_kwargs": {"refusal": null}, "response_metadata": {"token_usage": {"completion_tokens": 45, "prompt_tokens": 3610, "total_tokens": 3655, "completion_tokens_details": null, "prompt_tokens_details": null}, "model_provider": "openai", "model_name": "nvidia/nemotron-3-nano-30b-a3b", "system_fingerprint": null, "id": "chatcmpl-944d6cc7-a621-4820-b53c-1443d4f3257e", "finish_reason": "stop", "logprobs": null}, "type": "ai", "name": "general-purpose", "id": "lc_run--019f8bba-9b87-7dc1-92e2-89cdbf42b6ab-0", "tool_calls": [], "invalid_tool_calls": [], "usage_metadata": {"input_tokens": 3610, "output_tokens": 45, "total_tokens": 3655, "input_token_details": {}, "output_token_details": {}}}]}}}} |
There was a problem hiding this comment.
🔒 Security & Privacy | 🟠 Major | 🏗️ Heavy lift
Scrub internal trace content before committing these fixtures.
The committed traces expose internal system prompts, tool schemas, local filesystem paths, or private reasoning because the current recorder does not sanitize arbitrary string values.
streaming-poc/deepagents/native-events.jsonl#L9-L9: remove the raw system prompt, tool catalog, and local paths.streaming-poc/deepagents/events.atof.jsonl#L9-L9: remove the raw annotated request before committing the ATOF fixture.streaming-poc/hermes/events.atof.jsonl#L9-L9: removereasoning,reasoning_content, and unnecessary conversation data.streaming-poc/hermes/native-events.jsonl#L3-L3: redact the full request prompt, tool catalog, and local paths.
📍 Affects 4 files
streaming-poc/deepagents/native-events.jsonl#L9-L9(this comment)streaming-poc/deepagents/events.atof.jsonl#L9-L9streaming-poc/hermes/events.atof.jsonl#L9-L9streaming-poc/hermes/native-events.jsonl#L3-L3
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@streaming-poc/deepagents/native-events.jsonl` at line 9, Scrub internal trace
data from the listed fixtures before committing: in
streaming-poc/deepagents/native-events.jsonl lines 9-9 remove the raw system
prompt, tool catalog, and local paths; in
streaming-poc/deepagents/events.atof.jsonl lines 9-9 remove the raw annotated
request; in streaming-poc/hermes/events.atof.jsonl lines 9-9 remove reasoning,
reasoning_content, and unnecessary conversation data; and in
streaming-poc/hermes/native-events.jsonl lines 3-3 redact the full request
prompt, tool catalog, and local paths.
yczhang-nv
changed the title
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
streaming-poc/README.md (2)
38-42: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick winApply the required documentation formatting.
Use title case for table headers (
Harness,Mode,Status), formatnemo-relayand0.6.0as inline code, and add a blank line after## Fixture note.As per coding guidelines, technical-document headings and table headers use title case, and package names and versions use inline code.
Also applies to: 56-60
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@streaming-poc/README.md` around lines 38 - 42, Update the README documentation formatting: change the table headers to title case (`Harness`, `Mode`, `Status`), format every `nemo-relay` and `0.6.0` reference as inline code, and insert a blank line immediately after the `## Fixture note` heading. Apply these changes consistently in both referenced sections.Source: Coding guidelines
45-48: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick winMake the native-extension prerequisite platform-specific.
lib_native.dylibis macOS-only, but this prerequisite reads as generic. Scope it to macOS or add the Linux and Windows artifact names and copy commands too.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@streaming-poc/README.md` around lines 45 - 48, Update the prerequisites section to make the native-extension setup platform-specific: identify the macOS-only lib_native.dylib artifact and either scope its build/copy commands to macOS or provide equivalent Linux and Windows artifact names and copy commands. Keep the NVIDIA_API_KEY prerequisite unchanged.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@streaming-poc/implementation-spec.md`:
- Line 1: Add the repository-standard generated SPDX copyright and Apache-2.0
license header to implementation-spec.md, using Markdown-compatible comment
syntax, then rerun the pre-commit hook to verify and commit the generated
header.
---
Outside diff comments:
In `@streaming-poc/README.md`:
- Around line 38-42: Update the README documentation formatting: change the
table headers to title case (`Harness`, `Mode`, `Status`), format every
`nemo-relay` and `0.6.0` reference as inline code, and insert a blank line
immediately after the `## Fixture note` heading. Apply these changes
consistently in both referenced sections.
- Around line 45-48: Update the prerequisites section to make the
native-extension setup platform-specific: identify the macOS-only
lib_native.dylib artifact and either scope its build/copy commands to macOS or
provide equivalent Linux and Windows artifact names and copy commands. Keep the
NVIDIA_API_KEY prerequisite unchanged.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Enterprise
Run ID: 2d63d7ff-21e8-4578-9552-4fd27b8bd302
📒 Files selected for processing (2)
streaming-poc/README.mdstreaming-poc/implementation-spec.md
📜 Review details
⚠️ CI failures not shown inline (3)
GitHub Actions: Check / Pre-commit: feat: add streaming API POC (relay-only raw ATOF via invoke_stream)
Conclusion: failure
##[group]Run bail() {
�[36;1mbail() {�[0m
�[36;1m printf '::error::install-action: %s\n' "$*"�[0m
GitHub Actions: Check / 0_Pre-commit.txt: feat: add streaming API POC (relay-only raw ATOF via invoke_stream)
Conclusion: failure
##[group]Run bail() {
�[36;1mbail() {�[0m
�[36;1m printf '::error::install-action: %s\n' "$*"�[0m
GitHub Actions: Check / Pre-commit: feat: add streaming API POC (relay-only raw ATOF via invoke_stream)
Conclusion: failure
##[group]Run uv run --no-sync pre-commit run --all-files --show-diff-on-failure
�[36;1muv run --no-sync pre-commit run --all-files --show-diff-on-failure�[0m
shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
env:
UV_PYTHON_INSTALL_DIR: /home/runner/work/_temp/uv-python-dir
UV_CACHE_DIR: /home/runner/work/_temp/setup-uv-cache
GOTOOLCHAIN: local
CARGO_INCREMENTAL: 0
CARGO_PROFILE_DEV_DEBUG: 0
CARGO_TERM_COLOR: always
RUST_BACKTRACE: short
RUSTFLAGS: -D warnings
CARGO_UNSTABLE_SPARSE_REGISTRY: true
CARGO_REGISTRIES_CRATES_IO_PROTOCOL: sparse
##[endgroup]
copyright header.........................................................Failed
- hook id: copyright-header
- exit code: 1
- files were modified by this hook
Fixed copyright headers in:
streaming-poc/claude/findings.md
streaming-poc/codex/findings.md
Fixed copyright headers in:
streaming-poc/deepagents/findings.md
streaming-poc/.gitignore
Fixed copyright headers in:
streaming-poc/synthesis/README.md
streaming-poc/implementation-spec.md
Fixed copyright headers in:
streaming-poc/synthesis/work-breakdown.md
streaming-poc/hermes/findings.md
streaming-poc/common/README.md
streaming-poc/README.md
ruff check...............................................................Passed
Lint GitHub Actions workflow files.......................................Passed
Cargo.lock is up to date.................................................Passed
uv lockfiles are up to date..............................................Passed
ATTRIBUTIONS-Rust.md (Cargo.lock)........................................Passed
ATTRIBUTIONS-Python.md (uv.lock).........................................Passed
Dependency license diff..................................................Passed
pre-commit hook(s) made changes.
If you are seeing this message in CI, reproduce locally with: `pre-commit run --all-files`.
To run `pre-commit` as part of git workflow, use `pre-commit inst...
🧰 Additional context used
📓 Path-based instructions (8)
**/*.{md,rst}
📄 CodeRabbit inference engine (.agents/skills/contribute-api/SKILL.md)
Update documentation and examples in the same branch as the public API change.
Files:
streaming-poc/implementation-spec.mdstreaming-poc/README.md
**/*
📄 CodeRabbit inference engine (.agents/skills/karpathy-guidelines/SKILL.md)
**/*: Before implementing, explicitly state assumptions, surface ambiguity and tradeoffs, present multiple interpretations when relevant, and ask for clarification rather than silently deciding or proceeding when requirements are unclear.
Prefer the minimum code needed to solve the requested problem: avoid speculative features, unnecessary abstractions, unrequested flexibility, and handling of impossible scenarios; simplify overcomplicated solutions.
When editing existing code, make surgical changes only: do not modify unrelated code, comments, formatting, or pre-existing dead code; match the existing style, and remove only unused imports, variables, or functions introduced by your changes.
Define verifiable success criteria for each task, such as writing regression tests for bugs and invalid-input tests for validation, then verify the implementation against those criteria. For multi-step work, state a brief plan with a verification check for each step.
**/*: Always spellNVIDIAin all caps; do not useNvidia,nvidia,nVidia,nVIDIA, orNV.
Usean NVIDIAbefore a noun, because the name begins with an “en” sound.
Do not add a registered trademark symbol afterNVIDIAwhen referring to the company; use trademark symbols with product names only when required by the document type or legal guidance.
Verify official capitalization, spacing, hyphenation, and spelling for NVIDIA and third-party product names; do not rewrite official product names for grammar or title-case rules.
Precede NVIDIA product names withNVIDIAon first mention when natural and accurate, and link the first mention when the destination helps the reader.
On first use, include the company name and full model qualifier when it helps identify the model; preserve official capitalization and punctuation, and use shorter family names only after establishing the full name.
For learning-oriented and developer content, do not force trademark symbols unless explicitly required; for press, ...
Files:
streaming-poc/implementation-spec.mdstreaming-poc/README.md
**/*.{md,mdx,rst}
📄 CodeRabbit inference engine (.agents/skills/review-doc-style/assets/nvidia-style-guide.md)
**/*.{md,mdx,rst}: For NeMo Fabric documentation, verify technical claims against the current repository, public API, or documented command before reviewing style.
Always spellNVIDIAin all caps; do not useNvidia,nvidia, orNV.
Format commands, code elements, expressions, package names, file names, and paths as inline code.
Use descriptive link text; avoid raw URLs and weak anchors such ashereorread more.
Use title case consistently for technical documentation headings.
Introduce code blocks, lists, tables, and images with complete sentences.
Write procedures as imperative, parallel steps; split long procedures into smaller tasks.
Prefer active voice, present tense, short sentences, contractions, and plain English while preserving necessary technical precision.
Usecanfor possibility and reservemayfor permission.
Useafterfor temporal relationships instead ofonce, and preferrefer tooverseewhen directing readers to another resource.
Avoid culture-specific idioms, unnecessary Latinisms, jokes, and marketing exaggeration in technical documentation.
Spell out months in body text, avoid ordinal dates, and use clear time zones.
Spell out whole numbers from zero through nine unless they are technical values, parameters, versions, or UI values; use numerals for 10 or greater and commas in thousands.
Do not add trademark symbols to learning-oriented documentation unless the source, platform, or legal guidance explicitly requires them.
Do not replace precise technical terms with simpler words when doing so would lose precision.
Do not flag passive voice when the actor is unknown or the action is the important part.
Do not rewrite API names, package names, command flags, or code literals for style.
**/*.{md,mdx,rst}: Use consistent title case for technical-document headings and table headers; avoid quotation marks, ampersands, and exclamation marks in headings, while preserving official product, event, research, and whitepaper title ...
Files:
streaming-poc/implementation-spec.mdstreaming-poc/README.md
**/*.{md,rst,txt,adoc}
📄 CodeRabbit inference engine (.agents/skills/review-doc-style/assets/nvidia-style-language-mechanics.md)
**/*.{md,rst,txt,adoc}: For technical documentation, use professional, active, conversational, engaging, precise, and plain-English prose. Prefer active voice, present tense, short sentences, and scannable paragraphs. Avoid casual or imprecise language, swearing, threats, insults, jokes, puns, culture-specific idioms, marketing exaggeration, and unsupported third-party comparisons.
Usecanfor possibility and reservemayfor permission; useafterfor temporal order; userefer tofor cross-references; prefer short direct sentences and specific verbs; avoid unnecessarypleasein technical documentation.
Prefer active voice when the actor matters. Passive voice is acceptable when the actor is unknown or irrelevant, when the action or result is the focus, or in programmer documentation.
Use natural contractions in conversational technical prose, but do not force them in formal legal copy, API references, or generated text.
Prefer simpler English over Latinisms: usefor exampleorsuch asinstead ofe.g.,and so oninstead ofetc.,that isinstead ofi.e.,compared toinstead ofvs., andby,through, orusinginstead ofvia. Use industry-standard terms such as in silico, in vitro, and in vivo when appropriate, and italicize them in running text.
Usethatwithout commas for essential clauses, andwhichwith commas for nonessential clauses.
Format dates and times clearly: spell out months in body text; use forms such asJune 12, 2025; avoid numeric or ordinal dates; capitalize days; use 12-hour time when appropriate; include a space beforea.m.orp.m.; useETandPTfor needed time zones; avoid24/7; and preferfrom 12:30 to 1:00 p.m.for prose ranges.
Format numbers consistently: spell out zero through nine in body text, use numerals for 10 or greater and for technical values, use commas in thousands, do not begin a sentence with a numeral, spell out ordinals, and use numerals consistently within a category wh...
Files:
streaming-poc/implementation-spec.mdstreaming-poc/README.md
**/*.{md,mdx}
📄 CodeRabbit inference engine (.agents/skills/review-doc-style/SKILL.md)
**/*.{md,mdx}: Use the full product nameNVIDIA NeMo Fabricon its first usage, typically in the title or H1; useNeMo Fabricthereafter.
Usefabricby itself only when referring to the CLI tool, and surround those references with backticks.
CapitalizeNVIDIAcorrectly in public documentation.
Format commands, code elements, expressions, file names, paths, and filenames as inline code where needed.
Use title case consistently for headings in technical documentation.
Introduce code blocks, tables, and lists with complete lead-in sentences.
Use descriptive anchor text instead of raw URLs or generic link text such ashere.
Prefer active voice, present tense, short sentences, and plain English.
Use consistent terminology for the same concept throughout a document.
Write procedures as imperative, parallel, easy-to-scan steps, and split long sequences into smaller tasks.
Useafterinstead ofoncewhen expressing temporal sequence.
Usecaninstead ofmaywhen the intended meaning is possibility rather than permission.
Avoid ambiguous numeric dates and ordinal dates in body text.
For learning-oriented documentation, do not force trademark symbols unless the source document explicitly requires them.
Introduce examples' code blocks with full sentences and ensure examples match current APIs and build commands.For documentation-site changes, run
just docsto regenerate Python and Rust API references and validate Fern configuration.
Files:
streaming-poc/implementation-spec.mdstreaming-poc/README.md
**/{README.md,*.md,*.mdx}
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Update relevant SDK, API, adapter, example, integration, and embedded documentation when public behavior or the corresponding surface changes.
Files:
streaming-poc/implementation-spec.mdstreaming-poc/README.md
**/*.{html,md}
📄 CodeRabbit inference engine (CONTRIBUTING.md)
HTML and Markdown files must use the specified HTML comment form for SPDX license headers.
Files:
streaming-poc/implementation-spec.mdstreaming-poc/README.md
**/README.md
📄 CodeRabbit inference engine (.agents/skills/review-doc-style/SKILL.md)
Update relevant package, crate, adapter, and integration README files when public behavior or entry-point documentation changes.
Files:
streaming-poc/README.md
🪛 GitHub Actions: Check / Pre-commit
streaming-poc/implementation-spec.md
[error] 1-1: File was modified by pre-commit hook 'copyright-header' to add SPDX copyright/license headers.
streaming-poc/README.md
[error] 1-1: File was modified by pre-commit hook 'copyright-header' to add SPDX copyright/license headers.
🔇 Additional comments (3)
streaming-poc/README.md (3)
9-19: Duplicate: the contract example still uses the non-runnable API.The POC harness uses
start_streaming_runtime(fabric, config)instreaming-poc/common/run_harness.pyLines 105–106, notfabric.start_runtime(config). Update the example or label it as the proposed production SDK contract.
9-31: Duplicate: Markdown fence and heading spacing remains unresolved.Add blank lines around headings and fenced blocks, introduce each code block with a complete sentence, and specify
textfor the layout block.Also applies to: 43-55
1-7: LGTM!Also applies to: 17-19, 33-36
The two delegated subagents in the captured run execute sequentially (21:28:10.350-10.974 then 21:28:12.244-17.348; 1.3s gap, no scope overlap), so the "concurrent subagents interleave" statement was inference, not evidence. Reframe it as a design property of the astream namespace and flag genuine parallel evidence (overlapping sibling scopes + interleaved namespaces) as an open item still required. Attempts to force two parallel task calls were made: the fast model serializes the delegations and a 70B model exceeded the run budget before two subagents completed. Re-add a FABRIC_MODEL override to run_harness and document the parallel-attempt recipe. Nesting and delegation remain genuinely captured. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com>
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@streaming-poc/deepagents/findings.md`:
- Around line 109-118: Add blank lines immediately before and after the fenced
shell block in the findings content, preserving the existing bash language tag
and surrounding explanatory sentence.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Enterprise
Run ID: 078de252-ff7a-44a8-9711-b6522ad7683c
📒 Files selected for processing (3)
streaming-poc/common/run_harness.pystreaming-poc/deepagents/findings.mdstreaming-poc/synthesis/README.md
📜 Review details
⏰ Context from checks skipped due to timeout. (17)
- GitHub Check: Test (Python 3.14, windows-amd64)
- GitHub Check: Test (Python 3.12, windows-amd64)
- GitHub Check: Test (Python 3.13, windows-amd64)
- GitHub Check: Test (Python 3.11, windows-amd64)
- GitHub Check: Test (Python 3.14, macos-arm64)
- GitHub Check: Test (Python 3.12, linux-amd64)
- GitHub Check: Test (Python 3.13, linux-arm64)
- GitHub Check: Test (Python 3.11, linux-amd64)
- GitHub Check: Test (Python 3.14, linux-arm64)
- GitHub Check: Test (Python 3.14, linux-amd64)
- GitHub Check: Test (Python 3.12, macos-arm64)
- GitHub Check: Test (Python 3.11, macos-arm64)
- GitHub Check: Test (Python 3.11, linux-arm64)
- GitHub Check: Test (Python 3.13, linux-amd64)
- GitHub Check: Test (Python 3.13, macos-arm64)
- GitHub Check: Test (Python 3.12, linux-arm64)
- GitHub Check: Pre-commit
🧰 Additional context used
📓 Path-based instructions (15)
**/*.{md,rst}
📄 CodeRabbit inference engine (.agents/skills/contribute-api/SKILL.md)
Update documentation and examples in the same branch as the public API change.
Files:
streaming-poc/synthesis/README.mdstreaming-poc/deepagents/findings.md
**/*
📄 CodeRabbit inference engine (.agents/skills/karpathy-guidelines/SKILL.md)
**/*: Before implementing, explicitly state assumptions, surface ambiguity and tradeoffs, present multiple interpretations when relevant, and ask for clarification rather than silently deciding or proceeding when requirements are unclear.
Prefer the minimum code needed to solve the requested problem: avoid speculative features, unnecessary abstractions, unrequested flexibility, and handling of impossible scenarios; simplify overcomplicated solutions.
When editing existing code, make surgical changes only: do not modify unrelated code, comments, formatting, or pre-existing dead code; match the existing style, and remove only unused imports, variables, or functions introduced by your changes.
Define verifiable success criteria for each task, such as writing regression tests for bugs and invalid-input tests for validation, then verify the implementation against those criteria. For multi-step work, state a brief plan with a verification check for each step.
**/*: Always spellNVIDIAin all caps; do not useNvidia,nvidia,nVidia,nVIDIA, orNV.
Usean NVIDIAbefore a noun, because the name begins with an “en” sound.
Do not add a registered trademark symbol afterNVIDIAwhen referring to the company; use trademark symbols with product names only when required by the document type or legal guidance.
Verify official capitalization, spacing, hyphenation, and spelling for NVIDIA and third-party product names; do not rewrite official product names for grammar or title-case rules.
Precede NVIDIA product names withNVIDIAon first mention when natural and accurate, and link the first mention when the destination helps the reader.
On first use, include the company name and full model qualifier when it helps identify the model; preserve official capitalization and punctuation, and use shorter family names only after establishing the full name.
For learning-oriented and developer content, do not force trademark symbols unless explicitly required; for press, ...
Files:
streaming-poc/synthesis/README.mdstreaming-poc/common/run_harness.pystreaming-poc/deepagents/findings.md
**/*.{md,mdx,rst}
📄 CodeRabbit inference engine (.agents/skills/review-doc-style/assets/nvidia-style-guide.md)
**/*.{md,mdx,rst}: For NeMo Fabric documentation, verify technical claims against the current repository, public API, or documented command before reviewing style.
Always spellNVIDIAin all caps; do not useNvidia,nvidia, orNV.
Format commands, code elements, expressions, package names, file names, and paths as inline code.
Use descriptive link text; avoid raw URLs and weak anchors such ashereorread more.
Use title case consistently for technical documentation headings.
Introduce code blocks, lists, tables, and images with complete sentences.
Write procedures as imperative, parallel steps; split long procedures into smaller tasks.
Prefer active voice, present tense, short sentences, contractions, and plain English while preserving necessary technical precision.
Usecanfor possibility and reservemayfor permission.
Useafterfor temporal relationships instead ofonce, and preferrefer tooverseewhen directing readers to another resource.
Avoid culture-specific idioms, unnecessary Latinisms, jokes, and marketing exaggeration in technical documentation.
Spell out months in body text, avoid ordinal dates, and use clear time zones.
Spell out whole numbers from zero through nine unless they are technical values, parameters, versions, or UI values; use numerals for 10 or greater and commas in thousands.
Do not add trademark symbols to learning-oriented documentation unless the source, platform, or legal guidance explicitly requires them.
Do not replace precise technical terms with simpler words when doing so would lose precision.
Do not flag passive voice when the actor is unknown or the action is the important part.
Do not rewrite API names, package names, command flags, or code literals for style.
**/*.{md,mdx,rst}: Use consistent title case for technical-document headings and table headers; avoid quotation marks, ampersands, and exclamation marks in headings, while preserving official product, event, research, and whitepaper title ...
Files:
streaming-poc/synthesis/README.mdstreaming-poc/deepagents/findings.md
**/*.{md,rst,txt,adoc}
📄 CodeRabbit inference engine (.agents/skills/review-doc-style/assets/nvidia-style-language-mechanics.md)
**/*.{md,rst,txt,adoc}: For technical documentation, use professional, active, conversational, engaging, precise, and plain-English prose. Prefer active voice, present tense, short sentences, and scannable paragraphs. Avoid casual or imprecise language, swearing, threats, insults, jokes, puns, culture-specific idioms, marketing exaggeration, and unsupported third-party comparisons.
Usecanfor possibility and reservemayfor permission; useafterfor temporal order; userefer tofor cross-references; prefer short direct sentences and specific verbs; avoid unnecessarypleasein technical documentation.
Prefer active voice when the actor matters. Passive voice is acceptable when the actor is unknown or irrelevant, when the action or result is the focus, or in programmer documentation.
Use natural contractions in conversational technical prose, but do not force them in formal legal copy, API references, or generated text.
Prefer simpler English over Latinisms: usefor exampleorsuch asinstead ofe.g.,and so oninstead ofetc.,that isinstead ofi.e.,compared toinstead ofvs., andby,through, orusinginstead ofvia. Use industry-standard terms such as in silico, in vitro, and in vivo when appropriate, and italicize them in running text.
Usethatwithout commas for essential clauses, andwhichwith commas for nonessential clauses.
Format dates and times clearly: spell out months in body text; use forms such asJune 12, 2025; avoid numeric or ordinal dates; capitalize days; use 12-hour time when appropriate; include a space beforea.m.orp.m.; useETandPTfor needed time zones; avoid24/7; and preferfrom 12:30 to 1:00 p.m.for prose ranges.
Format numbers consistently: spell out zero through nine in body text, use numerals for 10 or greater and for technical values, use commas in thousands, do not begin a sentence with a numeral, spell out ordinals, and use numerals consistently within a category wh...
Files:
streaming-poc/synthesis/README.mdstreaming-poc/deepagents/findings.md
**/README.md
📄 CodeRabbit inference engine (.agents/skills/review-doc-style/SKILL.md)
Update relevant package, crate, adapter, and integration README files when public behavior or entry-point documentation changes.
Files:
streaming-poc/synthesis/README.md
**/*.{md,mdx}
📄 CodeRabbit inference engine (.agents/skills/review-doc-style/SKILL.md)
**/*.{md,mdx}: Use the full product nameNVIDIA NeMo Fabricon its first usage, typically in the title or H1; useNeMo Fabricthereafter.
Usefabricby itself only when referring to the CLI tool, and surround those references with backticks.
CapitalizeNVIDIAcorrectly in public documentation.
Format commands, code elements, expressions, file names, paths, and filenames as inline code where needed.
Use title case consistently for headings in technical documentation.
Introduce code blocks, tables, and lists with complete lead-in sentences.
Use descriptive anchor text instead of raw URLs or generic link text such ashere.
Prefer active voice, present tense, short sentences, and plain English.
Use consistent terminology for the same concept throughout a document.
Write procedures as imperative, parallel, easy-to-scan steps, and split long sequences into smaller tasks.
Useafterinstead ofoncewhen expressing temporal sequence.
Usecaninstead ofmaywhen the intended meaning is possibility rather than permission.
Avoid ambiguous numeric dates and ordinal dates in body text.
For learning-oriented documentation, do not force trademark symbols unless the source document explicitly requires them.
Introduce examples' code blocks with full sentences and ensure examples match current APIs and build commands.For documentation-site changes, run
just docsto regenerate Python and Rust API references and validate Fern configuration.
Files:
streaming-poc/synthesis/README.mdstreaming-poc/deepagents/findings.md
**/{README.md,*.md,*.mdx}
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Update relevant SDK, API, adapter, example, integration, and embedded documentation when public behavior or the corresponding surface changes.
Files:
streaming-poc/synthesis/README.mdstreaming-poc/deepagents/findings.md
**/*.{html,md}
📄 CodeRabbit inference engine (CONTRIBUTING.md)
HTML and Markdown files must use the specified HTML comment form for SPDX license headers.
Files:
streaming-poc/synthesis/README.mdstreaming-poc/deepagents/findings.md
**/*.{rs,py,pyi,json,yaml,yml}
📄 CodeRabbit inference engine (.agents/skills/contribute-api/SKILL.md)
Determine and update every affected public surface, including the CLI, PyO3 bindings, Python SDK, type stubs, schemas, and adapter contract, so they remain in parity.
Files:
streaming-poc/common/run_harness.py
**/*.{rs,py}
📄 CodeRabbit inference engine (.agents/skills/prepare-pr/SKILL.md)
For native binding changes, run
cargo check -p fabric-python --locked.
Files:
streaming-poc/common/run_harness.py
**/*.{py,pyi}
📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)
If Python code or a Python-facing adapter changes, run
just test-python.Use type annotations for public Python APIs and keep native Python binding declarations synchronized with their Rust implementations.
Files:
streaming-poc/common/run_harness.py
**/*.{rs,py,pyi}
📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)
**/*.{rs,py,pyi}: If public configuration types change, confirm schema snapshot tests injust test-rustpass and review generated schema diffs.
For schema or public contract changes, run both language suites and review changes underschemas/and generated API references.
**/*.{rs,py,pyi}: Usesnake_casefor Rust and Python functions and variables; usePascalCasefor Rust types and Python classes.
Keep native Python binding declarations synchronized with their Rust implementations when public contracts change.
Files:
streaming-poc/common/run_harness.py
**/*.{py,pyi,rs}
📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)
For Python SDK or PyO3 binding changes, use
python-tests, run focused pytest tests first, thenjust test-python; rebuild withjust build-pythonwhen native code or packaging changes.
Files:
streaming-poc/common/run_harness.py
**/*.{rs,py,toml}
📄 CodeRabbit inference engine (.agents/skills/update-project-version/SKILL.md)
When editing version helpers, verify every
nemo-fabric-*workspace package through Cargo metadata and reject a static version inpython/pyproject.toml.
Files:
streaming-poc/common/run_harness.py
**/*.{toml,rs,py}
📄 CodeRabbit inference engine (.agents/skills/update-project-version/SKILL.md)
Avoid blind repository-wide replacement of version-like strings; distinguish package-version references from examples and unrelated dependency versions.
Files:
streaming-poc/common/run_harness.py
🪛 markdownlint-cli2 (0.23.0)
streaming-poc/deepagents/findings.md
[warning] 36-36: Fenced code blocks should be surrounded by blank lines
(MD031, blanks-around-fences)
[warning] 36-36: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
[warning] 39-39: Fenced code blocks should be surrounded by blank lines
(MD031, blanks-around-fences)
[warning] 112-112: Fenced code blocks should be surrounded by blank lines
(MD031, blanks-around-fences)
[warning] 116-116: Fenced code blocks should be surrounded by blank lines
(MD031, blanks-around-fences)
🔇 Additional comments (3)
streaming-poc/common/run_harness.py (1)
1-50: LGTM!Also applies to: 52-65, 67-101, 103-137
streaming-poc/deepagents/findings.md (1)
30-35: LGTM!Also applies to: 72-74
streaming-poc/synthesis/README.md (1)
19-19: LGTM!Also applies to: 30-31
…ion/SSO Un-stub the Claude child POC with a real, gateway-captured ATOF fixture. The Relay gateway forwarded a signed-in Claude Code subscription (OAuth) session, so the run needed no ANTHROPIC_API_KEY. Adds streaming-poc/claude/events.atof.jsonl (18 records, 13 token-level llm.chunk marks) and a full findings.md documenting the native Anthropic SSE sequence embedded in the ATOF, the native->ATOF mapping, and the key fidelity result: llm.chunk marks carry event type/index/timing/usage live, but the per-delta token text is terminal-only in Relay's current projection (the "degraded granularity" v0.1 contract in practice). run_harness.py sets permission_mode=bypassPermissions for Claude and documents the SSO path (ANTHROPIC_CONFIG_DIR, key unset). Codex stays a stub, but its blocker is refined: SSO auth works; the installed Codex CLI is too old for the account model and the API-key path is out of quota. README/synthesis updated to match. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com>
There was a problem hiding this comment.
Actionable comments posted: 3
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (4)
streaming-poc/common/run_harness.py (3)
1-50: 🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick winIsolate each harness invocation’s working directory.
WORKis keyed only byADAPTER, so repeated or concurrent runs sharehome,workspace,artifacts, andrelay. This can mix state, overwrite artifacts, and interleave ATOF captures. Include a unique run identifier or use a temporary directory per invocation.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@streaming-poc/common/run_harness.py` around lines 1 - 50, Update the harness invocation setup that derives WORK from ADAPTER so each run uses a unique temporary directory or run-specific identifier. Ensure the resulting home, workspace, artifacts, and relay paths are isolated for repeated and concurrent invocations, while preserving the existing adapter-specific behavior.
108-142: 🔒 Security & Privacy | 🟠 Major | ⚡ Quick winMake raw ATOF capture explicitly opt-in or redact it.
The runner writes every Relay record to disk, including prompts, paths, session metadata, and potentially full request/response payloads. Add redaction before persistence or require an explicit capture flag and document the privacy implications.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@streaming-poc/common/run_harness.py` around lines 108 - 142, Update main and the raw ATOF persistence flow around OUT.open and json.dumps so writing Relay records is disabled by default unless an explicit capture flag is provided, or redact sensitive prompts, paths, session metadata, and request/response payloads before writing. Document the privacy implications and ensure console progress and result handling remain unchanged.
108-142: 🔒 Security & Privacy | 🟠 Major | ⚡ Quick winMake raw capture privacy-safe end to end.
The runner persists raw Relay records, and the committed fixture retains local paths, usernames, and session metadata.
streaming-poc/common/run_harness.py#L108-L142: add opt-in capture and redact sensitive fields before writing.streaming-poc/claude/events.atof.jsonl#L1-L2: replaceoutput_directory,cwd, andtranscript_pathwith stable placeholders.streaming-poc/claude/events.atof.jsonl#L18-L18: scrub repeated path and session metadata before committing the fixture.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@streaming-poc/common/run_harness.py` around lines 108 - 142, Update main in streaming-poc/common/run_harness.py (lines 108-142) to make raw record capture opt-in and redact sensitive fields before writing records to OUT, while preserving runtime execution and status reporting. Sanitize the committed fixture in streaming-poc/claude/events.atof.jsonl lines 1-2 by replacing output_directory, cwd, and transcript_path with stable placeholders; scrub repeated path and session metadata at line 18 as well.streaming-poc/synthesis/work-breakdown.md (1)
52-55: 🩺 Stability & Availability | 🟠 Major | ⚡ Quick winClarify early-exit semantics for
RunResult.aclose()should not cancel the task behindawait stream.result()if it is meant to be a non-interrupting detach; otherwise callers can getCancelledErrorand lose the terminalRunResult. Either keep result delivery separate from consumer cancellation, or explicitly document thataclose()forfeitsRunResultand add a test for that contract.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@streaming-poc/synthesis/work-breakdown.md` around lines 52 - 55, Clarify the early-exit contract in the section describing RunResult and aclose(): ensure aclose() or breaking from the stream detaches the consumer without cancelling the task that produces await stream.result(), preserving terminal RunResult delivery while discarding unread buffered events. If the implementation instead forfeits RunResult, explicitly document that behavior and add a test covering the resulting CancelledError contract.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@streaming-poc/claude/findings.md`:
- Around line 23-29: Update the ATOF description in findings.md to remove
“verbatim” and state that Relay stores a metadata projection of each Anthropic
SSE event, including event_type, indices, and provider, without claiming the
native text or thinking delta bodies are embedded. Apply the same correction to
the repeated passage around the later referenced section, while preserving the
distinction from in-process harnesses.
In `@streaming-poc/common/run_harness.py`:
- Around line 76-77: Update the Claude branch in the harness settings flow so
permission bypass is not enabled by default; require an explicit fixture-capture
opt-in before setting settings["permission_mode"] to "bypassPermissions", and
otherwise fail closed or retain the approval boundary for arbitrary INPUT
values.
In `@streaming-poc/README.md`:
- Around line 52-53: Update the Claude entry in the README comparison table to
replace “token-level ATOF” with wording that accurately states the fixture
preserves token-level event metadata while streamed text is available only in
terminal content; leave the remaining status and authentication details
unchanged.
---
Outside diff comments:
In `@streaming-poc/common/run_harness.py`:
- Around line 1-50: Update the harness invocation setup that derives WORK from
ADAPTER so each run uses a unique temporary directory or run-specific
identifier. Ensure the resulting home, workspace, artifacts, and relay paths are
isolated for repeated and concurrent invocations, while preserving the existing
adapter-specific behavior.
- Around line 108-142: Update main and the raw ATOF persistence flow around
OUT.open and json.dumps so writing Relay records is disabled by default unless
an explicit capture flag is provided, or redact sensitive prompts, paths,
session metadata, and request/response payloads before writing. Document the
privacy implications and ensure console progress and result handling remain
unchanged.
- Around line 108-142: Update main in streaming-poc/common/run_harness.py (lines
108-142) to make raw record capture opt-in and redact sensitive fields before
writing records to OUT, while preserving runtime execution and status reporting.
Sanitize the committed fixture in streaming-poc/claude/events.atof.jsonl lines
1-2 by replacing output_directory, cwd, and transcript_path with stable
placeholders; scrub repeated path and session metadata at line 18 as well.
In `@streaming-poc/synthesis/work-breakdown.md`:
- Around line 52-55: Clarify the early-exit contract in the section describing
RunResult and aclose(): ensure aclose() or breaking from the stream detaches the
consumer without cancelling the task that produces await stream.result(),
preserving terminal RunResult delivery while discarding unread buffered events.
If the implementation instead forfeits RunResult, explicitly document that
behavior and add a test covering the resulting CancelledError contract.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Enterprise
Run ID: f843ea5b-c609-4fcc-82d7-5e59e5b20e4d
📒 Files selected for processing (7)
streaming-poc/README.mdstreaming-poc/claude/events.atof.jsonlstreaming-poc/claude/findings.mdstreaming-poc/codex/findings.mdstreaming-poc/common/run_harness.pystreaming-poc/synthesis/README.mdstreaming-poc/synthesis/work-breakdown.md
📜 Review details
⏰ Context from checks skipped due to timeout. (16)
- GitHub Check: Test (Python 3.13, windows-amd64)
- GitHub Check: Test (Python 3.14, windows-amd64)
- GitHub Check: Test (Python 3.13, linux-arm64)
- GitHub Check: Test (Python 3.11, linux-amd64)
- GitHub Check: Test (Python 3.14, macos-arm64)
- GitHub Check: Test (Python 3.14, linux-arm64)
- GitHub Check: Test (Python 3.11, windows-amd64)
- GitHub Check: Test (Python 3.12, windows-amd64)
- GitHub Check: Test (Python 3.11, linux-arm64)
- GitHub Check: Test (Python 3.13, linux-amd64)
- GitHub Check: Test (Python 3.11, macos-arm64)
- GitHub Check: Test (Python 3.12, macos-arm64)
- GitHub Check: Test (Python 3.14, linux-amd64)
- GitHub Check: Test (Python 3.13, macos-arm64)
- GitHub Check: Test (Python 3.12, linux-amd64)
- GitHub Check: Pre-commit
🧰 Additional context used
📓 Path-based instructions (15)
**/*.{md,rst}
📄 CodeRabbit inference engine (.agents/skills/contribute-api/SKILL.md)
Update documentation and examples in the same branch as the public API change.
Files:
streaming-poc/synthesis/README.mdstreaming-poc/README.mdstreaming-poc/codex/findings.mdstreaming-poc/claude/findings.mdstreaming-poc/synthesis/work-breakdown.md
**/*
📄 CodeRabbit inference engine (.agents/skills/karpathy-guidelines/SKILL.md)
**/*: Before implementing, explicitly state assumptions, surface ambiguity and tradeoffs, present multiple interpretations when relevant, and ask for clarification rather than silently deciding or proceeding when requirements are unclear.
Prefer the minimum code needed to solve the requested problem: avoid speculative features, unnecessary abstractions, unrequested flexibility, and handling of impossible scenarios; simplify overcomplicated solutions.
When editing existing code, make surgical changes only: do not modify unrelated code, comments, formatting, or pre-existing dead code; match the existing style, and remove only unused imports, variables, or functions introduced by your changes.
Define verifiable success criteria for each task, such as writing regression tests for bugs and invalid-input tests for validation, then verify the implementation against those criteria. For multi-step work, state a brief plan with a verification check for each step.
**/*: Always spellNVIDIAin all caps; do not useNvidia,nvidia,nVidia,nVIDIA, orNV.
Usean NVIDIAbefore a noun, because the name begins with an “en” sound.
Do not add a registered trademark symbol afterNVIDIAwhen referring to the company; use trademark symbols with product names only when required by the document type or legal guidance.
Verify official capitalization, spacing, hyphenation, and spelling for NVIDIA and third-party product names; do not rewrite official product names for grammar or title-case rules.
Precede NVIDIA product names withNVIDIAon first mention when natural and accurate, and link the first mention when the destination helps the reader.
On first use, include the company name and full model qualifier when it helps identify the model; preserve official capitalization and punctuation, and use shorter family names only after establishing the full name.
For learning-oriented and developer content, do not force trademark symbols unless explicitly required; for press, ...
Files:
streaming-poc/synthesis/README.mdstreaming-poc/claude/events.atof.jsonlstreaming-poc/README.mdstreaming-poc/codex/findings.mdstreaming-poc/common/run_harness.pystreaming-poc/claude/findings.mdstreaming-poc/synthesis/work-breakdown.md
**/*.{md,mdx,rst}
📄 CodeRabbit inference engine (.agents/skills/review-doc-style/assets/nvidia-style-guide.md)
**/*.{md,mdx,rst}: For NeMo Fabric documentation, verify technical claims against the current repository, public API, or documented command before reviewing style.
Always spellNVIDIAin all caps; do not useNvidia,nvidia, orNV.
Format commands, code elements, expressions, package names, file names, and paths as inline code.
Use descriptive link text; avoid raw URLs and weak anchors such ashereorread more.
Use title case consistently for technical documentation headings.
Introduce code blocks, lists, tables, and images with complete sentences.
Write procedures as imperative, parallel steps; split long procedures into smaller tasks.
Prefer active voice, present tense, short sentences, contractions, and plain English while preserving necessary technical precision.
Usecanfor possibility and reservemayfor permission.
Useafterfor temporal relationships instead ofonce, and preferrefer tooverseewhen directing readers to another resource.
Avoid culture-specific idioms, unnecessary Latinisms, jokes, and marketing exaggeration in technical documentation.
Spell out months in body text, avoid ordinal dates, and use clear time zones.
Spell out whole numbers from zero through nine unless they are technical values, parameters, versions, or UI values; use numerals for 10 or greater and commas in thousands.
Do not add trademark symbols to learning-oriented documentation unless the source, platform, or legal guidance explicitly requires them.
Do not replace precise technical terms with simpler words when doing so would lose precision.
Do not flag passive voice when the actor is unknown or the action is the important part.
Do not rewrite API names, package names, command flags, or code literals for style.
**/*.{md,mdx,rst}: Use consistent title case for technical-document headings and table headers; avoid quotation marks, ampersands, and exclamation marks in headings, while preserving official product, event, research, and whitepaper title ...
Files:
streaming-poc/synthesis/README.mdstreaming-poc/README.mdstreaming-poc/codex/findings.mdstreaming-poc/claude/findings.mdstreaming-poc/synthesis/work-breakdown.md
**/*.{md,rst,txt,adoc}
📄 CodeRabbit inference engine (.agents/skills/review-doc-style/assets/nvidia-style-language-mechanics.md)
**/*.{md,rst,txt,adoc}: For technical documentation, use professional, active, conversational, engaging, precise, and plain-English prose. Prefer active voice, present tense, short sentences, and scannable paragraphs. Avoid casual or imprecise language, swearing, threats, insults, jokes, puns, culture-specific idioms, marketing exaggeration, and unsupported third-party comparisons.
Usecanfor possibility and reservemayfor permission; useafterfor temporal order; userefer tofor cross-references; prefer short direct sentences and specific verbs; avoid unnecessarypleasein technical documentation.
Prefer active voice when the actor matters. Passive voice is acceptable when the actor is unknown or irrelevant, when the action or result is the focus, or in programmer documentation.
Use natural contractions in conversational technical prose, but do not force them in formal legal copy, API references, or generated text.
Prefer simpler English over Latinisms: usefor exampleorsuch asinstead ofe.g.,and so oninstead ofetc.,that isinstead ofi.e.,compared toinstead ofvs., andby,through, orusinginstead ofvia. Use industry-standard terms such as in silico, in vitro, and in vivo when appropriate, and italicize them in running text.
Usethatwithout commas for essential clauses, andwhichwith commas for nonessential clauses.
Format dates and times clearly: spell out months in body text; use forms such asJune 12, 2025; avoid numeric or ordinal dates; capitalize days; use 12-hour time when appropriate; include a space beforea.m.orp.m.; useETandPTfor needed time zones; avoid24/7; and preferfrom 12:30 to 1:00 p.m.for prose ranges.
Format numbers consistently: spell out zero through nine in body text, use numerals for 10 or greater and for technical values, use commas in thousands, do not begin a sentence with a numeral, spell out ordinals, and use numerals consistently within a category wh...
Files:
streaming-poc/synthesis/README.mdstreaming-poc/README.mdstreaming-poc/codex/findings.mdstreaming-poc/claude/findings.mdstreaming-poc/synthesis/work-breakdown.md
**/README.md
📄 CodeRabbit inference engine (.agents/skills/review-doc-style/SKILL.md)
Update relevant package, crate, adapter, and integration README files when public behavior or entry-point documentation changes.
Files:
streaming-poc/synthesis/README.mdstreaming-poc/README.md
**/*.{md,mdx}
📄 CodeRabbit inference engine (.agents/skills/review-doc-style/SKILL.md)
**/*.{md,mdx}: Use the full product nameNVIDIA NeMo Fabricon its first usage, typically in the title or H1; useNeMo Fabricthereafter.
Usefabricby itself only when referring to the CLI tool, and surround those references with backticks.
CapitalizeNVIDIAcorrectly in public documentation.
Format commands, code elements, expressions, file names, paths, and filenames as inline code where needed.
Use title case consistently for headings in technical documentation.
Introduce code blocks, tables, and lists with complete lead-in sentences.
Use descriptive anchor text instead of raw URLs or generic link text such ashere.
Prefer active voice, present tense, short sentences, and plain English.
Use consistent terminology for the same concept throughout a document.
Write procedures as imperative, parallel, easy-to-scan steps, and split long sequences into smaller tasks.
Useafterinstead ofoncewhen expressing temporal sequence.
Usecaninstead ofmaywhen the intended meaning is possibility rather than permission.
Avoid ambiguous numeric dates and ordinal dates in body text.
For learning-oriented documentation, do not force trademark symbols unless the source document explicitly requires them.
Introduce examples' code blocks with full sentences and ensure examples match current APIs and build commands.For documentation-site changes, run
just docsto regenerate Python and Rust API references and validate Fern configuration.
Files:
streaming-poc/synthesis/README.mdstreaming-poc/README.mdstreaming-poc/codex/findings.mdstreaming-poc/claude/findings.mdstreaming-poc/synthesis/work-breakdown.md
**/{README.md,*.md,*.mdx}
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Update relevant SDK, API, adapter, example, integration, and embedded documentation when public behavior or the corresponding surface changes.
Files:
streaming-poc/synthesis/README.mdstreaming-poc/README.mdstreaming-poc/codex/findings.mdstreaming-poc/claude/findings.mdstreaming-poc/synthesis/work-breakdown.md
**/*.{html,md}
📄 CodeRabbit inference engine (CONTRIBUTING.md)
HTML and Markdown files must use the specified HTML comment form for SPDX license headers.
Files:
streaming-poc/synthesis/README.mdstreaming-poc/README.mdstreaming-poc/codex/findings.mdstreaming-poc/claude/findings.mdstreaming-poc/synthesis/work-breakdown.md
**/*.{rs,py,pyi,json,yaml,yml}
📄 CodeRabbit inference engine (.agents/skills/contribute-api/SKILL.md)
Determine and update every affected public surface, including the CLI, PyO3 bindings, Python SDK, type stubs, schemas, and adapter contract, so they remain in parity.
Files:
streaming-poc/common/run_harness.py
**/*.{rs,py}
📄 CodeRabbit inference engine (.agents/skills/prepare-pr/SKILL.md)
For native binding changes, run
cargo check -p fabric-python --locked.
Files:
streaming-poc/common/run_harness.py
**/*.{py,pyi}
📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)
If Python code or a Python-facing adapter changes, run
just test-python.Use type annotations for public Python APIs and keep native Python binding declarations synchronized with their Rust implementations.
Files:
streaming-poc/common/run_harness.py
**/*.{rs,py,pyi}
📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)
**/*.{rs,py,pyi}: If public configuration types change, confirm schema snapshot tests injust test-rustpass and review generated schema diffs.
For schema or public contract changes, run both language suites and review changes underschemas/and generated API references.
**/*.{rs,py,pyi}: Usesnake_casefor Rust and Python functions and variables; usePascalCasefor Rust types and Python classes.
Keep native Python binding declarations synchronized with their Rust implementations when public contracts change.
Files:
streaming-poc/common/run_harness.py
**/*.{py,pyi,rs}
📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)
For Python SDK or PyO3 binding changes, use
python-tests, run focused pytest tests first, thenjust test-python; rebuild withjust build-pythonwhen native code or packaging changes.
Files:
streaming-poc/common/run_harness.py
**/*.{rs,py,toml}
📄 CodeRabbit inference engine (.agents/skills/update-project-version/SKILL.md)
When editing version helpers, verify every
nemo-fabric-*workspace package through Cargo metadata and reject a static version inpython/pyproject.toml.
Files:
streaming-poc/common/run_harness.py
**/*.{toml,rs,py}
📄 CodeRabbit inference engine (.agents/skills/update-project-version/SKILL.md)
Avoid blind repository-wide replacement of version-like strings; distinguish package-version references from examples and unrelated dependency versions.
Files:
streaming-poc/common/run_harness.py
🪛 LanguageTool
streaming-poc/claude/findings.md
[style] ~75-~75: Consider an alternative for the overused word “exactly”.
Context: ... itself is terminal-only**. - This is exactly the **"degraded granularity" (option a)...
(EXACTLY_PRECISELY)
🪛 markdownlint-cli2 (0.23.0)
streaming-poc/synthesis/README.md
[warning] 18-18: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
[warning] 19-19: Tables should be surrounded by blank lines
(MD058, blanks-around-tables)
streaming-poc/README.md
[warning] 40-40: Fenced code blocks should be surrounded by blank lines
(MD031, blanks-around-fences)
streaming-poc/claude/findings.md
[warning] 13-13: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
[warning] 20-20: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
[warning] 35-35: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
[warning] 38-38: Fenced code blocks should be surrounded by blank lines
(MD031, blanks-around-fences)
[warning] 38-38: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
[warning] 43-43: Fenced code blocks should be surrounded by blank lines
(MD031, blanks-around-fences)
[warning] 48-48: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
[warning] 55-55: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
[warning] 56-56: Tables should be surrounded by blank lines
(MD058, blanks-around-tables)
[warning] 65-65: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
[warning] 80-80: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
[warning] 88-88: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
[warning] 95-95: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
[warning] 103-103: Fenced code blocks should be surrounded by blank lines
(MD031, blanks-around-fences)
[warning] 110-110: Fenced code blocks should be surrounded by blank lines
(MD031, blanks-around-fences)
🔇 Additional comments (11)
streaming-poc/claude/findings.md (1)
1-13: Resolve the remaining Markdown style violations.The title and headings are not consistently title case, headings are not separated from their content, and later tables and fenced blocks still need the reported spacing and language fixes. This is the same unresolved finding from the previous review.
As per coding guidelines, technical-documentation headings must use title case, and headings, tables, and fenced code blocks must be surrounded by blank lines.
Sources: Coding guidelines, Linters/SAST tools
streaming-poc/codex/findings.md (2)
1-6: Restore the SPDX header and title-case heading.This new Markdown file begins directly with the heading and uses sentence case. This is the same unresolved documentation finding from the previous review.
As per coding guidelines, all source files must include the applicable SPDX copyright and Apache-2.0 header, and technical-documentation headings must use title case.
Source: Coding guidelines
8-38: LGTM!streaming-poc/README.md (2)
17-24: Label this as the proposed production API, or use the runnable POC entry point.The current harness uses
start_streaming_runtime(fabric, config)to inject the listener and return the wrapper, but this README presentsfabric.start_runtime(config)as the POC contract. This remains the same unresolved documentation issue from the previous review.
6-18: Resolve the remaining Markdown layout violations.The title and contract/layout sections still contain the previously reported heading and fence-spacing issues. Add blank lines around headings and fences, introduce each block with a complete sentence, and use
textfor the tree-layout fence.As per coding guidelines, technical-documentation headings must use title case, and headings, tables, and fenced code blocks must be surrounded by blank lines.
Also applies to: 29-41
Sources: Coding guidelines, Linters/SAST tools
streaming-poc/synthesis/README.md (2)
6-19: Resolve the remaining Markdown style violations.The title is not title case and uses
&; the cross-harness heading is not separated from its table, and the later fenced block needs the required surrounding blank line.As per coding guidelines, technical-documentation headings and table headers must use title case, and headings, tables, and fenced code blocks must be surrounded by blank lines.
Also applies to: 55-61
Sources: Coding guidelines, Linters/SAST tools
24-35: 🎯 Functional CorrectnessKeep Codex granularity explicitly unverified.
The document correctly labels Codex as preliminary, but later states that gateway Claude/Codex streams are token-level and that native detail is preserved for all four harnesses. Qualify those statements as Claude-observed and Codex-expected until a real Codex capture exists.
Also applies to: 44-49
streaming-poc/synthesis/work-breakdown.md (2)
1-12: Resolve the remaining Markdown heading violations.The title and first section still use sentence case, and the first heading is not separated from its list.
As per coding guidelines, technical-documentation headings must use title case and be surrounded by blank lines.
Source: Coding guidelines
19-22: 🗄️ Data Integrity & IntegrationVerify the “no Rust/core change required” claim.
The supplied POC injects the listener in
streaming-poc/common/fabric_stream.py:start_streaming_runtimebefore callingfabric.start_runtime; that does not prove that moving injection intoFabric.start_runtimesurvives planning for all four adapters without core changes. Add adapter-level regression coverage or link the evidence before treating this as verified.streaming-poc/common/run_harness.py (2)
54-67: LGTM!
78-82: 🎯 Functional CorrectnessNo relay-version mismatch The
nemo-relay>=0.5.0,<0.7entry is the Python package dependency; the external CLI contract is already>=0.6.0,<0.7.0instreaming-poc/common/run_harness.py,adapters/common/src/nemo_fabric_adapters/common/relay_gateway.py, and the related docs. No update needed.> Likely an incorrect or invalid review comment.
| if "claude" in ADAPTER: # run the turn non-interactively | ||
| settings["permission_mode"] = "bypassPermissions" |
There was a problem hiding this comment.
🔒 Security & Privacy | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
sed -n '1,180p' streaming-poc/common/run_harness.py
printf '\n---\n'
rg -n "bypassPermissions|permission_mode|run_harness|ADAPTER" streaming-poc -SRepository: NVIDIA/NeMo-Fabric
Length of output: 9381
Do not default Claude runs to bypassPermissions.
This removes the approval boundary for arbitrary INPUT values; gate it behind an explicit opt-in for fixture capture, or fail closed when the opt-in is absent.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@streaming-poc/common/run_harness.py` around lines 76 - 77, Update the Claude
branch in the harness settings flow so permission bypass is not enabled by
default; require an explicit fixture-capture opt-in before setting
settings["permission_mode"] to "bypassPermissions", and otherwise fail closed or
retain the approval boundary for arbitrary INPUT values.
…on/SSO Un-stub the Codex child POC with a real, gateway-captured ATOF fixture, mirroring Claude. After upgrading the Codex CLI to 0.145.0 (older CLIs reject the ChatGPT account's gpt-5.6-sol model), the Relay gateway forwarded the ChatGPT OAuth session so the run needed no OPENAI_API_KEY. Adds streaming-poc/codex/events.atof.jsonl (22 records, 17 token-level llm.chunk marks spanning the canonical OpenAI Responses sequence response.created -> response.output_text.delta x9 -> response.completed) and a full findings.md. Same fidelity result as Claude: llm.chunk marks carry event type/index/timing/usage live, but the per-delta token text is terminal-only in Relay's current projection (text lands in the terminal openai.responses end scope output). Codex and Claude produce structurally identical ATOF from different SDK event models — confirming the cross-harness uniformity holds at the ATOF layer. README/synthesis/work-breakdown updated: all four harnesses now captured for real. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com>
There was a problem hiding this comment.
Actionable comments posted: 3
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
streaming-poc/synthesis/work-breakdown.md (1)
60-64: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick winDefine which terminal scope is authoritative.
The Codex fixture contains the same final
output[].content[].textin nestedopenai.responsesand outercodex-turnend records at Lines 21-22. The current contract covers delta-versus-terminal replacement but not nested terminal copies, so raw consumers can render the answer twice. Specify the authoritative scope or response ID and require deduplication; qualify “HIGH” as conditional because current live deltas contain no text.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@streaming-poc/synthesis/work-breakdown.md` around lines 60 - 64, Update the “Delta-vs-terminal” contract to identify the authoritative terminal scope or response ID for nested terminal records, require raw consumers to deduplicate identical final text from nested openai.responses and outer codex-turn records, and qualify the duplicate risk as conditional because current live deltas contain no text.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@streaming-poc/codex/events.atof.jsonl`:
- Around line 1-2: Sanitize the committed events fixture by replacing developer
usernames and all project-specific absolute filesystem paths in
output_directory, cwd, and transcript_path with stable placeholders. Preserve
the event structure and non-path values while ensuring no host-specific path
segments remain.
In `@streaming-poc/codex/findings.md`:
- Around line 14-17: Update the Markdown formatting in
streaming-poc/codex/findings.md at lines 14-17, 23, 39, 52, 59, 70, 85, 93, 102,
110, and 117: add blank lines around headings and fenced blocks, tag the
event-sequence fence as text, and title-case the affected headings. In
streaming-poc/synthesis/README.md at lines 18-19, add blank lines around the
Cross-Harness Evidence heading and its table.
- Around line 48-50: Replace token-level wording with delta-event granularity
wording wherever describing response.output_text.delta or ATOF: update
streaming-poc/codex/findings.md lines 48-50 and streaming-poc/README.md lines
51-73. Clarify that each SSE record is an incremental text delta without
implying one token per event.
---
Outside diff comments:
In `@streaming-poc/synthesis/work-breakdown.md`:
- Around line 60-64: Update the “Delta-vs-terminal” contract to identify the
authoritative terminal scope or response ID for nested terminal records, require
raw consumers to deduplicate identical final text from nested openai.responses
and outer codex-turn records, and qualify the duplicate risk as conditional
because current live deltas contain no text.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Enterprise
Run ID: c53c4595-2741-4260-ac65-fd7e56cd2dd0
📒 Files selected for processing (5)
streaming-poc/README.mdstreaming-poc/codex/events.atof.jsonlstreaming-poc/codex/findings.mdstreaming-poc/synthesis/README.mdstreaming-poc/synthesis/work-breakdown.md
📜 Review details
⏰ Context from checks skipped due to timeout. (11)
- GitHub Check: Test (Python 3.12, windows-amd64)
- GitHub Check: Test (Python 3.13, windows-amd64)
- GitHub Check: Test (Python 3.12, macos-arm64)
- GitHub Check: Test (Python 3.13, macos-arm64)
- GitHub Check: Test (Python 3.13, linux-amd64)
- GitHub Check: Test (Python 3.14, windows-amd64)
- GitHub Check: Test (Python 3.11, windows-amd64)
- GitHub Check: Test (Python 3.14, macos-arm64)
- GitHub Check: Test (Python 3.12, linux-arm64)
- GitHub Check: Test (Python 3.11, macos-arm64)
- GitHub Check: Pre-commit
🧰 Additional context used
📓 Path-based instructions (8)
**/*.{md,rst}
📄 CodeRabbit inference engine (.agents/skills/contribute-api/SKILL.md)
Update documentation and examples in the same branch as the public API change.
Files:
streaming-poc/synthesis/README.mdstreaming-poc/README.mdstreaming-poc/codex/findings.mdstreaming-poc/synthesis/work-breakdown.md
**/*
📄 CodeRabbit inference engine (.agents/skills/karpathy-guidelines/SKILL.md)
**/*: Before implementing, explicitly state assumptions, surface ambiguity and tradeoffs, present multiple interpretations when relevant, and ask for clarification rather than silently deciding or proceeding when requirements are unclear.
Prefer the minimum code needed to solve the requested problem: avoid speculative features, unnecessary abstractions, unrequested flexibility, and handling of impossible scenarios; simplify overcomplicated solutions.
When editing existing code, make surgical changes only: do not modify unrelated code, comments, formatting, or pre-existing dead code; match the existing style, and remove only unused imports, variables, or functions introduced by your changes.
Define verifiable success criteria for each task, such as writing regression tests for bugs and invalid-input tests for validation, then verify the implementation against those criteria. For multi-step work, state a brief plan with a verification check for each step.
**/*: Always spellNVIDIAin all caps; do not useNvidia,nvidia,nVidia,nVIDIA, orNV.
Usean NVIDIAbefore a noun, because the name begins with an “en” sound.
Do not add a registered trademark symbol afterNVIDIAwhen referring to the company; use trademark symbols with product names only when required by the document type or legal guidance.
Verify official capitalization, spacing, hyphenation, and spelling for NVIDIA and third-party product names; do not rewrite official product names for grammar or title-case rules.
Precede NVIDIA product names withNVIDIAon first mention when natural and accurate, and link the first mention when the destination helps the reader.
On first use, include the company name and full model qualifier when it helps identify the model; preserve official capitalization and punctuation, and use shorter family names only after establishing the full name.
For learning-oriented and developer content, do not force trademark symbols unless explicitly required; for press, ...
Files:
streaming-poc/synthesis/README.mdstreaming-poc/codex/events.atof.jsonlstreaming-poc/README.mdstreaming-poc/codex/findings.mdstreaming-poc/synthesis/work-breakdown.md
**/*.{md,mdx,rst}
📄 CodeRabbit inference engine (.agents/skills/review-doc-style/assets/nvidia-style-guide.md)
**/*.{md,mdx,rst}: For NeMo Fabric documentation, verify technical claims against the current repository, public API, or documented command before reviewing style.
Always spellNVIDIAin all caps; do not useNvidia,nvidia, orNV.
Format commands, code elements, expressions, package names, file names, and paths as inline code.
Use descriptive link text; avoid raw URLs and weak anchors such ashereorread more.
Use title case consistently for technical documentation headings.
Introduce code blocks, lists, tables, and images with complete sentences.
Write procedures as imperative, parallel steps; split long procedures into smaller tasks.
Prefer active voice, present tense, short sentences, contractions, and plain English while preserving necessary technical precision.
Usecanfor possibility and reservemayfor permission.
Useafterfor temporal relationships instead ofonce, and preferrefer tooverseewhen directing readers to another resource.
Avoid culture-specific idioms, unnecessary Latinisms, jokes, and marketing exaggeration in technical documentation.
Spell out months in body text, avoid ordinal dates, and use clear time zones.
Spell out whole numbers from zero through nine unless they are technical values, parameters, versions, or UI values; use numerals for 10 or greater and commas in thousands.
Do not add trademark symbols to learning-oriented documentation unless the source, platform, or legal guidance explicitly requires them.
Do not replace precise technical terms with simpler words when doing so would lose precision.
Do not flag passive voice when the actor is unknown or the action is the important part.
Do not rewrite API names, package names, command flags, or code literals for style.
**/*.{md,mdx,rst}: Use consistent title case for technical-document headings and table headers; avoid quotation marks, ampersands, and exclamation marks in headings, while preserving official product, event, research, and whitepaper title ...
Files:
streaming-poc/synthesis/README.mdstreaming-poc/README.mdstreaming-poc/codex/findings.mdstreaming-poc/synthesis/work-breakdown.md
**/*.{md,rst,txt,adoc}
📄 CodeRabbit inference engine (.agents/skills/review-doc-style/assets/nvidia-style-language-mechanics.md)
**/*.{md,rst,txt,adoc}: For technical documentation, use professional, active, conversational, engaging, precise, and plain-English prose. Prefer active voice, present tense, short sentences, and scannable paragraphs. Avoid casual or imprecise language, swearing, threats, insults, jokes, puns, culture-specific idioms, marketing exaggeration, and unsupported third-party comparisons.
Usecanfor possibility and reservemayfor permission; useafterfor temporal order; userefer tofor cross-references; prefer short direct sentences and specific verbs; avoid unnecessarypleasein technical documentation.
Prefer active voice when the actor matters. Passive voice is acceptable when the actor is unknown or irrelevant, when the action or result is the focus, or in programmer documentation.
Use natural contractions in conversational technical prose, but do not force them in formal legal copy, API references, or generated text.
Prefer simpler English over Latinisms: usefor exampleorsuch asinstead ofe.g.,and so oninstead ofetc.,that isinstead ofi.e.,compared toinstead ofvs., andby,through, orusinginstead ofvia. Use industry-standard terms such as in silico, in vitro, and in vivo when appropriate, and italicize them in running text.
Usethatwithout commas for essential clauses, andwhichwith commas for nonessential clauses.
Format dates and times clearly: spell out months in body text; use forms such asJune 12, 2025; avoid numeric or ordinal dates; capitalize days; use 12-hour time when appropriate; include a space beforea.m.orp.m.; useETandPTfor needed time zones; avoid24/7; and preferfrom 12:30 to 1:00 p.m.for prose ranges.
Format numbers consistently: spell out zero through nine in body text, use numerals for 10 or greater and for technical values, use commas in thousands, do not begin a sentence with a numeral, spell out ordinals, and use numerals consistently within a category wh...
Files:
streaming-poc/synthesis/README.mdstreaming-poc/README.mdstreaming-poc/codex/findings.mdstreaming-poc/synthesis/work-breakdown.md
**/README.md
📄 CodeRabbit inference engine (.agents/skills/review-doc-style/SKILL.md)
Update relevant package, crate, adapter, and integration README files when public behavior or entry-point documentation changes.
Files:
streaming-poc/synthesis/README.mdstreaming-poc/README.md
**/*.{md,mdx}
📄 CodeRabbit inference engine (.agents/skills/review-doc-style/SKILL.md)
**/*.{md,mdx}: Use the full product nameNVIDIA NeMo Fabricon its first usage, typically in the title or H1; useNeMo Fabricthereafter.
Usefabricby itself only when referring to the CLI tool, and surround those references with backticks.
CapitalizeNVIDIAcorrectly in public documentation.
Format commands, code elements, expressions, file names, paths, and filenames as inline code where needed.
Use title case consistently for headings in technical documentation.
Introduce code blocks, tables, and lists with complete lead-in sentences.
Use descriptive anchor text instead of raw URLs or generic link text such ashere.
Prefer active voice, present tense, short sentences, and plain English.
Use consistent terminology for the same concept throughout a document.
Write procedures as imperative, parallel, easy-to-scan steps, and split long sequences into smaller tasks.
Useafterinstead ofoncewhen expressing temporal sequence.
Usecaninstead ofmaywhen the intended meaning is possibility rather than permission.
Avoid ambiguous numeric dates and ordinal dates in body text.
For learning-oriented documentation, do not force trademark symbols unless the source document explicitly requires them.
Introduce examples' code blocks with full sentences and ensure examples match current APIs and build commands.For documentation-site changes, run
just docsto regenerate Python and Rust API references and validate Fern configuration.
Files:
streaming-poc/synthesis/README.mdstreaming-poc/README.mdstreaming-poc/codex/findings.mdstreaming-poc/synthesis/work-breakdown.md
**/{README.md,*.md,*.mdx}
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Update relevant SDK, API, adapter, example, integration, and embedded documentation when public behavior or the corresponding surface changes.
Files:
streaming-poc/synthesis/README.mdstreaming-poc/README.mdstreaming-poc/codex/findings.mdstreaming-poc/synthesis/work-breakdown.md
**/*.{html,md}
📄 CodeRabbit inference engine (CONTRIBUTING.md)
HTML and Markdown files must use the specified HTML comment form for SPDX license headers.
Files:
streaming-poc/synthesis/README.mdstreaming-poc/README.mdstreaming-poc/codex/findings.mdstreaming-poc/synthesis/work-breakdown.md
🪛 LanguageTool
streaming-poc/codex/findings.md
[style] ~17-~17: Consider an alternative to strengthen your wording.
Context: ...e Relay gateway. Final answer: "Hello! It's great to meet you." Requires Codex CLI **≥0...
(NICE_MEETING_YOU)
[style] ~80-~80: Consider an alternative for the overused word “exactly”.
Context: ...xt itself is terminal-only**. - This is exactly the **"degraded granularity" (option a)...
(EXACTLY_PRECISELY)
🪛 markdownlint-cli2 (0.23.0)
streaming-poc/synthesis/README.md
[warning] 18-18: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
[warning] 19-19: Tables should be surrounded by blank lines
(MD058, blanks-around-tables)
streaming-poc/codex/findings.md
[warning] 14-14: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
[warning] 23-23: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
[warning] 39-39: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
[warning] 42-42: Fenced code blocks should be surrounded by blank lines
(MD031, blanks-around-fences)
[warning] 42-42: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
[warning] 47-47: Fenced code blocks should be surrounded by blank lines
(MD031, blanks-around-fences)
[warning] 52-52: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
[warning] 59-59: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
[warning] 60-60: Tables should be surrounded by blank lines
(MD058, blanks-around-tables)
[warning] 70-70: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
[warning] 85-85: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
[warning] 93-93: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
[warning] 102-102: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
[warning] 110-110: Fenced code blocks should be surrounded by blank lines
(MD031, blanks-around-fences)
[warning] 117-117: Fenced code blocks should be surrounded by blank lines
(MD031, blanks-around-fences)
🔇 Additional comments (4)
streaming-poc/README.md (2)
51-51: 🎯 Functional CorrectnessQualify the Codex stream as token-level event metadata.
The Codex findings state that live
llm.chunkrecords preserve event metadata but omit per-delta text; assembled text appears only in terminal output. Replace “token-level ATOF” with “token-level event metadata; terminal-only text,” and clarify that event metadata and IDs—not delta text—are preserved. This repeats the earlier Claude finding.Also applies to: 71-73
9-13: LGTM!Also applies to: 36-36, 64-69
streaming-poc/codex/events.atof.jsonl (1)
3-22: LGTM!streaming-poc/synthesis/README.md (1)
8-16: LGTM!Also applies to: 44-48
| {"atof_version": "0.1", "category": null, "category_profile": null, "data": null, "data_schema": null, "kind": "mark", "metadata": {"agent_kind": "codex", "hook_event_name": "SessionStart", "model": "gpt-5.6-sol", "plugin_config": {"components": [{"config": {"atof": {"enabled": true, "sinks": [{"filename": "events.atof.jsonl", "mode": "append", "output_directory": "/private/tmp/claude-502/-Users-yuchenz-Desktop-Work-Project-NeMo-Fabric/68b9b7a7-7566-423a-b7e6-ede8bf5deb9c/scratchpad/cx-work/relay/runtime-1784771704939-7402-2", "type": "file"}, {"field_name_policy": "preserve", "header_env": {}, "headers": {}, "timeout_millis": 3000, "transport": "ndjson", "type": "stream", "url": "http://127.0.0.1:51477/atof"}]}, "version": 2}, "enabled": true, "kind": "observability"}], "version": 1}, "session_id": "019f8cae-fb13-7370-9de5-2365fe7f838a", "session_instance_id": "019f8cae-fee6-72b0-82f6-4150cac5a836"}, "name": "session.start", "parent_uuid": "019f8cae-fee6-72b0-82f6-4150cac5a836", "timestamp": "2026-07-23T01:55:06.598518+00:00", "uuid": "019f8cae-fee6-72b0-82f6-41603a3c34b6"} | ||
| {"atof_version": "0.1", "attributes": [], "category": "custom", "category_profile": null, "data": {"cwd": "/private/tmp/claude-502/-Users-yuchenz-Desktop-Work-Project-NeMo-Fabric/68b9b7a7-7566-423a-b7e6-ede8bf5deb9c/scratchpad/cx-work/workspace", "hook_event_name": "UserPromptSubmit", "model": "gpt-5.6-sol", "permission_mode": "default", "prompt": "Reply with a one-sentence greeting.", "session_id": "019f8cae-fb13-7370-9de5-2365fe7f838a", "transcript_path": "/Users/yuchenz/.codex/sessions/2026/07/22/rollout-2026-07-22T18-55-05-019f8cae-fb13-7370-9de5-2365fe7f838a.jsonl", "turn_id": "019f8cae-fbfb-7612-a873-e4525ed0f0c6"}, "data_schema": null, "kind": "scope", "metadata": {"agent_kind": "codex", "hook_event_name": "UserPromptSubmit", "model": "gpt-5.6-sol", "nemo_relay_scope_role": "turn", "plugin_config": {"components": [{"config": {"atof": {"enabled": true, "sinks": [{"filename": "events.atof.jsonl", "mode": "append", "output_directory": "/private/tmp/claude-502/-Users-yuchenz-Desktop-Work-Project-NeMo-Fabric/68b9b7a7-7566-423a-b7e6-ede8bf5deb9c/scratchpad/cx-work/relay/runtime-1784771704939-7402-2", "type": "file"}, {"field_name_policy": "preserve", "header_env": {}, "headers": {}, "timeout_millis": 3000, "transport": "ndjson", "type": "stream", "url": "http://127.0.0.1:51477/atof"}]}, "version": 2}, "enabled": true, "kind": "observability"}], "version": 1}, "session_id": "019f8cae-fb13-7370-9de5-2365fe7f838a", "session_instance_id": "019f8cae-fee6-72b0-82f6-4150cac5a836", "turn_index": 1, "turn_source": "user_prompt"}, "name": "codex-turn", "parent_uuid": "019f8cae-fee6-72b0-82f6-4150cac5a836", "scope_category": "start", "timestamp": "2026-07-23T01:55:06.724928+00:00", "uuid": "019f8cae-ff64-7592-81b4-fc002d610e82"} |
There was a problem hiding this comment.
🔒 Security & Privacy | 🟠 Major | ⚡ Quick win
Redact host paths and usernames from the committed fixture.
Lines 1-2 expose /Users/yuchenz and a project-specific absolute temporary path in output_directory, cwd, and transcript_path. Normalize these fields to stable placeholders before committing to avoid leaking developer identity and local filesystem layout.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@streaming-poc/codex/events.atof.jsonl` around lines 1 - 2, Sanitize the
committed events fixture by replacing developer usernames and all
project-specific absolute filesystem paths in output_directory, cwd, and
transcript_path with stable placeholders. Preserve the event structure and
non-path values while ensuring no host-specific path segments remain.
…rclaims Address reviewer findings on the streaming POC: - FABRIC-103: capture genuine raw native-event fixtures for the gateway harnesses (previously only ATOF projections). Tee the adapter SDK streams before Relay (Claude: include_partial_messages + StreamEvent; Codex: AsyncTurnHandle notifications) via a temporary POC seam, add claude/ + codex/ native-events.jsonl, and rewrite both findings as a measured native->ATOF diff: the native stream carries the per-delta token text (verbatim) that Relay's ATOF projection drops (text is terminal-only). One email in Claude's thinking was redacted; seams reverted. - FABRIC-104: stop marking Deep Agents "complete" in the root README; it ran subagents sequentially, so parallel interleaving is still unobserved (open). - Public wording: replace "token streaming"/"token-level" with "live delta-event structure/timing; terminal text" so it no longer implies renderable incremental text. - Prototype vs spec: bound the loopback queue by default (maxsize=1024, was 0/ unbounded) so backpressure is real; make InvokeStream.aclose() detach the consumer without cancelling the turn (matches the documented contract); correct the early-exit docs — runtime.stop() raises while a turn is in flight, so there is no in-flight cancellation in v0.1 (flagged as a production follow-up). - Two-turn isolation: add a checked-in artifact (two-turn-isolation.jsonl) and runner (common/two_turn_isolation.py) proving two turns on one persistent runtime have disjoint record uuids (overlap 0) and no sentinel leakage. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com>
…ify claims Address a second review pass: - P1 (leak): InvokeStream.aclose() only set a flag, so a turn's unread records leaked into the next invocation on the shared listener queue. Finalize now drains and discards through invocation completion, and StreamingRuntime enforces one active invocation (a concurrent invoke_stream raises FabricStateError). Verified: early-exit turn 1 (read 1 of N, aclose) no longer leaks its ALPHA sentinel or uuids into turn 2; concurrent invoke rejected. - P2 (artifact): two_turn_isolation.py now exercises the leak path directly — early exit with unread records + a concurrency guard — not just fully drained sequential turns. Regenerated two-turn-isolation.jsonl (isolated: true). - P2 (synthesis): drop "structurally identical" — Claude/Codex share the ATOF envelope but differ in event vocabulary/payloads; relabel Codex's captured native layer as app-server notifications (Relay taps Responses SSE at the gateway); and stop claiming "native detail preserved" (the per-delta token text is dropped). - P2 (privacy): scrub local identifying paths (/Users/<name>, session/workspace/ relay dirs, and dash-encoded project dirnames) from all committed fixtures to stable placeholders (/workspace/nemo-fabric, /home/user). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com>
…RIC-104) Close the last open review item with a real capture of concurrent, delegated subagents. meta/llama-3.1-70b-instruct emits two `task` calls in one assistant message (a provider probe found llama-3.3-70b serializes and gpt-oss-120b emits one), so LangGraph runs the two subagents concurrently. Adds deepagents/parallel-events.atof.jsonl + parallel-native-events.jsonl showing three independent parallelism signals: - two sibling `task` scopes starting 5us apart (same parent), - overlapping sibling scopes active together for 9.57s, - interleaved subagent namespaces in the native tee (tools:43d0a4d3 <-> tools:5c7a0931, A/B/A/B/A/B/A). Honest caveat documented: the run's terminal status is `failed` because the final combine model call trips the endpoint's "model only supports single tool-calls at once" limit — after, and independent of, the parallel delegation, whose sibling task scopes both close normally. The sequential nemotron-nano run is kept as the loss-analysis fixture. README/synthesis flip Deep Agents to parallel-observed. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com>
…it docs - InvokeStream._finalize() swallowed BaseException, so a cancelled aclose() returned normally while the cancellation propagated and left stream.result() cancelled. Now await asyncio.shield(self._task) (keep the invoke task running if the caller is cancelled) and re-raise CancelledError instead of suppressing it; only the invoke's own error is swallowed (still retrievable via result()). Verified: cancelling aclose() propagates CancelledError and result() stays valid; normal finalize and error-retrieval paths unchanged. - implementation-spec + work-breakdown said breaking the async for detaches/cleans up the stream. Correct: breaking only stops iteration; you must `await stream.aclose()` to finalize, and that call waits for the turn to complete before draining. The next invoke_stream raises until the stream is finalized. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com>
…+ redundancy Editorial pass over the POC docs — same facts, fewer words, no back-and-forth: - Drop process/defensive framing that referenced how findings evolved: "genuine native evidence", "measured, not inferred/assumed", "this is the point", "now observed", "verified by grepping/diffing", "(POC bug found + fixed)", the reviewer-process references, and the model-selection play-by-play in the Deep Agents parallel reproduce. - De-duplicate the fidelity finding (delta-text terminal-only), which was restated ~5x: fold the synthesis "unifying fact" into a single Conclusion, trim the README layout/fixture notes, and shorten each gateway findings' Recommendation to point at the cross-harness synthesis instead of restating it. - Remove the redundant "Prototype crossing the Fabric boundary" section (the mechanism lives in implementation-spec.md) and normalize scenario headers. Docs only; no fixtures or code touched. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com>
…ibility, style Must-fix: - Turn isolation is now presented as best-effort, not guaranteed: the 250 ms drain is a timing heuristic (a record arriving later can leak into the next turn). The spec, runner docstring, and fabric_stream comment say so, and the work breakdown requires a positive turn boundary (per-record turn attribution or a terminal delivery ack) for production. - Scrub + truncate two-turn-isolation.jsonl: the 205 KB request snapshot on line 3 carried an Anthropic device hash, account UUID, session ids, and the full request/tool body — now elided and identity values redacted (max line 1.8 KB). - Synthesis no longer claims ATOF "drops exactly one native detail" / preserves "everything a progress UI needs": ATOF is a lossy projection (also drops content-block bodies/types, app-server lifecycle); the one *rendering-relevant* shared loss is per-delta token text, so it is "enough for a structural progress UI, not incremental text rendering". - Separate the proposed v0.1 API (`fabric.start_runtime(...).invoke_stream(...)`) from the runnable prototype (`start_streaming_runtime()` / `StreamingRuntime`) with an explicit callout in the README, spec, and synthesis. - Reproduce procedures no longer overwrite committed fixtures or `git checkout --` adapters: add checked-in, reversible `patches/*-native-tee.patch` (verified apply forward + reverse) and rewrite all four recipes to `git apply` the patch, write to a scratch dir, then `git apply -R`. Should-fix: three child POCs / four harnesses; Hermes native layer relabeled lifecycle/plugin hooks; Deep Agents terminal shown as sequential-OK / parallel-failed; curly apostrophe in the Codex answer; title-case headings; NVIDIA NeMo Fabric at the top-level reference; table lead-ins; NDJSON and "credentials" standardized. The obsolete root STREAMING-POC.md (git-excluded) was replaced locally with a short pointer to streaming-poc/README.md; it is not part of this commit. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com>
…mmon/README The proposed-vs-runnable API callout in README.md and synthesis/README.md labeled a link `common/fabric_stream.py` but href'd common/README.md. Point both at the actual common/fabric_stream.py. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com>
yczhang-nv
changed the title
|
Superseded by #109 |
#### Overview Implements streaming backed by NVIDIA NeMo Relay for NVIDIA NeMo Fabric runtimes, promoting the streaming proof of concept from #104 into the supported Python SDK surface. `Runtime.invoke_stream()` exposes raw ATOF records while an invocation is running and keeps the normalized terminal `RunResult` out of band. Streaming is **opt-in**: callers pass `streaming=True` to `start_runtime(...)` (with NeMo Relay enabled) to provision it, so NeMo Relay-enabled runtimes that do not stream pay no additional cost and keep their existing telemetry shape. The streaming API change is additive — a new optional `streaming` parameter — and introduces no new dependencies. #### Details - Add an SDK-owned loopback HTTP listener for NeMo Relay NDJSON/ATOF delivery with a bounded, byte-budgeted record queue (16 MiB total / 1 MiB per record), TCP backpressure, and a bounded post-turn drain window. - Gate provisioning behind `start_runtime(..., streaming=True)`: a NeMo Relay-enabled runtime binds the loopback listener and injects the `nemo-fabric-stream` ATOF sink only when streaming is requested. Requesting `streaming=True` without NeMo Relay enabled raises `FabricConfigError`. - Add `Runtime.invoke_stream()`, `Runtime.supports_streaming`, `InvokeStream.result()`, and explicit early-exit cleanup via `InvokeStream.aclose()`. - Configure the listener host through an environment variable so deployments can override the loopback default when needed. - Correlate records to a single invocation by ATOF scope tree: Deep Agents and Hermes stamp a request ID on the turn's root scope; gateway harnesses match on NeMo Relay's turn role and index. The Hermes correlation scope is added only for streaming runs, so non-streaming NeMo Relay runs keep their prior trajectory shape. - Surface otherwise-silent failures as `RuntimeWarning`s: no NeMo Relay connection reached the listener, data arrived but no record matched the active turn, or a chunked upload was interrupted mid-turn. A clean stream completion does not warn. - Validate streaming requests before returning an `InvokeStream`, make stream iteration cancellation-safe, and allow a manual `Runtime.stop()` to finalize a completed stream after `result()`. - Document the public streaming contract, terminal failure semantics, adapter behavior, runnable examples, and the generated Python API reference. #### Validation - Focused streaming, adapter, generated-reference, and Fern-sync tests: 135 passed. - `uv run --no-sync pre-commit run --all-files --show-diff-on-failure` — all hooks passed, including Cargo/uv lockfile freshness, attribution generation, and the dependency license diff. - `just docs` — passed; Fern reported only the expected unauthenticated redirect check warning. - `just test-python` — 455 passed and 44 skipped locally. Two failures and two setup errors remain in `test_installed_adapter_discovery.py` because uv's local macOS Python 3.13 virtual environments cannot load `libpython3.13.dylib`; the streaming-focused tests pass independently. - End-to-end streaming turns run under `warnings.simplefilter("error", RuntimeWarning)` and assert non-empty records: - Deep Agents — `tests/e2e/test_deepagents.py` (persistent host, mock model). - Claude and Codex gateways — live tests in `tests/e2e/test_claude.py` and `tests/e2e/test_codex.py`, validating gateway turn correlation. #### Where should the reviewer start? Start with `python/src/nemo_fabric/streaming.py` for the listener, turn correlation, and stream lifecycle; then `python/src/nemo_fabric/client.py` for the `streaming=True` opt-in gate and `python/src/nemo_fabric/runtime.py` for eager validation and manual shutdown. The focused behavioral coverage is in `tests/python/test_streaming.py`, with end-to-end coverage under `tests/e2e/`. #### Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to) - Supersedes the implementation POC in #104 - [x] I confirm this contribution is my own work, or I have the right to submit it under this project's license. - [x] I searched existing issues and open pull requests, and this does not duplicate existing work. ## Summary by CodeRabbit * **New Features** * Added NeMo Relay-backed streaming to the Python SDK via `Runtime.invoke_stream()` with an `InvokeStream` async iterator of raw ATOF records and a terminal `await stream.result()`. * Added `Runtime.supports_streaming` and streaming-aware runtime startup/stop lifecycle handling. * Introduced typed v2 NeMo Relay ATOF sink configuration via file/stream sink models. * **Documentation** * Added the Streaming API reference and updated Python and integration guides with end-to-end streaming workflows. * Updated NeMo Relay ATOF examples to use the typed v2 `sinks` model. * **Tests** * Expanded/updated streaming, documentation contract, and adapter configuration tests to match the new streaming and typed sink behavior. Authors: - Yuchen Zhang (https://github.com/yczhang-nv) Approvers: - Anuradha Karuppiah (https://github.com/AnuradhaKaruppiah) - Ajay Thorve (https://github.com/AjayThorve) URL: #109
2 participants
Overview
Proof-of-concept for an NVIDIA NeMo Fabric streaming API built on NeMo Relay-generated ATOF, added as a self-contained
streaming-poc/folder. It validates a v0.1 contract:Runtime.invoke_stream()as syntactic sugar overinvoke()that yields raw ATOF records, withRunResultdelivered out of band, available only when Relay is enabled — no Fabric-specific event normalization in v0.1. This is a proposed API surface, not implemented in the SDK; the POC models it instreaming-poc/common/fabric_stream.py.All four harnesses were run for real. For each, the native SDK stream was captured before Relay (via a POC-only recorder) alongside Relay's ATOF from the same run, so the native→ATOF loss analysis is measured, not assumed. The two gateway harnesses (Claude, Codex) were captured on a subscription / SSO session forwarded by the Relay gateway — no API key. The single measured, rendering-relevant loss shared by both gateway harnesses is the per-delta token text (it lands only in the terminal scope), so the live stream is enough for a structural progress UI but not incremental text rendering.
This PR adds no product code — only the
streaming-poc/folder (31 files; nothing underadapters/,crates/, or the Python SDK).Details
streaming-poc/common/— the experimental prototype: a loopback NDJSON ATOF listener (bounded queue + TCP backpressure; handles >512 KB gateway records),invoke_stream()(loopback endpoint injected atstart_runtime; raw ATOF + out-of-bandresult(); one active invocation per runtime; early-exit via a cancellation-safeaclose()that drains through invocation completion),run_harness.py, a POC-onlynative_recorder.pythat tees the native SDK stream before Relay, andtwo_turn_isolation.py.streaming-poc/hermes/,deepagents/,claude/,codex/— each with realnative-events.jsonl+events.atof.jsonlfrom the same run and afindings.md(native→ATOF diff, measured loss, deltas-vs-terminal, duplicate-rendering risks, recommendation, reproduce steps). Deep Agents additionally includesparallel-*.jsonlcapturing concurrent sibling subagents (two delegations in one message, overlapping scopes, interleaved namespaces).streaming-poc/two-turn-isolation.jsonl— a checked-in artifact demonstrating turn isolation on one persistent runtime (early-exit with unread records + a concurrency guard). This is best-effort (the turn boundary is a 250 ms drain heuristic); the work breakdown flags a positive turn boundary as required for production.streaming-poc/patches/— checked-in, reversiblegit applypatches for the temporary native-tee seam per harness, so captures are reproducible and non-destructive (apply → run to a scratch dir → reverse-apply) rather than hand-editing adapters.streaming-poc/synthesis/— the cross-harness conclusion (v0.1 = raw ATOF pass-through; why normalization is deferred) and a production implementation work breakdown.adapters/,crates/, or the Python SDK: the temporary tee seams live only as the reversible patches above, not in committed adapter code.Known limitations (documented in the POC): turn isolation is best-effort pending a positive turn boundary; there is no in-flight cancellation in v0.1 (
runtime.stop()refuses while a turn is active).Validation
invoke_stream; the native recorder plus the Relay listener produced the committed fixtures. Per-harness reproduce steps (apply the patch, write to a scratch dir, reverse-apply) are in eachfindings.md.ruff checkandruff formatare clean on the prototype Python.validate-change),just test-rust, and native binding checks — this PR touches no product code, only the self-containedstreaming-poc/folder.Where should the reviewer start?
streaming-poc/synthesis/README.mdfor the cross-harness conclusion and the v0.1 recommendation, then a gateway findings (streaming-poc/claude/findings.mdorcodex/findings.md) for a measured native→ATOF diff. The prototype's streaming contract isstreaming-poc/common/fabric_stream.py.Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)
Relates to: none (exploratory POC; no public issue).
I confirm this contribution is my own work, or I have the right to submit it under this project's license.
I searched existing issues and open pull requests, and this does not duplicate existing work.
Summary by CodeRabbit
New Features
RunResultout of band.Documentation
events.atof.jsonl/native-events.jsonltraces.Chores
.gitignoreto exclude generated runtime artifacts and caches.