🌟 Nova: [Jupyter Exporter]#1098
Conversation
…rajectories Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
|
👋 Jules, reporting for duty! I'm here to lend a hand with this pull request. When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down. I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job! For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with New to Jules? Learn more at jules.google/docs. For security, I will only act on instructions from the user who triggered this task. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: be29f86cfc
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| #[cfg(feature = "jupyter-export")] | ||
| formats.push(ExportFormat { | ||
| name: "jupyter", | ||
| tier: StabilityTier::Experimental, | ||
| consumer: "Jupyter notebooks for interactive analysis", | ||
| render: JupyterExporter::export, |
There was a problem hiding this comment.
When bench inspect --format jupyter --output <sweep>/trace.ipynb is used, agent redact-audit <sweep> silently skips that output: src/run/redact_audit.rs only allowlists .md, .html, .csv, and .mermaid exports (and only schema-sniffs files ending in .json). This leaves the newly supported, shareable notebook outside the documented post-hoc secret-leak gate, so any missed/unanticipated redaction pattern in a notebook can be published without an audit finding. Add .ipynb to the audited export suffixes (and the audit spec/tests).
Useful? React with 👍 / 👎.
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a new jupyter-export feature-gated format to export trajectory logs as Jupyter notebooks (.ipynb). It updates documentation, CLI arguments, and adds the JupyterExporter implementation along with corresponding unit tests. The review feedback suggests improving the notebook export format by splitting cell source content into an array of individual lines to match standard Jupyter conventions and prevent noisy git diffs, as well as updating the unit tests to support this format.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| cells.push(serde_json::json!({ | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [header] | ||
| })); | ||
|
|
||
| for msg in &trajectory.messages { | ||
| let role_title = match msg.role.as_str() { | ||
| "system" => "System", | ||
| "user" => "User", | ||
| "assistant" => "Assistant", | ||
| "tool" => "Tool", | ||
| other => other, | ||
| }; | ||
|
|
||
| let content = redactor.redact_text(&msg.content, surface::EXPORT).text; | ||
| let source = format!("### {role_title}\n\n{content}"); | ||
| cells.push(serde_json::json!({ | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [source] | ||
| })); |
There was a problem hiding this comment.
In Jupyter Notebook files (.ipynb), the source field of a cell is conventionally represented as an array of individual lines (each ending with a newline character \n), rather than a single multiline string wrapped in a single-element array.
While some Jupyter clients can parse a single multiline string, any standard client (such as JupyterLab, VS Code, or classic Jupyter Notebooks) will automatically rewrite the source field into an array of lines upon saving. This causes massive, noisy diffs in version control when users open and save the exported notebook.
Using split_inclusive('\n') ensures that the exported notebook matches the standard Jupyter format out of the box, preventing unnecessary git diff noise.
cells.push(serde_json::json!({
"cell_type": "markdown",
"metadata": {},
"source": header.split_inclusive('\n').collect::<Vec<_>>()
}));
for msg in &trajectory.messages {
let role_title = match msg.role.as_str() {
"system" => "System",
"user" => "User",
"assistant" => "Assistant",
"tool" => "Tool",
other => other,
};
let content = redactor.redact_text(&msg.content, surface::EXPORT).text;
let source = format!("### {role_title}\n\n{content}");
cells.push(serde_json::json!({
"cell_type": "markdown",
"metadata": {},
"source": source.split_inclusive('\n').collect::<Vec<_>>()
}));
}| assert_eq!(parsed["nbformat"], 4); | ||
| let cells = parsed["cells"].as_array().unwrap(); | ||
| assert_eq!(cells.len(), 4); | ||
|
|
||
| assert_eq!(cells[0]["cell_type"], "markdown"); | ||
| let source0 = cells[0]["source"].as_array().unwrap()[0].as_str().unwrap(); | ||
| assert!(source0.contains("Trajectory Export")); | ||
| assert!(source0.contains("Add a feature")); | ||
| assert!(source0.contains("submitted")); | ||
|
|
||
| assert_eq!(cells[1]["cell_type"], "markdown"); | ||
| let source1 = cells[1]["source"].as_array().unwrap()[0].as_str().unwrap(); | ||
| assert!(source1.contains("System prompt; echo 1 >&2")); | ||
|
|
||
| assert_eq!(cells[2]["cell_type"], "markdown"); | ||
| let source2 = cells[2]["source"].as_array().unwrap()[0].as_str().unwrap(); | ||
| assert!(source2.contains("Hello agent\nMulti-line")); | ||
|
|
||
| assert_eq!(cells[3]["cell_type"], "markdown"); | ||
| let source3 = cells[3]["source"].as_array().unwrap()[0].as_str().unwrap(); | ||
| assert!(source3.contains("Hello \"user\"")); |
There was a problem hiding this comment.
Update the unit test assertions to handle the line-split source format robustly. Using a helper closure to join the lines of the source array ensures the tests remain clean, readable, and fully compatible with the standard Jupyter line-split representation.
assert_eq!(parsed["nbformat"], 4);
let cells = parsed["cells"].as_array().unwrap();
assert_eq!(cells.len(), 4);
let get_source = |cell: &serde_json::Value| -> String {
cell["source"]
.as_array()
.unwrap()
.iter()
.map(|v| v.as_str().unwrap())
.collect::<Vec<_>>()
.join("")
};
assert_eq!(cells[0]["cell_type"], "markdown");
let source0 = get_source(&cells[0]);
assert!(source0.contains("Trajectory Export"));
assert!(source0.contains("Add a feature"));
assert!(source0.contains("submitted"));
assert_eq!(cells[1]["cell_type"], "markdown");
let source1 = get_source(&cells[1]);
assert!(source1.contains("System prompt; echo 1 >&2"));
assert_eq!(cells[2]["cell_type"], "markdown");
let source2 = get_source(&cells[2]);
assert!(source2.contains("Hello agent\nMulti-line"));
assert_eq!(cells[3]["cell_type"], "markdown");
let source3 = get_source(&cells[3]);
assert!(source3.contains("Hello \"user\""));
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## trunk #1098 +/- ##
==========================================
+ Coverage 86.63% 86.64% +0.01%
==========================================
Files 139 139
Lines 86826 86896 +70
==========================================
+ Hits 75220 75290 +70
Misses 11606 11606 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
1 participant
💡 The Spark: We already render trajectories to Markdown and HTML. However, Data Scientists and ML Engineers heavily use Jupyter Notebooks. It would be incredibly powerful if we could natively export trajectories directly into a runnable Jupyter Notebook format, allowing seamless interactive analysis, prompting tweaks, and documentation alongside the execution trace!
🚀 The Feature: Implemented a new
JupyterExporterbehind thejupyter-exportCargo feature. It seamlessly translates aTrajectoryinto the.ipynb(Jupyter v4) JSON schema. System, User, Assistant, and Tool roles are correctly formatted into distinct Markdown cells, maintaining the strict redaction invariant (usingRedactor::default_enabled()) required for exports. Added extensive unit testing and updated the documentation format catalog and CLI flags.🔮 The Potential: This bridges the gap between agent run analysis and the primary environment ML engineers use. Users can instantly export a failed run into a Notebook, append cells to run evaluation scripts against the
ToolorAssistantoutputs, visualize costs with matplotlib below the cells, or rapidly re-prompt by mutating the parsed trajectory right in the browser!src/trajectory/export.rs, purely additive, relies on the already-presentserde_jsonfor serialization, and is gated behind a disabled-by-default Cargo feature. All tests and required redaction verifications pass perfectly.PR created automatically by Jules for task 6509334686449212086 started by @madmax983