From 417da191e602dbc4c599dc43a18ad820f1b3e0d0 Mon Sep 17 00:00:00 2001 From: Vinicius Zein Date: Thu, 9 Apr 2026 10:09:46 -0400 Subject: [PATCH 01/10] Add Renode emulator driver for embedded target simulation Introduce jumpstarter-driver-renode, a composite driver that enables Renode-based virtual hardware targets in Jumpstarter. Users can define any Renode-supported platform (STM32, S32K, Nucleo H753ZI, etc.) via exporter YAML configuration without modifying driver code. Key components: - RenodeMonitor: async telnet client for the Renode monitor interface - RenodePower: manages Renode process lifecycle and simulation control - RenodeFlasher: firmware loading via sysbus LoadELF / LoadBinary - RenodeClient: composite client with CLI extension for monitor commands Includes ADR-0001 documenting architectural decisions (control interface, UART exposure, configuration model, firmware loading strategy). Made-with: Cursor --- .../adr/0001-renode-integration.md | 154 +++++ python/docs/source/contributing/adr/index.md | 24 + .../reference/package-apis/drivers/renode.md | 1 + .../jumpstarter-driver-renode/.gitignore | 5 + .../jumpstarter-driver-renode/README.md | 125 ++++ .../examples/exporter.yaml | 64 ++ .../jumpstarter_driver_renode/__init__.py | 0 .../jumpstarter_driver_renode/client.py | 33 ++ .../jumpstarter_driver_renode/driver.py | 229 ++++++++ .../jumpstarter_driver_renode/driver_test.py | 555 ++++++++++++++++++ .../jumpstarter_driver_renode/monitor.py | 129 ++++ .../jumpstarter-driver-renode/pyproject.toml | 59 ++ 12 files changed, 1378 insertions(+) create mode 100644 python/docs/source/contributing/adr/0001-renode-integration.md create mode 100644 python/docs/source/contributing/adr/index.md create mode 120000 python/docs/source/reference/package-apis/drivers/renode.md create mode 100644 python/packages/jumpstarter-driver-renode/.gitignore create mode 100644 python/packages/jumpstarter-driver-renode/README.md create mode 100644 python/packages/jumpstarter-driver-renode/examples/exporter.yaml create mode 100644 python/packages/jumpstarter-driver-renode/jumpstarter_driver_renode/__init__.py create mode 100644 python/packages/jumpstarter-driver-renode/jumpstarter_driver_renode/client.py create mode 100644 python/packages/jumpstarter-driver-renode/jumpstarter_driver_renode/driver.py create mode 100644 python/packages/jumpstarter-driver-renode/jumpstarter_driver_renode/driver_test.py create mode 100644 python/packages/jumpstarter-driver-renode/jumpstarter_driver_renode/monitor.py create mode 100644 python/packages/jumpstarter-driver-renode/pyproject.toml diff --git a/python/docs/source/contributing/adr/0001-renode-integration.md b/python/docs/source/contributing/adr/0001-renode-integration.md new file mode 100644 index 000000000..faf75fc4f --- /dev/null +++ b/python/docs/source/contributing/adr/0001-renode-integration.md @@ -0,0 +1,154 @@ +# ADR-0001: Renode Integration Approach + +- **Status**: Accepted +- **Date**: 2026-04-06 +- **Authors**: Vinicius Zein + +## Context + +Jumpstarter provides a driver-based framework for interacting with +devices under test, both physical hardware and virtual systems. The +existing QEMU driver enables Linux-class virtual targets (aarch64, +x86_64) using full-system emulation with virtio devices and cloud-init +provisioning. + +There is growing demand for **microcontroller-class** virtual targets +running bare-metal firmware or RTOS (Zephyr, FreeRTOS, ThreadX) on +Cortex-M and RISC-V MCUs. [Renode](https://renode.io/) by Antmicro is +an open-source emulation framework designed specifically for this +domain, with extensive peripheral models for STM32, NXP S32K, Nordic, +SiFive, and other MCU platforms. + +The initial reference targets for validation are: + +- **STM32F407 Discovery** (Cortex-M4F) -- opensomeip FreeRTOS/ThreadX + ports, Renode built-in platform +- **NXP S32K388** (Cortex-M7) -- opensomeip Zephyr port, custom + platform description +- **Nucleo H753ZI** (Cortex-M7) -- openbsw-zephyr, Renode built-in + `stm32h743.repl` + +### Forces + +- The driver must follow Jumpstarter's established composite driver + pattern (as demonstrated by `jumpstarter-driver-qemu`) +- Users must be able to define new Renode targets through configuration + alone, without modifying driver code +- The solution should minimize external dependencies and runtime + requirements +- The UART/console interface must be compatible with Jumpstarter's + existing `PySerial` and `pexpect` tooling +- The async framework must be `anyio` (the project's standard) + +## Decisions + +### DD-1: Control Interface -- Telnet Monitor + +**Alternatives considered:** + +1. **Telnet monitor** -- Renode's built-in TCP monitor interface. + Simple socket connection, send text commands, read responses. + Lightweight, no extra runtime needed. +2. **pyrenode3** -- Python.NET bridge to Renode's C# internals. More + powerful but requires .NET runtime or Mono, heavy dependency, less + stable API surface. + +**Decision:** Telnet monitor. + +**Rationale:** It is the lowest-common-denominator interface that works +with any Renode installation. It mirrors the QEMU driver's pattern +where `Popen` starts the emulator process and a side-channel protocol +(QMP for QEMU, telnet monitor for Renode) provides programmatic +control. The monitor client uses `anyio.connect_tcp` with +`anyio.fail_after` for timeouts, consistent with `TcpNetwork` and +`grpc.py` in the project. No `telnetlib`, `telnetlib3`, or +`asynctelnet` is introduced since these are not used anywhere in the +project. + +### DD-2: UART Exposure -- PTY Terminal + +**Alternatives considered:** + +1. **PTY** (`emulation CreateUartPtyTerminal`) -- Creates a + pseudo-terminal file on the host. Reuses the existing `PySerial` + child driver exactly as QEMU does. Linux/macOS only. +2. **Socket** (`emulation CreateServerSocketTerminal`) -- Exposes UART + as a TCP socket. Cross-platform. Maps to `TcpNetwork` driver. Has + telnet IAC negotiation bytes to handle. + +**Decision:** PTY as the primary interface. + +**Rationale:** Consistency with the QEMU driver, which uses `-serial +pty` and wires a `PySerial` child driver to the discovered PTY path. +This reuses the same serial/pexpect/console tooling without any +adaptation. Socket terminal support can be added later as a fallback +for platforms without PTY support. + +### DD-3: Configuration Model -- Managed Mode + +**Alternatives considered:** + +1. **Managed mode** -- The driver constructs all Renode monitor + commands from YAML config parameters (`platform`, `uart`, firmware + path). The driver handles platform loading, UART wiring, and + firmware loading programmatically. +2. **Script mode** -- User provides a complete `.resc` script. The + driver runs it but still manages UART terminal setup. + +**Decision:** Managed mode as primary, with an `extra_commands` list +for target-specific customization. + +**Rationale:** Managed mode gives the driver full control over the UART +terminal setup (which must use PTY for Jumpstarter integration, not the +`CreateFileBackend` or `showAnalyzer` used in typical `.resc` scripts). +The `extra_commands` list covers target-specific needs like register +pokes (e.g., `sysbus WriteDoubleWord 0x40090030 0x0301` for S32K388 +PL011 UART enablement) and Ethernet switch setup. The opensomeip `.resc` +files are CI-oriented and their setup maps directly to managed-mode +config parameters. + +### DD-4: Firmware Loading -- Deferred to Flash + +**Alternatives considered:** + +1. `flash()` stores the firmware path, `on()` loads it into the + simulation and starts +2. `on()` starts the simulation, `flash()` loads firmware and resets + +**Decision:** Option 1 -- `flash()` stores the path, `on()` loads and +starts. + +**Rationale:** This matches the QEMU driver's semantic where you flash +a disk image first, then power on. It also allows re-flashing between +power cycles without restarting the Renode process. The `RenodeFlasher` +additionally supports hot-loading: if the simulation is already running, +`flash()` sends the `sysbus LoadELF` command and resets the machine. + +## Consequences + +### Positive + +- Single `jumpstarter-driver-renode` package supports any Renode target + through YAML configuration alone +- No .NET runtime or Mono dependency required +- Consistent user experience with the QEMU driver (same composite + pattern, same console/pexpect workflow) +- `extra_commands` provides an escape hatch for target-specific + customization without code changes + +### Negative + +- PTY-only UART exposure limits to Linux/macOS (acceptable since Renode + itself primarily targets these platforms) +- The telnet monitor protocol is text-based and less structured than + QMP's JSON -- error detection requires string matching +- Full `.resc` script support is deferred; users with complex Renode + setups must express their configuration as managed-mode parameters + plus `extra_commands` + +### Risks + +- Renode's monitor protocol has no formal specification; prompt + detection and error handling rely on observed behavior +- Renode's PTY terminal support on macOS may have edge cases not + covered in testing diff --git a/python/docs/source/contributing/adr/index.md b/python/docs/source/contributing/adr/index.md new file mode 100644 index 000000000..a5ad4ee52 --- /dev/null +++ b/python/docs/source/contributing/adr/index.md @@ -0,0 +1,24 @@ +# Architecture Decision Records + +This directory contains Architecture Decision Records (ADRs) for the +Jumpstarter project. ADRs document significant technical decisions, +their context, and consequences. + +## Format + +Each ADR follows a standard structure: + +- **Status**: Proposed, Accepted, Deprecated, or Superseded +- **Context**: The forces at play, including technical, political, and + project-specific constraints +- **Decision**: The change being proposed or decided +- **Consequences**: What follows from the decision, both positive and + negative + +## Records + +```{toctree} +:maxdepth: 1 + +0001-renode-integration +``` diff --git a/python/docs/source/reference/package-apis/drivers/renode.md b/python/docs/source/reference/package-apis/drivers/renode.md new file mode 120000 index 000000000..87f897f26 --- /dev/null +++ b/python/docs/source/reference/package-apis/drivers/renode.md @@ -0,0 +1 @@ +../../../../../packages/jumpstarter-driver-renode/README.md \ No newline at end of file diff --git a/python/packages/jumpstarter-driver-renode/.gitignore b/python/packages/jumpstarter-driver-renode/.gitignore new file mode 100644 index 000000000..73102fafe --- /dev/null +++ b/python/packages/jumpstarter-driver-renode/.gitignore @@ -0,0 +1,5 @@ +__pycache__/ +.coverage +coverage.xml +htmlcov/ +.pytest_cache/ diff --git a/python/packages/jumpstarter-driver-renode/README.md b/python/packages/jumpstarter-driver-renode/README.md new file mode 100644 index 000000000..3d20bb8a9 --- /dev/null +++ b/python/packages/jumpstarter-driver-renode/README.md @@ -0,0 +1,125 @@ +# Renode driver + +`jumpstarter-driver-renode` provides a Jumpstarter driver for the +[Renode](https://renode.io/) embedded systems emulation framework. It +enables microcontroller-class virtual targets (Cortex-M, RISC-V MCUs) +running bare-metal firmware or RTOS as Jumpstarter test targets. + +## Installation + +```{code-block} console +:substitutions: +$ pip3 install --extra-index-url {{index_url}} jumpstarter-driver-renode +``` + +Renode must be installed separately and available in `PATH`. See +[Renode installation](https://renode.readthedocs.io/en/latest/introduction/installing.html). + +## Architecture + +The driver follows the composite driver pattern: + +- **`Renode`** -- root composite driver, manages the simulation lifecycle +- **`RenodePower`** -- starts/stops the Renode process and controls the + simulation via the telnet monitor interface +- **`RenodeFlasher`** -- loads firmware (ELF/BIN/HEX) into the simulated MCU +- **`console`** -- UART output via PTY terminal, reusing the `PySerial` driver + +## Configuration + +Users define Renode targets entirely through YAML configuration. No +code changes are needed for new targets. + +### Configuration Parameters + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `platform` | `str` | *(required)* | Path to `.repl` file or Renode built-in platform name | +| `uart` | `str` | `sysbus.uart0` | Peripheral path for the console UART | +| `machine_name` | `str` | `machine-0` | Name of the Renode machine instance | +| `monitor_port` | `int` | `0` (auto) | TCP port for the Renode monitor (0 = auto-assign) | +| `extra_commands` | `list[str]` | `[]` | Additional monitor commands run after platform load | + +### Examples + +#### STM32F407 Discovery (opensomeip FreeRTOS/ThreadX) + +```yaml +export: + ecu: + type: jumpstarter_driver_renode.driver.Renode + config: + platform: "platforms/boards/stm32f4_discovery-kit.repl" + uart: "sysbus.usart2" +``` + +#### NXP S32K388 (opensomeip Zephyr) + +```yaml +export: + ecu: + type: jumpstarter_driver_renode.driver.Renode + config: + platform: "/path/to/s32k388_renode.repl" + uart: "sysbus.uart0" + extra_commands: + - "sysbus WriteDoubleWord 0x40090030 0x0301" +``` + +#### Nucleo H753ZI (openbsw-zephyr) + +```yaml +export: + ecu: + type: jumpstarter_driver_renode.driver.Renode + config: + platform: "platforms/cpus/stm32h743.repl" + uart: "sysbus.usart3" +``` + +## Usage + +### Programmatic (pytest) + +```python +from jumpstarter_driver_renode.driver import Renode +from jumpstarter.common.utils import serve + +with serve( + Renode( + platform="platforms/boards/stm32f4_discovery-kit.repl", + uart="sysbus.usart2", + ) +) as renode: + renode.flasher.flash("/path/to/firmware.elf") + renode.power.on() + + with renode.console.pexpect() as p: + p.expect("Hello from MCU", timeout=30) + + renode.power.off() +``` + +### Monitor Commands + +Send arbitrary Renode monitor commands via the client: + +```python +response = renode.monitor_cmd("sysbus GetRegistrationPoints sysbus.usart2") +``` + +The `monitor` CLI subcommand is also available inside a `jmp shell` session. + +## Design Decisions + +The architectural choices for this driver are documented in +[ADR-0001: Renode Integration Approach](../../docs/source/contributing/adr/0001-renode-integration.md). + +Key decisions: + +- **Control interface**: Telnet monitor via `anyio.connect_tcp` (no + pyrenode3 / .NET dependency) +- **UART exposure**: PTY terminal reusing `PySerial` (consistent with QEMU) +- **Configuration model**: Managed mode with `extra_commands` for + target-specific customization +- **Firmware loading**: `flash()` stores path, `on()` loads into simulation diff --git a/python/packages/jumpstarter-driver-renode/examples/exporter.yaml b/python/packages/jumpstarter-driver-renode/examples/exporter.yaml new file mode 100644 index 000000000..4180a5985 --- /dev/null +++ b/python/packages/jumpstarter-driver-renode/examples/exporter.yaml @@ -0,0 +1,64 @@ +# Renode driver exporter configuration examples +# +# Each example shows a different Renode target. The driver accepts any +# .repl platform description (built-in or custom) and any UART peripheral +# path -- new targets require only YAML configuration, no code changes. + +# --- STM32F407 Discovery (opensomeip FreeRTOS/ThreadX) --- +# Uses Renode's built-in platform, USART2 for console output. +# +# apiVersion: jumpstarter.dev/v1alpha1 +# kind: ExporterConfig +# metadata: +# name: renode-stm32f407 +# export: +# ecu: +# type: jumpstarter_driver_renode.driver.Renode +# config: +# platform: "platforms/boards/stm32f4_discovery-kit.repl" +# uart: "sysbus.usart2" + +# --- NXP S32K388 (opensomeip Zephyr) --- +# Uses a custom .repl and extra_commands to enable the PL011 UART. +# +# apiVersion: jumpstarter.dev/v1alpha1 +# kind: ExporterConfig +# metadata: +# name: renode-s32k388 +# export: +# ecu: +# type: jumpstarter_driver_renode.driver.Renode +# config: +# platform: "/path/to/s32k388_renode.repl" +# uart: "sysbus.uart0" +# extra_commands: +# - "sysbus WriteDoubleWord 0x40090030 0x0301" + +# --- Nucleo H753ZI (openbsw-zephyr) --- +# Uses Renode's built-in STM32H743 platform (close relative), USART3. +# +# apiVersion: jumpstarter.dev/v1alpha1 +# kind: ExporterConfig +# metadata: +# name: renode-nucleo-h753zi +# export: +# ecu: +# type: jumpstarter_driver_renode.driver.Renode +# config: +# platform: "platforms/cpus/stm32h743.repl" +# uart: "sysbus.usart3" + +# Minimal working example (uncomment and fill in token/endpoint): +apiVersion: jumpstarter.dev/v1alpha1 +kind: ExporterConfig +metadata: + name: renode-example + namespace: default +endpoint: grpc.jumpstarter.example.com:443 +token: "" +export: + ecu: + type: jumpstarter_driver_renode.driver.Renode + config: + platform: "platforms/boards/stm32f4_discovery-kit.repl" + uart: "sysbus.usart2" diff --git a/python/packages/jumpstarter-driver-renode/jumpstarter_driver_renode/__init__.py b/python/packages/jumpstarter-driver-renode/jumpstarter_driver_renode/__init__.py new file mode 100644 index 000000000..e69de29bb diff --git a/python/packages/jumpstarter-driver-renode/jumpstarter_driver_renode/client.py b/python/packages/jumpstarter-driver-renode/jumpstarter_driver_renode/client.py new file mode 100644 index 000000000..d6ed94751 --- /dev/null +++ b/python/packages/jumpstarter-driver-renode/jumpstarter_driver_renode/client.py @@ -0,0 +1,33 @@ +import click +from jumpstarter_driver_composite.client import CompositeClient + + +class RenodeClient(CompositeClient): + @property + def platform(self) -> str: + return self.call("get_platform") + + @property + def uart(self) -> str: + return self.call("get_uart") + + @property + def machine_name(self) -> str: + return self.call("get_machine_name") + + def monitor_cmd(self, command: str) -> str: + """Send an arbitrary command to the Renode monitor.""" + return self.call("monitor_cmd", command) + + def cli(self): + base = super().cli() + + @base.command(name="monitor") + @click.argument("command") + def monitor_command(command): + """Send a command to the Renode monitor.""" + result = self.monitor_cmd(command) + if result.strip(): + click.echo(result.strip()) + + return base diff --git a/python/packages/jumpstarter-driver-renode/jumpstarter_driver_renode/driver.py b/python/packages/jumpstarter-driver-renode/jumpstarter_driver_renode/driver.py new file mode 100644 index 000000000..f5e51c557 --- /dev/null +++ b/python/packages/jumpstarter-driver-renode/jumpstarter_driver_renode/driver.py @@ -0,0 +1,229 @@ +from __future__ import annotations + +import logging +import shutil +import socket +from collections.abc import AsyncGenerator +from dataclasses import dataclass, field +from pathlib import Path +from subprocess import PIPE, Popen, TimeoutExpired +from tempfile import TemporaryDirectory + +from anyio.streams.file import FileWriteStream +from jumpstarter_driver_opendal.driver import FlasherInterface +from jumpstarter_driver_power.driver import PowerInterface, PowerReading +from jumpstarter_driver_pyserial.driver import PySerial + +from .monitor import RenodeMonitor +from jumpstarter.driver import Driver, export + +logger = logging.getLogger(__name__) + + +def _find_free_port() -> int: + with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s: + s.bind(("127.0.0.1", 0)) + return s.getsockname()[1] + + +def _find_renode() -> str: + path = shutil.which("renode") + if path is None: + raise FileNotFoundError( + "renode executable not found in PATH. " + "Install Renode from https://renode.io/" + ) + return path + + +@dataclass(kw_only=True) +class RenodeFlasher(FlasherInterface, Driver): + parent: Renode + + @export + async def flash(self, source, load_command: str | None = None): + """Flash firmware to the simulated MCU. + + If the simulation is not yet running, stores the firmware for + loading during power-on. If already running, loads the firmware + and resets the machine. + """ + firmware_path = self.parent._tmp_dir.name + "/firmware" + async with await FileWriteStream.from_path(firmware_path) as stream: + async with self.resource(source) as res: + async for chunk in res: + await stream.send(chunk) + + cmd = load_command or "sysbus LoadELF" + self.parent._firmware_path = firmware_path + self.parent._load_command = cmd + + if hasattr(self.parent.children["power"], "_process"): + monitor = self.parent.children["power"]._monitor + if monitor is not None: + await monitor.execute(f'{cmd} @"{firmware_path}"') + await monitor.execute("machine Reset") + self.logger.info("firmware hot-loaded and machine reset") + + @export + async def dump(self, target, partition: str | None = None): + raise NotImplementedError("dump is not supported for Renode targets") + + +@dataclass(kw_only=True) +class RenodePower(PowerInterface, Driver): + parent: Renode + + _process: Popen | None = field(init=False, default=None, repr=False) + _monitor: RenodeMonitor | None = field(init=False, default=None, repr=False) + + @export + async def on(self) -> None: + if self._process is not None: + self.logger.warning("already powered on, ignoring request") + return + + renode_bin = _find_renode() + port = self.parent.monitor_port or _find_free_port() + self.parent._active_monitor_port = port + + cmdline = [ + renode_bin, + "--disable-xwt", + "--plain", + "--port", + str(port), + ] + + self.logger.info("starting Renode: %s", " ".join(cmdline)) + self._process = Popen(cmdline, stdin=PIPE, stdout=PIPE, stderr=PIPE) + + self._monitor = RenodeMonitor() + await self._monitor.connect("127.0.0.1", port) + + machine = self.parent.machine_name + await self._monitor.execute(f'mach create "{machine}"') + await self._monitor.execute( + f'machine LoadPlatformDescription @"{self.parent.platform}"' + ) + + pty_path = self.parent._pty + await self._monitor.execute( + f'emulation CreateUartPtyTerminal "term" "{pty_path}"' + ) + await self._monitor.execute( + f"connector Connect {self.parent.uart} term" + ) + + for cmd in self.parent.extra_commands: + await self._monitor.execute(cmd) + + if self.parent._firmware_path: + load_cmd = self.parent._load_command or "sysbus LoadELF" + await self._monitor.execute( + f'{load_cmd} @"{self.parent._firmware_path}"' + ) + + await self._monitor.execute("start") + self.logger.info("Renode simulation started") + + @export + async def off(self) -> None: + if self._process is None: + self.logger.warning("already powered off, ignoring request") + return + + if self._monitor is not None: + try: + await self._monitor.execute("quit") + except Exception: + pass + await self._monitor.disconnect() + self._monitor = None + + self._process.terminate() + try: + self._process.wait(timeout=5) + except TimeoutExpired: + self._process.kill() + self._process = None + + @export + async def read(self) -> AsyncGenerator[PowerReading, None]: + raise NotImplementedError + + def close(self): + if self._process is not None: + if self._monitor is not None: + self._monitor = None + self._process.terminate() + try: + self._process.wait(timeout=5) + except TimeoutExpired: + self._process.kill() + self._process = None + + +@dataclass(kw_only=True) +class Renode(Driver): + """Renode emulation framework driver for Jumpstarter. + + Provides a composite driver that manages a Renode simulation instance + with power control, firmware flashing, and serial console access. + + Users inject their Renode target configuration via YAML without + modifying driver code: + + - ``platform``: path to a ``.repl`` file or Renode built-in name + - ``uart``: peripheral path in the Renode object model + - ``extra_commands``: list of monitor commands for target-specific setup + """ + + platform: str + uart: str = "sysbus.uart0" + machine_name: str = "machine-0" + monitor_port: int = 0 + extra_commands: list[str] = field(default_factory=list) + + _tmp_dir: TemporaryDirectory = field( + init=False, default_factory=TemporaryDirectory + ) + _firmware_path: str | None = field(init=False, default=None) + _load_command: str | None = field(init=False, default=None) + _active_monitor_port: int = field(init=False, default=0) + + @classmethod + def client(cls) -> str: + return "jumpstarter_driver_renode.client.RenodeClient" + + def __post_init__(self): + if hasattr(super(), "__post_init__"): + super().__post_init__() + + self.children["power"] = RenodePower(parent=self) + self.children["flasher"] = RenodeFlasher(parent=self) + self.children["console"] = PySerial(url=self._pty, check_present=False) + + @property + def _pty(self) -> str: + return str(Path(self._tmp_dir.name) / "pty") + + @export + def get_platform(self) -> str: + return self.platform + + @export + def get_uart(self) -> str: + return self.uart + + @export + def get_machine_name(self) -> str: + return self.machine_name + + @export + async def monitor_cmd(self, command: str) -> str: + """Send an arbitrary command to the Renode monitor.""" + power: RenodePower = self.children["power"] + if power._monitor is None: + raise RuntimeError("Renode is not running") + return await power._monitor.execute(command) diff --git a/python/packages/jumpstarter-driver-renode/jumpstarter_driver_renode/driver_test.py b/python/packages/jumpstarter-driver-renode/jumpstarter_driver_renode/driver_test.py new file mode 100644 index 000000000..7a61ac706 --- /dev/null +++ b/python/packages/jumpstarter-driver-renode/jumpstarter_driver_renode/driver_test.py @@ -0,0 +1,555 @@ +from __future__ import annotations + +import os +import platform +import shutil +from pathlib import Path +from subprocess import TimeoutExpired +from unittest.mock import AsyncMock, MagicMock, patch + +import pytest + +from jumpstarter_driver_renode.driver import Renode, RenodeFlasher, RenodePower +from jumpstarter_driver_renode.monitor import RenodeMonitor + +from jumpstarter.common.utils import serve + + +@pytest.fixture +def anyio_backend(): + return "asyncio" + + +# --------------------------------------------------------------------------- +# 5.1 RenodeMonitor unit tests +# --------------------------------------------------------------------------- + + +class TestRenodeMonitor: + @pytest.mark.anyio + async def test_monitor_connect_retry(self): + """Monitor retries on OSError until connection succeeds.""" + monitor = RenodeMonitor() + call_count = 0 + + async def mock_connect_tcp(host, port): + nonlocal call_count + call_count += 1 + if call_count < 3: + raise OSError("Connection refused") + stream = AsyncMock() + stream.receive = AsyncMock( + return_value=b"Renode v1.15\n(monitor) \n" + ) + return stream + + with patch( + "jumpstarter_driver_renode.monitor.connect_tcp", + side_effect=mock_connect_tcp, + ): + with patch( + "jumpstarter_driver_renode.monitor.sleep", new_callable=AsyncMock + ): + await monitor.connect("127.0.0.1", 12345) + + assert call_count == 3 + assert monitor._stream is not None + + @pytest.mark.anyio + async def test_monitor_execute_command(self): + """Execute sends command and returns response text.""" + monitor = RenodeMonitor() + stream = AsyncMock() + responses = iter( + [b"some output\n(monitor) \n", b""] + ) + stream.receive = AsyncMock(side_effect=lambda size: next(responses)) + monitor._stream = stream + monitor._buffer = b"" + + result = await monitor.execute("mach create") + stream.send.assert_called_once_with(b"mach create\n") + assert "some output" in result + + @pytest.mark.anyio + async def test_monitor_execute_error_response(self): + """Monitor returns error text from Renode without raising.""" + monitor = RenodeMonitor() + stream = AsyncMock() + stream.receive = AsyncMock( + return_value=b"Could not find peripheral\n(monitor) \n" + ) + monitor._stream = stream + monitor._buffer = b"" + + result = await monitor.execute("bad command") + assert "Could not find peripheral" in result + + @pytest.mark.anyio + async def test_monitor_execute_not_connected(self): + """Execute raises RuntimeError when not connected.""" + monitor = RenodeMonitor() + with pytest.raises(RuntimeError, match="not connected"): + await monitor.execute("help") + + @pytest.mark.anyio + async def test_monitor_disconnect(self): + """Disconnect closes stream and is idempotent.""" + monitor = RenodeMonitor() + stream = AsyncMock() + monitor._stream = stream + + await monitor.disconnect() + stream.aclose.assert_called_once() + assert monitor._stream is None + + await monitor.disconnect() + + @pytest.mark.anyio + async def test_monitor_disconnect_ignores_errors(self): + """Disconnect handles errors during close gracefully.""" + monitor = RenodeMonitor() + stream = AsyncMock() + stream.aclose = AsyncMock(side_effect=OSError("already closed")) + monitor._stream = stream + + await monitor.disconnect() + assert monitor._stream is None + + +# --------------------------------------------------------------------------- +# 5.2 RenodePower unit tests +# --------------------------------------------------------------------------- + + +def _make_driver(**kwargs) -> Renode: + defaults = {"platform": "platforms/boards/stm32f4_discovery-kit.repl"} + defaults.update(kwargs) + return Renode(**defaults) + + +class TestRenodePower: + @pytest.mark.anyio + async def test_power_on_command_sequence(self): + """Verify the exact sequence of monitor commands during power on.""" + driver = _make_driver(uart="sysbus.usart2") + driver._firmware_path = "/tmp/test.elf" + driver._load_command = "sysbus LoadELF" + power: RenodePower = driver.children["power"] + + mock_monitor = AsyncMock(spec=RenodeMonitor) + + with patch( + "jumpstarter_driver_renode.driver._find_renode", + return_value="/usr/bin/renode", + ): + with patch( + "jumpstarter_driver_renode.driver._find_free_port", + return_value=54321, + ): + with patch( + "jumpstarter_driver_renode.driver.Popen" + ) as mock_popen: + mock_popen.return_value = MagicMock() + with patch( + "jumpstarter_driver_renode.driver.RenodeMonitor", + return_value=mock_monitor, + ): + await power.on() + + calls = [c.args[0] for c in mock_monitor.execute.call_args_list] + assert calls[0] == 'mach create "machine-0"' + assert "LoadPlatformDescription" in calls[1] + assert "stm32f4_discovery-kit.repl" in calls[1] + assert "CreateUartPtyTerminal" in calls[2] + assert "connector Connect sysbus.usart2 term" == calls[3] + assert "LoadELF" in calls[4] + assert calls[5] == "start" + + @pytest.mark.anyio + async def test_power_on_with_extra_commands(self): + """Extra commands are sent between connector Connect and LoadELF.""" + driver = _make_driver( + extra_commands=["sysbus WriteDoubleWord 0x40090030 0x0301"] + ) + driver._firmware_path = "/tmp/test.elf" + power: RenodePower = driver.children["power"] + mock_monitor = AsyncMock(spec=RenodeMonitor) + + with patch( + "jumpstarter_driver_renode.driver._find_renode", + return_value="/usr/bin/renode", + ): + with patch( + "jumpstarter_driver_renode.driver._find_free_port", + return_value=54321, + ): + with patch( + "jumpstarter_driver_renode.driver.Popen" + ) as mock_popen: + mock_popen.return_value = MagicMock() + with patch( + "jumpstarter_driver_renode.driver.RenodeMonitor", + return_value=mock_monitor, + ): + await power.on() + + calls = [c.args[0] for c in mock_monitor.execute.call_args_list] + connect_idx = next( + i for i, c in enumerate(calls) if "connector Connect" in c + ) + load_idx = next( + i for i, c in enumerate(calls) if "LoadELF" in c + ) + extra_idx = next( + i + for i, c in enumerate(calls) + if "WriteDoubleWord" in c + ) + assert connect_idx < extra_idx < load_idx + + @pytest.mark.anyio + async def test_power_on_without_firmware(self): + """When no firmware is set, LoadELF is skipped but start is sent.""" + driver = _make_driver() + power: RenodePower = driver.children["power"] + mock_monitor = AsyncMock(spec=RenodeMonitor) + + with patch( + "jumpstarter_driver_renode.driver._find_renode", + return_value="/usr/bin/renode", + ): + with patch( + "jumpstarter_driver_renode.driver._find_free_port", + return_value=54321, + ): + with patch( + "jumpstarter_driver_renode.driver.Popen" + ) as mock_popen: + mock_popen.return_value = MagicMock() + with patch( + "jumpstarter_driver_renode.driver.RenodeMonitor", + return_value=mock_monitor, + ): + await power.on() + + calls = [c.args[0] for c in mock_monitor.execute.call_args_list] + assert not any("LoadELF" in c for c in calls) + assert calls[-1] == "start" + + @pytest.mark.anyio + async def test_power_on_idempotent(self): + """Second on() call logs warning and does nothing.""" + driver = _make_driver() + power: RenodePower = driver.children["power"] + power._process = MagicMock() + + await power.on() + + @pytest.mark.anyio + async def test_power_off_terminates_process(self): + """off() terminates the process, waits, then kills on timeout.""" + driver = _make_driver() + power: RenodePower = driver.children["power"] + + mock_process = MagicMock() + mock_process.terminate = MagicMock() + mock_process.wait = MagicMock(side_effect=TimeoutExpired("renode", 5)) + mock_process.kill = MagicMock() + power._process = mock_process + + mock_monitor = AsyncMock(spec=RenodeMonitor) + power._monitor = mock_monitor + + await power.off() + + mock_monitor.execute.assert_called_with("quit") + mock_monitor.disconnect.assert_called_once() + mock_process.terminate.assert_called_once() + mock_process.kill.assert_called_once() + assert power._process is None + + @pytest.mark.anyio + async def test_power_off_clean_shutdown(self): + """off() with clean process exit does not call kill().""" + driver = _make_driver() + power: RenodePower = driver.children["power"] + + mock_process = MagicMock() + mock_process.wait = MagicMock() + power._process = mock_process + power._monitor = AsyncMock(spec=RenodeMonitor) + + await power.off() + + mock_process.terminate.assert_called_once() + mock_process.kill.assert_not_called() + assert power._process is None + + @pytest.mark.anyio + async def test_power_off_idempotent(self): + """Second off() call logs warning and does nothing.""" + driver = _make_driver() + power: RenodePower = driver.children["power"] + power._process = None + + await power.off() + + @pytest.mark.anyio + async def test_power_close_calls_off(self): + """close() terminates the process.""" + driver = _make_driver() + power: RenodePower = driver.children["power"] + mock_process = MagicMock() + mock_process.wait = MagicMock() + power._process = mock_process + + power.close() + + mock_process.terminate.assert_called_once() + assert power._process is None + + +# --------------------------------------------------------------------------- +# 5.3 RenodeFlasher unit tests +# --------------------------------------------------------------------------- + + +class TestRenodeFlasher: + @pytest.mark.anyio + async def test_flash_stores_firmware_path(self, tmp_path): + """flash() writes firmware to temp dir and stores the path.""" + driver = _make_driver() + + firmware_data = b"\x00" * 64 + firmware_file = tmp_path / "test.elf" + firmware_file.write_bytes(firmware_data) + + flasher: RenodeFlasher = driver.children["flasher"] + + with patch.object(flasher, "resource") as mock_resource: + mock_res = AsyncMock() + mock_res.__aiter__ = AsyncMock( + return_value=iter([firmware_data]) + ) + mock_resource.return_value.__aenter__ = AsyncMock( + return_value=mock_res + ) + mock_resource.return_value.__aexit__ = AsyncMock() + + await flasher.flash(str(firmware_file)) + + assert driver._firmware_path is not None + assert Path(driver._firmware_path).name == "firmware" + + @pytest.mark.anyio + async def test_flash_while_running_sends_load_and_reset(self): + """When simulation is running, flash() sends LoadELF + Reset.""" + driver = _make_driver() + power: RenodePower = driver.children["power"] + power._process = MagicMock() + mock_monitor = AsyncMock(spec=RenodeMonitor) + power._monitor = mock_monitor + + flasher: RenodeFlasher = driver.children["flasher"] + + with patch.object(flasher, "resource") as mock_resource: + mock_res = AsyncMock() + mock_res.__aiter__ = AsyncMock(return_value=iter([b"\x00"])) + mock_resource.return_value.__aenter__ = AsyncMock( + return_value=mock_res + ) + mock_resource.return_value.__aexit__ = AsyncMock() + + await flasher.flash("/some/firmware.elf") + + calls = [c.args[0] for c in mock_monitor.execute.call_args_list] + assert any("LoadELF" in c for c in calls) + assert any("Reset" in c for c in calls) + + @pytest.mark.anyio + async def test_flash_custom_load_command(self): + """flash() uses custom load_command when provided.""" + driver = _make_driver() + flasher: RenodeFlasher = driver.children["flasher"] + + with patch.object(flasher, "resource") as mock_resource: + mock_res = AsyncMock() + mock_res.__aiter__ = AsyncMock(return_value=iter([b"\x00"])) + mock_resource.return_value.__aenter__ = AsyncMock( + return_value=mock_res + ) + mock_resource.return_value.__aexit__ = AsyncMock() + + await flasher.flash( + "/some/firmware.bin", + load_command="sysbus LoadBinary", + ) + + assert driver._load_command == "sysbus LoadBinary" + + @pytest.mark.anyio + async def test_dump_not_implemented(self): + """dump() raises NotImplementedError.""" + driver = _make_driver() + flasher: RenodeFlasher = driver.children["flasher"] + + with pytest.raises(NotImplementedError, match="not supported"): + await flasher.dump("/dev/null") + + +# --------------------------------------------------------------------------- +# 5.4 Configuration validation tests +# --------------------------------------------------------------------------- + + +class TestRenodeConfig: + def test_renode_defaults(self): + """Default values are set correctly.""" + driver = _make_driver() + assert driver.uart == "sysbus.uart0" + assert driver.machine_name == "machine-0" + assert driver.monitor_port == 0 + assert driver.extra_commands == [] + assert driver._firmware_path is None + + def test_renode_children_wired(self): + """Children drivers are wired correctly.""" + driver = _make_driver() + assert "power" in driver.children + assert "flasher" in driver.children + assert "console" in driver.children + assert isinstance(driver.children["power"], RenodePower) + assert isinstance(driver.children["flasher"], RenodeFlasher) + + def test_renode_custom_config(self): + """Custom config values are applied.""" + driver = _make_driver( + uart="sysbus.usart2", + machine_name="my-machine", + monitor_port=9999, + extra_commands=["command1", "command2"], + ) + assert driver.uart == "sysbus.usart2" + assert driver.machine_name == "my-machine" + assert driver.monitor_port == 9999 + assert driver.extra_commands == ["command1", "command2"] + + def test_renode_pty_path(self): + """PTY path is inside the temp directory.""" + driver = _make_driver() + assert driver._pty.endswith("/pty") + assert driver._tmp_dir.name in driver._pty + + def test_renode_temp_directory_lifecycle(self): + """TemporaryDirectory is created and can be cleaned up.""" + driver = _make_driver() + tmp_path = driver._tmp_dir.name + assert Path(tmp_path).exists() + driver._tmp_dir.cleanup() + assert not Path(tmp_path).exists() + + def test_renode_get_platform(self): + """get_platform returns the platform path.""" + driver = _make_driver() + assert driver.get_platform() == "platforms/boards/stm32f4_discovery-kit.repl" + + def test_renode_get_uart(self): + """get_uart returns the UART peripheral path.""" + driver = _make_driver(uart="sysbus.usart3") + assert driver.get_uart() == "sysbus.usart3" + + def test_renode_get_machine_name(self): + """get_machine_name returns the machine name.""" + driver = _make_driver(machine_name="test-mcu") + assert driver.get_machine_name() == "test-mcu" + + +# --------------------------------------------------------------------------- +# 5.5 E2E test with serve() +# --------------------------------------------------------------------------- + + +@pytest.mark.skipif( + shutil.which("renode") is None, + reason="Renode not installed", +) +@pytest.mark.xfail( + platform.system() == "Darwin" and os.getenv("GITHUB_ACTIONS") == "true", + reason="Renode tests may be flaky on macOS CI", +) +def test_driver_renode_e2e(tmp_path): + """E2E: start Renode, verify power on/off cycle via serve().""" + with serve( + Renode( + platform="platforms/boards/stm32f4_discovery-kit.repl", + uart="sysbus.usart2", + ) + ) as renode: + assert renode.platform == "platforms/boards/stm32f4_discovery-kit.repl" + assert renode.uart == "sysbus.usart2" + assert renode.machine_name == "machine-0" + + renode.power.on() + renode.power.off() + + +# --------------------------------------------------------------------------- +# 5.6 Client and CLI tests +# --------------------------------------------------------------------------- + + +class TestRenodeClient: + def test_client_serve_properties(self): + """Client properties round-trip through serve().""" + driver = _make_driver(uart="sysbus.usart2") + + with serve(driver) as client: + assert client.platform == "platforms/boards/stm32f4_discovery-kit.repl" + assert client.uart == "sysbus.usart2" + assert client.machine_name == "machine-0" + + def test_client_children_accessible(self): + """Composite client exposes power, flasher, console children.""" + driver = _make_driver() + + with serve(driver) as client: + assert hasattr(client, "power") + assert hasattr(client, "flasher") + assert hasattr(client, "console") + + def test_client_monitor_cmd_not_running(self): + """monitor_cmd raises when Renode is not running.""" + from jumpstarter.client.core import DriverError + + driver = _make_driver() + + with serve(driver) as client: + with pytest.raises(DriverError, match="not running"): + client.monitor_cmd("help") + + def test_client_cli_renders(self): + """CLI group includes monitor command.""" + from click.testing import CliRunner + + driver = _make_driver() + + with serve(driver) as client: + cli = client.cli() + runner = CliRunner() + result = runner.invoke(cli, ["--help"]) + assert result.exit_code == 0 + assert "monitor" in result.output + + def test_client_cli_monitor_help(self): + """Monitor CLI subcommand shows help.""" + from click.testing import CliRunner + + driver = _make_driver() + + with serve(driver) as client: + cli = client.cli() + runner = CliRunner() + result = runner.invoke(cli, ["monitor", "--help"]) + assert result.exit_code == 0 + assert "COMMAND" in result.output diff --git a/python/packages/jumpstarter-driver-renode/jumpstarter_driver_renode/monitor.py b/python/packages/jumpstarter-driver-renode/jumpstarter_driver_renode/monitor.py new file mode 100644 index 000000000..7633afdc6 --- /dev/null +++ b/python/packages/jumpstarter-driver-renode/jumpstarter_driver_renode/monitor.py @@ -0,0 +1,129 @@ +from __future__ import annotations + +import logging + +from anyio import connect_tcp, fail_after, sleep +from anyio.abc import SocketStream + +logger = logging.getLogger(__name__) + +MONITOR_PROMPT = b"(monitor)" +MACHINE_PROMPT_PREFIX = b"(" +MACHINE_PROMPT_SUFFIX = b")" + + +class RenodeMonitorError(Exception): + """Raised when a Renode monitor command returns an error.""" + + +class RenodeMonitor: + """Async client for Renode's telnet monitor interface. + + Uses anyio.connect_tcp (the project's standard for TCP connections) + to communicate with Renode's built-in monitor port. The protocol is + line-oriented text over TCP. + """ + + _stream: SocketStream | None = None + _buffer: bytes = b"" + + async def connect(self, host: str, port: int, timeout: float = 10) -> None: + """Connect to the Renode monitor, retrying until the prompt appears.""" + with fail_after(timeout): + while True: + try: + self._stream = await connect_tcp(host, port) + self._buffer = b"" + await self._read_until_prompt() + logger.info("connected to Renode monitor at %s:%d", host, port) + return + except OSError: + await sleep(0.5) + + async def execute(self, command: str) -> str: + """Send a command and return the response text (excluding the prompt).""" + if self._stream is None: + raise RuntimeError("not connected to Renode monitor") + + logger.debug("monitor> %s", command) + await self._stream.send(f"{command}\n".encode()) + response = await self._read_until_prompt() + logger.debug("monitor< %s", response.strip()) + return response + + async def disconnect(self) -> None: + """Close the monitor connection.""" + if self._stream is not None: + try: + await self._stream.aclose() + except Exception: + pass + self._stream = None + self._buffer = b"" + + async def _read_until_prompt(self) -> str: + """Read from the stream until a monitor prompt line is detected. + + Returns the text received before the prompt. + """ + if self._stream is None: + raise RuntimeError("not connected to Renode monitor") + + while True: + prompt_pos = self._find_prompt() + if prompt_pos is not None: + text_before = self._buffer[:prompt_pos].decode(errors="replace") + self._buffer = self._buffer[prompt_pos:] + prompt_end = self._buffer.find(b"\n") + if prompt_end >= 0: + self._buffer = self._buffer[prompt_end + 1 :] + else: + self._buffer = b"" + return text_before + + data = await self._stream.receive(4096) + if not data: + raise ConnectionError("Renode monitor connection closed") + self._buffer += data + + def _find_prompt(self) -> int | None: + """Find a Renode monitor prompt in the buffer. + + Renode prompts look like "(monitor) " or "(machine-name) ". + """ + for line_start in self._iter_line_starts(): + line = self._buffer[line_start:] + line_end = line.find(b"\n") + if line_end < 0: + candidate = line + else: + candidate = line[:line_end] + candidate = candidate.rstrip() + if self._is_prompt(candidate): + return line_start + return None + + def _iter_line_starts(self): + """Yield byte offsets where lines begin in the buffer.""" + yield 0 + pos = 0 + while True: + nl = self._buffer.find(b"\n", pos) + if nl < 0: + break + yield nl + 1 + pos = nl + 1 + + @staticmethod + def _is_prompt(line: bytes) -> bool: + """Check if a line looks like a Renode prompt.""" + stripped = line.strip() + if not stripped: + return False + if stripped.startswith(MACHINE_PROMPT_PREFIX) and stripped.endswith( + MACHINE_PROMPT_SUFFIX + ): + inner = stripped[1:-1] + if inner and b")" not in inner: + return True + return False diff --git a/python/packages/jumpstarter-driver-renode/pyproject.toml b/python/packages/jumpstarter-driver-renode/pyproject.toml new file mode 100644 index 000000000..a11fa74d7 --- /dev/null +++ b/python/packages/jumpstarter-driver-renode/pyproject.toml @@ -0,0 +1,59 @@ +[project] +name = "jumpstarter-driver-renode" +dynamic = ["version", "urls"] +description = "Renode emulation framework driver for Jumpstarter" +readme = "README.md" +license = "Apache-2.0" +authors = [ + { name = "Vinicius Zein", email = "vtzein@gmail.com" } +] +requires-python = ">=3.11" +dependencies = [ + "jumpstarter", + "jumpstarter-driver-composite", + "jumpstarter-driver-network", + "jumpstarter-driver-opendal", + "jumpstarter-driver-power", + "jumpstarter-driver-pyserial", +] + +[project.entry-points."jumpstarter.drivers"] +Renode = "jumpstarter_driver_renode.driver:Renode" +RenodePower = "jumpstarter_driver_renode.driver:RenodePower" +RenodeFlasher = "jumpstarter_driver_renode.driver:RenodeFlasher" + +[tool.hatch.version] +source = "vcs" +raw-options = { 'root' = '../../../'} + +[tool.hatch.metadata.hooks.vcs.urls] +Homepage = "https://jumpstarter.dev" +source_archive = "https://github.com/jumpstarter-dev/repo/archive/{commit_hash}.zip" + +[tool.pytest.ini_options] +addopts = "--cov --cov-report=html --cov-report=xml" +log_cli = true +log_cli_level = "INFO" +testpaths = ["jumpstarter_driver_renode"] + +[tool.uv.sources] +jumpstarter = { workspace = true } +jumpstarter-driver-opendal = { workspace = true } +jumpstarter-driver-composite = { workspace = true } +jumpstarter-driver-network = { workspace = true } +jumpstarter-driver-pyserial = { workspace = true } +jumpstarter-driver-power = { workspace = true } + +[build-system] +requires = ["hatchling", "hatch-vcs", "hatch-pin-jumpstarter"] +build-backend = "hatchling.build" + +[tool.hatch.build.hooks.pin_jumpstarter] +name = "pin_jumpstarter" + +[dependency-groups] +dev = [ + "pytest>=8.3.2", + "pytest-cov>=5.0.0", + "pytest-anyio>=0.0.0", +] From c2f3199282c51353e96731e6648213c34b7da4b1 Mon Sep 17 00:00:00 2001 From: Vinicius Zein Date: Sat, 11 Apr 2026 13:31:45 -0400 Subject: [PATCH 02/10] Address CodeRabbit review feedback - Replace PIPE with DEVNULL in Popen to prevent deadlocks (pipes were never read) - Wrap monitor setup in try-except to teardown subprocess on failure, preventing process leaks - Use anyio.to_thread.run_sync for blocking wait() in off() to avoid blocking the event loop - Raise RenodeMonitorError on error responses instead of silently returning error text - Accept multi-word monitor commands in CLI via nargs=-1 - Rename test_power_close_calls_off to test_power_close_terminates_process - Add docstrings across all public APIs Made-with: Cursor --- .../jumpstarter_driver_renode/client.py | 10 +- .../jumpstarter_driver_renode/driver.py | 66 +++++++++----- .../jumpstarter_driver_renode/driver_test.py | 8 +- .../jumpstarter_driver_renode/monitor.py | 12 ++- python/uv.lock | 91 +++++++++++++++++++ 5 files changed, 155 insertions(+), 32 deletions(-) diff --git a/python/packages/jumpstarter-driver-renode/jumpstarter_driver_renode/client.py b/python/packages/jumpstarter-driver-renode/jumpstarter_driver_renode/client.py index d6ed94751..f921a0e0c 100644 --- a/python/packages/jumpstarter-driver-renode/jumpstarter_driver_renode/client.py +++ b/python/packages/jumpstarter-driver-renode/jumpstarter_driver_renode/client.py @@ -3,16 +3,21 @@ class RenodeClient(CompositeClient): + """Client for interacting with a Renode composite driver.""" + @property def platform(self) -> str: + """The Renode platform description path.""" return self.call("get_platform") @property def uart(self) -> str: + """The UART peripheral path in the Renode object model.""" return self.call("get_uart") @property def machine_name(self) -> str: + """The Renode machine name.""" return self.call("get_machine_name") def monitor_cmd(self, command: str) -> str: @@ -20,13 +25,14 @@ def monitor_cmd(self, command: str) -> str: return self.call("monitor_cmd", command) def cli(self): + """Extend the composite CLI with a ``monitor`` subcommand.""" base = super().cli() @base.command(name="monitor") - @click.argument("command") + @click.argument("command", nargs=-1, required=True) def monitor_command(command): """Send a command to the Renode monitor.""" - result = self.monitor_cmd(command) + result = self.monitor_cmd(" ".join(command)) if result.strip(): click.echo(result.strip()) diff --git a/python/packages/jumpstarter-driver-renode/jumpstarter_driver_renode/driver.py b/python/packages/jumpstarter-driver-renode/jumpstarter_driver_renode/driver.py index f5e51c557..50511889e 100644 --- a/python/packages/jumpstarter-driver-renode/jumpstarter_driver_renode/driver.py +++ b/python/packages/jumpstarter-driver-renode/jumpstarter_driver_renode/driver.py @@ -6,9 +6,10 @@ from collections.abc import AsyncGenerator from dataclasses import dataclass, field from pathlib import Path -from subprocess import PIPE, Popen, TimeoutExpired +from subprocess import DEVNULL, Popen, TimeoutExpired from tempfile import TemporaryDirectory +from anyio import to_thread from anyio.streams.file import FileWriteStream from jumpstarter_driver_opendal.driver import FlasherInterface from jumpstarter_driver_power.driver import PowerInterface, PowerReading @@ -67,11 +68,14 @@ async def flash(self, source, load_command: str | None = None): @export async def dump(self, target, partition: str | None = None): + """Not supported for Renode targets.""" raise NotImplementedError("dump is not supported for Renode targets") @dataclass(kw_only=True) class RenodePower(PowerInterface, Driver): + """Power controller that manages the Renode process lifecycle.""" + parent: Renode _process: Popen | None = field(init=False, default=None, repr=False) @@ -79,6 +83,7 @@ class RenodePower(PowerInterface, Driver): @export async def on(self) -> None: + """Start Renode, connect monitor, configure platform, and begin simulation.""" if self._process is not None: self.logger.warning("already powered on, ignoring request") return @@ -96,39 +101,44 @@ async def on(self) -> None: ] self.logger.info("starting Renode: %s", " ".join(cmdline)) - self._process = Popen(cmdline, stdin=PIPE, stdout=PIPE, stderr=PIPE) + self._process = Popen(cmdline, stdin=DEVNULL, stdout=DEVNULL, stderr=DEVNULL) self._monitor = RenodeMonitor() - await self._monitor.connect("127.0.0.1", port) - - machine = self.parent.machine_name - await self._monitor.execute(f'mach create "{machine}"') - await self._monitor.execute( - f'machine LoadPlatformDescription @"{self.parent.platform}"' - ) - - pty_path = self.parent._pty - await self._monitor.execute( - f'emulation CreateUartPtyTerminal "term" "{pty_path}"' - ) - await self._monitor.execute( - f"connector Connect {self.parent.uart} term" - ) + try: + await self._monitor.connect("127.0.0.1", port) - for cmd in self.parent.extra_commands: - await self._monitor.execute(cmd) + machine = self.parent.machine_name + await self._monitor.execute(f'mach create "{machine}"') + await self._monitor.execute( + f'machine LoadPlatformDescription @"{self.parent.platform}"' + ) - if self.parent._firmware_path: - load_cmd = self.parent._load_command or "sysbus LoadELF" + pty_path = self.parent._pty + await self._monitor.execute( + f'emulation CreateUartPtyTerminal "term" "{pty_path}"' + ) await self._monitor.execute( - f'{load_cmd} @"{self.parent._firmware_path}"' + f"connector Connect {self.parent.uart} term" ) - await self._monitor.execute("start") - self.logger.info("Renode simulation started") + for cmd in self.parent.extra_commands: + await self._monitor.execute(cmd) + + if self.parent._firmware_path: + load_cmd = self.parent._load_command or "sysbus LoadELF" + await self._monitor.execute( + f'{load_cmd} @"{self.parent._firmware_path}"' + ) + + await self._monitor.execute("start") + self.logger.info("Renode simulation started") + except Exception: + await self.off() + raise @export async def off(self) -> None: + """Stop simulation, disconnect monitor, and terminate the Renode process.""" if self._process is None: self.logger.warning("already powered off, ignoring request") return @@ -143,16 +153,18 @@ async def off(self) -> None: self._process.terminate() try: - self._process.wait(timeout=5) + await to_thread.run_sync(self._process.wait, 5) except TimeoutExpired: self._process.kill() self._process = None @export async def read(self) -> AsyncGenerator[PowerReading, None]: + """Not supported — Renode does not provide power readings.""" raise NotImplementedError def close(self): + """Synchronous cleanup for use during driver teardown.""" if self._process is not None: if self._monitor is not None: self._monitor = None @@ -194,6 +206,7 @@ class Renode(Driver): @classmethod def client(cls) -> str: + """Return the fully-qualified client class name.""" return "jumpstarter_driver_renode.client.RenodeClient" def __post_init__(self): @@ -210,14 +223,17 @@ def _pty(self) -> str: @export def get_platform(self) -> str: + """Return the Renode platform description path.""" return self.platform @export def get_uart(self) -> str: + """Return the UART peripheral path in the Renode object model.""" return self.uart @export def get_machine_name(self) -> str: + """Return the Renode machine name.""" return self.machine_name @export diff --git a/python/packages/jumpstarter-driver-renode/jumpstarter_driver_renode/driver_test.py b/python/packages/jumpstarter-driver-renode/jumpstarter_driver_renode/driver_test.py index 7a61ac706..6f6c42893 100644 --- a/python/packages/jumpstarter-driver-renode/jumpstarter_driver_renode/driver_test.py +++ b/python/packages/jumpstarter-driver-renode/jumpstarter_driver_renode/driver_test.py @@ -10,7 +10,7 @@ import pytest from jumpstarter_driver_renode.driver import Renode, RenodeFlasher, RenodePower -from jumpstarter_driver_renode.monitor import RenodeMonitor +from jumpstarter_driver_renode.monitor import RenodeMonitor, RenodeMonitorError from jumpstarter.common.utils import serve @@ -73,7 +73,7 @@ async def test_monitor_execute_command(self): @pytest.mark.anyio async def test_monitor_execute_error_response(self): - """Monitor returns error text from Renode without raising.""" + """Monitor raises RenodeMonitorError on error responses.""" monitor = RenodeMonitor() stream = AsyncMock() stream.receive = AsyncMock( @@ -82,8 +82,8 @@ async def test_monitor_execute_error_response(self): monitor._stream = stream monitor._buffer = b"" - result = await monitor.execute("bad command") - assert "Could not find peripheral" in result + with pytest.raises(RenodeMonitorError, match="Could not find peripheral"): + await monitor.execute("bad command") @pytest.mark.anyio async def test_monitor_execute_not_connected(self): diff --git a/python/packages/jumpstarter-driver-renode/jumpstarter_driver_renode/monitor.py b/python/packages/jumpstarter-driver-renode/jumpstarter_driver_renode/monitor.py index 7633afdc6..2f48e7fa8 100644 --- a/python/packages/jumpstarter-driver-renode/jumpstarter_driver_renode/monitor.py +++ b/python/packages/jumpstarter-driver-renode/jumpstarter_driver_renode/monitor.py @@ -40,8 +40,13 @@ async def connect(self, host: str, port: int, timeout: float = 10) -> None: except OSError: await sleep(0.5) + _ERROR_MARKERS = ("Could not find", "Error", "Invalid", "Failed", "Unknown") + async def execute(self, command: str) -> str: - """Send a command and return the response text (excluding the prompt).""" + """Send a command and return the response text (excluding the prompt). + + Raises RenodeMonitorError if the response indicates a command failure. + """ if self._stream is None: raise RuntimeError("not connected to Renode monitor") @@ -49,6 +54,11 @@ async def execute(self, command: str) -> str: await self._stream.send(f"{command}\n".encode()) response = await self._read_until_prompt() logger.debug("monitor< %s", response.strip()) + + stripped = response.strip() + if stripped and any(stripped.startswith(m) for m in self._ERROR_MARKERS): + raise RenodeMonitorError(stripped) + return response async def disconnect(self) -> None: diff --git a/python/uv.lock b/python/uv.lock index 234253413..2a35b9e76 100644 --- a/python/uv.lock +++ b/python/uv.lock @@ -33,12 +33,14 @@ members = [ "jumpstarter-driver-iscsi", "jumpstarter-driver-mitmproxy", "jumpstarter-driver-network", + "jumpstarter-driver-noyito-relay", "jumpstarter-driver-opendal", "jumpstarter-driver-pi-pico", "jumpstarter-driver-power", "jumpstarter-driver-probe-rs", "jumpstarter-driver-pyserial", "jumpstarter-driver-qemu", + "jumpstarter-driver-renode", "jumpstarter-driver-ridesx", "jumpstarter-driver-sdwire", "jumpstarter-driver-shell", @@ -1680,6 +1682,15 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/08/e7/ae38d7a6dfba0533684e0b2136817d667588ae3ec984c1a4e5df5eb88482/hatchling-1.27.0-py3-none-any.whl", hash = "sha256:d3a2f3567c4f926ea39849cdf924c7e99e6686c9c8e288ae1037c8fa2a5d937b", size = 75794, upload-time = "2024-12-15T17:08:10.364Z" }, ] +[[package]] +name = "hid" +version = "1.0.9" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/e9/f8/0357a8aa8874a243e96d08a8568efaf7478293e1a3441ddca18039b690c1/hid-1.0.9.tar.gz", hash = "sha256:f4471f11f0e176d1b0cb1b243e55498cc90347a3aede735655304395694ac182", size = 4973, upload-time = "2026-02-05T15:35:20.595Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/b8/c7/f0e1ad95179f44a6fc7a9140be025812cc7a62cf7390442b685a57ee1417/hid-1.0.9-py3-none-any.whl", hash = "sha256:6b9289e00bbc1e1589bec0c7f376a63fe03a4a4a1875575d0ad60e3e11a349f4", size = 4959, upload-time = "2026-02-05T15:35:19.269Z" }, +] + [[package]] name = "hpack" version = "4.1.0" @@ -2753,6 +2764,38 @@ dev = [ { name = "websocket-client", specifier = ">=1.8.0" }, ] +[[package]] +name = "jumpstarter-driver-noyito-relay" +source = { editable = "packages/jumpstarter-driver-noyito-relay" } +dependencies = [ + { name = "hid" }, + { name = "jumpstarter" }, + { name = "jumpstarter-driver-power" }, + { name = "pyserial" }, +] + +[package.dev-dependencies] +dev = [ + { name = "pytest" }, + { name = "pytest-cov" }, + { name = "pytest-mock" }, +] + +[package.metadata] +requires-dist = [ + { name = "hid", specifier = ">=1.0.4" }, + { name = "jumpstarter", editable = "packages/jumpstarter" }, + { name = "jumpstarter-driver-power", editable = "packages/jumpstarter-driver-power" }, + { name = "pyserial", specifier = ">=3.5" }, +] + +[package.metadata.requires-dev] +dev = [ + { name = "pytest", specifier = ">=8.3.3" }, + { name = "pytest-cov", specifier = ">=6.0.0" }, + { name = "pytest-mock", specifier = ">=3.14.0" }, +] + [[package]] name = "jumpstarter-driver-opendal" source = { editable = "packages/jumpstarter-driver-opendal" } @@ -2949,6 +2992,42 @@ dev = [ { name = "requests", specifier = ">=2.32.3" }, ] +[[package]] +name = "jumpstarter-driver-renode" +source = { editable = "packages/jumpstarter-driver-renode" } +dependencies = [ + { name = "jumpstarter" }, + { name = "jumpstarter-driver-composite" }, + { name = "jumpstarter-driver-network" }, + { name = "jumpstarter-driver-opendal" }, + { name = "jumpstarter-driver-power" }, + { name = "jumpstarter-driver-pyserial" }, +] + +[package.dev-dependencies] +dev = [ + { name = "pytest" }, + { name = "pytest-anyio" }, + { name = "pytest-cov" }, +] + +[package.metadata] +requires-dist = [ + { name = "jumpstarter", editable = "packages/jumpstarter" }, + { name = "jumpstarter-driver-composite", editable = "packages/jumpstarter-driver-composite" }, + { name = "jumpstarter-driver-network", editable = "packages/jumpstarter-driver-network" }, + { name = "jumpstarter-driver-opendal", editable = "packages/jumpstarter-driver-opendal" }, + { name = "jumpstarter-driver-power", editable = "packages/jumpstarter-driver-power" }, + { name = "jumpstarter-driver-pyserial", editable = "packages/jumpstarter-driver-pyserial" }, +] + +[package.metadata.requires-dev] +dev = [ + { name = "pytest", specifier = ">=8.3.2" }, + { name = "pytest-anyio", specifier = ">=0.0.0" }, + { name = "pytest-cov", specifier = ">=5.0.0" }, +] + [[package]] name = "jumpstarter-driver-ridesx" source = { editable = "packages/jumpstarter-driver-ridesx" } @@ -5257,6 +5336,18 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/0d/d2/dfc2f25f3905921c2743c300a48d9494d29032f1389fc142e718d6978fb2/pytest_httpserver-1.1.3-py3-none-any.whl", hash = "sha256:5f84757810233e19e2bb5287f3826a71c97a3740abe3a363af9155c0f82fdbb9", size = 21000, upload-time = "2025-04-10T08:17:13.906Z" }, ] +[[package]] +name = "pytest-mock" +version = "3.15.1" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "pytest" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/68/14/eb014d26be205d38ad5ad20d9a80f7d201472e08167f0bb4361e251084a9/pytest_mock-3.15.1.tar.gz", hash = "sha256:1849a238f6f396da19762269de72cb1814ab44416fa73a8686deac10b0d87a0f", size = 34036, upload-time = "2025-09-16T16:37:27.081Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/5a/cc/06253936f4a7fa2e0f48dfe6d851d9c56df896a9ab09ac019d70b760619c/pytest_mock-3.15.1-py3-none-any.whl", hash = "sha256:0a25e2eb88fe5168d535041d09a4529a188176ae608a6d249ee65abc0949630d", size = 10095, upload-time = "2025-09-16T16:37:25.734Z" }, +] + [[package]] name = "pytest-mqtt" version = "0.5.0" From 0be559970b9a36929df836d582365aebd9f85775 Mon Sep 17 00:00:00 2001 From: Vinicius Zein Date: Sat, 11 Apr 2026 15:03:53 -0400 Subject: [PATCH 03/10] Align ADR-0001 with JEP format from PR #423 Restructure the Renode integration ADR to follow the JEP template format: metadata table, Abstract, Motivation, Rejected Alternatives, Prior Art, Implementation History, and References sections. The DD-N design decisions and Consequences sections were already aligned. Made-with: Cursor --- .../adr/0001-renode-integration.md | 86 ++++++++++++++++--- 1 file changed, 74 insertions(+), 12 deletions(-) diff --git a/python/docs/source/contributing/adr/0001-renode-integration.md b/python/docs/source/contributing/adr/0001-renode-integration.md index faf75fc4f..f8d3b265c 100644 --- a/python/docs/source/contributing/adr/0001-renode-integration.md +++ b/python/docs/source/contributing/adr/0001-renode-integration.md @@ -1,10 +1,28 @@ # ADR-0001: Renode Integration Approach -- **Status**: Accepted -- **Date**: 2026-04-06 -- **Authors**: Vinicius Zein - -## Context +| Field | Value | +|--------------------|----------------------------------------------| +| **ADR** | 0001 | +| **Title** | Renode Integration Approach | +| **Author(s)** | @vtz (Vinicius Zein) | +| **Status** | Accepted | +| **Type** | Standards Track | +| **Created** | 2026-04-06 | +| **Updated** | 2026-04-11 | +| **Discussion** | [PR #533](https://github.com/jumpstarter-dev/jumpstarter/pull/533) | + +--- + +## Abstract + +This ADR documents the architectural decisions behind integrating the +[Renode](https://renode.io/) emulation framework into Jumpstarter as a +new driver package (`jumpstarter-driver-renode`). The driver enables +microcontroller-class virtual targets running bare-metal firmware or +RTOS on Cortex-M and RISC-V MCUs, complementing the existing QEMU +driver which targets Linux-capable SoCs. + +## Motivation Jumpstarter provides a driver-based framework for interacting with devices under test, both physical hardware and virtual systems. The @@ -14,12 +32,14 @@ provisioning. There is growing demand for **microcontroller-class** virtual targets running bare-metal firmware or RTOS (Zephyr, FreeRTOS, ThreadX) on -Cortex-M and RISC-V MCUs. [Renode](https://renode.io/) by Antmicro is -an open-source emulation framework designed specifically for this -domain, with extensive peripheral models for STM32, NXP S32K, Nordic, -SiFive, and other MCU platforms. +Cortex-M and RISC-V MCUs. Renode by Antmicro is an open-source +emulation framework designed specifically for this domain, with +extensive peripheral models for STM32, NXP S32K, Nordic, SiFive, and +other MCU platforms. + +### Reference Targets -The initial reference targets for validation are: +The initial targets for validation are: - **STM32F407 Discovery** (Cortex-M4F) -- opensomeip FreeRTOS/ThreadX ports, Renode built-in platform @@ -28,7 +48,7 @@ The initial reference targets for validation are: - **Nucleo H753ZI** (Cortex-M7) -- openbsw-zephyr, Renode built-in `stm32h743.repl` -### Forces +### Constraints - The driver must follow Jumpstarter's established composite driver pattern (as demonstrated by `jumpstarter-driver-qemu`) @@ -40,7 +60,7 @@ The initial reference targets for validation are: existing `PySerial` and `pexpect` tooling - The async framework must be `anyio` (the project's standard) -## Decisions +## Design Decisions ### DD-1: Control Interface -- Telnet Monitor @@ -152,3 +172,45 @@ additionally supports hot-loading: if the simulation is already running, detection and error handling rely on observed behavior - Renode's PTY terminal support on macOS may have edge cases not covered in testing + +## Rejected Alternatives + +Beyond the alternatives listed in each Design Decision above, the +high-level alternative of **not integrating Renode** and instead +extending the QEMU driver for MCU targets was considered. QEMU's MCU +support (e.g., `qemu-system-arm -M stm32vldiscovery`) is limited in +peripheral modeling and does not match Renode's breadth for embedded +platforms. The QEMU driver remains the right choice for Linux-capable +SoCs while Renode fills the MCU gap. + +## Prior Art + +- **jumpstarter-driver-qemu** -- The existing Jumpstarter QEMU driver + established the composite driver pattern, `Popen`-based process + management, and side-channel control protocol (QMP) that this ADR + follows. +- **Renode documentation** -- [Renode docs](https://renode.readthedocs.io/) + for monitor commands, platform descriptions, and UART terminal types. +- **opensomeip** -- [github.com/vtz/opensomeip](https://github.com/vtz/opensomeip) + provides the reference Renode targets (STM32F407, S32K388) used for + validation. + +## Implementation History + +- 2026-04-06: ADR proposed +- 2026-04-09: Initial implementation merged ([PR #533](https://github.com/jumpstarter-dev/jumpstarter/pull/533)) +- 2026-04-11: Address review feedback (DEVNULL, try-except cleanup, + async wait, RenodeMonitorError, multi-word CLI, docstrings) + +## References + +- [PR #533: Add Renode emulator driver](https://github.com/jumpstarter-dev/jumpstarter/pull/533) +- [Renode project](https://renode.io/) +- [Renode documentation](https://renode.readthedocs.io/) +- [JEP process (PR #423)](https://github.com/jumpstarter-dev/jumpstarter/pull/423) + +--- + +*This ADR is licensed under the +[Apache License, Version 2.0](https://www.apache.org/licenses/LICENSE-2.0), +consistent with the Jumpstarter project.* From 06c91bbab655e4337aab7370e91c6d271a985b68 Mon Sep 17 00:00:00 2001 From: Vinicius Zein Date: Sat, 11 Apr 2026 15:16:52 -0400 Subject: [PATCH 04/10] Fix async iterator mock pattern in flasher tests Replace incorrect AsyncMock(return_value=iter([...])) with idiomatic __aiter__/__anext__ pattern. The old pattern silently yielded nothing (with RuntimeWarning), the new one properly iterates the chunks. Made-with: Cursor --- .../jumpstarter_driver_renode/driver_test.py | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-) diff --git a/python/packages/jumpstarter-driver-renode/jumpstarter_driver_renode/driver_test.py b/python/packages/jumpstarter-driver-renode/jumpstarter_driver_renode/driver_test.py index 6f6c42893..0464ad7f6 100644 --- a/python/packages/jumpstarter-driver-renode/jumpstarter_driver_renode/driver_test.py +++ b/python/packages/jumpstarter-driver-renode/jumpstarter_driver_renode/driver_test.py @@ -329,8 +329,9 @@ async def test_flash_stores_firmware_path(self, tmp_path): with patch.object(flasher, "resource") as mock_resource: mock_res = AsyncMock() - mock_res.__aiter__ = AsyncMock( - return_value=iter([firmware_data]) + mock_res.__aiter__ = lambda self: self + mock_res.__anext__ = AsyncMock( + side_effect=[firmware_data, StopAsyncIteration()] ) mock_resource.return_value.__aenter__ = AsyncMock( return_value=mock_res @@ -355,7 +356,10 @@ async def test_flash_while_running_sends_load_and_reset(self): with patch.object(flasher, "resource") as mock_resource: mock_res = AsyncMock() - mock_res.__aiter__ = AsyncMock(return_value=iter([b"\x00"])) + mock_res.__aiter__ = lambda self: self + mock_res.__anext__ = AsyncMock( + side_effect=[b"\x00", StopAsyncIteration()] + ) mock_resource.return_value.__aenter__ = AsyncMock( return_value=mock_res ) @@ -375,7 +379,10 @@ async def test_flash_custom_load_command(self): with patch.object(flasher, "resource") as mock_resource: mock_res = AsyncMock() - mock_res.__aiter__ = AsyncMock(return_value=iter([b"\x00"])) + mock_res.__aiter__ = lambda self: self + mock_res.__anext__ = AsyncMock( + side_effect=[b"\x00", StopAsyncIteration()] + ) mock_resource.return_value.__aenter__ = AsyncMock( return_value=mock_res ) From 910d2492aea94f0038b346cd27fd3650d52e76f2 Mon Sep 17 00:00:00 2001 From: Vinicius Zein Date: Mon, 13 Apr 2026 03:29:35 -0400 Subject: [PATCH 05/10] Use Path-based assertion in test_renode_pty_path Mirror the driver's Path construction instead of string suffix matching. Made-with: Cursor --- .../jumpstarter_driver_renode/driver_test.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/python/packages/jumpstarter-driver-renode/jumpstarter_driver_renode/driver_test.py b/python/packages/jumpstarter-driver-renode/jumpstarter_driver_renode/driver_test.py index 0464ad7f6..984152d1b 100644 --- a/python/packages/jumpstarter-driver-renode/jumpstarter_driver_renode/driver_test.py +++ b/python/packages/jumpstarter-driver-renode/jumpstarter_driver_renode/driver_test.py @@ -445,8 +445,8 @@ def test_renode_custom_config(self): def test_renode_pty_path(self): """PTY path is inside the temp directory.""" driver = _make_driver() - assert driver._pty.endswith("/pty") - assert driver._tmp_dir.name in driver._pty + pty_path = Path(driver._pty) + assert pty_path == Path(driver._tmp_dir.name) / "pty" def test_renode_temp_directory_lifecycle(self): """TemporaryDirectory is created and can be cleaned up.""" From 44ed3d26c84613861a0e291f421159c84640ef6c Mon Sep 17 00:00:00 2001 From: Vinicius Zein Date: Wed, 15 Apr 2026 00:11:04 -0400 Subject: [PATCH 06/10] Address review findings: security hardening, packaging, and CI - Validate load_command against allowlist of known Renode load commands - Gate monitor_cmd behind allow_raw_monitor flag (default: false) - Auto-detect ELF vs binary firmware via magic bytes - Add timeout to monitor execute() with fail_after - Close leaked TCP streams on connect() retry - Reject newline characters in monitor commands - Check error markers per-line instead of just first line - Constrain prompt matching to registered machine names - Fix platform description path quoting (@path not @"path") - Add jumpstarter-driver-renode to workspace sources and jumpstarter-all - Add Renode installation to CI workflow (Linux + macOS) - Remove speculative xfail marker from E2E test Made-with: Cursor --- .github/workflows/python-tests.yaml | 11 ++ .../packages/jumpstarter-all/pyproject.toml | 1 + .../jumpstarter_driver_renode/driver.py | 45 ++++++- .../jumpstarter_driver_renode/driver_test.py | 122 ++++++++++++++++-- .../jumpstarter_driver_renode/monitor.py | 47 ++++--- python/pyproject.toml | 1 + python/uv.lock | 8 ++ 7 files changed, 206 insertions(+), 29 deletions(-) diff --git a/.github/workflows/python-tests.yaml b/.github/workflows/python-tests.yaml index 0742fc406..b24d52c9c 100644 --- a/.github/workflows/python-tests.yaml +++ b/.github/workflows/python-tests.yaml @@ -73,11 +73,22 @@ jobs: sudo apt-get update sudo apt-get install -y libgpiod-dev liblgpio-dev + - name: Install Renode (Linux) + if: runner.os == 'Linux' + run: | + wget https://builds.renode.io/renode_1.15.3+20241210git19e40b562_amd64.deb -O /tmp/renode.deb + sudo apt-get install -y /tmp/renode.deb + - name: Install Qemu (macOS) if: runner.os == 'macOS' run: | brew install qemu + - name: Install Renode (macOS) + if: runner.os == 'macOS' + run: | + brew install renode + - name: Cache Fedora Cloud images id: cache-fedora-cloud-images uses: actions/cache@v5 diff --git a/python/packages/jumpstarter-all/pyproject.toml b/python/packages/jumpstarter-all/pyproject.toml index a43fb8068..0ad6c20f4 100644 --- a/python/packages/jumpstarter-all/pyproject.toml +++ b/python/packages/jumpstarter-all/pyproject.toml @@ -28,6 +28,7 @@ dependencies = [ "jumpstarter-driver-probe-rs", "jumpstarter-driver-pyserial", "jumpstarter-driver-qemu", + "jumpstarter-driver-renode", "jumpstarter-driver-gpiod", "jumpstarter-driver-ridesx", "jumpstarter-driver-sdwire", diff --git a/python/packages/jumpstarter-driver-renode/jumpstarter_driver_renode/driver.py b/python/packages/jumpstarter-driver-renode/jumpstarter_driver_renode/driver.py index 50511889e..ef8475278 100644 --- a/python/packages/jumpstarter-driver-renode/jumpstarter_driver_renode/driver.py +++ b/python/packages/jumpstarter-driver-renode/jumpstarter_driver_renode/driver.py @@ -20,6 +20,26 @@ logger = logging.getLogger(__name__) +_ELF_MAGIC = b"\x7fELF" + +_ALLOWED_LOAD_COMMANDS = frozenset({ + "sysbus LoadELF", + "sysbus LoadBinary", + "sysbus LoadSymbolsFrom", +}) + + +def _detect_load_command(firmware_path: str) -> str: + """Choose the appropriate Renode load command based on file contents.""" + try: + with open(firmware_path, "rb") as f: + magic = f.read(4) + except OSError: + return "sysbus LoadELF" + if magic == _ELF_MAGIC: + return "sysbus LoadELF" + return "sysbus LoadBinary" + def _find_free_port() -> int: with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s: @@ -49,13 +69,22 @@ async def flash(self, source, load_command: str | None = None): loading during power-on. If already running, loads the firmware and resets the machine. """ + if load_command is not None and load_command not in _ALLOWED_LOAD_COMMANDS: + raise ValueError( + f"unsupported load_command {load_command!r}, " + f"allowed: {sorted(_ALLOWED_LOAD_COMMANDS)}" + ) + firmware_path = self.parent._tmp_dir.name + "/firmware" async with await FileWriteStream.from_path(firmware_path) as stream: async with self.resource(source) as res: async for chunk in res: await stream.send(chunk) - cmd = load_command or "sysbus LoadELF" + if load_command is not None: + cmd = load_command + else: + cmd = _detect_load_command(firmware_path) self.parent._firmware_path = firmware_path self.parent._load_command = cmd @@ -108,9 +137,10 @@ async def on(self) -> None: await self._monitor.connect("127.0.0.1", port) machine = self.parent.machine_name + self._monitor.add_expected_prompt(machine) await self._monitor.execute(f'mach create "{machine}"') await self._monitor.execute( - f'machine LoadPlatformDescription @"{self.parent.platform}"' + f"machine LoadPlatformDescription @{self.parent.platform}" ) pty_path = self.parent._pty @@ -196,6 +226,7 @@ class Renode(Driver): machine_name: str = "machine-0" monitor_port: int = 0 extra_commands: list[str] = field(default_factory=list) + allow_raw_monitor: bool = False _tmp_dir: TemporaryDirectory = field( init=False, default_factory=TemporaryDirectory @@ -238,7 +269,15 @@ def get_machine_name(self) -> str: @export async def monitor_cmd(self, command: str) -> str: - """Send an arbitrary command to the Renode monitor.""" + """Send a command to the Renode monitor. + + Requires ``allow_raw_monitor: true`` in the exporter configuration. + """ + if not self.allow_raw_monitor: + raise RuntimeError( + "raw monitor access is disabled; " + "set allow_raw_monitor: true in exporter config to enable" + ) power: RenodePower = self.children["power"] if power._monitor is None: raise RuntimeError("Renode is not running") diff --git a/python/packages/jumpstarter-driver-renode/jumpstarter_driver_renode/driver_test.py b/python/packages/jumpstarter-driver-renode/jumpstarter_driver_renode/driver_test.py index 984152d1b..5051a2cfa 100644 --- a/python/packages/jumpstarter-driver-renode/jumpstarter_driver_renode/driver_test.py +++ b/python/packages/jumpstarter-driver-renode/jumpstarter_driver_renode/driver_test.py @@ -1,7 +1,5 @@ from __future__ import annotations -import os -import platform import shutil from pathlib import Path from subprocess import TimeoutExpired @@ -9,7 +7,12 @@ import pytest -from jumpstarter_driver_renode.driver import Renode, RenodeFlasher, RenodePower +from jumpstarter_driver_renode.driver import ( + Renode, + RenodeFlasher, + RenodePower, + _detect_load_command, +) from jumpstarter_driver_renode.monitor import RenodeMonitor, RenodeMonitorError from jumpstarter.common.utils import serve @@ -116,6 +119,74 @@ async def test_monitor_disconnect_ignores_errors(self): await monitor.disconnect() assert monitor._stream is None + @pytest.mark.anyio + async def test_monitor_execute_rejects_newlines(self): + """execute() rejects commands containing newline characters.""" + monitor = RenodeMonitor() + monitor._stream = AsyncMock() + + with pytest.raises(ValueError, match="newline"): + await monitor.execute("cmd1\ncmd2") + + with pytest.raises(ValueError, match="newline"): + await monitor.execute("cmd1\rcmd2") + + @pytest.mark.anyio + async def test_monitor_connect_closes_stream_on_retry(self): + """connect() closes the previous stream before retrying.""" + monitor = RenodeMonitor() + streams = [] + call_count = 0 + + async def mock_connect_tcp(host, port): + nonlocal call_count + call_count += 1 + stream = AsyncMock() + streams.append(stream) + if call_count < 2: + stream.receive = AsyncMock(side_effect=OSError("not ready")) + else: + stream.receive = AsyncMock( + return_value=b"Renode v1.15\n(monitor) \n" + ) + return stream + + with patch( + "jumpstarter_driver_renode.monitor.connect_tcp", + side_effect=mock_connect_tcp, + ): + with patch( + "jumpstarter_driver_renode.monitor.sleep", new_callable=AsyncMock + ): + await monitor.connect("127.0.0.1", 12345) + + streams[0].aclose.assert_called_once() + + @pytest.mark.anyio + async def test_monitor_error_detection_per_line(self): + """Error markers are detected even when not on the first line.""" + monitor = RenodeMonitor() + stream = AsyncMock() + stream.receive = AsyncMock( + return_value=b"info text\nError executing command\n(monitor) \n" + ) + monitor._stream = stream + monitor._buffer = b"" + + with pytest.raises(RenodeMonitorError, match="Error executing"): + await monitor.execute("bad command") + + def test_monitor_prompt_matches_expected_only(self): + """_is_prompt only matches prompts in the expected set.""" + monitor = RenodeMonitor() + assert monitor._is_prompt(b"(monitor)") is True + assert monitor._is_prompt(b"(default)") is False + assert monitor._is_prompt(b"(enabled)") is False + + monitor.add_expected_prompt("my-machine") + assert monitor._is_prompt(b"(my-machine)") is True + assert monitor._is_prompt(b"(other)") is False + # --------------------------------------------------------------------------- # 5.2 RenodePower unit tests @@ -345,7 +416,7 @@ async def test_flash_stores_firmware_path(self, tmp_path): @pytest.mark.anyio async def test_flash_while_running_sends_load_and_reset(self): - """When simulation is running, flash() sends LoadELF + Reset.""" + """When simulation is running, flash() sends load + Reset.""" driver = _make_driver() power: RenodePower = driver.children["power"] power._process = MagicMock() @@ -353,12 +424,13 @@ async def test_flash_while_running_sends_load_and_reset(self): power._monitor = mock_monitor flasher: RenodeFlasher = driver.children["flasher"] + elf_data = b"\x7fELF" + b"\x00" * 60 with patch.object(flasher, "resource") as mock_resource: mock_res = AsyncMock() mock_res.__aiter__ = lambda self: self mock_res.__anext__ = AsyncMock( - side_effect=[b"\x00", StopAsyncIteration()] + side_effect=[elf_data, StopAsyncIteration()] ) mock_resource.return_value.__aenter__ = AsyncMock( return_value=mock_res @@ -395,6 +467,15 @@ async def test_flash_custom_load_command(self): assert driver._load_command == "sysbus LoadBinary" + @pytest.mark.anyio + async def test_flash_rejects_invalid_load_command(self): + """flash() rejects load_command values not in the allowlist.""" + driver = _make_driver() + flasher: RenodeFlasher = driver.children["flasher"] + + with pytest.raises(ValueError, match="unsupported load_command"): + await flasher.flash("/some/fw.elf", load_command="logFile @/tmp/evil") + @pytest.mark.anyio async def test_dump_not_implemented(self): """dump() raises NotImplementedError.""" @@ -404,6 +485,18 @@ async def test_dump_not_implemented(self): with pytest.raises(NotImplementedError, match="not supported"): await flasher.dump("/dev/null") + def test_detect_load_command_elf(self, tmp_path): + """ELF files are detected and use sysbus LoadELF.""" + elf = tmp_path / "fw.elf" + elf.write_bytes(b"\x7fELF" + b"\x00" * 60) + assert _detect_load_command(str(elf)) == "sysbus LoadELF" + + def test_detect_load_command_binary(self, tmp_path): + """Non-ELF files default to sysbus LoadBinary.""" + raw = tmp_path / "fw.bin" + raw.write_bytes(b"\x00" * 64) + assert _detect_load_command(str(raw)) == "sysbus LoadBinary" + # --------------------------------------------------------------------------- # 5.4 Configuration validation tests @@ -418,6 +511,7 @@ def test_renode_defaults(self): assert driver.machine_name == "machine-0" assert driver.monitor_port == 0 assert driver.extra_commands == [] + assert driver.allow_raw_monitor is False assert driver._firmware_path is None def test_renode_children_wired(self): @@ -481,10 +575,6 @@ def test_renode_get_machine_name(self): shutil.which("renode") is None, reason="Renode not installed", ) -@pytest.mark.xfail( - platform.system() == "Darwin" and os.getenv("GITHUB_ACTIONS") == "true", - reason="Renode tests may be flaky on macOS CI", -) def test_driver_renode_e2e(tmp_path): """E2E: start Renode, verify power on/off cycle via serve().""" with serve( @@ -525,12 +615,22 @@ def test_client_children_accessible(self): assert hasattr(client, "flasher") assert hasattr(client, "console") - def test_client_monitor_cmd_not_running(self): - """monitor_cmd raises when Renode is not running.""" + def test_client_monitor_cmd_disabled_by_default(self): + """monitor_cmd raises when allow_raw_monitor is False (default).""" from jumpstarter.client.core import DriverError driver = _make_driver() + with serve(driver) as client: + with pytest.raises(DriverError, match="raw monitor access is disabled"): + client.monitor_cmd("help") + + def test_client_monitor_cmd_not_running(self): + """monitor_cmd raises when Renode is not running (but monitor enabled).""" + from jumpstarter.client.core import DriverError + + driver = _make_driver(allow_raw_monitor=True) + with serve(driver) as client: with pytest.raises(DriverError, match="not running"): client.monitor_cmd("help") diff --git a/python/packages/jumpstarter-driver-renode/jumpstarter_driver_renode/monitor.py b/python/packages/jumpstarter-driver-renode/jumpstarter_driver_renode/monitor.py index 2f48e7fa8..1f110782c 100644 --- a/python/packages/jumpstarter-driver-renode/jumpstarter_driver_renode/monitor.py +++ b/python/packages/jumpstarter-driver-renode/jumpstarter_driver_renode/monitor.py @@ -7,10 +7,6 @@ logger = logging.getLogger(__name__) -MONITOR_PROMPT = b"(monitor)" -MACHINE_PROMPT_PREFIX = b"(" -MACHINE_PROMPT_SUFFIX = b")" - class RenodeMonitorError(Exception): """Raised when a Renode monitor command returns an error.""" @@ -26,6 +22,16 @@ class RenodeMonitor: _stream: SocketStream | None = None _buffer: bytes = b"" + _expected_prompts: set[bytes] + + def __init__(self) -> None: + self._stream = None + self._buffer = b"" + self._expected_prompts = {b"monitor"} + + def add_expected_prompt(self, name: str) -> None: + """Register a machine name so its prompt is recognised.""" + self._expected_prompts.add(name.encode()) async def connect(self, host: str, port: int, timeout: float = 10) -> None: """Connect to the Renode monitor, retrying until the prompt appears.""" @@ -38,26 +44,39 @@ async def connect(self, host: str, port: int, timeout: float = 10) -> None: logger.info("connected to Renode monitor at %s:%d", host, port) return except OSError: + if self._stream is not None: + try: + await self._stream.aclose() + except Exception: + pass + self._stream = None await sleep(0.5) _ERROR_MARKERS = ("Could not find", "Error", "Invalid", "Failed", "Unknown") - async def execute(self, command: str) -> str: + async def execute(self, command: str, timeout: float = 30) -> str: """Send a command and return the response text (excluding the prompt). Raises RenodeMonitorError if the response indicates a command failure. + Raises ValueError if the command contains newline characters. """ if self._stream is None: raise RuntimeError("not connected to Renode monitor") + if "\n" in command or "\r" in command: + raise ValueError("monitor commands must not contain newline characters") + logger.debug("monitor> %s", command) await self._stream.send(f"{command}\n".encode()) - response = await self._read_until_prompt() + with fail_after(timeout): + response = await self._read_until_prompt() logger.debug("monitor< %s", response.strip()) stripped = response.strip() - if stripped and any(stripped.startswith(m) for m in self._ERROR_MARKERS): - raise RenodeMonitorError(stripped) + if stripped: + for line in stripped.splitlines(): + if any(line.startswith(m) for m in self._ERROR_MARKERS): + raise RenodeMonitorError(stripped) return response @@ -100,6 +119,7 @@ def _find_prompt(self) -> int | None: """Find a Renode monitor prompt in the buffer. Renode prompts look like "(monitor) " or "(machine-name) ". + Only matches prompts whose inner text is in _expected_prompts. """ for line_start in self._iter_line_starts(): line = self._buffer[line_start:] @@ -124,16 +144,13 @@ def _iter_line_starts(self): yield nl + 1 pos = nl + 1 - @staticmethod - def _is_prompt(line: bytes) -> bool: - """Check if a line looks like a Renode prompt.""" + def _is_prompt(self, line: bytes) -> bool: + """Check if a line is a known Renode monitor prompt.""" stripped = line.strip() if not stripped: return False - if stripped.startswith(MACHINE_PROMPT_PREFIX) and stripped.endswith( - MACHINE_PROMPT_SUFFIX - ): + if stripped.startswith(b"(") and stripped.endswith(b")"): inner = stripped[1:-1] - if inner and b")" not in inner: + if inner in self._expected_prompts: return True return False diff --git a/python/pyproject.toml b/python/pyproject.toml index dbecb9b3b..afaddddbd 100644 --- a/python/pyproject.toml +++ b/python/pyproject.toml @@ -29,6 +29,7 @@ jumpstarter-driver-pi-pico = { workspace = true } jumpstarter-driver-probe-rs = { workspace = true } jumpstarter-driver-pyserial = { workspace = true } jumpstarter-driver-qemu = { workspace = true } +jumpstarter-driver-renode = { workspace = true } jumpstarter-driver-sdwire = { workspace = true } jumpstarter-driver-tasmota = { workspace = true } jumpstarter-driver-tmt = { workspace = true } diff --git a/python/uv.lock b/python/uv.lock index 2a35b9e76..b8255c8db 100644 --- a/python/uv.lock +++ b/python/uv.lock @@ -1994,6 +1994,7 @@ dependencies = [ { name = "jumpstarter-driver-probe-rs" }, { name = "jumpstarter-driver-pyserial" }, { name = "jumpstarter-driver-qemu" }, + { name = "jumpstarter-driver-renode" }, { name = "jumpstarter-driver-ridesx" }, { name = "jumpstarter-driver-sdwire" }, { name = "jumpstarter-driver-shell" }, @@ -2038,6 +2039,7 @@ requires-dist = [ { name = "jumpstarter-driver-probe-rs", editable = "packages/jumpstarter-driver-probe-rs" }, { name = "jumpstarter-driver-pyserial", editable = "packages/jumpstarter-driver-pyserial" }, { name = "jumpstarter-driver-qemu", editable = "packages/jumpstarter-driver-qemu" }, + { name = "jumpstarter-driver-renode", editable = "packages/jumpstarter-driver-renode" }, { name = "jumpstarter-driver-ridesx", editable = "packages/jumpstarter-driver-ridesx" }, { name = "jumpstarter-driver-sdwire", editable = "packages/jumpstarter-driver-sdwire" }, { name = "jumpstarter-driver-shell", editable = "packages/jumpstarter-driver-shell" }, @@ -2274,12 +2276,15 @@ source = { editable = "packages/jumpstarter-driver-ble" } dependencies = [ { name = "anyio" }, { name = "bleak" }, + { name = "click" }, { name = "jumpstarter" }, + { name = "jumpstarter-driver-network" }, ] [package.dev-dependencies] dev = [ { name = "pytest" }, + { name = "pytest-anyio" }, { name = "pytest-cov" }, ] @@ -2287,12 +2292,15 @@ dev = [ requires-dist = [ { name = "anyio", specifier = ">=4.10.0" }, { name = "bleak", specifier = ">=1.1.1" }, + { name = "click", specifier = ">=8.1.8" }, { name = "jumpstarter", editable = "packages/jumpstarter" }, + { name = "jumpstarter-driver-network", editable = "packages/jumpstarter-driver-network" }, ] [package.metadata.requires-dev] dev = [ { name = "pytest", specifier = ">=8.3.3" }, + { name = "pytest-anyio", specifier = ">=0.0.0" }, { name = "pytest-cov", specifier = ">=6.0.0" }, ] From 0f1f7b0c79a63cc1b2ca9d8235d1d36326bb1c4d Mon Sep 17 00:00:00 2001 From: Vinicius Zein Date: Wed, 15 Apr 2026 00:30:34 -0400 Subject: [PATCH 07/10] Replace ADR with JEP-0010 for Renode integration Move the Renode integration design document from python/docs/source/contributing/adr/ to the JEP directory at docs/internal/jeps/, following the JEP template from PR #423. Adds mandatory sections (Proposal, Design Details, Test Plan, Backward Compatibility, User Stories, Future Possibilities) and a new DD-5 for security decisions (allow_raw_monitor, load_command allowlist, newline rejection). Made-with: Cursor --- .../jeps/JEP-0010-renode-integration.md | 380 ++++++++++++++++++ .../adr/0001-renode-integration.md | 216 ---------- python/docs/source/contributing/adr/index.md | 24 -- 3 files changed, 380 insertions(+), 240 deletions(-) create mode 100644 docs/internal/jeps/JEP-0010-renode-integration.md delete mode 100644 python/docs/source/contributing/adr/0001-renode-integration.md delete mode 100644 python/docs/source/contributing/adr/index.md diff --git a/docs/internal/jeps/JEP-0010-renode-integration.md b/docs/internal/jeps/JEP-0010-renode-integration.md new file mode 100644 index 000000000..2be2b9225 --- /dev/null +++ b/docs/internal/jeps/JEP-0010-renode-integration.md @@ -0,0 +1,380 @@ +# JEP-0010: Renode Integration for Microcontroller Targets + +| Field | Value | +|--------------------|----------------------------------------------| +| **JEP** | 0010 | +| **Title** | Renode Integration for Microcontroller Targets | +| **Author(s)** | @vtz (Vinicius Zein) | +| **Status** | Implemented | +| **Type** | Standards Track | +| **Created** | 2026-04-06 | +| **Updated** | 2026-04-15 | +| **Discussion** | [PR #533](https://github.com/jumpstarter-dev/jumpstarter/pull/533) | + +--- + +## Abstract + +This JEP proposes integrating the [Renode](https://renode.io/) emulation +framework into Jumpstarter as a new driver package +(`jumpstarter-driver-renode`). The driver enables microcontroller-class +virtual targets running bare-metal firmware or RTOS on Cortex-M and +RISC-V MCUs, complementing the existing QEMU driver which targets +Linux-capable SoCs. + +## Motivation + +Jumpstarter provides a driver-based framework for interacting with +devices under test, both physical hardware and virtual systems. The +existing QEMU driver enables Linux-class virtual targets (aarch64, +x86_64) using full-system emulation with virtio devices and cloud-init +provisioning. + +There is growing demand for **microcontroller-class** virtual targets +running bare-metal firmware or RTOS (Zephyr, FreeRTOS, ThreadX) on +Cortex-M and RISC-V MCUs. QEMU's MCU support is limited in peripheral +modeling and does not match Renode's breadth for embedded platforms. +Renode by Antmicro is an open-source emulation framework designed +specifically for this domain, with extensive peripheral models for +STM32, NXP S32K, Nordic, SiFive, and other MCU platforms. + +### User Stories + +- **As a** firmware CI pipeline author, **I want to** run my Zephyr + firmware against a virtual STM32 target in Jumpstarter, **so that** + I can validate UART output and basic functionality without physical + hardware. +- **As a** SOME/IP stack developer, **I want to** configure virtual + NXP S32K388 and STM32F407 targets through exporter YAML, **so that** + I can test multi-platform firmware builds without modifying driver + code. +- **As a** Jumpstarter lab operator, **I want to** mix physical boards + and Renode virtual targets in the same test environment, **so that** + my team can scale testing beyond available hardware. + +### Reference Targets + +The initial targets for validation are: + +- **STM32F407 Discovery** (Cortex-M4F) -- opensomeip FreeRTOS/ThreadX + ports, Renode built-in platform +- **NXP S32K388** (Cortex-M7) -- opensomeip Zephyr port, custom + platform description +- **Nucleo H753ZI** (Cortex-M7) -- openbsw-zephyr, Renode built-in + `stm32h743.repl` + +### Constraints + +- The driver must follow Jumpstarter's established composite driver + pattern (as demonstrated by `jumpstarter-driver-qemu`) +- Users must be able to define new Renode targets through configuration + alone, without modifying driver code +- The solution should minimize external dependencies and runtime + requirements +- The UART/console interface must be compatible with Jumpstarter's + existing `PySerial` and `pexpect` tooling +- The async framework must be `anyio` (the project's standard) + +## Proposal + +The `jumpstarter-driver-renode` package provides a composite driver +(`Renode`) that manages a Renode simulation instance with three child +drivers: + +- **`RenodePower`** — controls the Renode process lifecycle (on/off) +- **`RenodeFlasher`** — handles firmware loading (`sysbus LoadELF` / + `sysbus LoadBinary`) +- **`PySerial` (console)** — serial access over a PTY terminal + +Users configure targets entirely through exporter YAML: + +```yaml +export: + Renode: + platform: "platforms/boards/stm32f4_discovery-kit.repl" + uart: "sysbus.usart2" + machine_name: "stm32f4" + extra_commands: + - "sysbus WriteDoubleWord 0x40090030 0x0301" + allow_raw_monitor: false +``` + +The driver starts Renode with `--disable-xwt --plain --port `, +connects to the telnet monitor via `anyio.connect_tcp`, and +programmatically sets up the platform, UART terminal, and firmware. + +A `RenodeMonitor` async client handles the telnet protocol: +- Retry-based connection with `fail_after(timeout)` +- Prompt detection using registered machine names +- Per-line error marker detection +- Newline injection prevention +- Configurable command timeout + +### API / Protocol Changes + +No gRPC protocol changes. The driver exposes standard Jumpstarter +interfaces (`PowerInterface`, `FlasherInterface`) plus: + +- `get_platform()`, `get_uart()`, `get_machine_name()` — read-only + config accessors +- `monitor_cmd(command)` — raw monitor access, gated behind + `allow_raw_monitor: true` (default: `false`) + +### Hardware Considerations + +No physical hardware required. Renode is a pure software emulator. +The driver uses PTY terminals for UART, which requires a POSIX +system (Linux or macOS). The Renode process is managed via +`subprocess.Popen`. + +## Design Decisions + +### DD-1: Control Interface — Telnet Monitor + +**Alternatives considered:** + +1. **Telnet monitor** — Renode's built-in TCP monitor interface. + Simple socket connection, send text commands, read responses. + Lightweight, no extra runtime needed. +2. **pyrenode3** — Python.NET bridge to Renode's C# internals. More + powerful but requires .NET runtime or Mono, heavy dependency, less + stable API surface. + +**Decision:** Telnet monitor. + +**Rationale:** It is the lowest-common-denominator interface that works +with any Renode installation. It mirrors the QEMU driver's pattern +where `Popen` starts the emulator process and a side-channel protocol +(QMP for QEMU, telnet monitor for Renode) provides programmatic +control. The monitor client uses `anyio.connect_tcp` with +`anyio.fail_after` for timeouts, consistent with `TcpNetwork` and +`grpc.py` in the project. + +### DD-2: UART Exposure — PTY Terminal + +**Alternatives considered:** + +1. **PTY** (`emulation CreateUartPtyTerminal`) — Creates a + pseudo-terminal file on the host. Reuses the existing `PySerial` + child driver exactly as QEMU does. Linux/macOS only. +2. **Socket** (`emulation CreateServerSocketTerminal`) — Exposes UART + as a TCP socket. Cross-platform. Maps to `TcpNetwork` driver. Has + telnet IAC negotiation bytes to handle. + +**Decision:** PTY as the primary interface. + +**Rationale:** Consistency with the QEMU driver, which uses `-serial +pty` and wires a `PySerial` child driver to the discovered PTY path. +This reuses the same serial/pexpect/console tooling without any +adaptation. Socket terminal support can be added later as a fallback +for platforms without PTY support. + +### DD-3: Configuration Model — Managed Mode + +**Alternatives considered:** + +1. **Managed mode** — The driver constructs all Renode monitor + commands from YAML config parameters (`platform`, `uart`, firmware + path). The driver handles platform loading, UART wiring, and + firmware loading programmatically. +2. **Script mode** — User provides a complete `.resc` script. The + driver runs it but still manages UART terminal setup. + +**Decision:** Managed mode as primary, with an `extra_commands` list +for target-specific customization. + +**Rationale:** Managed mode gives the driver full control over the UART +terminal setup (which must use PTY for Jumpstarter integration, not the +`CreateFileBackend` or `showAnalyzer` used in typical `.resc` scripts). +The `extra_commands` list covers target-specific needs like register +pokes (e.g., `sysbus WriteDoubleWord 0x40090030 0x0301` for S32K388 +PL011 UART enablement) and Ethernet switch setup. + +### DD-4: Firmware Loading — Deferred to Flash + +**Alternatives considered:** + +1. `flash()` stores the firmware path, `on()` loads it into the + simulation and starts +2. `on()` starts the simulation, `flash()` loads firmware and resets + +**Decision:** Option 1 — `flash()` stores the path, `on()` loads and +starts. + +**Rationale:** This matches the QEMU driver's semantic where you flash +a disk image first, then power on. It also allows re-flashing between +power cycles without restarting the Renode process. The `RenodeFlasher` +additionally supports hot-loading: if the simulation is already running, +`flash()` sends the load command and resets the machine. + +### DD-5: Security — Restricted Monitor Access + +**Alternatives considered:** + +1. **Open access** — Expose `monitor_cmd` to all authenticated clients +2. **Opt-in access** — Gate behind `allow_raw_monitor` config flag + +**Decision:** Opt-in with `allow_raw_monitor: false` by default. + +**Rationale:** The Renode monitor supports commands that interact with +the host filesystem (`logFile`, `include`, `CreateFileTerminal`). In a +shared lab environment, unrestricted monitor access from any +authenticated client poses a risk. The `load_command` parameter in +`flash()` is separately validated against an allowlist of known Renode +load commands. Newline characters are rejected in all monitor commands +to prevent command injection. + +## Design Details + +### Component Architecture + +``` +Renode (composite driver) +├── RenodePower → manages Popen lifecycle + RenodeMonitor +├── RenodeFlasher → writes firmware, sends LoadELF/LoadBinary +└── PySerial → console over PTY terminal +``` + +### Monitor Protocol + +The `RenodeMonitor` client connects to Renode's telnet port and +communicates via line-oriented text: + +1. **Connection**: retry loop with `fail_after(timeout)`, closing + leaked streams on retry +2. **Prompt detection**: matches `(monitor)` or registered machine + names only — no false positives from output like `(enabled)` +3. **Error detection**: per-line check against markers (`Could not + find`, `Error`, `Invalid`, `Failed`, `Unknown`) +4. **Timeout**: `execute()` wraps reads in `fail_after(30)` to prevent + indefinite blocking +5. **Injection prevention**: newline characters rejected in commands; + `load_command` validated against allowlist + +### Firmware Auto-Detection + +When `load_command` is not specified, the driver reads the first 4 +bytes of the firmware file. If they match the ELF magic (`\x7fELF`), +`sysbus LoadELF` is used; otherwise `sysbus LoadBinary`. + +## Test Plan + +### Unit Tests + +- `TestRenodeMonitor` — connection retry, command execution, error + detection (per-line), disconnect, newline rejection, stream cleanup + on retry, prompt matching against expected prompts only +- `TestRenodePower` — command sequence verification, extra commands + ordering, firmware-less boot, idempotent on/off, process termination + and cleanup +- `TestRenodeFlasher` — firmware path storage, hot-load with reset, + custom load command, invalid load command rejection, ELF magic + detection, dump not-implemented +- `TestRenodeConfig` — default values, children wiring, custom config, + PTY path construction, lifecycle + +### Integration Tests + +- `TestRenodeClient` — round-trip properties via `serve()`, children + accessibility, `monitor_cmd` disabled by default, `monitor_cmd` not + running error, CLI rendering + +### E2E Tests + +- `test_driver_renode_e2e` — full power on/off cycle with real Renode + process, skipped when Renode is not installed + +### CI + +Renode is installed in the `python-tests.yaml` workflow: +- Linux: `.deb` package from builds.renode.io +- macOS: `brew install renode` + +## Backward Compatibility + +This JEP introduces a new driver package with no changes to existing +packages. There are no breaking changes. The `jumpstarter-all` +meta-package includes the new driver as an optional dependency. + +## Consequences + +### Positive + +- Single `jumpstarter-driver-renode` package supports any Renode target + through YAML configuration alone +- No .NET runtime or Mono dependency required +- Consistent user experience with the QEMU driver (same composite + pattern, same console/pexpect workflow) +- `extra_commands` provides an escape hatch for target-specific + customization without code changes +- Security-by-default with `allow_raw_monitor: false` + +### Negative + +- PTY-only UART exposure limits to Linux/macOS (acceptable since Renode + itself primarily targets these platforms) +- The telnet monitor protocol is text-based and less structured than + QMP's JSON — error detection requires string matching +- Full `.resc` script support is deferred; users with complex Renode + setups must express their configuration as managed-mode parameters + plus `extra_commands` + +### Risks + +- Renode's monitor protocol has no formal specification; prompt + detection and error handling rely on observed behavior +- Renode's PTY terminal support on macOS may have edge cases not + covered in testing + +## Rejected Alternatives + +Beyond the alternatives listed in each Design Decision above, the +high-level alternative of **not integrating Renode** and instead +extending the QEMU driver for MCU targets was considered. QEMU's MCU +support (e.g., `qemu-system-arm -M stm32vldiscovery`) is limited in +peripheral modeling and does not match Renode's breadth for embedded +platforms. The QEMU driver remains the right choice for Linux-capable +SoCs while Renode fills the MCU gap. + +## Prior Art + +- **jumpstarter-driver-qemu** — The existing Jumpstarter QEMU driver + established the composite driver pattern, `Popen`-based process + management, and side-channel control protocol (QMP) that this JEP + follows. +- **Renode documentation** — [Renode docs](https://renode.readthedocs.io/) + for monitor commands, platform descriptions, and UART terminal types. +- **opensomeip** — [github.com/vtz/opensomeip](https://github.com/vtz/opensomeip) + provides the reference Renode targets (STM32F407, S32K388) used for + validation. + +## Future Possibilities + +- **Socket terminal fallback** for Windows/cross-platform UART access +- **`.resc` script mode** for users with complex existing Renode setups +- **Multi-machine simulation** for testing inter-MCU communication +- **Renode metrics integration** for performance profiling + +## Implementation History + +- 2026-04-06: JEP proposed +- 2026-04-09: Initial implementation merged + ([PR #533](https://github.com/jumpstarter-dev/jumpstarter/pull/533)) +- 2026-04-11: Address review feedback (DEVNULL, try-except cleanup, + async wait, RenodeMonitorError, multi-word CLI, docstrings) +- 2026-04-15: Security hardening (load_command allowlist, + allow_raw_monitor, newline rejection, per-line error detection, + prompt matching, execute timeout, stream leak fix) + +## References + +- [PR #533: Add Renode emulator driver](https://github.com/jumpstarter-dev/jumpstarter/pull/533) +- [Renode project](https://renode.io/) +- [Renode documentation](https://renode.readthedocs.io/) +- [JEP process (PR #423)](https://github.com/jumpstarter-dev/jumpstarter/pull/423) + +--- + +*This JEP is licensed under the +[Apache License, Version 2.0](https://www.apache.org/licenses/LICENSE-2.0), +consistent with the Jumpstarter project.* diff --git a/python/docs/source/contributing/adr/0001-renode-integration.md b/python/docs/source/contributing/adr/0001-renode-integration.md deleted file mode 100644 index f8d3b265c..000000000 --- a/python/docs/source/contributing/adr/0001-renode-integration.md +++ /dev/null @@ -1,216 +0,0 @@ -# ADR-0001: Renode Integration Approach - -| Field | Value | -|--------------------|----------------------------------------------| -| **ADR** | 0001 | -| **Title** | Renode Integration Approach | -| **Author(s)** | @vtz (Vinicius Zein) | -| **Status** | Accepted | -| **Type** | Standards Track | -| **Created** | 2026-04-06 | -| **Updated** | 2026-04-11 | -| **Discussion** | [PR #533](https://github.com/jumpstarter-dev/jumpstarter/pull/533) | - ---- - -## Abstract - -This ADR documents the architectural decisions behind integrating the -[Renode](https://renode.io/) emulation framework into Jumpstarter as a -new driver package (`jumpstarter-driver-renode`). The driver enables -microcontroller-class virtual targets running bare-metal firmware or -RTOS on Cortex-M and RISC-V MCUs, complementing the existing QEMU -driver which targets Linux-capable SoCs. - -## Motivation - -Jumpstarter provides a driver-based framework for interacting with -devices under test, both physical hardware and virtual systems. The -existing QEMU driver enables Linux-class virtual targets (aarch64, -x86_64) using full-system emulation with virtio devices and cloud-init -provisioning. - -There is growing demand for **microcontroller-class** virtual targets -running bare-metal firmware or RTOS (Zephyr, FreeRTOS, ThreadX) on -Cortex-M and RISC-V MCUs. Renode by Antmicro is an open-source -emulation framework designed specifically for this domain, with -extensive peripheral models for STM32, NXP S32K, Nordic, SiFive, and -other MCU platforms. - -### Reference Targets - -The initial targets for validation are: - -- **STM32F407 Discovery** (Cortex-M4F) -- opensomeip FreeRTOS/ThreadX - ports, Renode built-in platform -- **NXP S32K388** (Cortex-M7) -- opensomeip Zephyr port, custom - platform description -- **Nucleo H753ZI** (Cortex-M7) -- openbsw-zephyr, Renode built-in - `stm32h743.repl` - -### Constraints - -- The driver must follow Jumpstarter's established composite driver - pattern (as demonstrated by `jumpstarter-driver-qemu`) -- Users must be able to define new Renode targets through configuration - alone, without modifying driver code -- The solution should minimize external dependencies and runtime - requirements -- The UART/console interface must be compatible with Jumpstarter's - existing `PySerial` and `pexpect` tooling -- The async framework must be `anyio` (the project's standard) - -## Design Decisions - -### DD-1: Control Interface -- Telnet Monitor - -**Alternatives considered:** - -1. **Telnet monitor** -- Renode's built-in TCP monitor interface. - Simple socket connection, send text commands, read responses. - Lightweight, no extra runtime needed. -2. **pyrenode3** -- Python.NET bridge to Renode's C# internals. More - powerful but requires .NET runtime or Mono, heavy dependency, less - stable API surface. - -**Decision:** Telnet monitor. - -**Rationale:** It is the lowest-common-denominator interface that works -with any Renode installation. It mirrors the QEMU driver's pattern -where `Popen` starts the emulator process and a side-channel protocol -(QMP for QEMU, telnet monitor for Renode) provides programmatic -control. The monitor client uses `anyio.connect_tcp` with -`anyio.fail_after` for timeouts, consistent with `TcpNetwork` and -`grpc.py` in the project. No `telnetlib`, `telnetlib3`, or -`asynctelnet` is introduced since these are not used anywhere in the -project. - -### DD-2: UART Exposure -- PTY Terminal - -**Alternatives considered:** - -1. **PTY** (`emulation CreateUartPtyTerminal`) -- Creates a - pseudo-terminal file on the host. Reuses the existing `PySerial` - child driver exactly as QEMU does. Linux/macOS only. -2. **Socket** (`emulation CreateServerSocketTerminal`) -- Exposes UART - as a TCP socket. Cross-platform. Maps to `TcpNetwork` driver. Has - telnet IAC negotiation bytes to handle. - -**Decision:** PTY as the primary interface. - -**Rationale:** Consistency with the QEMU driver, which uses `-serial -pty` and wires a `PySerial` child driver to the discovered PTY path. -This reuses the same serial/pexpect/console tooling without any -adaptation. Socket terminal support can be added later as a fallback -for platforms without PTY support. - -### DD-3: Configuration Model -- Managed Mode - -**Alternatives considered:** - -1. **Managed mode** -- The driver constructs all Renode monitor - commands from YAML config parameters (`platform`, `uart`, firmware - path). The driver handles platform loading, UART wiring, and - firmware loading programmatically. -2. **Script mode** -- User provides a complete `.resc` script. The - driver runs it but still manages UART terminal setup. - -**Decision:** Managed mode as primary, with an `extra_commands` list -for target-specific customization. - -**Rationale:** Managed mode gives the driver full control over the UART -terminal setup (which must use PTY for Jumpstarter integration, not the -`CreateFileBackend` or `showAnalyzer` used in typical `.resc` scripts). -The `extra_commands` list covers target-specific needs like register -pokes (e.g., `sysbus WriteDoubleWord 0x40090030 0x0301` for S32K388 -PL011 UART enablement) and Ethernet switch setup. The opensomeip `.resc` -files are CI-oriented and their setup maps directly to managed-mode -config parameters. - -### DD-4: Firmware Loading -- Deferred to Flash - -**Alternatives considered:** - -1. `flash()` stores the firmware path, `on()` loads it into the - simulation and starts -2. `on()` starts the simulation, `flash()` loads firmware and resets - -**Decision:** Option 1 -- `flash()` stores the path, `on()` loads and -starts. - -**Rationale:** This matches the QEMU driver's semantic where you flash -a disk image first, then power on. It also allows re-flashing between -power cycles without restarting the Renode process. The `RenodeFlasher` -additionally supports hot-loading: if the simulation is already running, -`flash()` sends the `sysbus LoadELF` command and resets the machine. - -## Consequences - -### Positive - -- Single `jumpstarter-driver-renode` package supports any Renode target - through YAML configuration alone -- No .NET runtime or Mono dependency required -- Consistent user experience with the QEMU driver (same composite - pattern, same console/pexpect workflow) -- `extra_commands` provides an escape hatch for target-specific - customization without code changes - -### Negative - -- PTY-only UART exposure limits to Linux/macOS (acceptable since Renode - itself primarily targets these platforms) -- The telnet monitor protocol is text-based and less structured than - QMP's JSON -- error detection requires string matching -- Full `.resc` script support is deferred; users with complex Renode - setups must express their configuration as managed-mode parameters - plus `extra_commands` - -### Risks - -- Renode's monitor protocol has no formal specification; prompt - detection and error handling rely on observed behavior -- Renode's PTY terminal support on macOS may have edge cases not - covered in testing - -## Rejected Alternatives - -Beyond the alternatives listed in each Design Decision above, the -high-level alternative of **not integrating Renode** and instead -extending the QEMU driver for MCU targets was considered. QEMU's MCU -support (e.g., `qemu-system-arm -M stm32vldiscovery`) is limited in -peripheral modeling and does not match Renode's breadth for embedded -platforms. The QEMU driver remains the right choice for Linux-capable -SoCs while Renode fills the MCU gap. - -## Prior Art - -- **jumpstarter-driver-qemu** -- The existing Jumpstarter QEMU driver - established the composite driver pattern, `Popen`-based process - management, and side-channel control protocol (QMP) that this ADR - follows. -- **Renode documentation** -- [Renode docs](https://renode.readthedocs.io/) - for monitor commands, platform descriptions, and UART terminal types. -- **opensomeip** -- [github.com/vtz/opensomeip](https://github.com/vtz/opensomeip) - provides the reference Renode targets (STM32F407, S32K388) used for - validation. - -## Implementation History - -- 2026-04-06: ADR proposed -- 2026-04-09: Initial implementation merged ([PR #533](https://github.com/jumpstarter-dev/jumpstarter/pull/533)) -- 2026-04-11: Address review feedback (DEVNULL, try-except cleanup, - async wait, RenodeMonitorError, multi-word CLI, docstrings) - -## References - -- [PR #533: Add Renode emulator driver](https://github.com/jumpstarter-dev/jumpstarter/pull/533) -- [Renode project](https://renode.io/) -- [Renode documentation](https://renode.readthedocs.io/) -- [JEP process (PR #423)](https://github.com/jumpstarter-dev/jumpstarter/pull/423) - ---- - -*This ADR is licensed under the -[Apache License, Version 2.0](https://www.apache.org/licenses/LICENSE-2.0), -consistent with the Jumpstarter project.* diff --git a/python/docs/source/contributing/adr/index.md b/python/docs/source/contributing/adr/index.md deleted file mode 100644 index a5ad4ee52..000000000 --- a/python/docs/source/contributing/adr/index.md +++ /dev/null @@ -1,24 +0,0 @@ -# Architecture Decision Records - -This directory contains Architecture Decision Records (ADRs) for the -Jumpstarter project. ADRs document significant technical decisions, -their context, and consequences. - -## Format - -Each ADR follows a standard structure: - -- **Status**: Proposed, Accepted, Deprecated, or Superseded -- **Context**: The forces at play, including technical, political, and - project-specific constraints -- **Decision**: The change being proposed or decided -- **Consequences**: What follows from the decision, both positive and - negative - -## Records - -```{toctree} -:maxdepth: 1 - -0001-renode-integration -``` From fb6fc4f9b6af96e89767aa3339cf22e9ad17f8a7 Mon Sep 17 00:00:00 2001 From: Miguel Angel Ajo Pelayo Date: Wed, 15 Apr 2026 10:30:56 +0200 Subject: [PATCH 08/10] Update docs/internal/jeps/JEP-0010-renode-integration.md --- docs/internal/jeps/JEP-0010-renode-integration.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/internal/jeps/JEP-0010-renode-integration.md b/docs/internal/jeps/JEP-0010-renode-integration.md index 2be2b9225..56a7f3e8a 100644 --- a/docs/internal/jeps/JEP-0010-renode-integration.md +++ b/docs/internal/jeps/JEP-0010-renode-integration.md @@ -357,8 +357,8 @@ SoCs while Renode fills the MCU gap. ## Implementation History -- 2026-04-06: JEP proposed -- 2026-04-09: Initial implementation merged +- 2026-04-06 JEP proposed +- 2026-04-06: Initial implementation proposed ([PR #533](https://github.com/jumpstarter-dev/jumpstarter/pull/533)) - 2026-04-11: Address review feedback (DEVNULL, try-except cleanup, async wait, RenodeMonitorError, multi-word CLI, docstrings) From 90283dd7993207608c872b29cfb778c2cac8d9a2 Mon Sep 17 00:00:00 2001 From: Miguel Angel Ajo Pelayo Date: Wed, 15 Apr 2026 10:35:19 +0200 Subject: [PATCH 09/10] Apply suggestion from @mangelajo --- .github/workflows/python-tests.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/python-tests.yaml b/.github/workflows/python-tests.yaml index b24d52c9c..94419c388 100644 --- a/.github/workflows/python-tests.yaml +++ b/.github/workflows/python-tests.yaml @@ -76,7 +76,7 @@ jobs: - name: Install Renode (Linux) if: runner.os == 'Linux' run: | - wget https://builds.renode.io/renode_1.15.3+20241210git19e40b562_amd64.deb -O /tmp/renode.deb + wget https://github.com/renode/renode/releases/download/v1.16.1/renode_1.16.1_amd64.deb -O /tmp/renode.deb sudo apt-get install -y /tmp/renode.deb - name: Install Qemu (macOS) From 6c905a058f79f46f2822dc7cf31edc4cb6ba4b98 Mon Sep 17 00:00:00 2001 From: Miguel Angel Ajo Pelayo Date: Wed, 15 Apr 2026 10:41:55 +0200 Subject: [PATCH 10/10] Apply suggestion from @mangelajo --- .github/workflows/python-tests.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/python-tests.yaml b/.github/workflows/python-tests.yaml index 94419c388..33f4f5412 100644 --- a/.github/workflows/python-tests.yaml +++ b/.github/workflows/python-tests.yaml @@ -87,7 +87,7 @@ jobs: - name: Install Renode (macOS) if: runner.os == 'macOS' run: | - brew install renode + brew install renode/tap/renode - name: Cache Fedora Cloud images id: cache-fedora-cloud-images