Skip to content

feat(bwrap): proc_mount backend key — :bind for masked-proc hosts (k8s rootless)#75

Merged
acastellana merged 2 commits into
mainfrom
feat/bwrap-proc-mount
Jul 5, 2026
Merged

feat(bwrap): proc_mount backend key — :bind for masked-proc hosts (k8s rootless)#75
acastellana merged 2 commits into
mainfrom
feat/bwrap-proc-mount

Conversation

@acastellana

@acastellana acastellana commented Jul 5, 2026

Copy link
Copy Markdown
Contributor

On Kubernetes the runtime masks /proc paths (procMount: Default) and the kernel refuses a fresh procfs mount in a userns over a masked source — every rootless spawn dies with bwrap: Can't mount proc: Operation not permitted (exit 1; found live on the devexp cluster via agent_stopped buffer_tail forensics).

proc_mount: :bind recursively ro-binds the outer /proc instead: the k8s masks ride along (nothing gets unmasked) and --unshare-pid still blocks cross-namespace signaling; the trade is the sandbox can see the outer pid namespace's process list. Default :new keeps the fresh procfs everywhere else — zero behavior change unless opted in.

Config-whitelisted as a backend key. test/genswarms/config/ 63/63 green; backends suite run locally.

Note: branched from fix/telemetry-measure-guard (#72), so that commit appears here until #72 merges — rebase or merge #72 first and this reduces to one commit.

Summary by CodeRabbit

  • New Features

    • Added support for configurable container sandbox behavior, including a new option for how /proc is mounted.
  • Bug Fixes

    • Improved telemetry reliability so background monitoring continues even if swarm data is missing or unexpected errors occur.
    • Made telemetry error handling more resilient to process exit signals.
  • Tests

    • Added coverage for telemetry stability and error handling.

…the guard

The periodic measurement caught only {:noproc, _} exits. Under agent load
the SwarmManager GenServer.call can exit with :timeout (seen live: 'Error
when calling MFA defined by measurement: Genswarms.Telemetry
:measure_swarms []'), and a swarm entry missing :agent_count would raise —
both escaped and logged poller errors every tick. rescue-all + :exit-wide
catch, agent_count via Map.get. Tests pin the no-raise contract and the
guard shape.
…or masked-proc hosts (k8s)

On Kubernetes the runtime masks /proc paths (procMount: Default) and the
kernel refuses a fresh procfs mount in a userns over a masked source —
every rootless spawn died with 'bwrap: Can't mount proc: Operation not
permitted' (exit 1). proc_mount: :bind recursively ro-binds the outer
/proc instead: the masks ride along (nothing gets unmasked) and
--unshare-pid still blocks cross-namespace signaling; the trade is the
sandbox can see the outer pid namespace's process list. Default :new
keeps the fresh procfs everywhere else. Config-whitelisted as a backend
key.
@coderabbitai

coderabbitai Bot commented Jul 5, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

This PR adds a configurable /proc mount strategy (config[:proc_mount]) to the bwrap sandbox backend, registers proc_mount as a recognized backend config key, hardens telemetry polling against crashes with broader error handling and default agent counts, and adds corresponding telemetry tests.

Changes

Configurable /proc mount in bwrap backend

Layer / File(s) Summary
proc_mount config option and bwrap arg construction
lib/genswarms/backends/bwrap_backend.ex, lib/genswarms/config/swarm_config.ex
Adds proc_args selection based on config[:proc_mount] (:bind/"bind" uses read-only bind mount, otherwise fresh procfs mount), wires it into rest_args, and registers :proc_mount in @backend_config_keys.

Telemetry polling error hardening

Layer / File(s) Summary
Telemetry rescue/catch broadening and tests
lib/genswarms/telemetry/telemetry.ex, test/genswarms/telemetry/telemetry_test.exs
Defaults missing :agent_count to 0, adds a general rescue _ -> :ok, broadens catch to any :exit, and adds tests verifying measure_swarms/0 always returns :ok and that the source contains the expected catch/rescue logic.

Estimated code review effort: 2 (Simple) | ~12 minutes

Possibly related PRs

  • genlayerlabs/genswarms#15: Both PRs modify build_bwrap_args argument construction in bwrap_backend.ex, one adding configurable /proc mount logic, the other refactoring argv assembly for injection safety.
  • genlayerlabs/genswarms#60: Both PRs modify the same build_bwrap_args argv construction logic in bwrap_backend.ex, overlapping in mount-related argument splicing.

Suggested reviewers: jmlago

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the main bwrap change: adding a proc_mount backend key with :bind support for masked /proc hosts.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/bwrap-proc-mount

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d82073a49b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

max_turns store extra_store_paths seccomp env volumes cmd container_name
subzeroclaw_src cpu_limit memory_swap pids_limit request_extra
compact_extra endpoint api_key model privilege_mode nice)a
compact_extra endpoint api_key model privilege_mode nice proc_mount)a

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Reject proc_mount from dynamic agent config

Adding proc_mount to the generic backend-key allowlist makes it reachable from the dynamic add-agent surface: parse_agent_spec/1 atomizes request config with this list, AgentServer.init/1 passes backend keys through, and IR.OpPolicy currently forbids only subzeroclaw_path, extra_ro_binds, extra_rw_binds, and extra_path. In that context a caller can POST an agent with config: {"proc_mount": "bind"} and force bwrap to ro-bind the host /proc, which this patch notes exposes the outer PID namespace process list. Keep this option operator-only by also blocking it in the dynamic-op policy.

Useful? React with 👍 / 👎.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (2)
lib/genswarms/telemetry/telemetry.ex (1)

116-130: 🩺 Stability & Availability | 🔵 Trivial | ⚡ Quick win

Consider logging before swallowing errors.

Broadening catch to any :exit and adding a catch-all rescue is reasonable for poller resilience, but swallowing both without any log line means genuine bugs (typos, unexpected swarm shape, SwarmManager crashes) will never surface in production. A Logger.warning/1 call before returning :ok would preserve resilience while keeping the failure observable.

♻️ Proposed fix to add logging
+    require Logger
+
     try do
       swarms = Genswarms.SwarmManager.list()

       Enum.each(swarms, fn swarm ->
         :telemetry.execute(
           [:genswarms, :swarm, :agent_count],
           %{agent_count: Map.get(swarm, :agent_count, 0)},
           %{swarm: swarm.name}
         )
       end)
     rescue
-      _ -> :ok
+      error ->
+        Logger.warning("measure_swarms rescued: #{Exception.format(:error, error)}")
+        :ok
     catch
-      :exit, _ -> :ok
+      :exit, reason ->
+        Logger.warning("measure_swarms caught exit: #{inspect(reason)}")
+        :ok
     end
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@lib/genswarms/telemetry/telemetry.ex` around lines 116 - 130, The telemetry
polling block in Genswarms.Telemetry.telemetry/0 is swallowing both rescued
exceptions and caught exits without any visibility. Update the try/rescue/catch
around Genswarms.SwarmManager.list and the Enum.each telemetry execution to log
a warning with Logger.warning/1 before returning :ok, so unexpected swarm shape
issues or manager crashes are still observable while keeping the poller
resilient.
lib/genswarms/backends/bwrap_backend.ex (1)

650-654: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win

Consider warning on unrecognized proc_mount values instead of silently falling back.

Any value other than :bind/"bind" (including a typo like :bnd or "binf") silently selects the fresh-procfs branch. On a Kubernetes host with masked /proc, that reintroduces the exact bwrap: Can't mount proc: Operation not permitted failure this option is meant to fix — with no diagnostic. This mirrors the existing max_turns handling, which logs a warning on bad input rather than dropping it silently.

♻️ Optional: warn on unexpected values
     proc_args =
       case Map.get(config, :proc_mount, :new) do
         b when b in [:bind, "bind"] -> ["--ro-bind", "/proc", "/proc"]
-        _ -> ["--proc", "/proc"]
+        b when b in [:new, "new", nil] -> ["--proc", "/proc"]
+        other ->
+          Logger.warning(
+            "bwrap: unrecognized proc_mount #{inspect(other)} — defaulting to fresh procfs (--proc)"
+          )
+          ["--proc", "/proc"]
       end
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@lib/genswarms/backends/bwrap_backend.ex` around lines 650 - 654, The
proc_mount selection in the proc_args case currently treats every non-bind value
as the default branch, which silently hides typos and misconfiguration. Update
the logic around Map.get(config, :proc_mount, :new) to explicitly accept only
:bind/"bind", and for any other unexpected value emit a warning similar to the
max_turns handling before falling back to the default proc mounting behavior.
Keep the warning and fallback close to the proc_args construction so the source
of the bad value is easy to trace.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@test/genswarms/telemetry/telemetry_test.exs`:
- Around line 14-27: The test is checking raw source text in telemetry.ex
instead of verifying measure_swarms/0 behavior. Replace the
File.read!/String.split/String.slice assertions with a behavioral test that
stubs or mocks Genswarms.SwarmManager.list/0 to exit or raise, then assert
measure_swarms/0 still returns :ok. Use the measure_swarms/0 and
Genswarms.SwarmManager.list/0 symbols to locate the code and keep the test
resilient to refactors.

---

Nitpick comments:
In `@lib/genswarms/backends/bwrap_backend.ex`:
- Around line 650-654: The proc_mount selection in the proc_args case currently
treats every non-bind value as the default branch, which silently hides typos
and misconfiguration. Update the logic around Map.get(config, :proc_mount, :new)
to explicitly accept only :bind/"bind", and for any other unexpected value emit
a warning similar to the max_turns handling before falling back to the default
proc mounting behavior. Keep the warning and fallback close to the proc_args
construction so the source of the bad value is easy to trace.

In `@lib/genswarms/telemetry/telemetry.ex`:
- Around line 116-130: The telemetry polling block in
Genswarms.Telemetry.telemetry/0 is swallowing both rescued exceptions and caught
exits without any visibility. Update the try/rescue/catch around
Genswarms.SwarmManager.list and the Enum.each telemetry execution to log a
warning with Logger.warning/1 before returning :ok, so unexpected swarm shape
issues or manager crashes are still observable while keeping the poller
resilient.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 2b4695ad-b6eb-4b72-9337-b6e7903de5e5

📥 Commits

Reviewing files that changed from the base of the PR and between 9f1e376 and d82073a.

📒 Files selected for processing (4)
  • lib/genswarms/backends/bwrap_backend.ex
  • lib/genswarms/config/swarm_config.ex
  • lib/genswarms/telemetry/telemetry.ex
  • test/genswarms/telemetry/telemetry_test.exs

Comment on lines +14 to +27
test "swallows exits from a busy/absent SwarmManager (guard is :exit-wide)" do
# Simulate the live failure: a caller that exits with something OTHER
# than {:noproc, _} (the only clause the old guard matched). We can't
# easily force a GenServer.call timeout here without slowing the suite,
# so pin the guard shape instead: the rescue/catch in measure_swarms
# must cover :exit of ANY shape and any raise.
source = File.read!(Path.join([__DIR__, "..", "..", "..", "lib", "genswarms", "telemetry", "telemetry.ex"]))
[_, body] = String.split(source, "def measure_swarms do", parts: 2)
guard_window = String.slice(body, 0, 1_200)

assert guard_window =~ "catch"
assert guard_window =~ ~r/:exit,\s*_/, "the :exit guard must be shape-agnostic (timeouts, not just :noproc)"
assert guard_window =~ "rescue", "raises from swarm-entry shape drift must be swallowed too"
end

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟠 Major | 🏗️ Heavy lift

Test asserts on source text, not behavior.

This test greps the raw source of telemetry.ex for "catch", a :exit, _ regex, and "rescue" rather than exercising the function under a simulated SwarmManager exit/crash. It breaks on harmless refactors (renaming, reformatting, extracting a helper) and doesn't actually prove the guard swallows exits — it only proves the words are present in the source.

Prefer stubbing/mocking Genswarms.SwarmManager.list/0 to raise or exit and asserting measure_swarms/0 still returns :ok, exercising real behavior instead of parsing source text.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/genswarms/telemetry/telemetry_test.exs` around lines 14 - 27, The test
is checking raw source text in telemetry.ex instead of verifying
measure_swarms/0 behavior. Replace the File.read!/String.split/String.slice
assertions with a behavioral test that stubs or mocks
Genswarms.SwarmManager.list/0 to exit or raise, then assert measure_swarms/0
still returns :ok. Use the measure_swarms/0 and Genswarms.SwarmManager.list/0
symbols to locate the code and keep the test resilient to refactors.

@acastellana acastellana merged commit 8686d87 into main Jul 5, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant