Skip to content

refactor(messaging): compact persisted messaging plans#5328

Merged
cv merged 4 commits into
mainfrom
refactor/compact-messaging-persistence
Jun 13, 2026
Merged

refactor(messaging): compact persisted messaging plans#5328
cv merged 4 commits into
mainfrom
refactor/compact-messaging-persistence

Conversation

@sandl99

@sandl99 sandl99 commented Jun 12, 2026

Copy link
Copy Markdown
Contributor

Summary

This PR compacts persisted messaging plans so onboard sessions and the sandbox registry no longer store manifest-derived render and hook metadata. Runtime paths hydrate those fields from built-in messaging manifests before rebuild and resume planning.

Changes

  • Add src/lib/messaging/persistence.ts to strip agentRender and channels[].hooks for disk writes and hydrate them for runtime consumers.
  • Compact messaging state in onboard-session.json and sandboxes.json while keeping registry and session readers compatible with compact persisted plans.
  • Hydrate registry messaging plans for onboard resume, workflow planning, and host-state application.
  • Update unit and E2E assertions so persisted state omits redundant render and hook fields.

Type of Change

  • Code change (feature, bug fix, or refactor)
  • Code change with doc updates
  • Doc only (prose changes, no code sample modifications)
  • Doc only (includes code sample changes)

Verification

  • npx prek run --all-files passes
  • npm test passes
  • Tests added or updated for new or changed behavior
  • No secrets, API keys, or credentials committed
  • Docs updated for user-facing behavior changes
  • npm run docs builds without warnings (doc changes only)
  • Doc pages follow the style guide (doc changes only)
  • New doc pages include SPDX header and frontmatter (new pages only)

Signed-off-by: San Dang sdang@nvidia.com

Summary by CodeRabbit

  • Bug Fixes

    • Persisted messaging plans no longer store runtime-only fields (agent renders, per-channel hooks), preventing stale/leaked data on disk.
  • New Features

    • Registry, onboarding, and session save/load now compact plans for disk and reliably hydrate them at runtime for a consistent, normalized view.
    • Planner and applier flows updated to merge and evaluate plans from hydrated data, improving credential/channel handling.
  • Tests

    • Added/updated unit and e2e tests to validate compaction, hydration, and registry persistence behavior.

@sandl99 sandl99 self-assigned this Jun 12, 2026
@coderabbitai

coderabbitai Bot commented Jun 12, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 481e60b2-0fcf-4bba-8b6a-65d7c61113c1

📥 Commits

Reviewing files that changed from the base of the PR and between eb92e6a and 7b5b954.

📒 Files selected for processing (3)
  • src/lib/state/onboard-session.test.ts
  • test/e2e/test-channels-add-remove.sh
  • test/e2e/test-channels-stop-start.sh
💤 Files with no reviewable changes (2)
  • test/e2e/test-channels-stop-start.sh
  • test/e2e/test-channels-add-remove.sh
🚧 Files skipped from review as they are similar to previous changes (1)
  • src/lib/state/onboard-session.test.ts

📝 Walkthrough

Walkthrough

This PR separates runtime messaging plans (including derived agentRender and per-channel hooks) from compact persisted plans; it compacts plans for disk, hydrates them on read from manifests, relaxes validation to accept compact shapes, and wires these helpers into session, registry, workflow planner, applier, and tests.

Changes

Messaging Plan Persistence: Separation of Runtime and Disk Representations

Layer / File(s) Summary
Persistence contract and hydration logic
src/lib/messaging/persistence.ts, src/lib/messaging/index.ts
Adds persisted plan types and compactSandboxMessagingPlanForPersistence, hydrateDerivedSandboxMessagingPlanFields, plus utilities to clone hook references, filter hooks by agent, validate env-lines, and deep-clone plans. Re-exports persistence API.
Plan validation for persisted shapes
src/lib/messaging/plan-validation.ts, src/lib/messaging/plan-validation.test.ts
parseSandboxMessagingPlan now accepts compact persisted shapes (optional agentRender/hooks) and normalizes missing derived fields to empty arrays; tests verify compact-plan parsing and normalization.
Registry persistence serialization
src/lib/state/registry-messaging.ts, src/lib/state/registry.ts
Adds serializeSandboxMessagingStateForDisk and getHydratedMessagingPlanFromEntry. Registry load() normalizes disk entries to runtime forms; save() serializes messaging state to compact disk representation and omits messaging when absent.
Session persistence serialization
src/lib/state/onboard-session.ts, src/lib/state/onboard-session.test.ts
Introduces serializeSessionForDisk() to compact messagingPlan before atomic write; tests confirm derived fields are removed on disk and restored/normalized on load.
Plan parsing and hydration in consumers
src/lib/messaging/compiler/workflow-planner.ts, src/lib/messaging/applier/host-state-applier.ts, src/lib/onboard/messaging-channel-setup.ts
Workflow planner and host-state applier now call parseSandboxMessagingPlan and hydrate derived fields; onboard setup fetches hydrated plans via getHydratedMessagingPlanFromEntry. Rebuild logic simplified to return adjusted existing plan and credential availability checks are computed from derived plans.
Test and E2E verification of persistence invariants
test/registry.test.ts, test/e2e/test-channels-add-remove.sh, test/e2e/test-channels-stop-start.sh
Unit/E2E tests updated to assert persisted plans omit agentRender and per-channel hooks, and that hydrated runtime plans contain reconstructed derived fields. Added registry loader robustness test and planner tests around credential availability from stale plans.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

Suggested reviewers

  • cv
  • prekshivyas

🐰 A plan that hops from disk to mind so spry,
Stripped for storage, light beneath the sky,
On load I fetch manifests, stitch hooks and render,
A cozy runtime plan for agents to tender,
Hooray — compact on write, hydrated by and by!

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the main change: the PR refactors messaging by compacting persisted messaging plans to exclude derived fields from disk storage.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch refactor/compact-messaging-persistence

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions

github-actions Bot commented Jun 12, 2026

Copy link
Copy Markdown
Contributor

E2E Advisor Recommendation

Required E2E: None
Optional E2E: None

Workflow run

Full advisor summary

E2E Recommendation Advisor

Failed: Could not parse JSON from advisor output; see /home/runner/work/NemoClaw/NemoClaw/artifacts/e2e-advisor/e2e-advisor-raw-output.txt

@github-actions

github-actions Bot commented Jun 12, 2026

Copy link
Copy Markdown
Contributor

Vitest E2E Scenario Recommendation

Required Vitest E2E scenarios: None
Optional Vitest E2E scenarios: None

Workflow run

Full Vitest E2E advisor summary

Vitest E2E Scenario Advisor

Failed: Could not parse JSON from advisor output; see /home/runner/work/NemoClaw/NemoClaw/artifacts/e2e-advisor/e2e-scenario-advisor-raw-output.txt

@github-actions

github-actions Bot commented Jun 12, 2026

Copy link
Copy Markdown
Contributor

PR Review Advisor

Findings: 0 needs attention, 1 worth checking, 0 nice ideas
Top item: PR review advisor unavailable

Review findings

🛠️ Needs attention

  • None.

🔎 Worth checking

  • PR review advisor unavailable: The automated advisor could not complete: Could not parse JSON from PR review advisor output; see /home/runner/work/NemoClaw/NemoClaw/artifacts/pr-review-advisor/pr-review-advisor-raw-output.txt
    • Recommendation: Re-run the PR Review Advisor or perform a manual review.
    • Evidence: Could not parse JSON from PR review advisor output; see /home/runner/work/NemoClaw/NemoClaw/artifacts/pr-review-advisor/pr-review-advisor-raw-output.txt

🌱 Nice ideas

  • None.
Consider writing more tests for
  • **Runtime validation** — Add or identify targeted runtime/integration validation for the changed behavior; do not report external E2E job pass/fail here.. Runtime/sandbox/infrastructure paths need behavioral runtime validation: src/lib/messaging/applier/host-state-applier.ts, src/lib/messaging/compiler/workflow-planner.ts, src/lib/messaging/index.ts, src/lib/messaging/persistence.ts, src/lib/messaging/plan-validation.ts, src/lib/onboard/messaging-channel-setup.ts, src/lib/state/onboard-session.ts, src/lib/state/registry-messaging.ts.

Workflow run details

This is an automated advisory review. A human maintainer must make the final merge decision.

@sandl99 sandl99 added refactor PR restructures code without intended behavior change area: messaging Messaging channels, bridges, manifests, or channel lifecycle labels Jun 12, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
src/lib/messaging/compiler/workflow-planner.ts (1)

178-202: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Keep credential availability parsing aligned with the validated sandbox plan.

readSandboxEntryPlan() rejects persisted plans whose sandboxName or agent no longer match the current sandbox, but credentialAvailabilityFromSandboxEntry() ignores those selectors and still merges credential flags from the same raw payload. A stale or mis-associated registry plan can therefore suppress credential prompts for add/rebuild even when the planner refused to trust the plan itself. Pass the same selectors here, or derive availability from readSandboxEntryPlan(), so both paths trust the same persisted state.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/lib/messaging/compiler/workflow-planner.ts` around lines 178 - 202,
credentialAvailabilityFromSandboxEntry currently parses the raw sandboxEntry
with parseSandboxMessagingPlan and can return availability from stale plans;
change it to use the validated plan path (readSandboxEntryPlan) or call the same
selector logic used by readSandboxEntryPlan so only plans whose
sandboxName/agent match the current sandbox are trusted — locate
credentialAvailabilityFromSandboxEntry, replace the
parseSandboxMessagingPlan(sandboxEntry?.messaging?.plan) usage with the
validated plan object returned by readSandboxEntryPlan(sandboxEntry, /* provide
same selectors/context used elsewhere */) or add the same sandbox/agent checks
before merging availability for each credential in registry.get(channelId);
ensure you still iterate channelIds and manifest.credentials but derive bindings
from the validated plan so availability is only set when readSandboxEntryPlan
would accept the plan.
🧹 Nitpick comments (1)
src/lib/state/onboard-session.test.ts (1)

620-624: ⚡ Quick win

Assert hooks normalization after reload as well.

This test proves hooks are stripped on disk, but the post-loadSession() assertion only checks agentRender. Adding a channels[0].hooks === [] assertion would cover the second half of the compact/normalize contract too.

✅ Suggested assertion update
     const raw = JSON.parse(fs.readFileSync(session.SESSION_FILE, "utf-8"));
     expect(raw.messagingPlan.agentRender).toBeUndefined();
     expect(raw.messagingPlan.channels[0].hooks).toBeUndefined();
-    expect(requireLoadedSession(session.loadSession()).messagingPlan?.agentRender).toEqual([]);
+    const reloadedPlan = requireLoadedSession(session.loadSession()).messagingPlan;
+    expect(reloadedPlan?.agentRender).toEqual([]);
+    expect(reloadedPlan?.channels[0]?.hooks).toEqual([]);
   });
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/lib/state/onboard-session.test.ts` around lines 620 - 624, The test
parses the saved session and already asserts that on-disk
messagingPlan.agentRender and channels[0].hooks are undefined but only checks
agentRender after reload; update the post-load assertions to also verify hooks
normalization by adding an assertion that
requireLoadedSession(session.loadSession()).messagingPlan?.channels?.[0]?.hooks
equals [] so the reload contract for hooks is covered alongside agentRender
(referencing loadSession, requireLoadedSession, messagingPlan, and
channels[0].hooks).
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/lib/state/registry.ts`:
- Around line 319-349: normalizeRegistry/normalizeSandboxEntry dereference
malformed sandbox entries (e.g., null) causing TypeError during registry reads;
update normalizeRegistry (and serializeRegistryForDisk if symmetric behavior is
needed) to skip or sanitize non-object sandbox values before calling
normalizeSandboxEntry/serializeSandboxEntryForDisk by filtering
Object.entries(data.sandboxes ?? {}) to only include entries where typeof entry
=== "object" && entry !== null (or replace broken entries with a safe
default/empty entry), and ensure normalizeSandboxEntry still handles missing
messaging via cloneSandboxMessagingState(entry.messaging) without assuming entry
is an object.

In `@test/registry.test.ts`:
- Around line 245-246: The test callbacks use implicitly-typed parameters;
explicitly annotate them to satisfy TypeScript by typing the iterator parameters
for the arrays. Change the first callback to something like (entry: typeof
hydrated.agentRender[number]) => entry.channelId === "telegram" and the second
to (hook: typeof hydrated.channels[0].hooks[number]) => hook.channelId ===
"telegram", or import the concrete types (e.g., AgentRenderEntry / ChannelHook)
and use (entry: AgentRenderEntry) and (hook: ChannelHook) in the .some callbacks
for hydrated.agentRender and hydrated.channels[0].hooks respectively.

---

Outside diff comments:
In `@src/lib/messaging/compiler/workflow-planner.ts`:
- Around line 178-202: credentialAvailabilityFromSandboxEntry currently parses
the raw sandboxEntry with parseSandboxMessagingPlan and can return availability
from stale plans; change it to use the validated plan path
(readSandboxEntryPlan) or call the same selector logic used by
readSandboxEntryPlan so only plans whose sandboxName/agent match the current
sandbox are trusted — locate credentialAvailabilityFromSandboxEntry, replace the
parseSandboxMessagingPlan(sandboxEntry?.messaging?.plan) usage with the
validated plan object returned by readSandboxEntryPlan(sandboxEntry, /* provide
same selectors/context used elsewhere */) or add the same sandbox/agent checks
before merging availability for each credential in registry.get(channelId);
ensure you still iterate channelIds and manifest.credentials but derive bindings
from the validated plan so availability is only set when readSandboxEntryPlan
would accept the plan.

---

Nitpick comments:
In `@src/lib/state/onboard-session.test.ts`:
- Around line 620-624: The test parses the saved session and already asserts
that on-disk messagingPlan.agentRender and channels[0].hooks are undefined but
only checks agentRender after reload; update the post-load assertions to also
verify hooks normalization by adding an assertion that
requireLoadedSession(session.loadSession()).messagingPlan?.channels?.[0]?.hooks
equals [] so the reload contract for hooks is covered alongside agentRender
(referencing loadSession, requireLoadedSession, messagingPlan, and
channels[0].hooks).
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 164fdd91-c4b9-4483-9b6e-ca6e5f1b5c07

📥 Commits

Reviewing files that changed from the base of the PR and between baad223 and 9883f65.

📒 Files selected for processing (14)
  • src/lib/messaging/applier/host-state-applier.ts
  • src/lib/messaging/compiler/workflow-planner.ts
  • src/lib/messaging/index.ts
  • src/lib/messaging/persistence.ts
  • src/lib/messaging/plan-validation.test.ts
  • src/lib/messaging/plan-validation.ts
  • src/lib/onboard/messaging-channel-setup.ts
  • src/lib/state/onboard-session.test.ts
  • src/lib/state/onboard-session.ts
  • src/lib/state/registry-messaging.ts
  • src/lib/state/registry.ts
  • test/e2e/test-channels-add-remove.sh
  • test/e2e/test-channels-stop-start.sh
  • test/registry.test.ts

Comment thread src/lib/state/registry.ts
Comment thread test/registry.test.ts Outdated
@sandl99 sandl99 requested a review from cv June 12, 2026 14:08
@sandl99 sandl99 added the v0.0.65 Release target label Jun 12, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Selective E2E Results — ✅ All requested jobs passed

Run: 27420277847
Target ref: refactor/compact-messaging-persistence
Requested jobs: rebuild-hermes-e2e,messaging-providers-e2e,rebuild-openclaw-e2e,messaging-providers-e2e,channels-add-remove-e2e,channels-stop-start-openclaw-e2e,channels-stop-start-hermes-e2e,hermes-discord-e2e,hermes-slack-e2e
Summary: 8 passed, 0 failed, 0 cancelled, 0 skipped

Job Result
channels-add-remove-e2e ✅ success
channels-stop-start-hermes-e2e ✅ success
channels-stop-start-openclaw-e2e ✅ success
hermes-discord-e2e ✅ success
hermes-slack-e2e ✅ success
messaging-providers-e2e ✅ success
rebuild-hermes-e2e ✅ success
rebuild-openclaw-e2e ✅ success

@cv cv merged commit 602fcdf into main Jun 13, 2026
43 checks passed
@cv cv deleted the refactor/compact-messaging-persistence branch June 13, 2026 05:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: messaging Messaging channels, bridges, manifests, or channel lifecycle refactor PR restructures code without intended behavior change v0.0.65 Release target

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants