diff --git a/docs/architecture-audit-2026-07-12/AgentOrgRecovery.md b/docs/architecture-audit-2026-07-12/AgentOrgRecovery.md index 5664504e8..2b39ea351 100644 --- a/docs/architecture-audit-2026-07-12/AgentOrgRecovery.md +++ b/docs/architecture-audit-2026-07-12/AgentOrgRecovery.md @@ -1,444 +1,392 @@ -# Agent Org Recovery 与一致性加固：最终代码审计报告 +# Agent Org Recovery and Consistency Hardening — Architecture Audit -> 审计日期：2026-07-13 -> 分支：`fix/issue-272-agent-org-recovery-invariants` -> 基线：`develop` / `ea8516092` -> 范围：GitHub issue #272、相邻的 Agent Org 状态一致性、恢复、调度、任务原子性、CLI 能力边界和对应 UI 入口 -> 结论：依赖确认门、显式 Coordinator 指派、Planner 审批、生命周期收尾和 Pause/Resume 已通过连续真人 dev 验收；最终提交仍以本次全量检查与 Husky hooks 的实际结果为准。 +- Audit date: 2026-07-13, updated with final verification on 2026-07-18 +- Branch: `fix/issue-272-agent-org-recovery-invariants` +- Baseline: `develop` +- Scope: GitHub issue #272, adjacent Agent Org state consistency, recovery, scheduling, Task atomicity, CLI capability boundaries, and related UI entry points +- Status: The design described here is implemented and verified. The later review-safety audit in `docs/architecture-audit-2026-07-16/AgentOrgReviewSafetyAudit.md` is the final source for review findings and final test totals. -## 一句话结论 +## Executive conclusion -这次把 Agent Org 从“看见像卡住了就尝试叫人、甚至按时间抢任务”，改成了“先读数据库做无副作用诊断，再按持久预算、真实任务资格和 run 状态执行恢复”。Run、Session、Task、Inbox/Wake 四种状态不再互相冒充，重复 wake 会合并，terminal/paused run 不会被后台门铃复活，任务完成与 run 收尾也不会再互相穿透。真人运行暴露出的“蓝色运行态反复闪、成员成果没有可靠交给下游、coordinator 晚一拍仍认为 blocked、所有人完成后 run 不自动收尾”也已经按四批追加修复；随后又根据真实日志修掉了更上游的根因：普通 Coordinator 消息不再被误判成用户接管 worker。第二次 Breaking Bad 真人运行继续暴露出两个新根因：Coordinator 把有先后关系的 Implement/Review/Test 当成并行任务，以及 duplicate Wake 虽被 scheduler 合并、却在数据库留下永远 queued 的新 intent。第三次 The Boys 真人运行证明底层依赖调度正确，但 Coordinator 的文字说明写着“等写作和检查完成”，结构化 `dependency_task_ids` 却只填写 Planner；本轮因此新增无副作用的依赖确认门，漏列当前 open task 时先返回 guidance，只有补齐依赖或明确确认并行后才允许落库。 +Agent Org recovery changed from “something looks stalled, so wake somebody or reclaim old work by time” to “read a durable Snapshot without side effects, analyze the facts, then execute bounded recovery under a persistent budget, exact Task authority, and current Run state.” Run, Session, Task, and Inbox/Wake state no longer impersonate one another. Duplicate Wakes coalesce, Paused or terminal Runs cannot be revived by a queued doorbell, Task completion is handed to downstream members through durable output, and Run finality is serialized with Task mutation. -审计过程中还修复了相邻尾部问题：顶层 `org2` 编译路径错误、主动 shutdown 成员可能被 watchdog 复活、失败恢复曾直接唤醒 peer、损坏的 recovery deadline 会永久压住重试、启动恢复只扫描 500 个 run、CLI 保存错误没带 member_id、重复 Wake intent 不收口、旧 in-flight intent 不自愈、已完成 Run 重启后仍被通用策略暂停，以及 debug restart 与生产启动步骤漂移。最终设计进一步删除了失败后的 Worker 自动领取和自动 Wake，统一交回 Coordinator 明确指派。 +Real development runs exposed several failures that unit-level reasoning alone did not reveal: -## 大白话：现在这套设计是什么 +- a blue Running indicator repeatedly flashing because a WakeNoop loop fought an intervention guard; +- a normal message to the Coordinator being misclassified as direct takeover; +- completed upstream work lacking a durable handoff to the next worker; +- the Coordinator remaining one state behind Task completion; +- Review/Test Tasks silently starting early when dependencies were omitted; +- duplicate Wake requests coalescing in memory while leaving durable queued intents behind; +- completed Runs requiring manual Pause/Resume before finality became visible. -| 代码名词 | 大白话 | 数据库里的真相 | -| --------------- | ---------------------------------- | ---------------------------------------------- | -| `AgentOrgRun` | 一次团队项目 | `agent_org_runs` | -| `Session` | 某个成员当前是否在干活 | `agent_sessions` / 历史 CLI 的 `code_sessions` | -| `Task` | 看板工单，记录归属、依赖和完成状态 | `agent_org_tasks` | -| `AgentInbox` | 不会因为进程重启而丢失的信箱 | `agent_inbox` | -| Wake | 门铃，只负责让成员起来读信/接活 | scheduler 中带幂等 key 的 turn | -| Watchdog | 每分钟巡查的保安 | 纯分析 + 恢复执行器 | -| Recovery budget | 保安的持久重试登记簿 | `agent_org_recovery_attempts` | +All of those paths were traced to durable state and corrected. The final design also removes autonomous worker claiming after failure: ownerless work is an honest “currently unassigned” state and requires explicit Coordinator assignment. + +## Plain-language model + +| Code term | Plain-language meaning | Durable source of truth | +| --------------- | ---------------------------------------------------------- | ----------------------------------------------------- | +| `AgentOrgRun` | One team execution or project | `agent_org_runs` | +| `Session` | Whether one member is currently executing a turn | `agent_sessions`, plus historical CLI `code_sessions` | +| `Task` | A board item recording owner, dependencies, and completion | `agent_org_tasks` | +| `AgentInbox` | Durable mail that survives process restart | `agent_inbox` | +| Wake | A doorbell asking the scheduler to give a Session work | A scheduler turn with a stable idempotency key | +| Watchdog | A periodic recovery inspector | Pure analysis followed by a bounded executor | +| Recovery budget | Persistent retry accounting | `agent_org_recovery_attempts` | ```mermaid flowchart TD - DB["持久化快照
Run + Session + Task + Inbox + Budget"] - A["Recovery Analyzer
只读取，不写库、不 wake"] - P["Recovery Plan
同一轮可 wake、repair、reconcile"] - E["Recovery Executor
动作前重新检查 Run"] - W["Wake Dispatcher
确定性 key 合并重复门铃"] - C["Coordinator Notice
稳定指纹 + 1/5/15 分钟退避"] - F["Run Reconciler
writer lock + Immediate transaction"] - N["No-op
暂停、终态、重复或工作已消失"] + DB["Durable Snapshot\nRun + Session + Task + Inbox + Budget"] + A["Recovery Analyzer\nreads only; no write and no Wake"] + P["Recovery Plan\nmay contain Wake, repair, and reconcile actions"] + E["Recovery Executor\nrechecks current state before each action"] + W["Wake Dispatcher\nstable key coalesces duplicate doorbells"] + C["Coordinator Notice\nstable fingerprint + 1/5/15 minute backoff"] + F["Run Reconciler\nwriter lock + IMMEDIATE transaction"] + N["No-op\nPaused, terminal, duplicate, or work disappeared"] DB --> A --> P --> E E --> W E --> C E --> F - W -->|"Run 仍 Running 且有真实输入"| DB + W -->|"Run still Running with real input"| DB W -->|"Paused / terminal / coalesced / no work"| N ``` -核心原则是： +Core rules: -1. `Idle session` 只表示员工此刻没执行 turn，不等于项目结束。 -2. `Running run` 只表示项目允许继续，不等于每个成员都健康。 -3. `Pending task` 不等于任何人都能接；依赖、eligibility、现有 owner 工作量都要满足。 -4. `Unread inbox` 是持久事实；wake 只是门铃，排队失败也不能把信当成已送达。 -5. 任何 task mutation 和 run finality 都要经过同一 writer lock，不能出现 terminal run 下面又长出 open task。 +1. An Idle Session means that a worker is not executing a turn now; it does not mean the project ended. +2. A Running Run means that work is allowed to continue; it does not mean every member is healthy. +3. A Pending Task is not automatically claimable. Dependencies, eligibility, ownership, and authority still apply. +4. An unread Inbox row is durable truth. Wake is only a doorbell; enqueue failure cannot be presented as delivered work. +5. Task mutation and Run finality share the same writer serialization so a terminal Run cannot acquire new open work. -## 真人运行诊断后的四批追加修复 - -这四批不是换一套 Agent Org，也不是又加了两个 AI。它们是在现有 coordinator、member、task、inbox 之下补齐“门铃、交接单、完工回执、项目验收”四个机械环节。 +## Four recovery batches identified by real runs ```mermaid flowchart LR - A["第 1 批：门铃防抖
Wake 不再自我循环"] - B["第 2 批：成果交接单
TaskOutput 持久保存"] - C["第 3 批：完工回执
TaskCompleted 通知 coordinator"] - D["第 4 批：项目验收
Finality 自动且原子"] + A["Batch 1: Doorbell debounce\nWake cannot loop on itself"] + B["Batch 2: Durable handoff\nTaskOutput survives Sessions"] + C["Batch 3: Completion receipt\nTaskCompleted informs Coordinator"] + D["Batch 4: Project acceptance\nFinality is automatic and atomic"] A --> B --> C --> D ``` -### 第 1 批：切断 WakeNoop 自我循环 +### Batch 1 — Stop the WakeNoop loop -真人测试时的“蓝色运行态闪一下、停一下、又闪一下”，根因不是模型在反复思考，而是后台门铃在一个特殊窗口里自我循环：用户正在直接和某个 member 对话时，`intervention` 会故意暂停它的组织信箱读取；旧 race guard 又看到信仍未读，于是再次 wake；这个空 turn 结束后仍未读，于是继续 wake。 +The flashing blue state was not repeated model reasoning. During direct user intervention, Inbox drain is intentionally deferred so background work does not interrupt the user's turn. The old post-turn race guard still saw unread mail and immediately queued another Wake. That empty turn ended with the row unread and scheduled the next Wake indefinitely. -修复后： +The fix: -- `wake_one_member` 在排 scheduler 前检查 `AgentMemberInterventionStore`；直接用户对话期间返回 `DeferredIntervention`。 -- turn 结束后的 unread race guard 同时要求：run 仍为 Running、没有 intervention、确实有未读信。 -- 信仍保存在 `agent_inbox`，退出直接对话后再由正常路径读取，不会丢。 +- `wake_one_member` checks `AgentMemberInterventionStore` before scheduling and returns `DeferredIntervention`. +- The post-turn unread guard requires a Running Run, no active intervention, and actual unread data. +- The Inbox row remains durable and unread until Return to work or intervention expiry. ```mermaid flowchart TD - U["用户正在直接和 Member 对话"] --> I["Intervention active"] - I --> W["后台发现未读信，申请 Wake"] + U["User is directly controlling a worker"] --> I["Intervention active"] + I --> W["Background sees unread mail and requests Wake"] W --> D["DeferredIntervention"] - D --> K["信保留未读；不排空 turn；界面不闪"] - K --> R["Return to work 后正常 Wake + Drain"] + D --> K["Mail remains unread; no empty turn; no flashing"] + K --> R["Return to work enables normal Wake + drain"] ``` -### 实机日志复核：把 Coordinator intervention 的源头一起修掉 - -后来复查“龙之家族概括文”那次真实运行，确认闪烁时用户并没有继续给 Coordinator 发消息。准确时间线是：更早的一条普通 Coordinator 指令错误建立了 3 分钟 `intervention`；worker 随后给 Coordinator 写入未读信；旧系统一边禁止 intervention 中的 Coordinator 读信，一边又因未读信连续创建了 270 个顺序 Wake intent。也就是说，Wake 循环是后果，普通 Coordinator 消息被误分类才是更上游的根因。 +### Fix the upstream Coordinator-intervention cause -最终规则现在是： +A later log review proved that the user was not chatting with the Coordinator during the flashing period. An earlier normal Coordinator instruction had incorrectly created a three-minute intervention. Workers then wrote unread Coordinator mail. The system prohibited the Coordinator from reading it while continuously creating Wake intents for it. -| 用户动作 | 系统含义 | 是否建立 intervention | -| ------------------------------------------------------ | -------------------------------- | ----------------------------: | -| 在 root/Coordinator 对话框正常下指令 | 正常指挥整个 Agent Org | 否 | -| 切到 Planner/Implementer/Reviewer 等 worker 后直接聊天 | 用户临时接管这个 worker 的下一轮 | 是 | -| 在 Group Chat 发消息或 @成员 | 组织内正常投递 | 否，并清除目标旧 intervention | -| 点击 Return to work | 把 worker 交还给组织调度 | 清除 | +The final classification is: -```mermaid -flowchart TD - U["用户提交消息"] --> B["Backend 读取 session 的真实 member_id"] - B --> C{"是 coordinator 吗？"} - C -->|"是"| N["普通组织指令；不建立 intervention"] - C -->|"否，是 worker"| I["建立 worker intervention"] - N --> D["Coordinator turn 可正常 drain worker inbox"] - I --> P["暂缓后台 drain，直到 Return to work 或 TTL"] -``` +| User action | Meaning | Create intervention? | +| ----------------------------------------------------------- | --------------------------------------------- | ----------------------------------- | +| Normal instruction in Root/Coordinator input | Direct the Agent Org | No | +| Direct chat after switching to Planner/Implementer/Reviewer | Temporarily take over that worker's next turn | Yes | +| Group Chat message or member mention | Normal organization delivery | No; clear stale target intervention | +| Return to work | Return the worker to organization scheduling | Clear it | -实现上不是只改一个 UI 判断： +The backend uses the Session's canonical `member_id`; it does not infer role from the current page title. Generic `agent_send_message` no longer infers takeover from non-empty text. Duplicate intervention writes in the Rust adapter were removed. The Store refuses new Coordinator interventions and self-heals historical Coordinator records when read. Wake and lifecycle guards remain as defense for legitimate worker intervention. -- 前端 direct-submit 和 queued-dispatch 只负责报告“这是用户直接输入”； -- Backend 用持久 session 的 canonical `member_id` 做唯一分类，不能靠当前页面标题猜； -- 通用 `agent_send_message` 不再从“有文本”反推 intervention，避免计划续跑或内部 continuation 被误判为用户接管； -- Rust adapter 中重复写 intervention 的逻辑已删除，避免多层重复建立同一状态； -- store 层永久禁止新建 Coordinator intervention；旧版本留下的 Coordinator intervention 在读取时自动清除，升级后不会继续挡信； -- Wake 前置 gate 和 lifecycle race guard 仍保留，负责防住真正的 worker intervention，不再拿它们代替源头修复。 +### Batch 2 — Persist a cross-Session handoff -### 第 2 批：给跨 Session 的成果一张持久交接单 +A Task previously recorded only Pending, In progress, or Completed. It did not reliably preserve what was produced. Downstream workers cannot safely inspect arbitrary peer transcripts, so completion alone was insufficient. -以前 task 只有 `pending / in_progress / completed`，只知道“做完了”，没有可靠保存“做出了什么”。Reviewer 或 implementer 不能安全读取另一个 session 的聊天历史，所以 coordinator 可能看到上游 completed，却仍然不知道下游应该拿什么继续。 +`TaskOutput` adds: -新增 `TaskOutput`： +| Field | Meaning | +| ----------------------- | ------------------------------------------------ | +| `summary` | A short result summary | +| `content` | Bounded content that downstream work may consume | +| `artifact_ids` | Durable references for large files or artifacts | +| `produced_by_member_id` | Producer identity | +| `produced_at` | RFC3339 production time | -| 字段 | 大白话 | -| ----------------------- | ---------------------------------------- | -| `summary` | 一两句话的成果摘要 | -| `content` | 可直接交给下游的正文（最多 20,000 字符） | -| `artifact_ids` | 大文件或产物的持久引用 | -| `produced_by_member_id` | 谁做的 | -| `produced_at` | 什么时候做的，必须是 RFC3339 时间 | - -有下游依赖的上游任务，如果没有 `output`，系统不会接受 `completed`，而是返回明确 guidance。上游一旦合法完成，`TaskAssigned.dependency_outputs` 会把成果正文或 artifact 引用直接放进下游成员的真实 inbox turn；下游不需要猜 session id，也不需要翻别人的聊天记录。 +An upstream Task with downstream dependents cannot complete without a valid output. When it completes, `TaskAssigned.dependency_outputs` carries bounded content or artifact references into the next member's real Inbox turn. ```mermaid flowchart LR - P["Producer Session"] -->|"task_update completed + output"| T["Task 持久状态"] - T -->|"dependency_outputs"| I["下游 Inbox"] - I -->|"Wake + Drain"| R["Reviewer / Implementer Session"] + P["Producer Session"] -->|"task_update completed + output"| T["Durable Task state"] + T -->|"dependency_outputs"| I["Downstream Inbox"] + I -->|"Wake + drain"| R["Reviewer / Implementer Session"] ``` -### 第 3 批：把“成员空闲”和“任务完成”分成两封不同的回执 +### Batch 3 — Separate member idleness from Task completion + +`MemberIdle` means only that a member's current turn ended. It does not prove a Task completed. A system-only `TaskCompleted` receipt tells the Coordinator exactly which Task completed, who produced it, what its bounded output summary is, and how much open work remains. -`MemberIdle` 只表示“这个人这一轮结束了”，不能证明某张任务单已经完成。新增 system-only 的 `TaskCompleted`，明确告诉 coordinator：哪张 task、谁完成、输出摘要是什么、还剩多少 open task。 +- Only the current owner may set its Task to Completed. +- Completed Task state is monotonic; revisions create follow-up Tasks rather than reopening history. +- Transaction-local `TaskMutationOutcome` decides whether exactly one receipt is emitted. +- The Coordinator re-reads durable Task state before dispatching more work or replying to the user. -- 只有 task 当前 owner 能把它设为 completed；coordinator 不能替 member 猜完成。 -- completed task 的状态是单调的，store 层禁止退回 pending/in_progress；修改需求必须新建 follow-up task。 -- 完成事务产生的 before/after `TaskMutationOutcome` 决定是否发一次回执，避免并发更新重复通知。 -- coordinator 收到回执后重新读持久 task board，再决定继续派工还是给用户最终答复。 +### Batch 4 — Make Run finality a durable project acceptance step -### 第 4 批：把 run 的结束改成真正的“项目验收” +Previously all members could become Idle while the Run stayed Running. Manual Pause/Resume caused an extra Coordinator turn, making the system appear to discover completion only after user intervention. -过去所有成员都回到 Idle 后，run 可能仍保持 Running；用户 pause/resume 会额外叫醒 coordinator，于是看起来像“暂停再恢复才突然知道做完了”。现在每个 Agent Org turn 结束都会尝试 `reconcile_run_finality`，但只有下面条件在同一个 SQLite `IMMEDIATE` transaction 内仍同时成立，才会写 Completed： +Finality now evaluates durable facts under the same SQLite writer serialization as Task mutations. Completion requires: -1. task board 非空，且全部 task completed； -2. root coordinator 和所有 worker 都已静止； -3. 没有未读 inbox； -4. 没有 active intervention； -5. 没有 `optimistic / queued / running` 的未终结 turn intent； -6. coordinator 的最后一个完整 turn 晚于最后一次 task 更新，证明它已经看过最新事实并有机会给用户最终答复。 +1. a valid resolved Task board or an explicit empty-board completion intent; +2. quiescent Root and worker Sessions; +3. no unresolved unread Inbox delivery; +4. no active intervention; +5. no in-flight turn intent; +6. proof that the Coordinator observed the latest work revision and had a terminal opportunity to respond. -`reconcile_run_finality` 和所有 task mutation 共用 writer lock，所以“刚判定完成，另一个 turn 又创建 open task”的结果不可能写进数据库。 +The frontend projects explicit Run phases instead of making users infer state from animation: -前端新增 `runPhase`，用户不必再靠蓝色动画猜系统在做什么： +| Phase | Meaning | +| ----------------------------------------------------- | -------------------------------------------------------------------------- | +| `coordinating` | Coordinator is decomposing or organizing work. | +| `dispatching` | Durable messages are being delivered. | +| `members_working` | At least one member is executing or waiting for user/funds. | +| `waiting` | Open work exists but no member currently executes it. | +| `finalizing` | Tasks are resolved while final messages and Coordinator response converge. | +| `paused / completed / failed / cancelled / abandoned` | Durable Run state. | -| Phase | 界面含义 | -| ----------------------------------------------------- | ---------------------------------------------------------- | -| `coordinating` | coordinator 正在拆解/组织 | -| `dispatching` | 有持久消息正在派送 | -| `members_working` | 至少一个成员真正执行中或等待用户/额度 | -| `waiting` | 有 open task，但当前无人执行；恢复系统会继续检查 | -| `finalizing` | task 全部完成，正在等待最后消息排空和 coordinator 最终答复 | -| `paused / completed / failed / cancelled / abandoned` | run 的持久状态 | +### Additional defects found while auditing the four batches -### 四批追加审计发现并修正的尾部问题 +| Finding | Risk | Correction | +| ------------------------------------------------------------------- | ----------------------------------------------------------------------- | ------------------------------------------------------------------------------- | +| Historical tests reopened Completed Tasks | Tests encouraged an invalid lifecycle. | Use a legal subject update and release while preserving history-order coverage. | +| Finality initially checked only `queued` intents | `optimistic` and `running` windows could be missed. | Any non-terminal intent blocks finality. | +| New limits used UTF-8 byte length but documentation said characters | Non-ASCII content received a much smaller effective limit. | Character and byte limits are explicit and independently enforced. | +| `produced_at` deserialized without timestamp validation | Direct Store calls could persist unauditable time values. | Store enforces RFC3339 and tests malformed metadata. | +| UI active count omitted `waiting_for_funds` | Backend considered a member active while UI showed zero active members. | Phase and activity badge use the same status set. | -| 发现 | 为什么危险 | 最终修正 | -| ------------------------------------------- | ----------------------------------------------------------- | ------------------------------------------------------------------ | -| 旧 history 测试会把 completed task 重新打开 | 测试在鼓励已经禁止的非法生命周期 | 改成合法的 subject update 后再 release，仍覆盖 history 顺序 | -| Finality 最初只检查 `queued` intent | `optimistic` 或状态同步窗口里的 `running` intent 可能被漏掉 | 改为任何未终结 intent 都阻止收尾，并覆盖三种状态 | -| 新输出上限使用 UTF-8 byte 长度却写“字符” | 中文会被按 3 个 byte 计算，实际限制比文案小很多 | 所有新 output/dependency summary/content 限制统一按 Unicode 字符数 | -| `produced_at` 只反序列化、不验证时间 | 直接 store 调用可留下无法审计的伪时间 | store 层强制 RFC3339，并加 malformed metadata 回归 | -| UI 活跃数漏掉 `waiting_for_funds` | 后端认为成员工作中，界面却显示 active=0 | 前端 phase 和 active badge 使用同一状态集合 | +## Sequential dispatch and durable Wake-intent fixes -## Breaking Bad 真人运行后的顺序派工与 Wake intent 修复 +### Explicit dispatch policy -### 1. 任务不再因“漏写 blocked_by”静默抢跑 +The old `task_create` defaulted to `blocked_by=[]`. If the Coordinator omitted dependencies, Reviewer and Tester received `TaskAssigned` immediately even when their descriptions required upstream output. -旧 `task_create` 把 `blocked_by=[]` 当成默认值。Coordinator 只要忘记填这个数组，Reviewer 和 Tester 就会立刻收到 `TaskAssigned`，即使任务描述明明写着“在概述和审核结果基础上”。现在工具协议强制每次创建任务明确选择： +The tool now requires an explicit policy: -| 新字段 | 含义 | 何时派工 | -| -------------------------------------------------------------- | ---------------------------- | --------------------------- | -| `dispatch_policy="immediate"` | 当前信息已经足够、可独立开工 | 创建后立即派工 | -| `dispatch_policy="after_dependencies"` + `dependency_task_ids` | 必须消费上游持久成果 | 所有上游 completed 后才派工 | +| Field | Meaning | Dispatch timing | +| -------------------------------------------------------------- | ---------------------------------------------------------- | -------------------------------- | +| `dispatch_policy="immediate"` | Current information is sufficient and work is independent. | Immediately after creation | +| `dispatch_policy="after_dependencies"` + `dependency_task_ids` | Work must consume durable upstream results. | After every dependency completes | -`blocked_by` 仍是数据库里的底层表示，但不再暴露为 `task_create` 的易漏默认项。新协议还拒绝空依赖、未知 task id、self-cycle，以及 `immediate` 偷带 dependency id。最初使用 Rust tagged enum 会产生部分 LLM provider 不支持的 `oneOf`；审计测试捕获后，wire schema 改成扁平字符串 + 数组，在工具边界解析为 typed `TaskDispatchPolicy`。 +The wire schema uses provider-compatible flat fields and parses them into typed `TaskDispatchPolicy` at the tool boundary. It rejects empty dependencies, unknown Task ids, self-cycles, and dependency ids attached to `immediate`. A dependency-confirmation gate returns structured guidance when non-transitively open work appears to have been omitted; creation proceeds only after the Coordinator adds the dependency or explicitly confirms safe parallelism. ```mermaid flowchart LR - I["Implementer
immediate"] -->|"completed + TaskOutput"| R["Reviewer
after_dependencies"] - R -->|"completed + reviewed output"| T["Tester
after_dependencies"] - T -->|"completed"| C["Coordinator final"] + I["Implementer\nimmediate"] -->|"Completed + TaskOutput"| R["Reviewer\nafter_dependencies"] + R -->|"Completed + reviewed output"| T["Tester\nafter_dependencies"] + T -->|"Completed"| C["Coordinator final response"] ``` -三阶段回归测试证明：初始只有 Implementer 收到任务；Implementer 完成后 Reviewer 才收到且带上游 output；Reviewer 完成后 Tester 才收到且带审核 output。 - -### 2. Scheduler 负责把每个 enqueue 决定写成终态 - -旧路径会先写一条 `queued` intent，再调用 scheduler。scheduler 发现相同 `client_message_id` 时正确返回 duplicate，但新 intent id 没人更新，于是它永远停在 queued，阻止 `reconcile_run_finality`。现在新增两个精确终态： - -- `coalesced`：请求与已排队/运行的同一逻辑 turn 合并，本 intent 从未执行； -- `rejected`：queue full/closed，本 intent 从未被接受。 - -更新动作放进 Scheduler 本身，而不是要求 message、Wingman 或未来入口分别善后。`coalesced/rejected/stale` 都是 pre-durable terminal，不创建聊天 round，也不阻塞 Run 收尾。20 个并发相同 Wake 仍是 1 个 accepted、19 个 coalesced，但现在数据库 20 条 intent 全都有正确归宿。 +### Scheduler-owned terminal intent disposition -### 3. 升级后自动修旧数据，启动时先收尾再暂停 +The old path wrote a queued intent before calling the scheduler. The scheduler correctly detected a duplicate `client_message_id`, but the new durable intent remained queued forever and blocked finality. -进程重启后，内存 scheduler 已消失，因此旧 `optimistic/queued` 必然不可能再执行，启动时统一转为 `stale`；旧 `running` 转为 `failed`。实机数据库确认 Breaking Bad 的坏 intent `a7e69850-3f29-4e1c-a439-f35d1f53b659` 已从 `queued` 变为 `stale`。 +The scheduler now persists precise terminal outcomes: -启动顺序现在是：intent 自愈 → 用 durable terminal marker 修正已结束 session → 其余 in-flight session abandoned → task failure disposition → 清 intervention → 原子完成所有 task 已 resolved 的 Run → 只暂停仍需继续的 Run。这样真正未完成的项目仍可恢复，而已经完成、只是被假 queued 卡住的项目不再要求用户手动 Resume 才收尾。debug `simulate-app-restart` 与生产入口使用相同七步顺序，并通过 pause/resume rendered E2E。 +- `coalesced`: the logical turn was already queued or running, so this intent never executed; +- `rejected`: the queue was full or closed, so this intent was never accepted. -## 实际改了什么 +`coalesced`, `rejected`, and `stale` are pre-execution terminal states. They create no chat round and do not block finality. Twenty concurrent identical Wakes produce one accepted turn and nineteen coalesced intents, with every database row reaching an accurate disposition. -### 1. Watchdog 从“边看边改”改为“先分析、后执行” +### Startup healing -- `inspect_stalled_run` 只读取 run、tasks、sessions、inbox 和 budget，生成可组合计划。 -- 一轮可以同时唤醒 Alice、让 Bob 接 ready task、把坏任务交给 coordinator，并尝试 reconcile。 -- 所有 `SessionStatus` 都被显式分类；没有用 `_ => false` 把未来状态静默吞掉。 -- 保留已决定的 E3 限制：任一 worker 处于 `Running`、`WaitingForUser` 或 `WaitingForFunds` 时，不做 peer 自动恢复；只允许发 stale/unsupported 的观察性 coordinator notice。 -- `Pending` 有 2 分钟 materialization grace；`Paused` 不自动 wake；Missing、Archived、历史 CLI transport 会升级给 coordinator。 +After process restart, the previous in-memory scheduler no longer exists. Historical `optimistic` or `queued` intents become `stale`, while historical `running` intents become `failed`. -### 2. Wake 真正幂等，并把 Running 状态放回正确时刻 +Startup order is now: -- Agent Org wake 使用 `agent-org-wake:{run_id}:{member_id}`。 -- 20 个并发相同 wake 只接受一个 turn，其他返回 coalesced。 -- enqueue 前不再把 session 写成 Running；scheduler 真正执行 turn 时才更新。 -- 对 Agent Org wake，run 复核和 session→Running 在同一 writer lock + SQLite `IMMEDIATE` transaction 中完成。 -- wake 排队后 run 若变为 Paused/terminal，执行时直接 no-op，不 drain inbox，不调用 provider。 -- drain 后没有任何真实输入时返回空结果，不制造空 nudge，也不花一次 provider call。 +1. reconcile stale intents; +2. apply durable terminal markers to ended Sessions; +3. mark remaining in-flight Sessions abandoned; +4. apply Task failure disposition; +5. clear interventions; +6. atomically finish Runs whose work is resolved; +7. pause only Runs that still require work. -### 3. Recovery budget 持久化 +The debug restart simulator and production startup use the same ordered routine. -- 新增 `agent_org_recovery_attempts`，按 `(run, action_kind, target)` 保存指纹、次数和 UTC deadline。 -- 退避为 1/5/15 分钟，重启后仍然有效。 -- wake 只有真正 Enqueued 才消耗 member rewake attempt；coalesced 或 enqueue failure 不消耗。 -- member 成功 turn 后清除自己的预算；repair reason 指纹改变时 coordinator notice 自动获得新预算。 -- 非 Running run 的预算定期清理。 -- 损坏的 `next_allowed_at` 现在 fail-open 并告警；不会因为坏时间戳永久压住恢复，下一次成功动作会覆写为合法 UTC。 +## Implementation summary -### 4. Task eligibility 和显式指派逻辑统一 +### Recovery Analyzer and Executor + +- `inspect_stalled_run` reads Run, Task, Session, Inbox, and budget facts and returns a composable plan without side effects. +- One plan may contain member Wakes, Coordinator repair notices, and reconciliation. +- Every `SessionStatus` is explicitly classified. +- E3 remains an explicit limitation: while a worker is Active, the system does not perform general peer auto-recovery; unavailable unread recipients are still diagnosed. +- Pending Sessions receive a two-minute materialization grace. Paused Sessions are not woken. Missing, Archived, and historical CLI transport are escalated. +- Executor revalidates Run, Session, recipient, Task graph, fingerprint, and real work before committing an action. -> 2026-07-14 设计更新：后续真实运行证明自动领取会让任务责任边界变模糊，因此本节原有 claim 方案已被显式 Coordinator 指派取代。 - -- Ownerless 只表示“当前没人负责”，不会触发 Worker 自动领取、自动 Wake 或执行模式切换。 -- Watchdog 看到 ready ownerless task 时只通知 Coordinator 明确选择 `owner_member_id`；Inbox drain 和 resume 不改变任务 owner。 -- `eligible_member_ids` 现在是 Coordinator 可选择的候选白名单，不是 Worker 的抢单许可。 -- 新建 ownerless pending task 必须有非空 eligibility；owner 和 eligibility 必须属于 launch snapshot roster。 -- `metadata` 必须是 object，保留字段类型和值会在 store 层再次校验，不能只相信 tool 层。 - -### 5. 成员失败、shutdown 和 restart 使用明确 disposition - -成员异常失败： - -```text -open task - -> owner=NULL, status=pending, metadata 保留 - -> 不 wake failed owner，也不 wake eligible peer - -> coordinator 收到 awaiting_coordinator_assignment + task ids - -> coordinator 明确选择新的 owner 后，TaskAssigned 才能唤醒该成员 -``` - -这条最终语义刻意不再区分“是否存在 peer”：eligibility 只是 Coordinator 的候选白名单，不是 Worker 的抢单许可。 - -成员明确接受 shutdown： - -```text -有合法 peer -> released_to_pool -没有合法 peer -> owner=coordinator, status=pending -``` - -第二条是本轮审计补上的关键边界。主动 shutdown 是行政性停止，不应再被当成 provider failure 自动复活；交给 coordinator 后，任务仍合法、仍可见、但不会把已离开的成员叫回来。 - -App restart 时，遗留 Running session 先转为 Abandoned，再走相同的 failure disposition，然后 run 转为 Paused，等待用户明确恢复。扫描不再只取前 500 个 Running run。 - -### 6. Run finality 与 task mutation 原子化 - -- create/update/delete/reassign/requeue/shutdown disposition 都在 transaction 内检查 parent run 必须是 Running。 -- `reconcile_if_terminal` 在统一 writer lock + `IMMEDIATE` transaction 内读取 run、root session、worker sessions 和全部 task 状态，再 CAS 写 terminal status。 -- 新增 reconcile 与并发 task_create 测试；合法结果只能是：reconcile 先赢、create 被拒绝，或 create 先赢、run 因 open task 进入 Abandoned。 -- update 返回事务内的 `TaskMutationOutcome`，side effect 依据 before/after transition 触发，避免并发更新重复发送 TaskAssigned。 - -### 7. 删除 stale 自动抢任务 - -- 删除按 15 分钟年龄自动清 owner 的 production path、debug endpoint、常量和测试。 -- Running session 的旧任务只会提醒 coordinator，不会被 watchdog 偷走。 -- 只有明确 Failed/Cancelled/Abandoned/Timeout、accepted shutdown 或 restart recovery 才能改变 ownership。 -- 无法解析的 task/session timestamp 会告警并生成 repair，不会永久静默。 - -### 8. CLI Agent Org 能力边界变得诚实 - -当前 CLI member 没有 production inbox drain、Agent Org task tools 和正确的 resume bridge，因此本 PR 不再假装它能运行： - -- 新建/更新 org 时拒绝 CLI coordinator/member。 -- launch 前再次 preflight，必须在创建 run/root session 前失败。 -- 错误包含 `member_id` 和 `cli:*` transport。 -- Agent Org 设置和 Session Creator 的成员选择器不再展示 CLI agent。 -- 已有旧定义仍能反序列化和打开，用户可删除不支持的 CLI member。 -- 删除了宣称 CLI Agent Org 已可运行的正向 E2E；保留历史数据解析和负向 validation 覆盖。 - -## 用户现在看到的界面与行为 - -| 场景 | 现在的界面/行为 | -| ----------------------------------- | ------------------------------------------------------------------------------------------------------------------ | -| 新建 Agent Org | coordinator 和 member 下拉只显示 Rust-native built-in/custom agents，不显示 CLI agents。 | -| 编辑旧 CLI Agent Org | 旧数据仍能打开；CLI 选项不会继续提供，保存前需移除/替换不支持成员。 | -| 启动含 CLI member 的旧 Org | 启动前明确报错，且不会留下半成品 run 或 root session。 | -| Run 被暂停 | unread inbox 保留，后台 wake 不调用 provider；恢复后由正常 progress wake 继续。 | -| Run 已结束 | task mutation 返回结构化不可变 guidance；排队中的 wake 执行时 no-op，不会复活成员。 | -| 成员失败 | 看板任务按 eligibility 释放或保留；coordinator 收到含 disposition、eligible_member_ids、required_role 的恢复说明。 | -| 成员主动 shutdown | 有 peer 时回池；无 peer 时交给 coordinator，不再自动复活已停止成员。 | -| 多个来源同时 wake | 用户不会看到多个空 turn；scheduler 只执行一个，其余 coalesced。 | -| 用户给 Coordinator 普通指令 | 不显示 intervention pin；Coordinator 仍能在该 turn 读取 worker 发来的未读信。 | -| 用户直接切到 worker 聊天 | 只对该 worker 显示 intervention pin；后台信保留，Return to work 后继续。 | -| 升级前留下 Coordinator intervention | 第一次读取时自动清除，不再要求用户 Pause/Resume 才恢复。 | -| Running 成员长时间没更新 | 只提醒 coordinator，不自动清 owner。 | -| App 重启 | 旧 intent 先收口；任务已全部 resolved 的 run 尝试原子完成；仍有工作/未读交接的 run 才暂停并保持看板可见。 | - -## 本轮代码审计发现并修复的问题 - -| 优先级 | 发现 | 风险 | 修复 | 状态 | -| ------ | --------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------- | -| P0 | debug task seed 使用错误 crate 根路径 | `agent_core` check 通过但顶层 `org2` 无法编译 | 改为公开的 `agent_core::...` import，并补跑 `cargo check -p org2` | 已修复 | -| P0 | 普通 Coordinator 指令被当成 worker takeover | 未读 worker 回执与 3 分钟 intervention 冲突，形成 270 个顺序 Wake intent 和蓝色闪烁 | Backend 按 canonical member_id 分类；Coordinator 永不建立 intervention；store 自愈旧记录；真实 UI 回归覆盖 | 已修复 | -| P1 | 通用 `agent_send_message` 从“非空文本”推断 intervention | 自动 continuation 或内部程序化消息也可能被当成用户接管 worker | 只有 direct-submit / queued-dispatch 显式发 takeover 信号；generic command 不再推断；direct-member API 保留显式语义 | 已修复 | -| P0 | accepted shutdown 的 sole-owner task 仍留给 Cancelled member | watchdog 会把主动停止成员当异常终止重试 | 无 peer 时把 owner 改为 coordinator，并写 `escalated_to_coordinator` history | 已修复 | -| P1 | failure hook 按 eligibility 唤醒 peer | Worker 会在未经过 Coordinator 明确分配时收到任务 Wake | 删除失败后的 peer 自动领取/Wake；任务变成 ownerless pending，并向 Coordinator 报告具体 task ids | 已修复 | -| P1 | recovery deadline 损坏时 budget probe 返回错误 | member 既不重试也不升级，可能永久静默 | 告警并 fail-open，成功动作覆写 deadline；新增测试 | 已修复 | -| P1 | restart recovery 只扫 500 个 running run | 大量历史 run 时后面的遗留任务得不到 disposition | 使用 `usize::MAX`，SQL 安全 clamp 到 `i64::MAX` | 已修复 | -| P2 | CLI 保存错误只显示成员名字 | 同名成员难定位，未满足 validation contract | 错误加入 `member_id=` 和 `cli:*` | 已修复 | -| P2 | 全局 rustfmt/prettier 带入无关格式变化 | 增加 review 噪声 | 逐文件反向清除确认的 formatting-only diff | 已修复 | -| P0 | Review/Test task 漏写 `blocked_by` 会静默当成可立即执行 | Reviewer/Tester 在没有上游产物时反复空跑，真实顺序与任务描述矛盾 | `task_create` 强制 `dispatch_policy`；依赖策略映射为底层 `blocked_by`；三阶段顺序测试 | 已修复 | -| P0 | The Boys 实机中 Coordinator 文字说等待 Implement/Review，结构化依赖却只列 Planner | Scheduler 忠实执行错误图，Tester 在 Implementer 完成前先交付 | `after_dependencies` 漏列非传递 open task 时返回 `requires_dependency_confirmation` 且不落库；补齐依赖或显式 `allow_parallel_with_unlisted_open_tasks=true` 后才创建 | 已修复并经后续真人运行复验 | -| P0 | scheduler 合并 duplicate Wake 后新 intent 仍为 queued | 所有人完成后 Run 永久无法 finality，用户必须 Pause/Resume | Scheduler 原子收口为 `coalesced`；queue full/closed 为 `rejected` | 已修复 | -| P1 | 旧版本遗留 queued intent 升级后仍卡住 | 修复只对新数据有效，历史 Run 仍坏 | 启动时 queued/optimistic→stale、running→failed；实机坏行已自愈 | 已修复 | -| P1 | 所有 task 已完成的 Running Run 在 restart 后仍被通用策略暂停 | 用户仍需手动 Resume 才完成 | 启动 pause sweep 前先走正常原子 finality，仅暂停仍需工作的 Run | 已修复 | -| P1 | `simulate-app-restart` 与生产启动序列漂移 | rendered E2E 测到的是假流程 | debug endpoint 与生产统一为七步同序，并扩展 typed E2E result | 已修复 | -| P1 | tagged dispatch enum 生成 LLM provider 不兼容的 `oneOf` | Coordinator 可能无法调用 task_create | wire schema 扁平化，边界内再解析为 typed enum；schema portability 测试 | 已修复 | - -审计结束时没有遗留本次范围内的 P0/P1/P2 actionable finding。 - -## Architecture Audit：10 层检查 - -| Layer | 检查内容 | 结论 | -| -------------------------- | ----------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| 1. Compilation correctness | `org2`、`agent_core`、`e2e-test`、TypeScript、ESLint、Node syntax | 本次改动编译/类型/语法通过；Agent Core Clippy 有 36 条仓库既有基线，见验证章节。 | -| 2. Dead code / duplication | 从 watchdog、resume、task tools、failure finalize、scheduler 业务入口正向追踪 | 删除 dead `find_available`、`blockers_resolved`、旧 stale-release path、旧 wake wrapper、自动 claim helper 和失真的 CLI 正向 E2E。Ownerless 统一进入 Coordinator repair。 | -| 3. Naming consistency | 搜索 stale/release/wake/owner 旧名和注释 | stale 常量只表示 notice，不再暗示 release；failure 与 shutdown history 使用不同 disposition 名称。旧符号 sweep 为零。 | -| 4. Semantic overloading | Run / Session / Task / Delivery / Budget 对照 | 五个状态源独立持久化。Idle 不代表 terminal，unread 不代表 wake accepted，Pending/eligible 不代表 Worker 可以自领。 | -| 5. Default branches | 审查 `SessionStatus`、run status、wake outcome 和时间戳 fallback | Session recovery 显式穷举；DB/read error fail-closed；损坏 deadline/timestamp 不静默。E3 是具名且有测试的限制。 | -| 6. Cross-domain leakage | Rust/CLI transport、core task store、UI picker | 历史 CLI 只在边界被识别，不再落入 Rust wake；shared picker 用 `hideCliAgents` 参数，不改变其他 CLI-only surface。 | -| 7. New-developer clarity | 函数/事件/注释是否准确表达副作用 | Analyzer、executor、doorbell、disposition、finality transaction 的职责明确；shutdown 不再复用 failure 的 kept-owner 含义。 | -| 8. Wire / serialization | scheduler response、task metadata、inbox payload、tool guidance | scheduler 的 duplicate/rejected 映射为持久终态；task dispatch schema 为 provider-compatible 扁平字段，边界内解析 typed policy；依赖遗漏用成功形态的 structured guidance 返回，不制造 trajectory error；typed metadata 在 tool + store 双层校验。 | -| 9. Init parity | production setup、unit sandbox、debug seed、restart path | recovery schema 在 production 和测试初始化；production startup 与 `simulate-app-restart` 同为七步；debug seed 生成合法 eligibility；launch preflight 在 run 创建前执行。 | -| 10. Resolver symmetry | member identity、assignment、run gate、session resolution | watchdog/resume/task/failure 都把 ownerless 解释为等待 Coordinator 指派；身份始终以 member_id 为主；wake 的 run_id 用 typed 参数，不从字符串反解析。 | - -## 状态所有权表 - -| 状态维度 | 唯一真相源 | 允许从其他状态猜吗？ | -| ------------------- | ------------------------------ | -------------------- | -| Run 是否允许继续 | `agent_org_runs.status` | 不允许 | -| Member 是否正在执行 | `agent_sessions.status` | 不允许 | -| Task 归属/完成/依赖 | `agent_org_tasks` | 不允许 | -| 信是否未读 | `agent_inbox.read_at` | 不允许 | -| Wake 是否已排队 | scheduler idempotency registry | 不允许 | -| 自动恢复是否可再试 | `agent_org_recovery_attempts` | 不允许 | - -## 入口一致性矩阵 - -| 入口 | Running gate | 显式 assignment | 持久 inbox | 幂等 wake | Budget | -| ------------------------------ | ---------------------------------------------------------------------------: | --------------------------------: | -----------------: | --------------------------------: | ---------------------: | -| Watchdog | analyzer + executor + turn start | ownerless 只 repair Coordinator | 是 | 是 | 是 | -| Resume progress | dispatcher + turn start | 只投递已有 owner 的真实输入 | 是 | 是 | terminal retry 才计 | -| Task create/update side effect | mutation transaction + dispatcher | Coordinator 写 owner 后才投递 | 是 | 是 | terminal retry 才计 | -| Member failure finalize | task transaction + dispatcher | 清 owner，等待 Coordinator | coordinator notice | 是 | 不在 failure hook 消耗 | -| Accepted shutdown | task transaction | peer pool / coordinator ownership | MemberTerminated | coordinator wake 合并 | 不复活 stopped member | -| App restart | intent reconcile + failure disposition + resolved finality + remaining pause | 是 | 保留 | 旧队列收口；resume 后新 wake 生效 | 持久保留/按状态清理 | - -## 验证记录 - -| 命令/检查 | 结果 | -| -------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `cargo check -p org2` | 通过 | -| `cargo check -p e2e-test` | 通过 | -| Watchdog tests | 21 通过 | -| Inbox drain tests | 30 通过 | -| Lifecycle tests（单线程隔离） | 5 通过 | -| Task store/tool/run finality/scheduler targeted tests | 通过；41 个 task tool 测试含三阶段依赖链、The Boys 同形漏依赖确认、明确并行保留，scheduler 含 20 并发 wake，另含 startup intent/finality 与 reconcile/create race | -| `cargo test -p agent_core --lib -- --test-threads=1`（沙箱外） | **2951 通过，2 个既有基线失败** | -| `pnpm typecheck` | 通过 | -| `pnpm run lint` | 通过；修复 1 个本 diff 内的 Prettier 换行后全量复验通过 | -| `pnpm run check:circular` | 通过；5073 files，零 circular dependency | -| 两份修改的 `.mjs` 执行 `node --check` | 通过 | -| 隔离 desktop/WebDriver `agent-org-recovery-ui.spec.mjs` | **2/2 真实渲染场景通过**；证明 Coordinator 普通指令不建立 intervention、可 drain Planner 未读信，同时 Planner 直接聊天仍建立 intervention | -| 隔离 desktop/WebDriver `agent-org-pause-resume-ui.spec.mjs` | **8/8 真实渲染场景通过**；覆盖生产同序 restart simulation、暂停/恢复、历史 Run 自动恢复、成员/Coordinator 历史和侧栏 | -| `session_persistence` 单线程全量 | **31/32 通过**；唯一失败是既有测试硬编码另一开发者 `/private/var/folders/10/.../Users_junyu/...` 临时路径，与本 diff 无关 | -| `git diff --check` | 通过 | -| stale/dead symbol sweep | 旧 stale release、旧 wake helper、旧 availability helper均为 0 命中 | - -完整 Rust suite 的两个已知失败： - -1. `core::session::turn::entry::tests::skill_slash_command_accepts_newline_after_name`：测试期待 bundled e2e-testing skill content，但本地返回原始 slash command；与 Agent Org 改动无关。 -2. `core::tools::search_tool_tests::repo_path_and_repo_paths_conflict_returns_error`：search tool 既有参数冲突行为与测试期待不一致；与本 PR 无关。 - -`cargo clippy -p agent_core -p session_persistence --all-targets --no-deps -- -D warnings` 仍被仓库既有 Clippy 基线挡住，分布在 interaction、provider、memory、session launch 等旧代码；包含 `doc_lazy_continuation`、`too_many_arguments`、`clone_on_copy` 等。此次新增的 dispatch policy、scheduler intent 终态、startup recovery 和 parity 入口没有新增 Clippy 命中，按本 PR 约定未扩大为无关重构。 - -本机已构建带 WebDriver automation 的最新桌面二进制，并在全新隔离 home 上运行 `agent-org-recovery-ui.spec.mjs`：失败成员恢复 UI、默认 Agent Org 启动、生产 run phase、Coordinator 普通指令读取 Planner inbox、worker-only intervention 共 2 个 rendered 场景全部通过。数据库证据中 Planner→Coordinator 的目标信从 unread 变为 read，同时 intervention 表只出现 `sde-planner`，没有 Coordinator 记录。未运行与本轮直接修复无关的整套 `agent-org-settings-ui.spec.mjs`。 - -## 有意保留的设计边界 - -- **E3 仍延后**：一个 worker active 时，暂不做 member-level peer recovery。代码注释和测试明确锁定，未来应作为独立设计升级。 -- **Eligibility 仍在 metadata JSON**：目前通过 typed tool + store invariant + SQL JSON1 加固；join table 规范化留给独立 schema PR。 -- **CLI Agent Org 是明确禁用，不是“半支持”**：完整支持需要 CLI inbox drain、Agent Org tools、resume bridge 和 production E2E。 -- **DispatchCategoryPalette 有两套相近 option-building 实现**：本次为保持行为一致同时加了 `hideCliAgents`；后续可单独抽共享 selector，避免 UI picker 规则双写。本 PR 不为此扩大重构范围。 -- **`WakeNoop` 是执行期结果**：request 侧能区分 Enqueued/Coalesced/Paused/Terminal/Unavailable/Failed；排队后工作消失由 processor no-op 并记录日志，当前没有再向最初 caller 反向传播异步 NoWork。 - -## Commit-ready 判定 - -自动化层面可以提交，理由： - -- 本次范围内的编译、类型、lint、targeted tests 和 diff hygiene 均通过。 -- 完整 Rust suite 中所有 Agent Org/Recovery/Task/Wake 测试通过。 -- 两个全量失败和 36 条 Agent Core Clippy 报告均为可复现的既有基线，未由本 diff 引入；workspace Clippy 还会先被未修改的 `cursor-bridge-app` `question_mark` 告警阻断。 -- 审计发现的 P0/P1/P2 已全部修复并复验。 -- 无关 untracked 文件 `.atomcode/`、`docs/PR-GUIDE-issue-194.md`、`docs/cli-agent-launch-args-plan-2026-07-04.md` 未被修改或纳入本次范围。 - -但按仓库 `github-issue-fix-workflow`，The Boys 之后新增的依赖确认门属于可见调度行为变化，仍需用户在最新 dev 上做一次真人 Agent Org 验收。验收通过后才能把结论更新为最终 commit-ready 并执行 Husky commit。 - -建议提交标题： +### Wake idempotency and timing + +- Agent Org Wake uses `agent-org-wake:{run_id}:{member_id}`. +- Session Running is persisted only when the scheduler actually begins the turn. +- A queued Wake whose Run later becomes Paused or terminal exits without draining Inbox or calling the provider. +- A turn with no real durable input returns WakeNoop instead of injecting an empty nudge. +- Persistent 1/5/15-minute recovery budgets survive restart. Only Enqueued consumes an attempt; coalesced and rejected requests do not. +- Corrupt budget deadlines are diagnosed and cannot permanently suppress recovery. + +### Task authority and ownerless semantics + +- Ownerless means “currently unassigned.” It never authorizes worker self-claim, automatic Wake, or execution-mode change. +- Watchdog reports ready ownerless work to the Coordinator for explicit assignment. +- `eligible_member_ids` is a candidate allowlist for Coordinator assignment, not worker administrative authority. +- Owner and eligibility must belong to the immutable launch roster. +- Reserved metadata fields are typed and revalidated in the Store. + +### Failure, shutdown, and restart disposition + +Worker failure releases open work to ownerless Pending state, preserves metadata, and sends the Coordinator an explicit assignment notice. It does not wake the failed owner or an eligible peer automatically. + +Accepted shutdown uses administrative disposition: work may return to a valid peer pool, or ownership is escalated to the Coordinator when no peer can accept it. A deliberately stopped member is not revived as though it suffered provider failure. + +App restart marks leftover Running Sessions abandoned before applying the same failure disposition. Recovery queries select all Running Runs directly in SQL rather than truncating an arbitrary 500-row mixed-status list. + +### Atomic finality and Task mutation + +- Create, update, delete, reassign, requeue, and shutdown disposition validate a Running parent Run inside the transaction. +- Reconciliation reads Run, Root, workers, Tasks, Inbox, interventions, and intents under one serialized finality transaction. +- Concurrent reconcile/create tests permit only serializable outcomes. +- Transaction-local outcomes drive `TaskAssigned`, `TaskCompleted`, and dependency-unblock side effects exactly once. + +### Stale ownership and CLI boundary + +- A merely old Running Session no longer loses Task ownership. Staleness produces a Coordinator notice only. +- Ownership changes require explicit Failed, Cancelled, Abandoned, Timeout, accepted shutdown, or restart recovery semantics. +- Invalid timestamps produce diagnosis rather than silent behavior. +- New or updated Agent Orgs reject CLI Coordinator/member definitions because CLI lacks production Inbox drain, Agent Org tools, and a correct resume bridge. +- Launch preflight fails before creating a Run or Root Session and names the unsupported `member_id` and `cli:*` transport. +- Historical definitions remain readable so users can remove unsupported members. + +## User-visible behavior + +| Scenario | Current behavior | +| ----------------------------------- | ------------------------------------------------------------------------------------------------------------------------ | +| Create Agent Org | Coordinator/member selectors show supported Rust-native built-in and custom agents, not CLI transports. | +| Edit historical CLI Agent Org | Existing data opens, but unsupported members must be removed or replaced before saving. | +| Launch old Org containing CLI | Clear preflight error; no partial Run or Root Session is left behind. | +| Pause Run | Unread Inbox remains durable; background Wake does not call the provider; Resume continues through normal progress Wake. | +| Terminal Run | Task mutation returns structured immutable guidance; queued Wakes no-op and never revive members. | +| Member failure | Work becomes visible ownerless Pending work and the Coordinator receives exact assignment guidance. | +| Accepted shutdown | Work returns to a valid pool or the Coordinator; the stopped member is not revived. | +| Concurrent Wake sources | The user sees one real turn; duplicate requests become Coalesced. | +| Normal Coordinator instruction | No intervention is created; the Coordinator can drain worker replies in the same turn. | +| Direct worker chat | Only that worker gains intervention; Return to work restores organization scheduling. | +| Historical Coordinator intervention | The Store removes it on read so it cannot continue blocking delivery after upgrade. | +| Old Running member | Coordinator receives a stale notice; ownership is not stolen by time. | +| App restart | Intents heal first; resolved Runs finalize; only Runs with remaining work are paused. | + +## Audit findings and resolution + +| Priority | Finding | Risk | Resolution | Status | +| -------- | -------------------------------------------------------------------- | ---------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------- | ------ | +| P0 | Debug Task seed imported the wrong crate root | `agent_core` compiled while top-level `org2` did not. | Use the public `agent_core::...` path and check top-level `org2`. | Fixed | +| P0 | Normal Coordinator instructions created worker-takeover intervention | Unread replies conflicted with intervention and generated hundreds of Wakes. | Classify by canonical `member_id`; prohibit Coordinator intervention; self-heal old rows. | Fixed | +| P1 | Generic message code inferred intervention from non-empty text | Internal continuation could look like user takeover. | Only direct-submit paths send an explicit takeover signal. | Fixed | +| P0 | Accepted shutdown left sole-owner work on a Cancelled member | Watchdog could revive an administratively stopped worker. | Escalate ownership to Coordinator when no peer exists. | Fixed | +| P1 | Failure hook woke an eligible peer | Workers received work without explicit assignment. | Remove automatic peer claim/Wake; report exact Task ids to Coordinator. | Fixed | +| P1 | Corrupt recovery deadline suppressed all action | A member could remain permanently silent. | Diagnose, fail open for recovery evaluation, and overwrite with a valid UTC deadline after a successful action. | Fixed | +| P1 | Restart recovery scanned only 500 Runs | Older active Runs could miss disposition. | Query Running Runs directly without the arbitrary cap. | Fixed | +| P2 | CLI validation named only the display name | Duplicate names were not diagnosable. | Include `member_id` and `cli:*` transport. | Fixed | +| P2 | Global formatting introduced unrelated diffs | Review noise obscured behavioral changes. | Revert unrelated formatting-only changes and use changed-scope formatting. | Fixed | +| P0 | Omitted dependency silently dispatched Review/Test work | Downstream members repeatedly ran without upstream output. | Require explicit dispatch policy and validate dependencies. | Fixed | +| P0 | Narrative dependencies disagreed with structured dependency ids | Scheduler correctly executed the wrong graph. | Add a side-effect-free dependency-confirmation gate or explicit safe-parallel acknowledgement. | Fixed | +| P0 | Duplicate Wake left a durable queued intent | Finality was blocked until manual Pause/Resume. | Scheduler persists Coalesced or Rejected terminal outcomes. | Fixed | +| P1 | Historical queued intents remained stuck after upgrade | New fixes did not repair old Runs. | Startup maps queued/optimistic to Stale and running to Failed. | Fixed | +| P1 | Restart paused a Running Run whose Tasks were already resolved | User still needed manual Resume. | Attempt normal atomic finality before the pause sweep. | Fixed | +| P1 | Debug restart order diverged from production | Rendered E2E tested a false lifecycle. | Share the same seven-step restart routine. | Fixed | +| P1 | Tagged dispatch enum produced provider-incompatible `oneOf` | Some providers could not invoke `task_create`. | Use flat wire fields and parse into a typed policy at the boundary. | Fixed | + +No actionable P0/P1 finding in this audit remains open. + +## Ten-layer architecture audit + +| Layer | Audit focus | Conclusion | +| -------------------------- | -------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------- | +| 1. Compilation correctness | `org2`, `agent_core`, `e2e-test`, TypeScript, ESLint, Node syntax | Relevant code compiles and frontend gates pass. Strict workspace Clippy remains blocked by existing baseline outside this scope. | +| 2. Dead code / duplication | Trace Watchdog, resume, Task tools, lifecycle finalization, and scheduler entry points | Removed dead availability, stale release, old Wake wrappers, autonomous claim helpers, and misleading positive CLI E2E. | +| 3. Naming consistency | Sweep stale/release/Wake/owner terminology | Stale means notice, not ownership release. Failure and shutdown use distinct disposition names. | +| 4. Semantic overloading | Compare Run, Session, Task, Delivery, and Budget | Each dimension persists independently. Idle is not terminal, unread is not accepted Wake, and eligibility is not claim authority. | +| 5. Default branches | Review `SessionStatus`, Run state, Wake outcome, and timestamp fallback | Session recovery is explicit; database errors fail closed; corrupt deadlines and timestamps are diagnosed. E3 is named and tested. | +| 6. Cross-domain leakage | Rust/CLI transport, Task Store, and UI selectors | Historical CLI is recognized at the boundary but never sent through Rust-member Wake. Shared selectors hide CLI only where required. | +| 7. New-developer clarity | Check names and comments against side effects | Analyzer, Executor, doorbell, disposition, and finality transaction have distinct responsibilities. | +| 8. Wire / serialization | Scheduler response, Task metadata, Inbox payload, and tool guidance | Durable intent outcomes are exact, schema is provider-compatible, guidance is structured, and metadata is validated by tool and Store. | +| 9. Initialization parity | Production setup, unit sandbox, debug seed, restart | Recovery schema is shared; production and simulated restart use identical ordering; fixtures create legal eligibility. | +| 10. Resolver symmetry | Member identity, assignment, Run gate, Session resolution | All paths treat ownerless as Coordinator-owned assignment work, use canonical member identity, and carry typed Run ids. | + +## State ownership + +| State dimension | Single source of truth | May another state be used as a substitute? | +| ---------------------------------------- | ---------------------------------------------- | ------------------------------------------ | +| Whether the Run may continue | `agent_org_runs.status` | No | +| Whether a member is executing | `agent_sessions.status` | No | +| Task owner, completion, and dependencies | `agent_org_tasks` | No | +| Whether mail remains unread | `agent_inbox.read_at` plus delivery resolution | No | +| Whether a Wake was accepted | Scheduler intent and idempotency registry | No | +| Whether automatic recovery may retry | `agent_org_recovery_attempts` | No | + +## Entry-point consistency + +| Entry point | Running gate | Assignment policy | Durable Inbox | Idempotent Wake | Budget | +| ------------------------------ | ---------------------------------------------------------------------------- | ---------------------------------------------- | ------------------ | -------------------------------------------- | --------------------------------- | +| Watchdog | Analyzer + Executor + turn-start recheck | Ownerless only repairs/notifies Coordinator | Yes | Yes | Yes | +| Resume progress | Dispatcher + turn-start recheck | Delivers real work only to an existing owner | Yes | Yes | Terminal retry only | +| Task create/update side effect | Mutation transaction + dispatcher | Delivery begins after Coordinator writes owner | Yes | Yes | Terminal retry only | +| Member failure finalize | Task transaction + dispatcher | Clears owner and waits for Coordinator | Coordinator notice | Yes | Not consumed in failure hook | +| Accepted shutdown | Task transaction | Valid pool or Coordinator ownership | `MemberTerminated` | Coordinator Wake coalesces | Never revives stopped member | +| App restart | Intent reconcile + failure disposition + resolved finality + remaining pause | Explicit | Preserved | Old queue is closed; Resume creates new Wake | Persisted and pruned by Run state | + +## Final verification + +| Check | Final result | +| ----------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------- | +| `cargo check -p org2`, `cargo check -p e2e-test`, `cargo check -p org2 --lib` | Pass | +| Full `agent_core` suite in isolated environment | 3,155 / 3,155 pass | +| Full `session_persistence` suite outside restricted sandbox | 34 / 34 pass | +| Agent Org HTTP runtime E2E against real Debug App | 47 / 47 pass | +| Rendered Debug App E2E | 19 / 19 pass: Group Chat 6, Pause/Resume 8, Recovery 2, Settings 3 | +| Full frontend Vitest | 450 files, 5,181 / 5,181 pass | +| `pnpm typecheck` and `pnpm lint` | Pass | +| `git diff --check` | Pass | +| Strict Clippy | Existing develop findings remain; no remaining warning is on a line introduced by this review. | +| Full workspace rustfmt | Six unchanged develop files remain unformatted; changed-scope formatting passes. | +| Circular dependency check | Two existing cycles remain in unchanged Org2Cloud/SessionCore paths. | + +## Intentional design boundaries + +- **E3 remains deferred.** General member-level peer recovery is not attempted while another worker is Active. This is explicit and tested. +- **Eligibility remains in metadata JSON.** Typed tool checks, Store invariants, and SQLite JSON1 queries harden the current representation; normalization to a join table is a separate schema project. +- **CLI Agent Org is explicitly unsupported rather than partially supported.** Full parity requires CLI Inbox drain, Agent Org tools, a resume bridge, and production E2E. +- **WakeNoop is an execution-time result.** Request time distinguishes Enqueued, Coalesced, Paused, Terminal, Unavailable, and Failed. Work that disappears after enqueue becomes a logged no-op in the processor. +- **Full Revision Events remain outside #373.** See `docs/architecture-audit-2026-07-16/AgentOrgRevisionEventPlan.md`. + +## Commit readiness + +The approved #373 correctness scope is implementation- and test-complete: + +- relevant compilation, type, lint, focused tests, full Agent Core tests, runtime E2E, rendered E2E, and diff hygiene pass; +- no known P0/P1 correctness issue in the approved scope remains; +- strict workspace Clippy, workspace rustfmt, and circular checks are blocked only by explicitly recorded unchanged develop baseline; +- no unrelated local artifact, credential, database, screenshot, or generated report is included. + +Recommended commit title: ```text fix(agent-org): harden recovery, wake delivery, and run finality ``` - -提交前还需真人确认依赖遗漏会被 Coordinator 修正或明确确认并行；两份审计文档应随代码一并纳入 commit，无关本地文件继续排除。 diff --git a/docs/architecture-audit-2026-07-13/AgentOrgModernFamilyRecovery.md b/docs/architecture-audit-2026-07-13/AgentOrgModernFamilyRecovery.md index 58214e46e..065c57b2e 100644 --- a/docs/architecture-audit-2026-07-13/AgentOrgModernFamilyRecovery.md +++ b/docs/architecture-audit-2026-07-13/AgentOrgModernFamilyRecovery.md @@ -1,82 +1,80 @@ -# Agent Org “Modern Family” 五批修复架构审计 - -日期：2026-07-13 -分支：`fix/issue-272-agent-org-recovery-invariants` -范围：用户实测中出现的根 Coordinator 错误进入 Plan、任务链创建不完整、消息绕过任务、Reviewer 尚在运行却提前收尾、暂停与空筛选红错。 -结论：五批修复已接入生产工具装配和持久化路径；本范围没有遗留的 P0/P1 问题。审计额外发现并修复了“任务图检查与落库之间可能插入一张并发任务”的窄竞态。 - -## 验收标准与结果 - -| 用户看到的问题 | 必须成立的新行为 | 结果 | -| ------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------ | ---- | -| Group chat 要求用户切 Plan mode | 活跃 Agent Org 的根 Coordinator 不暴露通用 mode-switch 工具；计划由带 `execution_mode=plan` 的成员任务承载 | 通过 | -| Plan / 写作 / Review 的卡片对不上 | Coordinator 可用一个 `task_graph_create` 原子写入完整动态依赖图；失败时零任务落库 | 通过 | -| 创建 Review task 失败后，Coordinator 仍用聊天叫 Reviewer 干活 | 发给 Worker 的正式 plain 消息必须绑定真实、未完成、依赖已就绪且对收件人有权限的 `related_task_id` | 通过 | -| Reviewer 还在 Running，Coordinator 已宣布全部完成 | `task_list.run_summary.completion_ready` 同时检查 Task、Session、Inbox、Turn intent、Intervention 和 Plan approval | 通过 | -| `status=""` 形成红色工具失败；Pause 误伤未启动成员 | 空 status 当作未筛选；`OrgPause` 不把没有 live runtime 的惰性成员修成 Failed | 通过 | - -## 十层架构审计 - -| Layer | Line / Element | Verdict | Reason | Suggested change | -| ---------------- | ---------------------------------------------------------- | ---------------- | -------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------- | -| 1 编译与验证 | Rust / TypeScript / task tests | keep | `cargo check -p org2`、typecheck、格式和 diff 检查通过；任务工具 58/58、task store 43/43、消息 14/14 通过。 | 无。 | -| 1 编译与验证 | `cargo test -p agent_core --lib` | keep with reason | 2995/2997 通过；剩余 skill-content 与 search-tool 两项是本分支之前已存在且与 Agent Org 无关的基线。 | 另开基线清理，不混入本修复。 | -| 1 编译与验证 | strict Clippy | keep with reason | 新增任务图、消息门和完成证书没有产生 Clippy 诊断；全 crate 仍被 Provider、Memory、Channel 等既有告警阻塞。 | 另开 Clippy cleanup PR。 | -| 2 活代码/重复 | `task_dependency_closure` | fixed | 单任务、批量任务和 transaction-time recheck 曾各自可能定义“依赖覆盖”；现共享一个 closure 算法。 | 无。 | -| 2 活代码/重复 | `task_graph_create` production/debug/test wiring | keep | 工具已接到 builtin metadata、policy、production overlay、debug runtime、test API、Rust/TS tool names 与前端事件识别，不是只在测试里存在。 | 无。 | -| 3 命名 | `TaskGraphCreate` / `related_task_id` / `completion_ready` | keep | 名称分别表示“原子任务图”“消息所属任务”“完成证书”，没有用含糊的 `state` 或 `done`。 | 无。 | -| 4 语义维度 | Run / Session / Task / Delivery / Approval | keep | Run 是否继续、成员是否正在工作、任务是否完成、消息是否已送达、计划是否获批分别保存；不再用 `open=0` 猜整个组织完成。 | 无。 | -| 5 FSM 完整性 | `session_is_quiescent_for_completed_run` | keep | 每个 `SessionStatus` 显式分类；Running、Pending、Waiting、Paused 都会阻止完成证书。 | 新增状态时继续让编译器强制补齐 match。 | -| 5 FSM 完整性 | task dispatch | keep | 只有依赖已完成的根任务收到 assignment；下游由真实 TaskCompleted 事件解锁。 | 无。 | -| 6 跨域边界 | root Plan vs member Plan task | fixed | 通用 root mode switch 不再参与活跃 Agent Org 编排；用户显式以 Plan 启动普通 root session 的能力仍保留。 | 无。 | -| 6 跨域边界 | Chat routing vs task authority | fixed | Hierarchy Mode 仍决定“谁能联系谁”；Task authority 决定“谁能给谁派活”。可聊天不等于可绕过任务。 | 无。 | -| 7 新开发者理解 | Coordinator prompt / tool descriptions | fixed | Prompt 解释原子图、动态依赖、task-bound message、Plan task 和 completion certificate；非代码写作不会因工具存在就误入 GitHub issue workflow。 | 后续修改继续保留字符串契约测试。 | -| 8 Wire / schema | Rust + TS `TASK_GRAPH_CREATE` | keep | 工具名、provider JSON schema、event extraction 与前端识别一致；schema compatibility 测试通过。 | 无。 | -| 8 Wire / schema | recoverable guidance | keep | 依赖遗漏、plain 消息缺 task、空 after-dependencies 等返回结构化 guidance，不制造用户可见的红色失败卡。 | 长期可由 UI 渲染成专用提示卡。 | -| 9 初始化一致性 | production overlay / debug runtime / test API | keep | Coordinator 才注册跨成员 graph tool；所有测试与调试入口使用同一实现。 | 无。 | -| 10 Resolver 对称 | member owner / eligibility / recipient | keep | owner、eligible member 与 message recipient 都从 run roster 的稳定 member_id 解析；不接受 display name 猜测。 | 无。 | - -## 审计过程中额外修复的竞态 - -原本工具会先读取当前开放任务，再决定新图是否需要依赖它们，然后进入 transaction 插入新图。极窄情况下，另一条执行流可以刚好在两步之间新增任务：新图本身仍是完整的，但它会漏掉刚出现的开放工作。 - -现在 store 在真正写入的同一笔 SQLite IMMEDIATE transaction 里再检查一次： +# Agent Org “Modern Family” Five-Batch Recovery Architecture Audit + +- Date: 2026-07-13 +- Branch: `fix/issue-272-agent-org-recovery-invariants` +- Scope: The real-run failures in which the Root Coordinator entered Plan incorrectly, Task chains were created incompletely, messages bypassed formal Tasks, finality was declared while the Reviewer was still Running, and Pause or empty filters produced visible failures. +- Conclusion: All five batches are wired into production tool assembly and durable persistence paths. No P0/P1 issue in this scope remains. The audit also found and fixed a narrow race in which another Task could be inserted between graph preflight and graph persistence. + +## Acceptance criteria and result + +| User-visible failure | Required behavior | Result | +| ------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------- | ------ | +| Group Chat asked the user to switch the Coordinator to Plan mode. | The Root Coordinator of an active Agent Org does not expose the generic mode-switch tool. Planning is carried by a member Task with `execution_mode=plan`. | Pass | +| Plan, writing, and Review cards did not correspond. | The Coordinator can atomically create one complete, dynamic dependency graph with `task_graph_create`; any failure writes zero Tasks. | Pass | +| After Review Task creation failed, the Coordinator still told the Reviewer to work through chat. | A formal plain message to a worker must reference a real, incomplete, dependency-ready Task that the recipient is authorized to perform. | Pass | +| Coordinator announced completion while Reviewer was still Running. | `task_list.run_summary.completion_ready` checks Task, Session, Inbox, Turn intent, Intervention, and Plan approval state together. | Pass | +| `status=""` produced a red tool failure and Pause damaged unstarted members. | Empty status means no filter; `OrgPause` does not mark a lazy member with no live runtime as Failed. | Pass | + +## Ten-layer architecture audit + +| Layer | Line / element | Verdict | Reason | Suggested change | +| ------------------------------- | -------------------------------------------------------- | ------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ----------------------------------------------------- | +| 1. Compilation and verification | Rust, TypeScript, and Task suites | keep | Rust and frontend checks pass, and focused Task, Store, and message suites cover the behavior. | None. | +| 2. Live code / duplication | `task_dependency_closure` | fixed | Single-Task, graph, and transaction-time checks previously could define dependency coverage differently; they now share one closure algorithm. | None. | +| 2. Live code / duplication | `task_graph_create` production/debug/test wiring | keep | The tool is connected to builtin metadata, policy, production overlay, debug runtime, test API, Rust/TS tool names, extractor routing, and frontend event rendering. | None. | +| 3. Naming | `TaskGraphCreate`, `related_task_id`, `completion_ready` | keep | The names mean atomic graph creation, the Task associated with a message, and a multi-dimensional completion certificate. | None. | +| 4. Semantic dimensions | Run / Session / Task / Delivery / Approval | keep | Whether a Run may continue, a member is working, a Task is complete, a message was delivered, and a Plan was approved are persisted separately. | None. | +| 5. FSM completeness | `session_is_quiescent_for_completed_run` | keep | Every `SessionStatus` is explicit. Running, Pending, Waiting, and Paused all block a completion certificate. | Require exhaustive handling for future variants. | +| 5. FSM completeness | Task dispatch | keep | Only dependency-ready root Tasks receive assignment. Downstream work unlocks from durable completion. | None. | +| 6. Cross-domain boundary | Root Plan vs member Plan Task | fixed | Generic Root mode switching is not part of active Agent Org orchestration. Explicit Plan mode for an ordinary single-agent Root Session remains supported. | None. | +| 6. Cross-domain boundary | Chat routing vs Task authority | fixed | Hierarchy Mode controls who may communicate. Task authority controls who may assign or mutate work. Communication never bypasses Task ownership. | None. | +| 7. New-developer clarity | Coordinator prompt and tool descriptions | fixed | Contracts explain atomic graphs, dynamic dependencies, Task-bound messages, Plan Tasks, and completion certificates. Non-code work is not forced into a GitHub issue workflow. | Retain string-contract tests for prompt changes. | +| 8. Wire / schema | Rust and TS `TASK_GRAPH_CREATE` | keep | Tool name, provider schema, extraction, and frontend routing are aligned; schema portability is tested. | None. | +| 8. Wire / schema | Recoverable guidance | keep | Missing dependencies, missing related Task, and empty dependency policies return structured guidance instead of a visible red failure card. | A dedicated guidance UI card may be considered later. | +| 9. Initialization parity | Production overlay, debug runtime, test API | keep | Only the Coordinator receives the cross-member graph tool, and every debug/test entry point calls the production implementation. | None. | +| 10. Resolver symmetry | Member owner, eligibility, recipient | keep | Owner, eligible member, and message recipient resolve through stable Run-roster `member_id`, never display-name guesses. | None. | + +## Additional race fixed during audit + +The tool originally read open Tasks before entering the persistence transaction. Another execution could insert a Task between preflight and graph insertion. The graph itself remained internally valid but could omit the newly opened work. + +The Store now repeats the dependency-coverage check inside the same SQLite IMMEDIATE transaction that persists the graph: ```mermaid flowchart LR - A["工具预检查\n给模型友好 guidance"] --> B["取得统一 writer lock"] - B --> C["IMMEDIATE transaction\n重读现有 open tasks"] - C -->|"依赖覆盖完整"| D["验证整张图与环"] - D --> E["一次写入 tasks + history"] - C -->|"发现并发新增的遗漏任务"| F["整笔回滚\n返回重新确认 guidance"] + A["Tool preflight\nmodel-friendly guidance"] --> B["Acquire shared writer lock"] + B --> C["IMMEDIATE transaction\nreload existing open Tasks"] + C -->|"Coverage complete"| D["Validate full graph and cycles"] + D --> E["Write Tasks + history atomically"] + C -->|"Concurrent omitted Task found"| F["Rollback everything\nreturn confirmation guidance"] ``` -回归测试确认：事务内发现遗漏时，数据库只保留原来的任务，不会留下半张新图。 +A regression proves that when transaction-time validation finds an omission, the database retains only the original Task and no partial new graph. -## 最终流程 +## Final flow ```mermaid flowchart TD - U["用户在 Group chat 提交目标"] --> C["Coordinator 保持 Build\n设计动态工作图"] - C --> G["task_graph_create\n一次写入 Plan / Write / Review / Final 等实际需要的节点"] - G --> R["只唤醒依赖已满足的根任务"] - R --> M["Member 完成自己的 Task\n写 durable output"] - M --> N["后端解锁真正依赖它的下游 Task"] + U["User submits a goal in Group Chat"] --> C["Coordinator stays in Build\nand designs a dynamic work graph"] + C --> G["task_graph_create\natomically writes the Plan / Write / Review / Final nodes actually needed"] + G --> R["Wake only dependency-ready root Tasks"] + R --> M["Member completes its Task\nand writes durable output"] + M --> N["Backend unlocks only true downstream dependents"] N --> M - M --> S["Coordinator 调用 task_list"] + M --> S["Coordinator calls task_list"] S --> Q{"completion_ready?"} - Q -->|"false"| W["按 blocker 等待真实事件\n不假完成、不空 Wake"] + Q -->|"false"| W["Wait for the typed blocker\nno fake completion and no empty Wake"] W --> S - Q -->|"true"| F["向用户输出最终结果"] + Q -->|"true"| F["Return the final result to the user"] ``` -## 明确保留的边界 +## Intentional boundaries -1. 依赖链不是写死的 `Planner → Implementer → Reviewer → Tester`。Coordinator 根据本次请求动态生成；无依赖的任务可以并行，消费上游产物的任务必须声明依赖。 -2. `HierarchyMode::Soft` 没有被删除。它管理通信可达性，不授予跨成员任务修改权。 -3. `completion_ready` 是给 Coordinator 的持久化完成证书，不是新的大模型，也不取代 Coordinator 的判断和最终表达。 -4. 本报告不背书当前 dirty worktree 中 Git、Key Vault、Provider、其他 UI 等无关改动；本轮没有修改 `*.tsx`，因此没有新增 frontend-ui-audit 报告。 +1. The dependency chain is not hardcoded as Planner → Implementer → Reviewer → Tester. The Coordinator designs it per request. Independent Tasks may run in parallel; Tasks that consume upstream results must declare dependencies. +2. `HierarchyMode::Soft` remains. It controls communication reachability, not cross-member Task authority. +3. `completion_ready` is a durable completion certificate for the Coordinator. It is not another model and does not replace the Coordinator's final reasoning or response. +4. This report covers the Agent Org paths named above and does not endorse unrelated Git, Key Vault, Provider, or other worktree changes. -## 最终判断 +## Final verdict -这五批把 Agent Org 从“模型靠聊天和局部看板猜下一步”收紧为“Coordinator 设计任务图，数据库保证任务与依赖，消息只能补充上下文，完成必须拿到多维证书”。Modern Family 实测暴露的三条错误捷径——根 Plan、消息绕任务、Reviewer 未完先收尾——都已在代码边界被阻断，而不是只靠提示词劝模型别犯错。 +These five batches change orchestration from “the model guesses the next step from chat and a partial board” to “the Coordinator designs a graph, the database enforces Tasks and dependencies, messages add context without bypassing authority, and completion requires a multi-dimensional certificate.” The three shortcuts exposed by the Modern Family run—Root Plan switching, work assigned outside a Task, and finality before Reviewer completion—are blocked at code boundaries rather than discouraged only by prompts. diff --git a/docs/architecture-audit-2026-07-13/AgentOrgPlannerApproval.md b/docs/architecture-audit-2026-07-13/AgentOrgPlannerApproval.md index ffbeac8b0..cdc47bb6a 100644 --- a/docs/architecture-audit-2026-07-13/AgentOrgPlannerApproval.md +++ b/docs/architecture-audit-2026-07-13/AgentOrgPlannerApproval.md @@ -1,89 +1,90 @@ -# Agent Org Planner、审批与 Wake 加固架构审计 - -日期：2026-07-13 -范围：#272 恢复链路，以及本轮五批 Planner 任务、动态依赖、审批、Wake 和 Group chat 状态投影。 -结论：本范围内没有遗留的 P0/P1 架构问题；审计发现的三组“数据库写了一半”风险已经改成单事务。工作区中其余 Git、Key Vault、Provider 等改动不属于本报告，也没有被本轮修改或背书。 - -## 验收结果 - -| 检查 | 结果 | 说明 | -| ----------------------- | --------------------------- | -------------------------------------------------------------------------------------- | -| Rust 编译 | 通过 | `cargo check -p org2` | -| TypeScript | 通过 | `npm run typecheck` | -| Agent Org approval 单测 | 11/11 | 包含审批、退回、编辑、暂停、自动审批与事务回滚 | -| `create_plan` 单测 | 7/7 | 包含空标题、空内容、错误 session 与动态说明 | -| Agent Org 真实桌面 E2E | 通过 | task-driven Plan → 审批 → 下游解锁 | -| `agent_core` 全量单测 | 2984 通过，2 个既有基线失败 | 沙箱外运行后端口测试正常；剩余两项是无关的 skill 内容与 search 参数冲突基线 | -| Clippy | 项目基线未清零 | 全依赖被既有 dependency warning 阻塞；`--no-deps` 有既有警告。本轮新增的两条警告已修掉 | -| Diff 空白检查 | 通过 | `git diff --check` | - -## 十层审计 - -| Layer | Line / Element | Verdict | Reason | Suggested change | -| -------------------- | --------------------------------------------- | ---------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------- | -| 1 编译 | `src-tauri` / frontend | keep with reason | Rust 与 TypeScript 均编译；本轮新增代码没有留下项目 warning。全仓 Clippy 仍受既有基线影响。 | 本 PR 不顺手清理无关 Clippy 基线，单独建 cleanup PR。 | -| 2 死代码与重复 | `agent_org_plan_approvals.rs:256,286,391` | fixed | Coordinator 请求、自动批准、退回反馈原先存在各写各的风险；现在共享事务 helper。 | 无。 | -| 2 死代码与重复 | `agent_inbox.rs:714` | fixed | 新增 `insert_in_tx`，审批和 Inbox 不再各开一个 transaction。 | 无。 | -| 2 死代码与重复 | legacy `ExecModeSetRequest` reader | keep with reason | 当前生产端不再生成这种消息，但仍需读取旧数据库历史行；保留的是反序列化兼容，不是第二套调度机制。 | 等明确放弃旧本地 DB 兼容时再删除。 | -| 3 命名 | `agent_org_tasks::TaskExecutionMode` | keep | 名字明确表示“这张任务卡下一次应以 Plan 还是 Build 执行”，不会和整个 session 的通用执行模式混淆。 | 无。 | -| 3 命名 | `AgentOrgPlanInboxDelivery` | keep | 类型明确表达“审批状态与哪封持久信件一起提交”，没有用模糊的 `Result`/`Context`。 | 无。 | -| 4 语义维度 | Run / Session / Task / Approval / Delivery | keep | 五个状态分开保存：run 是否继续、session 是否工作、task 是否完成、plan 是否获批、inbox/wake 是否送达，互不冒充。 | 继续要求新 UI 标明自己显示的是哪一维。 | -| 4 语义维度 | `org_tasks.rs:118` `AwaitingPlanApproval` | keep | 这是从 durable approval 投影出的界面阶段，不会把 Run 改成 Paused，也不会把 Task 假装 Completed。 | 无。 | -| 5 默认分支 | `PlanApprovalPolicy` | keep | `Coordinator`、`User`、`Automatic` 三种策略在创建与提示词中显式匹配；没有 `_ => automatic` 之类危险兜底。 | 新增策略时让编译器强制所有 match 补齐。 | -| 5 默认分支 | historical missing `execution_mode` | keep with reason | 旧 task / inbox 行缺字段时按 Build 读取，只用于历史兼容；新 `task_create` 强制显式传值。 | 保持“旧读宽松、新写严格”。 | -| 6 跨域泄漏 | Agent Org approval vs top-level Plan approval | keep | 两者没有复用同一状态：单 agent 的 Build/Skip 机制保持原样；Agent Org 审批绑定 run + member + source task。 | 无。 | -| 7 新开发者理解 | `create_plan.rs:168` tool description | fixed | 工具说明现在写清：必须绑定 owned in-progress Plan task；提交后停止；批准负责完成任务；不会制造 fake Build turn。 | 无。 | -| 7 新开发者理解 | `section_builders.rs:680-768` | fixed | Coordinator 的 prompt 同时解释 dynamic dependency、dispatch policy、execution mode 和审批结果。 | 后续提示词变更继续有字符串契约测试。 | -| 8 Wire/序列化 | `TaskAssigned.execution_mode` | keep | 扁平 typed field，历史缺失默认 Build；成员从真实 assignment 决定 Plan/Build。 | 无。 | -| 8 Wire/序列化 | `task_create` schema | fixed | `dispatch_policy` 和 `execution_mode` 都是 required；依赖确认返回结构化 guidance，不制造 trajectory-visible tool failure。 | 无。 | -| 8 Wire/序列化 | Plan 内容边界 | keep | 审批保存完整 markdown 与 artifact path；注入 Inbox/TaskOutput 的文本有总量边界，避免无界 prompt 膨胀。 | 后续可加 payload-size telemetry，但不是正确性前提。 | -| 9 Init parity | setup / test env / debug Agent Org fixtures | fixed | 新审批表在 production setup、测试 DB 与 E2E 初始化一致；遗漏的 `plan_approval_policy` fixture 已补齐，`cargo check -p org2` 能抓住。 | 新增 OrgDefinition 字段时继续编译所有 desktop/E2E target。 | -| 9 Init parity | `external_import/tests.rs` 全局 HOME guard | fixed | 全量并行测试曾能在 lifecycle 建表过程中切走 `ORGII_HOME`，造成“no such table”假失败；所有本地 HOME guard 现先取得 workspace canonical lock。 | 新增 env-mutating test 必须使用 `test_helpers::test_env`。 | -| 10 Resolver symmetry | owner / member / task / execution mode | keep | `create_plan` 从 runtime member identity 找到同 run 的 owned in-progress Plan task；ownerless task 不参与 mode prepeek，也不会由成员自领。 | 无。 | -| 10 Resolver symmetry | explicit assignment | fixed | Watchdog、resume、Inbox drain 和 task side effect 都把 ownerless 解释为“等待 Coordinator 指派”；只有真实 `TaskAssigned` 决定 Worker 的下一次执行模式。 | 无。 | - -## 事务审计发现并已修复的问题 - -| 问题类 | 旧风险 | 现在 | -| -------------------- | --------------------------------------------------------------------------------- | ------------------------------------------------------------------- | -| 退回计划 | approval 先变 `changes_requested`，写 Planner Inbox 失败后 Planner 永远收不到意见 | approval 状态与反馈信同一 SQLite transaction；任一步失败全部回滚 | -| Coordinator 审批请求 | 可能有 pending approval，却没有给 Coordinator 的请求信 | approval revision 与请求信同事务创建 | -| Automatic 审批 | 可能 approval 已 approved，但 source Plan task 仍 in_progress | approval 创建、批准、TaskOutput 与 Plan task 完成在同一 transaction | - -对应测试会主动删除 `agent_inbox` 表制造失败，确认数据库不会留下“半成功”状态。这不是只测 happy path。 - -## 状态与事件边界 +# Agent Org Planner, Approval, and Wake Hardening — Architecture Audit + +- Date: 2026-07-13 +- Scope: The #272 recovery path and the five Planner Task, dynamic dependency, approval, Wake, and Group Chat projection batches. +- Conclusion: No P0/P1 architecture issue in this scope remains. Three “half-written database state” risks found during the audit now commit in a single transaction. Unrelated Git, Key Vault, Provider, and other worktree changes are outside this report. + +## Acceptance results + +| Check | Result | Notes | +| ----------------------------- | --------------------------------- | -------------------------------------------------------------------------- | +| Rust compilation | Pass | `cargo check -p org2` | +| TypeScript | Pass | `pnpm typecheck` | +| Agent Org approval tests | Pass | Approval, changes requested, edit, pause, automatic approval, and rollback | +| `create_plan` tests | Pass | Empty title/content, wrong Session, and dynamic guidance | +| Real Agent Org desktop E2E | Pass | Task-driven Plan → approval → downstream unlock | +| Final full `agent_core` suite | 3,155 / 3,155 | Isolated persistence and loopback-enabled environment | +| Clippy | Existing develop baseline remains | The warning introduced on a new review line was fixed. | +| Diff whitespace | Pass | `git diff --check` | + +## Ten-layer audit + +| Layer | Line / element | Verdict | Reason | Suggested change | +| ---------------------------- | --------------------------------------------- | ---------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------- | +| 1. Compilation | `src-tauri` / frontend | keep with reason | Rust and TypeScript compile; strict workspace Clippy remains affected by existing develop findings. | Address baseline lint in a separate cleanup PR. | +| 2. Dead code and duplication | Plan approval transaction helpers | fixed | Coordinator request, automatic approval, and changes-requested feedback previously risked independent partial writes; they now share transactional helpers. | None. | +| 2. Dead code and duplication | Inbox `insert_in_tx` | fixed | Approval and Inbox persistence participate in the same caller-owned transaction. | None. | +| 2. Dead code and duplication | Legacy `ExecModeSetRequest` reader | keep with reason | Production no longer creates this message, but historical database rows must remain readable. This is compatibility, not a second scheduler. | Remove only when historical local database compatibility is explicitly dropped. | +| 3. Naming | `TaskExecutionMode` | keep | The type says whether this Task's next turn runs in Plan or Build; it is not the generic mode of the whole Session. | None. | +| 3. Naming | `AgentOrgPlanInboxDelivery` | keep | The name explicitly binds approval state to a durable delivery instead of using a vague Result or Context type. | None. | +| 4. Semantic dimensions | Run / Session / Task / Approval / Delivery | keep | Whether work may continue, a Session is executing, a Task is done, a Plan is approved, and Inbox/Wake delivered input remain separate. | New UI should continue to name the dimension it displays. | +| 4. Semantic dimensions | `AwaitingPlanApproval` | keep | This UI phase is projected from durable approval state. It neither pauses the Run nor pretends the Plan Task is Completed. | None. | +| 5. Default branches | `PlanApprovalPolicy` | keep | Coordinator, User, and Automatic policies are exhaustively matched; there is no dangerous catch-all automatic approval. | Require exhaustive handling for new policies. | +| 5. Default branches | Historical missing `execution_mode` | keep with reason | Old Task/Inbox rows default to Build for compatibility; new creation requires an explicit value. | Keep historical reads tolerant and new writes strict. | +| 6. Cross-domain leakage | Agent Org approval vs top-level Plan approval | keep | Single-agent Build/Skip remains separate. Agent Org approval is scoped to Run, member, and source Task. | None. | +| 7. New-developer clarity | `create_plan` description | fixed | The contract requires an owned in-progress Plan Task, stops after submission, and makes approval responsible for completing the Task rather than producing a fake Build turn. | None. | +| 7. New-developer clarity | Coordinator prompt | fixed | The prompt explains dynamic dependencies, dispatch policy, execution mode, and approval outcomes together. | Retain string-contract tests. | +| 8. Wire / serialization | `TaskAssigned.execution_mode` | keep | A flat typed field controls the real assignment; historical omission defaults to Build. | None. | +| 8. Wire / serialization | `task_create` schema | fixed | `dispatch_policy` and `execution_mode` are required. Missing dependency confirmation returns structured guidance instead of a visible tool failure. | None. | +| 8. Wire / serialization | Plan content boundary | keep | Durable approval retains complete Markdown/artifact identity while text inserted into Inbox or TaskOutput is bounded. | Payload telemetry may be added later but is not a correctness prerequisite. | +| 9. Initialization parity | Setup / test environment / debug fixtures | fixed | Approval schema and `plan_approval_policy` initialization are consistent across production, tests, and E2E. | Compile every desktop/E2E target when adding OrgDefinition fields. | +| 9. Initialization parity | Global `ORGII_HOME` guard | fixed | Parallel tests could previously move HOME while lifecycle schema initialization ran, causing false “no such table” failures. Environment-mutating tests now use the canonical workspace lock. | Require `test_helpers::test_env` for new environment-mutating tests. | +| 10. Resolver symmetry | Owner / member / Task / execution mode | keep | `create_plan` finds an owned in-progress Plan Task through runtime member identity; ownerless work neither influences mode prepeek nor self-claims. | None. | +| 10. Resolver symmetry | Explicit assignment | fixed | Watchdog, resume, Inbox drain, and Task side effects all interpret ownerless as awaiting Coordinator assignment. Only a real `TaskAssigned` selects a worker's next mode. | None. | + +## Transaction findings and fixes + +| Class | Previous risk | Current behavior | +| ---------------------------- | ------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------- | +| Request Plan changes | Approval became `changes_requested`, then feedback delivery failed and Planner never received it. | Approval state and feedback Inbox row commit in one SQLite transaction; either both succeed or both roll back. | +| Coordinator approval request | Pending approval could exist without a durable request to Coordinator. | Approval revision and request Inbox row commit together. | +| Automatic approval | Approval could be Approved while the source Plan Task remained In progress. | Approval creation, approval decision, TaskOutput, and Plan Task completion commit together. | + +Tests deliberately remove the Inbox table to force failure and prove that no half-successful state remains. + +## State and event boundary ```mermaid flowchart LR - T["Plan Task\nin_progress"] --> P["create_plan\n写 approval revision"] - P --> C{"审批策略"} - C -->|Coordinator| I["给 Coordinator 一封持久请求信"] - C -->|User| U["Group chat 显示审批卡"] - C -->|Automatic| A["同事务自动批准"] - I --> D{"批准或退回"} + T["Plan Task\nin_progress"] --> P["create_plan\npersist approval revision"] + P --> C{"Approval policy"} + C -->|"Coordinator"| I["Durable request to Coordinator"] + C -->|"User"| U["Approval card in Group Chat"] + C -->|"Automatic"| A["Approve in same transaction"] + I --> D{"Approve or request changes"} U --> D - D -->|退回| F["同事务保留 Task + 写 Planner 反馈信"] + D -->|"Changes requested"| F["Keep Task + write Planner feedback atomically"] F --> T - D -->|批准| O["同事务写 TaskOutput + 完成 Plan Task"] + D -->|"Approved"| O["Write TaskOutput + complete Plan Task atomically"] A --> O - O --> N["只解锁真实依赖它的下游任务"] + O --> N["Unlock only real downstream dependents"] ``` -## Wake 审计结论 +## Wake audit conclusion -- 保留 #272 的恢复能力：未读 Inbox、刚解锁且已有 owner 的任务、审批反馈和受预算控制的真实恢复输入仍可触发 Wake；ownerless task 只提醒 Coordinator 明确指派。 -- 删除“只因还有一个未完成任务就每分钟叫醒”的假输入。 -- 等待用户审批时没有可消费输入，所以不调用模型、不闪烁、不消耗 recovery attempt。 -- Coordinator 或 member 真有未读持久信时仍会 Wake；Wake 只是门铃，Inbox row 才是事实。 -- 同一 member 的并发 Wake 使用确定性 key 合并；只有 Scheduler 真开始 turn 才把 session 写成 Running。 +- #272 recovery remains active for unread Inbox, newly ready owned work, approval feedback, and budgeted real recovery input. +- Ownerless Tasks only notify the Coordinator for explicit assignment. +- “An unfinished Task exists” alone is not a Wake reason. +- Awaiting user approval is an intentional quiet state: it does not call the model, flash, or consume recovery budget. +- A real unread durable row still wakes its recipient. Wake remains the doorbell; the Inbox row remains truth. +- Concurrent Wake requests for one member coalesce under a deterministic key, and Running is written only when the scheduler starts the turn. -## 保留项与明确边界 +## Intentional boundaries -1. SQLite 与计划 markdown 文件不能形成跨文件系统的真正原子 transaction。正常写文件失败会报错并保持 DB 未批准；进程在极窄窗口硬崩溃时，DB 中保存的 `plan_content` 仍是权威内容，文件可重建。这是已记录的边界，不是双真相。 -2. 历史 `ExecModeSetRequest` 只读兼容暂留；新代码没有 producer。 -3. 此报告只审计 Agent Org 范围；当前 worktree 的无关用户改动没有被格式化、删除或纳入结论。 +1. SQLite and a Markdown artifact cannot form one cross-filesystem transaction. The database retains canonical `plan_content`; managed artifact installation is staged and recoverable after a crash. +2. Historical `ExecModeSetRequest` remains read-only compatibility. New code has no producer. +3. This report audits Agent Org only and does not modify or endorse unrelated user changes in the worktree. -## 最终判断 +## Final verdict -这轮设计已经从“Coordinator 用消息临时命令成员切模式”变为“任务本身携带执行契约”。Planner 的 Plan 模式、审批等待、退回反馈、批准完成和依赖解锁都有持久状态与事务边界；Wake 只负责把真实事件送进 turn，不再负责猜测工作。范围内没有需要阻止提交的审计发现。 +Planning changed from “Coordinator sends an ad hoc message telling a member to switch mode” to “the Task carries an execution contract.” Planner mode, approval wait, feedback, approved completion, and dependency release all have durable state and transactional boundaries. Wake delivers real events into a turn; it no longer guesses that work exists. No blocking finding in this scope remains. diff --git a/docs/architecture-audit-2026-07-14/AgentOrgFinalAudit.md b/docs/architecture-audit-2026-07-14/AgentOrgFinalAudit.md index ecef4b47b..93d377db0 100644 --- a/docs/architecture-audit-2026-07-14/AgentOrgFinalAudit.md +++ b/docs/architecture-audit-2026-07-14/AgentOrgFinalAudit.md @@ -1,78 +1,69 @@ -# Agent Org #272 最终全量架构审计 - -日期：2026-07-14 -基线：`develop` -分支：`fix/issue-272-agent-org-recovery-invariants` -范围：从 `develop` 分出后，本分支涉及 Agent Org Run、Task、Inbox、Wake、Watchdog、成员生命周期、Planner 审批、任务权限、桌面端投影和 E2E 的全部差异。 - -## 最终结论 - -本分支范围内没有遗留的 P0/P1/P2 正确性问题，可以提交。最终审计发现并处理了两类收尾问题： - -1. 删除 25 个与 #272 无关的纯格式化或旧 Clippy 修写，避免把 Git、Key Vault、Provider、Session Memory 等无关代码混进提交。 -2. 将本次新增的 `member_view_from_parts` 十参数接口收束为一个 identity 参数对象，消除本分支唯一新增的 Clippy 告警。 - -最终设计坚持：Run、Session、Task、Approval、Inbox/Delivery 是五套不同状态；Coordinator 负责明确派工，Worker 不能自动抢 ownerless task；Watchdog 只恢复真实可处理事件；Plan 审批是持久状态，不靠空 Wake 或模型猜测推进。 - -## 十层审计 - -| Layer | Line / Element | Verdict | Reason | Suggested change | -| ------------------ | ------------------------------------------------------------- | ---------------- | ------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------- | -| 1 编译与警告 | Rust desktop / E2E / frontend | keep with reason | `org2`、`e2e-test`、TypeScript、ESLint 均通过；本分支新增 Clippy 告警为 0。全仓严格 Clippy 仍有 develop 基线告警。 | 基线告警另开 cleanup PR，不能混入 #272。 | -| 2 死代码与重复 | auto-claim、`find_available`、旧 blocker helpers | fixed | Worker 自动抢 ownerless task 的旧入口和失去调用者的 helper 已清除；全仓 sweep 没有第二套 auto-claim。 | 无。 | -| 3 命名与职责 | `RecoveryPlan` / analyzer / executor / wake outcome | keep | analyzer 只读并给出计划，executor 执行副作用，outcome 明确区分 queued、coalesced、paused、terminal、no-work。 | 无。 | -| 4 状态维度 | Run / Session / Task / Approval / Delivery | keep | 各自持久化、各自转换；Idle 不代表 Run 结束，Pending 不代表可抢，Wake 不代表消息已消费。 | 新功能继续避免用一个状态推断另一个状态。 | -| 5 FSM 与默认分支 | Session status / approval policy / run finality | keep | 关键状态显式分类；终态、Paused、等待审批和历史兼容路径都有明确语义，没有危险的“其他都继续”。 | 新增 enum variant 时依赖编译器强制补齐。 | -| 6 跨域边界 | hierarchy message routing vs task authority | fixed | Hierarchy Mode 只控制成员之间能否通信；Task authority 独立，Coordinator 管全局，Worker 只能推进自己的任务。 | 无。 | -| 7 新开发者可理解性 | prompt、tool schema、typed guidance | fixed | Coordinator 的派工、动态依赖、Planner 执行模式、审批责任和 ownerless 处理均写入提示词与 schema；可修正输入返回 guidance 而非红色失败卡。 | 提示词契约继续保留字符串测试。 | -| 8 Wire / 数据边界 | typed inbox payload、task mutation outcome、reserved metadata | fixed | 新写入严格校验 roster、eligibility、owner 和依赖；历史坏数据仍可读并交给 Watchdog 升级；side effect 使用事务内 outcome。 | eligibility join table 可在后续性能 PR 评估。 | -| 9 初始化一致性 | production / test / debug / E2E DB | fixed | recovery budget、plan approval 等表在各初始化路径一致；CLI Agent Org 在创建和 launch preflight 都明确拒绝。 | CLI parity 完整实现前不要重新开放。 | -| 10 Resolver 对称性 | run/member/session/task identity | keep | member id、session id、root session 和 run id 的解析方向一致；任务展示与 Inbox 显示优先使用 member identity，不用共享 agent type 冒充成员。 | 无。 | - -## 关键不变量复核 - -| 不变量 | 结果 | -| --------------------------------------------------------- | ------------------------------------------------------------------------- | -| 非 Running 的 Run 不能再创建、更新、删除、认领或重派 Task | 通过，Task mutation 与 reconcile 共用 writer lock + immediate transaction | -| Run 收尾与并发 Task create 不能同时成功 | 通过，并发测试锁定只有一个可串行化结果 | -| 同一成员的并发 Wake 不得制造多个空 turn | 通过，确定性 idempotency key 合并请求 | -| Wake 只有真正被 scheduler 接受才消耗恢复预算 | 通过，coalesced / rejected / paused 不计 attempt | -| scheduler 真正开始 turn 前 Session 不得伪装 Running | 通过，状态更新已移至实际执行边界 | -| Paused / terminal Run 的排队 Wake 不得复活 provider | 通过，执行前重新读取 Run 并 fail closed | -| Worker 不能自动认领 ownerless task | 通过，ownerless 只通知 Coordinator 明确指派 | -| Worker 不能修改其他成员的任务状态 | 通过，Coordinator 全局权限与 Worker 自有任务权限分离 | -| Worker 输出后忘记完成 Task 不能永久卡住 | 通过，有限自动修正；失败后通知 Coordinator，不做无限 Wake | -| 等待 Plan 审批时不得每分钟 Wake 或闪烁 | 通过，pending approval 是明确 quiet state | -| 失败成员任务重排不能丢 metadata 或制造半写入 | 通过，Task 与 history event 同事务 | -| Task 完成和 dependency unblock 不能重复发 TaskAssigned | 通过，side effect 依赖事务内 mutation outcome | - -## 验证结果 - -| 检查 | 结果 | 说明 | -| ---------------------------------------------------- | ---------------- | ----------------------------------------------------------------- | -| `pnpm typecheck` | 通过 | TypeScript 类型检查无错误 | -| `pnpm run lint` | 通过 | ESLint 无错误 | -| `pnpm run check:circular` | 通过 | 5,077 文件，无循环依赖 | -| 目标前端单测 | 25/25 | Group chat、Kanban、Task outcome | -| 前端全量单测 | 4,071/4,076 | 5 个失败均位于未修改的 develop 基线模块 | -| `cargo check -p org2` | 通过 | 桌面端完整编译 | -| `cargo check -p e2e-test` | 通过 | Rust E2E target 编译 | -| `cargo test -p agent_core --lib -- --test-threads=1` | 2,984/2,986 | 沙箱外运行；剩余 2 个是未修改的 skill-content 与 search-tool 基线 | -| Agent Org / Recovery / Approval / Lifecycle tests | 全部通过 | 包含并发、暂停、恢复、ownerless、审批与事务回滚 | -| changed Rust `rustfmt --check` | 通过 | 只检查本分支修改文件，不递归污染旧基线 | -| changed E2E `node --check` | 通过 | 变更的 `.mjs` 语法通过 | -| `git diff --check` | 通过 | 无空白错误 | -| `cargo clippy --all-targets -- -D warnings` | develop 基线阻断 | 5 个无关 crate 旧告警；本分支未修改这些文件 | -| Agent Core strict Clippy | develop 基线阻断 | 本次唯一新增告警已修；剩余均可在 develop 原代码复现 | - -## 全量测试中明确不属于本分支的失败 - -- 前端：Housekeeper settings manifest、Cursor external-history、planning-meta 旧断言、editUtils null 处理。 -- Rust：内置 `e2e-testing` skill 内容断言、search tool 的 `repo_path` / `repo_paths` 冲突断言。 -- Clippy：`orgtrack-core`、`cursor-bridge-app` 以及 Agent Core 旧模块的既有 lint debt。 - -这些文件没有被本分支修改。审计没有为追求“全绿数字”而把它们顺手改进来，避免扩大 #272 的评审范围。 - -## 提交判断 - -结论：**可提交**。本次范围的架构不变量、生产路径、事务边界、恢复策略、权限模型和 UI 投影均有实现与测试对应；未发现多余的目标外代码或会阻止提交的回归。 +# Agent Org #272 Final Architecture Audit + +- Date: 2026-07-14, updated with final verification on 2026-07-18 +- Baseline: `develop` +- Branch: `fix/issue-272-agent-org-recovery-invariants` +- Scope: Every branch difference affecting Agent Org Run, Task, Inbox, Wake, Watchdog, member lifecycle, Planner approval, Task authority, desktop projection, and E2E. + +## Final conclusion + +No known P0/P1 correctness issue remains in the approved #373 scope. Unrelated formatting and historical lint cleanup were removed from scope, while the implementation preserves five independent state dimensions: Run, Session, Task, Approval, and Inbox/Delivery. The Coordinator assigns work explicitly, workers cannot claim ownerless Tasks, Watchdog recovers only real actionable events, and Plan approval advances from durable state rather than empty Wake or model guesses. + +## Ten-layer audit + +| Layer | Line / element | Verdict | Reason | Suggested change | +| ---------------------------- | ------------------------------------------------- | ---------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------- | +| 1. Compilation and warnings | Rust desktop / E2E / frontend | keep with reason | `org2`, `e2e-test`, TypeScript, and ESLint pass. No remaining strict-Clippy finding is on a line introduced by this review; workspace baseline remains. | Clean baseline in a separate PR. | +| 2. Dead code and duplication | Auto-claim, `find_available`, old blocker helpers | fixed | Worker auto-claim paths and orphaned helpers are removed; no second ownerless auto-claim path remains. | None. | +| 3. Naming and responsibility | RecoveryPlan / Analyzer / Executor / Wake outcome | keep | Analyzer reads, Executor performs side effects, and outcomes distinguish Enqueued, Coalesced, Paused, Terminal, NoWork, and failure. | None. | +| 4. State dimensions | Run / Session / Task / Approval / Delivery | keep | Each persists and transitions independently. Idle is not Run completion, Pending is not claimable, and Wake is not consumption. | Preserve the separation in future features. | +| 5. FSM and defaults | Session status / approval policy / finality | keep | Critical states are explicit. Terminal, Paused, awaiting approval, malformed data, and historical compatibility have named behavior. | Require exhaustive enum handling. | +| 6. Cross-domain boundary | Hierarchy communication vs Task authority | fixed | Hierarchy Mode controls communication. Coordinator owns global Task administration; a worker may advance only its own assigned work. | None. | +| 7. New-developer clarity | Prompt, tool schema, typed guidance | fixed | Assignment, dynamic dependency, Planner mode, approval responsibility, and ownerless behavior are documented in contracts. Correctable misuse returns guidance instead of a red failure card. | Retain prompt/schema contract tests. | +| 8. Wire and data boundary | Typed Inbox, mutation outcome, reserved metadata | fixed | New writes validate roster, eligibility, owner, dependencies, identifiers, and payload size. Historical bad data remains readable and repairable. | A future schema PR may evaluate an eligibility join table. | +| 9. Initialization parity | Production / test / debug / E2E database | fixed | Recovery, approval, Inbox resolution, and intent schema use canonical initialization. Unsupported CLI Agent Org is rejected during both definition validation and launch preflight. | Do not reopen CLI until full parity exists. | +| 10. Resolver symmetry | Run/member/Session/Task identity | keep | Resolution consistently uses member id, Session id, Root Session, and Run id. UI and Inbox prefer member identity over shared agent definition identity. | None. | + +## Critical invariant review + +| Invariant | Result | +| --------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------- | +| A non-Running Run cannot create, update, delete, claim, or reassign a Task. | Pass: Task mutation and finality share writer serialization and an IMMEDIATE transaction. | +| Finality and concurrent Task creation cannot both succeed. | Pass: concurrency tests allow only one serializable order. | +| Concurrent Wake requests for one member cannot create multiple model turns. | Pass: stable idempotency key and durable Coalesced outcomes. | +| Recovery budget is consumed only when the scheduler accepts a Wake. | Pass: Coalesced, Rejected, Paused, and failed enqueue do not consume an attempt. | +| Session must not appear Running before the scheduler starts the turn. | Pass: state transition occurs at the execution boundary. | +| A queued Wake cannot revive a Paused or terminal Run. | Pass: execution re-reads Run state and fails closed. | +| A worker cannot auto-claim an ownerless Task. | Pass: ownerless produces Coordinator assignment guidance only. | +| A worker cannot modify another member's Task state. | Pass: Coordinator authority and worker-owned authority are separate. | +| A worker result without Task completion cannot stall forever. | Pass: bounded correction is attempted, then Coordinator is notified without infinite Wake. | +| Awaiting Plan approval cannot Wake every minute or flash. | Pass: pending approval is an explicit quiet state. | +| Failure requeue cannot lose metadata or commit half a history event. | Pass: Task and history/outbox share a transaction. | +| Dependency release cannot duplicate `TaskAssigned`. | Pass: side effects use a transaction-local mutation outcome. | +| Undeliverable Inbox cannot be falsely read or block finality forever. | Pass: explicit Cancelled/Superseded resolution preserves evidence and removes only the pending blocker. | +| Group Chat refresh cannot lose complete durable history. | Pass: bounded preview plus paginated durable history, verified with 230+ rows. | + +## Final verification + +| Check | Result | Notes | +| ----------------------------------------------- | ------------------------- | ----------------------------------------------------------------- | +| `pnpm typecheck` | Pass | Full TypeScript check | +| `pnpm lint` | Pass | Full ESLint check | +| Full frontend unit suite | 5,181 / 5,181 | 450 files | +| `cargo check -p org2 --lib` | Pass | Desktop library | +| `cargo check -p e2e-test` | Pass | Rust E2E target | +| Full `agent_core` suite | 3,155 / 3,155 | Isolated persistence and loopback-enabled environment | +| Full Agent Org HTTP E2E | 47 / 47 | Real isolated Debug App | +| Rendered Debug App E2E | 19 / 19 | Group Chat 6, Pause/Resume 8, Recovery 2, Settings 3 | +| Changed-scope formatting and `git diff --check` | Pass | No whitespace error in branch changes | +| Strict workspace Clippy | Existing develop baseline | No remaining finding is on a line introduced by this review. | +| Full workspace rustfmt | Existing develop baseline | Six unchanged files remain outside formatting baseline. | +| Circular dependency check | Existing develop baseline | Two cycles are confined to unchanged Org2Cloud/SessionCore files. | + +## Scope discipline + +The audit did not modify unrelated Git, Key Vault, Provider, Session Memory, OrgTrack, or other baseline modules merely to produce a green global number. Complete Revision Events, Event Projector, incremental subscriptions, and a full persistent Scheduler Outbox remain outside #373 and are documented separately. + +## Commit verdict + +**Implementation and verification are complete for the approved scope.** The local worktree still requires an intentional commit and push before GitHub PR #373 reflects the final fixes. No known P0/P1 blocker remains. diff --git a/docs/architecture-audit-2026-07-16/AgentOrgReviewSafetyAudit.md b/docs/architecture-audit-2026-07-16/AgentOrgReviewSafetyAudit.md index f620cd064..fca683f7c 100644 --- a/docs/architecture-audit-2026-07-16/AgentOrgReviewSafetyAudit.md +++ b/docs/architecture-audit-2026-07-16/AgentOrgReviewSafetyAudit.md @@ -1,139 +1,305 @@ -# Agent Org PR Review Correctness and Performance Audit +# Agent Org PR #373 Review Safety — Architecture Audit - Date: 2026-07-16 -- Scope: the two review rounds on PR #373 and the minimum dependencies needed to make those fixes compile, run, and remain testable -- Status: implementation audit and A-only verification complete, with documented clean-develop baselines -- Verification date: 2026-07-18 +- Scope: All safety fixes identified by the two PR #373 review rounds, excluding the full Revision Event architecture +- Conclusion: The correctness, concurrency, payload, polling, recovery, deletion-lifecycle, and test-fidelity issues found in this review are addressed on production paths. The full Revision Event architecture remains a separate follow-up PR by design. -## Executive summary +## Executive conclusion -This follow-up keeps Agent Org's product model unchanged: coordinators and workers still reason about the work, while Rust owns durable state transitions and the frontend renders persisted facts. The implementation and this report are limited to the two review rounds and the minimum dependencies required to make those fixes correct and independently testable. +This work does not change the product responsibilities of Agent Org, nor does it turn the Analyzer, Poller, or Snapshot into a new decision-making “brain.” Coordinators and workers still perform reasoning. The Rust backend constrains their actions into recoverable, concurrent, auditable state transitions, while the frontend reads and displays facts already persisted by the backend. -The reviewed production path is: +The review hardened this production chain: ```mermaid flowchart LR - MODEL["Coordinator or worker proposes an action"] - TOOL["Typed tool boundary"] - TX["Short SQLite transaction"] - WAKE["Budgeted Wake dispatch"] - VIEW["Read-only compact Run View"] - UI["Group Chat, Team Tasks, and Kanban"] - - MODEL --> TOOL --> TX --> WAKE + LLM["Coordinator / Worker\nproposes an action"] + TOOL["Typed Tool Boundary\nauthority + size + state validation"] + TX["SQLite IMMEDIATE Transaction\nRun + Task + Inbox + Approval"] + WAKE["Wake Dispatcher\nbudget + idempotency + execution-time recheck"] + VIEW["Read-only Compact Snapshot\nshared Poller"] + UI["Group Chat / Team Tasks / Kanban"] + + LLM --> TOOL --> TX --> WAKE TX --> VIEW --> UI ``` -## Scope boundary - -Included here: - -- the first review's read-path, polling, payload, locking, task-graph, finality, Wake-budget, and error-isolation findings; -- the second review's failure finality, Plan approval, Inbox receipt, Task authority, legacy dependency, extractor, Kanban, wire-parity, and test-fidelity findings; -- stable `useSyncExternalStore` Snapshot identity, keyset Group Chat history, canonical finality/progress helpers, transactional Task outbox, Inbox materialization receipts, and real-Run fixtures as required dependencies; -- dispatch-time user-intervention establishment for direct and queued user messages: intervention is persisted after the durable user event and immediately before provider dispatch; persistence failure stops dispatch; -- focused tests and these English audit reports. - -This audit intentionally contains only the two review rounds and their required integration dependencies. Later hardening and future architectural work are tracked outside this branch. - -## Review finding matrix - -### First review - -| Finding | Risk | In-scope resolution | -| ----------------------------------------------------- | ------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------- | -| Run View reads performed writes | Frequent UI polling could take the global writer lock and mutate lifecycle state. | Run View now reads a deferred, read-only Snapshot. Reconciliation and intervention cleanup remain explicit write operations. | -| `task_list` used the global writer path | A read-only tool could serialize unrelated Task, Session, and Inbox work. | `task_list` uses a read transaction and returns bounded summaries; `task_get` loads full detail on demand. | -| Poll payloads grew with Inbox, Plan, and Task content | Every poll copied large durable payloads and repeated frontend parsing. | Run View carries counts, previews, and Plan summaries. Full Plan, Task, Inbox, and Group Chat data use explicit detail or keyset APIs. | -| Task board loading repeated graph work | Repeated lists and per-Task dependency scans created N+1 behavior. | A transaction-local `TaskGraphIndex` is built once and shared by mutation and outbox decisions. | -| Finality checks disagreed | A Run could repeatedly reconcile without reaching a stable terminal decision. | Finality facts, assessment, and decision are centralized; minimal work revision and explicit empty-Run completion provide durable evidence. | -| Idle Wake bypassed recovery budget | Some paths could call the model repeatedly while watchdog paths backed off. | Production Wake sources use durable reserve, enqueue, and commit-or-refund accounting. | -| Plan and text persistence lacked direct bounds | One request could write an unexpectedly large Plan, feedback, Task description, or message. | Review-requested content fields are validated before file, SQLite, or Inbox persistence. Historical rows remain until Run deletion. | -| File I/O occurred while holding the writer lock | Slow Plan writes could block unrelated Agent Org updates. | Plan content is staged outside the lock; the lock protects only the short transaction and atomic installation step. | -| Synchronous SQLite ran on async execution threads | Database calls could stall unrelated async work. | Synchronous persistence work runs in a blocking section and reuses one connection per operation. | -| Frontend reparsed completed results | Large result wrappers were parsed repeatedly during render. | Explicit outcomes return early, and Task outcome resolution is memoized at the adapter boundary. | -| `task_graph_create` was not extracted consistently | Communication cards or replay could show raw JSON or erase existing Kanban state. | Rust extraction, Task cards, event routing, and additive Kanban replay recognize Task Graph outcomes. | -| One bad Run stopped watchdog scanning | A single database or analysis error could prevent later Runs from recovering. | Watchdog reports an inner per-Run error and continues scanning remaining Runs. | - -### Second review - -| Finding | Risk | In-scope resolution | -| --------------------------------------------------------------- | -------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| A failed or cancelled worker immediately abandoned the Run | Recoverable work could be declared terminal. | Run-terminalization paths covered by this review use the shared finality model; abandonment requires all relevant workers to be archived and no recovery path to remain. | -| Task Graph rendered inconsistently | Some surfaces showed raw JSON and historical replay could clear the board. | All three UI projections consume structured graph outcomes and merge graph creation additively. | -| Rust E2E fixtures skipped a real Run | Tests could pass while violating production Run invariants. | Fixtures initialize the canonical schema and create an actual Running Run before exercising Agent Org commands. | -| Watchdog hid errors or accepted an empty plan forever | Runs could remain stuck without an observable recovery action. | Errors remain visible per Run; a rejected reconcile continues to Wake valid work or emits the minimum coordinator repair notice. | -| Empty-task Runs could not complete | A coordinator that legitimately needed no Tasks had no terminal action. | The minimal `org_run_complete` path records explicit coordinator completion intent and still passes canonical finality checks. | -| Pause or restart cancelled pending Plan approval | User approval state disappeared even though the Run was not terminal. | Pending approval survives pause and restart; only terminal or missing Runs clear it. | -| Crash windows could duplicate Inbox delivery | The durable row and visible turn could diverge after restart. | Inbox materialization receipts and `causation_inbox_id` make acknowledgement conditional on successful materialization. | -| Legacy `blocks` did not participate in readiness | Old Tasks could unlock too early. | `blocks` is normalized into the canonical dependency graph used by every readiness check. | -| Workers could delete ownerless Tasks | A worker could remove coordinator-managed work it did not own. | Ownerless Task deletion is coordinator-only. | -| Approval persisted but notification failure returned UI failure | The user could retry an approval that was already committed. | Approval state and Inbox notification commit together; Wake happens after commit and stale revisions return a readable error. | -| TypeScript treated args-only operations as success | A failed tool attempt could mutate historical Kanban state. | New events require a structured outcome; legacy events require durable result evidence. Successful deletion removes the Task row from replay. | -| Rust and TypeScript wires drifted | `executionMode` and Task output could be absent or duplicated. | `executionMode` is explicit and Task output has one canonical wire location. | -| Tests initialized a weaker schema | Test-only behavior could mask missing tables or invalid Run state. | Production and tests share schema initialization and real-Run invariants; test-only bypasses are removed. | - -### Required integration dependency - -| Dependency | Why it is required | A-only implementation | -| ------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| User intervention at actual dispatch | A direct or queued user message must prevent a background Wake from racing the same member, but merely waiting in the queue must not suppress recovery. | Direct messages establish intervention after the durable user event and immediately before provider dispatch. Queued messages establish it only when dequeued for dispatch. Both paths fail closed if intervention persistence fails. | +## Acceptance checklist + +| Acceptance item | Result | Evidence | +| --------------------------------------------- | ------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Rust / TypeScript compilation | Pass | `cargo check -p agent_core --all-targets`, `cargo check -p e2e-test`, `cargo check -p org2 --lib`, and `pnpm typecheck` | +| New Clippy warnings introduced by this change | 0 | The one warning on a newly added line was fixed. Remaining strict-Clippy findings are existing develop baseline; see “Known develop and environment baseline.” | +| Frontend lint | Pass | `pnpm lint` | +| Full frontend unit suite | Pass | 450 files, 5,181 tests | +| Focused Agent Org Rust suites | Pass | Watchdog, Run, Plan, Task, Inbox, Lifecycle, Commands, and Send suites | +| Full `agent_core` suite | Pass | 3,155 / 3,155 with loopback access and an isolated persistence directory | +| Full `session_persistence` suite | Pass | 34 / 34, single-threaded outside the restricted sandbox | +| High-frequency Run View has no hidden writes | Pass | Snapshot uses a read-only deferred transaction. Historical Plan artifact reconciliation is an explicit, low-frequency operation limited to a managed directory. | +| Large payloads are bounded | Pass | Task summaries, Inbox drain batches, Plan detail, pagination, identifiers, and per-Run task counts all have explicit limits. | +| Runtime HTTP E2E | Pass | 47 / 47 scenarios against a real isolated Debug App | +| Rendered UI E2E | Pass | 19 / 19 WebDriver scenarios across Group Chat, pause/resume, recovery, and settings | +| Production return-to-work path | Pass | Real scheduler → processor → deterministic provider → Inbox drain path | +| Revision Event architecture | Explicitly excluded | See [AgentOrgRevisionEventPlan.md](./AgentOrgRevisionEventPlan.md). | ## Ten-layer architecture audit -| Layer | Result | Evidence and boundary | -| ------------------------------------- | --------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| 1. Compilation and static correctness | Verified with baseline exceptions | Rust compilation, TypeScript typecheck, formatting, frontend lint, and circular-dependency checks pass. Strict Clippy remains blocked by clean-develop findings documented below. | -| 2. Dead code and duplicate paths | Reviewed | Run reads, finality, Task graph evaluation, Task outbox, Plan persistence, Wake budgeting, and user-intervention dispatch were reviewed for competing paths; no duplicate A-only production path was found. | -| 3. Naming consistency | Reviewed | Run, Session, Task, delivery, Snapshot, Wake, and work revision remain distinct concepts. | -| 4. Semantic overloading | Reviewed | Run status does not stand in for Session, Task, or Inbox state; finality consumes typed facts across those dimensions. | -| 5. Default branches | Reviewed | Paused, terminal, failed, missing, stale approval, legacy outcome, and rejected tool states fail closed or return explicit guidance. | -| 6. Cross-domain leakage | Reviewed | Models reason; Rust validates and persists; the scheduler dispatches; the frontend projects durable state. | -| 7. New-developer comprehension | Reviewed | Mutation outcomes, finality blockers, Wake outcomes, and Plan summaries use typed structures rather than human-text parsing. | -| 8. Wire protocol and payloads | Reviewed | Polling returns summaries and exact counts; detail endpoints carry complete content; outcome evidence is explicit. | -| 9. Initialization parity | Reviewed | Production, unit tests, and HTTP E2E use the shared schema and real Run records. | -| 10. Resolver symmetry | Not applicable | This scope does not change a multi-field fallback resolver. Relevant identity and fallback paths were reviewed and no asymmetric resolution was introduced. | - -## Core invariants retained by this scope - -1. Only a Running Run may mutate Tasks. -2. Ownerless means unassigned, not available for arbitrary worker self-claim. -3. A normal worker may mutate only its own assigned work state. -4. Task mutation and its TaskAssigned or TaskCompleted outbox share one transaction result. -5. Wake attempts are charged only after scheduler acceptance; rejected and coalesced requests are refunded. -6. A Paused or terminal Run is rechecked before Wake execution and cannot be revived accidentally. -7. Plan approval is durable independently of transient Wake delivery. -8. Completion requires the coordinator to observe the latest work revision and pass the shared finality decision. -9. Historical Group Chat remains available through keyset pagination after reload. -10. Task and Plan UI state comes from structured persisted outcomes, not attempted tool arguments. -11. A user message establishes member intervention only when it is actually dispatched. A message waiting in the queue does not suppress Wake, and failure to persist intervention prevents provider dispatch. - -## Verification status - -The implementation was split after a larger combined audit. Only results produced from the A-only tree are reported here. - -| Gate | A-only result | -| ----------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| Rust compilation and focused review regressions | Pass. The Agent Org compilation and focused review suites complete successfully. | -| `agent_core` application suite | Pass: 3,028 / 3,028. The 23 `specialization::external_import` tests are filtered because the same nested `lock_home` deadlock is reproduced on clean develop. | -| `session_persistence` | Pass: 29 / 29. | -| Frontend unit and static verification | Pass: Vitest 5,318 / 5,318; TypeScript typecheck, ESLint, and circular-dependency checks pass. | -| Changed-scope Rust formatting | Pass: all 80 Rust files changed relative to develop pass `rustfmt --check`. | -| Strict Clippy | Baseline-blocked; no new A-only warning is known. See the baseline note below. | -| Isolated real Debug App Agent Org HTTP E2E | Pass: 46 / 46 against the real isolated Debug App and production behavior paths. | -| Rendered WebDriver Group Chat E2E | 5 / 6 pass. The remaining mention-menu scenario fails identically on clean develop and is not an A regression. | -| Rendered WebDriver Pause/Resume E2E | Pass: 8 / 8. | -| Rendered WebDriver Recovery E2E | Pass: 2 / 2. | - -### Clean-develop baseline exceptions - -- Strict workspace Clippy is blocked by seven `orgtrack_core` diagnostics reproduced on clean develop. -- `agent_core --no-deps` reports 45 diagnostics reproduced on clean develop. -- `e2e-test --no-deps` reports three diagnostics on A versus four on clean develop, so A does not add a Clippy diagnostic. -- Workspace-wide `cargo fmt --all --check` reports formatting drift in unchanged develop files; every Rust file changed by A passes an explicit `rustfmt --check`. -- The 23 filtered `specialization::external_import` tests enter the same nested non-reentrant `lock_home` deadlock on clean develop. They are reported explicitly rather than counted as passing. -- The rendered mention-menu scenario fails on A and clean develop with the same `Agent Org mention menu did not include both member and normal context options` assertion. It is not counted as passing and is outside this A-only follow-up. - -## Conclusion - -The A-only tree addresses the two review rounds and their minimum correctness dependencies. Source-level, unit, integration, real Debug App HTTP, and all in-scope rendered UI verification are complete. The one remaining rendered scenario is a reproduced clean-develop baseline outside this follow-up, not an A regression. +### Layer 1 — Compilation and static correctness + +- All three relevant Rust crates compile. +- TypeScript typecheck and ESLint pass. +- Strict Clippy found no warning on a line introduced by this review after the final cleanup. Remaining warnings are reproducible on existing develop code. +- Changed-scope rustfmt and `git diff --check` pass. + +Conclusion: no `allow` attribute was added to hide a new error, and unrelated modules were not changed merely to make a gate look green. + +### Layer 2 — Dead code and duplicate paths + +This review traced production entry points instead of relying on reference counts: + +| Business entry point | Unified production path | Duplicate behavior removed or avoided | +| --------------------------------------------- | ------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------- | +| Watchdog / resume / task tool requests a Wake | recovery disposition → shared budget → dispatcher → scheduler | Each entry point making its own decision, enqueueing directly, and consuming budget independently | +| Root / Member / Kanban reads a Run | one per-Run shared poller → compact snapshot → caller projection | Every session polling and copying the full Run independently | +| Single Task / Task Graph creation | typed validation → one transaction → one mutation outcome | Writing graph nodes one by one and leaving a partial graph after failure | +| Dependency release after Task completion | transaction outcome → exact transactional outbox / Wake | Re-reading state outside the transaction and duplicating `TaskAssigned` under concurrency | +| Plan artifact persistence | stage temporary file outside lock → short durable transaction → atomic install | Slow file I/O while holding the global writer lock and repeated connection setup | +| Plan artifact ownership | source session → managed Plan root → exact `.plan.md` child | Overwriting or deleting an external Markdown file or symlink referenced by historical bad data | +| Member intervention clear | clear + capture unread high-water mark in one IMMEDIATE transaction | Reading clear state and delivery boundary separately, creating a race | +| Turn intent ownership | persist nullable `org_run_id` on every intent | Guessing Run ownership from the Session tree and damaging nested Runs or missing Resume Wakes | + +Conclusion: Analyzer, Executor, Snapshot, Progress, and Finality types are wired into real production entry points. They are not abstractions kept alive only by tests. + +### Layer 3 — Naming consistency + +| Name | Exact meaning | +| --------------------------------- | ---------------------------------------------------------------------------------------------------------------- | +| `AgentOrgRun` | The durable business container for one team execution; it is not a model turn. | +| `RunView` / Snapshot | A compact, read-only projection of one Run at a point in time; it is not another source of truth. | +| `work_revision` | A monotonic proof that the Coordinator observed the latest work state; it is not a full event-stream revision. | +| `RecoveryAnalyzer` | A pure analysis step that reads a snapshot and proposes actions; it does not call a model or write the database. | +| `RecoveryExecutor` | Executes proposed recovery actions after revalidating current state; it does not reason about task content. | +| `Wake` | Requests a scheduler opportunity for a Session; it does not mean the turn is already Running. | +| `TaskSummary` | A bounded list/poll projection; full content is loaded through `task_get`. | +| `session_turn_intents.org_run_id` | Explicit ownership of a turn intent by an Agent Org Run; it is not inferred from Session ancestry. | + +Comments, wire fields, and TypeScript types were checked so that `running` does not ambiguously mean Run state, Session state, Task state, and UI activity. + +### Layer 4 — Semantic overloading + +The four state dimensions remain independent sources of truth: + +| Dimension | Source of truth | Invalid inference | +| -------------- | ------------------------------ | ----------------------------------------------------- | +| Run state | `agent_org_runs.status` | An Idle Session does not mean the Run is complete. | +| Session state | `agent_sessions.status` | A Running Run does not mean every worker is healthy. | +| Task state | `agent_org_tasks.status/owner` | A Pending Task does not mean any worker may claim it. | +| Delivery state | Inbox row + scheduler intent | Unread Inbox data does not prove a Wake was accepted. | + +Run finality is now evaluated from typed facts rather than treating one status as a substitute for the entire collaboration state. + +### Layer 5 — Default-branch audit + +- Recovery explicitly classifies `SessionStatus` as Active, Wakeable, Backoff, Exhausted, Paused, Pending materialization, Missing, or Archived. +- Paused, pending grace, Missing, Archived, and invalid timestamps have explicit behavior instead of falling into `_ => false`. +- Before a Wake begins execution, Running, Paused, terminal, missing, and database-error states are handled separately. Database errors fail closed. +- The frontend handles null, stale responses, and unknown or failed payloads separately; parse failure never defaults to “clear the board.” +- A Run View generation created by an abandoned React render and never subscribed is retired after 30 seconds. Late IPC responses cannot restore it as poll owner. + +Conclusion: new states and malformed input cannot silently become normal Idle or Running behavior. + +### Layer 6 — Cross-domain leakage + +- Coordinator and worker reasoning remains in the model layer. Recovery Analyzer, Recovery Executor, and Poller contain no business-reasoning prompts. +- The frontend does not directly mutate Task, Run, or Inbox truth. It invokes typed commands and projects durable outcomes. +- Incomplete CLI Agent Org runtime support still fails loudly instead of borrowing the Rust-member transport. +- A Plan file is an artifact of durable approval content. The SQLite approval row remains lifecycle truth. + +Conclusion: UI, model, scheduler, and persistence layers retain separate responsibilities. + +### Layer 7 — New-developer comprehension + +Important rules are now expressed through types, structured outcomes, and comments: + +- `WakeRequestOutcome` distinguishes Enqueued, Coalesced, DeferredPaused, DeferredBackoff, NoWork, RunTerminal, SessionUnavailable, and Failed. +- `TaskMutationOutcome` carries transaction-local previous/current state and meaningful transitions, so side effects do not guess after the transaction. +- Finality follows facts → assessment → decision and returns typed blockers explaining why completion is unsafe. +- Task tools return structured guidance for recoverable misuse instead of leaking red SQL or validation failures into the model trajectory. + +Conclusion: a new developer can distinguish an action that occurred from a request, deferral, or no-op without knowing the history of the bug. + +### Layer 8 — Wire protocol and payloads + +| Channel | Current boundary | +| ---------------- | ------------------------------------------------------------------------------------------------------------------------- | +| `task_list` | 50 rows by default, 200 maximum; descriptions are at most 512 Unicode characters and expose `description_truncated`. | +| `task_get` | Loads one Task with full detail on demand. | +| Run View | At most 200 Task summaries; exact totals and counts are returned separately and are never inferred from window length. | +| Task persistence | At most 200 Tasks per Run and 32 nodes per graph; existing + incoming counts are checked inside the same transaction. | +| Task identifier | At most 1,000 Unicode characters / 4,000 UTF-8 bytes across Task, dependency, cursor, and every Inbox task-id position. | +| Inbox | Message, array, and JSON fields have character and byte bounds; one drain is capped at 128 rows / 1 MiB. | +| Plan | Snapshot carries only a summary; detail is loaded and cached by `run + approval + revision`. | +| Tool outcome | New events prefer typed outcomes. Legacy events require durable success evidence and cannot succeed from arguments alone. | + +High-frequency polling no longer copies full TaskOutput, Plan, or Inbox payloads. Full information remains available through bounded, on-demand APIs. + +### Layer 9 — Initialization parity + +| Entry point | Run schema | Task / Inbox / Recovery | Plan approval | Session runtime | Production drain | +| ------------------------ | ----------------------- | ------------------------ | ------------------------- | ----------------------------------- | -------------------------------- | +| Tauri production | Yes | Yes | Yes | Yes | Yes | +| App restart | Yes | Yes, with reconciliation | Yes, with artifact repair | stale Running → failure disposition | Yes | +| Rust unit sandbox | Yes, shared initializer | Yes, shared initializer | Yes | Complete test schema | Production Store | +| HTTP E2E production path | Yes | Yes | Yes | Real Session registration | scheduler → processor → provider | + +`session_turn_intents.org_run_id` is included in both fresh DDL and historical database initialization. It is nullable for non-Agent-Org and Wingman intents, while root, member, Wake, steering, and normal follow-up Agent Org turns persist explicit Run ownership. Historical NULL rows may be backfilled safely; an existing, different non-NULL owner fails closed and cannot be silently reassigned. + +Fixtures that previously omitted tables or Run rows now use the canonical initializer and Store. Production invariants were not weakened to accommodate tests. + +### Layer 10 — Resolver symmetry + +| Resolver | Primary condition | Fallback / recheck | Symmetry result | +| ------------------- | -------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------ | +| Wake target | Run + member + session + real work | Re-read Run before execution; classify terminal, paused, and missing separately. | Every entry point uses the same dispatcher and budget. | +| Plan detail | run id + approval id + revision id | Validate Run ownership before artifact repair. | No cross-Run file read or mutation. | +| Run View poll owner | live Session + known Run id | On owner error/null/hang, hand off to another live Session in the same Run; unknown-Run bootstrap has a timeout. | Root and member use the same source chain. | +| Task availability | pending + ownerless + ready + eligible + member not busy | Coordinator assignment / recovery only; arbitrary workers no longer self-claim. | Watchdog, resume, and tools use the same definition. | +| Turn intent owner | explicit `org_run_id` | Explicit/runtime mismatch fails closed; non-Agent-Org intent remains NULL. | Deletion, finality, and nested Runs no longer infer ownership from Session ancestry. | + +Conclusion: related fields do not mix database truth with memory-only truth, and no production Wake entry point bypasses the shared recovery budget. + +## Mapping the two review rounds to fixes + +| Finding category | Original risk | Final fix | +| ------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Read View side effects | Opening the UI could repair or write state and block the global writer. | Read-only deferred Snapshot; repair moved to explicit write paths. | +| Duplicate polling by multiple Sessions | Root and every member repeatedly fetched the full Run. | One shared poller per Run with caller-specific `currentMemberId` projection. | +| Failed poll owner | A null/error/hung owner could clear the board or block another Run. | Sequence guard, same-Run failover, null ghost cleanup, and one-second bootstrap timeout. | +| Unstable Snapshot identity | `useSyncExternalStore` could repeatedly render or loop. | Create a stable entry on the first read and preserve reference identity until an actual publish. | +| Oversized Task / Inbox / Plan payloads | High-frequency polling copied long content, increasing memory and token pressure. | Summary by default, detail APIs, pagination, and independent row/byte limits. | +| Unbounded Task count | A Run could accumulate unlimited Tasks. | 200 per Run, 32 per graph, checked transactionally under concurrency. | +| Plan file write under lock | Slow file I/O blocked the Agent Org writer. | Stage outside the lock, perform a short transaction, then atomically install after commit. | +| Cross-Run Plan detail | A wrong id could repair an artifact belonging to another Run. | Validate the exact `run + approval + revision` relationship before read or repair. | +| Plan path escape | Historical or corrupt paths could reference arbitrary Markdown files, symlinks, or another workspace. | Permit only a direct `.plan.md` child under the source Session's managed root; external historical paths are warning-only and are never written or deleted. | +| Wake budget bypass | Some sources could enqueue directly while only Watchdog respected backoff. | Every production Wake source checks the durable budget; only Enqueued consumes an attempt. | +| Intervention-clear race | A new message between clear and boundary reads could be misclassified. | Clear and capture unread high-water in one IMMEDIATE transaction. | +| `block_in_place` on current-thread Tokio | Lifecycle code could panic on a current-thread runtime. | Shared blocking-section helper selects safe behavior for the active runtime flavor. | +| Finality guessed from status | Coordinator could finish without seeing the latest Task state. | Durable `work_revision`, observed revision, terminal turn, and explicit completion request. | +| Reconcile/create race | A terminal Run could end up with a newly created open Task. | Same writer lock and IMMEDIATE transaction; only serializable outcomes are allowed. | +| Incomplete deletion lifecycle | Deleting a worker could delete a Run, or deleting a root could retain Inbox/Plan history. | Worker deletion removes only the Session; root deletion removes the owned Run and cascades owned history by foreign key. | +| Session tree mistaken for Run boundary | Outer-Run deletion/finality could damage a nested Run, while Resume Wake intents could be missed. | Persist `org_run_id` on intents and query deletion/in-flight state by exact Run. | +| Abandoned React render leak | `getSnapshot` without subscribe retained a bounded preview forever and late IPC could revive poll ownership. | Retire zero-listener generations after 30 seconds and reject responses for retired or replaced generations. | +| Task identifier beyond delivery boundary | A Task could persist but fail to produce `TaskAssigned`, leaving unreachable work. | Store, tool, and Inbox independently validate Task, dependency, and cursor IDs before persistence. | +| False test path | Debug drain or partial SQL fixtures produced false positives and false negatives. | Canonical Store fixtures plus real scheduler/provider/Inbox-drain paths. | +| Undeliverable unread Inbox blocks finality | Missing, Archived, Paused, CLI, or expired-pending recipients could never read a row, while unread state blocked the Run forever. | Immutable roster + canonical member identity produces typed repair; the source remains unread and is never guessed or falsely acknowledged. | +| Analyzer/Executor TOCTOU | Member, Task, or Run state could change after analysis and produce stale Assignment, Continuation, or repair actions. | Revalidate Run, Session, transport, agent identity, Task graph, and typed fingerprint in the final writer transaction. | +| Coordinator notice budget race | Notice insert, unavailable diagnostic, and attempt accounting were separate writes. | Budget gate, notice/diagnostic, and attempt update commit in one IMMEDIATE transaction. | +| Legacy shared AgentDefinition double count | A historical Inbox row with only `agent_id` could be attributed to multiple members. | Member cards use canonical `member_id`; legacy rows remain visible only in Run-level history. | +| Old unread message treated as empty UI | Exact unread state outside the most recent 200 activities could show a member as `No tasks` and disable navigation. | Both member switchers share a pure predicate that checks recent activity and exact unread total. | + +## Core invariants + +1. Only a Running Run may mutate Tasks; Paused and terminal Runs reject writes. +2. Ownerless means “currently unassigned,” not “any worker may claim this Task.” +3. An `in_progress` Task must have a valid owner, and a normal worker may change only its own work state. +4. A Wake counts as an attempt only after scheduler acceptance; coalesced and rejected requests consume no attempt. +5. A queued Wake does not mean the Session is Running; Running is persisted only when the turn actually starts. +6. A Paused Run does not drain Inbox or call the provider; a terminal Run is never revived. +7. Without a real message, unlocked Task, approval response, or explicit work item, Wake returns no-op and does not call the model. +8. Every Task mutation and its side-effect outbox share one transaction outcome. +9. Completed requires proof that the Coordinator observed the latest work revision and that no finality blocker remains. +10. Historical Inbox rows, Plan revisions, and future Events live as long as the Run by default and cascade only on explicit Run deletion. +11. Every Agent Org turn intent carries explicit `org_run_id`; deletion and finality do not infer a Run from Session ancestry. +12. A Plan artifact must be under the source Session's managed directory; Agent Org never overwrites or deletes an external path or symlink. +13. Task identifiers independently satisfy the same character and byte limits in the Store, tool boundary, and every Inbox task-id position. +14. A retired frontend Run View generation never accepts late responses or regains poll ownership. + +## Performance and resource boundaries + +- Running Runs are selected directly with `WHERE status='running'` instead of fetching an arbitrary 500 rows and filtering in memory. +- Unread checks use `SELECT EXISTS`. +- Watchdog interval uses missed-tick skip, so a process stall does not replay a burst of historical ticks. +- Task readiness uses sets and a single scan instead of Task × member × full-list queries. +- Watchdog and high-frequency Run View unread statistics scan only the `read_at IS NULL` partial index, not all read history. +- Recovery `TaskAssigned` actions for one member are revalidated in one writer transaction against Session state, the Task graph, and existing delivery. +- One Run shares one UI Snapshot. Full Plan and Task bodies are fetched on demand. +- Lists enforce count/cursor bounds and independent byte/character bounds. + +## Test results + +### Rust suites + +| Suite | Result | +| -------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------- | +| Agent Org Watchdog | Pass, including budget, backoff, paused/terminal, invalid timestamp, undeliverable recipient, and stale-plan matrices | +| Agent Org Run / finality | Pass, including work revision, concurrent mutation, nested Run, explicit intent owner, empty board, and recoverable Failed/Cancelled sessions | +| Plan approvals | Pass, including external files, symlinks, managed-path deletion, paused restart, atomic notification, and historical compatibility | +| Task Store / tools | Pass, including graph atomicity, pagination, `task_get`, Run cap, identifier boundary, legacy `blocks`, and worker authority | +| Inbox production drain | Pass, including row/byte bounds, task-id message matrix, idempotent materialization, delivery resolution, and the real consumption path | +| Member idle / Lifecycle | Pass, including failure disposition, pause/resume, and restart recovery | +| Agent Org Tauri commands | Pass, including pure Snapshot, exact counts, Plan detail ownership, and execution mode wire projection | +| Send / Wake / turn intents | Pass, including coalescing, budget reservation/refund, execution-time Run checks, and explicit `org_run_id` | +| Full `agent_core` | 3,155 / 3,155 pass in an isolated environment | +| Full `session_persistence` | 34 / 34 pass, single-threaded outside the restricted sandbox | + +### Frontend + +- Full Vitest: 450 files, 5,181 / 5,181 tests. +- Typecheck: pass. +- ESLint: pass. +- Independent UI audit: [AgentOrgReviewSafetyAudit.md](../frontend-ui-audit-2026-07-16/AgentOrgReviewSafetyAudit.md). + +### Real isolated Debug App runtime E2E + +The Agent Org HTTP suite passed 47 / 47 scenarios against a real Debug App with isolated ports and an isolated `ORGII_HOME`. + +The production return-to-work scenario proved: + +- `TaskAssigned` begins unread; +- production return-to-work enqueues and completes a Wake; +- production Inbox drain materializes exactly one visible assignment input and acknowledges the source row only after success; +- the deterministic provider completes a real member turn; +- a second return-to-work with no new durable input returns no-op. + +This is a real Rust/Tauri runtime path with a deterministic provider. It is not presented as a paid external-provider integration test. + +### Rendered Debug App E2E + +WebDriver tests passed 19 / 19 against rendered Debug App instances: + +- Group Chat: 6 / 6, including real keyboard mention selection, 230+ durable history rows, and Plan approval. +- Pause / Resume: 8 / 8. +- Recovery: 2 / 2. +- Settings: 3 / 3. + +Debug helpers only seed or inspect state; the user-visible behaviors under assertion use production application paths. + +## Known develop and environment baseline + +The following were deliberately not “fixed along the way” in #373: + +1. Full workspace `cargo clippy --all-targets -- -D warnings` stops on existing findings in unchanged `integrations` and `orgtrack-core` code. +2. `cargo clippy -p agent_core --all-targets --no-deps -- -D warnings` reports existing warnings on lines not introduced by #373. The one warning on a newly added line was fixed, and a non-blocking Clippy run completed successfully. +3. `pnpm check:circular` reports two existing cycles in unchanged Org2Cloud and SessionCore paths; every file in those cycles has zero #373 diff. +4. Full workspace `cargo fmt --all -- --check` reports six unchanged develop files. Changed-scope formatting and `git diff --check` pass. +5. Tests that bind local ports require loopback access outside the restricted sandbox; final isolated runs were executed with that access. +6. Rendered and runtime E2E use deterministic fake-provider responses to make state-machine assertions reproducible. They do not make paid external model calls. + +## Explicitly outside this PR + +The full Revision Event architecture is not part of #373. The current `work_revision` answers only whether the Coordinator observed the latest work state; it is not an incremental UI event bus. + +The follow-up design will atomically commit canonical mutations and small, sequentially numbered Events, then drive the frontend with Snapshot + replay + real-time notification. Design, phases, and acceptance gates are documented in [AgentOrgRevisionEventPlan.md](./AgentOrgRevisionEventPlan.md). + +## Final verdict + +This review does not change Agent Org's macro collaboration model. It converges the implementation from “multiple paths independently guess current state” to: + +- the model proposes intent; +- a typed tool boundary validates it; +- one SQLite transaction writes canonical truth; +- recovery performs bounded actions from persisted facts only; +- the UI reads a small, consistent Snapshot; +- full content is loaded on demand; +- completion requires durable proof. + +Within the approved #373 scope, no known P0/P1 correctness issue remains. Deferred work is limited to the explicitly separated Revision Event architecture and existing develop baseline findings. diff --git a/docs/architecture-audit-2026-07-16/AgentOrgRevisionEventPlan.md b/docs/architecture-audit-2026-07-16/AgentOrgRevisionEventPlan.md new file mode 100644 index 000000000..fb913693d --- /dev/null +++ b/docs/architecture-audit-2026-07-16/AgentOrgRevisionEventPlan.md @@ -0,0 +1,224 @@ +# Agent Org Revision Event Follow-up Architecture Plan + +- Date: 2026-07-16 +- Target branch: A separate follow-up PR, not part of #373 +- Prerequisite: The #373 safety fixes must land and remain stable first. + +## Executive summary + +The current UI periodically reloads the latest compact Snapshot. The proposed Revision Event architecture changes synchronization as follows: **every durable state change appends one small, sequentially numbered Event in the same database transaction; the UI first loads a full Snapshot, then receives only Events after that Snapshot. If an Event is missed or the client disconnects, it replays the missing revision range and reloads a Snapshot only when continuous replay is unavailable.** + +The Event log does not replace Run, Task, Session, Inbox, or Plan Approval tables. Those canonical business tables remain the source of truth. Events reliably tell consumers what changed. + +```mermaid +flowchart LR + M["One business mutation\nfor example, Task completion"] --> TX["Same SQLite transaction"] + TX --> C["Update canonical tables\nTask / Run / Inbox / Approval"] + TX --> E["Append revision Event\nrun-7 / revision 42"] + C --> COMMIT["Commit together"] + E --> COMMIT + COMMIT --> PUSH["Notify live subscribers"] + PUSH --> UI["Shared frontend Run Store"] + UI -->|"Receives 42 immediately after 40"| REPLAY["Replay revisions 41..42"] + REPLAY --> UI +``` + +## Why this is not part of #373 + +PR #373 already changes recovery, finality, Task transactions, Inbox behavior, approvals, and frontend projections. Revision Events affect every mutation path and the frontend synchronization model. Combining both changes would make review, regression attribution, and rollback substantially harder. + +PR #373 establishes the required safety foundation: + +- Reads use a consistent, read-only Snapshot and do not mutate the database merely because a view is open. +- Every view of the same Run shares one Poller instead of reloading the full state for each member. +- Large Task, Inbox, and Plan content is paginated or loaded on demand. +- Mutations and their corresponding durable notifications commit together. +- A minimal `work_revision` proves whether the Coordinator observed the latest Task state. + +`work_revision` serves only as finality evidence. It is not a complete event-stream revision. A future Event revision covers every observable state change in a Run. + +## Design principles + +1. **Canonical tables are truth; Events are change notifications.** A mutation cannot write only an Event without updating the Task or other canonical state, and current state must not require replaying the entire Event history. +2. **State and Event commit atomically.** If either canonical mutation or Event insertion fails, the whole transaction rolls back. +3. **Each Run has an independent, strictly increasing sequence.** Revision 41 may only be followed by 42, allowing consumers to detect gaps, duplicates, and reordering. +4. **Events remain small.** They contain identifiers, state, and bounded summaries. Full Plans, TaskOutput, and long messages remain available through `task_get`, Plan detail, or artifact APIs. +5. **Delivery is at least once; consumers are idempotent.** Receiving the same revision twice is harmless. The system must not silently lose an Event in pursuit of exactly-once delivery. +6. **Disconnection is recoverable.** A client records its last applied revision and replays the gap after reconnecting. +7. **Backpressure requests resynchronization instead of unbounded buffering.** If the real-time queue overflows, consumers receive a resync signal. +8. **History lives as long as the Run by default.** No arbitrary 30- or 90-day TTL is introduced. Explicit Run deletion cascades its Events. + +## Persistence model + +The first version should add two tables: + +```sql +CREATE TABLE agent_org_run_event_cursors ( + org_run_id TEXT PRIMARY KEY, + current_revision INTEGER NOT NULL DEFAULT 0, + updated_at TEXT NOT NULL, + FOREIGN KEY(org_run_id) REFERENCES agent_org_runs(id) ON DELETE CASCADE +); + +CREATE TABLE agent_org_run_events ( + org_run_id TEXT NOT NULL, + revision INTEGER NOT NULL, + event_id TEXT NOT NULL UNIQUE, + kind TEXT NOT NULL, + entity_type TEXT NOT NULL, + entity_id TEXT, + causation_id TEXT, + payload_json TEXT NOT NULL, + created_at TEXT NOT NULL, + PRIMARY KEY(org_run_id, revision), + FOREIGN KEY(org_run_id) REFERENCES agent_org_runs(id) ON DELETE CASCADE +); +``` + +Mutation flow: + +1. Increment the Run's `current_revision` inside the active business transaction. +2. Append an Event using the new revision. +3. Update Task, Inbox, Approval, or other canonical business state. +4. Commit once. +5. Publish a lightweight in-process notification only after commit succeeds. + +`causation_id` groups the Events produced by one cause, such as Task completion producing `task.updated`, dependency readiness, and `inbox.inserted`. Every Event still has its own unique `event_id`. + +## Initial Event kinds + +| Event | Purpose | Bounded payload | +| ------------------------ | --------------------------------------------------------- | ------------------------------------------------------- | +| `run.status_changed` | Paused, Running, Completed, and other Run transitions | Previous status, new status, reason code | +| `task.created` | A Task appears | Task id, owner, status, dependency ids | +| `task.updated` | Owner, status, dependency, or other tracked fields change | Changed fields and a small summary | +| `task.deleted` | A Task is deleted | Task id and deletion reason | +| `inbox.inserted` | A member has new durable input | Inbox id, recipient, and kind; never the full long body | +| `inbox.read` | Input was successfully consumed | Inbox id and recipient | +| `approval.changed` | Plan waits, is approved, or receives requested changes | Approval id, revision id, and status | +| `member.runtime_changed` | A worker starts, becomes Idle, or fails | Member id, Session id, and status | +| `run.resync_required` | The real-time queue overflowed | Current highest revision | + +Watchdog scans, polling ticks, duplicate Wake requests, and Coalesced Wake requests do not generate business Events. Only a durable state change produces an Event, preventing millions of meaningless records. + +## Snapshot + Replay protocol + +### Initial open + +1. The frontend loads a compact Run Snapshot with `snapshotRevision = 40`. +2. It subscribes to live Events for that Run. +3. It requests Replay with `afterRevision=40`. +4. Replay and real-time delivery may overlap; the frontend deduplicates by `(runId, revision)`. +5. The shared Store then applies only revisions 41, 42, 43, and later. + +### Gap detection + +If the frontend has applied 40 but receives 42 first: + +- do not apply 42 immediately; +- request Replay with `afterRevision=40`; +- apply 41 and 42 in order; +- if the service explicitly cannot provide continuous history, reload a Snapshot. + +### Reconnection + +The frontend retains the last applied revision in memory. When the window regains focus or the subscription reconnects, it replays the missing range instead of immediately reloading the entire Task board. + +### Pagination and payload boundaries + +- Every Replay page has both a row limit and a total-byte limit. +- Responses return `nextRevision`, `hasMore`, and `currentRevision`. +- Each Event enforces strict character, byte, and array-length limits. +- Large content is represented by a detail handle and is never copied into the Event. + +## Frontend architecture + +```mermaid +flowchart TD + SNAP["Compact Snapshot @ revision R"] --> STORE["One shared Store per Run"] + LIVE["Real-time Event"] --> ORDER["Deduplicate + order + detect gaps"] + REPLAY["Bounded Replay"] --> ORDER + ORDER --> STORE + STORE --> ROOT["Root Group Chat"] + STORE --> A["Alice Member View"] + STORE --> B["Bob Member View"] + STORE --> K["Kanban / Monitor"] +``` + +The shared Store contains public Run facts. Caller identity such as `currentMemberId` is a lightweight projection on top of the Store; each Session must not copy an entire independent Run state. + +The reducer must: + +- no-op when the same revision is delivered twice; +- discard an older revision delivered late; +- pause application and Replay when an intermediate revision is missing; +- preserve the connection and emit a compatibility warning for an unknown Event kind instead of clearing the board; +- isolate reducer failure to the affected Run rather than disrupting other Runs. + +## Phased implementation + +### PR A — Event contract and database foundation + +- Add tables, shared initialization, and deletion cascade. +- Define a typed Event envelope, payload limits, and stable serialization. +- Add a helper that allocates a revision and appends an Event inside the active business transaction. +- Integrate Task, Run, and Approval mutation paths first without changing UI behavior. + +### PR B — Replay and real-time bridge + +- Return `snapshotRevision` with the compact Snapshot. +- Add a bounded keyset Replay API. +- Broadcast in-process only after commit; emit `resync_required` under backpressure. +- Add authority checks, reconnect handling, and per-Run error isolation. + +### PR C — Shared frontend Event Store + +- Implement the Snapshot + subscribe + Replay handshake. +- Add an idempotent reducer, gap recovery, and per-Run isolation. +- Share one Store across Root, Member, and Kanban views. +- Keep the current slow Snapshot Poller as verification and fallback. + +### PR D — Gradually reduce polling + +- Compare Event Store output with direct Snapshot output using consistency metrics. +- During the stability period, reduce polling from 2.5 seconds to a 30–60 second safety check. +- Remove high-frequency polling only after loss, reordering, reconnection, and pressure metrics are stable. + +## Required tests + +| Scenario | Required proof | +| ------------------------ | ----------------------------------------------------------------------- | +| Task transaction fails | Neither Task mutation nor Event persists. | +| Event insertion fails | The canonical mutation rolls back with it. | +| 20 concurrent writes | Every Run revision is unique, continuous, and replayable in order. | +| Duplicate delivery | The reducer applies the revision exactly once. | +| Out-of-order 42 → 41 | The client fills the gap before applying the sequence. | +| Disconnect and reconnect | The client resumes from the last revision without clearing the board. | +| Replay pagination | Row and byte limits are both enforced. | +| Large Plan / TaskOutput | Event omits the full body and the detail API retrieves it. | +| Pause / Resume | Status order is correct and Paused never calls the model. | +| Run deletion | Snapshot-owned state, Events, Inbox, and Plan history cascade together. | +| App restart | Revision and unconsumed Events persist and recovery remains continuous. | +| Real-time queue overflow | A resync signal is emitted and memory does not grow without bound. | +| Multiple active Runs | A malformed Event in one Run does not affect any other Run. | + +## Acceptance gates + +- Every canonical mutation and its Event commit atomically. +- Applying Snapshot + Replay produces the same UI state as loading a fresh Snapshot directly. +- A Run has no duplicated, decreasing, or permanently missing revision. +- Event storage contains no unbounded long text or binary content. +- Event-subscription failure cannot block Coordinator, worker, Watchdog, or Task writes. +- Low-frequency Snapshot verification remains until the Event path is proven stable with production evidence. + +## Expected user experience + +Users do not need to know the term “Revision Event.” They should notice only that: + +- Task completion appears in the UI sooner; +- Root, member, and Kanban views no longer disagree briefly; +- pause, resume, approval, and finality changes do not wait for the next full poll; +- reconnecting or restarting the App resumes from an exact point; +- keeping the UI synchronized no longer requires repeatedly reading large TaskOutput or Plan bodies. + +This is the relationship between #373 and the final architecture: **#373 first makes every read and write safe and consistent. The Revision Event follow-up then makes the delivery of those committed changes reliable, ordered, and incremental.** diff --git a/docs/architecture-audit-2026-07-24/AgentOrgRedTeamPort.md b/docs/architecture-audit-2026-07-24/AgentOrgRedTeamPort.md new file mode 100644 index 000000000..3aefc0708 --- /dev/null +++ b/docs/architecture-audit-2026-07-24/AgentOrgRedTeamPort.md @@ -0,0 +1,187 @@ +# Agent Org Red-Team Hardening Port — Architecture Audit + +## Scope + +- Target branch: `develop` +- Source: PR #424, commit `df158006ad76271a0cc60440dc4c611e54774f8a` +- Port strategy: preserve the current split Inbox, Plan Approval, Task Store, + Session Message, Org Tasks, Watchdog, and Turn Executor modules. No deleted + monolith is restored. +- Acceptance boundary: durable recovery semantics, run/member identity, + dispatch revalidation, bounded resources, schema compatibility, and the + corresponding frontend projections. + +## Red-Team Finding Mapping + +| # | Finding / contract | Port verdict | Implementation evidence | Test evidence | +| --- | ----------------------------------------------------------------------- | ------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------- | +| 1 | Coordinator-only Inbox repair with `inspect`, `cancel`, and `supersede` | Ported | `agent_org/inbox_repair.rs`; tool policy, assembly, metadata, and tool-name registration | Inbox repair unit tests cover role rejection, inspection, cancel, and supersede | +| 2 | Preserve original Inbox evidence while resolving delivery | Ported | `agent_inbox_delivery_resolutions` is append-only relative to Inbox rows; no synthetic `read_at` mutation | Inbox store tests assert unread evidence remains and resolution is returned | +| 3 | Resolved delivery must not block unread wake/finality | Ported | Inbox unread queries, wake selection, lifecycle resume, Run View, and finality exclude rows with a delivery resolution | Inbox, run finality, wake, and group-chat settlement tests | +| 4 | Bind turn Intent to exactly one Run without guessing legacy ownership | Ported | Nullable `session_turn_intents.org_run_id`; ordinary turns remain `NULL`; one-time `NULL` binding is allowed; reassignment is rejected | Fresh-schema, upgrade-path, NULL backfill, and cross-Run rejection tests | +| 5 | Use exact `member_id` plus Run identity instead of shared `agent_id` | Ported | Inbox recipient/sender member fields, task authorization, Run View, and wake/drain queries use `(org_run_id, member_id)` | Shared-agent-id fixtures verify member isolation | +| 6 | Revalidate dispatch-time ownership and eligibility | Ported | Task assignment/requeue and continuation insertion re-read Run, member, task, and eligibility state inside the writer transaction | Task store and orchestration tool tests cover stale/invalid assignment | +| 7 | Make Wake budget consumption atomic with recovery writes | Ported | Watchdog re-inspection, budget insertion, and recovery notice insertion share one SQLite writer transaction | Watchdog budget/fingerprint tests, including exhausted budget | +| 8 | Use stable, bounded recovery fingerprints | Ported | Typed facts, sorted task/Inbox fingerprints, work revision, bounded reason and identifier previews | Watchdog fingerprint and recovery-plan tests | +| 9 | Fail closed on corrupt historical task/Inbox data | Ported | Operational task projection validates identifiers, timestamps, dependencies, metadata, and payloads; unread scans reject corrupt payloads | Corrupt task and historical Inbox integration tests | +| 10 | Enforce input and projection resource limits | Ported | Central Agent Org payload limits; task count, identifier, JSON, page-row, page-byte, and preview caps | Boundary tests for task payloads, listing, history, and Inbox snapshots | +| 11 | Protect managed Plan paths and repair durable artifacts | Ported | Component-wise lexical validation, canonical-root checks, race-bounded install, startup artifact reconciliation | Plan artifact traversal/symlink/reconciliation tests | +| 12 | Bound Run View caching and reject stale async generations | Ported | Retention eviction, retired-generation tombstone, request ordering, shared polling, and bounded bootstrap join | Run View store tests cover abandoned render, eviction, late IPC, and shared polling | +| 13 | Keep Plan Approval detail cache bounded | Equivalent retained | Current `develop` already has both 64-entry and 8 MiB LRU bounds, stronger than the source entry-only bound | Existing detail-cache tests retained | +| 14 | Preserve complete history and shared member activity semantics | Ported | Cursor pagination has bounded gap frontiers; delivery resolutions merge monotonically; both member selectors use the shared exact-unread activity predicate | History overlap/gap tests and shared member-activity tests | +| 15 | Deduplicate mention targets and fix WebKit inline `@` parsing | Ported | Shared target-key merge and `getInlineMentionQuery` normalize WebKit keydown/input ordering | Mention option and inline mention query tests | + +## Ten-Layer Audit + +### Layer 1 — Compilation Correctness + +The split modules compile as the production graph rather than as isolated test +copies. The delivery gate includes workspace Rust check/clippy, TypeScript +typecheck/lint/circular checks, and changed-file formatting. Final command +results are recorded in the Draft PR because they must describe the exact +pushed commit. + +Verdict: **fix** — cherry-pick conflicts were resolved in the active modular +files; no parallel legacy file is needed for compilation. + +### Layer 2 — Dead Code and Structural Deduplication + +Production call chains were traced from: + +1. `org_inbox_repair` registration → policy → tool assembly → executor → Inbox + store. +2. Tauri message send/wake → turn Intent persistence → Turn Executor → Inbox + drain. +3. lifecycle/watchdog tick → coherent inspection → recovery plan → transactional + revalidation/write. +4. Run View/group-chat commands → bounded store projections → shared frontend + caches and selectors. + +The old monolithic Inbox, Plan Approval, Task Store, Session Message, and Org +Tasks files remain deleted. Recovery uses the batch assignment and transactional +writer paths; the redundant single-task recovery helper was removed. + +Verdict: **fix**, with no unwired compatibility abstraction introduced. + +### Layer 3 — Naming Consistency + +| Term | Meaning in this port | Verdict | +| --------------------- | ------------------------------------------------------------------------------------------- | ------- | +| `delivery_resolution` | Durable `cancelled`/`superseded` disposition of an Inbox delivery; never equivalent to read | Keep | +| `org_run_id` | Durable ownership boundary for a turn Intent or Agent Org row | Keep | +| `member_id` | Materialized member identity inside one Run; not interchangeable with reusable `agent_id` | Keep | +| `work_revision` | Monotonic task-board observation used for stale recovery/finality rejection | Keep | +| `fingerprint` | Stable bounded digest input for one recovery fact set | Keep | +| `corrupt` | Explicit count/state that blocks operational projection and finality | Keep | + +Rust uses snake_case wire fields and TypeScript exposes the established +camelCase Tauri contract. Historical comments that implied `agent_id` was a +unique member identity were not copied into the modular implementation. + +Verdict: **keep with reason**. + +### Layer 4 — Semantic Overloading + +The critical overloaded words were audited across Inbox, tasks, turns, and UI: + +- `read` means a recipient consumed a delivery. `resolved` means a coordinator + intentionally cancelled or replaced an undeliverable delivery. They remain + distinct columns and behaviors. +- `session` is execution identity; `member` is Run roster identity; `agent` is a + reusable definition identity. Authorization and delivery use the member + dimension. +- `completed` applies independently to task, session turn, and Run. Run finality + is derived from durable task/Inbox/session facts and does not reuse a UI + session status. + +Verdict: **fix** — the port restores the missing distinctions instead of +mapping them to existing overloaded fields. + +### Layer 5 — Default Branch Analysis + +- Unknown delivery-resolution strings return an error instead of falling + through to `cancelled`. +- Unknown/corrupt task status, timestamp, identifiers, dependencies, and payload + shapes fail closed instead of defaulting to pending/empty. +- A legacy turn Intent defaults to `org_run_id = NULL`; no Run is inferred. +- A missing or permanently unavailable Inbox recipient remains unresolved + until an explicit coordinator action; it is not silently marked read. +- Future tool callers are denied `org_inbox_repair` unless the current context + is the coordinator for the exact Run. + +Verdict: **fix**. + +### Layer 6 — Cross-Domain Concept Leakage + +Persistence owns storage compatibility and typed records. Coordination modules +own Run/member/task/Inbox semantics. Tauri command modules only project those +facts, and frontend utilities only decide presentation behavior from typed +fields. Plan filesystem validation remains in the Plan Approval artifact +module, not in generic path helpers. + +Verdict: **keep with reason** — Agent Org-specific concepts cross a shared +boundary only through explicit fields (`org_run_id`, member ids, `corrupt`, +`delivery_resolution`). + +### Layer 7 — New Developer Confusion Test + +The new names state their postconditions: `ResolveInboxDeliveryParams`, +`delivery_resolution_for_inbox`, `list_operational`, +`ensure_task_rows_safe_for_operational_projection`, +`coordinator_repair_*_fingerprint`, `isAgentOrgMemberEmpty`, and +`getInlineMentionQuery`. Comments explain why durable unread and bounded recent +activity are both required. + +Verdict: **keep with reason**. + +### Layer 8 — Wire Protocol and Serialization + +| Boundary | Added/changed fields | Safety check | +| ------------------------- | -------------------------------------------------- | ------------------------------------------------------------------------------------------- | +| Tool schema | `org_inbox_repair` action and replacement evidence | Explicit enum-like actions; coordinator/run validation occurs before storage | +| Turn Intent SQLite record | nullable `org_run_id` | Existing rows and ordinary turns serialize as `NULL`; conflicting non-NULL ownership errors | +| Run View Tauri payload | task `corrupt`; Inbox `deliveryResolution` | Rust snake_case is projected through the existing camelCase TypeScript wrapper | +| Group-chat history | `deliveryResolution` | Merge is monotonic so a refresh cannot erase a known resolution | +| Recovery notice | bounded typed facts/fingerprints | No unbounded task payload is copied into the wake reason | + +No external provider request schema is changed by this port. Tool UI metadata +and tool-name tests pin registration symmetry. + +Verdict: **fix**. + +### Layer 9 — Initialization Parity + +| Entry point | Inbox resolution schema | turn Intent upgrade | Plan artifact repair | expired intervention cleanup | watchdog recovery | +| ---------------------------- | ------------------------: | --------------------------------: | --------------------------: | ---------------------------: | -------------------------: | +| Production app startup | Yes | Yes | Yes | Yes | Yes | +| Fresh test database | Yes | Yes | Exercised by artifact tests | Exercised | Exercised | +| Existing-schema upgrade test | Existing Inbox init path | Yes, nullable and non-inferential | N/A | N/A | N/A | +| Rust runtime E2E | Production initialization | Production initialization | Production launch path | Production launch path | Production wake/drain path | +| Rendered Tauri E2E | Production initialization | Production send path | Production launch path | Production lifecycle path | Production wake/drain path | + +Verdict: **fix** — startup repair and cleanup are wired into lifecycle +initialization, not a test-only helper. + +### Layer 10 — Resolver Symmetry + +| Resolved fact | Run | Member | Task / Inbox row | Durable fallback | Default | +| ------------------------- | -------------------------------: | ----------------------------------: | --------------------------------------------: | ------------------------------------: | --------------------------------------- | +| Inbox delivery authority | Required | Required for materialized recipient | Exact Inbox id | SQLite row | None | +| task assignment authority | Required | Required owner/eligible member | Exact task id | SQLite snapshot in writer transaction | None | +| turn Intent ownership | Required only for Agent Org turn | Derived from runtime context | Exact Intent id | Existing non-NULL `org_run_id` | `NULL`, never guessed | +| watchdog recovery target | Required | Exact member/coordinator | Task + Inbox fingerprints | Re-inspection in writer transaction | Abort stale action | +| Run View member activity | Required | Exact member | task counts + bounded activity + exact unread | Tauri Run View | Empty only when every dimension is zero | + +Verdict: **fix** — every mutating resolver uses the same Run/member/row +dimensions; stale or asymmetric evidence aborts rather than falling back to +`agent_id`. + +## Audit Result + +- Fix: 7 layers +- Keep with reason: 3 layers +- Abstract/defer: 0 layers +- Open architecture findings: 0 + +The acceptance test report in the Draft PR remains the authority for final +PASS/BLOCKED status on the exact pushed commit. diff --git a/docs/frontend-ui-audit-2026-07-12/AgentOrgCliMemberParity.md b/docs/frontend-ui-audit-2026-07-12/AgentOrgCliMemberParity.md index 4c4766104..2daf1d32f 100644 --- a/docs/frontend-ui-audit-2026-07-12/AgentOrgCliMemberParity.md +++ b/docs/frontend-ui-audit-2026-07-12/AgentOrgCliMemberParity.md @@ -1,37 +1,37 @@ # Agent Org CLI Member UI Boundary Audit -> 日期：2026-07-12 -> 范围：本次修改的 Agent Org 设置、成员选择器和对应 rendered E2E。 -> 方法说明：仓库 `AGENTS.md` 指向的 `~/.orgii/skills/frontend-ui-audit/SKILL.md` 在当前环境不存在；以下按仓库规定的逐元素表格格式进行人工替代审计。 +- Date: 2026-07-12 +- Scope: Agent Org settings, member selectors, and their rendered E2E coverage +- Method note: The `frontend-ui-audit` skill referenced by `AGENTS.md` is not present in the current environment. This report does not claim that the missing skill was executed; it applies the repository's established per-element audit table manually. ## Verdict summary -| Verdict | Count | -| ----------------- | ----: | -| fix（本次已完成） | 4 | -| keep with reason | 5 | -| abstract later | 1 | -| open finding | 0 | +| Verdict | Count | +| ---------------- | ----: | +| Fix, completed | 4 | +| Keep with reason | 5 | +| Abstract later | 1 | +| Open finding | 0 | ## Audit table -| Line | Element | Verdict | Reason | Suggested change | -| --------------------------------------- | ------------------------------------- | ---------------- | ------------------------------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------- | -| `SessionCreatorOrgMembersPanel.tsx:403` | Agent Org member picker | fix（已完成） | Session Creator 原先复用通用 picker，会展示当前后端无法运行的 CLI member。 | 传入 `hideCliAgents`，保留 Rust built-in/custom 和现有视觉结构。 | -| `AgentTeamWizard.tsx:88` | Agent Org 新建/编辑 wizard options | fix（已完成） | 设置页此前显式拉取并注入 installed CLI agents，界面能力声明高于 runtime。 | 只使用 Rust-native built-in/custom definitions。 | -| `OrgDetailView.tsx:101` | 旧 Org 详情编辑 options | fix（已完成） | 详情页也必须与 wizard 使用相同的 capability boundary。 | 使用相同的 Rust-only `buildAgentOptions`。 | -| `AgentOrgs/index.tsx:124` | CLI agent fetching state/effect | fix（已完成） | 数据仅服务于已禁用的 Agent Org CLI picker，继续请求会造成死状态和误导。 | 删除 `cliAgents` state、fetch/refresh effect 和 prop plumbing。 | -| `DispatchCategoryPalette/index.tsx:99` | `hideCliAgents` prop | keep with reason | 共享 picker 在普通会话中仍要显示 CLI；不能全局删除 CLI category。显式 capability flag 能把限制控制在 Agent Org surface。 | 保持默认 `false`，只在 Agent Org member picker 传 `true`。 | -| `DispatchCategoryDropdown.tsx:125` | dropdown parity | keep with reason | Dropdown 和 modal palette 是两个现有渲染入口；只改一个会产生响应式/入口差异。 | 保持 prop 透传和默认值一致。 | -| `useDispatchCategoryOptions.tsx:340` | option filtering | keep with reason | hook 版本仍被其他 picker 使用，需要与 component 版本一致过滤 CLI option/header。 | 保持 `hideCliAgents` 同时影响 options 和 group header。 | -| `config.ts:22` | `buildAgentOptions` Rust-only builder | keep with reason | 这是 Agent Org 专用 builder，不是全局 Agent picker；在这里收窄不会影响普通 CLI session。 | 注释继续明确 Rust-native 范围。 | -| `agentOrgUiDriver.mjs:988` | rendered negative assertion | keep with reason | 仅检查保存失败不足以证明 UI 没撒谎；生产 picker 必须在 DOM 层不出现 `cli-*` option。 | 在可用桌面 E2E 环境运行对应 rendered spec。 | -| Palette component + hook | duplicated option composition | abstract later | 两套相近组合逻辑是既有结构，本次必须双改才能保证入口一致；继续扩展 flags 会增加漂移风险。 | 独立 PR 抽出纯 `buildDispatchCategoryGroups` selector，并让 modal/dropdown 共用。 | +| Line / surface | Element | Verdict | Reason | Suggested change | +| ----------------------------------- | ----------------------------- | ---------------- | ------------------------------------------------------------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------- | +| `SessionCreatorOrgMembersPanel.tsx` | Agent Org member picker | fix | The Session Creator reused a generic picker and exposed CLI members that the backend could not run in Agent Org. | Pass `hideCliAgents` while preserving Rust built-in/custom options and the existing visual structure. | +| `AgentTeamWizard.tsx` | Create/edit wizard options | fix | Settings explicitly fetched and injected installed CLI agents, presenting more capability than runtime supported. | Use Rust-native built-in/custom definitions only. | +| `OrgDetailView.tsx` | Historical Org edit options | fix | Detail editing must enforce the same runtime capability boundary as the wizard. | Use the same Rust-only `buildAgentOptions`. | +| `AgentOrgs/index.tsx` | CLI fetching state/effect | fix | The data existed only for the now-disabled Agent Org CLI picker and continued to create dead state and misleading refresh behavior. | Remove CLI state, fetch/refresh effect, and prop plumbing from this surface. | +| `DispatchCategoryPalette` | `hideCliAgents` prop | keep with reason | Ordinary Sessions must still display CLI choices. An explicit capability flag limits the restriction to Agent Org without removing CLI globally. | Keep the default `false` and pass `true` only in Agent Org member selection. | +| `DispatchCategoryDropdown` | Dropdown parity | keep with reason | Dropdown and modal palette are existing responsive entry points; changing only one would create inconsistent options. | Keep prop forwarding and defaults aligned. | +| `useDispatchCategoryOptions` | Option filtering | keep with reason | Other pickers use the hook path, so CLI options and the CLI group header must be filtered consistently. | Keep `hideCliAgents` applied to both options and group headers. | +| `config.ts` | Rust-only `buildAgentOptions` | keep with reason | This builder is specific to Agent Org, not the global agent picker, so narrowing it does not affect ordinary CLI Sessions. | Keep the Rust-native scope explicit in comments. | +| `agentOrgUiDriver.mjs` | Rendered negative assertion | keep with reason | A backend save rejection is insufficient proof that UI capability is honest. The production picker must not render `cli-*` choices. | Retain the rendered negative assertion. | +| Palette component + hook | Duplicated option composition | abstract later | Two existing composition paths must currently receive the flag for parity; more flags would increase drift risk. | In a separate PR, extract a pure `buildDispatchCategoryGroups` selector used by modal and dropdown. | ## UI behavior conclusion -- Agent Org 新建、编辑和 Session Creator member override 不再展示 CLI agents。 -- 普通 CLI session 和 CLI-only picker 不受影响，因为 `hideCliAgents` 默认关闭。 -- 旧 CLI Agent Org 仍可加载并删除；保存或启动时由后端给出明确 unsupported validation。 -- 没有新增 arbitrary Tailwind value、视觉 token、交互控件或 a11y pattern；本次 UI diff 是能力过滤和无用数据流删除。 -- TypeScript、目标 ESLint 和修改 `.mjs` 的语法检查通过；完整 rendered desktop E2E 待具备运行环境后执行。 +- Agent Org create, edit, and Session Creator member overrides do not display CLI agents. +- Ordinary CLI Sessions and CLI-only pickers remain unchanged because `hideCliAgents` defaults to false. +- Historical CLI Agent Org definitions remain readable and deletable; save and launch return explicit unsupported validation. +- No arbitrary Tailwind value, visual token, interaction primitive, or accessibility pattern was introduced. The UI diff filters unsupported capability and removes dead data flow. +- TypeScript, ESLint, and rendered Settings E2E pass; the final rendered suite includes three Settings scenarios. diff --git a/docs/frontend-ui-audit-2026-07-13/AgentOrgPlannerApproval.md b/docs/frontend-ui-audit-2026-07-13/AgentOrgPlannerApproval.md index 38759080e..0bd82de89 100644 --- a/docs/frontend-ui-audit-2026-07-13/AgentOrgPlannerApproval.md +++ b/docs/frontend-ui-audit-2026-07-13/AgentOrgPlannerApproval.md @@ -1,45 +1,45 @@ -# Agent Org Planner Approval 前端 UI 审计 - -日期：2026-07-13 -范围：Agent Org 创建/编辑页的审批策略、Group chat 审批卡、run phase 与任务展示。 -说明：仓库路由要求的 `frontend-ui-audit` skill 在 workspace 和用户目录都不存在，因此本报告按 AGENTS.md 要求手工执行同等检查，未伪称运行了缺失 skill。 - -## 审计表 - -| Line | Element | Verdict | Reason | Suggested change | -| -------------------------------------- | ---------------------- | ------- | -------------------------------------------------------------------------------------------- | ---------------- | -| `PlanApprovalPolicySelector.tsx:17` | 审批策略选择器 | keep | 复用项目 `Select` 与现有表单 section；三种策略用 typed union，不用自由文本。 | 无。 | -| `AgentTeamFormSections.tsx:266` | 创建 Agent Org | fixed | 创建时可选择 Coordinator / User / Automatic，默认 Coordinator。 | 无。 | -| `OrgDetailView.tsx:85-199` | 编辑 Agent Org | fixed | 编辑和创建使用同一字段与 selector，避免只在 wizard 生效。 | 无。 | -| `AgentOrgOverviewPanel.tsx:287` | 用户审批区域 | keep | 只有 policy=user 且存在 pending approval 才显示；不把 Coordinator 审批重复展示给用户。 | 无。 | -| `AgentOrgPlanApprovalCard.tsx:21` | 审批卡 | keep | 使用项目 Button、Textarea、Markdown；显示计划标题与成员 display name，不暴露内部 member id。 | 无。 | -| `AgentOrgPlanApprovalCard.tsx:72` | 卡片状态标识 | keep | 稳定 test id 与 approval id 方便 E2E，不影响视觉。 | 无。 | -| `AgentOrgPlanApprovalCard.tsx:97,112` | 编辑与反馈输入 | keep | 有 `aria-label`、disabled 和 loading；长内容限制高度并可滚动。 | 无。 | -| `AgentOrgPlanApprovalCard.tsx:126` | 错误反馈 | keep | command 失败在卡内以 `role=alert` 显示，不会静默吞错。 | 无。 | -| `AgentOrgPlanApprovalCard.tsx:142-191` | 要求修改 / 编辑 / 批准 | fixed | 三条用户动作各自明确；空反馈不能发送、空计划不能批准、提交中防重复点击。 | 无。 | -| `org_tasks.rs:118` + UI badge | `AwaitingPlanApproval` | fixed | 等审批时 Run 仍是 Running，界面显示“等待计划审批”，不会误显示 Paused 或 Done。 | 无。 | -| `sessions.json` / `integrations.json` | 中英文文案 | keep | 新标签、提示、策略说明同时提供英文和中文。 | 无。 | -| Group chat approval E2E | rendered interaction | fixed | E2E 真实打开 overview，并点击要求修改、输入反馈、重新提交、编辑、批准，再检查 task 解锁。 | 无。 | - -## 设计一致性 +# Agent Org Planner Approval — Frontend UI Audit + +- Date: 2026-07-13 +- Scope: Approval policy in Agent Org create/edit, Group Chat approval card, Run phase, and Task presentation. +- Method note: The routed `frontend-ui-audit` skill is absent from both workspace and user paths. This report performs the equivalent `AGENTS.md` checks manually and does not claim to have executed the missing skill. + +## Audit table + +| Line / surface | Element | Verdict | Reason | Suggested change | +| -------------------------------- | -------------------------------- | ------- | ----------------------------------------------------------------------------------------------------------------------------------------- | ---------------- | +| `PlanApprovalPolicySelector.tsx` | Approval policy selector | keep | Reuses the project `Select` and existing form section; the three policies use a typed union rather than free text. | None. | +| `AgentTeamFormSections.tsx` | Create Agent Org | fixed | Creation supports Coordinator, User, and Automatic, defaulting to Coordinator. | None. | +| `OrgDetailView.tsx` | Edit Agent Org | fixed | Create and edit share the same field and selector, avoiding wizard-only behavior. | None. | +| `AgentOrgOverviewPanel.tsx` | User approval region | keep | Appears only for `policy=user` with a pending approval; Coordinator-owned approvals are not duplicated to the user. | None. | +| `AgentOrgPlanApprovalCard.tsx` | Approval card | keep | Reuses project Button, Textarea, and Markdown; shows Plan title and member display name rather than internal member id. | None. | +| `AgentOrgPlanApprovalCard.tsx` | Card state identity | keep | Stable test id and approval id support E2E without changing visual presentation. | None. | +| `AgentOrgPlanApprovalCard.tsx` | Edit and feedback input | keep | Includes `aria-label`, disabled, and loading states; long content is height-bounded and scrollable. | None. | +| `AgentOrgPlanApprovalCard.tsx` | Error feedback | keep | Command failure appears in the card with `role="alert"` and is not silently swallowed. | None. | +| `AgentOrgPlanApprovalCard.tsx` | Request changes / edit / approve | fixed | Each action is explicit; empty feedback cannot submit, an empty Plan cannot be approved, and submitting disables duplicate clicks. | None. | +| Run projection + UI badge | `AwaitingPlanApproval` | fixed | The Run remains Running while the UI says “Awaiting plan approval”; it is not mislabeled Paused or Done. | None. | +| Locale files | English and Chinese product copy | keep | Product labels, guidance, and policy descriptions remain available in both supported locales. This audit document itself is English-only. | None. | +| Group Chat approval E2E | Rendered interaction | fixed | E2E opens the real overview, requests changes, enters feedback, resubmits, edits, approves, and verifies downstream release. | None. | + +## Design consistency ```mermaid flowchart LR - S["Settings\n选择审批策略"] --> R["Run 启动时固定策略"] - R -->|User| G["Group chat 审批卡"] - G --> X["要求修改"] - G --> E["编辑后批准"] - G --> A["直接批准"] - X --> W["Planner 收到一次反馈 Wake"] - E --> N["下游任务解锁"] + S["Settings\nselect approval policy"] --> R["Policy fixed at Run launch"] + R -->|"User"| G["Approval card in Group Chat"] + G --> X["Request changes"] + G --> E["Edit and approve"] + G --> A["Approve directly"] + X --> W["Planner receives one feedback Wake"] + E --> N["Downstream work unlocks"] A --> N ``` -## 统计 +## Verdict summary -- fix：6 -- keep with reason：6 -- 需要抽象：0 -- 未解决的阻塞级 UI 问题：0 +- Fix: 6 +- Keep with reason: 6 +- New abstraction required: 0 +- Unresolved blocking UI finding: 0 -没有新增任意色值、任意 spacing 或另一套按钮/输入组件。审批卡有明确 loading、disabled、错误与无内容保护；对用户而言，它是 Group chat 任务概览的一部分，不会跳到 Planner session 要求用户手动切换 mode。 +No arbitrary color, spacing system, or second button/input implementation was added. The card has explicit loading, disabled, error, and empty-content protection. For the user it is part of the Group Chat work overview; it does not force a manual switch into the Planner Session. diff --git a/docs/frontend-ui-audit-2026-07-14/AgentOrgFinalAudit.md b/docs/frontend-ui-audit-2026-07-14/AgentOrgFinalAudit.md index 4a3b753db..f94a11a20 100644 --- a/docs/frontend-ui-audit-2026-07-14/AgentOrgFinalAudit.md +++ b/docs/frontend-ui-audit-2026-07-14/AgentOrgFinalAudit.md @@ -1,44 +1,49 @@ -# Agent Org #272 最终前端 UI 审计 - -日期：2026-07-14 -范围：本分支修改的 Agent Org Settings、Group chat overview、Plan approval、Task block、Kanban、Monitor 与中英文文案。 -说明：仓库指定的 `frontend-ui-audit` skill 在 workspace 和用户目录均不存在，因此按 AGENTS.md 的同等检查项手工审计，没有伪称运行缺失 skill。 - -## 审计表 - -| Line | Element | Verdict | Reason | Suggested change | -| --------------------------------------------- | ------------------------ | ---------------- | ---------------------------------------------------------------------------------------------------- | ---------------- | -| `PlanApprovalPolicySelector.tsx` | 审批策略 | keep | 使用现有 Select，typed 三选一；创建和编辑共用，不存在两套 UI。 | 无。 | -| `AgentOrgPlanApprovalCard.tsx` | 用户审批卡 | keep | 有 loading、disabled、空输入保护、`aria-label`、卡内错误和明确的批准/编辑/退回动作。 | 无。 | -| `AgentOrgOverviewPanel.tsx` | Run 总览与 phase | fixed | Waiting for approval、Finalizing、Paused、Waiting for work 不再互相冒充；用户能看出系统为什么安静。 | 无。 | -| `OrgTaskBlock/index.tsx` | Task tool 结果 | fixed | 可预期的权限/依赖纠正使用结构化 guidance，不再渲染成吓人的红色系统失败卡。 | 无。 | -| `TodoKanban.tsx` + `orgTaskOutcome.ts` | 看板状态 | fixed | Completed/Cancelled/Open 从真实 Task event 投影；同一任务后续事件覆盖旧状态，避免历史卡片永久 Open。 | 无。 | -| `AgentEventBubbles.tsx` / `MessageViewer.tsx` | 成员事件 | keep | 关键协作状态可读，内部恢复细节不作为普通失败轰炸用户。 | 无。 | -| Agent Org 创建/编辑成员选择 | CLI member | fixed | 未完成的 CLI transport 从可运行选择中排除；旧定义仍能打开并删除。 | 无。 | -| `sessions.json` / `integrations.json` | 中英文文案 | keep | 新状态、审批、错误和策略说明中英文同步。 | 无。 | -| Group chat E2E | rendered production path | keep | Debug helper 只搭建前置数据；审批、暂停、恢复和任务展示通过真实 UI 操作验证。 | 无。 | -| arbitrary styles / accessibility sweep | 变更 TSX | keep with reason | 未新增另一套按钮、输入或选择器；没有发现任意色值/spacing 扩散；交互控件具备可访问名称和禁用态。 | 无。 | - -## 用户现在看到的流程 +# Agent Org #272 Final Frontend UI Audit + +- Date: 2026-07-14, updated with final verification on 2026-07-18 +- Scope: Agent Org Settings, Group Chat overview and history, Plan approval, Task blocks, Kanban, Monitor, member navigation, mention input, and localized product copy changed by this branch. +- Method note: The repository's `frontend-ui-audit` skill is absent from workspace and user paths. The equivalent `AGENTS.md` checks were performed manually without claiming the missing skill was executed. + +## Audit table + +| Line / surface | Element | Verdict | Reason | Suggested change | +| --------------------------------------- | ---------------------------------- | ---------------- | --------------------------------------------------------------------------------------------------------------------------------------------- | ---------------- | +| `PlanApprovalPolicySelector.tsx` | Approval policy | keep | Uses the existing Select and one typed three-option field shared by create and edit. | None. | +| `AgentOrgPlanApprovalCard.tsx` | User approval card | keep | Provides loading, disabled, empty-input protection, accessible labels, inline errors, and distinct approve/edit/request-changes actions. | None. | +| `AgentOrgOverviewPanel.tsx` | Run overview and phase | fixed | Awaiting approval, Finalizing, Paused, and Waiting for work no longer impersonate one another. | None. | +| Group Chat history hooks and pagination | Durable transcript | fixed | Refresh and “Load older” preserve complete durable history while the compact Run View remains bounded. | None. | +| Composer mention path | Member mention | fixed | WebKit trigger parsing and member-option identity allow real keyboard selection of every Agent Org member. | None. | +| `OrgTaskBlock/index.tsx` | Task tool result | fixed | Expected authority or dependency correction appears as structured guidance rather than a frightening red system failure. | None. | +| `TodoKanban.tsx` + `orgTaskOutcome.ts` | Board state | fixed | Open/Completed/Delete state projects from durable typed outcomes; rejected and args-only legacy calls cannot mutate replay state. | None. | +| `AgentEventBubbles.tsx` | Member events | keep | Important collaboration state is readable while internal recovery details do not flood users as ordinary failures. | None. | +| Agent Org create/edit member selection | CLI member | fixed | Incomplete CLI transport is hidden from runnable choices; historical definitions remain readable and deletable. | None. | +| Member switchers | Exact unread state | fixed | A member with an old unread row outside the recent preview remains visible and selectable in Group Chat and WorkStation. | None. | +| Locale files | English and Chinese product copy | keep | Product UI continues to ship matching localized status, approval, error, and policy copy. Audit Markdown is English-only. | None. | +| Rendered Agent Org E2E | Production UI path | keep | Debug helpers seed or inspect only; mention, history, approval, pause/resume, recovery, settings, and Task presentation use the rendered App. | None. | +| Changed TSX sweep | Arbitrary styles and accessibility | keep with reason | No second button/input/selector system, arbitrary color expansion, or new inaccessible control was introduced. | None. | + +## User flow ```mermaid flowchart LR - S["Settings\n配置组织与审批策略"] --> G["Group chat\n查看 Run 总体状态"] - G --> T["Team tasks\n查看 owner、依赖和进度"] - T --> P{"需要计划审批?"} - P -->|是| A["审批卡\n批准 / 编辑 / 要求修改"] - P -->|否| W["成员按真实分配工作"] + S["Settings\nconfigure organization and approval policy"] --> G["Group Chat\nview overall Run state"] + G --> T["Team Tasks\nview owner, dependencies, and progress"] + T --> P{"Plan approval required?"} + P -->|"Yes"| A["Approval card\napprove / edit / request changes"] + P -->|"No"| W["Members execute real assignments"] A --> W - W --> K["Kanban / Monitor\n显示同一 Task 的最新状态"] - K --> F["Finalizing\nCoordinator 收尾"] + W --> K["Kanban / Monitor\nshow the same durable Task state"] + K --> F["Finalizing\nCoordinator completes the Run"] ``` -## 统计 +## Verification + +- Full frontend Vitest: 450 files, 5,181 / 5,181 tests. +- TypeScript typecheck: pass. +- ESLint: pass. +- Rendered Debug App Agent Org E2E: 19 / 19. +- Group Chat includes real keyboard mention selection and 230+ durable history rows across refresh/pagination. -- fix：4 -- keep：5 -- keep with reason：1 -- 需要抽象：0 -- 未解决的阻塞级 UI 问题：0 +## Verdict -结论：**可提交**。界面没有引入新的设计系统分叉；状态展示与后端事实对齐，等待审批、暂停和收尾都能用用户能理解的方式呈现。 +**Implementation and UI verification are complete for the approved scope.** The UI introduces no new design-system fork. Displayed state matches durable backend facts, and approval wait, pause, recovery, finality, historical navigation, and member selection are understandable to users. diff --git a/docs/frontend-ui-audit-2026-07-16/AgentOrgReviewSafetyAudit.md b/docs/frontend-ui-audit-2026-07-16/AgentOrgReviewSafetyAudit.md index 56a2575f8..c904211d9 100644 --- a/docs/frontend-ui-audit-2026-07-16/AgentOrgReviewSafetyAudit.md +++ b/docs/frontend-ui-audit-2026-07-16/AgentOrgReviewSafetyAudit.md @@ -1,72 +1,116 @@ -# Agent Org PR Review — Frontend UI Audit +# Agent Org Review Safety — Frontend UI Audit - Date: 2026-07-16 -- Scope: frontend changes required by the two PR #373 review rounds -- Status: source audit and A-only verification complete, with a documented clean-develop rendered baseline -- Verification date: 2026-07-18 +- Scope: All TypeScript and TSX files changed by the PR #373 review-safety work, including Run View, Plan approval, Group Chat history, mention input, Task events, Kanban projection, member navigation, and their unit tests. + +> The `frontend-ui-audit` skill referenced by the repository's `AGENTS.md` is not present at either `.orgii/skills/frontend-ui-audit/SKILL.md` or `~/.orgii/skills/frontend-ui-audit/SKILL.md`. This report does not claim that the missing skill was executed. It preserves the repository's established frontend-audit table format and applies the equivalent checks manually: design-system reuse, arbitrary Tailwind values, basic accessibility, state/error/empty presentation, duplicated UI patterns, and cross-surface projection consistency. ## Audit table -| Line / surface | Element | Verdict | Reason | Suggested change | -| ----------------------------------------------- | ------------------------------------ | ---------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------- | -| `src/api/tauri/agent/orgTasks.ts` | Compact Run View contract | keep with reason | Task totals, unread totals, Plan summaries, and Inbox previews match the backend's read-only Snapshot. Complete content is loaded through explicit detail or history APIs. | Keep summary-by-default and avoid restoring full payloads to the poll response. | -| `AgentOrgOverviewPanel.tsx` | Exact overview counters | keep with reason | Counters come from backend totals instead of the visible Task or Inbox window. The user is told when only a Task prefix is shown. | None. | -| `agentOrgRunViewStore.ts` | Shared Run poller | keep with reason | Root and member consumers share one per-Run poll path while preserving caller-specific `currentMemberId`. | Preserve one durable Snapshot source per Run. | -| `agentOrgRunViewStore.test.ts` | Snapshot identity | fix | `useSyncExternalStore` requires repeated reads to return the same object until state changes. The regression verifies stable identity before a member subscribes. | Keep the referential-equality assertion. | -| `useAgentOrgPlanApprovalDetail.ts` | Plan summary/detail split | keep with reason | Polling carries immutable revision identity and title only. The full Plan loads once per revision, concurrent loads coalesce, and failures are retryable. | Keep detail loading explicit; cache-cap policy is outside this follow-up. | -| `AgentOrgPlanApprovalCard.tsx` | Plan loading and submission states | keep with reason | Loading, retry, edit, feedback, submit, paused, and stale-revision states remain distinct. Existing `Button`, `Textarea`, and `Markdown` primitives are reused. | None. | -| `useAgentOrgGroupChatHistory.ts` | Durable keyset history | fix | Reload no longer relies on a bounded Run View preview. History pages merge by Inbox id, preserve older rows, retain cursor frontiers across refresh gaps, and expose retry state. | Keep cursor pagination and explicit Load older behavior. Extra cache caps belong to later hardening. | -| `ChatView.tsx` | Load older and retry controls | keep with reason | Existing button and pagination surfaces expose history progress without introducing a parallel navigation pattern. | None. | -| `useGroupChatMergedEvents.ts` | Durable user-message projection | keep with reason | User Group Chat turns are built from persisted history rows, while compact Inbox previews are used only to annotate reply cutoffs. | Do not parse full Inbox payload JSON during render. | -| `useAgentOrgGroupChatController.ts` | Optimistic send plus durable refresh | keep with reason | A pending message is rendered immediately, then replaced by the persisted Inbox id and refreshed through the keyset history source. | None. | -| `useUserIntentSubmit.ts` | Direct-send intervention boundary | fix | A direct user message must establish member intervention after its durable event is appended and immediately before provider dispatch. Failure to persist intervention must stop dispatch. | Keep the operation fail closed and do not move it back to composer-submit time. | -| `useQueueDispatch.ts` | Queued-send intervention boundary | fix | A queued message must not suppress background Wake while merely waiting. Intervention is established only when the item is actually dequeued for provider dispatch. | Preserve dispatch-time ordering and fail closed on persistence failure. | -| Direct/queued intervention tests | Dispatch-order regression coverage | fix | Five focused tests cover direct dispatch, queued dispatch, queue waiting, ordering, and persistence failure behavior. | Retain all five tests as required dependency coverage. | -| `OrgTaskAdapter.tsx` | Task outcome memoization | fix | The structured outcome is resolved once per render and reused by every Task branch instead of reparsing a large wrapped result. | Keep explicit outcomes authoritative. | -| `orgTaskOutcome.ts` | Legacy success evidence | fix | Persisted legacy events must show durable result evidence. Args alone cannot make a failed create, update, delete, list, or get operation appear successful. | Retain the action matrix regression. | -| `TaskUpdateCard.tsx` / `ToolCallBlock/types.ts` | Task Graph card | fix | Atomic graph creation uses a typed `graph` discriminant and existing Task-list primitives instead of raw JSON. | None. | -| `AgentEventBubbles.tsx` | Task Graph routing | fix | `task_graph_create` is registered through shared tool names and reaches the Agent Org renderer on live and replay surfaces. | None. | -| `TodoKanban.tsx` | Additive Task replay | fix | Graph creation adds rows without clearing earlier Tasks; successful deletion removes a row; rejected or failed operations leave the board unchanged. | Keep durable Run View as the live source of truth. | -| Changed TSX set | Design-system reuse | keep with reason | New controls reuse repository buttons, text areas, Markdown, Task cards, Kanban, and pagination components. No new color, spacing, or input system is introduced. | If compact typography is standardized later, use a repository-wide design-token sweep. | -| Changed TSX set | Accessibility and error state | keep with reason | Plan errors use alerts, fields retain labels, buttons expose disabled/loading state, and history failures have a visible retry action. | None. | +| Line / surface | Element | Verdict | Reason | Suggested change | +| --------------------------------------------- | ----------------------------------------------- | ---------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------- | +| `src/api/tauri/agent/orgTasks.ts` | Run View wire contract | keep with reason | `taskOverview`, exact unread counts, Plan summary/detail separation, execution mode, and optional `payloadJson` match the backend's compact Snapshot boundary. Polling no longer carries complete Plan or Inbox payloads. | Keep summary-by-default and load complete content only through detail APIs. | +| `useGroupChatMergedEvents.ts` | Group Chat Inbox preview | keep with reason | Missing `payloadJson` explicitly becomes an empty string, backend `displayText` is preferred, and JSON parse failure cannot destroy the whole feed. The projection supports both direct message responses and compact Run View rows. | None. | +| `useAgentOrgGroupChatHistory.ts` | Durable Group Chat history | keep with reason | Refresh restores persisted history, “Load older” follows durable pagination, concurrent loads are coalesced, and a recent preview is never presented as the complete transcript. | Preserve keyset pagination and keep page-loading state visible. | +| `TurnPaginationControls.tsx` | History navigation and member switching | keep with reason | Controls use durable counts and shared empty-state semantics. A member with an old unread message outside the recent preview remains selectable. | None. | +| `AgentOrgOverviewPanel.tsx` | Exact overview counters | keep with reason | Task and unread counts come from exact backend totals rather than being inferred from at most 200 visible rows. A visible/total notice appears when the Task window is truncated. | None. | +| `AgentOrgOverviewPanel.tsx` | Truncated Task-window notice | keep with reason | The notice reuses compact explanatory text, theme tokens, and i18n. It is read-only state information rather than a new control pattern. | None. | +| `AgentOrgPlanApprovalCard.tsx` | Plan summary/detail UI | keep with reason | Run View carries only summary data. Opening the card loads immutable revision detail on demand. Loading, retry, paused, submitting, and stale-revision states are distinct. | None. | +| `AgentOrgPlanApprovalCard.tsx` | Error and form accessibility | keep with reason | Existing `Button`, `Textarea`, and `Markdown` components are reused. Errors use `role="alert"`, inputs have localized accessible labels, and disabled/loading states prevent duplicate submission. | If a global async loading announcer is introduced later, implement it in the component library rather than locally. | +| `useAgentOrgPlanApprovalDetail.ts` | Revision detail cache | keep with reason | Cache keys include approval id and immutable revision id. Concurrent requests for one revision are coalesced, failures are explicitly retryable, inactive revisions use a bounded 32-entry cache, and Snapshot identity remains stable. | None. | +| `useAgentOrgPlanApprovalDetail.test.ts` | Plan detail regressions | keep with reason | Tests cover same-revision coalescing, rerender reuse, and retry after failure, proving that detail is not reloaded on every Run View poll. | None. | +| `useAgentOrgRunView.ts` | Shared per-Run Poller | keep with reason | Root and member views of one Run share one poll owner while each Session projects its own `currentMemberId`. Request sequencing prevents late responses from overwriting newer Snapshots. | None. | +| `useAgentOrgRunView.ts` | `useSyncExternalStore` Snapshot identity | fix | Before the fix, borrowing a same-Run Snapshot before the member entry existed created a new wrapper on every `getSnapshot`, violating the stable-reference contract. The first read now creates a stable entry that changes only after a real publish. | Fixed and locked by referential-equality tests. | +| `useAgentOrgRunView.ts` | Abandoned-render cache lifecycle | fix | React may call `getSnapshot` and abandon the render without subscribing. The old entry could retain up to 200 Task/Inbox previews indefinitely, and late IPC could restore it as poll owner. | Fixed with a 30-second zero-listener retirement timer, retired-generation tombstones, and late-response rejection. | +| `useAgentOrgRunView.ts` | Null response and ghost-board cleanup | keep with reason | A null owner response first hands ownership to another live Session in the same Run. The shared entry is cleared only when no live replacement exists, preventing both accidental board loss and permanent ghost state. | None. | +| `useAgentOrgRunView.ts` | Poll-owner failure isolation | keep with reason | IPC failure preserves the last valid Snapshot, exposes an error, and allows another live Session to become owner. A dead root cannot monopolize the Poller or erase Task/Plan UI. | None. | +| `useAgentOrgRunView.ts` | Bootstrap join timeout | keep with reason | An unknown-Run request shares a bootstrap owner briefly, then releases it after one second so an unrelated Run is not blocked by a hung request. Request sequencing still rejects late responses. | None. | +| `useAgentOrgRunView.test.ts` | Run View coordination regressions | keep with reason | Coverage includes one Poller, caller-specific member projection, stable Snapshot identity, same-Session coalescing, root/member bootstrap, bootstrap timeout, owner failover, null cleanup, and bounded-description signals. | None. | +| `mentionQuery.ts` / `ComposerInput/index.tsx` | WebKit mention trigger parsing | fix | WebKit input could leave the literal trigger `@` inside the search query, so `@Planner` did not match `Planner`. Only the actual trigger character is now removed. | Fixed with direct query tests and rendered real-keyboard coverage. | +| `openedTabMentionOptions.ts` | Agent Org member option identity | fix | Member entries without ordinary tab selector fields all collapsed to an `undefined:undefined` deduplication key, leaving only one visible member. Agent Org members now use `custom:${memberId}`. | Fixed with a multi-member regression test. | +| `ContextMenuPortal.tsx` | Mention menu projection | keep with reason | The menu consumes the stable member options and continues to use the existing context-menu primitives. It does not create a separate Agent Org picker. | None. | +| `useAgentOrgGroupChatController.ts` | Mention send and Group Chat orchestration | keep with reason | Mention tokens are resolved through the normal Composer path, while Run-specific recipient information remains typed. Sending does not bypass normal validation or persistence. | None. | +| `OrgTaskBlock/index.tsx` | Task delete event card | keep with reason | Deletion is modeled as an operation rather than a fake `cancelled` lifecycle state. Existing EventBlock and CompactTaskCard primitives are reused. | None. | +| `TaskUpdateCard.tsx` | Atomic Task Graph card | keep with reason | Multi-Task atomic creation renders as one collapsible graph card using existing `TaskListCard` and `ExpandableItemList` primitives, preventing a large graph from flooding the transcript. | None. | +| `ToolCallBlock/types.ts` | Task card discriminant | keep with reason | The narrow `"graph"` extension makes rendering exhaustive and typed instead of inferring graph behavior from text or payload shape. | None. | +| `OrgTaskAdapter.tsx` | Structured Task outcome projection | keep with reason | The outcome is computed once and shared by create/list/get/delete branches. Graph creation, deletion, and failed fallbacks use existing renderer primitives. | None. | +| `orgTaskOutcome.ts` | Legacy outcome fail-closed | keep with reason | New events trust Rust typed outcomes. Legacy events must contain durable success evidence; arguments alone, validation rejection, or infrastructure failure cannot mutate the replay Kanban. Explicit outcomes also avoid parsing a potentially large wrapped result. | None. | +| `orgTaskOutcome.test.ts` | Legacy outcome matrix | keep with reason | Tests cover structured rejection, wrapped legacy success, args-only false success, create/graph/update/delete/list/get evidence, and explicit outcomes that avoid reading a large result. | None. | +| `AgentEventBubbles.tsx` | Agent-event routing | keep with reason | Graph creation is registered through shared `TOOL_NAMES`, preventing string drift. The Task Graph reuses the Agent Org renderer and existing ChatBubble layout primitives. | None. | +| `AgentEventBubbles.test.ts` | Graph routing regression | keep with reason | The test proves that `task_graph_create` does not fall back to an opaque raw JSON tool card. | None. | +| `TodoKanban.tsx` | Replay Task/Kanban projection | keep with reason | Live Agent Org boards use durable Run View as truth. Historical replay additively merges graph creation, removes a successful delete, and ignores rejected or failed mutations. | None. | +| `TodoKanban.test.ts` | Kanban projection regressions | keep with reason | Coverage includes rejected creation, additive graph merge, legacy graph preservation, delete removal, and durable Agent Org Tasks taking precedence over replay. | None. | +| `memberActivity.ts` | Shared member empty-state predicate | keep with reason | This pure, style-free projection centralizes six count fields so Group Chat and WorkStation cannot disagree. Exact unread total prevents old unread mail from being hidden by a recent-activity window. | Keep the helper pure and retain the old-unread regression. | +| Changed TSX sweep | Design-system and arbitrary-value usage | keep with reason | New interactions reuse repository `Button`, `Textarea`, `KanbanBoard`, EventBlock, ChatBubble, and context-menu primitives. No raw hex/RGB color, arbitrary shadow, or parallel input/button system was added. Existing compact `text-[10px]/[11px]/[13px]` and `gap-1.5` values follow adjacent Agent Org cards. | If compact typography tokens are standardized later, perform a repository-wide sweep rather than inventing local tokens in this PR. | +| Changed TSX sweep | Keyboard, labels, motion, and responsive layout | keep with reason | Icon-only controls have `aria-label`/`title`; Plan inputs have labels and alert errors; buttons support disabled/loading; spinners honor reduced motion; actions wrap; long titles truncate with a title; Kanban retains the existing responsive read-only container. | None. | ## Changed-file coverage -This A-only audit covers the frontend implementation and tests for: - -- compact Agent Org Run View types and overview counters; -- stable shared Snapshot identity; -- Plan summary/detail loading and approval submission; -- keyset Group Chat history, reload, pagination, retry, and persisted display text; -- Task Graph extraction projection, Task cards, Task outcomes, deletion, and Kanban replay; -- direct and queued user-intervention timing at the canonical provider-dispatch boundary, including five focused regressions; -- rendered Group Chat, Plan approval, pause/resume, and recovery fixtures required by these review fixes. +This report covers the following 34 changed or newly added TypeScript/TSX files: -This report covers only the frontend changes required by the two review rounds and their minimum integration dependencies. Later UI hardening is tracked outside this branch. +1. `src/api/tauri/agent/orgTasks.ts` +2. `src/app/root/e2e/helpers/agentOrgs.ts` +3. `src/app/root/e2e/types.ts` +4. `src/components/ComposerInput/__tests__/mentionQuery.test.ts` +5. `src/components/ComposerInput/index.tsx` +6. `src/components/ComposerInput/mentionQuery.ts` +7. `src/engines/ChatPanel/ChatHistory/GroupChatView/useGroupChatMergedEvents.ts` +8. `src/engines/ChatPanel/ChatHistory/components/TurnPaginationControls.tsx` +9. `src/engines/ChatPanel/ChatView.tsx` +10. `src/engines/ChatPanel/InputArea/components/AgentOrgOverviewPanel.tsx` +11. `src/engines/ChatPanel/InputArea/components/AgentOrgPlanApprovalCard.tsx` +12. `src/engines/ChatPanel/InputArea/components/ContextMenuPortal.tsx` +13. `src/engines/ChatPanel/InputArea/components/useAgentOrgPlanApprovalDetail.test.ts` +14. `src/engines/ChatPanel/InputArea/components/useAgentOrgPlanApprovalDetail.ts` +15. `src/engines/ChatPanel/InputArea/components/useAgentOrgRunView.test.ts` +16. `src/engines/ChatPanel/InputArea/components/useAgentOrgRunView.ts` +17. `src/engines/ChatPanel/InputArea/openedTabMentionOptions.ts` +18. `src/engines/ChatPanel/InputArea/utils/__tests__/openedTabMentionOptions.test.ts` +19. `src/engines/ChatPanel/blocks/OrgTaskBlock/index.tsx` +20. `src/engines/ChatPanel/blocks/ToolCallBlock/cards/TaskUpdateCard.tsx` +21. `src/engines/ChatPanel/blocks/ToolCallBlock/types.ts` +22. `src/engines/ChatPanel/hooks/useAgentOrgGroupChatController.ts` +23. `src/engines/ChatPanel/hooks/useAgentOrgGroupChatHistory.test.ts` +24. `src/engines/ChatPanel/hooks/useAgentOrgGroupChatHistory.ts` +25. `src/engines/ChatPanel/rendering/adapters/OrgTaskAdapter.tsx` +26. `src/engines/SessionCore/rendering/orgTaskOutcome.test.ts` +27. `src/engines/SessionCore/rendering/orgTaskOutcome.ts` +28. `src/modules/WorkStation/Chat/Communication/AgentEventBubbles.test.ts` +29. `src/modules/WorkStation/Chat/Communication/AgentEventBubbles.tsx` +30. `src/modules/WorkStation/Chat/Communication/TodoKanban.test.ts` +31. `src/modules/WorkStation/Chat/Communication/TodoKanban.tsx` +32. `src/modules/WorkStation/shared/AppSwitcherWrappers.tsx` +33. `src/util/agentOrg/memberActivity.test.ts` +34. `src/util/agentOrg/memberActivity.ts` -## Verification status +The rendered `.mjs` WebDriver specs are validated separately by the E2E methodology and are not counted as TypeScript/TSX UI-audit files. -Results from the previous combined tree are not treated as A-only evidence. +## Final review addendum -| Gate | A-only result | -| ----------------------------------------- | -------------------------------------------------------------------------------------------------------------- | -| Direct/queued intervention regressions | Pass: 5 / 5. | -| Full Vitest suite | Pass: 5,318 / 5,318. | -| TypeScript typecheck | Pass. | -| ESLint and circular-dependency check | Pass. | -| Isolated real Debug App HTTP E2E | Pass: 46 / 46. | -| Rendered Group Chat WebDriver scenarios | 5 / 6 pass. The remaining mention-menu scenario fails identically on clean develop and is not an A regression. | -| Rendered Pause/Resume WebDriver scenarios | Pass: 8 / 8. | -| Rendered Recovery WebDriver scenarios | Pass: 2 / 2. | +| Line / surface | Element | Verdict | Reason | Suggested change | +| ---------------------------------------------- | --------------------------------- | ---------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------- | +| `TurnPaginationControls.tsx` member switcher | `No tasks` / disabled decision | fix | `inboxActivityCount` covers only the recent 200-row window, while `unreadInboxCount` is exact. Activity alone incorrectly disabled a member with an older unread message. | Fixed by calling `isAgentOrgMemberEmpty` and requiring Task, recent activity, and exact unread counts all to be zero. | +| `AppSwitcherWrappers.tsx` WorkStation switcher | Empty sorting / disabled decision | fix | Group Chat and WorkStation must apply the same empty-state semantics or present contradictory navigation for the same member. | Fixed by reusing the same pure helper. | +| `memberActivity.ts` | Shared activity predicate | keep with reason | The helper introduces no DOM, styling, or side effects. It centralizes domain projection and prevents the two UI entry points from drifting again. | None. | +| Mention follow-up sweep | Product and visual impact | fix | Rendered E2E exposed two real product issues: WebKit trigger parsing and unstable member deduplication. Both were corrected without adding a new control or styling system. | Retain the focused unit tests and rendered keyboard scenario. | +| Changed TSX follow-up sweep | Visual/design-system impact | keep with reason | The follow-up changes alter predicates and option identity only. They add no new color, spacing system, motion pattern, or interactive primitive. | None. | ## Verdict summary -- Review fixes represented: compact polling, durable history, Plan detail, structured Task outcomes, Task Graph rendering, additive Kanban replay, and dispatch-time user intervention. +- Fix findings: 6, all resolved in the final tree with focused regressions. - New design-system abstractions: 0. -- Out-of-scope follow-up UI changes included: 0. -- Confirmed blocking source/UI findings: 0. -- Rendered acceptance: all A-scope scenarios pass; one clean-develop mention-menu baseline remains explicitly excluded. +- Systematic visual sweep candidates: 0. +- Unresolved blocking UI findings: 0. + +Validation: + +- Full frontend Vitest: 450 files, 5,181 / 5,181 tests. +- TypeScript typecheck: pass. +- ESLint: pass. +- Rendered Debug App E2E: 19 / 19, including real keyboard mention selection, 230+ durable Group Chat history rows, Plan approval, pause/resume, recovery, and settings. ## Conclusion -The frontend remains a projection layer over durable backend facts. High-frequency reads stay compact, complete history and Plan content load on demand, failed Task attempts cannot masquerade as persisted success, and direct or queued user takeover is established only at actual dispatch. Unit, static, HTTP, and in-scope rendered verification are complete. The one remaining rendered mention-menu failure is reproduced on clean develop and is not attributed to this A-only follow-up. +The frontend keeps a clear boundary: Run View is a small, stable, shared Snapshot; full Plan and Task details are loaded on demand; and the UI projects durable backend facts instead of inventing lifecycle truth. A failed Session cannot clear another valid view of the same Run, a hung bootstrap cannot block an unrelated Run, and Task/Kanban projections no longer present a model's attempted tool call as a successful database mutation. + +The final tree adds no design-system fork, blocking accessibility defect, or unbounded payload-retention point. Snapshot identity, abandoned-render cache lifecycle, exact unread member navigation, WebKit mention parsing, and Agent Org member-option identity are all fixed and covered by focused regressions. diff --git a/docs/frontend-ui-audit-2026-07-24/AgentOrgRedTeamPort.md b/docs/frontend-ui-audit-2026-07-24/AgentOrgRedTeamPort.md new file mode 100644 index 000000000..765765b3d --- /dev/null +++ b/docs/frontend-ui-audit-2026-07-24/AgentOrgRedTeamPort.md @@ -0,0 +1,32 @@ +# Agent Org Red-Team Hardening Port — Frontend UI Audit + +## Scope + +The audit covers the Run View cache, Group Chat history, member navigation, +mention options, Composer inline mentions, and Plan Approval detail cache +touched or semantically depended on by the PR #424 port. This is independent +from the architecture audit. + +| Line | Element | Verdict | Reason | Suggested change | +| ------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------- | ---------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------- | --- | ---------------------------------- | +| `src/api/tauri/agent/orgTasks.ts:128,254` | Run View and Inbox wire fields | fix | The UI must distinguish corrupt task state and a resolved delivery from ordinary unread/read state. | Keep `corrupt` and `deliveryResolution` typed at the Tauri boundary and avoid component-local casts. | +| `src/engines/ChatPanel/InputArea/components/agentOrgRunViewStore.ts:10-60` | Shared Run View cache | fix | Abandoned React renders and late IPC responses could retain or repopulate stale Run state. Retention, generation retirement, request ordering, and bounded bootstrap joining close those races. | Keep cache lifetime and request ordering in this store; components should only subscribe. | +| `src/engines/ChatPanel/hooks/useAgentOrgGroupChatHistory.ts:9-218` | Historical pagination | fix | Refresh overlap can open cursor gaps; an unbounded continuation stack would trade correctness for memory growth. The bounded frontier plus scan-through fallback preserves rows and bounds client state. | Keep pagination state in the hook model and retain overlap/gap regression tests. | +| `src/engines/ChatPanel/hooks/useAgentOrgGroupChatHistory.ts:237-263` | Delivery settlement projection | fix | Read and coordinator resolution are different durable outcomes, but both settle a pending delivery indicator. | Use the shared settlement predicates instead of duplicating `readAt | | deliveryResolution` in components. | +| `src/util/agentOrg/memberActivity.ts:3-30` | Member emptiness predicate | abstract | Two navigation surfaces previously duplicated a partial task/activity check and missed old durable unread Inbox rows. | Reuse `isAgentOrgMemberEmpty` for all member navigation surfaces. | +| `src/engines/ChatPanel/ChatHistory/components/TurnPaginationControls.tsx:317-333` and `src/modules/WorkStation/shared/AppSwitcherWrappers.tsx:183-190` | Member navigation availability | fix | Both surfaces now use the exact same task, recent activity, and durable unread dimensions, preventing contradictory disabled/crossed-out state. | Keep both consumers on the shared predicate; do not infer activity from runtime status. | +| `src/engines/ChatPanel/InputArea/openedTabMentionOptions.ts:102-123` | Mention target merge | abstract | Duplicated options must be keyed by actionable target, while targetless custom actions remain distinct by id. | Keep `mergeCustomMentionOptions` as the only merge implementation. | +| `src/engines/ChatPanel/InputArea/components/ContextMenuPortal.tsx:22-25` | Mention menu composition | keep with reason | The existing design-system menu/portal is appropriate; the correctness issue was duplicate merge logic, now delegated to the shared helper. | No visual abstraction change. Continue importing the shared merge helper. | +| `src/components/ComposerInput/mentionQuery.ts:14-24` | WebKit inline `@` query parsing | fix | WebKit may record the trigger before the literal `@` input event. Normalizing the query start prevents `@` from leaking into the search term. | Keep the pure helper and its event-order regression tests; do not patch DOM text directly. | +| `src/engines/ChatPanel/InputArea/components/useAgentOrgPlanApprovalDetail.ts:28-55` | Plan Approval detail cache | keep with reason | Current `develop` already bounds the cache by both 64 entries and 8 MiB, which is stronger than the source PR's entry-only LRU. | Preserve the current byte-aware cache and existing tests; do not replace it with the older implementation. | + +## Counts + +- fix: 6 +- keep with reason: 2 +- abstract: 2 + +No arbitrary visual token, design-system bypass, or new accessibility pattern +was introduced. The rendered E2E gate exercises the real Composer and Agent Org +navigation paths; debug helpers are limited to deterministic seeding and +inspection. diff --git a/src-tauri/crates/agent-core/src/core/coordination/agent_inbox/message.rs b/src-tauri/crates/agent-core/src/core/coordination/agent_inbox/message.rs index 79de60388..3f79f5270 100644 --- a/src-tauri/crates/agent-core/src/core/coordination/agent_inbox/message.rs +++ b/src-tauri/crates/agent-core/src/core/coordination/agent_inbox/message.rs @@ -366,7 +366,7 @@ impl AgentMessage { /// via tool error reporting. pub fn validate(&self) -> Result<(), String> { if let Some(request_id) = self.request_id() { - validate_non_empty_identifier("request_id", request_id.as_str())?; + limits::validate_message_identifier("request_id", request_id.as_str())?; } match self { Self::Plain { summary, text } => { @@ -414,12 +414,15 @@ impl AgentMessage { limits::PLAN_TITLE_MAX_CHARS, limits::PLAN_TITLE_MAX_BYTES, )?; - validate_non_empty_identifier("PlanApprovalRequest.approval_id", approval_id)?; - validate_non_empty_identifier( + limits::validate_message_identifier( + "PlanApprovalRequest.approval_id", + approval_id, + )?; + limits::validate_message_identifier( "PlanApprovalRequest.plan_revision_id", plan_revision_id, )?; - validate_non_empty_identifier( + limits::validate_task_identifier( "PlanApprovalRequest.source_task_id", source_task_id, )?; @@ -461,7 +464,7 @@ impl AgentMessage { member_name, .. } => { - validate_non_empty_identifier("MemberTerminated.member_id", member_id)?; + limits::validate_message_identifier("MemberTerminated.member_id", member_id)?; limits::validate_required_text( "MemberTerminated.member_name", member_name, @@ -478,7 +481,7 @@ impl AgentMessage { unfinished_task_ids, .. } => { - validate_non_empty_identifier("MemberIdle.member_id", member_id)?; + limits::validate_message_identifier("MemberIdle.member_id", member_id)?; limits::validate_required_text( "MemberIdle.member_name", member_name, @@ -491,16 +494,15 @@ impl AgentMessage { limits::MEMBER_SUMMARY_MAX_CHARS, limits::MEMBER_SUMMARY_MAX_BYTES, )?; - if unfinished_task_ids.len() > 32 - || unfinished_task_ids - .iter() - .any(|task_id| task_id.trim().is_empty() || task_id.len() > 200) - { + if unfinished_task_ids.len() > 32 { return Err( - "MemberIdle.unfinished_task_ids must contain at most 32 non-empty task ids of <= 200 chars" - .into(), + "MemberIdle.unfinished_task_ids must contain at most 32 task ids".into(), ); } + limits::validate_task_identifier_list( + "MemberIdle.unfinished_task_ids", + unfinished_task_ids, + )?; match reason { MemberIdleReason::Failed => { let fr = failure_reason.as_deref().unwrap_or("").trim(); @@ -529,7 +531,7 @@ impl AgentMessage { dependency_outputs, .. } => { - validate_non_empty_identifier("TaskAssigned.task_id", task_id)?; + limits::validate_task_identifier("TaskAssigned.task_id", task_id)?; if dependency_outputs.len() > limits::TASK_DEPENDENCY_OUTPUT_MAX_COUNT { return Err(format!( "TaskAssigned.dependency_outputs must contain at most {} entries", @@ -565,6 +567,14 @@ impl AgentMessage { .into(), ); } + limits::validate_task_identifier( + "TaskAssigned dependency task_id", + &output.task_id, + )?; + limits::validate_message_identifier( + "TaskAssigned dependency produced_by_member_id", + &output.produced_by_member_id, + )?; limits::validate_required_text( "TaskAssigned dependency subject", &output.subject, @@ -583,6 +593,10 @@ impl AgentMessage { limits::TASK_OUTPUT_CONTENT_MAX_CHARS, limits::TASK_OUTPUT_CONTENT_MAX_BYTES, )?; + limits::validate_task_artifact_ids( + "TaskAssigned dependency artifact_ids", + &output.artifact_ids, + )?; } let total_inline_chars: usize = dependency_outputs .iter() @@ -614,14 +628,14 @@ impl AgentMessage { output_summary, .. } => { - validate_non_empty_identifier("TaskCompleted.task_id", task_id)?; + limits::validate_task_identifier("TaskCompleted.task_id", task_id)?; limits::validate_required_text( "TaskCompleted.subject", subject, limits::TASK_SUBJECT_MAX_CHARS, limits::TASK_SUBJECT_MAX_BYTES, )?; - validate_non_empty_identifier( + limits::validate_message_identifier( "TaskCompleted.completed_by_member_id", completed_by_member_id, )?; @@ -648,16 +662,233 @@ impl AgentMessage { } } -pub(super) fn validate_non_empty_identifier(field: &str, value: &str) -> Result<(), String> { - if value.trim().is_empty() { - return Err(format!("{field} must not be empty")); - } - Ok(()) -} - #[cfg(test)] mod tests { + use super::super::{ + init_schema, AgentInboxDeliveryResolutionKind, AgentInboxRecord, AgentInboxStore, + InsertInboxParams, ResolveInboxDeliveryParams, SYSTEM_SENDER_ID, + }; use super::*; + use crate::coordination::agent_org_runs::COORDINATOR_MEMBER_ID; + use database::db::get_connection; + use rusqlite::params; + + fn sandbox_with_inbox_schema() -> test_helpers::test_env::SandboxGuard { + let sandbox = test_helpers::test_env::sandbox(); + let conn = get_connection().expect("open sandbox database"); + init_schema(&conn).expect("initialize agent inbox schema"); + sandbox + } + + fn seed_minimal_running_run_for_delivery_resolution(run_id: &str) { + let conn = get_connection().expect("open sandbox database"); + conn.execute_batch( + "CREATE TABLE IF NOT EXISTS agent_org_runs ( + id TEXT PRIMARY KEY, + status TEXT NOT NULL, + org_snapshot_json TEXT, + root_session_id TEXT + ); + CREATE TABLE IF NOT EXISTS agent_sessions ( + session_id TEXT PRIMARY KEY, + status TEXT NOT NULL, + updated_at TEXT NOT NULL, + parent_session_id TEXT, + agent_definition_id TEXT, + org_member_id TEXT + ); + CREATE TABLE IF NOT EXISTS code_sessions ( + session_id TEXT PRIMARY KEY, + cli_agent_type TEXT NOT NULL, + status TEXT NOT NULL, + parent_session_id TEXT, + org_member_id TEXT, + updated_at TEXT NOT NULL + ); + CREATE TABLE IF NOT EXISTS agent_org_tasks ( + id TEXT PRIMARY KEY, + org_run_id TEXT NOT NULL + );", + ) + .expect("initialize minimal delivery-repair dependencies"); + crate::coordination::agent_member_interventions::init_schema(&conn) + .expect("initialize intervention lookup for delivery repair"); + let root_session_id = format!("root-{run_id}"); + conn.execute( + "INSERT INTO agent_sessions (session_id, status, updated_at) + VALUES (?1, 'idle', ?2)", + params![&root_session_id, chrono::Utc::now().to_rfc3339()], + ) + .expect("seed coordinator session"); + conn.execute( + "INSERT INTO agent_org_runs (id, status, org_snapshot_json, root_session_id) + VALUES (?1, 'running', NULL, ?2)", + params![run_id, &root_session_id], + ) + .expect("seed running run"); + } + + fn seed_legacy_orphan_inbox_row(run_id: &str, summary: &str, text: &str) -> AgentInboxRecord { + let message = AgentMessage::Plain { + summary: summary.to_string(), + text: text.to_string(), + }; + let payload_json = serde_json::to_string(&message).expect("serialize legacy payload"); + let conn = get_connection().expect("open sandbox database"); + conn.execute( + "INSERT INTO agent_inbox ( + recipient_agent_id, recipient_member_id, + sender_agent_id, sender_member_id, org_run_id, + payload_kind, payload_json, request_id, + created_at, read_at, causation_inbox_id + ) VALUES (?1, NULL, ?2, ?3, ?4, 'plain', ?5, NULL, ?6, NULL, NULL)", + params![ + "missing-agent", + "coordinator-agent", + "coordinator", + run_id, + payload_json, + chrono::Utc::now().to_rfc3339(), + ], + ) + .expect("seed historical orphan Inbox row"); + let inbox_id = conn.last_insert_rowid(); + AgentInboxStore::get_by_id_for_run(run_id, inbox_id) + .expect("load historical orphan row") + .expect("historical orphan row exists") + } + + fn messages_for_each_task_identifier_position( + task_id: &str, + ) -> Vec<(&'static str, AgentMessage)> { + vec![ + ( + "PlanApprovalRequest.source_task_id", + AgentMessage::PlanApprovalRequest { + request_id: RequestId("request-task-id-boundary".into()), + approval_id: "approval-task-id-boundary".into(), + plan_revision_id: "revision-task-id-boundary".into(), + source_task_id: task_id.into(), + plan_title: "Task identifier boundary".into(), + plan_path: "/tmp/task-id-boundary.plan.md".into(), + plan_content: "# Plan".into(), + }, + ), + ( + "MemberIdle.unfinished_task_ids[]", + AgentMessage::MemberIdle { + member_id: "worker".into(), + member_name: "Worker".into(), + reason: MemberIdleReason::Available, + current_mode: Some(AgentExecMode::Build), + summary: None, + failure_reason: None, + unfinished_task_ids: vec![task_id.into()], + }, + ), + ( + "TaskAssigned.task_id", + AgentMessage::TaskAssigned { + task_id: task_id.into(), + subject: "Assigned task".into(), + description: "Validate the task identifier boundary".into(), + assigned_by: "Coordinator".into(), + execution_mode: crate::coordination::agent_org_tasks::TaskExecutionMode::Build, + dependency_outputs: Vec::new(), + }, + ), + ( + "TaskAssigned.dependency_outputs[].task_id", + AgentMessage::TaskAssigned { + task_id: "dependent-task".into(), + subject: "Dependent task".into(), + description: "Consume a completed dependency".into(), + assigned_by: "Coordinator".into(), + execution_mode: crate::coordination::agent_org_tasks::TaskExecutionMode::Build, + dependency_outputs: vec![TaskDependencyOutput { + task_id: task_id.into(), + subject: "Dependency".into(), + summary: "Dependency completed".into(), + content: None, + artifact_ids: Vec::new(), + produced_by_member_id: "producer".into(), + }], + }, + ), + ( + "TaskAssigned.dependency_outputs[].produced_by_member_id", + AgentMessage::TaskAssigned { + task_id: "dependent-producer-task".into(), + subject: "Dependent task".into(), + description: "Consume a completed dependency".into(), + assigned_by: "Coordinator".into(), + execution_mode: crate::coordination::agent_org_tasks::TaskExecutionMode::Build, + dependency_outputs: vec![TaskDependencyOutput { + task_id: "dependency-task".into(), + subject: "Dependency".into(), + summary: "Dependency completed".into(), + content: None, + artifact_ids: Vec::new(), + produced_by_member_id: task_id.into(), + }], + }, + ), + ( + "MemberTerminated.member_id", + AgentMessage::MemberTerminated { + member_id: task_id.into(), + member_name: "Worker".into(), + reason: MemberTerminationReason::Shutdown, + }, + ), + ( + "MemberIdle.member_id", + AgentMessage::MemberIdle { + member_id: task_id.into(), + member_name: "Worker".into(), + reason: MemberIdleReason::Available, + current_mode: Some(AgentExecMode::Build), + summary: None, + failure_reason: None, + unfinished_task_ids: Vec::new(), + }, + ), + ( + "TaskCompleted.task_id", + AgentMessage::TaskCompleted { + task_id: task_id.into(), + subject: "Completed task".into(), + completed_by_member_id: "worker".into(), + output_summary: Some("Done".into()), + remaining_open_task_count: 0, + }, + ), + ( + "TaskCompleted.completed_by_member_id", + AgentMessage::TaskCompleted { + task_id: "completed-member-boundary".into(), + subject: "Completed task".into(), + completed_by_member_id: task_id.into(), + output_summary: Some("Done".into()), + remaining_open_task_count: 0, + }, + ), + ] + } + + fn insert_boundary_message( + run_id: &str, + message: AgentMessage, + ) -> Result { + AgentInboxStore::insert(InsertInboxParams { + recipient_agent_id: "recipient-agent".into(), + recipient_member_id: Some("recipient-member".into()), + sender_agent_id: SYSTEM_SENDER_ID.into(), + sender_member_id: None, + org_run_id: Some(run_id.into()), + message, + }) + } #[test] fn plain_validation() { @@ -958,6 +1189,71 @@ mod tests { assert!(ok.request_id().is_none()); } + #[test] + fn every_task_identifier_position_accepts_the_exact_char_and_byte_limit() { + let _sandbox = sandbox_with_inbox_schema(); + let run_id = format!("run-task-id-exact-{}", uuid::Uuid::new_v4()); + // One emoji is four UTF-8 bytes, so this value simultaneously reaches + // the exact character and byte ceilings (1000 chars / 4000 bytes). + let exact_limit = "😀".repeat(limits::TASK_IDENTIFIER_MAX_CHARS); + + for (position, message) in messages_for_each_task_identifier_position(&exact_limit) { + insert_boundary_message(&run_id, message) + .unwrap_or_else(|error| panic!("{position} rejected the exact limit: {error}")); + } + + assert_eq!( + AgentInboxStore::list_by_run(&run_id) + .expect("list exact-limit inbox rows") + .len(), + 9 + ); + } + + #[test] + fn every_task_identifier_position_rejects_over_char_limit_before_persistence() { + let _sandbox = sandbox_with_inbox_schema(); + let run_id = format!("run-task-id-char-over-{}", uuid::Uuid::new_v4()); + let over_char_limit = "x".repeat(limits::TASK_IDENTIFIER_MAX_CHARS + 1); + + for (position, message) in messages_for_each_task_identifier_position(&over_char_limit) { + let error = match insert_boundary_message(&run_id, message) { + Ok(_) => panic!("{position} accepted an oversized task id"), + Err(error) => error, + }; + assert!(error.contains("chars"), "{position}: {error}"); + } + + assert!( + AgentInboxStore::list_by_run(&run_id) + .expect("list rejected char-limit rows") + .is_empty(), + "invalid task identifiers must not reach durable inbox storage" + ); + } + + #[test] + fn every_task_identifier_position_rejects_over_byte_limit_before_persistence() { + let _sandbox = sandbox_with_inbox_schema(); + let run_id = format!("run-task-id-byte-over-{}", uuid::Uuid::new_v4()); + let over_byte_limit = "😀".repeat(limits::TASK_IDENTIFIER_MAX_CHARS + 1); + + for (position, message) in messages_for_each_task_identifier_position(&over_byte_limit) { + let error = match insert_boundary_message(&run_id, message) { + Ok(_) => panic!("{position} accepted an oversized task id"), + Err(error) => error, + }; + assert!(error.contains("bytes"), "{position}: {error}"); + } + + assert!( + AgentInboxStore::list_by_run(&run_id) + .expect("list rejected byte-limit rows") + .is_empty(), + "byte-oversized task identifiers must not reach durable inbox storage" + ); + } + #[test] fn task_assigned_rejects_blank_required_fields() { let bad_id = AgentMessage::TaskAssigned { @@ -1014,6 +1310,347 @@ mod tests { assert!(bad_description.validate().is_err()); } + #[test] + fn cancelled_delivery_stays_unread_as_evidence_but_leaves_pending_queries() { + let _sandbox = sandbox_with_inbox_schema(); + let run_id = "run-delivery-cancel"; + seed_minimal_running_run_for_delivery_resolution(run_id); + let row = + seed_legacy_orphan_inbox_row(run_id, "Undeliverable", "Preserve this exact message"); + + let resolution = AgentInboxStore::resolve_delivery(ResolveInboxDeliveryParams { + inbox_id: row.id, + org_run_id: run_id.into(), + resolved_by_member_id: "coordinator".into(), + resolution_kind: AgentInboxDeliveryResolutionKind::Cancelled, + reason: "Member was removed and the work is no longer required".into(), + replacement_inbox_id: None, + replacement_task_id: None, + }) + .expect("cancel delivery"); + assert_eq!( + resolution.resolution_kind, + AgentInboxDeliveryResolutionKind::Cancelled + ); + let conn = get_connection().expect("open sandbox database"); + assert!( + AgentInboxStore::unread_counts_by_recipient_with_connection(&conn, run_id) + .expect("pending delivery snapshot") + .is_empty() + ); + let stored = AgentInboxStore::get_by_id_for_run(run_id, row.id) + .expect("load evidence") + .expect("source row remains"); + assert!( + stored.read_at.is_none(), + "resolution must not fake a read receipt" + ); + + let same = AgentInboxStore::resolve_delivery(ResolveInboxDeliveryParams { + inbox_id: row.id, + org_run_id: run_id.into(), + resolved_by_member_id: "coordinator".into(), + resolution_kind: AgentInboxDeliveryResolutionKind::Cancelled, + reason: "Member was removed and the work is no longer required".into(), + replacement_inbox_id: None, + replacement_task_id: None, + }) + .expect("exact retry is idempotent"); + assert_eq!(same, resolution); + let conflict = AgentInboxStore::resolve_delivery(ResolveInboxDeliveryParams { + inbox_id: row.id, + org_run_id: run_id.into(), + resolved_by_member_id: "coordinator".into(), + resolution_kind: AgentInboxDeliveryResolutionKind::Cancelled, + reason: "A different decision".into(), + replacement_inbox_id: None, + replacement_task_id: None, + }) + .expect_err("different retry must conflict"); + assert!(conflict + .to_string() + .contains("different delivery resolution")); + } + + #[test] + fn delivery_resolution_invalidates_stale_materialization_guard() { + let _sandbox = sandbox_with_inbox_schema(); + let run_id = "run-delivery-stale-guard"; + seed_minimal_running_run_for_delivery_resolution(run_id); + let row = seed_legacy_orphan_inbox_row( + run_id, + "Stale receipt", + "Do not acknowledge after repair", + ); + let conn = get_connection().expect("open sandbox database"); + conn.execute( + "INSERT INTO agent_inbox_materializations ( + inbox_id, session_id, transcript_message_id, + transcript_intent_id, materialized_at + ) VALUES (?1, 'old-session', 'message-1', 'intent-1', ?2)", + params![row.id, chrono::Utc::now().to_rfc3339()], + ) + .expect("seed stale receipt"); + + AgentInboxStore::resolve_delivery(ResolveInboxDeliveryParams { + inbox_id: row.id, + org_run_id: run_id.into(), + resolved_by_member_id: "coordinator".into(), + resolution_kind: AgentInboxDeliveryResolutionKind::Cancelled, + reason: "Recipient permanently unavailable".into(), + replacement_inbox_id: None, + replacement_task_id: None, + }) + .expect("resolve delivery"); + assert_eq!( + AgentInboxStore::mark_many_read_for_session(&[row.id], "old-session") + .expect("resolved row is an acknowledgement no-op"), + 0 + ); + let stored = AgentInboxStore::get_by_id_for_run(run_id, row.id) + .expect("load source") + .expect("source remains"); + assert!(stored.read_at.is_none()); + } + + #[test] + fn delivery_resolution_rejects_a_healthy_canonical_recipient() { + let _sandbox = sandbox_with_inbox_schema(); + let run_id = "run-delivery-healthy"; + seed_minimal_running_run_for_delivery_resolution(run_id); + let row = AgentInboxStore::insert(InsertInboxParams { + recipient_agent_id: "coordinator-agent".into(), + recipient_member_id: Some("coordinator".into()), + sender_agent_id: "worker-agent".into(), + sender_member_id: Some("worker".into()), + org_run_id: Some(run_id.into()), + message: AgentMessage::Plain { + summary: "Healthy delivery".into(), + text: "This must reach the coordinator".into(), + }, + }) + .expect("insert healthy row"); + let error = AgentInboxStore::resolve_delivery(ResolveInboxDeliveryParams { + inbox_id: row.id, + org_run_id: run_id.into(), + resolved_by_member_id: "coordinator".into(), + resolution_kind: AgentInboxDeliveryResolutionKind::Cancelled, + reason: "The model changed its mind".into(), + replacement_inbox_id: None, + replacement_task_id: None, + }) + .expect_err("healthy recipient cannot be discarded by the model"); + assert!(error + .to_string() + .contains("recoverable canonical recipient")); + assert!( + AgentInboxStore::has_unread_for_member("coordinator", run_id) + .expect("healthy delivery remains pending") + ); + } + + #[test] + fn superseded_delivery_requires_an_existing_same_run_replacement() { + let _sandbox = sandbox_with_inbox_schema(); + let run_id = "run-delivery-supersede"; + seed_minimal_running_run_for_delivery_resolution(run_id); + let source = seed_legacy_orphan_inbox_row(run_id, "Original", "Original work"); + let conn = get_connection().expect("open sandbox database"); + conn.execute( + "INSERT INTO agent_org_tasks (id, org_run_id) VALUES ('replacement-task', ?1)", + params![run_id], + ) + .expect("seed replacement task"); + + let missing = AgentInboxStore::resolve_delivery(ResolveInboxDeliveryParams { + inbox_id: source.id, + org_run_id: run_id.into(), + resolved_by_member_id: "coordinator".into(), + resolution_kind: AgentInboxDeliveryResolutionKind::Superseded, + reason: "Moved to a durable replacement".into(), + replacement_inbox_id: None, + replacement_task_id: Some("missing-task".into()), + }) + .expect_err("missing replacement rejected"); + assert!(missing.to_string().contains("does not exist")); + + let resolution = AgentInboxStore::resolve_delivery(ResolveInboxDeliveryParams { + inbox_id: source.id, + org_run_id: run_id.into(), + resolved_by_member_id: "coordinator".into(), + resolution_kind: AgentInboxDeliveryResolutionKind::Superseded, + reason: "Moved to a durable replacement".into(), + replacement_inbox_id: None, + replacement_task_id: Some("replacement-task".into()), + }) + .expect("supersede delivery"); + assert_eq!( + resolution.replacement_task_id.as_deref(), + Some("replacement-task") + ); + } + + #[test] + fn superseded_delivery_can_follow_a_real_replacement_chain_but_not_cycle() { + use crate::definitions::orgs::{ + HierarchyMode, OrgDefinition, OrgMember, PlanApprovalPolicy, + }; + + let _sandbox = sandbox_with_inbox_schema(); + let run_id = "run-delivery-chain"; + seed_minimal_running_run_for_delivery_resolution(run_id); + let org = OrgDefinition { + id: "org-delivery-chain".into(), + name: "Delivery Chain".into(), + role: "Coordinator".into(), + agent_id: "coordinator-agent".into(), + description: None, + hierarchy_mode: HierarchyMode::Soft, + plan_approval_policy: PlanApprovalPolicy::Coordinator, + children: vec![ + OrgMember { + id: "member-a".into(), + name: "Member A".into(), + role: "worker".into(), + agent_id: "agent-a".into(), + runtime_config: None, + children: Vec::new(), + }, + OrgMember { + id: "member-b".into(), + name: "Member B".into(), + role: "worker".into(), + agent_id: "agent-b".into(), + runtime_config: None, + children: Vec::new(), + }, + OrgMember { + id: "member-c".into(), + name: "Member C".into(), + role: "worker".into(), + agent_id: "agent-c".into(), + runtime_config: None, + children: Vec::new(), + }, + ], + }; + let conn = get_connection().expect("open sandbox database"); + conn.execute( + "UPDATE agent_org_runs SET org_snapshot_json=?1 WHERE id=?2", + params![serde_json::to_string(&org).unwrap(), run_id], + ) + .expect("seed roster snapshot"); + let now = chrono::Utc::now().to_rfc3339(); + for (session_id, member_id, agent_id) in [ + ("session-a", "member-a", "agent-a"), + ("session-b", "member-b", "agent-b"), + ("session-c", "member-c", "agent-c"), + ] { + conn.execute( + "INSERT INTO agent_sessions ( + session_id, status, updated_at, parent_session_id, + agent_definition_id, org_member_id + ) VALUES (?1, 'idle', ?2, ?3, ?4, ?5)", + params![ + session_id, + &now, + format!("root-{run_id}"), + agent_id, + member_id + ], + ) + .expect("seed healthy replacement member"); + } + + let row_a = AgentInboxStore::insert(InsertInboxParams { + recipient_agent_id: "agent-a".into(), + recipient_member_id: Some("member-a".into()), + sender_agent_id: "coordinator-agent".into(), + sender_member_id: Some(COORDINATOR_MEMBER_ID.into()), + org_run_id: Some(run_id.into()), + message: AgentMessage::Plain { + summary: "A".into(), + text: "Original delivery".into(), + }, + }) + .expect("insert original delivery"); + let row_b = AgentInboxStore::insert(InsertInboxParams { + recipient_agent_id: "agent-b".into(), + recipient_member_id: Some("member-b".into()), + sender_agent_id: "coordinator-agent".into(), + sender_member_id: Some(COORDINATOR_MEMBER_ID.into()), + org_run_id: Some(run_id.into()), + message: AgentMessage::Plain { + summary: "B".into(), + text: "First replacement".into(), + }, + }) + .expect("insert first replacement"); + let row_c = AgentInboxStore::insert(InsertInboxParams { + recipient_agent_id: "agent-c".into(), + recipient_member_id: Some("member-c".into()), + sender_agent_id: "coordinator-agent".into(), + sender_member_id: Some(COORDINATOR_MEMBER_ID.into()), + org_run_id: Some(run_id.into()), + message: AgentMessage::Plain { + summary: "C".into(), + text: "Second replacement".into(), + }, + }) + .expect("insert second replacement"); + + conn.execute( + "UPDATE agent_sessions SET status='archived' WHERE session_id='session-a'", + [], + ) + .expect("archive original recipient"); + AgentInboxStore::resolve_delivery(ResolveInboxDeliveryParams { + inbox_id: row_a.id, + org_run_id: run_id.into(), + resolved_by_member_id: COORDINATOR_MEMBER_ID.into(), + resolution_kind: AgentInboxDeliveryResolutionKind::Superseded, + reason: "Moved from A to B".into(), + replacement_inbox_id: Some(row_b.id), + replacement_task_id: None, + }) + .expect("supersede A with B"); + + conn.execute( + "UPDATE agent_sessions SET status='archived' WHERE session_id='session-b'", + [], + ) + .expect("archive first replacement recipient"); + AgentInboxStore::resolve_delivery(ResolveInboxDeliveryParams { + inbox_id: row_b.id, + org_run_id: run_id.into(), + resolved_by_member_id: COORDINATOR_MEMBER_ID.into(), + resolution_kind: AgentInboxDeliveryResolutionKind::Superseded, + reason: "Moved from B to C after B became unavailable".into(), + replacement_inbox_id: Some(row_c.id), + replacement_task_id: None, + }) + .expect("supersede unavailable B with C"); + + conn.execute( + "UPDATE agent_sessions SET status='archived' WHERE session_id='session-c'", + [], + ) + .expect("archive second replacement recipient"); + let cycle = AgentInboxStore::resolve_delivery(ResolveInboxDeliveryParams { + inbox_id: row_c.id, + org_run_id: run_id.into(), + resolved_by_member_id: COORDINATOR_MEMBER_ID.into(), + resolution_kind: AgentInboxDeliveryResolutionKind::Superseded, + reason: "Attempt to cycle back to A".into(), + replacement_inbox_id: Some(row_a.id), + replacement_task_id: None, + }) + .expect_err("a replacement chain must not cycle into an already resolved row"); + assert!(cycle + .to_string() + .contains("already has a delivery resolution")); + } + #[test] fn task_assigned_round_trips_through_serde() { let msg = AgentMessage::TaskAssigned { diff --git a/src-tauri/crates/agent-core/src/core/coordination/agent_inbox/mod.rs b/src-tauri/crates/agent-core/src/core/coordination/agent_inbox/mod.rs index 2be514d69..dd17cedeb 100644 --- a/src-tauri/crates/agent-core/src/core/coordination/agent_inbox/mod.rs +++ b/src-tauri/crates/agent-core/src/core/coordination/agent_inbox/mod.rs @@ -46,8 +46,9 @@ pub use message::{ }; pub(crate) use record::AgentInboxUnreadRecipientCounts; pub use record::{ - AgentInboxBatch, AgentInboxPage, AgentInboxPreviewRecord, AgentInboxRecipientCounts, - AgentInboxRecord, InsertInboxParams, + AgentInboxBatch, AgentInboxDeliveryResolution, AgentInboxDeliveryResolutionKind, + AgentInboxPage, AgentInboxPreviewRecord, AgentInboxRecipientCounts, AgentInboxRecord, + InsertInboxParams, ResolveInboxDeliveryError, ResolveInboxDeliveryParams, }; pub use schema::init_schema; diff --git a/src-tauri/crates/agent-core/src/core/coordination/agent_inbox/record.rs b/src-tauri/crates/agent-core/src/core/coordination/agent_inbox/record.rs index d404618a5..af96eb178 100644 --- a/src-tauri/crates/agent-core/src/core/coordination/agent_inbox/record.rs +++ b/src-tauri/crates/agent-core/src/core/coordination/agent_inbox/record.rs @@ -2,7 +2,8 @@ //! row mappers shared by the store submodules. use rusqlite::Result as SqliteResult; -use serde::Serialize; +use serde::{Deserialize, Serialize}; +use thiserror::Error; use super::AgentMessage; @@ -74,6 +75,7 @@ pub struct AgentInboxPreviewRecord { pub created_at: String, pub read_at: Option, pub display_preview: Option, + pub delivery_resolution: Option, } #[derive(Debug, Clone)] @@ -90,6 +92,71 @@ pub struct AgentInboxPage { pub next_cursor: Option, } +/// Explicit disposition for an Inbox row that can no longer be delivered to +/// its original canonical recipient. The source row remains immutable and +/// unread for audit/history; operational readers treat a row with one of +/// these append-only records as no longer pending delivery. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)] +#[serde(rename_all = "snake_case")] +pub enum AgentInboxDeliveryResolutionKind { + Cancelled, + Superseded, +} + +impl AgentInboxDeliveryResolutionKind { + pub const fn as_str(self) -> &'static str { + match self { + Self::Cancelled => "cancelled", + Self::Superseded => "superseded", + } + } + + pub(super) fn parse(value: &str) -> Result { + match value { + "cancelled" => Ok(Self::Cancelled), + "superseded" => Ok(Self::Superseded), + other => Err(format!( + "unknown Agent Inbox delivery resolution kind: {other:?}" + )), + } + } +} + +#[derive(Debug, Clone, PartialEq, Eq, Serialize)] +#[serde(rename_all = "camelCase")] +pub struct AgentInboxDeliveryResolution { + pub inbox_id: i64, + pub org_run_id: String, + pub resolution_kind: AgentInboxDeliveryResolutionKind, + pub resolved_by_member_id: String, + pub reason: String, + pub replacement_inbox_id: Option, + pub replacement_task_id: Option, + pub created_at: String, +} + +#[derive(Debug, Clone)] +pub struct ResolveInboxDeliveryParams { + pub inbox_id: i64, + pub org_run_id: String, + pub resolved_by_member_id: String, + pub resolution_kind: AgentInboxDeliveryResolutionKind, + pub reason: String, + pub replacement_inbox_id: Option, + pub replacement_task_id: Option, +} + +/// Typed boundary between expected coordinator corrections and infrastructure +/// failures. The LLM tool renders `Constraint` as recoverable guidance while +/// keeping SQLite/schema/locking failures as real execution failures. +#[derive(Debug, Error)] +pub enum ResolveInboxDeliveryError { + #[error("{0}")] + Constraint(String), + #[error("{0}")] + Storage(String), +} + impl AgentInboxRecord { /// Re-hydrate the typed `AgentMessage` from the persisted JSON /// payload. Returns a stable error string on schema drift; callers @@ -143,5 +210,6 @@ pub(super) fn row_to_preview_record( created_at: row.get(8)?, read_at: row.get(9)?, display_preview: row.get(10)?, + delivery_resolution: row.get(11)?, }) } diff --git a/src-tauri/crates/agent-core/src/core/coordination/agent_inbox/schema.rs b/src-tauri/crates/agent-core/src/core/coordination/agent_inbox/schema.rs index 913593568..5a0a56c59 100644 --- a/src-tauri/crates/agent-core/src/core/coordination/agent_inbox/schema.rs +++ b/src-tauri/crates/agent-core/src/core/coordination/agent_inbox/schema.rs @@ -23,6 +23,28 @@ pub fn init_schema(conn: &Connection) -> SqliteResult<()> { transcript_intent_id TEXT NOT NULL, materialized_at TEXT NOT NULL ); + CREATE TABLE IF NOT EXISTS agent_inbox_delivery_resolutions ( + inbox_id INTEGER PRIMARY KEY, + org_run_id TEXT NOT NULL, + resolution_kind TEXT NOT NULL + CHECK(resolution_kind IN ('cancelled', 'superseded')), + resolved_by_member_id TEXT NOT NULL, + reason TEXT NOT NULL, + replacement_inbox_id INTEGER, + replacement_task_id TEXT, + created_at TEXT NOT NULL, + CHECK( + (resolution_kind='cancelled' + AND replacement_inbox_id IS NULL + AND replacement_task_id IS NULL) + OR + (resolution_kind='superseded' + AND ((replacement_inbox_id IS NOT NULL) + <> (replacement_task_id IS NOT NULL))) + ) + ); + CREATE INDEX IF NOT EXISTS idx_agent_inbox_delivery_resolutions_run + ON agent_inbox_delivery_resolutions(org_run_id, inbox_id); CREATE INDEX IF NOT EXISTS idx_agent_inbox_materializations_session ON agent_inbox_materializations(session_id, inbox_id); CREATE INDEX IF NOT EXISTS idx_agent_inbox_recipient_member_unread @@ -89,6 +111,10 @@ pub fn init_schema(conn: &Connection) -> SqliteResult<()> { SELECT 1 FROM agent_inbox inbox WHERE inbox.id=receipt.inbox_id AND inbox.read_at IS NULL + AND NOT EXISTS ( + SELECT 1 FROM agent_inbox_delivery_resolutions resolution + WHERE resolution.inbox_id=inbox.id + ) ) OR NOT EXISTS ( SELECT 1 FROM agent_messages message diff --git a/src-tauri/crates/agent-core/src/core/coordination/agent_inbox/store_drain.rs b/src-tauri/crates/agent-core/src/core/coordination/agent_inbox/store_drain.rs index 59616a324..2f102d7bc 100644 --- a/src-tauri/crates/agent-core/src/core/coordination/agent_inbox/store_drain.rs +++ b/src-tauri/crates/agent-core/src/core/coordination/agent_inbox/store_drain.rs @@ -8,6 +8,8 @@ use rusqlite::{params, OptionalExtension}; use database::db::{get_connection, with_sessions_writer}; +use crate::coordination::agent_org_payload_limits as limits; + #[cfg(test)] use super::record::AgentInboxRecord; use super::record::{row_to_record, AgentInboxBatch}; @@ -28,6 +30,10 @@ impl AgentInboxStore { WHERE recipient_member_id = ?1 AND org_run_id = ?2 AND read_at IS NULL + AND NOT EXISTS ( + SELECT 1 FROM agent_inbox_delivery_resolutions resolution + WHERE resolution.inbox_id=agent_inbox.id + ) )", params![recipient_member_id, org_run_id], |row| row.get::<_, bool>(0), @@ -58,6 +64,10 @@ impl AgentInboxStore { WHERE recipient_member_id = ?1 AND org_run_id = ?2 AND read_at IS NULL + AND NOT EXISTS ( + SELECT 1 FROM agent_inbox_delivery_resolutions resolution + WHERE resolution.inbox_id=agent_inbox.id + ) ORDER BY id ASC", ) .map_err(|err| err.to_string())?; @@ -83,7 +93,11 @@ impl AgentInboxStore { "SELECT MAX(id) FROM agent_inbox WHERE recipient_member_id=?1 AND org_run_id=?2 - AND read_at IS NULL", + AND read_at IS NULL + AND NOT EXISTS ( + SELECT 1 FROM agent_inbox_delivery_resolutions resolution + WHERE resolution.inbox_id=agent_inbox.id + )", params![recipient_member_id, org_run_id], |row| row.get(0), ) @@ -105,7 +119,11 @@ impl AgentInboxStore { WHERE recipient_member_id=?1 AND org_run_id=?2 AND id<=?3 - AND read_at IS NULL", + AND read_at IS NULL + AND NOT EXISTS ( + SELECT 1 FROM agent_inbox_delivery_resolutions resolution + WHERE resolution.inbox_id=agent_inbox.id + )", params![recipient_member_id, org_run_id, boundary_id], |row| row.get(0), ) @@ -130,8 +148,20 @@ impl AgentInboxStore { sender_agent_id, sender_member_id, org_run_id, - payload_kind, - payload_json, + CASE WHEN length(CAST(payload_json AS BLOB))<=?4 + THEN payload_kind ELSE 'oversized_payload' END, + CASE WHEN length(CAST(payload_json AS BLOB))<=?4 + THEN payload_json + ELSE json_object( + 'kind', 'plain', + 'summary', 'Oversized historical inbox payload', + 'text', printf( + 'Inbox row %d contained %d bytes, above the supported delivery limit. The original row remains durable; this bounded diagnostic replaces its body. Raw prefix: %s', + id, + length(CAST(payload_json AS BLOB)), + substr(payload_json,1,4096) + ) + ) END, request_id, created_at, read_at @@ -139,6 +169,10 @@ impl AgentInboxStore { WHERE recipient_member_id = ?1 AND org_run_id = ?2 AND read_at IS NULL + AND NOT EXISTS ( + SELECT 1 FROM agent_inbox_delivery_resolutions resolution + WHERE resolution.inbox_id=agent_inbox.id + ) ORDER BY id ASC LIMIT ?3", ) @@ -149,6 +183,7 @@ impl AgentInboxStore { recipient_member_id, org_run_id, (MAX_INBOX_DRAIN_ROWS + 1) as i64, + limits::AGENT_INBOX_PAYLOAD_MAX_BYTES as i64, ], row_to_record, ) @@ -223,18 +258,23 @@ impl AgentInboxStore { FROM agent_inbox_materializations receipt WHERE receipt.inbox_id=agent_inbox.id AND receipt.session_id=?2 + ), + EXISTS( + SELECT 1 + FROM agent_inbox_delivery_resolutions resolution + WHERE resolution.inbox_id=agent_inbox.id ) FROM agent_inbox WHERE id=?1", ) .map_err(|err| err.to_string())?; for id in ids { - let state: Option<(Option, bool)> = preflight + let state: Option<(Option, bool, bool)> = preflight .query_row(params![id, session_id], |row| { - Ok((row.get(0)?, row.get(1)?)) + Ok((row.get(0)?, row.get(1)?, row.get(2)?)) }) .optional() .map_err(|err| err.to_string())?; - if matches!(state, Some((None, false))) { + if matches!(state, Some((None, false, false))) { return Err(format!( "Agent Org Inbox row {id} has no materialization receipt owned by session {session_id}; refusing partial acknowledgement" )); @@ -245,6 +285,10 @@ impl AgentInboxStore { "UPDATE agent_inbox SET read_at=?1 WHERE id=?2 AND read_at IS NULL + AND NOT EXISTS ( + SELECT 1 FROM agent_inbox_delivery_resolutions resolution + WHERE resolution.inbox_id=agent_inbox.id + ) AND EXISTS ( SELECT 1 FROM agent_inbox_materializations receipt WHERE receipt.inbox_id=agent_inbox.id @@ -277,7 +321,11 @@ impl AgentInboxStore { .prepare( "UPDATE agent_inbox SET read_at=?1 - WHERE id=?2 AND read_at IS NULL", + WHERE id=?2 AND read_at IS NULL + AND NOT EXISTS ( + SELECT 1 FROM agent_inbox_delivery_resolutions resolution + WHERE resolution.inbox_id=agent_inbox.id + )", ) .map_err(|err| err.to_string())?; for id in ids { diff --git a/src-tauri/crates/agent-core/src/core/coordination/agent_inbox/store_read.rs b/src-tauri/crates/agent-core/src/core/coordination/agent_inbox/store_read.rs index 13746e2de..7bb2fc3bd 100644 --- a/src-tauri/crates/agent-core/src/core/coordination/agent_inbox/store_read.rs +++ b/src-tauri/crates/agent-core/src/core/coordination/agent_inbox/store_read.rs @@ -6,15 +6,16 @@ use rusqlite::{params, Connection, OptionalExtension}; use std::collections::HashSet; use crate::coordination::agent_org_payload_limits as limits; -use database::db::get_connection; +use database::db::{get_connection, with_sessions_writer}; use super::message::AgentMessage; +#[cfg(test)] +use super::record::AgentInboxRecipientCounts; use super::record::{ - row_to_preview_record, row_to_record, AgentInboxPage, AgentInboxPreviewRecord, - AgentInboxUnreadRecipientCounts, + row_to_preview_record, row_to_record, AgentInboxDeliveryResolution, + AgentInboxDeliveryResolutionKind, AgentInboxPage, AgentInboxPreviewRecord, AgentInboxRecord, + AgentInboxUnreadRecipientCounts, ResolveInboxDeliveryError, ResolveInboxDeliveryParams, }; -#[cfg(test)] -use super::record::{AgentInboxRecipientCounts, AgentInboxRecord}; use super::{ AgentInboxStore, MAX_INBOX_HISTORY_PAGE_BYTES, MAX_INBOX_HISTORY_PAGE_ROWS, MAX_RUN_INBOX_PREVIEW_CHARS, MAX_RUN_INBOX_SNAPSHOT_ROWS, @@ -27,6 +28,10 @@ pub(super) const UNREAD_COUNTS_BY_RECIPIENT_SQL: &str = "SELECT recipient_agent_ FROM agent_inbox INDEXED BY idx_agent_inbox_run_unread_recipient WHERE org_run_id = ?1 AND read_at IS NULL + AND NOT EXISTS ( + SELECT 1 FROM agent_inbox_delivery_resolutions resolution + WHERE resolution.inbox_id=agent_inbox.id + ) GROUP BY recipient_member_id, recipient_agent_id ORDER BY recipient_member_id ASC, recipient_agent_id ASC"; @@ -57,6 +62,110 @@ pub(super) fn task_assignment_lookup_sql() -> String { ) } +fn inbox_recipient_is_permanently_unavailable( + conn: &Connection, + org_run_id: &str, + recipient_member_id: Option<&str>, +) -> Result { + use crate::coordination::agent_org_runs::{AgentOrgRunStore, COORDINATOR_MEMBER_ID}; + use crate::session::SessionStatus; + + let Some(member_id) = recipient_member_id.filter(|value| !value.trim().is_empty()) else { + return Ok(true); + }; + let roster = AgentOrgRunStore::snapshot_member_ids_with_connection(conn, org_run_id)?; + if member_id != COORDINATOR_MEMBER_ID + && roster + .as_ref() + .is_some_and(|members| !members.contains(member_id)) + { + return Ok(true); + } + if member_id == COORDINATOR_MEMBER_ID { + return Ok( + AgentOrgRunStore::find_coordinator_session_by_member_id_with_connection( + conn, + org_run_id, + COORDINATOR_MEMBER_ID, + )? + .is_none_or(|runtime| runtime.status == SessionStatus::Archived), + ); + } + Ok( + AgentOrgRunStore::list_descendant_worker_sessions_with_connection(conn, org_run_id)? + .into_iter() + .find(|runtime| runtime.member_id.as_deref() == Some(member_id)) + .is_none_or(|runtime| { + runtime.cli_agent_type.is_some() || runtime.status == SessionStatus::Archived + }), + ) +} + +fn load_delivery_resolution( + conn: &Connection, + org_run_id: &str, + inbox_id: i64, +) -> Result, String> { + type DeliveryResolutionRow = ( + i64, + String, + String, + String, + String, + Option, + Option, + String, + ); + let raw: Option = conn + .query_row( + "SELECT inbox_id, org_run_id, resolution_kind, + resolved_by_member_id, reason, + replacement_inbox_id, replacement_task_id, created_at + FROM agent_inbox_delivery_resolutions + WHERE inbox_id=?1 AND org_run_id=?2 + LIMIT 1", + params![inbox_id, org_run_id], + |row| { + Ok(( + row.get(0)?, + row.get(1)?, + row.get(2)?, + row.get(3)?, + row.get(4)?, + row.get(5)?, + row.get(6)?, + row.get(7)?, + )) + }, + ) + .optional() + .map_err(|err| err.to_string())?; + raw.map( + |( + inbox_id, + org_run_id, + resolution_kind, + resolved_by_member_id, + reason, + replacement_inbox_id, + replacement_task_id, + created_at, + )| { + Ok(AgentInboxDeliveryResolution { + inbox_id, + org_run_id, + resolution_kind: AgentInboxDeliveryResolutionKind::parse(&resolution_kind)?, + resolved_by_member_id, + reason, + replacement_inbox_id, + replacement_task_id, + created_at, + }) + }, + ) + .transpose() +} + impl AgentInboxStore { /// Test-only convenience wrapper for assertions that intentionally seed a /// tiny number of rows. Production and debug paths use bounded pages. @@ -104,16 +213,38 @@ impl AgentInboxStore { let mut stmt = conn .prepare( "SELECT id, - recipient_agent_id, - recipient_member_id, - sender_agent_id, - sender_member_id, + CASE WHEN length(CAST(recipient_agent_id AS BLOB))<=?4 + THEN recipient_agent_id + ELSE printf('[oversized recipient agent row %d]', id) END, + CASE WHEN recipient_member_id IS NULL THEN NULL + WHEN length(CAST(recipient_member_id AS BLOB))<=?4 + THEN recipient_member_id + ELSE printf('[oversized recipient member row %d]', id) END, + CASE WHEN length(CAST(sender_agent_id AS BLOB))<=?4 + THEN sender_agent_id + ELSE printf('[oversized sender agent row %d]', id) END, + CASE WHEN sender_member_id IS NULL THEN NULL + WHEN length(CAST(sender_member_id AS BLOB))<=?4 + THEN sender_member_id + ELSE printf('[oversized sender member row %d]', id) END, ?1, - payload_kind, - payload_json, - request_id, - created_at, - read_at + CASE WHEN length(CAST(payload_json AS BLOB))<=?5 + THEN substr(payload_kind,1,128) ELSE 'oversized_payload' END, + CASE WHEN length(CAST(payload_json AS BLOB))<=?5 + THEN payload_json + ELSE json_object( + 'kind', 'plain', + 'summary', 'Oversized historical inbox payload', + 'text', printf( + 'Inbox row %d contained %d bytes, above the supported delivery limit. The original row remains durable; this bounded diagnostic replaces its body. Raw prefix: %s', + id, + length(CAST(payload_json AS BLOB)), + substr(payload_json,1,4096) + ) + ) END, + CASE WHEN request_id IS NULL THEN NULL ELSE substr(request_id,1,1000) END, + substr(created_at,1,64), + CASE WHEN read_at IS NULL THEN NULL ELSE substr(read_at,1,64) END FROM agent_inbox WHERE org_run_id=?1 AND id>?2 ORDER BY id ASC @@ -126,6 +257,8 @@ impl AgentInboxStore { org_run_id, after_id.unwrap_or(0), (bounded_limit + 1) as i64, + limits::MESSAGE_IDENTIFIER_MAX_BYTES as i64, + limits::AGENT_INBOX_PAYLOAD_MAX_BYTES as i64, ], row_to_record, ) @@ -178,6 +311,322 @@ impl AgentInboxStore { usize::try_from(count).map_err(|_| format!("invalid Inbox row count: {count}")) } + /// Read one durable Inbox row for an explicit repair/diagnostic action. + /// The stored row is never mutated and oversized historical payloads are + /// replaced in the response by the same bounded diagnostic used by the + /// cursor history API. + pub fn get_by_id_for_run( + org_run_id: &str, + inbox_id: i64, + ) -> Result, String> { + if inbox_id <= 0 { + return Err("inbox_id must be a positive integer".to_string()); + } + let conn = get_connection().map_err(|err| err.to_string())?; + conn.query_row( + "SELECT id, + CASE WHEN length(CAST(recipient_agent_id AS BLOB))<=?3 + THEN recipient_agent_id + ELSE printf('[oversized recipient agent row %d]', id) END, + CASE WHEN recipient_member_id IS NULL THEN NULL + WHEN length(CAST(recipient_member_id AS BLOB))<=?3 + THEN recipient_member_id + ELSE printf('[oversized recipient member row %d]', id) END, + CASE WHEN length(CAST(sender_agent_id AS BLOB))<=?3 + THEN sender_agent_id + ELSE printf('[oversized sender agent row %d]', id) END, + CASE WHEN sender_member_id IS NULL THEN NULL + WHEN length(CAST(sender_member_id AS BLOB))<=?3 + THEN sender_member_id + ELSE printf('[oversized sender member row %d]', id) END, + ?1, + CASE WHEN length(CAST(payload_json AS BLOB))<=?4 + THEN substr(payload_kind,1,128) ELSE 'oversized_payload' END, + CASE WHEN length(CAST(payload_json AS BLOB))<=?4 + THEN payload_json + ELSE json_object( + 'kind', 'plain', + 'summary', 'Oversized historical inbox payload', + 'text', printf( + 'Inbox row %d contained %d bytes, above the supported delivery limit. The original row remains durable; this bounded diagnostic replaces its body. Raw prefix: %s', + id, + length(CAST(payload_json AS BLOB)), + substr(payload_json,1,4096) + ) + ) END, + CASE WHEN request_id IS NULL THEN NULL ELSE substr(request_id,1,1000) END, + substr(created_at,1,64), + CASE WHEN read_at IS NULL THEN NULL ELSE substr(read_at,1,64) END + FROM agent_inbox + WHERE org_run_id=?1 AND id=?2 + LIMIT 1", + params![ + org_run_id, + inbox_id, + limits::MESSAGE_IDENTIFIER_MAX_BYTES as i64, + limits::AGENT_INBOX_PAYLOAD_MAX_BYTES as i64, + ], + row_to_record, + ) + .optional() + .map_err(|err| err.to_string()) + } + + pub fn delivery_resolution_for_inbox( + org_run_id: &str, + inbox_id: i64, + ) -> Result, String> { + let conn = get_connection().map_err(|err| err.to_string())?; + load_delivery_resolution(&conn, org_run_id, inbox_id) + } + + /// Resolve an otherwise-undeliverable source row without falsifying its + /// read receipt. Only the canonical coordinator may call this store path; + /// the LLM tool also enforces that authority before entering the blocking + /// transaction. + pub fn resolve_delivery( + params: ResolveInboxDeliveryParams, + ) -> Result { + use crate::coordination::agent_org_runs::{ + AgentOrgRunStatus, AgentOrgRunStore, COORDINATOR_MEMBER_ID, + }; + + let constraint = ResolveInboxDeliveryError::Constraint; + let storage = ResolveInboxDeliveryError::Storage; + + if params.inbox_id <= 0 { + return Err(constraint( + "inbox_id must be a positive integer".to_string(), + )); + } + limits::validate_message_identifier("org_run_id", ¶ms.org_run_id) + .map_err(constraint)?; + limits::validate_message_identifier("resolved_by_member_id", ¶ms.resolved_by_member_id) + .map_err(constraint)?; + if params.resolved_by_member_id != COORDINATOR_MEMBER_ID { + return Err(constraint( + "Inbox delivery resolution is coordinator-only".to_string(), + )); + } + limits::validate_required_text( + "reason", + ¶ms.reason, + limits::PLAN_FEEDBACK_MAX_CHARS, + limits::PLAN_FEEDBACK_MAX_BYTES, + ) + .map_err(constraint)?; + if let Some(task_id) = params.replacement_task_id.as_deref() { + limits::validate_task_identifier("replacement_task_id", task_id).map_err(constraint)?; + } + if params.replacement_inbox_id.is_some_and(|id| id <= 0) { + return Err(constraint( + "replacement_inbox_id must be a positive integer".to_string(), + )); + } + match params.resolution_kind { + AgentInboxDeliveryResolutionKind::Cancelled => { + if params.replacement_inbox_id.is_some() || params.replacement_task_id.is_some() { + return Err(constraint( + "cancelled delivery resolution cannot name a replacement".to_string(), + )); + } + } + AgentInboxDeliveryResolutionKind::Superseded => { + if params.replacement_inbox_id.is_some() == params.replacement_task_id.is_some() { + return Err(constraint( + "superseded delivery resolution requires exactly one of replacement_inbox_id or replacement_task_id" + .to_string(), + )); + } + if params.replacement_inbox_id == Some(params.inbox_id) { + return Err(constraint( + "an Inbox row cannot supersede itself".to_string(), + )); + } + } + } + + with_sessions_writer( + || -> Result { + let mut conn = get_connection().map_err(|err| storage(err.to_string()))?; + let tx = conn + .transaction_with_behavior(rusqlite::TransactionBehavior::Immediate) + .map_err(|err| storage(err.to_string()))?; + let run_status = + AgentOrgRunStore::get_run_status_with_connection(&tx, ¶ms.org_run_id) + .map_err(storage)?; + if run_status != Some(AgentOrgRunStatus::Running) { + return Err(constraint(format!( + "Agent Org run {} is not Running; Inbox delivery repair was not applied", + params.org_run_id + ))); + } + + let source: Option<(Option, Option)> = tx + .query_row( + "SELECT recipient_member_id, read_at + FROM agent_inbox + WHERE id=?1 AND org_run_id=?2 + LIMIT 1", + params![params.inbox_id, ¶ms.org_run_id], + |row| Ok((row.get(0)?, row.get(1)?)), + ) + .optional() + .map_err(|err| storage(err.to_string()))?; + let Some((source_recipient_member_id, read_at)) = source else { + return Err(constraint(format!( + "Inbox row {} does not belong to Agent Org run {}", + params.inbox_id, params.org_run_id + ))); + }; + if read_at.is_some() { + return Err(constraint(format!( + "Inbox row {} was already delivered and cannot be resolved as undeliverable", + params.inbox_id + ))); + } + + if let Some(existing) = + load_delivery_resolution(&tx, ¶ms.org_run_id, params.inbox_id) + .map_err(storage)? + { + let is_same = existing.resolution_kind == params.resolution_kind + && existing.resolved_by_member_id == params.resolved_by_member_id + && existing.reason == params.reason + && existing.replacement_inbox_id == params.replacement_inbox_id + && existing.replacement_task_id == params.replacement_task_id; + if is_same { + tx.commit().map_err(|err| storage(err.to_string()))?; + return Ok(existing); + } + return Err(constraint(format!( + "Inbox row {} already has a different delivery resolution", + params.inbox_id + ))); + } + + // A model-visible repair tool must not be able to discard healthy + // work merely because the coordinator changed its mind. Only + // identities that are provably outside a deliverable production + // path may be resolved here. Recoverable states (Idle, terminal + // retry candidates, Pending, Paused, Running/waiting) must instead + // be resumed/retried or explicitly archived by the user first. + let permanently_unavailable = inbox_recipient_is_permanently_unavailable( + &tx, + ¶ms.org_run_id, + source_recipient_member_id.as_deref(), + ) + .map_err(storage)?; + if !permanently_unavailable { + return Err(constraint(format!( + "Inbox row {} still has a recoverable canonical recipient. Resume/retry that recipient instead of discarding or superseding healthy delivery; archive it explicitly first only if the user has decided it is permanently unavailable.", + params.inbox_id + ))); + } + + if let Some(replacement_inbox_id) = params.replacement_inbox_id { + let replacement: Option<(Option, Option, bool)> = tx + .query_row( + "SELECT inbox.recipient_member_id, + inbox.read_at, + EXISTS( + SELECT 1 + FROM agent_inbox_delivery_resolutions resolution + WHERE resolution.inbox_id=inbox.id + ) + FROM agent_inbox inbox + WHERE inbox.id=?1 AND inbox.org_run_id=?2 + LIMIT 1", + params![replacement_inbox_id, ¶ms.org_run_id], + |row| Ok((row.get(0)?, row.get(1)?, row.get(2)?)), + ) + .optional() + .map_err(|err| storage(err.to_string()))?; + let Some((Some(replacement_member_id), replacement_read_at, is_resolved)) = + replacement + else { + return Err(constraint(format!( + "replacement Inbox row {replacement_inbox_id} must exist in the same run and name a canonical recipient_member_id" + ))); + }; + if is_resolved { + return Err(constraint(format!( + "replacement Inbox row {replacement_inbox_id} already has a delivery resolution and cannot be used as a live replacement" + ))); + } + let replacement_is_unavailable = inbox_recipient_is_permanently_unavailable( + &tx, + ¶ms.org_run_id, + Some(&replacement_member_id), + ) + .map_err(storage)?; + if replacement_read_at.is_none() && replacement_is_unavailable { + return Err(constraint(format!( + "replacement Inbox row {replacement_inbox_id} has not been delivered and its recipient {replacement_member_id:?} is permanently unavailable" + ))); + } + } + if let Some(replacement_task_id) = params.replacement_task_id.as_deref() { + let replacement_exists: bool = tx + .query_row( + "SELECT EXISTS( + SELECT 1 FROM agent_org_tasks + WHERE id=?1 AND org_run_id=?2 + )", + params![replacement_task_id, ¶ms.org_run_id], + |row| row.get(0), + ) + .map_err(|err| storage(err.to_string()))?; + if !replacement_exists { + return Err(constraint(format!( + "replacement task {replacement_task_id:?} does not exist in Agent Org run {}", + params.org_run_id + ))); + } + } + + let created_at = chrono::Utc::now().to_rfc3339(); + tx.execute( + "INSERT INTO agent_inbox_delivery_resolutions ( + inbox_id, org_run_id, resolution_kind, + resolved_by_member_id, reason, + replacement_inbox_id, replacement_task_id, created_at + ) VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8)", + params![ + params.inbox_id, + ¶ms.org_run_id, + params.resolution_kind.as_str(), + ¶ms.resolved_by_member_id, + ¶ms.reason, + params.replacement_inbox_id, + params.replacement_task_id.as_deref(), + &created_at, + ], + ) + .map_err(|err| storage(err.to_string()))?; + // A Session that materialized the old row before this repair must + // not later acknowledge it as delivered. The guarded mark-read + // path also rechecks the resolution table. + tx.execute( + "DELETE FROM agent_inbox_materializations WHERE inbox_id=?1", + params![params.inbox_id], + ) + .map_err(|err| storage(err.to_string()))?; + tx.commit().map_err(|err| storage(err.to_string()))?; + Ok(AgentInboxDeliveryResolution { + inbox_id: params.inbox_id, + org_run_id: params.org_run_id, + resolution_kind: params.resolution_kind, + resolved_by_member_id: params.resolved_by_member_id, + reason: params.reason, + replacement_inbox_id: params.replacement_inbox_id, + replacement_task_id: params.replacement_task_id, + created_at, + }) + }, + ) + } + /// Return a bounded tail of one run's inbox history in chronological /// order. The inner descending query lets SQLite stop at `limit`; the /// outer query restores the order expected by transcript projections. @@ -199,8 +648,20 @@ impl AgentInboxStore { sender_agent_id, sender_member_id, org_run_id, - payload_kind, - payload_json, + CASE WHEN length(CAST(payload_json AS BLOB))<=?2 + THEN payload_kind ELSE 'oversized_payload' END, + CASE WHEN length(CAST(payload_json AS BLOB))<=?2 + THEN payload_json + ELSE json_object( + 'kind', 'plain', + 'summary', 'Oversized historical inbox payload', + 'text', printf( + 'Inbox row %d contained %d bytes, above the supported delivery limit. The original row remains durable; this bounded diagnostic replaces its body. Raw prefix: %s', + id, + length(CAST(payload_json AS BLOB)), + substr(payload_json,1,4096) + ) + ) END, request_id, created_at, read_at @@ -271,7 +732,8 @@ impl AgentInboxStore { request_id, created_at, read_at, - display_preview + display_preview, + delivery_resolution FROM ( SELECT id, recipient_agent_id, @@ -323,7 +785,13 @@ impl AgentInboxStore { ) ELSE NULL END ELSE NULL END - ELSE NULL END AS display_preview + ELSE NULL END AS display_preview, + ( + SELECT resolution.resolution_kind + FROM agent_inbox_delivery_resolutions resolution + WHERE resolution.inbox_id=agent_inbox.id + LIMIT 1 + ) AS delivery_resolution FROM agent_inbox WHERE org_run_id = ?1 ORDER BY id DESC @@ -370,7 +838,13 @@ impl AgentInboxStore { "SELECT recipient_agent_id, recipient_member_id, COUNT(*) AS activity_count, - SUM(CASE WHEN read_at IS NULL THEN 1 ELSE 0 END) AS unread_count + SUM(CASE WHEN read_at IS NULL + AND NOT EXISTS ( + SELECT 1 + FROM agent_inbox_delivery_resolutions resolution + WHERE resolution.inbox_id=agent_inbox.id + ) + THEN 1 ELSE 0 END) AS unread_count FROM agent_inbox WHERE org_run_id = ?1 GROUP BY recipient_member_id, recipient_agent_id @@ -423,6 +897,35 @@ impl AgentInboxStore { Ok(out) } + /// Load only the durable task ids that have ever received a TaskAssigned + /// envelope in this run. Recovery uses this instead of decoding the full + /// inbox history in Rust. + #[cfg(test)] + pub(super) fn task_assignment_ids_by_run(org_run_id: &str) -> Result, String> { + let conn = get_connection().map_err(|err| err.to_string())?; + let mut stmt = conn + .prepare( + "SELECT DISTINCT json_extract(payload_json, '$.task_id') + FROM agent_inbox + WHERE org_run_id=?1 + AND payload_kind='task_assigned' + AND json_valid(payload_json) + AND json_type(payload_json, '$.task_id')='text'", + ) + .map_err(|err| err.to_string())?; + let rows = stmt + .query_map(params![org_run_id], |row| row.get::<_, String>(0)) + .map_err(|err| err.to_string())?; + let mut task_ids = HashSet::new(); + for row in rows { + let task_id = row.map_err(|err| err.to_string())?; + if !task_id.trim().is_empty() { + task_ids.insert(task_id); + } + } + Ok(task_ids) + } + /// Return only current open task ids whose *current owner* has a valid, /// durable `TaskAssigned` envelope. The expression index turns this into /// bounded lookups from the current task board instead of re-running @@ -434,7 +937,6 @@ impl AgentInboxStore { conn: &Connection, org_run_id: &str, ) -> Result, String> { - const TASK_PAGE_SIZE: i64 = 200; let mut task_stmt = conn .prepare( "SELECT id, owner @@ -442,55 +944,54 @@ impl AgentInboxStore { WHERE org_run_id=?1 AND status IN ('pending','in_progress') AND owner IS NOT NULL - AND id>?2 ORDER BY id ASC - LIMIT ?3", + LIMIT ?2", + ) + .map_err(|err| err.to_string())?; + let open_tasks = task_stmt + .query_map( + params![ + org_run_id, + (crate::coordination::agent_org_payload_limits::TASK_RUN_MAX_TASKS + 1) as i64, + ], + |row| Ok((row.get::<_, String>(0)?, row.get::<_, String>(1)?)), ) + .map_err(|err| err.to_string())? + .collect::, _>>() .map_err(|err| err.to_string())?; + if open_tasks.len() > crate::coordination::agent_org_payload_limits::TASK_RUN_MAX_TASKS { + return Err( + "Agent Org task board exceeds the supported assignment snapshot limit".to_string(), + ); + } - // Page the current board in bounded reads. SQLite does not bind an - // outer task.id into the third expression-index column, so one reused - // exact probe per current task is faster than scanning historical - // Inbox JSON and does not impose a run-level task capacity policy. + // At most 200 exact probes are preferable to one nominally joined + // query here. SQLite does not bind an outer task.id into the third + // expression-index column and otherwise scans historical Inbox rows + // plus a temp sort. One reused prepared statement with ?3 uses all + // `(run, member, task_id)` keys and traverses rowid newest-first. let lookup_sql = task_assignment_lookup_sql(); let mut assignment_stmt = conn.prepare(&lookup_sql).map_err(|err| err.to_string())?; let mut task_ids = HashSet::new(); - let mut after_task_id = String::new(); - loop { - let open_tasks = task_stmt - .query_map(params![org_run_id, &after_task_id, TASK_PAGE_SIZE], |row| { - Ok((row.get::<_, String>(0)?, row.get::<_, String>(1)?)) + for (task_id, owner) in open_tasks { + let payload_json = assignment_stmt + .query_row(params![org_run_id, &owner, &task_id], |row| { + row.get::<_, String>(0) }) - .map_err(|err| err.to_string())? - .collect::, _>>() + .optional() .map_err(|err| err.to_string())?; - if open_tasks.is_empty() { - break; + let Some(payload_json) = payload_json else { + continue; + }; + let Ok(message) = serde_json::from_str::(&payload_json) else { + continue; + }; + if message.validate().is_err() { + continue; } - after_task_id = open_tasks - .last() - .map(|(task_id, _)| task_id.clone()) - .unwrap_or_default(); - for (task_id, owner) in open_tasks { - let payload_json = assignment_stmt - .query_row(params![org_run_id, &owner, &task_id], |row| { - row.get::<_, String>(0) - }) - .optional() - .map_err(|err| err.to_string())?; - let Some(payload_json) = payload_json else { - continue; - }; - let Ok(message) = serde_json::from_str::(&payload_json) else { - continue; - }; - if message.validate().is_err() { - continue; - } - if matches!(message, AgentMessage::TaskAssigned { task_id: ref id, .. } if id == &task_id) - { - task_ids.insert(task_id); - } + if matches!(message, AgentMessage::TaskAssigned { task_id: ref id, .. } if id == &task_id) + { + task_ids.insert(task_id); } } Ok(task_ids) @@ -520,7 +1021,11 @@ impl AgentInboxStore { FROM agent_inbox WHERE recipient_member_id=?1 AND org_run_id=?2 - AND read_at IS NULL", + AND read_at IS NULL + AND NOT EXISTS ( + SELECT 1 FROM agent_inbox_delivery_resolutions resolution + WHERE resolution.inbox_id=agent_inbox.id + )", params![recipient_member_id, org_run_id], |row| Ok((row.get(0)?, row.get(1)?)), ) diff --git a/src-tauri/crates/agent-core/src/core/coordination/agent_inbox/store_write.rs b/src-tauri/crates/agent-core/src/core/coordination/agent_inbox/store_write.rs index f51af1881..41225e0dc 100644 --- a/src-tauri/crates/agent-core/src/core/coordination/agent_inbox/store_write.rs +++ b/src-tauri/crates/agent-core/src/core/coordination/agent_inbox/store_write.rs @@ -6,7 +6,6 @@ use rusqlite::{params, Connection}; use crate::coordination::agent_org_payload_limits as limits; use database::db::{get_connection, with_sessions_writer}; -use super::message::validate_non_empty_identifier; use super::record::row_to_record; use super::{AgentInboxRecord, AgentInboxStore, InsertInboxParams}; @@ -99,15 +98,30 @@ impl AgentInboxStore { params: InsertInboxParams, causation_inbox_id: Option, ) -> Result<(AgentInboxRecord, bool), String> { - validate_non_empty_identifier("recipient_agent_id", ¶ms.recipient_agent_id)?; - validate_non_empty_identifier("sender_agent_id", ¶ms.sender_agent_id)?; + limits::validate_required_text( + "recipient_agent_id", + ¶ms.recipient_agent_id, + limits::MESSAGE_IDENTIFIER_MAX_CHARS, + limits::MESSAGE_IDENTIFIER_MAX_BYTES, + )?; + limits::validate_required_text( + "sender_agent_id", + ¶ms.sender_agent_id, + limits::MESSAGE_IDENTIFIER_MAX_CHARS, + limits::MESSAGE_IDENTIFIER_MAX_BYTES, + )?; for (field, value) in [ ("recipient_member_id", params.recipient_member_id.as_deref()), ("sender_member_id", params.sender_member_id.as_deref()), ("org_run_id", params.org_run_id.as_deref()), ] { if let Some(value) = value { - validate_non_empty_identifier(field, value)?; + limits::validate_required_text( + field, + value, + limits::MESSAGE_IDENTIFIER_MAX_CHARS, + limits::MESSAGE_IDENTIFIER_MAX_BYTES, + )?; } } if params.org_run_id.is_some() && params.recipient_member_id.is_none() { diff --git a/src-tauri/crates/agent-core/src/core/coordination/agent_inbox/tests.rs b/src-tauri/crates/agent-core/src/core/coordination/agent_inbox/tests.rs index 6a9286711..44e9b1d6b 100644 --- a/src-tauri/crates/agent-core/src/core/coordination/agent_inbox/tests.rs +++ b/src-tauri/crates/agent-core/src/core/coordination/agent_inbox/tests.rs @@ -341,6 +341,59 @@ fn caused_inbox_insert_coalesces_only_the_same_recipient_member() { ); } +#[test] +fn preview_and_assignment_scan_tolerate_corrupt_historical_payloads() { + let _sandbox = sandbox_with_inbox_schema(); + let run_id = format!("run-{}", uuid::Uuid::new_v4()); + let conn = get_connection().expect("test database"); + for (kind, payload) in [ + ("plain", "{not valid json"), + ("task_assigned", "also-not-json"), + ] { + conn.execute( + "INSERT INTO agent_inbox ( + recipient_agent_id, recipient_member_id, sender_agent_id, + org_run_id, payload_kind, payload_json, created_at + ) VALUES ('worker', 'member-worker', 'sender', ?1, ?2, ?3, ?4)", + params![&run_id, kind, payload, chrono::Utc::now().to_rfc3339()], + ) + .expect("seed corrupt historical inbox row"); + } + conn.execute_batch("DROP INDEX idx_agent_inbox_run_task_assignment_v4") + .expect("drop assignment index to simulate upgrade"); + init_schema(&conn).expect("schema upgrade tolerates corrupt historical payloads"); + AgentInboxStore::insert(InsertInboxParams { + recipient_agent_id: "worker".into(), + recipient_member_id: Some("member-worker".into()), + sender_agent_id: "sender".into(), + sender_member_id: None, + org_run_id: Some(run_id.clone()), + message: AgentMessage::TaskAssigned { + task_id: "valid-task".into(), + subject: "Valid assignment".into(), + description: String::new(), + assigned_by: "Coordinator".into(), + dependency_outputs: Vec::new(), + execution_mode: crate::coordination::agent_org_tasks::TaskExecutionMode::Build, + }, + }) + .expect("insert valid assignment"); + + let previews = AgentInboxStore::list_recent_previews_by_run(&run_id, 10) + .expect("corrupt rows degrade to empty previews"); + assert_eq!(previews.len(), 3); + assert!(previews[0].display_preview.is_none()); + assert!(previews[1].display_preview.is_none()); + assert_eq!( + previews[2].display_preview.as_deref(), + Some("Valid assignment") + ); + + let assigned = AgentInboxStore::task_assignment_ids_by_run(&run_id) + .expect("corrupt assignment payload is skipped"); + assert_eq!(assigned, HashSet::from(["valid-task".to_string()])); +} + #[test] fn open_assignment_snapshot_uses_current_tasks_and_expression_index() { let _sandbox = sandbox_with_inbox_schema(); diff --git a/src-tauri/crates/agent-core/src/core/coordination/agent_member_interventions.rs b/src-tauri/crates/agent-core/src/core/coordination/agent_member_interventions.rs index 985e43e9e..0a029657e 100644 --- a/src-tauri/crates/agent-core/src/core/coordination/agent_member_interventions.rs +++ b/src-tauri/crates/agent-core/src/core/coordination/agent_member_interventions.rs @@ -208,7 +208,11 @@ impl AgentMemberInterventionStore { "SELECT MAX(id) FROM agent_inbox WHERE recipient_member_id=?1 AND org_run_id=?2 - AND read_at IS NULL", + AND read_at IS NULL + AND NOT EXISTS ( + SELECT 1 FROM agent_inbox_delivery_resolutions resolution + WHERE resolution.inbox_id=agent_inbox.id + )", params![member_id, org_run_id], |row| row.get(0), ) diff --git a/src-tauri/crates/agent-core/src/core/coordination/agent_org_payload_limits.rs b/src-tauri/crates/agent-core/src/core/coordination/agent_org_payload_limits.rs index b3afb169d..29f04de5f 100644 --- a/src-tauri/crates/agent-core/src/core/coordination/agent_org_payload_limits.rs +++ b/src-tauri/crates/agent-core/src/core/coordination/agent_org_payload_limits.rs @@ -12,6 +12,7 @@ pub const TASK_DESCRIPTION_MAX_BYTES: usize = TASK_DESCRIPTION_MAX_CHARS * 4; /// frequently-polled Run View. Full durable descriptions remain available via /// `task_get`. pub const TASK_SUMMARY_DESCRIPTION_MAX_CHARS: usize = 512; +pub const TASK_SUMMARY_ID_PREVIEW_MAX_CHARS: usize = 256; pub const TASK_SUMMARY_DEPENDENCY_PREVIEW_MAX_COUNT: usize = 8; pub const TASK_SUMMARY_ELIGIBILITY_PREVIEW_MAX_COUNT: usize = 16; pub const TASK_SUMMARY_ARTIFACT_PREVIEW_MAX_COUNT: usize = 16; @@ -19,9 +20,34 @@ pub const TASK_SUMMARY_PAGE_MAX_BYTES: usize = 512 * 1024; pub const TASK_OPEN_ID_PREVIEW_MAX_BYTES: usize = 16 * 1024; pub const TASK_ACTIVE_FORM_MAX_CHARS: usize = 1_000; pub const TASK_ACTIVE_FORM_MAX_BYTES: usize = TASK_ACTIVE_FORM_MAX_CHARS * 4; +/// A task identifier must remain small enough to cross every durable delivery +/// boundary, including `TaskAssigned` inbox messages. Existing historical +/// rows remain readable; all new task ids, dependency ids, and cursors are +/// bounded at their write/tool boundary. +pub const TASK_IDENTIFIER_MAX_CHARS: usize = MESSAGE_IDENTIFIER_MAX_CHARS; +pub const TASK_IDENTIFIER_MAX_BYTES: usize = MESSAGE_IDENTIFIER_MAX_BYTES; pub const TASK_METADATA_MAX_BYTES: usize = 64 * 1024; pub const TASK_REQUIRED_ROLE_MAX_CHARS: usize = 200; pub const TASK_REQUIRED_ROLE_MAX_BYTES: usize = TASK_REQUIRED_ROLE_MAX_CHARS * 4; +pub const TASK_DEPENDENCY_MAX_COUNT: usize = 128; +pub const TASK_DEPENDENCY_TOTAL_MAX_CHARS: usize = 8_000; +pub const TASK_DEPENDENCY_TOTAL_MAX_BYTES: usize = TASK_DEPENDENCY_TOTAL_MAX_CHARS * 4; +pub const TASK_ELIGIBILITY_MAX_COUNT: usize = 128; +pub const TASK_ELIGIBILITY_TOTAL_MAX_CHARS: usize = 8_000; +pub const TASK_ELIGIBILITY_TOTAL_MAX_BYTES: usize = TASK_ELIGIBILITY_TOTAL_MAX_CHARS * 4; +/// Serialized JSON can expand quotes/control characters beyond the decoded +/// identifier total. This is the pre-parse ceiling used for historical rows. +pub const TASK_DEPENDENCY_JSON_MAX_BYTES: usize = 256 * 1024; +pub const RFC3339_TIMESTAMP_MAX_CHARS: usize = 64; +pub const RFC3339_TIMESTAMP_MAX_BYTES: usize = RFC3339_TIMESTAMP_MAX_CHARS * 4; +/// Maximum number of durable task rows that one Agent Org run may retain. +/// This is a run-level storage boundary, not a recommendation for how many +/// tasks a coordinator should create in one model tool call. +pub const TASK_RUN_MAX_TASKS: usize = 200; +/// LLM-facing request limit for one atomic `task_graph_create` call. Keeping +/// this separate from [`TASK_RUN_MAX_TASKS`] avoids teaching coordinators to +/// create 200-node graphs merely because the database can retain that many. +pub const TASK_GRAPH_CREATE_MAX_TASKS: usize = 32; pub const PLAIN_SUMMARY_MAX_CHARS: usize = 200; pub const PLAIN_SUMMARY_MAX_BYTES: usize = PLAIN_SUMMARY_MAX_CHARS * 4; @@ -43,6 +69,11 @@ pub const TASK_OUTPUT_SUMMARY_MAX_CHARS: usize = 1_000; pub const TASK_OUTPUT_SUMMARY_MAX_BYTES: usize = TASK_OUTPUT_SUMMARY_MAX_CHARS * 4; pub const TASK_OUTPUT_CONTENT_MAX_CHARS: usize = 20_000; pub const TASK_OUTPUT_CONTENT_MAX_BYTES: usize = TASK_OUTPUT_CONTENT_MAX_CHARS * 4; +pub const TASK_ARTIFACT_ID_MAX_CHARS: usize = 1_000; +pub const TASK_ARTIFACT_ID_MAX_BYTES: usize = TASK_ARTIFACT_ID_MAX_CHARS * 4; +pub const TASK_ARTIFACT_ID_MAX_COUNT: usize = 64; +pub const TASK_ARTIFACT_IDS_TOTAL_MAX_CHARS: usize = 16_000; +pub const TASK_ARTIFACT_IDS_TOTAL_MAX_BYTES: usize = TASK_ARTIFACT_IDS_TOTAL_MAX_CHARS * 4; pub const MEMBER_SUMMARY_MAX_CHARS: usize = 500; pub const MEMBER_SUMMARY_MAX_BYTES: usize = MEMBER_SUMMARY_MAX_CHARS * 4; @@ -56,6 +87,8 @@ pub const EXEC_MODE_REASON_MAX_CHARS: usize = 500; pub const EXEC_MODE_REASON_MAX_BYTES: usize = EXEC_MODE_REASON_MAX_CHARS * 4; pub const SHUTDOWN_NOTE_MAX_CHARS: usize = 2_000; pub const SHUTDOWN_NOTE_MAX_BYTES: usize = SHUTDOWN_NOTE_MAX_CHARS * 4; +pub const MESSAGE_IDENTIFIER_MAX_CHARS: usize = 1_000; +pub const MESSAGE_IDENTIFIER_MAX_BYTES: usize = MESSAGE_IDENTIFIER_MAX_CHARS * 4; pub const TASK_DEPENDENCY_OUTPUT_MAX_COUNT: usize = 64; pub const TASK_DEPENDENCY_TOTAL_CONTENT_MAX_CHARS: usize = 50_000; pub const TASK_DEPENDENCY_TOTAL_CONTENT_MAX_BYTES: usize = @@ -107,6 +140,107 @@ pub fn validate_optional_text( Ok(()) } +pub fn validate_task_identifier(field: &str, value: &str) -> Result<(), String> { + validate_message_identifier(field, value) +} + +pub fn validate_message_identifier(field: &str, value: &str) -> Result<(), String> { + validate_required_text( + field, + value, + MESSAGE_IDENTIFIER_MAX_CHARS, + MESSAGE_IDENTIFIER_MAX_BYTES, + )?; + if value != value.trim() { + return Err(format!( + "{field} must not contain leading or trailing whitespace" + )); + } + Ok(()) +} + +pub fn validate_task_identifier_list(field: &str, values: &[String]) -> Result<(), String> { + for (index, value) in values.iter().enumerate() { + validate_task_identifier(&format!("{field}[{index}]"), value)?; + } + Ok(()) +} + +pub fn validate_task_dependency_ids(field: &str, values: &[String]) -> Result<(), String> { + if values.len() > TASK_DEPENDENCY_MAX_COUNT { + return Err(format!( + "task_dependency_limit: {field} must contain at most {TASK_DEPENDENCY_MAX_COUNT} task ids" + )); + } + validate_task_identifier_list(field, values) + .map_err(|error| format!("task_dependency_limit: {error}"))?; + let total_chars = values.iter().fold(0usize, |total, value| { + total.saturating_add(value.chars().count()) + }); + let total_bytes = values + .iter() + .fold(0usize, |total, value| total.saturating_add(value.len())); + if total_chars > TASK_DEPENDENCY_TOTAL_MAX_CHARS + || total_bytes > TASK_DEPENDENCY_TOTAL_MAX_BYTES + { + return Err(format!( + "task_dependency_limit: {field} must total <= {TASK_DEPENDENCY_TOTAL_MAX_CHARS} chars and <= {TASK_DEPENDENCY_TOTAL_MAX_BYTES} bytes" + )); + } + Ok(()) +} + +pub fn validate_task_eligible_member_ids(field: &str, values: &[String]) -> Result<(), String> { + if values.len() > TASK_ELIGIBILITY_MAX_COUNT { + return Err(format!( + "{field} must contain at most {TASK_ELIGIBILITY_MAX_COUNT} member ids" + )); + } + validate_task_identifier_list(field, values)?; + let total_chars = values.iter().fold(0usize, |total, value| { + total.saturating_add(value.chars().count()) + }); + let total_bytes = values + .iter() + .fold(0usize, |total, value| total.saturating_add(value.len())); + if total_chars > TASK_ELIGIBILITY_TOTAL_MAX_CHARS + || total_bytes > TASK_ELIGIBILITY_TOTAL_MAX_BYTES + { + return Err(format!( + "{field} must total <= {TASK_ELIGIBILITY_TOTAL_MAX_CHARS} chars and <= {TASK_ELIGIBILITY_TOTAL_MAX_BYTES} bytes" + )); + } + Ok(()) +} + +pub fn validate_task_artifact_ids(field: &str, values: &[String]) -> Result<(), String> { + if values.len() > TASK_ARTIFACT_ID_MAX_COUNT { + return Err(format!( + "{field} must contain at most {TASK_ARTIFACT_ID_MAX_COUNT} entries" + )); + } + let mut total_chars = 0usize; + let mut total_bytes = 0usize; + for (index, value) in values.iter().enumerate() { + validate_required_text( + &format!("{field}[{index}]"), + value, + TASK_ARTIFACT_ID_MAX_CHARS, + TASK_ARTIFACT_ID_MAX_BYTES, + )?; + total_chars = total_chars.saturating_add(value.chars().count()); + total_bytes = total_bytes.saturating_add(value.len()); + } + if total_chars > TASK_ARTIFACT_IDS_TOTAL_MAX_CHARS + || total_bytes > TASK_ARTIFACT_IDS_TOTAL_MAX_BYTES + { + return Err(format!( + "{field} must total <= {TASK_ARTIFACT_IDS_TOTAL_MAX_CHARS} chars and <= {TASK_ARTIFACT_IDS_TOTAL_MAX_BYTES} bytes" + )); + } + Ok(()) +} + #[cfg(test)] mod tests { use super::*; @@ -121,4 +255,16 @@ mod tests { .unwrap_err() .contains("4 bytes")); } + + #[test] + fn task_identifier_uses_the_delivery_boundary() { + assert!( + validate_task_identifier("task id", &"a".repeat(TASK_IDENTIFIER_MAX_CHARS)).is_ok() + ); + assert!( + validate_task_identifier("task id", &"a".repeat(TASK_IDENTIFIER_MAX_CHARS + 1)) + .unwrap_err() + .contains("task id must be <= 1000 chars") + ); + } } diff --git a/src-tauri/crates/agent-core/src/core/coordination/agent_org_plan_approvals/artifact.rs b/src-tauri/crates/agent-core/src/core/coordination/agent_org_plan_approvals/artifact.rs index 2bf927a5e..b733723e7 100644 --- a/src-tauri/crates/agent-core/src/core/coordination/agent_org_plan_approvals/artifact.rs +++ b/src-tauri/crates/agent-core/src/core/coordination/agent_org_plan_approvals/artifact.rs @@ -4,6 +4,11 @@ use std::sync::OnceLock; use rusqlite::{params, Connection, OptionalExtension}; +use database::db::{get_connection, with_sessions_writer}; + +use super::persistence::query_record; +use super::AgentOrgPlanApproval; + /// A fully-written, fsynced artifact waiting for its short atomic install. /// /// The temporary file lives beside the target so `rename` never crosses a @@ -409,7 +414,7 @@ pub(super) fn finish_committed_artifact( .map(|artifact| artifact.target_path.display().to_string()) .unwrap_or_default(), error = %err, - "Agent Org plan DB commit succeeded but its derived artifact could not be installed" + "Agent Org plan DB commit succeeded but artifact installation needs repair" ); } Ok(value) @@ -417,3 +422,144 @@ pub(super) fn finish_committed_artifact( Err(err) => Err(err), } } + +pub(super) fn list_distinct_plan_paths_after( + after_path: Option<&str>, + limit: usize, +) -> Result, String> { + let conn = get_connection().map_err(|err| err.to_string())?; + let mut stmt = conn + .prepare( + "SELECT DISTINCT plan_path + FROM agent_org_plan_approvals + WHERE (?1 IS NULL OR plan_path > ?1) + ORDER BY plan_path ASC + LIMIT ?2", + ) + .map_err(|err| err.to_string())?; + let rows = stmt + .query_map(params![after_path, limit as i64], |row| { + row.get::<_, String>(0) + }) + .map_err(|err| err.to_string())?; + rows.collect::, _>>() + .map_err(|err| err.to_string()) +} + +fn latest_plan_revision_for_path_with_connection( + conn: &Connection, + plan_path: &str, +) -> Result, String> { + query_record( + conn, + "WHERE plan_path=?1 ORDER BY created_at DESC, rowid DESC", + params![plan_path], + ) +} + +fn latest_plan_revision_for_path(plan_path: &str) -> Result, String> { + let conn = get_connection().map_err(|err| err.to_string())?; + latest_plan_revision_for_path_with_connection(&conn, plan_path) +} + +fn stage_plan_artifact_if_needed( + source_session_id: &str, + plan_path: &str, + canonical_content: &str, +) -> Result, String> { + let conn = get_connection().map_err(|err| err.to_string())?; + let Some(owned) = + owned_plan_path_for_existing_revision_with_connection(&conn, source_session_id, plan_path)? + else { + return Ok(None); + }; + let target_path = match resolve_owned_plan_target(&owned, true) { + Ok(Some(target)) => target, + Ok(None) => return Ok(None), + Err(err) => { + tracing::warn!( + source_session_id, + plan_path, + error = %err, + "skipping unsafe Agent Org plan artifact repair" + ); + return Ok(None); + } + }; + match std::fs::read(&target_path) { + Ok(existing) if existing == canonical_content.as_bytes() => return Ok(None), + Ok(_) => {} + Err(err) if err.kind() == std::io::ErrorKind::NotFound => {} + Err(err) => { + return Err(format!( + "failed to inspect Agent Org plan artifact {}: {err}", + target_path.display() + )) + } + } + stage_owned_plan_artifact(owned, target_path, canonical_content).map(Some) +} + +pub(super) fn repair_latest_plan_artifact_for_path(plan_path: &str) -> Result { + const MAX_REPAIR_RACES: usize = 4; + + for _ in 0..MAX_REPAIR_RACES { + let Some(canonical) = latest_plan_revision_for_path(plan_path)? else { + return Ok(false); + }; + let staged = stage_plan_artifact_if_needed( + &canonical.source_session_id, + plan_path, + &canonical.plan_content, + )?; + if staged.is_none() { + let latest = latest_plan_revision_for_path(plan_path)?; + if latest.as_ref().is_some_and(|record| { + record.approval_id == canonical.approval_id + && record.plan_revision_id == canonical.plan_revision_id + && record.plan_content == canonical.plan_content + }) { + return Ok(false); + } + continue; + } + + let _artifact_guard = plan_artifact_install_lock().lock(); + let should_install = with_sessions_writer(|| -> Result { + let conn = get_connection().map_err(|err| err.to_string())?; + let latest = latest_plan_revision_for_path_with_connection(&conn, plan_path)?; + let still_current = latest.as_ref().is_some_and(|record| { + record.approval_id == canonical.approval_id + && record.plan_revision_id == canonical.plan_revision_id + && record.plan_content == canonical.plan_content + }); + if !still_current { + return Ok(false); + } + let Some(latest) = latest.as_ref() else { + return Ok(false); + }; + if let Err(err) = validate_owned_plan_path_with_connection( + &conn, + &latest.source_session_id, + &latest.plan_path, + ) { + tracing::warn!( + source_session_id = %latest.source_session_id, + plan_path = %latest.plan_path, + error = %err, + "skipping Agent Org plan artifact repair after ownership changed" + ); + return Ok(false); + } + Ok(true) + })?; + if should_install { + install_staged_plan_artifact(staged.as_ref())?; + return Ok(true); + } + } + Err(format!( + "Agent Org plan artifact kept changing while being repaired: {plan_path}" + )) +} diff --git a/src-tauri/crates/agent-core/src/core/coordination/agent_org_plan_approvals/store.rs b/src-tauri/crates/agent-core/src/core/coordination/agent_org_plan_approvals/store.rs index c83f9daf1..e8cbaae53 100644 --- a/src-tauri/crates/agent-core/src/core/coordination/agent_org_plan_approvals/store.rs +++ b/src-tauri/crates/agent-core/src/core/coordination/agent_org_plan_approvals/store.rs @@ -16,7 +16,8 @@ use crate::definitions::orgs::PlanApprovalPolicy; use super::artifact::{ expected_plan_root_with_connection, finish_committed_artifact, install_staged_plan_artifact, - plan_artifact_install_lock, resolve_owned_plan_target, + list_distinct_plan_paths_after, plan_artifact_install_lock, + repair_latest_plan_artifact_for_path, resolve_owned_plan_target, stage_plan_artifact_for_existing_revision_with_connection, stage_plan_artifact_with_connection, sync_parent_directory, validate_owned_plan_path_with_connection, validate_plan_file_name, }; @@ -33,6 +34,13 @@ use super::{ pub struct AgentOrgPlanApprovalStore; +#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)] +pub struct AgentOrgPlanArtifactRepairReport { + pub inspected: usize, + pub repaired: usize, + pub failed: usize, +} + impl AgentOrgPlanApprovalStore { /// Resolve a filename under the exact Plan root owned by a persisted /// source session. Callers use this when they need a fresh path after a @@ -342,8 +350,8 @@ impl AgentOrgPlanApprovalStore { // SQLite is the durable source of truth. Prepare and fsync the slow // file bytes before taking the sessions writer, commit SQLite first, // then perform only the same-directory rename while writes remain - // serialized. SQLite remains authoritative if the derived artifact - // cannot be installed after the commit. + // serialized. A process crash in the tiny commit -> rename window is + // healed from `plan_content` on startup or the next detail read. let canonical_content = edited_content .clone() .unwrap_or_else(|| current.plan_content.clone()); @@ -511,34 +519,105 @@ impl AgentOrgPlanApprovalStore { query_record(&conn, "WHERE approval_id=?1", params![approval_id]) } - /// Read one immutable plan revision. The durable database content is the - /// source of truth for approval detail views. + /// Read one immutable plan revision and best-effort reconcile the shared + /// plan artifact to the latest revision stored for that path. + /// + /// Historical rows remain immutable and are returned exactly as stored; + /// only the derived filesystem artifact is repaired. A repair failure is + /// logged rather than turning an otherwise valid detail read into a false + /// user-visible failure. pub fn get_revision( approval_id: &str, plan_revision_id: &str, ) -> Result, String> { let conn = get_connection().map_err(|err| err.to_string())?; - query_record( + let record = query_record( &conn, "WHERE approval_id=?1 AND plan_revision_id=?2", params![approval_id, plan_revision_id], - ) + )?; + drop(conn); + if let Some(record) = record.as_ref() { + if let Err(err) = repair_latest_plan_artifact_for_path(&record.plan_path) { + tracing::warn!( + approval_id, + plan_revision_id, + plan_path = %record.plan_path, + error = %err, + "failed to reconcile Agent Org plan artifact during detail read" + ); + } + } + Ok(record) } /// Run-scoped detail lookup for user-facing/API callers. The ownership /// predicate is part of the SQLite query, so an approval from another Run - /// cannot be exposed to the caller. + /// cannot trigger even the best-effort filesystem repair performed after + /// an authorized detail read. pub fn get_revision_for_run( org_run_id: &str, approval_id: &str, plan_revision_id: &str, ) -> Result, String> { let conn = get_connection().map_err(|err| err.to_string())?; - query_record( + let record = query_record( &conn, "WHERE org_run_id=?1 AND approval_id=?2 AND plan_revision_id=?3", params![org_run_id, approval_id, plan_revision_id], - ) + )?; + drop(conn); + if let Some(record) = record.as_ref() { + if let Err(err) = repair_latest_plan_artifact_for_path(&record.plan_path) { + tracing::warn!( + org_run_id, + approval_id, + plan_revision_id, + plan_path = %record.plan_path, + error = %err, + "failed to reconcile Agent Org plan artifact during run-scoped detail read" + ); + } + } + Ok(record) + } + + /// Reconcile every physical plan artifact from the latest durable SQLite + /// revision for its path. The query is paged so retained approval history + /// cannot create one unbounded allocation. Individual corrupt/unwritable + /// paths are isolated and reported without preventing other plans from + /// being repaired. + pub fn repair_latest_plan_artifacts() -> Result { + const PAGE_SIZE: usize = 64; + + let mut report = AgentOrgPlanArtifactRepairReport::default(); + let mut after_path: Option = None; + loop { + let paths = list_distinct_plan_paths_after(after_path.as_deref(), PAGE_SIZE)?; + if paths.is_empty() { + break; + } + for path in &paths { + report.inspected += 1; + match repair_latest_plan_artifact_for_path(path) { + Ok(true) => report.repaired += 1, + Ok(false) => {} + Err(err) => { + report.failed += 1; + tracing::warn!( + plan_path = %path, + error = %err, + "failed to reconcile one Agent Org plan artifact" + ); + } + } + } + after_path = paths.last().cloned(); + if paths.len() < PAGE_SIZE { + break; + } + } + Ok(report) } /// Cancel approvals whose parent run is gone or terminal. A paused run is diff --git a/src-tauri/crates/agent-core/src/core/coordination/agent_org_runs/finality.rs b/src-tauri/crates/agent-core/src/core/coordination/agent_org_runs/finality.rs index 9f4651bd4..bce23aa77 100644 --- a/src-tauri/crates/agent-core/src/core/coordination/agent_org_runs/finality.rs +++ b/src-tauri/crates/agent-core/src/core/coordination/agent_org_runs/finality.rs @@ -31,6 +31,7 @@ pub struct AgentOrgFinalityFacts { pub worker_sessions: Vec, pub task_count: usize, pub unresolved_task_count: usize, + pub corrupt_task_count: usize, pub pending_task_count: usize, pub in_progress_task_count: usize, pub completed_task_count: usize, @@ -86,6 +87,9 @@ pub enum AgentOrgFinalityBlocker { OpenTasks { count: usize, }, + CorruptTaskData { + count: usize, + }, EmptyTaskBoardRequiresCompletionIntent, StaleCompletionIntent { requested_work_revision: Option, @@ -117,6 +121,7 @@ pub enum AgentOrgFinalityBlocker { root_session_missing: bool, active_session_count: usize, open_task_count: usize, + corrupt_task_count: usize, unread_inbox_count: usize, active_intervention_count: usize, in_flight_turn_intent_count: usize, @@ -132,6 +137,177 @@ pub struct AgentOrgFinalityAssessment { pub blockers: Vec, } +#[derive(Debug, Clone, Serialize)] +#[serde(rename_all = "camelCase")] +pub struct AgentOrgFinalityProjection { + pub decision: AgentOrgFinalityDecision, + pub blockers: Vec, +} + +/// Exact effects that the currently executing coordinator turn will commit +/// if (and only if) that turn succeeds. These counts are not caller hints: +/// they are revalidated from durable intent/materialization rows inside the +/// same read transaction as the finality snapshot. +#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)] +pub struct AgentOrgGuaranteedTurnEffects { + pub current_coordinator_turn: bool, + pub in_flight_turn_intents: usize, + pub unread_inbox_rows: usize, +} + +impl AgentOrgFinalityAssessment { + /// Canonical prospective certificate used by the coordinator inside its + /// current turn. It answers one narrow question: "if this coordinator + /// turn succeeds now, will the strict reconciler be able to complete?" + /// + /// Only effects guaranteed by successful turn finalization are projected: + /// the root session becomes quiescent, and the revision staged into this + /// prompt becomes observed. Every worker, task, inbox, approval, + /// intervention, corruption, and turn-intent blocker remains unchanged. + pub fn after_successful_coordinator_turn(&self) -> AgentOrgFinalityProjection { + self.after_successful_coordinator_turn_with_effects(AgentOrgGuaranteedTurnEffects { + current_coordinator_turn: true, + ..AgentOrgGuaranteedTurnEffects::default() + }) + } + + pub fn after_successful_coordinator_turn_with_effects( + &self, + effects: AgentOrgGuaranteedTurnEffects, + ) -> AgentOrgFinalityProjection { + if self.facts.run_status != Some(AgentOrgRunStatus::Running) + || !effects.current_coordinator_turn + { + return AgentOrgFinalityProjection { + decision: self.decision, + blockers: self.blockers.clone(), + }; + } + let root_session_id = self.facts.root_session_id.as_deref(); + let presented_current_revision = self.facts.progress.as_ref().is_some_and(|progress| { + progress.coordinator_presented_work_revision == Some(progress.work_revision) + }); + let mut blockers = Vec::new(); + for blocker in &self.blockers { + match blocker { + AgentOrgFinalityBlocker::SessionsActive { session_ids } => { + let remaining = session_ids + .iter() + .filter(|session_id| Some(session_id.as_str()) != root_session_id) + .cloned() + .collect::>(); + if !remaining.is_empty() { + blockers.push(AgentOrgFinalityBlocker::SessionsActive { + session_ids: remaining, + }); + } + } + AgentOrgFinalityBlocker::CoordinatorHasNotObservedLatestWork { .. } + if presented_current_revision => {} + AgentOrgFinalityBlocker::UnreadInbox { count } => { + let remaining = count.saturating_sub(effects.unread_inbox_rows); + if remaining > 0 { + blockers.push(AgentOrgFinalityBlocker::UnreadInbox { count: remaining }); + } + } + AgentOrgFinalityBlocker::InFlightTurnIntents { count } => { + let remaining = count.saturating_sub(effects.in_flight_turn_intents); + if remaining > 0 { + blockers.push(AgentOrgFinalityBlocker::InFlightTurnIntents { + count: remaining, + }); + } + } + other => blockers.push(other.clone()), + } + } + AgentOrgFinalityProjection { + decision: if blockers.is_empty() { + AgentOrgFinalityDecision::Complete + } else { + AgentOrgFinalityDecision::KeepRunning + }, + blockers, + } + } +} + +pub(crate) fn guaranteed_current_turn_effects_with_connection( + conn: &Connection, + run_id: &str, + root_session_id: Option<&str>, + dispatching_session_id: &str, + turn_intent_id: &str, + projected_inbox_ids: &[i64], +) -> Result { + if root_session_id != Some(dispatching_session_id) + || dispatching_session_id.trim().is_empty() + || turn_intent_id.trim().is_empty() + { + return Ok(AgentOrgGuaranteedTurnEffects::default()); + } + + let in_flight_turn_intents: bool = conn + .query_row( + "SELECT EXISTS( + SELECT 1 FROM session_turn_intents + WHERE session_id=?1 AND turn_intent_id=?2 AND org_run_id=?3 + AND status IN (?4, ?5, ?6) + )", + params![ + dispatching_session_id, + turn_intent_id, + run_id, + IN_FLIGHT_TURN_INTENT_STATUSES[0], + IN_FLIGHT_TURN_INTENT_STATUSES[1], + IN_FLIGHT_TURN_INTENT_STATUSES[2], + ], + |row| row.get(0), + ) + .map_err(|err| err.to_string())?; + + // The production drain batch is bounded. Deduplicate the typed ids and + // validate each exact row/receipt pair instead of interpolating an IN + // list or counting every receipt ever owned by this Session. + let mut unique_ids = projected_inbox_ids + .iter() + .copied() + .filter(|id| *id > 0) + .collect::>(); + unique_ids.sort_unstable(); + unique_ids.dedup(); + let mut unread_inbox_rows = 0usize; + let mut stmt = conn + .prepare( + "SELECT EXISTS( + SELECT 1 + FROM agent_inbox inbox + JOIN agent_inbox_materializations receipt + ON receipt.inbox_id=inbox.id AND receipt.session_id=?2 + WHERE inbox.id=?1 AND inbox.org_run_id=?3 AND inbox.read_at IS NULL + AND NOT EXISTS ( + SELECT 1 FROM agent_inbox_delivery_resolutions resolution + WHERE resolution.inbox_id=inbox.id + ) + )", + ) + .map_err(|err| err.to_string())?; + for inbox_id in unique_ids { + let is_guaranteed: bool = stmt + .query_row(params![inbox_id, dispatching_session_id, run_id], |row| { + row.get(0) + }) + .map_err(|err| err.to_string())?; + unread_inbox_rows += usize::from(is_guaranteed); + } + + Ok(AgentOrgGuaranteedTurnEffects { + current_coordinator_turn: in_flight_turn_intents, + in_flight_turn_intents: usize::from(in_flight_turn_intents), + unread_inbox_rows, + }) +} + pub(super) fn load_and_assess( conn: &Connection, run_id: &str, @@ -152,6 +328,7 @@ pub(super) fn load_and_assess( worker_sessions: Vec::new(), task_count: 0, unresolved_task_count: 0, + corrupt_task_count: 0, pending_task_count: 0, in_progress_task_count: 0, completed_task_count: 0, @@ -196,26 +373,64 @@ pub(super) fn load_and_assess( }) .collect(); - let task_counts_sql = "SELECT COUNT(*), + let corrupt_task_predicate = + crate::coordination::agent_org_tasks::corrupt_task_row_predicate_sql(); + let persisted_task_count: i64 = conn + .query_row( + "SELECT COUNT(*) FROM agent_org_tasks WHERE org_run_id=?1", + params![run_id], + |row| row.get(0), + ) + .map_err(|err| err.to_string())?; + let corruption_projection = if persisted_task_count + > crate::coordination::agent_org_payload_limits::TASK_RUN_MAX_TASKS as i64 + { + // A historical board above the supported cap is one run-level + // corruption. Do not parse every legacy JSON row merely to enumerate + // extra violations. + "1".to_string() + } else { + format!("COALESCE(SUM(CASE WHEN {corrupt_task_predicate} THEN 1 ELSE 0 END), 0)") + }; + let dependency_json_max = + crate::coordination::agent_org_payload_limits::TASK_DEPENDENCY_JSON_MAX_BYTES; + let metadata_max = crate::coordination::agent_org_payload_limits::TASK_METADATA_MAX_BYTES; + let task_counts_sql = format!( + "SELECT COUNT(*), COALESCE(SUM(CASE WHEN status <> 'completed' THEN 1 ELSE 0 END), 0), + {corruption_projection}, COALESCE(SUM(CASE WHEN status='pending' THEN 1 ELSE 0 END), 0), COALESCE(SUM(CASE WHEN status='in_progress' THEN 1 ELSE 0 END), 0), COALESCE(SUM(CASE WHEN status='completed' THEN 1 ELSE 0 END), 0) - FROM agent_org_tasks WHERE org_run_id=?1"; + FROM ( + SELECT id, subject, description, active_form, owner, status, + created_at, updated_at, + CASE WHEN length(CAST(blocks_json AS BLOB))<={dependency_json_max} + THEN blocks_json ELSE '!' END AS blocks_json, + CASE WHEN length(CAST(blocked_by_json AS BLOB))<={dependency_json_max} + THEN blocked_by_json ELSE '!' END AS blocked_by_json, + CASE WHEN metadata_json IS NULL + OR length(CAST(metadata_json AS BLOB))<={metadata_max} + THEN metadata_json ELSE '!' END AS metadata_json + FROM agent_org_tasks WHERE org_run_id=?1 + ) AS bounded_tasks" + ); let ( task_count, unresolved_task_count, + corrupt_task_count, pending_task_count, in_progress_task_count, completed_task_count, - ): (i64, i64, i64, i64, i64) = conn - .query_row(task_counts_sql, params![run_id], |row| { + ): (i64, i64, i64, i64, i64, i64) = conn + .query_row(&task_counts_sql, params![run_id], |row| { Ok(( row.get(0)?, row.get(1)?, row.get(2)?, row.get(3)?, row.get(4)?, + row.get(5)?, )) }) .map_err(|err| err.to_string())?; @@ -223,7 +438,11 @@ pub(super) fn load_and_assess( .query_row( "SELECT COUNT(*) FROM agent_inbox - WHERE org_run_id=?1 AND read_at IS NULL", + WHERE org_run_id=?1 AND read_at IS NULL + AND NOT EXISTS ( + SELECT 1 FROM agent_inbox_delivery_resolutions resolution + WHERE resolution.inbox_id=agent_inbox.id + )", params![run_id], |row| row.get(0), ) @@ -248,16 +467,8 @@ pub(super) fn load_and_assess( }; let in_flight_turn_intent_count: i64 = conn .query_row( - "WITH RECURSIVE org_sessions(session_id) AS ( - SELECT root_session_id FROM agent_org_runs WHERE id=?1 - UNION ALL - SELECT session.session_id - FROM agent_sessions session - JOIN org_sessions parent ON session.parent_session_id=parent.session_id - ) - SELECT COUNT(*) FROM session_turn_intents intent - WHERE intent.session_id IN (SELECT session_id FROM org_sessions) - AND intent.status IN (?2, ?3, ?4)", + "SELECT COUNT(*) FROM session_turn_intents + WHERE org_run_id=?1 AND status IN (?2, ?3, ?4)", params![ run_id, IN_FLIGHT_TURN_INTENT_STATUSES[0], @@ -283,6 +494,7 @@ pub(super) fn load_and_assess( worker_sessions, task_count: count_to_usize("task", task_count)?, unresolved_task_count: count_to_usize("unresolved task", unresolved_task_count)?, + corrupt_task_count: count_to_usize("corrupt task", corrupt_task_count)?, pending_task_count: count_to_usize("pending task", pending_task_count)?, in_progress_task_count: count_to_usize("in-progress task", in_progress_task_count)?, completed_task_count: count_to_usize("completed task", completed_task_count)?, @@ -365,6 +577,11 @@ pub(super) fn assess(facts: AgentOrgFinalityFacts) -> AgentOrgFinalityAssessment count: facts.unresolved_task_count, }); } + if facts.corrupt_task_count > 0 { + blockers.push(AgentOrgFinalityBlocker::CorruptTaskData { + count: facts.corrupt_task_count, + }); + } if facts.unread_inbox_count > 0 { blockers.push(AgentOrgFinalityBlocker::UnreadInbox { count: facts.unread_inbox_count, @@ -451,6 +668,7 @@ fn terminal_state_inconsistency( let inconsistent = root_session_missing || active_session_count > 0 || facts.unresolved_task_count > 0 + || facts.corrupt_task_count > 0 || facts.unread_inbox_count > 0 || !facts.active_intervention_member_ids.is_empty() || facts.in_flight_turn_intent_count > 0 @@ -460,6 +678,7 @@ fn terminal_state_inconsistency( root_session_missing, active_session_count, open_task_count: facts.unresolved_task_count, + corrupt_task_count: facts.corrupt_task_count, unread_inbox_count: facts.unread_inbox_count, active_intervention_count: facts.active_intervention_member_ids.len(), in_flight_turn_intent_count: facts.in_flight_turn_intent_count, @@ -503,6 +722,7 @@ mod tests { }], task_count: 1, unresolved_task_count: 0, + corrupt_task_count: 0, pending_task_count: 0, in_progress_task_count: 0, completed_task_count: 1, @@ -524,12 +744,45 @@ mod tests { } } + #[test] + fn prospective_certificate_allows_current_coordinator_turn_only() { + let assessment = assess(completed_board_facts(Some(2), SessionStatus::Idle)); + assert_eq!(assessment.decision, AgentOrgFinalityDecision::KeepRunning); + let prospective = assessment.after_successful_coordinator_turn(); + assert_eq!(prospective.decision, AgentOrgFinalityDecision::Complete); + assert!(prospective.blockers.is_empty()); + } + + #[test] + fn prospective_certificate_rejects_stale_presented_revision() { + let assessment = assess(completed_board_facts(Some(1), SessionStatus::Idle)); + let prospective = assessment.after_successful_coordinator_turn(); + assert_eq!(prospective.decision, AgentOrgFinalityDecision::KeepRunning); + assert!(prospective.blockers.iter().any(|blocker| matches!( + blocker, + AgentOrgFinalityBlocker::CoordinatorHasNotObservedLatestWork { .. } + ))); + } + + #[test] + fn prospective_certificate_never_hides_active_worker() { + let assessment = assess(completed_board_facts(Some(2), SessionStatus::Running)); + let prospective = assessment.after_successful_coordinator_turn(); + assert_eq!(prospective.decision, AgentOrgFinalityDecision::KeepRunning); + assert!(prospective.blockers.iter().any(|blocker| matches!( + blocker, + AgentOrgFinalityBlocker::SessionsActive { session_ids } + if session_ids == &["worker".to_string()] + ))); + } + #[test] fn completed_run_stays_terminal_but_reports_inconsistent_retained_facts() { let mut facts = completed_board_facts(Some(2), SessionStatus::Idle); facts.run_status = Some(AgentOrgRunStatus::Completed); facts.root_status = Some(SessionStatus::Idle); facts.unresolved_task_count = 1; + facts.corrupt_task_count = 1; facts.unread_inbox_count = 2; let assessment = assess(facts); @@ -539,6 +792,7 @@ mod tests { [AgentOrgFinalityBlocker::TerminalStateInconsistent { status: AgentOrgRunStatus::Completed, open_task_count: 1, + corrupt_task_count: 1, unread_inbox_count: 2, .. }] diff --git a/src-tauri/crates/agent-core/src/core/coordination/agent_org_runs/mod.rs b/src-tauri/crates/agent-core/src/core/coordination/agent_org_runs/mod.rs index ce811c525..5e2a0851a 100644 --- a/src-tauri/crates/agent-core/src/core/coordination/agent_org_runs/mod.rs +++ b/src-tauri/crates/agent-core/src/core/coordination/agent_org_runs/mod.rs @@ -12,13 +12,16 @@ mod worker; #[cfg(test)] mod tests; +pub(crate) use finality::guaranteed_current_turn_effects_with_connection; pub use finality::{ AgentOrgFinalityAssessment, AgentOrgFinalityBlocker, AgentOrgFinalityDecision, - AgentOrgFinalityFacts, AgentOrgFinalitySessionFact, + AgentOrgFinalityFacts, AgentOrgFinalityProjection, AgentOrgFinalitySessionFact, + AgentOrgGuaranteedTurnEffects, }; pub(crate) use progress::bump_work_revision_in_tx; pub use progress::AgentOrgRunProgress; pub use store::AgentOrgRunStore; +pub(crate) use worker::recovery_dispatch_recipient_is_available; pub use worker::{WorkerSessionInfo, WorkerSessionRuntime}; use rusqlite::{Connection, Result as SqliteResult}; diff --git a/src-tauri/crates/agent-core/src/core/coordination/agent_org_runs/store.rs b/src-tauri/crates/agent-core/src/core/coordination/agent_org_runs/store.rs index a672b3008..c42cfbafe 100644 --- a/src-tauri/crates/agent-core/src/core/coordination/agent_org_runs/store.rs +++ b/src-tauri/crates/agent-core/src/core/coordination/agent_org_runs/store.rs @@ -11,8 +11,8 @@ use database::db::{get_connection, with_sessions_writer}; use super::finality::load_and_assess; use super::helpers::{ - context_for_run_record, insert_run, load_by_id, load_by_root_session, parent_session_id_of, - row_to_run, validate_entry_mode, validate_status, + context_for_run_record, flatten_members, insert_run, load_by_id, load_by_root_session, + parent_session_id_of, row_to_run, validate_entry_mode, validate_status, }; use super::progress::{ ensure_progress_in_conn, load_progress_with_conn, mark_coordinator_observed_revision_with_conn, @@ -659,17 +659,12 @@ impl AgentOrgRunStore { .map_err(|err| err.to_string())? }; + // A session tree can contain another Agent Org run. Intent rows + // therefore carry explicit run ownership: deleting this run must + // remove all of its direct and wake/resume turns without touching + // a nested run merely because its root is a session descendant. tx.execute( - "WITH RECURSIVE org_sessions(session_id) AS ( - SELECT root_session_id FROM agent_org_runs WHERE id=?1 - UNION ALL - SELECT session.session_id - FROM agent_sessions session - JOIN org_sessions parent - ON session.parent_session_id=parent.session_id - ) - DELETE FROM session_turn_intents - WHERE session_id IN (SELECT session_id FROM org_sessions)", + "DELETE FROM session_turn_intents WHERE org_run_id=?1", params![run_id], ) .map_err(|err| err.to_string())?; @@ -690,9 +685,11 @@ impl AgentOrgRunStore { "agent_org_recovery_attempts", "agent_org_task_events", "agent_org_tasks", + "agent_inbox_delivery_resolutions", "agent_inbox", "agent_member_interventions", "agent_org_run_progress", + "agent_org_task_run_schema_migrations", ] { tx.execute( &format!("DELETE FROM {table} WHERE org_run_id=?1"), @@ -829,6 +826,40 @@ impl AgentOrgRunStore { .collect()) } + /// Canonical member ids captured in the immutable launch snapshot. + /// + /// Recovery must not consult the user's current Agent Org definition: a + /// team can be edited while an older run is still alive. `None` is kept + /// for historical rows that predate launch snapshots; callers may still + /// classify a materialized session, but must not invent roster membership. + pub(crate) fn snapshot_member_ids_with_connection( + conn: &Connection, + org_run_id: &str, + ) -> Result>, String> { + let snapshot_json: Option = conn + .query_row( + "SELECT org_snapshot_json FROM agent_org_runs WHERE id=?1", + params![org_run_id], + |row| row.get(0), + ) + .optional() + .map_err(|err| err.to_string())? + .flatten(); + let Some(snapshot_json) = snapshot_json else { + return Ok(None); + }; + let snapshot: crate::definitions::orgs::OrgDefinition = + serde_json::from_str(&snapshot_json).map_err(|err| { + format!("failed to parse Agent Org launch snapshot for run {org_run_id}: {err}") + })?; + Ok(Some( + flatten_members(&snapshot.children, None) + .into_iter() + .map(|member| member.member_id) + .collect(), + )) + } + pub fn list_descendant_worker_sessions( org_run_id: &str, ) -> Result, String> { @@ -864,10 +895,20 @@ impl AgentOrgRunStore { SELECT session_id FROM agent_sessions child WHERE child.parent_session_id = ?1 + AND NOT EXISTS ( + SELECT 1 FROM agent_org_runs nested + WHERE nested.id <> ?2 + AND nested.root_session_id = child.session_id + ) UNION SELECT s.session_id FROM agent_sessions s JOIN descendants d ON s.parent_session_id = d.session_id + WHERE NOT EXISTS ( + SELECT 1 FROM agent_org_runs nested + WHERE nested.id <> ?2 + AND nested.root_session_id = s.session_id + ) ), ranked AS ( SELECT s.agent_definition_id, s.org_member_id, @@ -894,7 +935,7 @@ impl AgentOrgRunStore { .map_err(|err| err.to_string())?; let rows = stmt - .query_map(params![root.clone()], |row| { + .query_map(params![root.clone(), org_run_id], |row| { let status_raw: String = row.get(3)?; let status = crate::core::session::SessionStatus::parse(&status_raw).ok_or_else(|| { diff --git a/src-tauri/crates/agent-core/src/core/coordination/agent_org_runs/tests.rs b/src-tauri/crates/agent-core/src/core/coordination/agent_org_runs/tests.rs index f79e94b06..3a472e7e9 100644 --- a/src-tauri/crates/agent-core/src/core/coordination/agent_org_runs/tests.rs +++ b/src-tauri/crates/agent-core/src/core/coordination/agent_org_runs/tests.rs @@ -69,6 +69,7 @@ fn ensure_runtime_schemas() { session_id TEXT NOT NULL, turn_intent_id TEXT NOT NULL, client_message_id TEXT, + org_run_id TEXT, source TEXT NOT NULL, status TEXT NOT NULL, created_at TEXT NOT NULL, @@ -153,6 +154,8 @@ fn delete_by_id_cascades_all_run_owned_state_and_plan_artifact() { std::fs::create_dir_all(&plan_root).expect("create managed Plan root"); let plan_path = plan_root.join("delete-cascade.plan.md"); std::fs::write(&plan_path, "# disposable plan").unwrap(); + let external_notes = sandbox.path().join("notes.md"); + std::fs::write(&external_notes, "user-owned notes").unwrap(); let conn = database::db::get_connection().unwrap(); conn.execute( "UPDATE agent_sessions SET workspace_path=?1 WHERE session_id='worker-delete-cascade'", @@ -173,6 +176,19 @@ fn delete_by_id_cascades_all_run_owned_state_and_plan_artifact() { params![&run.id, plan_path.to_string_lossy().as_ref(), &now], ) .unwrap(); + conn.execute( + "INSERT INTO agent_org_plan_approvals ( + approval_id, plan_revision_id, request_id, org_run_id, + source_task_id, source_member_id, source_session_id, + root_session_id, policy, status, plan_title, plan_path, + plan_content, created_at + ) VALUES ('external-approval','external-revision','external-request',?1, + 'delete-task','member-w1','worker-delete-cascade', + 'root-delete-cascade','coordinator','superseded','Historical notes',?2, + '# historical corrupt path',?3)", + params![&run.id, external_notes.to_string_lossy().as_ref(), &now], + ) + .unwrap(); conn.execute( "INSERT INTO agent_org_recovery_attempts (org_run_id, action_kind, target_key, reason_fingerprint, attempts, @@ -181,11 +197,18 @@ fn delete_by_id_cascades_all_run_owned_state_and_plan_artifact() { params![&run.id, &now], ) .unwrap(); + conn.execute( + "INSERT INTO agent_org_task_run_schema_migrations + (name, org_run_id, applied_at) + VALUES ('delete-test', ?1, ?2)", + params![&run.id, &now], + ) + .unwrap(); conn.execute( "INSERT INTO session_turn_intents - (session_id,turn_intent_id,source,status,created_at,updated_at) - VALUES ('worker-delete-cascade','delete-intent','resume','queued',?1,?1)", - params![&now], + (session_id,turn_intent_id,org_run_id,source,status,created_at,updated_at) + VALUES ('worker-delete-cascade','delete-intent',?1,'resume','queued',?2,?2)", + params![&run.id, &now], ) .unwrap(); @@ -199,6 +222,7 @@ fn delete_by_id_cascades_all_run_owned_state_and_plan_artifact() { "agent_member_interventions", "agent_org_plan_approvals", "agent_org_recovery_attempts", + "agent_org_task_run_schema_migrations", ] { let count: i64 = conn .query_row( @@ -220,6 +244,146 @@ fn delete_by_id_cascades_all_run_owned_state_and_plan_artifact() { .unwrap(); assert_eq!(intent_count, 0); assert!(!plan_path.exists()); + assert_eq!( + std::fs::read_to_string(&external_notes).expect("read external notes after deletion"), + "user-owned notes", + "run deletion must never remove an unmanaged historical path" + ); +} + +#[test] +fn delete_by_id_preserves_nested_run_intents_and_finality_isolation() { + let _sandbox = test_helpers::test_env::sandbox(); + let org = sample_org(); + let outer = create_run_for_root(&org, "outer-root"); + upsert_session_row_full("outer-root", None, Some("agent-coord"), "idle"); + upsert_session_row_for_member( + "outer-worker", + Some("outer-root"), + Some("agent-w1"), + Some("member-w1"), + "idle", + ); + + let nested = create_run_for_root(&org, "nested-root"); + // A nested run root is deliberately also a descendant in the session UI + // tree. Session ancestry is presentation/navigation state, not ownership. + upsert_session_row_full( + "nested-root", + Some("outer-worker"), + Some("agent-coord"), + "idle", + ); + upsert_session_row_for_member( + "nested-worker", + Some("nested-root"), + Some("agent-w1"), + Some("member-w1"), + "running", + ); + + let outer_workers = AgentOrgRunStore::list_descendant_worker_sessions(&outer.id) + .expect("list only sessions owned by the outer run"); + assert_eq!(outer_workers.len(), 1); + assert_eq!(outer_workers[0].session_id, "outer-worker"); + assert_eq!(outer_workers[0].status, SessionStatus::Idle); + + let conn = database::db::get_connection().expect("test sqlite connection"); + let now = chrono::Utc::now().to_rfc3339(); + conn.execute( + "INSERT INTO session_turn_intents + (session_id,turn_intent_id,org_run_id,source,status,created_at,updated_at) + VALUES ('outer-worker','outer-wake',?1,'resume','queued',?3,?3), + ('nested-root','nested-turn',?2,'agent_org','queued',?3,?3)", + params![&outer.id, &nested.id, &now], + ) + .expect("seed independently owned intents"); + + let outer_assessment = + AgentOrgRunStore::assess_run_finality(&outer.id).expect("assess outer run finality"); + assert_eq!( + outer_assessment.facts.in_flight_turn_intent_count, 1, + "nested run work must not block outer run finality" + ); + assert_eq!(outer_assessment.facts.worker_sessions.len(), 1); + assert_eq!( + outer_assessment.facts.worker_sessions[0].session_id, "outer-worker", + "a Running worker owned by a nested run must not block outer finality" + ); + + AgentOrgRunStore::delete_by_id(&outer.id).expect("delete outer run"); + + assert!(load_by_id(&outer.id).unwrap().is_none()); + assert!(load_by_id(&nested.id).unwrap().is_some()); + let outer_intent_count: i64 = conn + .query_row( + "SELECT COUNT(*) FROM session_turn_intents WHERE org_run_id=?1", + params![&outer.id], + |row| row.get(0), + ) + .expect("count outer intents"); + let nested_intent_count: i64 = conn + .query_row( + "SELECT COUNT(*) FROM session_turn_intents WHERE org_run_id=?1", + params![&nested.id], + |row| row.get(0), + ) + .expect("count nested intents"); + assert_eq!(outer_intent_count, 0); + assert_eq!(nested_intent_count, 1); +} + +#[test] +fn recursive_session_queries_terminate_on_parent_cycle() { + let _sandbox = test_helpers::test_env::sandbox(); + let org = sample_org(); + let run = create_run_for_root(&org, "cycle-root"); + upsert_session_row_full("cycle-root", Some("cycle-b"), Some("agent-coord"), "idle"); + upsert_session_row_for_member( + "cycle-a", + Some("cycle-root"), + Some("agent-w1"), + Some("member-w1"), + "idle", + ); + upsert_session_row_for_member( + "cycle-b", + Some("cycle-a"), + Some("agent-w2"), + Some("member-w2"), + "idle", + ); + + let descendants = AgentOrgRunStore::list_descendant_worker_sessions(&run.id) + .expect("cyclic descendant scan terminates"); + assert!( + descendants.len() <= 3, + "cycle must not duplicate descendants" + ); + AgentOrgRunStore::assess_run_finality(&run.id).expect("cyclic finality scan terminates"); + + let conn = database::db::get_connection().expect("test sqlite connection"); + let now = chrono::Utc::now().to_rfc3339(); + for session_id in ["cycle-root", "cycle-a", "cycle-b"] { + conn.execute( + "INSERT INTO session_turn_intents + (session_id,turn_intent_id,org_run_id,source,status,created_at,updated_at) + VALUES (?1,?2,?3,'agent_org','queued',?4,?4)", + params![session_id, format!("intent-{session_id}"), &run.id, &now], + ) + .expect("seed cyclic session intent"); + } + + AgentOrgRunStore::delete_by_id(&run.id).expect("delete cyclic run"); + let remaining: i64 = conn + .query_row( + "SELECT COUNT(*) FROM session_turn_intents + WHERE session_id IN ('cycle-root','cycle-a','cycle-b')", + [], + |row| row.get(0), + ) + .expect("count cyclic intents"); + assert_eq!(remaining, 0); } fn upsert_session_row(session_id: &str, parent_session_id: Option<&str>) { @@ -808,9 +972,9 @@ fn reconcile_run_finality_completes_run_when_all_tasks_completed() { let now = chrono::Utc::now().to_rfc3339(); conn.execute( "INSERT INTO session_turn_intents ( - session_id, turn_intent_id, source, status, created_at, updated_at - ) VALUES (?1, 'final-turn', 'agent_org', 'optimistic', ?2, ?2)", - params!["coord-root-final-complete", &now], + session_id, turn_intent_id, org_run_id, source, status, created_at, updated_at + ) VALUES (?1, 'final-turn', ?2, 'agent_org', 'optimistic', ?3, ?3)", + params!["coord-root-final-complete", &run.id, &now], ) .expect("seed pending turn intent"); for pending_status in ["optimistic", "queued", "running"] { @@ -960,6 +1124,100 @@ fn reconcile_completes_normal_idle_run_only_after_inbox_is_drained() { ); } +#[test] +fn resolved_undeliverable_inbox_stays_unread_but_no_longer_blocks_finality() { + use crate::coordination::agent_inbox::{ + AgentInboxDeliveryResolutionKind, AgentInboxStore, AgentMessage, ResolveInboxDeliveryParams, + }; + use crate::coordination::agent_org_tasks::{AgentOrgTaskStore, CreateTaskParams, TaskStatus}; + + let _sandbox = test_helpers::test_env::sandbox(); + let org = sample_org(); + let run = create_run_for_root(&org, "coord-root-resolved-inbox"); + upsert_session_row_full( + "coord-root-resolved-inbox", + None, + Some("agent-coord"), + "idle", + ); + upsert_session_row_for_member( + "worker-resolved-inbox", + Some("coord-root-resolved-inbox"), + Some("agent-w1"), + Some("member-w1"), + "idle", + ); + AgentOrgTaskStore::create(CreateTaskParams { + id: "resolved-inbox-done".into(), + org_run_id: run.id.clone(), + subject: "done".into(), + description: String::new(), + active_form: None, + owner: Some("member-w1".into()), + status: TaskStatus::Completed, + blocks: Vec::new(), + blocked_by: Vec::new(), + metadata: None, + }) + .expect("create completed task"); + mark_coordinator_observed_current_work(&run.id); + stamp_coordinator_terminal_turn("coord-root-resolved-inbox"); + + let message = AgentMessage::Plain { + summary: "Undeliverable historical row".into(), + text: "Keep this exact evidence unread".into(), + }; + let conn = database::db::get_connection().expect("test sqlite connection"); + conn.execute( + "INSERT INTO agent_inbox ( + recipient_agent_id, recipient_member_id, + sender_agent_id, sender_member_id, org_run_id, + payload_kind, payload_json, created_at + ) VALUES ( + 'removed-agent', NULL, + 'agent-coord', 'coordinator', ?1, + 'plain', ?2, ?3 + )", + params![ + &run.id, + serde_json::to_string(&message).unwrap(), + chrono::Utc::now().to_rfc3339(), + ], + ) + .expect("seed historical orphan row"); + let inbox_id = conn.last_insert_rowid(); + + let before = AgentOrgRunStore::assess_run_finality(&run.id).expect("assess before repair"); + assert_eq!(before.facts.unread_inbox_count, 1); + assert_eq!(before.decision, AgentOrgFinalityDecision::KeepRunning); + + AgentInboxStore::resolve_delivery(ResolveInboxDeliveryParams { + inbox_id, + org_run_id: run.id.clone(), + resolved_by_member_id: COORDINATOR_MEMBER_ID.into(), + resolution_kind: AgentInboxDeliveryResolutionKind::Cancelled, + reason: "Removed recipient and work intentionally abandoned".into(), + replacement_inbox_id: None, + replacement_task_id: None, + }) + .expect("resolve undeliverable delivery"); + + let after = AgentOrgRunStore::assess_run_finality(&run.id).expect("assess after repair"); + assert_eq!(after.facts.unread_inbox_count, 0); + assert_eq!(after.decision, AgentOrgFinalityDecision::Complete); + assert_eq!( + AgentOrgRunStore::reconcile_run_finality(&run.id).expect("reconcile repaired run"), + Some(AgentOrgRunStatus::Completed) + ); + let evidence = AgentInboxStore::get_by_id_for_run(&run.id, inbox_id) + .unwrap() + .unwrap(); + assert!( + evidence.read_at.is_none(), + "repair must not forge a read receipt" + ); +} + #[test] fn startup_reconcile_completes_empty_board_with_explicit_completion_intent() { let _sandbox = test_helpers::test_env::sandbox(); diff --git a/src-tauri/crates/agent-core/src/core/coordination/agent_org_runs/worker.rs b/src-tauri/crates/agent-core/src/core/coordination/agent_org_runs/worker.rs index a1a955e32..93fb54aad 100644 --- a/src-tauri/crates/agent-core/src/core/coordination/agent_org_runs/worker.rs +++ b/src-tauri/crates/agent-core/src/core/coordination/agent_org_runs/worker.rs @@ -28,3 +28,95 @@ pub struct WorkerSessionRuntime { #[serde(skip_serializing_if = "Option::is_none")] pub intervention: Option, } + +/// Whether the freshest persisted session for one Agent Org member can +/// receive a recovery-derived Inbox row and then be woken safely. +/// +/// Recovery analyzers work from snapshots. A member can be paused, archived, +/// replaced by an unsupported CLI session, or start another turn before the +/// executor acquires the writer lock. Every recovery outbox producer shares +/// this final transaction-time predicate so stale plans cannot manufacture +/// new orphan Inbox rows. +pub(crate) fn recovery_dispatch_recipient_is_available( + sessions: &[WorkerSessionRuntime], + member_id: &str, + recipient_agent_id: &str, +) -> bool { + sessions + .iter() + .find(|session| session.member_id.as_deref() == Some(member_id)) + .is_some_and(|session| { + session.cli_agent_type.is_none() + && session.agent_definition_id.as_deref() == Some(recipient_agent_id) + && session.status.is_agent_org_wakeable() + }) +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::core::session::SessionStatus; + + fn runtime(status: SessionStatus) -> WorkerSessionRuntime { + WorkerSessionRuntime { + agent_definition_id: Some("agent-a".to_string()), + cli_agent_type: None, + member_id: Some("member-a".to_string()), + session_id: "session-a".to_string(), + parent_session_id: Some("root".to_string()), + status, + updated_at: "2026-07-17T00:00:00Z".to_string(), + intervention: None, + } + } + + #[test] + fn recovery_recipient_requires_wakeable_rust_session_and_exact_identity() { + for status in [ + SessionStatus::Idle, + SessionStatus::Completed, + SessionStatus::Failed, + SessionStatus::Cancelled, + SessionStatus::Abandoned, + SessionStatus::Timeout, + ] { + assert!(recovery_dispatch_recipient_is_available( + &[runtime(status)], + "member-a", + "agent-a" + )); + } + for status in [ + SessionStatus::Pending, + SessionStatus::Running, + SessionStatus::WaitingForUser, + SessionStatus::WaitingForFunds, + SessionStatus::Paused, + SessionStatus::Archived, + ] { + assert!(!recovery_dispatch_recipient_is_available( + &[runtime(status)], + "member-a", + "agent-a" + )); + } + + let mut cli = runtime(SessionStatus::Idle); + cli.cli_agent_type = Some("claude_code".to_string()); + assert!(!recovery_dispatch_recipient_is_available( + &[cli], + "member-a", + "agent-a" + )); + assert!(!recovery_dispatch_recipient_is_available( + &[runtime(SessionStatus::Idle)], + "member-a", + "agent-b" + )); + assert!(!recovery_dispatch_recipient_is_available( + &[runtime(SessionStatus::Idle)], + "member-b", + "agent-a" + )); + } +} diff --git a/src-tauri/crates/agent-core/src/core/coordination/agent_org_tasks/helpers.rs b/src-tauri/crates/agent-core/src/core/coordination/agent_org_tasks/helpers.rs index e34997a9d..f8b5a2ff4 100644 --- a/src-tauri/crates/agent-core/src/core/coordination/agent_org_tasks/helpers.rs +++ b/src-tauri/crates/agent-core/src/core/coordination/agent_org_tasks/helpers.rs @@ -9,7 +9,16 @@ pub(super) fn now_rfc3339() -> String { } pub(super) fn encode_json_array(values: &[String]) -> Result { - serde_json::to_string(values).map_err(|err| format!("encode JSON array: {err}")) + let encoded = + serde_json::to_string(values).map_err(|err| format!("encode JSON array: {err}"))?; + let max_bytes = crate::coordination::agent_org_payload_limits::TASK_DEPENDENCY_JSON_MAX_BYTES; + if encoded.len() > max_bytes { + return Err(format!( + "encoded task dependency array must be <= {max_bytes} bytes (got {} bytes)", + encoded.len() + )); + } + Ok(encoded) } pub(super) fn decode_json_array(raw: &str) -> Result, String> { diff --git a/src-tauri/crates/agent-core/src/core/coordination/agent_org_tasks/mod.rs b/src-tauri/crates/agent-core/src/core/coordination/agent_org_tasks/mod.rs index 1084e9ce8..5c275ada9 100644 --- a/src-tauri/crates/agent-core/src/core/coordination/agent_org_tasks/mod.rs +++ b/src-tauri/crates/agent-core/src/core/coordination/agent_org_tasks/mod.rs @@ -7,13 +7,19 @@ //! Org task tools and recovery paths. Ownerless tasks are durable //! "awaiting assignment" rows; workers never claim them autonomously. -use std::collections::HashSet; +use std::collections::{HashMap, HashSet}; -use rusqlite::{Connection, Result as SqliteResult}; +use database::db::{get_connection, with_sessions_writer}; +use rusqlite::{params, Connection, Result as SqliteResult}; + +use crate::coordination::agent_org_runs::{ + recovery_dispatch_recipient_is_available, AgentOrgRunStore, +}; pub(super) mod graph; pub(super) mod helpers; mod store; +pub(crate) use graph::validate_dependency_graph; pub use graph::TaskGraphIndex; pub use store::AgentOrgTaskStore; @@ -21,15 +27,332 @@ pub use store::AgentOrgTaskStore; mod tests; pub const TASK_DEPENDENCY_CYCLE_ERROR: &str = "task_dependency_cycle"; +pub const TASK_DEPENDENCY_LIMIT_ERROR: &str = "task_dependency_limit"; +pub const TASK_RUN_TASK_LIMIT_ERROR: &str = "task_run_task_limit"; pub const TASK_GRAPH_OPEN_WORK_CONFLICT_ERROR: &str = "task_graph_unlisted_open_tasks"; pub const TASK_COMPLETED_IMMUTABLE_ERROR: &str = "task_completed_immutable"; pub const TASK_MUTATION_CONFLICT_ERROR: &str = "task_mutation_conflict"; pub const TASK_DELETE_HAS_DEPENDENTS_ERROR: &str = "task_delete_has_dependents"; +pub const TASK_DELETE_IS_DELIVERY_REPLACEMENT_ERROR: &str = "task_delete_is_delivery_replacement"; pub const TASK_METADATA_ELIGIBLE_MEMBER_IDS: &str = "eligible_member_ids"; pub const TASK_METADATA_REQUIRED_ROLE: &str = "required_role"; pub const TASK_METADATA_OUTPUT: &str = "output"; pub const TASK_METADATA_EXECUTION_MODE: &str = "execution_mode"; +/// SQL predicate shared by finality and watchdog repair discovery. +/// +/// Historical/manual SQLite rows can bypass the typed write boundary. Keep +/// this predicate in one place so the finality count and the watchdog's +/// concrete repair identities cannot disagree about whether a row is safe to +/// deserialize. The numeric values are interpolated from the same payload +/// constants used by new writes. +pub(crate) fn corrupt_task_row_predicate_sql() -> String { + use crate::coordination::agent_org_payload_limits as limits; + + format!( + r#"( + status NOT IN ('pending','in_progress','completed') + OR (status='in_progress' AND owner IS NULL) + OR trim(id)='' + OR id<>trim(id) + OR length(id)>{id_chars} + OR length(CAST(id AS BLOB))>{id_bytes} + OR trim(subject)='' + OR length(subject)>{subject_chars} + OR length(CAST(subject AS BLOB))>{subject_bytes} + OR length(description)>{description_chars} + OR length(CAST(description AS BLOB))>{description_bytes} + OR (active_form IS NOT NULL AND ( + length(active_form)>{active_chars} + OR length(CAST(active_form AS BLOB))>{active_bytes} + )) + OR (owner IS NOT NULL AND ( + trim(owner)='' OR owner<>trim(owner) OR length(owner)>{id_chars} + OR length(CAST(owner AS BLOB))>{id_bytes} + )) + OR length(created_at)>{timestamp_chars} + OR length(CAST(created_at AS BLOB))>{timestamp_bytes} + OR datetime(created_at) IS NULL + OR length(updated_at)>{timestamp_chars} + OR length(CAST(updated_at AS BLOB))>{timestamp_bytes} + OR datetime(updated_at) IS NULL + OR CASE WHEN length(CAST(blocks_json AS BLOB))>{dependency_json_bytes} + THEN 1 ELSE json_valid(blocks_json)=0 END + OR CASE WHEN length(CAST(blocked_by_json AS BLOB))>{dependency_json_bytes} + THEN 1 ELSE json_valid(blocked_by_json)=0 END + OR (metadata_json IS NOT NULL AND CASE + WHEN length(CAST(metadata_json AS BLOB))>{metadata_bytes} + THEN 1 ELSE json_valid(metadata_json)=0 END) + OR json_type(CASE WHEN length(CAST(blocks_json AS BLOB))<={dependency_json_bytes} AND json_valid(blocks_json)=1 THEN blocks_json ELSE '[]' END)<>'array' + OR json_type(CASE WHEN length(CAST(blocked_by_json AS BLOB))<={dependency_json_bytes} AND json_valid(blocked_by_json)=1 THEN blocked_by_json ELSE '[]' END)<>'array' + OR (metadata_json IS NOT NULL AND json_type( + CASE WHEN length(CAST(metadata_json AS BLOB))<={metadata_bytes} AND json_valid(metadata_json)=1 THEN metadata_json ELSE '{{}}' END + )<>'object') + OR EXISTS ( + SELECT 1 FROM json_each( + CASE WHEN length(CAST(blocks_json AS BLOB))<={dependency_json_bytes} AND json_valid(blocks_json)=1 THEN blocks_json ELSE '[]' END + ) WHERE type<>'text' OR trim(value)='' + OR value<>trim(value) + OR length(value)>{id_chars} + OR length(CAST(value AS BLOB))>{id_bytes} + ) + OR EXISTS ( + SELECT 1 FROM json_each( + CASE WHEN length(CAST(blocked_by_json AS BLOB))<={dependency_json_bytes} AND json_valid(blocked_by_json)=1 THEN blocked_by_json ELSE '[]' END + ) WHERE type<>'text' OR trim(value)='' + OR value<>trim(value) + OR length(value)>{id_chars} + OR length(CAST(value AS BLOB))>{id_bytes} + ) + OR (SELECT COUNT(*) FROM json_each( + CASE WHEN length(CAST(blocks_json AS BLOB))<={dependency_json_bytes} AND json_valid(blocks_json)=1 THEN blocks_json ELSE '[]' END + ))>{dependency_count} + OR (SELECT COUNT(*) FROM json_each( + CASE WHEN length(CAST(blocked_by_json AS BLOB))<={dependency_json_bytes} AND json_valid(blocked_by_json)=1 THEN blocked_by_json ELSE '[]' END + ))>{dependency_count} + OR (SELECT COALESCE(SUM(length(value)),0) FROM json_each( + CASE WHEN length(CAST(blocks_json AS BLOB))<={dependency_json_bytes} AND json_valid(blocks_json)=1 THEN blocks_json ELSE '[]' END + ) WHERE type='text')>{dependency_chars} + OR (SELECT COALESCE(SUM(length(value)),0) FROM json_each( + CASE WHEN length(CAST(blocked_by_json AS BLOB))<={dependency_json_bytes} AND json_valid(blocked_by_json)=1 THEN blocked_by_json ELSE '[]' END + ) WHERE type='text')>{dependency_chars} + OR (SELECT COALESCE(SUM(length(CAST(value AS BLOB))),0) FROM json_each( + CASE WHEN length(CAST(blocks_json AS BLOB))<={dependency_json_bytes} AND json_valid(blocks_json)=1 THEN blocks_json ELSE '[]' END + ) WHERE type='text')>{dependency_bytes} + OR (SELECT COALESCE(SUM(length(CAST(value AS BLOB))),0) FROM json_each( + CASE WHEN length(CAST(blocked_by_json AS BLOB))<={dependency_json_bytes} AND json_valid(blocked_by_json)=1 THEN blocked_by_json ELSE '[]' END + ) WHERE type='text')>{dependency_bytes} + OR (json_type( + CASE WHEN json_valid(metadata_json)=1 THEN metadata_json ELSE '{{}}' END, + '$.eligible_member_ids' + ) IS NOT NULL + AND json_type( + CASE WHEN json_valid(metadata_json)=1 THEN metadata_json ELSE '{{}}' END, + '$.eligible_member_ids' + )<>'array') + OR EXISTS ( + SELECT 1 FROM json_each( + CASE WHEN json_valid(metadata_json)=1 THEN metadata_json ELSE '{{}}' END, + '$.eligible_member_ids' + ) WHERE type<>'text' OR trim(value)='' + OR value<>trim(value) + OR length(value)>{id_chars} + OR length(CAST(value AS BLOB))>{id_bytes} + ) + OR (SELECT COUNT(*) FROM json_each( + CASE WHEN json_valid(metadata_json)=1 THEN metadata_json ELSE '{{}}' END, + '$.eligible_member_ids' + ))>{eligibility_count} + OR (SELECT COALESCE(SUM(length(value)),0) FROM json_each( + CASE WHEN json_valid(metadata_json)=1 THEN metadata_json ELSE '{{}}' END, + '$.eligible_member_ids' + ) WHERE type='text')>{eligibility_chars} + OR (SELECT COALESCE(SUM(length(CAST(value AS BLOB))),0) FROM json_each( + CASE WHEN json_valid(metadata_json)=1 THEN metadata_json ELSE '{{}}' END, + '$.eligible_member_ids' + ) WHERE type='text')>{eligibility_bytes} + OR (json_type( + CASE WHEN json_valid(metadata_json)=1 THEN metadata_json ELSE '{{}}' END, + '$.required_role' + ) IS NOT NULL + AND (json_type( + CASE WHEN json_valid(metadata_json)=1 THEN metadata_json ELSE '{{}}' END, + '$.required_role' + )<>'text' + OR trim(json_extract( + CASE WHEN json_valid(metadata_json)=1 THEN metadata_json ELSE '{{}}' END, + '$.required_role' + ))='' + OR length(json_extract( + CASE WHEN json_valid(metadata_json)=1 THEN metadata_json ELSE '{{}}' END, + '$.required_role' + ))>{role_chars} + OR length(CAST(json_extract( + CASE WHEN json_valid(metadata_json)=1 THEN metadata_json ELSE '{{}}' END, + '$.required_role' + ) AS BLOB))>{role_bytes})) + OR (json_type( + CASE WHEN json_valid(metadata_json)=1 THEN metadata_json ELSE '{{}}' END, + '$.execution_mode' + ) IS NOT NULL + AND (json_type( + CASE WHEN json_valid(metadata_json)=1 THEN metadata_json ELSE '{{}}' END, + '$.execution_mode' + )<>'text' + OR json_extract( + CASE WHEN json_valid(metadata_json)=1 THEN metadata_json ELSE '{{}}' END, + '$.execution_mode' + ) NOT IN ('build','plan'))) + OR (json_type( + CASE WHEN json_valid(metadata_json)=1 THEN metadata_json ELSE '{{}}' END, + '$.output' + ) IS NOT NULL + AND ( + json_type( + CASE WHEN json_valid(metadata_json)=1 THEN metadata_json ELSE '{{}}' END, + '$.output' + )<>'object' + OR status<>'completed' + OR json_type( + CASE WHEN json_valid(metadata_json)=1 THEN metadata_json ELSE '{{}}' END, + '$.output.summary' + ) IS NULL + OR json_type( + CASE WHEN json_valid(metadata_json)=1 THEN metadata_json ELSE '{{}}' END, + '$.output.summary' + )<>'text' + OR trim(json_extract( + CASE WHEN json_valid(metadata_json)=1 THEN metadata_json ELSE '{{}}' END, + '$.output.summary' + ))='' + OR length(json_extract( + CASE WHEN json_valid(metadata_json)=1 THEN metadata_json ELSE '{{}}' END, + '$.output.summary' + ))>{output_summary_chars} + OR length(CAST(json_extract( + CASE WHEN json_valid(metadata_json)=1 THEN metadata_json ELSE '{{}}' END, + '$.output.summary' + ) AS BLOB))>{output_summary_bytes} + OR (json_type( + CASE WHEN json_valid(metadata_json)=1 THEN metadata_json ELSE '{{}}' END, + '$.output.content' + ) IS NOT NULL + AND json_type( + CASE WHEN json_valid(metadata_json)=1 THEN metadata_json ELSE '{{}}' END, + '$.output.content' + ) NOT IN ('null','text')) + OR length(COALESCE(json_extract( + CASE WHEN json_valid(metadata_json)=1 THEN metadata_json ELSE '{{}}' END, + '$.output.content' + ),''))>{output_content_chars} + OR length(CAST(COALESCE(json_extract( + CASE WHEN json_valid(metadata_json)=1 THEN metadata_json ELSE '{{}}' END, + '$.output.content' + ),'') AS BLOB))>{output_content_bytes} + OR json_type( + CASE WHEN json_valid(metadata_json)=1 THEN metadata_json ELSE '{{}}' END, + '$.output.artifactIds' + ) IS NULL + OR json_type( + CASE WHEN json_valid(metadata_json)=1 THEN metadata_json ELSE '{{}}' END, + '$.output.artifactIds' + )<>'array' + OR (SELECT COUNT(*) FROM json_each( + CASE WHEN json_type( + CASE WHEN json_valid(metadata_json)=1 THEN metadata_json ELSE '{{}}' END, + '$.output.artifactIds' + )='array' THEN json_extract( + CASE WHEN json_valid(metadata_json)=1 THEN metadata_json ELSE '{{}}' END, + '$.output.artifactIds' + ) ELSE '[]' END + ))>{artifact_count} + OR EXISTS (SELECT 1 FROM json_each( + CASE WHEN json_type( + CASE WHEN json_valid(metadata_json)=1 THEN metadata_json ELSE '{{}}' END, + '$.output.artifactIds' + )='array' THEN json_extract( + CASE WHEN json_valid(metadata_json)=1 THEN metadata_json ELSE '{{}}' END, + '$.output.artifactIds' + ) ELSE '[]' END + ) WHERE type<>'text' OR trim(value)='' + OR length(value)>{artifact_chars} + OR length(CAST(value AS BLOB))>{artifact_bytes}) + OR (SELECT COALESCE(SUM(length(value)),0) FROM json_each( + CASE WHEN json_type( + CASE WHEN json_valid(metadata_json)=1 THEN metadata_json ELSE '{{}}' END, + '$.output.artifactIds' + )='array' THEN json_extract( + CASE WHEN json_valid(metadata_json)=1 THEN metadata_json ELSE '{{}}' END, + '$.output.artifactIds' + ) ELSE '[]' END + ) WHERE type='text')>{artifact_total_chars} + OR (SELECT COALESCE(SUM(length(CAST(value AS BLOB))),0) FROM json_each( + CASE WHEN json_type( + CASE WHEN json_valid(metadata_json)=1 THEN metadata_json ELSE '{{}}' END, + '$.output.artifactIds' + )='array' THEN json_extract( + CASE WHEN json_valid(metadata_json)=1 THEN metadata_json ELSE '{{}}' END, + '$.output.artifactIds' + ) ELSE '[]' END + ) WHERE type='text')>{artifact_total_bytes} + OR json_type( + CASE WHEN json_valid(metadata_json)=1 THEN metadata_json ELSE '{{}}' END, + '$.output.producedByMemberId' + ) IS NULL + OR json_type( + CASE WHEN json_valid(metadata_json)=1 THEN metadata_json ELSE '{{}}' END, + '$.output.producedByMemberId' + )<>'text' + OR trim(json_extract( + CASE WHEN json_valid(metadata_json)=1 THEN metadata_json ELSE '{{}}' END, + '$.output.producedByMemberId' + ))='' + OR length(json_extract( + CASE WHEN json_valid(metadata_json)=1 THEN metadata_json ELSE '{{}}' END, + '$.output.producedByMemberId' + ))>{id_chars} + OR length(CAST(json_extract( + CASE WHEN json_valid(metadata_json)=1 THEN metadata_json ELSE '{{}}' END, + '$.output.producedByMemberId' + ) AS BLOB))>{id_bytes} + OR json_type( + CASE WHEN json_valid(metadata_json)=1 THEN metadata_json ELSE '{{}}' END, + '$.output.producedAt' + ) IS NULL + OR json_type( + CASE WHEN json_valid(metadata_json)=1 THEN metadata_json ELSE '{{}}' END, + '$.output.producedAt' + )<>'text' + OR datetime(json_extract( + CASE WHEN json_valid(metadata_json)=1 THEN metadata_json ELSE '{{}}' END, + '$.output.producedAt' + )) IS NULL + OR (json_extract( + CASE WHEN json_valid(metadata_json)=1 THEN metadata_json ELSE '{{}}' END, + '$.output.producedAt' + ) NOT GLOB '*T*Z' + AND json_extract( + CASE WHEN json_valid(metadata_json)=1 THEN metadata_json ELSE '{{}}' END, + '$.output.producedAt' + ) NOT GLOB '*T*+??:??' + AND json_extract( + CASE WHEN json_valid(metadata_json)=1 THEN metadata_json ELSE '{{}}' END, + '$.output.producedAt' + ) NOT GLOB '*T*-??:??') + )) + )"#, + id_chars = limits::TASK_IDENTIFIER_MAX_CHARS, + id_bytes = limits::TASK_IDENTIFIER_MAX_BYTES, + subject_chars = limits::TASK_SUBJECT_MAX_CHARS, + subject_bytes = limits::TASK_SUBJECT_MAX_BYTES, + description_chars = limits::TASK_DESCRIPTION_MAX_CHARS, + description_bytes = limits::TASK_DESCRIPTION_MAX_BYTES, + active_chars = limits::TASK_ACTIVE_FORM_MAX_CHARS, + active_bytes = limits::TASK_ACTIVE_FORM_MAX_BYTES, + metadata_bytes = limits::TASK_METADATA_MAX_BYTES, + dependency_json_bytes = limits::TASK_DEPENDENCY_JSON_MAX_BYTES, + dependency_count = limits::TASK_DEPENDENCY_MAX_COUNT, + dependency_chars = limits::TASK_DEPENDENCY_TOTAL_MAX_CHARS, + dependency_bytes = limits::TASK_DEPENDENCY_TOTAL_MAX_BYTES, + eligibility_count = limits::TASK_ELIGIBILITY_MAX_COUNT, + eligibility_chars = limits::TASK_ELIGIBILITY_TOTAL_MAX_CHARS, + eligibility_bytes = limits::TASK_ELIGIBILITY_TOTAL_MAX_BYTES, + role_chars = limits::TASK_REQUIRED_ROLE_MAX_CHARS, + role_bytes = limits::TASK_REQUIRED_ROLE_MAX_BYTES, + output_summary_chars = limits::TASK_OUTPUT_SUMMARY_MAX_CHARS, + output_summary_bytes = limits::TASK_OUTPUT_SUMMARY_MAX_BYTES, + output_content_chars = limits::TASK_OUTPUT_CONTENT_MAX_CHARS, + output_content_bytes = limits::TASK_OUTPUT_CONTENT_MAX_BYTES, + artifact_count = limits::TASK_ARTIFACT_ID_MAX_COUNT, + artifact_chars = limits::TASK_ARTIFACT_ID_MAX_CHARS, + artifact_bytes = limits::TASK_ARTIFACT_ID_MAX_BYTES, + artifact_total_chars = limits::TASK_ARTIFACT_IDS_TOTAL_MAX_CHARS, + artifact_total_bytes = limits::TASK_ARTIFACT_IDS_TOTAL_MAX_BYTES, + timestamp_chars = limits::RFC3339_TIMESTAMP_MAX_CHARS, + timestamp_bytes = limits::RFC3339_TIMESTAMP_MAX_BYTES, + ) +} + /// Transaction-time scheduling decision for a single task created through /// the Agent Org tool boundary. /// @@ -382,6 +705,7 @@ pub fn init_schema(conn: &Connection) -> SqliteResult<()> { )?; add_column_if_missing(conn, "agent_org_tasks", "metadata_json", "TEXT")?; add_column_if_missing(conn, "agent_org_task_events", "actor_member_id", "TEXT")?; + store::normalize_legacy_dependency_rows(conn)?; Ok(()) } @@ -470,6 +794,132 @@ pub fn enqueue_task_assigned_to_with_tasks( ) } +/// Batch recovery assignments for one member in one writer transaction. +/// +/// Assignment actions are already grouped by member. Rechecking the session, +/// task graph, and existing durable deliveries once per member avoids opening +/// one IMMEDIATE transaction and rescanning the whole board for every task. +/// Invalidated tasks are skipped independently; all still-current deliveries +/// commit together. +#[allow(clippy::too_many_arguments)] +pub(crate) fn enqueue_task_assignments_if_still_ready_for_recovery( + org_run_id: &str, + task_ids: &[String], + recipient_agent_id: &str, + recipient_member_id: &str, + sender_agent_id: &str, + sender_member_id: Option<&str>, + assigned_by_display_name: &str, +) -> Result, String> { + if task_ids.is_empty() { + return Ok(Vec::new()); + } + with_sessions_writer(|| -> Result, String> { + let mut conn = get_connection().map_err(|err| err.to_string())?; + let tx = conn + .transaction_with_behavior(rusqlite::TransactionBehavior::Immediate) + .map_err(|err| err.to_string())?; + let running: bool = tx + .query_row( + "SELECT EXISTS( + SELECT 1 FROM agent_org_runs WHERE id=?1 AND status='running' + )", + params![org_run_id], + |row| row.get(0), + ) + .map_err(|err| err.to_string())?; + if !running { + tx.commit().map_err(|err| err.to_string())?; + return Ok(Vec::new()); + } + + let sessions = + AgentOrgRunStore::list_descendant_worker_sessions_with_connection(&tx, org_run_id)?; + if !recovery_dispatch_recipient_is_available( + &sessions, + recipient_member_id, + recipient_agent_id, + ) { + tx.commit().map_err(|err| err.to_string())?; + return Ok(Vec::new()); + } + + let all_tasks = AgentOrgTaskStore::list_with_connection(&tx, org_run_id)?; + let graph = TaskGraphIndex::new(&all_tasks); + let requested_task_ids = task_ids.iter().map(String::as_str).collect::>(); + let current_tasks = all_tasks + .iter() + .filter(|task| requested_task_ids.contains(task.id.as_str())) + .filter(|task| { + task.status == TaskStatus::Pending + && task.owner.as_deref() == Some(recipient_member_id) + && graph.is_ready(task) + }) + .collect::>(); + if current_tasks.is_empty() { + tx.commit().map_err(|err| err.to_string())?; + return Ok(Vec::new()); + } + + // Another producer may have delivered one or more exact assignments + // after the analyzer snapshot. Load the recipient's unread assignment + // set once and reuse those durable rows instead of inserting copies. + let mut existing_stmt = tx + .prepare( + "SELECT id, + CASE + WHEN json_valid(payload_json) + AND json_type(payload_json, '$.task_id')='text' + THEN json_extract(payload_json, '$.task_id') + END + FROM agent_inbox INDEXED BY idx_agent_inbox_run_unread_recipient + WHERE org_run_id=?1 + AND recipient_member_id=?2 + AND payload_kind='task_assigned' + AND read_at IS NULL + AND NOT EXISTS ( + SELECT 1 FROM agent_inbox_delivery_resolutions resolution + WHERE resolution.inbox_id=agent_inbox.id + ) + ORDER BY id ASC", + ) + .map_err(|err| err.to_string())?; + let existing_rows = existing_stmt + .query_map(params![org_run_id, recipient_member_id], |row| { + Ok((row.get::<_, i64>(0)?, row.get::<_, Option>(1)?)) + }) + .map_err(|err| err.to_string())?; + let mut existing_by_task_id = HashMap::new(); + for row in existing_rows { + let (row_id, task_id) = row.map_err(|err| err.to_string())?; + if let Some(task_id) = task_id { + existing_by_task_id.entry(task_id).or_insert(row_id); + } + } + drop(existing_stmt); + + let mut row_ids = Vec::with_capacity(current_tasks.len()); + for task in current_tasks { + if let Some(existing_row_id) = existing_by_task_id.get(&task.id) { + row_ids.push(*existing_row_id); + continue; + } + row_ids.push(enqueue_task_assigned_to_with_tasks_impl( + task, + &all_tasks, + recipient_agent_id, + recipient_member_id, + sender_agent_id, + sender_member_id, + assigned_by_display_name, + TaskAssignedInsert::Transaction(&tx), + )?); + } + tx.commit().map_err(|err| err.to_string())?; + Ok(row_ids) + }) +} + /// Transaction-aware variant used when a task mutation and its TaskAssigned /// outbox row must commit together. #[allow(clippy::too_many_arguments)] diff --git a/src-tauri/crates/agent-core/src/core/coordination/agent_org_tasks/store/create.rs b/src-tauri/crates/agent-core/src/core/coordination/agent_org_tasks/store/create.rs index eb27eb081..365346823 100644 --- a/src-tauri/crates/agent-core/src/core/coordination/agent_org_tasks/store/create.rs +++ b/src-tauri/crates/agent-core/src/core/coordination/agent_org_tasks/store/create.rs @@ -7,6 +7,10 @@ use std::collections::HashSet; use database::db::{get_connection, with_sessions_writer}; use rusqlite::params; +use crate::coordination::agent_org_payload_limits::{ + validate_task_dependency_ids, validate_task_identifier, +}; + use super::super::helpers::{ encode_json_array, encode_metadata, insert_task_history_event, list_tasks_with_conn, now_rfc3339, @@ -17,8 +21,8 @@ use super::super::{ }; use super::dependencies::{canonicalize_dependencies, persist_dependency_projection}; use super::validation::{ - ensure_run_allows_task_mutation, reject_writable_blocks, validate_task_persistence_invariants, - validate_task_text_fields, + ensure_run_allows_task_mutation, ensure_task_run_capacity, reject_writable_blocks, + validate_task_persistence_invariants, validate_task_text_fields, }; use super::AgentOrgTaskStore; @@ -66,9 +70,8 @@ impl AgentOrgTaskStore { scheduling_policy: Option, effects: impl FnOnce(&rusqlite::Connection, &Task, &[Task]) -> Result, ) -> Result<(Task, T), String> { - if params.id.trim().is_empty() { - return Err("task id must be non-empty".into()); - } + validate_task_identifier("task id", ¶ms.id)?; + validate_task_dependency_ids("blocked_by", ¶ms.blocked_by)?; if params.org_run_id.trim().is_empty() { return Err("org_run_id must be non-empty".into()); } @@ -100,6 +103,7 @@ impl AgentOrgTaskStore { )?; let mut candidate_tasks = list_tasks_with_conn(&tx, ¶ms.org_run_id)?; let existing_task_count = candidate_tasks.len(); + ensure_task_run_capacity(existing_task_count, 1)?; candidate_tasks.push(Task { id: params.id.clone(), org_run_id: params.org_run_id.clone(), @@ -208,6 +212,7 @@ impl AgentOrgTaskStore { if params_list.is_empty() { return Err("task graph must contain at least one task".to_string()); } + ensure_task_run_capacity(0, params_list.len())?; let org_run_id = params_list[0].org_run_id.clone(); if org_run_id.trim().is_empty() { return Err("org_run_id must be non-empty".to_string()); @@ -216,9 +221,8 @@ impl AgentOrgTaskStore { if params.org_run_id != org_run_id { return Err("every task in a graph must belong to the same org run".to_string()); } - if params.id.trim().is_empty() { - return Err("task id must be non-empty".to_string()); - } + validate_task_identifier("task id", ¶ms.id)?; + validate_task_dependency_ids("blocked_by", ¶ms.blocked_by)?; validate_task_text_fields( ¶ms.subject, ¶ms.description, @@ -238,6 +242,7 @@ impl AgentOrgTaskStore { ensure_run_allows_task_mutation(&tx, &org_run_id)?; let existing_tasks = list_tasks_with_conn(&tx, &org_run_id)?; + ensure_task_run_capacity(existing_tasks.len(), params_list.len())?; if !allow_parallel_with_existing_open_tasks { let existing_ids = existing_tasks .iter() diff --git a/src-tauri/crates/agent-core/src/core/coordination/agent_org_tasks/store/delete.rs b/src-tauri/crates/agent-core/src/core/coordination/agent_org_tasks/store/delete.rs index e5443ab89..c9a1077f9 100644 --- a/src-tauri/crates/agent-core/src/core/coordination/agent_org_tasks/store/delete.rs +++ b/src-tauri/crates/agent-core/src/core/coordination/agent_org_tasks/store/delete.rs @@ -6,7 +6,10 @@ use database::db::{get_connection, with_sessions_writer}; use rusqlite::params; use super::super::helpers::{insert_task_history_event, list_tasks_with_conn, now_rfc3339}; -use super::super::{TaskGraphIndex, TASK_DELETE_HAS_DEPENDENTS_ERROR, TASK_EVENT_DELETED}; +use super::super::{ + TaskGraphIndex, TASK_DELETE_HAS_DEPENDENTS_ERROR, TASK_DELETE_IS_DELIVERY_REPLACEMENT_ERROR, + TASK_EVENT_DELETED, +}; use super::validation::ensure_run_allows_task_mutation; use super::AgentOrgTaskStore; @@ -70,6 +73,25 @@ impl AgentOrgTaskStore { dependent_task_ids.join(",") )); } + // Fail closed if the delivery-resolution schema is missing or + // unreadable. Treating a schema failure as "not referenced" could + // permanently delete the only durable replacement for an Inbox row. + let is_delivery_replacement: bool = tx + .query_row( + "SELECT EXISTS( + SELECT 1 + FROM agent_inbox_delivery_resolutions + WHERE org_run_id=?1 AND replacement_task_id=?2 + )", + params![org_run_id, task_id], + |row| row.get(0), + ) + .map_err(|err| err.to_string())?; + if is_delivery_replacement { + return Err(format!( + "{TASK_DELETE_IS_DELIVERY_REPLACEMENT_ERROR}: task {task_id} is durable replacement evidence for a resolved Inbox delivery and cannot be deleted" + )); + } let n = tx .execute( "DELETE FROM agent_org_tasks WHERE org_run_id = ?1 AND id = ?2", diff --git a/src-tauri/crates/agent-core/src/core/coordination/agent_org_tasks/store/dependencies.rs b/src-tauri/crates/agent-core/src/core/coordination/agent_org_tasks/store/dependencies.rs index e77b635fb..ef2fc8590 100644 --- a/src-tauri/crates/agent-core/src/core/coordination/agent_org_tasks/store/dependencies.rs +++ b/src-tauri/crates/agent-core/src/core/coordination/agent_org_tasks/store/dependencies.rs @@ -3,10 +3,14 @@ //! projection recomputed here and written back only when it drifts. use rusqlite::params; +use tracing::warn; use super::super::graph::validate_dependency_graph; -use super::super::helpers::encode_json_array; +use super::super::helpers::{encode_json_array, list_tasks_with_conn, now_rfc3339}; use super::super::{Task, TaskGraphIndex}; +use crate::coordination::agent_org_payload_limits::{ + validate_task_dependency_ids, TASK_RUN_MAX_TASKS, +}; pub(super) fn canonicalize_dependencies( tasks: &mut [Task], @@ -20,6 +24,10 @@ pub(super) fn canonicalize_dependencies( } let graph = TaskGraphIndex::new(tasks); graph.apply_projection(tasks); + for task in tasks.iter() { + validate_task_dependency_ids("blocked_by", &task.blocked_by)?; + validate_task_dependency_ids("derived blocks", &task.blocks)?; + } validate_dependency_graph(tasks, org_run_id) } @@ -48,3 +56,138 @@ pub(super) fn persist_dependency_projection( } Ok(()) } + +/// One-time migration for the historical dual-write dependency fields. +/// Legacy `blocks`-only edges are folded into canonical `blocked_by`, then +/// both stored columns are rewritten as a consistent forward/reverse pair. +pub(super) fn normalize_legacy_dependency_rows( + conn: &rusqlite::Connection, +) -> rusqlite::Result<()> { + const MIGRATION_NAME: &str = "canonical_blocked_by_v1"; + conn.execute_batch( + "CREATE TABLE IF NOT EXISTS agent_org_task_run_schema_migrations ( + name TEXT NOT NULL, + org_run_id TEXT NOT NULL, + applied_at TEXT NOT NULL, + PRIMARY KEY (name, org_run_id) + );", + )?; + + let mut after_run_id: Option = None; + loop { + let run_ids = { + let mut stmt = conn.prepare( + "SELECT task.org_run_id + FROM agent_org_tasks task + WHERE NOT EXISTS ( + SELECT 1 FROM agent_org_task_run_schema_migrations migration + WHERE migration.name=?1 + AND migration.org_run_id=task.org_run_id + ) + AND (?2 IS NULL OR task.org_run_id>?2) + GROUP BY task.org_run_id + ORDER BY task.org_run_id + LIMIT 256", + )?; + let rows = stmt.query_map(params![MIGRATION_NAME, after_run_id.as_deref()], |row| { + row.get::<_, String>(0) + })?; + rows.collect::, _>>()? + }; + if run_ids.is_empty() { + break; + } + after_run_id = run_ids.last().cloned(); + + for run_id in run_ids { + conn.execute_batch("BEGIN IMMEDIATE")?; + let normalized = (|| -> Result<(), String> { + let already_applied: bool = conn + .query_row( + "SELECT EXISTS( + SELECT 1 FROM agent_org_task_run_schema_migrations + WHERE name=?1 AND org_run_id=?2 + )", + params![MIGRATION_NAME, &run_id], + |row| row.get(0), + ) + .map_err(|err| err.to_string())?; + if already_applied { + return Ok(()); + } + + if !run_is_safe_for_dependency_normalization(conn, &run_id)? { + return Err( + "task board exceeds current resource/integrity limits; repair is required before dependency normalization" + .to_string(), + ); + } + + let mut tasks = list_tasks_with_conn(conn, &run_id)?; + canonicalize_dependencies(&mut tasks, &run_id)?; + persist_dependency_projection(conn, &tasks)?; + conn.execute( + "INSERT INTO agent_org_task_run_schema_migrations( + name, org_run_id, applied_at + ) VALUES (?1, ?2, ?3)", + params![MIGRATION_NAME, &run_id, now_rfc3339()], + ) + .map_err(|err| err.to_string())?; + Ok(()) + })(); + + match normalized { + Ok(()) => conn.execute_batch("COMMIT")?, + Err(error) => { + let _ = conn.execute_batch("ROLLBACK"); + warn!( + org_run_id = %run_id, + error = %error, + "deferring corrupt Agent Org task board dependency normalization" + ); + } + } + } + } + Ok(()) +} + +/// Preflight historical rows without deserializing their JSON. +pub(super) fn run_is_safe_for_dependency_normalization( + conn: &rusqlite::Connection, + run_id: &str, +) -> Result { + let count: i64 = conn + .query_row( + "SELECT COUNT(*) FROM agent_org_tasks WHERE org_run_id=?1", + params![run_id], + |row| row.get(0), + ) + .map_err(|err| err.to_string())?; + if count > TASK_RUN_MAX_TASKS as i64 { + return Ok(false); + } + let predicate = super::super::corrupt_task_row_predicate_sql(); + let dependency_json_max = + crate::coordination::agent_org_payload_limits::TASK_DEPENDENCY_JSON_MAX_BYTES; + let metadata_max = crate::coordination::agent_org_payload_limits::TASK_METADATA_MAX_BYTES; + let sql = format!( + "SELECT COALESCE(SUM(CASE WHEN {predicate} THEN 1 ELSE 0 END),0) + FROM ( + SELECT id, subject, description, active_form, owner, status, + created_at, updated_at, + CASE WHEN length(CAST(blocks_json AS BLOB))<={dependency_json_max} + THEN blocks_json ELSE '!' END AS blocks_json, + CASE WHEN length(CAST(blocked_by_json AS BLOB))<={dependency_json_max} + THEN blocked_by_json ELSE '!' END AS blocked_by_json, + CASE WHEN metadata_json IS NULL + OR length(CAST(metadata_json AS BLOB))<={metadata_max} + THEN metadata_json ELSE '!' END AS metadata_json + FROM agent_org_tasks WHERE org_run_id=?1 + ) AS bounded_tasks" + ); + let corrupt: i64 = conn + .query_row(&sql, params![run_id], |row| row.get(0)) + .map_err(|err| err.to_string())?; + Ok(corrupt == 0) +} diff --git a/src-tauri/crates/agent-core/src/core/coordination/agent_org_tasks/store/mod.rs b/src-tauri/crates/agent-core/src/core/coordination/agent_org_tasks/store/mod.rs index 5f8a9c6d2..87228aee7 100644 --- a/src-tauri/crates/agent-core/src/core/coordination/agent_org_tasks/store/mod.rs +++ b/src-tauri/crates/agent-core/src/core/coordination/agent_org_tasks/store/mod.rs @@ -26,6 +26,12 @@ mod requeue; mod update; mod validation; +pub(super) fn normalize_legacy_dependency_rows( + conn: &rusqlite::Connection, +) -> rusqlite::Result<()> { + dependencies::normalize_legacy_dependency_rows(conn) +} + // Names referenced with an explicit `super::` prefix inside the submodule // bodies below. Re-binding them here keeps those references verbatim: from a // submodule, `super::` resolves to this module. diff --git a/src-tauri/crates/agent-core/src/core/coordination/agent_org_tasks/store/read.rs b/src-tauri/crates/agent-core/src/core/coordination/agent_org_tasks/store/read.rs index a59b42d77..b9b05c54d 100644 --- a/src-tauri/crates/agent-core/src/core/coordination/agent_org_tasks/store/read.rs +++ b/src-tauri/crates/agent-core/src/core/coordination/agent_org_tasks/store/read.rs @@ -20,6 +20,7 @@ use super::super::{ Task, TaskExecutionMode, TaskGraphIndex, TaskOutputSummary, TaskStatus, TaskSummary, TaskSummaryPage, TASK_METADATA_ELIGIBLE_MEMBER_IDS, }; +use super::validation::ensure_task_rows_safe_for_operational_projection; use super::AgentOrgTaskStore; fn decode_summary_array(raw: String, column: usize) -> rusqlite::Result> { @@ -29,9 +30,24 @@ fn decode_summary_array(raw: String, column: usize) -> rusqlite::Result String { + use crate::coordination::agent_org_payload_limits as limits; + format!( "{alias}.status IN ('pending','in_progress','completed') - AND trim({alias}.id)<>''" + AND trim({alias}.id)<>'' + AND {alias}.id=trim({alias}.id) + AND length({alias}.id)<={} + AND length(CAST({alias}.id AS BLOB))<={} + AND length({alias}.created_at)<={} + AND length(CAST({alias}.created_at AS BLOB))<={} + AND length({alias}.updated_at)<={} + AND length(CAST({alias}.updated_at AS BLOB))<={}", + limits::TASK_IDENTIFIER_MAX_CHARS, + limits::TASK_IDENTIFIER_MAX_BYTES, + limits::RFC3339_TIMESTAMP_MAX_CHARS, + limits::RFC3339_TIMESTAMP_MAX_BYTES, + limits::RFC3339_TIMESTAMP_MAX_CHARS, + limits::RFC3339_TIMESTAMP_MAX_BYTES, ) } @@ -72,6 +88,7 @@ impl AgentOrgTaskStore { conn: &rusqlite::Connection, org_run_id: &str, ) -> Result, String> { + ensure_task_rows_safe_for_operational_projection(conn, org_run_id)?; Self::list_operational_after_validated_with_connection(conn, org_run_id) } @@ -99,38 +116,58 @@ impl AgentOrgTaskStore { ELSE '[]' END, task.created_at, task.updated_at - FROM agent_org_tasks task + FROM ( + SELECT id, org_run_id, subject, description, active_form, + owner, status, created_at, updated_at, + CASE WHEN length(CAST(blocks_json AS BLOB))<=?2 + THEN blocks_json ELSE '!' END AS blocks_json, + CASE WHEN length(CAST(blocked_by_json AS BLOB))<=?2 + THEN blocked_by_json ELSE '!' END AS blocked_by_json, + CASE WHEN metadata_json IS NULL + OR length(CAST(metadata_json AS BLOB))<=?3 + THEN metadata_json ELSE '!' END AS metadata_json + FROM agent_org_tasks + ) task WHERE task.org_run_id=?1 ORDER BY task.created_at ASC, task.id ASC", ) .map_err(|err| err.to_string())?; let rows = stmt - .query_map(params![org_run_id], |row| { - let status_raw: String = row.get(4)?; - let eligible = decode_summary_array(row.get(7)?, 7)?; - let metadata = (!eligible.is_empty()) - .then(|| serde_json::json!({ (TASK_METADATA_ELIGIBLE_MEMBER_IDS): eligible })); - Ok(Task { - id: row.get(0)?, - org_run_id: row.get(1)?, - subject: row.get(2)?, - description: String::new(), - active_form: None, - owner: row.get(3)?, - status: TaskStatus::from_wire(&status_raw).map_err(|error| { - rusqlite::Error::FromSqlConversionFailure( - 4, - rusqlite::types::Type::Text, - error.into(), - ) - })?, - blocks: decode_summary_array(row.get(5)?, 5)?, - blocked_by: decode_summary_array(row.get(6)?, 6)?, - metadata, - created_at: row.get(8)?, - updated_at: row.get(9)?, - }) - }) + .query_map( + params![ + org_run_id, + crate::coordination::agent_org_payload_limits::TASK_DEPENDENCY_JSON_MAX_BYTES + as i64, + crate::coordination::agent_org_payload_limits::TASK_METADATA_MAX_BYTES as i64, + ], + |row| { + let status_raw: String = row.get(4)?; + let eligible = decode_summary_array(row.get(7)?, 7)?; + let metadata = (!eligible.is_empty()).then( + || serde_json::json!({ (TASK_METADATA_ELIGIBLE_MEMBER_IDS): eligible }), + ); + Ok(Task { + id: row.get(0)?, + org_run_id: row.get(1)?, + subject: row.get(2)?, + description: String::new(), + active_form: None, + owner: row.get(3)?, + status: TaskStatus::from_wire(&status_raw).map_err(|error| { + rusqlite::Error::FromSqlConversionFailure( + 4, + rusqlite::types::Type::Text, + error.into(), + ) + })?, + blocks: decode_summary_array(row.get(5)?, 5)?, + blocked_by: decode_summary_array(row.get(6)?, 6)?, + metadata, + created_at: row.get(8)?, + updated_at: row.get(9)?, + }) + }, + ) .map_err(|err| err.to_string())?; let mut tasks = rows .map(|row| row.map_err(|err| err.to_string())) @@ -173,14 +210,30 @@ impl AgentOrgTaskStore { .map(|task_id| { conn.query_row( "SELECT created_at, id FROM agent_org_tasks - WHERE org_run_id=?1 AND id=?2", - params![org_run_id, task_id], + WHERE org_run_id=?1 AND id=?2 + AND length(id)<=?3 AND length(CAST(id AS BLOB))<=?4 + AND length(created_at)<=?5 + AND length(CAST(created_at AS BLOB))<=?6", + params![ + org_run_id, + task_id, + crate::coordination::agent_org_payload_limits::TASK_IDENTIFIER_MAX_CHARS + as i64, + crate::coordination::agent_org_payload_limits::TASK_IDENTIFIER_MAX_BYTES + as i64, + crate::coordination::agent_org_payload_limits::RFC3339_TIMESTAMP_MAX_CHARS + as i64, + crate::coordination::agent_org_payload_limits::RFC3339_TIMESTAMP_MAX_BYTES + as i64, + ], |row| Ok((row.get::<_, String>(0)?, row.get::<_, String>(1)?)), ) .optional() .map_err(|err| err.to_string())? .ok_or_else(|| { - format!("task_list after_task_id '{task_id}' does not exist in this run") + format!( + "task_list after_task_id '{task_id}' does not exist or is corrupt in this run" + ) }) }) .transpose()?; @@ -297,7 +350,23 @@ impl AgentOrgTaskStore { AND json_type(task.metadata_json, '$.output.artifactIds')='array' THEN (SELECT COUNT(*) FROM json_each(task.metadata_json, '$.output.artifactIds') WHERE type='text') ELSE 0 END - FROM agent_org_tasks task + FROM ( + SELECT id, org_run_id, subject, description, active_form, + CASE WHEN owner IS NULL THEN NULL + WHEN trim(owner)<>'' + AND length(owner)<={id_chars} + AND length(CAST(owner AS BLOB))<={id_bytes} + THEN owner ELSE NULL END AS owner, + status, created_at, updated_at, + CASE WHEN length(CAST(blocks_json AS BLOB))<=?11 + THEN blocks_json ELSE '!' END AS blocks_json, + CASE WHEN length(CAST(blocked_by_json AS BLOB))<=?11 + THEN blocked_by_json ELSE '!' END AS blocked_by_json, + CASE WHEN metadata_json IS NULL + OR length(CAST(metadata_json AS BLOB))<=?12 + THEN metadata_json ELSE '!' END AS metadata_json + FROM agent_org_tasks + ) task WHERE task.org_run_id=?1 AND {summary_scalar_predicate} AND (?2 IS NULL OR task.status=?2) @@ -309,6 +378,10 @@ impl AgentOrgTaskStore { ) ORDER BY task.created_at ASC, task.id ASC LIMIT ?7" + .replace("{id_chars}", &crate::coordination::agent_org_payload_limits::TASK_IDENTIFIER_MAX_CHARS.to_string()) + .replace("{id_bytes}", &crate::coordination::agent_org_payload_limits::TASK_IDENTIFIER_MAX_BYTES.to_string()) + .replace("{timestamp_chars}", &crate::coordination::agent_org_payload_limits::RFC3339_TIMESTAMP_MAX_CHARS.to_string()) + .replace("{timestamp_bytes}", &crate::coordination::agent_org_payload_limits::RFC3339_TIMESTAMP_MAX_BYTES.to_string()) .replace("{summary_scalar_predicate}", &summary_scalar_predicate); let mut stmt = conn.prepare(&summary_sql).map_err(|err| err.to_string())?; let rows = stmt @@ -324,6 +397,9 @@ impl AgentOrgTaskStore { TASK_SUMMARY_DEPENDENCY_PREVIEW_MAX_COUNT as i64, TASK_SUMMARY_ELIGIBILITY_PREVIEW_MAX_COUNT as i64, TASK_SUMMARY_ARTIFACT_PREVIEW_MAX_COUNT as i64, + crate::coordination::agent_org_payload_limits::TASK_DEPENDENCY_JSON_MAX_BYTES + as i64, + crate::coordination::agent_org_payload_limits::TASK_METADATA_MAX_BYTES as i64, ], |row| { let status_raw: String = row.get(6)?; @@ -433,14 +509,22 @@ impl AgentOrgTaskStore { "SELECT id FROM agent_org_tasks WHERE org_run_id=?1 AND status<>'completed' AND trim(id)<>'' + AND length(id)<=?3 + AND length(CAST(id AS BLOB))<=?4 ORDER BY created_at ASC, id ASC LIMIT ?2", ) .map_err(|err| err.to_string())?; let rows = stmt - .query_map(params![org_run_id, (bounded_limit + 1) as i64], |row| { - row.get::<_, String>(0) - }) + .query_map( + params![ + org_run_id, + (bounded_limit + 1) as i64, + crate::coordination::agent_org_payload_limits::TASK_IDENTIFIER_MAX_CHARS as i64, + crate::coordination::agent_org_payload_limits::TASK_IDENTIFIER_MAX_BYTES as i64, + ], + |row| row.get::<_, String>(0), + ) .map_err(|err| err.to_string())?; let mut ids = Vec::new(); let mut bytes = 2usize; // surrounding JSON array diff --git a/src-tauri/crates/agent-core/src/core/coordination/agent_org_tasks/store/requeue.rs b/src-tauri/crates/agent-core/src/core/coordination/agent_org_tasks/store/requeue.rs index be87587de..af4a6ece8 100644 --- a/src-tauri/crates/agent-core/src/core/coordination/agent_org_tasks/store/requeue.rs +++ b/src-tauri/crates/agent-core/src/core/coordination/agent_org_tasks/store/requeue.rs @@ -228,3 +228,114 @@ impl AgentOrgTaskStore { Ok(updated_rows) } } + +#[cfg(test)] +mod migration_tests { + use super::*; + + #[test] + fn dependency_migration_skips_corrupt_run_and_normalizes_valid_run() { + let conn = rusqlite::Connection::open_in_memory().expect("open in-memory database"); + super::super::super::init_schema(&conn).expect("create task schema"); + + let now = now_rfc3339(); + for (id, blocks_json, blocked_by_json) in + [("task-a", r#"["task-b"]"#, "[]"), ("task-b", "[]", "[]")] + { + conn.execute( + "INSERT INTO agent_org_tasks ( + id, org_run_id, subject, description, status, + blocks_json, blocked_by_json, created_at, updated_at + ) VALUES (?1, 'valid-run', ?1, '', 'pending', ?2, ?3, ?4, ?4)", + params![id, blocks_json, blocked_by_json, &now], + ) + .expect("seed valid legacy task"); + } + conn.execute( + "INSERT INTO agent_org_tasks ( + id, org_run_id, subject, description, status, + blocks_json, blocked_by_json, created_at, updated_at + ) VALUES ( + 'corrupt-task', 'corrupt-run', 'corrupt', '', 'pending', + 'not-json', '[]', ?1, ?1 + )", + params![&now], + ) + .expect("seed corrupt historical task"); + + super::super::super::init_schema(&conn) + .expect("schema init must survive one corrupt historical run"); + + let (a_blocks, b_blocked_by): (String, String) = conn + .query_row( + "SELECT a.blocks_json, b.blocked_by_json + FROM agent_org_tasks a + JOIN agent_org_tasks b + ON b.org_run_id=a.org_run_id AND b.id='task-b' + WHERE a.org_run_id='valid-run' AND a.id='task-a'", + [], + |row| Ok((row.get(0)?, row.get(1)?)), + ) + .expect("read normalized valid board"); + assert_eq!(a_blocks, r#"["task-b"]"#); + assert_eq!(b_blocked_by, r#"["task-a"]"#); + + let corrupt_blocks: String = conn + .query_row( + "SELECT blocks_json FROM agent_org_tasks + WHERE org_run_id='corrupt-run' AND id='corrupt-task'", + [], + |row| row.get(0), + ) + .expect("corrupt row remains available for runtime repair"); + assert_eq!(corrupt_blocks, "not-json"); + + let valid_marked: bool = conn + .query_row( + "SELECT EXISTS( + SELECT 1 FROM agent_org_task_run_schema_migrations + WHERE name='canonical_blocked_by_v1' AND org_run_id='valid-run' + )", + [], + |row| row.get(0), + ) + .expect("read valid marker"); + let corrupt_marked: bool = conn + .query_row( + "SELECT EXISTS( + SELECT 1 FROM agent_org_task_run_schema_migrations + WHERE name='canonical_blocked_by_v1' AND org_run_id='corrupt-run' + )", + [], + |row| row.get(0), + ) + .expect("read corrupt marker"); + assert!(valid_marked, "healthy run receives its own success marker"); + assert!( + !corrupt_marked, + "corrupt run remains unmarked so a later startup can retry" + ); + + conn.execute( + "UPDATE agent_org_tasks SET blocks_json='[]' + WHERE org_run_id='corrupt-run' AND id='corrupt-task'", + [], + ) + .expect("repair corrupt historical row"); + super::super::super::init_schema(&conn).expect("retry repaired run"); + let corrupt_marked_after_retry: bool = conn + .query_row( + "SELECT EXISTS( + SELECT 1 FROM agent_org_task_run_schema_migrations + WHERE name='canonical_blocked_by_v1' AND org_run_id='corrupt-run' + )", + [], + |row| row.get(0), + ) + .expect("read retry marker"); + assert!( + corrupt_marked_after_retry, + "a repaired run is retried and marked independently" + ); + } +} diff --git a/src-tauri/crates/agent-core/src/core/coordination/agent_org_tasks/store/update.rs b/src-tauri/crates/agent-core/src/core/coordination/agent_org_tasks/store/update.rs index a1faf6b9c..71b4b918c 100644 --- a/src-tauri/crates/agent-core/src/core/coordination/agent_org_tasks/store/update.rs +++ b/src-tauri/crates/agent-core/src/core/coordination/agent_org_tasks/store/update.rs @@ -6,6 +6,8 @@ use database::db::{get_connection, with_sessions_writer}; use rusqlite::{params, OptionalExtension}; +use crate::coordination::agent_org_payload_limits::validate_task_dependency_ids; + use super::super::helpers::{ encode_json_array, encode_metadata, insert_task_history_event, list_tasks_with_conn, now_rfc3339, row_to_task, SELECT_COLUMNS, @@ -212,6 +214,9 @@ impl AgentOrgTaskStore { expected_updated_at: Option<&str>, effects: impl FnOnce(&rusqlite::Connection, &TaskMutationOutcome, &[Task]) -> Result, ) -> Result<(TaskMutationOutcome, T), String> { + if let Some(blocked_by) = patch.blocked_by.as_ref() { + validate_task_dependency_ids("blocked_by", blocked_by)?; + } let mut conn = get_connection().map_err(|err| err.to_string())?; let tx = conn .transaction_with_behavior(rusqlite::TransactionBehavior::Immediate) diff --git a/src-tauri/crates/agent-core/src/core/coordination/agent_org_tasks/store/validation.rs b/src-tauri/crates/agent-core/src/core/coordination/agent_org_tasks/store/validation.rs index d7400fa1c..8f8ce06ef 100644 --- a/src-tauri/crates/agent-core/src/core/coordination/agent_org_tasks/store/validation.rs +++ b/src-tauri/crates/agent-core/src/core/coordination/agent_org_tasks/store/validation.rs @@ -8,13 +8,46 @@ use std::collections::HashSet; use rusqlite::{params, OptionalExtension}; use crate::coordination::agent_org_payload_limits::{ - validate_optional_text, validate_required_text, validate_text_len, TASK_ACTIVE_FORM_MAX_BYTES, + validate_optional_text, validate_required_text, validate_task_eligible_member_ids, + validate_task_identifier, validate_text_len, TASK_ACTIVE_FORM_MAX_BYTES, TASK_ACTIVE_FORM_MAX_CHARS, TASK_DESCRIPTION_MAX_BYTES, TASK_DESCRIPTION_MAX_CHARS, TASK_OUTPUT_CONTENT_MAX_BYTES, TASK_OUTPUT_CONTENT_MAX_CHARS, TASK_OUTPUT_SUMMARY_MAX_BYTES, - TASK_OUTPUT_SUMMARY_MAX_CHARS, TASK_SUBJECT_MAX_BYTES, TASK_SUBJECT_MAX_CHARS, + TASK_OUTPUT_SUMMARY_MAX_CHARS, TASK_RUN_MAX_TASKS, TASK_SUBJECT_MAX_BYTES, + TASK_SUBJECT_MAX_CHARS, }; -use super::super::TaskStatus; +use super::super::{TaskStatus, TASK_RUN_TASK_LIMIT_ERROR}; + +pub(super) fn ensure_task_rows_safe_for_operational_projection( + conn: &rusqlite::Connection, + run_id: &str, +) -> Result<(), String> { + if super::dependencies::run_is_safe_for_dependency_normalization(conn, run_id)? { + Ok(()) + } else { + Err( + "Agent Org task board contains oversized or corrupt rows; operational projection refused" + .to_string(), + ) + } +} + +pub(super) fn ensure_task_run_capacity( + existing_count: usize, + incoming_count: usize, +) -> Result<(), String> { + let projected_count = existing_count.checked_add(incoming_count).ok_or_else(|| { + format!( + "{TASK_RUN_TASK_LIMIT_ERROR}: task count overflow while checking the Agent Org run capacity" + ) + })?; + if projected_count <= TASK_RUN_MAX_TASKS { + return Ok(()); + } + Err(format!( + "{TASK_RUN_TASK_LIMIT_ERROR}: run retains {existing_count} tasks and this mutation would add {incoming_count}; maximum total is {TASK_RUN_MAX_TASKS}" + )) +} pub(super) fn reject_writable_blocks(blocks: &[String]) -> Result<(), String> { if blocks.is_empty() { @@ -93,6 +126,9 @@ pub(super) fn validate_task_persistence_invariants( status: TaskStatus, metadata: Option<&serde_json::Value>, ) -> Result<(), String> { + if let Some(owner) = owner { + validate_task_identifier("task owner_member_id", owner)?; + } let metadata_object = match metadata { None => None, Some(serde_json::Value::Object(object)) => Some(object), @@ -106,6 +142,15 @@ pub(super) fn validate_task_persistence_invariants( let values = value.as_array().ok_or_else(|| { "eligible_member_ids must be an array of member_id strings".to_string() })?; + let raw_member_ids = values + .iter() + .map(|value| { + value.as_str().map(str::to_string).ok_or_else(|| { + "eligible_member_ids must contain only non-empty member_id strings".to_string() + }) + }) + .collect::, _>>()?; + validate_task_eligible_member_ids("eligible_member_ids", &raw_member_ids)?; let mut seen = HashSet::new(); for value in values { let member_id = value @@ -173,12 +218,17 @@ pub(super) fn validate_task_persistence_invariants( TASK_OUTPUT_CONTENT_MAX_CHARS, TASK_OUTPUT_CONTENT_MAX_BYTES, )?; - if output.produced_by_member_id.trim().is_empty() { - return Err("task output produced_by_member_id must not be empty".to_string()); - } + validate_task_identifier( + "task output produced_by_member_id", + &output.produced_by_member_id, + )?; if chrono::DateTime::parse_from_rfc3339(&output.produced_at).is_err() { return Err("task output produced_at must be a valid RFC3339 timestamp".to_string()); } + crate::coordination::agent_org_payload_limits::validate_task_artifact_ids( + "task output artifact_ids", + &output.artifact_ids, + )?; } let snapshot_json: Option = conn diff --git a/src-tauri/crates/agent-core/src/core/coordination/agent_org_tasks/tests.rs b/src-tauri/crates/agent-core/src/core/coordination/agent_org_tasks/tests.rs index 533df56ff..047e5b0b1 100644 --- a/src-tauri/crates/agent-core/src/core/coordination/agent_org_tasks/tests.rs +++ b/src-tauri/crates/agent-core/src/core/coordination/agent_org_tasks/tests.rs @@ -41,6 +41,18 @@ fn make_eligible_params( params } +fn make_task_batch(org_run_id: &str, prefix: &str, count: usize) -> Vec { + let template = make_params(org_run_id, "template", "template"); + (0..count) + .map(|index| { + let mut task = template.clone(); + task.id = format!("{prefix}-{index}"); + task.subject = format!("{prefix} {index}"); + task + }) + .collect() +} + fn task_store_sandbox() -> test_helpers::test_env::SandboxGuard { let sandbox = test_helpers::test_env::sandbox(); let conn = get_connection().expect("test sqlite connection"); @@ -62,6 +74,49 @@ fn task_status_wire_round_trip() { assert!(TaskStatus::from_wire("garbage").is_err()); } +#[test] +fn create_rejects_task_ids_that_cannot_cross_the_inbox_boundary() { + let _sandbox = task_store_sandbox(); + let oversized_id = + "x".repeat(crate::coordination::agent_org_payload_limits::TASK_IDENTIFIER_MAX_CHARS + 1); + let error = AgentOrgTaskStore::create(make_eligible_params( + "run-task-id-limit", + &oversized_id, + "ownerless bounded task", + &["member-a"], + )) + .expect_err("an undeliverable task id must not be persisted"); + + assert!(error.contains("task id must be <= 1000 chars"), "{error}"); + assert!(AgentOrgTaskStore::list("run-task-id-limit") + .expect("inspect rejected run") + .is_empty()); +} + +#[test] +fn create_rejects_oversized_dependency_ids_before_persistence() { + let _sandbox = task_store_sandbox(); + let mut params = make_eligible_params( + "run-dependency-id-limit", + "bounded-task", + "bounded dependency", + &["member-a"], + ); + params.blocked_by = + vec!["x" + .repeat(crate::coordination::agent_org_payload_limits::TASK_IDENTIFIER_MAX_CHARS + 1)]; + let error = AgentOrgTaskStore::create(params) + .expect_err("an oversized dependency id must not be persisted"); + + assert!( + error.contains("blocked_by[0] must be <= 1000 chars"), + "{error}" + ); + assert!(AgentOrgTaskStore::list("run-dependency-id-limit") + .expect("inspect rejected run") + .is_empty()); +} + #[test] fn task_mutations_require_running_parent_run() { let _sandbox = task_store_sandbox(); @@ -322,6 +377,67 @@ fn delete_rejects_task_still_referenced_by_canonical_dependencies() { .is_some()); } +#[test] +fn delete_rejects_task_used_as_an_inbox_delivery_replacement() { + let _sandbox = task_store_sandbox(); + let run_id = format!("run-{}", uuid::Uuid::new_v4()); + AgentOrgTaskStore::create(make_params(&run_id, "replacement-task", "Replacement work")) + .expect("create replacement task"); + + let conn = get_connection().expect("test sqlite connection"); + let now = chrono::Utc::now().to_rfc3339(); + conn.execute( + "INSERT INTO agent_inbox ( + recipient_agent_id, sender_agent_id, org_run_id, + payload_kind, payload_json, created_at + ) VALUES ( + 'removed-agent', 'coordinator-agent', ?1, + 'plain', '{\"kind\":\"plain\",\"summary\":\"old\",\"text\":\"old\"}', ?2 + )", + rusqlite::params![&run_id, &now], + ) + .expect("seed source inbox evidence"); + let inbox_id = conn.last_insert_rowid(); + conn.execute( + "INSERT INTO agent_inbox_delivery_resolutions ( + inbox_id, org_run_id, resolution_kind, resolved_by_member_id, + reason, replacement_task_id, created_at + ) VALUES (?1, ?2, 'superseded', 'coordinator', 'Moved to task', + 'replacement-task', ?3)", + rusqlite::params![inbox_id, &run_id, &now], + ) + .expect("seed task replacement resolution"); + + let error = AgentOrgTaskStore::delete(&run_id, "replacement-task") + .expect_err("replacement evidence must not be deleted"); + assert!(error.contains(TASK_DELETE_IS_DELIVERY_REPLACEMENT_ERROR)); + assert!(AgentOrgTaskStore::get(&run_id, "replacement-task") + .expect("reload task") + .is_some()); +} + +#[test] +fn delete_fails_closed_when_delivery_resolution_schema_is_missing() { + let _sandbox = task_store_sandbox(); + let run_id = format!("run-{}", uuid::Uuid::new_v4()); + AgentOrgTaskStore::create(make_params( + &run_id, + "schema-guarded-task", + "Must survive schema failure", + )) + .expect("create guarded task"); + let conn = get_connection().expect("test sqlite connection"); + conn.execute("DROP TABLE agent_inbox_delivery_resolutions", []) + .expect("simulate damaged delivery-resolution schema"); + + let error = AgentOrgTaskStore::delete(&run_id, "schema-guarded-task") + .expect_err("schema failure must not be treated as an unreferenced task"); + assert!(error.contains("agent_inbox_delivery_resolutions")); + assert!(AgentOrgTaskStore::get(&run_id, "schema-guarded-task") + .expect("reload guarded task") + .is_some()); +} + #[test] fn create_batch_rechecks_existing_open_work_inside_transaction() { let _sandbox = task_store_sandbox(); @@ -375,6 +491,97 @@ fn create_rejects_in_progress_without_owner() { ); } +#[test] +fn create_rejects_noncanonical_eligibility_member_ids() { + let _sandbox = task_store_sandbox(); + let run_id = format!("run-{}", uuid::Uuid::new_v4()); + let mut params = make_params(&run_id, "task-1", "Noncanonical eligibility"); + params.metadata = Some(serde_json::json!({ + TASK_METADATA_ELIGIBLE_MEMBER_IDS: [" member-default "] + })); + let err = AgentOrgTaskStore::create(params) + .expect_err("surrounding whitespace must not be silently canonicalized"); + assert!(err.contains("leading or trailing whitespace"), "got {err}"); +} + +#[test] +fn corrupt_predicate_flags_ownerless_in_progress_and_spaced_eligibility() { + let _sandbox = task_store_sandbox(); + let run_id = format!("run-{}", uuid::Uuid::new_v4()); + let _ = make_params(&run_id, "template", "seed parent run"); + let conn = get_connection().expect("task database"); + let now = chrono::Utc::now().to_rfc3339(); + conn.execute( + "INSERT INTO agent_org_tasks + (id, org_run_id, subject, description, active_form, owner, status, + blocks_json, blocked_by_json, metadata_json, created_at, updated_at) + VALUES ('ownerless-running', ?1, 'bad running row', '', NULL, NULL, + 'in_progress', '[]', '[]', ?2, ?3, ?3)", + rusqlite::params![ + &run_id, + serde_json::json!({TASK_METADATA_ELIGIBLE_MEMBER_IDS: ["member-default"]}).to_string(), + &now + ], + ) + .expect("seed ownerless in-progress row"); + conn.execute( + "INSERT INTO agent_org_tasks + (id, org_run_id, subject, description, active_form, owner, status, + blocks_json, blocked_by_json, metadata_json, created_at, updated_at) + VALUES ('spaced-eligibility', ?1, 'bad eligibility row', '', NULL, NULL, + 'pending', '[]', '[]', ?2, ?3, ?3)", + rusqlite::params![ + &run_id, + serde_json::json!({TASK_METADATA_ELIGIBLE_MEMBER_IDS: [" member-default "]}) + .to_string(), + &now + ], + ) + .expect("seed spaced eligibility row"); + + let predicate = corrupt_task_row_predicate_sql(); + let corrupt_count: i64 = conn + .query_row( + &format!("SELECT COUNT(*) FROM agent_org_tasks WHERE org_run_id=?1 AND {predicate}"), + rusqlite::params![&run_id], + |row| row.get(0), + ) + .expect("count corrupt historical rows"); + assert_eq!(corrupt_count, 2); +} + +#[test] +fn summary_filtered_total_matches_rows_after_scalar_corruption_filtering() { + let _sandbox = task_store_sandbox(); + let run_id = format!("run-{}", uuid::Uuid::new_v4()); + AgentOrgTaskStore::create(make_params(&run_id, "valid-task", "Visible task")) + .expect("create visible task"); + let conn = get_connection().expect("task database"); + let now = chrono::Utc::now().to_rfc3339(); + let oversized_id = + "x".repeat(crate::coordination::agent_org_payload_limits::TASK_IDENTIFIER_MAX_CHARS + 1); + conn.execute( + "INSERT INTO agent_org_tasks + (id, org_run_id, subject, description, active_form, owner, status, + blocks_json, blocked_by_json, metadata_json, created_at, updated_at) + VALUES (?1, ?2, 'hidden corrupt row', '', NULL, NULL, 'pending', + '[]', '[]', ?3, ?4, ?4)", + rusqlite::params![ + oversized_id, + &run_id, + serde_json::json!({TASK_METADATA_ELIGIBLE_MEMBER_IDS: ["member-default"]}).to_string(), + now + ], + ) + .expect("seed oversized historical id"); + + let page = AgentOrgTaskStore::list_summary_page(&run_id, None, None, None, 200) + .expect("bounded summary page"); + assert_eq!(page.filtered_total, 1); + assert_eq!(page.tasks.len(), 1); + assert_eq!(page.tasks[0].id, "valid-task"); +} + #[test] fn store_rejects_ownerless_pending_without_eligibility() { let _sandbox = task_store_sandbox(); @@ -428,6 +635,98 @@ fn store_rejects_malformed_reserved_dispatch_metadata() { })); let err = AgentOrgTaskStore::create(timezone_less_output).unwrap_err(); assert!(err.contains("valid RFC3339"), "got {err}"); + + let mut oversized_producer = + make_params(&run_id, "task-output-producer", "oversized output producer"); + oversized_producer.owner = Some("member-default".to_string()); + oversized_producer.status = TaskStatus::Completed; + oversized_producer.metadata = Some(serde_json::json!({ + TASK_METADATA_ELIGIBLE_MEMBER_IDS: ["member-default"], + TASK_METADATA_OUTPUT: { + "summary": "done", + "content": null, + "artifactIds": [], + "producedByMemberId": "x".repeat( + crate::coordination::agent_org_payload_limits::TASK_IDENTIFIER_MAX_CHARS + 1 + ), + "producedAt": chrono::Utc::now().to_rfc3339(), + }, + })); + let err = AgentOrgTaskStore::create(oversized_producer).unwrap_err(); + assert!( + err.contains("task output produced_by_member_id must be <= 1000 chars"), + "got {err}" + ); + + let historical_metadata = serde_json::json!({ + TASK_METADATA_ELIGIBLE_MEMBER_IDS: ["member-default"], + TASK_METADATA_OUTPUT: { + "summary": "done", + "content": null, + "artifactIds": [], + "producedByMemberId": "x".repeat( + crate::coordination::agent_org_payload_limits::TASK_IDENTIFIER_MAX_CHARS + 1 + ), + "producedAt": chrono::Utc::now().to_rfc3339(), + }, + }); + let conn = get_connection().expect("task database"); + let now = chrono::Utc::now().to_rfc3339(); + conn.execute( + "INSERT INTO agent_org_tasks + (id, org_run_id, subject, description, active_form, owner, status, + blocks_json, blocked_by_json, metadata_json, created_at, updated_at) + VALUES ('historical-output-producer', ?1, 'historical', '', NULL, + 'member-default', 'completed', '[]', '[]', ?2, ?3, ?3)", + rusqlite::params![&run_id, historical_metadata.to_string(), now], + ) + .expect("seed historical oversized producer"); + let predicate = corrupt_task_row_predicate_sql(); + let classified: bool = conn + .query_row( + &format!( + "SELECT {predicate} FROM agent_org_tasks + WHERE org_run_id=?1 AND id='historical-output-producer'" + ), + rusqlite::params![&run_id], + |row| row.get(0), + ) + .expect("classify historical producer"); + assert!( + classified, + "historical oversized producer must block finality" + ); + + let timezone_less_metadata = serde_json::json!({ + TASK_METADATA_ELIGIBLE_MEMBER_IDS: ["member-default"], + TASK_METADATA_OUTPUT: { + "summary": "done", + "content": null, + "artifactIds": [], + "producedByMemberId": "member-default", + "producedAt": "2026-07-17 12:34:56", + }, + }); + conn.execute( + "INSERT INTO agent_org_tasks + (id, org_run_id, subject, description, active_form, owner, status, + blocks_json, blocked_by_json, metadata_json, created_at, updated_at) + VALUES ('historical-output-zone', ?1, 'historical', '', NULL, + 'member-default', 'completed', '[]', '[]', ?2, ?3, ?3)", + rusqlite::params![&run_id, timezone_less_metadata.to_string(), now], + ) + .expect("seed historical timezone-less output"); + let classified: bool = conn + .query_row( + &format!( + "SELECT {predicate} FROM agent_org_tasks + WHERE org_run_id=?1 AND id='historical-output-zone'" + ), + rusqlite::params![&run_id], + |row| row.get(0), + ) + .expect("classify historical timezone-less output"); + assert!(classified, "timezone-less output must block finality"); } #[test] @@ -645,6 +944,153 @@ fn create_rejects_self_dependency_cycle() { assert!(err.contains(TASK_DEPENDENCY_CYCLE_ERROR), "got {err}"); } +#[test] +fn create_rejects_dependency_fan_in_above_persistence_limit() { + let _sandbox = task_store_sandbox(); + let run_id = format!("run-{}", uuid::Uuid::new_v4()); + let mut params = make_params(&run_id, "too-many-inputs", "too many inputs"); + params.blocked_by = (0 + ..=crate::coordination::agent_org_payload_limits::TASK_DEPENDENCY_MAX_COUNT) + .map(|index| format!("dependency-{index}")) + .collect(); + + let error = AgentOrgTaskStore::create(params).expect_err("fan-in must be bounded"); + assert!( + error.starts_with(TASK_DEPENDENCY_LIMIT_ERROR), + "got {error}" + ); + assert!(AgentOrgTaskStore::list(&run_id).unwrap().is_empty()); +} + +#[test] +fn create_batch_rejects_derived_fan_out_above_persistence_limit() { + let _sandbox = task_store_sandbox(); + let run_id = format!("run-{}", uuid::Uuid::new_v4()); + let root = make_params(&run_id, "root", "root"); + let mut graph = vec![root]; + for index in 0..=crate::coordination::agent_org_payload_limits::TASK_DEPENDENCY_MAX_COUNT { + let mut downstream = make_params(&run_id, &format!("downstream-{index}"), "downstream"); + downstream.blocked_by = vec!["root".to_string()]; + graph.push(downstream); + } + + let error = AgentOrgTaskStore::create_batch(graph, true) + .expect_err("derived fan-out projection must be bounded"); + assert!( + error.starts_with(TASK_DEPENDENCY_LIMIT_ERROR), + "got {error}" + ); + assert!(AgentOrgTaskStore::list(&run_id).unwrap().is_empty()); +} + +#[test] +fn create_batch_rejects_oversized_internal_graph_before_writes() { + let _sandbox = task_store_sandbox(); + let run_id = format!("run-{}", uuid::Uuid::new_v4()); + let template = make_params(&run_id, "template", "template"); + let graph = (0..=crate::coordination::agent_org_payload_limits::TASK_RUN_MAX_TASKS) + .map(|index| { + let mut task = template.clone(); + task.id = format!("task-{index}"); + task.subject = format!("Task {index}"); + task + }) + .collect(); + + let error = AgentOrgTaskStore::create_batch(graph, true) + .expect_err("oversized internal graph must be rejected"); + assert!(error.starts_with(TASK_RUN_TASK_LIMIT_ERROR), "got {error}"); + assert!(AgentOrgTaskStore::list(&run_id).unwrap().is_empty()); +} + +#[test] +fn run_task_capacity_applies_to_single_and_existing_plus_batch_create() { + let _sandbox = task_store_sandbox(); + let maximum = crate::coordination::agent_org_payload_limits::TASK_RUN_MAX_TASKS; + + let single_run_id = format!("run-single-capacity-{}", uuid::Uuid::new_v4()); + AgentOrgTaskStore::create_batch(make_task_batch(&single_run_id, "seed", maximum - 1), true) + .expect("seed one slot below the run capacity"); + AgentOrgTaskStore::create(make_params( + &single_run_id, + "last-slot", + "Last available slot", + )) + .expect("single create may fill the final run slot"); + let single_error = AgentOrgTaskStore::create(make_params( + &single_run_id, + "over-capacity", + "Must not be inserted", + )) + .expect_err("single create must not exceed the durable run capacity"); + assert!( + single_error.starts_with(TASK_RUN_TASK_LIMIT_ERROR), + "got {single_error}" + ); + assert_eq!( + AgentOrgTaskStore::list(&single_run_id).unwrap().len(), + maximum + ); + + let batch_run_id = format!("run-batch-capacity-{}", uuid::Uuid::new_v4()); + AgentOrgTaskStore::create_batch(make_task_batch(&batch_run_id, "seed", maximum - 1), true) + .expect("seed one slot below the run capacity"); + let batch_error = + AgentOrgTaskStore::create_batch(make_task_batch(&batch_run_id, "overflow", 2), true) + .expect_err("existing plus batch size must be checked inside the transaction"); + assert!( + batch_error.starts_with(TASK_RUN_TASK_LIMIT_ERROR), + "got {batch_error}" + ); + assert_eq!( + AgentOrgTaskStore::list(&batch_run_id).unwrap().len(), + maximum - 1, + "rejected batch must leave every row uncommitted" + ); +} + +#[test] +fn concurrent_single_creates_cannot_cross_run_task_capacity() { + let _sandbox = task_store_sandbox(); + let maximum = crate::coordination::agent_org_payload_limits::TASK_RUN_MAX_TASKS; + let run_id = format!("run-concurrent-capacity-{}", uuid::Uuid::new_v4()); + AgentOrgTaskStore::create_batch(make_task_batch(&run_id, "seed", maximum - 1), true) + .expect("seed one slot below the run capacity"); + + let first = make_params(&run_id, "concurrent-a", "Concurrent A"); + let second = make_params(&run_id, "concurrent-b", "Concurrent B"); + let barrier = std::sync::Arc::new(std::sync::Barrier::new(2)); + let results = std::thread::scope(|scope| { + let first_barrier = std::sync::Arc::clone(&barrier); + let first_handle = scope.spawn(move || { + first_barrier.wait(); + AgentOrgTaskStore::create(first) + }); + let second_barrier = std::sync::Arc::clone(&barrier); + let second_handle = scope.spawn(move || { + second_barrier.wait(); + AgentOrgTaskStore::create(second) + }); + vec![ + first_handle.join().expect("first create thread"), + second_handle.join().expect("second create thread"), + ] + }); + + assert_eq!(results.iter().filter(|result| result.is_ok()).count(), 1); + let errors = results + .iter() + .filter_map(|result| result.as_ref().err()) + .collect::>(); + assert_eq!(errors.len(), 1); + assert!( + errors[0].starts_with(TASK_RUN_TASK_LIMIT_ERROR), + "got {}", + errors[0] + ); + assert_eq!(AgentOrgTaskStore::list(&run_id).unwrap().len(), maximum); +} + #[test] fn update_rejects_dependency_cycle_in_canonical_blocked_by() { let _sandbox = task_store_sandbox(); diff --git a/src-tauri/crates/agent-core/src/core/coordination/agent_org_watchdog.rs b/src-tauri/crates/agent-core/src/core/coordination/agent_org_watchdog.rs index b11f43ad2..f2a3b2dd0 100644 --- a/src-tauri/crates/agent-core/src/core/coordination/agent_org_watchdog.rs +++ b/src-tauri/crates/agent-core/src/core/coordination/agent_org_watchdog.rs @@ -37,9 +37,9 @@ mod tests; pub use budget::clear_rewake_budget; pub use budget::init_schema; +pub(crate) use budget::member_rewake_fingerprint; #[cfg(test)] pub use budget::test_only_mark_failed_rewake_attempt; -pub(crate) use budget::member_rewake_fingerprint; pub use inspect::inspect_stalled_run; pub use plan::{MemberContinuationAction, MemberTaskAssignmentAction, StallRecoveryPlan}; pub use recover::{recover_stalled_run, spawn}; @@ -48,7 +48,7 @@ pub(crate) use reservation::{ reserve_member_rewake_dispatch, MemberRewakeReservationOutcome, }; -use std::collections::{HashMap, HashSet}; +use std::collections::{BTreeSet, HashMap, HashSet}; use std::time::Duration; use chrono::{DateTime, Duration as ChronoDuration, Utc}; @@ -61,8 +61,9 @@ use crate::coordination::agent_inbox::{ }; use crate::coordination::agent_org_plan_approvals::AgentOrgPlanApprovalStore; use crate::coordination::agent_org_runs::{ - AgentOrgFinalityBlocker, AgentOrgFinalityDecision, AgentOrgRunRecord, AgentOrgRunStatus, - AgentOrgRunStore, COORDINATOR_MEMBER_ID, + recovery_dispatch_recipient_is_available, AgentOrgFinalityBlocker, AgentOrgFinalityDecision, + AgentOrgRunRecord, AgentOrgRunStatus, AgentOrgRunStore, WorkerSessionRuntime, + COORDINATOR_MEMBER_ID, }; use crate::coordination::agent_org_tasks::{self, Task, TaskStatus}; use crate::core::session::SessionStatus; @@ -71,6 +72,7 @@ use crate::tools::impls::orchestration::org_send_message::InboxWakeHook; const WATCHDOG_INTERVAL_SECS: u64 = 60; const RECOVERY_DELAYS_SECS: [i64; 3] = [60, 5 * 60, 15 * 60]; +const PENDING_MATERIALIZATION_GRACE_SECS: i64 = 2 * 60; const MEMBER_REWAKE: &str = "member_rewake"; const COORDINATOR_NOTICE: &str = "coordinator_notice"; diff --git a/src-tauri/crates/agent-core/src/core/coordination/agent_org_watchdog/budget.rs b/src-tauri/crates/agent-core/src/core/coordination/agent_org_watchdog/budget.rs index b7db994fc..49ad41c19 100644 --- a/src-tauri/crates/agent-core/src/core/coordination/agent_org_watchdog/budget.rs +++ b/src-tauri/crates/agent-core/src/core/coordination/agent_org_watchdog/budget.rs @@ -57,6 +57,7 @@ pub(super) enum BudgetDisposition { Exhausted, } +#[cfg(test)] pub(super) fn budget_disposition( run_id: &str, action_kind: &str, @@ -121,6 +122,7 @@ pub(super) fn budget_disposition_with_connection( }) } +#[cfg(test)] pub(super) fn record_attempt( run_id: &str, action_kind: &str, @@ -222,6 +224,7 @@ pub fn test_only_mark_failed_rewake_attempt(run_id: &str, member_id: &str) -> Re Ok(true) } +#[cfg(test)] pub(super) fn delayed_rewake_allowed( run_id: &str, member_id: &str, @@ -239,6 +242,7 @@ pub(super) fn delayed_rewake_allowed( /// backoff window": an exhausted budget never recovers without a /// successful member turn (which clears it), so it marks the member as /// beyond autonomous recovery. +#[cfg(test)] pub(super) fn rewake_budget_exhausted( run_id: &str, member_id: &str, @@ -250,6 +254,7 @@ pub(super) fn rewake_budget_exhausted( )) } +#[cfg(test)] pub(super) fn reason_fingerprint(reason: &str) -> String { blake3::hash(reason.as_bytes()).to_hex().to_string() } @@ -270,6 +275,7 @@ pub(super) fn coordinator_notice_allowed(run_id: &str, reason: &str) -> Result, ) -> String { diff --git a/src-tauri/crates/agent-core/src/core/coordination/agent_org_watchdog/inspect.rs b/src-tauri/crates/agent-core/src/core/coordination/agent_org_watchdog/inspect.rs index 10bd1d599..36b29ad67 100644 --- a/src-tauri/crates/agent-core/src/core/coordination/agent_org_watchdog/inspect.rs +++ b/src-tauri/crates/agent-core/src/core/coordination/agent_org_watchdog/inspect.rs @@ -4,23 +4,865 @@ //! This is the read-only decision half of the watchdog; [`super::recover`] //! carries out the plan it returns. -use super::budget::{delayed_rewake_allowed, reason_fingerprint, rewake_budget_exhausted}; +use super::budget::{ + budget_disposition_with_connection, member_rewake_fingerprint_from_unread, BudgetDisposition, +}; use super::plan::ready_unassigned_repair_reason; use super::*; +/// A machine-stable recovery fact. Human-readable repair prose is excluded so +/// copy edits do not reset retry budgets. +#[derive(Debug, Clone, PartialEq, Eq)] +struct RecoveryRepairFact { + kind: &'static str, + fields: Vec>, +} + +impl RecoveryRepairFact { + fn new(kind: &'static str, fields: impl IntoIterator>) -> Self { + Self { + kind, + fields: fields.into_iter().collect(), + } + } + + fn marker(kind: &'static str) -> Self { + Self::new(kind, std::iter::empty()) + } + + fn digest(&self) -> String { + fn write_bytes(hasher: &mut blake3::Hasher, bytes: &[u8]) { + hasher.update(&(bytes.len() as u64).to_le_bytes()); + hasher.update(bytes); + } + + let mut hasher = blake3::Hasher::new(); + write_bytes(&mut hasher, b"agent-org-recovery-fact-v1"); + write_bytes(&mut hasher, self.kind.as_bytes()); + hasher.update(&(self.fields.len() as u64).to_le_bytes()); + for field in &self.fields { + match field { + Some(value) => { + hasher.update(&[1]); + write_bytes(&mut hasher, value.as_bytes()); + } + None => { + hasher.update(&[0]); + } + } + } + hasher.finalize().to_hex().to_string() + } +} + +fn recovery_repair_fingerprint(facts: &[RecoveryRepairFact]) -> Option { + if facts.is_empty() { + return None; + } + let mut digests = facts + .iter() + .map(RecoveryRepairFact::digest) + .collect::>(); + digests.sort(); + digests.dedup(); + let mut hasher = blake3::Hasher::new(); + hasher.update(b"agent-org-recovery-set-v1"); + hasher.update(&(digests.len() as u64).to_le_bytes()); + for digest in digests { + hasher.update(&(digest.len() as u64).to_le_bytes()); + hasher.update(digest.as_bytes()); + } + Some(hasher.finalize().to_hex().to_string()) +} + +fn corrupt_task_repair_facts( + conn: &Connection, + run_id: &str, +) -> Result, String> { + let task_count: i64 = conn + .query_row( + "SELECT COUNT(*) FROM agent_org_tasks WHERE org_run_id=?1", + params![run_id], + |row| row.get(0), + ) + .map_err(|err| err.to_string())?; + if task_count > crate::coordination::agent_org_payload_limits::TASK_RUN_MAX_TASKS as i64 { + return Ok(vec![RecoveryRepairFact::new( + "task_run_limit_exceeded", + [Some(task_count.to_string())], + )]); + } + let predicate = agent_org_tasks::corrupt_task_row_predicate_sql(); + let dependency_json_max = + crate::coordination::agent_org_payload_limits::TASK_DEPENDENCY_JSON_MAX_BYTES; + let metadata_max = crate::coordination::agent_org_payload_limits::TASK_METADATA_MAX_BYTES; + let sql = format!( + "SELECT substr(id,1,1024), length(CAST(id AS BLOB)), + substr(status,1,128), + length(CAST(blocks_json AS BLOB)), hex(substr(blocks_json,1,1024)), + length(CAST(blocked_by_json AS BLOB)), hex(substr(blocked_by_json,1,1024)), + length(CAST(COALESCE(metadata_json,'') AS BLOB)), + hex(substr(COALESCE(metadata_json,''),1,1024)) + FROM ( + SELECT id, subject, description, active_form, owner, status, + created_at, updated_at, + CASE WHEN length(CAST(blocks_json AS BLOB))<={dependency_json_max} + THEN blocks_json ELSE '!' END AS blocks_json, + CASE WHEN length(CAST(blocked_by_json AS BLOB))<={dependency_json_max} + THEN blocked_by_json ELSE '!' END AS blocked_by_json, + CASE WHEN metadata_json IS NULL + OR length(CAST(metadata_json AS BLOB))<={metadata_max} + THEN metadata_json ELSE '!' END AS metadata_json + FROM agent_org_tasks WHERE org_run_id=?1 + ) AS bounded_tasks + WHERE {predicate} + ORDER BY id ASC" + ); + let mut stmt = conn.prepare(&sql).map_err(|err| err.to_string())?; + let rows = stmt + .query_map(params![run_id], |row| { + Ok(RecoveryRepairFact::new( + "corrupt_task_data", + [ + Some(row.get::<_, String>(0)?), + Some(row.get::<_, i64>(1)?.to_string()), + Some(row.get::<_, String>(2)?), + Some(row.get::<_, i64>(3)?.to_string()), + Some(row.get::<_, String>(4)?), + Some(row.get::<_, i64>(5)?.to_string()), + Some(row.get::<_, String>(6)?), + Some(row.get::<_, i64>(7)?.to_string()), + Some(row.get::<_, String>(8)?), + ], + )) + }) + .map_err(|err| err.to_string())?; + rows.collect::, _>>() + .map_err(|err| err.to_string()) +} + +fn bounded_id_list_preview(ids: &[String], max_items: usize, max_chars_per_id: usize) -> String { + let preview = ids + .iter() + .take(max_items) + .map(|id| crate::utils::safe_truncate_chars_to_string(id, max_chars_per_id)) + .collect::>() + .join(", "); + let omitted = ids.len().saturating_sub(max_items); + if omitted > 0 { + format!("{preview}, +{omitted} more (use task_list/task_get)") + } else { + preview + } +} + +fn bounded_recovery_reason_text(reasons: &[String]) -> String { + const MAX_REASON_CHARS: usize = 15_000; + let mut out = String::new(); + let mut used = 0usize; + let mut included = 0usize; + for reason in reasons { + let separator = usize::from(!out.is_empty()); + let remaining = MAX_REASON_CHARS.saturating_sub(used.saturating_add(separator)); + if remaining == 0 { + break; + } + let bounded = crate::utils::safe_truncate_chars_to_string(reason, remaining); + if !out.is_empty() { + out.push('\n'); + used += 1; + } + used = used.saturating_add(bounded.chars().count()); + out.push_str(&bounded); + included += 1; + if bounded.chars().count() < reason.chars().count() { + break; + } + } + let omitted = reasons.len().saturating_sub(included); + if omitted > 0 { + let suffix = format!( + "\n+{omitted} additional repair item(s); use task_list/task_get for the full board." + ); + let keep = MAX_REASON_CHARS.saturating_sub(suffix.chars().count()); + out = crate::utils::safe_truncate_chars_to_string(&out, keep); + out.push_str(&suffix); + } + out +} + +pub(super) fn task_snapshot_fingerprint(tasks: &[Task]) -> String { + fn hash_field(hasher: &mut blake3::Hasher, field_kind: &str, value: &str) { + hasher.update(&(field_kind.len() as u64).to_le_bytes()); + hasher.update(field_kind.as_bytes()); + hasher.update(&(value.len() as u64).to_le_bytes()); + hasher.update(value.as_bytes()); + } + + fn hash_list(hasher: &mut blake3::Hasher, field_kind: &str, values: &[String]) { + hasher.update(&(field_kind.len() as u64).to_le_bytes()); + hasher.update(field_kind.as_bytes()); + hasher.update(&(values.len() as u64).to_le_bytes()); + for value in values { + hash_field(hasher, "item", value); + } + } + + let mut hasher = blake3::Hasher::new(); + hasher.update(b"agent-org-task-snapshot-v2"); + let mut ordered_tasks = tasks.iter().collect::>(); + ordered_tasks.sort_by(|left, right| left.id.cmp(&right.id)); + hasher.update(&(ordered_tasks.len() as u64).to_le_bytes()); + for task in ordered_tasks { + let mut blocked_by = task.blocked_by.clone(); + blocked_by.sort(); + blocked_by.dedup(); + let mut eligible_member_ids = agent_org_tasks::eligible_member_ids(task); + eligible_member_ids.sort(); + eligible_member_ids.dedup(); + hash_field(&mut hasher, "task_id", &task.id); + hash_field(&mut hasher, "status", task.status.as_wire()); + match task.owner.as_deref() { + Some(owner) => hash_field(&mut hasher, "owner_some", owner), + None => hash_field(&mut hasher, "owner_none", ""), + } + hash_list(&mut hasher, "blocked_by", &blocked_by); + hash_list(&mut hasher, "eligible_member_ids", &eligible_member_ids); + hash_field(&mut hasher, "updated_at", &task.updated_at); + } + hasher.finalize().to_hex().to_string() +} + +fn append_dependency_integrity_repairs( + tasks: &[Task], + reasons: &mut Vec, + facts: &mut Vec, +) { + let known_ids = tasks + .iter() + .map(|task| task.id.as_str()) + .collect::>(); + let mut missing_edges = Vec::<(String, String, String)>::new(); + for task in tasks { + for blocker_id in &task.blocked_by { + if !known_ids.contains(blocker_id.as_str()) { + missing_edges.push(( + "blocked_by".to_string(), + task.id.clone(), + blocker_id.clone(), + )); + } + } + for downstream_id in &task.blocks { + if !known_ids.contains(downstream_id.as_str()) { + missing_edges.push(("blocks".to_string(), task.id.clone(), downstream_id.clone())); + } + } + } + missing_edges.sort(); + missing_edges.dedup(); + if !missing_edges.is_empty() { + let mut hasher = blake3::Hasher::new(); + hasher.update(b"agent-org-missing-dependency-edges-v1"); + for (relation, task_id, missing_id) in &missing_edges { + for field in [relation, task_id, missing_id] { + hasher.update(&(field.len() as u64).to_le_bytes()); + hasher.update(field.as_bytes()); + } + } + facts.push(RecoveryRepairFact::new( + "missing_dependency_edges", + [ + Some(missing_edges.len().to_string()), + Some(hasher.finalize().to_hex().to_string()), + ], + )); + let preview = missing_edges + .iter() + .take(8) + .map(|(relation, task_id, missing_id)| { + format!("{task_id}.{relation} -> missing task {missing_id}") + }) + .collect::>() + .join("; "); + let remainder = missing_edges.len().saturating_sub(8); + reasons.push(format!( + "the task graph contains {} dependency reference(s) to task ids that do not exist: {}{}. Repair those persisted edges before continuing; the watchdog will not guess which task was intended.", + missing_edges.len(), + preview, + if remainder > 0 { + format!("; +{remainder} more (use task_list/task_get)") + } else { + String::new() + } + )); + } + + let Some(run_id) = tasks.first().map(|task| task.org_run_id.as_str()) else { + return; + }; + if let Err(error) = agent_org_tasks::validate_dependency_graph(tasks, run_id) { + let mut edges = tasks + .iter() + .flat_map(|task| { + task.blocked_by + .iter() + .map(move |blocker| (task.id.clone(), blocker.clone())) + }) + .collect::>(); + edges.sort(); + edges.dedup(); + let mut hasher = blake3::Hasher::new(); + hasher.update(b"agent-org-dependency-cycle-v1"); + for (task_id, blocker_id) in &edges { + for field in [task_id, blocker_id] { + hasher.update(&(field.len() as u64).to_le_bytes()); + hasher.update(field.as_bytes()); + } + } + facts.push(RecoveryRepairFact::new( + "dependency_cycle", + [Some(hasher.finalize().to_hex().to_string())], + )); + reasons.push(format!( + "the persisted task dependency graph contains a cycle ({}). Break the cycle explicitly before continuing; cyclic tasks can never become ready.", + crate::utils::safe_truncate_chars_to_string(&error, 2_000) + )); + } +} + +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub(super) enum PendingMaterializationDisposition { + Grace, + Expired, + InvalidTimestamp, +} + +pub(super) fn pending_materialization_disposition( + owner_updated_at: Option<&str>, +) -> PendingMaterializationDisposition { + let Some(owner_updated_at) = owner_updated_at else { + return PendingMaterializationDisposition::InvalidTimestamp; + }; + let updated_at = match DateTime::parse_from_rfc3339(owner_updated_at) { + Ok(parsed) => parsed.with_timezone(&Utc), + Err(err) => { + tracing::warn!( + timestamp = %owner_updated_at, + error = %err, + "[agent_org_watchdog] unparseable Pending member updated_at; escalating repair" + ); + return PendingMaterializationDisposition::InvalidTimestamp; + } + }; + if Utc::now() - updated_at <= ChronoDuration::seconds(PENDING_MATERIALIZATION_GRACE_SECS) { + PendingMaterializationDisposition::Grace + } else { + PendingMaterializationDisposition::Expired + } +} + +#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord)] +enum UnreadRecipientUnavailableReason { + MissingCanonicalMemberId, + UnknownRosterMember, + MissingSession, + ArchivedSession, + UnsupportedTransport, + AdministrativelyPaused, + PendingMaterializationExpired, + InvalidSessionTimestamp, + RecoveryBudgetExhausted, +} + +impl UnreadRecipientUnavailableReason { + fn as_key(self) -> &'static str { + match self { + Self::MissingCanonicalMemberId => "missing_canonical_member_id", + Self::UnknownRosterMember => "unknown_roster_member", + Self::MissingSession => "missing_session", + Self::ArchivedSession => "archived_session", + Self::UnsupportedTransport => "unsupported_transport", + Self::AdministrativelyPaused => "administratively_paused", + Self::PendingMaterializationExpired => "pending_materialization_expired", + Self::InvalidSessionTimestamp => "invalid_session_timestamp", + Self::RecoveryBudgetExhausted => "recovery_budget_exhausted", + } + } +} + +#[derive(Debug, Clone, PartialEq, Eq)] +struct UnreadRecipientRepair { + recipient_member_id: Option, + recipient_agent_id: String, + unread_count: usize, + max_unread_id: i64, + reason: UnreadRecipientUnavailableReason, +} + +impl UnreadRecipientRepair { + fn repair_fact(&self) -> RecoveryRepairFact { + RecoveryRepairFact::new( + "unread_recipient", + [ + Some(self.reason.as_key().to_string()), + self.recipient_member_id.clone(), + self.recipient_member_id + .is_none() + .then(|| self.recipient_agent_id.clone()), + ], + ) + } + + fn stable_key(&self) -> String { + self.repair_fact().digest() + } + + fn snapshot_fact(&self) -> RecoveryRepairFact { + RecoveryRepairFact::new( + "unread_recipient_snapshot", + [ + Some(self.reason.as_key().to_string()), + self.recipient_member_id.clone(), + self.recipient_member_id + .is_none() + .then(|| self.recipient_agent_id.clone()), + Some(self.unread_count.to_string()), + Some(self.max_unread_id.to_string()), + ], + ) + } + + fn coordinator_reason(&self) -> String { + let recipient = self + .recipient_member_id + .as_deref() + .map(|member_id| format!("member {member_id}")) + .unwrap_or_else(|| { + format!( + "a legacy Inbox recipient without recipient_member_id (recipient_agent_id={})", + self.recipient_agent_id + ) + }); + let repair = match self.reason { + UnreadRecipientUnavailableReason::MissingCanonicalMemberId => { + "the durable row has no canonical member identity. Do not guess from agent_id because multiple roster members may share one AgentDefinition; restore the intended member identity or cancel the run" + } + UnreadRecipientUnavailableReason::UnknownRosterMember => { + "that member is not present in this run's immutable launch roster; inspect the corrupted routing identity or cancel the run" + } + UnreadRecipientUnavailableReason::MissingSession => { + "no materialized session exists for that roster member; restore/materialize the member session or cancel the run" + } + UnreadRecipientUnavailableReason::ArchivedSession => { + "the recipient session is Archived and cannot be woken; reopen the member or cancel the run" + } + UnreadRecipientUnavailableReason::UnsupportedTransport => { + "the recipient is a historical CLI member, whose Agent Org Inbox transport is unsupported; move the work to a Rust member or cancel the run" + } + UnreadRecipientUnavailableReason::AdministrativelyPaused => { + "the recipient session is administratively Paused; resume it explicitly or cancel the run" + } + UnreadRecipientUnavailableReason::PendingMaterializationExpired => { + "the recipient session remained Pending beyond the materialization grace period; retry materialization or cancel the run" + } + UnreadRecipientUnavailableReason::InvalidSessionTimestamp => { + "the Pending recipient has a missing or invalid timestamp, so automatic recovery cannot safely wait; repair the session or cancel the run" + } + UnreadRecipientUnavailableReason::RecoveryBudgetExhausted => { + "automatic Wake attempts for the current unread set are exhausted; explicitly retry/reopen the recipient or cancel the run" + } + }; + format!( + "{recipient} has {} pending Agent Org Inbox message(s), but {repair}. The watchdog preserves those rows as unread because the intended recipient did not read them. Inspect the newest affected inbox_id {} with org_inbox_repair. If recovery is impossible, create any legitimate replacement work first and explicitly supersede it, or explicitly cancel that delivery; never mark it read by guessing the recipient.", + self.unread_count, + self.max_unread_id + ) + } +} + +fn unread_fingerprints_by_member( + counts: &[crate::coordination::agent_inbox::AgentInboxUnreadRecipientCounts], +) -> HashMap { + let mut aggregate = HashMap::::new(); + for counts in counts { + let Some(member_id) = counts + .recipient_member_id + .as_deref() + .filter(|member_id| !member_id.trim().is_empty()) + else { + continue; + }; + let entry = aggregate + .entry(member_id.to_string()) + .or_insert((counts.max_unread_id, 0)); + entry.0 = entry.0.max(counts.max_unread_id); + entry.1 = entry.1.saturating_add(counts.unread_count); + } + aggregate + .into_iter() + .map(|(member_id, (max_id, count))| (member_id, format!("{max_id}:{count}"))) + .collect() +} + +fn unavailable_unread_recipient_repairs_from_counts_with_connection( + conn: &Connection, + run_id: &str, + workers: &[WorkerSessionRuntime], + counts: &[crate::coordination::agent_inbox::AgentInboxUnreadRecipientCounts], +) -> Result, String> { + let unread_fingerprints_by_member = unread_fingerprints_by_member(counts); + let roster_member_ids = AgentOrgRunStore::snapshot_member_ids_with_connection(conn, run_id)?; + let coordinator = AgentOrgRunStore::find_coordinator_session_by_member_id_with_connection( + conn, + run_id, + COORDINATOR_MEMBER_ID, + )?; + let mut repairs = Vec::new(); + let mut canonical = HashMap::, usize, i64)>::new(); + let mut legacy = Vec::new(); + for count in counts.iter().filter(|counts| counts.unread_count > 0) { + if let Some(member_id) = count + .recipient_member_id + .as_deref() + .filter(|member_id| !member_id.trim().is_empty()) + { + let entry = canonical + .entry(member_id.to_string()) + .or_insert_with(|| (BTreeSet::new(), 0, count.max_unread_id)); + entry.0.insert(count.recipient_agent_id.clone()); + entry.1 = entry.1.saturating_add(count.unread_count); + entry.2 = entry.2.max(count.max_unread_id); + } else { + legacy.push(count.clone()); + } + } + let mut normalized_counts = legacy; + normalized_counts.extend(canonical.into_iter().map( + |(member_id, (agent_ids, unread_count, max_unread_id))| { + crate::coordination::agent_inbox::AgentInboxUnreadRecipientCounts { + recipient_agent_id: agent_ids.into_iter().collect::>().join(","), + recipient_member_id: Some(member_id), + unread_count, + max_unread_id, + } + }, + )); + normalized_counts.sort_by(|left, right| { + left.recipient_member_id + .cmp(&right.recipient_member_id) + .then_with(|| left.recipient_agent_id.cmp(&right.recipient_agent_id)) + }); + + for counts in &normalized_counts { + let Some(member_id) = counts + .recipient_member_id + .as_deref() + .filter(|member_id| !member_id.trim().is_empty()) + else { + repairs.push(UnreadRecipientRepair { + recipient_member_id: None, + recipient_agent_id: counts.recipient_agent_id.clone(), + unread_count: counts.unread_count, + max_unread_id: counts.max_unread_id, + reason: UnreadRecipientUnavailableReason::MissingCanonicalMemberId, + }); + continue; + }; + + if member_id != COORDINATOR_MEMBER_ID + && roster_member_ids + .as_ref() + .is_some_and(|roster| !roster.contains(member_id)) + { + repairs.push(UnreadRecipientRepair { + recipient_member_id: Some(member_id.to_string()), + recipient_agent_id: counts.recipient_agent_id.clone(), + unread_count: counts.unread_count, + max_unread_id: counts.max_unread_id, + reason: UnreadRecipientUnavailableReason::UnknownRosterMember, + }); + continue; + } + + let runtime = if member_id == COORDINATOR_MEMBER_ID { + coordinator + .as_ref() + .map(|runtime| (runtime.status, runtime.updated_at.as_str(), false)) + } else { + workers + .iter() + .find(|runtime| runtime.member_id.as_deref() == Some(member_id)) + .map(|runtime| { + ( + runtime.status, + runtime.updated_at.as_str(), + runtime.cli_agent_type.is_some(), + ) + }) + }; + let Some((status, updated_at, unsupported_transport)) = runtime else { + repairs.push(UnreadRecipientRepair { + recipient_member_id: Some(member_id.to_string()), + recipient_agent_id: counts.recipient_agent_id.clone(), + unread_count: counts.unread_count, + max_unread_id: counts.max_unread_id, + reason: UnreadRecipientUnavailableReason::MissingSession, + }); + continue; + }; + + let reason = if unsupported_transport { + Some(UnreadRecipientUnavailableReason::UnsupportedTransport) + } else { + match status { + SessionStatus::Pending => { + match pending_materialization_disposition(Some(updated_at)) { + PendingMaterializationDisposition::Grace => None, + PendingMaterializationDisposition::Expired => { + Some(UnreadRecipientUnavailableReason::PendingMaterializationExpired) + } + PendingMaterializationDisposition::InvalidTimestamp => { + Some(UnreadRecipientUnavailableReason::InvalidSessionTimestamp) + } + } + } + SessionStatus::Paused => { + Some(UnreadRecipientUnavailableReason::AdministrativelyPaused) + } + SessionStatus::Archived => Some(UnreadRecipientUnavailableReason::ArchivedSession), + SessionStatus::Idle + | SessionStatus::Completed + | SessionStatus::Failed + | SessionStatus::Cancelled + | SessionStatus::Abandoned + | SessionStatus::Timeout => { + let unread_fingerprint = unread_fingerprints_by_member + .get(member_id) + .map(|fingerprint| format!("unread:{fingerprint}")) + .ok_or_else(|| { + format!( + "unread recipient {member_id} was missing from grouped snapshot" + ) + })?; + match budget_disposition_with_connection( + conn, + run_id, + MEMBER_REWAKE, + member_id, + &unread_fingerprint, + )? { + BudgetDisposition::Exhausted => { + Some(UnreadRecipientUnavailableReason::RecoveryBudgetExhausted) + } + BudgetDisposition::Allowed | BudgetDisposition::Backoff => None, + } + } + SessionStatus::Running + | SessionStatus::WaitingForUser + | SessionStatus::WaitingForFunds => None, + } + }; + + if let Some(reason) = reason { + repairs.push(UnreadRecipientRepair { + recipient_member_id: Some(member_id.to_string()), + recipient_agent_id: counts.recipient_agent_id.clone(), + unread_count: counts.unread_count, + max_unread_id: counts.max_unread_id, + reason, + }); + } + } + + repairs.sort_by_key(UnreadRecipientRepair::stable_key); + Ok(repairs) +} + +pub(super) fn unavailable_unread_recipient_repair_fingerprint_with_connection( + conn: &Connection, + run_id: &str, + workers: &[WorkerSessionRuntime], +) -> Result, String> { + let counts = AgentInboxStore::unread_counts_by_recipient_with_connection(conn, run_id)?; + let repairs = unavailable_unread_recipient_repairs_from_counts_with_connection( + conn, run_id, workers, &counts, + )?; + Ok(unread_recipient_repair_snapshot_fingerprint(&repairs)) +} + +fn unread_recipient_repair_snapshot_fingerprint( + repairs: &[UnreadRecipientRepair], +) -> Option { + let facts = repairs + .iter() + .map(UnreadRecipientRepair::snapshot_fact) + .collect::>(); + recovery_repair_fingerprint(&facts) +} + +fn append_unread_recipient_repairs( + repairs: &[UnreadRecipientRepair], + reasons: &mut Vec, + facts: &mut Vec, +) { + for repair in repairs { + reasons.push(repair.coordinator_reason()); + facts.push(repair.repair_fact()); + } +} + +fn coordinator_notice_budget_exists_with_connection( + conn: &Connection, + run_id: &str, +) -> Result { + conn.query_row( + "SELECT EXISTS( + SELECT 1 FROM agent_org_recovery_attempts + WHERE org_run_id=?1 AND action_kind=?2 AND target_key='coordinator' + )", + params![run_id, COORDINATOR_NOTICE], + |row| row.get(0), + ) + .map_err(|err| err.to_string()) +} + +fn coordinator_unread_recovery_with_connection( + conn: &Connection, + run_id: &str, + unread_fingerprints_by_member: &HashMap, +) -> Result<(bool, Vec), String> { + let Some(unread_fingerprint) = unread_fingerprints_by_member.get(COORDINATOR_MEMBER_ID) else { + return Ok((false, Vec::new())); + }; + let Some(info) = AgentOrgRunStore::find_coordinator_session_by_member_id_with_connection( + conn, + run_id, + COORDINATOR_MEMBER_ID, + )? + else { + return Ok((true, Vec::new())); + }; + let fingerprint = member_rewake_fingerprint_from_unread(info.status, Some(unread_fingerprint)); + let wake = is_wakeable_status(info.status) + && budget_disposition_with_connection( + conn, + run_id, + MEMBER_REWAKE, + COORDINATOR_MEMBER_ID, + &fingerprint, + )? == BudgetDisposition::Allowed; + Ok(( + true, + wake.then(|| COORDINATOR_MEMBER_ID.to_string()) + .into_iter() + .collect(), + )) +} + pub fn inspect_stalled_run(run_id: &str) -> Result { - if AgentOrgRunStore::get_run_status(run_id)? != Some(AgentOrgRunStatus::Running) { + let mut conn = get_connection().map_err(|err| err.to_string())?; + let tx = conn + .transaction_with_behavior(rusqlite::TransactionBehavior::Deferred) + .map_err(|err| err.to_string())?; + let plan = inspect_stalled_run_with_connection(&tx, run_id)?; + tx.commit().map_err(|err| err.to_string())?; + Ok(plan) +} + +/// Analyze one run from one coherent SQLite read snapshot. The executor still +/// opens short writer transactions and revalidates every derived action before +/// committing it; this function intentionally performs no writes. +pub(super) fn inspect_stalled_run_with_connection( + conn: &Connection, + run_id: &str, +) -> Result { + if AgentOrgRunStore::get_run_status_with_connection(conn, run_id)? + != Some(AgentOrgRunStatus::Running) + { return Ok(StallRecoveryPlan::default()); } - let finality_assessment = AgentOrgRunStore::assess_run_finality(run_id)?; - let tasks = agent_org_tasks::AgentOrgTaskStore::list(run_id)?; + let finality_assessment = AgentOrgRunStore::finality_assessment_with_connection(conn, run_id)?; + let unread_counts = AgentInboxStore::unread_counts_by_recipient_with_connection(conn, run_id)?; + let unread_fingerprints_by_member = unread_fingerprints_by_member(&unread_counts); + let (coordinator_unread, coordinator_unread_wake_member_ids) = + coordinator_unread_recovery_with_connection(conn, run_id, &unread_fingerprints_by_member)?; + let workers = AgentOrgRunStore::list_descendant_worker_sessions_with_connection(conn, run_id)?; + let unavailable_unread_repairs = + unavailable_unread_recipient_repairs_from_counts_with_connection( + conn, + run_id, + &workers, + &unread_counts, + )?; + let unavailable_unread_fingerprint = + unread_recipient_repair_snapshot_fingerprint(&unavailable_unread_repairs); + let coordinator_unread_is_unavailable = unavailable_unread_repairs + .iter() + .any(|repair| repair.recipient_member_id.as_deref() == Some(COORDINATOR_MEMBER_ID)); + let coordinator_unread_suppresses_notice = + coordinator_unread && !coordinator_unread_is_unavailable; + + if finality_assessment.facts.corrupt_task_count > 0 { + let count = finality_assessment.facts.corrupt_task_count; + let mut reasons = vec![format!( + "The Agent Org task board has {count} persisted integrity or run-limit violation(s). The watchdog refused to guess task state or declare completion. Use task_list to identify bounded diagnostics. Ordinary task tools intentionally cannot rewrite malformed rows; cancel/delete this run or use a trusted maintenance path to repair the database before continuing." + )]; + let mut repair_facts = corrupt_task_repair_facts(conn, run_id)?; + append_unread_recipient_repairs( + &unavailable_unread_repairs, + &mut reasons, + &mut repair_facts, + ); + let has_new_notice = !coordinator_unread_suppresses_notice; + let work_revision = finality_assessment + .facts + .progress + .as_ref() + .map(|progress| progress.work_revision); + return Ok(StallRecoveryPlan { + wake_member_ids: coordinator_unread_wake_member_ids, + continuation_actions: Vec::new(), + assignment_actions: Vec::new(), + coordinator_repair_reason: has_new_notice + .then(|| bounded_recovery_reason_text(&reasons)), + coordinator_repair_fingerprint: has_new_notice + .then(|| { + recovery_repair_fingerprint(&repair_facts).ok_or_else(|| { + format!( + "finality reported {count} corrupt task row(s), but no corrupt identity was found" + ) + }) + }) + .transpose()?, + coordinator_repair_work_revision: has_new_notice.then_some(work_revision).flatten(), + coordinator_repair_task_fingerprint: None, + coordinator_repair_inbox_fingerprint: has_new_notice + .then_some(unavailable_unread_fingerprint) + .flatten(), + coordinator_repair_active: true, + clear_coordinator_notice_budget: false, + terminal_candidate: false, + }); + } + + let tasks = + agent_org_tasks::AgentOrgTaskStore::list_operational_after_validated_with_connection( + conn, run_id, + )?; + let task_snapshot_work_revision = finality_assessment + .facts + .progress + .as_ref() + .map(|progress| progress.work_revision); + let task_snapshot_fingerprint = task_snapshot_fingerprint(&tasks); let task_graph = agent_org_tasks::TaskGraphIndex::new(&tasks); - let pending_plan_task_ids = AgentOrgPlanApprovalStore::list_pending_by_run(run_id)? - .into_iter() - .map(|approval| approval.source_task_id) - .collect::>(); - let workers = AgentOrgRunStore::list_descendant_worker_sessions(run_id)?; + let pending_plan_task_ids = + AgentOrgPlanApprovalStore::list_pending_summaries_by_run_with_connection(conn, run_id)? + .into_iter() + .map(|approval| approval.source_task_id) + .collect::>(); let has_active_worker = workers.iter().any(|worker| is_active_status(worker.status)); let mut member_status = HashMap::new(); @@ -28,8 +870,12 @@ pub fn inspect_stalled_run(run_id: &str) -> Result { let mut unsupported_transport_members = HashSet::new(); for worker in &workers { if let Some(member_id) = worker.member_id.as_deref() { - member_status.insert(member_id.to_string(), worker.status); - member_updated_at.insert(member_id.to_string(), worker.updated_at.clone()); + member_status + .entry(member_id.to_string()) + .or_insert(worker.status); + member_updated_at + .entry(member_id.to_string()) + .or_insert_with(|| worker.updated_at.clone()); if worker.cli_agent_type.is_some() { unsupported_transport_members.insert(member_id.to_string()); } @@ -43,24 +889,34 @@ pub fn inspect_stalled_run(run_id: &str) -> Result { // never used to steal ownership. if has_active_worker { let mut reasons = Vec::new(); - let mut keys = Vec::new(); + let mut repair_facts = Vec::new(); + append_unread_recipient_repairs( + &unavailable_unread_repairs, + &mut reasons, + &mut repair_facts, + ); + append_dependency_integrity_repairs(&tasks, &mut reasons, &mut repair_facts); for task in &tasks { let Some(owner) = task.owner.as_deref() else { let ready = task.status == TaskStatus::Pending && task_graph.is_ready(task); if ready { let mut eligible = agent_org_tasks::eligible_member_ids(task); eligible.sort(); - keys.push(format!( - "awaiting_coordinator_assignment:{}:{}", - task.id, - eligible.join(",") + let mut fields = vec![Some(task.id.clone())]; + fields.extend(eligible.into_iter().map(Some)); + repair_facts.push(RecoveryRepairFact::new( + "awaiting_coordinator_assignment", + fields, )); reasons.push(ready_unassigned_repair_reason(task)); } continue; }; if unsupported_transport_members.contains(owner) && !task.status.is_resolved() { - keys.push(format!("unsupported_transport:{}:{}", task.id, owner)); + repair_facts.push(RecoveryRepairFact::new( + "unsupported_transport", + [Some(task.id.clone()), Some(owner.to_string())], + )); reasons.push(format!( "task {} is owned by historical CLI member {}, whose Agent Org transport is unsupported; reassign it to a Rust member.", task.id, owner @@ -71,24 +927,42 @@ pub fn inspect_stalled_run(run_id: &str) -> Result { || task.status != TaskStatus::InProgress || member_status.get(owner) != Some(&SessionStatus::Running) || !is_stale_in_progress(task.updated_at.as_str(), member_updated_at.get(owner)) - || has_unread_for_member(run_id, owner)? + || unread_fingerprints_by_member.contains_key(owner) { continue; } - keys.push(format!("stale_running_owner:{}:{}", task.id, owner)); + repair_facts.push(RecoveryRepairFact::new( + "stale_running_owner", + [Some(task.id.clone()), Some(owner.to_string())], + )); reasons.push(format!( "task {} is still in_progress under Running member {} but appears stale; the watchdog will not steal it based on age. Ask the owner to continue/retry or explicitly reassign it.", task.id, owner )); } - keys.sort(); + let coordinator_repair_active = !reasons.is_empty(); + let clear_coordinator_notice_budget = !coordinator_repair_active + && coordinator_notice_budget_exists_with_connection(conn, run_id)?; + let has_new_notice = coordinator_repair_active && !coordinator_unread_suppresses_notice; return Ok(StallRecoveryPlan { - wake_member_ids: Vec::new(), + wake_member_ids: coordinator_unread_wake_member_ids, continuation_actions: Vec::new(), assignment_actions: Vec::new(), - coordinator_repair_reason: (!reasons.is_empty()).then(|| reasons.join("\n")), - coordinator_repair_fingerprint: (!keys.is_empty()) - .then(|| reason_fingerprint(&keys.join("|"))), + coordinator_repair_reason: has_new_notice + .then(|| bounded_recovery_reason_text(&reasons)), + coordinator_repair_fingerprint: has_new_notice + .then(|| recovery_repair_fingerprint(&repair_facts)) + .flatten(), + coordinator_repair_work_revision: has_new_notice + .then_some(task_snapshot_work_revision) + .flatten(), + coordinator_repair_task_fingerprint: has_new_notice + .then(|| task_snapshot_fingerprint.clone()), + coordinator_repair_inbox_fingerprint: has_new_notice + .then_some(unavailable_unread_fingerprint) + .flatten(), + coordinator_repair_active, + clear_coordinator_notice_budget, terminal_candidate: false, }); } @@ -100,12 +974,8 @@ pub fn inspect_stalled_run(run_id: &str) -> Result { .into_iter() .map(|task| task.id.clone()) .collect(); - let assignment_conn = get_connection().map_err(|err| err.to_string())?; let historically_assigned_task_ids = - AgentInboxStore::task_assignment_ids_for_open_tasks_with_connection( - &assignment_conn, - run_id, - )?; + AgentInboxStore::task_assignment_ids_for_open_tasks_with_connection(conn, run_id)?; let mut owned_open_tasks_by_member: HashMap<&str, Vec> = HashMap::new(); let mut ready_pending_tasks_by_member: HashMap<&str, Vec> = HashMap::new(); for task in &tasks { @@ -148,7 +1018,8 @@ pub fn inspect_stalled_run(run_id: &str) -> Result { if wake_member_ids.iter().any(|existing| existing == member_id) { continue; } - let has_unread = has_unread_for_member(run_id, member_id)?; + let unread_fingerprint = unread_fingerprints_by_member.get(member_id); + let has_unread = unread_fingerprint.is_some(); let continuation_task_ids = owned_open_tasks_by_member.get(member_id); let assignment_task_ids = ready_pending_tasks_by_member.get(member_id); let needs_assignment = assignment_task_ids.is_some_and(|task_ids| !task_ids.is_empty()); @@ -173,8 +1044,18 @@ pub fn inspect_stalled_run(run_id: &str) -> Result { if !has_unread && !needs_assignment && !needs_terminal_continuation { continue; } - let rewake_fingerprint = member_rewake_fingerprint(run_id, member_id, worker.status)?; - if !delayed_rewake_allowed(run_id, member_id, worker.status, &rewake_fingerprint)? { + let rewake_fingerprint = member_rewake_fingerprint_from_unread( + worker.status, + unread_fingerprint.map(String::as_str), + ); + if budget_disposition_with_connection( + conn, + run_id, + MEMBER_REWAKE, + member_id, + &rewake_fingerprint, + )? != BudgetDisposition::Allowed + { continue; } if !has_unread && needs_assignment { @@ -203,28 +1084,16 @@ pub fn inspect_stalled_run(run_id: &str) -> Result { // row with a quiescent coordinator session means a wake was lost // (e.g. dropped at shutdown). Redeliver instead of inserting more // notices on top of it. - let coordinator_unread = has_unread_for_member(run_id, COORDINATOR_MEMBER_ID)?; - if coordinator_unread { - if let Some(info) = - AgentOrgRunStore::find_coordinator_session_by_member_id(run_id, COORDINATOR_MEMBER_ID)? - { - let rewake_fingerprint = - member_rewake_fingerprint(run_id, COORDINATOR_MEMBER_ID, info.status)?; - if is_wakeable_status(info.status) - && delayed_rewake_allowed( - run_id, - COORDINATOR_MEMBER_ID, - info.status, - &rewake_fingerprint, - )? - { - wake_member_ids.push(COORDINATOR_MEMBER_ID.to_string()); - } - } - } + wake_member_ids.extend(coordinator_unread_wake_member_ids); let mut needs_repair = Vec::new(); - let mut repair_keys = Vec::new(); + let mut repair_facts = Vec::new(); + append_unread_recipient_repairs( + &unavailable_unread_repairs, + &mut needs_repair, + &mut repair_facts, + ); + append_dependency_integrity_repairs(&tasks, &mut needs_repair, &mut repair_facts); for task in &tasks { if task.status.is_resolved() { continue; @@ -232,26 +1101,93 @@ pub fn inspect_stalled_run(run_id: &str) -> Result { if let Some(owner) = task.owner.as_deref() { let owner_status = member_status.get(owner).copied(); if unsupported_transport_members.contains(owner) { - repair_keys.push(format!("unsupported_transport:{}:{}", task.id, owner)); + repair_facts.push(RecoveryRepairFact::new( + "unsupported_transport", + [Some(task.id.clone()), Some(owner.to_string())], + )); needs_repair.push(format!( "task {} is owned by historical CLI member {}, whose Agent Org transport is unsupported; reassign owner_member_id to a Rust member", task.id, owner )); } else if owner_status.is_none() || owner_status == Some(SessionStatus::Archived) { - repair_keys.push(format!("missing_owner:{}:{}", task.id, owner)); + repair_facts.push(RecoveryRepairFact::new( + "missing_owner", + [Some(task.id.clone()), Some(owner.to_string())], + )); needs_repair.push(format!( "task {} is owned by unavailable member {}; reassign owner_member_id or repair eligible_member_ids", task.id, owner )); + } else if owner_status == Some(SessionStatus::Paused) { + repair_facts.push(RecoveryRepairFact::new( + "paused_owner", + [Some(task.id.clone()), Some(owner.to_string())], + )); + needs_repair.push(format!( + "task {} is owned by administratively paused member {}. The watchdog will not wake a paused member; resume that member or explicitly reassign owner_member_id.", + task.id, owner + )); + } else if owner_status == Some(SessionStatus::Pending) { + match pending_materialization_disposition( + member_updated_at.get(owner).map(String::as_str), + ) { + PendingMaterializationDisposition::Grace => {} + PendingMaterializationDisposition::Expired => { + repair_facts.push(RecoveryRepairFact::new( + "pending_owner_timeout", + [Some(task.id.clone()), Some(owner.to_string())], + )); + needs_repair.push(format!( + "task {} is owned by member {}, but that session remained Pending beyond the {}-second materialization grace period. Retry materialization or explicitly reassign owner_member_id.", + task.id, owner, PENDING_MATERIALIZATION_GRACE_SECS + )); + } + PendingMaterializationDisposition::InvalidTimestamp => { + repair_facts.push(RecoveryRepairFact::new( + "pending_owner_invalid_timestamp", + [Some(task.id.clone()), Some(owner.to_string())], + )); + needs_repair.push(format!( + "task {} is owned by Pending member {}, whose session timestamp is missing or invalid. Repair the session or explicitly reassign owner_member_id.", + task.id, owner + )); + } + } } else if match owner_status { - Some(status) if status.is_terminal() => rewake_budget_exhausted( - run_id, - owner, - &member_rewake_fingerprint(run_id, owner, status)?, - )?, - _ => false, + Some( + status @ (SessionStatus::Completed + | SessionStatus::Failed + | SessionStatus::Cancelled + | SessionStatus::Abandoned + | SessionStatus::Timeout + | SessionStatus::Archived), + ) => { + let fingerprint = member_rewake_fingerprint_from_unread( + status, + unread_fingerprints_by_member.get(owner).map(String::as_str), + ); + budget_disposition_with_connection( + conn, + run_id, + MEMBER_REWAKE, + owner, + &fingerprint, + )? == BudgetDisposition::Exhausted + } + Some( + SessionStatus::Pending + | SessionStatus::Idle + | SessionStatus::Running + | SessionStatus::WaitingForUser + | SessionStatus::WaitingForFunds + | SessionStatus::Paused, + ) + | None => false, } { - repair_keys.push(format!("terminal_owner:{}:{}", task.id, owner)); + repair_facts.push(RecoveryRepairFact::new( + "terminal_owner", + [Some(task.id.clone()), Some(owner.to_string())], + )); needs_repair.push(format!( "task {} is owned by terminal member {} whose automatic retry budget is exhausted; retry the owner, reassign owner_member_id, or repair eligible_member_ids", task.id, owner @@ -259,14 +1195,17 @@ pub fn inspect_stalled_run(run_id: &str) -> Result { } else if task.status == TaskStatus::InProgress && !pending_plan_task_ids.contains(&task.id) && is_stale_in_progress(task.updated_at.as_str(), member_updated_at.get(owner)) - && !has_unread_for_member(run_id, owner)? + && !unread_fingerprints_by_member.contains_key(owner) { - repair_keys.push(format!("stale_owner:{}:{}", task.id, owner)); + repair_facts.push(RecoveryRepairFact::new( + "stale_owner", + [Some(task.id.clone()), Some(owner.to_string())], + )); let eligible = agent_org_tasks::eligible_member_ids(task); let eligible = if eligible.is_empty() { "none".to_string() } else { - eligible.join(", ") + bounded_id_list_preview(&eligible, 8, 160) }; needs_repair.push(format!( "task {} is still in_progress under member {} but appears stale; task_updated_at={}, owner_updated_at={}, eligible_member_ids=[{}]. The watchdog does not steal work from a Running member based on age alone. Ask the owner to continue/retry, reassign owner_member_id, or repair eligible_member_ids.", @@ -283,11 +1222,11 @@ pub fn inspect_stalled_run(run_id: &str) -> Result { && historically_assigned_task_ids.contains(&task.id) && owner_status == Some(SessionStatus::Idle) && is_stale_in_progress(task.updated_at.as_str(), member_updated_at.get(owner)) - && !has_unread_for_member(run_id, owner)? + && !unread_fingerprints_by_member.contains_key(owner) { - repair_keys.push(format!( - "consumed_assignment_without_start:{}:{}", - task.id, owner + repair_facts.push(RecoveryRepairFact::new( + "consumed_assignment_without_start", + [Some(task.id.clone()), Some(owner.to_string())], )); needs_repair.push(format!( "task {} was assigned to member {}, its assignment was consumed, but the task never entered in_progress. Ask the owner for status or explicitly retry/reassign it.", @@ -306,25 +1245,23 @@ pub fn inspect_stalled_run(run_id: &str) -> Result { let eligible_member_ids = agent_org_tasks::eligible_member_ids(task); let mut stable_eligible = eligible_member_ids.clone(); stable_eligible.sort(); - repair_keys.push(format!( - "awaiting_coordinator_assignment:{}:{}", - task.id, - stable_eligible.join(",") + let mut fields = vec![Some(task.id.clone())]; + fields.extend(stable_eligible.into_iter().map(Some)); + repair_facts.push(RecoveryRepairFact::new( + "awaiting_coordinator_assignment", + fields, )); needs_repair.push(ready_unassigned_repair_reason(task)); } - // A terminal reconciliation may legitimately decline even when every - // Task is resolved (for example, the coordinator has not observed the - // latest work revision). Convert the actionable canonical blockers into - // one bounded coordinator repair instead of returning an empty plan that - // leaves the run permanently Running. for blocker in &finality_assessment.blockers { match blocker { AgentOrgFinalityBlocker::EmptyTaskBoardRequiresCompletionIntent => { - repair_keys.push("empty_board_requires_completion_intent".to_string()); + repair_facts.push(RecoveryRepairFact::marker( + "empty_board_requires_completion_intent", + )); needs_repair.push( - "the Agent Org task board is empty. If the mission truly needs no durable tasks, call org_run_complete with a concise summary; otherwise create the missing task graph." + "the Agent Org task board is empty. If the mission truly requires no durable tasks, call org_run_complete with a concise summary; otherwise create the missing task graph." .to_string(), ); } @@ -332,42 +1269,83 @@ pub fn inspect_stalled_run(run_id: &str) -> Result { requested_work_revision, current_work_revision, } => { - repair_keys.push(format!( - "stale_completion_intent:{requested_work_revision:?}:{current_work_revision}" + repair_facts.push(RecoveryRepairFact::new( + "stale_completion_intent", + [ + requested_work_revision.map(|revision| revision.to_string()), + Some(current_work_revision.to_string()), + ], )); needs_repair.push(format!( - "the previous completion request observed work revision {requested_work_revision:?}, but the board is now revision {current_work_revision}. Re-inspect the board before calling org_run_complete again." + "the previous completion request observed work revision {requested_work_revision:?}, but the board is now revision {current_work_revision}. Re-inspect the current task board and call org_run_complete again only if it is still finished." )); } AgentOrgFinalityBlocker::CoordinatorHasNotObservedLatestWork { observed_work_revision, current_work_revision, } if tasks.iter().all(|task| task.status.is_resolved()) => { - repair_keys.push(format!( - "coordinator_observation:{observed_work_revision:?}:{current_work_revision}" + repair_facts.push(RecoveryRepairFact::new( + "coordinator_observation", + [ + observed_work_revision.map(|revision| revision.to_string()), + Some(current_work_revision.to_string()), + ], )); needs_repair.push(format!( "all durable tasks are resolved, but the coordinator has only observed work revision {observed_work_revision:?}; the current revision is {current_work_revision}. Refresh task_list and produce the final user-facing synthesis." )); } - _ => {} + AgentOrgFinalityBlocker::CorruptTaskData { count } => { + repair_facts.extend(corrupt_task_repair_facts(conn, run_id)?); + needs_repair.push(format!( + "{count} task row(s) contain invalid persisted JSON. Do not declare completion; inspect and repair the task records." + )); + } + AgentOrgFinalityBlocker::ProgressStateMissing => { + repair_facts.push(RecoveryRepairFact::marker("missing_run_progress")); + needs_repair.push( + "the run is missing its durable work-revision record. Do not declare completion until the state is repaired." + .to_string(), + ); + } + AgentOrgFinalityBlocker::RootSessionMissing => { + repair_facts.push(RecoveryRepairFact::marker("missing_coordinator_session")); + needs_repair.push( + "the run has no materialized coordinator session, so final completion cannot be safely presented." + .to_string(), + ); + } + AgentOrgFinalityBlocker::RunMissing + | AgentOrgFinalityBlocker::RunNotRunning { .. } + | AgentOrgFinalityBlocker::SessionsActive { .. } + | AgentOrgFinalityBlocker::OpenTasks { .. } + | AgentOrgFinalityBlocker::CoordinatorHasNotObservedLatestWork { .. } + | AgentOrgFinalityBlocker::UnreadInbox { .. } + | AgentOrgFinalityBlocker::ActiveInterventions { .. } + | AgentOrgFinalityBlocker::InFlightTurnIntents { .. } + | AgentOrgFinalityBlocker::PendingPlanApprovals { .. } + | AgentOrgFinalityBlocker::TerminalStateInconsistent { .. } => {} } } - let coordinator_repair_reason = if !needs_repair.is_empty() && !coordinator_unread { - Some(needs_repair.join("\n")) - } else { - None - }; - repair_keys.sort(); + let coordinator_repair_reason = + if !needs_repair.is_empty() && !coordinator_unread_suppresses_notice { + Some(bounded_recovery_reason_text(&needs_repair)) + } else { + None + }; let coordinator_repair_fingerprint = coordinator_repair_reason .as_ref() - .map(|_| reason_fingerprint(&repair_keys.join("|"))); + .and_then(|_| recovery_repair_fingerprint(&repair_facts)); let terminal_candidate = matches!( finality_assessment.decision, AgentOrgFinalityDecision::Complete | AgentOrgFinalityDecision::Abandon ); + let has_coordinator_repair = coordinator_repair_reason.is_some(); + let coordinator_repair_active = !needs_repair.is_empty(); + let clear_coordinator_notice_budget = !coordinator_repair_active + && coordinator_notice_budget_exists_with_connection(conn, run_id)?; Ok(StallRecoveryPlan { wake_member_ids, @@ -375,6 +1353,16 @@ pub fn inspect_stalled_run(run_id: &str) -> Result { assignment_actions, coordinator_repair_reason, coordinator_repair_fingerprint, + coordinator_repair_work_revision: has_coordinator_repair + .then_some(task_snapshot_work_revision) + .flatten(), + coordinator_repair_task_fingerprint: has_coordinator_repair + .then_some(task_snapshot_fingerprint), + coordinator_repair_inbox_fingerprint: has_coordinator_repair + .then_some(unavailable_unread_fingerprint) + .flatten(), + coordinator_repair_active, + clear_coordinator_notice_budget, terminal_candidate, }) } diff --git a/src-tauri/crates/agent-core/src/core/coordination/agent_org_watchdog/plan.rs b/src-tauri/crates/agent-core/src/core/coordination/agent_org_watchdog/plan.rs index 95c1d280d..fd799a60b 100644 --- a/src-tauri/crates/agent-core/src/core/coordination/agent_org_watchdog/plan.rs +++ b/src-tauri/crates/agent-core/src/core/coordination/agent_org_watchdog/plan.rs @@ -26,9 +26,32 @@ pub struct StallRecoveryPlan { /// inbox rows (an unread notice already covers redelivery via /// `wake_member_ids`). pub coordinator_repair_reason: Option, - /// Stable hash of the repair state keys used to reset the notice budget - /// when the underlying stalled board changes. + /// Stable hash of typed repair facts (task/reason/member ids), excluding + /// prose and timestamps so copy edits do not reset the retry budget. pub coordinator_repair_fingerprint: Option, + /// Work revision observed with the task snapshot used to compose the + /// coordinator repair. The executor compares it again under the shared + /// writer lock before persisting the notice. + pub coordinator_repair_work_revision: Option, + /// Stable fingerprint of the canonical task state/graph used to compose + /// the repair. This catches stale analyzer output even when a historical + /// writer failed to bump `work_revision`. + pub coordinator_repair_task_fingerprint: Option, + /// Stable fingerprint of typed unavailable-unread recipient facts used to + /// compose the repair. The executor recomputes it before inserting a + /// notice so a concurrently restored session or drained Inbox cannot + /// produce stale guidance. + pub coordinator_repair_inbox_fingerprint: Option, + /// Whether the coherent snapshot still contained any coordinator repair + /// condition, including one temporarily suppressed by an already-unread + /// coordinator message. A false value ends the previous fault episode and + /// clears its notice budget so the same fault can be reported if it later + /// genuinely recurs. + pub coordinator_repair_active: bool, + /// End a previously persisted coordinator fault episode. Set only when + /// the analyzer sees no current repair *and* a budget row actually exists, + /// avoiding one no-op writer transaction per healthy run per tick. + pub clear_coordinator_notice_budget: bool, /// Every task resolved + every worker terminal: the run can be /// reconciled to a terminal status. pub terminal_candidate: bool, @@ -41,6 +64,10 @@ impl StallRecoveryPlan { && self.assignment_actions.is_empty() && self.coordinator_repair_reason.is_none() && self.coordinator_repair_fingerprint.is_none() + && self.coordinator_repair_work_revision.is_none() + && self.coordinator_repair_task_fingerprint.is_none() + && self.coordinator_repair_inbox_fingerprint.is_none() + && !self.clear_coordinator_notice_budget && !self.terminal_candidate } } diff --git a/src-tauri/crates/agent-core/src/core/coordination/agent_org_watchdog/recover.rs b/src-tauri/crates/agent-core/src/core/coordination/agent_org_watchdog/recover.rs index 122c6e275..3f6d6db36 100644 --- a/src-tauri/crates/agent-core/src/core/coordination/agent_org_watchdog/recover.rs +++ b/src-tauri/crates/agent-core/src/core/coordination/agent_org_watchdog/recover.rs @@ -2,11 +2,43 @@ //! carrying out a [`super::plan::StallRecoveryPlan`] returned by //! [`super::inspect::inspect_stalled_run`]. -use super::budget::{coordinator_notice_budget_allows, prune_recovery_budgets, record_attempt}; +use super::budget::{ + budget_disposition_with_connection, prune_recovery_budgets, record_attempt_with_connection, + BudgetDisposition, +}; +use super::inspect::{ + inspect_stalled_run_with_connection, pending_materialization_disposition, + task_snapshot_fingerprint, unavailable_unread_recipient_repair_fingerprint_with_connection, + PendingMaterializationDisposition, +}; use super::*; pub fn spawn(app_handle: AppHandle) { tauri::async_runtime::spawn(async move { + match tokio::task::spawn_blocking(|| { + AgentOrgPlanApprovalStore::repair_latest_plan_artifacts() + }) + .await + { + Ok(Ok(report)) => { + if report.repaired > 0 || report.failed > 0 { + tracing::info!( + inspected = report.inspected, + repaired = report.repaired, + failed = report.failed, + "[agent_org_watchdog] reconciled durable plan artifacts at startup" + ); + } + } + Ok(Err(err)) => tracing::warn!( + error = %err, + "[agent_org_watchdog] startup plan artifact reconciliation failed" + ), + Err(err) => tracing::warn!( + error = %err, + "[agent_org_watchdog] startup plan artifact worker failed" + ), + } let mut interval = tokio::time::interval(Duration::from_secs(WATCHDOG_INTERVAL_SECS)); // A slow scan must not be "repaid" with back-to-back burst // ticks afterwards; the next scheduled tick is enough. @@ -30,6 +62,10 @@ pub fn spawn(app_handle: AppHandle) { fn recover_all_stalled_runs(app_handle: AppHandle) -> Result<(), String> { let runs = AgentOrgRunStore::list_running_runs(usize::MAX)?; run_best_effort_cleanup("prune recovery budgets", prune_recovery_budgets); + run_best_effort_cleanup("clear expired member interventions", || { + crate::coordination::agent_member_interventions::AgentMemberInterventionStore::clear_expired_and_legacy() + .map(|_| ()) + }); run_best_effort_cleanup("cancel stale plan approvals", || { AgentOrgPlanApprovalStore::cancel_pending_for_terminal_or_missing_runs().map(|_| ()) }); @@ -83,7 +119,18 @@ pub fn recover_stalled_run( run_id: &str, ) -> Result { let plan = inspect_stalled_run(run_id)?; + let wake_hook = AppHandleInboxWakeHook::new(app_handle); + execute_stall_recovery_plan(run_id, plan, wake_hook.as_ref()) +} +/// Execute an advisory analyzer plan through a caller-supplied Wake hook. +/// Keeping orchestration here makes the full reconcile → revalidate → persist +/// → wake ordering directly testable without constructing a Tauri runtime. +fn execute_stall_recovery_plan( + run_id: &str, + plan: StallRecoveryPlan, + wake_hook: &dyn InboxWakeHook, +) -> Result { // Reconcile first: when the run actually closes there is nothing // left to wake or repair. When reconciliation declines (e.g. the // coordinator root session is still open), fall through and deliver @@ -95,41 +142,235 @@ pub fn recover_stalled_run( } } + // Analyzer output is advisory. Every derived inbox row is revalidated + // under the same writer lock + IMMEDIATE transaction as its insert. A + // task completed, reassigned, or re-blocked after inspection therefore + // produces neither stale input nor a spurious wake. + let action_member_ids = plan + .assignment_actions + .iter() + .map(|action| action.member_id.as_str()) + .chain( + plan.continuation_actions + .iter() + .map(|action| action.member_id.as_str()), + ) + .collect::>(); + let mut wake_member_ids = HashSet::new(); + + for action in &plan.assignment_actions { + let has_current_assignment = + !agent_org_tasks::enqueue_task_assignments_if_still_ready_for_recovery( + run_id, + &action.task_ids, + &action.recipient_agent_id, + &action.member_id, + SYSTEM_SENDER_ID, + None, + "Agent Org recovery", + )? + .is_empty(); + if has_current_assignment || has_unread_for_member(run_id, &action.member_id)? { + wake_member_ids.insert(action.member_id.clone()); + } + } + + for action in &plan.continuation_actions { + if insert_member_continuation_if_tasks_current(run_id, action)? { + wake_member_ids.insert(action.member_id.clone()); + } + } + + // Members without a derived action were selected only because the + // analyzer observed unread durable input. Recheck that input rather than + // waking from the stale plan alone. if AgentOrgRunStore::get_run_status(run_id)? == Some(AgentOrgRunStatus::Running) { - let recovery_tasks = agent_org_tasks::AgentOrgTaskStore::list(run_id)?; - for action in &plan.assignment_actions { - if has_unread_for_member(run_id, &action.member_id)? { - continue; - } - for task_id in &action.task_ids { - let Some(task) = recovery_tasks.iter().find(|task| &task.id == task_id) else { - continue; - }; - let ready = task.status == TaskStatus::Pending - && task.owner.as_deref() == Some(action.member_id.as_str()) - && task.blocked_by.iter().all(|blocker_id| { - recovery_tasks - .iter() - .find(|candidate| &candidate.id == blocker_id) - .is_some_and(|candidate| candidate.status.is_resolved()) - }); - if ready { - agent_org_tasks::enqueue_task_assigned_to( - task, - &action.recipient_agent_id, - &action.member_id, - SYSTEM_SENDER_ID, - None, - "Agent Org recovery", - )?; - } + for member_id in &plan.wake_member_ids { + if !action_member_ids.contains(member_id.as_str()) + && has_unread_for_member(run_id, member_id)? + { + wake_member_ids.insert(member_id.clone()); } } - for action in &plan.continuation_actions { - if has_unread_for_member(run_id, &action.member_id)? { - continue; + } + + if !wake_member_ids.is_empty() { + for member_id in &wake_member_ids { + wake_hook.wake_member(member_id, run_id); + } + } + + if let Some(reason) = plan.coordinator_repair_reason.as_deref() { + let fingerprint = plan + .coordinator_repair_fingerprint + .as_deref() + .unwrap_or(reason); + match insert_coordinator_stall_notice( + run_id, + reason, + fingerprint, + plan.coordinator_repair_work_revision, + plan.coordinator_repair_task_fingerprint.as_deref(), + plan.coordinator_repair_inbox_fingerprint.as_deref(), + )? { + CoordinatorNoticeDispatch::Inserted | CoordinatorNoticeDispatch::ExistingUnread => { + wake_hook.wake_member(COORDINATOR_MEMBER_ID, run_id); } - AgentInboxStore::insert(InsertInboxParams { + CoordinatorNoticeDispatch::Deferred => { + tracing::debug!( + run_id = %run_id, + "[agent_org_watchdog] coordinator notice deferred during session materialization grace" + ); + } + CoordinatorNoticeDispatch::RecipientUnavailable => { + tracing::warn!( + run_id = %run_id, + repair_reason = %reason, + "[agent_org_watchdog] coordinator repair cannot be delivered because the coordinator session is unavailable" + ); + } + CoordinatorNoticeDispatch::BudgetSuppressed => { + tracing::debug!( + run_id = %run_id, + "[agent_org_watchdog] coordinator stall notice suppressed by budget (reason unchanged)" + ); + } + CoordinatorNoticeDispatch::Stale => {} + } + } + + if plan.clear_coordinator_notice_budget { + clear_coordinator_notice_budget_if_recovered(run_id)?; + } + + Ok(plan) +} + +fn clear_coordinator_notice_budget_if_recovered(run_id: &str) -> Result<(), String> { + with_sessions_writer(|| -> Result<(), String> { + let mut conn = get_connection().map_err(|err| err.to_string())?; + let tx = conn + .transaction_with_behavior(rusqlite::TransactionBehavior::Immediate) + .map_err(|err| err.to_string())?; + if !inspect_stalled_run_with_connection(&tx, run_id)?.coordinator_repair_active { + tx.execute( + "DELETE FROM agent_org_recovery_attempts + WHERE org_run_id=?1 AND action_kind=?2 AND target_key='coordinator'", + params![run_id, COORDINATOR_NOTICE], + ) + .map_err(|err| err.to_string())?; + } + tx.commit().map_err(|err| err.to_string())?; + Ok(()) + }) +} + +fn bounded_id_list_preview(ids: &[String], max_items: usize, max_chars_per_id: usize) -> String { + let preview = ids + .iter() + .take(max_items) + .map(|id| crate::utils::safe_truncate_chars_to_string(id, max_chars_per_id)) + .collect::>() + .join(", "); + let omitted = ids.len().saturating_sub(max_items); + if omitted > 0 { + format!("{preview}, +{omitted} more (use task_list/task_get)") + } else { + preview + } +} + +/// Persist a terminal-member continuation only when the analyzed tasks still +/// have the same owner, remain unresolved, and have no unresolved blockers. +fn insert_member_continuation_if_tasks_current( + run_id: &str, + action: &MemberContinuationAction, +) -> Result { + with_sessions_writer(|| -> Result { + let mut conn = get_connection().map_err(|err| err.to_string())?; + let tx = conn + .transaction_with_behavior(rusqlite::TransactionBehavior::Immediate) + .map_err(|err| err.to_string())?; + let running: bool = tx + .query_row( + "SELECT EXISTS( + SELECT 1 FROM agent_org_runs WHERE id=?1 AND status='running' + )", + params![run_id], + |row| row.get(0), + ) + .map_err(|err| err.to_string())?; + if !running { + tx.commit().map_err(|err| err.to_string())?; + return Ok(false); + } + + let sessions = + AgentOrgRunStore::list_descendant_worker_sessions_with_connection(&tx, run_id)?; + if !recovery_dispatch_recipient_is_available( + &sessions, + &action.member_id, + &action.recipient_agent_id, + ) { + tx.commit().map_err(|err| err.to_string())?; + return Ok(false); + } + + let has_unread: bool = tx + .query_row( + "SELECT EXISTS( + SELECT 1 FROM agent_inbox + WHERE org_run_id=?1 AND recipient_member_id=?2 AND read_at IS NULL + AND NOT EXISTS ( + SELECT 1 FROM agent_inbox_delivery_resolutions resolution + WHERE resolution.inbox_id=agent_inbox.id + ) + )", + params![run_id, &action.member_id], + |row| row.get(0), + ) + .map_err(|err| err.to_string())?; + if has_unread { + tx.commit().map_err(|err| err.to_string())?; + return Ok(true); + } + + let tasks = + agent_org_tasks::AgentOrgTaskStore::list_operational_with_connection(&tx, run_id)?; + let graph = agent_org_tasks::TaskGraphIndex::new(&tasks); + let planned_ids = action.task_ids.iter().collect::>(); + let pending_plan_task_ids = { + let mut stmt = tx + .prepare( + "SELECT source_task_id FROM agent_org_plan_approvals + WHERE org_run_id=?1 AND status='pending'", + ) + .map_err(|err| err.to_string())?; + let rows = stmt + .query_map(params![run_id], |row| row.get::<_, String>(0)) + .map_err(|err| err.to_string())?; + rows.collect::, _>>() + .map_err(|err| err.to_string())? + }; + let current_task_ids = tasks + .iter() + .filter(|task| planned_ids.contains(&task.id)) + .filter(|task| task.owner.as_deref() == Some(action.member_id.as_str())) + .filter(|task| { + matches!(task.status, TaskStatus::Pending | TaskStatus::InProgress) + && graph.unresolved_blockers(&task.id).is_empty() + && !pending_plan_task_ids.contains(&task.id) + }) + .map(|task| task.id.clone()) + .collect::>(); + if current_task_ids.is_empty() { + tx.commit().map_err(|err| err.to_string())?; + return Ok(false); + } + + AgentInboxStore::insert_in_tx( + &tx, + InsertInboxParams { recipient_agent_id: action.recipient_agent_id.clone(), recipient_member_id: Some(action.member_id.clone()), sender_agent_id: SYSTEM_SENDER_ID.to_string(), @@ -139,61 +380,267 @@ pub fn recover_stalled_run( summary: "Retry assigned Agent Org work".to_string(), text: format!( "A previous turn ended before your owned task(s) were resolved. Continue only these durable task ids: {}. Refresh task_list/task_get first, then update each task from its current state. Do not create replacement duplicates.", - action.task_ids.join(", ") + bounded_id_list_preview(¤t_task_ids, 8, 1_000) ), }, - })?; + }, + )?; + tx.commit().map_err(|err| err.to_string())?; + Ok(true) + }) +} + +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +enum CoordinatorNoticeDispatch { + Inserted, + ExistingUnread, + Deferred, + RecipientUnavailable, + BudgetSuppressed, + Stale, +} + +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +enum CoordinatorRecipientDisposition { + Available, + Deferred, + Unavailable, +} + +fn insert_coordinator_stall_notice( + run_id: &str, + reason: &str, + reason_fingerprint: &str, + expected_work_revision: Option, + expected_task_fingerprint: Option<&str>, + expected_inbox_fingerprint: Option<&str>, +) -> Result { + with_sessions_writer(|| -> Result { + let mut conn = get_connection().map_err(|err| err.to_string())?; + let tx = conn + .transaction_with_behavior(rusqlite::TransactionBehavior::Immediate) + .map_err(|err| err.to_string())?; + let current_plan = inspect_stalled_run_with_connection(&tx, run_id)?; + if current_plan.coordinator_repair_fingerprint.as_deref() != Some(reason_fingerprint) { + tx.commit().map_err(|err| err.to_string())?; + return Ok(CoordinatorNoticeDispatch::Stale); } - } + let coordinator_runtime: Option<(String, Option, Option)> = tx + .query_row( + "SELECT run.coordinator_agent_id, session.status, session.updated_at + FROM agent_org_runs run + LEFT JOIN agent_sessions session + ON session.session_id=run.root_session_id + WHERE run.id=?1 AND run.status='running'", + params![run_id], + |row| Ok((row.get(0)?, row.get(1)?, row.get(2)?)), + ) + .optional() + .map_err(|err| err.to_string())?; + let Some((coordinator_agent_id, coordinator_status, coordinator_updated_at)) = + coordinator_runtime + else { + tx.commit().map_err(|err| err.to_string())?; + return Ok(CoordinatorNoticeDispatch::Stale); + }; - if !plan.wake_member_ids.is_empty() { - let wake_hook = AppHandleInboxWakeHook::new(app_handle.clone()); - for member_id in &plan.wake_member_ids { - wake_hook.wake_member(member_id, run_id); + let current_work_revision = tx + .query_row( + "SELECT work_revision FROM agent_org_run_progress WHERE org_run_id=?1", + params![run_id], + |row| row.get::<_, i64>(0), + ) + .optional() + .map_err(|err| err.to_string())?; + if current_work_revision != expected_work_revision { + tx.commit().map_err(|err| err.to_string())?; + return Ok(CoordinatorNoticeDispatch::Stale); } - } - if let Some(reason) = plan.coordinator_repair_reason.as_deref() { - let fingerprint = plan - .coordinator_repair_fingerprint + if let Some(expected_task_fingerprint) = expected_task_fingerprint { + let current_tasks = + agent_org_tasks::AgentOrgTaskStore::list_operational_with_connection(&tx, run_id)?; + if task_snapshot_fingerprint(¤t_tasks) != expected_task_fingerprint { + tx.commit().map_err(|err| err.to_string())?; + return Ok(CoordinatorNoticeDispatch::Stale); + } + } + + if let Some(expected_inbox_fingerprint) = expected_inbox_fingerprint { + let workers = + AgentOrgRunStore::list_descendant_worker_sessions_with_connection(&tx, run_id)?; + if unavailable_unread_recipient_repair_fingerprint_with_connection( + &tx, run_id, &workers, + )? .as_deref() - .unwrap_or(reason); - if coordinator_notice_budget_allows(run_id, fingerprint)? { - if insert_coordinator_stall_notice(run_id, reason)? { - record_attempt(run_id, COORDINATOR_NOTICE, "coordinator", fingerprint)?; - AppHandleInboxWakeHook::new(app_handle).wake_member(COORDINATOR_MEMBER_ID, run_id); + != Some(expected_inbox_fingerprint) + { + tx.commit().map_err(|err| err.to_string())?; + return Ok(CoordinatorNoticeDispatch::Stale); } - } else { - tracing::debug!( - run_id = %run_id, - "[agent_org_watchdog] coordinator stall notice suppressed by budget (reason unchanged)" - ); } - } - Ok(plan) -} + let coordinator_disposition = match coordinator_status { + None => CoordinatorRecipientDisposition::Unavailable, + Some(coordinator_status) => { + let coordinator_status = + SessionStatus::parse(&coordinator_status).ok_or_else(|| { + format!( + "unknown coordinator session status for run {run_id}: {coordinator_status:?}" + ) + })?; + let disposition = match coordinator_status { + SessionStatus::Pending => { + match pending_materialization_disposition(coordinator_updated_at.as_deref()) + { + PendingMaterializationDisposition::Grace => { + CoordinatorRecipientDisposition::Deferred + } + PendingMaterializationDisposition::Expired + | PendingMaterializationDisposition::InvalidTimestamp => { + CoordinatorRecipientDisposition::Unavailable + } + } + } + SessionStatus::Paused | SessionStatus::Archived => { + CoordinatorRecipientDisposition::Unavailable + } + SessionStatus::Idle + | SessionStatus::Running + | SessionStatus::WaitingForUser + | SessionStatus::WaitingForFunds + | SessionStatus::Completed + | SessionStatus::Failed + | SessionStatus::Cancelled + | SessionStatus::Abandoned + | SessionStatus::Timeout => CoordinatorRecipientDisposition::Available, + }; + if disposition == CoordinatorRecipientDisposition::Available + && matches!( + coordinator_status, + SessionStatus::Idle + | SessionStatus::Completed + | SessionStatus::Failed + | SessionStatus::Cancelled + | SessionStatus::Abandoned + | SessionStatus::Timeout + ) + { + let unread_fingerprint = + AgentInboxStore::unread_fingerprint_for_member_with_connection( + &tx, + COORDINATOR_MEMBER_ID, + run_id, + )?; + if let Some(unread_fingerprint) = unread_fingerprint { + let fingerprint = format!("unread:{unread_fingerprint}"); + if budget_disposition_with_connection( + &tx, + run_id, + MEMBER_REWAKE, + COORDINATOR_MEMBER_ID, + &fingerprint, + )? == BudgetDisposition::Exhausted + { + CoordinatorRecipientDisposition::Unavailable + } else { + disposition + } + } else { + disposition + } + } else { + disposition + } + } + }; + match coordinator_disposition { + CoordinatorRecipientDisposition::Deferred => { + tx.commit().map_err(|err| err.to_string())?; + return Ok(CoordinatorNoticeDispatch::Deferred); + } + CoordinatorRecipientDisposition::Unavailable => { + if budget_disposition_with_connection( + &tx, + run_id, + COORDINATOR_NOTICE, + "coordinator", + reason_fingerprint, + )? != BudgetDisposition::Allowed + { + tx.commit().map_err(|err| err.to_string())?; + return Ok(CoordinatorNoticeDispatch::BudgetSuppressed); + } + record_attempt_with_connection( + &tx, + run_id, + COORDINATOR_NOTICE, + "coordinator", + reason_fingerprint, + )?; + tx.commit().map_err(|err| err.to_string())?; + return Ok(CoordinatorNoticeDispatch::RecipientUnavailable); + } + CoordinatorRecipientDisposition::Available => {} + } -fn insert_coordinator_stall_notice(run_id: &str, reason: &str) -> Result { - if AgentOrgRunStore::get_run_status(run_id)? != Some(AgentOrgRunStatus::Running) { - return Ok(false); - } - let store = crate::definitions::orgs::orgs_store(); - let Some(context) = AgentOrgRunStore::context_for_run(run_id, &store)? else { - return Ok(false); - }; - AgentInboxStore::insert(InsertInboxParams { - recipient_agent_id: context.coordinator_agent_id, - recipient_member_id: Some(COORDINATOR_MEMBER_ID.to_string()), - sender_agent_id: SYSTEM_SENDER_ID.to_string(), - sender_member_id: None, - org_run_id: Some(run_id.to_string()), - message: AgentMessage::Plain { - summary: "Agent Org recovery needed".to_string(), - text: format!( - "The Agent Org watchdog detected stalled work that needs coordinator repair.\n\n{reason}\n\nUse task_list/task_get to inspect the task board, then use task_update owner_member_id or eligible_member_ids to repair dispatch. Never assign work outside eligible_member_ids." - ), - }, - })?; - Ok(true) + let coordinator_has_unread: bool = tx + .query_row( + "SELECT EXISTS( + SELECT 1 FROM agent_inbox + WHERE org_run_id=?1 + AND recipient_member_id=?2 + AND read_at IS NULL + AND NOT EXISTS ( + SELECT 1 FROM agent_inbox_delivery_resolutions resolution + WHERE resolution.inbox_id=agent_inbox.id + ) + )", + params![run_id, COORDINATOR_MEMBER_ID], + |row| row.get(0), + ) + .map_err(|err| err.to_string())?; + if coordinator_has_unread { + tx.commit().map_err(|err| err.to_string())?; + return Ok(CoordinatorNoticeDispatch::ExistingUnread); + } + + if budget_disposition_with_connection( + &tx, + run_id, + COORDINATOR_NOTICE, + "coordinator", + reason_fingerprint, + )? != BudgetDisposition::Allowed + { + tx.commit().map_err(|err| err.to_string())?; + return Ok(CoordinatorNoticeDispatch::BudgetSuppressed); + } + + AgentInboxStore::insert_in_tx( + &tx, + InsertInboxParams { + recipient_agent_id: coordinator_agent_id, + recipient_member_id: Some(COORDINATOR_MEMBER_ID.to_string()), + sender_agent_id: SYSTEM_SENDER_ID.to_string(), + sender_member_id: None, + org_run_id: Some(run_id.to_string()), + message: AgentMessage::Plain { + summary: "Agent Org recovery needed".to_string(), + text: format!( + "The Agent Org watchdog detected stalled work that needs coordinator repair.\n\n{reason}\n\nUse task_list/task_get to inspect the task board, then use task_update owner_member_id or eligible_member_ids to repair dispatch. Never assign work outside eligible_member_ids." + ), + }, + }, + )?; + record_attempt_with_connection( + &tx, + run_id, + COORDINATOR_NOTICE, + "coordinator", + reason_fingerprint, + )?; + tx.commit().map_err(|err| err.to_string())?; + Ok(CoordinatorNoticeDispatch::Inserted) + }) } diff --git a/src-tauri/crates/agent-core/src/core/coordination/agent_org_watchdog/tests.rs b/src-tauri/crates/agent-core/src/core/coordination/agent_org_watchdog/tests.rs index 3bce6e817..4a777bbf8 100644 --- a/src-tauri/crates/agent-core/src/core/coordination/agent_org_watchdog/tests.rs +++ b/src-tauri/crates/agent-core/src/core/coordination/agent_org_watchdog/tests.rs @@ -1,4 +1,6 @@ -use super::budget::{budget_disposition, coordinator_notice_allowed, BudgetDisposition}; +use super::budget::{ + budget_disposition, coordinator_notice_allowed, rewake_budget_exhausted, BudgetDisposition, +}; use super::inspect::is_wakeable_status; use super::recover::{recover_listed_runs, run_best_effort_cleanup}; use super::*; @@ -129,3 +131,31 @@ fn coordinator_notice_budget_backs_off_and_resets_on_new_reason() { assert!(!coordinator_notice_allowed(&run_id, "task a stuck").expect("backoff")); assert!(coordinator_notice_allowed(&run_id, "task b stuck").expect("new reason")); } + +#[test] +fn rewake_budget_exhaustion_requires_all_attempts_and_an_expired_cooldown() { + let _sandbox = test_helpers::test_env::sandbox(); + let conn = get_connection().expect("db"); + init_schema(&conn).expect("schema"); + let run_id = format!("run-{}", uuid::Uuid::new_v4()); + let member_id = "member-exhausted"; + let fingerprint = "same-input"; + assert!(!rewake_budget_exhausted(&run_id, member_id, fingerprint).expect("initial budget")); + let expired_at = (Utc::now() - ChronoDuration::seconds(1)).to_rfc3339(); + conn.execute( + "INSERT INTO agent_org_recovery_attempts + (org_run_id, action_kind, target_key, reason_fingerprint, attempts, + next_allowed_at, updated_at) + VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?6)", + params![ + &run_id, + MEMBER_REWAKE, + member_id, + fingerprint, + RECOVERY_DELAYS_SECS.len() as i64, + &expired_at, + ], + ) + .expect("seed exhausted budget"); + assert!(rewake_budget_exhausted(&run_id, member_id, fingerprint).expect("exhausted budget")); +} diff --git a/src-tauri/crates/agent-core/src/core/session/goal_loop.rs b/src-tauri/crates/agent-core/src/core/session/goal_loop.rs index c1cd066a5..5d2803cca 100644 --- a/src-tauri/crates/agent-core/src/core/session/goal_loop.rs +++ b/src-tauri/crates/agent-core/src/core/session/goal_loop.rs @@ -439,6 +439,7 @@ async fn enqueue_continuation( None, None, None, + None, crate::foundation::session_bridge::TurnIntentBridgeSource::Queue, ) .await; diff --git a/src-tauri/crates/agent-core/src/core/session/launch/launch_org.rs b/src-tauri/crates/agent-core/src/core/session/launch/launch_org.rs index a2eb8dc1e..719c27a02 100644 --- a/src-tauri/crates/agent-core/src/core/session/launch/launch_org.rs +++ b/src-tauri/crates/agent-core/src/core/session/launch/launch_org.rs @@ -304,6 +304,7 @@ pub(super) async fn send_initial_turn( ide_context: Option, agent_definition_id: Option, sub_agent_ids: Vec, + intent_org_run_id: Option, source: crate::foundation::session_bridge::TurnIntentBridgeSource, ) -> Result<(), String> { if sub_agent_ids.is_empty() { @@ -326,6 +327,7 @@ pub(super) async fn send_initial_turn( None, None, None, + intent_org_run_id, source, ) .await?; @@ -364,6 +366,7 @@ pub(super) async fn send_initial_turn( None, None, None, + intent_org_run_id, crate::foundation::session_bridge::TurnIntentBridgeSource::AgentOrg, ) .await?; diff --git a/src-tauri/crates/agent-core/src/core/session/launch/mod.rs b/src-tauri/crates/agent-core/src/core/session/launch/mod.rs index b2397f759..b8933b089 100644 --- a/src-tauri/crates/agent-core/src/core/session/launch/mod.rs +++ b/src-tauri/crates/agent-core/src/core/session/launch/mod.rs @@ -659,6 +659,7 @@ pub(crate) async fn launch_rust_agent_run( ide_context_for_send, agent_definition_id_for_send, sub_agent_ids_for_send, + agent_org_run_id_for_background.clone(), crate::foundation::session_bridge::TurnIntentBridgeSource::AgentOrg, ) .await; @@ -784,6 +785,7 @@ pub(crate) async fn launch_rust_agent_run( ide_context_for_send, agent_definition_id_for_send, sub_agent_ids_for_send, + agent_org_run_id_for_send.clone(), crate::foundation::session_bridge::TurnIntentBridgeSource::UserSubmit, ) .await; diff --git a/src-tauri/crates/agent-core/src/core/session/turn/processor/execute.rs b/src-tauri/crates/agent-core/src/core/session/turn/processor/execute.rs index d215d8687..2a854a3fd 100644 --- a/src-tauri/crates/agent-core/src/core/session/turn/processor/execute.rs +++ b/src-tauri/crates/agent-core/src/core/session/turn/processor/execute.rs @@ -39,6 +39,8 @@ impl UnifiedMessageProcessor { turn_id: &str, messages: &mut Vec, reasoning_trigger: Option, + turn_intent_id: &str, + projected_inbox_ids: Vec, ) -> Result<(TurnResult, UnifiedEventHandler), String> { let effective_policy = self.effective_tool_policy(); @@ -69,6 +71,8 @@ impl UnifiedMessageProcessor { }; let turn_config = TurnConfig { + turn_intent_id: turn_intent_id.to_string(), + projected_inbox_ids, model: turn_model, account_id: self.runtime.account_id.clone(), context_window_override: self diff --git a/src-tauri/crates/agent-core/src/core/session/turn/processor/inbox_drain/guard.rs b/src-tauri/crates/agent-core/src/core/session/turn/processor/inbox_drain/guard.rs index 89ad46f45..1ea018c45 100644 --- a/src-tauri/crates/agent-core/src/core/session/turn/processor/inbox_drain/guard.rs +++ b/src-tauri/crates/agent-core/src/core/session/turn/processor/inbox_drain/guard.rs @@ -72,6 +72,14 @@ impl DrainGuard { !self.pending_ids.is_empty() } + /// Exact source rows this turn will acknowledge only after successful + /// provider execution. Threaded into tool-call context so prospective + /// finality can project this turn's guaranteed commit without treating + /// unrelated unread mail as consumed. + pub fn pending_ids(&self) -> &[i64] { + &self.pending_ids + } + pub fn new_materialization_ids(&self) -> &[i64] { &self.new_materialization_ids } diff --git a/src-tauri/crates/agent-core/src/core/session/turn/processor/inbox_drain/render.rs b/src-tauri/crates/agent-core/src/core/session/turn/processor/inbox_drain/render.rs index 49b010833..adac20bc4 100644 --- a/src-tauri/crates/agent-core/src/core/session/turn/processor/inbox_drain/render.rs +++ b/src-tauri/crates/agent-core/src/core/session/turn/processor/inbox_drain/render.rs @@ -34,18 +34,50 @@ pub(super) fn render_inbox_attachment( pub(super) fn render_inbox_transcript(rows: &[AgentInboxRecord]) -> String { rows.iter() - .filter_map(|row| match row.decode_payload() { + .map(|row| match row.decode_payload() { Ok(message) => { let body = render_payload_for_transcript(&message); let trimmed = body.trim(); - (!trimmed.is_empty()).then(|| trimmed.to_string()) + if trimmed.is_empty() { + render_transcript_fallback(row, "decoded payload rendered empty") + } else { + render_transcript_entry(row, trimmed) + } } - Err(_) => None, + Err(err) => render_transcript_fallback(row, &err), }) .collect::>() .join("\n\n") } +/// Every Inbox source row needs a durable transcript representation before +/// it can ever be acknowledged. Historical/manual DB corruption therefore +/// degrades to a bounded diagnostic instead of being filtered out (which +/// would leave an unread row spinning forever without a receipt). +fn render_transcript_fallback(row: &AgentInboxRecord, reason: &str) -> String { + const MAX_RAW_PREVIEW_CHARS: usize = 4_096; + let raw = crate::utils::safe_truncate_chars_to_string(&row.payload_json, MAX_RAW_PREVIEW_CHARS); + render_transcript_entry( + row, + &format!("Undecodable payload: {reason}\nRaw payload: {raw}"), + ) +} + +fn render_transcript_entry(row: &AgentInboxRecord, body: &str) -> String { + let sender = row.sender_member_id.as_deref().unwrap_or_else(|| { + if row.sender_agent_id == USER_SENDER_ID { + "user" + } else { + "system" + } + }); + let request_id = row.request_id.as_deref().unwrap_or("none"); + format!( + "[Agent Org inbox message id={} from_member_id={} kind={} request_id={} created_at={}]\n{}", + row.id, sender, row.payload_kind, request_id, row.created_at, body + ) +} + fn render_one_row(row: &AgentInboxRecord) -> String { let request_id_attr = match &row.request_id { Some(rid) => format!(" request_id=\"{}\"", xml_escape(rid)), @@ -54,11 +86,20 @@ fn render_one_row(row: &AgentInboxRecord) -> String { let body = match row.decode_payload() { Ok(msg) => render_payload(&msg), - Err(err) => format!( - "{}", - xml_escape(&err), - xml_escape(&row.payload_json) - ), + Err(err) => { + const MAX_RAW_PREVIEW_CHARS: usize = 4_096; + let raw = crate::utils::safe_truncate_chars_to_string( + &row.payload_json, + MAX_RAW_PREVIEW_CHARS, + ); + let truncated = raw.len() < row.payload_json.len(); + format!( + "{}", + xml_escape(&err), + truncated, + xml_escape(&raw) + ) + } }; let sender_label = row.sender_member_id.as_deref().unwrap_or_else(|| { diff --git a/src-tauri/crates/agent-core/src/core/session/turn/processor/inbox_drain/tests.rs b/src-tauri/crates/agent-core/src/core/session/turn/processor/inbox_drain/tests.rs index 29272f5b2..1a9cef2b0 100644 --- a/src-tauri/crates/agent-core/src/core/session/turn/processor/inbox_drain/tests.rs +++ b/src-tauri/crates/agent-core/src/core/session/turn/processor/inbox_drain/tests.rs @@ -942,13 +942,8 @@ fn deferred_drain_without_commit_leaves_rows_unread() { // committing — simulates a turn that failed before completion. let mut first: Vec = Vec::new(); { - let guard = drain_and_render_deferred( - &ctx, - "worker-1", - Some("member-worker-1"), - &mut first, - None, - ); + let guard = + drain_and_render_deferred(&ctx, "worker-1", Some("member-worker-1"), &mut first, None); assert_eq!(guard.drained_count(), 1); // Guard goes out of scope here without `.commit()` — rows // stay unread. @@ -1079,8 +1074,8 @@ fn materialized_batch_replay_delivers_only_rows_that_arrived_later() { ) .expect("materialize later row"); - let transcript = crate::session::persistence::load_messages(&session_id) - .expect("load durable transcript"); + let transcript = + crate::session::persistence::load_messages(&session_id).expect("load durable transcript"); assert_eq!(transcript.len(), 2, "one transcript per materialized batch"); let combined = transcript .iter() @@ -1345,9 +1340,7 @@ fn shutdown_notification_failure_leaves_source_unread_and_retries_before_deliver /// reassignment. Tasks that were already completed are left alone. #[test] fn drain_releases_member_tasks_on_accepted_shutdown() { - use crate::coordination::agent_org_tasks::{ - AgentOrgTaskStore, CreateTaskParams, TaskStatus, - }; + use crate::coordination::agent_org_tasks::{AgentOrgTaskStore, CreateTaskParams, TaskStatus}; let _sandbox = test_helpers::test_env::sandbox(); let ctx = running_ctx_for_members(&[ @@ -1537,9 +1530,7 @@ fn drain_does_not_steal_task_from_stale_running_worker() { use crate::coordination::agent_org_runs::{ AgentOrgRunEntryMode, AgentOrgRunStatus, AgentOrgRunStore, CreateAgentOrgRunParams, }; - use crate::coordination::agent_org_tasks::{ - AgentOrgTaskStore, CreateTaskParams, TaskStatus, - }; + use crate::coordination::agent_org_tasks::{AgentOrgTaskStore, CreateTaskParams, TaskStatus}; use crate::core::definitions::orgs::{OrgDefinition, OrgMember}; use crate::session::persistence::{session_type, upsert_session, UnifiedSessionRecord}; @@ -1671,9 +1662,7 @@ fn drain_does_not_steal_task_from_stale_running_worker() { /// must not mutate ownership or manufacture a TaskAssigned message. #[test] fn ownerless_task_is_not_claimed_by_inbox_drain() { - use crate::coordination::agent_org_tasks::{ - AgentOrgTaskStore, CreateTaskParams, TaskStatus, - }; + use crate::coordination::agent_org_tasks::{AgentOrgTaskStore, CreateTaskParams, TaskStatus}; let _sandbox = test_helpers::test_env::sandbox(); let ctx = running_ctx_for_members(&[("member-alice", "alice", "Alice")]); @@ -1817,9 +1806,7 @@ fn drain_drops_exec_mode_request_from_non_coordinator_sender() { /// coordinator. Assignment happens only through task tools. #[test] fn coordinator_inbox_drain_does_not_assign_ownerless_task() { - use crate::coordination::agent_org_tasks::{ - AgentOrgTaskStore, CreateTaskParams, TaskStatus, - }; + use crate::coordination::agent_org_tasks::{AgentOrgTaskStore, CreateTaskParams, TaskStatus}; let _sandbox = test_helpers::test_env::sandbox(); let ctx = running_ctx_for_members(&[("member-worker", "worker", "Worker")]); diff --git a/src-tauri/crates/agent-core/src/core/session/turn/processor/mod.rs b/src-tauri/crates/agent-core/src/core/session/turn/processor/mod.rs index 0a83412ac..192078165 100644 --- a/src-tauri/crates/agent-core/src/core/session/turn/processor/mod.rs +++ b/src-tauri/crates/agent-core/src/core/session/turn/processor/mod.rs @@ -896,12 +896,18 @@ impl UnifiedMessageProcessor { // Reasoning trigger words are detected on the CURRENT user input // only (never history) so escalation stays per-turn. let reasoning_trigger = crate::providers::thinking_mode::detect_reasoning_trigger(content); + let projected_inbox_ids = inbox_guard + .as_ref() + .map(|guard| guard.pending_ids().to_vec()) + .unwrap_or_default(); let turn_result = self .execute_turn_with_reactive_retry( session_id, &turn_id, &mut messages, reasoning_trigger, + &context.turn_intent_id, + projected_inbox_ids, ) .await; if let Some(prefetch_hook) = self.turn_prefetch_hook.lock().await.take() { diff --git a/src-tauri/crates/agent-core/src/core/session/wingman/loop_runner.rs b/src-tauri/crates/agent-core/src/core/session/wingman/loop_runner.rs index 791a26f93..a6423486e 100644 --- a/src-tauri/crates/agent-core/src/core/session/wingman/loop_runner.rs +++ b/src-tauri/crates/agent-core/src/core/session/wingman/loop_runner.rs @@ -248,6 +248,7 @@ impl WingmanLoop { &self.session_id, &msg.turn_intent_id, None, + None, crate::foundation::session_bridge::TurnIntentBridgeSource::Wingman, crate::foundation::session_bridge::TurnIntentBridgeStatus::Queued, ); diff --git a/src-tauri/crates/agent-core/src/core/tools/builtin_tools/table/agent_org_tasks.rs b/src-tauri/crates/agent-core/src/core/tools/builtin_tools/table/agent_org_tasks.rs index c8ecfef6e..c9a15519d 100644 --- a/src-tauri/crates/agent-core/src/core/tools/builtin_tools/table/agent_org_tasks.rs +++ b/src-tauri/crates/agent-core/src/core/tools/builtin_tools/table/agent_org_tasks.rs @@ -99,4 +99,20 @@ pub(super) static TOOLS: &[ToolEntry] = &[ required_capability: CapOrch, ..DEFAULT_TOOL_ENTRY }, + ToolEntry { + name: tool_names::ORG_INBOX_REPAIR, + description: "Repair an undeliverable Agent Org Inbox row.", + description_detail: "Coordinator-only. Inspects, cancels, or supersedes an Inbox delivery while preserving the original row as audit evidence.", + category: tool_categories::ORCHESTRATION, + icon_id: "mail-warning", + simulator_app: AppChannels, + app_subtool: SubTodo, + chat_block: CbOrgTask, + hidden: true, + label_running: "tools.taskUpdateRunning", + label_done: "tools.taskUpdateDone", + label_failed: "tools.taskUpdateFailed", + required_capability: CapOrch, + ..DEFAULT_TOOL_ENTRY + }, ]; diff --git a/src-tauri/crates/agent-core/src/core/tools/call_context.rs b/src-tauri/crates/agent-core/src/core/tools/call_context.rs index c8fab379f..c6ee85adb 100644 --- a/src-tauri/crates/agent-core/src/core/tools/call_context.rs +++ b/src-tauri/crates/agent-core/src/core/tools/call_context.rs @@ -27,6 +27,11 @@ //! `ToolRegistry::set_session_key` shared mutable state that was the //! root cause of the `create_plan` subagent-misattribution saga. //! +//! - `turn_intent_id` and `projected_inbox_ids`: identify the exact durable +//! turn and Inbox batch whose effects become committed only after this turn +//! succeeds. Agent Org finality uses them to build a prospective +//! completion certificate without guessing or subtracting unrelated work. +//! //! ## Defaults //! //! `CallContext::default()` is a zero-value context (empty strings). @@ -48,6 +53,12 @@ pub struct CallContext { pub call_id: String, /// Identifier of the dispatching session. pub session_id: String, + /// Durable lifecycle identity of the dispatching turn. Empty for + /// maintenance/direct test calls that do not belong to a real turn. + pub turn_intent_id: String, + /// Exact Agent Org Inbox rows held by this turn's deferred drain guard. + /// These rows are acknowledged only when the turn succeeds. + pub projected_inbox_ids: Vec, } impl CallContext { @@ -56,6 +67,22 @@ impl CallContext { Self { call_id: call_id.into(), session_id: session_id.into(), + turn_intent_id: String::new(), + projected_inbox_ids: Vec::new(), + } + } + + pub fn for_turn( + call_id: impl Into, + session_id: impl Into, + turn_intent_id: impl Into, + projected_inbox_ids: Vec, + ) -> Self { + Self { + call_id: call_id.into(), + session_id: session_id.into(), + turn_intent_id: turn_intent_id.into(), + projected_inbox_ids, } } } diff --git a/src-tauri/crates/agent-core/src/core/tools/impls/orchestration/agent/mod.rs b/src-tauri/crates/agent-core/src/core/tools/impls/orchestration/agent/mod.rs index f57406a04..475ac3bea 100644 --- a/src-tauri/crates/agent-core/src/core/tools/impls/orchestration/agent/mod.rs +++ b/src-tauri/crates/agent-core/src/core/tools/impls/orchestration/agent/mod.rs @@ -922,6 +922,8 @@ impl Tool for AgentTool { .map(|sm| sm.max_iterations) .unwrap_or(DEFAULT_SUBAGENT_MAX_ITERATIONS); let turn_config = TurnConfig { + turn_intent_id: String::new(), + projected_inbox_ids: Vec::new(), model: model.clone(), account_id: self.config.session_account_id.clone(), context_window_override: agent.context_window, diff --git a/src-tauri/crates/agent-core/src/core/tools/impls/orchestration/agent_org/inbox_repair.rs b/src-tauri/crates/agent-core/src/core/tools/impls/orchestration/agent_org/inbox_repair.rs new file mode 100644 index 000000000..c3aada258 --- /dev/null +++ b/src-tauri/crates/agent-core/src/core/tools/impls/orchestration/agent_org/inbox_repair.rs @@ -0,0 +1,402 @@ +use std::sync::Arc; + +use async_trait::async_trait; +use schemars::JsonSchema; +use serde::Deserialize; +use serde_json::{json, Value}; + +use crate::coordination::agent_inbox::{ + AgentInboxDeliveryResolutionKind, AgentInboxStore, ResolveInboxDeliveryError, + ResolveInboxDeliveryParams, +}; +use crate::coordination::agent_org_runs::COORDINATOR_MEMBER_ID; +use crate::tools::names as tool_names; +use crate::tools::traits::{params_schema, parse_params, CallContext, Tool, ToolError}; + +use super::TaskToolsContext; + +/// Explicit operator action for an Inbox row that cannot reach its original +/// recipient. This is intentionally not an automatic forwarding API: typed +/// messages can carry task/approval semantics that must be reconstructed by +/// the normal task/message tools rather than copied to a guessed identity. +#[derive(Debug, Deserialize, JsonSchema)] +#[serde(tag = "action", rename_all = "snake_case", deny_unknown_fields)] +pub enum OrgInboxRepairParams { + Inspect { + inbox_id: i64, + }, + Cancel { + inbox_id: i64, + reason: String, + }, + Supersede { + inbox_id: i64, + reason: String, + #[serde(default)] + replacement_inbox_id: Option, + #[serde(default)] + replacement_task_id: Option, + }, +} + +pub struct OrgInboxRepairTool { + ctx: Arc, +} + +impl OrgInboxRepairTool { + pub fn new(ctx: Arc) -> Self { + Self { ctx } + } +} + +#[async_trait] +impl Tool for OrgInboxRepairTool { + fn name(&self) -> &str { + tool_names::ORG_INBOX_REPAIR + } + + fn description(&self) -> &str { + "Inspect or explicitly resolve an undeliverable Agent Org Inbox row. Coordinator-only. The original row remains durable and unread for audit. Use cancel only when the delivery is intentionally abandoned; use supersede only after creating a valid replacement message with org_send_message or replacement task with task_create/task_update." + } + + fn category(&self) -> &str { + crate::tools::categories::ORCHESTRATION + } + + fn parameters(&self) -> Value { + params_schema::() + } + + async fn execute_text( + &self, + params_value: Value, + _ctx: &CallContext, + ) -> Result { + if !self.ctx.is_coordinator() { + return Err(ToolError::InvalidParams( + "org_inbox_repair is coordinator-only".to_string(), + )); + } + let params: OrgInboxRepairParams = parse_params(params_value)?; + let run_id = self.ctx.org_context.run_id.clone(); + + match params { + OrgInboxRepairParams::Inspect { inbox_id } => { + let inspect_run_id = run_id.clone(); + let (row, resolution) = tokio::task::spawn_blocking(move || { + let row = AgentInboxStore::get_by_id_for_run(&inspect_run_id, inbox_id)?; + let resolution = + AgentInboxStore::delivery_resolution_for_inbox(&inspect_run_id, inbox_id)?; + Ok::<_, String>((row, resolution)) + }) + .await + .map_err(|err| { + ToolError::ExecutionFailed(format!( + "org_inbox_repair inspect worker failed: {err}" + )) + })? + .map_err(ToolError::ExecutionFailed)?; + let row = row.ok_or_else(|| { + ToolError::InvalidParams(format!( + "Inbox row {inbox_id} does not belong to the current Agent Org run" + )) + })?; + serde_json::to_string(&json!({ + "outcome": "inspected", + "org_run_id": run_id, + "inbox_row": row, + "delivery_resolution": resolution, + "guidance": "If the original recipient can be restored, leave this row pending. Otherwise create a valid replacement through org_send_message or task tools, then call org_inbox_repair with action=supersede; use cancel only for an intentional discard." + })) + .map_err(|err| { + ToolError::ExecutionFailed(format!( + "org_inbox_repair inspect serialization failed: {err}" + )) + }) + } + OrgInboxRepairParams::Cancel { inbox_id, reason } => { + resolve( + &run_id, + inbox_id, + AgentInboxDeliveryResolutionKind::Cancelled, + reason, + None, + None, + ) + .await + } + OrgInboxRepairParams::Supersede { + inbox_id, + reason, + replacement_inbox_id, + replacement_task_id, + } => { + resolve( + &run_id, + inbox_id, + AgentInboxDeliveryResolutionKind::Superseded, + reason, + replacement_inbox_id, + replacement_task_id, + ) + .await + } + } + } +} + +async fn resolve( + run_id: &str, + inbox_id: i64, + resolution_kind: AgentInboxDeliveryResolutionKind, + reason: String, + replacement_inbox_id: Option, + replacement_task_id: Option, +) -> Result { + let params = ResolveInboxDeliveryParams { + inbox_id, + org_run_id: run_id.to_string(), + resolved_by_member_id: COORDINATOR_MEMBER_ID.to_string(), + resolution_kind, + reason, + replacement_inbox_id, + replacement_task_id, + }; + let resolution = tokio::task::spawn_blocking(move || AgentInboxStore::resolve_delivery(params)) + .await + .map_err(|err| { + ToolError::ExecutionFailed(format!("org_inbox_repair worker failed: {err}")) + })? + .map_err(|error| match error { + ResolveInboxDeliveryError::Constraint(message) => ToolError::InvalidParams(format!( + "Agent Org Inbox repair was not applied: {message}" + )), + ResolveInboxDeliveryError::Storage(message) => ToolError::ExecutionFailed(format!( + "Agent Org Inbox repair storage failed: {message}" + )), + })?; + serde_json::to_string(&json!({ + "outcome": resolution.resolution_kind.as_str(), + "org_run_id": run_id, + "delivery_resolution": resolution, + "guidance": "The original Inbox row remains durable and unread as audit evidence, but no longer blocks delivery/finality. Re-inspect task_list and the replacement work before requesting completion." + })) + .map_err(|err| { + ToolError::ExecutionFailed(format!( + "org_inbox_repair result serialization failed: {err}" + )) + }) +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::coordination::agent_inbox::AgentMessage; + use crate::coordination::agent_org_runs::{ + AgentOrgContextMember, AgentOrgRunContext, AgentOrgRunEntryMode, AgentOrgRunStatus, + AgentOrgRunStore, CreateAgentOrgRunParams, + }; + use crate::definitions::orgs::{HierarchyMode, OrgDefinition, OrgMember, PlanApprovalPolicy}; + use crate::session::persistence::{upsert_session, UnifiedSessionRecord}; + use crate::tools::impls::orchestration::org_send_message::NoopInboxWakeHook; + use crate::tools::traits::Tool; + use database::db::get_connection; + use rusqlite::params; + + struct Fixture { + _sandbox: test_helpers::test_env::SandboxGuard, + run_id: String, + inbox_id: i64, + coordinator: Arc, + worker: Arc, + } + + fn fixture() -> Fixture { + let sandbox = test_helpers::test_env::sandbox(); + let conn = get_connection().expect("test sqlite connection"); + crate::persistence::test_schema::ensure_agent_sessions_schema(&conn); + crate::session::persistence::init(&conn).expect("session schema"); + crate::coordination::init_agent_org_schemas(&conn).expect("Agent Org schemas"); + + let org = OrgDefinition { + id: format!("org-inbox-repair-{}", uuid::Uuid::new_v4()), + name: "Inbox Repair Org".into(), + role: "Coordinator".into(), + agent_id: "coordinator-agent".into(), + description: None, + hierarchy_mode: HierarchyMode::Soft, + plan_approval_policy: PlanApprovalPolicy::Coordinator, + children: vec![OrgMember { + id: "worker".into(), + name: "Worker".into(), + role: "Implementer".into(), + agent_id: "worker-agent".into(), + runtime_config: None, + children: Vec::new(), + }], + }; + let run = AgentOrgRunStore::create(CreateAgentOrgRunParams { + org_id: org.id.clone(), + coordinator_agent_id: org.agent_id.clone(), + root_session_id: Some("root-inbox-repair".into()), + org_snapshot: org, + entry_mode: AgentOrgRunEntryMode::StandaloneSession, + status: AgentOrgRunStatus::Running, + work_item_id: None, + project_slug: None, + routine_fire_id: None, + }) + .expect("create run"); + let now = chrono::Utc::now().to_rfc3339(); + upsert_session(&UnifiedSessionRecord { + session_id: "root-inbox-repair".into(), + name: "Coordinator".into(), + status: "idle".into(), + created_at: now.clone(), + updated_at: now, + session_type: "sde".into(), + org_member_id: Some(COORDINATOR_MEMBER_ID.into()), + agent_definition_id: Some("coordinator-agent".into()), + ..Default::default() + }) + .expect("seed coordinator session"); + + let message = AgentMessage::Plain { + summary: "Undeliverable work".into(), + text: "Preserve this original message".into(), + }; + conn.execute( + "INSERT INTO agent_inbox ( + recipient_agent_id, recipient_member_id, + sender_agent_id, sender_member_id, org_run_id, + payload_kind, payload_json, created_at + ) VALUES ( + 'removed-agent', NULL, + 'coordinator-agent', 'coordinator', ?1, + 'plain', ?2, ?3 + )", + params![ + &run.id, + serde_json::to_string(&message).unwrap(), + chrono::Utc::now().to_rfc3339(), + ], + ) + .expect("seed legacy undeliverable row"); + let inbox_id = conn.last_insert_rowid(); + + let org_context = Arc::new(AgentOrgRunContext { + run_id: run.id.clone(), + org_id: "org-inbox-repair".into(), + org_name: "Inbox Repair Org".into(), + org_role: "Coordinator".into(), + coordinator_agent_id: "coordinator-agent".into(), + coordinator_name: "Coordinator".into(), + coordinator_role: "Coordinator".into(), + members: vec![AgentOrgContextMember { + member_id: "worker".into(), + name: "Worker".into(), + role: "Implementer".into(), + agent_id: "worker-agent".into(), + parent_member_id: None, + }], + hierarchy_mode: HierarchyMode::Soft, + plan_approval_policy: PlanApprovalPolicy::Coordinator, + root_session_id: Some("root-inbox-repair".into()), + }); + let make_context = |member_id: &str, agent_id: &str| { + Arc::new(TaskToolsContext { + org_context: Arc::clone(&org_context), + caller_agent_id: agent_id.into(), + caller_member_id: member_id.into(), + wake_hook: Arc::new(NoopInboxWakeHook), + }) + }; + + Fixture { + _sandbox: sandbox, + run_id: run.id, + inbox_id, + coordinator: make_context(COORDINATOR_MEMBER_ID, "coordinator-agent"), + worker: make_context("worker", "worker-agent"), + } + } + + #[tokio::test] + async fn coordinator_can_cancel_an_undeliverable_row_without_faking_read() { + let fixture = fixture(); + let result = OrgInboxRepairTool::new(fixture.coordinator) + .execute_text( + json!({ + "action": "cancel", + "inbox_id": fixture.inbox_id, + "reason": "The removed member's work is intentionally abandoned" + }), + &CallContext::default(), + ) + .await + .expect("coordinator repair succeeds"); + assert_eq!( + serde_json::from_str::(&result).unwrap()["outcome"], + "cancelled" + ); + let row = AgentInboxStore::get_by_id_for_run(&fixture.run_id, fixture.inbox_id) + .unwrap() + .unwrap(); + assert!(row.read_at.is_none()); + assert!( + AgentInboxStore::delivery_resolution_for_inbox(&fixture.run_id, fixture.inbox_id) + .unwrap() + .is_some() + ); + } + + #[tokio::test] + async fn worker_cannot_resolve_inbox_delivery() { + let fixture = fixture(); + let error = OrgInboxRepairTool::new(fixture.worker) + .execute_text( + json!({ + "action": "cancel", + "inbox_id": fixture.inbox_id, + "reason": "Worker must not discard it" + }), + &CallContext::default(), + ) + .await + .expect_err("worker repair is denied"); + assert!(matches!(error, ToolError::InvalidParams(_))); + assert!( + AgentInboxStore::delivery_resolution_for_inbox(&fixture.run_id, fixture.inbox_id) + .unwrap() + .is_none() + ); + } + + #[tokio::test] + async fn terminal_run_rejects_inbox_delivery_mutation() { + let fixture = fixture(); + let conn = get_connection().expect("test sqlite connection"); + conn.execute( + "UPDATE agent_org_runs SET status='completed' WHERE id=?1", + params![&fixture.run_id], + ) + .expect("complete run"); + let error = OrgInboxRepairTool::new(fixture.coordinator) + .execute_text( + json!({ + "action": "cancel", + "inbox_id": fixture.inbox_id, + "reason": "Too late" + }), + &CallContext::default(), + ) + .await + .expect_err("terminal run mutation is denied"); + assert!(matches!(error, ToolError::InvalidParams(_))); + assert!( + AgentInboxStore::delivery_resolution_for_inbox(&fixture.run_id, fixture.inbox_id) + .unwrap() + .is_none() + ); + } +} diff --git a/src-tauri/crates/agent-core/src/core/tools/impls/orchestration/agent_org/mod.rs b/src-tauri/crates/agent-core/src/core/tools/impls/orchestration/agent_org/mod.rs index 2340bbec9..ba7ae09af 100644 --- a/src-tauri/crates/agent-core/src/core/tools/impls/orchestration/agent_org/mod.rs +++ b/src-tauri/crates/agent-core/src/core/tools/impls/orchestration/agent_org/mod.rs @@ -1,11 +1,11 @@ //! Agent Org orchestration tools — persistent inter-agent coordination. //! -//! This sub-module owns the two Agent Org tools that deal with **persistent, +//! This sub-module owns the Agent Org tools that deal with **persistent, //! multi-turn collaboration** between org members, distinct from the //! one-shot subagent spawning in [`super::agent`]: //! //! - [`send_message`] — `org_send_message`: typed inbox messages between org participants -//! - [`tasks`] — `task_graph_create` / `task_create` / `task_update` / `task_list` / `task_get` +//! - [`tasks`] — task-board, run-completion, and inbox-repair tools //! //! The split from the top-level `orchestration/` flat file layout clarifies //! the conceptual boundary: `agent/` spawns ephemeral workers; `agent_org/` @@ -16,6 +16,6 @@ pub mod tasks; pub use send_message::{NoopInboxWakeHook, NoopSelfAbortHook, OrgSendMessageTool}; pub use tasks::{ - OrgRunCompleteTool, TaskCreateTool, TaskGetTool, TaskGraphCreateTool, TaskListTool, - TaskToolsContext, TaskUpdateTool, + OrgInboxRepairTool, OrgRunCompleteTool, TaskCreateTool, TaskGetTool, TaskGraphCreateTool, + TaskListTool, TaskToolsContext, TaskUpdateTool, }; diff --git a/src-tauri/crates/agent-core/src/core/tools/impls/orchestration/agent_org/send_message.rs b/src-tauri/crates/agent-core/src/core/tools/impls/orchestration/agent_org/send_message.rs index 2b50d8dd6..43d1b49e1 100644 --- a/src-tauri/crates/agent-core/src/core/tools/impls/orchestration/agent_org/send_message.rs +++ b/src-tauri/crates/agent-core/src/core/tools/impls/orchestration/agent_org/send_message.rs @@ -15,7 +15,9 @@ use async_trait::async_trait; use serde_json::{json, Value}; use std::sync::Arc; -use crate::coordination::agent_inbox::{is_supported_agent_org_remote_mode, AgentMessage, RequestId}; +use crate::coordination::agent_inbox::{ + is_supported_agent_org_remote_mode, AgentMessage, RequestId, +}; use crate::coordination::agent_org_plan_approvals::{ AgentOrgPlanApprovalStore, AgentOrgPlanDecisionBy, AgentOrgPlanInboxDelivery, }; @@ -665,4 +667,3 @@ impl Tool for OrgSendMessageTool { false } } - diff --git a/src-tauri/crates/agent-core/src/core/tools/impls/orchestration/agent_org/send_message/persistence.rs b/src-tauri/crates/agent-core/src/core/tools/impls/orchestration/agent_org/send_message/persistence.rs index 0bf75c5c8..a2c7a2d8f 100644 --- a/src-tauri/crates/agent-core/src/core/tools/impls/orchestration/agent_org/send_message/persistence.rs +++ b/src-tauri/crates/agent-core/src/core/tools/impls/orchestration/agent_org/send_message/persistence.rs @@ -8,7 +8,9 @@ use rusqlite::{params, OptionalExtension}; use serde_json::json; use crate::coordination::agent_inbox::{AgentInboxStore, AgentMessage, InsertInboxParams}; -use crate::coordination::agent_org_runs::{AgentOrgParticipant, AgentOrgRunStore, COORDINATOR_MEMBER_ID}; +use crate::coordination::agent_org_runs::{ + AgentOrgParticipant, AgentOrgRunStore, COORDINATOR_MEMBER_ID, +}; use crate::coordination::agent_org_tasks::AgentOrgTaskStore; use crate::core::session::SessionStatus; use crate::tools::traits::ToolError; diff --git a/src-tauri/crates/agent-core/src/core/tools/impls/orchestration/agent_org/send_message/tests.rs b/src-tauri/crates/agent-core/src/core/tools/impls/orchestration/agent_org/send_message/tests.rs index 61748e7ac..9f6cb4d14 100644 --- a/src-tauri/crates/agent-core/src/core/tools/impls/orchestration/agent_org/send_message/tests.rs +++ b/src-tauri/crates/agent-core/src/core/tools/impls/orchestration/agent_org/send_message/tests.rs @@ -6,8 +6,7 @@ use super::*; use crate::coordination::agent_inbox::AgentInboxStore; use crate::coordination::agent_org_runs::{AgentOrgContextMember, COORDINATOR_MEMBER_ID}; use crate::coordination::agent_org_tasks::{ - new_task_id, AgentOrgTaskStore, CreateTaskParams, TaskStatus, - TASK_METADATA_ELIGIBLE_MEMBER_IDS, + new_task_id, AgentOrgTaskStore, CreateTaskParams, TaskStatus, TASK_METADATA_ELIGIBLE_MEMBER_IDS, }; use crate::definitions::orgs::HierarchyMode; use std::sync::Mutex; @@ -253,16 +252,13 @@ fn llm_description_recipient_hints_follow_strict_hierarchy_mode() { #[test] fn llm_description_restricts_kind_by_sender_role() { - let coordinator_tool = - OrgSendMessageTool::new(context(), COORDINATOR_MEMBER_ID.to_string()); + let coordinator_tool = OrgSendMessageTool::new(context(), COORDINATOR_MEMBER_ID.to_string()); let member_tool = OrgSendMessageTool::new(context(), "builder".to_string()); assert!(coordinator_tool .llm_description() .expect("description") - .contains( - "kind enum for this sender: [plain, shutdown_request, plan_approval_response]" - )); + .contains("kind enum for this sender: [plain, shutdown_request, plan_approval_response]")); assert!(member_tool .llm_description() .expect("description") diff --git a/src-tauri/crates/agent-core/src/core/tools/impls/orchestration/agent_org/task_create.rs b/src-tauri/crates/agent-core/src/core/tools/impls/orchestration/agent_org/task_create.rs index 3ec5b12f3..261cc9960 100644 --- a/src-tauri/crates/agent-core/src/core/tools/impls/orchestration/agent_org/task_create.rs +++ b/src-tauri/crates/agent-core/src/core/tools/impls/orchestration/agent_org/task_create.rs @@ -5,6 +5,9 @@ use schemars::JsonSchema; use serde::Deserialize; use serde_json::{json, Value}; +use crate::coordination::agent_org_payload_limits::{ + validate_task_identifier, validate_task_identifier_list, +}; use crate::coordination::agent_org_tasks::{ self, task_dependency_closure, AgentOrgTaskStore, CreateTaskParams, TaskCreateSchedulingPolicy, TaskExecutionMode, TaskStatus, TASK_GRAPH_OPEN_WORK_CONFLICT_ERROR, @@ -80,7 +83,7 @@ impl TaskDispatchPolicy { #[derive(Debug, Deserialize, JsonSchema)] #[serde(deny_unknown_fields)] pub struct TaskCreateParams { - /// Optional caller-supplied identifier. Defaults to a freshly + /// Optional caller-supplied bounded identifier. Defaults to a freshly /// minted v4 UUID. Use only when porting an external task or stamping a /// deterministic id in tests. #[serde(default)] @@ -348,6 +351,9 @@ impl Tool for TaskCreateTool { .clone() .filter(|s| !s.trim().is_empty()) .unwrap_or_else(agent_org_tasks::new_task_id); + validate_task_identifier("task_create.id", &id).map_err(ToolError::InvalidParams)?; + validate_task_identifier_list("task_create.dependency_task_ids", &blocked_by) + .map_err(ToolError::InvalidParams)?; if blocked_by.iter().any(|dependency_id| dependency_id == &id) { return Err(ToolError::InvalidParams(format!( "{}: task '{id}' cannot depend on itself", diff --git a/src-tauri/crates/agent-core/src/core/tools/impls/orchestration/agent_org/task_graph_create.rs b/src-tauri/crates/agent-core/src/core/tools/impls/orchestration/agent_org/task_graph_create.rs index 02101e62b..e8192f0dc 100644 --- a/src-tauri/crates/agent-core/src/core/tools/impls/orchestration/agent_org/task_graph_create.rs +++ b/src-tauri/crates/agent-core/src/core/tools/impls/orchestration/agent_org/task_graph_create.rs @@ -6,6 +6,9 @@ use schemars::JsonSchema; use serde::Deserialize; use serde_json::{json, Value}; +use crate::coordination::agent_org_payload_limits::{ + validate_task_identifier_list, TASK_GRAPH_CREATE_MAX_TASKS, +}; use crate::coordination::agent_org_tasks::{ self, task_dependency_closure, AgentOrgTaskStore, CreateTaskParams, TaskExecutionMode, TaskStatus, TASK_GRAPH_OPEN_WORK_CONFLICT_ERROR, @@ -48,7 +51,7 @@ pub struct TaskGraphNodeParams { #[derive(Debug, Deserialize, JsonSchema)] #[serde(deny_unknown_fields)] pub struct TaskGraphCreateParams { - /// Complete graph patch to create atomically. Use 1..=32 nodes. + /// Complete graph patch to create atomically. Use at most 32 nodes. pub tasks: Vec, /// Required only when this graph intentionally starts a new independent /// branch while older open tasks remain outside the graph. @@ -115,11 +118,19 @@ impl Tool for TaskGraphCreateTool { "Only the coordinator may create a cross-member task graph. Send the proposed graph to the coordinator.", ); } - if params.tasks.is_empty() || params.tasks.len() > 32 { - return Err(ToolError::InvalidParams( - "task_graph_create requires 1..=32 tasks".to_string(), - )); + if params.tasks.is_empty() || params.tasks.len() > TASK_GRAPH_CREATE_MAX_TASKS { + return Err(ToolError::InvalidParams(format!( + "task_graph_create requires 1..={TASK_GRAPH_CREATE_MAX_TASKS} tasks per request" + ))); + } + for (index, node) in params.tasks.iter().enumerate() { + validate_task_identifier_list( + &format!("task_graph_create.tasks[{index}].depends_on"), + &node.depends_on, + ) + .map_err(ToolError::InvalidParams)?; } + let read_run_id = self.ctx.org_context.run_id.clone(); let existing_tasks = tokio::task::spawn_blocking(move || AgentOrgTaskStore::list(&read_run_id)) diff --git a/src-tauri/crates/agent-core/src/core/tools/impls/orchestration/agent_org/task_list_get.rs b/src-tauri/crates/agent-core/src/core/tools/impls/orchestration/agent_org/task_list_get.rs index 397fc8e77..5c3bc6b4d 100644 --- a/src-tauri/crates/agent-core/src/core/tools/impls/orchestration/agent_org/task_list_get.rs +++ b/src-tauri/crates/agent-core/src/core/tools/impls/orchestration/agent_org/task_list_get.rs @@ -7,7 +7,10 @@ use schemars::JsonSchema; use serde::Deserialize; use serde_json::{json, Value}; -use crate::coordination::agent_org_runs::{AgentOrgFinalityDecision, AgentOrgRunStore}; +use crate::coordination::agent_org_payload_limits::validate_task_identifier; +use crate::coordination::agent_org_runs::{ + guaranteed_current_turn_effects_with_connection, AgentOrgFinalityDecision, AgentOrgRunStore, +}; use crate::coordination::agent_org_tasks::AgentOrgTaskStore; use crate::tools::names as tool_names; use crate::tools::traits::{params_schema, parse_params, CallContext, Tool, ToolError}; @@ -93,7 +96,7 @@ impl Tool for TaskListTool { async fn execute_text( &self, params_value: Value, - _call_ctx: &CallContext, + call_ctx: &CallContext, ) -> Result { let params: TaskListParams = parse_params(params_value)?; let normalized_status = params @@ -129,6 +132,11 @@ impl Tool for TaskListTool { .map(str::trim) .filter(|value| !value.is_empty()) .map(str::to_string); + if let Some(after_task_id) = after_task_id.as_deref() { + validate_task_identifier("task_list.after_task_id", after_task_id) + .map_err(ToolError::InvalidParams)?; + } + // Task summaries and run finality facts must describe the same // database moment. Use one deferred read transaction and project only // bounded columns; routine task_list calls never deserialize full @@ -136,46 +144,67 @@ impl Tool for TaskListTool { let run_id = self.ctx.org_context.run_id.clone(); let read_owner_filter = owner_filter.clone(); let read_after_task_id = after_task_id.clone(); - let (completion, page, open_task_ids_preview, open_task_ids_truncated) = - tokio::task::spawn_blocking(move || { - let mut conn = get_connection().map_err(|err| err.to_string())?; - let tx = conn - .transaction_with_behavior(TransactionBehavior::Deferred) - .map_err(|err| err.to_string())?; - let completion = - AgentOrgRunStore::finality_assessment_with_connection(&tx, &run_id)?; - let page = AgentOrgTaskStore::list_summary_page_with_connection( - &tx, - &run_id, - status_filter, - read_owner_filter.as_deref(), - read_after_task_id.as_deref(), - limit, - )?; - let (open_task_ids_preview, open_task_ids_truncated) = - AgentOrgTaskStore::open_task_ids_preview_with_connection(&tx, &run_id, 200)?; - tx.commit().map_err(|err| err.to_string())?; - Ok::<_, String>(( - completion, - page, - open_task_ids_preview, - open_task_ids_truncated, - )) - }) - .await - .map_err(|err| { - ToolError::ExecutionFailed(format!("task_list snapshot worker failed: {err}")) - })? - .map_err(|error| { - if error.starts_with("task_list after_task_id '") { - ToolError::InvalidParams(error) - } else { - ToolError::ExecutionFailed(error) - } - })?; + let dispatching_session_id = call_ctx.session_id.clone(); + let turn_intent_id = call_ctx.turn_intent_id.clone(); + let projected_inbox_ids = call_ctx.projected_inbox_ids.clone(); + let ( + completion, + guaranteed_turn_effects, + page, + open_task_ids_preview, + open_task_ids_truncated, + ) = tokio::task::spawn_blocking(move || { + let mut conn = get_connection().map_err(|err| err.to_string())?; + let tx = conn + .transaction_with_behavior(TransactionBehavior::Deferred) + .map_err(|err| err.to_string())?; + let completion = AgentOrgRunStore::finality_assessment_with_connection(&tx, &run_id)?; + let guaranteed_turn_effects = guaranteed_current_turn_effects_with_connection( + &tx, + &run_id, + completion.facts.root_session_id.as_deref(), + &dispatching_session_id, + &turn_intent_id, + &projected_inbox_ids, + )?; + let page = AgentOrgTaskStore::list_summary_page_with_connection( + &tx, + &run_id, + status_filter, + read_owner_filter.as_deref(), + read_after_task_id.as_deref(), + limit, + )?; + let (open_task_ids_preview, open_task_ids_truncated) = + AgentOrgTaskStore::open_task_ids_preview_with_connection(&tx, &run_id, 200)?; + tx.commit().map_err(|err| err.to_string())?; + Ok::<_, String>(( + completion, + guaranteed_turn_effects, + page, + open_task_ids_preview, + open_task_ids_truncated, + )) + }) + .await + .map_err(|err| { + ToolError::ExecutionFailed(format!("task_list snapshot worker failed: {err}")) + })? + .map_err(|error| { + if error.starts_with("task_list after_task_id '") { + ToolError::InvalidParams(error) + } else { + ToolError::ExecutionFailed(error) + } + })?; let active_member_ids = completion.facts.active_member_ids(); - let completion_ready = matches!(completion.decision, AgentOrgFinalityDecision::Complete); + let completion_after_turn = + completion.after_successful_coordinator_turn_with_effects(guaranteed_turn_effects); + let completion_ready = matches!( + completion_after_turn.decision, + AgentOrgFinalityDecision::Complete + ); let body = json!({ "tasks": page.tasks.iter().map(compact_task_summary_to_json).collect::>(), "total": page.tasks.len(), @@ -198,6 +227,7 @@ impl Tool for TaskListTool { "pending": completion.facts.pending_task_count, "in_progress": completion.facts.in_progress_task_count, "completed": completion.facts.completed_task_count, + "corrupt_task_count": completion.facts.corrupt_task_count, "open_task_ids": open_task_ids_preview, "open_task_ids_truncated": open_task_ids_truncated, "active_member_ids": active_member_ids, @@ -208,7 +238,7 @@ impl Tool for TaskListTool { "completion_ready": completion_ready, "finality_decision": completion.decision, "current_finality_blockers": &completion.blockers, - "completion_blockers": &completion.blockers, + "completion_blockers": &completion_after_turn.blockers, }, "org_run_id": self.ctx.org_context.run_id, }); @@ -272,6 +302,7 @@ impl Tool for TaskGetTool { "task_get requires a non-empty `id`".into(), )); } + validate_task_identifier("task_get.id", &task_id).map_err(ToolError::InvalidParams)?; let run_id = self.ctx.org_context.run_id.clone(); let read_task_id = task_id.clone(); let task = diff --git a/src-tauri/crates/agent-core/src/core/tools/impls/orchestration/agent_org/task_tests.rs b/src-tauri/crates/agent-core/src/core/tools/impls/orchestration/agent_org/task_tests.rs index 3ed4e562f..63b151077 100644 --- a/src-tauri/crates/agent-core/src/core/tools/impls/orchestration/agent_org/task_tests.rs +++ b/src-tauri/crates/agent-core/src/core/tools/impls/orchestration/agent_org/task_tests.rs @@ -2,7 +2,7 @@ use std::sync::{Arc, Mutex}; use serde_json::{json, Value}; -use crate::coordination::agent_inbox::{AgentInboxStore, AgentMessage}; +use crate::coordination::agent_inbox::{AgentInboxStore, AgentMessage, InsertInboxParams}; use crate::coordination::agent_org_runs::{ AgentOrgContextMember, AgentOrgRunContext, COORDINATOR_MEMBER_ID, }; @@ -246,6 +246,7 @@ fn task_tools_sandbox() -> test_env::SandboxGuard { session_id TEXT NOT NULL, turn_intent_id TEXT NOT NULL, client_message_id TEXT, + org_run_id TEXT, source TEXT NOT NULL, status TEXT NOT NULL, created_at TEXT NOT NULL, @@ -400,6 +401,32 @@ async fn task_create_rejects_missing_dispatch_policy() { ); } +#[tokio::test] +async fn task_create_rejects_ownerless_task_with_undeliverable_id() { + let _sandbox = task_tools_sandbox(); + let tool = TaskCreateTool::new(ctx(COORDINATOR_MEMBER_ID)); + let oversized_id = + "x".repeat(crate::coordination::agent_org_payload_limits::TASK_IDENTIFIER_MAX_CHARS + 1); + let error = tool + .execute_text( + json!({ + "id": oversized_id, + "subject": "Ownerless bounded task", + "eligible_member_ids": ["m-alice"], + }), + &test_ctx(), + ) + .await + .expect_err("task id must fit every TaskAssigned delivery path"); + + assert!( + matches!(error, ToolError::InvalidParams(message) if message.contains("task_create.id must be <= 1000 chars")) + ); + assert!(AgentOrgTaskStore::list("run-tools-1") + .expect("inspect rejected board") + .is_empty()); +} + #[test] fn task_create_schema_requires_unambiguous_dispatch_policy() { let tool = ProductionTaskCreateTool::new(ctx(COORDINATOR_MEMBER_ID)); @@ -2809,6 +2836,275 @@ async fn task_list_defaults_to_fifty_compact_rows() { assert!(value["page"]["next_cursor"].is_string()); } +fn seed_task_list_current_turn_finality_fixture(materialize_inbox: bool) -> i64 { + let now = chrono::Utc::now().to_rfc3339(); + let conn = database::db::get_connection().expect("test sqlite connection"); + conn.execute( + "UPDATE agent_org_runs + SET root_session_id='root-tools-1', status='running', updated_at=?2 + WHERE id=?1", + rusqlite::params!["run-tools-1", &now], + ) + .expect("reset running Agent Org run"); + upsert_session(&UnifiedSessionRecord { + session_id: "root-tools-1".to_string(), + name: "Coordinator".to_string(), + status: "running".to_string(), + session_type: "agent".to_string(), + created_at: now.clone(), + updated_at: now.clone(), + ..Default::default() + }) + .expect("seed running coordinator session"); + AgentOrgTaskStore::create(CreateTaskParams { + id: "current-turn-finished-task".to_string(), + org_run_id: "run-tools-1".to_string(), + subject: "Finished work presented to the coordinator".to_string(), + description: String::new(), + active_form: None, + owner: Some("m-alice".to_string()), + status: TaskStatus::Completed, + blocks: Vec::new(), + blocked_by: Vec::new(), + metadata: None, + }) + .expect("seed resolved task board"); + conn.execute( + "INSERT INTO session_turn_intents + (session_id, turn_intent_id, org_run_id, source, status, created_at, updated_at) + VALUES ('root-tools-1', 'current-coordinator-turn', 'run-tools-1', + 'agent_org', 'running', ?1, ?1)", + rusqlite::params![&now], + ) + .expect("seed current coordinator turn intent"); + let inbox = AgentInboxStore::insert(InsertInboxParams { + recipient_agent_id: "coord-1".to_string(), + recipient_member_id: Some(COORDINATOR_MEMBER_ID.to_string()), + sender_agent_id: "alice-1".to_string(), + sender_member_id: Some("m-alice".to_string()), + org_run_id: Some("run-tools-1".to_string()), + message: AgentMessage::Plain { + summary: "Worker result".to_string(), + text: "The finished work is in this coordinator turn.".to_string(), + }, + }) + .expect("seed unread coordinator inbox row"); + if materialize_inbox { + crate::session::persistence::materialize_agent_org_inbox_transcript( + "root-tools-1", + &[inbox.id], + "current-turn-inbox-transcript", + "current-turn-inbox-intent", + "worker result", + ) + .expect("materialize current coordinator inbox row"); + } + crate::coordination::agent_org_runs::AgentOrgRunStore::stage_coordinator_work_revision( + "run-tools-1", + ) + .expect("stage the current work revision"); + inbox.id +} + +fn has_finality_blocker(value: &Value, field: &str, kind: &str, count: i64) -> bool { + value["run_summary"][field] + .as_array() + .expect("finality blocker array") + .iter() + .any(|blocker| blocker["kind"] == kind && blocker["count"] == count) +} + +#[tokio::test] +async fn task_list_projects_exact_current_root_turn_and_materialized_inbox() { + let _sandbox = task_tools_sandbox(); + let inbox_id = seed_task_list_current_turn_finality_fixture(true); + let call_ctx = crate::tools::call_context::CallContext::for_turn( + "task-list-current-turn", + "root-tools-1", + "current-coordinator-turn", + vec![inbox_id], + ); + + let result = TaskListTool::new(ctx(COORDINATOR_MEMBER_ID)) + .execute_text(json!({}), &call_ctx) + .await + .expect("list tasks from the current coordinator turn"); + let value: Value = serde_json::from_str(&result).expect("decode task_list result"); + + assert_eq!(value["run_summary"]["open"], 0); + assert_eq!( + value["run_summary"]["pending_worker_turn_intent_count"], 1, + "the raw snapshot still includes the currently running coordinator intent" + ); + assert_eq!(value["run_summary"]["unread_inbox_count"], 1); + assert!(has_finality_blocker( + &value, + "current_finality_blockers", + "in_flight_turn_intents", + 1, + )); + assert!(has_finality_blocker( + &value, + "current_finality_blockers", + "unread_inbox", + 1, + )); + assert_eq!(value["run_summary"]["completion_ready"], true); + assert_eq!(value["run_summary"]["completion_blockers"], json!([])); +} + +#[tokio::test] +async fn task_list_current_turn_projection_keeps_unrelated_durable_blockers() { + let _sandbox = task_tools_sandbox(); + let projected_inbox_id = seed_task_list_current_turn_finality_fixture(true); + let now = chrono::Utc::now().to_rfc3339(); + let conn = database::db::get_connection().expect("test sqlite connection"); + conn.execute( + "INSERT INTO session_turn_intents + (session_id, turn_intent_id, org_run_id, source, status, created_at, updated_at) + VALUES ('reviewer-session', 'unrelated-queued-turn', 'run-tools-1', + 'agent_org', 'queued', ?1, ?1)", + rusqlite::params![&now], + ) + .expect("seed unrelated queued turn"); + let unrelated_inbox = AgentInboxStore::insert(InsertInboxParams { + recipient_agent_id: "coord-1".to_string(), + recipient_member_id: Some(COORDINATOR_MEMBER_ID.to_string()), + sender_agent_id: "alice-1".to_string(), + sender_member_id: Some("m-alice".to_string()), + org_run_id: Some("run-tools-1".to_string()), + message: AgentMessage::Plain { + summary: "Later worker result".to_string(), + text: "This row was not materialized into the current turn.".to_string(), + }, + }) + .expect("seed unrelated unread inbox row"); + let call_ctx = crate::tools::call_context::CallContext::for_turn( + "task-list-current-turn", + "root-tools-1", + "current-coordinator-turn", + vec![projected_inbox_id], + ); + + let result = TaskListTool::new(ctx(COORDINATOR_MEMBER_ID)) + .execute_text(json!({}), &call_ctx) + .await + .expect("list tasks without hiding unrelated blockers"); + let value: Value = serde_json::from_str(&result).expect("decode task_list result"); + + assert_ne!(unrelated_inbox.id, projected_inbox_id); + assert_eq!(value["run_summary"]["pending_worker_turn_intent_count"], 2); + assert_eq!(value["run_summary"]["unread_inbox_count"], 2); + assert_eq!(value["run_summary"]["completion_ready"], false); + assert!(has_finality_blocker( + &value, + "completion_blockers", + "in_flight_turn_intents", + 1, + )); + assert!(has_finality_blocker( + &value, + "completion_blockers", + "unread_inbox", + 1, + )); +} + +#[tokio::test] +async fn task_list_current_turn_projection_fails_closed_for_wrong_identity_or_receipt() { + let _sandbox = task_tools_sandbox(); + let inbox_id = seed_task_list_current_turn_finality_fixture(true); + let list = TaskListTool::new(ctx(COORDINATOR_MEMBER_ID)); + + let wrong_session_ctx = crate::tools::call_context::CallContext::for_turn( + "task-list-wrong-session", + "reviewer-session", + "current-coordinator-turn", + vec![inbox_id], + ); + let wrong_session: Value = serde_json::from_str( + &list + .execute_text(json!({}), &wrong_session_ctx) + .await + .expect("wrong-session call still returns diagnostics"), + ) + .expect("decode wrong-session result"); + assert_eq!(wrong_session["run_summary"]["completion_ready"], false); + assert!(has_finality_blocker( + &wrong_session, + "completion_blockers", + "in_flight_turn_intents", + 1, + )); + assert!(has_finality_blocker( + &wrong_session, + "completion_blockers", + "unread_inbox", + 1, + )); + + let wrong_intent_ctx = crate::tools::call_context::CallContext::for_turn( + "task-list-wrong-intent", + "root-tools-1", + "another-turn-intent", + vec![inbox_id], + ); + let wrong_intent: Value = serde_json::from_str( + &list + .execute_text(json!({}), &wrong_intent_ctx) + .await + .expect("wrong-intent call still returns diagnostics"), + ) + .expect("decode wrong-intent result"); + assert_eq!(wrong_intent["run_summary"]["completion_ready"], false); + assert!(has_finality_blocker( + &wrong_intent, + "completion_blockers", + "in_flight_turn_intents", + 1, + )); + assert!(has_finality_blocker( + &wrong_intent, + "completion_blockers", + "unread_inbox", + 1, + )); + + database::db::get_connection() + .expect("test sqlite connection") + .execute( + "DELETE FROM agent_inbox_materializations WHERE inbox_id=?1", + rusqlite::params![inbox_id], + ) + .expect("remove the receipt to exercise fail-closed validation"); + let missing_receipt_ctx = crate::tools::call_context::CallContext::for_turn( + "task-list-missing-receipt", + "root-tools-1", + "current-coordinator-turn", + vec![inbox_id], + ); + let missing_receipt: Value = serde_json::from_str( + &list + .execute_text(json!({}), &missing_receipt_ctx) + .await + .expect("missing-receipt call still returns diagnostics"), + ) + .expect("decode missing-receipt result"); + assert_eq!(missing_receipt["run_summary"]["completion_ready"], false); + assert!(has_finality_blocker( + &missing_receipt, + "completion_blockers", + "unread_inbox", + 1, + )); + assert!(!has_finality_blocker( + &missing_receipt, + "completion_blockers", + "in_flight_turn_intents", + 1, + )); +} + #[tokio::test] async fn task_list_completion_certificate_blocks_while_reviewer_is_running() { let _sandbox = task_tools_sandbox(); @@ -2825,7 +3121,7 @@ async fn task_list_completion_certificate_blocks_while_reviewer_is_running() { UnifiedSessionRecord { session_id: "root-tools-1".to_string(), name: "Coordinator".to_string(), - status: "idle".to_string(), + status: "running".to_string(), session_type: "agent".to_string(), created_at: now.clone(), updated_at: now.clone(), @@ -2859,7 +3155,24 @@ async fn task_list_completion_certificate_blocks_while_reviewer_is_running() { metadata: None, }) .unwrap(); - let call_ctx = test_ctx(); + crate::coordination::agent_org_runs::AgentOrgRunStore::stage_coordinator_work_revision( + "run-tools-1", + ) + .expect("stage current work revision"); + conn.execute( + "INSERT INTO session_turn_intents + (session_id, turn_intent_id, org_run_id, source, status, created_at, updated_at) + VALUES ('root-tools-1', 'current-review-turn', 'run-tools-1', + 'agent_org', 'running', ?1, ?1)", + rusqlite::params![chrono::Utc::now().to_rfc3339()], + ) + .expect("seed the current coordinator intent"); + let call_ctx = crate::tools::call_context::CallContext::for_turn( + "task-list-review-turn", + "root-tools-1", + "current-review-turn", + Vec::new(), + ); let list = TaskListTool::new(ctx(COORDINATOR_MEMBER_ID)); let result = list @@ -2878,6 +3191,102 @@ async fn task_list_completion_certificate_blocks_while_reviewer_is_running() { .unwrap() .iter() .any(|blocker| blocker["kind"] == "sessions_active")); + + conn.execute( + "UPDATE agent_sessions SET status='idle', updated_at=?2 WHERE session_id=?1", + rusqlite::params!["reviewer-session", chrono::Utc::now().to_rfc3339()], + ) + .unwrap(); + conn.execute( + "INSERT INTO session_turn_intents + (session_id, turn_intent_id, org_run_id, source, status, created_at, updated_at) + VALUES (?1, 'queued-review', 'run-tools-1', 'resume', 'queued', ?2, ?2)", + rusqlite::params!["reviewer-session", chrono::Utc::now().to_rfc3339()], + ) + .unwrap(); + let result = list.execute_text(json!({}), &call_ctx).await.unwrap(); + let value: Value = serde_json::from_str(&result).unwrap(); + assert_eq!(value["run_summary"]["completion_ready"], false); + assert_eq!(value["run_summary"]["pending_worker_turn_intent_count"], 2); + assert!(value["run_summary"]["completion_blockers"] + .as_array() + .unwrap() + .iter() + .any(|blocker| blocker["kind"] == "in_flight_turn_intents")); + + conn.execute( + "UPDATE session_turn_intents SET status='completed', updated_at=?2 + WHERE session_id=?1 AND turn_intent_id='queued-review'", + rusqlite::params!["reviewer-session", chrono::Utc::now().to_rfc3339()], + ) + .unwrap(); + let result = list.execute_text(json!({}), &call_ctx).await.unwrap(); + let value: Value = serde_json::from_str(&result).unwrap(); + assert_eq!(value["run_summary"]["completion_ready"], true); + assert_eq!(value["run_summary"]["completion_blockers"], json!([])); +} + +#[tokio::test] +async fn task_list_surfaces_corrupt_task_data_without_false_empty_completion() { + let _sandbox = task_tools_sandbox(); + let now = chrono::Utc::now().to_rfc3339(); + let conn = database::db::get_connection().unwrap(); + conn.execute( + "INSERT OR REPLACE INTO agent_org_runs + (id, org_id, coordinator_agent_id, root_session_id, entry_mode, status, created_at, updated_at) + VALUES ('run-tools-1', 'org-tools-1', 'coord-1', 'root-tools-1', 'standalone_session', 'running', ?1, ?1)", + rusqlite::params![&now], + ) + .unwrap(); + upsert_session(&UnifiedSessionRecord { + session_id: "root-tools-1".to_string(), + name: "Coordinator".to_string(), + status: "running".to_string(), + session_type: "agent".to_string(), + created_at: now.clone(), + updated_at: now, + ..Default::default() + }) + .unwrap(); + AgentOrgTaskStore::create(CreateTaskParams { + id: "corrupt-task".to_string(), + org_run_id: "run-tools-1".to_string(), + subject: "Corrupt persisted task".to_string(), + description: String::new(), + active_form: None, + owner: Some("m-alice".to_string()), + status: TaskStatus::Completed, + blocks: Vec::new(), + blocked_by: Vec::new(), + metadata: None, + }) + .unwrap(); + conn.execute( + "UPDATE agent_org_tasks SET blocks_json='not-json' WHERE id='corrupt-task'", + [], + ) + .unwrap(); + crate::coordination::agent_org_runs::AgentOrgRunStore::stage_coordinator_work_revision( + "run-tools-1", + ) + .unwrap(); + + let result = TaskListTool::new(ctx(COORDINATOR_MEMBER_ID)) + .execute_text(json!({}), &test_ctx()) + .await + .expect("corrupt board still returns canonical diagnostics"); + let value: Value = serde_json::from_str(&result).unwrap(); + assert_eq!(value["tasks"].as_array().unwrap().len(), 1); + assert_eq!(value["tasks"][0]["id"], "corrupt-task"); + assert_eq!(value["tasks"][0]["blocks"], json!([])); + assert_eq!(value["run_summary"]["total"], 1); + assert_eq!(value["run_summary"]["corrupt_task_count"], 1); + assert_eq!(value["run_summary"]["completion_ready"], false); + assert!(value["run_summary"]["completion_blockers"] + .as_array() + .unwrap() + .iter() + .any(|blocker| blocker["kind"] == "corrupt_task_data")); } #[tokio::test] diff --git a/src-tauri/crates/agent-core/src/core/tools/impls/orchestration/agent_org/task_update.rs b/src-tauri/crates/agent-core/src/core/tools/impls/orchestration/agent_org/task_update.rs index abace8b9d..208002b9b 100644 --- a/src-tauri/crates/agent-core/src/core/tools/impls/orchestration/agent_org/task_update.rs +++ b/src-tauri/crates/agent-core/src/core/tools/impls/orchestration/agent_org/task_update.rs @@ -5,6 +5,9 @@ use schemars::JsonSchema; use serde::Deserialize; use serde_json::{json, Map, Value}; +use crate::coordination::agent_org_payload_limits::{ + validate_task_identifier, validate_task_identifier_list, +}; use crate::coordination::agent_org_tasks::{ task_output, AgentOrgTaskStore, Task, TaskOutput, TaskStatus, UpdateTaskPatch, }; @@ -268,6 +271,7 @@ impl Tool for TaskUpdateTool { "task_update requires a non-empty `id`".into(), )); } + validate_task_identifier("task_update.id", &task_id).map_err(ToolError::InvalidParams)?; let org_run_id = self.ctx.org_context.run_id.clone(); let output_requested = params.output.is_some(); let output_params = params.output; @@ -315,6 +319,8 @@ impl Tool for TaskUpdateTool { patch.status = Some(parse_status(status).map_err(ToolError::InvalidParams)?); } if let Some(blocked_by) = params.blocked_by { + validate_task_identifier_list("task_update.blocked_by", &blocked_by) + .map_err(ToolError::InvalidParams)?; patch.blocked_by = Some(blocked_by); } let freeform_metadata_patch = params.metadata; diff --git a/src-tauri/crates/agent-core/src/core/tools/impls/orchestration/agent_org/tasks.rs b/src-tauri/crates/agent-core/src/core/tools/impls/orchestration/agent_org/tasks.rs index 725dff3fc..0b59505f9 100644 --- a/src-tauri/crates/agent-core/src/core/tools/impls/orchestration/agent_org/tasks.rs +++ b/src-tauri/crates/agent-core/src/core/tools/impls/orchestration/agent_org/tasks.rs @@ -34,13 +34,16 @@ use crate::coordination::agent_org_runs::{AgentOrgRunContext, COORDINATOR_MEMBER use crate::coordination::agent_org_tasks::{ self, eligible_member_ids as task_eligible_member_ids, Task, TaskExecutionMode, TaskMutationOutcome, TaskOutput, TaskStatus, TaskSummary, TASK_COMPLETED_IMMUTABLE_ERROR, - TASK_DELETE_HAS_DEPENDENTS_ERROR, TASK_DEPENDENCY_CYCLE_ERROR, - TASK_METADATA_ELIGIBLE_MEMBER_IDS, TASK_METADATA_EXECUTION_MODE, TASK_METADATA_OUTPUT, - TASK_METADATA_REQUIRED_ROLE, TASK_MUTATION_CONFLICT_ERROR, + TASK_DELETE_HAS_DEPENDENTS_ERROR, TASK_DELETE_IS_DELIVERY_REPLACEMENT_ERROR, + TASK_DEPENDENCY_CYCLE_ERROR, TASK_DEPENDENCY_LIMIT_ERROR, TASK_METADATA_ELIGIBLE_MEMBER_IDS, + TASK_METADATA_EXECUTION_MODE, TASK_METADATA_OUTPUT, TASK_METADATA_REQUIRED_ROLE, + TASK_MUTATION_CONFLICT_ERROR, TASK_RUN_TASK_LIMIT_ERROR, }; use crate::tools::impls::orchestration::org_send_message::InboxWakeHook; use crate::tools::traits::ToolError; +#[path = "inbox_repair.rs"] +pub mod inbox_repair; #[path = "run_complete.rs"] pub mod run_complete; #[path = "task_create.rs"] @@ -55,14 +58,16 @@ mod task_tests; #[path = "task_update.rs"] pub mod task_update; +pub use inbox_repair::{OrgInboxRepairParams, OrgInboxRepairTool}; pub use run_complete::{OrgRunCompleteParams, OrgRunCompleteTool}; pub use task_create::{TaskCreateParams, TaskCreateTool, TaskDispatchPolicy}; pub use task_graph_create::{TaskGraphCreateParams, TaskGraphCreateTool, TaskGraphNodeParams}; pub use task_list_get::{TaskGetParams, TaskGetTool, TaskListParams, TaskListTool}; pub use task_update::{TaskUpdateParams, TaskUpdateTool}; -/// Shared context for the four task tools. Cloned cheaply via `Arc` — -/// every tool stores its own clone so registry slots stay independent. +/// Shared context for Agent Org task, run-completion, and inbox-repair tools. +/// Cloned cheaply via `Arc` — every tool stores its own clone so registry +/// slots stay independent. pub struct TaskToolsContext { pub org_context: Arc, /// Backing agent definition id of the calling session. This is transport @@ -262,6 +267,10 @@ impl TaskToolsContext { &self, raw_member_ids: Vec, ) -> Result, String> { + crate::coordination::agent_org_payload_limits::validate_task_eligible_member_ids( + "eligible_member_ids", + &raw_member_ids, + )?; let mut resolved = Vec::new(); for raw_member_id in raw_member_ids { let member_id = raw_member_id.trim(); @@ -547,6 +556,9 @@ pub(crate) fn map_task_write_error(err: String) -> ToolError { || err.starts_with(TASK_COMPLETED_IMMUTABLE_ERROR) || err.starts_with(TASK_MUTATION_CONFLICT_ERROR) || err.starts_with(TASK_DELETE_HAS_DEPENDENTS_ERROR) + || err.starts_with(TASK_DELETE_IS_DELIVERY_REPLACEMENT_ERROR) + || err.starts_with(TASK_DEPENDENCY_LIMIT_ERROR) + || err.starts_with(TASK_RUN_TASK_LIMIT_ERROR) { ToolError::InvalidParams(err) } else { diff --git a/src-tauri/crates/agent-core/src/core/tools/policy.rs b/src-tauri/crates/agent-core/src/core/tools/policy.rs index 9023488b9..7de182745 100644 --- a/src-tauri/crates/agent-core/src/core/tools/policy.rs +++ b/src-tauri/crates/agent-core/src/core/tools/policy.rs @@ -68,6 +68,7 @@ pub const TOOL_GROUPS: &[(&str, &[&str])] = &[ tool_names::TASK_LIST, tool_names::TASK_GET, tool_names::ORG_RUN_COMPLETE, + tool_names::ORG_INBOX_REPAIR, ], ), ( diff --git a/src-tauri/crates/agent-core/src/core/tools/tests/ui_metadata_tests.rs b/src-tauri/crates/agent-core/src/core/tools/tests/ui_metadata_tests.rs index 8452101ac..08ec2c6c1 100644 --- a/src-tauri/crates/agent-core/src/core/tools/tests/ui_metadata_tests.rs +++ b/src-tauri/crates/agent-core/src/core/tools/tests/ui_metadata_tests.rs @@ -54,6 +54,7 @@ fn invokable_canonical_tool_names() -> BTreeSet<&'static str> { names::TASK_LIST, names::TASK_GET, names::ORG_RUN_COMPLETE, + names::ORG_INBOX_REPAIR, ]) } diff --git a/src-tauri/crates/agent-core/src/core/turn_executor/execute.rs b/src-tauri/crates/agent-core/src/core/turn_executor/execute.rs index 450913884..228622445 100644 --- a/src-tauri/crates/agent-core/src/core/turn_executor/execute.rs +++ b/src-tauri/crates/agent-core/src/core/turn_executor/execute.rs @@ -23,11 +23,11 @@ use super::stream_normalizer::{NormalizedStreamEvent, TurnStreamNormalizer}; use crate::model_context::microcompact; use super::backoff::{MAX_CONTEXT_RESCUE_ATTEMPTS, MAX_REPEAT_STREAK}; +use super::context_accounting::ContextUsageSnapshot; use super::continuation::{ - should_auto_continue, should_inject_todo_reminder, ANTI_RUSH_MIN_PERCENT, - ANTI_RUSH_MIN_WINDOW, MAX_AUTO_CONTINUATIONS, + should_auto_continue, should_inject_todo_reminder, ANTI_RUSH_MIN_PERCENT, ANTI_RUSH_MIN_WINDOW, + MAX_AUTO_CONTINUATIONS, }; -use super::context_accounting::ContextUsageSnapshot; use super::file_tracker; use super::helpers::{add_assistant_message, add_tool_result}; use super::length_recovery::{maybe_recover_from_length, LengthRecoveryOutcome}; @@ -748,6 +748,8 @@ pub async fn execute_turn( tools, policy, session_id, + &config.turn_intent_id, + &config.projected_inbox_ids, handler, permission_provider, cancel_flag, diff --git a/src-tauri/crates/agent-core/src/core/turn_executor/tool_execution/mod.rs b/src-tauri/crates/agent-core/src/core/turn_executor/tool_execution/mod.rs index 5a2251414..829a2e0d1 100644 --- a/src-tauri/crates/agent-core/src/core/turn_executor/tool_execution/mod.rs +++ b/src-tauri/crates/agent-core/src/core/turn_executor/tool_execution/mod.rs @@ -247,6 +247,8 @@ pub(crate) async fn execute_tool_calls( tools: &ToolRegistry, policy: &ResolvedToolPolicy, session_id: &str, + turn_intent_id: &str, + projected_inbox_ids: &[i64], handler: &dyn TurnEventHandler, permission_provider: Option<&dyn PermissionProvider>, cancel_flag: Option<&Arc>, @@ -269,6 +271,8 @@ pub(crate) async fn execute_tool_calls( tools, policy, session_id, + turn_intent_id, + projected_inbox_ids, handler, permission_provider, cancel_flag, @@ -298,6 +302,8 @@ pub(crate) async fn execute_tool_calls( tools, policy, session_id, + turn_intent_id, + projected_inbox_ids, handler, permission_provider, cancel_flag, @@ -325,6 +331,8 @@ pub(crate) async fn execute_tool_calls( tools, policy, session_id, + turn_intent_id, + projected_inbox_ids, handler, permission_provider, cancel_flag, diff --git a/src-tauri/crates/agent-core/src/core/turn_executor/tool_execution/parallel.rs b/src-tauri/crates/agent-core/src/core/turn_executor/tool_execution/parallel.rs index 871c161e6..31cfad42f 100644 --- a/src-tauri/crates/agent-core/src/core/turn_executor/tool_execution/parallel.rs +++ b/src-tauri/crates/agent-core/src/core/turn_executor/tool_execution/parallel.rs @@ -44,6 +44,8 @@ pub(super) async fn execute_parallel_group( tools: &ToolRegistry, policy: &ResolvedToolPolicy, session_id: &str, + turn_intent_id: &str, + projected_inbox_ids: &[i64], handler: &dyn TurnEventHandler, permission_provider: Option<&dyn PermissionProvider>, cancel_flag: Option<&Arc>, @@ -225,7 +227,12 @@ pub(super) async fn execute_parallel_group( .map(|(idx, effective_args, _display_name)| { let tool_name = &calls[*idx].name; let call_id = &calls[*idx].id; - let ctx = crate::tools::call_context::CallContext::new(call_id, session_id); + let ctx = crate::tools::call_context::CallContext::for_turn( + call_id, + session_id, + turn_intent_id, + projected_inbox_ids.to_vec(), + ); let args = effective_args.clone(); async move { let start = Instant::now(); diff --git a/src-tauri/crates/agent-core/src/core/turn_executor/tool_execution/single.rs b/src-tauri/crates/agent-core/src/core/turn_executor/tool_execution/single.rs index a8148c516..c309f716b 100644 --- a/src-tauri/crates/agent-core/src/core/turn_executor/tool_execution/single.rs +++ b/src-tauri/crates/agent-core/src/core/turn_executor/tool_execution/single.rs @@ -43,6 +43,8 @@ pub(super) async fn execute_single_tool( tools: &ToolRegistry, policy: &ResolvedToolPolicy, session_id: &str, + turn_intent_id: &str, + projected_inbox_ids: &[i64], handler: &dyn TurnEventHandler, permission_provider: Option<&dyn PermissionProvider>, cancel_flag: Option<&Arc>, @@ -238,7 +240,12 @@ pub(super) async fn execute_single_tool( ); let exec_start = Instant::now(); - let ctx = crate::tools::call_context::CallContext::new(&tool_call.id, session_id); + let ctx = crate::tools::call_context::CallContext::for_turn( + &tool_call.id, + session_id, + turn_intent_id, + projected_inbox_ids.to_vec(), + ); let raw_outcome = tools .execute_with_policy(&tool_call.name, effective_args.clone(), policy, &ctx) .await; @@ -474,6 +481,8 @@ mod tests { &tools, &policy, "session-test", + "", + &[], &handler, None, None, diff --git a/src-tauri/crates/agent-core/src/core/turn_executor/types.rs b/src-tauri/crates/agent-core/src/core/turn_executor/types.rs index eab9383c5..4830951e6 100644 --- a/src-tauri/crates/agent-core/src/core/turn_executor/types.rs +++ b/src-tauri/crates/agent-core/src/core/turn_executor/types.rs @@ -93,6 +93,11 @@ pub type SteeringQueue = Arc>>; /// Configuration for a single agent turn. #[derive(Clone)] pub struct TurnConfig { + /// Durable lifecycle identity for this exact scheduled turn. Empty when + /// the caller is not executing through the dialog scheduler. + pub turn_intent_id: String, + /// Agent Org Inbox rows that this turn will acknowledge on success. + pub projected_inbox_ids: Vec, /// Model identifier (provider-specific). pub model: String, /// KeyVault account id backing this turn. Threaded through so @@ -443,6 +448,8 @@ mod tests { #[test] fn turn_config_unlimited_iterations() { let config = TurnConfig { + turn_intent_id: String::new(), + projected_inbox_ids: Vec::new(), model: "test".to_string(), account_id: None, context_window_override: None, @@ -465,6 +472,8 @@ mod tests { #[test] fn turn_config_limited_iterations() { let config = TurnConfig { + turn_intent_id: String::new(), + projected_inbox_ids: Vec::new(), model: "test".to_string(), account_id: None, context_window_override: None, diff --git a/src-tauri/crates/agent-core/src/foundation/session_bridge.rs b/src-tauri/crates/agent-core/src/foundation/session_bridge.rs index ba17573bb..5dbfb5245 100644 --- a/src-tauri/crates/agent-core/src/foundation/session_bridge.rs +++ b/src-tauri/crates/agent-core/src/foundation/session_bridge.rs @@ -459,6 +459,7 @@ pub type UpsertTurnIntentFn = fn( session_id: &str, turn_intent_id: &str, client_message_id: Option<&str>, + org_run_id: Option<&str>, source: TurnIntentBridgeSource, status: TurnIntentBridgeStatus, ); @@ -490,6 +491,7 @@ pub fn upsert_turn_intent( session_id: &str, turn_intent_id: &str, client_message_id: Option<&str>, + org_run_id: Option<&str>, source: TurnIntentBridgeSource, status: TurnIntentBridgeStatus, ) { @@ -501,6 +503,7 @@ pub fn upsert_turn_intent( session_id, turn_intent_id, client_message_id, + org_run_id, source, status, ); diff --git a/src-tauri/crates/agent-core/src/init/tool_assembly.rs b/src-tauri/crates/agent-core/src/init/tool_assembly.rs index 0aaa424fa..c81da8f63 100644 --- a/src-tauri/crates/agent-core/src/init/tool_assembly.rs +++ b/src-tauri/crates/agent-core/src/init/tool_assembly.rs @@ -29,8 +29,8 @@ use crate::state::{AgentAppState, AgentSession}; use crate::tools::impls::meta::tool_search::ToolSearchTool; use crate::tools::impls::orchestration::agent::{AgentTool, AgentToolConfig}; use crate::tools::impls::orchestration::agent_tasks::{ - OrgRunCompleteTool, TaskCreateTool, TaskGetTool, TaskGraphCreateTool, TaskListTool, - TaskToolsContext, TaskUpdateTool, + OrgInboxRepairTool, OrgRunCompleteTool, TaskCreateTool, TaskGetTool, TaskGraphCreateTool, + TaskListTool, TaskToolsContext, TaskUpdateTool, }; use crate::tools::impls::orchestration::inbox_wake::AppHandleInboxWakeHook; use crate::tools::impls::orchestration::member_shutdown::AppHandleSelfAbortHook; @@ -245,6 +245,13 @@ pub(super) fn assemble_overlay( &task_tools_ctx, )))); } + if task_tools_ctx.is_coordinator() + && !ctx.disabled_set.contains(names::ORG_INBOX_REPAIR) + { + overlay.register(Box::new(OrgInboxRepairTool::new(Arc::clone( + &task_tools_ctx, + )))); + } } else { tracing::warn!( session_id = %ctx.session_id, diff --git a/src-tauri/crates/agent-core/src/lifecycle.rs b/src-tauri/crates/agent-core/src/lifecycle.rs index 30db9142c..fd80db128 100644 --- a/src-tauri/crates/agent-core/src/lifecycle.rs +++ b/src-tauri/crates/agent-core/src/lifecycle.rs @@ -827,6 +827,7 @@ mod tests { session_id TEXT NOT NULL, turn_intent_id TEXT NOT NULL, client_message_id TEXT, + org_run_id TEXT, source TEXT NOT NULL, status TEXT NOT NULL, created_at TEXT NOT NULL, diff --git a/src-tauri/crates/agent-core/src/specialization/memory/workspace_memory/auto_dream.rs b/src-tauri/crates/agent-core/src/specialization/memory/workspace_memory/auto_dream.rs index 355c38b51..606582693 100644 --- a/src-tauri/crates/agent-core/src/specialization/memory/workspace_memory/auto_dream.rs +++ b/src-tauri/crates/agent-core/src/specialization/memory/workspace_memory/auto_dream.rs @@ -181,6 +181,8 @@ pub async fn run_consolidation(params: super::super::MemoryAgentParams<'_>) -> R // Turn config let turn_config = TurnConfig { + turn_intent_id: String::new(), + projected_inbox_ids: Vec::new(), model: params.model.to_string(), account_id: None, context_window_override: None, diff --git a/src-tauri/crates/agent-core/src/specialization/memory/workspace_memory/extract/runner.rs b/src-tauri/crates/agent-core/src/specialization/memory/workspace_memory/extract/runner.rs index ed913eb90..b70f8b496 100644 --- a/src-tauri/crates/agent-core/src/specialization/memory/workspace_memory/extract/runner.rs +++ b/src-tauri/crates/agent-core/src/specialization/memory/workspace_memory/extract/runner.rs @@ -93,6 +93,8 @@ pub async fn run_extraction( })); let turn_config = TurnConfig { + turn_intent_id: String::new(), + projected_inbox_ids: Vec::new(), model: params.model.to_string(), account_id: None, context_window_override: None, diff --git a/src-tauri/crates/agent-core/src/state/commands/session/interaction.rs b/src-tauri/crates/agent-core/src/state/commands/session/interaction.rs index c923fd700..4c36a6db7 100644 --- a/src-tauri/crates/agent-core/src/state/commands/session/interaction.rs +++ b/src-tauri/crates/agent-core/src/state/commands/session/interaction.rs @@ -557,6 +557,7 @@ pub(crate) async fn plan_approval_response_impl( None, None, None, + None, crate::foundation::session_bridge::TurnIntentBridgeSource::UserSubmit, ) .await diff --git a/src-tauri/crates/agent-core/src/state/commands/session/message/entry_points.rs b/src-tauri/crates/agent-core/src/state/commands/session/message/entry_points.rs index 9d1e24247..27ab6d7e1 100644 --- a/src-tauri/crates/agent-core/src/state/commands/session/message/entry_points.rs +++ b/src-tauri/crates/agent-core/src/state/commands/session/message/entry_points.rs @@ -48,6 +48,7 @@ pub async fn send_message_impl_for_subagent_wake( None, None, None, + None, TurnIntentBridgeSource::Resume, ) .await @@ -77,6 +78,7 @@ pub async fn send_message_impl_for_org_wake( Some(format!("agent-org-wake:{org_run_id}:{member_id}")), None, Some(org_run_id.to_string()), + Some(org_run_id.to_string()), TurnIntentBridgeSource::Resume, ) .await @@ -118,6 +120,7 @@ pub async fn send_message_impl_for_test( None, None, None, + None, TurnIntentBridgeSource::UserSubmit, ) .await diff --git a/src-tauri/crates/agent-core/src/state/commands/session/message/org_wake.rs b/src-tauri/crates/agent-core/src/state/commands/session/message/org_wake.rs index 3d2cbef3b..505dd4486 100644 --- a/src-tauri/crates/agent-core/src/state/commands/session/message/org_wake.rs +++ b/src-tauri/crates/agent-core/src/state/commands/session/message/org_wake.rs @@ -20,11 +20,24 @@ pub(super) fn promote_agent_org_wake_session_to_running( let wakeable = SessionStatus::AGENT_ORG_WAKEABLE; let now = chrono::Utc::now().to_rfc3339(); conn.execute( - "WITH run_anchor(root_session_id) AS ( + "WITH RECURSIVE + run_anchor(root_session_id) AS ( SELECT root_session_id FROM agent_org_runs WHERE id=?4 AND status=?5 AND root_session_id IS NOT NULL ), + descendants(session_id) AS ( + SELECT root_session_id FROM run_anchor + UNION + SELECT child.session_id + FROM agent_sessions child + JOIN descendants parent ON child.parent_session_id=parent.session_id + WHERE NOT EXISTS ( + SELECT 1 FROM agent_org_runs nested + WHERE nested.id<>?4 + AND nested.root_session_id=child.session_id + ) + ), ranked(session_id, member_rank) AS ( SELECT session.session_id, ROW_NUMBER() OVER ( @@ -36,10 +49,10 @@ pub(super) fn promote_agent_org_wake_session_to_running( ORDER BY session.updated_at DESC, session.session_id DESC ) FROM agent_sessions session + JOIN descendants USING (session_id) CROSS JOIN run_anchor anchor WHERE session.session_id=anchor.root_session_id - OR (session.parent_session_id=anchor.root_session_id - AND session.agent_definition_id IS NOT NULL + OR (session.agent_definition_id IS NOT NULL AND session.org_member_id IS NOT NULL) ) UPDATE agent_sessions @@ -118,6 +131,10 @@ pub(super) fn resolve_agent_org_wake_mode( WHERE org_run_id=?1 AND recipient_member_id=?2 AND read_at IS NULL + AND NOT EXISTS ( + SELECT 1 FROM agent_inbox_delivery_resolutions resolution + WHERE resolution.inbox_id=agent_inbox.id + ) ORDER BY id ASC LIMIT ?3 ), delivery_window AS ( diff --git a/src-tauri/crates/agent-core/src/state/commands/session/message/send.rs b/src-tauri/crates/agent-core/src/state/commands/session/message/send.rs index a31d26000..093f010cf 100644 --- a/src-tauri/crates/agent-core/src/state/commands/session/message/send.rs +++ b/src-tauri/crates/agent-core/src/state/commands/session/message/send.rs @@ -54,6 +54,7 @@ pub(crate) async fn send_message_impl( client_message_id: Option, turn_intent_id: Option, org_wake_run_id: Option, + intent_org_run_id: Option, source: TurnIntentBridgeSource, ) -> Result { // Canonical user-intent id: callers that already mint one at the @@ -110,14 +111,27 @@ pub(crate) async fn send_message_impl( let runtime = crate::init::init_session(state, launch_spec).await?; - // Scheduler messages keep the current run id in memory so post-turn - // finality can reconcile the run that actually produced this turn. The - // durable turn-intent bridge remains session-scoped; nested-run intent - // ownership belongs to the later red-team hardening change. - let scheduled_org_run_id = runtime + // Turn intent ownership is independent from wake behavior. Explicit + // callers (initial Org launch, direct member message, wake) pass the run + // id before the runtime necessarily exists; ordinary messages recover it + // from the canonical runtime context. Never allow a retry to cross runs. + let runtime_org_run_id = runtime .agent_org_context .as_ref() .map(|context| context.run_id.clone()); + let effective_intent_org_run_id = match ( + intent_org_run_id.as_deref(), + runtime_org_run_id.as_deref(), + ) { + (Some(explicit), Some(runtime_id)) if explicit != runtime_id => { + return Err(format!( + "Agent Org turn intent run mismatch for session {session_id}: explicit run {explicit}, runtime run {runtime_id}" + )); + } + (Some(_), _) => intent_org_run_id, + (None, Some(_)) => runtime_org_run_id, + (None, None) => None, + }; // Wingman resume: reopen the bottom bar. On fresh start the frontend // sends `wingman_start` which opens the bar, but after app restart @@ -219,6 +233,7 @@ pub(crate) async fn send_message_impl( &session_id, &effective_turn_intent_id, client_message_id.as_deref(), + effective_intent_org_run_id.as_deref(), source, crate::foundation::session_bridge::TurnIntentBridgeStatus::Queued, ); @@ -513,7 +528,7 @@ pub(crate) async fn send_message_impl( generation: 0, client_message_id, turn_intent_id: effective_turn_intent_id.clone(), - org_run_id: scheduled_org_run_id, + org_run_id: effective_intent_org_run_id.clone(), content, execute, }; @@ -527,6 +542,7 @@ pub(crate) async fn send_message_impl( &session_id, &effective_turn_intent_id, msg.client_message_id.as_deref(), + effective_intent_org_run_id.as_deref(), source, crate::foundation::session_bridge::TurnIntentBridgeStatus::Queued, ); diff --git a/src-tauri/crates/agent-core/src/state/commands/session/mod.rs b/src-tauri/crates/agent-core/src/state/commands/session/mod.rs index ec89eb589..34cddffba 100644 --- a/src-tauri/crates/agent-core/src/state/commands/session/mod.rs +++ b/src-tauri/crates/agent-core/src/state/commands/session/mod.rs @@ -214,6 +214,7 @@ pub async fn agent_send_message( clientMessageId, turnIntentId, None, + None, source, ) .await diff --git a/src-tauri/crates/agent-core/src/state/commands/session/org_tasks/group_chat.rs b/src-tauri/crates/agent-core/src/state/commands/session/org_tasks/group_chat.rs index 657a9a2c7..7fb67a4ce 100644 --- a/src-tauri/crates/agent-core/src/state/commands/session/org_tasks/group_chat.rs +++ b/src-tauri/crates/agent-core/src/state/commands/session/org_tasks/group_chat.rs @@ -40,6 +40,8 @@ pub struct AgentOrgGroupChatHistoryRow { pub display_text: String, pub created_at: String, pub read_at: Option, + #[serde(skip_serializing_if = "Option::is_none")] + pub delivery_resolution: Option, } #[derive(Debug, Clone, Serialize)] @@ -103,7 +105,10 @@ pub(super) fn load_group_chat_history_page( let mut stmt = conn .prepare( "SELECT inbox.id, - inbox.recipient_member_id, + CASE WHEN inbox.recipient_member_id IS NULL THEN NULL + WHEN length(CAST(inbox.recipient_member_id AS BLOB))<=?7 + THEN substr(inbox.recipient_member_id, 1, ?8) + ELSE NULL END AS recipient_member_id, CASE WHEN length(CAST(inbox.payload_json AS BLOB))<=?4 AND json_valid(inbox.payload_json) @@ -117,14 +122,17 @@ pub(super) fn load_group_chat_history_page( THEN substr(inbox.display_text, 1, ?5) ELSE NULL END AS display_text, substr(inbox.created_at, 1, 64), - CASE WHEN inbox.read_at IS NULL THEN NULL ELSE substr(inbox.read_at, 1, 64) END + CASE WHEN inbox.read_at IS NULL THEN NULL ELSE substr(inbox.read_at, 1, 64) END, + resolution.resolution_kind FROM agent_inbox inbox + LEFT JOIN agent_inbox_delivery_resolutions resolution + ON resolution.inbox_id=inbox.id WHERE inbox.org_run_id=?1 AND inbox.sender_agent_id=?2 AND inbox.payload_kind='plain' AND (?3 IS NULL OR inbox.id>(3)?, row.get::<_, String>(4)?, row.get::<_, Option>(5)?, + row.get::<_, Option>(6)?, )) }, ) @@ -155,16 +167,39 @@ pub(super) fn load_group_chat_history_page( let mut serialized_bytes = 2usize; let mut has_more = false; for row in rows { - let (inbox_id, target_member_id, text, stored_display_text, created_at, read_at) = - row.map_err(|err| err.to_string())?; + let ( + inbox_id, + target_member_id, + text, + stored_display_text, + created_at, + read_at, + delivery_resolution, + ) = row.map_err(|err| err.to_string())?; if newest_first.len() == limit { has_more = true; break; } + let target_member_id = target_member_id.filter(|value| { + crate::coordination::agent_org_payload_limits::validate_message_identifier( + "group_chat_history.target_member_id", + value, + ) + .is_ok() + }); let target_member_name = target_member_id .as_deref() .and_then(|member_id| context.participant_display_name(member_id)) .or_else(|| target_member_id.clone()) + .filter(|value| { + crate::coordination::agent_org_payload_limits::validate_text_len( + "group_chat_history.target_member_name", + value, + crate::coordination::agent_org_payload_limits::MEMBER_DISPLAY_NAME_MAX_CHARS, + crate::coordination::agent_org_payload_limits::MEMBER_DISPLAY_NAME_MAX_BYTES, + ) + .is_ok() + }) .unwrap_or_else(|| "Unknown recipient".to_string()); let text = text .filter(|value| { @@ -196,6 +231,7 @@ pub(super) fn load_group_chat_history_page( display_text, created_at, read_at, + delivery_resolution, }; let row_bytes = serde_json::to_vec(&history_row) .map_err(|err| format!("serialize Group Chat history row failed: {err}"))? diff --git a/src-tauri/crates/agent-core/src/state/commands/session/org_tasks/intervention.rs b/src-tauri/crates/agent-core/src/state/commands/session/org_tasks/intervention.rs index c00486113..4335760be 100644 --- a/src-tauri/crates/agent-core/src/state/commands/session/org_tasks/intervention.rs +++ b/src-tauri/crates/agent-core/src/state/commands/session/org_tasks/intervention.rs @@ -200,6 +200,7 @@ pub async fn agent_org_send_user_message_to_member_impl( let view = agent_org_session_run_view_impl(state, &session_id) .await? .ok_or_else(|| format!("Session {session_id} is not part of an Agent Org run"))?; + let org_run_id = view.context.run_id.clone(); let member = view .members .into_iter() @@ -229,6 +230,7 @@ pub async fn agent_org_send_user_message_to_member_impl( None, None, None, + Some(org_run_id), crate::foundation::session_bridge::TurnIntentBridgeSource::AgentOrg, ) .await?; diff --git a/src-tauri/crates/agent-core/src/state/commands/session/org_tasks/lifecycle.rs b/src-tauri/crates/agent-core/src/state/commands/session/org_tasks/lifecycle.rs index e3957c68a..8c567f979 100644 --- a/src-tauri/crates/agent-core/src/state/commands/session/org_tasks/lifecycle.rs +++ b/src-tauri/crates/agent-core/src/state/commands/session/org_tasks/lifecycle.rs @@ -323,6 +323,10 @@ fn seed_coordinator_resume_inbox_in_tx( WHERE recipient_member_id=?1 AND org_run_id=?2 AND read_at IS NULL + AND NOT EXISTS ( + SELECT 1 FROM agent_inbox_delivery_resolutions resolution + WHERE resolution.inbox_id=agent_inbox.id + ) )", params![coordinator_member_id, &context.run_id], |row| row.get(0), diff --git a/src-tauri/crates/agent-core/src/state/commands/session/org_tasks/run_view.rs b/src-tauri/crates/agent-core/src/state/commands/session/org_tasks/run_view.rs index ab09ee44e..01d743df6 100644 --- a/src-tauri/crates/agent-core/src/state/commands/session/org_tasks/run_view.rs +++ b/src-tauri/crates/agent-core/src/state/commands/session/org_tasks/run_view.rs @@ -102,6 +102,8 @@ pub struct AgentOrgInboxPreviewRow { pub created_at: String, #[serde(skip_serializing_if = "Option::is_none")] pub read_at: Option, + #[serde(skip_serializing_if = "Option::is_none")] + pub delivery_resolution: Option, pub recipient_name: String, pub sender_name: String, pub display_text: String, @@ -131,6 +133,7 @@ pub struct AgentOrgRunTaskOverview { pub pending: usize, pub in_progress: usize, pub completed: usize, + pub corrupt: usize, pub visible: usize, pub truncated: bool, } @@ -218,6 +221,7 @@ fn build_agent_org_run_view( pending: finality.facts.pending_task_count, in_progress: finality.facts.in_progress_task_count, completed: finality.facts.completed_task_count, + corrupt: finality.facts.corrupt_task_count, visible: task_page.tasks.len(), truncated: task_page.has_more, }; @@ -339,7 +343,8 @@ pub(super) fn project_run_phase( AgentOrgRunStatus::Running => { let all_tasks_completed = task_overview.total > 0 && task_overview.pending == 0 - && task_overview.in_progress == 0; + && task_overview.in_progress == 0 + && task_overview.corrupt == 0; if all_tasks_completed { return AgentOrgRunPhase::Finalizing; } @@ -362,7 +367,10 @@ pub(super) fn project_run_phase( if unread_inbox_count > 0 { return AgentOrgRunPhase::Dispatching; } - if task_overview.pending > 0 || task_overview.in_progress > 0 { + if task_overview.pending > 0 + || task_overview.in_progress > 0 + || task_overview.corrupt > 0 + { AgentOrgRunPhase::Waiting } else { AgentOrgRunPhase::Coordinating @@ -573,6 +581,7 @@ fn enrich_inbox_preview_rows( request_id: row.request_id, created_at: row.created_at, read_at: row.read_at, + delivery_resolution: row.delivery_resolution, recipient_name, sender_name, display_text, @@ -653,6 +662,9 @@ fn member_view_from_parts( } = identity; let (inbox_activity_count, unread_inbox_count) = inbox_counts .iter() + // member_id is the only canonical Agent Org identity. A legacy row + // without it remains visible in the bounded Run Inbox, but is not + // copied onto every roster member that happens to share agent_id. .filter(|counts| counts.recipient_member_id.as_deref() == Some(member_id.as_str())) .fold((0usize, 0usize), |(activity, unread), counts| { ( diff --git a/src-tauri/crates/agent-core/src/state/commands/session/org_tasks/tests.rs b/src-tauri/crates/agent-core/src/state/commands/session/org_tasks/tests.rs index 7efd4e356..b5bf8d3a4 100644 --- a/src-tauri/crates/agent-core/src/state/commands/session/org_tasks/tests.rs +++ b/src-tauri/crates/agent-core/src/state/commands/session/org_tasks/tests.rs @@ -173,6 +173,7 @@ fn run_phase_projects_all_completed_running_board_as_finalizing() { pending: 0, in_progress: 0, completed: 1, + corrupt: 0, visible: 1, truncated: false, }; @@ -189,6 +190,7 @@ fn run_phase_projects_all_completed_running_board_as_finalizing() { pending: 0, in_progress: 0, completed: 0, + corrupt: 0, visible: 0, truncated: false, }, @@ -261,6 +263,7 @@ fn run_view_inbox_preview_omits_durable_payload_json() { request_id: None, created_at: "2026-05-28T00:00:00Z".to_string(), read_at: None, + delivery_resolution: None, recipient_name: "Alice".to_string(), sender_name: "User".to_string(), display_text: "hello".to_string(), @@ -287,6 +290,7 @@ fn run_phase_projects_quiet_user_plan_gate_as_awaiting_approval() { pending: 0, in_progress: 1, completed: 0, + corrupt: 0, visible: 1, truncated: false, }; diff --git a/src-tauri/crates/agent-core/src/state/commands/session/persistence.rs b/src-tauri/crates/agent-core/src/state/commands/session/persistence.rs index ad20ff16c..597cc8993 100644 --- a/src-tauri/crates/agent-core/src/state/commands/session/persistence.rs +++ b/src-tauri/crates/agent-core/src/state/commands/session/persistence.rs @@ -476,6 +476,7 @@ mod tests { session_id TEXT NOT NULL, turn_intent_id TEXT NOT NULL, client_message_id TEXT, + org_run_id TEXT, source TEXT NOT NULL, status TEXT NOT NULL, created_at TEXT NOT NULL, diff --git a/src-tauri/crates/agent-core/src/tests/turn_executor_retry_tests.rs b/src-tauri/crates/agent-core/src/tests/turn_executor_retry_tests.rs index b9491a6a6..02a3db79c 100644 --- a/src-tauri/crates/agent-core/src/tests/turn_executor_retry_tests.rs +++ b/src-tauri/crates/agent-core/src/tests/turn_executor_retry_tests.rs @@ -208,6 +208,8 @@ fn empty_policy() -> ResolvedToolPolicy { fn test_config() -> TurnConfig { TurnConfig { + turn_intent_id: String::new(), + projected_inbox_ids: Vec::new(), model: "mock-model".to_string(), account_id: None, context_window_override: None, diff --git a/src-tauri/crates/e2e-test/src/agent_org.rs b/src-tauri/crates/e2e-test/src/agent_org.rs index 7c479fcfc..393ba3213 100644 --- a/src-tauri/crates/e2e-test/src/agent_org.rs +++ b/src-tauri/crates/e2e-test/src/agent_org.rs @@ -12,6 +12,11 @@ //! Pairing strategy: positive AND negative pins per behavior. Every //! successful send is followed by a `list-by-run` read so a future //! refactor of either side surfaces here, not just in unit tests. +//! Production caller-path coverage lives in +//! `production_return_to_work_drains_inbox_into_member_transcript` below. It +//! launches a real materialized member and uses the debug-only deterministic +//! provider; full live-provider coordinator behavior belongs in rendered UI +//! E2E, not this deterministic runtime contract suite. use super::config::Config; use super::harness; @@ -23,6 +28,8 @@ const CHECK_MEMBER_SPAWN_GATE_PATH: &str = "/agent/test/agent-org/check-member-s const POST_MEMBER_IDLE_PATH: &str = "/agent/test/agent-org/post-member-idle"; const SEED_ORG_PATH: &str = "/agent/test/agent-org/seed"; const LAUNCH_COORDINATOR_PATH: &str = "/agent/test/agent-org/launch-coordinator"; +const SESSION_RETURN_TO_WORK_PATH: &str = "/agent/test/agent-org/session-return-to-work"; +const TASK_TOOL_DIRECT_PATH: &str = "/agent/test/agent-org/task-tool-direct"; const RUN_SEED_PATH: &str = "/agent/test/agent-org/run/seed"; const E2E_RUN_FIXTURE_ORG_PREFIX: &str = "e2e-agent-org-fixture:"; const RUN_VIEW_PATH: &str = "/agent/test/agent-org/run-view"; @@ -598,6 +605,332 @@ pub async fn launch_materializes_member_sessions_in_run_view(cfg: &Config) -> bo ) } +/// Production caller-path pin for Agent Org wake delivery. +/// +/// Debug helpers establish only the durable preconditions: a real org/run, +/// a materialized member session, and an assigned task. The action under test +/// is `session-return-to-work`, which invokes the same implementation as the +/// Tauri command. It must traverse the production idempotent scheduler, +/// initialize the member runtime, drain the inbox inside +/// `UnifiedMessageProcessor`, call the deterministic debug provider, persist +/// the visible inbox transcript, and mark the source row read. The scenario +/// never calls the helper-only `drain-inbox` endpoint. +pub async fn production_return_to_work_drains_inbox_into_member_transcript(cfg: &Config) -> bool { + let label = "Agent-Org: production return-to-work drains visible member input"; + let fixture_suffix = unique_run_id("production-wake"); + let org_id = format!("e2e-agent-org-fixture:{fixture_suffix}"); + let coordinator_agent_id = "builtin:sde"; + let worker_agent_id = "builtin:explore"; + let worker_member_id = "m-production-worker"; + let task_id = format!("task-{fixture_suffix}"); + let task_subject = "Production wake delivery marker"; + let fake_model = format!("e2e-fake-provider-agent-org-wake-{fixture_suffix}"); + + let seed_resp = match post_agent_org_json( + cfg, + SEED_ORG_PATH, + serde_json::json!({ + "id": org_id, + "name": "Production Wake E2E Org", + "coordinator_agent_id": coordinator_agent_id, + "members": [{ + "id": worker_member_id, + "name": "Wake Worker", + "role": "implementer", + "agent_id": worker_agent_id + }] + }), + ) + .await + { + Ok(response) => response, + Err(error) => return harness::print_error(label, &error), + }; + if seed_resp.get("ok").and_then(serde_json::Value::as_bool) != Some(true) { + return harness::print_error(label, &seed_resp.to_string()); + } + + let launch_resp = match post_agent_org_json( + cfg, + LAUNCH_COORDINATOR_PATH, + serde_json::json!({ + "agent_org_id": org_id, + "workspace_path": tmp_agent_org_workspace("production-wake"), + "content": "", + "model": fake_model, + "sync_turn": false, + "name": "Production Wake E2E" + }), + ) + .await + { + Ok(response) => response, + Err(error) => return harness::print_error(label, &error), + }; + let root_session_id = match launch_resp + .get("session_id") + .and_then(serde_json::Value::as_str) + { + Some(value) if !value.is_empty() => value.to_string(), + _ => return harness::print_error(label, &launch_resp.to_string()), + }; + let org_run_id = match launch_resp + .get("agent_org_run_id") + .and_then(serde_json::Value::as_str) + { + Some(value) if !value.is_empty() => value.to_string(), + _ => return harness::print_error(label, &launch_resp.to_string()), + }; + + // Member materialization is intentionally background work in production. + // Poll the production lookup rather than assuming it completed before the + // launch HTTP response returned. + let materialization_started = std::time::Instant::now(); + let worker_session_id = loop { + let lookup = match post_agent_org_json( + cfg, + FIND_WORKER_SESSION_PATH, + serde_json::json!({ + "org_run_id": org_run_id, + "member_id": worker_member_id + }), + ) + .await + { + Ok(response) => response, + Err(error) => return harness::print_error(label, &error), + }; + if let Some(session_id) = lookup + .get("session_id") + .and_then(serde_json::Value::as_str) + .filter(|value| !value.is_empty()) + { + break session_id.to_string(); + } + if materialization_started.elapsed() >= std::time::Duration::from_secs(15) { + return harness::print_error( + label, + &format!("member materialization timed out: {lookup}"), + ); + } + tokio::time::sleep(std::time::Duration::from_millis(100)).await; + }; + + // Use the production task tool to create the canonical task and its + // TaskAssigned outbox row. This endpoint installs a Noop wake hook so the + // row remains unread until the return-to-work action below. + let task_create_resp = match post_agent_org_json( + cfg, + TASK_TOOL_DIRECT_PATH, + serde_json::json!({ + "org_run_id": org_run_id, + "org_id": org_id, + "org_name": "Production Wake E2E Org", + "org_role": "coordinator", + "coordinator_agent_id": coordinator_agent_id, + "coordinator_name": "Coordinator", + "coordinator_role": "coordinator", + "members": [{ + "member_id": worker_member_id, + "name": "Wake Worker", + "role": "implementer", + "agent_id": worker_agent_id + }], + "sender_agent_id": coordinator_agent_id, + "sender_member_id": "coordinator", + "operation": "create", + "params": { + "id": task_id, + "subject": task_subject, + "description": "This exact task must become visible in the member transcript.", + "owner_member_id": worker_member_id, + "dispatch_policy": "immediate", + "execution_mode": "build" + } + }), + ) + .await + { + Ok(response) => response, + Err(error) => return harness::print_error(label, &error), + }; + if task_create_resp + .get("ok") + .and_then(serde_json::Value::as_bool) + != Some(true) + { + return harness::print_error(label, &task_create_resp.to_string()); + } + + let before_inbox = match list_inbox(cfg, &org_run_id).await { + Ok(response) => response, + Err(error) => return harness::print_error(label, &error), + }; + let source_row_before = messages_array(&before_inbox).ok().and_then(|messages| { + messages.iter().find(|row| { + row.get("payload_kind").and_then(serde_json::Value::as_str) == Some("task_assigned") + && row + .get("recipient_member_id") + .and_then(serde_json::Value::as_str) + == Some(worker_member_id) + && row + .get("payload_decoded") + .and_then(|payload| payload.get("task_id")) + .and_then(serde_json::Value::as_str) + == Some(task_id.as_str()) + }) + }); + let assignment_was_unread = source_row_before + .and_then(|row| row.get("read_at")) + .is_some_and(serde_json::Value::is_null); + + let wake_url = format!("{}{}", cfg.base_url, SESSION_RETURN_TO_WORK_PATH); + let wake_resp = match reqwest::Client::builder() + .timeout(std::time::Duration::from_secs(120)) + .build() + .expect("return-to-work client") + .post(wake_url) + .json(&serde_json::json!({ "session_id": worker_session_id })) + .send() + .await + { + Ok(response) => match response.json::().await { + Ok(json) => json, + Err(error) => return harness::print_error(label, &error.to_string()), + }, + Err(error) => return harness::print_error(label, &error.to_string()), + }; + + let after_inbox = match list_inbox(cfg, &org_run_id).await { + Ok(response) => response, + Err(error) => return harness::print_error(label, &error), + }; + let assignment_marked_read = messages_array(&after_inbox) + .ok() + .and_then(|messages| { + messages.iter().find(|row| { + row.get("payload_kind").and_then(serde_json::Value::as_str) == Some("task_assigned") + && row + .get("recipient_member_id") + .and_then(serde_json::Value::as_str) + == Some(worker_member_id) + && row + .get("payload_decoded") + .and_then(|payload| payload.get("task_id")) + .and_then(serde_json::Value::as_str) + == Some(task_id.as_str()) + }) + }) + .and_then(|row| row.get("read_at")) + .and_then(serde_json::Value::as_str) + .is_some_and(|value| !value.is_empty()); + + let transcript_url = format!( + "{}/agent/test/sde/transcript/{}", + cfg.base_url, + urlencoding::encode(&worker_session_id) + ); + let transcript = match http_client().get(transcript_url).send().await { + Ok(response) => match response.json::().await { + Ok(json) => json, + Err(error) => return harness::print_error(label, &error.to_string()), + }, + Err(error) => return harness::print_error(label, &error.to_string()), + }; + let transcript_messages = transcript + .get("messages") + .and_then(serde_json::Value::as_array); + let visible_assignment_inputs = transcript_messages + .map(|messages| { + messages + .iter() + .filter(|message| { + message.get("role").and_then(serde_json::Value::as_str) == Some("user") + && message.to_string().contains(task_subject) + && message.to_string().contains(&task_id) + }) + .count() + }) + .unwrap_or_default(); + let fake_provider_replied = transcript_messages.is_some_and(|messages| { + messages.iter().any(|message| { + message.get("role").and_then(serde_json::Value::as_str) == Some("assistant") + && message.to_string().contains("E2E_FAKE_PROVIDER_REPLY") + }) + }); + + // A second return-to-work with no new durable input must be a NoWork + // result, not a second empty provider turn or duplicate visible message. + let second_wake_resp = match reqwest::Client::builder() + .timeout(std::time::Duration::from_secs(30)) + .build() + .expect("second return-to-work client") + .post(format!("{}{}", cfg.base_url, SESSION_RETURN_TO_WORK_PATH)) + .json(&serde_json::json!({ "session_id": worker_session_id })) + .send() + .await + { + Ok(response) => response + .json::() + .await + .unwrap_or_default(), + Err(error) => return harness::print_error(label, &error.to_string()), + }; + let second_wake_was_noop = second_wake_resp + .get("ok") + .and_then(serde_json::Value::as_bool) + == Some(true) + && second_wake_resp + .get("woke") + .and_then(serde_json::Value::as_bool) + == Some(false); + + let details = serde_json::json!({ + "root_session_id": root_session_id, + "worker_session_id": worker_session_id, + "org_run_id": org_run_id, + "task_create": task_create_resp, + "before_inbox": before_inbox, + "wake": wake_resp, + "after_inbox": after_inbox, + "transcript": transcript, + "second_wake": second_wake_resp, + }); + + harness::print_result( + label, + &details.to_string(), + &[ + ("TaskAssigned precondition is unread", assignment_was_unread), + ( + "production return-to-work enqueued and completed a wake", + wake_resp.get("ok").and_then(serde_json::Value::as_bool) == Some(true) + && wake_resp.get("woke").and_then(serde_json::Value::as_bool) == Some(true), + ), + ( + "production drain acknowledged the source inbox row", + assignment_marked_read, + ), + ( + "member transcript contains exactly one visible assignment input", + visible_assignment_inputs == 1, + ), + ( + "deterministic provider completed the real member turn", + fake_provider_replied, + ), + ( + "return-to-work without new durable input is a no-op", + second_wake_was_noop, + ), + ], + ) +} + +/// Run-view task counts distinguish queued owned work from the currently active +/// task. This pins the read model needed for running members: assigning more +/// work to a member that already has an in-progress task must be visible as a +/// pending queue, not as a second active turn. pub async fn run_view_distinguishes_pending_and_in_progress_tasks(cfg: &Config) -> bool { let label = "Agent-Org: run view separates pending and in-progress tasks"; let org_id = unique_run_id("task-count-org"); diff --git a/src-tauri/crates/e2e-test/src/agent_org_tasks_and_exec_mode.rs b/src-tauri/crates/e2e-test/src/agent_org_tasks_and_exec_mode.rs index 5c237a996..ce9557e6e 100644 --- a/src-tauri/crates/e2e-test/src/agent_org_tasks_and_exec_mode.rs +++ b/src-tauri/crates/e2e-test/src/agent_org_tasks_and_exec_mode.rs @@ -7,6 +7,11 @@ //! `/test/agent-org/drain-inbox`, and assert on post-state via //! `/test/agent-org/tasks/list` and `/test/agent-org/inbox/list-by-run`. //! These pin the tool/store/helper contract only. +//! - caller-path: `agent_org.rs`'s production return-to-work scenario drives +//! `/test/agent-org/launch-coordinator` followed by the same +//! `agent_org_session_return_to_work_impl` used by the Tauri command. That +//! scenario proves a real materialized member scheduler/turn reaches the +//! production drain instead of calling this file's drain helper endpoint. //! - rendered UI: frontend rendered E2E must assert user-visible history and //! run-view badges/cards. Helper endpoints may seed or inspect, but must not //! be the side-effect path for rendered history assertions. diff --git a/src-tauri/crates/e2e-test/src/main.rs b/src-tauri/crates/e2e-test/src/main.rs index 17167368d..71bdf2e0a 100644 --- a/src-tauri/crates/e2e-test/src/main.rs +++ b/src-tauri/crates/e2e-test/src/main.rs @@ -668,6 +668,11 @@ fn all_scenarios() -> Vec { "agent-org-launch-materializes-member-sessions", agent_org::launch_materializes_member_sessions_in_run_view ), + scenario!( + "agent-org", + "agent-org-production-return-to-work-drains-visible-input", + agent_org::production_return_to_work_drains_inbox_into_member_transcript + ), scenario!( "agent-org", "agent-org-run-pause-resume-toggles-status", diff --git a/src-tauri/crates/session-persistence/src/agent_core_bridge.rs b/src-tauri/crates/session-persistence/src/agent_core_bridge.rs index 1e5347610..ab94e6b43 100644 --- a/src-tauri/crates/session-persistence/src/agent_core_bridge.rs +++ b/src-tauri/crates/session-persistence/src/agent_core_bridge.rs @@ -118,6 +118,7 @@ fn upsert_turn_intent_adapter( session_id: &str, turn_intent_id: &str, client_message_id: Option<&str>, + org_run_id: Option<&str>, source: session_bridge::TurnIntentBridgeSource, status: session_bridge::TurnIntentBridgeStatus, ) { @@ -125,6 +126,7 @@ fn upsert_turn_intent_adapter( session_id, turn_intent_id, client_message_id, + org_run_id, map_bridge_source(source), map_bridge_status(status), ) { diff --git a/src-tauri/crates/session-persistence/src/schema.rs b/src-tauri/crates/session-persistence/src/schema.rs index 73657797b..4fa5e22cc 100644 --- a/src-tauri/crates/session-persistence/src/schema.rs +++ b/src-tauri/crates/session-persistence/src/schema.rs @@ -165,6 +165,7 @@ pub fn init_session_tables(conn: &Connection) -> SqliteResult<()> { session_id TEXT NOT NULL, turn_intent_id TEXT NOT NULL, client_message_id TEXT, + org_run_id TEXT, source TEXT NOT NULL, status TEXT NOT NULL, created_at TEXT NOT NULL, @@ -173,11 +174,26 @@ pub fn init_session_tables(conn: &Connection) -> SqliteResult<()> { )", [], )?; + // Existing databases predate explicit Agent Org ownership. The column is + // nullable because ordinary session turns do not belong to an Org run. + // In-flight legacy rows are reconciled to terminal state on restart, so + // no unsafe session-tree backfill is attempted here. + conn.execute( + "ALTER TABLE session_turn_intents ADD COLUMN org_run_id TEXT", + [], + ) + .ok(); conn.execute( "CREATE INDEX IF NOT EXISTS idx_session_turn_intents_session_status ON session_turn_intents(session_id, status)", [], )?; + conn.execute( + "CREATE INDEX IF NOT EXISTS idx_session_turn_intents_org_run_status + ON session_turn_intents(org_run_id, status) + WHERE org_run_id IS NOT NULL", + [], + )?; drop_events_fts(conn); @@ -500,6 +516,18 @@ mod tests { .expect("query table existence") } + fn column_exists(conn: &Connection, table_name: &str, column_name: &str) -> bool { + let mut stmt = conn + .prepare(&format!("PRAGMA table_info({table_name})")) + .expect("prepare table info"); + let exists = stmt + .query_map([], |row| row.get::<_, String>(1)) + .expect("query table info") + .filter_map(Result::ok) + .any(|name| name == column_name); + exists + } + #[test] fn init_session_tables_creates_usage_telemetry_tables_and_indexes() { let conn = Connection::open_in_memory().expect("open in-memory sqlite"); @@ -512,6 +540,46 @@ mod tests { assert!(index_exists(&conn, "idx_stool_session_turn")); assert!(index_exists(&conn, "idx_stool_session_call")); assert!(index_exists(&conn, "idx_stool_session_iteration")); + assert!(column_exists(&conn, "session_turn_intents", "org_run_id")); + assert!(index_exists( + &conn, + "idx_session_turn_intents_org_run_status" + )); + } + + #[test] + fn init_session_tables_adds_org_run_id_to_existing_turn_intents() { + let conn = Connection::open_in_memory().expect("open in-memory sqlite"); + conn.execute_batch( + "CREATE TABLE session_turn_intents ( + session_id TEXT NOT NULL, + turn_intent_id TEXT NOT NULL, + client_message_id TEXT, + source TEXT NOT NULL, + status TEXT NOT NULL, + created_at TEXT NOT NULL, + updated_at TEXT NOT NULL, + PRIMARY KEY (session_id, turn_intent_id) + ); + INSERT INTO session_turn_intents ( + session_id, turn_intent_id, source, status, created_at, updated_at + ) VALUES ('legacy-session', 'legacy-intent', 'agent_org', 'queued', + '2026-01-01T00:00:00Z', '2026-01-01T00:00:00Z');", + ) + .expect("seed legacy turn-intent schema"); + + init_session_tables(&conn).expect("upgrade session schema"); + + assert!(column_exists(&conn, "session_turn_intents", "org_run_id")); + let legacy_owner: Option = conn + .query_row( + "SELECT org_run_id FROM session_turn_intents + WHERE session_id='legacy-session' AND turn_intent_id='legacy-intent'", + [], + |row| row.get(0), + ) + .expect("legacy row remains readable"); + assert_eq!(legacy_owner, None, "legacy ownership must not be guessed"); } #[test] diff --git a/src-tauri/crates/session-persistence/src/turn_intents.rs b/src-tauri/crates/session-persistence/src/turn_intents.rs index c5eeb284b..9109cee4b 100644 --- a/src-tauri/crates/session-persistence/src/turn_intents.rs +++ b/src-tauri/crates/session-persistence/src/turn_intents.rs @@ -177,6 +177,7 @@ pub struct TurnIntentRow { pub session_id: String, pub turn_intent_id: String, pub client_message_id: Option, + pub org_run_id: Option, pub source: TurnIntentSource, pub status: TurnIntentStatus, pub created_at: String, @@ -196,6 +197,15 @@ pub enum IntentError { NotFound(String, String), #[error("invalid stored status {0:?} for turn intent {1}")] InvalidStoredStatus(String, String), + #[error( + "turn intent {turn_intent_id} for session {session_id} already belongs to Agent Org run {existing_org_run_id}, not {requested_org_run_id}" + )] + OrgRunMismatch { + session_id: String, + turn_intent_id: String, + existing_org_run_id: String, + requested_org_run_id: String, + }, } /// Whitelist of state transitions. Anything outside this list is rejected. @@ -231,18 +241,18 @@ fn transition_allowed(from: TurnIntentStatus, to: TurnIntentStatus) -> bool { // ============================================ fn row_from_sql(row: &rusqlite::Row<'_>) -> SqliteResult { - let source_str: String = row.get(3)?; - let status_str: String = row.get(4)?; + let source_str: String = row.get(4)?; + let status_str: String = row.get(5)?; let source = TurnIntentSource::parse(&source_str).ok_or_else(|| { rusqlite::Error::FromSqlConversionFailure( - 3, + 4, rusqlite::types::Type::Text, format!("unknown turn_intents.source value: {source_str}").into(), ) })?; let status = TurnIntentStatus::parse(&status_str).ok_or_else(|| { rusqlite::Error::FromSqlConversionFailure( - 4, + 5, rusqlite::types::Type::Text, format!("unknown turn_intents.status value: {status_str}").into(), ) @@ -251,10 +261,11 @@ fn row_from_sql(row: &rusqlite::Row<'_>) -> SqliteResult { session_id: row.get(0)?, turn_intent_id: row.get(1)?, client_message_id: row.get(2)?, + org_run_id: row.get(3)?, source, status, - created_at: row.get(5)?, - updated_at: row.get(6)?, + created_at: row.get(6)?, + updated_at: row.get(7)?, }) } @@ -268,6 +279,7 @@ pub fn upsert_initial( session_id: &str, turn_intent_id: &str, client_message_id: Option<&str>, + org_run_id: Option<&str>, source: TurnIntentSource, status: TurnIntentStatus, ) -> Result { @@ -275,13 +287,14 @@ pub fn upsert_initial( let now = Utc::now().to_rfc3339(); let inserted = conn.execute( "INSERT OR IGNORE INTO session_turn_intents - (session_id, turn_intent_id, client_message_id, source, status, - created_at, updated_at) - VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?6)", + (session_id, turn_intent_id, client_message_id, org_run_id, + source, status, created_at, updated_at) + VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?7)", params![ session_id, turn_intent_id, client_message_id, + org_run_id, source.as_str(), status.as_str(), now, @@ -292,14 +305,38 @@ pub fn upsert_initial( session_id: session_id.to_string(), turn_intent_id: turn_intent_id.to_string(), client_message_id: client_message_id.map(str::to_string), + org_run_id: org_run_id.map(str::to_string), source, status, created_at: now.clone(), updated_at: now, }); } - get_intent(&conn, session_id, turn_intent_id)? - .ok_or_else(|| IntentError::NotFound(turn_intent_id.to_string(), session_id.to_string())) + // A row may have been created by an earlier bridge stage or by an older + // app version before explicit run ownership existed. Backfill only NULL; + // never overwrite a different durable run id. + if let Some(org_run_id) = org_run_id { + conn.execute( + "UPDATE session_turn_intents SET org_run_id=?3 + WHERE session_id=?1 AND turn_intent_id=?2 AND org_run_id IS NULL", + params![session_id, turn_intent_id, org_run_id], + )?; + } + let existing = get_intent(&conn, session_id, turn_intent_id)? + .ok_or_else(|| IntentError::NotFound(turn_intent_id.to_string(), session_id.to_string()))?; + if let (Some(existing_org_run_id), Some(requested_org_run_id)) = + (existing.org_run_id.as_deref(), org_run_id) + { + if existing_org_run_id != requested_org_run_id { + return Err(IntentError::OrgRunMismatch { + session_id: session_id.to_string(), + turn_intent_id: turn_intent_id.to_string(), + existing_org_run_id: existing_org_run_id.to_string(), + requested_org_run_id: requested_org_run_id.to_string(), + }); + } + } + Ok(existing) } /// Patch the status of an existing intent. Transition must be in the @@ -380,8 +417,8 @@ pub fn get_intent( turn_intent_id: &str, ) -> SqliteResult> { let mut stmt = conn.prepare_cached( - "SELECT session_id, turn_intent_id, client_message_id, source, status, - created_at, updated_at + "SELECT session_id, turn_intent_id, client_message_id, org_run_id, + source, status, created_at, updated_at FROM session_turn_intents WHERE session_id = ?1 AND turn_intent_id = ?2", )?; @@ -394,8 +431,8 @@ pub fn get_intent( pub fn list_for_session(session_id: &str) -> SqliteResult> { let conn = get_connection()?; let mut stmt = conn.prepare_cached( - "SELECT session_id, turn_intent_id, client_message_id, source, status, - created_at, updated_at + "SELECT session_id, turn_intent_id, client_message_id, org_run_id, + source, status, created_at, updated_at FROM session_turn_intents WHERE session_id = ?1 ORDER BY created_at ASC, turn_intent_id ASC", @@ -444,6 +481,7 @@ mod tests { session, intent, Some("client-1"), + None, TurnIntentSource::UserSubmit, TurnIntentStatus::Queued, ) @@ -488,6 +526,7 @@ mod tests { session, intent, Some("client-2"), + None, TurnIntentSource::Queue, TurnIntentStatus::Running, ) @@ -498,6 +537,59 @@ mod tests { }); } + #[test] + fn retry_can_backfill_missing_org_run_but_cannot_reassign_it() { + with_temp_orgii_home(|| { + let session = "test-session-org-run-ownership"; + let intent = "intent-org-run-ownership"; + let original = upsert_initial( + session, + intent, + None, + None, + TurnIntentSource::AgentOrg, + TurnIntentStatus::Queued, + ) + .expect("legacy-style upsert succeeds"); + assert_eq!(original.org_run_id, None); + + let backfilled = upsert_initial( + session, + intent, + None, + Some("run-a"), + TurnIntentSource::AgentOrg, + TurnIntentStatus::Queued, + ) + .expect("retry may fill a previously unknown run owner"); + assert_eq!(backfilled.org_run_id.as_deref(), Some("run-a")); + + let error = upsert_initial( + session, + intent, + None, + Some("run-b"), + TurnIntentSource::AgentOrg, + TurnIntentStatus::Queued, + ) + .expect_err("a durable intent must never move between runs"); + assert!(matches!( + error, + IntentError::OrgRunMismatch { + existing_org_run_id, + requested_org_run_id, + .. + } if existing_org_run_id == "run-a" && requested_org_run_id == "run-b" + )); + + let conn = get_connection().expect("open sessions DB"); + let stored = get_intent(&conn, session, intent) + .expect("read intent") + .expect("intent exists"); + assert_eq!(stored.org_run_id.as_deref(), Some("run-a")); + }); + } + #[test] fn legal_transitions_walk_to_terminal() { with_temp_orgii_home(|| { @@ -599,6 +691,7 @@ mod tests { session, "pending-a", None, + None, TurnIntentSource::UserSubmit, TurnIntentStatus::Queued, ) @@ -607,6 +700,7 @@ mod tests { session, "pending-b", None, + None, TurnIntentSource::Queue, TurnIntentStatus::Optimistic, ) @@ -615,6 +709,7 @@ mod tests { session, "running-c", None, + None, TurnIntentSource::UserSubmit, TurnIntentStatus::Queued, ) @@ -649,6 +744,7 @@ mod tests { session, intent, None, + None, TurnIntentSource::AgentOrg, TurnIntentStatus::Queued, ) diff --git a/src-tauri/crates/types/src/tool_names.rs b/src-tauri/crates/types/src/tool_names.rs index 05da4e2f1..c2a806f21 100644 --- a/src-tauri/crates/types/src/tool_names.rs +++ b/src-tauri/crates/types/src/tool_names.rs @@ -126,6 +126,8 @@ pub const TASK_GET: &str = "task_get"; /// Record a coordinator completion request at the current durable work /// revision. Finality checks remain authoritative. pub const ORG_RUN_COMPLETE: &str = "org_run_complete"; +/// Inspect or explicitly resolve an undeliverable Agent Org Inbox row. +pub const ORG_INBOX_REPAIR: &str = "org_inbox_repair"; // ── Channel workspace tools ───────────────────────────────────────── /// List known workspace paths seen in recent sessions. diff --git a/src-tauri/crates/types/src/tool_names_tests.rs b/src-tauri/crates/types/src/tool_names_tests.rs index 752a37480..f5e380ac8 100644 --- a/src-tauri/crates/types/src/tool_names_tests.rs +++ b/src-tauri/crates/types/src/tool_names_tests.rs @@ -70,6 +70,7 @@ fn tool_name_constants_are_stable_wire_strings() { assert_eq!(TASK_LIST, "task_list"); assert_eq!(TASK_GET, "task_get"); assert_eq!(ORG_RUN_COMPLETE, "org_run_complete"); + assert_eq!(ORG_INBOX_REPAIR, "org_inbox_repair"); // ── Channel workspace tools ── assert_eq!(LIST_KNOWN_WORKSPACES, "list_known_workspaces"); diff --git a/src-tauri/src/api/agent/mod.rs b/src-tauri/src/api/agent/mod.rs index 4f200c0f2..0a4d82928 100644 --- a/src-tauri/src/api/agent/mod.rs +++ b/src-tauri/src/api/agent/mod.rs @@ -694,6 +694,10 @@ pub fn create_routes() -> Router { "/test/agent-org/drain-inbox", post(test::agent_org::test_agent_org_drain_inbox), ) + .route( + "/test/agent-org/session-return-to-work", + post(test::agent_org::test_agent_org_session_return_to_work), + ) .route( "/test/agent-org/post-member-idle", post(test::agent_org::test_agent_org_post_member_idle), diff --git a/src-tauri/src/api/agent/test/agent_org.rs b/src-tauri/src/api/agent/test/agent_org.rs index 048106bc0..14de10dc5 100644 --- a/src-tauri/src/api/agent/test/agent_org.rs +++ b/src-tauri/src/api/agent/test/agent_org.rs @@ -3,9 +3,14 @@ //! Inter-agent E2E observability probes for Agent Org runs. Most //! endpoints in this module are **helper-isolation / symbol-pinning** //! probes (driving an `AgentInboxStore` / `AgentOrgRunContext` helper -//! directly, no live session, no LLM). The caller-path exception is -//! `launch-coordinator`, which drives the canonical `session_launch_impl`. -//! Each endpoint's individual doc states which kind it is. +//! directly, no live session, no LLM). The caller-path exceptions are +//! `launch-coordinator`, which drives the canonical `session_launch_impl`, +//! and `session-return-to-work`, which drives the production wake scheduler +//! and inbox drain on a materialized member session. Each endpoint's +//! individual doc states which kind it is. Helper-isolation probes catch +//! contract drift cheaply; the deterministic fake-provider return-to-work +//! scenario catches regressions where the real turn processor stops draining +//! or persisting inbox input. //! //! Currently exposed: //! @@ -33,6 +38,11 @@ //! set. Init parity is automatic because we drive the same path the //! production frontend uses; we never re-implement runtime assembly //! here. +//! - `POST /test/agent-org/session-return-to-work` — call the same +//! `agent_org_session_return_to_work_impl` as the Tauri command. This is a +//! narrow HTTP bridge, not a replacement drain helper: the production +//! scheduler, turn processor, inbox persistence, and provider path all run. +//! //! `payload_kind` and `payload_decoded` are returned alongside the raw //! row so a corrupted serde tag (anti-pattern caught by //! `kind_tag_matches_serde_tag` in unit tests) shows up here too — @@ -869,6 +879,11 @@ pub async fn test_agent_org_inbox_seed( /// caller (`UnifiedMessageProcessor::process`) actually invokes /// `drain_and_render_deferred` with the correct `org_context` and /// `recipient_agent_id` at the start of every turn. The full +/// caller-path is exercised by +/// `agent_org::production_return_to_work_drains_inbox_into_member_transcript`, +/// which launches a real member session and uses the deterministic debug +/// provider while leaving the scheduler and turn processor unchanged. +/// /// Response shape: `{ ok: true, drained_count: usize, rendered: usize, messages: Value[] }`. pub async fn test_agent_org_drain_inbox( Json(body): Json, @@ -1044,6 +1059,60 @@ pub async fn test_agent_org_drain_inbox( })) } +/// `POST /test/agent-org/session-return-to-work` +/// +/// Debug HTTP bridge to the production return-to-work command implementation. +/// It deliberately does not drain or mark any inbox row itself. The invoked +/// implementation must resolve the persisted member session, enqueue the +/// idempotent Agent Org wake, run the real scheduler/turn processor/provider, +/// and observe the production drain's durable acknowledgement. +/// +/// Body: `{ "session_id": "agent-..." }`. +/// Response: `{ "ok": true, "woke": true }` on a real wake, or a structured +/// error. E2E preconditions may use a seeded inbox row, but the behavior under +/// test starts at this bridge and cannot pass through `drain-inbox`. +pub async fn test_agent_org_session_return_to_work( + Json(body): Json, +) -> Json { + use tauri::Manager; + + let session_id = match body.get("session_id").and_then(|value| value.as_str()) { + Some(value) if !value.trim().is_empty() => value.to_string(), + _ => { + return Json(serde_json::json!({ + "ok": false, + "error": "session_id is required (non-empty string)" + })) + } + }; + + let Some(handle) = crate::api::get_app_handle() else { + return Json(serde_json::json!({ + "ok": false, + "error": "AppHandle not initialized." + })); + }; + let state = handle.state::(); + + match agent_core::state::commands::session::org_tasks::agent_org_session_return_to_work_impl( + &state, + session_id.clone(), + ) + .await + { + Ok(woke) => Json(serde_json::json!({ + "ok": true, + "session_id": session_id, + "woke": woke, + })), + Err(error) => Json(serde_json::json!({ + "ok": false, + "session_id": session_id, + "error": error, + })), + } +} + /// Render a `ToolError` into the stable `{error_kind, error_message}` /// shape used by the inter-agent E2E scenarios. Mirrors the desktop /// probes' [`tool_err_kind`] in `agent/test/desktop.rs` — keeping the @@ -1763,8 +1832,9 @@ pub async fn test_agent_org_seed_stale_worker_run( /// [`AgentOrgRunStore::find_worker_session_by_member_id`]. Runtime scenarios /// use it to poll production background member materialization and to obtain /// the real session id needed for a subsequent caller-path action. A status -/// value alone is not accepted as proof of inbox delivery; callers must also -/// inspect the durable `read_at` and transcript materialization evidence. +/// value alone is not accepted as proof of inbox delivery; the production +/// return-to-work scenario also checks `read_at` and persisted transcript +/// input. pub async fn test_agent_org_find_worker_session( Json(body): Json, ) -> Json { @@ -2049,7 +2119,9 @@ pub async fn test_agent_org_check_member_spawn_gate( /// [`agent_core::core::session::turn::processor::UnifiedMessageProcessor::process`] /// actually invokes `maybe_emit_member_idle` at turn end with the /// right `idle_reason` (Cancelled → Interrupted, Completed → -/// Available). Those lifecycle classifications remain pinned by focused +/// Available). The completed/Available caller path is exercised incidentally +/// by the production return-to-work scenario, which runs a real member turn; +/// Interrupted and Failed lifecycle classification remain pinned by focused /// lifecycle/unit coverage rather than this helper endpoint. /// /// Body shape mirrors the other agent-org probes (`org_run_id`, diff --git a/src/api/tauri/agent/orgTasks.ts b/src/api/tauri/agent/orgTasks.ts index 7114e0c4f..a2185b322 100644 --- a/src/api/tauri/agent/orgTasks.ts +++ b/src/api/tauri/agent/orgTasks.ts @@ -125,6 +125,7 @@ export interface AgentOrgRunTaskOverview { pending: number; inProgress: number; completed: number; + corrupt: number; visible: number; truncated: boolean; } @@ -250,6 +251,7 @@ export interface AgentOrgInboxPreviewRow { requestId?: string | null; createdAt: string; readAt?: string | null; + deliveryResolution?: "cancelled" | "superseded" | null; } export interface AgentOrgInboxRuntimeRow extends AgentOrgInboxPreviewRow { @@ -265,6 +267,7 @@ export interface AgentOrgGroupChatHistoryRow { displayText: string; createdAt: string; readAt?: string | null; + deliveryResolution?: "cancelled" | "superseded" | null; } export interface AgentOrgGroupChatHistoryPage { diff --git a/src/components/ComposerInput/__tests__/mentionQuery.test.ts b/src/components/ComposerInput/__tests__/mentionQuery.test.ts new file mode 100644 index 000000000..06b22754e --- /dev/null +++ b/src/components/ComposerInput/__tests__/mentionQuery.test.ts @@ -0,0 +1,32 @@ +import { describe, expect, it } from "vitest"; + +import { getInlineMentionQuery } from "../mentionQuery"; + +describe("getInlineMentionQuery", () => { + it("excludes the @ trigger when the keydown fallback ran before input", () => { + expect( + getInlineMentionQuery("@", 1, { startOffset: 0, hasAtChar: true }) + ).toBe(""); + expect( + getInlineMentionQuery("@planner", 8, { + startOffset: 0, + hasAtChar: true, + }) + ).toBe("planner"); + }); + + it("preserves normal inline and trigger-button query offsets", () => { + expect( + getInlineMentionQuery("before @planner", 15, { + startOffset: 8, + hasAtChar: true, + }) + ).toBe("planner"); + expect( + getInlineMentionQuery("planner", 7, { + startOffset: 0, + hasAtChar: false, + }) + ).toBe("planner"); + }); +}); diff --git a/src/components/ComposerInput/composerInput.inputHandler.ts b/src/components/ComposerInput/composerInput.inputHandler.ts index a12cf12b4..3683427c2 100644 --- a/src/components/ComposerInput/composerInput.inputHandler.ts +++ b/src/components/ComposerInput/composerInput.inputHandler.ts @@ -12,6 +12,7 @@ * `keyboard.ts`. */ import { type MentionState, canStartSlashCommand } from "./keyboard"; +import { getInlineMentionQuery } from "./mentionQuery"; import { caretTextOffset, rangeInsideHost } from "./selection"; import { PILL_DATA_ATTR, extractPlainText } from "./utils"; @@ -122,9 +123,11 @@ export function createInputHandler(ctx: InputHandlerContext) { ctx.getOnAtMentionClose()?.(); } } else { - const query = text - .slice(ctx.getAtMention().startOffset, caretOffset) - .replace(/\u200B/g, ""); + const query = getInlineMentionQuery( + text, + caretOffset, + ctx.getAtMention() + ); if (/\s/.test(query)) { if (!openedRecently) { ctx.setAtMention({ active: false, startOffset: 0 }); diff --git a/src/components/ComposerInput/mentionQuery.ts b/src/components/ComposerInput/mentionQuery.ts new file mode 100644 index 000000000..fa45994b1 --- /dev/null +++ b/src/components/ComposerInput/mentionQuery.ts @@ -0,0 +1,24 @@ +interface InlineMentionQueryState { + startOffset: number; + hasAtChar?: boolean; +} + +/** + * Return only the text typed after an inline `@` trigger. + * + * Some WebKit event schedules can run the keydown fallback before the `@` + * input event lands. In that case the fallback records the trigger's offset, + * not the first query-character offset. `hasAtChar` lets us normalize both + * schedules without exposing the literal `@` as the search query. + */ +export function getInlineMentionQuery( + text: string, + caretOffset: number, + mention: InlineMentionQueryState +): string { + const queryStart = + mention.hasAtChar && text[mention.startOffset] === "@" + ? mention.startOffset + 1 + : mention.startOffset; + return text.slice(queryStart, caretOffset).replace(/\u200B/g, ""); +} diff --git a/src/engines/ChatPanel/ChatHistory/GroupChatView/useGroupChatMergedEvents.ts b/src/engines/ChatPanel/ChatHistory/GroupChatView/useGroupChatMergedEvents.ts index 6f5ef6435..e0c53eced 100644 --- a/src/engines/ChatPanel/ChatHistory/GroupChatView/useGroupChatMergedEvents.ts +++ b/src/engines/ChatPanel/ChatHistory/GroupChatView/useGroupChatMergedEvents.ts @@ -48,6 +48,7 @@ function inboxRowToGroupChatUserEvent( actionType: "raw", args: { recipientMemberId: row.targetMemberId, + deliveryResolution: row.deliveryResolution ?? null, agentOrgGroupChatMessage: true, }, result: { diff --git a/src/engines/ChatPanel/ChatHistory/components/TurnPaginationControls.tsx b/src/engines/ChatPanel/ChatHistory/components/TurnPaginationControls.tsx index e31b5c759..db886f03a 100644 --- a/src/engines/ChatPanel/ChatHistory/components/TurnPaginationControls.tsx +++ b/src/engines/ChatPanel/ChatHistory/components/TurnPaginationControls.tsx @@ -33,6 +33,7 @@ import { DETAIL_PANEL_TOKENS } from "@src/config/detailPanelTokens"; import { SURFACE_TOKENS } from "@src/config/surfaceTokens"; import { useDropdownEngine } from "@src/hooks/dropdown"; import { WorkstationHeaderSectionSeparator } from "@src/modules/WorkStation/shared"; +import { isAgentOrgMemberEmpty } from "@src/util/agentOrg/memberActivity"; interface TurnPaginationControlsProps { agentName?: string | null; @@ -316,11 +317,7 @@ const TurnPaginationControls: React.FC = memo( : member.name; const hasNoTasksAndNoInbox = !member.isCoordinator && - member.activeTaskCount === 0 && - member.pendingTaskCount === 0 && - member.inProgressTaskCount === 0 && - member.completedTaskCount === 0 && - member.inboxActivityCount === 0; + isAgentOrgMemberEmpty(member); const runtimeStatusLabelKey = MEMBER_RUNTIME_STATUS_LABEL_KEYS[runtimeStatus]; const runtimeStatusLabel = hasNoTasksAndNoInbox @@ -332,8 +329,9 @@ const TurnPaginationControls: React.FC = memo( ? t(`sessions:${runtimeStatusLabelKey}`) : formatFallbackStatusLabel(runtimeStatus) : ""; - // Members with no tasks or inbox activity cannot be - // switched to — opening their session would render a + // Members with no tasks, recent activity, or durable + // unread Inbox cannot be switched to — opening their + // session would render a // chat panel with no events and a "session may not // have loaded" reload prompt. Coordinator is always // selectable (the parent session, never empty). diff --git a/src/engines/ChatPanel/InputArea/components/ContextMenuPortal.tsx b/src/engines/ChatPanel/InputArea/components/ContextMenuPortal.tsx index b853ec38e..b8ec8aac7 100644 --- a/src/engines/ChatPanel/InputArea/components/ContextMenuPortal.tsx +++ b/src/engines/ChatPanel/InputArea/components/ContextMenuPortal.tsx @@ -22,7 +22,10 @@ import { mainPaneTabsAtom, } from "@src/store/workstation/tabs"; -import { getOpenedTabMentionOptions } from "../openedTabMentionOptions"; +import { + getOpenedTabMentionOptions, + mergeCustomMentionOptions, +} from "../openedTabMentionOptions"; import type { FloatingPlacementStrategy } from "./floatingPlacement"; import { useFloatingPortalPosition } from "./useFloatingPortalPosition"; @@ -63,27 +66,6 @@ function getOpenedTabRecentFiles( })); } -function getMentionOptionTargetKey( - option: ContextMenuCustomMentionOption -): string { - return `${option.selectType}:${option.selectValue}`; -} - -function mergeCustomMentionOptions( - primaryOptions: ReadonlyArray, - secondaryOptions: ReadonlyArray = [] -): ContextMenuCustomMentionOption[] { - const merged: ContextMenuCustomMentionOption[] = []; - const seenTargets = new Set(); - for (const option of [...primaryOptions, ...secondaryOptions]) { - const targetKey = getMentionOptionTargetKey(option); - if (seenTargets.has(targetKey)) continue; - seenTargets.add(targetKey); - merged.push(option); - } - return merged; -} - const VisibleContextMenuPortal: React.FC< Omit > = ({ diff --git a/src/engines/ChatPanel/InputArea/components/agentOrgRunViewStore.test.ts b/src/engines/ChatPanel/InputArea/components/agentOrgRunViewStore.test.ts index 9b55dc380..b859de01d 100644 --- a/src/engines/ChatPanel/InputArea/components/agentOrgRunViewStore.test.ts +++ b/src/engines/ChatPanel/InputArea/components/agentOrgRunViewStore.test.ts @@ -1,8 +1,10 @@ import { afterEach, describe, expect, it, vi } from "vitest"; import { + AGENT_ORG_BOOTSTRAP_JOIN_TIMEOUT_MS, AGENT_ORG_RUN_VIEW_FALLBACK_MS, AGENT_ORG_RUN_VIEW_PUSH_DEBOUNCE_MS, + agentOrgRunViewStoreTestApi, getAgentOrgRunViewSnapshot, subscribeAgentOrgRunView, } from "./agentOrgRunViewStore"; @@ -31,7 +33,7 @@ async function flushPromises(): Promise { } function runView( - runStatus: "running" | "completed", + runStatus: "running" | "paused" | "completed", interventionResumeAfter?: string ) { return { @@ -113,6 +115,7 @@ function deferred() { } afterEach(() => { + agentOrgRunViewStoreTestApi.reset(); vi.useRealTimers(); vi.clearAllMocks(); }); @@ -193,92 +196,54 @@ describe("Agent Org run-view store", () => { expect(mocks.unsubscribeBackendChanges).toHaveBeenCalledTimes(3); }); - it("keeps a borrowed member snapshot referentially stable before subscribe", async () => { + it("stops probing a non-org session after its initial discovery", async () => { + vi.useFakeTimers(); mocks.subscribeAgentOrgStateChanges.mockReturnValue( mocks.unsubscribeStateChanges ); - mocks.getAgentOrgSessionRunView.mockResolvedValue(runView("running")); + mocks.getAgentOrgSessionRunView.mockResolvedValue(null); - const unsubscribeRoot = subscribeAgentOrgRunView("session-root", vi.fn()); + const unsubscribe = subscribeAgentOrgRunView("ordinary-session", vi.fn()); + expect(mocks.getAgentOrgSessionRunView).toHaveBeenCalledTimes(1); await flushPromises(); - // React reads getSnapshot before subscribe. Repeated reads must return the - // same wrapper until the backend actually publishes a different view. - const first = getAgentOrgRunViewSnapshot("session-worker"); - const second = getAgentOrgRunViewSnapshot("session-worker"); - expect(second).toBe(first); - expect(first.view?.currentMemberId).toBe("worker"); - - const unsubscribeWorker = subscribeAgentOrgRunView( - "session-worker", - vi.fn() - ); - expect(getAgentOrgRunViewSnapshot("session-worker")).toBe(first); + await vi.advanceTimersByTimeAsync(AGENT_ORG_RUN_VIEW_FALLBACK_MS * 5); expect(mocks.getAgentOrgSessionRunView).toHaveBeenCalledTimes(1); - unsubscribeWorker(); - unsubscribeRoot(); + unsubscribe(); }); - it("preserves the bounded-description signal in a shared snapshot", async () => { - const compactView = { - ...runView("running"), - tasks: [ - { - id: "task-1", - orgRunId: "run-1", - subject: "Compact task", - description: "bounded preview", - descriptionTruncated: true, - owner: "worker", - status: "pending", - blocks: [], - blockedBy: [], - executionMode: "build", - createdAt: "2026-07-17T00:00:00Z", - updatedAt: "2026-07-17T00:00:00Z", - }, - ], - }; + it("refreshes a retained view immediately when the session is reopened", async () => { + vi.useFakeTimers(); mocks.subscribeAgentOrgStateChanges.mockReturnValue( mocks.unsubscribeStateChanges ); - mocks.getAgentOrgSessionRunView.mockResolvedValue(compactView); + mocks.getAgentOrgSessionRunView + .mockResolvedValueOnce(runView("running")) + .mockResolvedValueOnce(runView("paused")); - const unsubscribeRoot = subscribeAgentOrgRunView("session-root", vi.fn()); + const unsubscribeFirst = subscribeAgentOrgRunView("session-root", vi.fn()); await flushPromises(); - const unsubscribeWorker = subscribeAgentOrgRunView( - "session-worker", - vi.fn() + expect(getAgentOrgRunViewSnapshot("session-root").view?.runStatus).toBe( + "running" ); - expect( - getAgentOrgRunViewSnapshot("session-worker").view?.tasks[0] - .descriptionTruncated - ).toBe(true); - - unsubscribeWorker(); - unsubscribeRoot(); - }); - - it("stops probing a non-org session after its initial discovery", async () => { - vi.useFakeTimers(); - mocks.subscribeAgentOrgStateChanges.mockReturnValue( - mocks.unsubscribeStateChanges + unsubscribeFirst(); + const unsubscribeReopened = subscribeAgentOrgRunView( + "session-root", + vi.fn() ); - mocks.getAgentOrgSessionRunView.mockResolvedValue(null); - - const unsubscribe = subscribeAgentOrgRunView("ordinary-session", vi.fn()); - expect(mocks.getAgentOrgSessionRunView).toHaveBeenCalledTimes(1); + expect(mocks.getAgentOrgSessionRunView).toHaveBeenCalledTimes(2); await flushPromises(); - await vi.advanceTimersByTimeAsync(AGENT_ORG_RUN_VIEW_FALLBACK_MS * 5); - expect(mocks.getAgentOrgSessionRunView).toHaveBeenCalledTimes(1); - - unsubscribe(); + expect(getAgentOrgRunViewSnapshot("session-root").view?.runStatus).toBe( + "paused" + ); + unsubscribeReopened(); }); it("rejects an older discovery response that resolves after a newer one", async () => { + vi.useFakeTimers(); const rootRequest = deferred>(); const workerRequest = deferred>(); mocks.subscribeAgentOrgStateChanges.mockReturnValue( @@ -294,6 +259,10 @@ describe("Agent Org run-view store", () => { vi.fn() ); + // Unknown sessions initially share one bootstrap request. If the first + // discovery hangs, the second is released after the bounded join timeout; + // request ordering must still reject the first request's late result. + await vi.advanceTimersByTimeAsync(AGENT_ORG_BOOTSTRAP_JOIN_TIMEOUT_MS); workerRequest.resolve(runView("completed")); await flushPromises(); rootRequest.resolve(runView("running")); diff --git a/src/engines/ChatPanel/InputArea/components/agentOrgRunViewStore.ts b/src/engines/ChatPanel/InputArea/components/agentOrgRunViewStore.ts index e968d4804..38252a6e8 100644 --- a/src/engines/ChatPanel/InputArea/components/agentOrgRunViewStore.ts +++ b/src/engines/ChatPanel/InputArea/components/agentOrgRunViewStore.ts @@ -8,6 +8,8 @@ import { export const AGENT_ORG_RUN_VIEW_FALLBACK_MS = 60_000; export const AGENT_ORG_RUN_VIEW_PUSH_DEBOUNCE_MS = 50; +export const AGENT_ORG_RUN_VIEW_CACHE_RETENTION_MS = 30_000; +export const AGENT_ORG_BOOTSTRAP_JOIN_TIMEOUT_MS = 1_000; const MAX_NON_ORG_DISCOVERY_ATTEMPTS = 1; const TERMINAL_RUN_STATUSES: ReadonlySet = new Set([ @@ -30,6 +32,12 @@ interface RunViewEntry { serializedView: string | null; subscribers: Set; discoveryAttempts: number; + evictionTimer: ReturnType | null; + bootstrapWait: Promise | null; + lastAppliedRequestId: number; + hasBeenSubscribed: boolean; + /** Set once this cache generation is evicted; late IPC responses are ignored. */ + retired: boolean; } const EMPTY_SNAPSHOT: AgentOrgRunViewSnapshot = { @@ -48,6 +56,7 @@ const interventionExpiryTimers = new Map< >(); let nextRequestId = 0; let activeSubscriberCount = 0; +let bootstrapOwner: RunViewEntry | null = null; let pollingTimer: ReturnType | undefined; let unsubscribeStateChanges: (() => void) | undefined; let unsubscribeBackendChanges: (() => void) | undefined; @@ -101,6 +110,53 @@ function findEntryCoveringSession(sessionId: string): RunViewEntry | undefined { return undefined; } +function isCurrentEntry(entry: RunViewEntry): boolean { + return !entry.retired && entriesBySessionId.get(entry.sessionId) === entry; +} + +function cancelEntryEviction(entry: RunViewEntry): void { + if (entry.evictionTimer === null) return; + clearTimeout(entry.evictionTimer); + entry.evictionTimer = null; +} + +function scheduleEntryEviction(entry: RunViewEntry): void { + if ( + !isCurrentEntry(entry) || + entry.subscribers.size > 0 || + entry.evictionTimer !== null + ) { + return; + } + entry.evictionTimer = setTimeout(() => { + entry.evictionTimer = null; + evictEntry(entry); + }, AGENT_ORG_RUN_VIEW_CACHE_RETENTION_MS); +} + +function evictEntry(entry: RunViewEntry): void { + if (!isCurrentEntry(entry) || entry.subscribers.size > 0) return; + entry.retired = true; + entriesBySessionId.delete(entry.sessionId); + if (bootstrapOwner === entry) bootstrapOwner = null; + + const sessionKey = `session:${entry.sessionId}`; + inFlightByRunOrSession.delete(sessionKey); + refreshAfterInFlight.delete(sessionKey); + + const runId = entry.snapshot.view?.context.runId; + if (!runId) return; + const hasOtherRunEntry = Array.from(entriesBySessionId.values()).some( + (candidate) => candidate.snapshot.view?.context.runId === runId + ); + if (!hasOtherRunEntry) { + const runKey = `run:${runId}`; + inFlightByRunOrSession.delete(runKey); + refreshAfterInFlight.delete(runKey); + latestRequestIdByRun.delete(runId); + } +} + function getOrCreateEntry(sessionId: string): RunViewEntry { const existing = entriesBySessionId.get(sessionId); if (existing) return existing; @@ -117,35 +173,96 @@ function getOrCreateEntry(sessionId: string): RunViewEntry { serializedView: seededView ? JSON.stringify(seededView) : null, subscribers: new Set(), discoveryAttempts: seededView ? MAX_NON_ORG_DISCOVERY_ATTEMPTS : 0, + evictionTimer: null, + bootstrapWait: null, + lastAppliedRequestId: 0, + hasBeenSubscribed: false, + retired: false, }; entriesBySessionId.set(sessionId, entry); + // React can call getSnapshot for a render that is abandoned before + // subscribe. Keep that generation bounded just like an unsubscribed entry. + scheduleEntryEviction(entry); return entry; } function publishEntry( entry: RunViewEntry, view: AgentOrgRunView | null, - error: string | null -): void { + error: string | null, + requestId?: number +): boolean { + if (!isCurrentEntry(entry)) return false; + if (requestId !== undefined) { + if (requestId < entry.lastAppliedRequestId) return false; + entry.lastAppliedRequestId = requestId; + } const serializedView = view ? JSON.stringify(view) : null; if ( entry.serializedView === serializedView && entry.snapshot.error === error ) { - return; + return true; } entry.snapshot = { view, error }; entry.serializedView = serializedView; for (const subscriber of entry.subscribers) subscriber(); + return true; } -function publishRunView(view: AgentOrgRunView): void { +function publishRunView(view: AgentOrgRunView, requestId: number): void { scheduleInterventionExpiryRefresh(view); for (const entry of entriesBySessionId.values()) { - if (!runViewContainsSession(view, entry.sessionId)) continue; + if ( + !runViewContainsSession(view, entry.sessionId) && + entry.snapshot.view?.context.runId !== view.context.runId + ) { + continue; + } entry.discoveryAttempts = MAX_NON_ORG_DISCOVERY_ATTEMPTS; - publishEntry(entry, viewForSession(view, entry.sessionId), null); + publishEntry(entry, viewForSession(view, entry.sessionId), null, requestId); + } +} + +function liveReplacementForRun( + runId: string, + excluded: RunViewEntry +): RunViewEntry | undefined { + return Array.from(entriesBySessionId.values()).find( + (candidate) => + candidate !== excluded && + candidate.subscribers.size > 0 && + candidate.snapshot.view?.context.runId === runId + ); +} + +function publishMissingRun( + entry: RunViewEntry, + requestId: number +): RunViewEntry | null { + if (!isCurrentEntry(entry) || requestId < entry.lastAppliedRequestId) { + return null; + } + const previousRunId = entry.snapshot.view?.context.runId; + entry.discoveryAttempts += 1; + publishEntry(entry, null, null, requestId); + if (!previousRunId) return null; + + const replacement = liveReplacementForRun(previousRunId, entry); + if (replacement) { + // One session disappearing does not prove that the Run disappeared. Ask a + // live peer before clearing the shared projection. + return replacement; + } + + // No live session can verify the old Run. Clear every retained projection + // so an inactive member panel cannot keep a ghost board indefinitely. + for (const related of entriesBySessionId.values()) { + if (related.snapshot.view?.context.runId !== previousRunId) continue; + related.discoveryAttempts = MAX_NON_ORG_DISCOVERY_ATTEMPTS; + publishEntry(related, null, null, requestId); } + return null; } function scheduleInterventionExpiryRefresh(view: AgentOrgRunView): void { @@ -177,6 +294,31 @@ function requestKey(entry: RunViewEntry): string { return runId ? `run:${runId}` : `session:${entry.sessionId}`; } +function waitForBootstrapOwner( + owner: RunViewEntry, + request: Promise +): Promise { + return new Promise((resolve) => { + const timeout = setTimeout(() => { + // A hung discovery for one session must not serialize every unrelated + // session forever. Its eventual response remains guarded by generation + // and request ordering. + if (bootstrapOwner === owner) bootstrapOwner = null; + resolve(); + }, AGENT_ORG_BOOTSTRAP_JOIN_TIMEOUT_MS); + request.then( + () => { + clearTimeout(timeout); + resolve(); + }, + () => { + clearTimeout(timeout); + resolve(); + } + ); + }); +} + function refreshAgentOrgRunViewInternal( sessionId: string, queueIfBusy: boolean @@ -191,28 +333,56 @@ function refreshAgentOrgRunViewInternal( return existing; } + if ( + entry.snapshot.view === null && + bootstrapOwner && + bootstrapOwner !== entry + ) { + if (entry.bootstrapWait) return entry.bootstrapWait; + const owner = bootstrapOwner; + const ownerRequest = inFlightByRunOrSession.get(requestKey(owner)); + if (ownerRequest) { + const wait = waitForBootstrapOwner(owner, ownerRequest).then(async () => { + if (!isCurrentEntry(entry) || entry.snapshot.view !== null) return; + if (entry.subscribers.size > 0 || queueIfBusy) { + await refreshAgentOrgRunViewInternal(entry.sessionId, queueIfBusy); + } + }); + entry.bootstrapWait = wait; + void wait.finally(() => { + if (entry.bootstrapWait === wait) entry.bootstrapWait = null; + }); + return wait; + } + bootstrapOwner = null; + } + const requestId = ++nextRequestId; const knownRunId = entry.snapshot.view?.context.runId; if (knownRunId) latestRequestIdByRun.set(knownRunId, requestId); + if (!knownRunId) bootstrapOwner = entry; + let missingRunReplacement: RunViewEntry | null = null; const request = getAgentOrgSessionRunView(entry.sessionId) .then((view) => { + if (!isCurrentEntry(entry)) return; if (view) { const runId = view.context.runId; const latestRequestId = latestRequestIdByRun.get(runId) ?? 0; if (requestId < latestRequestId) return; latestRequestIdByRun.set(runId, requestId); - publishRunView(view); + publishRunView(view, requestId); return; } - entry.discoveryAttempts += 1; - publishEntry(entry, null, null); + missingRunReplacement = publishMissingRun(entry, requestId); }) .catch((error: unknown) => { + if (!isCurrentEntry(entry)) return; const message = error instanceof Error ? error.message : String(error); - publishEntry(entry, entry.snapshot.view, message); + publishEntry(entry, entry.snapshot.view, message, requestId); }) .finally(() => { + if (bootstrapOwner === entry) bootstrapOwner = null; if (inFlightByRunOrSession.get(key) === request) { inFlightByRunOrSession.delete(key); } @@ -224,8 +394,21 @@ function refreshAgentOrgRunViewInternal( const shouldRefreshAgain = refreshAfterInFlight.delete(key) || (runKey ? refreshAfterInFlight.delete(runKey) : false); - if (shouldRefreshAgain && entry.subscribers.size > 0) { + if ( + shouldRefreshAgain && + isCurrentEntry(entry) && + entry.subscribers.size > 0 + ) { void refreshAgentOrgRunViewInternal(entry.sessionId, false); + } else if ( + missingRunReplacement && + isCurrentEntry(missingRunReplacement) && + missingRunReplacement.subscribers.size > 0 + ) { + void refreshAgentOrgRunViewInternal( + missingRunReplacement.sessionId, + false + ); } }); inFlightByRunOrSession.set(key, request); @@ -361,35 +544,28 @@ export function subscribeAgentOrgRunView( subscriber: Subscriber ): () => void { const entry = getOrCreateEntry(sessionId); + cancelEntryEviction(entry); + const isReturningSubscriber = + entry.hasBeenSubscribed && entry.subscribers.size === 0; + entry.hasBeenSubscribed = true; const subscription = () => subscriber(); entry.subscribers.add(subscription); activeSubscriberCount += 1; if (activeSubscriberCount === 1) startScheduler(); if (entry.snapshot.view === null && entry.discoveryAttempts === 0) { void refreshAgentOrgRunViewInternal(sessionId, false); + } else if (isReturningSubscriber && entry.snapshot.view !== null) { + // Retained entries stop receiving push events after their last subscriber + // leaves. Refresh immediately when the UI reopens the session instead of + // showing a stale Run status until the fallback poll fires. + void refreshAgentOrgRunViewInternal(sessionId, false); } return () => { if (!entry.subscribers.delete(subscription)) return; activeSubscriberCount -= 1; if (activeSubscriberCount === 0) stopScheduler(); - queueMicrotask(() => { - if ( - entry.subscribers.size === 0 && - entriesBySessionId.get(entry.sessionId) === entry - ) { - entriesBySessionId.delete(entry.sessionId); - const runId = entry.snapshot.view?.context.runId; - if ( - runId && - !Array.from(entriesBySessionId.values()).some( - (candidate) => candidate.snapshot.view?.context.runId === runId - ) - ) { - latestRequestIdByRun.delete(runId); - } - } - }); + scheduleEntryEviction(entry); }; } @@ -398,3 +574,37 @@ export function getAgentOrgRunViewSnapshot( ): AgentOrgRunViewSnapshot { return getOrCreateEntry(sessionId).snapshot; } + +/** Narrow test seam for cache-generation and shared-poller invariants. */ +export const agentOrgRunViewStoreTestApi = { + subscribe: subscribeAgentOrgRunView, + getSnapshot: getAgentOrgRunViewSnapshot, + refresh: refreshAgentOrgRunView, + hasEntry(sessionId: string): boolean { + return entriesBySessionId.has(sessionId); + }, + ownerSessionId(runId: string): string | null { + return ( + Array.from(entriesBySessionId.values()).find( + (entry) => + entry.subscribers.size > 0 && + entry.snapshot.view?.context.runId === runId + )?.sessionId ?? null + ); + }, + reset(): void { + stopScheduler(); + for (const entry of entriesBySessionId.values()) { + cancelEntryEviction(entry); + entry.subscribers.clear(); + entry.retired = true; + } + entriesBySessionId.clear(); + inFlightByRunOrSession.clear(); + latestRequestIdByRun.clear(); + refreshAfterInFlight.clear(); + bootstrapOwner = null; + nextRequestId = 0; + activeSubscriberCount = 0; + }, +}; diff --git a/src/engines/ChatPanel/InputArea/components/useAgentOrgIntervention.test.ts b/src/engines/ChatPanel/InputArea/components/useAgentOrgIntervention.test.ts index 3e5f3917c..2df0842b7 100644 --- a/src/engines/ChatPanel/InputArea/components/useAgentOrgIntervention.test.ts +++ b/src/engines/ChatPanel/InputArea/components/useAgentOrgIntervention.test.ts @@ -78,6 +78,7 @@ function runView(): AgentOrgRunView { pending: 0, inProgress: 0, completed: 0, + corrupt: 0, visible: 0, truncated: false, }, diff --git a/src/engines/ChatPanel/InputArea/components/useAgentOrgPlanApprovalDetail.test.ts b/src/engines/ChatPanel/InputArea/components/useAgentOrgPlanApprovalDetail.test.ts index c47589c0b..7c79b505c 100644 --- a/src/engines/ChatPanel/InputArea/components/useAgentOrgPlanApprovalDetail.test.ts +++ b/src/engines/ChatPanel/InputArea/components/useAgentOrgPlanApprovalDetail.test.ts @@ -109,13 +109,6 @@ describe("Agent Org plan approval detail cache", () => { loading: false, }); - // A Run View poll returns a new summary object for the same immutable - // revision. That must not turn a visible failure into an automatic retry. - await agentOrgPlanApprovalDetailCacheTestApi.load("root-session", { - ...approval, - }); - expect(mockedGetDetail).toHaveBeenCalledTimes(1); - await agentOrgPlanApprovalDetailCacheTestApi.load( "root-session", approval, diff --git a/src/engines/ChatPanel/InputArea/components/useAgentOrgRunView.test.ts b/src/engines/ChatPanel/InputArea/components/useAgentOrgRunView.test.ts new file mode 100644 index 000000000..7042e46cc --- /dev/null +++ b/src/engines/ChatPanel/InputArea/components/useAgentOrgRunView.test.ts @@ -0,0 +1,403 @@ +import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; + +import { + type AgentOrgRunView, + getAgentOrgSessionRunView, +} from "@src/api/tauri/agent/orgTasks"; + +import { + AGENT_ORG_RUN_VIEW_FALLBACK_MS, + agentOrgRunViewStoreTestApi, +} from "./agentOrgRunViewStore"; + +vi.mock("@src/api/tauri/agent/orgTasks", () => ({ + getAgentOrgSessionRunView: vi.fn(), + subscribeAgentOrgStateChanges: vi.fn(() => () => undefined), +})); + +vi.mock("@src/api/realtime/codeEditorWebSocket", () => ({ + getCodeEditorWebSocket: () => ({ on: () => () => undefined }), +})); + +const mockedGetRunView = vi.mocked(getAgentOrgSessionRunView); +const agentOrgRunViewPollingTestApi = agentOrgRunViewStoreTestApi; + +function runView(): AgentOrgRunView { + return { + context: { + runId: "run-1", + orgId: "org-1", + orgName: "Test Org", + orgRole: "team", + coordinatorAgentId: "coord-agent", + coordinatorName: "Coordinator", + coordinatorRole: "lead", + members: [ + { memberId: "m1", name: "Alice", role: "worker", agentId: "alice" }, + { memberId: "m2", name: "Bob", role: "worker", agentId: "bob" }, + ], + hierarchyMode: "flat", + planApprovalPolicy: "coordinator", + rootSessionId: "root-session", + }, + runStatus: "running", + runPhase: "members_working", + currentMemberId: "coordinator", + members: [ + { + memberId: "m1", + name: "Alice", + role: "worker", + agentId: "alice", + isCoordinator: false, + sessionRuntime: { + sessionId: "alice-session", + status: "idle", + updatedAt: "2026-07-16T00:00:00Z", + }, + unreadInboxCount: 0, + inboxActivityCount: 0, + activeTaskCount: 0, + pendingTaskCount: 0, + inProgressTaskCount: 0, + completedTaskCount: 0, + }, + { + memberId: "m2", + name: "Bob", + role: "worker", + agentId: "bob", + isCoordinator: false, + sessionRuntime: { + sessionId: "bob-session", + status: "idle", + updatedAt: "2026-07-16T00:00:00Z", + }, + unreadInboxCount: 0, + inboxActivityCount: 0, + activeTaskCount: 0, + pendingTaskCount: 0, + inProgressTaskCount: 0, + completedTaskCount: 0, + }, + ], + tasks: [], + taskOverview: { + total: 0, + pending: 0, + inProgress: 0, + completed: 0, + corrupt: 0, + visible: 0, + truncated: false, + }, + inbox: [], + unreadInboxCount: 0, + pendingPlanApprovals: [], + }; +} + +describe("Agent Org run-view polling coordinator", () => { + beforeEach(() => { + vi.useFakeTimers(); + agentOrgRunViewPollingTestApi.reset(); + mockedGetRunView.mockReset(); + }); + + afterEach(() => { + agentOrgRunViewPollingTestApi.reset(); + vi.useRealTimers(); + }); + + it("shares one run poller while preserving caller-specific currentMemberId", async () => { + mockedGetRunView.mockResolvedValue(runView()); + const unsubscribeRoot = agentOrgRunViewPollingTestApi.subscribe( + "root-session", + () => undefined + ); + await vi.waitFor(() => expect(mockedGetRunView).toHaveBeenCalledTimes(1)); + + const unsubscribeAlice = agentOrgRunViewPollingTestApi.subscribe( + "alice-session", + () => undefined + ); + const unsubscribeBob = agentOrgRunViewPollingTestApi.subscribe( + "bob-session", + () => undefined + ); + + expect(mockedGetRunView).toHaveBeenCalledTimes(1); + expect( + agentOrgRunViewPollingTestApi.getSnapshot("root-session").view + ?.currentMemberId + ).toBe("coordinator"); + expect( + agentOrgRunViewPollingTestApi.getSnapshot("alice-session").view + ?.currentMemberId + ).toBe("m1"); + expect( + agentOrgRunViewPollingTestApi.getSnapshot("bob-session").view + ?.currentMemberId + ).toBe("m2"); + + unsubscribeBob(); + unsubscribeAlice(); + unsubscribeRoot(); + }); + + it("preserves the bounded-description signal in shared Run View snapshots", async () => { + const compactView = runView(); + compactView.tasks = [ + { + id: "task-1", + orgRunId: "run-1", + subject: "Compact task", + description: "bounded preview", + descriptionTruncated: true, + owner: "m1", + status: "pending", + blocks: [], + blockedBy: [], + executionMode: "build", + createdAt: "2026-07-16T00:00:00Z", + updatedAt: "2026-07-16T00:00:00Z", + }, + ]; + mockedGetRunView.mockResolvedValue(compactView); + + const unsubscribeRoot = agentOrgRunViewPollingTestApi.subscribe( + "root-session", + () => undefined + ); + await vi.waitFor(() => expect(mockedGetRunView).toHaveBeenCalledTimes(1)); + const unsubscribeAlice = agentOrgRunViewPollingTestApi.subscribe( + "alice-session", + () => undefined + ); + + expect( + agentOrgRunViewPollingTestApi.getSnapshot("alice-session").view?.tasks[0] + .descriptionTruncated + ).toBe(true); + + unsubscribeAlice(); + unsubscribeRoot(); + }); + + it("keeps a shared member snapshot referentially stable before subscribe", async () => { + mockedGetRunView.mockResolvedValue(runView()); + const unsubscribeRoot = agentOrgRunViewPollingTestApi.subscribe( + "root-session", + () => undefined + ); + await vi.waitFor(() => expect(mockedGetRunView).toHaveBeenCalledTimes(1)); + + // React's useSyncExternalStore reads getSnapshot before it subscribes. A + // member can already borrow the root session's Run projection at that + // point, and repeated reads must return the exact same object identity. + const first = agentOrgRunViewPollingTestApi.getSnapshot("alice-session"); + const second = agentOrgRunViewPollingTestApi.getSnapshot("alice-session"); + expect(second).toBe(first); + expect(first.view?.currentMemberId).toBe("m1"); + + const unsubscribeAlice = agentOrgRunViewPollingTestApi.subscribe( + "alice-session", + () => undefined + ); + expect(agentOrgRunViewPollingTestApi.getSnapshot("alice-session")).toBe( + first + ); + expect(mockedGetRunView).toHaveBeenCalledTimes(1); + + unsubscribeAlice(); + unsubscribeRoot(); + }); + + it("evicts a getSnapshot-only generation when React never subscribes", async () => { + agentOrgRunViewPollingTestApi.getSnapshot("abandoned-render-session"); + expect( + agentOrgRunViewPollingTestApi.hasEntry("abandoned-render-session") + ).toBe(true); + + await vi.advanceTimersByTimeAsync(29_999); + expect( + agentOrgRunViewPollingTestApi.hasEntry("abandoned-render-session") + ).toBe(true); + + await vi.advanceTimersByTimeAsync(1); + expect( + agentOrgRunViewPollingTestApi.hasEntry("abandoned-render-session") + ).toBe(false); + }); + + it("ignores a late IPC response from an evicted cache generation", async () => { + let resolveRequest: ((view: AgentOrgRunView) => void) | undefined; + mockedGetRunView.mockImplementation( + () => + new Promise((resolve) => { + resolveRequest = resolve; + }) + ); + + const unsubscribe = agentOrgRunViewPollingTestApi.subscribe( + "root-session", + () => undefined + ); + // Capture the shared in-flight request so the test can await its late + // completion without creating a replacement cache entry after eviction. + const inFlight = agentOrgRunViewPollingTestApi.refresh("root-session"); + expect(mockedGetRunView).toHaveBeenCalledTimes(1); + + unsubscribe(); + await vi.advanceTimersByTimeAsync(30_000); + expect(agentOrgRunViewPollingTestApi.hasEntry("root-session")).toBe(false); + + resolveRequest?.(runView()); + await inFlight; + + expect(agentOrgRunViewPollingTestApi.hasEntry("root-session")).toBe(false); + expect(agentOrgRunViewPollingTestApi.ownerSessionId("run-1")).toBeNull(); + }); + + it("coalesces same-session subscribers before the first response", async () => { + let resolveRequest: ((view: AgentOrgRunView) => void) | undefined; + mockedGetRunView.mockImplementation( + () => + new Promise((resolve) => { + resolveRequest = resolve; + }) + ); + + const unsubscribeFirst = agentOrgRunViewPollingTestApi.subscribe( + "root-session", + () => undefined + ); + const unsubscribeSecond = agentOrgRunViewPollingTestApi.subscribe( + "root-session", + () => undefined + ); + + expect(mockedGetRunView).toHaveBeenCalledTimes(1); + resolveRequest?.(runView()); + await vi.waitFor(() => + expect( + agentOrgRunViewPollingTestApi.getSnapshot("root-session").view + ).not.toBeNull() + ); + + unsubscribeSecond(); + unsubscribeFirst(); + }); + + it("coalesces root and member subscribers during run-id bootstrap", async () => { + let resolveRequest: ((view: AgentOrgRunView) => void) | undefined; + mockedGetRunView.mockImplementation( + () => + new Promise((resolve) => { + resolveRequest = resolve; + }) + ); + + const unsubscribeRoot = agentOrgRunViewPollingTestApi.subscribe( + "root-session", + () => undefined + ); + const unsubscribeAlice = agentOrgRunViewPollingTestApi.subscribe( + "alice-session", + () => undefined + ); + const unsubscribeBob = agentOrgRunViewPollingTestApi.subscribe( + "bob-session", + () => undefined + ); + + expect(mockedGetRunView).toHaveBeenCalledTimes(1); + resolveRequest?.(runView()); + await vi.waitFor(() => + expect( + agentOrgRunViewPollingTestApi.getSnapshot("bob-session").view + ).not.toBeNull() + ); + expect(mockedGetRunView).toHaveBeenCalledTimes(1); + + unsubscribeBob(); + unsubscribeAlice(); + unsubscribeRoot(); + }); + + it("releases an unrelated bootstrap when the global owner hangs", async () => { + mockedGetRunView.mockImplementation((sessionId) => { + if (sessionId === "stuck-session") { + return new Promise(() => undefined); + } + return Promise.resolve(null); + }); + + const unsubscribeStuck = agentOrgRunViewPollingTestApi.subscribe( + "stuck-session", + () => undefined + ); + const unsubscribeOther = agentOrgRunViewPollingTestApi.subscribe( + "unrelated-session", + () => undefined + ); + expect(mockedGetRunView).toHaveBeenCalledTimes(1); + + await vi.advanceTimersByTimeAsync(1_000); + await vi.waitFor(() => expect(mockedGetRunView).toHaveBeenCalledTimes(2)); + expect(mockedGetRunView).toHaveBeenLastCalledWith("unrelated-session"); + + unsubscribeOther(); + unsubscribeStuck(); + }); + + it("moves polling to another member when the current owner fails", async () => { + mockedGetRunView + .mockResolvedValueOnce(runView()) + .mockRejectedValueOnce(new Error("session disappeared")) + .mockResolvedValue(runView()); + const unsubscribeRoot = agentOrgRunViewPollingTestApi.subscribe( + "root-session", + () => undefined + ); + await vi.waitFor(() => expect(mockedGetRunView).toHaveBeenCalledTimes(1)); + const unsubscribeAlice = agentOrgRunViewPollingTestApi.subscribe( + "alice-session", + () => undefined + ); + + await agentOrgRunViewPollingTestApi.refresh("root-session"); + expect(mockedGetRunView).toHaveBeenCalledTimes(2); + await vi.advanceTimersByTimeAsync(AGENT_ORG_RUN_VIEW_FALLBACK_MS); + await vi.waitFor(() => expect(mockedGetRunView).toHaveBeenCalledTimes(3)); + expect(mockedGetRunView).toHaveBeenLastCalledWith("alice-session"); + + unsubscribeAlice(); + unsubscribeRoot(); + }); + + it("clears every cached member view when all live sessions return no run", async () => { + mockedGetRunView.mockResolvedValueOnce(runView()).mockResolvedValue(null); + const unsubscribeRoot = agentOrgRunViewPollingTestApi.subscribe( + "root-session", + () => undefined + ); + await vi.waitFor(() => expect(mockedGetRunView).toHaveBeenCalledTimes(1)); + const unsubscribeAlice = agentOrgRunViewPollingTestApi.subscribe( + "alice-session", + () => undefined + ); + + await agentOrgRunViewPollingTestApi.refresh("root-session"); + await vi.waitFor(() => expect(mockedGetRunView).toHaveBeenCalledTimes(3)); + expect( + agentOrgRunViewPollingTestApi.getSnapshot("root-session").view + ).toBeNull(); + expect( + agentOrgRunViewPollingTestApi.getSnapshot("alice-session").view + ).toBeNull(); + + unsubscribeAlice(); + unsubscribeRoot(); + }); +}); diff --git a/src/engines/ChatPanel/InputArea/openedTabMentionOptions.ts b/src/engines/ChatPanel/InputArea/openedTabMentionOptions.ts index 3fa122e8e..fc407e184 100644 --- a/src/engines/ChatPanel/InputArea/openedTabMentionOptions.ts +++ b/src/engines/ChatPanel/InputArea/openedTabMentionOptions.ts @@ -99,9 +99,26 @@ export const getOpenedTabMentionOption = ( }; function getMentionOptionTargetKey(option: CustomMentionOption): string { - return `${option.selectType}:${option.selectValue}`; + return option.selectType && option.selectValue + ? `target:${option.selectType}:${option.selectValue}` + : `custom:${option.id}`; } +export const mergeCustomMentionOptions = ( + primaryOptions: ReadonlyArray, + secondaryOptions: ReadonlyArray = [] +): CustomMentionOption[] => { + const merged: CustomMentionOption[] = []; + const seenTargets = new Set(); + for (const option of [...primaryOptions, ...secondaryOptions]) { + const targetKey = getMentionOptionTargetKey(option); + if (seenTargets.has(targetKey)) continue; + seenTargets.add(targetKey); + merged.push(option); + } + return merged; +}; + export const getOpenedTabMentionOptions = ( workstationTabs: ReadonlyArray ): CustomMentionOption[] => { diff --git a/src/engines/ChatPanel/InputArea/utils/__tests__/openedTabMentionOptions.test.ts b/src/engines/ChatPanel/InputArea/utils/__tests__/openedTabMentionOptions.test.ts index 1c40f3b87..e784e87d9 100644 --- a/src/engines/ChatPanel/InputArea/utils/__tests__/openedTabMentionOptions.test.ts +++ b/src/engines/ChatPanel/InputArea/utils/__tests__/openedTabMentionOptions.test.ts @@ -2,7 +2,10 @@ import { describe, expect, it, vi } from "vitest"; import type { WorkStationTab } from "@src/store/workstation/tabs"; -import { getOpenedTabMentionOptions } from "../../openedTabMentionOptions"; +import { + getOpenedTabMentionOptions, + mergeCustomMentionOptions, +} from "../../openedTabMentionOptions"; vi.mock("@src/components/TerminalInteractive/bufferCache", () => ({ hasNonEmptyTerminalBuffer: vi.fn(() => true), @@ -21,6 +24,36 @@ function makeTab(overrides: Partial): WorkStationTab { } describe("getOpenedTabMentionOptions", () => { + it("keeps distinct member mentions while deduplicating shared targets", () => { + const options = mergeCustomMentionOptions( + [ + { id: "coordinator", label: "Coordinator" }, + { id: "planner", label: "Planner" }, + { + id: "file-primary", + label: "Primary file", + selectType: "files", + selectValue: "/repo/src/index.tsx", + }, + ], + [ + { id: "planner", label: "Planner duplicate" }, + { + id: "file-secondary", + label: "Duplicate target", + selectType: "files", + selectValue: "/repo/src/index.tsx", + }, + ] + ); + + expect(options.map((option) => option.id)).toEqual([ + "coordinator", + "planner", + "file-primary", + ]); + }); + it("deduplicates tabs that point to the same mention target", () => { const options = getOpenedTabMentionOptions([ makeTab({ diff --git a/src/engines/ChatPanel/hooks/useAgentOrgGroupChatController.ts b/src/engines/ChatPanel/hooks/useAgentOrgGroupChatController.ts index 3b453dc99..89f610671 100644 --- a/src/engines/ChatPanel/hooks/useAgentOrgGroupChatController.ts +++ b/src/engines/ChatPanel/hooks/useAgentOrgGroupChatController.ts @@ -12,7 +12,10 @@ import { sendAgentOrgGroupChatMessage, } from "@src/api/tauri/agent"; import { useGroupChatMergedEvents } from "@src/engines/ChatPanel/ChatHistory/GroupChatView/useGroupChatMergedEvents"; -import { useAgentOrgGroupChatHistory } from "@src/engines/ChatPanel/hooks/useAgentOrgGroupChatHistory"; +import { + isGroupChatPendingDeliverySettled, + useAgentOrgGroupChatHistory, +} from "@src/engines/ChatPanel/hooks/useAgentOrgGroupChatHistory"; import type { CustomMentionOption, SubmitOverrideInput, @@ -53,10 +56,6 @@ function normalizeMentionToken(value: string): string { .replace(/[^a-z0-9_-]+/g, ""); } -function isInboxRowRead(row: { readAt?: string | null } | undefined): boolean { - return Boolean(row?.readAt && row.readAt.trim()); -} - function timestampMs(value: string | null | undefined): number | null { if (!value) return null; const ms = new Date(value).getTime(); @@ -222,7 +221,9 @@ export function useAgentOrgGroupChatController({ const rows = agentOrgRunView?.inbox ?? []; return rows .filter((row) => row.senderAgentId === AGENT_ORG_USER_SENDER_ID) - .map((row) => `${row.id}:${row.readAt ?? ""}`) + .map( + (row) => `${row.id}:${row.readAt ?? ""}:${row.deliveryResolution ?? ""}` + ) .join("|"); }, [agentOrgRunView?.inbox]); const { @@ -256,6 +257,7 @@ export function useAgentOrgGroupChatController({ displayText: groupChatPendingMessage.displayText, createdAt: groupChatPendingMessage.createdAt, readAt: null, + deliveryResolution: null, }, ].sort((left, right) => left.inboxId - right.inboxId); }, [durableGroupChatHistoryRows, groupChatPendingMessage]); @@ -292,7 +294,13 @@ export function useAgentOrgGroupChatController({ const pendingRow = agentOrgRunView.inbox.find( (row) => row.id === groupChatPendingMessage.rowId ); - if (isInboxRowRead(pendingRow)) { + if ( + isGroupChatPendingDeliverySettled( + groupChatPendingMessage.rowId, + pendingRow, + durableGroupChatHistoryRows + ) + ) { setGroupChatPendingMessage(null); return; } @@ -321,6 +329,7 @@ export function useAgentOrgGroupChatController({ } }, [ agentOrgRunView, + durableGroupChatHistoryRows, groupChatMergedEvents, groupChatPendingMessage, sessionId, diff --git a/src/engines/ChatPanel/hooks/useAgentOrgGroupChatHistory.test.ts b/src/engines/ChatPanel/hooks/useAgentOrgGroupChatHistory.test.ts index 7042da539..c486845b5 100644 --- a/src/engines/ChatPanel/hooks/useAgentOrgGroupChatHistory.test.ts +++ b/src/engines/ChatPanel/hooks/useAgentOrgGroupChatHistory.test.ts @@ -16,6 +16,7 @@ function row( displayText: `@Reviewer message-${inboxId}`, createdAt: `2026-07-17T00:00:${String(inboxId).padStart(2, "0")}Z`, readAt: null, + deliveryResolution: null, ...overrides, }; } @@ -62,7 +63,7 @@ describe("Agent Org Group Chat durable history", () => { ), request, { - rows: [row(3), row(4), row(5)], + rows: [row(3, { deliveryResolution: "superseded" }), row(4), row(5)], hasMore: true, nextBeforeId: 3, } @@ -80,6 +81,9 @@ describe("Agent Org Group Chat durable history", () => { expect(loaded.rows.find((item) => item.inboxId === 3)?.readAt).toBe( "2026-07-17T00:01:00Z" ); + expect( + loaded.rows.find((item) => item.inboxId === 3)?.deliveryResolution + ).toBe("superseded"); expect(loaded).toMatchObject({ hasMore: false, nextBeforeId: null }); }); @@ -101,16 +105,16 @@ describe("Agent Org Group Chat durable history", () => { older, request, { - rows: [row(4, { readAt: "2026-07-17T00:02:00Z" }), row(5)], + rows: [row(4, { deliveryResolution: "cancelled" }), row(5)], hasMore: true, nextBeforeId: 4, } ); expect(refreshed.rows.map((item) => item.inboxId)).toEqual([1, 2, 3, 4, 5]); - expect(refreshed.rows.find((item) => item.inboxId === 4)?.readAt).toBe( - "2026-07-17T00:02:00Z" - ); + expect( + refreshed.rows.find((item) => item.inboxId === 4)?.deliveryResolution + ).toBe("cancelled"); expect(refreshed).toMatchObject({ hasMore: false, nextBeforeId: null }); }); @@ -235,6 +239,29 @@ describe("Agent Org Group Chat durable history", () => { }); }); + it("bounds stacked refresh gaps and falls back to a complete cursor scan", () => { + let model = agentOrgGroupChatHistoryTestApi.applyRefreshPage( + agentOrgGroupChatHistoryTestApi.createHistoryModel( + request.scopeKey, + request.generation + ), + request, + { rows: [row(1)], hasMore: false } + ); + for (let page = 1; page <= 33; page += 1) { + const inboxId = page * 100 + 1; + model = agentOrgGroupChatHistoryTestApi.applyRefreshPage(model, request, { + rows: [row(inboxId)], + hasMore: true, + nextBeforeId: inboxId, + }); + } + + expect(model.continuationFrontiers).toHaveLength(0); + expect(model.scanThroughLoadedRows).toBe(true); + expect(model.nextBeforeId).toBe(3301); + }); + it("ignores a response from a previous session or enablement generation", () => { const current = agentOrgGroupChatHistoryTestApi.applyRefreshPage( agentOrgGroupChatHistoryTestApi.createHistoryModel( @@ -294,4 +321,44 @@ describe("Agent Org Group Chat durable history", () => { rows: [expect.objectContaining({ inboxId: 8 })], }); }); + + it("recognizes cancelled and superseded rows as resolved pending delivery", () => { + const rows = [ + row(10), + row(11, { deliveryResolution: "cancelled" }), + row(12, { deliveryResolution: "superseded" }), + ]; + + expect( + agentOrgGroupChatHistoryTestApi.isGroupChatDeliveryResolved(10, rows) + ).toBe(false); + expect( + agentOrgGroupChatHistoryTestApi.isGroupChatDeliveryResolved(11, rows) + ).toBe(true); + expect( + agentOrgGroupChatHistoryTestApi.isGroupChatDeliveryResolved(12, rows) + ).toBe(true); + expect( + agentOrgGroupChatHistoryTestApi.isGroupChatPendingDeliverySettled( + 10, + { + id: 10, + recipientAgentId: "reviewer-agent", + recipientMemberId: "reviewer", + senderAgentId: "_user", + senderMemberId: null, + recipientName: "Reviewer", + senderName: "User", + displayText: "@Reviewer message-10", + orgRunId: "run-1", + payloadKind: "plain", + requestId: null, + createdAt: "2026-07-17T00:00:10Z", + readAt: null, + deliveryResolution: "cancelled", + }, + rows + ) + ).toBe(true); + }); }); diff --git a/src/engines/ChatPanel/hooks/useAgentOrgGroupChatHistory.ts b/src/engines/ChatPanel/hooks/useAgentOrgGroupChatHistory.ts index f5b1a79b5..fe2c44701 100644 --- a/src/engines/ChatPanel/hooks/useAgentOrgGroupChatHistory.ts +++ b/src/engines/ChatPanel/hooks/useAgentOrgGroupChatHistory.ts @@ -2,9 +2,12 @@ import { useCallback, useEffect, useRef, useState } from "react"; import { type AgentOrgGroupChatHistoryRow, + type AgentOrgInboxPreviewRow, getAgentOrgGroupChatHistoryPage, } from "@src/api/tauri/agent"; +const MAX_HISTORY_GAP_FRONTIERS = 32; + interface AgentOrgGroupChatHistoryState { rows: AgentOrgGroupChatHistoryRow[]; hasMore: boolean; @@ -26,6 +29,7 @@ interface HistoryModel { error: string | null; errorKind: "refresh" | "older" | null; continuationFrontiers: HistoryFrontier[]; + scanThroughLoadedRows: boolean; } interface HistoryFrontier { @@ -54,6 +58,7 @@ function createHistoryModel( error: null, errorKind: null, continuationFrontiers: [], + scanThroughLoadedRows: false, }; } @@ -82,6 +87,8 @@ function mergeRows( ...existing, ...row, readAt: row.readAt ?? existing.readAt, + deliveryResolution: + row.deliveryResolution ?? existing.deliveryResolution, } : row ); @@ -130,15 +137,24 @@ function applyRefreshPage( // before the gap once an older request overlaps already-loaded rows. const adoptsPageFrontier = !model.initialized || model.rows.length === 0 || opensNewGap; - const continuationFrontiers = opensNewGap - ? [ - ...model.continuationFrontiers, - { - hasMore: model.hasMore, - nextBeforeId: model.nextBeforeId, - }, - ] - : model.continuationFrontiers; + const frontierLimitReached = + opensNewGap && + !model.scanThroughLoadedRows && + model.continuationFrontiers.length >= MAX_HISTORY_GAP_FRONTIERS; + // Extremely fragmented histories fall back to walking the server cursor to + // the end. That may refetch bounded pages, but cannot skip rows or grow this + // client-side continuation stack without limit. + const continuationFrontiers = frontierLimitReached + ? [] + : opensNewGap && !model.scanThroughLoadedRows + ? [ + ...model.continuationFrontiers, + { + hasMore: model.hasMore, + nextBeforeId: model.nextBeforeId, + }, + ] + : model.continuationFrontiers; return { ...model, rows: mergeRows(model.rows, page.rows), @@ -151,6 +167,7 @@ function applyRefreshPage( error: model.errorKind === "refresh" ? null : model.error, errorKind: model.errorKind === "refresh" ? null : model.errorKind, continuationFrontiers, + scanThroughLoadedRows: model.scanThroughLoadedRows || frontierLimitReached, }; } @@ -183,7 +200,9 @@ function applyOlderPage( ); const continuationFrontiers = [...model.continuationFrontiers]; const resumedFrontier = - overlapsLoadedRows && continuationFrontiers.length > 0 + !model.scanThroughLoadedRows && + overlapsLoadedRows && + continuationFrontiers.length > 0 ? continuationFrontiers.pop() : undefined; return { @@ -198,6 +217,7 @@ function applyOlderPage( error: model.errorKind === "older" ? null : model.error, errorKind: model.errorKind === "older" ? null : model.errorKind, continuationFrontiers, + scanThroughLoadedRows: model.scanThroughLoadedRows && page.hasMore, }; } @@ -217,6 +237,27 @@ function applyRequestFailure( }; } +export function isGroupChatDeliveryResolved( + inboxId: number, + historyRows: ReadonlyArray +): boolean { + return historyRows.some( + (row) => row.inboxId === inboxId && Boolean(row.deliveryResolution) + ); +} + +export function isGroupChatPendingDeliverySettled( + inboxId: number, + previewRow: AgentOrgInboxPreviewRow | undefined, + historyRows: ReadonlyArray +): boolean { + return Boolean( + (previewRow?.readAt && previewRow.readAt.trim()) || + previewRow?.deliveryResolution || + isGroupChatDeliveryResolved(inboxId, historyRows) + ); +} + /** * Bounded, explicit Group Chat history reader. The frequently-polled Run View * only supplies delivery previews; this hook fetches full user text one page @@ -370,4 +411,6 @@ export const agentOrgGroupChatHistoryTestApi = { beginLoadOlder, applyOlderPage, applyRequestFailure, + isGroupChatDeliveryResolved, + isGroupChatPendingDeliverySettled, }; diff --git a/src/modules/WorkStation/shared/AppSwitcherWrappers.tsx b/src/modules/WorkStation/shared/AppSwitcherWrappers.tsx index 733a88a93..baf9f23de 100644 --- a/src/modules/WorkStation/shared/AppSwitcherWrappers.tsx +++ b/src/modules/WorkStation/shared/AppSwitcherWrappers.tsx @@ -21,6 +21,7 @@ import { sessionMapAtom, workstationActiveSessionIdAtom, } from "@src/store/session"; +import { isAgentOrgMemberEmpty } from "@src/util/agentOrg/memberActivity"; import { getRustAgentType, isCursorIdeSession, @@ -181,12 +182,7 @@ const SimulatorAgentChipComponent: React.FC = () => { (member) => !member.isCoordinator ); const withEmptyState = nonCoordinators.map((member) => { - const hasNoTasksAndNoInbox = - member.activeTaskCount === 0 && - member.pendingTaskCount === 0 && - member.inProgressTaskCount === 0 && - member.completedTaskCount === 0 && - member.inboxActivityCount === 0; + const hasNoTasksAndNoInbox = isAgentOrgMemberEmpty(member); return { member, hasNoTasksAndNoInbox }; }); withEmptyState.sort((a, b) => { diff --git a/src/util/agentOrg/memberActivity.test.ts b/src/util/agentOrg/memberActivity.test.ts new file mode 100644 index 000000000..a45c01b22 --- /dev/null +++ b/src/util/agentOrg/memberActivity.test.ts @@ -0,0 +1,43 @@ +import { describe, expect, it } from "vitest"; + +import { isAgentOrgMemberEmpty } from "./memberActivity"; + +const EMPTY_MEMBER_ACTIVITY = { + activeTaskCount: 0, + pendingTaskCount: 0, + inProgressTaskCount: 0, + completedTaskCount: 0, + inboxActivityCount: 0, + unreadInboxCount: 0, +}; + +describe("isAgentOrgMemberEmpty", () => { + it("treats a worker with no task or inbox state as empty", () => { + expect(isAgentOrgMemberEmpty(EMPTY_MEMBER_ACTIVITY)).toBe(true); + }); + + it("keeps an old unread message switchable outside the recent activity window", () => { + expect( + isAgentOrgMemberEmpty({ + ...EMPTY_MEMBER_ACTIVITY, + inboxActivityCount: 0, + unreadInboxCount: 1, + }) + ).toBe(false); + }); + + it("keeps recent inbox activity and task history switchable", () => { + expect( + isAgentOrgMemberEmpty({ + ...EMPTY_MEMBER_ACTIVITY, + inboxActivityCount: 1, + }) + ).toBe(false); + expect( + isAgentOrgMemberEmpty({ + ...EMPTY_MEMBER_ACTIVITY, + completedTaskCount: 1, + }) + ).toBe(false); + }); +}); diff --git a/src/util/agentOrg/memberActivity.ts b/src/util/agentOrg/memberActivity.ts new file mode 100644 index 000000000..3007f8155 --- /dev/null +++ b/src/util/agentOrg/memberActivity.ts @@ -0,0 +1,30 @@ +import type { AgentOrgRunMemberView } from "@src/api/tauri/agent"; + +type AgentOrgMemberActivity = Pick< + AgentOrgRunMemberView, + | "activeTaskCount" + | "pendingTaskCount" + | "inProgressTaskCount" + | "completedTaskCount" + | "inboxActivityCount" + | "unreadInboxCount" +>; + +/** + * Whether switching to a worker would open a genuinely empty session. + * + * `inboxActivityCount` is intentionally a bounded recent-history window, + * while `unreadInboxCount` is the exact durable total. Both must be empty: + * an old unread row can fall outside the recent window and is still real + * work the user must be able to inspect. + */ +export function isAgentOrgMemberEmpty(member: AgentOrgMemberActivity): boolean { + return ( + member.activeTaskCount === 0 && + member.pendingTaskCount === 0 && + member.inProgressTaskCount === 0 && + member.completedTaskCount === 0 && + member.inboxActivityCount === 0 && + member.unreadInboxCount === 0 + ); +} diff --git a/tests/e2e/specs/core/agent-org-group-chat-ui.spec.mjs b/tests/e2e/specs/core/agent-org-group-chat-ui.spec.mjs index 608bfef37..b2986997e 100644 --- a/tests/e2e/specs/core/agent-org-group-chat-ui.spec.mjs +++ b/tests/e2e/specs/core/agent-org-group-chat-ui.spec.mjs @@ -446,19 +446,14 @@ describe("Agent Org group chat and plan rendered UI", () => { await waitForAgentOrgRunView( sessionId, (view) => - (view?.inbox ?? []).some((row) => { - const payload = parseInboxPayload( - row, - "default coordinator group chat" - ); - return ( + (view?.inbox ?? []).some( + (row) => + row.id === coordinatorInboxRow.id && row.senderAgentId === "_user" && row.senderName === "User" && row.recipientMemberId === AGENT_ORG_COORDINATOR_MEMBER_ID && - row.payloadKind === "plain" && - payload.text === coordinatorMessage - ); - }), + row.payloadKind === "plain" + ), "default coordinator group chat inbox row persisted" ); await assertNoMemberIntervention( diff --git a/tests/e2e/specs/core/agent-org-pause-resume-ui.spec.mjs b/tests/e2e/specs/core/agent-org-pause-resume-ui.spec.mjs index 5007a6a10..31badbd65 100644 --- a/tests/e2e/specs/core/agent-org-pause-resume-ui.spec.mjs +++ b/tests/e2e/specs/core/agent-org-pause-resume-ui.spec.mjs @@ -884,6 +884,7 @@ describe("Agent Org pause, resume, and sidebar rendered UI", () => { ); } + await openRenderedGroupChatView(); await waitForGroupChatPausedBanner( "historical paused run after reopening from sidebar" ); diff --git a/tests/e2e/support/core/agentOrgUiDriver.mjs b/tests/e2e/support/core/agentOrgUiDriver.mjs index 1ac27427a..6fe4ebbb0 100644 --- a/tests/e2e/support/core/agentOrgUiDriver.mjs +++ b/tests/e2e/support/core/agentOrgUiDriver.mjs @@ -1642,33 +1642,64 @@ export async function selectRenderedTurnPageByPreview(previewSnippet, label) { async function waitForAgentOrgMentionMenuOption(memberName, label) { let state = null; - await browser.waitUntil( - async () => { - state = await execJS(` - const menu = document.querySelector('.context-menu'); - const options = Array.from(document.querySelectorAll('[data-testid="agent-org-mention-option"]')); - return { - menuText: menu?.textContent || '', - options: options.map((option) => ({ - text: option.textContent || '', - mentionId: option.getAttribute('data-mention-id') || '', - })), - }; - `); - const hasMember = (state?.options ?? []).some((option) => - String(option.text ?? "").includes(memberName) - ); - const hasNormalContextEntry = String(state?.menuText ?? "").includes( - "Files & Folders" - ); - return hasMember && hasNormalContextEntry; - }, - { - timeout: RENDER_TIMEOUT_MS, - interval: 250, - timeoutMsg: `Agent Org mention menu did not include both member and normal context options for ${label}: ${JSON.stringify(state)}`, - } - ); + try { + await browser.waitUntil( + async () => { + state = await execJS(` + const menus = Array.from(document.querySelectorAll('.context-menu')); + const menu = menus.find((candidate) => candidate.closest('[data-context-menu-portal]')) ?? menus[menus.length - 1] ?? null; + const options = Array.from(menu?.querySelectorAll('[data-testid="agent-org-mention-option"]') ?? []); + return { + menuText: menu?.textContent || '', + options: options.map((option) => ({ + text: option.textContent || '', + mentionId: option.getAttribute('data-mention-id') || '', + })), + }; + `); + const hasMember = (state?.options ?? []).some((option) => + String(option.text ?? "").includes(memberName) + ); + const hasNormalContextEntry = String(state?.menuText ?? "").includes( + "Files & Folders" + ); + return hasMember && hasNormalContextEntry; + }, + { + timeout: RENDER_TIMEOUT_MS, + interval: 250, + timeoutMsg: `Agent Org mention menu did not become ready for ${label}`, + } + ); + } catch (error) { + const diagnostic = await execJS(` + const isVisible = (element) => { + const rect = element.getBoundingClientRect(); + const style = window.getComputedStyle(element); + return rect.width > 0 && rect.height > 0 && style.visibility !== "hidden" && style.display !== "none"; + }; + const shells = Array.from(document.querySelectorAll('[data-testid="chat-input"]')) + .filter(isVisible) + .map((shell, index) => { + const editor = shell.querySelector('[contenteditable="true"]'); + return { + index, + text: editor?.textContent || '', + active: document.activeElement === editor, + memberPills: Array.from(shell.querySelectorAll('[data-composer-pill="true"][data-icon-type="member"]')) + .map((pill) => pill.getAttribute('data-file-name') || ''), + }; + }); + return { + shells, + triggerText: document.querySelector('[data-testid="agent-org-member-switcher-trigger"]')?.textContent || '', + portalPresent: Boolean(document.querySelector('[data-context-menu-portal]')), + }; + `); + throw new Error( + `${error instanceof Error ? error.message : String(error)}; lastMenu=${JSON.stringify(state)}; composer=${JSON.stringify(diagnostic)}` + ); + } } export async function sendRenderedGroupChatMentionPrompt( @@ -1686,7 +1717,37 @@ export async function sendRenderedGroupChatMentionPrompt( if (clearResult !== "typed") { throw new Error(`chat input did not clear for ${label}: ${clearResult}`); } + const focusResult = await execJS(` + const visibleInputShells = Array.from(document.querySelectorAll('[data-testid="chat-input"]')).filter((inputShell) => { + const rect = inputShell.getBoundingClientRect(); + const style = window.getComputedStyle(inputShell); + return rect.width > 0 && rect.height > 0 && style.visibility !== "hidden" && style.display !== "none"; + }); + const editor = visibleInputShells[visibleInputShells.length - 1]?.querySelector('[contenteditable="true"]') ?? null; + if (!editor) return "missing"; + editor.focus(); + return document.activeElement === editor ? "focused" : "focus-failed"; + `); + if (focusResult !== "focused") { + throw new Error(`chat input did not focus for ${label}: ${focusResult}`); + } await browser.keys("@"); + await browser.pause(100); + let mentionTriggerText = await execJS(js.editorText(inputSelector)); + if (!String(mentionTriggerText ?? "").includes("@")) { + const mentionTriggerResult = await execJS(js.type(inputSelector, "@")); + if (mentionTriggerResult !== "typed") { + throw new Error( + `chat input did not accept @ mention trigger for ${label}: ${mentionTriggerResult}` + ); + } + mentionTriggerText = await execJS(js.editorText(inputSelector)); + } + if (!String(mentionTriggerText ?? "").includes("@")) { + throw new Error( + `chat input did not render @ mention trigger for ${label}: ${mentionTriggerText}` + ); + } await waitForAgentOrgMentionMenuOption(memberName, label); const clickResult = await execJS(` const options = Array.from(document.querySelectorAll('[data-testid="agent-org-mention-option"]')); @@ -1882,31 +1943,43 @@ export async function waitForGroupChatPendingTarget(targetName, label) { export async function waitForGroupChatPausedBanner(label) { let state = null; - await browser.waitUntil( - async () => { - state = await execJS(` - const banner = document.querySelector('[data-testid="agent-org-group-chat-paused-banner"]'); - const resume = document.querySelector('[data-testid="agent-org-group-chat-resume-button"]'); - return { - bannerText: banner ? (banner.textContent || '') : '', - resumeVisible: Boolean(resume), - resumeDisabled: resume ? Boolean(resume.disabled) : null, - }; - `); - const text = String(state?.bannerText ?? "").toLowerCase(); - return ( - text.includes("new work is paused") && - text.includes("pause stops active replies") && - state?.resumeVisible === true && - state?.resumeDisabled === false - ); - }, - { - timeout: RENDER_TIMEOUT_MS, - interval: 250, - timeoutMsg: `group chat paused banner did not render for ${label}: ${JSON.stringify(state)}`, - } - ); + try { + await browser.waitUntil( + async () => { + try { + state = await execJS(` + const banner = document.querySelector('[data-testid="agent-org-group-chat-paused-banner"]'); + const resume = document.querySelector('[data-testid="agent-org-group-chat-resume-button"]'); + return { + bannerText: banner ? (banner.textContent || '') : '', + resumeVisible: Boolean(resume), + resumeDisabled: resume ? Boolean(resume.disabled) : null, + bodyText: document.body?.textContent?.slice(0, 1200) || '', + }; + `); + } catch (error) { + state = { error: String(error?.message ?? error) }; + return false; + } + const text = String(state?.bannerText ?? "").toLowerCase(); + return ( + text.includes("new work is paused") && + text.includes("pause stops active replies") && + state?.resumeVisible === true && + state?.resumeDisabled === false + ); + }, + { + timeout: RENDER_TIMEOUT_MS, + interval: 250, + timeoutMsg: `group chat paused banner did not render for ${label}`, + } + ); + } catch (_error) { + throw new Error( + `group chat paused banner did not render for ${label}: ${JSON.stringify(state)}` + ); + } } export async function clickGroupChatResumeButton(label) {