Reapply #167 for follow-up fixes

CHANGELOG.md

@@ -1,5 +1,45 @@
 # Changelog
 
+## Unreleased
+
+### Public Upstream Sync - 2026-04-11
+
+#### Included in this upstream-safe branch
+- Chat composer queue and steer workflow:
+  - Added durable queued-turn data structures and queue reconciliation helpers.
+  - Added steer promotion, paused queue handling, queued-turn resume, and queue cleanup after turn settlement.
+  - Added queue/session-scope tests for `codexQueue`, `sessionLoadGuards`, `sessionSnapshotCache`, and `sessionScope`.
+- Session stability and protocol hardening:
+  - Added explicit WebSocket lifecycle protocol messages: `session-accepted`, `session-busy`, `session-state-changed`.
+  - Added project-scoped event enrichment for session lifecycle payloads to avoid cross-session/cross-project ambiguity.
+  - Added lifecycle projection from provider completion/error messages into normalized session-state events.
+  - Added session-created `projectName` metadata for Claude/Cursor/Gemini session initialization paths.
+  - Removed unused `projectPath` from `session-accepted` payloads; downstream should rely on `projectName` + scoped identifiers.
+- Nano chain compatibility:
+  - UI provider/model selection no longer surfaces Nano by default in the upstream-safe branch, while server-side `nano-command` handling remains for compatibility.
+  - Preserved Nano command path and active session reporting in WebSocket session status flows.
+  - Ensured session lifecycle protocol is emitted consistently for Nano just like other providers.
+
+#### Migration notes
+- **Message cache key change**: Session message cache keys are now scoped by provider (`chat_messages_{project}_{provider}_{session}`). Existing localStorage entries keyed the old way (`chat_messages_{project}_{session}`) will not be read by default. To migrate, set `allowLegacyFallback: true` when calling `getSessionMessageCacheKeys()`. Users upgrading from builds prior to this sync may see empty transcript history on first load for previously-active sessions; the data is still in localStorage and can be recovered by enabling the legacy fallback or manually re-keying the entries.
+
+#### Explicitly excluded (local/private only, not part of upstream PR)
+- Codex-only product strategy and provider lock-in controls.
+- External auth / license refresh-heartbeat-offline-grace stack.
+- Project root hard-cut and Codex session backfill private policy behavior.
+- LingZhi/LingzhiLab branding replacements and private demo/link route changes.
+
+#### Cross-platform notes
+- No platform-specific server behavior was hardcoded for this sync.
+- Session protocol additions are transport-level and provider-agnostic; Windows-specific stability paths do not alter macOS behavior.
+
+#### Validation
+- Passed: `node --check server/index.js`
+- Passed: `node --check server/claude-sdk.js`
+- Passed: `node --check server/cursor-cli.js`
+- Passed: `node --check server/gemini-cli.js`
+- Not runnable in current environment (missing local dev dependencies): `vitest`, `tsc`
+
 ## Dr. Claw v1.1.1 - 2026-03-30
 
 ### Highlights

README.md

@@ -644,7 +644,7 @@ When you first open Dr. Claw you will see the **Projects** sidebar. You have two
 - **Open an existing project** — Dr. Claw auto-discovers registered projects and linked sessions from Claude Code, Codex, and Gemini.
 - **Create a new project** — Click the **"+"** button, choose a directory on your machine, and Dr. Claw will set up the workspace: agent folders such as `.claude/`, `.agents/`, `.gemini/`, standard workspace metadata, linked `skills/` directories, preset research dirs (`Survey/references`, `Survey/reports`, `Ideation/ideas`, `Ideation/references`, `Experiment/code_references`, `Experiment/datasets`, `Experiment/core_code`, `Experiment/analysis`, `Publication/paper`, `Promotion/homepage`, `Promotion/slides`, `Promotion/audio`, `Promotion/video`), and **instance.json** at the project root with absolute paths for those directories. Cursor agent support is coming soon.
 
-> **Default project storage path:** New projects are stored under `~/dr-claw` by default. You can change this in **Settings → Appearance → Default Project Path**, or set the `WORKSPACES_ROOT` environment variable. The setting is persisted in `~/.claude/project-config.json`.
+> **Default project storage path:** New projects are stored under `~/dr-claw` by default. You can change this in **Settings → Appearance → Default Project Path**, or set the `WORKSPACES_ROOT` environment variable. The setting is persisted in `~/.dr-claw/project-config.json` (with automatic one-time migration from `~/.claude/project-config.json`).
 
 </details>
 

README.zh-CN.md

@@ -640,7 +640,7 @@ Dr. Claw 的核心功能是 **Research Lab**。
 - **打开已有项目** — Dr. Claw 会自动发现已注册项目，以及来自 Claude Code、Codex、Gemini 的关联会话。
 - **创建新项目** — 点击 **"+"** 按钮，选择本机的一个目录，Dr. Claw 会创建：`.claude/`、`.agents/`、`.gemini/` 等 Agent 目录、标准工作区元数据、链接的 `skills/` 目录、预设研究目录（`Survey/references`、`Survey/reports`、`Ideation/ideas`、`Ideation/references`、`Experiment/code_references`、`Experiment/datasets`、`Experiment/core_code`、`Experiment/analysis`、`Publication/paper`、`Promotion/homepage`、`Promotion/slides`、`Promotion/audio`、`Promotion/video`），以及项目根目录下的 **instance.json**（上述目录的绝对路径写入其中）。Cursor Agent 支持即将推出。
 
-> **默认项目存储路径：** 新项目默认存储在 `~/dr-claw` 目录下。可在 **Settings → Appearance → Default Project Path** 中修改，也可通过环境变量 `WORKSPACES_ROOT` 设置。该配置持久化在 `~/.claude/project-config.json` 中。
+> **默认项目存储路径：** 新项目默认存储在 `~/dr-claw` 目录下。可在 **Settings → Appearance → Default Project Path** 中修改，也可通过环境变量 `WORKSPACES_ROOT` 设置。该配置持久化在 `~/.dr-claw/project-config.json` 中（会自动一次性迁移 `~/.claude/project-config.json`）。
 
 </details>
 

server/__tests__/codex-session-events.test.mjs

@@ -0,0 +1,36 @@
+import { describe, expect, it } from 'vitest';
+
+import { buildCodexSessionCreatedEvent } from '../utils/codexSessionEvents.js';
+
+describe('codex session event payloads', () => {
+  it('includes projectName when provided', () => {
+    const projectName = 'C--Users-test-user-dr-claw-project';
+    const event = buildCodexSessionCreatedEvent({
+      sessionId: '019d82e8-1ee3-7860-baa1-24603f424ade',
+      sessionMode: 'research',
+      projectName,
+    });
+
+    expect(event).toEqual({
+      type: 'session-created',
+      sessionId: '019d82e8-1ee3-7860-baa1-24603f424ade',
+      provider: 'codex',
+      mode: 'research',
+      projectName,
+    });
+  });
+
+  it('keeps backward-compatible payload shape when projectName is missing', () => {
+    const event = buildCodexSessionCreatedEvent({
+      sessionId: 'session-no-project',
+      sessionMode: 'workspace_qa',
+    });
+
+    expect(event).toEqual({
+      type: 'session-created',
+      sessionId: 'session-no-project',
+      provider: 'codex',
+      mode: 'workspace_qa',
+    });
+  });
+});

server/__tests__/gemini-session-index.test.mjs

@@ -8,12 +8,57 @@ const originalUserProfile = process.env.USERPROFILE;
 const originalDatabasePath = process.env.DATABASE_PATH;
 
 let tempRoot = null;
+let activeDatabaseModule = null;
+
+function sleep(ms) {
+  return new Promise((resolve) => setTimeout(resolve, ms));
+}
+
+async function closeTestDatabase() {
+  if (!activeDatabaseModule?.db?.close) {
+    return;
+  }
+
+  const maxAttempts = 6;
+  for (let attempt = 1; attempt <= maxAttempts; attempt += 1) {
+    try {
+      activeDatabaseModule.db.close();
+      activeDatabaseModule = null;
+      return;
+    } catch (error) {
+      if (attempt === maxAttempts) {
+        throw error;
+      }
+      await sleep(30 * attempt);
+    }
+  }
+}
+
+async function removeTempRootWithRetry(targetPath) {
+  if (!targetPath) {
+    return;
+  }
+
+  const maxAttempts = 8;
+  for (let attempt = 1; attempt <= maxAttempts; attempt += 1) {
+    try {
+      await rm(targetPath, { recursive: true, force: true });
+      return;
+    } catch (error) {
+      if (error?.code !== 'EBUSY' || attempt === maxAttempts) {
+        throw error;
+      }
+      await sleep(50 * attempt);
+    }
+  }
+}
 
 async function loadTestModules() {
   vi.resetModules();
   const projects = await import('../projects.js');
   const database = await import('../database/db.js');
   await database.initializeDatabase();
+  activeDatabaseModule = database;
   return { projects, database };
 }
 
@@ -26,6 +71,8 @@ describe('Gemini API session indexing', () => {
   });
 
   afterEach(async () => {
+    await closeTestDatabase();
+
     vi.resetModules();
 
     if (originalHome === undefined) delete process.env.HOME;
@@ -38,7 +85,7 @@ describe('Gemini API session indexing', () => {
     else process.env.DATABASE_PATH = originalDatabasePath;
 
     if (tempRoot) {
-      await rm(tempRoot, { recursive: true, force: true });
+      await removeTempRootWithRetry(tempRoot);
       tempRoot = null;
     }
   });

server/__tests__/project-config-path.test.mjs

@@ -0,0 +1,134 @@
+import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
+import { mkdir, mkdtemp, readFile, rm, writeFile } from 'fs/promises';
+import os from 'os';
+import path from 'path';
+
+const originalHome = process.env.HOME;
+const originalUserProfile = process.env.USERPROFILE;
+const originalDatabasePath = process.env.DATABASE_PATH;
+
+let tempRoot = null;
+let activeDatabaseModule = null;
+
+function sleep(ms) {
+  return new Promise((resolve) => setTimeout(resolve, ms));
+}
+
+async function closeTestDatabase() {
+  if (!activeDatabaseModule?.db?.close) {
+    return;
+  }
+
+  const maxAttempts = 6;
+  for (let attempt = 1; attempt <= maxAttempts; attempt += 1) {
+    try {
+      activeDatabaseModule.db.close();
+      activeDatabaseModule = null;
+      return;
+    } catch (error) {
+      if (attempt === maxAttempts) {
+        throw error;
+      }
+      await sleep(30 * attempt);
+    }
+  }
+}
+
+async function removeTempRootWithRetry(targetPath) {
+  if (!targetPath) {
+    return;
+  }
+
+  const maxAttempts = 8;
+  for (let attempt = 1; attempt <= maxAttempts; attempt += 1) {
+    try {
+      await rm(targetPath, { recursive: true, force: true });
+      return;
+    } catch (error) {
+      if (error?.code !== 'EBUSY' || attempt === maxAttempts) {
+        throw error;
+      }
+      await sleep(50 * attempt);
+    }
+  }
+}
+
+async function loadProjectsModule() {
+  vi.resetModules();
+  const projects = await import('../projects.js');
+  activeDatabaseModule = await import('../database/db.js');
+  return projects;
+}
+
+describe('project config path migration', () => {
+  beforeEach(async () => {
+    tempRoot = await mkdtemp(path.join(os.tmpdir(), 'dr-claw-project-config-'));
+    process.env.HOME = tempRoot;
+    process.env.USERPROFILE = tempRoot;
+    process.env.DATABASE_PATH = path.join(tempRoot, 'db', 'auth.db');
+  });
+
+  afterEach(async () => {
+    await closeTestDatabase();
+    vi.resetModules();
+
+    if (originalHome === undefined) delete process.env.HOME;
+    else process.env.HOME = originalHome;
+
+    if (originalUserProfile === undefined) delete process.env.USERPROFILE;
+    else process.env.USERPROFILE = originalUserProfile;
+
+    if (originalDatabasePath === undefined) delete process.env.DATABASE_PATH;
+    else process.env.DATABASE_PATH = originalDatabasePath;
+
+    if (tempRoot) {
+      await removeTempRootWithRetry(tempRoot);
+      tempRoot = null;
+    }
+  });
+
+  it('prefers ~/.dr-claw/project-config.json when both current and legacy files exist', async () => {
+    const currentConfigPath = path.join(tempRoot, '.dr-claw', 'project-config.json');
+    const legacyConfigPath = path.join(tempRoot, '.claude', 'project-config.json');
+
+    await mkdir(path.dirname(currentConfigPath), { recursive: true });
+    await mkdir(path.dirname(legacyConfigPath), { recursive: true });
+
+    await writeFile(currentConfigPath, JSON.stringify({ marker: 'current', _workspacesRoot: path.join(tempRoot, 'dr-claw') }, null, 2), 'utf8');
+    await writeFile(legacyConfigPath, JSON.stringify({ marker: 'legacy', _workspacesRoot: path.join(tempRoot, 'legacy-root') }, null, 2), 'utf8');
+
+    const projects = await loadProjectsModule();
+    const config = await projects.loadProjectConfig();
+
+    expect(config.marker).toBe('current');
+    expect(config._workspacesRoot).toBe(path.join(tempRoot, 'dr-claw'));
+  });
+
+  it('migrates legacy ~/.claude/project-config.json into ~/.dr-claw/project-config.json once', async () => {
+    const currentConfigPath = path.join(tempRoot, '.dr-claw', 'project-config.json');
+    const legacyConfigPath = path.join(tempRoot, '.claude', 'project-config.json');
+    const legacyConfig = {
+      marker: 'legacy-only',
+      _workspacesRoot: path.join(tempRoot, 'workspaces'),
+    };
+
+    await mkdir(path.dirname(legacyConfigPath), { recursive: true });
+    await writeFile(legacyConfigPath, JSON.stringify(legacyConfig, null, 2), 'utf8');
+
+    const projects = await loadProjectsModule();
+    const loadedConfig = await projects.loadProjectConfig();
+    expect(loadedConfig).toEqual(legacyConfig);
+
+    const migratedRaw = await readFile(currentConfigPath, 'utf8');
+    expect(JSON.parse(migratedRaw)).toEqual(legacyConfig);
+
+    const updated = { ...loadedConfig, marker: 'saved-to-current' };
+    await projects.saveProjectConfig(updated);
+
+    const currentAfterSave = JSON.parse(await readFile(currentConfigPath, 'utf8'));
+    expect(currentAfterSave.marker).toBe('saved-to-current');
+
+    const legacyAfterSave = JSON.parse(await readFile(legacyConfigPath, 'utf8'));
+    expect(legacyAfterSave.marker).toBe('legacy-only');
+  });
+});