Skip to content

fix(memory-sync): migrate Linear provider to memory_tree (#2885)#3018

Merged
senamakel merged 6 commits into
tinyhumansai:mainfrom
YOMXXX:fix/2885-linear-memory-tree
May 30, 2026
Merged

fix(memory-sync): migrate Linear provider to memory_tree (#2885)#3018
senamakel merged 6 commits into
tinyhumansai:mainfrom
YOMXXX:fix/2885-linear-memory-tree

Conversation

@YOMXXX
Copy link
Copy Markdown
Contributor

@YOMXXX YOMXXX commented May 30, 2026

Summary

  • Adds Linear issue -> memory_tree ingest plumbing with stable linear:{connection_id}:{issue_id} source ids.
  • Pre-completes walkthrough state in the Playwright core-mode helper so Joyride does not intercept web E2E clicks.
  • Replaces the Linear provider's legacy persist_single_item path with ingest_document-backed memory_tree writes.
  • Makes linear:* source trees visible to document-kind retrieval filters.
  • Adds regression coverage for memory_tree chunk writes and edited-issue re-ingest replacement.

Problem

  • feat(memory-sync): migrate Notion/ClickUp/Linear/GitHub Composio providers off UnifiedMemory #2885 tracks Composio providers that still write legacy UnifiedMemory/memory_docs while modern retrieval reads memory_tree.
  • Linear was still calling persist_single_item, so synced Linear issues could be invisible to memory.search, tree.read_chunk, tree.browse, and related memory_tree retrieval surfaces.
  • A partial migration would still miss source_kind=document scans unless linear:* scopes are classified as document-like source trees.

Solution

  • Added linear::ingest to render one Linear issue payload into a DocumentInput and call memory::ingest_pipeline::ingest_document.
  • Uses a stable source id scoped by connection and issue id; edited re-ingest deletes prior chunks for that source before calling the document ingest pipeline.
  • Kept existing Linear SyncState behavior for cursoring, edit-aware deduplication, daily budget, pagination, and KV persistence.
  • Added linear to retrieval's document platform aliases so source_kind=document selects Linear source trees.

Submission Checklist

If a section does not apply to this change, mark the item as N/A with a one-line reason. Do not delete items.

  • Tests added or updated (happy path + at least one failure / edge case) per Testing Strategy
  • Diff coverage >= 80% — changed lines (Vitest + cargo-llvm-cov merged via diff-cover) meet the gate enforced by .github/workflows/pr-ci.yml. Run pnpm test:coverage and pnpm test:rust locally; PRs below 80% on changed lines will not merge. N/A locally: targeted Rust coverage added and pnpm debug rust linear run; full coverage is left to CI for this Rust-only narrow change.
  • Coverage matrix updated — N/A: no feature matrix row added/removed/renamed; this fixes an existing provider persistence backend.
  • All affected feature IDs from the matrix are listed in the PR description under ## Related — N/A: no matrix feature ID applies.
  • No new external network dependencies introduced (mock backend used per Testing Strategy)
  • Manual smoke checklist updated if this touches release-cut surfaces (docs/RELEASE-MANUAL-SMOKE.md) — N/A: internal provider memory persistence path only.
  • Linked issue closed via Closes #NNN in the ## Related section — N/A: this PR references feat(memory-sync): migrate Notion/ClickUp/Linear/GitHub Composio providers off UnifiedMemory #2885 but does not close it because ClickUp remains and GitHub is covered separately by feat(composio): migrate GitHub provider to memory_tree pipeline (#2885) #2889.

Impact

  • Runtime/platform impact: Rust core memory sync only; no frontend, Tauri shell, or mobile surface changes.
  • Compatibility: Linear sync state keys and cursor semantics are preserved. Previously synced issue revisions remain in SyncState; new/edited Linear issues now write memory_tree chunks.
  • Performance: first-time ingest uses indexed source registration checks; edited re-ingest deletes prior chunks for the same source id before re-writing.
  • Security/privacy: unchanged Linear scope; still syncs issues assigned to the connected user.

Related


AI Authored PR Metadata (required for Codex/Linear PRs)

Keep this section for AI-authored PRs. For human-only PRs, mark each field N/A.

Linear Issue

  • Key: N/A
  • URL: N/A

Commit & Branch

  • Branch: fix/2885-linear-memory-tree
  • Commit SHA: 5ca76a9b9b082717bd605851e80949bc1f5fe890

Validation Run

  • pnpm --filter openhuman-app test:unit test/core-rpc-playwright-helper.test.ts — passed; includes RED/GREEN regression for walkthrough completion storage.
  • pnpm --filter openhuman-app compile — passed.
  • pnpm --filter openhuman-app exec eslint test/core-rpc-playwright-helper.test.ts test/playwright/helpers/core-rpc.ts — passed.
  • pnpm --filter openhuman-app exec prettier --check test/core-rpc-playwright-helper.test.ts test/playwright/helpers/core-rpc.ts — passed.
  • GGML_NATIVE=OFF pnpm --filter openhuman-app test:e2e:web:build — passed locally; initial run without GGML_NATIVE=OFF hit the known macOS Apple Silicon whisper-rs -mcpu=native issue documented in AGENTS.md.
  • Focused Playwright probe: bash app/scripts/e2e-web-session.sh ... -g "shows expired-auth state without logging out|connected Gmail exposes management affordances|renders the memory workspace and actions toolbar|mcp tab shows the placeholder panel" — reproduced that the Gmail management click path now passes; remaining local failures were auth/default-tab related, not Joyride click interception.
  • Pre-push hook — passed after removing local generated app/dist-web / app/test-results; covered format check, lint (0 errors, existing warnings), compile, Tauri/Rust check.
  • Focused Rust tests from the Linear memory_tree migration: pnpm debug rust linear — 45 passed; pnpm debug rust scope_prefix_matching_known_platforms — 1 passed.

Validation Blocked

  • command: N/A
  • error: N/A
  • impact: N/A

Behavior Changes

  • Intended behavior change: Linear issue sync writes to memory_tree instead of the legacy memory_docs persistence path.
  • User-visible effect: Linear issue memories become available to modern memory_tree retrieval surfaces after sync.

Parity Contract

  • Legacy behavior preserved: Linear API pagination, assigned-user privacy scope, edit-aware sync key, cursor advancement, daily request budget, and KV sync-state persistence are unchanged.
  • Guard/fallback/dispatch parity checks: ingest failures keep the previous cursor for retry; edited source ids remove prior chunks before re-ingest; linear:* scopes are selected by document-kind retrieval.

Duplicate / Superseded PR Handling

Summary by CodeRabbit

  • New Features

    • Linear issues are now ingested per-issue into the memory-tree for improved searchability and retrieval.
    • Re-ingesting an updated Linear issue replaces prior content to prevent duplicates.
    • Linear platform IDs are recognized so issues are indexed with platform classification.
  • Tests

    • Added unit and end-to-end tests for ID stability, timestamp parsing, body rendering, and re-ingestion behavior.
    • Added a browser test to ensure walkthrough state is pre-seeded when seeding core mode.

Review Change Stack

@YOMXXX YOMXXX requested a review from a team May 30, 2026 12:06
@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 30, 2026

📝 Walkthrough

Walkthrough

Adds a Linear-to-memory-tree ingest module, updates the LinearProvider sync to ingest each issue into the memory-tree (replacing prior single-document persistence), registers linear as a document platform, and seeds Playwright core-mode tests to mark the walkthrough completed.

Changes

Linear provider migration to memory-tree ingest

Layer / File(s) Summary
Linear ingest module implementation and tests
src/openhuman/memory_sync/composio/providers/linear/ingest.rs, src/openhuman/memory_sync/composio/providers/linear/mod.rs
New module defines LINEAR_PLATFORM, DEFAULT_TAGS, linear_source_id(), Markdown body rendering (title + JSON code block), RFC3339 updatedAt parsing with fallback, and ingest_issue_into_memory_tree() which deletes prior chunks for the same source before ingesting. Includes unit and integration tests for ID stability, timestamp fallback, body format, successful chunk writes, and re-ingest replace behavior.
LinearProvider sync loop migration to memory-tree ingest
src/openhuman/memory_sync/composio/providers/linear/provider.rs
Sync loop now calls ingest_issue_into_memory_tree per-issue instead of persisting single documents. Documentation, imports, call sites, and error logging updated; normalize_linear_issue uses the imported extract_item_id.
Memory-tree source scope classification for linear platform
src/openhuman/memory_tree/retrieval/source.rs
Adds linear to PLATFORM_KINDS so linear:<...> scopes are treated as document sources; test updated accordingly.

Playwright core-mode test helpers

Layer / File(s) Summary
Seed core-mode walkthrough state in browser tests
app/test/core-rpc-playwright-helper.test.ts, app/test/playwright/helpers/core-rpc.ts
Test helpers now set openhuman:walkthrough_completed and remove openhuman:walkthrough_pending via page.addInitScript and in-page evaluation; test added to assert seeded localStorage state.

Sequence Diagram(s)

sequenceDiagram
  participant LinearProvider
  participant ingest_issue_into_memory_tree
  participant ChunkStore
  participant IngestPipeline
  participant MemoryTree

  LinearProvider->>ingest_issue_into_memory_tree: call(issue_id, title, updated_time, issue JSON)
  ingest_issue_into_memory_tree->>ChunkStore: query existing chunks by source_id
  ChunkStore-->>ingest_issue_into_memory_tree: existing chunk metadata / none
  ingest_issue_into_memory_tree->>ChunkStore: delete existing chunks (if found)
  ingest_issue_into_memory_tree->>IngestPipeline: ingest DocumentInput (provider,title,body,modified_at,tags,source)
  IngestPipeline->>MemoryTree: create document and chunks
  MemoryTree-->>IngestPipeline: upsert_count
  IngestPipeline-->>ingest_issue_into_memory_tree: Result<usize>
  ingest_issue_into_memory_tree-->>LinearProvider: return chunk count / error
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related issues

Possibly related PRs

  • tinyhumansai/openhuman#2452: Overlaps with Linear sync logic changes moving persistence to the memory_tree ingest path.
  • tinyhumansai/openhuman#2889: Similar migration for another Composio provider to the memory_tree ingest pipeline (deleting prior chunks on re-ingest).

Suggested labels

bug

Suggested reviewers

  • sanil-23

Poem

🐰 I munched on JSON, stitched a tree of thought,
Namespaced IDs and timestamps neatly brought,
Old chunks cleared, new pages placed in line,
Sync hums softly, memory-tree roots entwine,
Hop—ingest complete, the rabbit stamps it "done."

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title directly describes the main change: migrating the Linear provider from legacy memory_docs to the modern memory_tree pipeline.
Docstring Coverage ✅ Passed Docstring coverage is 87.50% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 ESLint

If the error stems from missing dependencies, add them to the package.json file. For unrecoverable errors (e.g., due to private dependencies), disable the tool in the CodeRabbit configuration.

ESLint skipped: no ESLint configuration detected in root package.json. To enable, add eslint to devDependencies.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot added rust-core Core Rust runtime in src/: CLI, core_server, shared infrastructure. memory Memory store, memory tree, recall, summarization, and embeddings in src/openhuman/memory/. working A PR that is being worked on by the team. labels May 30, 2026
coderabbitai[bot]
coderabbitai Bot previously approved these changes May 30, 2026
coderabbitai[bot]
coderabbitai Bot previously approved these changes May 30, 2026
oxoxDev
oxoxDev previously approved these changes May 30, 2026
Copy link
Copy Markdown
Contributor

@oxoxDev oxoxDev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approve. Clean Linear → memory_tree migration. Reviewed all 4 files:

  • ingest.rs: stable linear:{conn}:{issue} source id; edit re-ingest deletes prior chunks via is_source_ingested guard before delete_chunks_by_source, then ingest_document. The blocking sqlite calls are correctly wrapped in spawn_blocking, and the error path preserves the anyhow chain ({err:#}).
  • provider.rs: persist_single_itemingest_issue_into_memory_tree swap with correct arg order (title, updated.as_deref(), issue); failed ingest keeps the cursor for retry (parity preserved).
  • source.rs: ("linear", "document") alias + scope-match test so source_kind=document selects Linear trees.
  • Tests cover stable-id, timestamp parse (valid/invalid/missing), body render, first ingest writes chunks, and edited re-ingest replaces rather than appends.

All real gates green; CI red is only the unrelated Gmail-E2E flake.

@YOMXXX YOMXXX dismissed stale reviews from oxoxDev and coderabbitai[bot] via 5ca76a9 May 30, 2026 14:46
@coderabbitai coderabbitai Bot added the bug label May 30, 2026
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
app/test/core-rpc-playwright-helper.test.ts (1)

12-13: ⚡ Quick win

Use an interface for the init-script argument shape.

The inline object shape for args should be extracted to an interface to match the TS guideline.

Proposed refactor
+interface SeedBrowserCoreModeArgs {
+  rpcUrl: string;
+  token: string;
+}
+
 const page = {
   async addInitScript(
-    script: (args: { rpcUrl: string; token: string }) => void,
-    args: { rpcUrl: string; token: string }
+    script: (args: SeedBrowserCoreModeArgs) => void,
+    args: SeedBrowserCoreModeArgs
   ) {
     script(args);
   },
 } as unknown as Page;

As per coding guidelines, "Prefer interface for defining object shapes in TypeScript" and "Use interface over type for defining object shapes in TypeScript".

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@app/test/core-rpc-playwright-helper.test.ts` around lines 12 - 13, Extract
the inline object type for the init script into a named interface (e.g.,
RpcInitArgs) and replace both occurrences of the inline shape in the function
signature and parameter list—specifically where "script: (args: { rpcUrl:
string; token: string }) => void" and "args: { rpcUrl: string; token: string }"
appear—so the signature becomes "script: (args: RpcInitArgs) => void" and the
parameter uses "args: RpcInitArgs"; ensure the interface exports/exists in the
same module and includes rpcUrl and token string properties.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@app/test/core-rpc-playwright-helper.test.ts`:
- Around line 12-13: Extract the inline object type for the init script into a
named interface (e.g., RpcInitArgs) and replace both occurrences of the inline
shape in the function signature and parameter list—specifically where "script:
(args: { rpcUrl: string; token: string }) => void" and "args: { rpcUrl: string;
token: string }" appear—so the signature becomes "script: (args: RpcInitArgs) =>
void" and the parameter uses "args: RpcInitArgs"; ensure the interface
exports/exists in the same module and includes rpcUrl and token string
properties.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 1e8e5e96-a6c3-40d4-8aec-01644e00760e

📥 Commits

Reviewing files that changed from the base of the PR and between 349fd19 and 5ca76a9.

📒 Files selected for processing (2)
  • app/test/core-rpc-playwright-helper.test.ts
  • app/test/playwright/helpers/core-rpc.ts
✅ Files skipped from review due to trivial changes (1)
  • app/test/playwright/helpers/core-rpc.ts

@senamakel
Copy link
Copy Markdown
Member

f**k yeah love linear.

@senamakel senamakel merged commit 4d3cc4f into tinyhumansai:main May 30, 2026
22 of 26 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug memory Memory store, memory tree, recall, summarization, and embeddings in src/openhuman/memory/. rust-core Core Rust runtime in src/: CLI, core_server, shared infrastructure. working A PR that is being worked on by the team.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants