Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -152,7 +152,7 @@ Rules:
Agents can and should self-test the running app — don't stop at a successful compile. The fast path skips the slow parts (web login, sidebar click-through):

1. **Build + launch a named bundle:** `cd desktop/macos && OMI_APP_NAME="omi-<feature>" ./run.sh` (add `OMI_SKIP_TUNNEL=1` for a local backend without a tunnel; `OMI_SKIP_BACKEND=1 OMI_DESKTOP_API_URL=…` to point at a remote backend).
2. **Boot signed-in (no browser):** sign into "Omi Dev" once, then clone the session into the named bundle **before launch** (UserDefaults is read at startup):
2. **Boot signed-in (no browser):** sign into "Omi Dev" once; `./run.sh` auto-clones auth/onboarding into named bundles **before launch** (UserDefaults is read at startup). To do it manually:
```bash
cd desktop/macos && ./scripts/omi-auth-dump.sh # capture the Omi Dev session
./scripts/omi-auth-seed.sh com.omi.omi-<feature> # replay into the test bundle
Expand Down
4 changes: 2 additions & 2 deletions desktop/macos/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -127,9 +127,9 @@ This creates `/Applications/omi-fix-rewind.app` with bundle ID `com.omi.omi-fix-
- NEVER use the default `./run.sh` (which overwrites "Omi Dev") when testing a specific feature — always set `OMI_APP_NAME`
- **ALWAYS prefix the name with `omi-`** (e.g., `omi-fix-rewind`, `omi-6512-polling`, `omi-vision-test`) so named bundles are visually grouped in `/Applications/` alongside "Omi Dev" and "Omi Beta"
- Keep the name short and descriptive (it becomes both the app name and bundle ID suffix)
- The named bundle gets its own permissions, database, and auth state — the user may need to re-grant permissions and sign in
- The named bundle gets its own permissions and database. `./run.sh` auto-seeds auth/onboarding from "Omi Dev" unless `OMI_SKIP_AUTH_SEED=1` is set.
- To connect agent-swift: `agent-swift connect --bundle-id com.omi.omi-fix-rewind`
- **Skip the web login:** sign into "Omi Dev" once, then `./scripts/omi-auth-dump.sh && ./scripts/omi-auth-seed.sh com.omi.omi-fix-rewind` clones the session so the named bundle boots signed-in
- **Skip the web login:** sign into "Omi Dev" once; named bundles launched by `./run.sh` clone that session before launch.
- **Jump to a screen without clicking:** the automation bridge auto-enables on non-prod bundles — `./scripts/omi-ctl navigate <screen>` (e.g. `rewind`, `memories`, `settings rewind`). See "Fast-Path for Local Iteration" in `e2e/SKILL.md`.

### After Implementing Changes
Expand Down
65 changes: 6 additions & 59 deletions desktop/macos/Desktop/Sources/Chat/ChatPrompts.swift
Original file line number Diff line number Diff line change
Expand Up @@ -273,6 +273,11 @@ struct ChatPrompts {
* "Do I like dogs?" → get_memories_tool (FACT/preference)
* "When did a dog bite me?" → search_conversations_tool (EVENT)

**Task-Agent Awareness:**
- Omi can run local task-chat agents/subagents in the desktop task panel. If the user says "your subagents", "task agents", "running agents", or mentions task-agent errors/timeouts, do NOT deny that you have subagents.
- Call get_task_agent_status to inspect the current Omi task-agent registry before answering.
- If a task agent timed out or failed, summarize the affected task and error, then suggest a concrete retry/recovery step.

**Limit and Transcript Guidance:**
- **Summarization queries** ("summarize my week", "recap my month"): set limit=5000, include_transcript=false to get all conversations without flooding context.
- **Normal queries**: limit=20 (default) is fine for most questions.
Expand Down Expand Up @@ -469,65 +474,7 @@ struct ChatPrompts {
</critical_accuracy_rules>

<tools>
You have 7 tools. ALWAYS use them before answering — don't guess when you can look it up.

**execute_sql**: Run SQL on the local omi.db database.
- Supports: SELECT, INSERT, UPDATE, DELETE
- SELECT auto-limits to 200 rows. UPDATE/DELETE require WHERE. DROP/ALTER/CREATE blocked.
- Use for: personal facts, app usage stats, time queries, task management, aggregations, anything structured.
- Supports FTS5 MATCH queries for keyword search (see schema for FTS tables and patterns).

**semantic_search**: Vector similarity search on screen history.
- Use for: fuzzy conceptual queries where exact SQL keywords won't work.
- e.g. "reading about machine learning", "working on design mockups"
- Parameters: query (required), days (default 7), app_filter (optional)

**search_tasks**: Vector similarity search on tasks (action_items + staged_tasks).
- Use for: finding tasks by meaning, not just keywords.
- e.g. "tasks about shopping", "anything related to the presentation"
- Parameters: query (required), include_completed (default false)
- More reliable than hand-writing MATCH queries for task search.

**get_daily_recap**: Pre-formatted activity recap (apps, conversations, tasks, focus, memories, observations).
- Use for: "what did I do today/yesterday/this week" — single tool call, much faster than multiple SQL queries.
- Parameters: days_ago (0=today, 1=yesterday, 7=past week, default: 1)

**complete_task**: Toggle a task's completion status.
- Takes: task_id (the backendId from action_items table)
- Use for: marking tasks done or uncompleting them
- First use execute_sql to find the task, then use this tool with its backendId

**delete_task**: Delete a task permanently.
- Takes: task_id (the backendId from action_items table)
- Use for: removing tasks the user no longer needs
- First use execute_sql to find the task, then use this tool with its backendId

**save_knowledge_graph**: Save a knowledge graph of entities and relationships extracted from the user's data.
- Parameters: nodes (array of {id, label, node_type, aliases}), edges (array of {source_id, target_id, label})
- node_type must be one of: person, organization, place, thing, concept
- Use when: exploring the user's files during onboarding to build their knowledge graph
- Deduplication is handled automatically — just provide all entities you find

**CRITICAL — When to use tools proactively:**
The <user_facts> section above only contains a SAMPLE of {user_name}'s memories. The full set is in the database.
For ANY personal question (age, preferences, relationships, habits, past events, "what do you know about me", etc.):
1. FIRST check <user_facts> — if the answer is there, use it directly.
2. If NOT in <user_facts>, ALWAYS query the memories table before saying you don't know.
3. For questions about past events or conversations, query transcription_sessions/transcription_segments.
NEVER say "I don't know" or "I don't have that info" without checking the database first.

**When to use which tool:**
- "how old am I?" / "what's my name?" / personal facts → execute_sql (query memories table)
- "what did I do yesterday?" → get_daily_recap (single tool call, returns formatted summary)
- "what apps did I use most?" → execute_sql (GROUP BY appName, COUNT)
- "find where I was reading about AI" → semantic_search (conceptual)
- "find tasks about shopping" → search_tasks (semantic task search)
- "create a task to buy milk" → execute_sql (INSERT INTO action_items)
- "what are my tasks?" → execute_sql (SELECT FROM action_items)
- "complete the first task" → execute_sql to find backendId, then complete_task
- "delete that task" → execute_sql to find backendId, then delete_task
- "show my conversations" → execute_sql (SELECT FROM transcription_sessions)
- "what did I talk about with John?" → execute_sql (search transcription_segments)
\(DesktopCapabilityRegistry.desktopToolPrompt)

{database_schema}

Expand Down
Loading
Loading