Skip to content

operations.md

Latest commit

 

History

History
276 lines (201 loc) · 9.91 KB

operations.md

File metadata and controls

276 lines (201 loc) · 9.91 KB

Operations

Daily Operations

Available Tools

Tool Purpose Confirmation Required
memory_search Search memories by query No
memory_delete Delete a specific memory by ID Yes (confirm=true)
memory_clear Clear all memories in a scope Yes (confirm=true)
memory_stats View statistics and index health No
memory_effectiveness View effectiveness metrics No
memory_feedback_missing Report missing memory No
memory_feedback_wrong Report incorrect memory No
memory_feedback_useful Report useful memory No
memory_scope_promote Promote memory to global scope No
memory_scope_demote Demote memory from global scope No
memory_global_list List global memories No
memory_consolidate Merge near-duplicate memories in a scope Yes (confirm=true)
memory_consolidate_all Consolidate global + current project scope Yes (confirm=true)

Common Workflows

Check System Health

# View statistics
memory_stats

# Check effectiveness
memory_effectiveness

Scheduled Consolidation (Daily Cleanup)

The memory system performs opportunistic consolidation on session.compacted events. For environments with long-running sessions or infrequent activity, a daily cron job ensures duplicates are merged even without session compaction.

Example cron script (~/.config/opencode/consolidate-cron.sh):

#!/bin/bash
# Consolidate duplicates daily at 03:00 UTC
# Run via: opencode --memory-consolidate-all --confirm
#
# Note: Requires OpenCode CLI with lancedb-opencode-pro plugin installed.
# The AI will invoke memory_consolidate_all tool automatically when you ask.

0 3 * * * /path/to/opencode --memory-consolidate-all --confirm >> ~/.opencode/logs/consolidation.log 2>&1

Key configuration options (via environment or lancedb-opencode-pro.json):

Variable Default Description
LANCEDB_OPENCODE_PRO_DEDUP_ENABLED true Enable/disable dedup
LANCEDB_OPENCODE_PRO_DEDUP_WRITE_THRESHOLD 0.92 Similarity threshold for flagging new memories
LANCEDB_OPENCODE_PRO_DEDUP_CONSOLIDATE_THRESHOLD 0.95 Similarity threshold for merging in consolidation

Viewing consolidation metrics:

memory_effectiveness
# Look for: duplicates.flaggedCount (memories flagged as potential duplicates)
# Look for: duplicates.consolidatedCount (memories merged via consolidation)

Search and Verify

# Search for specific topic
memory_search query="nginx configuration" limit=5

# Search with broader terms
memory_search query="deployment" limit=10

Clean Up Old Memories

# List memories in scope
memory_global_list limit=50

# Delete specific memory (requires confirmation)
memory_delete id="<memory-id>" confirm=true

# Clear entire scope (requires confirmation)
memory_clear scope="project:old-project" confirm=true

Embedding Model Migration Behavior

  • Records persist schemaVersion, embeddingModel, and vectorDim.
  • On startup, the store initializes against the configured embedding dimension.
  • Query-time vector similarity is only meaningful when query vector dimension matches record vector dimension.
  • If embedding backend is unavailable, retrieval degrades to lexical scoring only.

Index Health And Stale Recovery

  • Store exposes index status via memory_stats.
  • Vector index creation is best effort; if unavailable, retrieval still works with in-process scoring.
  • FTS index creation is best effort; if unavailable, lexical fallback is used.

Retention Strategy

  • Auto-capture writes into active project scope by default.
  • Scope retention is count-based (maxEntriesPerScope, default 3000).
  • After each auto-capture write, older entries in the same scope are pruned.

Effectiveness Reporting

  • Runtime effectiveness events are stored in the same LanceDB database as memories under a dedicated events table.
  • Capture flow records considered, stored, and skipped outcomes with normalized skip reasons when the store is initialized.
  • Recall flow records request count, result count, and whether optional memory context was injected.
  • User feedback is recorded through memory_feedback_missing, memory_feedback_wrong, and memory_feedback_useful.
  • Operators can inspect the aggregated machine-readable summary with memory_effectiveness for the active project scope.

System Health vs Product Value

  • System health metrics: capture.successRate, capture.skipReasons, recall.hitRate, recall.injectionRate, recall.auto.*, recall.manual.*, and recall.manualRescueRatio.
  • Product value metrics: repeated-context reduction, clarification burden reduction, manual memory rescue rate, correction-signal rate, and sampled recall usefulness.
  • High recall availability means the store can return something; it does not prove that the injected memory helped the conversation.
  • Zero feedback.* counts mean the workflow lacks direct labels, not that memory quality is confirmed.

Example Workflow

memory_search query="stale token api gateway" limit=5
memory_feedback_useful id="<memory-id>" helpful=true
memory_feedback_missing text="This project prefers blue-green deploys." labels=["preference"]
memory_effectiveness

Expected summary fields:

  • capture.considered, capture.stored, capture.skipped
  • capture.skipReasons
  • recall.requested, recall.returnedResults, recall.injected
  • recall.auto.requested, recall.auto.injected, recall.auto.returnedResults, recall.auto.hitRate, recall.auto.injectionRate
  • recall.manual.requested, recall.manual.returnedResults, recall.manual.hitRate
  • recall.manualRescueRatio
  • feedback.missing, feedback.wrong, feedback.useful
  • feedback.falsePositiveRate, feedback.falseNegativeRate

Low-Feedback Proxy Metrics

Use these proxy metrics when users rarely submit memory_feedback_* commands:

Proxy metric What it means Current evidence source
Repeated-context reduction Users repeat less project context across sessions or follow-up turns Manual conversation review; not instrumented yet
Clarification burden Agent asks fewer reminder or context-recovery questions Manual conversation review; not instrumented yet
Manual memory rescue rate Users still need memory_search after automatic recall Instrumented: recall.manual.requested / recall.manualRescueRatio in memory_effectiveness output
Correction-signal rate Users say the recalled context is wrong, stale, or irrelevant memory_feedback_wrong, memory_feedback_missing, or conversation review
Sampled recall usefulness Audited recalled memories appear relevant and actually help move work forward Sample audit of recalled memories

Sample Audit Workflow

When explicit feedback is sparse, run a bounded audit instead of assuming quality:

  1. Sample 10-20 recent recall injections from the same project scope.
  2. For each sample, inspect the recalled memory text and the next assistant reply.
  3. Mark whether the memory was relevant, neutral noise, or misleading.
  4. Sample 10-20 skipped captures, especially no-positive-signal, and check whether important durable knowledge was missed.
  5. Treat the audit as release input alongside memory_effectiveness, not as a replacement for runtime metrics.

Monitoring And Diagnostics

Key Metrics to Monitor

Metric Healthy Range Warning Sign
capture.successRate > 0.8 < 0.5
recall.hitRate > 0.7 < 0.3
recall.injectionRate > 0.6 < 0.2
recall.manualRescueRatio < 0.3 > 0.5

Diagnostic Commands

# Check index health
memory_stats
# Look for: indexHealth.vector = true, indexHealth.fts = true

# Verify search works
memory_search query="test" limit=1
# Should return results if memories exist

# Check effectiveness
memory_effectiveness
# Review capture and recall metrics

Troubleshooting Common Issues

No Search Results

Symptoms: memory_search returns empty array

Possible Causes:

  1. No memories stored yet
  2. Vector dimension mismatch (after embedding model change)
  3. Scope filtering issue

Diagnosis:

# Check if memories exist
memory_stats

# Try broader search
memory_search query="" limit=10

Slow Search Performance

Symptoms: Search takes > 500ms

Possible Causes:

  1. Large number of memories (> 10K)
  2. Index not created
  3. Embedding backend slow

Diagnosis:

# Check index status
memory_stats
# Look for: indexHealth.vector = true

# Check memory count
memory_stats
# Look for: recentCount

Embedding Errors

Symptoms: Ollama embedding request failed or OpenAI embedding request failed

Possible Causes:

  1. Backend unavailable
  2. Wrong configuration
  3. Network issues

Diagnosis:

# Check configuration
cat ~/.config/opencode/lancedb-opencode-pro.json

# Test Ollama connection (if using Ollama)
curl http://127.0.0.1:11434/api/tags

# Test OpenAI connection (if using OpenAI)
curl -H "Authorization: Bearer $OPENAI_API_KEY" https://api.openai.com/v1/models

Rollback Criteria

Disable provider in opencode.json when any of the following is true:

  • Retrieval causes prompt contamination (memory injection overrides user intent).
  • Scope filtering leaks memories across project boundaries.
  • Embedding backend instability causes repeated hook failures.
  • Index creation fails and operational warnings exceed acceptable threshold.

Rollback Procedure

  1. Set memory.provider to a different provider or remove memory block.
  2. Restart OpenCode session.
  3. Preserve local DB files under ~/.opencode/memory/lancedb for later analysis.

Operator Diagnostics

  • Run memory_stats to inspect index health and active configuration.
  • Run memory_search with a known prior phrase to verify recall behavior.
  • Run memory_clear with confirm=true only when decommissioning a scope.