Skip to content

feat(memory): optional Dongtian memory retrieval for system prompt context#72

Open
siaochuan wants to merge 11 commits intoHKUDS:mainfrom
siaochuan:feat/dongtian-memory
Open

feat(memory): optional Dongtian memory retrieval for system prompt context#72
siaochuan wants to merge 11 commits intoHKUDS:mainfrom
siaochuan:feat/dongtian-memory

Conversation

@siaochuan
Copy link
Copy Markdown
Contributor

Closes #67

Summary

Add an optional integration layer that retrieves relevant conversation snippets from a local Dongtian memory database and injects them into the system prompt. Default behavior is completely unchanged — the feature is off unless explicitly enabled.

How it works

User sends a message
  → OH searches the local Dongtian palace DB (FTS5 keyword match)
  → Top-N relevant snippets injected into system prompt as context
  → Agent sees historical conversation context from past sessions

Configuration

{
  "memory": {
    "enabled": true,
    "dongtian_enabled": true,
    "dongtian_db_path": "~/.dongtian/palace.db",
    "dongtian_limit": 5,
    "dongtian_max_chars": 1200
  }
}

All fields have safe defaults. Set `dongtian_enabled: true` to activate.

Design decisions

  • Off by default — zero behavior change for existing installs
  • Zero dependencies — uses only Python stdlib `sqlite3`; no `dongtian` package import needed
  • Read-only — opens the DB in `?mode=ro` URI mode; never writes
  • FTS5 only — no embedding API calls, no network, works fully offline
  • Timeout protected — 200ms SQLite timeout, graceful fallback to empty on any error
  • Injection safety — snippets are wrapped with an untrusted-content warning telling the model not to follow instructions inside retrieved memories
  • Chinese-aware tokenizer — FTS query builder handles CJK characters

Relation to #67

This addresses the same need as #67 (external memory retrieval for system prompt) with a different backend:

PowerMem (#67) Dongtian (this PR)
Backend HTTP API server Local SQLite file
Dependencies Running server + SDK None (stdlib sqlite3)
Search Vector + full-text + graph FTS5 keyword (+ optional embeddings via MCP)
Setup Install + configure server Just point to a `.db` file
Offline No Yes

Both approaches are valid for different use cases. A future `MemoryProvider` protocol could unify them — see discussion on #67.

Files

File Purpose
`memory/dongtian.py` FTS5 search against palace DB (166 lines)
`config/settings.py` `MemorySettings` fields for dongtian config
`prompts/context.py` Snippet injection into system prompt
`tests/test_prompts/test_dongtian_memory.py` End-to-end tests (creates DB, searches, verifies injection)

What is Dongtian?

Dongtian is a lightweight AI conversation memory system (~1200 lines, SQLite + FTS5 + optional embeddings). It ingests sessions from Claude Code, Codex, OpenCode/DeepSeek, ChatGPT, and Slack into a structured searchable database. It also runs as an MCP server with 15 tools for search, ingestion, knowledge graph, and SSH remote sync.

siaochuan and others added 11 commits April 8, 2026 14:29
…DS#138)

`asyncio.gather` was called without `return_exceptions=True`, so a single
escaping exception from one parallel tool propagated out of the gather,
abandoned its sibling coroutines, and left the assistant turn with one or
more `tool_use` blocks that never received a matching `tool_result`. The
Anthropic Messages API rejects the next request on the session with a 400
in that state, so any unhandled tool exception during a multi-tool turn
effectively bricks the session.

Pass `return_exceptions=True` and synthesize an `is_error=True`
`ToolResultBlock` for each raised exception, matched to the originating
`tool_use_id`. The exception is logged via `log.exception` so the failure
is still observable.

Co-authored-by: José Maia <glitch-ux@users.noreply.github.com>
(cherry picked from commit a2dea03)
Files under ~/.openharness/ — credentials, settings, session snapshots,
cron registry, memory index — were written with `Path.write_text()` in
truncating mode. A crash, SIGKILL, power loss, or out-of-disk error
during the write leaves a truncated file on disk; concurrent writers
clobber each other's updates; and the credentials file spent a brief
window at the default umask mode (commonly 0o644) before chmod-to-0600
ran.

Introduce `openharness.utils.fs.atomic_write_text` / `atomic_write_bytes`
which write to a same-directory temp file, fsync, apply the target mode
while the file is still private, and `os.replace` into place. Thread
them through all persistence writers. For read-modify-write on shared
files (credentials, settings, cron, memory index), pair atomic writes
with the existing `exclusive_file_lock` primitive so two `oh` processes
no longer race.

The generic lock helper moves from `openharness.swarm.lockfile` to
`openharness.utils.file_lock`. `swarm.lockfile` is retained as a thin
re-export so existing callers keep working.

Co-authored-by: José Maia <glitch-ux@users.noreply.github.com>
(cherry picked from commit 4857439)
…fresh line (HKUDS#133)

* fix(tui): write newline on exit so shell prompt starts on fresh line

Ink hides the cursor during rendering and restores it on exit, but does
not emit a trailing newline.  When the TUI process ends, the terminal
cursor sits at the end of the last rendered line, causing the shell
prompt to appear directly concatenated with the TUI output.

Rename restoreCursor → restoreTerminal and append '\n' alongside the
cursor-show escape sequence so the parent shell always receives the
prompt on a clean new line.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* test(tui): add regression guard for exit newline; update changelog

- Add tests/test_ui/test_tui_exit_sequence.py with two tests:
  * test_tui_exit_handler_writes_newline: asserts the cleanup write
    includes the trailing \n alongside \x1B[?25h
  * test_tui_exit_handler_registered_for_all_signals: asserts cleanup
    is registered for 'exit', SIGINT, and SIGTERM
- Add entry to CHANGELOG.md [Unreleased] Fixed section

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(tests): update ohmo cli test inputs for allow_remote_admin_commands prompt

Commit dd1d235 added a new 'Allow explicitly listed administrative slash
commands from remote channels?' confirmation step to the gateway config
wizard, but the four interactive test cases in test_ohmo/test_cli.py were
not updated to provide an answer for it. This caused stdin to be exhausted
and click to emit Abort(), making all four tests fail with exit_code=1.

Add 'n' as the answer for allow_remote_admin_commands in each affected
test, and move the existing 'restart gateway' answer after it in
test_ohmo_config_interactive_can_restart_gateway.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
(cherry picked from commit ed0b5f0)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature]: (memory): optional PowerMem retrieval for system prompt context

4 participants