Skip to content

docs: add memory backend guide #3

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Viperdk2020 merged 1 commit into from
Sep 1, 2025
Merged

docs: add memory backend guide #3

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,25 +62,26 @@ You can also use Codex with an API key, but this requires [additional setup](./d

Codex CLI supports [MCP servers](./docs/advanced.md#model-context-protocol-mcp). Enable by adding an `mcp_servers` section to your `~/.codex/config.toml`.


### Configuration

Codex CLI supports a rich set of configuration options, with preferences stored in `~/.codex/config.toml`. For full configuration options, see [Configuration](./docs/config.md).

### Per‑repo memory (local)

Codex keeps a lightweight, per‑repository memory of key actions to help you recall decisions and changes in each project. Entries are written locally to `<repo>/.codex/memory/memory.jsonl` after tool use (shell exec, MCP tool calls) and patch application. The file is plain JSONL so you can search, back up, or clear it easily. See [Per‑repo memory](./docs/config.md#per-repo-memory-local) for details.
Codex keeps a lightweight, per‑repository memory of key actions to help you recall decisions and changes in each project. Entries are written locally to `<repo>/.codex/memory/memory.jsonl` after tool use (shell exec, MCP tool calls) and patch application. The file is plain JSONL so you can search, back up, or clear it easily. For choosing the optional SQLite backend or migrating existing data, see [Memory backends](./docs/memory-backends.md). See [Per‑repo memory](./docs/config.md#per-repo-memory-local) for configuration details.

Toggle per run (TUI or exec): add `--memory off` to disable, or `--memory on` to force‑enable. You can also set `CODEX_PER_REPO_MEMORY=0|1`.

Durable memory and smarter preamble

- Durable items: In addition to action logs (exec, tool, change), Codex records durable items you and Codex can reuse across turns:
- `pref`: your preferences and decisions (e.g., “Prefer Ruff + Black”).
- `summary`: short outcomes/facts captured on task completion.
- `decision`: approval checkpoints and key decisions.
- Smarter preamble: On new sessions, Codex injects a short “project memory” preamble built from recent `pref` and `summary` items. It deduplicates, merges tags, caps counts, and enforces a length limit so your prompt stays concise.

TUI: memory slash commands

- Add preference: `/memory add Use Ruff + Black`
- List recent durable: `/memory list 10`
- Search durable: `/memory search ruff`
Expand All @@ -89,6 +90,7 @@ TUI: memory slash commands
- Help: `/memory help`

What gets logged automatically (TUI)

- On task complete: a `summary` durable item with a compact preview of the last assistant message (kept brief).
- On approval request: a `decision` durable item noting the request (exec/patch).

Expand Down
110 changes: 56 additions & 54 deletions docs/config.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
# Config


Codex supports several mechanisms for setting config values:

- Config-specific command-line flags, such as `--model o3` (highest precedence).
Expand Down Expand Up @@ -388,10 +387,10 @@ set = { CI = "1" }
include_only = ["PATH", "HOME"]
```

| Field | Type | Default | Description |
| ------------------------- | -------------------------- | ------- | ----------------------------------------------------------------------------------------------------------------------------------------------- |
| `inherit` | string | `all` | Starting template for the environment:<br>`all` (clone full parent env), `core` (`HOME`, `PATH`, `USER`, …), or `none` (start empty). |
| `ignore_default_excludes` | boolean | `false` | When `false`, Codex removes any var whose **name** contains `KEY`, `SECRET`, or `TOKEN` (case-insensitive) before other rules run. |
| Field | Type | Default | Description |
| ------------------------- | -------------------- | ------- | ----------------------------------------------------------------------------------------------------------------------------------------------- |
| `inherit` | string | `all` | Starting template for the environment:<br>`all` (clone full parent env), `core` (`HOME`, `PATH`, `USER`, …), or `none` (start empty). |
| `ignore_default_excludes` | boolean | `false` | When `false`, Codex removes any var whose **name** contains `KEY`, `SECRET`, or `TOKEN` (case-insensitive) before other rules run. |
| `exclude` | array<string> | `[]` | Case-insensitive glob patterns to drop after the default filter.<br>Examples: `"AWS_*"`, `"AZURE_*"`. |
| `set` | table<string,string> | `{}` | Explicit key/value overrides or additions – always win over inherited values. |
| `include_only` | array<string> | `[]` | If non-empty, a whitelist of patterns; only variables that match _one_ pattern survive the final step. (Generally used with `inherit = "all"`.) |
Expand Down Expand Up @@ -566,55 +565,55 @@ Options that are specific to the TUI.

## Config reference

| Key | Type / Values | Notes |
| --- | --- | --- |
| `model` | string | Model to use (e.g., `gpt-5`). |
| `model_provider` | string | Provider id from `model_providers` (default: `openai`). |
| `model_context_window` | number | Context window tokens. |
| `model_max_output_tokens` | number | Max output tokens. |
| `approval_policy` | `untrusted` | `on-failure` | `on-request` | `never` | When to prompt for approval. |
| `sandbox_mode` | `read-only` | `workspace-write` | `danger-full-access` | OS sandbox policy. |
| `sandbox_workspace_write.writable_roots` | array<string> | Extra writable roots in workspace‑write. |
| `sandbox_workspace_write.network_access` | boolean | Allow network in workspace‑write (default: false). |
| `sandbox_workspace_write.exclude_tmpdir_env_var` | boolean | Exclude `$TMPDIR` from writable roots (default: false). |
| `sandbox_workspace_write.exclude_slash_tmp` | boolean | Exclude `/tmp` from writable roots (default: false). |
| `disable_response_storage` | boolean | Required for ZDR orgs. |
| `notify` | array<string> | External program for notifications. |
| `instructions` | string | Currently ignored; use `experimental_instructions_file` or `AGENTS.md`. |
| `mcp_servers.<id>.command` | string | MCP server launcher command. |
| `mcp_servers.<id>.args` | array<string> | MCP server args. |
| `mcp_servers.<id>.env` | map<string,string> | MCP server env vars. |
| `model_providers.<id>.name` | string | Display name. |
| `model_providers.<id>.base_url` | string | API base URL. |
| `model_providers.<id>.env_key` | string | Env var for API key. |
| `model_providers.<id>.wire_api` | `chat` | `responses` | Protocol used (default: `chat`). |
| `model_providers.<id>.query_params` | map<string,string> | Extra query params (e.g., Azure `api-version`). |
| `model_providers.<id>.http_headers` | map<string,string> | Additional static headers. |
| `model_providers.<id>.env_http_headers` | map<string,string> | Headers sourced from env vars. |
| `model_providers.<id>.request_max_retries` | number | Per‑provider HTTP retry count (default: 4). |
| `model_providers.<id>.stream_max_retries` | number | SSE stream retry count (default: 5). |
| `model_providers.<id>.stream_idle_timeout_ms` | number | SSE idle timeout (ms) (default: 300000). |
| `project_doc_max_bytes` | number | Max bytes to read from `AGENTS.md`. |
| `profile` | string | Active profile name. |
| `profiles.<name>.*` | various | Profile‑scoped overrides of the same keys. |
| `history.persistence` | `save-all` | `none` | History file persistence (default: `save-all`). |
| `history.max_bytes` | number | Currently ignored (not enforced). |
| `file_opener` | `vscode` | `vscode-insiders` | `windsurf` | `cursor` | `none` | URI scheme for clickable citations (default: `vscode`). |
| `tui` | table | TUI‑specific options (reserved). |
| `hide_agent_reasoning` | boolean | Hide model reasoning events. |
| `show_raw_agent_reasoning` | boolean | Show raw reasoning (when available). |
| `model_reasoning_effort` | `minimal` | `low` | `medium` | `high` | Responses API reasoning effort. |
| `model_reasoning_summary` | `auto` | `concise` | `detailed` | `none` | Reasoning summaries. |
| `model_verbosity` | `low` | `medium` | `high` | GPT‑5 text verbosity (Responses API). |
| `model_supports_reasoning_summaries` | boolean | Force‑enable reasoning summaries. |
| `chatgpt_base_url` | string | Base URL for ChatGPT auth flow. |
| `experimental_resume` | string (path) | Resume JSONL path (internal/experimental). |
| `experimental_instructions_file` | string (path) | Replace built‑in instructions (experimental). |
| `experimental_use_exec_command_tool` | boolean | Use experimental exec command tool. |
| `responses_originator_header_internal_override` | string | Override `originator` header value. |
| `projects.<path>.trust_level` | string | Mark project/worktree as trusted (only `"trusted"` is recognized). |
| `preferred_auth_method` | `chatgpt` | `apikey` | Select default auth method (default: `chatgpt`). |
| `tools.web_search` | boolean | Enable web search tool (alias: `web_search_request`) (default: false). |
| Key | Type / Values | Notes |
| ------------------------------------------------ | ------------------ | ----------------------------------------------------------------------- | ------------------------------------------------ | ------------------------------------- | ------------------------------- | ------------------------------------------------------- |
| `model` | string | Model to use (e.g., `gpt-5`). |
| `model_provider` | string | Provider id from `model_providers` (default: `openai`). |
| `model_context_window` | number | Context window tokens. |
| `model_max_output_tokens` | number | Max output tokens. |
| `approval_policy` | `untrusted` | `on-failure` | `on-request` | `never` | When to prompt for approval. |
| `sandbox_mode` | `read-only` | `workspace-write` | `danger-full-access` | OS sandbox policy. |
| `sandbox_workspace_write.writable_roots` | array<string> | Extra writable roots in workspace‑write. |
| `sandbox_workspace_write.network_access` | boolean | Allow network in workspace‑write (default: false). |
| `sandbox_workspace_write.exclude_tmpdir_env_var` | boolean | Exclude `$TMPDIR` from writable roots (default: false). |
| `sandbox_workspace_write.exclude_slash_tmp` | boolean | Exclude `/tmp` from writable roots (default: false). |
| `disable_response_storage` | boolean | Required for ZDR orgs. |
| `notify` | array<string> | External program for notifications. |
| `instructions` | string | Currently ignored; use `experimental_instructions_file` or `AGENTS.md`. |
| `mcp_servers.<id>.command` | string | MCP server launcher command. |
| `mcp_servers.<id>.args` | array<string> | MCP server args. |
| `mcp_servers.<id>.env` | map<string,string> | MCP server env vars. |
| `model_providers.<id>.name` | string | Display name. |
| `model_providers.<id>.base_url` | string | API base URL. |
| `model_providers.<id>.env_key` | string | Env var for API key. |
| `model_providers.<id>.wire_api` | `chat` | `responses` | Protocol used (default: `chat`). |
| `model_providers.<id>.query_params` | map<string,string> | Extra query params (e.g., Azure `api-version`). |
| `model_providers.<id>.http_headers` | map<string,string> | Additional static headers. |
| `model_providers.<id>.env_http_headers` | map<string,string> | Headers sourced from env vars. |
| `model_providers.<id>.request_max_retries` | number | Per‑provider HTTP retry count (default: 4). |
| `model_providers.<id>.stream_max_retries` | number | SSE stream retry count (default: 5). |
| `model_providers.<id>.stream_idle_timeout_ms` | number | SSE idle timeout (ms) (default: 300000). |
| `project_doc_max_bytes` | number | Max bytes to read from `AGENTS.md`. |
| `profile` | string | Active profile name. |
| `profiles.<name>.*` | various | Profile‑scoped overrides of the same keys. |
| `history.persistence` | `save-all` | `none` | History file persistence (default: `save-all`). |
| `history.max_bytes` | number | Currently ignored (not enforced). |
| `file_opener` | `vscode` | `vscode-insiders` | `windsurf` | `cursor` | `none` | URI scheme for clickable citations (default: `vscode`). |
| `tui` | table | TUI‑specific options (reserved). |
| `hide_agent_reasoning` | boolean | Hide model reasoning events. |
| `show_raw_agent_reasoning` | boolean | Show raw reasoning (when available). |
| `model_reasoning_effort` | `minimal` | `low` | `medium` | `high` | Responses API reasoning effort. |
| `model_reasoning_summary` | `auto` | `concise` | `detailed` | `none` | Reasoning summaries. |
| `model_verbosity` | `low` | `medium` | `high` | GPT‑5 text verbosity (Responses API). |
| `model_supports_reasoning_summaries` | boolean | Force‑enable reasoning summaries. |
| `chatgpt_base_url` | string | Base URL for ChatGPT auth flow. |
| `experimental_resume` | string (path) | Resume JSONL path (internal/experimental). |
| `experimental_instructions_file` | string (path) | Replace built‑in instructions (experimental). |
| `experimental_use_exec_command_tool` | boolean | Use experimental exec command tool. |
| `responses_originator_header_internal_override` | string | Override `originator` header value. |
| `projects.<path>.trust_level` | string | Mark project/worktree as trusted (only `"trusted"` is recognized). |
| `preferred_auth_method` | `chatgpt` | `apikey` | Select default auth method (default: `chatgpt`). |
| `tools.web_search` | boolean | Enable web search tool (alias: `web_search_request`) (default: false). |

## Per-repo memory (local)

Expand All @@ -635,11 +634,14 @@ Example (one line):
```

Notes

- This store is local-only and intended for project memory/history.
- Clearing: delete or truncate the file at `<repo>/.codex/memory/memory.jsonl`.
- Backups/exports: copy the JSONL file anywhere (each line is an entry).
- Alternate backends: JSONL is default; a SQLite store is available for faster queries. See [Memory backends](./memory-backends.md).

Toggle

- Per run (both TUI and exec): pass `--memory off` to disable or `--memory on` to force-enable.
- Environment variable (both modes): set `CODEX_PER_REPO_MEMORY=0|1` (also accepts `on|off`, `true|false`). Alias: `CODEX_MEMORY`.
- Examples:
Expand Down
41 changes: 41 additions & 0 deletions docs/memory-backends.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# Memory backends

Codex stores per-repository memory to help recall decisions and context. Two storage formats are available:

## JSONL (default)

- One line per entry at `.codex/memory/memory.jsonl` inside each repo.
- Human-readable and easy to back up or edit with standard tools.

## SQLite (optional)

- Stores entries in `.codex/memory/memory.db`.
- Offers atomic updates, indexing and faster queries.
- Requires a build with SQLite support and enabling at runtime.

### Enabling SQLite

1. Ensure your Codex build includes the `sqlite` feature (pre-built binaries include it).
2. Set `CODEX_MEMORY_BACKEND=sqlite` in the environment.

### Migrating existing data

Convert an existing JSONL store to SQLite:

```bash
codex memory migrate
```

This reads `memory.jsonl` and writes `memory.db` in the same directory.

### Compaction

To reclaim free space in the SQLite database:

```bash
codex memory compact
```

This runs `VACUUM` on `memory.db`.

Unsetting `CODEX_MEMORY_BACKEND` returns Codex to the default JSONL backend.
Loading