Skip to content

feat(parser): add gptme agent parser#678

Merged
wesm merged 5 commits into
kenn-io:mainfrom
TimeToBuildBob:feat/gptme-parser
Jun 15, 2026
Merged

feat(parser): add gptme agent parser#678
wesm merged 5 commits into
kenn-io:mainfrom
TimeToBuildBob:feat/gptme-parser

Conversation

@TimeToBuildBob

Copy link
Copy Markdown
Contributor

Summary

Adds gptme as a supported agent in AgentsView.

gptme is an open-source terminal AI assistant (Spring 2023, one of the earliest agent CLIs) supporting Anthropic, OpenAI, Google, DeepSeek, and local models via OpenRouter. Sessions are stored as JSONL files with one JSON object per message.

Session format

{"role": "system", "content": "...", "timestamp": "2026-06-13T10:00:00.000000", "pinned": true}
{"role": "user", "content": "Write a hello world program in Python.", "timestamp": "..."}
{"role": "assistant", "content": "I'll write...", "timestamp": "...", "metadata": {"model": "openrouter/anthropic/claude-sonnet-4-6", "usage": {"input_tokens": 120, "output_tokens": 42, "cache_read_tokens": 80, "cache_creation_tokens": 0}}}
{"role": "tool", "content": "Saved file: hello.py", "timestamp": "..."}

Session discovery path: ~/.local/share/gptme/logs/*/conversation.jsonl

Changes

  • internal/parser/gptme.goDiscoverGptmeSessions, FindGptmeSourceFile, ParseGptmeSession with full token accounting (input/output/cache read/cache write tokens)
  • internal/parser/types.go — register AgentGptme in the Registry with GPTME_DIR env var and gptme_dirs config key
  • internal/parser/gptme_test.go — tests for parse, discover, find, and session-name-to-project extraction
  • internal/parser/testdata/gptme/ — fixture JSONL with 7 messages covering all role types and token_usage fields

Implementation notes

  • Tool messages ("role": "tool") are mapped to RoleUser with IsSystem: true, following the same pattern used for other agents' tool output
  • System messages are skipped (they're context injections, not conversation turns)
  • Session name format: YYYY-MM-DD-topic or YYYY-MM-DD-HHMMSS-topic — the parser extracts the topic part as the project name
  • Token field names: gptme uses input_tokens/output_tokens/cache_read_tokens/cache_creation_tokens (vs. Anthropic API's cache_read_input_tokens)

Testing

go test ./internal/parser/... -run "TestParseGptme|TestDiscoverGptme|TestFindGptme|TestGptme" -v

All 4 tests pass.

Adds gptme as a supported agent in AgentsView. gptme stores sessions as
JSONL files under ~/.local/share/gptme/logs/<session-id>/conversation.jsonl
with one JSON object per message containing role, content, timestamp, and
optional metadata (model, usage) on assistant messages.

- internal/parser/gptme.go: DiscoverGptmeSessions, FindGptmeSourceFile,
  ParseGptmeSession with full token accounting (input/output/cache tokens)
- internal/parser/types.go: register AgentGptme in the Registry
- internal/parser/gptme_test.go: tests for parse, discover, find, and
  session-name-to-project extraction
- internal/parser/testdata/gptme/: fixture JSONL with 7 messages covering
  system/user/assistant/tool roles and token_usage fields

gptme is an open-source terminal AI assistant (https://gptme.org) with
support for Anthropic, OpenAI, Google, and local models via OpenRouter.
@roborev-ci

roborev-ci Bot commented Jun 13, 2026

Copy link
Copy Markdown

roborev: Combined Review (0a02eca)

High-level verdict: changes are not ready; gptme is registered but not wired into sync, and its fixture timestamp format is not parsed.

High

  • Location: internal/parser/types.go:456
  • Problem: The new agent is registered for discovery, but the sync engine has no AgentGptme case or processGptme path. Discovered gptme files will fall into the engine's unknown-agent branch instead of being parsed and indexed.
  • Fix: Add sync engine support analogous to the other file-backed JSONL agents and cover it with a sync integration test.

Medium

  • Location: internal/parser/gptme.go:97
  • Problem: The parser uses parseTimestamp, but the added fixture uses timestamps like 2026-06-13T10:00:00.000000, which that helper does not accept. Those gptme records will persist with nil session/message timestamps, breaking chronological ordering and date-based filtering for this format.
  • Fix: Add a gptme-specific timestamp parser or extend accepted layouts, then assert parsed session/message timestamps in the test.

Panel: ci_default_security | Synthesis: codex, 8s | Members: codex_default (codex/default, done, 2m51s), codex_security (codex/security, done, 1m35s) | Total: 4m34s

…sing

- Add classifyOnePath case for AgentGptme so discovered gptme files route
  to the sync engine instead of the unknown-agent branch
- Add processGptme to the engine's processFile dispatch
- Extend timestampLayouts with "2006-01-02T15:04:05.000000" to handle
  gptme's microsecond-precision timestamps (no timezone suffix)
- Assert session.StartedAt, session.EndedAt, and msgs[0].Timestamp in
  TestParseGptmeSession to confirm timestamp parsing end-to-end

Addresses roborev High + Medium findings on PR kenn-io#678.
@TimeToBuildBob

Copy link
Copy Markdown
Contributor Author

Addressed the two roborev findings in 832fa1c:

High — sync engine missing AgentGptme dispatch
Added a classifyOnePath case for AgentGptme (iterates e.agentDirs[parser.AgentGptme], matches <logsDir>/<session-dir>/conversation.jsonl) and wired processGptme into the processFile switch, following the same pattern as other file-backed JSONL agents.

Medium — microsecond timestamp format not parsed
Extended timestampLayouts with "2006-01-02T15:04:05.000000" to handle gptme's timezone-less microsecond timestamps. Added assertions in TestParseGptmeSession for sess.StartedAt, sess.EndedAt, and msgs[0].Timestamp to confirm end-to-end parsing. All 4 parser tests pass.

@roborev-ci

roborev-ci Bot commented Jun 13, 2026

Copy link
Copy Markdown

roborev: Combined Review (832fa1c)

No issues found.


Panel: ci_default_security | Synthesis: codex | Members: codex_default (codex/default, done, 8m59s), codex_security (codex/security, done, 3m14s) | Total: 12m13s

@roborev-ci

roborev-ci Bot commented Jun 15, 2026

Copy link
Copy Markdown

roborev: Combined Review (746bb5e)

No Medium, High, or Critical findings were reported.

The only findings were Low severity and are omitted per instructions.


Panel: ci_default_security | Synthesis: codex, 5s | Members: codex_default (codex/default, done, 5m54s), codex_security (codex/security, done, 2m17s) | Total: 8m16s

@roborev-ci

roborev-ci Bot commented Jun 15, 2026

Copy link
Copy Markdown

roborev: Combined Review (d76fab2)

Medium: gptme tool output is being hidden from visible transcripts.

  • Medium: internal/parser/gptme.go:164 stores gptme tool messages with IsSystem: true, causing UI/export/search paths to treat visible tool output, such as command results, as hidden system messages. Store gptme tool output as visible transcript content, or attach it as tool-result data if gptme provides a stable tool call ID.

Panel: ci_default_security | Synthesis: codex, 8s | Members: codex_default (codex/default, done, 5m27s), codex_security (codex/security, done, 2m21s) | Total: 7m56s

@TimeToBuildBob

TimeToBuildBob commented Jun 15, 2026

Copy link
Copy Markdown
Contributor Author

Addressed the latest roborev Medium finding in c91507a5.

gptme does not expose a stable tool-call ID in these JSONL lines, so I did not synthesize ToolResults. Instead, I changed standalone role:"tool" entries to stay visible as assistant transcript content rather than hidden IsSystem messages. That keeps command output like Saved file: hello.py and Hello, World! visible in UI/export/search without inflating UserMessageCount.

Verified with:

  • go test ./internal/parser -run "TestParseGptmeSession|TestDiscoverGptmeSessions|TestFindGptmeSourceFile|TestGptmeProjectFromSessionName"

@roborev-ci

roborev-ci Bot commented Jun 15, 2026

Copy link
Copy Markdown

roborev: Combined Review (c91507a)

Verdict: No Medium, High, or Critical findings to report.

The only reported issue was Low severity and has been omitted per instructions.


Panel: ci_default_security | Synthesis: codex, 7s | Members: codex_default (codex/default, done, 6m30s), codex_security (codex/security, done, 2m58s) | Total: 9m35s

@wesm wesm merged commit 6b7931d into kenn-io:main Jun 15, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants