Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
560aec6
feat: add push-only PostgreSQL sync
Mar 17, 2026
2fd802b
docs: add PostgreSQL sync refactor design spec
wesm Mar 17, 2026
29d1629
docs: address spec review findings
wesm Mar 17, 2026
29ba551
docs: add implementation plan for PG sync refactor
wesm Mar 17, 2026
643ed61
build: add BurntSushi/toml dependency
wesm Mar 17, 2026
af3bda8
refactor: migrate config from JSON to TOML
wesm Mar 17, 2026
476b0e5
refactor: create internal/postgres package with connect, schema, time
wesm Mar 17, 2026
f504ba8
refactor: move push/sync into internal/postgres
wesm Mar 17, 2026
e406131
refactor: move read-only store into internal/postgres
wesm Mar 17, 2026
a6937a3
refactor: rename PGSyncConfig to PGConfig, update fields
wesm Mar 17, 2026
5b69b6d
fix: add SearchSession to Store interface, remove periodic PG sync
wesm Mar 17, 2026
5de036d
refactor: add pg command group, remove PG from sync/serve
wesm Mar 17, 2026
a3bf629
refactor: delete old pgdb, pgsync, pgutil packages
wesm Mar 17, 2026
6d939b0
docs: update documentation for PG sync refactor
wesm Mar 17, 2026
5edeff8
chore: remove spec and plan files before push
wesm Mar 17, 2026
7543d19
test: add docker-compose PG integration testing infrastructure
wesm Mar 17, 2026
805e5f2
fix: address golangci-lint modernize findings
wesm Mar 17, 2026
b739614
fix: update stale config key names in error/warning messages
wesm Mar 17, 2026
4c69ec8
fix: address timezone, sort stability, and pg serve issues
wesm Mar 17, 2026
007b031
docs: fix stale references in CLAUDE.md and README.md
wesm Mar 17, 2026
7327b6a
fix: pg serve remote access and search sort stability
wesm Mar 17, 2026
5b8e83d
fix: use isLoopbackHost for pg serve remote access check
wesm Mar 17, 2026
e772a02
feat: add schema version tracking to PG schema
wesm Mar 17, 2026
1fefdd9
feat: add --base-path flag for reverse-proxy subpath deployment
wesm Mar 18, 2026
151c9c5
perf: batch push transactions and multi-row inserts
wesm Mar 18, 2026
c4d56ef
fix: base-path prefix matching and frontend API base derivation
wesm Mar 18, 2026
37a5e91
fix: isolate per-session push failures and bump updated_at on msg rew…
wesm Mar 18, 2026
2888503
fix: public_origins should also expand host allowlist
wesm Mar 18, 2026
5c017d7
fix: sanitize UTF-8 before PG insert and fix byte-based truncation
wesm Mar 18, 2026
f7be64a
fix: address review feedback on push retry, host check, and sanitization
wesm Mar 18, 2026
cec9164
fix: detect PG schema reset and auto-force full push
wesm Mar 18, 2026
6a94d34
fix: sanitize tool_call fields and fix schema reset detection
wesm Mar 18, 2026
0674c48
fix: clear schemaDone on full push so schema is recreated after drop
wesm Mar 18, 2026
9f3fa5b
fix: remove erroneous ctx arg from UpsertSession call in test
wesm Mar 18, 2026
65a4f21
fix: CI failures and absolute asset paths in frontend
wesm Mar 18, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 45 additions & 3 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,9 @@ jobs:
go-version-file: go.mod

- name: Stub frontend embed dir
run: mkdir -p internal/web/dist && echo ok > internal/web/dist/stub.html
run: |
mkdir -p internal/web/dist
echo '<!doctype html><html><head></head><body><div id="app"></div></body></html>' > internal/web/dist/index.html

- uses: golangci/golangci-lint-action@1e7e51e771db61008b38414a730f564565cf7c20 # v9.2.0
with:
Expand Down Expand Up @@ -55,7 +57,9 @@ jobs:
path-type: inherit

- name: Stub frontend embed dir
run: mkdir -p internal/web/dist && echo ok > internal/web/dist/stub.html
run: |
mkdir -p internal/web/dist
echo '<!doctype html><html><head></head><body><div id="app"></div></body></html>' > internal/web/dist/index.html

- name: Run Go tests
run: go test -tags fts5 ./... -v -count=1
Expand All @@ -80,7 +84,9 @@ jobs:
go-version-file: go.mod

- name: Stub frontend embed dir
run: mkdir -p internal/web/dist && echo ok > internal/web/dist/stub.html
run: |
mkdir -p internal/web/dist
echo '<!doctype html><html><head></head><body><div id="app"></div></body></html>' > internal/web/dist/index.html

- name: Test with coverage
run: go test -tags fts5 -race -coverprofile=coverage.out ./...
Expand All @@ -98,6 +104,42 @@ jobs:
if: steps.codecov.outcome == 'failure'
run: echo "::warning::Codecov upload failed"

integration:
runs-on: ubuntu-latest
services:
postgres:
image: postgres:16-alpine
env:
POSTGRES_USER: agentsview_test
POSTGRES_PASSWORD: agentsview_test_password
POSTGRES_DB: agentsview_test
ports:
- 5433:5432
options: >-
--health-cmd "pg_isready -U agentsview_test -d agentsview_test"
--health-interval 2s
--health-timeout 5s
--health-retries 10
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false

- uses: actions/setup-go@4b73464bb391d4059bd26b0524d20df3927bd417 # v6.3.0
with:
go-version-file: go.mod

- name: Stub frontend embed dir
run: |
mkdir -p internal/web/dist
echo '<!doctype html><html><head></head><body><div id="app"></div></body></html>' > internal/web/dist/index.html

- name: Run PostgreSQL integration tests
run: make test-postgres-ci
env:
CGO_ENABLED: "1"
TEST_PG_URL: postgres://agentsview_test:agentsview_test_password@localhost:5433/agentsview_test?sslmode=disable

e2e:
runs-on: ubuntu-latest
steps:
Expand Down
47 changes: 43 additions & 4 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,24 +8,30 @@ agentsview is a local web viewer for AI agent sessions (Claude Code, Codex, Copi

```
CLI (agentsview) → Config → DB (SQLite/FTS5)
File Watcher → Sync Engine → Parser (Claude, Codex, Copilot, Gemini, OpenCode, Amp)
HTTP Server → REST API + SSE + Embedded SPA
PG Push Sync → PostgreSQL (optional)
HTTP Server (pg serve) ← PostgreSQL
```

- **Server**: HTTP server with auto-port discovery (default 8080)
- **Storage**: SQLite with WAL mode, FTS5 for full-text search
- **Storage**: SQLite with WAL mode, FTS5 for full-text search; optional PostgreSQL for multi-machine shared access
- **Sync**: File watcher + periodic sync (15min) for session directories
- **PG Sync**: On-demand push sync from SQLite to PostgreSQL via `pg push`
- **Frontend**: Svelte 5 SPA embedded in the Go binary at build time
- **Config**: Env vars (`AGENT_VIEWER_DATA_DIR`, `CLAUDE_PROJECTS_DIR`, `CODEX_SESSIONS_DIR`, `COPILOT_DIR`, `GEMINI_DIR`, `OPENCODE_DIR`, `AMP_DIR`) and CLI flags

## Project Structure

- `cmd/agentsview/` - Go server entrypoint
- `cmd/testfixture/` - Test data generator for E2E tests
- `internal/config/` - Config loading, flag registration, legacy migration
- `internal/config/` - Config loading (TOML, JSON migration), flag registration
- `internal/db/` - SQLite operations (sessions, messages, search, analytics)
- `internal/postgres/` - PostgreSQL support: push sync, read-only store, schema, connection helpers
- `internal/parser/` - Session file parsers (Claude, Codex, Copilot, Gemini, OpenCode, Amp, content extraction)
- `internal/server/` - HTTP handlers, SSE, middleware, search, export
- `internal/sync/` - Sync engine, file watcher, discovery, hashing
Expand All @@ -39,6 +45,7 @@ CLI (agentsview) → Config → DB (SQLite/FTS5)
| Path | Purpose |
|------|---------|
| `cmd/agentsview/main.go` | CLI entry point, server startup, file watcher |
| `cmd/agentsview/pg.go` | pg command group (push, status, serve) |
| `internal/server/server.go` | HTTP router and handler setup |
| `internal/server/sessions.go` | Session list/detail API handlers |
| `internal/server/search.go` | Full-text search API |
Expand All @@ -51,6 +58,15 @@ CLI (agentsview) → Config → DB (SQLite/FTS5)
| `internal/parser/codex.go` | Codex session parser |
| `internal/parser/copilot.go` | Copilot CLI session parser |
| `internal/parser/amp.go` | Amp session parser |
| `internal/postgres/connect.go` | Connection setup, SSL checks, DSN helpers |
| `internal/postgres/schema.go` | PG DDL, schema management |
| `internal/postgres/push.go` | Push logic, fingerprinting |
| `internal/postgres/sync.go` | Push sync lifecycle |
| `internal/postgres/store.go` | PostgreSQL read-only store |
| `internal/postgres/sessions.go` | PG session queries (read side) |
| `internal/postgres/messages.go` | PG message queries, ILIKE search |
| `internal/postgres/analytics.go` | PG analytics queries |
| `internal/postgres/time.go` | Timestamp conversion helpers |
| `internal/config/config.go` | Config loading, flag registration |

## Development
Expand Down Expand Up @@ -78,6 +94,29 @@ make lint # golangci-lint
make vet # go vet
```

### PostgreSQL Integration Tests

PG integration tests require a real PostgreSQL instance and the `pgtest`
build tag. The easiest way to run them is with docker-compose:

```bash
make test-postgres # Starts PG container, runs tests, leaves container running
make postgres-down # Stop the test container when done
```

Or manually with an existing PostgreSQL instance:

```bash
TEST_PG_URL="postgres://user:pass@host:5432/dbname?sslmode=disable" \
CGO_ENABLED=1 go test -tags "fts5,pgtest" ./internal/postgres/... -v
```

Tests create and drop the `agentsview` schema, so use a dedicated
database or one where schema changes are acceptable.

The CI pipeline runs these tests automatically via a GitHub Actions
service container (see `.github/workflows/ci.yml`, `integration` job).

### Test Guidelines

- Table-driven tests for Go code
Expand Down
24 changes: 23 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ LDFLAGS := -X main.version=$(VERSION) \
LDFLAGS_RELEASE := $(LDFLAGS) -s -w
DESKTOP_DIST_DIR := dist/desktop

.PHONY: build build-release install frontend frontend-dev dev desktop-dev desktop-build desktop-macos-app desktop-macos-dmg desktop-windows-installer desktop-linux-appimage desktop-app test test-short e2e vet lint tidy clean release release-darwin-arm64 release-darwin-amd64 release-linux-amd64 install-hooks ensure-embed-dir help
.PHONY: build build-release install frontend frontend-dev dev desktop-dev desktop-build desktop-macos-app desktop-macos-dmg desktop-windows-installer desktop-linux-appimage desktop-app test test-short test-postgres test-postgres-ci postgres-up postgres-down e2e vet lint tidy clean release release-darwin-arm64 release-darwin-amd64 release-linux-amd64 install-hooks ensure-embed-dir help

# Ensure go:embed has at least one file (no-op if frontend is built)
ensure-embed-dir:
Expand Down Expand Up @@ -141,6 +141,25 @@ test: ensure-embed-dir
test-short: ensure-embed-dir
go test -tags fts5 ./... -short -count=1

# Start test PostgreSQL container
postgres-up:
docker compose -f docker-compose.test.yml up -d --wait

# Stop test PostgreSQL container
postgres-down:
docker compose -f docker-compose.test.yml down

# Run PostgreSQL integration tests (starts postgres automatically)
test-postgres: ensure-embed-dir postgres-up
@echo "Waiting for postgres to be ready..."
@sleep 2
TEST_PG_URL="postgres://agentsview_test:agentsview_test_password@localhost:5433/agentsview_test?sslmode=disable" \
CGO_ENABLED=1 go test -tags "fts5,pgtest" -v ./internal/postgres/... -count=1

# PostgreSQL integration tests for CI (postgres already running as service)
test-postgres-ci: ensure-embed-dir
CGO_ENABLED=1 go test -tags "fts5,pgtest" -v ./internal/postgres/... -count=1

# Run Playwright E2E tests
e2e:
cd frontend && npx playwright test
Expand Down Expand Up @@ -224,6 +243,9 @@ help:
@echo ""
@echo " test - Run all tests"
@echo " test-short - Run fast tests only"
@echo " test-postgres - Run PostgreSQL integration tests"
@echo " postgres-up - Start test PostgreSQL container"
@echo " postgres-down - Stop test PostgreSQL container"
@echo " e2e - Run Playwright E2E tests"
@echo " vet - Run go vet"
@echo " lint - Run golangci-lint"
Expand Down
90 changes: 73 additions & 17 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -123,23 +123,18 @@ agentsview -host 127.0.0.1 -port 8080 \
-allowed-subnet 192.168.1.0/24
```

You can persist the same settings in `~/.agentsview/config.json`:

```json
{
"public_url": "https://viewer.example.test",
"proxy": {
"mode": "caddy",
"bind_host": "0.0.0.0",
"public_port": 8443,
"tls_cert": "/home/user/.certs/viewer.crt",
"tls_key": "/home/user/.certs/viewer.key",
"allowed_subnets": [
"10.0/16",
"192.168.1.0/24"
]
}
}
You can persist the same settings in `~/.agentsview/config.toml`:

```toml
public_url = "https://viewer.example.test"

[proxy]
mode = "caddy"
bind_host = "0.0.0.0"
public_port = 8443
tls_cert = "/home/user/.certs/viewer.crt"
tls_key = "/home/user/.certs/viewer.key"
allowed_subnets = ["10.0/16", "192.168.1.0/24"]
```

`public_origins` remains available as an advanced override when you
Expand Down Expand Up @@ -169,6 +164,66 @@ need to allow additional browser origins beyond the main `public_url`.
| `r` | Sync sessions |
| `?` | Show all shortcuts |

## PostgreSQL Sync

agentsview can push session data from the local SQLite database to a
remote PostgreSQL instance, enabling shared team dashboards and
centralized search across multiple machines.

### Push Sync (SQLite to PG)

Configure `pg` in `~/.agentsview/config.toml`:

```toml
[pg]
url = "postgres://user:pass@host:5432/dbname?sslmode=require"
machine_name = "my-laptop"
```

Use `sslmode=require` (or `verify-full` for CA-verified connections)
for non-local PostgreSQL instances. Only use `sslmode=disable` for
trusted local/loopback connections.

The `machine_name` identifies which machine pushed each session
(must not be `"local"`, which is reserved).

CLI commands:

```bash
agentsview pg push # push now
agentsview pg push --full # force full re-push (bypasses heuristic)
agentsview pg status # show sync status
```

Push is on-demand — run `pg push` whenever you want to sync to
PostgreSQL. There is no automatic background push.

### PG Read-Only Mode

Serve the web UI directly from PostgreSQL with no local SQLite.
Configure `[pg].url` in config (as shown above), then:

```bash
agentsview pg serve # default: 127.0.0.1:8080
agentsview pg serve -port 9090 # custom port
```

This mode is useful for shared team viewers where multiple machines
push to a central PG database and one or more read-only instances
serve the UI. Uploads, file watching, and local sync are disabled.
By default, `pg serve` binds to `127.0.0.1`. When a non-loopback
`-host` is specified, remote access is enabled automatically and an
auth token is generated and printed to stdout.

### Known Limitations

- **Deleted sessions**: Sessions permanently pruned from SQLite
(via `agentsview prune`) are not propagated as deletions to PG.
Sessions soft-deleted with `deleted_at` are synced correctly.
- **Change detection**: Push uses aggregate length statistics
rather than content hashes. Use `-full` to force a complete
re-push if content was rewritten in-place.

## Documentation

Full documentation is available at
Expand Down Expand Up @@ -218,6 +273,7 @@ PATH/API keys overrides).
cmd/agentsview/ CLI entrypoint
internal/config/ Configuration loading
internal/db/ SQLite operations (sessions, search, analytics)
internal/postgres/ PostgreSQL support (push sync, read-only store, schema)
internal/parser/ Session parsers (all supported agents)
internal/server/ HTTP handlers, SSE, middleware
internal/sync/ Sync engine, file watcher, discovery
Expand Down
Loading
Loading