Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 8 additions & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
@@ -1,3 +1,10 @@
# CH_VERSION selects the bundled ClickHouse server image tag, baked at build
# time. Override with `--build-arg CH_VERSION=24.3` or via docker-compose
# (`CH_VERSION=24.3 docker compose build`). Defaults to the latest release.
# Declared before the first FROM so it is a global ARG usable in the runtime
# stage's FROM below.
ARG CH_VERSION=latest

# Build stage
FROM node:20-alpine AS builder
WORKDIR /app
Expand All @@ -7,7 +14,7 @@ COPY . .
RUN npm run build

# Runtime stage
FROM clickhouse/clickhouse-server:latest
FROM clickhouse/clickhouse-server:${CH_VERSION}

# Install nginx and supervisor
RUN apt-get update && apt-get install -y nginx supervisor && rm -rf /var/lib/apt/lists/*
Expand Down
16 changes: 16 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,22 @@ docker run -d -p 8080:80 rowbinary-explorer

Open http://localhost:8080

### Bundled ClickHouse version

The image bundles a ClickHouse server, pinned to `latest` by default. Choose a
specific version at **build time** with the `CH_VERSION` build argument (it maps
to the `clickhouse/clickhouse-server:<tag>` image tag):

```bash
# docker build
docker build --build-arg CH_VERSION=24.3 -t rowbinary-explorer .

# docker compose
CH_VERSION=24.3 docker compose build
```

The version is baked into the image — rebuild to change it.

## Desktop App

For developers who already run ClickHouse locally. Download the latest release for your platform from the [Releases](../../releases) page:
Expand Down
34 changes: 34 additions & 0 deletions docker-compose.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
# Self-contained deployment: bundled ClickHouse + nginx + capture server.
#
# docker compose up -d --build # build and start (http://<host>:8080)
# docker compose logs -f # follow logs
# docker compose down # stop and remove
#
# Override the web port, read-only mode, or bundled ClickHouse version without
# editing this file:
# WEB_PORT=9000 docker compose up -d --build
# READONLY=0 docker compose up -d --build # allow INSERT/DDL (internal use)
# CH_VERSION=24.3 docker compose build # pin the bundled server version
services:
explorer:
build:
context: .
args:
# Bundled ClickHouse server image tag, baked at build time (default latest).
CH_VERSION: "${CH_VERSION:-latest}"
image: rowbinary-explorer
ports:
# host:container — the app/nginx listens on 80 inside the container.
- "${WEB_PORT:-8080}:80"
environment:
# READONLY=1 (default) serves the read-only viewer user.
# Set READONLY=0 for an internal deployment that may INSERT / run DDL.
READONLY: "${READONLY:-1}"
restart: unless-stopped
volumes:
# Persist the bundled ClickHouse data across restarts (drop this line for
# a fully ephemeral instance).
- clickhouse-data:/var/lib/clickhouse

volumes:
clickhouse-data:
95 changes: 95 additions & 0 deletions docs/cli-spec.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
# CLI & Tooling Spec

Captures the decisions from the requirements session for the agent-usable CLI
and related work (todo items 1, 2, 4, 5). **Item 3 (configurable TCP protocol
version) is dropped** — clickhouse-client negotiates `min(client, server)` on
its own and exposes no way to force a version, and clamping it in the proxy is
out of scope.

## 1. CLI — `chfx`

A single npm `bin` named **`chfx`**, run via Node/tsx. Decoders are imported
directly from `src/core` (one source of truth — see §Output). The package is
prepared **publish-ready**: a build step compiles `src/cli` + the needed core to
`dist`, and `package.json` (`bin`, `files`, `exports`) is wired so `npm publish`
works. Local use via `npx chfx` / `npm link`.

### Agent-usability requirements (item 1)
- **Deterministic JSON on stdout.** Primary output is one well-formed JSON
document with stable key ordering. Diagnostics/logs go to **stderr** only.
- **Non-interactive by default.** No prompts, spinners, color, or progress on
stdout. Everything is flag-controllable so an agent can script it.
- **Self-describing output.** The JSON carries a tool/schema version marker, the
format, the (for captures) negotiated protocol version, and documents the
`byteRange` convention inline.
- (Errors are emitted as JSON on stderr with a sane exit code — not a raw stack
trace — but elaborate exit-code taxonomy was not prioritized.)

### Commands

#### `chfx decode`
Import a binary dump from a file **or stdin** and emit structured JSON.
- Inputs: `.chproto` captures, raw **Native** bodies, raw **RowBinary**
(`RowBinaryWithNamesAndTypes`) bodies.
- Format detection: **autodetect with override.** `.chproto` is detected by its
magic header; raw bodies are autodetected best-effort with
`--format native|rowbinary|chproto` to force it (the reliable path when the
Native/RowBinary heads are ambiguous).
- Accepts binary on **stdin** (e.g. piped from clickhouse-client) as well as a
file path argument.

#### `chfx query`
Run SQL against a server and decode the result in one step.
- Transport: **both, `--transport http|native`.**
- `http`: POST to ClickHouse HTTP, request the chosen format, decode the body.
- `native`: drive clickhouse-client through the capture proxy and decode the
full `.chproto` packet stream.
- Default `--format native` (richest). **Experimental type settings**
(Variant/Dynamic/JSON enablement) are **sent by default**, with
`--no-experimental-settings` to disable for read-only/strict servers that
reject them.
- Remote connection flags: `--host/--port/--user/--password`, env-var fallbacks,
and HTTPS/TLS where applicable.

#### `chfx proxy` (item 5 — standalone capture proxy)
A listener that forwards to a target server and captures the native TCP stream.
**Any** native client (clickhouse-client, Go/JDBC/Python drivers, …) connects
through it — the proxy does not spawn the client itself.
- Configurable lifecycle:
- **Default: single-shot** — accept one connection, capture, write a
`.chproto` (and decode if asked), then exit.
- Flags opt into **persistent** mode (long-running, one `.chproto` per
connection to an output dir) and **live decode** (`--decode` streams decoded
JSON per connection to stdout).
- Flags: `--listen`, `--target host:port`, `--out`/`--save-dir`, `--decode`,
`--persistent`/`--once`, plus remote `--user/--password`/TLS where applicable.
- **Plaintext/uncompressed only** (same constraint as today). TLS/compressed
streams are unsupported — error clearly and document it.

#### `chfx schema` / `--help`
Machine-readable description of commands, flags, and the output JSON shape for
agent discovery.

### Output (item 4)
- **Reuse the web `ParsedData`/`AstNode` shape verbatim**, serialized to JSON,
wrapped with top-level metadata: tool/schema version, format, negotiated
protocol version (for captures), and the raw bytes.
- **Raw bytes inline, once**, as a top-level **hex** string. Agents read a node's
`byteRange {start, end}` (exclusive end) and slice the hex to inspect bytes —
no second command or sidecar file.

### Tests & docs (cross-cutting)
- **Thorough CLI tests/fixtures**: decode against the existing
`src/core/decoder/fixtures/protocol/*.chproto`, golden-JSON output assertions,
plus query/proxy coverage.
- **README**: a short, practical **quick-start** near the top, and further down a
full reference of all commands, options, settings, and defaults.

## 2. Docker — configurable server version (item 2)
- `ARG CH_VERSION=latest` → `FROM clickhouse/clickhouse-server:${CH_VERSION}`.
- Surfaced through docker-compose, e.g. `CH_VERSION=24.3 docker compose build`.
- Version is baked at **build time** (no runtime/CI-matrix publishing for now).

## 3. Configurable TCP/native protocol version — **dropped**
No reliable way to force the negotiated version via clickhouse-client; the
existing Native HTTP `client_protocol_version` selector is sufficient.
23 changes: 23 additions & 0 deletions todo.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# TODO

Spec for items 1, 2, 4, 5: [docs/cli-spec.md](docs/cli-spec.md)

1. **CLI (`chfx`) usable by agent** — npm bin via Node/tsx, publish-ready.
Deterministic JSON on stdout, non-interactive, self-describing. Commands:
`decode`, `query`, `proxy`, `schema`/`--help`.
2. **Configurable server version in the image** — `ARG CH_VERSION` →
`FROM clickhouse/clickhouse-server:${CH_VERSION}`, surfaced via docker-compose.
3. ~~Configurable protocol version in TCP + native web interface~~ — **dropped**
(clickhouse-client can't force the negotiated version; HTTP selector suffices).
4. **Import binary dump in CLI → structured output** — `chfx decode` (and
`query`): autodetect `.chproto`/Native/RowBinary with `--format` override,
stdin supported. Emits the web `ParsedData`/`AstNode` JSON plus the full raw
buffer inline as one hex string; agents slice it via each node's `byteRange`.
5. **Use with external clients, in the CLI** — `chfx proxy`: standalone capture
proxy any native client connects through. Single-shot by default; persistent
and live-decode via flags. Plaintext/uncompressed only.

Cross-cutting: thorough CLI tests/fixtures; README quick-start + full options
reference; remote auth/TLS flags for `query`/`proxy`.

6. Deploy similar to play.clickhouse.com?
Loading