From 4d01e2e4053255d11b0fb9d9242a379a26e029de Mon Sep 17 00:00:00 2001 From: alex-lum Date: Mon, 22 Jun 2026 12:07:17 -0700 Subject: [PATCH 1/4] feat: align clickhouse-data-analyst plugin with direct Query API Replace clickhousectl cloud service query with the direct ClickHouse Query API (CH_API_KEY / CH_API_SECRET via curl) for Cloud analytics. - data-analyst/SKILL.md: add explicit note that Cloud queries use the direct Query API, not clickhousectl cloud service query - skills/clickhouse/SKILL.md: rewrite to document two access paths: Path A (Cloud) = direct Query API via curl Path B (local/host-port) = clickhousectl local client Remove clickhousectl cloud auth and cloud service query from the Cloud workflow entirely Fixes the FORBIDDEN provisioning error on first use and removes the admin-key requirement for new users. --- .../skills/data-analyst/SKILL.md | 31 ++-- .../data-analyst/skills/clickhouse/SKILL.md | 151 +++++++++++------- 2 files changed, 115 insertions(+), 67 deletions(-) diff --git a/plugins/clickhouse-data-analyst/skills/data-analyst/SKILL.md b/plugins/clickhouse-data-analyst/skills/data-analyst/SKILL.md index 9e1b8b28..3a5bc76f 100644 --- a/plugins/clickhouse-data-analyst/skills/data-analyst/SKILL.md +++ b/plugins/clickhouse-data-analyst/skills/data-analyst/SKILL.md @@ -7,7 +7,12 @@ description: Act as an interactive data analyst for ClickHouse-backed analytics. Act as an interactive data analyst over ClickHouse. The job is not to run the first query you can think of; it is to figure out the question the user actually has, answer it with a correct and bounded query, and report the definitions and caveats behind the number. -CRITICAL: this skill never uses ClickHouse MCP tools. All database connections, queries, schema discovery, and data access go through the `clickhousectl` CLI (`skills/clickhouse/`). If ClickHouse MCP tools (`mcp-clickhouse__*`) are available in the environment, ignore them completely. Always run queries via `clickhousectl local client` or `clickhousectl cloud service query`. +CRITICAL: this skill never uses ClickHouse MCP tools. All queries go through the paths defined in `skills/clickhouse/`: + +- **ClickHouse Cloud (production analytics):** use the **direct ClickHouse Query API** with per-user credentials (`CH_API_KEY` / `CH_API_SECRET`). Do **not** use `clickhousectl cloud service query` for production analytics — it auto-provisions keys and creates key sprawl. +- **Local or host/port server:** use `clickhousectl local client`. + +If ClickHouse MCP tools (`mcp-clickhouse__*`) are available in the environment, ignore them completely. Sub-skills live in `skills/`. Load only the sub-skill directory needed for the current step, then follow that directory's `SKILL.md`. Referenced paths are relative to this skill directory (`/skills/data-analyst/`), not the user's workspace. For example, read plotting guidance at `/skills/data-analyst/skills/plotting/SKILL.md`. @@ -15,20 +20,20 @@ Sub-skills live in `skills/`. Load only the sub-skill directory needed for the c Authored for this analyst workflow: -- `skills/clickhouse/` - connect to ClickHouse (local or ClickHouse Cloud) via the `clickhousectl` CLI and run safe, bounded queries. Load before executing any SQL. -- `skills/reading-data-dict/` - resolve business and product terms to concrete models, columns, and metric definitions when the project documents its data (dbt repo, data dictionary, model docs). -- `skills/steering-user-elicitation/` - fill the Intent block well, phrase good pushback, and handle metrics that are missing or commonly misunderstood. -- `skills/analyzer/` - turn query results into trends, comparisons, distributions, funnels, sanity checks, and report-ready findings. -- `skills/plotting/` - create chart or visual artifacts from query results. -- `skills/artifact-management/` - save CSVs, charts, and report assets to a stable location and report their paths. +- `skills/clickhouse/` — connect to ClickHouse (local or ClickHouse Cloud) via the `clickhousectl` CLI and run safe, bounded queries. Load before executing any SQL. +- `skills/reading-data-dict/` — resolve business and product terms to concrete models, columns, and metric definitions when the project documents its data (dbt repo, data dictionary, model docs). +- `skills/steering-user-elicitation/` — fill the Intent block well, phrase good pushback, and handle metrics that are missing or commonly misunderstood. +- `skills/analyzer/` — turn query results into trends, comparisons, distributions, funnels, sanity checks, and report-ready findings. +- `skills/plotting/` — create chart or visual artifacts from query results. +- `skills/artifact-management/` — save CSVs, charts, and report assets to a stable location and report their paths. Bundled official ClickHouse skills (from [ClickHouse/agent-skills](https://github.com/ClickHouse/agent-skills), Apache-2.0, vendored via a git submodule). Load these when the corresponding need arises: -- `skills/clickhouse-best-practices/` - schema, query, and ingestion rules plus an agent schema-discovery and query-safety workflow. Consult when writing or optimizing non-trivial SQL. -- `skills/chdb-sql/` - run ClickHouse SQL on local files (parquet/csv/json), S3, and remote databases in Python with no server. Use for ad-hoc analysis over files or cross-source data. -- `skills/chdb-datastore/` - pandas-style API on a ClickHouse engine and cross-source DataFrames. Use when the user has DataFrames/files and wants fast, SQL-grade aggregation that feeds plotting. -- `skills/clickhousectl-local-dev/` - install ClickHouse and run a local server. Use when the user needs a local instance to load and analyze data. -- `skills/clickhousectl-cloud-deploy/`, `skills/clickhouse-architecture-advisor/`, `skills/clickhouse-js-node-coding/`, `skills/clickhouse-js-node-troubleshooting/` - also bundled; less central to ad-hoc analysis (deployment, production architecture, and JS client work). +- `skills/clickhouse-best-practices/` — schema, query, and ingestion rules plus an agent schema-discovery and query-safety workflow. Consult when writing or optimizing non-trivial SQL. +- `skills/chdb-sql/` — run ClickHouse SQL on local files (parquet/csv/json), S3, and remote databases in Python with no server. Use for ad-hoc analysis over files or cross-source data. +- `skills/chdb-datastore/` — pandas-style API on a ClickHouse engine and cross-source DataFrames. Use when the user has DataFrames/files and wants fast, SQL-grade aggregation that feeds plotting. +- `skills/clickhousectl-local-dev/` — install ClickHouse and run a local server. Use when the user needs a local instance to load and analyze data. +- `skills/clickhousectl-cloud-deploy/`, `skills/clickhouse-architecture-advisor/`, `skills/clickhouse-js-node-coding/`, `skills/clickhouse-js-node-troubleshooting/` — also bundled; less central to ad-hoc analysis (deployment, production architecture, and JS client work). See `examples.md` for realistic example prompts that show the elicitation-first style. @@ -76,7 +81,7 @@ Elicitation is an invariant, not just step 1. At any step, if a new ambiguity su ## Core rules -- Never use ClickHouse MCP tools. All SQL execution goes through the `clickhousectl` CLI as described in `skills/clickhouse/`. Do not call `mcp-clickhouse__run_query`, `mcp-clickhouse__list_databases`, `mcp-clickhouse__list_tables`, or any other ClickHouse MCP function, even if they are available in the environment. +- Never use ClickHouse MCP tools. Do not call `mcp-clickhouse__run_query`, `mcp-clickhouse__list_databases`, `mcp-clickhouse__list_tables`, or any other ClickHouse MCP function, even if they are available in the environment. For ClickHouse Cloud, use the direct Query API with `CH_API_KEY`/`CH_API_SECRET` as described in `skills/clickhouse/`. For local servers, use `clickhousectl local client`. Do not use `clickhousectl cloud service query` for production analytics. - Prefer curated, documented models and metrics over raw event or log tables. - State the definitions, filters, time window, and assumptions used. - Start with schema discovery, previews, or aggregates before broad result dumps. diff --git a/plugins/clickhouse-data-analyst/skills/data-analyst/skills/clickhouse/SKILL.md b/plugins/clickhouse-data-analyst/skills/data-analyst/skills/clickhouse/SKILL.md index 4232c2ae..60cc9ab3 100644 --- a/plugins/clickhouse-data-analyst/skills/data-analyst/skills/clickhouse/SKILL.md +++ b/plugins/clickhouse-data-analyst/skills/data-analyst/skills/clickhouse/SKILL.md @@ -1,11 +1,20 @@ --- name: clickhouse -description: Connect to and query ClickHouse (a local server or a ClickHouse Cloud service) from the terminal using the official clickhousectl CLI, including the browser OAuth login flow. Use when the user wants to run SQL against ClickHouse, explore schemas and tables, inspect Cloud services, or authenticate clickhousectl. For building a local dev environment or deploying to Cloud, defer to the official ClickHouse skills (see Scope). +description: Connect to and query ClickHouse (a local server or a ClickHouse Cloud service) from the terminal. For ClickHouse Cloud production analytics, use the direct Query API with CH_API_KEY and CH_API_SECRET. For local or host/port servers, use the clickhousectl CLI. Use when the user wants to run SQL against ClickHouse, explore schemas and tables, inspect Cloud services, or authenticate. For building a local dev environment or deploying to Cloud, defer to the official ClickHouse skills (see Scope). --- -# ClickHouse via clickhousectl +# ClickHouse Connection and Queries -Connect to ClickHouse and run queries using `clickhousectl`, the official ClickHouse CLI. This skill covers the parts a data analyst needs: authenticating, pointing at the right server, and running safe SQL. It does not use the ClickHouse MCP server; everything goes through the CLI. +Two access paths — pick the right one for the target: + +| Target | Path | +| ------ | ---- | +| ClickHouse Cloud (production analytics) | **Direct Query API** — `CH_API_KEY` / `CH_API_SECRET` via `curl` | +| Local server or any host/port | **clickhousectl CLI** — `clickhousectl local client` | + +Do **not** use `clickhousectl cloud service query` for production analytics. It auto-provisions per-service query-endpoint keys on first use, which creates key sprawl and fails without local state. + +This skill does not use the ClickHouse MCP server. ## Scope @@ -18,98 +27,132 @@ This skill is for connecting and querying. For these other flows, use the bundle These are vendored from https://github.com/ClickHouse/agent-skills (Apache-2.0). They can also be installed standalone with `clickhousectl skills` or `npx skills add clickhouse/agent-skills`. -## Step 1: Ensure clickhousectl is installed +--- + +## Path A: ClickHouse Cloud — Direct Query API (production analytics) + +### Credentials + +Read from the local environment: ```bash -which clickhousectl +CH_API_KEY # required — ClickHouse Cloud API key ID +CH_API_SECRET # required — ClickHouse Cloud API key secret +CH_QUERY_API_URL # optional — override the default endpoint ``` -If not found, install it (downloads the right build for the OS, installs to `~/.local/bin/clickhousectl`, and creates a `chctl` alias): +If `CH_API_KEY` or `CH_API_SECRET` is missing, stop and tell the user: -```bash -curl -fsSL https://clickhouse.com/cli | sh +```text +Missing ClickHouse Query API credentials. +Set CH_API_KEY and CH_API_SECRET in your local environment. +Use your per-user ClickHouse Cloud API key. Do not use a shared key +or clickhousectl cloud service query for production analytics. ``` -If the command is still not found after install, `~/.local/bin` is not on PATH for this session: +Never print `CH_API_SECRET`. + +### Running queries ```bash -export PATH="$HOME/.local/bin:$PATH" +curl -X POST -s \ + --user "$CH_API_KEY:$CH_API_SECRET" \ + "${CH_QUERY_API_URL:-}?format=JSONEachRow" \ + -H 'Content-Type: application/json' \ + -d '{ "sql": "SELECT 1 AS ok" }' ``` -## Step 2: Identify the target - -Decide what you are querying before authenticating. There are three cases: +> Replace `` with your org's ClickHouse Cloud Query API endpoint URL, or set `CH_QUERY_API_URL` in the environment. -- A local ClickHouse server managed by `clickhousectl` (started via `clickhousectl local server start`). Query it by name. No cloud auth needed. -- Any reachable ClickHouse over host/port (local or remote). No cloud auth needed. -- A ClickHouse Cloud service. Requires cloud authentication (Step 3). +Output format: `JSONEachRow` — parse line-by-line; each non-empty line is a JSON object. -List local servers and their ports: +Connectivity check: ```bash -clickhousectl local server list +curl -X POST -s \ + --user "$CH_API_KEY:$CH_API_SECRET" \ + "${CH_QUERY_API_URL:-}?format=JSONEachRow" \ + -H 'Content-Type: application/json' \ + -d '{ "sql": "SELECT 1 AS ok" }' +# Expected: {"ok":1} ``` -## Step 3: Authenticate to ClickHouse Cloud (only for Cloud targets) +### Read-only guardrails (defense-in-depth) + +Before sending any query, check that the first SQL token is not one of: + +```text +INSERT, ALTER, DROP, TRUNCATE, CREATE, DELETE, SYSTEM, OPTIMIZE, RENAME, GRANT, REVOKE +``` -Skip this entirely for local or host/port targets. +If it is, refuse. The real enforcement layer is the read-only endpoint database role in ClickHouse Cloud — client-side checks are defense-in-depth only. -`clickhousectl` has two cloud auth modes. The distinction matters: +### Error handling -- OAuth login (browser device flow), read-only. The agent can run this directly; it opens the user's browser. It can list and inspect resources (orgs, services, service details) but cannot create, modify, or delete. +| HTTP status | Likely cause | Action | +| ----------- | ------------ | ------ | +| `{"ok":1}` | Success | Proceed | +| 401 | Wrong key ID or secret; extra whitespace; key disabled | Re-check credentials | +| 403 | Key not authorized on this endpoint; IP allowlist | Confirm key is in endpoint's authorized list | +| 4xx/5xx | Query error or service issue | Surface full error body to user | - ```bash - clickhousectl cloud auth login - ``` +Surface the full HTTP status and response body on errors. Do not retry silently. -- API key login, read and write. Needed for any write, and also for running SQL via `cloud service query` on first use (see the note in Step 4). Keep the secret out of the chat: ask the user to run this in a separate terminal in the same directory, or set the env vars themselves. +--- - ```bash - # Either: run in a separate terminal (keeps the secret out of this session) - clickhousectl cloud auth login --api-key --api-secret +## Path B: Local or host/port server — clickhousectl CLI - # Or: environment variables (good for scripts/agents) - export CLICKHOUSE_CLOUD_API_KEY= - export CLICKHOUSE_CLOUD_API_SECRET= - ``` +### Step 1: Ensure clickhousectl is installed -If the user has no account yet, `clickhousectl cloud auth signup` opens the sign-up page. +```bash +which clickhousectl +``` -Verify auth and list what you can reach: +If not found, install it (downloads the right build for the OS, installs to `~/.local/bin/clickhousectl`, and creates a `chctl` alias): ```bash -clickhousectl cloud auth status -clickhousectl cloud org list -clickhousectl cloud service list # get the service name / id you will query +curl -fsSL https://clickhouse.com/cli | sh ``` -Credential resolution order: CLI flags > OAuth tokens > `.clickhousectl/credentials.json` > environment variables. Credentials are stored project-locally under `.clickhousectl/`. +If the command is still not found after install, `~/.local/bin` is not on PATH for this session: + +```bash +export PATH="$HOME/.local/bin:$PATH" +``` -## Step 4: Run queries +### Step 2: Identify the target -Prefer `--format` (e.g. `JSONEachRow`, `CSV`, `TabSeparated`) or `--json` when you need to parse results in a later step. SQL precedence for the query commands is `--query` > `--queries-file` > stdin. +Decide what you are querying before connecting. -Cloud service (over HTTP, no local binary or service password required): +List local servers and their ports: ```bash -clickhousectl cloud service query --name -q "SHOW DATABASES" -clickhousectl cloud service query --id -q "SELECT count() FROM events" --format JSONEachRow -clickhousectl cloud service query --name --database analytics --queries-file query.sql +clickhousectl local server list ``` -Note on Cloud auth and querying: `cloud service query` uses a per-service query-endpoint API key that is auto-provisioned on first use and stored in `.clickhousectl/credentials.json`. Provisioning is a write, so a read-only OAuth login is not sufficient for the first query against a service. Use API key auth (Step 3) to run SQL, or pass `--no-auto-enable` to fail fast instead of attempting to provision. Once provisioned, later queries reuse the stored key. +### Step 3: Run queries -Local or host/port server (uses `clickhouse-client`): +Local named server (uses `clickhouse-client`): ```bash -clickhousectl local client --name -q "SHOW TABLES" # named local server -clickhousectl local client --host myhost --port 9000 -q "SELECT 1" # any reachable server +clickhousectl local client --name -q "SHOW TABLES" +clickhousectl local client --name -q "SELECT count() FROM events" --format JSONEachRow clickhousectl local client --name --queries-file query.sql ``` +Any reachable host/port: + +```bash +clickhousectl local client --host myhost --port 9000 -q "SELECT 1" +``` + +Prefer `--format` (e.g. `JSONEachRow`, `CSV`, `TabSeparated`) when you need to parse results in a later step. SQL precedence: `--query` > `--queries-file` > stdin. + +--- + ## Safe query practices -Keep queries safe, explainable, and bounded. +Keep queries safe, explainable, and bounded — regardless of which path you use. 1. Discover schema/table shape if unknown: `SHOW DATABASES`, `SHOW TABLES`, `DESCRIBE TABLE `. 2. Draft SQL using documented definitions when available (see `../reading-data-dict/`). @@ -125,7 +168,7 @@ Safety checks: - Filter by time window whenever possible. - Avoid `SELECT *` except tiny schema previews. - Check row counts before exporting large result sets. -- Add `--json` or a `--format` for machine-readable output you intend to parse downstream. +- Add `--format` or `?format=JSONEachRow` for machine-readable output you intend to parse downstream. ## Bounded queries and large tables @@ -151,6 +194,6 @@ When returning query results, include: ## Auth and secret handling - Never print API keys, secrets, or passwords into the conversation or commit them. -- Prefer the browser OAuth flow for read-only exploration; it puts no secret in the chat. -- When write access or SQL querying requires an API key, prefer a separate terminal or environment variables over pasting the secret into this session. -- `clickhousectl cloud auth logout` clears all saved credentials (OAuth tokens and API keys). +- For Cloud: use `CH_API_KEY` / `CH_API_SECRET` from local environment variables. If missing, stop and prompt the user to set them. Do not fall back to shared or default credentials. +- For local servers: no cloud credentials needed. +- `clickhousectl cloud auth logout` clears saved clickhousectl OAuth tokens and API keys. From cb711e9ce7329624f0ee0b9c54f1cf0de06a7091 Mon Sep 17 00:00:00 2001 From: alex-lum Date: Mon, 29 Jun 2026 13:04:42 -0700 Subject: [PATCH 2/4] docs(clickhouse-data-analyst): replace bold markdown with plain text in SKILL.md MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Convert bold markdown formatting (`**text**`) and em-dash separators (`—`) to plain text and hyphens (`-`) in list items across SKILL.md files for improved compatibility and readability. --- .../skills/data-analyst/SKILL.md | 26 +++++++++---------- .../data-analyst/skills/clickhouse/SKILL.md | 24 ++++++++--------- 2 files changed, 25 insertions(+), 25 deletions(-) diff --git a/plugins/clickhouse-data-analyst/skills/data-analyst/SKILL.md b/plugins/clickhouse-data-analyst/skills/data-analyst/SKILL.md index 3a5bc76f..80585e32 100644 --- a/plugins/clickhouse-data-analyst/skills/data-analyst/SKILL.md +++ b/plugins/clickhouse-data-analyst/skills/data-analyst/SKILL.md @@ -9,8 +9,8 @@ Act as an interactive data analyst over ClickHouse. The job is not to run the fi CRITICAL: this skill never uses ClickHouse MCP tools. All queries go through the paths defined in `skills/clickhouse/`: -- **ClickHouse Cloud (production analytics):** use the **direct ClickHouse Query API** with per-user credentials (`CH_API_KEY` / `CH_API_SECRET`). Do **not** use `clickhousectl cloud service query` for production analytics — it auto-provisions keys and creates key sprawl. -- **Local or host/port server:** use `clickhousectl local client`. +- ClickHouse Cloud (production analytics): use the direct ClickHouse Query API with per-user credentials (`CH_API_KEY` / `CH_API_SECRET`). Do not use `clickhousectl cloud service query` for production analytics - it auto-provisions keys and creates key sprawl. +- Local or host/port server: use `clickhousectl local client`. If ClickHouse MCP tools (`mcp-clickhouse__*`) are available in the environment, ignore them completely. @@ -20,20 +20,20 @@ Sub-skills live in `skills/`. Load only the sub-skill directory needed for the c Authored for this analyst workflow: -- `skills/clickhouse/` — connect to ClickHouse (local or ClickHouse Cloud) via the `clickhousectl` CLI and run safe, bounded queries. Load before executing any SQL. -- `skills/reading-data-dict/` — resolve business and product terms to concrete models, columns, and metric definitions when the project documents its data (dbt repo, data dictionary, model docs). -- `skills/steering-user-elicitation/` — fill the Intent block well, phrase good pushback, and handle metrics that are missing or commonly misunderstood. -- `skills/analyzer/` — turn query results into trends, comparisons, distributions, funnels, sanity checks, and report-ready findings. -- `skills/plotting/` — create chart or visual artifacts from query results. -- `skills/artifact-management/` — save CSVs, charts, and report assets to a stable location and report their paths. +- `skills/clickhouse/` - connect to ClickHouse (local or ClickHouse Cloud) via the `clickhousectl` CLI and run safe, bounded queries. Load before executing any SQL. +- `skills/reading-data-dict/` - resolve business and product terms to concrete models, columns, and metric definitions when the project documents its data (dbt repo, data dictionary, model docs). +- `skills/steering-user-elicitation/` - fill the Intent block well, phrase good pushback, and handle metrics that are missing or commonly misunderstood. +- `skills/analyzer/` - turn query results into trends, comparisons, distributions, funnels, sanity checks, and report-ready findings. +- `skills/plotting/` - create chart or visual artifacts from query results. +- `skills/artifact-management/` - save CSVs, charts, and report assets to a stable location and report their paths. Bundled official ClickHouse skills (from [ClickHouse/agent-skills](https://github.com/ClickHouse/agent-skills), Apache-2.0, vendored via a git submodule). Load these when the corresponding need arises: -- `skills/clickhouse-best-practices/` — schema, query, and ingestion rules plus an agent schema-discovery and query-safety workflow. Consult when writing or optimizing non-trivial SQL. -- `skills/chdb-sql/` — run ClickHouse SQL on local files (parquet/csv/json), S3, and remote databases in Python with no server. Use for ad-hoc analysis over files or cross-source data. -- `skills/chdb-datastore/` — pandas-style API on a ClickHouse engine and cross-source DataFrames. Use when the user has DataFrames/files and wants fast, SQL-grade aggregation that feeds plotting. -- `skills/clickhousectl-local-dev/` — install ClickHouse and run a local server. Use when the user needs a local instance to load and analyze data. -- `skills/clickhousectl-cloud-deploy/`, `skills/clickhouse-architecture-advisor/`, `skills/clickhouse-js-node-coding/`, `skills/clickhouse-js-node-troubleshooting/` — also bundled; less central to ad-hoc analysis (deployment, production architecture, and JS client work). +- `skills/clickhouse-best-practices/` - schema, query, and ingestion rules plus an agent schema-discovery and query-safety workflow. Consult when writing or optimizing non-trivial SQL. +- `skills/chdb-sql/` - run ClickHouse SQL on local files (parquet/csv/json), S3, and remote databases in Python with no server. Use for ad-hoc analysis over files or cross-source data. +- `skills/chdb-datastore/` - pandas-style API on a ClickHouse engine and cross-source DataFrames. Use when the user has DataFrames/files and wants fast, SQL-grade aggregation that feeds plotting. +- `skills/clickhousectl-local-dev/` - install ClickHouse and run a local server. Use when the user needs a local instance to load and analyze data. +- `skills/clickhousectl-cloud-deploy/`, `skills/clickhouse-architecture-advisor/`, `skills/clickhouse-js-node-coding/`, `skills/clickhouse-js-node-troubleshooting/` - also bundled; less central to ad-hoc analysis (deployment, production architecture, and JS client work). See `examples.md` for realistic example prompts that show the elicitation-first style. diff --git a/plugins/clickhouse-data-analyst/skills/data-analyst/skills/clickhouse/SKILL.md b/plugins/clickhouse-data-analyst/skills/data-analyst/skills/clickhouse/SKILL.md index 60cc9ab3..e0c548a8 100644 --- a/plugins/clickhouse-data-analyst/skills/data-analyst/skills/clickhouse/SKILL.md +++ b/plugins/clickhouse-data-analyst/skills/data-analyst/skills/clickhouse/SKILL.md @@ -5,14 +5,14 @@ description: Connect to and query ClickHouse (a local server or a ClickHouse Clo # ClickHouse Connection and Queries -Two access paths — pick the right one for the target: +Two access paths - pick the right one for the target: | Target | Path | | ------ | ---- | -| ClickHouse Cloud (production analytics) | **Direct Query API** — `CH_API_KEY` / `CH_API_SECRET` via `curl` | -| Local server or any host/port | **clickhousectl CLI** — `clickhousectl local client` | +| ClickHouse Cloud (production analytics) | Direct Query API - `CH_API_KEY` / `CH_API_SECRET` via `curl` | +| Local server or any host/port | clickhousectl CLI - `clickhousectl local client` | -Do **not** use `clickhousectl cloud service query` for production analytics. It auto-provisions per-service query-endpoint keys on first use, which creates key sprawl and fails without local state. +Do not use `clickhousectl cloud service query` for production analytics. It auto-provisions per-service query-endpoint keys on first use, which creates key sprawl and fails without local state. This skill does not use the ClickHouse MCP server. @@ -29,16 +29,16 @@ These are vendored from https://github.com/ClickHouse/agent-skills (Apache-2.0). --- -## Path A: ClickHouse Cloud — Direct Query API (production analytics) +## Path A: ClickHouse Cloud - Direct Query API (production analytics) ### Credentials Read from the local environment: ```bash -CH_API_KEY # required — ClickHouse Cloud API key ID -CH_API_SECRET # required — ClickHouse Cloud API key secret -CH_QUERY_API_URL # optional — override the default endpoint +CH_API_KEY # required - ClickHouse Cloud API key ID +CH_API_SECRET # required - ClickHouse Cloud API key secret +CH_QUERY_API_URL # optional - override the default endpoint ``` If `CH_API_KEY` or `CH_API_SECRET` is missing, stop and tell the user: @@ -64,7 +64,7 @@ curl -X POST -s \ > Replace `` with your org's ClickHouse Cloud Query API endpoint URL, or set `CH_QUERY_API_URL` in the environment. -Output format: `JSONEachRow` — parse line-by-line; each non-empty line is a JSON object. +Output format: `JSONEachRow` - parse line-by-line; each non-empty line is a JSON object. Connectivity check: @@ -85,7 +85,7 @@ Before sending any query, check that the first SQL token is not one of: INSERT, ALTER, DROP, TRUNCATE, CREATE, DELETE, SYSTEM, OPTIMIZE, RENAME, GRANT, REVOKE ``` -If it is, refuse. The real enforcement layer is the read-only endpoint database role in ClickHouse Cloud — client-side checks are defense-in-depth only. +If it is, refuse. The real enforcement layer is the read-only endpoint database role in ClickHouse Cloud - client-side checks are defense-in-depth only. ### Error handling @@ -100,7 +100,7 @@ Surface the full HTTP status and response body on errors. Do not retry silently. --- -## Path B: Local or host/port server — clickhousectl CLI +## Path B: Local or host/port server - clickhousectl CLI ### Step 1: Ensure clickhousectl is installed @@ -152,7 +152,7 @@ Prefer `--format` (e.g. `JSONEachRow`, `CSV`, `TabSeparated`) when you need to p ## Safe query practices -Keep queries safe, explainable, and bounded — regardless of which path you use. +Keep queries safe, explainable, and bounded - regardless of which path you use. 1. Discover schema/table shape if unknown: `SHOW DATABASES`, `SHOW TABLES`, `DESCRIBE TABLE `. 2. Draft SQL using documented definitions when available (see `../reading-data-dict/`). From dfc7de1c412d10445d4d9e21fd4265406e4d4b21 Mon Sep 17 00:00:00 2001 From: alex-lum Date: Mon, 29 Jun 2026 13:28:09 -0700 Subject: [PATCH 3/4] docs(clickhouse-data-analyst): update skill to use clickhousectl for Cloud analytics Replace direct Query API usage with `clickhousectl cloud service query` using explicit per-user API key/secret credentials for ClickHouse Cloud analytics. Restrict `CH_QUERY_API_URL` usage to only when users explicitly provide a saved-query endpoint URL for preconfigured queries. Update README, SKILL.md, and clickhouse sub-skill docs to reflect the new recommended workflow and remove references to the old direct Query API approach. --- plugins/clickhouse-data-analyst/README.md | 6 +- .../skills/data-analyst/SKILL.md | 7 +- .../data-analyst/skills/clickhouse/SKILL.md | 104 +++++++++++------- 3 files changed, 74 insertions(+), 43 deletions(-) diff --git a/plugins/clickhouse-data-analyst/README.md b/plugins/clickhouse-data-analyst/README.md index 7babd43f..524fc102 100644 --- a/plugins/clickhouse-data-analyst/README.md +++ b/plugins/clickhouse-data-analyst/README.md @@ -32,13 +32,13 @@ Cline automatically uses the `data-analyst` skill and its supporting ClickHouse ## Requirements -- `clickhousectl` for ClickHouse server and Cloud workflows. +- `clickhousectl` for ClickHouse Cloud, local, and host/port workflows. +- Per-user ClickHouse Cloud API credentials (`CH_API_KEY` / `CH_API_SECRET`) and a service name or ID for Cloud analytics. - Python dependencies as required by the bundled chDB skills when analyzing local files. -- ClickHouse credentials or browser OAuth depending on the target connection. ## Security Notes -The bundled skill instructs agents to use `clickhousectl` instead of ClickHouse MCP tools, ask before expensive or unbounded queries, and avoid echoing credentials or secrets. +The bundled skill instructs agents to avoid ClickHouse MCP tools, use explicit per-user API key/secret credentials for ClickHouse Cloud analytics, avoid requiring `CH_QUERY_API_URL` for the default analyst workflow, ask before expensive or unbounded queries, and avoid echoing credentials or secrets. ## Attribution diff --git a/plugins/clickhouse-data-analyst/skills/data-analyst/SKILL.md b/plugins/clickhouse-data-analyst/skills/data-analyst/SKILL.md index 80585e32..fe27d72e 100644 --- a/plugins/clickhouse-data-analyst/skills/data-analyst/SKILL.md +++ b/plugins/clickhouse-data-analyst/skills/data-analyst/SKILL.md @@ -9,8 +9,9 @@ Act as an interactive data analyst over ClickHouse. The job is not to run the fi CRITICAL: this skill never uses ClickHouse MCP tools. All queries go through the paths defined in `skills/clickhouse/`: -- ClickHouse Cloud (production analytics): use the direct ClickHouse Query API with per-user credentials (`CH_API_KEY` / `CH_API_SECRET`). Do not use `clickhousectl cloud service query` for production analytics - it auto-provisions keys and creates key sprawl. +- ClickHouse Cloud analytics: use `clickhousectl cloud service query` with explicit per-user ClickHouse Cloud API credentials (`CH_API_KEY` / `CH_API_SECRET`) and a service `--name` or `--id`. Do not require or recommend `CH_QUERY_API_URL` for the analyst workflow. - Local or host/port server: use `clickhousectl local client`. +- Saved-query Query API endpoint URLs: use only when the user explicitly provides one for the exact preconfigured query or parameterized query needed. If ClickHouse MCP tools (`mcp-clickhouse__*`) are available in the environment, ignore them completely. @@ -20,7 +21,7 @@ Sub-skills live in `skills/`. Load only the sub-skill directory needed for the c Authored for this analyst workflow: -- `skills/clickhouse/` - connect to ClickHouse (local or ClickHouse Cloud) via the `clickhousectl` CLI and run safe, bounded queries. Load before executing any SQL. +- `skills/clickhouse/` - connect to ClickHouse Cloud via `clickhousectl cloud service query` with explicit API key/secret credentials, to local/host-port servers via `clickhousectl local client`, or to explicit saved-query endpoint URLs only for preconfigured queries. Load before executing any SQL. - `skills/reading-data-dict/` - resolve business and product terms to concrete models, columns, and metric definitions when the project documents its data (dbt repo, data dictionary, model docs). - `skills/steering-user-elicitation/` - fill the Intent block well, phrase good pushback, and handle metrics that are missing or commonly misunderstood. - `skills/analyzer/` - turn query results into trends, comparisons, distributions, funnels, sanity checks, and report-ready findings. @@ -81,7 +82,7 @@ Elicitation is an invariant, not just step 1. At any step, if a new ambiguity su ## Core rules -- Never use ClickHouse MCP tools. Do not call `mcp-clickhouse__run_query`, `mcp-clickhouse__list_databases`, `mcp-clickhouse__list_tables`, or any other ClickHouse MCP function, even if they are available in the environment. For ClickHouse Cloud, use the direct Query API with `CH_API_KEY`/`CH_API_SECRET` as described in `skills/clickhouse/`. For local servers, use `clickhousectl local client`. Do not use `clickhousectl cloud service query` for production analytics. +- Never use ClickHouse MCP tools. Do not call `mcp-clickhouse__run_query`, `mcp-clickhouse__list_databases`, `mcp-clickhouse__list_tables`, or any other ClickHouse MCP function, even if they are available in the environment. For ClickHouse Cloud analytics, use `clickhousectl cloud service query` with explicit per-user API key/secret credentials as described in `skills/clickhouse/`. For local servers, use `clickhousectl local client`. Do not require or recommend `CH_QUERY_API_URL` for open-ended analysis or schema exploration. - Prefer curated, documented models and metrics over raw event or log tables. - State the definitions, filters, time window, and assumptions used. - Start with schema discovery, previews, or aggregates before broad result dumps. diff --git a/plugins/clickhouse-data-analyst/skills/data-analyst/skills/clickhouse/SKILL.md b/plugins/clickhouse-data-analyst/skills/data-analyst/skills/clickhouse/SKILL.md index e0c548a8..54fc0deb 100644 --- a/plugins/clickhouse-data-analyst/skills/data-analyst/skills/clickhouse/SKILL.md +++ b/plugins/clickhouse-data-analyst/skills/data-analyst/skills/clickhouse/SKILL.md @@ -1,18 +1,19 @@ --- name: clickhouse -description: Connect to and query ClickHouse (a local server or a ClickHouse Cloud service) from the terminal. For ClickHouse Cloud production analytics, use the direct Query API with CH_API_KEY and CH_API_SECRET. For local or host/port servers, use the clickhousectl CLI. Use when the user wants to run SQL against ClickHouse, explore schemas and tables, inspect Cloud services, or authenticate. For building a local dev environment or deploying to Cloud, defer to the official ClickHouse skills (see Scope). +description: Connect to and query ClickHouse (a local server or a ClickHouse Cloud service) from the terminal. For ClickHouse Cloud analytics, use clickhousectl cloud service query with explicit CH_API_KEY and CH_API_SECRET credentials plus a service name or ID; do not require CH_QUERY_API_URL. For local or host/port servers, use clickhousectl local client. Use saved-query Query API endpoint URLs only when the user explicitly provides an endpoint for the exact preconfigured query needed. Use when the user wants to run SQL against ClickHouse, explore schemas and tables, inspect Cloud services, or authenticate. For building a local dev environment or deploying to Cloud, defer to the official ClickHouse skills (see Scope). --- # ClickHouse Connection and Queries -Two access paths - pick the right one for the target: +Pick the right access path for the target: | Target | Path | | ------ | ---- | -| ClickHouse Cloud (production analytics) | Direct Query API - `CH_API_KEY` / `CH_API_SECRET` via `curl` | -| Local server or any host/port | clickhousectl CLI - `clickhousectl local client` | +| ClickHouse Cloud analytics | `clickhousectl cloud service query` with explicit `CH_API_KEY` / `CH_API_SECRET` and service `--name` or `--id` | +| Local server or any host/port | `clickhousectl local client` | +| Saved-query Query API endpoint URL | Only for the exact preconfigured query or parameterized query exposed by that endpoint | -Do not use `clickhousectl cloud service query` for production analytics. It auto-provisions per-service query-endpoint keys on first use, which creates key sprawl and fails without local state. +Do not require or recommend `CH_QUERY_API_URL` for the analyst workflow. Saved-query Query API endpoint URLs are not the default path for open-ended analysis or schema exploration. This skill does not use the ClickHouse MCP server. @@ -29,54 +30,78 @@ These are vendored from https://github.com/ClickHouse/agent-skills (Apache-2.0). --- -## Path A: ClickHouse Cloud - Direct Query API (production analytics) +## Path A: ClickHouse Cloud - clickhousectl with API key/secret + +Use `clickhousectl cloud service query` with explicit per-user ClickHouse Cloud API credentials. This path supports the analyst workflow's ad-hoc SQL, including schema discovery, previews, metric queries, and sanity checks, without asking the user for a Query API endpoint URL. + +Do not require or recommend `CH_QUERY_API_URL` for open-ended analysis. Saved-query Query API endpoint URLs are a narrow exception for preconfigured queries only. ### Credentials Read from the local environment: ```bash -CH_API_KEY # required - ClickHouse Cloud API key ID -CH_API_SECRET # required - ClickHouse Cloud API key secret -CH_QUERY_API_URL # optional - override the default endpoint +CH_API_KEY # required - ClickHouse Cloud API key ID +CH_API_SECRET # required - ClickHouse Cloud API key secret +CH_SERVICE # required unless passing --id - ClickHouse Cloud service name +CH_SERVICE_ID # required unless passing --name - ClickHouse Cloud service ID +CH_DATABASE # optional - target database ``` If `CH_API_KEY` or `CH_API_SECRET` is missing, stop and tell the user: ```text -Missing ClickHouse Query API credentials. +Missing ClickHouse Cloud API credentials. Set CH_API_KEY and CH_API_SECRET in your local environment. -Use your per-user ClickHouse Cloud API key. Do not use a shared key -or clickhousectl cloud service query for production analytics. +Use your per-user ClickHouse Cloud API key. Do not paste secrets into chat. ``` +If neither `CH_SERVICE` nor `CH_SERVICE_ID` is available, ask the user which ClickHouse Cloud service to query. + Never print `CH_API_SECRET`. ### Running queries +Use a service name or service ID. Prefer service ID when available because names can be ambiguous. + +By service name: + ```bash -curl -X POST -s \ - --user "$CH_API_KEY:$CH_API_SECRET" \ - "${CH_QUERY_API_URL:-}?format=JSONEachRow" \ - -H 'Content-Type: application/json' \ - -d '{ "sql": "SELECT 1 AS ok" }' +clickhousectl cloud service query \ + --api-key "$CH_API_KEY" \ + --api-secret "$CH_API_SECRET" \ + --name "$CH_SERVICE" \ + ${CH_DATABASE:+--database "$CH_DATABASE"} \ + -q "SELECT 1 AS ok" \ + --format JSONEachRow ``` -> Replace `` with your org's ClickHouse Cloud Query API endpoint URL, or set `CH_QUERY_API_URL` in the environment. +By service ID: -Output format: `JSONEachRow` - parse line-by-line; each non-empty line is a JSON object. +```bash +clickhousectl cloud service query \ + --api-key "$CH_API_KEY" \ + --api-secret "$CH_API_SECRET" \ + --id "$CH_SERVICE_ID" \ + ${CH_DATABASE:+--database "$CH_DATABASE"} \ + -q "SELECT 1 AS ok" \ + --format JSONEachRow +``` -Connectivity check: +Expected connectivity-check output: -```bash -curl -X POST -s \ - --user "$CH_API_KEY:$CH_API_SECRET" \ - "${CH_QUERY_API_URL:-}?format=JSONEachRow" \ - -H 'Content-Type: application/json' \ - -d '{ "sql": "SELECT 1 AS ok" }' -# Expected: {"ok":1} +```json +{"ok":1} ``` +Prefer `--format JSONEachRow`, `CSV`, or `TabSeparated` when you need to parse results in a later step. SQL precedence for query commands is `--query` > `--queries-file` > stdin. + +### Query endpoint provisioning note + +`clickhousectl cloud service query` runs SQL over the Cloud Query API and does not require a local `clickhouse-client` binary or database password. Depending on local state, the first query for a service may provision or reuse a per-service query endpoint binding managed by `clickhousectl`. Use explicit per-user API credentials and avoid shared/default credentials. + +If a workflow must fail instead of provisioning missing query-endpoint state, add `--no-auto-enable` and surface the error to the user. Do not silently retry without it. + ### Read-only guardrails (defense-in-depth) Before sending any query, check that the first SQL token is not one of: @@ -85,18 +110,22 @@ Before sending any query, check that the first SQL token is not one of: INSERT, ALTER, DROP, TRUNCATE, CREATE, DELETE, SYSTEM, OPTIMIZE, RENAME, GRANT, REVOKE ``` -If it is, refuse. The real enforcement layer is the read-only endpoint database role in ClickHouse Cloud - client-side checks are defense-in-depth only. +If it is, refuse. The real enforcement layer should be least-privilege credentials and service-side access controls - client-side checks are defense-in-depth only. ### Error handling -| HTTP status | Likely cause | Action | -| ----------- | ------------ | ------ | -| `{"ok":1}` | Success | Proceed | -| 401 | Wrong key ID or secret; extra whitespace; key disabled | Re-check credentials | -| 403 | Key not authorized on this endpoint; IP allowlist | Confirm key is in endpoint's authorized list | -| 4xx/5xx | Query error or service issue | Surface full error body to user | +Surface the full command error and exit status on errors, but redact secrets. Do not retry silently. + +### Saved-query Query API endpoint URLs - narrow exception + +Use a ClickHouse Cloud saved-query Query API endpoint URL only when the user explicitly provides an endpoint for the exact saved query or parameterized saved query needed. Do not use this path for schema discovery or open-ended analyst SQL. + +When using this exception: -Surface the full HTTP status and response body on errors. Do not retry silently. +- Treat the endpoint URL as required and endpoint-specific; do not invent or require `CH_QUERY_API_URL` as part of the default analyst setup. +- Confirm that the saved query's semantics, parameters, and output format match the user's request. +- Do not send arbitrary `{ "sql": "..." }` payloads; the endpoint runs its configured query. +- Keep credentials in local environment variables and never print secrets. --- @@ -168,7 +197,7 @@ Safety checks: - Filter by time window whenever possible. - Avoid `SELECT *` except tiny schema previews. - Check row counts before exporting large result sets. -- Add `--format` or `?format=JSONEachRow` for machine-readable output you intend to parse downstream. +- Add `--format JSONEachRow` or another machine-readable output format you intend to parse downstream. ## Bounded queries and large tables @@ -194,6 +223,7 @@ When returning query results, include: ## Auth and secret handling - Never print API keys, secrets, or passwords into the conversation or commit them. -- For Cloud: use `CH_API_KEY` / `CH_API_SECRET` from local environment variables. If missing, stop and prompt the user to set them. Do not fall back to shared or default credentials. +- For Cloud analytics: use explicit per-user `CH_API_KEY` / `CH_API_SECRET` credentials with `clickhousectl cloud service query`. If required connection details are missing, stop and prompt the user to set them. Do not fall back to shared or default credentials. +- For saved-query Query API endpoint URLs: use only an explicit endpoint provided for the exact query needed; do not require or recommend `CH_QUERY_API_URL` for the default analyst workflow. - For local servers: no cloud credentials needed. - `clickhousectl cloud auth logout` clears saved clickhousectl OAuth tokens and API keys. From bbdcd4b70c7e3b91472755df04a5fc855d111ade Mon Sep 17 00:00:00 2001 From: alex-lum Date: Mon, 29 Jun 2026 14:00:25 -0700 Subject: [PATCH 4/4] docs(clickhouse-data-analyst): switch Cloud analytics to direct Query API endpoint Replace `clickhousectl cloud service query` with the configured direct ClickHouse Query API endpoint for Cloud analytics workflows. The previous approach relied on local service-query-key state and could auto-provision keys, which is undesirable. Update README, SKILL.md, and clickhouse skill docs to reflect the new preferred path using per-user `CH_API_KEY` / `CH_API_SECRET` credentials against the configured Query API URL directly. --- plugins/clickhouse-data-analyst/README.md | 6 +- .../skills/data-analyst/SKILL.md | 7 +- .../data-analyst/skills/clickhouse/SKILL.md | 115 +++++++++--------- 3 files changed, 64 insertions(+), 64 deletions(-) diff --git a/plugins/clickhouse-data-analyst/README.md b/plugins/clickhouse-data-analyst/README.md index 524fc102..f125e37a 100644 --- a/plugins/clickhouse-data-analyst/README.md +++ b/plugins/clickhouse-data-analyst/README.md @@ -32,13 +32,13 @@ Cline automatically uses the `data-analyst` skill and its supporting ClickHouse ## Requirements -- `clickhousectl` for ClickHouse Cloud, local, and host/port workflows. -- Per-user ClickHouse Cloud API credentials (`CH_API_KEY` / `CH_API_SECRET`) and a service name or ID for Cloud analytics. +- Per-user ClickHouse Cloud API credentials (`CH_API_KEY` / `CH_API_SECRET`) authorized on the configured Prod Query API endpoint for Cloud analytics. +- `clickhousectl` for local and host/port ClickHouse workflows. - Python dependencies as required by the bundled chDB skills when analyzing local files. ## Security Notes -The bundled skill instructs agents to avoid ClickHouse MCP tools, use explicit per-user API key/secret credentials for ClickHouse Cloud analytics, avoid requiring `CH_QUERY_API_URL` for the default analyst workflow, ask before expensive or unbounded queries, and avoid echoing credentials or secrets. +The bundled skill instructs agents to avoid ClickHouse MCP tools, use the configured direct Query API endpoint with per-user API key/secret credentials for ClickHouse Cloud analytics, avoid `clickhousectl cloud service query` and its automatic service-query-key provisioning, ask before expensive or unbounded queries, and avoid echoing credentials or secrets. ## Attribution diff --git a/plugins/clickhouse-data-analyst/skills/data-analyst/SKILL.md b/plugins/clickhouse-data-analyst/skills/data-analyst/SKILL.md index fe27d72e..edad4463 100644 --- a/plugins/clickhouse-data-analyst/skills/data-analyst/SKILL.md +++ b/plugins/clickhouse-data-analyst/skills/data-analyst/SKILL.md @@ -9,9 +9,8 @@ Act as an interactive data analyst over ClickHouse. The job is not to run the fi CRITICAL: this skill never uses ClickHouse MCP tools. All queries go through the paths defined in `skills/clickhouse/`: -- ClickHouse Cloud analytics: use `clickhousectl cloud service query` with explicit per-user ClickHouse Cloud API credentials (`CH_API_KEY` / `CH_API_SECRET`) and a service `--name` or `--id`. Do not require or recommend `CH_QUERY_API_URL` for the analyst workflow. +- ClickHouse Cloud analytics: use the configured direct ClickHouse Query API endpoint with per-user credentials (`CH_API_KEY` / `CH_API_SECRET`). Do not use `clickhousectl cloud service query`; it depends on local service-query-key state and may auto-provision keys. - Local or host/port server: use `clickhousectl local client`. -- Saved-query Query API endpoint URLs: use only when the user explicitly provides one for the exact preconfigured query or parameterized query needed. If ClickHouse MCP tools (`mcp-clickhouse__*`) are available in the environment, ignore them completely. @@ -21,7 +20,7 @@ Sub-skills live in `skills/`. Load only the sub-skill directory needed for the c Authored for this analyst workflow: -- `skills/clickhouse/` - connect to ClickHouse Cloud via `clickhousectl cloud service query` with explicit API key/secret credentials, to local/host-port servers via `clickhousectl local client`, or to explicit saved-query endpoint URLs only for preconfigured queries. Load before executing any SQL. +- `skills/clickhouse/` - connect to ClickHouse Cloud via the configured direct Query API endpoint with per-user API key/secret credentials, or to local/host-port servers via `clickhousectl local client`. Load before executing any SQL. - `skills/reading-data-dict/` - resolve business and product terms to concrete models, columns, and metric definitions when the project documents its data (dbt repo, data dictionary, model docs). - `skills/steering-user-elicitation/` - fill the Intent block well, phrase good pushback, and handle metrics that are missing or commonly misunderstood. - `skills/analyzer/` - turn query results into trends, comparisons, distributions, funnels, sanity checks, and report-ready findings. @@ -82,7 +81,7 @@ Elicitation is an invariant, not just step 1. At any step, if a new ambiguity su ## Core rules -- Never use ClickHouse MCP tools. Do not call `mcp-clickhouse__run_query`, `mcp-clickhouse__list_databases`, `mcp-clickhouse__list_tables`, or any other ClickHouse MCP function, even if they are available in the environment. For ClickHouse Cloud analytics, use `clickhousectl cloud service query` with explicit per-user API key/secret credentials as described in `skills/clickhouse/`. For local servers, use `clickhousectl local client`. Do not require or recommend `CH_QUERY_API_URL` for open-ended analysis or schema exploration. +- Never use ClickHouse MCP tools. Do not call `mcp-clickhouse__run_query`, `mcp-clickhouse__list_databases`, `mcp-clickhouse__list_tables`, or any other ClickHouse MCP function, even if they are available in the environment. For ClickHouse Cloud analytics, use the configured direct Query API endpoint with per-user `CH_API_KEY` / `CH_API_SECRET` credentials as described in `skills/clickhouse/`. For local servers, use `clickhousectl local client`. Do not use `clickhousectl cloud service query` for Cloud analytics. - Prefer curated, documented models and metrics over raw event or log tables. - State the definitions, filters, time window, and assumptions used. - Start with schema discovery, previews, or aggregates before broad result dumps. diff --git a/plugins/clickhouse-data-analyst/skills/data-analyst/skills/clickhouse/SKILL.md b/plugins/clickhouse-data-analyst/skills/data-analyst/skills/clickhouse/SKILL.md index 54fc0deb..2aec0f75 100644 --- a/plugins/clickhouse-data-analyst/skills/data-analyst/skills/clickhouse/SKILL.md +++ b/plugins/clickhouse-data-analyst/skills/data-analyst/skills/clickhouse/SKILL.md @@ -1,6 +1,6 @@ --- name: clickhouse -description: Connect to and query ClickHouse (a local server or a ClickHouse Cloud service) from the terminal. For ClickHouse Cloud analytics, use clickhousectl cloud service query with explicit CH_API_KEY and CH_API_SECRET credentials plus a service name or ID; do not require CH_QUERY_API_URL. For local or host/port servers, use clickhousectl local client. Use saved-query Query API endpoint URLs only when the user explicitly provides an endpoint for the exact preconfigured query needed. Use when the user wants to run SQL against ClickHouse, explore schemas and tables, inspect Cloud services, or authenticate. For building a local dev environment or deploying to Cloud, defer to the official ClickHouse skills (see Scope). +description: Connect to and query ClickHouse (a local server or a ClickHouse Cloud service) from the terminal. For ClickHouse Cloud analytics, use the configured direct ClickHouse Query API endpoint with per-user CH_API_KEY and CH_API_SECRET credentials; do not use clickhousectl cloud service query. For local or host/port servers, use clickhousectl local client. Use when the user wants to run SQL against ClickHouse, explore schemas and tables, inspect Cloud services, or authenticate. For building a local dev environment or deploying to Cloud, defer to the official ClickHouse skills (see Scope). --- # ClickHouse Connection and Queries @@ -9,11 +9,10 @@ Pick the right access path for the target: | Target | Path | | ------ | ---- | -| ClickHouse Cloud analytics | `clickhousectl cloud service query` with explicit `CH_API_KEY` / `CH_API_SECRET` and service `--name` or `--id` | +| ClickHouse Cloud analytics | Configured direct ClickHouse Query API endpoint with per-user `CH_API_KEY` / `CH_API_SECRET` | | Local server or any host/port | `clickhousectl local client` | -| Saved-query Query API endpoint URL | Only for the exact preconfigured query or parameterized query exposed by that endpoint | -Do not require or recommend `CH_QUERY_API_URL` for the analyst workflow. Saved-query Query API endpoint URLs are not the default path for open-ended analysis or schema exploration. +Do not use `clickhousectl cloud service query` for Cloud analytics. It depends on locally stored service-query-key state and may auto-provision keys, which causes key sprawl and inconsistent behavior across machines. This skill does not use the ClickHouse MCP server. @@ -30,77 +29,74 @@ These are vendored from https://github.com/ClickHouse/agent-skills (Apache-2.0). --- -## Path A: ClickHouse Cloud - clickhousectl with API key/secret +## Path A: ClickHouse Cloud - direct Query API with per-user key -Use `clickhousectl cloud service query` with explicit per-user ClickHouse Cloud API credentials. This path supports the analyst workflow's ad-hoc SQL, including schema discovery, previews, metric queries, and sanity checks, without asking the user for a Query API endpoint URL. +Use the configured direct ClickHouse Query API endpoint with the user's per-user ClickHouse Cloud API key. This is the required Cloud path for the Data Analyst skill. It avoids `clickhousectl cloud service query`, local service-query-key state, automatic key provisioning, and key sprawl. -Do not require or recommend `CH_QUERY_API_URL` for open-ended analysis. Saved-query Query API endpoint URLs are a narrow exception for preconfigured queries only. +The Query API endpoint is team-managed and should be authorized for each `data-agent-` API key with a read-only database role. Users should only need to store their own key ID and secret locally. ### Credentials Read from the local environment: ```bash -CH_API_KEY # required - ClickHouse Cloud API key ID -CH_API_SECRET # required - ClickHouse Cloud API key secret -CH_SERVICE # required unless passing --id - ClickHouse Cloud service name -CH_SERVICE_ID # required unless passing --name - ClickHouse Cloud service ID -CH_DATABASE # optional - target database +CH_API_KEY # required - per-user ClickHouse Cloud API key ID +CH_API_SECRET # required - per-user ClickHouse Cloud API key secret ``` If `CH_API_KEY` or `CH_API_SECRET` is missing, stop and tell the user: ```text -Missing ClickHouse Cloud API credentials. +Missing ClickHouse Query API credentials. Set CH_API_KEY and CH_API_SECRET in your local environment. -Use your per-user ClickHouse Cloud API key. Do not paste secrets into chat. +Use your per-user data-agent- ClickHouse Cloud API key. +Do not paste secrets into chat and do not use a shared org-wide key. ``` -If neither `CH_SERVICE` nor `CH_SERVICE_ID` is available, ask the user which ClickHouse Cloud service to query. - Never print `CH_API_SECRET`. -### Running queries - -Use a service name or service ID. Prefer service ID when available because names can be ambiguous. +### Endpoint -By service name: +Use the configured Prod Query API endpoint for this skill. Do not ask each user to create their own endpoint or use `clickhousectl cloud service query`. ```bash -clickhousectl cloud service query \ - --api-key "$CH_API_KEY" \ - --api-secret "$CH_API_SECRET" \ - --name "$CH_SERVICE" \ - ${CH_DATABASE:+--database "$CH_DATABASE"} \ - -q "SELECT 1 AS ok" \ - --format JSONEachRow +CH_QUERY_ENDPOINT="https://queries.clickhouse.cloud/service/b14c43d1-44f9-4446-ae00-483bcbc6952a/run" ``` -By service ID: +Keep endpoint configuration separate from user credentials. `CH_API_KEY` / `CH_API_SECRET` identify who is calling; the endpoint's database role controls what SQL can run. + +### Running queries ```bash -clickhousectl cloud service query \ - --api-key "$CH_API_KEY" \ - --api-secret "$CH_API_SECRET" \ - --id "$CH_SERVICE_ID" \ - ${CH_DATABASE:+--database "$CH_DATABASE"} \ - -q "SELECT 1 AS ok" \ - --format JSONEachRow +curl -X POST -s \ + --user "$CH_API_KEY:$CH_API_SECRET" \ + "$CH_QUERY_ENDPOINT?format=JSONEachRow" \ + -H 'Content-Type: application/json' \ + -d '{ "sql": "SELECT 1 AS ok" }' ``` -Expected connectivity-check output: +Connectivity check: -```json -{"ok":1} +```bash +curl -X POST -s \ + --user "$CH_API_KEY:$CH_API_SECRET" \ + "$CH_QUERY_ENDPOINT?format=JSONEachRow" \ + -H 'Content-Type: application/json' \ + -d '{ "sql": "SELECT 1 AS ok" }' +# Expected: {"ok":1} ``` -Prefer `--format JSONEachRow`, `CSV`, or `TabSeparated` when you need to parse results in a later step. SQL precedence for query commands is `--query` > `--queries-file` > stdin. - -### Query endpoint provisioning note +Schema access check: -`clickhousectl cloud service query` runs SQL over the Cloud Query API and does not require a local `clickhouse-client` binary or database password. Depending on local state, the first query for a service may provision or reuse a per-service query endpoint binding managed by `clickhousectl`. Use explicit per-user API credentials and avoid shared/default credentials. +```bash +curl -X POST -s \ + --user "$CH_API_KEY:$CH_API_SECRET" \ + "$CH_QUERY_ENDPOINT?format=JSONEachRow" \ + -H 'Content-Type: application/json' \ + -d '{ "sql": "SHOW DATABASES" }' +``` -If a workflow must fail instead of provisioning missing query-endpoint state, add `--no-auto-enable` and surface the error to the user. Do not silently retry without it. +Output format: `JSONEachRow` - parse line-by-line; each non-empty line is a JSON object. ### Read-only guardrails (defense-in-depth) @@ -110,22 +106,27 @@ Before sending any query, check that the first SQL token is not one of: INSERT, ALTER, DROP, TRUNCATE, CREATE, DELETE, SYSTEM, OPTIMIZE, RENAME, GRANT, REVOKE ``` -If it is, refuse. The real enforcement layer should be least-privilege credentials and service-side access controls - client-side checks are defense-in-depth only. +If it is, refuse. The real enforcement layer is the read-only database role configured on the Query API endpoint - client-side checks are defense-in-depth only. ### Error handling -Surface the full command error and exit status on errors, but redact secrets. Do not retry silently. - -### Saved-query Query API endpoint URLs - narrow exception +| HTTP status | Likely cause | Action | +| ----------- | ------------ | ------ | +| `200` with `{"ok":1}` | Success | Proceed | +| 401 | Wrong key ID or secret; extra whitespace; key disabled | Re-check local credentials | +| 403 | Key is not authorized on the endpoint; IP allowlist; database role denial | Confirm the user's key is authorized on the Prod Query API endpoint | +| 4xx/5xx | Query error or service issue | Surface full error body to user, with secrets redacted | -Use a ClickHouse Cloud saved-query Query API endpoint URL only when the user explicitly provides an endpoint for the exact saved query or parameterized saved query needed. Do not use this path for schema discovery or open-ended analyst SQL. +Surface the full HTTP status and response body on errors, but redact secrets. Do not retry silently. -When using this exception: +### Credential policy -- Treat the endpoint URL as required and endpoint-specific; do not invent or require `CH_QUERY_API_URL` as part of the default analyst setup. -- Confirm that the saved query's semantics, parameters, and output format match the user's request. -- Do not send arbitrary `{ "sql": "..." }` payloads; the endpoint runs its configured query. -- Keep credentials in local environment variables and never print secrets. +- Use one per-user API key named `data-agent-`. +- Authorize that key on the Prod Query API endpoint. +- Keep the endpoint database role read-only. +- Do not use shared org-wide keys. +- Do not rely on automatic `clickhousectl cloud service query` provisioning. +- Rotate or revoke the user's key for offboarding. --- @@ -197,7 +198,7 @@ Safety checks: - Filter by time window whenever possible. - Avoid `SELECT *` except tiny schema previews. - Check row counts before exporting large result sets. -- Add `--format JSONEachRow` or another machine-readable output format you intend to parse downstream. +- Add `?format=JSONEachRow` for Cloud Query API calls, or `--format JSONEachRow` for `clickhousectl local client`, when you need machine-readable output downstream. ## Bounded queries and large tables @@ -223,7 +224,7 @@ When returning query results, include: ## Auth and secret handling - Never print API keys, secrets, or passwords into the conversation or commit them. -- For Cloud analytics: use explicit per-user `CH_API_KEY` / `CH_API_SECRET` credentials with `clickhousectl cloud service query`. If required connection details are missing, stop and prompt the user to set them. Do not fall back to shared or default credentials. -- For saved-query Query API endpoint URLs: use only an explicit endpoint provided for the exact query needed; do not require or recommend `CH_QUERY_API_URL` for the default analyst workflow. +- For Cloud analytics: use the configured direct Query API endpoint with explicit per-user `CH_API_KEY` / `CH_API_SECRET` credentials. If credentials are missing, stop and prompt the user to set them. Do not fall back to shared or default credentials. +- Do not use `clickhousectl cloud service query` for Cloud analytics; it depends on local service-query-key state and may auto-provision keys. - For local servers: no cloud credentials needed. -- `clickhousectl cloud auth logout` clears saved clickhousectl OAuth tokens and API keys. +- `clickhousectl cloud auth logout` clears saved clickhousectl OAuth tokens and API keys, but does not manage the per-user `CH_API_KEY` / `CH_API_SECRET` environment variables.