Prepare updates for main PR

.gitignore

@@ -38,6 +38,7 @@ docker-compose.override.yml
 .env.*
 tmp/
 .qdrant_db/
+.cache/dataset_sources/
 
 # Regenerable KB intermediate artifacts (see data/kb/README.md)
 data/kb/raw/

README.en.md

@@ -52,6 +52,7 @@ what you send to that client and on your organization's security policies.
 - Expose ClickAdvisor to AI agents as a local MCP server.
 - Add local links to ClickHouse docs / Altinity KB through embedded Qdrant.
 - Compare rewrite candidates with `EXPLAIN ESTIMATE` when `--connect` is explicitly provided.
+- Build prototype workload reports from sanitized `system.query_log` CSV exports.
 
 ## Project Numbers
 
@@ -308,6 +309,24 @@ Available MCP tools:
 
 More details: [docs/MCP.md](docs/MCP.md).
 
+## Workload Analyzer Prototype
+
+To move from single-query review toward a DBA review queue, ClickAdvisor includes
+a prototype for sanitized `system.query_log` CSV exports:
+
+```bash
+poetry run chadvisor workload \
+  --query-log examples/query_log_sample.csv \
+  --output-format markdown \
+  --top-n 3
+```
+
+It groups similar queries by normalized fingerprint, computes executions,
+total/avg/p95 latency, read rows/bytes, and memory, then runs the representative
+query through the rule engine and ranks top opportunities.
+
+More details: [docs/workload.md](docs/workload.md).
+
 ## Quality Metrics
 
 Current reproducible metrics as of 2026-06-30:
@@ -317,6 +336,8 @@ Current reproducible metrics as of 2026-06-30:
 | Rule detection | `222` synthetic/schema/env benchmark cases | precision `1.000`, recall `1.000`, F1 `1.000` |
 | ML classifier | `synthetic_expanded_v1` train/test split | best test macro F1 `0.691`, best test micro F1 `0.988` |
 | Retrieval | `20` explicit query -> relevant docs pairs | MRR@3 `0.517` with MiniLM-L6 |
+| Risk-label DS pipeline | `20 235` SQL records, group split + holdout | RF holdout macro F1 `0.949`, measured-only macro F1 `0.595` |
+| Workload prototype | sample sanitized `query_log` CSV | normalized groups + top-N risk report |
 
 What was evaluated:
 
@@ -334,7 +355,8 @@ poetry run python scripts/eval/run_benchmark.py \
 
 Methodology: [docs/evaluation.md](docs/evaluation.md),
 [docs/experiments/classifier_ablation.md](docs/experiments/classifier_ablation.md),
-[docs/experiments/retrieval_ablation.md](docs/experiments/retrieval_ablation.md).
+[docs/experiments/retrieval_ablation.md](docs/experiments/retrieval_ablation.md),
+[docs/experiments/risk_labeling_ds_summary.md](docs/experiments/risk_labeling_ds_summary.md).
 
 ## Security And Data Handling
 
@@ -353,6 +375,8 @@ SQL, DDL, EXPLAIN, or environment context to external LLM/API services. This
 reduces risks around compliance, banking secrecy, personal data, trade secrets,
 and internal naming conventions.
 
+More details: [docs/security-local-first.md](docs/security-local-first.md).
+
 ## Development
 
 ```bash
@@ -376,7 +400,7 @@ the rule engine, ML evaluation surface, and local retrieval.
 ## Not Claimed As Ready
 
 - Product generative LLM in the trusted runtime path.
-- Automatic `query_log` analysis.
+- Live `query_log` analysis through `--connect --since`; a CSV prototype exists.
 - Automatic DDL changes.
 - Running `ANALYZE` or replaying queries on user data.
 - Automatically applying Tier 2 design/storage recommendations without DBA review.

README.md

@@ -54,6 +54,7 @@ CI-запуск внутри своего контура, пользовател
 - Подключаться к AI-агентам как локальный MCP-сервер.
 - Добавлять локальные ссылки на документацию ClickHouse и Altinity KB через встроенный Qdrant.
 - Сравнивать варианты переписывания через `EXPLAIN ESTIMATE`, если явно передан `--connect`.
+- Строить prototype workload-отчеты по sanitized CSV export из `system.query_log`.
 
 ## Цифры проекта
 
@@ -311,6 +312,24 @@ poetry run chadvisor mcp-server
 
 Подробности: [docs/MCP.md](docs/MCP.md).
 
+## Workload analyzer prototype
+
+Для перехода от анализа одного SQL к DBA review queue есть прототип анализа
+sanitized `system.query_log` CSV:
+
+```bash
+poetry run chadvisor workload \
+  --query-log examples/query_log_sample.csv \
+  --output-format markdown \
+  --top-n 3
+```
+
+Он группирует похожие запросы по normalized fingerprint, считает executions,
+total/avg/p95 latency, read rows/bytes и memory, затем прогоняет representative
+query через rule engine и ранжирует top opportunities.
+
+Подробности: [docs/workload.md](docs/workload.md).
+
 ## Метрики качества
 
 Текущие воспроизводимые метрики на 2026-06-30:
@@ -320,6 +339,8 @@ poetry run chadvisor mcp-server
 | Детерминированные правила | `222` SQL/schema/env benchmark-кейса | precision `1.000`, recall `1.000`, F1 `1.000` |
 | ML-классификатор | train/test split `synthetic_expanded_v1` | лучший test macro F1 `0.691`, лучший test micro F1 `0.988` |
 | Поиск по документации | `20` явных пар запрос -> релевантная документация | MRR@3 `0.517` на MiniLM-L6 |
+| Risk-label DS контур | `20 235` SQL-записей, group split + holdout | RF holdout macro F1 `0.949`, measured-only macro F1 `0.595` |
+| Workload prototype | sample sanitized `query_log` CSV | normalized groups + top-N risk report |
 
 Что именно оценивалось:
 
@@ -337,7 +358,8 @@ poetry run python scripts/eval/run_benchmark.py \
 
 Подробная методика: [docs/evaluation.md](docs/evaluation.md),
 [docs/experiments/classifier_ablation.md](docs/experiments/classifier_ablation.md),
-[docs/experiments/retrieval_ablation.md](docs/experiments/retrieval_ablation.md).
+[docs/experiments/retrieval_ablation.md](docs/experiments/retrieval_ablation.md),
+[docs/experiments/risk_labeling_ds_summary.md](docs/experiments/risk_labeling_ds_summary.md).
 
 ## Безопасность и данные
 
@@ -356,6 +378,8 @@ EXPLAIN и environment-контекста во внешние LLM или API-с
 compliance, банковской тайны, персональных данных, коммерческой тайны и
 внутренних naming conventions.
 
+Подробности: [docs/security-local-first.md](docs/security-local-first.md).
+
 ## Разработка
 
 ```bash
@@ -380,7 +404,7 @@ Codex и Claude использовались системно в разрабо
 ## Что не заявляется как готовое
 
 - Продуктовая генеративная LLM в доверенном пути выполнения.
-- Автоматический анализ `query_log`.
+- Live-анализ `query_log` через `--connect --since`; сейчас есть CSV prototype.
 - Автоматические DDL-изменения.
 - Выполнение `ANALYZE` или реальный replay запросов на данных.
 - Автоматическое применение Tier 2 design/storage рекомендаций без проверки DBA.

clickadvisor/cli/main.py

@@ -155,6 +155,47 @@ def analyze(
         return
 
 
+@app.command()
+def workload(
+    query_log: Annotated[
+        Path,
+        typer.Option(help="Sanitized CSV export from system.query_log"),
+    ],
+    top_n: Annotated[int, typer.Option(help="Number of normalized query groups to show")] = 10,
+    ch_version: Annotated[str | None, typer.Option(help="24.3")] = None,
+    output_format: Annotated[str, typer.Option(help="console|json|markdown")] = "console",
+    output: Annotated[Path | None, typer.Option(help="Write report to file")] = None,
+) -> None:
+    """Analyze a sanitized query_log CSV export and rank workload risks."""
+    from clickadvisor.workload.analyzer import (
+        analyze_query_log_csv,
+        render_workload_json,
+        render_workload_markdown,
+        workload_report_to_dict,
+    )
+
+    if output_format not in {"console", "json", "markdown"}:
+        raise typer.BadParameter("output_format must be one of: console, json, markdown")
+    if top_n <= 0:
+        raise typer.BadParameter("top_n must be positive")
+
+    report = analyze_query_log_csv(query_log, top_n=top_n, ch_version=ch_version)
+    if output_format == "json":
+        rendered = render_workload_json(report)
+    else:
+        rendered = render_workload_markdown(report)
+
+    if output is not None:
+        output.write_text(rendered + "\n", encoding="utf-8")
+        console.print(f"Wrote workload report to {output}")
+        return
+
+    if output_format == "json":
+        console.print_json(data=workload_report_to_dict(report))
+    else:
+        console.print(rendered)
+
+
 @app.command()
 def index_kb(
     chunks_dir: Annotated[

clickadvisor/workload/__init__.py

@@ -0,0 +1,13 @@
+from clickadvisor.workload.analyzer import (
+    QueryGroup,
+    WorkloadReport,
+    analyze_query_log_csv,
+    render_workload_markdown,
+)
+
+__all__ = [
+    "QueryGroup",
+    "WorkloadReport",
+    "analyze_query_log_csv",
+    "render_workload_markdown",
+]