Drive ClickHouse metrics DDL from .sql templates + preinit bulker metrics destination

[absorbb]

Summary

Two related pieces of ClickHouse metrics-infra cleanup.

Console — single source of truth for the metrics ClickHouse DDL

clickhouse-init.ts previously hardcoded the events-log / metrics DDL as query strings, and prisma/events_log.sql / prisma/metrics.sql were separate, hand-applied copies that had drifted (the .sql file was missing task_log and dead_letter, which the TS created). The metrics pipeline had no code counterpart at all — its DDL was duplicated a third time, inlined in the integration test harness.

This makes the .sql files the single source, rendered and executed by the TS:

• Rename prisma/{events_log,metrics}.sql → *.clickhouse.sql and templatize them with placeholders the runner fills in:
  • ${db}, ${onCluster}, ${engine:Family:znode} (→ Replicated<Family>(...) on a cluster, plain <Family>() single-node), ${user}/${password}, and a -- @ch-settings k=v directive for per-statement ClickHouse settings (the TTL's allow_suspicious_ttl_expressions, the refresh MV's allow_experimental_refreshable_materialized_view).
• clickhouse-init.ts renders → splits → runs the templates. initEventsLogTables now also emits task_log/dead_letter; new initMetricsTables runs the metrics pipeline (ordered dependencies-first).
• admin/events-log-init calls both; the integration harness now runs the same templates instead of its inlined DDL copy.
• Add a vitest raw-.sql loader (Next already had raw-loader) and a *.sql type declaration.
• Remove the unused mv_active_incoming3 / to_mv_active_incoming3 objects (referenced nowhere).

Helm — preinitialize bulker's metrics ClickHouse destination

Render BULKER_INTERNAL_METRICS_DESTINATION into the bulker Deployment (mirrors the prod config), so bulker's special metrics destination and its predefined tables are set up. Connection resolves per field, highest precedence first:

1. explicit bulker.internalMetricsDestination.*;
2. derived from env.common.CLICKHOUSE_URL when set — hostname, user:password userinfo, /database path, and scheme → protocol (https → clickhouse-secure, else clickhouse);
3. in-cluster helm-deps ClickHouse fallback.

So pointing the stack at an external ClickHouse via CLICKHOUSE_URL moves the metrics destination with it — no split-brain where bulker writes metrics to one ClickHouse while the console reads from another.

Testing

• tsc --noEmit: clean.
• Console unit tests: 22/22 pass.
• Console integration tests (real Postgres + ClickHouse 25.4 via testcontainers): the templates apply against real ClickHouse — including the experimental refresh MV. Result 31 pass / 6 fail, identical to newjitsu before this change (the 6 are a pre-existing default auth failure on the cutoff dictionary in this environment, not a regression).
• helm lint clean; helm template verified for in-cluster fallback, external http (→ native clickhouse), external https (→ clickhouse-secure), and explicit-override-wins.

🤖 Generated with Claude Code

[jitsu-code-review]

Reviewed the ClickHouse init/template migration, integration setup updates, and Helm metrics destination wiring. I did not find clear correctness, security, or user-visible regression issues in this diff. The template rendering and statement execution order look consistent with existing behavior, and the Helm fallback/override precedence appears internally coherent.

[vklimontovich]

How about something like this

create table IF NOT EXISTS ${db}.task_log${onCluster}
(
    task_id String,
    sync_id LowCardinality(String),
    timestamp DateTime64(3)
)
engine = ${engine:MergeTree:task_log}
PARTITION BY toYYYYMM(timestamp)
ORDER BY (task_id, sync_id, timestamp)
TTL toDateTime(timestamp) + INTERVAL 3 MONTH DELETE;

ALTER TABLE ${db}.task_log${onCluster} ADD COLUMN IF NOT EXISTS level LowCardinality(String);
ALTER TABLE ${db}.task_log${onCluster} ADD COLUMN IF NOT EXISTS logger LowCardinality(String);
ALTER TABLE ${db}.task_log${onCluster} ADD COLUMN IF NOT EXISTS message String;

It establishing a pattern for further schema changes (adding columns).

[vklimontovich]

How changes in the view is handled? E.g. we add a column, will the column be added to a view after deploy?