Drift - Click Analytics Pipeline

Data Flow

Generator → Filebeat → Kafka → Logstash → Elasticsearch + ClickHouse

The generator produces random click events every 0.5-3 seconds.

Quick Start

docker compose up -d

Verify Data Flow

ClickHouse

# Row count
docker exec drift-clickhouse clickhouse-client -q "SELECT count() FROM analytics.clicks"

# Sample data
docker exec drift-clickhouse clickhouse-client -q "SELECT * FROM analytics.clicks LIMIT 5"

# Clicks by country
docker exec drift-clickhouse clickhouse-client -q "SELECT country_code, count() FROM analytics.clicks GROUP BY country_code"

Elasticsearch

# Document count
curl -s "localhost:9200/clicks-*/_count" | jq .count

# Sample documents
curl -s "localhost:9200/clicks-*/_search?size=3&pretty"

Dashboards

• ClickHouse Play UI: http://localhost:8123/play

Services

Service	Port
Kafka	9092
Elasticsearch	9200
ClickHouse HTTP	8123
ClickHouse Native	9000