feat(analytics): add support for analytics

[jagan-jaya]

This pull request introduces a comprehensive analytics infrastructure for the Decision Engine project, leveraging Kafka and ClickHouse for real-time event tracking and data aggregation. It also includes updates to configuration files, Docker setup, and database migrations to support the new analytics system.

Analytics Infrastructure Setup

• analytics/README.md: Added detailed documentation for the analytics system architecture, components, configuration, database schema, and troubleshooting steps. This includes instructions for enabling analytics and querying data.
• analytics/migrations/001_routing_events.sql: Created a consolidated SQL migration file for setting up the ClickHouse database schema, Kafka integration, and materialized views for real-time event processing.
• analytics/run_migrations.sh: Added a shell script to automate the execution of analytics migrations in ClickHouse.

Dependency and Configuration Updates

• Cargo.toml: Added new dependencies (http-body-util, kafka, and clickhouse) required for analytics integration.
• config/development.toml and config/docker-configuration.toml: Updated configurations to include analytics settings for Kafka and ClickHouse. Enabled analytics in development but disabled it in the Docker configuration by default. [1] [2]

Docker and Service Updates

• docker-compose.yaml:
  • Renamed container names for clarity (e.g., open-router-local, postgres-db).
  • Added new services for analytics infrastructure: Zookeeper, Kafka, ClickHouse, and an analytics migrator. Defined health checks and profiles for these services. [1] [2] [3] [4] [5]
  • Introduced a new volume for ClickHouse data storage.

Run ./scripts/test_analytics.sh and Change below configs to test directly with application using cargo r

In cargo.toml

[features]
default = ["middleware","postgres"]

In config/development.toml

[analytics]
enabled = true