Purpose
This repository provides a unified foundation for experimentation, supporting both traditional ML projects (model training, fine-tuning, evaluation) and conversational AI projects (chatbots, LLM-powered assistants). It is designed for senior developers who may not have prior MLOps experience.
This platform supports two project types through a unified experimentation framework:
| Project Type | Use Case | Execution Strategy | Quick Start |
|---|---|---|---|
| Traditional ML | Model training, fine-tuning, offline evaluation | mlflow_model |
Training Route |
| Conversational AI | Chatbots, LLM assistants, structured dialogues | prompt_template |
Conversational AI Route |
Not sure which to choose? → Choosing Project Type Guide
| Your Goal | Entry Point |
|---|---|
| Choose project type | Choosing Project Type |
| Learn the full system end-to-end | Sequential Learning Path |
| Build conversational AI | Conversational AI Route |
| Work on experiments only | Experiment Route |
| Work on event logging | Event Logging Route |
| Work on training/fine-tuning | Training Route |
| Work on offline evaluation | Offline Evaluation Route |
| Work on model promotion | Model Promotion Route |
| Work on analytics/dashboards | Analytics Route |
Follow this order for complete understanding:
- Architecture Overview
- Data Model
- Experiments
- Assignment Service
- Event Ingestion Service
- MLflow Guide
- Training Workflow
- Offline Evaluation
- Model Promotion
- Metabase & Analytics
/
├── README.md ← You are here
├── docs/ ← Core documentation
│ ├── README.md ← Documentation index
│ ├── choosing-project-type.md ← Project type decision guide
│ ├── architecture.md ← System architecture
│ ├── data-model.md ← PostgreSQL schema
│ ├── experiments.md ← Experiment concepts
│ ├── prompts-guide.md ← Prompt management (Conversational AI)
│ ├── conversation-flows.md ← Flow orchestration (Conversational AI)
│ ├── assignment-service.md ← Assignment service overview
│ ├── event-ingestion-service.md ← Event service overview
│ ├── mlflow-guide.md ← MLflow usage (ML projects)
│ ├── training-workflow.md ← Training process (ML projects)
│ ├── offline-evaluation.md ← Offline replay evaluation
│ ├── model-promotion.md ← Promotion lifecycle
│ ├── mlops-concepts.md ← MLOps primer
│ └── routes/ ← Parallel exploration routes
│ ├── conversational-ai-route.md ← Conversational AI quickstart
│ └── ...
├── services/ ← Service specifications
│ ├── assignment-service/
│ ├── event-ingestion-service/
│ ├── prompt-service/ ← Prompt retrieval (Conversational AI)
│ └── flow-orchestrator/ ← Conversation management (Conversational AI)
├── prompts/ ← Prompt template files (Conversational AI)
├── flows/ ← Conversation flow definitions (Conversational AI)
├── examples/ ← End-to-end example projects
│ └── conversational-assistant/ ← Conversational AI example
├── infra/ ← Infrastructure setup
├── pipelines/ ← Pipeline specifications
├── training/ ← Training templates (ML projects)
├── datasets/ ← Dataset conventions (ML projects)
├── analytics/ ← Metabase & queries
└── config/ ← Configuration examples
| Component | Purpose |
|---|---|
| PostgreSQL | Primary metadata store (experiments, events, policies, prompts) |
| MLflow | Model registry and experiment tracking (ML projects) |
| Metabase | Dashboards and experiment visualisation |
| LLM Providers | Anthropic Claude, OpenAI GPT (Conversational AI) |
| Open-source LLMs | LLaMA, Mistral, Qwen, etc. (ML projects) |
| Redis | Session storage for conversation flows (Conversational AI) |
| Object Storage | S3/MinIO for datasets and ML artefacts |
| Orchestrator | Airflow or Prefect |
| Task Queue | Redis/worker for async jobs |
| Docker | Containerisation |
- Choose your project type: Start with Choosing Project Type
- Conversational AI projects: Follow Conversational AI Route
- ML projects: Read MLOps Concepts if unfamiliar with ML workflows
- Using AI assistants? See AI Learning Prompts for active learning strategies
- Follow the Sequential Learning Path for complete understanding
- Explore dashboards in Metabase
- Check out Example Projects for complete working examples
Jump directly to the relevant Exploration Route based on your task.
- Full Onboarding Specification:
onboarding-spec.md - Training & Fine-Tuning Specification:
training-spec.md
Note: Infrastructure endpoints (MLflow, Metabase, API services) will be provided here once the production environment is deployed. For local development, refer to the Infrastructure Guide.