🧠 MemVault - AI-Powered Digital Memory System

Your personal AI memory assistant via WhatsApp - Store, search, and retrieve memories using natural language through text and voice.

🎯 Overview

MemVault is a multi-tenant digital memory system that integrates with WhatsApp to provide seamless memory storage and retrieval. Send text or voice messages to store memories, and query them using natural language—all encrypted and secure.

Key Features

• 🔐 Multi-Tenant Authentication - Per-user isolation with WhatsApp phone verification
• 🎤 Voice-to-Memory - Automatic speech-to-text conversion using OpenAI Whisper
• 🧠 Intelligent Intent Detection - LLM-powered classification of store vs. query intents
• 🔍 Hybrid Search - BM25 + Vector search for optimal retrieval accuracy
• 🔒 Encryption at Rest - AES-256-GCM encryption for all stored memories
• ⚡ Scalable Architecture - Designed for 1000s of concurrent users

🏗️ Architecture

┌─────────────────────────────────────────────────────────────────────────────┐
│                              MEMVAULT ARCHITECTURE                          │
├─────────────────────────────────────────────────────────────────────────────┤
│                                                                             │
│  ┌─────────────┐     ┌─────────────────┐     ┌──────────────────────────┐  │
│  │  WhatsApp   │────▶│   Twilio API    │────▶│     FastAPI Backend      │  │
│  │   Client    │◀────│   (Webhook)     │◀────│                          │  │
│  └─────────────┘     └─────────────────┘     │  ┌────────────────────┐  │  │
│                                              │  │   Authentication   │  │  │
│                                              │  │   (Phone-based)    │  │  │
│                                              │  └────────────────────┘  │  │
│                                              │           │              │  │
│                                              │           ▼              │  │
│                                              │  ┌────────────────────┐  │  │
│                                              │  │    LangGraph Flow  │  │  │
│  ┌─────────────────────────────────────┐    │  │                    │  │  │
│  │         LANGGRAPH WORKFLOW          │    │  │  ┌──────────────┐  │  │  │
│  │                                     │    │  │  │ Voice→Text   │  │  │  │
│  │  ┌─────────┐    ┌─────────────┐    │    │  │  │  (Whisper)   │  │  │  │
│  │  │  Input  │───▶│   Intent    │    │    │  │  └──────┬───────┘  │  │  │
│  │  │ Process │    │  Classifier │    │    │  │         │          │  │  │
│  │  └─────────┘    │   (LLM)     │    │    │  │         ▼          │  │  │
│  │                 └──────┬──────┘    │    │  │  ┌──────────────┐  │  │  │
│  │                        │           │    │  │  │   Intent     │  │  │  │
│  │           ┌────────────┴───────┐   │    │  │  │  Classifier  │  │  │  │
│  │           │                    │   │    │  │  │    (LLM)     │  │  │  │
│  │           ▼                    ▼   │    │  │  └──────┬───────┘  │  │  │
│  │    ┌──────────┐         ┌─────────┐│    │  │         │          │  │  │
│  │    │  STORE   │         │  QUERY  ││    │  │    ┌────┴────┐     │  │  │
│  │    │  Memory  │         │ Search  ││    │  │    │         │     │  │  │
│  │    └────┬─────┘         └────┬────┘│    │  │    ▼         ▼     │  │  │
│  │         │                    │     │    │  │  Store    Search   │  │  │
│  │         └────────┬───────────┘     │    │  │  Memory   Memory   │  │  │
│  │                  ▼                 │    │  └────────────────────┘  │  │
│  │         ┌─────────────┐            │    └──────────────────────────┘  │
│  │         │  Response   │            │                                  │
│  │         │  Generator  │            │                                  │
│  │         └─────────────┘            │                                  │
│  └─────────────────────────────────────┘                                 │
│                                                                          │
│  ┌───────────────────────────────────────────────────────────────────┐   │
│  │                         DATA LAYER                                │   │
│  │                                                                   │   │
│  │   ┌──────────────────┐    ┌───────────────────────────────────┐   │   │
│  │   │   PostgreSQL     │    │           Qdrant                  │   │   │
│  │   │                  │    │                                   │   │   │
│  │   │  • User accounts │    │  • Vector embeddings (encrypted)  │   │   │
│  │   │  • Auth tokens   │    │  • BM25 sparse vectors            │   │   │
│  │   │  • Audit logs    │    │  • Multi-tenant collections       │   │   │
│  │   │  • Usage metrics │    │  • Hybrid search indexes          │   │   │
│  │   └──────────────────┘    └───────────────────────────────────┘   │    │
│  │                                                                   │    │
│  └───────────────────────────────────────────────────────────────────┘    │
│                                                                           │
│                                                                           │
└───────────────────────────────────────────────────────────────────────────┘

📁 Project Structure

memvault/
├── backend/
│   ├── app/
│   │   ├── api/              # FastAPI routes & webhooks
│   │   │   ├── __init__.py
│   │   │   ├── routes.py     # API endpoints
│   │   │   └── webhooks.py   # WhatsApp/Twilio webhooks
│   │   ├── core/             # Core configuration
│   │   │   ├── __init__.py
│   │   │   ├── config.py     # Settings & environment
│   │   │   ├── security.py   # Encryption & auth
│   │   │   └── exceptions.py # Custom exceptions
│   │   ├── db/               # Database layer
│   │   │   ├── __init__.py
│   │   │   ├── postgres.py   # PostgreSQL models & queries
│   │   │   └── qdrant.py     # Qdrant vector DB operations
│   │   ├── llm/              # LangGraph & AI components
│   │   │   ├── __init__.py
│   │   │   ├── graph.py      # LangGraph workflow
│   │   │   ├── embeddings.py # Text embeddings
│   │   │   └── whisper.py    # Speech-to-text
│   │   ├── services/         # Business logic
│   │   │   ├── __init__.py
│   │   │   ├── memory.py     # Memory CRUD operations
│   │   │   └── search.py     # Hybrid search implementation 
│   │   └── main.py           # FastAPI application entry
│   ├── tests/                # Test suite
│   │   ├── __init__.py
│   │   ├── conftest.py       # Pytest fixtures
│   │   ├── test_api.py       # API endpoint tests
│   │   ├── test_graph.py     # LangGraph workflow tests
│   │   ├── test_search.py    # Search functionality tests
│   │   └── test_security.py  # Encryption tests
│   ├── Dockerfile
│   ├── requirements.txt
│   └── pyproject.toml                 # Kubernetes manifests
├── docs/                     # Documentation
│   ├── api.md                # API documentation
│   ├── deployment.md         # Deployment guide
│   └── architecture.png      # Architecture diagram
├── scripts/                  # Utility scripts
│   ├── setup.sh              # Development setup
│   └── migrate.py            # Database migrations
├── .env.example              # Environment template
├── .gitignore
├── LICENSE
└── README.md

🚀 Quick Start

Prerequisites

• Python 3.11+
• Docker & Docker Compose
• Twilio account with WhatsApp sandbox
• OpenAI API key
• Qdrant Cloud or local instance

1. Clone & Setup

git clone https://github.com/yourusername/memvault.git
cd memvault

# Copy environment template
cp .env.example .env

2. Configure Environment Variables

Edit .env with your credentials:

# API Keys
OPENAI_API_KEY=sk-...
TWILIO_ACCOUNT_SID=AC...
TWILIO_AUTH_TOKEN=...
TWILIO_WHATSAPP_NUMBER=

# Database
POSTGRES_URL=postgresql://user:pass@localhost:5432/memvault
QDRANT_URL=http://localhost:6333
QDRANT_API_KEY=your-qdrant-key

# Security
ENCRYPTION_KEY=your-32-byte-base64-key
JWT_SECRET=your-jwt-secret

# Observability
OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317
PROMETHEUS_PORT=9090

3. Start with Docker Compose

# Start all services (dev mode)
docker-compose up -d

# Check logs
docker-compose logs -f backend

4. Configure Twilio Webhook

1. Go to Twilio Console
2. Set webhook URL: https://your-domain.com/api/v1/webhook/whatsapp
3. Join the sandbox: Send "join " to the WhatsApp number

5. Test It Out

Send a message to your WhatsApp bot:

• Store a memory: "Remember that my favorite restaurant is Sushi Ko on Main Street"
• Query memories: "What's my favorite restaurant?"
• Voice notes: Send a voice message and it will be transcribed and processed

🔧 Configuration

Environment Variables Reference

Variable	Description	Required
OPENAI_API_KEY	OpenAI API key for embeddings & LLM	✅
TWILIO_ACCOUNT_SID	Twilio account SID	✅
TWILIO_AUTH_TOKEN	Twilio auth token	✅
TWILIO_WHATSAPP_NUMBER	WhatsApp-enabled number	✅
POSTGRES_URL	PostgreSQL connection string	✅
QDRANT_URL	Qdrant server URL	✅
QDRANT_API_KEY	Qdrant API key	❌
ENCRYPTION_KEY	32-byte base64 key for AES-256	✅
JWT_SECRET	Secret for JWT tokens	✅
LOG_LEVEL	Logging level (DEBUG/INFO/WARNING)	❌
MAX_MEMORIES_PER_USER	Memory limit per user	❌
OTEL_EXPORTER_OTLP_ENDPOINT	OpenTelemetry endpoint	❌

Generating Encryption Key

python -c "import secrets; import base64; print(base64.b64encode(secrets.token_bytes(32)).decode())"

🧪 Testing

# Run all tests
pytest

# Run with coverage
pytest --cov=app --cov-report=html

# Run specific test file
pytest tests/test_api.py -v

# Run integration tests
pytest tests/ -m integration

🚢 Deployment

Docker (Recommended)

# Build production image
docker build -t memvault:latest ./backend

# Run with production compose
docker-compose -f infra/docker-compose.prod.yml up -d

Environment-Specific Configs

Environment	Config File	Notes
Development	docker-compose.yml	Hot reload, debug logging
Staging	docker-compose.staging.yml	Test webhooks, sandbox
Production	docker-compose.prod.yml	Full HA, autoscaling

🔐 Security

Encryption

• At Rest: AES-256-GCM encryption for all memory content
• In Transit: TLS 1.3 for all connections
• Keys: Environment-based key management (use Vault in production)

Authentication

• Phone number verification via Twilio
• JWT tokens with configurable expiry
• Rate limiting per user

Multi-Tenancy

• Complete data isolation per user
• Separate Qdrant collections per tenant
• Row-level security in PostgreSQL

📈 Scaling

Database Scaling

• PostgreSQL: Connection pooling with PgBouncer
• Qdrant: Distributed mode with sharding

Horizontal Scaling

• Stateless FastAPI workers behind load balancer
• Redis for session state (if needed)
• Kubernetes HPA for auto-scaling

🤝 Contributing

1. Fork the repository
2. Create a feature branch (git checkout -b feature/great-feature)
3. Commit changes (git commit -m 'Add great feature')
4. Push to branch (git push origin feature/great-feature)
5. Open a Pull Request

📄 License

This project is licensed under the MIT License - see the LICENSE file.

🙏 Acknowledgments

• LangGraph - Workflow orchestration
• Qdrant - Vector database
• FastAPI - Web framework
• Twilio - WhatsApp integration

---

---

vibe coded with ❤️ for the future of personal memory