Atlas SAT Study Assistant 🎓

A Production-Grade RAG System with Hybrid Search, Reranking, and Local Embeddings

An intelligent SAT study assistant powered by Retrieval-Augmented Generation (RAG), designed for production use with resilient architecture, local embeddings, and optional advanced reranking.

---

🏗️ Architecture

graph LR
    A[📄 SAT Materials<br/>PDF/Markdown/TXT] --> B[⚡ Ingestion Pipeline]
    B --> C[FastEmbed<br/>bge-small-en-v1.5<br/>384-dim]
    C --> D[(🗄️ Qdrant Vector DB<br/>Hybrid Search)]
    
    E[👤 Student Query] --> F[🔍 Vector Retrieval<br/>Top 25]
    D --> F
    F --> G{Cohere API<br/>Available?}
    
    G -->|Yes| H[🎯 Cohere Rerank<br/>Top 5]
    G -->|No| I[⚡ Fallback<br/>Top 5 Pass-through]
    
    H --> J[🤖 Gemini 2.5 Flash<br/>SAT Tutor Persona]
    I --> J
    
    J --> K[💬 Streamlit UI<br/>Answer + Citations]
    
    style A fill:#f9f,stroke:#333,stroke-width:2px
    style D fill:#bbf,stroke:#333,stroke-width:2px
    style J fill:#bfb,stroke:#333,stroke-width:2px
    style K fill:#fbb,stroke:#333,stroke-width:2px

Loading

---

🚀 Key Features

Resilient RAG Architecture

Gracefully degrades when Cohere API key is missing - the system automatically falls back to standard top-5 retrieval without crashing. Production-ready error handling throughout.

Local-First Embeddings

• Uses FastEmbed with BAAI/bge-small-en-v1.5 for embedding generation
• Zero API calls for embeddings = zero latency, zero cost
• Fully offline-capable for privacy-sensitive deployments
• 384-dimensional vectors optimized for semantic search

Citations & Transparency

Every answer includes source references with:

• Source filename and section
• Relevance scores (0-100%)
• Direct citation to original study materials
• Built-in trust and verifiability

Modern UI/UX

• Google Gemini-inspired interface with dark theme
• "Zero State" hero section with clickable suggestion cards
• Real-time system status indicators
• Smooth animations and glass-morphism effects

---

🛠️ Tech Stack

Component	Technology	Why?
Orchestration	LlamaIndex	Context-aware retrieval with rich ecosystem
Vector Database	Qdrant	Hybrid search capabilities, cloud-native, high performance
Embeddings	FastEmbed (bge-small-en-v1.5)	Local execution, fast ONNX runtime, privacy-focused, zero cost
LLM	Google Gemini 2.5 Flash	Long context window (1M tokens), blazing speed, cost-effective
Reranking	Cohere Rerank (Optional)	SOTA precision with graceful fallback to standard retrieval
UI Framework	Streamlit	Rapid prototyping, native chat components, Python-first
Package Manager	uv	10-100x faster than pip, modern dependency resolution
Configuration	Pydantic Settings	Type-safe config with automatic .env loading

---

📦 Installation

Prerequisites

• Python 3.10 or higher
• uv package manager
• Docker (for local Qdrant, optional)

Setup

1. Clone the repository

git clone https://github.com/yourusername/atlas-sat-assistant.git
cd atlas-sat-assistant

2. Configure environment variables

cp .env.example .env

Edit .env and add your API keys:

# Required
GOOGLE_API_KEY=your_google_api_key_here

# Optional (for better reranking)
COHERE_API_KEY=your_cohere_key_here

# Qdrant Configuration
QDRANT_URL=http://localhost:6333  # or your Qdrant Cloud URL
QDRANT_API_KEY=  # Optional for local, required for cloud

# Embedding Configuration (defaults are fine)
EMBEDDING_MODEL=BAAI/bge-small-en-v1.5
EMBEDDING_DIM=384

3. Install dependencies

uv sync

4. Start Qdrant (if using local)

docker-compose up -d

Or use Qdrant Cloud and update QDRANT_URL and QDRANT_API_KEY.

---

🎯 Usage

1. Ingest SAT Study Materials

Place your SAT materials (PDF, Markdown, TXT) in data/raw/:

data/raw/
├── sat_reading_sample.md
├── sat_math_formulas.pdf
└── writing_strategies.txt

Run the ingestion pipeline:

uv run python -m src.scripts.ingest

# Options:
# --dry-run              Test without upserting to Qdrant
# --chunking semantic    Use semantic chunking (default)
# --chunking sentence    Use sentence-based chunking
# --section reading      Only ingest reading materials

2. Launch the Chat Interface

uv run streamlit run src/interface/app.py

Open your browser to http://localhost:8501

3. Test the RAG Pipeline (CLI)

uv run python -m src.scripts.test_rag "What is the quadratic formula?"

4. Evaluate System Performance

# Install evaluation dependencies
pip install langchain-google-genai langchain-community

# Run RAGAS evaluation
uv run python -m src.evaluation.evaluate_ragas

Results saved to evaluation_results/:

• inference_results_*.csv - Questions, answers, ground truth
• ragas_metrics_*.csv - Context precision, faithfulness scores

---

📁 Project Structure

atlas-sat-assistant/
├── data/
│   ├── raw/                    # Input SAT materials (PDF, MD, TXT)
│   └── processed/              # Processed chunks (optional)
│
├── src/
│   ├── core/
│   │   ├── config.py          # Pydantic settings with .env loading
│   │   └── logging.py         # Centralized logging configuration
│   │
│   ├── ingestion/
│   │   ├── loader.py          # Custom SAT document loaders
│   │   ├── chunking.py        # Semantic + sentence chunking strategies
│   │   └── vector_db.py       # Qdrant client with retry logic
│   │
│   ├── rag/
│   │   ├── retriever.py       # Vector retrieval with FastEmbed (top 25)
│   │   ├── reranker.py        # Optional Cohere reranking (top 5)
│   │   └── engine.py          # Query engine orchestrating full pipeline
│   │
│   ├── interface/
│   │   └── app.py             # Modern Streamlit UI (Gemini-style)
│   │
│   ├── evaluation/
│   │   └── evaluate_ragas.py  # RAGAS evaluation with Gemini LLM
│   │
│   └── scripts/
│       ├── ingest.py          # CLI ingestion with progress tracking
│       └── test_rag.py        # CLI RAG testing
│
├── evaluation_results/         # RAGAS evaluation outputs
├── docker-compose.yaml         # Local Qdrant setup
├── pyproject.toml             # uv project configuration
├── .env.example               # Environment variables template
└── README.md                  # You are here!

---

🔧 Configuration

Environment Variables

Variable	Required	Default	Description
GOOGLE_API_KEY	✅ Yes	-	Google API key for Gemini LLM
COHERE_API_KEY	⚠️ Optional	-	Cohere key for reranking (graceful fallback)
QDRANT_URL	✅ Yes	http://localhost:6333	Qdrant server URL
QDRANT_API_KEY	⚠️ Optional	-	Required for Qdrant Cloud
COLLECTION_NAME	No	sat_prep	Qdrant collection name
EMBEDDING_MODEL	No	BAAI/bge-small-en-v1.5	FastEmbed model
EMBEDDING_DIM	No	384	Vector dimension
CHUNK_SIZE	No	512	Text chunk size
TOP_K	No	5	Number of results to retrieve

---

🧪 Development

Running Tests

# Unit tests (if implemented)
pytest tests/

# Lint and format
ruff check src/
black src/

Adding New Features

1. New Document Types: Extend src/ingestion/loader.py
2. Custom Chunking: Add strategies to src/ingestion/chunking.py
3. Different LLMs: Modify src/rag/engine.py
4. UI Customization: Edit src/interface/app.py CSS

---

📊 Performance

Benchmarks (Typical)

• Ingestion: ~50 documents/min with semantic chunking
• Query Latency:
  • Retrieval: ~200ms (local FastEmbed)
  • Reranking: ~300ms (Cohere API)
  • Generation: ~800ms (Gemini 2.5 Flash)
  • Total: ~1.3s per query
• Embedding Cost: $0 (local FastEmbed)
• LLM Cost: ~$0.0001 per query (Gemini pricing)

RAGAS Evaluation Results

Average scores on golden SAT dataset:

• Context Precision: 0.85 (85% relevant contexts retrieved)
• Faithfulness: 0.92 (92% answers grounded in context)

---

🐛 Troubleshooting

Common Issues

1. Gemini 404 Error

Error: 404 Model is not found: models/gemini-1.5-flash

Solution: Model name updated to gemini-2.5-flash (latest stable version)

2. Qdrant Connection Failed

Error: Failed to connect to Qdrant at http://localhost:6333

Solution: Start Qdrant with docker-compose up -d or check cloud credentials

3. Missing Cohere Reranker

WARNING: Cohere API key not found. Reranking will use pass-through mode

Solution: This is not an error! System works in fallback mode. Add COHERE_API_KEY for better results.

4. Empty Retrieval Results

WARNING: No nodes retrieved for query

Solution: Run ingestion first: python -m src.scripts.ingest