CodeRAG Navigator

A Retrieval-Augmented Generation (RAG) tool that lets you upload a codebase and ask questions about it using natural language. It combines semantic vector search with BM25 keyword matching to retrieve the most relevant code snippets, then uses Google Gemini to generate grounded answers with source citations.

Features

• Hybrid Retrieval -- blends semantic vector search (Chroma) with BM25 keyword search via an ensemble retriever. Adjust the weight slider in the UI to favor either signal.
• Token-Aware Chunking -- splits code using tiktoken (cl100k_base) at ~1000-token boundaries with 120-token overlap, respecting language-specific constructs (functions, classes, blocks).
• Multi-Language Support -- language-aware separators for Python, JavaScript, Java, C/C++, Go, Ruby, PHP, HTML, CSS, Markdown, JSON, YAML, Bash, and Dockerfiles.
• Rich Metadata -- each chunk carries file path, language, line numbers, and inferred symbol paths (Python via AST, JS/Java via regex) so answers can point to exact locations.
• Repository Inventory -- automatically generates a synthetic document listing all files with preview lines, helping the model answer broad structural questions.
• Conversational Context -- retains the last 10 Q&A pairs per session so follow-up questions stay grounded.
• Source Attribution -- every answer includes expandable source snippets showing the retrieved chunks.

Architecture

Upload (.zip / single file)
        |
        v
  ┌─────────────┐     ┌──────────────────┐
  │  Loader &    │────>│  Chroma Cloud     │  (vector store)
  │  Chunker     │     │  + HuggingFace    │
  │  (tiktoken)  │     │  Embeddings       │
  └─────────────┘     └──────────────────┘
        |                       |
        v                       v
  ┌─────────────┐     ┌──────────────────┐
  │  BM25        │     │  Vector           │
  │  Retriever   │     │  Retriever        │
  └──────┬──────┘     └────────┬─────────┘
         └──────┬──────────────┘
                v
       ┌────────────────┐
       │  Ensemble       │  (weighted merge)
       │  Retriever      │
       └───────┬────────┘
               v
       ┌────────────────┐
       │  Gemini 2.5    │
       │  Flash (LLM)   │
       └───────┬────────┘
               v
         Answer + Sources

Tech Stack

Component	Technology
LLM	Google Gemini 2.5 Flash
Embeddings	HuggingFace all-MiniLM-L6-v2 (384-dim)
Vector Store	Chroma Cloud
Orchestration	LangChain
Tokenizer	tiktoken (cl100k_base)
UI	Streamlit

Project Structure

CodeRAG-Navigator/
├── app/
│   └── main_ui.py                  # Streamlit web interface (entry point)
├── src/
│   ├── config.py                   # Environment variable loading
│   ├── data_processing/
│   │   └── loader.py               # File loading, chunking, metadata enrichment
│   ├── vector_store/
│   │   └── chroma_cloud_manager.py # Chroma Cloud client wrapper
│   └── rag_pipeline/
│       └── qa_chain.py             # RAG chain creation & query execution
├── scripts/
│   └── inspect_chroma.py           # CLI tool for vector store inspection
├── requirements.txt
└── .env.example

Getting Started

Prerequisites

• Python 3.11+
• A Google Gemini API key
• A Chroma Cloud account (API key, tenant ID, and database name)

Installation

python3.11 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

Configuration

Copy the example env file and fill in your credentials:

cp .env.example .env

GEMINI_API_KEY=your-gemini-api-key
CHROMA_API_KEY=your-chroma-api-key
CHROMA_TENANT_ID=your-chroma-tenant-id
CHROMA_DATABASE=your-chroma-database

Run

streamlit run app/main_ui.py

Opens at http://localhost:8501.

Usage

1. Upload -- drag a .zip archive or a single code file into the sidebar.
2. Name the collection -- give it a memorable name (or let one be auto-generated).
3. Adjust retrieval weight -- slide between pure keyword (BM25) and pure semantic search.
4. Ask questions -- type natural-language questions in the chat. Follow-ups use conversation context automatically.
5. View sources -- expand the "Source Documents" section under any answer to see the exact chunks used.

Inspecting the Vector Store

Use the helper script to browse stored chunks or run ad-hoc similarity searches:

# List all collections
python scripts/inspect_chroma.py --list-collections --client cloud

# View chunks in a collection
python scripts/inspect_chroma.py --collection my_codebase --limit 5 --client cloud

# Similarity search
python scripts/inspect_chroma.py --collection my_codebase --query "how does auth work" --limit 3 --format table --client cloud

Pass --width 0 to print full (untruncated) chunk contents. Omit --client cloud to inspect a local Chroma directory instead.