Skip to content

Commit ba7afbd

Browse files
hallelx2claude
andcommitted
docs: add visual diagrams and improve all READMEs
SVG diagrams (Excalidraw-style): - docs/how-it-works.svg: RAG vs Vectorless comparison - docs/architecture.svg: system architecture (SDK → API → infra) - docs/retrieval-flow.svg: 3-step query flow (ToC → Reason → Fetch) README improvements: - Main: inline SVG diagrams, better comparison table, architecture section - TS SDK: retrieval flow diagram, full LLM integration example, error handling, ToC strategies - Python SDK: matching flow diagram, OpenAI integration example, error handling, ToC strategies Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent c9a1c11 commit ba7afbd

6 files changed

Lines changed: 597 additions & 22 deletions

File tree

README.md

Lines changed: 46 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -32,19 +32,52 @@ The result: more accurate retrieval, complete context, and every choice is trace
3232

3333
## How It Works
3434

35-
```
36-
Document uploaded --> Structure extracted --> ToC map generated --> Sections stored
37-
|
38-
At query time: v
39-
Your LLM reads the map --> Picks relevant sections --> Fetches full content
40-
```
35+
<div align="center">
36+
37+
![How Vectorless Works](docs/how-it-works.svg)
38+
39+
</div>
40+
41+
Traditional RAG shatters documents into arbitrary chunks and relies on vector similarity -- a black box that destroys structure and loses context. Vectorless takes a fundamentally different approach:
42+
43+
| | Traditional RAG | Vectorless |
44+
|---|----------------|------------|
45+
| **Splitting** | Arbitrary chunks | Natural sections |
46+
| **Retrieval** | Vector similarity | LLM reasoning |
47+
| **Structure** | Destroyed | Preserved |
48+
| **Traceability** | Black box ranking | Every choice explainable |
49+
| **Context** | Fragments | Complete sections |
50+
51+
## Architecture
52+
53+
<div align="center">
54+
55+
![System Architecture](docs/architecture.svg)
56+
57+
</div>
58+
59+
Vectorless is designed as a modular stack. Your application talks to the API through one of the official SDKs (TypeScript or Python). The API orchestrates document processing across four infrastructure services:
60+
61+
- **Neon (PostgreSQL)** -- stores documents, sections, ToC maps, and metadata
62+
- **Cloudflare R2** -- holds uploaded document files (S3-compatible)
63+
- **Upstash QStash** -- manages background jobs for async document processing
64+
- **Gemini / Claude** -- LLM used to generate summaries and ToC maps
65+
66+
## The Retrieval Flow
67+
68+
<div align="center">
69+
70+
![Query Retrieval Flow](docs/retrieval-flow.svg)
71+
72+
</div>
73+
74+
Retrieval happens in three steps:
75+
76+
1. **Get the Document Map** -- call `getToC()` to receive a structured Table of Contents with section titles, summaries, and IDs. This is lightweight metadata, not the full content.
77+
78+
2. **LLM Reasons Over the Map** -- pass the ToC to your LLM along with the user's query. The LLM reads the summaries and selects exactly the sections relevant to the question. Every choice is visible and explainable.
4179

42-
| Traditional RAG | Vectorless |
43-
|----------------|------------|
44-
| Arbitrary chunks | Natural sections |
45-
| Vector similarity | LLM reasoning |
46-
| Structure destroyed | Structure preserved |
47-
| Black box ranking | Every choice traceable |
80+
3. **Fetch Complete Sections** -- call `fetchSections()` with the IDs your LLM selected. You get back full, unbroken section content -- no fragments, no missing context.
4881

4982
## Quick Start
5083

@@ -155,6 +188,7 @@ vectorless/
155188
openapi/ # OpenAPI 3.1 specification
156189
sdks/
157190
python/ # Python SDK (PyPI: vectorless-sdk)
191+
docs/ # SVG diagrams and documentation assets
158192
```
159193

160194
## Self-Hosting

docs/architecture.svg

Lines changed: 90 additions & 0 deletions
Loading

docs/how-it-works.svg

Lines changed: 106 additions & 0 deletions
Loading

0 commit comments

Comments
 (0)