Create a Chatbot using LLM to chat with the help documents

[authorjapps]

✅ Understand the Scope

• Goal: User asks a question → Chatbot understands it → Fetches answer from the documentation pages
• Also: Suggest solutions, provides link to specific doc page, and even generate snippets if user query intends so.

---

AC1:

✅ Recommended Tech Stack:

Frontend (A Simple Chat UI preferably using a OpenSource lib)

Example:
Chatbot UI Template: Chatbot UI by mckaywrigley (great starting point)
(Chose if there is anything simpler or more easier than this)

Backend / API:

Python (FastAPI or anything simpler)
or
Node js
(depending on which you're more comfortable with)

Vector DB for document embeddings:

Use a OpenSource Vector DB.

Example:
Open source: Weaviate, Qdrant, ChromaDB
(or any other one you are comfortable with)

LLM (Language Model)

OpenAI GPT-4 (with API KEY)
or
Anthropics : Claude 3 (Opus, Sonnet, Haiku)
(or any other OpenSource LLM you are comfortable with)

OR use open-source models like Mistral, LLaMA, Gemma with Ollama or Hugging Face Inference API for local hosting

How It Works (Architecture)

Ingest Docs: Crawl or parse your documentation (Markdown, HTML, etc.)

Split & Embed: Chunk the content (e.g., 300–500 tokens) and embed using:

OpenAIEmbeddings (if using OpenAI)

HuggingFaceEmbeddings for local models

Store Embeddings in Vector DB

Query Flow:

User asks a question

Embed the query

Perform similarity search in Vector DB

Retrieve relevant chunks

Send context + query to LLM

Return the answer

(or any other simpler or better architecture you suggest)

🧰 Tools & Libraries

LangChain : Orchestrates everything (embedding, retrieval, prompt templates)

Playwright or BeautifulSoup: For crawling doc website pages

Markdown Parser: For MD-based docs (like markdown-it, mistune, or markdown)

⚙️ Hosting

Frontend: Vercel / Netlify

Backend: Vercel (Next.js) or a AWS VPS or or any other choice

Vector DB: Qdrant (Docker/self-hosted) or any other choice

(or any other free or economically viable hostings if you can suggest)

AC2:

Code repo for Chatbox:
https://github.com/authorjapps/zerocode-tdd-chat-box

Create or ask for more repos if necessary.

[JaynilJaiswal]

Can you please assign me this issue? I have previously worked on RAG-based code generation using embeddings from Stack Overflow stored in ChromaDB. I used open-source Mistral 7B.

[authorjapps]

Can you please assign me this issue? I have previously worked on RAG-based code generation using embeddings from Stack Overflow stored in ChromaDB. I used open-source Mistral 7B.

Hi @JaynilJaiswal, I've added the new repo details for you to push your code when you're ready.
If you need additional repos, just let us know.

Please join the community channel here in Slack and ping me when you get a chance—
happy to collaborate on the solution!