refactor: Extract shared utilities into ai_brain package#2
Open
Ovbosire wants to merge 2 commits into
Open
Conversation
- Add Knowledge Brain system: ingest AI conversation history (Gemini, Claude, DeepSeek) and local documents into a GraphRAG pipeline - Postgres + pgvector for vector storage and hybrid search - Neo4j knowledge graph with entity/relationship extraction via Ollama - FastAPI web UI with SSE streaming chat + graph visualization (static/index.html) - Unified CLI (cli.py) with interactive menu, query console, ingestion, and daemon commands - Extraction daemon (extract_daemon.py) with thermal throttling and auto-resume - Hybrid GraphRAG search (graph_query.py): vector + keyword + 1-hop graph traversal - Temporal filter syntax in query console (last week, since june, in 2025) - Domain filtering support - Switch file organizer from Gemini to local Ollama (llama3.2) - Add document vault ingester with PDF support (pypdf) - Pinned all dependency versions in requirements.txt - Expanded configuration in .env.example with Postgres, Neo4j, Ollama, thermal throttling settings - Comprehensive README covering architecture, setup, and usage for both components
- Create ai_brain/ package with graph/, query/, ingestion/ subpackages - Extract time-filter parsing into ai_brain/query/time_filter.py - Extract Neo4j driver into ai_brain/graph/driver.py (lazy init + liveness check) - Extract flat RAG search into ai_brain/query/flat_rag.py - Extract shared STOPWORDS into ai_brain/query/text_utils.py - Update app.py, query_brain.py, graph_query.py, extract_entities.py, extract_daemon.py, neo4j_setup.py to import from shared modules - Eliminates ~100 lines of duplicated code across 4+ files
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Extract duplicated code across
app.py,query_brain.py,graph_query.py, and ingestion scripts into a sharedai_brain/package structure, eliminating ~100 lines of duplicated code.Changes
ai_brain/query/time_filter.py— Shared time-filter parsing (was duplicated inapp.pyandquery_brain.py)ai_brain/graph/driver.py— Shared Neo4j driver with lazy initialization and liveness check (was duplicated across 4 files). RaisesConnectionErrorinstead of callingsys.exit(1).ai_brain/query/flat_rag.py— Shared flat RAG vector+keyword search (was duplicated inapp.pyandquery_brain.py)ai_brain/query/text_utils.py— SharedSTOPWORDSset (was duplicated with slight variations inquery_brain.pyandgraph_query.py)app.py— Imports from shared modules; removes ~50 lines of duplicated logicquery_brain.py— Imports from shared modules; removes ~45 lines of duplicated logicgraph_query.py— Imports STOPWORDS and Neo4j driver from shared modules; removes ~25 linesextract_entities.py— Imports Neo4j driver from shared module; removes ~10 linesextract_daemon.py— Imports Neo4j driver from shared moduleneo4j_setup.py— Imports Neo4j driver from shared module; removes ~15 linesValidation
ai_brain/,ai_brain/graph/,ai_brain/query/,ai_brain/ingestion/File Structure