Skip to content

Feature/vectors split#3

Open
scottgal wants to merge 10 commits into
mainfrom
feature/vectors_split
Open

Feature/vectors split#3
scottgal wants to merge 10 commits into
mainfrom
feature/vectors_split

Conversation

@scottgal
Copy link
Copy Markdown
Owner

This pull request introduces significant documentation improvements and workflow automation for the LucidRAG and DoomSummarizer projects. The main changes include adding detailed design documents for background learning and vector store abstraction, updating the semantic query classifier documentation with new tuning and test details, introducing a new GitHub Actions workflow for NuGet publishing, and removing an obsolete project from the solution.

Documentation Enhancements:

  • Added a comprehensive design document describing the background learning loop for query classification and the abstraction of vector store backends, including implementation phases, CLI integration, and expected impacts.
  • Introduced a new document outlining the three-tier vector store architecture (Qdrant/Postgres for server, DuckDB for research, sqlite-vec for CLI), with configuration, migration, and interface details.
  • Updated the semantic query classifier documentation:
    • Improved composite detection logic and heuristics.
    • Added a detailed tuning checklist, exemplar design rules, and clarified score naming conventions.
    • Increased the diagnostic test matrix to 88 queries and expanded vibe detection coverage. [1] [2]

Automation and Build System:

  • Added a new GitHub Actions workflow (.github/workflows/publish-lucidrag-nuget.yml) to automate publishing LucidRAG Core and Postgres NuGet packages, supporting both tag-based and manual dispatch triggers.

Solution Maintenance:

  • Removed the obsolete LucidRAG project from the solution file (LucidRAG.sln) and cleaned up related configuration entries. [1] [2] [3]

…uctured logging for embedding disagreements with a learning loop for exemplar generation.
…rketing page.

- Added collection auto-routing logic in `AgenticSearchService.cs` to automatically determine and use the best collection if not specified.
- Implemented salient segment caching for follow-up queries in ongoing conversations to improve result relevance.
- Enhanced error handling in `PostgresBm25Service.cs` by checking for valid `topK` values.
- Created a new marketing page (`marketing/index.html`) to promote features and capabilities of the product.
The AI-powered response engine, deterministic resolver, and related augmented response systems have been deleted. This includes associated helper classes, embedding cache logic, and unit tests. These modules are likely deprecated or no longer needed in the runtime core services.
Introduce new marketing components to showcase product use cases, answer FAQs, and present pricing tiers. These include visually rich layouts and interactive elements to enhance user understanding and engagement.
Improved middleware logic for API authentication, including HMAC headers and alternative routing prefixes. Updated UI elements across marketing views for better responsiveness and consistency in typography, spacing, and visual hierarchy. Enhanced pricing tiers with added features, file upload size limits, and detailed overage options.
This commit introduces hosted search pages and improves widget configurability via the new `_DashboardWidgetConfig` partial. It also restructures the dashboard to include tabs for streamlined navigation and updates the key management section with hosted URL options. Enhancements support user-facing customization features such as custom CSS, logos, and domain settings.
Introduces Typesense-based search extensions, replacing Lucene for indexing and search. Adds widget and domain customization entities, middleware improvements for CORS handling, and schema migrations to support new features. Includes enhancements for query parsing and search result curation using Typesense.
Revised shadow DOM handling, added safety checks for CSS and URLs, limited in-flight requests, and improved resilience with request retries and input validations. Updated pricing plans, analytics, and test cases to align with new features.
Updated SaaS plan configurations with stress-test limits and added new features such as query retention and export options. Introduced LucidEncrypt.Core with KMS provider support and tracking for encryption migration progress. Expanded LucidRAG.App UI with document explorer, detail views, and a knowledge graph powered by D3.js.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant