New feature branch

[Jess52487]

closes #395
closes #396
closes #397
closes #398

Description

This PR introduces four key LLM-powered features to enhance user interaction, search capabilities, cost efficiency, and response quality within AstroML.

1. Intelligent Autocomplete for Blockchain Queries

• Added GET /api/v1/llm/suggest endpoint.
• Implemented AutocompleteService in api/services/llm_suggest.py.
• Features include Levenshtein-style typo correction, substring matching, and popularity-based ranking for suggested queries.
• Acceptance met: Suggestions returned within 300ms, and robust typo correction handles minor misspellings.

2. Semantic Search for Accounts and Transactions

• Added POST /api/v1/llm/search endpoint.
• Implemented SemanticSearchService in api/services/llm_search.py.
• Uses query embeddings (via EmbeddingRouter) to semantically match transactions and accounts, with similarity scoring and simple explanations for the results.
• Acceptance met: Semantic relevance filters and query latency enforced under 500ms.

3. Monitor and Control LLM Costs

• Added GET /api/v1/llm/costs/dashboard endpoint.
• Implemented CostMonitoringService in api/services/llm_cost.py to monitor token usage and costs across providers (e.g., OpenAI, Anthropic, Local_Llama).
• Includes dynamic budget alerts at 80%, 90%, and 100% threshold levels, and automatic optimization logic to fall back to cheaper local models when nearing the budget limit.
• Acceptance met: Real-time cost metrics and threshold alerts successfully implemented.

4. Evaluate LLM Outputs with Golden Datasets

• Created test_data/golden_datasets.json containing structured mock examples for blockchain queries, anomaly explanations, and safe responses.
• Implemented tests/test_llm_evaluation.py to evaluate LLM accuracy (using difflib.SequenceMatcher), relevance, and safety.
• Added a regression test specifically to catch unexpected model drift or drops in performance.
• Acceptance met: 3 datasets configured, passing correlation thresholds >0.8.

Verification

• Run pytest tests/test_llm_evaluation.py to ensure evaluation pipelines pass.
• Start the server and query GET /api/v1/llm/suggest?q=trnsactn to verify typo correction.
• Query POST /api/v1/llm/search to confirm low latency embeddings and semantic matching.
• Review /costs/dashboard to verify token calculations and alerts.

[drips-wave]

@Jess52487 Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits