Lumina (formerly LesionRec) 🔬✨

AI-Powered Skin Analysis & Personalized Product Recommendation System

Lumina is a modern, full-stack application that leverages advanced AI to analyze skin conditions and generate personalized skincare routines. It combines a privacy-focused image processing pipeline with a sophisticated recommendation engine to offer users actionable insights and product bundles that fit their budget.

---

🚀 Project Status: MVP Complete

This project has successfully reached Minimum Viable Product (MVP) status. It features a fully functional frontend, a robust backend API, and integrated AI services for real-time analysis.

---

🏗️ Architecture & Tech Stack

The application is built using a modern decoupled architecture:

Frontend (Client-Side)

• Framework: React (Vite) with TypeScript
• Styling: Tailwind CSS for responsive, modern UI
• State Management: React Hooks + LocalStorage for persistence
• Key Components:
  • ImageUpload: Handles camera capture and file uploads.
  • RecommendedProducts: Displays analysis results, product bundles, and full catalog with sorting/pagination.
  • ProductRoutine: Visualizes the step-by-step skincare routine.
  • Chatbot: Context-aware AI assistant for skin health Q&A.
  • ErrorBoundary: Robust error handling to prevent app crashes.

Backend (Server-Side)

• Framework: FastAPI (Python)
• AI Services:
  • Google Gemini 2.0 Flash: Primary engine for dermatological analysis (condition detection, severity assessment).
  • Google Vision API: Used for object localization (finding blemishes).
• Data Processing: Pandas for product dataset manipulation.
• Storage:
  • S3 (AWS): Secure storage for uploaded images (presigned URLs).
  • Local Data: Curated CSV datasets for skincare products.

---

✨ Key Features

1. Privacy-First Analysis:
   • Client-Side Background Removal: Uses WebAssembly (WASM) to remove the background from images before upload. This ensures no personal room details or bystanders are ever sent to the server.
   • Metadata Scrubbing: Automatically strips EXIF data (GPS, device info) to protect user privacy.
2. Multi-Modal Input: Supports both file uploads and live camera capture.
3. AI-Driven Insights: Detects conditions like Acne, Rosacea, Eczema, etc., and determines severity (Mild, Moderate, Severe).
4. Smart Budgeting & Bundling:
   • Dynamic Bundling: Users can set a specific budget (e.g., $50), and the system recalculates the optimal "Bundle" (Cleanser + Treatment + Moisturizer) to fit the sum within that limit.
   • Infinite Budget Mode: Defaults to showing the absolute best products if no budget is set.
5. Advanced Product Discovery:
   • Personalized Bundle: A cohesive routine where the total cost fits your budget.
   • Full Catalog View: Browse all matching products with pagination.
   • Smart Sorting: Sort products by Price (Low/High), Rating, or Popularity (Reviews).
6. AI Chatbot Assistant: Integrated chat interface to ask follow-up questions about the diagnosis or skincare advice.
7. Persistence: Analysis results are saved locally, allowing page refreshes without re-uploading images.

---

📂 Project Structure

Root Directory

• backend/: Python FastAPI server and logic.
• frontend/: React application.
• archive/: Legacy code (Streamlit, YOLO models) and documentation.
• docs/: Current project documentation.
• start-dev.sh: Script to launch both frontend and backend.

Backend Breakdown (backend/src/)

• main.py: The entry point for the FastAPI server. Defines endpoints /upload, /recommend, and /api/chat.
• services/:
  • analysis.py: Handles interaction with Google Gemini API.
  • privacy.py: Utilities for metadata scrubbing.
  • product_recommender.py: The core logic engine. Contains the "Knapsack-style" algorithm for bundling and filtering logic.
  • chatbot.py: Manages the conversational AI logic.
  • product_data_cleaner.py: Utilities for cleaning and loading CSV data.
• data/: Contains the CSV files for different skin conditions (e.g., acne_products.csv, rosacea_products.csv).

Frontend Breakdown (frontend/src/)

• App.tsx: Main application controller. Handles routing between Dashboard, Upload, Results, and Chat views.
• components/:
  • ImageUpload.tsx: Manages file selection, camera streaming, and API upload calls.
  • RecommendedProducts.tsx: The results dashboard. Manages the budget state, sorting, pagination, and displays the bundle/list.
  • ProductRoutine.tsx: Renders the "Bundle" as a visual step-by-step card grid.
  • Chatbot.tsx: The chat interface component.
  • ErrorBoundary.tsx: Catches runtime errors to display a friendly fallback UI.

---

🔄 Data Flow Guide

1. User Action: User uploads an image or captures a photo via ImageUpload.tsx.
2. Frontend: Sends POST /upload request with the image file to the Backend.
3. Backend (Privacy): privacy.py strips any remaining metadata.
4. Backend (Storage): Uploads the processed image to AWS S3.
5. Backend (AI Analysis): analysis.py sends the scrubbed image to Gemini 2.0 Flash.
   • Prompt: "Analyze this skin image for conditions..."
   • Response: JSON containing condition (e.g., "acne"), severity, and characterization.
6. Backend (Recommendation): product_recommender.py takes the analysis:
   • Loads the relevant product CSV (e.g., acne_products.csv).
   • Bundle Logic: Selects a Cleanser, Treatment, and Moisturizer such that Sum(Prices) <= Budget.
   • List Logic: Selects top-rated items where Item_Price <= Budget.
7. Response: Backend returns the Analysis + Bundle + Recommendations to Frontend.
8. Frontend: App.tsx saves data to localStorage and switches to RecommendedProducts.tsx view.
9. User Interaction: User enters a new budget (e.g., ).
10. Update: Frontend calls POST /recommend with the existing analysis text and new budget. Backend recalculates and returns the new bundle.

---

🧩 Planned Feature: Interactive Weekly Routines & Community Reviews

Overview

This feature will allow users to turn their recommended product routines into shareable, customizable weekly templates. Users can:

• Create a weekly routine based on their AI-recommended bundle.
• Add a personal description/breakdown for each routine.
• Upload progress photos (e.g., 1 week, 1 month after starting routine).
• Share routines for others to review, rate, and comment.
• Copy routines from other users and substitute products as needed.

1. Backend Architecture Changes

Current State: Routines are generated on-the-fly and stored in client-side localStorage. Required Change: Introduce a persistent database (PostgreSQL recommended) to store User Generated Content (UGC).

Database Schema (Draft)

• Routines Table:

  • id: UUID (Primary Key)
  • user_id: String (Owner)
  • title: String (e.g., "My Acne Fighting Journey")
  • description: Text (User's breakdown/explanation)
  • products: JSONB (List of products with id, name, category, image_url)
  • schedule: JSONB (e.g., {"Monday": ["cleanser", "treatment"], ...})
  • condition_tags: Array (e.g., ["Acne", "Oily Skin"])
  • is_public: Boolean
  • created_at: Timestamp

• RoutineReviews Table:

  • id: UUID
  • routine_id: UUID (Foreign Key)
  • reviewer_id: String
  • rating: Integer (1-5)
  • comment: Text
  • created_at: Timestamp

• ProgressPhotos Table:

  • id: UUID
  • routine_id: UUID
  • s3_key: String (Path to image in S3)
  • stage: String (e.g., "Week 1", "Month 1")
  • caption: Text

2. API Endpoints (New)

• Routine Management:

  • POST /routines: Save a current recommendation as a new Routine.
  • GET /routines: Search/List public routines (filter by condition, rating).
  • GET /routines/{id}: Get full details of a specific routine.
  • PUT /routines/{id}: Update description, schedule, or products.
  • POST /routines/{id}/fork: Copy another user's routine to your library (allows substitution).

• Community Interaction:

  • POST /routines/{id}/reviews: Submit a rating and comment.
  • POST /routines/{id}/photos: Upload a progress transition photo.

3. Frontend Implementation Plan

• "Save as Routine" Action:

  • Add button to RecommendedProducts.tsx and Dashboard.tsx.
  • Opens a "Create Routine" wizard to add a Title and Description.

• Routine Editor:

  • Interface to drag-and-drop products into a weekly schedule.
  • "Substitute Product" feature: Click a product -> Search Catalog -> Replace.

• Community Hub:

  • New main navigation tab: "Community".
  • Feed of top-rated routines and success stories (Progress Photos).

• Routine Detail View:

  • Shows the "Before/After" photos.
  • Displays the product list with "Buy Now" links.
  • Comments section for user reviews.

---

🛠️ Getting Started

Prerequisites

• Node.js (v16+)
• Python (v3.9+)
• Google Cloud Credentials (for Vision & Gemini)
• AWS Credentials (for S3)

Installation

1. Clone the repository:

   git clone https://github.com/adonisja/LesionRec.git
   cd LesionRec

2. Environment Setup: Create a .env file in the root (or backend/) with the following:

   GOOGLE_APPLICATION_CREDENTIALS="path/to/your/google-creds.json"
   GEMINI_API_KEY="your_gemini_key"
   AWS_ACCESS_KEY_ID="your_aws_key"
   AWS_SECRET_ACCESS_KEY="your_aws_secret"
   AWS_REGION="us-east-1"
   S3_BUCKET_NAME="your-bucket-name"

3. Run the Application: We have provided a convenience script to start both servers:

   ./start-dev.sh

   This will start the Backend on http://localhost:8000 and the Frontend on http://localhost:5173.

---

📝 License

Personal & Educational Use License

Copyright (c) 2025 Akkeem

This project is designed to be a learning resource and is available for personal and educational use only.

✅ You are free to:

• Download & Run: Install and run the application locally on your machine.
• Study & Learn: Review the source code to understand how the AI, Backend, and Frontend components work together.
• Modify: Experiment with the code for your own personal learning and hobby projects.

❌ You may NOT:

• Commercial Use: Use this source code, in whole or in part, for any commercial purpose, business, or revenue-generating activity.
• Redistribute for Profit: Sell, license, or monetize this code or any derivative works based on it.

If you wish to use this software for commercial purposes, please contact the author for permission.

---

⚠️ Medical Disclaimer

This application is for educational and informational purposes only.

• Not Medical Advice: The analysis, insights, and product recommendations provided by Lumina are generated by Artificial Intelligence and are not a substitute for professional medical advice, diagnosis, or treatment.
• Consult a Professional: Always seek the advice of a physician or other qualified health provider with any questions you may have regarding a medical condition. Never disregard professional medical advice or delay in seeking it because of something you have read on this application.
• No Doctor-Patient Relationship: Use of this application does not create a doctor-patient relationship.

General Disclaimer

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.