Tongue Analysis API

Overview

The Tongue Analysis API provides automated tongue health analysis using computer vision and machine learning techniques. This FastAPI-based service analyzes tongue images and extracts quantitative metrics related to tongue characteristics.

The system combines multiple AI models and image-processing techniques to perform:

• Tongue segmentation
• White coating analysis
• Papillae detection
• Crack detection
• Tongue jaggedness measurement
• Health score estimation
• AI-generated health summaries

The project leverages the Segment Anything Model (SAM), Roboflow-based tongue detection, and Gemini API integration for summary generation.

---

Features

Tongue Segmentation

Accurately isolates the tongue region from an input image using the Segment Anything Model (SAM).

White Coating Analysis

Detects and quantifies the percentage of white coating present on the tongue surface.

Papillae Detection

Identifies and analyzes tongue papillae characteristics including count, size, and redness.

Crack Detection

Detects tongue cracks and generates severity-related metrics.

Jaggedness Analysis

Measures irregularities along the tongue boundary.

Health Scoring

Generates quantitative health-related scores from extracted tongue features.

AI Summary Generation

Uses Gemini API to generate human-readable summaries of analysis results.

Chat Interface

Provides an interactive interface for health-related discussions and interpretation of results.

---

Technology Stack

• Python
• FastAPI
• OpenCV
• PyTorch
• Segment Anything Model (SAM)
• Roboflow
• Gemini API
• NumPy
• Pandas
• Matplotlib

---

Installation

Prerequisites

• Python 3.8 or higher
• CUDA-capable GPU (recommended)
• SAM model checkpoint (sam_vit_h.pth)

Setup

1. Clone the repository

git clone https://github.com/ansh-codes11/HealthLingue.git
cd HealthLingue

2. Install dependencies

pip install -r requirements.txt

3. Download the SAM model checkpoint

Download:

https://dl.fbaipublicfiles.com/segment_anything/sam_vit_h_4b8939.pth

Rename the file to:

sam_vit_h.pth

Place it in the project root directory.

4. Create a .env file

Create a file named .env in the project root:

GEMINI_API_KEY=your_gemini_api_key
ROBOFLOW_API_KEY=your_roboflow_api_key

5. Create output directory

mkdir output

---

Running the API Server

Start the server using:

uvicorn FastApiServer:app --host 0.0.0.0 --port 8000

The API will be available at:

http://localhost:8000

---

API Endpoints

POST /analyze_tongue

Analyzes an uploaded tongue image and returns detailed metrics.

Request

• file: Tongue image

Example Response

{
  "Jaggedness": 25.0,
  "Summary": "AI-generated summary of tongue health",
  "Cracks": {
    "morph": "path/to/visualization",
    "score": 3.5
  },
  "NutritionScore": 75.5,
  "MantleScore": 82.5,
  "redness": 6.8,
  "segmented_image_path": "path/to/segmented_image",
  "white_coating": {
    "white_coating_percentage": 15.3,
    "visualization_path": "path/to/visualization"
  },
  "papillae_analysis": {
    "total_papillae": 127,
    "avg_size": 32.5,
    "avg_redness": 0.68
  }
}

---

POST /chat

Allows users to ask health-related questions.

Example Request

{
  "message": "What does high tongue coating indicate?"
}

Example Response

{
  "reply": "AI-generated response about tongue coating"
}

---

GET /health

Returns API health status and component status information.

---

GET /image/{image_path}

Retrieves generated visualization images.

---

GET /csv/{csv_path}

Retrieves generated CSV analysis data.

---

Project Structure

HealthLingue/
│
├── FastApiServer.py
├── segment.py
├── llmcall.py
├── chat.py
├── Tongue_crack_detection_model.py
├── tongue_papillae_analyzer.py
├── tongue_jagged.py
├── jaggedScore.py
├── evaluate.py
├── Image_dataset/
├── requirements.txt
├── .env.example
└── README.md

---

Dataset

The repository contains sample tongue images inside:

Image_dataset/

These images are provided for demonstration and testing purposes.

---

Dependencies

Major dependencies include:

• FastAPI
• Uvicorn
• OpenCV
• PyTorch
• NumPy
• Pandas
• Pillow
• Matplotlib
• Requests
• Segment Anything
• Inference SDK (Roboflow)
• Python Dotenv

Install all dependencies using:

pip install -r requirements.txt

---

Notes

• Internet access is required for Gemini API functionality.
• Internet access is required for Roboflow inference services.
• GPU acceleration is strongly recommended for SAM-based segmentation.
• Processing speed depends on image size and hardware configuration.

---

Disclaimer

This project is intended for educational and research purposes only.

The results generated by this system are not medical diagnoses and should not be considered a substitute for professional medical advice, diagnosis, or treatment.

Always consult a qualified healthcare professional for medical concerns.