🧠 AI Judge — Internet Judge

An AI agent that evaluates the credibility of information on the internet.

The system is built using 'LangChain Agents' with a sequential multi-agent workflow. Each agent specializes in a specific aspect of fact-checking, from breaking down questions to providing a credibility score.

🎯 Idea

The user asks a question, and the AI:
1. Breaks it down into sub-questions.
2. Searches for reliable sources on the internet.
3. Evaluates credibility on a scale of 0-100%.
4. Assigns a star rating (0-5).
5. Provides 3 specific reasons for the evaluation.

🧠 How the Architecture Works

The system uses a **sequential multi-agent workflow** where each agent builds on the output of the previous one:
[User Question] → [Research Agent] → [Source Checker Agent] → [Scoring Agent] → [Report]

Key features of this implementation:
- Specialized Agents: Each agent has a single, well-defined responsibility.
- Sequential Flow: Information flows from one agent to the next, with each adding value.
- Tool Integration: The Source Checker uses real-time web search tools (Web Search, URL Reader, Current Date).
- Memory Base: The Scoring Agent can access previous evaluations via Memory Base.
- Test of Absurdity: The system recognizes obvious myths and assigns 0% credibility.

🛠️ Technologies

- LangChain 1.2.18 – framework for building AI agents
- OpenAI API – GPT models
- Tavily – internet search
- LangFlow – visual environment
- ChromaDB – vector memory

📦 Installation

1. Clone
git clone https://github.com/AIResearchForge/AI-JUDGE.git
cd AI-JUDGE

2. Virtual environment
python -m venv venv
source venv/bin/activate  # Linux/Mac
venv\Scripts\activate   # Windows

3. Install dependencies (in this exact order to avoid conflicts)
pip install langchain==1.2.18
pip install langchain-community==0.3.0
pip install langchain-openai==0.2.0
pip install tavily-python python-dotenv pydantic requests beautifulsoup4 openai chromadb

4. Configuration
Edit the .env file and enter your API keys

5. Run
python -m src.main

🚀 EXAMPLE USAGE:

🧠 AI JUDGE — Internet Judge
📦 Model: gpt-4o-mini
Type 'exit' to quit.

❓ Ask: Was Napoleon short?

✅ Is the claim true? NO  
Truth Score: 0%  
Credibility: ☆☆☆☆☆  
Reasons:  
✓ Napoleon Bonaparte's height was approximately 5 feet 6 inches, which is average for his time.  
✓ The average height of men in early 19th-century France was around 5 feet 5 inches, making him taller than average.  
✓ The myth of Napoleon being short was largely a result of British propaganda and caricatures.

============================================================
📋 REPORT:
============================================================
Is the claim true? NO  
Truth Score: 0%  
Credibility: ☆☆☆☆☆  
Reasons:  
✓ Napoleon Bonaparte's height was approximately 5 feet 6 inches, which is average for his time.  
✓ The average height of men in early 19th-century France was around 5 feet 5 inches, making him taller than average.  
✓ The myth of Napoleon being short was largely a result of British propaganda and caricatures.
============================================================

PROJECT STRUCTURE:

AI-JUDGE/
├── README.md
├── .env
├── .gitignore
├── config/
│   └── openai_config.py          # OpenAI API configuration
├── src/
│   ├── __init__.py               # Package initialization
│   ├── agents.py                 # LangChain agent definitions (3 agents)
│   ├── memory.py                 # Simple memory for conversation history
│   ├── prompts.py                # System prompts for all agents
│   ├── tools.py                  # Tools (Web Search, URL, Date)
│   ├── main.py                   # Entry point (CLI)
│   └── utils.py                  # Utility functions
├── examples/
│   ├── __init__.py
│   └── example_questions.txt     # Sample questions for testing
└── langflow/
    └── ai_judge_flow.json        # LangFlow export for visual prototyping

🎨 LangFlow Version (GUI):

The project includes a fully functional LangFlow export file: langflow/ai_judge_flow.json
This allows you to build and test the same workflow visually without writing any code.

To use the LangFlow version:

Import the flow:
- Run LangFlow in your browser: langflow run - type this in the 'cmd' command line
- Open your browser at http://localhost:7860
- Click "Import Flow"
- Select langflow/business_analyst_flow.json

The visual workflow will load with all 3 agents and their connections

Test in Playground:
- Enter a question
- Watch the agents execute step by step in the visual interface

What you can do in LangFlow:
- Modify agent prompts in real-time
- Adjust model parameters (temperature, max tokens)
- Add or remove tools visually
- Debug the flow step by step
- Export the modified flow back to JSON

Important difference:

The CLI version uses LangChain Agents with AgentExecutor and is ideal for production deployment.
The LangFlow (GUI) version is best for prototyping, testing, and demonstrating the workflow to non-technical stakeholders.