CURe Dataset

This repository contains the implementation of multiple models and approaches for biomedical claim verification using the CURe dataset.

Dataset

The CURe dataset (dataset/CURe_dataset_sample.json) is a comprehensive dataset for biomedical claim verification. It contains:

• Claims extracted from medical forum discussions
• Supporting evidence from PubMed abstracts
• Multi-label annotations for claim veracity (Supports/Refutes/NEI)
• Natural language inference (NLI) labels for evidence relevance

Models and Implementations

The codebase contains multiple approaches to biomedical claim verification:

Code file description:

• cross-encoder-pubmed-bert.py: Cross-encoder using PubMedBERT for biomedical domain adaptation
• cross-encoder-roberta.py: Cross-encoder using RoBERTa with medical fine-tuning
• graph-fusion.py: Graph neural network for evidence aggregation and reasoning
• multi_hop_debarta_reasoner.py: Multi-hop reasoning using DeBERTa with graph structure
• set-transformer-debarta.py: Set Transformer architecture for permutation-invariant evidence processing
• Zero_shot_pipeline_LLAMA3.18b_RAG.py: Zero-shot approach using LLAMA
• zero-shot-high-parameter-models.py: Zero-shot testing with various large models
• RAG_best_sentence_selection.py: Retrieval-augmented generation with evidence selection
• Multi-Intent_SHAP-Integrated_Cross-Attentive_Reasoning.py: Multi-intent reasoning with SHAP attribution
• multi_agentic_pipeline.py: Multi-agent pipeline for complex reasoning
• reasoning_persona_generation.py: Persona-based reasoning approach

Reproducibility

To ensure full reproducibility of our experiments, we have taken the following measures:

1. Environment Setup

   • Fixed random seeds across all frameworks (PyTorch, NumPy, Python)
   • Deterministic CUDA operations enabled
   • GPU memory management optimizations

2. Dependencies

   • Pinned package versions in requirements.txt tested for python 3.10.x
   • Virtual environment setup scripts for isolation
   • Docker container definition for complete environment replication

Environment Setup Instructions

Linux / macOS

# Create a virtual environment
python3 -m venv myenv

# Activate the virtual environment
source myenv/bin/activate

# Install dependencies with pinned versions
pip install -r requirements.txt

Windows

# Create a virtual environment
python -m venv myenv

# Activate the virtual environment
myenv\Scripts\activate

# Install dependencies with pinned versions
pip install -r requirements.txt