Labeling queue library for managing human labeling workflows

A self-hosted multi-rater data labeling platform for structured annotation workflows.

⚖️ Dual-Judge: 让AI测试结果真正有说服力 | 双LLM交叉验证消除单模型偏见 | 独立于具体Agent的通用评估框架 | Making AI Evaluation Trustworthy

An open multi-rater benchmark for characterizing architectural fingerprints in identity-scaffolded LLMs.

Multi-axis taxonomy of Australian coronial recommendations 1998-2026 — code, codebook, and pre-registered analysis outputs. Pre-registered at OSF DOI 10.17605/OSF.IO/NEX85 under CC-BY 4.0.

Open evaluation harness for mental health LLM responses. 5 clinically-grounded rubrics, LLM-as-judge with bias controls, crisis-detection routing to 988 protocols.

Statistical validation of labeling consistency across three independent raters for a handwritten digit classification dataset.

Python package implementing measure from my working paper "Kappa-IoU: Inter-Rater Reliability for Spatial Annotation"

Statistical analysis of inter-rater reliability and quality patterns in LLM evaluation systems using R

Browser-based inter-rater reliability calculator for systematic literature reviews. Computes Krippendorff's Alpha using a pooled coincidence matrix across all (Paper, RQ, Field) units. No installation required — single HTML file, fully client-side. Built for the GenAI Evidence Hub at Learning Data Insights.

The R scripts for my MSc dissertation "Developmental Trends in Children’s Understanding of COVID-19: A Draw, Write & Tell Study"

TRIP(U): Emotional travel recommendation system using NLP. Features a custom Gold Standard corpus of 500 labeled texts and supervised classification (SVM/Naive Bayes) for emotionally adaptive itineraries. Master's Thesis (TFM) - Distinction.

AI-assisted distractor classification and Cohen's Kappa reliability analysis tool for cognitive science and education research.