📚 LLM Papers Update - 2026-03-13

[github-actions]

📚 Daily LLM Paper Curation Summary

Overview

• Total Papers Added: 7
• Average Significance Score: 91.4/100
• Categories Updated: 3
• Date Range: Last 1 day(s)

Selection Criteria

Papers are automatically selected based on:

• Innovation Score: Novel methods, breakthrough approaches
• Impact Score: Practical applications, real-world significance
• Technical Quality: Mathematical rigor, comprehensive analysis
• Sentiment Analysis: Positive reception indicators
• Minimum Threshold: 90.0/100 significance score

Papers Added

Evaluation (5 new papers)

• ThReadMed-QA: A Multi-Turn Medical Dialogue Benchmark from Real Patient Questions (Score: 92.0)
  • This paper addresses a crucial gap in medical QA benchmarks by focusing on multi-turn dialogues, mirroring real-world patient interactions. The use of real patient-physician conversations from Reddit is a strong methodological choice, providing authentic data. While the evaluation using LLMs as judges is common, the low performance of even GPT-5 highlights the challenge and the need for further research, and the calibrated rubric adds rigor. The clear presentation and focus on a relevant problem suggest a high likelihood of positive reception.
• Stop Listening to Me! How Multi-turn Conversations Can Degrade Diagnostic Reasoning (Score: 92.0)
  • This paper addresses a crucial and timely problem – the degradation of LLM performance in realistic, multi-turn healthcare conversations. The 'stick-or-switch' evaluation framework appears novel and well-suited to capture the nuances of conversational reasoning. The observed 'conversation tax' is a significant finding with implications for the safe and effective deployment of LLMs in clinical settings, and the evaluation of 17 LLMs provides a robust empirical basis.
• Measuring Intent Comprehension in LLMs (Score: 92.0)
  • This paper tackles a crucial problem in LLM evaluation – moving beyond surface-level matching to assess true intent comprehension. The proposed framework for evaluating robustness to semantically equivalent prompts is well-motivated and addresses a significant weakness in current LLM assessment methods. While the abstract doesn't detail the specifics of the framework, the problem statement is compelling and suggests a rigorous approach. The focus on high-stakes settings further elevates the importance of this work.
• LifeSim: Long-Horizon User Life Simulator for Personalized Assistant Evaluation (Score: 90.0)
  • This paper addresses a crucial bottleneck in LLM development – the lack of realistic evaluation benchmarks for personalized assistants. The use of a BDI model to simulate user cognition and the creation of a large, multi-scenario benchmark (LifeSim-Eval) are strong points. While the BDI model isn't entirely novel, its application within a simulated life context for assistant evaluation appears to be a significant advancement, and the scale of the benchmark is impressive. The focus on long-horizon interactions and implicit intentions is particularly valuable.
• Human-Centred LLM Privacy Audits: Findings and Frictions (Score: 90.0)
  • This paper tackles a crucial and timely problem – LLM privacy – with a user-centered approach. The development of LMP2 and the user studies demonstrating the ability to predict personal attributes are strong methodological contributions. The acknowledgement of a broader 'generative AI evaluation crisis' adds depth and suggests the work is aware of the larger context, though the abstract ends abruptly.

Alignment (1 new papers)

• LLLMs: A Data-Driven Survey of Evolving Research on Limitations of Large Language Models (Score: 92.0)
  • This paper addresses a highly relevant and rapidly evolving area – the limitations of LLMs. The data-driven, semi-automated approach to surveying the literature is rigorous and addresses the challenge of keeping pace with the field's growth. The combination of keyword filtering, LLM-based classification, expert validation, and topic clustering demonstrates a strong methodology, and the observed trends (reasoning as a primary limitation, shift towards security concerns) are insightful.

Training (1 new papers)

• RECAP: Reproducing Copyrighted Data from LLMs Training with an Agentic Pipeline (Score: 92.0)
  • This paper tackles a crucial problem in LLM research – understanding what data these models have memorized – in a novel way. The agentic pipeline with feedback and jailbreaking is a clever approach to elicit memorized content, and the EchoTrace benchmark provides a solid evaluation platform. While the core idea of iterative prompting isn't entirely new, the combination with an agentic loop and targeted correction hints demonstrates a significant advancement. The potential implications for copyright, privacy, and model auditing are substantial.

Categories

Evaluation (5), Alignment (1), Training (1)

---

This PR was automatically generated by the LLM Paper Curation workflow
Review the papers and merge if the selection looks appropriate