Paper Tracker Agent: LLM Academic Paper Tracker

Paper Tracker Agent is an open-source paper-tracker and academic paper tracker for automated literature review, research monitoring, and paper alerts. It pulls recent papers from arXiv, OpenAlex, Crossref, DBLP, and Semantic Scholar, asks an LLM to judge each paper against a research profile you define, and emails you a digest. Topic, keywords, LLM provider, email backend, and schedule are all configurable, so the same code works for any research area.

Related keywords: paper-tracker, academic-paper-tracker, academic paper tracker, research paper tracker, literature review agent, arxiv paper alert, LLM research agent, OpenAlex, Crossref, DBLP, Semantic Scholar, Gemini, OpenAI-compatible API.

一个可配置的学术论文追踪器：从多个开放学术 API 拉取最新论文，用大模型（LLM）按你定义的 研究画像（profile） 逐篇评估，并把结果汇总成邮件发给你。主题、关键词、大模型供应商、 邮件后端和定时任务全部可配置。

---

Features

• Multi-source fetching — arXiv, OpenAlex, Crossref, DBLP, Semantic Scholar (toggle each on/off, or multi/auto).
• Configurable topic & keywords — every research area lives in a YAML profile; ship your own without touching code.
• Pluggable LLM — Google Gemini, or any OpenAI-compatible endpoint including domestic (China) APIs such as DeepSeek / Qwen / Zhipu GLM / Moonshot / SiliconFlow / Ollama.
• Scheduling (定时启动) — run once, daily at a fixed time, or every N hours; or drive it from cron / Windows Task Scheduler.
• Email digest (发送到邮箱) — via SMTP (any provider) or the Resend API; prioritized list plus a "watch list"; labels in Chinese or English (EMAIL_LANGUAGE).
• De-duplication & memory — a local SQLite DB means each paper is processed and emailed only once.
• No secrets in the repo — all credentials come from a git-ignored .env.

---

Common Use Cases

• Build a personal academic paper tracker for any research topic.
• Monitor arXiv, OpenAlex, Crossref, DBLP, and Semantic Scholar with one workflow.
• Run an LLM-based literature review agent that scores and summarizes new papers.
• Send daily or scheduled paper alerts by SMTP or Resend.
• Maintain watch lists for papers that need manual review.

---

Project layout

paper-tracker/
├── run.py                  # CLI entry point
├── profiles/
│   ├── codec.yaml          # example profile: video/image coding
│   └── llm_agents.yaml     # example profile: LLM agents
├── paper_tracker/
│   ├── config.py           # env-driven configuration
│   ├── profile.py          # YAML profile loader
│   ├── models.py           # Paper / PaperAnalysis schemas
│   ├── sources.py          # academic API fetchers
│   ├── llm.py              # Gemini + OpenAI-compatible backends
│   ├── report.py           # HTML email + markdown export
│   ├── email_sender.py     # SMTP + Resend
│   ├── storage.py          # SQLite de-duplication
│   ├── scheduler.py        # built-in scheduler
│   └── runner.py           # one end-to-end pass
├── .env.example            # copy to .env and fill in
└── requirements.txt

---

Quick start

# 1. Install dependencies (Python 3.9+)
pip install -r requirements.txt

# 2. Configure
cp .env.example .env
#   edit .env: choose LLM_BACKEND, set the API key, and email settings

# 3. Run once
python run.py --once

The first run creates data/papers.db. Subsequent runs skip already-seen papers.

---

1. Choosing what to track (profiles)

A profile is a YAML file describing one research area. Two examples are included: profiles/codec.yaml and profiles/llm_agents.yaml.

Select one via .env:

PROFILE=profiles/codec.yaml

…or per run:

python run.py --profile profiles/llm_agents.yaml

To track something new, copy a profile and edit these fields:

Field	Purpose
name	Title shown in the email subject/header.
language	zh or en — language of the LLM's free-text output.
topic_description	One-line description of the area.
expert_persona	The expert role the LLM should adopt.
analysis_guidance	Extra reasoning rules for relevance.
custom_focus	Your specific research interest, evaluated per paper.
scoring_guidance	The 0–100 scoring rubric.
arxiv_categories	arXiv category filter (when arXiv is enabled).
keywords	Cheap local pre-filter before calling the LLM.
search_queries	Queries sent to OpenAlex / Crossref / DBLP / S2.

---

2. Choosing the LLM (incl. domestic APIs)

Google Gemini

LLM_BACKEND=gemini
GEMINI_API_KEY=your-gemini-api-key
GEMINI_MODEL=gemini-2.5-flash

OpenAI-compatible / domestic (China) APIs

Any provider exposing a /chat/completions endpoint works. Set:

LLM_BACKEND=openai
OPENAI_API_KEY=your-api-key
OPENAI_BASE_URL=...
OPENAI_MODEL=...

Provider	OPENAI_BASE_URL	Example OPENAI_MODEL
DeepSeek	https://api.deepseek.com/v1	deepseek-chat
Qwen (DashScope)	https://dashscope.aliyuncs.com/compatible-mode/v1	qwen-plus
Zhipu GLM	https://open.bigmodel.cn/api/paas/v4	glm-4-flash
Moonshot (Kimi)	https://api.moonshot.cn/v1	moonshot-v1-8k
SiliconFlow	https://api.siliconflow.cn/v1	(any hosted model)
Ollama (local)	http://localhost:11434/v1	qwen2.5

google-genai is only required for the Gemini backend; you can skip installing it if you only use the OpenAI-compatible backend.

---

3. Email delivery

SMTP (any provider)

EMAIL_BACKEND=smtp
SMTP_HOST=smtp.gmail.com
SMTP_PORT=587          # 587 = STARTTLS, 465 = SSL
SMTP_USER=your-email@example.com
SMTP_PASSWORD=your-app-password
MAIL_FROM=your-email@example.com
MAIL_TO=recipient@example.com   # comma-separated for multiple recipients

For Gmail/Outlook use an app password, not your login password.

Resend API

EMAIL_BACKEND=resend
RESEND_API_KEY=your-resend-key
MAIL_FROM=Paper Tracker <onboarding@resend.dev>
MAIL_TO=recipient@example.com

Email language (中文 / English)

Set the language of the entire digest — both the template labels (Source / 来源, Performance / 性能情况, …) and the LLM-generated text (summaries, reasons, …):

EMAIL_LANGUAGE=en   # "zh", "en", or empty to follow the profile's `language`

When EMAIL_LANGUAGE is set it overrides the profile's language for that run. When left empty, the profile's language field is used.

Note: papers analyzed on a previous run are cached in the database and are not re-analyzed, so switching the language only affects newly fetched papers. (Already-seen papers are skipped entirely and don't appear in the email.)

---

4. Scheduling (定时启动)

Built-in scheduler

Daily at a fixed time:

RUN_MODE=schedule
SCHEDULE_TIME=08:00
SCHEDULE_INTERVAL_HOURS=0

Or every N hours:

RUN_MODE=schedule
SCHEDULE_INTERVAL_HOURS=12

Then start the long-running process:

python run.py --schedule

OS scheduler (recommended for servers)

Keep RUN_MODE=once and let the OS trigger it:

Linux/macOS cron — daily at 08:00:

0 8 * * *  cd /path/to/paper-tracker && /path/to/python run.py --once >> data/cron.log 2>&1

Windows Task Scheduler (PowerShell):

schtasks /Create /SC DAILY /ST 08:00 /TN PaperTracker `
  /TR "cmd /c cd /d D:\path\to\paper-tracker && python run.py --once"

---

CLI reference

python run.py [--profile PATH] [--source NAME] [--once | --schedule]

Flag	Effect
--profile PATH	Use a specific profile (overrides PROFILE).
--source NAME	Override PAPER_SOURCE (arxiv, openalex, multi, …).
--once	Force a single run.
--schedule	Run continuously on the configured schedule.

---

How it works

1. Fetch recent papers from the enabled sources (de-duplicated by title/ID).
2. Pre-filter locally with the profile's keywords (skipped if ANALYZE_ALL_CANDIDATES=true).
3. Analyze each remaining paper with the LLM → a structured JSON judgement (relevance score, priority, summary, performance, datasets, baselines, reason…).
4. Select papers scoring ≥ PUSH_SCORE_THRESHOLD; collect a watch list in the WATCHLIST_MIN_SCORE–WATCHLIST_MAX_SCORE band.
5. Email a prioritized HTML digest and export the watch list to data/.
6. Remember everything in SQLite so nothing is processed or sent twice.

---

Notes & etiquette

• Set CONTACT_EMAIL so requests to OpenAlex / Crossref / arXiv carry a polite mailto (these APIs ask for it and give better rate limits in return).
• Be reasonable with MAX_RESULTS — each candidate costs one LLM call.
• data/, .env and generated reports are git-ignored by default.

License

MIT