ai-study: ELF: Embedded Language Flows (Hu et al., 2026)

Layered study artifacts — interview prep, math deep dive, opinion capture template, LaTeX literature-review entry, proposed extensions with runnable prototypes, and a sandbox — for arxiv:2605.10938.

Compiled PDFs: pdfs/04-literature-review.pdf · pdfs/05-improvements.pdf · pdfs/interview-handout.pdf — auto-built by GitHub Actions on every push to .tex / .bib files.

📍 Start here

New to this paper? Read the tour — a guided walk through the math foundations you'll need, the paper's own concepts, and the proposed improvements (each validated by a formal proof or a measured code prototype).

Also available: tour.pdf (printable, CI-rendered) · tour.ipynb (interactive, runnable measurement cells).

Generated by the study-paper Claude Code skill.

---

The paper

Title: ELF: Embedded Language Flows Authors: Keya Hu, Linlu Qiu, Yiyang Lu, Hanhong Zhao, Tianhong Li, Yoon Kim, Jacob Andreas, Kaiming He (MIT) Venue: arXiv preprint, May 2026 Official code: https://github.com/lillian039/ELF

Headline claim. Continuous diffusion language models can be made effective with minimal adaptation to the discrete domain: run continuous-time flow matching in a pretrained embedding space, keep the trajectory continuous until $t = 1$, and reuse the same network's final timestep as the decoder via the weight-tied unembedding matrix. At $\sim 105$M params and $\sim 10\times$ fewer training tokens, ELF outperforms leading discrete DLMs (MDLM, SEDD) and prior continuous DLMs at fewer sampling steps.

What's in this repo

Path	Purpose
01-interview-prep.md	~500-word, opinionated 6-section talking-points doc (includes "My proposed extensions")
02-math-deep-dive.md	Mathematician-grade walk-through: definitions, derivations, load-bearing assumptions, gaps flagged
03-opinions.md	Opinion-capture template (filled in by hand, not by AI)
04-literature-review.tex	Research-ready LaTeX literature-review entry, standalone-compilable
05-improvements.tex	Forward-looking proposals — math, code, experimental, theoretical improvements
improvements/	Runnable Python prototype: scheduled CFG demo on a 2D Gaussian mixture
interview-handout.tex	One-page printable summary (with "My One Proposed Extension" section)
references.bib	BibTeX entries: the paper + 10 prior works it engages with
metadata.json	Slug, arxiv ID, authors, study date, completed stages
source.pdf	The paper (CC BY 4.0, redistributed under the same license)
sandbox/	CPU-runnable PyTorch experiment: toy flow matching with weight-tied unembedding decoding

Build the LaTeX artifacts (PDF)

sudo apt install texlive-latex-base texlive-fonts-recommended texlive-publishers
pdflatex 04-literature-review.tex && bibtex 04-literature-review && pdflatex 04-literature-review.tex && pdflatex 04-literature-review.tex
pdflatex 05-improvements.tex && bibtex 05-improvements && pdflatex 05-improvements.tex && pdflatex 05-improvements.tex
pdflatex interview-handout.tex

GitHub Actions does this automatically on every push that touches .tex or .bib and commits the PDFs to pdfs/.

Run the sandbox

cd sandbox && pip install -r requirements.txt && python experiment.py

Trains a tiny MLP velocity model on synthetic 32-dim embeddings of a 64-token vocabulary, then sweeps Euler step counts to demonstrate the weight-tied-unembedding decoding pathway.

Run the proposed-improvement prototype

cd improvements && pip install -r requirements.txt && python cfg-schedule.py

Trains a conditional flow-matching model on a 2D Gaussian mixture with CFG dropout, then compares constant-$\omega$ to U-shape and linear-ramp $\omega(t)$ schedules. Demonstrates that scheduled CFG can improve quality and diversity simultaneously.

License

• Paper (source.pdf): CC BY 4.0 — © Hu et al. 2026, redistributed under the same license.
• All other files in this repo: MIT.