Suggestion: WFGY as a debugging overlay for LLMs working on source code

[onestardao]

Hi, thanks for curating this awesome list on machine learning for source code.

I would like to propose WFGY as a complementary framework for people using LLMs on code, especially when they wrap them in tools, agents or RAG systems.

• Project: WFGY
• Repo: https://github.com/onestardao/WFGY
• License: MIT
• Stars: about 1.4k+

WFGY is a text-only “semantic firewall” that you load into an LLM. It helps you diagnose why a code-oriented assistant is failing: retrieval issues, chunking issues, long-chain drift, etc.

Key component:

ProblemMap (WFGY 2.0)
https://github.com/onestardao/WFGY/tree/main/ProblemMap

• 16 named failure modes for LLM / RAG pipelines.
• Each entry treats the system as a pipeline (ingestion → embedding → store → retrieval → reasoning), not just a black-box model.
• This is especially relevant for LLMs that browse codebases through vector stores or code search.

There is also a more formal:

• WFGY 1.0 PDF explaining the math and structure, and
• WFGY 3.0 Singularity Demo for long-horizon reasoning tests.

If you consider frameworks that help debug code-oriented LLM tools in scope, WFGY might be worth a short mention. I can open a PR if helpful.

[onestardao]

Hi, just a small follow-up to clarify the direction.

Over the past few weeks, the focus of WFGY has evolved more explicitly into the WFGY 2.0 “RAG 16 Problem Map” — a structured failure taxonomy for LLM / RAG pipelines.

In several ecosystems (RAGFlow, LlamaIndex, ToolUniverse, Rankify, Multimodal RAG Survey, and 20+ repos overall), it is now referenced specifically as a structured debugging vocabulary rather than as a standalone framework.

For code-oriented LLM systems — especially those using:

• vector stores for code search
• chunked repositories
• long-horizon retrieval + reasoning
• tool-wrapped agents

the 16-problem map is used to systematically classify failure modes such as retrieval mismatch, chunk boundary drift, embedding misalignment, or reasoning collapse.

So rather than positioning WFGY broadly, I wanted to narrow the suggestion:

If this list includes structured debugging or evaluation resources for LLMs working on source code, the WFGY 2.0 Problem Map might fit specifically in that context.

If helpful, I can submit a minimal PR framed purely as a failure-mode checklist for code-centric RAG systems.

Happy to adjust to your scope.