preprocessor: fix unbounded MacroExpand/PrescanMacroArg mutual recursion (stack overflow)

[ashketchumwashere]

Summary

Fixes a stack-overflow vulnerability in the GLSL preprocessor where
MacroExpand and PrescanMacroArg could recurse into each other without
bound, exhausting the process stack.

Root Cause

macro->busy was set to 1 after the PrescanMacroArg loop
(Pp.cpp, previously after pushInput). But PrescanMacroArg itself calls
MacroExpand to expand tokens it finds in macro arguments. Because
busy == 0 during that pre-scan, a macro whose argument contains an
invocation of itself (directly or through a chain) causes:

MacroExpand → PrescanMacroArg → MacroExpand → PrescanMacroArg → ...

until the process stack is exhausted. Confirmed via OSS-Fuzz:

==ERROR: AddressSanitizer: stack-overflow
#7  MacroExpand  Pp.cpp:1295
#8  PrescanMacroArg  Pp.cpp:1109
#9  MacroExpand  Pp.cpp:1392
#10 PrescanMacroArg  Pp.cpp:1109
... [248 alternating frames]

Fix

Fix 1 (root cause) — PpContext.h + Pp.cpp:
Move macro->busy = 1 to before the PrescanMacroArg loop. This
matches standard C preprocessor semantics (C17 §6.10.3.4): a macro
currently being expanded must not be re-expanded, including during argument
pre-scanning. GCC and Clang both enforce this.

Fix 2 (defense-in-depth) — PpContext.h + Pp.cpp:
Add a macroExpandDepth counter (bounded at 200) to TPpContext. A RAII
DepthGuard increments/decrements it on each MacroExpand entry. This
independently prevents any future unbounded recursion path not caught by
the busy flag, and makes the limit explicit and auditable.

Reproduction

// triggers the bug before this fix
#define A(x) A(x)
void main() { A(1); }

[CLAassistant]

All committers have signed the CLA.

[arcady-lunarg]

Before anything else, you're going to need to a) get rid of extraneous whitespace diffs, and b) add a test that exercises your new check.

[arcady-lunarg]

Get rid of these extraneous whitespace changes

[ashketchumwashere]

Fixed. I've removed all extraneous whitespace changes and added Test/preprocessor.macro.recursion.vert — a 210-level nested F(F(...F(1)...)) call that exercises the macroExpandDepth guard. With the fix it produces 10 "macro expansion depth limit exceeded" errors cleanly; without it the same input causes a stack overflow (SIGSEGV with 4 MB stack / timeout).

[ashketchumwashere]

Hi @arcady-lunarg — just wanted to flag that I've addressed your feedback: all extraneous whitespace changes have been removed and I've added Test/preprocessor.macro.recursion.vert to exercise the depth guard (210-level nested F(F(...F(1)...)) — produces 10 clean "macro expansion depth limit exceeded" errors with the fix, stack overflow without it). Would appreciate a re-review when you get a chance. Thanks!

[dneto0]

This looks good to me. Neither GLSL nor C++98 specify a minimum number of macro expansion nesting levels. 200 seems fine.