feat: unify name demangling with single Lean implementation#12539
feat: unify name demangling with single Lean implementation#12539
Conversation
07dc564 to
d5e99dc
Compare
|
Mathlib CI status (docs):
|
|
Reference manual CI status:
|
hargoniX
left a comment
There was a problem hiding this comment.
I think the approach in NameDemandling.lean looks generally sensible. It would be great if we could use the new String API to as opposed to the legacy one.
For the linking details in object.cpp I would like to defer to @Kha
5aca15a to
8e51b9b
Compare
@Rob23oba thanks for your very helpful comments. Can you point to where you would want to call a version of this with disabled post-processing? I'd like to understand the use case so I can know exactly what should be disabled. |
|
Alright, here's a script that should demonstrate the ambiguity involved: open Lean Name Demangle
run_meta
let ext := Compiler.LCNF.impureSigExt
let env ← getEnv
let mut map : Std.HashMap String Nat := {}
for idx in *...env.header.modules.size do
for decl in ext.getModuleEntries env idx do
map := map.alter (postprocessNameParts (nameToNameParts decl.name)) fun
| none => some 1
| some a => some (a + 1)
let arr := map.toArray.qsort fun (a, b) (c, d) => b > d ∨ (b = d ∧ a < c)
let arr := arr.take 100
Lean.logInfo m!"Result: {arr}"The top results are So if |
Thanks for the explanation here @Rob23oba. I think with |
This PR replaces three independent name demangling implementations (Lean, C++, Python) with a single source of truth in Lean. The new `Lean.Compiler.NameDemangling` module handles the full pipeline: prefix parsing, postprocessing (suffix flags, private names, hygienic stripping, specialization contexts), backtrace line parsing, and C exports. The C++ runtime backtrace handler now calls `@[export]`ed Lean functions instead of its own reimplementation. The Python profiler demangler is replaced with a thin subprocess wrapper around a Lean CLI tool. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…n link dependency The `@[export lean_demangle_bt_line_cstr]` function is defined in `Lean.Compiler.NameDemangling` (in `libLean`) but called from `libleanrt` (`print_backtrace`). On Linux these are in separate linker groups, so downstream executables fail to link with undefined reference. Use a weak symbol with a no-op stub so `leanrt` doesn't require `libLean` at link time. When the Lean demangler is linked in (as it always is for Lean executables), the real implementation overrides the stub. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This PR fixes issues found during code review: - add re-entrancy guard in C runtime panic handler to prevent recursive panic when demangling itself panics - expand test coverage from 44 to 118 test cases - fix bug in findLpSplit where _private was absorbed into package name - replace hardcoded byte offsets with dropPrefix? - remove unnecessary partial annotations using bounded for loops - move public import from Lean.lean to Lean/Compiler.lean - remove AI-slop stylistic artifacts (section dividers, verbose comments) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This PR addresses review feedback from hargoniX and Kha: - add missing lean_dec(s) in the weak symbol stub (object.cpp) - add Char.isHexDigit to Init and use it instead of custom isHexChar - reuse Lean.NamePart instead of defining a local Component type - remove _lambda_N, _jp_N, _closed_N numbered suffix handling (compiler only produces _lam_N and _elam_N) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…mangling This PR removes the custom raw-position string helpers (rawGet, rawNext, rawAtEnd, rawExtract, rawEnd) and rewrites all string traversal to use the modern Lean 4 String API: s.positions iterator, s.Pos with proofs, s.find? with char/predicate/substring patterns, s.extract, and a skipWhile helper with termination proof. Net reduction of 52 lines. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The g_in_demangle guard was preventing recursive demangling if the demangler itself panicked. The demangler is pure string manipulation that should not panic, and if it does we'd rather discover the bug than silently fall back to raw output. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This PR addresses three issues raised in review: 1. Use `Name.toString` for correct escaping of demangled names (#9): replace manual `String.intercalate "."` formatting with `Name.toString`, which handles `«»` escaping for names containing special characters. Add `namePartsToName` helper and guard `formatNameParts` against empty input. 2. Use `Name.demangle?` for robust `lp_` package splitting (#8): replace `findLpSplit`/`hasUpperStart`/`unmanglePkg` with a simpler `demangleWithPkg` that tries each underscore as a split point and validates both package and body via `Name.demangle?` round-trip. 3. Fix suffix flags lost after hygienic sections: move suffix flag collection (`_boxed`, `_lam`, etc.) before `_@` hygienic suffix stripping, so flags appearing after hygienic sections are recognized. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
Needs a rebase... |
d684699 to
25e8e04
Compare
This PR replaces three independent name demangling implementations (Lean, C++, Python) with a single source of truth in
Lean.Compiler.NameDemangling. The new module handles the full pipeline: prefix parsing (l_,lp_,_init_,initialize_,lean_apply_N,_lean_main), postprocessing (suffix flags, private name stripping, hygienic suffix stripping, specialization contexts), backtrace line parsing, and C exports via@[export].The C++ runtime backtrace handler now calls the Lean-exported functions instead of its own 792-line reimplementation. This is safe because
print_backtraceis only called fromlean_panic_impl(soft panics), notlean_internal_panic.The Python profiler demangler (
script/profiler/lean_demangle.py) is replaced with a thin subprocess wrapper around a Lean CLI tool, preserving thedemangle_lean_nameAPI so downstream scripts work unchanged.New files:
src/Lean/Compiler/NameDemangling.lean— single source of truth (483 lines)tests/lean/run/demangling.lean— comprehensive tests (281 lines)script/profiler/lean_demangle_cli.lean—c++filt-style CLI toolDeleted files:
src/runtime/demangle.cpp(792 lines)src/runtime/demangle.h(26 lines)script/profiler/test_demangle.py(670 lines)Net: −1,381 lines of duplicated C++/Python code.
🤖 Prepared with Claude Code