mangling: mangle impl params w/ v0 scheme by davidtwco · Pull Request #75675 · rust-lang/rust

davidtwco · 2020-08-18T18:31:29Z

This PR modifies v0 symbol mangling to include all generic parameters from impl blocks (not just those used in the self type) - an alternative fix to #75326.

original:
   _RNCNvXCs4fqI2P2rA04_19impl_param_manglingINtB4_3FooppENtNtNtNtCsfnEnqCNU58Z_4core4iter6traits8iterator8Iterator4next0B4_
//        |------------ B4_ ----------------|
// _R (N C (N v (X (C ((s 4fqI2p2rA04_) 19impl_param_mangling)) (I (N t B4_ 3Foo) pp E) (N t (N t (N t (N t (C ((s fnEnqCNU58Z_) 4core)) 4iter) 6traits) 8iterator) 8Iterator)) 4next) 0) B4_

modified:
   _RNvXINICs4fqI2P2rA04_11issue_753260pppEINtB5_3FooppENtNtNtNtCsfnEnqCNU58Z_4core4iter6traits8iterator8Iterator4nextB5_
// _R (N v (X (I (N I (C ((s 4fqI2P2rA04_) 11issue_75326)) 0) ppp E) (I (N t B5_ 3Foo) pp E) (N t (N t (N t (N t (C ((s fnEnqCNU58Z_) 4core)) 4iter) 6traits) 8iterator) 8Iterator)) 4next) B5_
//            |     ^                                              |
//            |     |                                              |
//            |     new impl namespace                             |

~~Submitted as a draft as after some discussion w/ @eddyb, I'm going to do some investigation into (yet more alternative) changes to polymorphization that might remove the necessity for this.~~

r? @eddyb

lcnr · 2020-08-18T20:39:20Z

Would this allow us to revert #75518?

davidtwco · 2020-08-18T20:40:27Z

Would this allow us to revert #75518?

Yeah, it could.

~~Didn't see these CI failures locally - seems like the mangling is still correct but the discriminants have changed.~~ Fixed now, you can't use std/core types in these tests and have to normalize for bitness.

tesuji · 2020-08-19T08:11:33Z

Do we need to update the v0 RFC and related references?

davidtwco · 2020-08-20T10:47:02Z

After some more experimentation, I've opened #75737 which removes predicate logic from polymorphization and depends on this PR. I think we should proceed with landing this.

src/librustc_symbol_mangling/v0.rs

src/test/ui/symbol-names/impl1.legacy.stderr

src/test/ui/symbol-names/impl1.rs

src/test/ui/symbol-names/issue-60925.rs

src/test/ui/symbol-names/issue-75326.rs

davidtwco

Noting some things that need changed after chatting with @eddyb.

compiler/rustc_symbol_mangling/src/v0.rs

compiler/rustc_symbol_mangling/src/test.rs

@eddyb

…ms, r=eddyb mangling: mangle impl params w/ v0 scheme This PR modifies v0 symbol mangling to include all generic parameters from impl blocks (not just those used in the self type) - an alternative fix to rust-lang#75326. ``` original: _RNCNvXCs4fqI2P2rA04_19impl_param_manglingINtB4_3FooppENtNtNtNtCsfnEnqCNU58Z_4core4iter6traits8iterator8Iterator4next0B4_ // |------------ B4_ ----------------| // _R (N C (N v (X (C ((s 4fqI2p2rA04_) 19impl_param_mangling)) (I (N t B4_ 3Foo) pp E) (N t (N t (N t (N t (C ((s fnEnqCNU58Z_) 4core)) 4iter) 6traits) 8iterator) 8Iterator)) 4next) 0) B4_ modified: _RNvXINICs4fqI2P2rA04_11issue_753260pppEINtB5_3FooppENtNtNtNtCsfnEnqCNU58Z_4core4iter6traits8iterator8Iterator4nextB5_ // _R (N v (X (I (N I (C ((s 4fqI2P2rA04_) 11issue_75326)) 0) ppp E) (I (N t B5_ 3Foo) pp E) (N t (N t (N t (N t (C ((s fnEnqCNU58Z_) 4core)) 4iter) 6traits) 8iterator) 8Iterator)) 4next) B5_ // | ^ | // | | | // | new impl namespace | ``` ~~Submitted as a draft as after some discussion w/ @eddyb, I'm going to do some investigation into (yet more alternative) changes to polymorphization that might remove the necessity for this.~~ r? @eddyb

@eddyb

…ms, r=eddyb mangling: mangle impl params w/ v0 scheme This PR modifies v0 symbol mangling to include all generic parameters from impl blocks (not just those used in the self type) - an alternative fix to rust-lang#75326. ``` original: _RNCNvXCs4fqI2P2rA04_19impl_param_manglingINtB4_3FooppENtNtNtNtCsfnEnqCNU58Z_4core4iter6traits8iterator8Iterator4next0B4_ // |------------ B4_ ----------------| // _R (N C (N v (X (C ((s 4fqI2p2rA04_) 19impl_param_mangling)) (I (N t B4_ 3Foo) pp E) (N t (N t (N t (N t (C ((s fnEnqCNU58Z_) 4core)) 4iter) 6traits) 8iterator) 8Iterator)) 4next) 0) B4_ modified: _RNvXINICs4fqI2P2rA04_11issue_753260pppEINtB5_3FooppENtNtNtNtCsfnEnqCNU58Z_4core4iter6traits8iterator8Iterator4nextB5_ // _R (N v (X (I (N I (C ((s 4fqI2P2rA04_) 11issue_75326)) 0) ppp E) (I (N t B5_ 3Foo) pp E) (N t (N t (N t (N t (C ((s fnEnqCNU58Z_) 4core)) 4iter) 6traits) 8iterator) 8Iterator)) 4next) B5_ // | ^ | // | | | // | new impl namespace | ``` ~~Submitted as a draft as after some discussion w/ @eddyb, I'm going to do some investigation into (yet more alternative) changes to polymorphization that might remove the necessity for this.~~ r? @eddyb

Rollup of 14 pull requests Successful merges: - rust-lang#75023 (ensure arguments are included in count mismatch span) - rust-lang#75265 (Add `str::{Split,RSplit,SplitN,RSplitN,SplitTerminator,RSplitTerminator,SplitInclusive}::as_str` methods) - rust-lang#75675 (mangling: mangle impl params w/ v0 scheme) - rust-lang#76084 (Refactor io/buffered.rs into submodules) - rust-lang#76119 (Stabilize move_ref_pattern) - rust-lang#77493 (ICEs should always print the top of the query stack) - rust-lang#77619 (Use futex-based thread-parker for Wasm32.) - rust-lang#77646 (For backtrace, use StaticMutex instead of a raw sys Mutex.) - rust-lang#77648 (Static mutex is static) - rust-lang#77657 (Cleanup cloudabi mutexes and condvars) - rust-lang#77672 (Simplify doc-cfg rendering based on the current context) - rust-lang#77780 (rustc_parse: fix spans on cast and range exprs with attrs) - rust-lang#77935 (BTreeMap: make PartialCmp/PartialEq explicit and tested) - rust-lang#77980 (Fix intra doc link for needs_drop) Failed merges: r? `@ghost`

This commit removes all logic which marks parameters as used based on their presence in predicates - given rust-lang#75675, this will enable more polymorphization and avoid the symbol clashes that predicate logic previously sidestepped. Signed-off-by: David Wood <david@davidtw.co>

…edicates, r=lcnr polymorphization: shims and predicates Supersedes rust-lang#75737 and rust-lang#75414. This pull request includes up some changes to polymorphization which hadn't landed previously and gets stage2 bootstrapping and the test suite passing when polymorphization is enabled. There are still issues with `type_id` and polymorphization to investigate but this should get polymorphization in a reasonable state to work on. - rust-lang#75737 and rust-lang#75414 both worked but were blocked on having the rest of the test suite pass (with polymorphization enabled) with and without the PRs. It makes more sense to just land these so that the changes are in. - rust-lang#75737's changes remove the restriction of `InstanceDef::Item` on polymorphization, so that shims can now be polymorphized. This won't have much of an effect until polymorphization's analysis is more advanced, but it doesn't hurt. - rust-lang#75414's changes remove all logic which marks parameters as used based on their presence in predicates - given rust-lang#75675, this will enable more polymorphization and avoid the symbol clashes that predicate logic previously sidestepped. - Polymorphization now explicitly checks (and skips) foreign items, this is necessary for stage2 bootstrapping to work when polymorphization is enabled. - The conditional determining the emission of a note adding context to a post-monomorphization error has been modified. Polymorphization results in `optimized_mir` running for shims during collection where that wouldn't happen previously, some errors are emitted during `optimized_mir` and these were considered post-monomorphization errors with the existing logic (more errors and shims have a `DefId` coming from the std crate, not the local crate), adding a note that resulted in tests failing. It isn't particularly feasible to change where polymorphization runs or prevent it from using `optimized_mir`, so it seemed more reasonable to not change the conditional. - `characteristic_def_id_of_type` was being invoked during partitioning for self types of impl blocks which had projections that depended on the value of unused generic parameters of a function - this caused a ICE in a debuginfo test. If partitioning is enabled and the instance needs substitution then this is skipped. That test still fails for me locally, but not with an ICE, but it fails in a fresh checkout too, so 🤷‍♂️. r? `@lcnr`

…icates, r=lcnr polymorphization: shims and predicates Supersedes rust-lang#75737 and rust-lang#75414. This pull request includes up some changes to polymorphization which hadn't landed previously and gets stage2 bootstrapping and the test suite passing when polymorphization is enabled. There are still issues with `type_id` and polymorphization to investigate but this should get polymorphization in a reasonable state to work on. - rust-lang#75737 and rust-lang#75414 both worked but were blocked on having the rest of the test suite pass (with polymorphization enabled) with and without the PRs. It makes more sense to just land these so that the changes are in. - rust-lang#75737's changes remove the restriction of `InstanceDef::Item` on polymorphization, so that shims can now be polymorphized. This won't have much of an effect until polymorphization's analysis is more advanced, but it doesn't hurt. - rust-lang#75414's changes remove all logic which marks parameters as used based on their presence in predicates - given rust-lang#75675, this will enable more polymorphization and avoid the symbol clashes that predicate logic previously sidestepped. - Polymorphization now explicitly checks (and skips) foreign items, this is necessary for stage2 bootstrapping to work when polymorphization is enabled. - The conditional determining the emission of a note adding context to a post-monomorphization error has been modified. Polymorphization results in `optimized_mir` running for shims during collection where that wouldn't happen previously, some errors are emitted during `optimized_mir` and these were considered post-monomorphization errors with the existing logic (more errors and shims have a `DefId` coming from the std crate, not the local crate), adding a note that resulted in tests failing. It isn't particularly feasible to change where polymorphization runs or prevent it from using `optimized_mir`, so it seemed more reasonable to not change the conditional. - `characteristic_def_id_of_type` was being invoked during partitioning for self types of impl blocks which had projections that depended on the value of unused generic parameters of a function - this caused a ICE in a debuginfo test. If partitioning is enabled and the instance needs substitution then this is skipped. That test still fails for me locally, but not with an ICE, but it fails in a fresh checkout too, so 🤷‍♂️. r? `@lcnr`

…ngling-scheme, r=wesleywiser sess: default to v0 symbol mangling on nightly cc #60705 rust-lang/compiler-team#938 Rust's current mangling scheme depends on compiler internals; loses information about generic parameters (and other things) which makes for a worse experience when using external tools that need to interact with Rust symbol names; is inconsistent; and can contain `.` characters which aren't universally supported. Therefore, Rust has defined its own symbol mangling scheme which is defined in terms of the Rust language, not the compiler implementation; encodes information about generic parameters in a reversible way; has a consistent definition; and generates symbols that only use the characters `A-Z`, `a-z`, `0-9`, and `_`. Support for the new Rust symbol mangling scheme has been added to upstream tools that will need to interact with Rust symbols (e.g. debuggers). This pull request changes the default symbol mangling scheme from the legacy scheme to the new Rust mangling scheme on nightly. The following pull requests implemented v0 mangling in rustc (if I'm missing any, let me know): - #57967 - #63559 - #75675 - #77452 - #77554 - #83767 - #87194 - #87789 Rust's symbol mangling scheme has support in the following external tools: - `binutils`/`gdb` (GNU `libiberty`) - [[PATCH] Move rust_{is_mangled,demangle_sym} to a private libiberty header. ](https://gcc.gnu.org/pipermail/gcc-patches/2019-June/523011.html) committed as gcc-mirror/gcc@979526c - [[PATCH] Simplify and generalize rust-demangle's unescaping logic. ](https://gcc.gnu.org/pipermail/gcc-patches/2019-August/527835.html) committed as gcc-mirror/gcc@42bf58b - [[PATCH] Remove some restrictions from rust-demangle. ](https://gcc.gnu.org/pipermail/gcc-patches/2019-September/530445.html) committed as gcc-mirror/gcc@e1cb00d - [[PATCH] Refactor rust-demangle to be independent of C++ demangling. ](https://gcc.gnu.org/pipermail/gcc-patches/2019-November/533719.html) ([original submission](https://gcc.gnu.org/pipermail/gcc-patches/2019-October/532388.html)) committed as gcc-mirror/gcc@32fc371 - [[PATCH] Support the new ("v0") mangling scheme in rust-demangle. ](https://gcc.gnu.org/pipermail/gcc-patches/2020-November/558905.html) ([original submission](https://gcc.gnu.org/pipermail/gcc-patches/2020-March/542012.html)) committed as gcc-mirror/gcc@8409649 - `lldb`/`llvm-objdump`/`llvm-nm`/`llvm-symbolizer`/`llvm-cxxfilt`/etc - llvm/llvm-project@7310403 - llvm/llvm-project@c8c2b46 - llvm/llvm-project@0a2d4f3 - Linux `perf` - `valgrind` - [Update demangler to support Rust v0 name mangling.](https://bugs.kde.org/show_bug.cgi?id=431306) #85530 (comment) contains a summary of the most recent crater run of the v0 mangling, and the remaining issues from that were fixed by #87194 (confirmed by follow-up crater run, #85530 (comment)). `@rustbot` label +T-compiler r? `@michaelwoerister`

…ngling-scheme, r=wesleywiser sess: default to v0 symbol mangling on nightly cc rust-lang/rust#60705 rust-lang/compiler-team#938 Rust's current mangling scheme depends on compiler internals; loses information about generic parameters (and other things) which makes for a worse experience when using external tools that need to interact with Rust symbol names; is inconsistent; and can contain `.` characters which aren't universally supported. Therefore, Rust has defined its own symbol mangling scheme which is defined in terms of the Rust language, not the compiler implementation; encodes information about generic parameters in a reversible way; has a consistent definition; and generates symbols that only use the characters `A-Z`, `a-z`, `0-9`, and `_`. Support for the new Rust symbol mangling scheme has been added to upstream tools that will need to interact with Rust symbols (e.g. debuggers). This pull request changes the default symbol mangling scheme from the legacy scheme to the new Rust mangling scheme on nightly. The following pull requests implemented v0 mangling in rustc (if I'm missing any, let me know): - rust-lang/rust#57967 - rust-lang/rust#63559 - rust-lang/rust#75675 - rust-lang/rust#77452 - rust-lang/rust#77554 - rust-lang/rust#83767 - rust-lang/rust#87194 - rust-lang/rust#87789 Rust's symbol mangling scheme has support in the following external tools: - `binutils`/`gdb` (GNU `libiberty`) - [[PATCH] Move rust_{is_mangled,demangle_sym} to a private libiberty header. ](https://gcc.gnu.org/pipermail/gcc-patches/2019-June/523011.html) committed as gcc-mirror/gcc@979526c - [[PATCH] Simplify and generalize rust-demangle's unescaping logic. ](https://gcc.gnu.org/pipermail/gcc-patches/2019-August/527835.html) committed as gcc-mirror/gcc@42bf58b - [[PATCH] Remove some restrictions from rust-demangle. ](https://gcc.gnu.org/pipermail/gcc-patches/2019-September/530445.html) committed as gcc-mirror/gcc@e1cb00d - [[PATCH] Refactor rust-demangle to be independent of C++ demangling. ](https://gcc.gnu.org/pipermail/gcc-patches/2019-November/533719.html) ([original submission](https://gcc.gnu.org/pipermail/gcc-patches/2019-October/532388.html)) committed as gcc-mirror/gcc@32fc371 - [[PATCH] Support the new ("v0") mangling scheme in rust-demangle. ](https://gcc.gnu.org/pipermail/gcc-patches/2020-November/558905.html) ([original submission](https://gcc.gnu.org/pipermail/gcc-patches/2020-March/542012.html)) committed as gcc-mirror/gcc@8409649 - `lldb`/`llvm-objdump`/`llvm-nm`/`llvm-symbolizer`/`llvm-cxxfilt`/etc - llvm/llvm-project@7310403 - llvm/llvm-project@c8c2b46 - llvm/llvm-project@0a2d4f3 - Linux `perf` - `valgrind` - [Update demangler to support Rust v0 name mangling.](https://bugs.kde.org/show_bug.cgi?id=431306) rust-lang/rust#85530 (comment) contains a summary of the most recent crater run of the v0 mangling, and the remaining issues from that were fixed by rust-lang/rust#87194 (confirmed by follow-up crater run, rust-lang/rust#85530 (comment)). `@rustbot` label +T-compiler r? `@michaelwoerister`

…symbol-mangling-scheme, r=wesleywiser sess: default to v0 symbol mangling on nightly cc rust-lang#60705 rust-lang/compiler-team#938 Rust's current mangling scheme depends on compiler internals; loses information about generic parameters (and other things) which makes for a worse experience when using external tools that need to interact with Rust symbol names; is inconsistent; and can contain `.` characters which aren't universally supported. Therefore, Rust has defined its own symbol mangling scheme which is defined in terms of the Rust language, not the compiler implementation; encodes information about generic parameters in a reversible way; has a consistent definition; and generates symbols that only use the characters `A-Z`, `a-z`, `0-9`, and `_`. Support for the new Rust symbol mangling scheme has been added to upstream tools that will need to interact with Rust symbols (e.g. debuggers). This pull request changes the default symbol mangling scheme from the legacy scheme to the new Rust mangling scheme on nightly. The following pull requests implemented v0 mangling in rustc (if I'm missing any, let me know): - rust-lang#57967 - rust-lang#63559 - rust-lang#75675 - rust-lang#77452 - rust-lang#77554 - rust-lang#83767 - rust-lang#87194 - rust-lang#87789 Rust's symbol mangling scheme has support in the following external tools: - `binutils`/`gdb` (GNU `libiberty`) - [[PATCH] Move rust_{is_mangled,demangle_sym} to a private libiberty header. ](https://gcc.gnu.org/pipermail/gcc-patches/2019-June/523011.html) committed as gcc-mirror/gcc@979526c - [[PATCH] Simplify and generalize rust-demangle's unescaping logic. ](https://gcc.gnu.org/pipermail/gcc-patches/2019-August/527835.html) committed as gcc-mirror/gcc@42bf58b - [[PATCH] Remove some restrictions from rust-demangle. ](https://gcc.gnu.org/pipermail/gcc-patches/2019-September/530445.html) committed as gcc-mirror/gcc@e1cb00d - [[PATCH] Refactor rust-demangle to be independent of C++ demangling. ](https://gcc.gnu.org/pipermail/gcc-patches/2019-November/533719.html) ([original submission](https://gcc.gnu.org/pipermail/gcc-patches/2019-October/532388.html)) committed as gcc-mirror/gcc@32fc371 - [[PATCH] Support the new ("v0") mangling scheme in rust-demangle. ](https://gcc.gnu.org/pipermail/gcc-patches/2020-November/558905.html) ([original submission](https://gcc.gnu.org/pipermail/gcc-patches/2020-March/542012.html)) committed as gcc-mirror/gcc@8409649 - `lldb`/`llvm-objdump`/`llvm-nm`/`llvm-symbolizer`/`llvm-cxxfilt`/etc - llvm/llvm-project@7310403 - llvm/llvm-project@c8c2b46 - llvm/llvm-project@0a2d4f3 - Linux `perf` - `valgrind` - [Update demangler to support Rust v0 name mangling.](https://bugs.kde.org/show_bug.cgi?id=431306) rust-lang#85530 (comment) contains a summary of the most recent crater run of the v0 mangling, and the remaining issues from that were fixed by rust-lang#87194 (confirmed by follow-up crater run, rust-lang#85530 (comment)). `@rustbot` label +T-compiler r? `@michaelwoerister`

…ngling-scheme, r=wesleywiser sess: default to v0 symbol mangling on nightly cc rust-lang/rust#60705 rust-lang/compiler-team#938 Rust's current mangling scheme depends on compiler internals; loses information about generic parameters (and other things) which makes for a worse experience when using external tools that need to interact with Rust symbol names; is inconsistent; and can contain `.` characters which aren't universally supported. Therefore, Rust has defined its own symbol mangling scheme which is defined in terms of the Rust language, not the compiler implementation; encodes information about generic parameters in a reversible way; has a consistent definition; and generates symbols that only use the characters `A-Z`, `a-z`, `0-9`, and `_`. Support for the new Rust symbol mangling scheme has been added to upstream tools that will need to interact with Rust symbols (e.g. debuggers). This pull request changes the default symbol mangling scheme from the legacy scheme to the new Rust mangling scheme on nightly. The following pull requests implemented v0 mangling in rustc (if I'm missing any, let me know): - rust-lang/rust#57967 - rust-lang/rust#63559 - rust-lang/rust#75675 - rust-lang/rust#77452 - rust-lang/rust#77554 - rust-lang/rust#83767 - rust-lang/rust#87194 - rust-lang/rust#87789 Rust's symbol mangling scheme has support in the following external tools: - `binutils`/`gdb` (GNU `libiberty`) - [[PATCH] Move rust_{is_mangled,demangle_sym} to a private libiberty header. ](https://gcc.gnu.org/pipermail/gcc-patches/2019-June/523011.html) committed as gcc-mirror/gcc@979526c - [[PATCH] Simplify and generalize rust-demangle's unescaping logic. ](https://gcc.gnu.org/pipermail/gcc-patches/2019-August/527835.html) committed as gcc-mirror/gcc@42bf58b - [[PATCH] Remove some restrictions from rust-demangle. ](https://gcc.gnu.org/pipermail/gcc-patches/2019-September/530445.html) committed as gcc-mirror/gcc@e1cb00d - [[PATCH] Refactor rust-demangle to be independent of C++ demangling. ](https://gcc.gnu.org/pipermail/gcc-patches/2019-November/533719.html) ([original submission](https://gcc.gnu.org/pipermail/gcc-patches/2019-October/532388.html)) committed as gcc-mirror/gcc@32fc371 - [[PATCH] Support the new ("v0") mangling scheme in rust-demangle. ](https://gcc.gnu.org/pipermail/gcc-patches/2020-November/558905.html) ([original submission](https://gcc.gnu.org/pipermail/gcc-patches/2020-March/542012.html)) committed as gcc-mirror/gcc@8409649 - `lldb`/`llvm-objdump`/`llvm-nm`/`llvm-symbolizer`/`llvm-cxxfilt`/etc - llvm/llvm-project@7310403 - llvm/llvm-project@c8c2b46 - llvm/llvm-project@0a2d4f3 - Linux `perf` - `valgrind` - [Update demangler to support Rust v0 name mangling.](https://bugs.kde.org/show_bug.cgi?id=431306) rust-lang/rust#85530 (comment) contains a summary of the most recent crater run of the v0 mangling, and the remaining issues from that were fixed by rust-lang/rust#87194 (confirmed by follow-up crater run, rust-lang/rust#85530 (comment)). `@rustbot` label +T-compiler r? `@michaelwoerister`

rust-highfive assigned eddyb Aug 18, 2020

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Aug 18, 2020

davidtwco force-pushed the symbol-mangling-impl-params branch 5 times, most recently from 2fdea0f to 78108f9 Compare August 19, 2020 13:47

davidtwco marked this pull request as ready for review August 20, 2020 10:44

davidtwco mentioned this pull request Aug 20, 2020

polymorphize: remove predicate logic #75737

Closed