linker: More systematic handling of CRT objects

[petrochenkov]

Document which kinds of crt0.o-like objects we link and in which cases, discovering bugs in process.
src/librustc_target/spec/crt_objects.rs is the place to start reading from.

This PR also automatically contains half of the -static-pie support (#70740), because that's one of the six cases that we need to consider when linking CRT objects.

This is a breaking change for custom target specifications that specify CRT objects.

Closes #30868

[rust-highfive]

r? @eddyb

(rust_highfive has picked a reviewer for you, use r? to override)

[petrochenkov]

The only case of rustc interpreting user-provided raw linker arguments is removed here.
First, -static wins over -pie so we could still pass -pie if the user passed -static with the same effect.
Second, it was a hack, this static guessing wasn't done consistently in other places, equivalent cases like -Wl,-static weren't recognized, and we shouldn't be interpreting raw linker args in general.

[petrochenkov]

No idea which libc flavor Fuchsia uses, just kept the status quo behavior.

[bors]

☔ The latest upstream changes (presumably #71807) made this pull request unmergeable. Please resolve the merge conflicts.

[petrochenkov]

@smaeul @shizmob @mixi (people who participated in #40113 and related PRs), which of you I can ping on PRs/issues like this one?

I have a question.
From the paper trail (#71586 (comment)) it looks like it's strongly preferable to use a musl-aware system gcc as a linker if it's available, and rely on it for pulling CRT objects. (And system libc.a as well, but it's a separate question.)
At the same time, for backward compatibility in some cases we have to fall back to gcc with -nostdlib and manually linking CRT objects distributed with rustc.

What would be a good and safe approximation for "musl-aware system gcc is available" that rustc could use to apply the fallback only when it's really necessary?

• host == target (both musl) seems like a good heuristic
• anything else?

[smaeul]

Checking to see if $GCC -dumpmachine contains "-musl" may be somewhat less fragile, as it would handle musl-to-musl cross compiling, and would handle the user overriding the linker (say, to a real musl cross toolchain when cross compiling glibc-to-musl).

[petrochenkov]

Checking to see if $GCC -dumpmachine contains "-musl" may be somewhat less fragile

Clarification: without expensive environment probing like running processes (but checking files is ok).

Ideally such check is performed by a build system, and the result is cached, and then passed to the compiler.
Cargo cannot do such things unfortunately, but it would be useful for other things as well (e.g. detecting support of -static-pie and -no-pie).

and would handle the user overriding the linker

Explicitly passed -C linker may be a good heuristic actually.

[petrochenkov]

So, I've measured the slowdown from the gcc -dumpmachine probing on a subset of UI tests in rustc (./x.py test --bless --stage 1 src/test/ui --test-args f, that's at least 3206 invocations of rustc, some don't do linking though).
The measurements were done on Windows, which is known for its slow process creation, so the slowdown on Linux should be smaller.

No probing (3 runs):

real    1m46.719s
real    1m47.304s
real    1m49.113s

Probing (3 runs):

real    1m50.650s
real    1m51.125s
real    1m50.946s

That's ~3% slower.

Should be pretty acceptable if it only happens in setups in which other heuristics fail.
I'll try to implement it in a separate PR.

In this PR some FIXMEs and comments need to be updated before it's ready.

[petrochenkov]

r? @cuviper
eddyb is nowhere in sight since april

[cuviper]

I don't have deep knowledge of all of these targets, but it looks reasonable to me.

@bors r+

[bors]

📌 Commit 49eb35c has been approved by cuviper

[bors]

⌛ Testing commit 49eb35c with merge 64ad709...

[bors]

☀️ Test successful - checks-azure
Approved by: cuviper
Pushing 64ad709 to master...

[tchebb]

Am I missing something, or is this logic inverted from what it should be? If crt_static() returns true, aren't we only allowed to create a DynamicDylib when crt_static_allows_dylibs is also true?

[petrochenkov]

This case is here for consistency, but we should never hit it right now because if invalid_output_for_target is true then we skip the crate type and never get to linking.
But if we didn't skip it (like we don't skip PIC executables when +crt-static is enabled but not supported), then we would fall back from StaticDylib to DynamicDylib.