Upgrade Rust toolchain to nightly-2024-03-21

[adpaco-aws]

Upgrades the Rust toolchain to nightly-2024-03-21. The relevant changes in Rust are:

• Avoid various uses of Option<Span> in favor of using DUMMY_SP in the few cases that used None rust-lang/rust#122480
• Reduce pub usage in rustc_session. rust-lang/rust#122748
• cargo: prevent dashes in lib.name rust-lang/cargo#12783

I wasn't confident that our regression testing could detect differences in the file paths being generated with the new logic, so I added code similar to the following just to check they were equivalent:

             let base_filename = tcx.output_filenames(()).output_path(OutputType::Object);
+            let binding = tcx.output_filenames(()).path(OutputType::Object);
+            let base_filename2 = binding.as_path();
+            assert_eq!(base_filename, base_filename2);

Note that this was done for each instance where we used output_path, and completed regression testing with the previous toolchain version. I also checked manually the instance where we generate a .dot graph for debug purposes following the instructions here (a libmain.dot file was generated for the main.rs in one of my projects).

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 and MIT licenses.

[celinval]

Thanks Adrian. It might be worth creating an issue in the Cargo project to ensure cargo metadata contains the information we need.

I believe someone mentioned that in the original PR.

[adpaco-aws]

It might be worth creating an issue in the Cargo project to ensure cargo metadata contains the information we need.

Thanks for the suggestion! I created oli-obk/cargo_metadata#259 to follow up on this 😄