Explain how to get the discriminant out of a #[repr(T)] enum with payload

[lukas-code]

example stolen from rust-lang/reference#1055

@rustbot label A-docs

[rustbot]

r? @scottmcm

(rustbot has picked a reviewer for you, use r? to override)

[rustbot]

Hey! It looks like you've submitted a new PR for the library teams!

If this PR contains changes to any rust-lang/rust public library APIs then please comment with @rustbot label +T-libs-api -T-libs to tag it appropriately. If this PR contains changes to any unstable APIs please edit the PR description to add a link to the relevant API Change Proposal or create one if you haven't already. If you're unsure where your change falls no worries, just leave it as is and the reviewer will take a look and make a decision to forward on if necessary.

Examples of T-libs-api changes:

• Stabilizing library features
• Introducing insta-stable changes such as new implementations of existing stable traits on existing stable types
• Introducing new or changing existing unstable library APIs (excluding permanently unstable features / features without a tracking issue)
• Changing public documentation in ways that create new stability guarantees
• Changing observable runtime behavior of library APIs

[scottmcm]

Thanks for the PR! Overall it looks good, though I have a few hopefully-simple requests.

@rustbot author

[scottmcm]

We're trying to avoid as on pointers now, where possible, in favour of methods. So please change this to something using cast & friends, maybe

Suggested change

	/// unsafe { *(self as *const Self as *const u8) }
	/// unsafe { <*const _>::from(self).cast::<u8>().read() }

Or you could consider something like

    pub fn discriminant(&self) -> &u8 {
        unsafe { NonNull::from(self).cast::<u8>().as_ref() }
    }

[scottmcm]

Oh, and it would be good to have a comment on the unsafe block repeating why it's sound. Maybe

// SAFETY: because `Self` is marked `repr(u8)`,
// its layout is a `repr(C)` `union` between `repr(C)` structs,
// each of which has the `u8` discriminant as its first field,
// so we can read the discriminant without offsetting the pointer.

[lukas-code]

TIL that <*const _>::from(reference) works. Since there is no impl From<&T> for *const T, it probably selects From<*const T> for *const T and then coerces the parameter?

We should probably document the "canonical" way to convert a reference to a raw pointer when coercion isn't an option somewhere. So far I've seen reference as *const _, ptr::addr_of!(*reference) and now <*const _>::from(reference).

[lukas-code]

I've removed the as cast on pointers now, but kept *ptr over ptr.read() since u8 is Copy and that seems "safer" to me.

[scottmcm]

Note that the test below currently "passes" even without the repr, though it's UB without the repr -- and MIRI doesn't complain about it either.

So thus I think it's important to put extra emphasis on how this only applies for non-default reprs, and it'd be nice to link to some documentation about those other reprs.

Here's a first draft of an idea, please improve it:

Note that it is undefined behavior to [transmute] from [Discriminant] to a primitive.

If an enum has opted-in to having a defined layout, then it's possible to use pointers to read the memory location storing the discriminant. That cannot be done for anything using the default enum layout, however, as it's undefined where the discriminant is stored -- it might not even be stored at all!

[lukas-code]

This section doesn't exist until rust-lang/reference#1055 is merged, but the linked page still contains some relevant information.

[lukas-code]

The CI didn't like this, so we need to revert 946d51e later.

[lukas-code]

I've updated it now to include some links to the Reference on enum reprs. I'd rather not link to the RFC, since RFCs tend to get outdated over time and that has led to confusion before. I intentionally left out repr(C) enums here, since the ABI of C enums is notably underspecified and may or may not be c_int.

I also added the easy way to get the discriminant of a c-like enum now, to avoid having someone use unsafe code when it isn't necessary.

Docs preview available here: https://lukas-code.github.io/rust-docs/core/mem/fn.discriminant.html

[lukas-code]

@rustbot ready

[scottmcm]

Thanks for the updates! Having the safe way there first is a great addition.

@bors r+ rollup

[bors]

📌 Commit 946d51e has been approved by scottmcm

It is now in the queue for this repository.