Resolve layout of single variant enums

[gnzlbg]

Addresses part of #79. Addresses part of #37 .

I don't see any reasons not to guarantee this, and AFAICT nobody has mentioned any either.

---

EDIT: Note that this doesn't say anything about the layout of enums without variants, or where all variants are uninhabited. That should better be tackled in a different PR.

[danielhenrymantilla]

What about #[non_exhaustive] enums?

[gnzlbg]

AFAICT #[non_exhaustive] does not interact with layout.

[hanna-kruppe]

To elaborate: users of non-exhaustive enums can't rely on the number of variants not changing under their feet, so for their purposes this part of the UCG would not guarantee anything about the layout of the non-exhaustive enum, since it is not guaranteed to meet the precondition (only one variant). Similarly, most guarantees about the layout of structs automatically wouldn't apply to non-exhaustive structs since fields might be added to them.

(Within the same crate/module that defines the non-exhaustive enum, it would technically be allowed, though I guess it's a bit fragile to rely on it when you intend to expend the set of variants later.)

[gnzlbg]

@rkruppe

I don't know what value would we add by not allowing such code to rely on the #[non_exhaustive] enums layout; AFAICT there aren't any extra optimizations that this would allow.

So I think I disagree (you kind of hint things both ways) ? I think that the layout guarantees specified here apply to both #[non_exhaustive] and "exhaustive" enums because #[non_exhaustive] does not affect layout and unsafe code is allowed to rely on layout guarantees.

Whether that unsafe code is safe or not would be a different issue - e.g. if the layout changes and the unsafe code is not updated, then it would probably be incorrect, but this is true for all unsafe code. Still, unsafe code written in the same module as a #[non_exhaustive] enum can make use of layout guarantees and be safe.

[hanna-kruppe]

I didn't mean to imply layout computation or what reliance on layout is considered UB by the Rust language should be affected by non_exhaustive. I was just speaking of contracts between libraries: an external library that assumes for soundness that a non_exhaustive struct or enum has a certain layout (based on information that is strictly ) is just as wrong as if it relies on private fields or other information not part of the API. As we both noted, code within the same API boundary as the non-exhaustive type is not subject to the same restrictions.

[gnzlbg]

@rkruppe

I was just speaking of contracts between libraries: an external library that assumes for soundness that a non_exhaustive struct or enum has a certain layout (based on information that is strictly ) is just as wrong as if it relies on private fields or other information not part of the API.

Such a library is ugly, brittle, non-portable (to different library versions / toolchain), etc. but it is only wrong if its assumptions are incorrect.

If a Rust library exposes a #[non_exhaustive] #[repr(C)] enum, and one wants to interface with that library from C, then one can make it work, e.g., by just fixing the exact version of the dependency and toolchain and manually inspecting the variants of the enum.

Ideally, nobody would ever need to do this, and such APIs are obviously bad, but doing this types of things is the bread and butter of, for example, the libc crate.

I don't think we can do much about this either. Making a type public leaks its layout, and while features like #[non_exhaustive] protect users from accidental misuse, they can't prevent a motivated user from exploiting layout for profit, nor should that be their goal. The only way I can think of in which we could help, would be to make the rules clearer, so that those who need to exploit API details know where they stand, and maybe give them better tools, so that they don't have to rely on undefined behavior for their hacks.

[RalfJung]

The file says at the beginning that we try to mark what is RFC'ed and what not. Should these new sections get such comments?

[gnzlbg]

Should these new sections get such comments?

They probably should, I'll add that these sections have not gone through the RFC process, although the other sections more explicitly call what has gone through the process.

Before turning the reference or a subset of it into an RFC, we are going to have to do multiple rounds of making everything consistent, and that will include deciding to call out either what has been RFC'ed already, what hasn't, or both.

As layout has shown, some of the fundamental definitions evolve over time as we work through newer topics like validity. Each time they change, we kind of have to go through everything.

[gnzlbg]

I think all issues have been resolved. Please take a look.

[gnzlbg]

I've recorded the proposal for multi-variant enums with one inhabited variant in the tracking issue: #79 (comment)

[RalfJung]

r=me with the nit resolved.

[gnzlbg]

@RalfJung nit resolved

[RalfJung]

@rkruppe was the merge here fine with you? (Your last comment in this PR was quite a while ago.)

[hanna-kruppe]

Yes!