Calculate spare bits for multi-payload enums from first principles

[tbkka]

This adds a getSpareBits method to all the TypeInfo classes
that returns a suitable bitmask indicating the spare bits available
in values of this type. This gives us a way to recursively explore
the type tree and build up a full spare bit mask for an arbitrary type.

Happily, it appears we actually do have enough information to do this
calculation entirely from first principles, without requiring additional
reflection information. So once this is stable, we should remove my
earlier incomplete effort to publish spare bit mask info in the reflection
data, as that adds unnecessary metadata to every binary.

This doubtless still has plenty of holes, but seems sufficient
to handle a few basic enum types, including the stdlib DecodingError
which was used as an example to work out some key issues.

Resolves rdar://126563813

[slavapestov]

One difficulty you might run into is that this depends on resilience boundaries. Eg,

public struct S {
  var x: AnyObject
}

public enum E {
  case first(S)
  case second(S)
}

The layout of E will use spare bits if and only if E is resilient or S is not resilient. That is, we have to consider whether any potential client of E "knows" what the spare bits of S are:

• S resilient, E not resilient: no spare bits
• S resilient, E resilient: spare bits ok
• S not resilient, E resilient: spare bits ok
• S not resilient, E not resilient: spare bits ok

This doesn't come up with extra inhabitants, because those are tracked dynamically.

[tbkka]

One difficulty you might run into is that this depends on resilience boundaries.

Thanks for pointing that out. If necessary, we can try adding some resilience markers to the type and/or reflection metadata (which should be cheaper than putting the entire spare bit mask in the metadata).

[tbkka]

@swift-ci Please test

[tbkka]

@slavapestov Here's a related question: Do we ever under-utilize an added tag field? That is, if I have 100 cases will the compiler ever choose to add a tag byte and then only partially use the tag byte by depending on spare bits as well? It seems reasonable that once you've paid the memory cost of adding a tag byte/word/etc then you would take advantage of it to reduce code size and improve performance. If so, then RemoteInspection can assume tag-only semantics whenever it sees the compiler has added a suitably-large tag. Which would in turn partially address your concern above.

[tbkka]

Answer: Yes, we do under-utilize the tag field. We always prefer spare bits and spill over to the tag field only as many bits as needed.

So eventually, we'll need to find a bit somewhere to stash whether or not a particular enum uses spare bits at all. But we should still be able to calculate the actual mask directly in RemoteMirror. We can still rip out my old experiment to store the full compiler-computed spare bit mask in the reflection data, which would be good.

[tbkka]

@swift-ci Please test

[kastiglione]

This works for the use case I have run into:

(lldb) p error
(DecodingError) error = keyNotFound {
  keyNotFound = {
    0 = num
    1 = {
      codingPath = 0 values {}
      debugDescription = "No value associated with key CodingKeys(stringValue: \"num\", intValue: nil) (\"num\")."
      underlyingError = nil
    }
  }
}

[tbkka]

@kastiglione I'll doubtless need your help to test this further and identify cases that are still unsupported. @slavapestov pointed out one big hole above that will impact certain resilient types. Otherwise, I believe that it should be pretty routine to address any missing cases in this code.

[tbkka]

One difficulty you might run into is that this depends on resilience boundaries.

A-ha! Fortunately, such cases don't emit FixedTypeDescriptors, and we already have logic to map any MPE that lacks a FixedTypeDescriptor to the tag-only layout strategy. So we actually don't need any new logic to handle that correctly. (I do now have a test case, though. I'll certainly add that.)