Skip to content

Expand the work from #73491 to support more MPE layouts. #74145

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Jun 6, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 10 additions & 1 deletion include/swift/RemoteInspection/BitMask.h
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,16 @@ class BitMask {

BitMask(unsigned sizeInBytes, uint64_t sourceMask): size(sizeInBytes) {
mask = (uint8_t *)calloc(1, sizeInBytes);
memcpy(mask, &sourceMask, sizeInBytes);
if (!mask) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is just here to further harden the bit mask code against allocation failures. Allocation failures can occur in this code when clients try to use RemoteMirror to decipher type data from random chunks of memory.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we also have a limit on sizeInBytes so we don't read garbage and then successfully, but unhappily, allocate 500TB of memory?

assert(false && "Failed to allocate Bitmask");
size = 0;
return;
}
size_t toCopy = sizeInBytes;
if (toCopy > sizeof(sourceMask)) {
toCopy = sizeof(sourceMask);
}
memcpy(mask, &sourceMask, toCopy);
}

// Construct a bitmask of the appropriate number of bytes
Expand Down
68 changes: 26 additions & 42 deletions stdlib/public/RemoteInspection/TypeLowering.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -315,6 +315,7 @@ BitMask BuiltinTypeInfo::getSpareBits(TypeConverter &TC, bool &hasAddrOnly) cons
mask.keepOnlyMostSignificantBits(getSize() * 8 - intSize);
return mask;
} else if (
Name == "ypXp" || // Any.Type
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are doubtless other pointer types that need to expose spare bits. I expect some further iteration of this specific test here.

Name == "yyXf" // 'yyXf' = @thin () -> Void function
) {
// Builtin types that expose pointer spare bits
Expand Down Expand Up @@ -578,7 +579,9 @@ class NoPayloadEnumTypeInfo: public EnumTypeInfo {
}

BitMask getSpareBits(TypeConverter &TC, bool &hasAddrOnly) const override {
return BitMask::zeroMask(getSize());
auto mask = BitMask(getSize(), maskForCount(getNumCases()));
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug fix: Non-payload enums do in fact export spare bits from the tag value.

mask.complement();
return mask;
}

bool projectEnumValue(remote::MemoryReader &reader,
Expand Down Expand Up @@ -648,7 +651,17 @@ class SinglePayloadEnumTypeInfo: public EnumTypeInfo {
}

BitMask getSpareBits(TypeConverter &TC, bool &hasAddrOnly) const override {
return BitMask::zeroMask(getSize());
FieldInfo PayloadCase = getCases()[0];
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Single payload enums also export spare bits from the tag. This is a bit more complex than the non-payload case above just because of the manipulations needed to ensure we have a tag and to ensure we don't inadvertently try to export spare bits from the payload.

size_t payloadSize = PayloadCase.TI.getSize();
if (getSize() <= payloadSize) {
return BitMask::zeroMask(getSize());
}
size_t tagSize = getSize() - payloadSize;
auto mask = BitMask::oneMask(getSize());
mask.keepOnlyMostSignificantBits(tagSize * 8); // Clear payload bits
auto tagMaskUsedBits = BitMask(getSize(), maskForCount(getNumCases()));
mask.andNotMask(tagMaskUsedBits, payloadSize); // Clear used tag bits
return mask;
}

// Think of a single-payload enum as being encoded in "pages".
Expand Down Expand Up @@ -2046,11 +2059,10 @@ class EnumTypeInfoBuilder {
//

// Do we have a fixed layout?
// TODO: Test whether a missing FixedDescriptor is actually relevant.
auto FixedDescriptor = TC.getBuilder().getBuiltinTypeDescriptor(TR);
if (!FixedDescriptor || GenericPayloadCases > 0) {
// This is a "dynamic multi-payload enum". For example,
// this occurs with:
// this occurs with generics such as:
// ```
// class ClassWithEnum<T> {
// enum E {
Expand All @@ -2060,6 +2072,12 @@ class EnumTypeInfoBuilder {
// var e: E?
// }
// ```
// and when we have a resilient inner enum, such as:
// ```
// enum E2 {
// case y(E1_resilient)
// case z(Int)
// }
auto tagCounts = getEnumTagCounts(Size, EffectiveNoPayloadCases,
EffectivePayloadCases);
Size += tagCounts.numTagBytes;
Expand Down Expand Up @@ -2091,7 +2109,6 @@ class EnumTypeInfoBuilder {
unsigned Stride = ((Size + Alignment - 1) & ~(Alignment - 1));
if (Stride == 0)
Stride = 1;
auto PayloadSize = EnumTypeInfo::getPayloadSizeForCases(Cases);

// Compute the spare bit mask and determine if we have any address-only fields
auto localSpareBitMask = BitMask::oneMask(Size);
Expand All @@ -2103,44 +2120,11 @@ class EnumTypeInfoBuilder {
}
}

// See if we have MPE bit mask information from the compiler...
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Previously, I tried having the compiler export a "multi-payload enum descriptor" with a complete copy of the spare bit mask. I was never happy with this approach because it potentially adds a lot of bulk to the reflection metadata. Fortunately, the new code has shown we shouldn't need such information from the compiler and this PR rips all of that logic out of RemoteMirror. I'll file a separate PR to remove the never-completely-implemented compiler support for generating this data.

// TODO: drop this?

// Uncomment the following line to dump the MPE section every time we come through here...
//TC.getBuilder().dumpMultiPayloadEnumSection(std::cerr); // DEBUG helper

auto MPEDescriptor = TC.getBuilder().getMultiPayloadEnumDescriptor(TR);
if (MPEDescriptor && MPEDescriptor->usesPayloadSpareBits()) {
// We found compiler-provided spare bit data...
auto PayloadSpareBitMaskByteCount = MPEDescriptor->getPayloadSpareBitMaskByteCount();
auto PayloadSpareBitMaskByteOffset = MPEDescriptor->getPayloadSpareBitMaskByteOffset();
auto SpareBitMask = MPEDescriptor->getPayloadSpareBits();
BitMask compilerSpareBitMask(PayloadSize, SpareBitMask,
PayloadSpareBitMaskByteCount, PayloadSpareBitMaskByteOffset);

if (compilerSpareBitMask.isZero() || hasAddrOnly) {
// If there are no spare bits, use the "simple" tag-only implementation.
return TC.makeTypeInfo<TaggedMultiPayloadEnumTypeInfo>(
Size, Alignment, Stride, NumExtraInhabitants,
BitwiseTakable, Cases, EffectivePayloadCases);
}

#if 0 // TODO: This should be !defined(NDEBUG)
// Verify that compiler provided and local spare bit info agree...
// TODO: If we could make this actually work, then we wouldn't need the
// bulky compiler-provided info, would we?
assert(localSpareBitMask == compilerSpareBitMask);
#endif

// Use compiler-provided spare bit information
return TC.makeTypeInfo<MultiPayloadEnumTypeInfo>(
Size, Alignment, Stride, NumExtraInhabitants,
BitwiseTakable, Cases, compilerSpareBitMask,
EffectivePayloadCases);
}

if (localSpareBitMask.isZero() || hasAddrOnly) {
// Simple case that does not use spare bits
// Simple tag-only layout does not use spare bits.
// Either:
// * There are no spare bits, or
// * We can't copy it to strip spare bits.
return TC.makeTypeInfo<TaggedMultiPayloadEnumTypeInfo>(
Size, Alignment, Stride, NumExtraInhabitants,
BitwiseTakable, Cases, EffectivePayloadCases);
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@

public enum E1_resilient {
case a
case b
}

public enum E2_resilient {
case c(E1_resilient)
case d(E1_resilient)
}
74 changes: 74 additions & 0 deletions validation-test/Reflection/reflect_Enum_values_resilient.swift
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
// RUN: %empty-directory(%t)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New test case builds a separate resilient library and verifies that we can project values from enums defined in that library.


// RUN: %target-build-swift-dylib(%t/%target-library-name(resilient_enums)) -enable-library-evolution %S/Inputs/reflect_Enum_values_resilient_enums.swift -emit-module -emit-module-path %t/resilient_enums.swiftmodule -module-name resilient_enums
// RUN: %target-codesign %t/%target-library-name(resilient_enums)

// RUN: %target-build-swift -lswiftSwiftReflectionTest %s -L %t -I %t -lresilient_enums -o %t/reflect_Enum_values_resilient %target-rpath(%t)
// RUN: %target-codesign %t/reflect_Enum_values_resilient

// RUN: %target-run %target-swift-reflection-test %t/reflect_Enum_values_resilient | tee /dev/stderr | %FileCheck %s --dump-input=fail

// REQUIRES: executable_test
// UNSUPPORTED: use_os_stdlib

import SwiftReflectionTest

import resilient_enums

// Non-resilient enum wrapping a resilient enum
// This doesn't use spare bits of the inner enum
enum E2 {
case y(E1_resilient)
case z(E1_resilient)
}

// Contrast:
// E2_resilient is a resilient enum wrapping a resilient enum
// This does use spare bits of the inner enum


// CHECK: Reflecting an enum value.
// CHECK-NEXT: Type reference:
// CHECK-NEXT: (enum reflect_Enum_values_resilient.E2)
// CHECK-NEXT: Value: .y(.a)

reflect(enumValue: E2.y(.a))

// CHECK: Reflecting an enum value.
// CHECK-NEXT: Type reference:
// CHECK-NEXT: (enum reflect_Enum_values_resilient.E2)
// CHECK-NEXT: Value: .z(.b)

reflect(enumValue: E2.z(.b))

// CHECK: Reflecting an enum value.
// CHECK-NEXT: Type reference:
// CHECK-NEXT: (enum resilient_enums.E1_resilient)
// CHECK-NEXT: Value: .a

reflect(enumValue: E1_resilient.a)

// CHECK: Reflecting an enum value.
// CHECK-NEXT: Type reference:
// CHECK-NEXT: (enum resilient_enums.E1_resilient)
// CHECK-NEXT: Value: .b

reflect(enumValue: E1_resilient.b)

// CHECK: Reflecting an enum value.
// CHECK-NEXT: Type reference:
// CHECK-NEXT: (enum resilient_enums.E2_resilient)
// CHECK-NEXT: Value: .c(.a)

reflect(enumValue: E2_resilient.c(.a))

// CHECK: Reflecting an enum value.
// CHECK-NEXT: Type reference:
// CHECK-NEXT: (enum resilient_enums.E2_resilient)
// CHECK-NEXT: Value: .d(.b)

reflect(enumValue: E2_resilient.d(.b))

doneReflecting()

// CHECK: Done.