Fix layouts in flambda1 partial application simplification #1549
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
(I'm not confident this is correct, and if it is other problems remain.)This addresses an flambda1-specific failure I'm seeing over in #1528, which adds unboxed floats to the front-end. In that PR, this program trips an assert in
Inline_and_simplify.simplify_partial_application
:The assert tripped is:
This assert is surprising to me. As far as I can tell,
result_layout
is the result of the application. Since we're looking at a partial application, the result type is a function and the result layout will always be value. On the other hand, thereturn_layout
of the function being partially applied may not be value.This PR attempts to fix this by dropping the assert and correcting a layout in the resulting function. I can't test it here since there are no other layouts allowed by the front-end. I have tested it in #1528, and it seems to work.
But I'm also seeing other flambda1 problems that I'm less sure how to fix, so I don't feel confident - I'll write a slack message about those.Edit: The things that worried me in the original message did turn out to be a distinct issue, now addressed in #1557