Middle-end support for local allocs

[stedolan]

This PR adds support for local allocations to the middle-end, following on from #478.

The main change is a change to Lambda to introduce the following features, as well as the corresponding middle-end changes to compile them:

• Allocation modes (Alloc_heap / Alloc_local) on allocating primitives
• Region boundaries (Lregion e) delimiting lifetime of local allocs
• Tail call markers (Apply_tail) on applications

The last of these is the trickiest: tail calls from within a region should end the region after the arguments to the tail call have been evaluated, but before control is transferred.

I'm afraid that the middle-end changes are large and complicated. However, the complicated bit (handling overapplications and tail calls of local closures) is not large, and the large bit (plumbing alloc modes through every single allocating primitive) is not complicated. (There are detailed tests, but not in this PR as they depend also on frontend support)

This PR only adds support for Closure and Flambda1. PR #490 has a draft by @mshinwell of Flambda2 support, but in this PR there is only stub support in Flambda2.

The remaining changes to ocaml/ are the minimal ones required for CI. Full support for local allocs in the ocaml/ subtree will appear later.

(As with #478, this code is extracted from the local allocs branch, whose dev history and previous PRs are here)

[xclerc]

(For the files I own.)

[mshinwell]

Apart from the backend changes I'll review the whole of this PR.

[gretay-js]

My review is for backend changes only.

Question: mode replaces a place-holder Alloc_heap throughout cmmgen.ml in this PR, except a couple of places related to unboxed numbers: is_unboxed_number_cmm and transl_let. Is it intentional?

[mshinwell]

My review is for backend changes only.

Question: mode replaces a place-holder Alloc_heap throughout cmmgen.ml in this PR, except a couple of places related to unboxed numbers: is_unboxed_number_cmm and transl_let. Is it intentional?

I'll look at this, thanks for spotting that.

[stedolan]

My review is for backend changes only.

Question: mode replaces a place-holder Alloc_heap throughout cmmgen.ml in this PR, except a couple of places related to unboxed numbers: is_unboxed_number_cmm and transl_let. Is it intentional?

Thanks for pointing this out, I'll have a proper look on Monday. The Cmm number unboxing is tricky, and there might well be something wrong here.

But as a quick note: it is always safe to allocate a float or boxed integer as Alloc_heap, although it is not always optimal. (The invariant that must be upheld is that there may never be heap -> local pointers. Floats and boxed integers never contain pointers, so making them heap can never break this invariant).

There are a couple of places where Alloc_heap is unconditionally used, including:

• C externals that return unboxed floats when used in a position requiring a boxed float cause an allocation, which is always Alloc_heap (even if the context would allow a local one)
• 32-bit fallback implementations of 64-bit primitives use Alloc_heap for the boxed 64-bit values
• Accesses to flat float arrays which box the result generate an Alloc_heap box

(All of these could be made to use local allocations with even more propagation of alloc_mode through the compiler, this is left for a future patch)

[mshinwell]

My review is for backend changes only.
Question: mode replaces a place-holder Alloc_heap throughout cmmgen.ml in this PR, except a couple of places related to unboxed numbers: is_unboxed_number_cmm and transl_let. Is it intentional?

Thanks for pointing this out, I'll have a proper look on Monday. The Cmm number unboxing is tricky, and there might well be something wrong here.

I also think there might be something wrong, but expect to fix it today. (update: we need to discuss on Monday!)

[stedolan]

Question: mode replaces a place-holder Alloc_heap throughout cmmgen.ml in this PR, except a couple of places related to unboxed numbers: is_unboxed_number_cmm and transl_let. Is it intentional?

I've just had a look:

• is_unboxed_number_cmm uses Alloc_heap when unboxing a constant (found via Cconst_symbol). This is correct: constants count as "heap" because there are no restrictions on where they may escape, as they are never deallocated.

• transl_let uses Alloc_heap when unboxing a mutable float or boxed integer. This is correct but somewhat conservative: if the resulting value is used in a context where boxing is required, then the box will be Alloc_heap even if an Alloc_local is allowed. For instance:

  let f (g : local_ float -> int) =
    let x = local_ ref 0. in
    for i = 1 to 10 do x := !x +. 1. done;
    let r = g !x in
    r

  Here, x is unboxed, mutable and local, and a boxing operation is re-introduced in the call to g !x. Because of the conservative use of Alloc_heap, this box will be Alloc_heap even though Alloc_local would be OK. (It should be possible to improve this later by tracking more information about allocation mode, but this seems OK for now)

[mshinwell]

This is ok to merge once CI passes.