Root scanning fixes for flambda2 #57

mshinwell · 2021-06-24T17:19:32Z

Originally written by @poechsel in conjunction with @mshinwell

We may want to tidy this up a bit, but it is believed to be correct. It supports two things which Flambda 2 needs:

registration of closures as static GC roots
the ability for such closures to be not the first in the corresponding OCaml value (only one closure per set may be registered because of limitations in the compactor; this allows Flambda 2 to choose a closure that might be one of the Infix_tag ones, which makes things easier).

git-subtree-dir: ocaml git-subtree-split: 1ed86d502f3e635fa238652dbb72ae61e2e2969f

…ac3c)

…d81d)

…74322eefd9af8)

…b6f1a67704cdec) Change the default policy to best-fit and space overhead to 100

…7d32d251a903bde44fbd8a)

…4603a9105ebc7f0fc5e225a5c1)

…-pick 388524f451aea03051c90cc9f66857d4333a249d) Add named text sections for caml_system__code_begin/end symbols

Ensure that string representations of scopes are shared.

Diff from 9ef80263c7d8ee789f93ae9f81584b70e16743bb to 897c24c1380eaebf78fa615b1699fba1716621b4

…o pattern match of mutable field in curried functions (#9751) Introduce new warning 68 (cherry picked from commit 49aa87c)

…s/unix/

…bs/systhreads/, toplevel/toploop.ml

…for systhreads/

…ops files (still needs .gitignore + way of generating these files)

* Add intrinsics for ext_pointer and native_pointer * Add two_args * remove XCR * Add intrinisics for storing and loading int64,int32,nativeint Fix up names * Remove "unsafe" from the name of one of the intrinsics * Address review comments Use Word_int instead of Word_val for {load,store}_immediate intrinsics Co-authored-by: Mark Shinwell <[email protected]>

Reduces peak memory usage, which is important when a build system tries to run many large links in parallel.

* Import the code of ocamlcfg. * Fix imported code. * Optionally go through ocamlcfg. * Merge in the latest CFG version * Fix compilation warning 18. * Add copyright header to cfg files. The head is a copy of the LICENSE file at the root of ocamlcfg library Co-authored-by: xclerc <[email protected]>

Fix error "integer literal exceeds the range of representable integers of type nativeint" in the code that runs only on 64-bit targets, but need to compile on 32-bit target.

Some `Long_val` conversions were missing. This was setting the wrong values for the `custom_` parameters, causing the major GC to work too much, slowing down some programs. Add regression test. Fixes: #9326 Co-authored-by: Xavier Leroy <[email protected]>

Print unary operations in mach

…ded to compiler libs (#36)

Allow the deletion of blocks with exceptional successors.

Remove the `fun_tailrec_entry_point_label` field from `Cfg.t`, and explicitly store the destination label in the `Self` constructor of `Cfg_intf.tail_call_operation`.

* Add missing case for Ispecific rdtsc in an exhaustive match * Remove Cmm.memory_chunk.Double_u All current targets treat Double_u the same as Double. * Update comment for Double * Remove Cmm.memory_chunk.Double_u All current targets treat Double_u the same as Double. * Update comment for Double * Fixup after a rebase

mshinwell · 2021-06-24T17:26:33Z

I forgot, this needs 4.12, so converted to draft for the moment.

1924269795 Several fixes for partial application and currying 4fee6ae2e8 Pprintast support for new local syntax 8df43e93e5 Quieten Makefile when runtime dep files are not present 88ec84e29e Typecheck x |> f y as (f y x), not ((f y) x) 87a10e3348 Remove autogeneration of @ocaml.curry c656dc9 Merge flambda-backend changes 11b5424 Avoid printing double spaces in function argument lists 7751faa Restore locations to Typedtree.{pat,let}_bound_idents_full e450b6c add build_ocaml_compiler.sexp 0403bb3 Revert PR 9895 to continue installing VERSION b3447db Ensure new local attributes are namespaced properly 7f213fc Allow empty functions again 8f22ad8 Bugfix: ensure local domain state is initialised 80f54dd Bugfix for Selectgen with regions e8133a1 Fix external-external signature inclusion 9840051 Bootstrap d879f23 Merge remote-tracking branch 'jane/local-reviewed' into local-merge 94454f5 Use Local_store for the local allocations ref 54a164c Create fewer regions, according to typechecking (oxcaml#59) 1c2479b Merge flambda-backend changes ce34678 Fix printing of modes in return types 91f2281 Hook mode variable solving into Btype.snapshot/backtrack 54e4b09 Move Alloc_mode and Value_mode to Btype ff4611e Merge flambda-backend changes ce62e45 Ensure allocations are initialised, even dead ones 6b6ec5a Fix the alloc.ml test on 32-bit builds 81e9879 Merge flambda-backend changes 40a7f89 Update repo URL for ocaml-jst, and rename script. 0454ee7 Add some new locally-allocating primitives (oxcaml#57) 8acdda1 Reset the local stack pointer in exception handlers (oxcaml#56) 8dafa98 Improve typing for (||) and (&&) (oxcaml#55) 8c64754 Fix make_check_all_arches (oxcaml#54) b50cd45 Allow arguments to primitives to be local even in tail position (oxcaml#53) cad125d Fix modes from or-patterns (oxcaml#50) 4efdb72 Fix tailcalls tests with inlining (oxcaml#52) 4a795cb Flambda support (oxcaml#49) 74722cb Add [@ocaml.principal] and [@ocaml.noprincipal] attributes, and use in oo.mli 6d7d3b8 Ensure that functions are evaluated after their arguments (flambda-backend oxcaml#353) 89bda6b Keep Sys.opaque_identity in Cmm and Mach (port upstream PR 9412) a39126a Fix tailcalls within regions (oxcaml#48) 4ac4cfd Fix stdlib manpages build 3a95f5e Merge flambda-backend changes efe80c9 Add jane/pull-flambda-patches script fca94c4 Register allocations for Omitted parameter closures (oxcaml#47) 103b139 Remove various FIXMEs (oxcaml#46) 62ba2c1 Bootstrap a0062ad Allow local allocations for various primitives (oxcaml#43) 7a2165e Allow primitives to be poly-moded (oxcaml#43) 2af3f55 Fix a flaky test by refactoring TypePairs (ocaml/ocaml#10638) 58dd807 Bootstrap ee3be10 Fix modes in build_apply for partial applications fe73656 Tweak for evaluation order of labelled partial applications (#10653) 0527570 Fix caml_modify on local allocations (oxcaml#40) e657e99 Relax modes for `as` patterns (oxcaml#42) f815bf2 Add special mode handling for tuples in matches and let bindings (oxcaml#38) 39f1211 Only take the upper bounds of modes associated with allocations (oxcaml#37) aec6fde Interpret arrow types in "local positions" differently c4f3319 Bootstrap ff6fdad Add some missing regions 40d586d Bootstrap 66d8110 Switch to a system with 3 modes for values f2c5a85 Bugfix for Comballoc with local allocations. (oxcaml#41) 83bcd09 Fix bug with root scanning during compaction (oxcaml#39) 1b5ec83 Track modes in Lambda.lfunction and onwards (oxcaml#33) f1e2e97 Port ocaml/ocaml#10728 56703cd Port ocaml/ocaml#10081 eb66785 Support local allocations in i386 and fix amd64 bug (oxcaml#31) c936b19 Disallow local recursive non-functions (oxcaml#30) c7a193a GC support for local allocations (oxcaml#29) 8dd7270 Nonlocal fields (oxcaml#28) e19a2f0 Bootstrap 694b9ac Add syntax to the parser for local allocations (oxcaml#26) f183008 Lower initial stack size 918226f Allow local closure allocations (oxcaml#27) 2552e7d Introduce mode variables (oxcaml#25) bc41c99 Minor fixes for local allocations (oxcaml#24) a2a4e60 Runtime and compiler support for more local allocations (oxcaml#23) d030554 Typechecking for local allocations (oxcaml#21) 9ee2332 Bugfix missing from oxcaml#20 02c4cef Retain block-structured local regions until Mach. 86dbe1c amd64: Move stack realloc calls out-of-line 324d218 More typing modes and locking of environments a4080b8 Initial version of local allocation (unsafe) git-subtree-dir: ocaml git-subtree-split: 1924269795db2450be5c084f7799340e0e003e19

173842c Merge flambda-backend changes ed7eba2 Remove leading space from LINE. (#484) bd61170 Bump magic numbers (#5) c50c47d Add CI builds with local allocations enabled 1412792 Move local allocations support behind '-extension local' 6d8e42a Better tail call behaviour in caml_applyN c7dac3d Typemod: toplevel bindings escape even if no variables are bound 82d6c3e Several fixes for partial application and currying d05c70c Pprintast support for new local syntax e0e62fc Typecheck x |> f y as (f y x), not ((f y) x) d7e34ce Remove autogeneration of @ocaml.curry b9a0593 Port #493 0a872d9 Code review fixes from #491 6c168bb Remove local allocation counting 3c6e7f0 Code review fixes from #478 bb97207 Rename Lambda.apply_position a7cb650 Quieten Makefile when runtime dep files are not present c656dc9 Merge flambda-backend changes 11b5424 Avoid printing double spaces in function argument lists 7751faa Restore locations to Typedtree.{pat,let}_bound_idents_full e450b6c add build_ocaml_compiler.sexp 0403bb3 Revert PR 9895 to continue installing VERSION b3447db Ensure new local attributes are namespaced properly 7f213fc Allow empty functions again 8f22ad8 Bugfix: ensure local domain state is initialised 80f54dd Bugfix for Selectgen with regions e8133a1 Fix external-external signature inclusion 9840051 Bootstrap d879f23 Merge remote-tracking branch 'jane/local-reviewed' into local-merge 94454f5 Use Local_store for the local allocations ref 54a164c Create fewer regions, according to typechecking (#59) 1c2479b Merge flambda-backend changes ce34678 Fix printing of modes in return types 91f2281 Hook mode variable solving into Btype.snapshot/backtrack 54e4b09 Move Alloc_mode and Value_mode to Btype ff4611e Merge flambda-backend changes ce62e45 Ensure allocations are initialised, even dead ones 6b6ec5a Fix the alloc.ml test on 32-bit builds 81e9879 Merge flambda-backend changes 40a7f89 Update repo URL for ocaml-jst, and rename script. 0454ee7 Add some new locally-allocating primitives (#57) 8acdda1 Reset the local stack pointer in exception handlers (#56) 8dafa98 Improve typing for (||) and (&&) (#55) 8c64754 Fix make_check_all_arches (#54) b50cd45 Allow arguments to primitives to be local even in tail position (#53) cad125d Fix modes from or-patterns (#50) 4efdb72 Fix tailcalls tests with inlining (#52) 4a795cb Flambda support (#49) 74722cb Add [@ocaml.principal] and [@ocaml.noprincipal] attributes, and use in oo.mli 6d7d3b8 Ensure that functions are evaluated after their arguments (flambda-backend #353) 89bda6b Keep Sys.opaque_identity in Cmm and Mach (port upstream PR 9412) a39126a Fix tailcalls within regions (#48) 4ac4cfd Fix stdlib manpages build 3a95f5e Merge flambda-backend changes efe80c9 Add jane/pull-flambda-patches script fca94c4 Register allocations for Omitted parameter closures (#47) 103b139 Remove various FIXMEs (#46) 62ba2c1 Bootstrap a0062ad Allow local allocations for various primitives (#43) 7a2165e Allow primitives to be poly-moded (#43) 2af3f55 Fix a flaky test by refactoring TypePairs (ocaml/ocaml#10638) 58dd807 Bootstrap ee3be10 Fix modes in build_apply for partial applications fe73656 Tweak for evaluation order of labelled partial applications (#10653) 0527570 Fix caml_modify on local allocations (#40) e657e99 Relax modes for `as` patterns (#42) f815bf2 Add special mode handling for tuples in matches and let bindings (#38) 39f1211 Only take the upper bounds of modes associated with allocations (#37) aec6fde Interpret arrow types in "local positions" differently c4f3319 Bootstrap ff6fdad Add some missing regions 40d586d Bootstrap 66d8110 Switch to a system with 3 modes for values f2c5a85 Bugfix for Comballoc with local allocations. (#41) 83bcd09 Fix bug with root scanning during compaction (#39) 1b5ec83 Track modes in Lambda.lfunction and onwards (#33) f1e2e97 Port ocaml/ocaml#10728 56703cd Port ocaml/ocaml#10081 eb66785 Support local allocations in i386 and fix amd64 bug (#31) c936b19 Disallow local recursive non-functions (#30) c7a193a GC support for local allocations (#29) 8dd7270 Nonlocal fields (#28) e19a2f0 Bootstrap 694b9ac Add syntax to the parser for local allocations (#26) f183008 Lower initial stack size 918226f Allow local closure allocations (#27) 2552e7d Introduce mode variables (#25) bc41c99 Minor fixes for local allocations (#24) a2a4e60 Runtime and compiler support for more local allocations (#23) d030554 Typechecking for local allocations (#21) 9ee2332 Bugfix missing from #20 02c4cef Retain block-structured local regions until Mach. 86dbe1c amd64: Move stack realloc calls out-of-line 324d218 More typing modes and locking of environments a4080b8 Initial version of local allocation (unsafe) git-subtree-dir: ocaml git-subtree-split: 173842c

mshinwell and others added 30 commits November 12, 2020 12:34

Initial commit

f1a4086

Squashed 'ocaml/' content from commit 1ed86d502

3535bae

git-subtree-dir: ocaml git-subtree-split: 1ed86d502f3e635fa238652dbb72ae61e2e2969f

Merge commit '3535bae4f6c538eb8054da4520d04c0375fbcd33' as 'ocaml'

53e3a8d

permanent/js-config (cherry-pick 288028727b68c7e6c30587727ce888b75f65…

0f58a04

…ac3c)

permanent/gc-tuning (cherry-pick e9d6d2f145438dd6a82b56e9b41c8a01e752…

cbc3099

…d81d)

permanent/reraise-by-default (cherry-pick 50e94902ca6bb84c33982db858b…

ca65a11

…74322eefd9af8)

permanent/default-to-best-fit (cherry-pick 64240fd716a9d0db57d779ebe5…

bbaaad7

…b6f1a67704cdec) Change the default policy to best-fit and space overhead to 100

ocaml#8939 from gretay-js/save-linear (cherry-pick 1200de97288b212e80…

32c53a2

…7d32d251a903bde44fbd8a)

ocaml#9003 from gretay-js/start-from-emit (cherry-pick 21d5d26f27ed45…

bef6029

…4603a9105ebc7f0fc5e225a5c1)

ocaml#9631 from gretay-js/named_sections_for_caml_system_code (cherry…

6980153

…-pick 388524f451aea03051c90cc9f66857d4333a249d) Add named text sections for caml_system__code_begin/end symbols

ocaml#9748 from xclerc/share-ev_defname (cherry-pick 3e937fcb562)

ad0f6aa

Ensure that string representations of scopes are shared.

Tracing probes (4.11)

c6c1936

Diff from 9ef80263c7d8ee789f93ae9f81584b70e16743bb to 897c24c1380eaebf78fa615b1699fba1716621b4

ocaml#9751 Introduce warning 68 to warn about hidden allocation due t…

5430bb7

…o pattern match of mutable field in curried functions (#9751) Introduce new warning 68 (cherry picked from commit 49aa87c)

fixup f3971e492ac98d8e8e0cb9844c74e74777717ee2

a7fca54

Fix VERSION: this compiler is 4.11.1

3c3c22f

Already upstreamed: stdlib capitalisation patch

6754325

To upstream: remove Compflags, add attributes, etc.

c75517d

To upstream: use nolabels attribute instead of -nolabels for otherlib…

d76c78a

…s/unix/

To upstream: flambda_o3 attribute support (includes bootstrap)

7429336

To upstream: use flambda_o3 attribute in stdlib

4530adc

To upstream: stabilise filenames used in backtraces: stdlib/, otherli…

1d1d97d

…bs/systhreads/, toplevel/toploop.ml

To upstream: remove threadUnix

8fe015b

To upstream: fix dependency problem with Instruct

d0676b8

To upstream: fix to Ocaml_modifiers in ocamltest

656fd6c

To upstream: refactor otherlibs/dynlink/, removing byte/ and native/

835845c

To upstream: dune build fixes (depends on previous to-upstream patches)

ed96141

To upstream: use flambda_o3 attribute instead of -O3 in the Makefile …

e39274e

…for systhreads/

To upstream: stop using -O3 for dynlink/

fafcb6e

To upstream: stop using -O3 in otherlibs/Makefile.otherlibs.common

e364a82

To upstream: export-dynamic for otherlibs/dynlink/ via the natdynlink…

c97ea91

…ops files (still needs .gitignore + way of generating these files)

gretay-js and others added 22 commits May 17, 2021 17:47

Compact heap before calling C linker (#33)

7aec5bb

Reduces peak memory usage, which is important when a build system tries to run many large links in parallel.

Install cfg files in lib/ocaml for compiler-libs (#35)

9134cb4

Fix typo in Cpopcnt constructor in Proc.operation_supported (#37)

65d64c7

Replace tuple with record in Cextcall (targets other than amd64) (#38)

abc0a8c

Fix integer literal that is too big for 32-bit targets (#39)

eddfdc1

Fix error "integer literal exceeds the range of representable integers of type nativeint" in the code that runs only on 64-bit targets, but need to compile on 32-bit target.

Set CAML_LD_LIBRARY_PATH to load stublibs (#40)

e8b3b67

Update HACKING.md with how to add new intrinsics. (#41)

cedc37f

Fix printing of mach for intrinsics (#45)

12554de

Print unary operations in mach

Workaround for a GNU make 3.81 bug. (oxcaml#47)

0093d19

Make matches exhaustive (amd64) (#46)

f07447a

Add a little script that helps update dune file when a new file is ad…

aaee677

…ded to compiler libs (#36)

Put the result of rdtsc and rdpmc in rax instead of rdx

745cc26

Arbitrary result register for rdtsc and rdpmc.

13b0226

Delete "backend/cfg/ocamlcfg.ml{,i}".

5d56f06

Disconnect blocks with exceptional successors (oxcaml#49)

ab557b8

Allow the deletion of blocks with exceptional successors.

Change the way the label for tailrec calls is stored (oxcaml#51)

8ff9e54

Remove the `fun_tailrec_entry_point_label` field from `Cfg.t`, and explicitly store the destination label in the `Self` constructor of `Cfg_intf.tail_call_operation`.

Add missing case for Ispecific rdtsc in an exhaustive match (oxcaml#52)

b4c0256

Root scanning fixes for flambda2

301fb8c

mshinwell added the flambda2 Prerequisite for, or part of, flambda2 label Jun 24, 2021

mshinwell marked this pull request as draft June 24, 2021 17:26

mshinwell changed the base branch from 4.11 to 4.12 June 25, 2021 09:43

mshinwell closed this Jun 25, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Root scanning fixes for flambda2 #57

Root scanning fixes for flambda2 #57

Uh oh!

mshinwell commented Jun 24, 2021

Uh oh!

mshinwell commented Jun 24, 2021

Uh oh!

Uh oh!

Root scanning fixes for flambda2 #57

Root scanning fixes for flambda2 #57

Uh oh!

Conversation

mshinwell commented Jun 24, 2021

Uh oh!

mshinwell commented Jun 24, 2021

Uh oh!

Uh oh!