Compatibility with julia nightly #10
Conversation
This used to implicitly refer to a module-level name, but lowering is now expected to wrap it in a `globalref`. Part of JuliaLang/julia#54772
JuliaLang/julia#54773, JuliaLang/julia#56713, JuliaLang/julia#57470. Some changes omitted from `expand-decls` and `expand-assignment`.
JuliaLang/julia#56746. Also call :slot and :static_parameter valid (for now)
Part of JuliaLang/julia#56497. `insert-struct-shim` (for the a self-reference case) is omitted for now.
For method defs, `latestworld` is produced in desugaring rather than closure conversion for now (our closure conversion doesn't seem to cover the same cases as lisp lowering yet). Covers JuliaLang/julia#56523, JuliaLang/julia#56509, JuliaLang/julia#57299. Also includes changes from JuliaLang/julia#57102 (bpart: Start enforcing minimum world age for const bparts) and JuliaLang/julia#57150 (bpart: Start enforcing min_world for global variable definitions) since the lowering changes from those appear to be amendments to the changes above.
`Core._typebody!` now takes a new "prev" argument, which we don't use yet here. Changes from JuliaLang/julia#57253
Fix segfaulting test. Thanks for the TODO
Was missing increments after closure type declarations and a kwarg function declaration. Fixes the world age warnings when running tests. Part of JuliaLang/julia#56509
Signature changed in JuliaLang/julia#57230. Thanks @aviatesk for the help!
Differentiate it from AST K"const"
As of JuliaLang/julia#57765, `jl_module_public` is no longer exported. Change our runtime to handle it like `public` and `export` like we handle `import` or `using` for now
I believe this was a world age issue
Adapt to Core.Binding changes in JuliaLang/julia#54788. Also remove `is_defined_nothrow_global` (JuliaLang/julia#56746)
Part of JuliaLang/julia#56497 that I ignored before. I have doubts about how long this solution will stay in the base repository, and how complete it is (doesn't seem to work with M1.M2.S), but we are testing for it here. Also change the expected value of a test changed in the same PR.
Changes from JuliaLang/julia#57253 (bpart: Fully switch to partitioned semantics). This fixes one failing test and realigns struct desugaring to match lisp for now. Also changed: the expected result of redefining a primitive type (now allowed).
|
All tests (excluding IR equality tests) are passing, so I'm marking this as ready for review! |
mlechu
changed the title
Once the corresponding bindings are merged into julia, `@activate JuliaLowering`
will set both `Core._parse` and `Core._lower` so that parsing produces our
`SyntaxTree`s and we become the default lowerer.
The parser hook copied from JuliaSyntax is unfortunate, but I think moving this
hook there and keeping it in sync would be a bigger headache while
JuliaLowering is still under development.
Moved to [JuliaLowering](c42f/JuliaLowering.jl#10) instead (the data structure we need is defined there).
Moved to [JuliaLowering](c42f/JuliaLowering.jl#10) instead (the data structure we need is defined there).
|
This is pretty epic, thanks! Doing this port was going to be my first task back to get me back into the swing of things 😆 Regarding the TODOs - definitely best to do them later, except for the tests. We should keep the IR tests working. I'm keen on moving to the use of utility functions like I think we should have |
Sure, will update them
Already done (albeit buried), see aa32c09. I didn't do this at first because I thought both const forms were the same 🙃. Note that this form is probably getting eliminated once JuliaLang/julia#58187 is merged. |
Amazing, thanks! I did see this as soon as I started to do an in depth review. For the tests, you can use the utility function in test/util.jl In addition to existing tests, please add new tests if there's new features to cover. I imagine Can you explain the |
|
Generally, I'll note that I'm taking the occasional liberty with compatibility when I really feel some obscure feature is poorly motivated in modern Julia. Especially when it's an "emergent feature" of the implementation, rather than something done purposefully. There's not many such cases and it's still always a really hard call, but I feel a brand new implementation does provide a little extra freedom here, especially when the code complexity of implementing a mis-feature is high! |
Will do. Once we can use JuliaLowering in core JuliaLang/julia#58207, we can also try using some of the lowering tests from the julia repository without huge manual porting effort.
I'm pretty happy to delete |
As far as I'm aware, all this plumbing was just to support `const a,b,c = 1,2,3`
having `b` and `c` inherit the `const`.
Fix `let; const x = 1; end`, which should throw. Moves the check up to scope
analysis (lisp has it in `compile`).
Too many to count. Also fix `latestworld` printing.
|
OK, this is ready for more review. |
This PR along with [the associated JuliaLowering PR](c42f/JuliaLowering.jl#10) allow us to experiment with JuliaLowering as the core lowering implementation, mainly for JETLS development at the moment (cc @aviatesk, @c42f). ## Changes JuliaLowering works with `JuliaLowering.SyntaxTree` input instead of Expr. This change allows`SyntaxTree`s out of parsing and into lowering, hopefully without disturbing existing code expecting Expr. The current design wraps `x::SyntaxTree` like `Expr(:syntaxtree, x)` because otherwise `toplevel_eval_flex` wouldn't know what `x` is, but once we're past the experimental phase, it would only make sense for the compiler to actually know about the types it's operating on. ### New: `Core._lower` It (a julia entrypoint), `jl_lower` (C entrypoint), `jl_fl_lower`, and `fl_lower` all mirror the existing parsing API which was added in #35243. <details> <summary> Click for pre-PR call graph </summary>  </details> After this change:  Like `Core._parse`, `Core._lower` returns an `svec` with the first element as the expected result, and the remaining elements can hold extra information like the lowered SyntaxTree with provenance info (I've left the specification of extra return values for later, when JuliaLowering is more stable and JETLS knows what it needs, because we may opt to pack provenance into the lowered CodeInfo object instead). Parsing only uses the svec to return an offset, but it could actually make use of the the flexibility to return diagnostics in the future. It's unfortunately not clear what types we can expect to go into and come out of lowering (I got Expr, LineNumberNode, String, Symbol, and Nothing in testing.) ### Remove `jl_lower_expr_mod` I realize this isn't being called from C, and Core._lower is a more sensible entry point from julia. We could probably also remove some uncalled parse and jl_load functions. ## Limitations These belong to JuliaLowering, but are listed here as known issues for anyone who wants to try it out. - `(module ...)` forms don't go through JuliaLowering yet. Modules are lowered to a call to `JuliaLowering.eval_module(parentmod, modname, syntaxtree)`, which is currently a stub function that converts to Expr for lowering and evaluation, which JuliaLowering can't use. - Macros work differently. Defining and calling macros should work, but invoking normal macros defined pre-JuliaLowering will not. - Compilation is currently slow (~30 seconds for me).
|
A couple of opaque closure and generated function tests are re-broken on nightly, lol. I will take a look. Edit: Generated function IR tests just need to stay in sync with JuliaSyntax changes (I'm using JuliaLang/JuliaSyntax.jl#552, which hasn't been released). Opaque closures hit assertion failures in |
|
If this gets us closer to master can we merge this and then work on the remaining points after? I'd like to start playing with this so that we can have a concrete plan for 1.13. |
The goal of this change is to have JuliaLowering work on julia nightly again,
particularly for the JETLS development that's going on, by porting recent
changes from julia-syntax.scm. It will be an unfortunately large diff, but a
smaller PR would leave JuliaLowering broken for all versions of julia, so I've
settled on keeping the commit history readable.
Major changes
globalreflatestworld)globaldeclglobalform is still emitted from both lowering implementations, but may not be in the futureconstIR form that takes two arguments like=, not to be confused with the one-argumentconstAST form.constsemantics changed to implicitlyglobalin 1.12-DEV JuliaLang/julia#56613 (comment)isdefinedglobalbuiltin::GeneratedFunctionStubandCore._typebody!@activate JuliaLoweringTODO
These can be done in this PR or a later one.
_defaultctors? (internals: add _defaultctor function for defining ctors JuliaLang/julia#57317)Commits considered
This list was generated so I could keep track of my work, but it might be helpful here to list bugfixes we don't have yet. Note that it only includes changes to julia-syntax.scm, which doesn't cover all breaking commits. N/A means I intentionally skipped it for some reason (e.g. already done before this PR).
Long list
outerrefintermediate form (#54772)exportand similar (#54812)Moduleinto method name slot (#54856)@opaque Tuple{T,U...}->RT (...)->...syntax for explicit arg/return types (#54947)PartialOpaquesupport viaExpr(:opaque_closure, ...)(#55068)finallyblocks for exceptional control-flow (#55876):latestworld(#56523)(pop-handler-list ...)(#55871)linearize(#57416)...expressions (#57480)>:inwhere(#57554)functiondeclaration implicitlyglobal(#57562)importorusing(#57774)constassignment:methodExpr return value between interpreter and codegen (#58076)