Add expression compiler #22597
Merged
Add expression compiler #22597
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
adpi2
force-pushed
the
add-expression-compiler
branch
from
37985f6 to
839713b
Compare
|
Looks like the tests are not compiling due to usage of newer JDK API? |
I need to configure sbt-jdi-tools to enable JDI on JDK 8. |
adpi2
force-pushed
the
add-expression-compiler
branch
3 times, most recently
from
55e7262 to
df7f853
Compare
adpi2
force-pushed
the
add-expression-compiler
branch
6 times, most recently
from
d60954c to
b207244
Compare
sjrd
requested changes
Mar 5, 2025
sjs-compiler-tests/test/scala/dotty/tools/dotc/ScalaJSCompilationTests.scala
Outdated
Show resolved
Hide resolved
adpi2
force-pushed
the
add-expression-compiler
branch
2 times, most recently
from
11e4259 to
587896a
Compare
debugMode allows to connect to runner processes with JDI
Each time we disconnect the debugger, the child process opens a new port that we must read to connect a new debugger
We use ParallelTesting.cleanup instead
adpi2
force-pushed
the
add-expression-compiler
branch
from
587896a to
9fcf8f1
Compare
sjrd
approved these changes
Mar 10, 2025
tgodzik
reviewed
Mar 10, 2025
| val errorWithNewPos = new Diagnostic.Error(error.msg, newPos) | ||
| reportError(stripColor(messageAndPos(errorWithNewPos))) | ||
| case _ => | ||
| // TODO report the warnings in the expression |
There was a problem hiding this comment.
Should we report warnings? Maybe let's add a follow up issue later if this really necessary, but I would say it's not.
| override def phaseName: String = InsertExpression.name | ||
| override def isCheckable: Boolean = false | ||
|
|
||
| // TODO move reflection methods (callMethod, getField, etc) to scala3-library |
There was a problem hiding this comment.
Let's maybe create an issue?
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR migrates the expression compiler from scalacenter/scala-debug-adapter to the scala3 repository. This change was agreed upon a few months ago during a core meeting to simplify the release process. By doing so, we eliminate the need to release parts of scalacenter/scala-debug-adapter during each Scala 3 release.
The
ExpressionCompilerandExpressionCompilerBridgeare now parts of the main compiler module. This integration streamlines usage for external tools like sbt, Metals, Bloop, IJ Scala plugin. They can now rely on the compiler JAR directly without needing to download additional artifacts.The expression compiler
The expression compiler extends the main compiler with 3 phases:
InsertExpression:
def evaluate: Anymethod.ExtractExpression:
Typer, the pickler phases,ExtensionMethodsandElimByName.LambdaLiftto prevent interference: the expression itself can change how lambdas are lifted. Notably it can contains lambda that needs to be lifted to the expression class.ResolveReflectEval:
reflectEvalcalls into actual reflective calls based onReflectEvalStrategymetadata.Usage of the expression compiler
The expression compiler powers the debug console in Metals and the IJ Scala plugin, enabling evaluation of arbitrary Scala expressions at runtime (even macros). The expression compiler produce class files that can be loaded the running Scala program, to compute the evaluation output.
The
ExpressionCompilerBridgecan be invoked by reflection with the following parameters:outputDir: Path: Directory where compiled classes are writtenclassPath: String: Classpath used during compilationoptions: Array[String]: Compiler optionssourceFile: Path: Source file where the expression is evaluatedconfig: ExpressionCompilerConfig:packageName: Package name of the current evaluation frameoutputClassName: String: Name of the main generated class, to be loaded by the debuggeebreakpointLine: Int: Line number where the expression is evaluatedexpression: String: Expression to compilelocalVariables: Set[String]: Set of visible local variables (used for checking evaluation of captured variables)errorReporter: Consumer[String]: Callback for reporting errorsThe
ExpressionCompilerConfigis designed as a factory class to allow binary-compatible evolution across versions, enabling cross-compatibility between different debugger and compiler versions.The debug tests
A significant part of this PR is the migration of ScalaEvaluationTests, notably the integration with the vulpix test infrastructure:
dotty.tools.vulpix.RunnerOrchestration:debugModeto start runners with JVM debugging optionsdebugMainmethod to connect a debugger, start a main method and execute the debug stepsdotty.tools.debug.Debugger: a lightweight implementation of a JVM debuggerdotty.tools.debug.ExpressionEvaluator: to compile and evaluate expressions insidedotty.tools.debug.Debuggerdotty.tools.debug.DebugStepAssert: to describe a debug scenario as a series of steps (break,step,nextandeval)dotty.tools.debug.DebugTeststest class:tests/debug.checkfile as a series ofDebugStepAssertdebugMain, and executes the debug stepsI migrated all relevant tests from ScalaEvaluationTests to
tests/debugfolder. Thanks to Vulpix parallel execution, the 47 debug tests now complete in approximately 30-40 seconds.