Language Server appears to be out of sync with command line gradle

[mcraveiro]

Hi Kotlin Language Server developers,

Many thanks for a great project. I have been trying to set it up against emacs, but I must be doing something very silly because I get errors with KLS that I do not see when I manually run gradle. Let me first explain my setup. I am using Debian testing on x64 and I installed kotlin using SDK Man [0]. I have tried two distinct setups though, sadly, with more or less the same outcome.

Setup using Java 11

I have read through the many posts detailing how it is not possible to use Java 11 to build KLS - e.g. [1, 2, 3] - but it was still unclear to me if this limitation also applied to building user projects or only to building KLS itself [4]. So I started first by installing openjdk-11 and using the binaries supplied on GitHub [5]. Though there is no out of the box support on emacs for KLS [6], thanks to the help of the emacs lsp community, I managed to get emacs to start KLS and report errors on my kotlin code.

The weird thing was that it all worked well at first, but as I started to add more dependencies and change the project files, I started to see errors in KLS that I do not see in gradle. In particular, I see:

 Server.kt    20  27 error    INTER… kotlin: Calls to static methods in Java interfaces are prohibited in JVM target 1.6. Recompile with '-jvm-target 1.8' (lsp-ui)
 Server.kt    27  33 error    INTER… kotlin: Calls to static methods in Java interfaces are prohibited in JVM target 1.6. Recompile with '-jvm-target 1.8' (lsp-ui)

However, my gradle file states:

tasks.withType<KotlinCompile> {
    kotlinOptions.jvmTarget = "1.8"
}

In addition, my kts (gradle Kotlin script files) are riddled with errors:

 build.gra…     1  29 error    UNRES… kotlin: Unresolved reference: gradle (lsp-ui)
 build.gra…     3   1 error    UNRES… kotlin: Unresolved reference: plugins (lsp-ui)
 build.gra…     4   5 error    UNRES… kotlin: Unresolved reference: application (lsp-ui)
 build.gra…     5   5 error    UNRES… kotlin: Unresolved reference: id (lsp-ui)
...

The first error stems from this line:

import org.jetbrains.kotlin.gradle.tasks.KotlinCompile

To the untrained eye, it seems like the gradle kotlin DSL files are just not being picked up correctly.

Setup using Java 8

I then decided to investigate two questions:

• check if there have been fixes in master that address my problem;
• check if Java 11 is also not supported in user projects.

For this I relied on openjdk-8, but sadly it is not (entirely) available on Debian testing. The missing packages were downloaded manually from their FTP server [7]:

openjdk-8-jdk_8u212-b03-2_deb9u1_amd64.deb
openjdk-8-jdk-headless_8u212-b03-2_deb9u1_amd64.deb

Once installed I selected the java version via alternatives:

update-alternatives --config java

And set it to Java 8. I confirmed that my binaries were running from Java 8, as well as my local build using gradle. I then git cloned KLS and built it as per my understanding of the instructions:

./gradlew server:build
./gradlew server:installDist

It all worked fine. I then installed the resulting binaries, restarted emacs and picked up the new KLS. However, I am getting exactly the same errors as I had before, as per section above.

Conclusions

So it seems to me my problems are actually not related to the java version, but to my project. The weird thing is that when I run gradle build (or via the gradle wrapper) it all works. So I'm trying to figure out what KLS is doing differently. Can you point out to mistakes I have done on my setup, or if nothing obvious can be spotted, on ways to get more debug information about what KLS is doing?

Thanks very much for all your help.

Marco

[0] https://kotlinlang.org/docs/tutorials/command-line.html
[1] #125
[2] #108
[3] #114
[4] #124
[5] https://github.com/fwcd/KotlinLanguageServer/releases/tag/0.1.13
[6] emacs-lsp/lsp-mode#900
[7] http://ftp.debian.org/debian/pool/main/o/openjdk-8/

[mcraveiro]

Actually, an important omission in the previous commentary is that there likely are some problems in the setup of my kotlin project. I have created as a way to learn how kotlin projects work, (mostly) by hand, and with the objective of supporting multi-project builds [1]. Maybe I have done something that is confusing KLS.

My file structure is as follows (files that were generated by gradle are marked as [generated]):

./gradlew.bat                                                 [generated]
./build.gradle.kts
./gradle.properties
./settings.gradle.kts
./gradlew                                                     [generated]
./gradle                                                      [generated]
./gradle/wrapper                                              [generated]
./gradle/wrapper/gradle-wrapper.properties                    [generated]
./gradle/wrapper/gradle-wrapper.jar                           [generated]
./projects
./projects/cli
./projects/cli/build.gradle.kts
./projects/cli/src
./projects/cli/src/test
./projects/cli/src/test/kotlin
./projects/cli/src/test/kotlin/abc
./projects/cli/src/test/kotlin/abc/cli
./projects/cli/src/test/kotlin/abc/cli/HelloTest.kt
./projects/cli/src/main
./projects/cli/src/main/kotlin
./projects/cli/src/main/kotlin/abc
./projects/cli/src/main/kotlin/abc/cli
./projects/cli/src/main/kotlin/abc/cli/Server.kt
./projects/core
./projects/core/build.gradle.kts
./projects/core/src
./projects/core/src/test
./projects/core/src/test/kotlin
./projects/core/src/test/kotlin/abc
./projects/core/src/test/kotlin/abc/core
./projects/core/src/test/kotlin/abc/core/HelloTest.kt
./projects/core/src/main
./projects/core/src/main/kotlin
./projects/core/src/main/kotlin/abc
./projects/core/src/main/kotlin/abc/core
./projects/core/src/main/kotlin/abc/core/Hello.kt

All my targets (build, test, etc) work from the gradle CLI. I will try to clean up the project and put it on a stand alone GitHub project if it helps.

[1] https://docs.gradle.org/current/userguide/multi_project_builds.html

[mcraveiro]

Yet another interesting point: I see a few errors in *kotlin-ls::stderr* that look puzzling:

ERROR: Exception while analyzing expression at (1,492) in /dummy.kt
org.jetbrains.kotlin.utils.KotlinExceptionWithAttachments: Exception while analyzing expression at (1,492) in /dummy.kt
	at org.jetbrains.kotlin.types.expressions.ExpressionTypingVisitorDispatcher.logOrThrowException(ExpressionTypingVisitorDispatcher.java:233)
	at org.jetbrains.kotlin.types.expressions.ExpressionTypingVisitorDispatcher.lambda$getTypeInfo$0(ExpressionTypingVisitorDispatcher.java:211)
	at org.jetbrains.kotlin.util.PerformanceCounter.time(PerformanceCounter.kt:91)
	at org.jetbrains.kotlin.types.expressions.ExpressionTypingVisitorDispatcher.getTypeInfo(ExpressionTypingVisitorDispatcher.java:161)
	at org.jetbrains.kotlin.types.expressions.ExpressionTypingVisitorDispatcher.getTypeInfo(ExpressionTypingVisitorDispatcher.java:145)
	at org.jetbrains.kotlin.types.expressions.ExpressionTypingServices.getTypeInfo(ExpressionTypingServices.java:114)
	at org.jetbrains.kotlin.types.expressions.ExpressionTypingServices.getTypeInfo(ExpressionTypingServices.java:93)
	at org.javacs.kt.Compiler.compileExpression(Compiler.kt:148)
	at org.javacs.kt.CompiledFile.bindingContextOf(CompiledFile.kt:39)
	at org.javacs.kt.CompiledFile.referenceAtPoint(CompiledFile.kt:53)
	at org.javacs.kt.hover.HoversKt.hoverAt(Hovers.kt:37)
	at org.javacs.kt.KotlinTextDocumentService$hover$1.invoke(KotlinTextDocumentService.kt:80)
	at org.javacs.kt.KotlinTextDocumentService$hover$1.invoke(KotlinTextDocumentService.kt:26)
	at org.javacs.kt.util.AsyncExecutorKt$sam$java_util_function_Supplier$0.get(AsyncExecutor.kt)
	at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1590)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.UnsupportedOperationException: Should not be called
	at org.jetbrains.kotlin.resolve.lazy.NoTopLevelDescriptorProvider.shouldNotBeCalled(TopLevelDescriptorProvider.kt:36)
	at org.jetbrains.kotlin.resolve.lazy.NoTopLevelDescriptorProvider.getPackageFragment(TopLevelDescriptorProvider.kt:39)
	at org.jetbrains.kotlin.resolve.DeclarationResolver.getTopLevelDescriptorsByFqName(DeclarationResolver.kt:107)
	at org.jetbrains.kotlin.resolve.DeclarationResolver.checkRedeclarationsInPackages(DeclarationResolver.kt:86)
	at org.jetbrains.kotlin.resolve.LazyTopDownAnalyzer.analyzeDeclarations(LazyTopDownAnalyzer.kt:214)
	at org.jetbrains.kotlin.types.expressions.LocalClassifierAnalyzer.processClassOrObject(LocalClassifierAnalyzer.kt:106)
	at org.jetbrains.kotlin.types.expressions.ExpressionTypingVisitorForStatements.visitObjectDeclaration(ExpressionTypingVisitorForStatements.java:102)
	at org.jetbrains.kotlin.types.expressions.ExpressionTypingVisitorForStatements.visitObjectDeclaration(ExpressionTypingVisitorForStatements.java:62)
	at org.jetbrains.kotlin.psi.KtObjectDeclaration.accept(KtObjectDeclaration.kt:63)
	at org.jetbrains.kotlin.types.expressions.ExpressionTypingVisitorDispatcher.lambda$getTypeInfo$0(ExpressionTypingVisitorDispatcher.java:172)
	... 16 more
Jul 16, 2019 8:16:13 PM org.eclipse.lsp4j.jsonrpc.RemoteEndpoint fallbackResponseError

And

SEVERE: Internal error: org.javacs.kt.util.KotlinLSException: Error while analyzing: object Server {
...
	at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:273)
	at java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:280)
	at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1592)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: org.javacs.kt.util.KotlinLSException: Error while analyzing: object Server {

The second error is related to plain kotlin code, the first I'm not too sure.

[mcraveiro]

As a control test, I've decided to clone the KotlinQuickStart repo [1] and validate KTS from there. I experience the same problems with the Kotlin DSL. Upon further investigation, it seems the kotlin DSL is actually not yet supported [2], so I will switch off LSP for these files. On the other hand, I do seem to get good error checking on the regular kotlin files. I will try to bring across code from my project and see if I can get it to work...

[1]https://github.com/fwcd/KotlinQuickStart
[2] #6

[mcraveiro]

OK, I've managed to reproduce my problems using the KotlinQuickStart repo. I modified the code as follows:

diff --git a/build.gradle b/build.gradle
index 936ec53..679d111 100644
--- a/build.gradle
+++ b/build.gradle
@@ -16,14 +16,29 @@ version = '0.1.1'
 mainClassName = 'MainKt'
 description = 'My Application'
 
-sourceCompatibility = 1.8
-targetCompatibility = 1.8
-
 repositories {
 	mavenCentral()
+    jcenter()
+
 }
 
 dependencies {
 	implementation 'org.jetbrains.kotlin:kotlin-stdlib'
 	testImplementation 'junit:junit:4.12'
+
+    compile 'io.vertx:vertx-core:3.7.1'
+    compile 'io.vertx:vertx-web:3.7.1'
+    compile 'io.vertx:vertx-jdbc-client:3.7.1'
+
+}
+
+compileKotlin {
+    sourceCompatibility = JavaVersion.VERSION_1_8
+    targetCompatibility = JavaVersion.VERSION_1_8
+
+    kotlinOptions {
+        jvmTarget = "1.8"
+        apiVersion = "1.1"
+        languageVersion = "1.1"
+    }
 }
diff --git a/src/main/kotlin/Main.kt b/src/main/kotlin/Main.kt
index 46dfdfc..a0ecc39 100644
--- a/src/main/kotlin/Main.kt
+++ b/src/main/kotlin/Main.kt
@@ -1,3 +1,34 @@
+import io.vertx.core.AsyncResult
+import io.vertx.core.Future
+import io.vertx.core.Vertx
+import io.vertx.core.json.Json
+import io.vertx.core.json.JsonArray
+import io.vertx.core.json.JsonObject
+import io.vertx.ext.jdbc.JDBCClient
+import io.vertx.ext.sql.ResultSet
+import io.vertx.ext.sql.SQLConnection
+import io.vertx.ext.web.Router
+import io.vertx.ext.web.RoutingContext
+import io.vertx.ext.web.handler.BodyHandler
+import java.util.*
+import kotlin.reflect.KClass
+
+object Server {
+    @JvmStatic fun main(args: Array<String>) {
+
+        val vertx = Vertx.vertx()
+        val server = vertx.createHttpServer()
+        val port = 9000
+        val router = Router.router(vertx)
+
+        router.get("/").handler { it.response().end("Hello world!") }
+        server.requestHandler { router.accept(it) }.listen(port) {
+            if (it.succeeded()) println("Server listening at $port")
+            else println(it.cause())
+        }
+    }
+}
+
 fun main(args: Array<String>) {
 	println("Hello world")
 }

I had to add jvmTarget etc to get the code to compile with gradle. However, KTS is now complaining of unresolved references:

 Main.kt     1   8 error    UNRES… kotlin: Unresolved reference: io (lsp-ui)
 Main.kt     2   8 error    UNRES… kotlin: Unresolved reference: io (lsp-ui)
 Main.kt     3   8 error    UNRES… kotlin: Unresolved reference: io (lsp-ui)
 Main.kt     4   8 error    UNRES… kotlin: Unresolved reference: io (lsp-ui)
 Main.kt     5   8 error    UNRES… kotlin: Unresolved reference: io (lsp-ui)
 Main.kt     6   8 error    UNRES… kotlin: Unresolved reference: io (lsp-ui)
 Main.kt     7   8 error    UNRES… kotlin: Unresolved reference: io (lsp-ui)
 Main.kt     8   8 error    UNRES… kotlin: Unresolved reference: io (lsp-ui)
 Main.kt     9   8 error    UNRES… kotlin: Unresolved reference: io (lsp-ui)
 Main.kt    10   8 error    UNRES… kotlin: Unresolved reference: io (lsp-ui)
 Main.kt    11   8 error    UNRES… kotlin: Unresolved reference: io (lsp-ui)
 Main.kt    12   8 error    UNRES… kotlin: Unresolved reference: io (lsp-ui)
 Main.kt    19  21 error    UNRES… kotlin: Unresolved reference: Vertx (lsp-ui)
 Main.kt    22  22 error    UNRES… kotlin: Unresolved reference: Router (lsp-ui)
 Main.kt    24  35 error    UNRES… kotlin: Unresolved reference: it (lsp-ui)
 Main.kt    25  47 error    UNRES… kotlin: Unresolved reference: it (lsp-ui)
 Main.kt    26  17 error    UNRES… kotlin: Unresolved reference: it (lsp-ui)
 Main.kt    27  26 error    UNRES… kotlin: Unresolved reference: it (lsp-ui)

These are all of the vertx imports. It seems I need to "resync" KTS after gradle downloaded the new jars?

[mcraveiro]

OK, I now realised that by restarting my workspace I can get KLS to pick up the dependencies (lsp-restart-workspace in emacs). However, the only snag is that I still can't get it to recognise my Java 1.8 settings:

 Main.kt    19  27 error    INTER… kotlin: Calls to static methods in Java interfaces are prohibited in JVM target 1.6. Recompile with '-jvm-target 1.8' (lsp-ui)
 Main.kt    23  29 error    INTER… kotlin: Calls to static methods in Java interfaces are prohibited in JVM target 1.6. Recompile with '-jvm-target 1.8' (lsp-ui)
 Main.kt    26  40 warning  DEPRE… kotlin: 'accept(HttpServerRequest!): Unit' is deprecated. Deprecated in Java (lsp-ui)

For good measure, I installed VS code just to see if this was an emacs problem but it also complains about the same error.

[mcraveiro]

OK right so this ticket is quite long and rambling-y, so I think its worth summarising my findings:

1. As stated in many other places, you need Java 8 to build KLS itself. (Build instructions imply Java 8+ support, but really KLS only supports Java 8 #125, problem building using java 11.0.2 #108, java 12 is not working #114)
2. Kotlin DSL files are not supported by KLS. (Better build instructions w.r.t Java version #124)
3. It seems building user projects using Java 11 gives the exact same behaviour as using Java 8 - e.g. they are supported by KLS.
4. Adding new dependencies (jars) to a project appears to require a restart of KLS when used under emacs (lsp-restart-workspace).

Which means that the only real problem I have is this one:

5. Warnings related to using Java 8 features appear even though the project has jvmTarget = "1.8" set in the gradle files. This behaviour is present in both in v0.1.13 and master.

To avoid confusion, I think I will close this ticket and open a new one focused only on this problem.

[mcraveiro]

A ticket for this already exists [1], so I'll move the discussion there.

[1] #72

[fwcd]

Thanks for investigating and providing such a detailed description of the issue!

I have to admit that I haven't got around to test the compiled KLS on Java 11 projects yet, though this will definitely be part of #109, since KLS should always support its own codebase.