kotlinx.coroutines.debug's DebugProbes fails when loaded from a different ClassLoader than kotlin.stdlib

[ArchangelX360]

Describe the bug

DebugProbes.install() works, but the code crashed when spawning the first coroutine after installation, with java.lang.ClassNotFoundException: kotlinx.coroutines.debug.internal.DebugProbesImpl.

First, a quick context on our different module layers hierarchy (we use JPMS), our module layers are structured like such:

Layer "fleet.dock.bootstrap" (`AppClassLoader`)
    -> Module `fleet.dock.bootstrap`
    -> Module `kotlin.stdlib`
    -> [...]
  └── Layer "fleet.dock.api"
      -> Module `fleet.dock.api`
      -> Module `kotlinx.coroutines.core`
      -> Module `kotlinx.coroutines.debug`
      -> [...]

Note: Module layers does not have names, I just name them by the most significant module that is part of it for convenience.

Here is what I think is happening, internally, when doing DebugProbes.install(), kotlinx.coroutines.debug uses ByteBuddy to swap kotlin.stdlib's kotlin.coroutines.jvm.internal.DebugProbesKt code with the code from kotlinx.coroutines.core's kotlinx.coroutines.debug.internal.DebugProbesKt.
This runtime injected code tries to use DebugProbesImpl object from kotlinx.coroutines.core module. But it does that at the kotlin.stdlib level, so using the fleet.dock.bootstrap's ClassLoader (AppClassLoader), which does not know about kotlinx.coroutines.core module and so it crashes with ClassNotFoundException.

[qwwdfsad]

Thanks! DebugProbes, indeed, were not designed/implemented to work with multiple classloaders and with module separation in mind.

Could you please elaborate on the following:

• Is it correct that this problem prevents you from using debug probes in Fleet? Could you please share FL ticket?
• Would you be able to test an intermediate -dev build w.r.t. this issue? Classloading is a nuanced matter and it would be nice to have an extra check before shipping the fix

[ArchangelX360]

Is it correct that this problem prevents you from using debug probes in Fleet? Could you please share FL ticket?

Indeed, we solve the package split internally, but we are stuck on that classloader issue, and it seems there is nothing easy we can do on our side to address it. Here is the ticket: https://youtrack.jetbrains.com/issue/FL-23308

Would you be able to test an intermediate -dev build w.r.t. this issue? Classloading is a nuanced matter and it would be nice to have an extra check before shipping the fix

Yes, that's most probably possible, and we would be our pleasure to help testing it out.

[qwwdfsad]

Thanks!

[qwwdfsad]

We've discussed it further and I also investigated why IDEA does not have this problem.

The short summary:

• There is nothing we can do on kotlinx-coroutines side. Indeed, DebugProbes from stdlib should be replaced with something and this something is located in kotlinx-coroutines and depends on it. We cannot reasonably isolate the debugger from kotlinx-coroutines, thus, this is the intrinsic requirement of the debugger -- to be in the same CL hierarchy/JPMS reachability as the standard library. This is the reason docs tag is here -- we'll explicitly mention that
• IntelliJ right now loads both stdlib and coroutines in the same classloader, but previously used child-first classloader (see com.intellij.ide.plugins.cl.PluginClassLoader) and, when requested to load a class by a classloader holding standard library, first looked the class up in children classloader (not unconditionally, but with specific filtering)

If you or anybody else have any other ideas how we can patch these things up, I'm open to suggestions