deal with classpath issues during bootstrapping

[adriaanm]

core dependencies are dropped by ivy because of faux circular dependencies

to avoid problem with circular dependency when bootstrapping the modules

compilers in (Compile, doc) := {
  val instance = scalaInstance.value
  val cp = (packageBin in Compile).value +: instance.allJars
  println(s"Classpath for scalaInstance =\n\t * ${cp mkString "\n\t * "}")
  val launcher = appConfiguration.value.provider.scalaProvider.launcher
  val newInstance = ScalaInstance(instance.libraryJar, instance.compilerJar, launcher, cp:_*)
  Compiler.compilers(newInstance, classpathOptions.value, javaHome.value)(appConfiguration.value, streams.value.log)
}

[adriaanm]

or:

val scalaInstanceFromUpdate = TaskKey[ScalaInstance]("scala-instance-from-update", "Defines the Scala instance to use for compilation, running, and testing.")

scalaInstanceFromUpdate := Defaults.scalaInstanceFromUpdate.value

scalaInstance := {
 val s = state.value
 val extracted = Project.extract(s)
 val newState = extracted.append(List(version := "1.0.0-BLA", libraryDependencies += "junit" % "junit" % "4.10" % "scala-tool"), s)
 val newState2 = extracted.runTask(Keys.update, newState)._1
 val inst = extracted.runTask(scalaInstanceFromUpdate, newState2)._2
 println(inst.allJars.toList)
 inst
}

[gkossakowski]

Won't the first method result in picking up Scala version used by sbt itself?

[gkossakowski]

core dependencies are dropped by ivy because of faux circular dependencies

What does it mean? For a while I thought that the problem is that a module (let's say scala-xml) defines a project "org.scala-lang.modules" %% "scala-xml" % xxx but then this project depends on scala-tools configuration (to resolve scala compiler) and scala-compiler depends on scala-xml so we get a circular dependency. I thought that the solution would be to resolve scala-tools configuration in isolation from the project so Ivy wouldn't see the dependency and wouldn't complain. This is safe because scala-tools consits of JARs used at runtime of scala compiler/scaladoc/repl. They should never be part of the classpath used for compiling.

[adriaanm]

This is unrelated to the testing issues we were talking about. This is about bootstrapping a release, and this is confirmed to resolve the problem.

The current workaround is: https://github.com/scala/jenkins-scripts/blob/master/job/scala-release-2.11.x#L124 -- set a dummy version for scala-xml, build docs, then set real version and build the release. The dummy version is needed because ivy drops jars involved in a circular dependency from the classpath and thus scaladoc fails because it can't find any scala-xml classes. The cycle occurs because scaladoc depends on the very same version of scala-xml that it's documenting.

[SethTisue]

@szeiger is this no longer an issue in the 2.12 build?

[szeiger]

It's the first time I look at this ticket and I don't understand what it's about. The link above is not stable. I tried to look in the history but it wasn't obvious to me what it's supposed to point at. Since the dependencies on scala-xml haven't changed in 2.12, I have no reason to believe that it would behave any differently than in 2.11.

[adriaanm]

It would affect 2.12 in the same way as 2.11. My recollection is that we had to put temporary version number hacks in place in the bootstrap script when documenting the modules because we would be depending on scala-xml (for running scaladoc) while documenting scala-xml (same version). This cycle caused ivy to drop this artifact from the class path, which would cause scaladoc to fail. The same hack that resolved this in 2.11 is also in place for 2.12 AFAIK

[ashawley]

Is this the same issue that causes scala/scala-xml#112 (among others)? Or a different one?

[adriaanm]

I haven't investigated closely, but that looks unrelated.

…

On Thu, Dec 1, 2016 at 13:19 Aaron S. Hawley ***@***.***> wrote: Is this the same issue that causes scala/scala-xml#112 <scala/scala-xml#112> (among others)? Or a different one? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#5 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAFjy3IZPYC8PEoVHnJazz6d8gvsOCyaks5rDznHgaJpZM4Bn9ix> .

[SethTisue]

I am closing this, because I think the solution is to finally just cut the remaining thin dependency of the scaladoc tool on scala-xml, and forbid any such circular dependencies between scala and it modules from ever arising again. circular dependencies are what Pandora's box was full of