Skip to content

Scaladoc generation fails when extending some Java-defined or Scala-2-defined classes #15927

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
GitZinger opened this issue Aug 27, 2022 · 33 comments · Fixed by #16759
Closed

Scaladoc generation fails when extending some Java-defined or Scala-2-defined classes #15927

GitZinger opened this issue Aug 27, 2022 · 33 comments · Fixed by #16759
Assignees
@SethTisue @jan-pieter
Labels
area:doctool itype:bug itype:crash Spree Suitable for a future Spree
Milestone
3.3.1

Comments

@GitZinger
Copy link

GitZinger commented Aug 27, 2022 •
edited by dwijnand
Loading

Compiler version

3.2.0.rc4 and sbt 1.7.1,

If you're not sure what version you're using, run print scalaVersion from sbt
(if you're running scalac manually, use scalac -version instead).

Minimized code

sbt clean compile doc package

//the following is **BugTest.scala**


//::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
/** @author  Gitzinger
 *
 *  @title   `BugTest` for Handle Dates and Times
 */

package test4education



//::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
/** The `BugTest`  class 
 */
abstract class BugTest () extends Numeric [BugTest] with Serializable:

    //::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
    /**  a `BugTest` 
     */
    def what() = println("a bug")
  
    

end BugTest

Output (click arrow to expand)

// TODO add output here
[info] Main Scala API documentation to ../.../scala-3.2.0-RC4/api...
Problem parsing src/main/scala/BugTest.scala:[159..167..529], documentation may not be generated.
java.lang.AssertionError: assertion failed
  | => rat scala.runtime.Scala3RunTime$.assertFailed(Scala3RunTime.scala:11)
	at dotty.tools.dotc.util.Spans$Span$.start$extension(Spans.scala:45)
	at dotty.tools.dotc.util.SourcePosition.start(SourcePosition.scala:52)
	at scala.quoted.runtime.impl.QuotesImpl$reflect$PositionMethods$.start(QuotesImpl.scala:2813)
	at scala.quoted.runtime.impl.QuotesImpl$reflect$PositionMethods$.start(QuotesImpl.scala:2813)
	at dotty.tools.scaladoc.tasty.ClassLikeSupport.getParentsAsTreeSymbolTuples$$anonfun$1(ClassLikeSupport.scala:260)
	at scala.collection.Iterator$$anon$6.hasNext(Iterator.scala:472)
	at scala.collection.Iterator$$anon$9.hasNext(Iterator.scala:576)
	at scala.collection.immutable.List.prependedAll(List.scala:152)
	at scala.collection.immutable.List$.from(List.scala:684)
	at scala.collection.immutable.List$.from(List.scala:681)
	at scala.collection.IterableOps$WithFilter.map(Iterable.scala:891)
	at dotty.tools.scaladoc.tasty.ClassLikeSupport.getParentsAsTreeSymbolTuples(ClassLikeSupport.scala:264)
	at dotty.tools.scaladoc.tasty.ClassLikeSupport.getParentsAsTreeSymbolTuples$(ClassLikeSupport.scala:15)
	at dotty.tools.scaladoc.tasty.TastyParser.getParentsAsTreeSymbolTuples(TastyParser.scala:169)
	at dotty.tools.scaladoc.tasty.ClassLikeSupport.getParentsAsLinkToTypes(ClassLikeSupport.scala:253)
	at dotty.tools.scaladoc.tasty.ClassLikeSupport.getParentsAsLinkToTypes$(ClassLikeSupport.scala:15)
	at dotty.tools.scaladoc.tasty.TastyParser.getParentsAsLinkToTypes(TastyParser.scala:169)
	at dotty.tools.scaladoc.tasty.ClassLikeSupport.mkClass(ClassLikeSupport.scala:106)
	at dotty.tools.scaladoc.tasty.ClassLikeSupport.mkClass$(ClassLikeSupport.scala:15)
	at dotty.tools.scaladoc.tasty.TastyParser.mkClass(TastyParser.scala:169)
	at dotty.tools.scaladoc.tasty.ClassLikeSupport.parseClasslike(ClassLikeSupport.scala:290)
	at dotty.tools.scaladoc.tasty.ClassLikeSupport.parseClasslike$(ClassLikeSupport.scala:15)
	at dotty.tools.scaladoc.tasty.TastyParser.parseClasslike(TastyParser.scala:169)
	at dotty.tools.scaladoc.tasty.ClassLikeSupport.parseInheritedMember$$anonfun$1(ClassLikeSupport.scala:190)
	at dotty.tools.scaladoc.tasty.TastyParser.processTreeOpt(TastyParser.scala:204)
	at dotty.tools.scaladoc.tasty.ClassLikeSupport.parseInheritedMember(ClassLikeSupport.scala:202)
	at dotty.tools.scaladoc.tasty.ClassLikeSupport.extractMembers$$anonfun$2(ClassLikeSupport.scala:221)
	at scala.collection.immutable.List.flatMap(List.scala:293)
	at dotty.tools.scaladoc.tasty.ClassLikeSupport.extractMembers(ClassLikeSupport.scala:221)
	at dotty.tools.scaladoc.tasty.ClassLikeSupport.extractMembers$(ClassLikeSupport.scala:15)
	at dotty.tools.scaladoc.tasty.TastyParser.extractMembers(TastyParser.scala:169)
	at dotty.tools.scaladoc.tasty.ClassLikeSupport.extractPatchedMembers(ClassLikeSupport.scala:226)
	at dotty.tools.scaladoc.tasty.ClassLikeSupport.extractPatchedMembers$(ClassLikeSupport.scala:15)
	at dotty.tools.scaladoc.tasty.TastyParser.extractPatchedMembers(TastyParser.scala:169)
	at dotty.tools.scaladoc.tasty.ClassLikeSupport.mkClass(ClassLikeSupport.scala:113)
	at dotty.tools.scaladoc.tasty.ClassLikeSupport.mkClass$(ClassLikeSupport.scala:15)
	at dotty.tools.scaladoc.tasty.TastyParser.mkClass(TastyParser.scala:169)
	at dotty.tools.scaladoc.tasty.ClassLikeSupport.parseClasslike(ClassLikeSupport.scala:290)
	at dotty.tools.scaladoc.tasty.ClassLikeSupport.parseClasslike$(ClassLikeSupport.scala:15)
	at dotty.tools.scaladoc.tasty.TastyParser.parseClasslike(TastyParser.scala:169)
	at dotty.tools.scaladoc.tasty.TastyParser$Traverser$2$.traverseTree(TastyParser.scala:223)
	at scala.quoted.Quotes$reflectModule$TreeTraverser.foldTree(Quotes.scala:4666)
	at scala.quoted.Quotes$reflectModule$TreeTraverser.foldTree$(Quotes.scala:4662)
	at dotty.tools.scaladoc.tasty.TastyParser$Traverser$2$.foldTree(TastyParser.scala:211)
	at dotty.tools.scaladoc.tasty.TastyParser$Traverser$2$.foldTree(TastyParser.scala:211)
	at scala.quoted.Quotes$reflectModule$TreeAccumulator.foldTrees$$anonfun$1(Quotes.scala:4554)
	at scala.collection.LinearSeqOps.foldLeft(LinearSeq.scala:169)
	at scala.collection.LinearSeqOps.foldLeft$(LinearSeq.scala:165)
	at scala.collection.immutable.List.foldLeft(List.scala:79)
	at scala.quoted.Quotes$reflectModule$TreeAccumulator.foldTrees(Quotes.scala:4554)
	at scala.quoted.Quotes$reflectModule$TreeAccumulator.foldTrees$(Quotes.scala:4549)
	at dotty.tools.scaladoc.tasty.TastyParser$Traverser$2$.foldTrees(TastyParser.scala:211)
	at scala.quoted.Quotes$reflectModule$TreeAccumulator.foldOverTree(Quotes.scala:4617)
	at scala.quoted.Quotes$reflectModule$TreeAccumulator.foldOverTree$(Quotes.scala:4549)
	at dotty.tools.scaladoc.tasty.TastyParser$Traverser$2$.foldOverTree(TastyParser.scala:211)
	at scala.quoted.Quotes$reflectModule$TreeTraverser.traverseTreeChildren(Quotes.scala:4668)
	at scala.quoted.Quotes$reflectModule$TreeTraverser.traverseTreeChildren$(Quotes.scala:4662)
	at dotty.tools.scaladoc.tasty.TastyParser$Traverser$2$.traverseTreeChildren(TastyParser.scala:211)
	at scala.quoted.Quotes$reflectModule$TreeTraverser.traverseTree(Quotes.scala:4664)
	at scala.quoted.Quotes$reflectModule$TreeTraverser.traverseTree$(Quotes.scala:4662)
	at dotty.tools.scaladoc.tasty.TastyParser$Traverser$2$.traverseTree(TastyParser.scala:219)
	at dotty.tools.scaladoc.tasty.TastyParser.parseRootTree(TastyParser.scala:228)
	at dotty.tools.scaladoc.tasty.ScaladocTastyInspector.$anonfun$4(TastyParser.scala:123)
	at scala.runtime.function.JProcedure1.apply(JProcedure1.java:15)
	at scala.runtime.function.JProcedure1.apply(JProcedure1.java:10)
	at dotty.tools.scaladoc.tasty.ScaladocTastyInspector.postProcess$$anonfun$2(TastyParser.scala:42)
	at scala.runtime.function.JProcedure1.apply(JProcedure1.java:15)
	at scala.runtime.function.JProcedure1.apply(JProcedure1.java:10)
	at scala.collection.immutable.List.foreach(List.scala:333)
	at dotty.tools.scaladoc.tasty.ScaladocTastyInspector.postProcess(TastyParser.scala:42)
	at scala.tasty.inspector.OldTastyInspector$TastyInspectorFinishPhase$1.runOn(OldTastyInspector.scala:91)
	at dotty.tools.dotc.Run.runPhases$1$$anonfun$1(Run.scala:234)
	at scala.runtime.function.JProcedure1.apply(JProcedure1.java:15)
	at scala.runtime.function.JProcedure1.apply(JProcedure1.java:10)
	at scala.collection.ArrayOps$.foreach$extension(ArrayOps.scala:1328)
	at dotty.tools.dotc.Run.runPhases$1(Run.scala:245)
	at dotty.tools.dotc.Run.compileUnits$$anonfun$1(Run.scala:253)
	at dotty.tools.dotc.Run.compileUnits$$anonfun$adapted$1(Run.scala:262)
	at dotty.tools.dotc.util.Stats$.maybeMonitored(Stats.scala:68)
	at dotty.tools.dotc.Run.compileUnits(Run.scala:262)
	at dotty.tools.dotc.Run.compileUnits(Run.scala:192)
	at dotty.tools.dotc.fromtasty.TASTYRun.compile(TASTYRun.scala:14)
	at dotty.tools.dotc.Driver.doCompile(Driver.scala:35)
	at dotty.tools.dotc.Driver.process(Driver.scala:195)
	at scala.tasty.inspector.OldTastyInspector.inspectFilesInContext(OldTastyInspector.scala:72)
	at scala.tasty.inspector.OldTastyInspector.inspectFilesInContext$(OldTastyInspector.scala:22)
	at scala.tasty.inspector.DocTastyInspector.inspectFilesInContext(DocTastyInspector.scala:5)
	at dotty.tools.scaladoc.tasty.ScaladocTastyInspector.result(TastyParser.scala:147)
	at dotty.tools.scaladoc.ScalaModuleProvider$.mkModule(ScalaModuleProvider.scala:11)
	at dotty.tools.scaladoc.Scaladoc$.run(Scaladoc.scala:230)
	at dotty.tools.scaladoc.Scaladoc$.run$$anonfun$1(Scaladoc.scala:72)
	at scala.runtime.function.JProcedure1.apply(JProcedure1.java:15)
	at scala.runtime.function.JProcedure1.apply(JProcedure1.java:10)
	at scala.Option.map(Option.scala:242)
	at dotty.tools.scaladoc.Scaladoc$.run(Scaladoc.scala:76)
	at dotty.tools.dottydoc.Main$.process(Main.scala:25)
	at dotty.tools.dottydoc.Main.process(Main.scala)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at xsbt.DottydocRunner.run(DottydocRunner.java:61)
	at xsbt.ScaladocInterface.run(ScaladocInterface.java:11)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at sbt.internal.inc.AnalyzingCompiler.invoke(AnalyzingCompiler.scala:329)
	at sbt.internal.inc.AnalyzingCompiler.doc(AnalyzingCompiler.scala:175)
	at sbt.internal.inc.AnalyzingCompiler.doc(AnalyzingCompiler.scala:133)
	at sbt.Doc$.$anonfun$scaladoc$1(Doc.scala:52)
	at sbt.Doc$.$anonfun$scaladoc$1$adapted(Doc.scala:40)
	at sbt.RawCompileLike$.$anonfun$prepare$1(RawCompileLike.scala:79)
	at sbt.RawCompileLike$.$anonfun$prepare$1$adapted(RawCompileLike.scala:72)
	at sbt.RawCompileLike$.$anonfun$cached$4(RawCompileLike.scala:63)
	at sbt.RawCompileLike$.$anonfun$cached$4$adapted(RawCompileLike.scala:61)
	at sbt.util.Tracked$.$anonfun$inputChangedW$1(Tracked.scala:219)
	at sbt.RawCompileLike$.$anonfun$cached$1(RawCompileLike.scala:68)
	at sbt.RawCompileLike$.$anonfun$cached$1$adapted(RawCompileLike.scala:52)
	at sbt.Defaults$.$anonfun$docTaskSettings$4(Defaults.scala:2157)
	at scala.Function1.$anonfun$compose$1(Function1.scala:49)
	at sbt.internal.util.$tilde$greater.$anonfun$$u2219$1(TypeFunctions.scala:62)
	at sbt.std.Transform$$anon$4.work(Transform.scala:68)
	at sbt.Execute.$anonfun$submit$2(Execute.scala:282)
	at sbt.internal.util.ErrorHandling$.wideConvert(ErrorHandling.scala:23)
	at sbt.Execute.work(Execute.scala:291)
	at sbt.Execute.$anonfun$submit$1(Execute.scala:282)
	at sbt.ConcurrentRestrictions$$anon$4.$anonfun$submitValid$1(ConcurrentRestrictions.scala:265)
	at sbt.CompletionService$$anon$2.call(CompletionService.scala:64)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:833)
[warn] Companion for class xxx exists but is missing in classlike map
@GitZinger GitZinger added itype:bug itype:crash stat:needs triage Every issue needs to have an "area" and "itype" label labels Aug 27, 2022
@SethTisue
Copy link
Member

How would we reproduce the problem on our own computers?

What is the minimum code xxx.scala must contain for the bug to be triggered?

@KacperFKorban KacperFKorban added the stat:needs minimization Needs a self contained minimization label Aug 28, 2022
@GitZinger
Copy link
Author

GitZinger commented Aug 29, 2022 •
edited
Loading

How would we reproduce the problem on our own computers?

What is the minimum code xxx.scala must contain for the bug to be triggered?

@SethTisue
Hey sorry about that. I just updated the codes. if you run sbt clean compile doc package on a project with codes like this, you will end up with the same issue.

Could you look into it please? I can even push a repo with this the simplified code.

@Kordyjan Kordyjan added area:doctool and removed stat:needs triage Every issue needs to have an "area" and "itype" label labels Aug 29, 2022
@Kordyjan Kordyjan assigned szymon-rd and unassigned szymon-rd Aug 29, 2022
@SethTisue
Copy link
Member

SethTisue commented Aug 29, 2022 •
edited
Loading

Could you look into it please?

It's not my area of expertise, but perhaps one of the Scaladoc maintainers will have a look.

@SethTisue
Copy link
Member

You've improved the report greatly by supplying code, but presumably most of that code presumably isn't actually necessary for reproducing the bug. The report would be even higher quality, and much likelier to be acted upon, if you reduced it to the absolute minimum necessary code to demonstrate the problem. (And that's why Paweł added the "needs minimization" label.)

@GitZinger
Copy link
Author

You've improved the report greatly by supplying code, but presumably most of that code presumably isn't actually necessary for reproducing the bug. The report would be even higher quality, and much likelier to be acted upon, if you reduced it to the absolute minimum necessary code to demonstrate the problem. (And that's why Paweł added the "needs minimization" label.)

@SethTisue
Thanks for the advice. Just updated it. Please help me out.

@prolativ
Copy link
Contributor

A slightly simpler minimization:

trait Foo extends Numeric[Any]

It crashes with just sbt clean doc - no package required.

@prolativ prolativ removed the stat:needs minimization Needs a self contained minimization label Aug 30, 2022
@GitZinger
Copy link
Author

GitZinger commented Sep 4, 2022 •
edited
Loading

A slightly simpler minimization:

trait Foo extends Numeric[Any]

It crashes with just sbt clean doc - no package required.

@SethTisue @prolativ
Thanks. Are you looking into it?
or is it already fixed with the next release version?

@prolativ
Copy link
Contributor

prolativ commented Sep 5, 2022

Currently I have other priorities so I cannot give you any guarantee about when this would be fixed. For now this looks like a very strange corner case. I played with this example a bit and didn't manage to make this crash when Numeric gets replaced with some other type

@SethTisue
Copy link
Member

is it already fixed with the next release version?

you can find that out for yourself by trying it in the latest nightly build

@GitZinger
Copy link
Author

GitZinger commented Sep 5, 2022 •
edited
Loading

Currently I have other priorities so I cannot give you any guarantee about when this would be fixed. For now this looks like a very strange corner case. I played with this example a bit and didn't manage to make this crash when Numeric gets replaced with some other type

There are a lot of other types that can cause this too. For instance,

import javax.swing.JPanel

class bug2 extends JPanel:

end bug2

and with sbt clean doc
this will cause

scala-3.2.0-RC4/api...
Problem parsing src/main/scala/Bug2.scala:<0..65>, documentation may not be generated.
java.lang.AssertionError: assertion failed
  | => rat scala.runtime.Scala3RunTime$.assertFailed(Scala3RunTime.scala:11)
	at dotty.tools.dotc.util.Spans$Span$.start$extension(Spans.scala:45)
	at dotty.tools.dotc.util.SourcePosition.start(SourcePosition.scala:52)
	at scala.quoted.runtime.impl.QuotesImpl$reflect$PositionMethods$.start(QuotesImpl.scala:2813)
	at scala.quoted.runtime.impl.QuotesImpl$reflect$PositionMethods$.start(QuotesImpl.scala:2813)
	at dotty.tools.scaladoc.tasty.ClassLikeSupport.getParentsAsTreeSymbolTuples$$anonfun$1(ClassLikeSupport.scala:260)
	at scala.collection.Iterator$$anon$6.hasNext(Iterator.scala:472)
	at scala.collection.Iterator$$anon$9.hasNext(Iterator.scala:576)
	at scala.collection.immutable.List.prependedAll(List.scala:152)
	at scala.collection.immutable.List$.from(List.scala:684)
	at scala.collection.immutable.List$.from(List.scala:681)
	at scala.collection.IterableOps$WithFilter.map(Iterable.scala:891)
	at dotty.tools.scaladoc.tasty.ClassLikeSupport.getParentsAsTreeSymbolTuples(ClassLikeSupport.scala:264)
	at dotty.tools.scaladoc.tasty.ClassLikeSupport.getParentsAsTreeSymbolTuples$(ClassLikeSupport.scala:15)
	at dotty.tools.scaladoc.tasty.TastyParser.getParentsAsTreeSymbolTuples(TastyParser.scala:169)
	at dotty.tools.scaladoc.tasty.ClassLikeSupport.getParentsAsLinkToTypes(ClassLikeSupport.scala:253)
	at dotty.tools.scaladoc.tasty.ClassLikeSupport.getParentsAsLinkToTypes$(ClassLikeSupport.scala:15)
	at dotty.tools.scaladoc.tasty.TastyParser.getParentsAsLinkToTypes(TastyParser.scala:169)
	at dotty.tools.scaladoc.tasty.ClassLikeSupport.mkClass(ClassLikeSupport.scala:106)
	at dotty.tools.scaladoc.tasty.ClassLikeSupport.mkClass$(ClassLikeSupport.scala:15)
	at dotty.tools.scaladoc.tasty.TastyParser.mkClass(TastyParser.scala:169)
	at dotty.tools.scaladoc.tasty.ClassLikeSupport.parseClasslike(ClassLikeSupport.scala:290)
	at dotty.tools.scaladoc.tasty.ClassLikeSupport.parseClasslike$(ClassLikeSupport.scala:15)
	at dotty.tools.scaladoc.tasty.TastyParser.parseClasslike(TastyParser.scala:169)
	at dotty.tools.scaladoc.tasty.ClassLikeSupport.parseInheritedMember$$anonfun$1(ClassLikeSupport.scala:190)
	at dotty.tools.scaladoc.tasty.TastyParser.processTreeOpt(TastyParser.scala:204)
	at dotty.tools.scaladoc.tasty.ClassLikeSupport.parseInheritedMember(ClassLikeSupport.scala:202)
	at dotty.tools.scaladoc.tasty.ClassLikeSupport.extractMembers$$anonfun$2(ClassLikeSupport.scala:221)
	at scala.collection.immutable.List.flatMap(List.scala:293)
	at dotty.tools.scaladoc.tasty.ClassLikeSupport.extractMembers(ClassLikeSupport.scala:221)
	at dotty.tools.scaladoc.tasty.ClassLikeSupport.extractMembers$(ClassLikeSupport.scala:15)
	at dotty.tools.scaladoc.tasty.TastyParser.extractMembers(TastyParser.scala:169)
	at dotty.tools.scaladoc.tasty.ClassLikeSupport.extractPatchedMembers(ClassLikeSupport.scala:226)
	at dotty.tools.scaladoc.tasty.ClassLikeSupport.extractPatchedMembers$(ClassLikeSupport.scala:15)
	at dotty.tools.scaladoc.tasty.TastyParser.extractPatchedMembers(TastyParser.scala:169)
	at dotty.tools.scaladoc.tasty.ClassLikeSupport.mkClass(ClassLikeSupport.scala:113)
	at dotty.tools.scaladoc.tasty.ClassLikeSupport.mkClass$(ClassLikeSupport.scala:15)
	at dotty.tools.scaladoc.tasty.TastyParser.mkClass(TastyParser.scala:169)
	at dotty.tools.scaladoc.tasty.ClassLikeSupport.parseClasslike(ClassLikeSupport.scala:290)
	at dotty.tools.scaladoc.tasty.ClassLikeSupport.parseClasslike$(ClassLikeSupport.scala:15)
	at dotty.tools.scaladoc.tasty.TastyParser.parseClasslike(TastyParser.scala:169)
	at dotty.tools.scaladoc.tasty.TastyParser$Traverser$2$.traverseTree(TastyParser.scala:223)
	at scala.quoted.Quotes$reflectModule$TreeTraverser.foldTree(Quotes.scala:4666)
	at scala.quoted.Quotes$reflectModule$TreeTraverser.foldTree$(Quotes.scala:4662)
	at dotty.tools.scaladoc.tasty.TastyParser$Traverser$2$.foldTree(TastyParser.scala:211)
	at dotty.tools.scaladoc.tasty.TastyParser$Traverser$2$.foldTree(TastyParser.scala:211)
	at scala.quoted.Quotes$reflectModule$TreeAccumulator.foldTrees$$anonfun$1(Quotes.scala:4554)
	at scala.collection.LinearSeqOps.foldLeft(LinearSeq.scala:169)
	at scala.collection.LinearSeqOps.foldLeft$(LinearSeq.scala:165)
	at scala.collection.immutable.List.foldLeft(List.scala:79)
	at scala.quoted.Quotes$reflectModule$TreeAccumulator.foldTrees(Quotes.scala:4554)
	at scala.quoted.Quotes$reflectModule$TreeAccumulator.foldTrees$(Quotes.scala:4549)
	at dotty.tools.scaladoc.tasty.TastyParser$Traverser$2$.foldTrees(TastyParser.scala:211)
	at scala.quoted.Quotes$reflectModule$TreeAccumulator.foldOverTree(Quotes.scala:4617)
	at scala.quoted.Quotes$reflectModule$TreeAccumulator.foldOverTree$(Quotes.scala:4549)
	at dotty.tools.scaladoc.tasty.TastyParser$Traverser$2$.foldOverTree(TastyParser.scala:211)
	at scala.quoted.Quotes$reflectModule$TreeTraverser.traverseTreeChildren(Quotes.scala:4668)
	at scala.quoted.Quotes$reflectModule$TreeTraverser.traverseTreeChildren$(Quotes.scala:4662)
	at dotty.tools.scaladoc.tasty.TastyParser$Traverser$2$.traverseTreeChildren(TastyParser.scala:211)
	at scala.quoted.Quotes$reflectModule$TreeTraverser.traverseTree(Quotes.scala:4664)
	at scala.quoted.Quotes$reflectModule$TreeTraverser.traverseTree$(Quotes.scala:4662)
	at dotty.tools.scaladoc.tasty.TastyParser$Traverser$2$.traverseTree(TastyParser.scala:219)
	at dotty.tools.scaladoc.tasty.TastyParser.parseRootTree(TastyParser.scala:228)
	at dotty.tools.scaladoc.tasty.ScaladocTastyInspector.$anonfun$4(TastyParser.scala:123)
	at scala.runtime.function.JProcedure1.apply(JProcedure1.java:15)
	at scala.runtime.function.JProcedure1.apply(JProcedure1.java:10)
	at dotty.tools.scaladoc.tasty.ScaladocTastyInspector.postProcess$$anonfun$2(TastyParser.scala:42)
	at scala.runtime.function.JProcedure1.apply(JProcedure1.java:15)
	at scala.runtime.function.JProcedure1.apply(JProcedure1.java:10)
	at scala.collection.immutable.List.foreach(List.scala:333)
	at dotty.tools.scaladoc.tasty.ScaladocTastyInspector.postProcess(TastyParser.scala:42)
	at scala.tasty.inspector.OldTastyInspector$TastyInspectorFinishPhase$1.runOn(OldTastyInspector.scala:91)
	at dotty.tools.dotc.Run.runPhases$1$$anonfun$1(Run.scala:234)
	at scala.runtime.function.JProcedure1.apply(JProcedure1.java:15)
	at scala.runtime.function.JProcedure1.apply(JProcedure1.java:10)
	at scala.collection.ArrayOps$.foreach$extension(ArrayOps.scala:1328)
	at dotty.tools.dotc.Run.runPhases$1(Run.scala:245)
	at dotty.tools.dotc.Run.compileUnits$$anonfun$1(Run.scala:253)
	at dotty.tools.dotc.Run.compileUnits$$anonfun$adapted$1(Run.scala:262)
	at dotty.tools.dotc.util.Stats$.maybeMonitored(Stats.scala:68)
	at dotty.tools.dotc.Run.compileUnits(Run.scala:262)
	at dotty.tools.dotc.Run.compileUnits(Run.scala:192)
	at dotty.tools.dotc.fromtasty.TASTYRun.compile(TASTYRun.scala:14)
	at dotty.tools.dotc.Driver.doCompile(Driver.scala:35)
	at dotty.tools.dotc.Driver.process(Driver.scala:195)
	at scala.tasty.inspector.OldTastyInspector.inspectFilesInContext(OldTastyInspector.scala:72)
	at scala.tasty.inspector.OldTastyInspector.inspectFilesInContext$(OldTastyInspector.scala:22)
	at scala.tasty.inspector.DocTastyInspector.inspectFilesInContext(DocTastyInspector.scala:5)
	at dotty.tools.scaladoc.tasty.ScaladocTastyInspector.result(TastyParser.scala:147)
	at dotty.tools.scaladoc.ScalaModuleProvider$.mkModule(ScalaModuleProvider.scala:11)
	at dotty.tools.scaladoc.Scaladoc$.run(Scaladoc.scala:230)
	at dotty.tools.scaladoc.Scaladoc$.run$$anonfun$1(Scaladoc.scala:72)
	at scala.runtime.function.JProcedure1.apply(JProcedure1.java:15)
	at scala.runtime.function.JProcedure1.apply(JProcedure1.java:10)
	at scala.Option.map(Option.scala:242)
	at dotty.tools.scaladoc.Scaladoc$.run(Scaladoc.scala:76)
	at dotty.tools.dottydoc.Main$.process(Main.scala:25)
	at dotty.tools.dottydoc.Main.process(Main.scala)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at xsbt.DottydocRunner.run(DottydocRunner.java:61)
	at xsbt.ScaladocInterface.run(ScaladocInterface.java:11)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at sbt.internal.inc.AnalyzingCompiler.invoke(AnalyzingCompiler.scala:329)
	at sbt.internal.inc.AnalyzingCompiler.doc(AnalyzingCompiler.scala:175)
	at sbt.internal.inc.AnalyzingCompiler.doc(AnalyzingCompiler.scala:133)
	at sbt.Doc$.$anonfun$scaladoc$1(Doc.scala:52)
	at sbt.Doc$.$anonfun$scaladoc$1$adapted(Doc.scala:40)
	at sbt.RawCompileLike$.$anonfun$prepare$1(RawCompileLike.scala:79)
	at sbt.RawCompileLike$.$anonfun$prepare$1$adapted(RawCompileLike.scala:72)
	at sbt.RawCompileLike$.$anonfun$cached$4(RawCompileLike.scala:63)
	at sbt.RawCompileLike$.$anonfun$cached$4$adapted(RawCompileLike.scala:61)
	at sbt.util.Tracked$.$anonfun$inputChangedW$1(Tracked.scala:219)
	at sbt.RawCompileLike$.$anonfun$cached$1(RawCompileLike.scala:68)
	at sbt.RawCompileLike$.$anonfun$cached$1$adapted(RawCompileLike.scala:52)
	at sbt.Defaults$.$anonfun$docTaskSettings$4(Defaults.scala:2157)
	at scala.Function1.$anonfun$compose$1(Function1.scala:49)
	at sbt.internal.util.$tilde$greater.$anonfun$$u2219$1(TypeFunctions.scala:62)
	at sbt.std.Transform$$anon$4.work(Transform.scala:68)
	at sbt.Execute.$anonfun$submit$2(Execute.scala:282)
	at sbt.internal.util.ErrorHandling$.wideConvert(ErrorHandling.scala:23)
	at sbt.Execute.work(Execute.scala:291)
	at sbt.Execute.$anonfun$submit$1(Execute.scala:282)
	at sbt.ConcurrentRestrictions$$anon$4.$anonfun$submitValid$1(ConcurrentRestrictions.scala:265)
	at sbt.CompletionService$$anon$2.call(CompletionService.scala:64)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:833)
[info] Main Scala API documentation successful.

Yeah there are more..
@prolativ I really appreciate it. Take time for your high priority stuff. Thanks a lot

@SethTisue
Copy link
Member

One thing I wonder is if the parent must be Java-defined to trigger the bug.

Not every Java-defined parent triggers it, though (for example, the problem goes away if you change JPanel to JFrame).

@anatoliykmetyuk might be a good spree ticket?

@anatoliykmetyuk anatoliykmetyuk added the Spree Suitable for a future Spree label Sep 6, 2022
@jgongo jgongo mentioned this issue Oct 13, 2022
Closed
@SethTisue SethTisue mentioned this issue Nov 11, 2022
Closed
@SethTisue
Copy link
Member

reported again at #16290 (on extends Iterator[String])

@godenji
Copy link

godenji commented Dec 22, 2022

I'm seeing this as well, with traits/abstract classes extending java.io.Closeable.

Same stacktrace as above:

Scala 3.2.2-RC1
sbt 1.8.0
OpenJDK 19

@SethTisue SethTisue mentioned this issue Jan 5, 2023
Closed
@SethTisue
Copy link
Member

SethTisue commented Jan 5, 2023 •
edited
Loading

reported again at #16546 and again at #16547

maybe duplicate, maybe not: #16180

@SethTisue SethTisue mentioned this issue Jan 5, 2023
Closed
@scala-center-bot
Copy link

This issue was picked for the Issue Spree No. 25 of 24 January 2023 which takes place in a week from now. @SethTisue, @jan-pieter, @yzia2000 will be working on it. If you have any insight into the issue or guidance on how to fix it, please leave it here.

@SethTisue
Copy link
Member

When preparing for a spree, one tries to strike a balance between being unprepared, and over-preparing in the sense that you figure out what's going on before the spree has even started :-)

I wasn't sure how the Scaladoc tool was tested, and there didn't seem to be a relevant directory under tests.

so I searched recently merged PRs for "Scaladoc": https://github.com/lampepfl/dotty/pulls?q=is%3Apr+sort%3Aupdated-desc+is%3Amerged+scaladoc since presumably these PRs would touch test cases

and indeed, that led me to discover the scaladoc-testcases directory, which is at the top level rather than under tests

then I wondered how they were run, and git grep scaladoc-testcases led me to build.sbt where there's a scaladoc-testcases subproject. I tried scaladoc-testcases/test but that didn't do anything. hmm, what to try next?

after further poking around I found scaladoc/README.md, but it still wasn't absolutely clear to me, after reading that, how the scaladoc-testcases functions. I thought I'd look some more at a recent PR that did a bug fix to the Scaladoc tool, so I looked at #14810 and found an exchange between Julien and Quentin (@Sporarum) where Quentin says:

This works by comparing the text of the file with the output of scala-meta, similarly to what is done in the compiler with the tests/pos, tests/neg folders. This is therefore both the declaration and the specification for the tests !

wow, cool! but still not clear to me how to run the tests?

as a guess, maybe scaladoc/test? that seems promising; the output contains some references to the scaladoc-testcases directory

so I tried creating scaladoc-testcases/src/tests/numeric.scala containing trait Foo extends Numeric[Any] , hoping scaladoc/test would fail... but it didn't

another thing that git grep scaladoc-testcases turned up was the existence of project/scripts/cmdScaladocTests, so I tried running that script but it didn't fail 🤔

perhaps I'm expected to somehow tell the testing stuff that I added a file, maybe files in scaladoc-testcases/src/tests aren't picked up automatically? I tried git greping for the names of some of the source files there and I did find that they seem to explicitly be mentioned in source files under scaladoc/test/dotty/tools/scaladoc

but it's not clear to me in this case where the right place to add my new file would be. maybe to scaladoc/test/dotty/tools/scaladoc/signatures/TranslatableSignaturesTestCases.scala, again based on looking at what was done in #14810

so at the end of that file I added:

class Numeric extends SignatureTest("numeric", SignatureTest.all)

and I ran scaladoc/testOnly *scaladoc.signatures.Numeric

and it said "The test name is incorrect or scaladoc-testcases were not recompiled"

I looked at another test file in the same directory and I noticed it started with a package declaration, so I tried changing numeric.scala to include one too:

package tests
package numeric

trait Foo extends Numeric[Any]

and lo and behold, now when I run scaladoc/testOnly *scaladoc.signatures.Numeric, it fails with "Problem parsing scaladoc-testcases/src/tests/numeric.scala:[0..8..61], documentation may not be generated", which is the desired mode of failure

and I also verified that if I changed the test file to something harmless like extends AnyRef, that it doesn't fail

so, I think at this point we know one way to add a test for this — maybe not the best way, but enough to proceed with attempting a fix at the spree

@jan-pieter it looks like you have one previous contribution here, and @yzia2000 it looks like it's your first time? though I might be mistaken? if I'm right, we should let @yzia2000 drive to make sure he gets up to speed on basics of contributing

yzia2000 added a commit to yzia2000/scala3 that referenced this issue Jan 24, 2023
@yzia2000
Added jpath check to ClassLikeSupport getParentsAsTreeSymbolTuples
646546a 
Fixes scala#15927

Check for whether the java parent exists before checking the start
and end of the span to confirm whether the span exists in
getParentsAsTreeSymbolTuples.
@yzia2000 yzia2000 mentioned this issue Jan 24, 2023
Merged
@SethTisue
Copy link
Member

SethTisue commented Jan 24, 2023 •
edited
Loading

We believe the problem is in ClassLikeSupport.scala, in this line:

        parentTree <- c.parents if parentTree.pos.start != parentTree.pos.end // We assume here that order is correct

if .pos doesn't exist (is NoSpan), call .start and .end blow up

What we want is to call .exists first, but .exists doesn't exist in the PositionMethods trait in Quotes.scala. It should exist, but we can't add it until 3.4, since 3.3.0-RC1 is already imminent.

In the long run, we should add probably add .exists to the API, but it would be nice to have a shorter-term fix in the meantime since 3.4 is a ways off and this bug is being frequently reported.

@SethTisue
Copy link
Member

Caveat: not crashing isn't enough, we also need to verify that the parent still shows up in the generated Scaladoc.

@SethTisue
Copy link
Member

SethTisue commented Jan 24, 2023 •
edited
Loading

One thing I wonder is if the parent must be Java-defined to trigger the bug

No, because for example Numericand Iterator aren't Java-defined, they're Scala-2-defined (in the Scala 2.13 standard library). But I don't think we have a failure example where the parent is Scala-3-defined.

@SethTisue
Copy link
Member

in the sbt build, scaladoc/run is how we can manually run Scaladoc against a TASTy file — I hadn't found that before

yzia2000 added a commit to yzia2000/scala3 that referenced this issue Jan 24, 2023
@yzia2000
Added jpath check to ClassLikeSupport getParentsAsTreeSymbolTuples
60747c3 
Fixes scala#15927

Check for whether the java parent exists before checking the start
and end of the span to confirm whether the span exists in
getParentsAsTreeSymbolTuples.
yzia2000 added a commit to yzia2000/scala3 that referenced this issue Jan 24, 2023
@yzia2000
Added jpath check to ClassLikeSupport getParentsAsTreeSymbolTuples
1978bcb 
Fixes scala#15927

Check for whether the non-scala3 parent exists before checking the start
and end of the span to confirm whether the span exists in
getParentsAsTreeSymbolTuples.
jan-pieter added a commit to jan-pieter/dotty that referenced this issue Jan 24, 2023
@jan-pieter
Extend the PositionMethods with an exists method, since the other met…
ea906aa 
…hods are not safe to call on non-existing Spans.

Fixes scala#15927
@SethTisue
Copy link
Member

SethTisue commented Jan 24, 2023 •
edited
Loading

#16759 is the short-term fix which we can backport to the 3.3.x series

#16761 is the better long-term fix

jan-pieter added a commit to jan-pieter/dotty that referenced this issue Jan 24, 2023
@jan-pieter
Extend the PositionMethods with an exists method, since the other met…
030b726 
…hods are not safe to call on non-existing Spans.

Fixes scala#15927
@jan-pieter jan-pieter mentioned this issue Jan 24, 2023
Closed
@SethTisue SethTisue mentioned this issue Jan 26, 2023
Merged
@julienrf
Copy link
Contributor

Minor comment about wording: would isDefined: Boolean be more consistent with the standard library? (like isDefined on Option. exists usually takes a predicate as a parameter).

@SethTisue
Copy link
Member

SethTisue commented Jan 26, 2023 •
edited
Loading

Minor comment about wording: would isDefined: Boolean be more consistent with the standard library

good comment/question, but: .exists is widely used within the compiler (namely, on symbols and types), and it's probably more important to be consistent with that, especially since this isn't a brand new method; we're only making an existing method visible via an interface

@SethTisue SethTisue changed the title Problem parsing xxx.scala, documentation may not be generated. Scaladoc generation fails when extending some Java-defined or Scala-2-defined classes Jan 31, 2023
@Sporarum
Copy link
Contributor

Sporarum commented Feb 1, 2023

@SethTisue

wow, cool! but still not clear to me how to run the tests?

I lived through the same, so this seems to indicate some documentation is missing

I had actually isolated a commit to make it only about setting up the test: 5671d26

Something that pointed in the right direction was the old behaviour of running test in dotty, it used to say something like:

Running all the tests takes a very long time, it's probably not what you want to do, instead you can use:
testCompilation - runs all compile tests
someWayToStartTheScaladocTests - runs all scaladoc tests
...

I don't know why the behavior was changed

I do not have time at the moment, do you think you could investigate what makes it hard to start the scaladoc tests, and address it ?

(Probably through creating some documentation in the readme or in https://docs.scala-lang.org/scala3/guides/contribution/procedures-intro.html, in particular https://docs.scala-lang.org/scala3/guides/contribution/procedures-testing.html does not mention the commands to test other things like scaladoc)

@nicolasstucki
Copy link
Contributor

The tree that has a non-existent position should have a position.

@nicolasstucki
Copy link
Contributor

The way to debug this is to set -Ydebug-tree-with-id -2 and figure out the uniqueId of the tree that does not have a position. If the id is 234, re-run with -Ydebug-tree-with-id 234. This will print the stack trace of the location where the tree was created. If it is part of a tree copy, we need to figure out the id of the original tree without a position and repeat. There is probably a tree for which we are forgetting to set the span (withSpan).

@nicolasstucki
Copy link
Contributor

It seems that the problematic trees are generated here:
https://github.com/lampepfl/dotty/blob/main/compiler/src/dotty/tools/dotc/quoted/reflect/FromSymbol.scala#L33

We should probably set the span of those trees to cls.span.

@SethTisue
Copy link
Member

I lived through the same, so this seems to indicate some documentation is missing

(working on it over at #16769)

@Kordyjan Kordyjan added this to the 3.3.0 backports milestone Feb 2, 2023
Kordyjan added a commit that referenced this issue Feb 2, 2023
@Kordyjan
Fix Scaladoc crash when extending non-Scala-3 classes (short-term fix…
04f5a0b 
… for 3.3.x) (#16759)

Added jpath check to `ClassLikeSupport` getParentsAsTreeSymbolTuples
— check for whether the non-scala3 parent exists before checking the
start and end of the span to confirm whether the span exists in
getParentsAsTreeSymbolTuples.

Fixes #15927
@SethTisue
Copy link
Member

Thanks @nicolasstucki for digging into the underlying cause. I'll get back to it (but not right away).

Kordyjan pushed a commit to dotty-staging/dotty that referenced this issue Feb 15, 2023
@yzia2000 @Kordyjan
Added jpath check to ClassLikeSupport getParentsAsTreeSymbolTuples
c188f1d 
Fixes scala#15927

Check for whether the non-scala3 parent exists before checking the start
and end of the span to confirm whether the span exists in
getParentsAsTreeSymbolTuples.
@Kordyjan Kordyjan modified the milestones: 3.3.0 backports, 3.3.1 Aug 2, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@SethTisue SethTisue

@jan-pieter jan-pieter

Labels
area:doctool itype:bug itype:crash Spree Suitable for a future Spree
Projects
None yet
Milestone
3.3.1