Change HK types scheme #731

odersky · 2015-07-22T14:13:56Z

I recently found out that the original scheme for hk types in dotty
that I proposed does not work. Here is a counter example:

trait Iterable[IA, C <: Iterable]
trait Seq[SA, C <: Seq] extends Iterable[A, C]
val s: Seq[Int, Seq] = ???
val i: Iterable[Int, Iterable]

Now, we said that the type C in Iterable has two type parameters, the
IA and the C. So if C <: Iterable, then

C[T, D] would translate to C { type IA = D, type C = D }.

Analogously, if C <: Seq, then

C[T, D] would translate to C { type SA = D, type C = D }

The problem is when we see a C defined to be <: Seq as a subtype of Iterable.
The two versions have the same number parameters, but the fields are different!
We'd be translating then C[T, D] to C { type IA = D, type C = D } which makes no
sense, it would just give us a double binding of IA. It turns out this
cannot be worked around in general; asSeenFrom interacts in fatal ways
with the scheme.

To fix this, we now always use the (formerly backup) scheme of type lambdas, paired
by a new notion of "eta-reduction" to keep types small.

Review by @adriaanm

Avoid forcing info if the reference goes to a class. This avoided a CyclicReference when reading Scala's standard library form pos/collections when fiddling with the hk logic.

In Namer, eta expand any type argument that corresponds to a higher-kinded type parameter. Also, check that all type parameter lists are fully applied.

With the hk-types schem changed, we need to make sure that actual and formal argument lists of parameterized types have the same length.

It used to be "assertion error: NoType" whenever sigName git a projection of a missing member. Now we find out about what type was projected.

Now catches attempts to recursively force a LazyRef type that's in train of being evaluated.

Erreneous programs could have a difference in lengths between type parameters and type args, but this is tested anyway in Typer.

... and move to TypeApplications. isLambda test was the wrong way before.

Without the additional `typeParams.nonEmpty` condition we got a crash in t1439.scala

Previously, we did this only in applications in rhs of type definitions. Need to do it everywhere.

This would lead to a crash. Example is in Predef: object Pair type Pair

EtaReduce will be used to keep applications on eta expanded methods small.

…lass.

parameterizeWith picked between simple hk types and lambda abstraction. No longer needed because now we always lambda abstract.

Needed to avoid cycles involving F-boundes hk-types when reading Scala2 collection classes with new hk-scheme.

When unpickling from Scala2 TypeRefs with arguments which do not refer to classes, use EtaExpand instead of LambdaAbstract. Lambda Abstrct is wrong since it drops type arguments.

New hk-scheme caused cycles in elimExistentials which are fixed by this patch.

Derived types already contain the lambda abstractoion; lambda abstracting them again would cause a double lambda.

Discrepancies between numbers of formal and actual type arguments were observed when typing partialFunctions.scala under new scheme. Should come back to this when subtyping is rewrittem/simplified to work with new hk-scheme. Maybe testLifted is no longer needed at all.

This aligns typeParams and rawTypeParams. It's not strictly to necessary, though.

Rewrite a type application like ([HK$0] => C[HK$0])(T) to C[T] Avoids application cahins to become unnecessarly large.

Used to be just instantiated lambdas. With the new scheme every type with a kind higher than * needs to be projected with #Apply.

Turn a possible NPE into an AssertionError. The latter are caught in pickleTree, so an error leaves a trace about what was pickled.

Previously, only pattern bound arguments were adapated. This was an oversight. Also, change logix so that we survive empty type parameter lists. This was also an oversight before.

Want to have a unique name for Apply, so that tests for higher-kinded types become cheaper.

by removing dead case.

These are not user-accessible types, so no need to follow type convention and write in upper case. Also, rename occurrences of lambda to Lambda, to make clear we mean a type lambda.

It's no longer needed with new hk scheme.

Replace occurrences of isLambda with isHK, because isHK is a bit faster and simplier.

Accidentally forwarded to rawTypeParams. This solved the problem with mismatching type params in appliedTo that was caught in testLifted.

Now also allows to reduce something like ([T] => Map[T, String]) to Map[_, String]

...unless the HK type can be eta-reduced to a class type.

1) Lambda abstract now records bounds of abstracted type parameters in TypeLambda 2) Eta-reduce likewise keeps the bounds it finds in the TypeLambda 3) Eta-reduce now also translates hk$i references to type parameters of the reduced type.

The original IterableSelfRec is not syntactically legal after the hk changes. I attempted to fix, but there's still a type error. Need to investigate whether this is a true error or a bug.

smarter · 2015-07-22T20:49:10Z

src/dotty/tools/dotc/core/TypeApplications.scala

+   *   - `bounds` consists of type declarations `type hk$i >: toHK(L) <: toHK(U),
+   *     one for each type parameter in `T` with non-trivial bounds L,U.
+   *   - `toHK` is a substitution that replaces every bound symbol sym_i by
+   *     `this.Arg$i`.


Should be this.hk$i.

Yes, I fixed the comments in #802

odersky · 2015-09-21T11:42:16Z

Closed in favor of #802

odersky added 30 commits July 16, 2015 17:48

Clean up of logic in typeDefSig

a3f1e56

Avoid cyclic references in containsRefinedThis

b07727b

Avoid forcing info if the reference goes to a class. This avoided a CyclicReference when reading Scala's standard library form pos/collections when fiddling with the hk logic.

Eta expand type arguments corresponding to lambdas

2b75c95

In Namer, eta expand any type argument that corresponds to a higher-kinded type parameter. Also, check that all type parameter lists are fully applied.

Check argument lengths in typedAppliedTypeTree

c11e20d

With the hk-types schem changed, we need to make sure that actual and formal argument lists of parameterized types have the same length.

Better diagnostics for failed sigName calls.

d02d383

It used to be "assertion error: NoType" whenever sigName git a projection of a missing member. Now we find out about what type was projected.

Black hole detection for LazyRefs

6b5ff03

Now catches attempts to recursively force a LazyRef type that's in train of being evaluated.

Turn assertion into a test in etaExpandArgs.

fc8e20a

Erreneous programs could have a difference in lengths between type parameters and type args, but this is tested anyway in Typer.

Fix argument eta expansion

ce831f8

... and move to TypeApplications. isLambda test was the wrong way before.

Strenghten condition in EtaExpandIfLambda

6c992f0

Without the additional `typeParams.nonEmpty` condition we got a crash in t1439.scala

Adapt arguments in all type applications

43377ad

Previously, we did this only in applications in rhs of type definitions. Need to do it everywhere.

Avoid generating companion-methods for non-class types.

35adb64

This would lead to a crash. Example is in Predef: object Pair type Pair

Add EtaReduce method.

38ac13a

EtaReduce will be used to keep applications on eta expanded methods small.

Add comment.

8029f65

Add a test that logs the classpath to pinpoint setup failures.

c5958aa

Don't eta expand in appliedTo

24e3403

Always lambda abstract TempPolyTypes representing types.

5e45dc5

Lambda abstract all TypeRefs when unpickling.

968999a

A non-class typeref has type parameters only if subtype of a lambda c…

d474100

…lass.

Drop parameterizeWith

accbed2

parameterizeWith picked between simple hk types and lambda abstraction. No longer needed because now we always lambda abstract.

Allow to use safe substitution in LambdaAbstract

c2a124f

Needed to avoid cycles involving F-boundes hk-types when reading Scala2 collection classes with new hk-scheme.

Use EtaExpand instead of LambdaAbstract when unpickling typerefs

9bb3f52

When unpickling from Scala2 TypeRefs with arguments which do not refer to classes, use EtaExpand instead of LambdaAbstract. Lambda Abstrct is wrong since it drops type arguments.

Avoid cycles when eliminating existentials in unpickling

8610b4d

New hk-scheme caused cycles in elimExistentials which are fixed by this patch.

Don't lambda abstract derived types

7b751f8

Derived types already contain the lambda abstractoion; lambda abstracting them again would cause a double lambda.

Follow alias types when computing type params.

f679830

This aligns typeParams and rawTypeParams. It's not strictly to necessary, though.

Try to eta reduce type applications.

484afef

Rewrite a type application like ([HK$0] => C[HK$0])(T) to C[T] Avoids application cahins to become unnecessarly large.

Project all high-kinded types with #Apply

5f779f6

Used to be just instantiated lambdas. With the new scheme every type with a kind higher than * needs to be projected with #Apply.

Better error reporting in TreePickler

7cb91c7

Turn a possible NPE into an AssertionError. The latter are caught in pickleTree, so an error leaves a trace about what was pickled.

Adapt type arguments in typedAppliedTypeTree

8b1d320

Previously, only pattern bound arguments were adapated. This was an oversight. Also, change logix so that we survive empty type parameter lists. This was also an oversight before.

Add missing position to error

4b06dea

odersky added 12 commits July 20, 2015 10:36

Rename Apply -> hkApply

e24d164

Want to have a unique name for Apply, so that tests for higher-kinded types become cheaper.

Simplify EtaReduce

7582c0e

by removing dead case.

Rename of hk args HK$x -> hk$x

4a5bf98

These are not user-accessible types, so no need to follow type convention and write in upper case. Also, rename occurrences of lambda to Lambda, to make clear we mean a type lambda.

Remove magic from derivedRefinedType.

974c503

It's no longer needed with new hk scheme.

Replace isLambda with ifHK

7eaa515

Replace occurrences of isLambda with isHK, because isHK is a bit faster and simplier.

Rewrite compareHK to be kind-correct

ab55221

Fix problem in typeParams

03be249

Accidentally forwarded to rawTypeParams. This solved the problem with mismatching type params in appliedTo that was caught in testLifted.

Generalize eta reduction to partially applied types.

9c75b53

Now also allows to reduce something like ([T] => Map[T, String]) to Map[_, String]

Disallow wildcard arguments to higher-kinded types...

fecd591

...unless the HK type can be eta-reduced to a class type.

Avoid orphan RefinedThis types in glb/lub.

fd0fa8e

Move failing test to pending.

887f815

The original IterableSelfRec is not syntactically legal after the hk changes. I attempted to fix, but there's still a type error. Need to investigate whether this is a true error or a bug.

smarter mentioned this pull request Jul 22, 2015

Inference of higher-kinded type parameters is not precise enough #732

Closed

smarter reviewed Jul 22, 2015
View reviewed changes

odersky mentioned this pull request Sep 21, 2015

Change hk take 2 #802

Merged

odersky closed this Sep 21, 2015

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Change HK types scheme #731

Change HK types scheme #731

odersky commented Jul 22, 2015

smarter Jul 22, 2015

odersky Sep 21, 2015

odersky commented Sep 21, 2015

Change HK types scheme #731

Change HK types scheme #731

Conversation

odersky commented Jul 22, 2015

smarter Jul 22, 2015

Choose a reason for hiding this comment

odersky Sep 21, 2015

Choose a reason for hiding this comment

odersky commented Sep 21, 2015