Pre-SIP: Improvements to Type Classes #19395

odersky · 2024-01-08T13:10:59Z

Various proposed improvements to how Scala handles generic programming and type classes. They are summarized in this doc page:

https://github.com/dotty-staging/dotty/blob/typeclass-experiments/docs/_docs/reference/experimental/typeclasses.md

This PR is based on #18958.

jducoeur · 2024-01-08T14:06:34Z

Oooh, I like it. I can quibble with a detail or two, but overall, from the user (and teacher) POV these look like major wins...

docs/_docs/reference/experimental/typeclasses.md

odersky · 2024-01-08T16:41:05Z

@ritschwumm @dwijnand Thanks for reporting the typos!

bishabosha · 2024-01-09T13:52:35Z

tests/pos/hylolib/Collection.scala

+    * @complexity
+    *   O(1)
+    */
+  def headAndTail: Option[(Self.Element, Slice[Self])] =


just a cool observation: so this is type member selection on a synthetic Collection term parameter called Self, i.e. not the Self type, very elegant

arturopala · 2024-01-09T22:33:07Z

what about making it even more clear and concise by replacing

trait Value:
  type Self

with just one-liner:

typeclass Value:

?

odersky · 2024-01-09T22:53:13Z

That's not clearer, and requires more explanations. Where does the Self come from?

arturopala · 2024-01-09T23:06:20Z

That's not clearer, and requires more explanations. Where does the Self come from?

It comes straight from your proposal stated in docs/_docs/reference/experimental/typeclasses.md; if it is a type class, it has a type member named Self, so why type it all over and again?

Proposal: Allow type classes to define an abstract type member named Self instead of a type parameter.

odersky · 2024-01-09T23:26:07Z

My proposal is irrelevant for people reading the code. typeclass is just one more thing to learn, with no associated benefit.

arturopala · 2024-01-10T00:09:35Z

If someone changes a type member name from Self to any arbitrary, would the trait still behave the same way?

bishabosha · 2024-01-10T07:56:01Z

If someone changes a type member name from Self to any arbitrary, would the trait still behave the same way?

no, the context bound desugaring requires a member called Self (or the old way with a type parameter)

arturopala · 2024-01-10T09:03:44Z

@bishabosha, so this is still something non-trivial to learn, and IMHO, the separate (soft) keyword would convey the intention better than the trait + Self type member name fragile convention. I remember Martin long arguing about the new givens declaring intention vs. the low-level nature of the old implicits. Does the same applies here?

bmeesters · 2024-01-10T09:04:53Z

I think it is great to look at how type classes can be improved. Here are some of my (unasked) two cents:

Might be good to start a contributors topic on this. Type classes are used a lot and I think there might be ideas or frustrations that could be missed now.
I have to get used to replacing the type parameter with a type member instead, but it feels quite Scala like. I wonder though how we could get people to use this syntax instead of the current syntax. Since traits with type parameters are still valid it cannot be deprecated, and might be harder to move people over if the compiler doesn't give any hints on what is preferred. Having multiple ways to do the same thing is in my experience what people consider complex about Scala.
I think changing abstract givens to deferred it fine. But I had to look in the tests to confirm whether my intuition on how would instantiate it was correct. Could be good to add that to the docs.
Nitpick but I find given Int is FromString = _.toInt a little weird to read. I think given Int has FromString = _.toInt would be more clear. Often when talking about type classes I find people talk about having an instance, not being an instance.
There are some nice QoL improvements there like type bounds on abstract type definitions, auto naming the instance that I think are definitely worth it, even without the new syntax.

bishabosha · 2024-01-10T09:25:44Z

@bishabosha, so this is still something non-trivial to learn, and IMHO, the separate (soft) keyword would convey the intention better than the trait + Self type member name fragile convention. I remember Martin long arguing about the new givens declaring intention vs. the low-level nature of the old implicits. Does the same applies here?

This is likely to be highly subjective, but in my opinion, "a trait with a Self type" is more simple than a new category of things, because all the other components that make it behave as a type class are explained in terms of other things you may already know. "what is Int is Show?" - click on is in the IDE and it tells you its Show trait where Self is refined, then you can go to Show to find the Self definition. If you introduce typeclass then this is all opaque or become builtins.

I guess you could say the same about the context bound syntax being opaque at first glance, but then you will see the explanation of desugaring to real things that you can see.

arturopala · 2024-01-10T09:30:22Z

@bishabosha Really? IMHO "a trait with a Self type" is becoming a totally new category of things with this PR, not just "a trait with a Self type"

odersky · 2024-01-10T09:41:15Z

Imagine the ridicule: Only scala has "types" and "classes" and "typeclasses", but in fact "typeclasses" are neither types nor classes, they are traits! No, that won't fly.

arturopala · 2024-01-10T09:55:06Z

@odersky you can say the same about guinea pig, neither pig nor from Guinea and can't fly, right, but people like them!

bishabosha · 2024-01-10T11:26:45Z

IMHO "a trait with a Self type" is becoming a totally new category of things with this PR, not just "a trait with a Self type"

The Self type you can think of as a "convention" - i.e. it's nothing magic inherently, there is some help in the standard library (is type alias) to make it more ergonomic to follow the convention. Context bounds you could say are what makes it magical, but then again it is only a convenience, they are not necessary.

If you bake in the typeclass then all this stuff becomes necessary. So this is what is meant by "simple" - its compositional

sasha2048 · 2024-01-10T11:51:58Z

There are many good things in this proposal. But, to be honest, I would rather prefer one of the following things instead of type Self:

Variant A:
```
def f[T: Requirement[X, Y]] = ???
```
is automatically desugared into
```
def f[T](using T: Requirement[T, X, Y]) = ???
```
when Requirement[T] is not available (but Requirement[T, …] is).

Variant B:

Possibility to declare a trait (or class, case class, enum, etc) like:

trait Requirement[X, Y][T] {…}

instead of

trait PreRequirement[T, X, Y] {…}
type Requirement[X, Y] = [T] =>> PreRequirement[T, X, Y]

There also could be additional sugaring (inspired by arturopala): type trait Requirement[X, Y] {…} (which would be automatically desugared either into trait Requirement[Self, X, Y] {…} or into trait Requirement[X, Y][Self] {…}, depending of whether we choose A or B) – however that's non-primary thing. (P.S.: There could be even type[S] trait Requirement[X, Y] {…}, which would give explicit name S instead of Self – that would partially resemble extension syntax.)

lihaoyi · 2024-01-10T14:39:54Z

I have to agree with the comment that is is a bit weird and goes against how typeclasses are generally named. has is a lot better:

given Int is upickle.default.ReadWriter vs given Int has upickle.default.ReadWriter

given String is mainargs.TokensReader vs given String has mainargs.TokensReader

given Boolean is scalasql.TypeMapper vs given Boolean has scalasql.TypeMapper

lihaoyi · 2024-01-10T14:54:36Z

Although given that the whole point of type Self is syntactic sugar, should we provide sugar for Foo{type Self = Bar} as well, rather than be limited by what we can implement in user-land?

e.g. what if we say Bar: Foo is the general syntax for typeclasses, such that [T: Foo] is how you refer to typeclasses for generic parameters, but (given ev: Bar: Foo) or (given: Bar: Foo) is how you refer to typeclass parameters with concrete types?

odersky · 2024-01-10T17:15:52Z

I hesitated a lot between is and has. Both are appropriate in some circumstances and less so in others. It also depends on the type class name.

For instance, here is is better

    Int is Monoid
    List is Monad
    Int is Ordered
    List[T] is Collection
    Int is Readable
    String is Writable

But here has is better:

   Int has upickle.default.ReadWriter 
   String has mainargs.TokensReader
   Boolean has scalasql.TypeMapper 
   Int has Ordering

On the other hand, the type class is relationship is one of extension. It's meant to establish the same kind of relationship as extends, but without the restriction that all the traits extended by a class have to be given in the class itself. Also, the type Self strongly suggests is. One could argue that for instance in the

   Int is upickle.default.ReadWriter

example, it's also weird if the ReaderWriter was expanded like this:

  ReadWriter { type Self = Int }

Self suggests its a kind of ReadWriter, which Int certainly is not.

So in summary I think that is and Self go together well, and also fit well with the idea of typeclasses as ad-hoc extensions. (Swift even uses the extension keyword for this). If the name of a prospective typeclass does not fit that mold, then maybe it's better to leave it as a parameterized class. A ReadWriter[T] makes perfect sense. But then we don't need is for these traits anyway.

bishabosha · 2024-01-10T21:53:18Z

I've had another thought which is how this will interact with derives? as far as I know this has some fairly delicate rules that we managed to distill in https://docs.scala-lang.org/scala3/reference/contextual/derivation.html#exact-mechanism-1

joroKr21 · 2024-01-10T22:07:55Z

For instance, here is is better

If we are talking about the classic typeclasses borrowed from category theory I like forms the most:

    Int forms Monoid
    List forms Monad

Monoid for Int is maybe not the best example because we have at least two that are both natural - addition and multiplication.

Well, in fact one could even say Group is a Monoid and Monoid is a Semigroup because here we do have an inheritance relationship. Or Monad is Applicative. Although in this context maybe "implies" is the correct verb.

But the cool thing about this proposal is that if someone feels too strongly about it, they could create the preferred type alias in user space.

P.S. An example where the current ecosystem is breaking the conventions - Foldable vs Traverse. Can't really win, can we 😄

Co-authored-by: Dale Wijnand <[email protected]>

This is a trial balloon to see whether `forms` works better than `this`. My immediate reaction is meh. Sometimes it's OK, at other times I liked `is` better. But I admit there's bias since the examples were chosen to work well with `is`.

This is a trial as a first step for other refactorings down the line.

This if for better understandability only since it avoids the deeply nested return. Also, add a new test showing how to deal with tracked parameters in typeclass arguments.

Shows choice between parameterized and higher-kinded type classes.

That way they can be implemented by operations of the form `given T = f(...)`, which also map to lazy values. And strict abstract values cannot be overridden by lazy values since that could undermine realizability and with it type soundness.

1. Use the constrained type's name as a term name only for single context bounds 2. Apply the same scheme to deferred givens

When expanding a context bound of a member type, don't use the member name as the name of the generated deferred given. The same member type might have different single context bounds in different traits. A class inheriting several of these traits would then get double definitions in its given clauses. Partial revert of "Changes to default names for context bound witnesses"

@main

Consider the following program: ```scala class A class B extends A class C extends A given A = A() given B = B() given C = C() def f(using a: A, b: B, c: C) = println(a.getClass) println(b.getClass) println(c.getClass) @main def Test = f ``` With the current rules, this would fail with an ambiguity error between B and C when trying to synthesize the A parameter. This is a problem without an easy remedy. We can fix this problem by flipping the priority for implicit arguments. Instead of requiring an argument to be most _specific_, we now require it to be most _general_ while still conforming to the formal parameter. There are three justifications for this change, which at first glance seems quite drastic: - It gives us a natural way to deal with inheritance triangles like the one in the code above. Such triangles are quite common. - Intuitively, we want to get the closest possible match between required formal parameter type and synthetisized argument. The "most general" rule provides that. - We already do a crucial part of this. Namely, with current rules we interpolate all type variables in an implicit argument downwards, no matter what their variance is. This makes no sense in theory, but solves hairy problems with contravariant typeclasses like `Comparable`. Instead of this hack, we now do something more principled, by flipping the direction everywhere, preferring general over specific, instead of just flipping contravariant type parameters. The behavior is dependent on the Scala version - Old behavior: up to 3.4 - New behavior: from 3.5, 3.5-migration warns on behavior change The CB builds under the new rules. One fix was needed for a shapeless 3 deriving test. There was a typo: mkInstances instead of mkProductInstances, which previously got healed by accident because of the most specific rule. Also: Don't flip contravariant type arguments for overloading resolution Flipping contravariant type arguments was needed for implicit search where it will be replaced by a more general scheme. But it makes no sense for overloading resolution. For overloading resolution, we want to pick the most specific alternative, analogous to us picking the most specific instantiation when we force a fully defined type. Also: Disable implicit search everywhere for disambiaguation Previously, one disambiguation step missed that, whereas implicits were turned off everywhere else.

Also, fix the unmangling of UniqueExtNames, which seemingly never worked.

Add a config setting whether or not to use the type name for unary context bounds as default name. It's on by default. If it is off, context bound companions are created instead. After fixing several problems, the test suite was verified to compile with the setting set to off.

Align handling of tracked with constructors. This means - Do it in Namer instead of Desugar - Only add tracked to context bound witnesses if they have abstract type members.

odersky · 2024-04-04T08:53:14Z

Superseded by #20061