proposal: Go 2: range-over-function + range over types implementing iterator interface

[chad-bekmezian-snap]

Go Programming Experience

Intermediate

Other Languages Experience

Python, C#, Javascript, Java

Related Idea

• Has this idea, or one like it, been proposed before?
• Does this affect error handling?
• Is this about generics?
• Is this change backward compatible? Breaking the Go 1 compatibility guarantee is a large cost and requires a large benefit

Has this idea, or one like it, been proposed before?

I don't believe so, but it feels likely

Does this affect error handling?

no

Is this about generics?

not really

Proposal

I think many of us Gophers can agree, the experimental range-over-func features, along with the iter package are seriously exciting changes! As I have tested these changes out with the release of Go 1.22 I have found implementation incredibly easy and very much inline with Go's historically readable design. However, I have also found the use of iterator functions somewhat obtuse in some scenarios.

As an example, let's use an ordered map as that's one of the example motivations for this change:

type OrderedMap[K comparable, V any] struct{}

func (m OrderedMap[K, V]) Iterate() iter.Seq2[K, V] {
	return func(yield func(K, V) bool) {
		// ... for each key-value pair ...
		// if !yield(key, value) {
		//     return
		// }
	}
}

func Foo(m OrderedMap[string, Bar]) {
   for k, v := range m.Iterate() {
      // do something with k and v
   }
}

As you can see, I have to call m.Iterate() to actually use the iterator. That's not a huge deal, but the built-in map[K]V has no such limitation. Imo in an ideal world, interacting with my own types should be as simple and clean as interacting with built-in types. Hence this proposal. I propose the following:

1. We introduce 3 new types to the iter package:

type Iterator0 interface{ Iterate() Seq0 }
type Iterator[V any] interface { Iterate() Seq[V] }
type Iterator2[K, V any] interface { Iterate() Seq[K, V] }

2. Go adds a language feature that, in addition to iterator functions, recognizes types that implement any of the 3 interfaces defined above as valid to range over. At compile time, the range function should be resolved/treated as if it were written out as a fully qualified function call to the type's Iterator.Iterate() method.

This change would allow for my previous OrderedMap for loop to be written:

func Foo(m OrderedMap[string, Bar]) {
   for k, v := range m {
      // do something with k and v
   }
}

While at first glances this may appear to be a trivial reward for the work, I believe this change could make iterators feel incredibly polished, in addition to decluttering code. I admit that when looking at a for ... range loop that iterates over an iter.Iterator, it is less clear what is happening. However, with IDEs and intellisense what they are today, I believe this would be a very small lost in clarity of code.

Note: the only restrictions to this change would be that any type that is already iterable (e.g. slices, arrays, maps, etc...) only ever use their typical behavior on iteration, even if they implement the Iterator interface.

Language Spec Changes

The compiler, static type checker, probably some others

Informal Change

The purpose of this change is to simplify iterating over custom types by allowing any type to implement the Iterator interface, allowing any type to very easily be iterated over in a for ... range loop. It even allows overriding default iteration behavior on a type, such as on type MyType []string by allowing that type to implement the Iterator interface.

Is this change backward compatible?

I believe so.

Orthogonality: How does this change interact or overlap with existing features?

No response

Would this change make Go easier or harder to learn, and why?

It would make it harder, but only marginally so. I think you would have to see this one time, and you would understand it.

Cost Description

No response

Changes to Go ToolChain

No response

Performance Costs

Pretty much no difference

Prototype

No response

[Jorropo]

I'm pretty sure the same arguments have been refused for operator overloading.
In other words, I don't think this makes sense if we don't also allow:

func (T) Add(T) T

To overload the + operator.

[fzipp]

Note: the only restrictions to this change would be that any type that is already iterable (e.g. slices, arrays, maps, etc...) only ever use their typical behavior on iteration, even if they implement the Iterator interface.

I find this exception rule quite undesirable, and it would be something that every developer must additionally be aware of.

By the way, the naming convention for the method returning the sequence of all elements is All(), not Iterate().

[chad-bekmezian-snap]

I'm pretty sure the same arguments have been refused for operator overloading. In other words, I don't think this makes sense if we don't also allow:

func (T) Add(T) T

To overload the + operator.

I’m not sure I would consider these two comparable changes, but I understand the issue you perceive

[chad-bekmezian-snap]

Note: the only restrictions to this change would be that any type that is already iterable (e.g. slices, arrays, maps, etc...) only ever use their typical behavior on iteration, even if they implement the Iterator interface.

I find this exception rule quite undesirable, and it would be something that every developer must additionally be aware of.

By the way, the naming convention for the method returning the sequence of all elements is All(), not Iterate().

I don’t love the exception myself, but it’s necessary to preserve backward compatibility.

I don’t really care if it’s Iterate or All. I chose Iterate because it fits well into Go’s single method interface naming conventions. An interface named Aller feels gross compared to Iterator

[seankhliao]

this appears to just be a rehash of #54245 ?

[earthboundkid]

To be specific, #54245 ran into the problem that it wasn't backwards compatible. If you have type Ints []int and it defines Iterate, then range myInts will change behavior between versions of Go. I don't think its worth opening that can of worms just to avoid doing .All(). Also, doing .All() lets you have other methods, like .Reverse() or .ByReverseCron() or whatever. I think the savings here aren't worth the costs in clarity.

[chad-bekmezian-snap]

this appears to just be a rehash of #54245 ?

Ah, I hadn't seen #54245. I'll give this a little more time, but I'll probably end up closing