https://xnacly.me/posts/2024/fun-with-iterators/ Google is actively hurting the open web with its browser chromium, read more here. Consider switching to a better browser, such as Firefox~ ~/archive ~/tags ~/about Fun with Go Iterators Oct 7, 2024 * 1207 words * 6 minute read Table of Contents * The Annoyance * The Solution (sort of) * The Logic + Each + Reverse + Map + Filter * Examples and Thoughts Go version 1.23 added iterator support ^1 and the iter package ^2. We can now loop over constants, containers (maps, slices, arrays, strings) and functions. At first I found the iterator creation clunky, while consuming the iterator seems straightforward. My issue with the go way of iterators is, you can't chain them like you would in JavaScript: 1[1,2,3,4] 2 .reverse() 3 .map(e => e*e) 4 .filter(e => e % 2 == 0) 5 .forEach(e => console.log(e)) The Annoyance ## Writing this in go would require us to chain 5 function calls: 1slices.ForEach( 2 slices.Filter( 3 slices.Map( 4 slices.Reverse(slices.All([]int{1,2,3,4})), 5 func(i int) int { return i * i}, 6 ), 7 func(i int) bool { return i % 2 == 0 } 8 ), 9 func(i int) { fmt.Println(i) } 10) This is an example, there are are no Map, Filter or ForEach functions in the slices package ^3. The Solution (sort of) ## Because i have a big distaste for writing chained "functional" operations like this, looking at you python (don't come at me haskell bros) - I wanted to use the new iterators and the iter package and wrap it with a structure to allow the nice and clean chaining JavaScript provides. Below are the same operations, but instead of using the iter and slices package, I use my abstraction: 1func TestIterator(t *testing.T) { 2 From([]int{1, 2, 3, 4}). 3 Reverse(). 4 Map(func(i int) int { return i * i }). 5 Filter(func(i int) bool { return i%2 == 0 }). 6 Each(func(a int) { println(a) }) 7 // 16 8 // 4 9} The Logic ## Lets take a look a the implementation and let me introduce the Iteratorstruct. It wraps the iterator (*Iterator).iter and thus allows me to callfunctions on this structure, instead of having to use each iterator functionas a param to the next one. 1type Iterator[V any] struct { 2 iter iter.Seq[V] 3} Lets take a look at the first functions coming to mind when talking about iterators, creating one from a slice and collection one into a slice: 1func (i Iterator[V]) Collect() []V { 2 collect := make([]V, 0) 3 for e := range i.iter { 4 collect = append(collect, e) 5 } 6 return collect 7} 8 9func From[V any](slice []V) *Iterator[V] { 10 return &Iterator[V]{ 11 iter: func(yield func(V) bool) { 12 for _, v := range slice { 13 if !yield(v) { 14 return 15 } 16 } 17 }, 18 } 19} The first one is as straight forward as possible - create a slice, consume the iterator, append each element, return the slice. The second highlights the weird way iterators are created in go. Lets first take a look at the signature, we are returning a pointer to the struct, so the callee can invoke all methods without having to use a temporary variable for each. In the function itself, the iterator is created by returning a closure, that loops over the param and returning, which stops the iterator, when the yield function returns false. Each ### The ForEach / Each method is the next function I want, It simply executes the passed in function for every element of the iterator. 1func (i *Iterator[V]) Each(f func(V)) { 2 for i := range i.iter { 3 f(i) 4 } 5} It can be used like this: 1From([]int{1, 2, 3, 4}).Each(func(a int) { println(a) }) 2// 1 3// 2 4// 3 5// 4 Reverse ### A way to reverse the iterator, we first need to collect all elements and after that construct a new iterator from the collected slice, luckily we have functions for exactly this: 1func (i *Iterator[V]) Reverse() *Iterator[V] { 2 collect := i.Collect() 3 counter := len(collect) - 1 4 for e := range i.iter { 5 collect[counter] = e 6 counter-- 7 } 8 return From(collect) 9} Again, useful to reverse a slice: 1From([]int{1, 2, 3, 4}).Reverse().Each(func(a int) { println(a) }) 2// 4 3// 3 4// 2 5// 1 Map ### Mutating every element of the iterator is a necessity, too: 1func (i *Iterator[V]) Map(f func(V) V) *Iterator[V] { 2 cpy := i.iter 3 i.iter = func(yield func(V) bool) { 4 for v := range cpy { 5 v = f(v) 6 if !yield(v) { 7 return 8 } 9 } 10 } 11 return i 12} At first we copy the previous iterator, by doing this, we skip causing a stack overflow by referencing the i.iter iterator in the iterator itself. Map is works by overwriting the i.iter with a new iterator thats consuming every field of the cpy iterator and overwriting the iterator value with the result of passing v to f, thus mapping over the iterator. Filter ### After mapping, possibly the most used streaming / functional api method is Filter. So lets take a look at our final operation: 1func (i *Iterator[V]) Filter(f func(V) bool) *Iterator[V] { 2 cpy := i.iter 3 i.iter = func(yield func(V) bool) { 4 for v := range cpy { 5 if f(v) { 6 if !yield(v) { 7 return 8 } 9 } 10 } 11 } 12 return i 13} Similar to Map, we consume the copied iterator and invoke f with v as the param for each one, if f returns true, we keep it in the new iterator. Examples and Thoughts ## The slices and the iter package both play very good together with the generic system introduced in go 1.18 ^4. While this API is easier to use, I understand the reasoning of the go team for not implementing iterators like this. Below are some tests that serve as examples and the result of running them. 1package iter1 2 3import ( 4 "fmt" 5 "testing" 6 "unicode" 7) 8 9func TestIteratorNumbers(t *testing.T) { 10 From([]int{1, 2, 3, 4}). 11 Reverse(). 12 Map(func(i int) int { return i * i }). 13 Filter(func(i int) bool { return i%2 == 0 }). 14 Each(func(a int) { println(a) }) 15} 16 17func TestIteratorRunes(t *testing.T) { 18 r := From([]rune("Hello World!")). 19 Reverse(). 20 // remove all spaces 21 Filter(func(r rune) bool { return !unicode.IsSpace(r) }). 22 // convert every rune to uppercase 23 Map(func(r rune) rune { return unicode.ToUpper(r) }). 24 Collect() 25 fmt.Println(string(r)) 26} 27 28func TestIteratorStructs(t *testing.T) { 29 type User struct { 30 Id int 31 Name string 32 Hash int 33 } 34 35 u := []User{ 36 {0, "xnacly", 0}, 37 {1, "hans", 0}, 38 {2, "gedigedagedeio", 0}, 39 } 40 41 From(u). 42 // computing the hash for each user 43 Map(func(u User) User { 44 h := 0 45 for i, r := range u.Name { 46 h += int(r)*31 ^ (len(u.Name) - i - 1) 47 } 48 u.Hash = h 49 return u 50 }). 51 Each(func(u User) { fmt.Printf("%#+v\n", u) }) 52} Running these, results in: 1$ go test ./... -v 2=== RUN TestIteratorNumbers 316 44 5--- PASS: TestIteratorNumbers (0.00s) 6=== RUN TestIteratorRunes 7!DLROWOLLEH 8--- PASS: TestIteratorRunes (0.00s) 9=== RUN TestIteratorStructs 10&iter1.User{Id:0, Name:"xnacly", Hash:20314} 11&iter1.User{Id:1, Name:"hans", Hash:13208} 12&iter1.User{Id:2, Name:"gedigedagedeio", Hash:44336} 13--- PASS: TestIteratorStructs (0.00s) 14PASS 15ok iter1 0.263s So there u have it, a wrapper around iter and slices to mirror the way JavaScript provides streaming, only in go. --------------------------------------------------------------------- 1. https://go.dev/doc/go1.23#language -[?] 2. https://go.dev/doc/go1.23#iterators -[?] 3. https://pkg.go.dev/slices -[?] 4. https://go.dev/doc/go1.18 -[?] go (c) Copyright 2024 <3 xnacly