[HN Gopher] Fun with Go Iterators
___________________________________________________________________
Fun with Go Iterators
Author : xnacly
Score : 121 points
Date : 2024-10-07 18:27 UTC (3 days ago)
(HTM) web link (xnacly.me)
(TXT) w3m dump (xnacly.me)
| pdimitar wrote:
| Might be interesting to make a library that competes with
| https://github.com/samber/lo?
| gtramont wrote:
| Back when the Go team announced generics, I had a go (pun
| intended) at it: https://github.com/gtramontina/go-extlib --
| might resurrect it one day. `lo` is pretty comprehensive
| though.
| mihaitodor wrote:
| Here's one: https://github.com/szmcdull/glinq It doesn't do
| function chaining though.
| mseepgood wrote:
| It's not interesting. Boring people did this a million times,
| the result is always the same: just use for loops.
| acheong08 wrote:
| > Tips for lazy developers > I cannot recommend it, but in case
| you are too lazy for repeating lo. everywhere, you can import
| the entire library into the namespace. import
| ( . "github.com/samber/lo" )
|
| TIL. Crazy
| Savageman wrote:
| I like how the author uses a test to run arbitrary code, this is
| exactly how I do it too!
| gtramont wrote:
| Unfortunately the chain approach breaks down when if you need to
| `map` to a different type, for example. Go does not allow generic
| typed methods.
| daghamm wrote:
| I never understood why that limitation exist.
|
| Can someone explain the reason for this?
| jerf wrote:
| Straight from the designers: https://go.googlesource.com/prop
| osal/+/refs/heads/master/des...
| thefounder wrote:
| https://go.googlesource.com/proposal/+/refs/heads/master/des.
| ..
| mihaitodor wrote:
| I recall reading some details around this on
| https://blog.merovius.de. Maybe it was this article:
| https://blog.merovius.de/posts/2018-06-03-why-doesnt-go-
| have.... Compilation speed plays a big factor when deciding
| which features are added to Golang and I think they'd have a
| hard time maintaining the current compilation speed if they
| remove this limitation.
| mseepgood wrote:
| https://go.dev/doc/faq#generic_methods
| foldr wrote:
| It's broadly similar to the reason why Rust won't allow
| generic methods on traits used to construct trait objects. It
| seems superficially like a reasonable thing to want, but
| actually isn't when you consider the details. (The sister
| comments link to the specific reasons why in the case of Go.)
| tapirl wrote:
| https://github.com/golang/go/issues/49085
| simiones wrote:
| You can actually (almost) make it work, you just need to add an
| extra type parameter to keep Go's silly limitations happy [0]:
| strs := []string{"abc", "defgh", "klmnopqrst"} ints :=
| From[string, int](strs). Map(func(a string) int
| { return len(a) }). Filter(func(a int) bool {
| return a >= 4 }) Iterator[int, float32](ints).
| Map(func(a int) float32 { return float32(a) }).
| Each(func(a float32) { fmt.Printf("%v\n", a) }) //prints
| 5, then 10
|
| If they allowed the postfix cast syntax to work for non-
| interface types too, it could have been a single chain,
| actually (you could do `.(Iterator[int, float32])` inline
| instead of needing the extra variable.
|
| Note that the original implementation in the article modifies
| the collections in place, in which case this issue doesn't come
| up at all: you can't put strings in an array of ints. My
| implementation creates copies of these collections so that the
| concept of mapping to a new type actually makes sense.
|
| [0] https://go.dev/play/p/ggWrokAk7nS
| dilap wrote:
| The Reverse implementation seems off to me -- it runs through the
| iterator twice, once collecting into a slice, and then a second
| time filling the same slice in reverse. (So basically the first
| Collect call is only being used to find the length of the
| iterated sequence.) I'm not sure about Go conventions+, but I
| imagine it would be considered better form to only run through
| the iterator once, reversing the collected slice in-place via a
| series of swaps.
|
| (+ Are iterators even expected/required to be reusable? If they
| are reusable, are they expected to be stable?)
| jerf wrote:
| That eliminates one of the main reasons to use this approach.
| Function chaining as most people write is awful for performance
| because it involves creating a separate array for each step in
| the chain. Given that most programs are actually more memory-
| blocked than CPU blocked, this is a bad tradeoff. Composing Go
| iterators, and composing iterators in general, is preferable
| because it doesn't have to create all the intermediate arrays.
| A bad reverse wrecks that back up.
|
| Still, you need the option, and while reverse is one of the
| more common iterators, it's still usually avoidable if you need
| to. But if at all possible I'd suggest a "reverse" type-
| specialized to slices, and as necessary and possible, type-
| specialized to whatever other types you are using to _actually_
| crawl a value "backwards" rather than collecting a full
| iterator into a slice.
|
| (Then again, I'm not a fan of this approach in imperative
| languages in general, due to the generalized difficulty in
| refactoring code written in this style and the fact that
| observationally, people just don't refactor it once written and
| it affords a style rife with repetition. One of _the_ most
| important considerations about code is how easily it can be
| refactored. In Haskell, this approach is excellent, precisely
| _because_ it refactors very, very well in Haskell. In
| imperative languages, it tends not to, thus, serving as another
| example of why you can 't just blindly bring over nice things
| from one language into another without verifying they haven't
| become sour in the process.)
| dilap wrote:
| Yeah, that all makes sense!, but I think it's not relevent to
| the code in question, which is this: func
| (i *Iterator[V]) Reverse() *Iterator[V] { collect :=
| i.Collect() counter := len(collect) - 1 for
| e := range i.iter { collect[counter] = e
| counter-- } return From(collect) }
|
| So this code creates a slice from the iterator in the call to
| Collect(), and then fills the slice _again_ in reverse by
| running the iterator _again_ , which I think is wrong (or at
| least not-ideal).
|
| (Your broader point about wanting to avoid creating an
| intermediate array at all for iterators and using type
| information to intelligently reverse "at the source"
| definitely still stands, in a broader context, though.)
| jerf wrote:
| Oh, yes, sorry. The level of error was more than I
| expected.
|
| Edit: I should add that a "Reverse" that takes an iterator
| has no choice but to manifest the list and then reverse on
| the list. (Goodness help you if you try to reverse an
| infinite list.) But you can do type-dependent reverse
| iterators that don't have to if the data structure doesn't
| force it, and if you can do that you should, in general.
| This paragraph isn't about Go; this is the iterator
| protocol itself.
| dilap wrote:
| I guess if you're doing something like the original
| post's "type Iterator[V] struct" you could somehow extend
| it capture the extra information to make an efficient
| Reverse work.
|
| (FWIW, in C#, the IEnumerable extension method Reverse()
| just captures the whole sequence to an array and then
| iterates it in reverse. And yeah, woe unto you if you try
| to reverse an infinite sequence.)
| wwalexander wrote:
| Worth comparing to Swift's LazySequenceProtocol [0]. Various
| Sequences have a .lazy property that returns some
| LazySequenceProtocol<Element>, so you can opt in to this sort
| of fluent-style method chaining where it makes sense.
|
| [0] https://developer.apple.com/documentation/swift/lazyseque
| nce...
| atombender wrote:
| Why reverse the slice at all? Collect the input into a slice,
| then return an iterator that navigates the slice in backward
| order.
|
| What this sort of thing lacks is any ability to optimize. For
| example, let's say the operation is Reverse().Take(20). There's
| no reason for the reverser to keep more than 20 elements in its
| buffer. But to express that, you have to make sure the iterator
| can be introspected and then rewritten to merge the operators
| and maybe unroll some of the loops to get better cache
| locality. This is what Haskell can achieve via Stream Fusion,
| which is pretty neat. But not so viable in Go.
| dilap wrote:
| Thanks, good points.
| ted_dunning wrote:
| Julia does similar optimizations with the broadcast operator.
| The results can be mind bogglingly fast.
| skybrian wrote:
| I think it would be more idiomatic to use statements, not
| expressions. That is, it's ok to use local variables for
| intermediate values in a pipeline.
| simiones wrote:
| You often end up with a lot of extraneous variables with no
| useful names if you do that. A lot of the time the intermediate
| results in a pipeline are almost meaningless.
| skybrian wrote:
| The main thing is not to do things that would be fine in
| other languages when they result in complications in the one
| you're using. Some people want to write an entire library
| rather than writing statements. Why?
|
| Also, local names can sometimes be useful documentation when
| the function names don't really get the point across (perhaps
| because they're at a different level of abstraction). Or
| alternatively, in Go it's idiomatic to keep them short.
| simiones wrote:
| > Some people want to write an entire library rather than
| writing statements. Why?
|
| Because you want to make code more readable, and getting
| rid of extraneous intermediate results is one way of
| achieving that.
| integrii wrote:
| Call me crazy, but I don't like any of this. Make more named
| functions. Keep your logic flat and explicit. I believe go wants
| you to code this way as well. Imagine the horrors this kind of
| function chaining creates. Actually, you don't have to. It's
| JavaScript.
| lifthrasiir wrote:
| Even JS doesn't really use functional styles that much. In
| fact, the whole reason for functional styles is the
| decomposability: any language with easy function chaining like
| `x |> f |> g |> h` will also have an easy extraction of any
| part of the chain (say, `x |> fg |> h where fg = f . g`). It is
| not a good idea to use chaining when the language supports no
| such feature, as it would be much harder to work with such
| chains then.
| arethuza wrote:
| .Net seems to handle it pretty well with LINQ?
|
| https://medium.com/@sanjanasw99/an-in-depth-guide-to-linq-in...
| bilinguliar wrote:
| When Go eventually lets all this horrible syntax sugar into the
| standard library, we will meet again in the Zig community.
| tapirl wrote:
| It is a sad fact that Go is becoming more and more like
| JavaScript (and deviating from C).
| openasocket wrote:
| I work with Go a lot at my job, and I definitely prefer
| functional programming so I chafe at the language a bit. I was
| excited to incorporate iterators into our code. Unfortunately,
| I'm also working in an environment where we are memory
| constrained, so on our hot path we need to not allocate at all. I
| tried playing with the iterators a bit, and couldn't get it to
| produce something that didn't allocate. I got close, but as much
| as a tried I couldn't get below 1 allocation per loop (not per
| loop iteration, per loop). Which in any other setting would be
| fine, but not for our use case.
| acheong08 wrote:
| This is probably one of the areas where Zig shines. I'm mostly
| a Gopher but reading Zig is just as easy while ensuring that no
| allocations are hidden
| kunley wrote:
| I know complaining about downvote is not the right thing, but
| the above is someone else's comment, not mine. Why was it
| downvoted (I see it grayed) ? It's an useful information
| without zealotry or "we-know-betterism".
| atomic128 wrote:
| I have not looked at Go's iterator range loop implementation
| yet. So somebody tell me if I'm wrong here.
|
| My guess is that Go is probably wrapping the body of the
| range loop into a closure, and passing that closure into the
| iterator function as the yield function. A break in the body
| of the loop becomes a "return false" in the closure.
|
| The allocation is probably the closure environment struct
| (giving access to variables prior to the range loop).
|
| This closure might escape through the iterator function so Go
| can't just put the environment struct onto the stack, it has
| to escape to the heap.
|
| The cost is small but it's not free. Usually, not having to
| think about the allocation is an advantage. In the rare case
| can't afford the iterator, do it differently.
|
| Go is great.
| raggi wrote:
| Same, I work in Go most of the time in performance and memory
| sensitive areas and the lack of usable abstractions grates so
| hard. I'm increasingly fed up of long hand everywhere and the
| bug volume and frequency it inherently produces particularly
| under change. I was worried this would be the outcome of the
| iterator proposal and am not surprised at all to find that yes,
| it is in practice. I was similarly disappointed with rand/v2
| where I provided feedback thah the interface in the middle of
| it had been seldom used and could have been eradicated and the
| choice was to preserve that part of the API because the
| compiler should in the future eradicate these costs. I see no
| actual plan for how that will be done though, and I'm skeptical
| it will happen in practice. I'll finish with the comment that
| will lead to comment downvoting, but I derive tons of value for
| resource constrained work from rusts lazy iterator design
| combined with the aggressively optimizing compiler folding nice
| low cost of maintenance abstraction here into tight sometimes
| even vectorized or unrolled loops - it makes work I do
| demonstrably easier.
| glenjamin wrote:
| A question for both you and the parent: If you are heavily
| performance and memory constrained, why are you using a
| language that gives you relatively little control over
| allocations?
| openasocket wrote:
| In my case it was a choice made long ago, when the exact
| consequences weren't fully apparent. I don't think when
| making those initial decisions people understood how
| important low memory usage would turn out to be, or that Go
| would be an obstacle to that. And we've got so much code in
| Go at this point it would be a huge lift to switch
| languages. The language does have some nice features that
| make things easier. It's just in certain portions of the
| code, in the really hot loops.
| zmj wrote:
| This is usually the case with C#'s equivalent as well.
| Enumerables and LINQ are nice options to concisely express
| logic, but you won't see them in hot paths.
| neonsunset wrote:
| Unfortunately, the baseline allocation cost is hard to avoid
| due to IEnumerable<T> being an interface which all LINQ
| methods return save for scalar values, and IEnumerable<T>
| itself returning an interface-typed IEnumerator<T>. Even with
| escape analysis, the iterator implementation selection logic
| is quite complex, which ends up being opaque to compiler so
| at most it can get rid of the IEnumerable<T> allocation but
| not the enumerator itself, and only when inlining allows so.
|
| There are community libraries that implement similar API
| surface with structs that can be completely allocation-free
| and frequently dispatched statically.
|
| Moreover, with the advent of `T where T : allows ref struct`
| you can finally write proper LINQ-like abstraction for
| Span<T>s, even if it's a bit less pretty. I have been playing
| with a small toy prototype[0] recently and it looks like
| this: // Efectively C's array constant
| var numbers = (ReadOnlySpan<int>)[1, 2, 3, 4, 5, 6, 7, 8, 9,
| 10]; var iter = numbers .Where((int
| n) => n % 2 == 0) .Select((int n) => n * 2);
| // Inspired by Zig :) using var vec = NVec<int,
| Global>.Collect(iter);
|
| The argument types for lambdas need to be provided to work
| around C# lacking full Hindler-Milner type inference, but
| this iterator expression is fully statically dispatched and
| monomorphized save for the lambdas themselves. Luckily, JIT
| can profile the exact method types passed to Funcs and
| perform further guarded devirtualization, putting this code
| painfully close to the way Rust's iterators are compiled.
|
| At the end of the day, .NET's GC implementations can sustain
| 4-10x allocation throughput when compared to Go one (it's not
| strictly better - just different tradeoffs), with further
| tuning options available, so one allocation here and there is
| not the end of the world, and not all LINQ methods allocate
| in the first place, and many of them allocate very little
| thanks to optimizations made in that area in all recent
| releases.
|
| [0]: https://github.com/neon-sunset/project-
| anvil/blob/master/Sou...
| Spivak wrote:
| It's funny the author throws a dig at Python for its syntax that
| actively discourages this kind of code. Like... my guy you're not
| taking the hint. Python makes things it doesn't want you to do
| ugly as sin. It's why lambda is so awkward and clunky.
| rolux wrote:
| Yes. The following is a lot more concise: a =
| [1, 2, 3, 4] print([v*v for v in reversed(a) if v*v % 2
| == 0])
| libria wrote:
| I think the above is a good idea of what's wrong with python
| (and Go), because in your example the list comp is evaluated
| in what seems to be this order: FOURTH->
| print([THIRD-> v*v for v in FIRST-> reversed(a) if SECOND->
| v*v % 2 == 0])
|
| Which is all over the place. I'd rather see:
| a = [1, 2, 3, 4] a = reversed(a) a = [v*v for
| v in a] a = [w for w in a if a % 2 == 0]
| print(a)
| alfons_foobar wrote:
| This.
|
| I often use generator expressions for the intermediate
| values (so I don't allocate a new list for each step), but
| I find this to be much more readable.
| xnacly wrote:
| I get that, i still dont like to write f5(f4(f3(f2(f1()))))
| instead of writing f1().f2().f3().f4().f5()
| mbrumlow wrote:
| > My issue with the go way of iterators is, you can't chain them
| like you would in JavaScrip
|
| Because it's not JavaScript, and that is a good thing.
| eweise wrote:
| But not because you can't easily chain functions. That's just a
| deficiency in the language design.
| Varriount wrote:
| It always bugs me when I see that pattern in JavaScript,
| because _each_ `map`, etc. call is an array allocation. Yeah,
| yeah, I know that the VM 's memory allocators are likely
| optimized for fast allocation, but that doesn't make the
| allocation completely free.
| binary132 wrote:
| I'm trying to understand whether this is intended to make Go seem
| bad or whether it's just coming across that way to me.
| ugjka wrote:
| I looked at the code and it was giving me a headache, been
| messing around with Go for a decade, i do not like this
| binary132 wrote:
| Yeah, same, used Go fulltime professionally for 10 years, not
| into this newer stuff.
| arp242 wrote:
| People have been trying to do this sort of thing in Go for
| as long as I remember; it's nothing new and has not and
| most likely never will gain much uptake.
|
| The stages of learning a language are something along the
| lines of:
|
| 1. Force language patterns from previous language
|
| 2. Frustration
|
| 3. Write library to make it easier
|
| 4. Anger
|
| 5. Acceptance
| pragma_x wrote:
| I absolutely love it when we can take advantage of Go's type
| system and add additional traits and behaviors to existing types
| like this.
|
| That said, I noticed something odd here. In order for a module
| like this to really shine, I think all these operations need to
| be functionally pure. Right now, some of these mutate the
| iterator's `iter` method mid-stream, which is about as side-
| effect-ful as you can get.
|
| ```
|
| func (i _Iterator[V]) Map(f func(V) V)_ Iterator[V] {
|
| cpy := i.iter
|
| i.iter = func(yield func(V) bool) { for v :=
| range cpy { v = f(v) if !yield(v) {
| return } } } return i
|
| } ```
|
| Unless I'm misreading that, `i.iter` has new behavior after this
| call. A better way would be to return a new _iterator_ with the
| custom iter behavior instead.
|
| ``` func (i _Iterator[V]) Map(f func(V) V)_ Iterator[V] {
| // create a fresh iterator around a custom closure (NewIterator()
| is hypothetical in this case) return
| NewIterator(func(yield func(V) bool) { for v := range
| i.iter { v = f(v) if !yield(v) {
| return } } })
|
| } ```
| tantivy wrote:
| This flagged for me right away too. I would be badly surprised
| if a Map-style chained method mutated the memory of the
| receiver.
| kunley wrote:
| Might be written in a hurry and maybe the author was thinking
| that not allocating new Iterator, which is in fact wrapper
| around iter.Seq, will produce less heap garbage. But I guess
| it's the same amount of garbage, because the size of Iterator
| is the same as iter.Seq which is allocated anyway, it's just
| different type of the object being discarded
| kubb wrote:
| Go people will do this and they'll be content: a
| := []int{1,2,3,4} it := slices.All(a) it =
| slices.Reverse(it) it = slices.Map(it) it =
| slices.Filter(it, func(i int) bool { return i % 2 == 0 })
| slices.ForEach(it, func(i int) { fmt.Println(i) })
|
| I don't judge the Go enjoyers, but I prefer writing TypeScript to
| Go which says it all.
|
| Type-inferred arrow lambda for function arguments would go such a
| long way in making this code nicer... And not make compilation
| slower at all. it = slices.Filter(it, i => i % 2
| == 0) slices.ForEach(it, i => fmt.Println(i))
| jerf wrote:
| No, Go programmers would write a := []int{1,
| 2, 3, 4} out := []int{} for idx := range a
| { val := a[len(a)-idx-1] if mappedVal
| := Map(val); mappedVal % 2 == 0 { out =
| append(out, mappedVal) fmt.Println(mappedVal)
| } }
|
| In modern Go I might write a reverse iterator, that index is a
| bit hairy, which would cut out the 'val :=' line, but as there
| is not yet a standard library option for that I'll leave it
| out.
| kubb wrote:
| Many Go programmers would claim this is cleaner and more
| understandable, unfortunately I'm not one of them.
| int_19h wrote:
| The bigger problem is that it's not efficiently composable.
| If your caller needs to additionally filter `out`, that's
| another loop with a copy.
| thegeekpirate wrote:
| The proper term is deforestation https://en.wikipedia.org
| /wiki/Deforestation_(computer_scienc..., and I have seen
| Go libraries which do this
| nemo1618 wrote:
| ah, _that 's_ what it's called! I did this in my compile-
| to-Go language, but I didn't know the name for it:
| https://github.com/lukechampine/ply?tab=readme-ov-
| file#suppo...
| jerf wrote:
| In most imperative languages, writing
| .Map().Map().Filter().Map() is another full copy for
| _each call_ anyhow. As a sibling notes, it is possible to
| "fix" that, but the fix is not even remotely free. It is
| quite complicated to do it generically. (Although there
| are other benefits; the most natural approach to that
| largely fixes my grumblings about refactoring I mentioned
| in another post.)
|
| Plus, in a for loop approach, it is not true that the
| caller may need another loop with a copy. They may just
| loop over the result and skip over the things they don't
| need. They only need a copy if they are going to pass
| that on to something else.
|
| A drum I can not stop banging on is that you can not just
| take a neat technique out of one language and slam it
| into another without examining the end result to make
| sure that you haven't murdered the cost/benefits
| tradeoff. You can slam together all the maps and filters
| and reduces you want in Haskell, and applicatives and
| monads and all the fun, due to a combination of the
| laziness and various safe optimizations like loop fusion.
| In an eager context that lacks loop fusion, going
| .Map().Map().Map().Map() has _radically_ different
| performance implications. For instance, "take 10 $ map f
| list" in Haskell will only call "f" 10 times.
| .Map().Take(10) in most implementations will create the
| full array, however large it is, and slice ten off the
| end after that.
|
| In imperative languages, contrary to frequent claims from
| the functional programming crowd, for loops are actually
| often better in practice. The solution to their
| pathologies is to be aware of them and not do them. But
| it is far, _far_ easier to get good performance out of a
| for loop in an imperative language than to contort one 's
| code into a pale parody of functional programming.
| akkad33 wrote:
| > In most imperative languages, writing
| .Map().Map().Filter().Map() is another full copy for each
| call anyhow.
|
| Which imperative language? In Java and rust, two
| languages I know, all these operations are lazy until the
| final collect. So no copy is made
| nine_k wrote:
| Nor in Python. Nor in C++20's std::views. The very point
| of iterators us to proceed by one element, avoiding
| intermediate copies of collections.
|
| One case where the for-loop would be much more efficient
| is a simple transformation like incrementing each element
| of an array, or adding two arrays element-wise. A serious
| C compiler could unroll such a loop into several vector
| instructions which work on several elements at once.
| Maybe even LLVM can recognize something like that in Go
| code.
| jerf wrote:
| That's why I specified the code. If you're writing
| .Map().Map().Map().Map(), you are usually getting a lot
| of intermediate arrays.
|
| If you have a .Collect() call, you're in the
| deforestation branch. This has its own issues with
| stressing the inliner (turning simple, direct for loops
| over large collections into traversals that include
| several indirect method calls per item in addition to the
| payload is pretty easy), but that's still generally
| better than stressing the RAM.
|
| Rust's map doesn't operate on arrays at all from what I
| can see but operates on iterators directly. This is good
| and generally more correct. However there's a lot of
| languages that don't support that. Rust is generally also
| going to be more reliable about compiling it all away
| than a lot of other languages where it will be really
| easy to spill over what the inliner can handle. Those
| long compile times in Rust do have their benefits.
|
| There's also languages that sort of split the difference,
| e.g., it is not that difficult in Python to use itertools
| and generators to correctly write something that will not
| generate a lot of intermediate arrays, _but_ it is also
| easy to write a series of calls and list comprehensions
| to write otherwise equivalent code that _will_ create a
| lot of intermediate arrays.
|
| I expect as we continue to build new languages over time
| they're going to all look like Rust here. It's pretty
| obvious that conceiving of loops as iteration over some
| sequence is the way to go. However, that is a result that
| we got to precisely because of our experiences with a lot
| of languages that don't support it as well, or support it
| inconsistently, or as is actually quite common, the
| language nominally supports it but the ecosystem tends to
| assume concrete values a lot more often than it should,
| and all these languages are still around.
|
| Writing in this style _correctly_ in imperative code is
| more difficult than a lot of people jumping up and down
| about how we should rewrite all our loops as maps and
| filters tend to account for. It can be done, but it 's
| often harder than it looks, in at least one of the
| writing and the performance if not both, and the harder
| it is, the more the costs stack up on the costs side, and
| the harder the costs/benefits analysis becomes. And I
| still don't like how it refactors in most cases.
| margalabargala wrote:
| The original version relied on no fewer than five different
| imported functions from a package that, if you're not
| already familiar with them, you would have to go read to
| see exactly what they do.
|
| The updated version has just one such function.
|
| A function gets less readable as you have to go context
| switch to more and more different functions to understand
| what it's doing.
|
| Yes, the `slices` functions are named such that you can
| make a decent educated guess as to what they're doing, but
| if there's a problem you're trying to debug that doesn't
| save you from having to dive into each one.
| scubbo wrote:
| > A function gets less readable as you have to go context
| switch to more and more different functions to understand
| what it's doing.
|
| This is true.
|
| A function also gets less readable every time you have to
| mentally translate from low-level implementation code, to
| a high-/human-level description of "what is this actually
| doing?" (e.g. `val := a[len(a)-idx-1]` - "ohhh, ok,
| that's iterating in reverse"). Extracting out common
| patterns to methods with well-known names like `Filter`
| or `Map` or `ForEach` short-circuits that repeated
| parsing.
|
| > that doesn't save you from having to dive into each
| one.
|
| 99% of the time, it really does. If you can't trust
| something as basic and well-defined as `Map` to do what
| you expect it to do, you've chosen a poor library on
| which to take a dependency.
| margalabargala wrote:
| > 99% of the time, it really does. If you can't trust
| something as basic and well-defined as `Map` to do what
| you expect it to do, you've chosen a poor library on
| which to take a dependency.
|
| The original comment said: it =
| slices.Map(it)
|
| I understand Map taking an array and a function, as in
| the article. I don't understand Map taking simply an
| array.
|
| Then I went to look it up, and turns out that slices.Map
| does not in fact exist. https://pkg.go.dev/slices
|
| Personally I feel like this exchange is a vivid
| illustration of exactly my original point.
| scubbo wrote:
| The facts that GoLang is missing such a fundamental tool
| from its standard library, and that it's easy to make
| typos when coding not in an IDE, hardly support your
| point.
|
| In fact, the very fact that you originally thought of the
| correct interpretation despite the fact that it had been
| misrepresented is a great example of why common
| shorthands are useful.
| parhamn wrote:
| I think the killer case for this is not hiding the
| computational complexity of mapping/reversing/etc.
|
| I write a lot of typescript and golang and I often notice in
| typescript my chains end up iterating the same array N times
| when it couldbe been done in one smarter iteration.
|
| I don't think I care though -- both styles are way more than
| performant enough. You can be more careful about it in cases
| it is not.
| tsimionescu wrote:
| Iterating an array 10 times doing 1 operation on each
| element each time is equivalent to iterating it 1 time
| while doing 10 operations on each element.
| parhamn wrote:
| This isn't true outside the theoretical best case.
| Definitely not true in most interpreted languages.
| Consider the performance difference of .reverse() (in-
| place or copy) vs using a descending for loop. Generally
| with interpreter chaining there are more "hidden"
| allocations/operations.
| thrww120956 wrote:
| Found the theoretical informatician... Nope, not in real
| world software engineering. Especially not in JIT
| runtimes like V8. Try it!
|
| I tried it, code: https://pastebin.com/cA8YkE8R
|
| Result: process1: 2s 251.327833ms
| process2: 1s 537.721625ms
| throwitaway1123 wrote:
| It's a little bit faster (on my machine at least) if you
| combine the filter and map into a flatMap (it's still not
| as performant as the imperative solution though).
| function process3(input) { return input
| .flatMap((n) => (n % 2 === 0 ? n * 2 : []))
| .reduce((a, b) => a + b, 0) }
| nine_k wrote:
| But this is apples and oranges. Process 1 creates and
| calls lambdas, etc.
|
| A proper comparison could be: # Case 1.
| for n in range(len(data)): data[n] =
| transform1(data[n]) data[n] = transform2(data[n])
| # Case 2. for n in range(len(data)):
| data[n] = transform1(data[n]) for n in
| range(len(data)): data[n] = transform2(data[n])
| nick__m wrote:
| In an ideal machine you are absolutely correct! But when
| executed by a real physical CPU, if the array doesn't fit
| I the cache, iteration strategies matters!
| gpderetta wrote:
| https://en.m.wikipedia.org/wiki/Loop_fission_and_fusion
| ehaliewicz2 wrote:
| Until the array no longer fits in your cache :)
| everybodyknows wrote:
| > it = slices.Map(it)
|
| There is no such function as "slices.Map()".
|
| https://pkg.go.dev/slices@go1.23.2
| cyberax wrote:
| > I don't judge the Go enjoyers, but I prefer writing
| TypeScript to Go which says it all.
|
| The functional style is fine for simple filter/map algorithms,
| but once you get into zipping multiple iterators together,
| classic for-loops are often easier.
|
| My company's hiring coding task had this "trap", at one point
| you needed to write an algorithm to merge consecutive "empty"
| cells in a data structure that is used to represent a simple
| table. This is dead easy with a regular for-loop, but writing
| it in a "functional" style could easily become extremely hairy
| and/or have O(N^2) complexity.
| kubb wrote:
| You can use loops in typescript if you need to.
| scubbo wrote:
| Nit - I think you've missed the `func` argument in the
| `slices.Map` line.
|
| (But - yes, bravo, correct, you're right and you should say it)
| kubb wrote:
| I did, but I believe some people understood what I meant
| anyway.
| vyskocilm wrote:
| Shameless plug. I had experimented with Go iterators a while ago
| and did a https://github.com/gomoni/it
|
| It was updated to 1.23, so it is as idiomatic as I can get. And
| yes it has a map method between two types. Just a single simple
| trick used.
| kunley wrote:
| Nice rewrite for 1.23. Btw, just sent you a PR for a typo in
| the readme.
| indulona wrote:
| > My issue with the go way of iterators is, you can't chain them
| like you would in JavaScript
|
| You are not supposed to chain them. This addiction to try and
| chain everything everywhere all the time is so freaking weird and
| has been for a very long time.
|
| Not only you are completely losing grasp on what is going on and
| write code prone to errors, but you are making it unreadable for
| other people that will be maintaining or just reading your code
| who will come long after you are gone from the company or abandon
| your library.
|
| This is where Go's simplicity approach and splitting each action
| into its own for loop or block of code is a godsend for
| maintainability.
| AndyKluger wrote:
| 1. That's a good looking Hugo theme!
|
| 2. Implicitly chain everything all the time!
|
| In Factor, you might do it as: reverse [ sq ] [
| even? ] map-filter [ . ] each
|
| Or with a little less optimizing: reverse [ sq
| ] map [ even? ] filter [ . ] each
|
| The least obvious thing is that the period is the pretty-print
| function.
| qudat wrote:
| We just released a go pkg that uses the new iter pkg. We were so
| excited by the interface in large part because of how simple
| iterators are to use.
|
| https://github.com/picosh/pubsub/blob/main/pubsub.go#L18
|
| We have seen in other languages like JS and python the power of
| iterators and we are happy to see it in Go
___________________________________________________________________
(page generated 2024-10-10 23:01 UTC)