[HN Gopher] Goroutines: The concurrency model we wanted all along
       ___________________________________________________________________
        
       Goroutines: The concurrency model we wanted all along
        
       Author : ingve
       Score  : 81 points
       Date   : 2023-07-07 18:40 UTC (4 hours ago)
        
 (HTM) web link (jayconrod.com)
 (TXT) w3m dump (jayconrod.com)
        
       | synthetigram wrote:
       | The StreamObserver API came at a time (2015) when it seems liked
       | RxJava was going to take over. That didn't end up happening, but
       | the API is still around. While it is more cumbersome, some things
       | are /impossible/ to do with the Go style blocking. For example,
       | try cancelling out of a Recv() call. The only way is to tear the
       | entire Stream down. Goroutines never successfully married select
       | {} and sync.Cond, or context Cancel. These are needed to
       | successfully back out of a blocking statement. Unfortunately,
       | that can't be done, and a goroutine that blocks is really stuck
       | there. The only saving grace is that goroutines are relatively
       | cheap (2-4K of memory?), and it's okay if a few O(100K) of them
       | get stuck.
        
       | adeon wrote:
       | Haskell threads have some similarities to Goroutines. They are
       | also real cheap and there's no problem creating lots of threads
       | (M:N scheduling behind the scenes). There's also no function
       | color problem with the threads.
       | 
       | There's one special thing that Haskell threads have that I think
       | I've only seen in Erlang: it's safe to kill threads. Things will
       | be cleaned up properly. You can safely expect that Haskell code
       | created by someone else has been written to not go into a bad
       | state if it's suddenly cancelled.
       | 
       | For example, you can write a generic timeout function that works
       | by launching the work in a new thread, then wait N seconds, then
       | kill that thread if it's still running. You don't need to make
       | any kind of signalling thing where you send "please stop" message
       | to a Goroutine.
       | 
       | The article talks about Goroutines in context of serving lots of
       | requests. I like to think Haskell's threads and the fact that you
       | can kill them safely makes it much easier to develop servers or
       | job orchestration with simpler code.
        
         | aseipp wrote:
         | To be fair, there have been plenty of bugs in Haskell libraries
         | wrt exception safety. Cancellation and asynchronous exceptions
         | are actually really hard in practice. But yes, actually having
         | the possibility can make some code a lot easier to write and
         | think about.
        
       | ryanthedev wrote:
       | Well articulated.
       | 
       | Few people realized what callback hell was. Web devs got lucky
       | they added await to JS.
       | 
       | I always appreciated Go for its coroutines. Does make for simpler
       | code.
       | 
       | You know a language has done well when the biggest consistent
       | argument is that it's "boring"
        
       | dekhn wrote:
       | goroutines are just m:n threads with message passing queues.
       | please correct me if I'm missing some obvious detail that makes
       | them unique in any way. in that case, I've been using this model
       | for a couple decades, as they are similar to green threads. They
       | are in fact a wonderful model for concurrency but I think they
       | leave a number of expert-programmer aspects of threads off the
       | table.
        
         | titzer wrote:
         | IMO thread pools are anti-pattern in several ways. They don't
         | globally load-balance properly and can be exhausted. I think
         | the Go concurrency model with many lightweight threads is
         | really nice. The only issue I have is that race conditions in
         | Go code can violate the type system and cause horror-crashes.
         | I've never used Go in production but have talked to outfits who
         | do and have confirmed that race conditions are a serious issue.
        
         | klooney wrote:
         | "Just m:n threads" is serious downplaying, most operating
         | systems and languages that had that retired it eventually. The
         | go implementation of goroutines is really really good.
        
           | dekhn wrote:
           | I'm not saying their implementation isn't good, it is good.
           | And probably message-passing is the right concurrency
           | primitive to expose at a language level for people who want
           | to take advantage of multi-core machines running concurrent
           | code.
           | 
           | I'm not downplaying anything; I'm expressing merely that with
           | a title like "Goroutines: The concurrency model we all..."
           | makes it sounds like go invented something new, but afaict,
           | they're just a well-engineered implemention of already well-
           | understood concurrency principles.
        
             | zwieback wrote:
             | 100%, as I was reading the article I was remembering the
             | different types of concurrency mechanisms I've used and I
             | think it's helpful to have something like goroutines as a
             | first-class language feature but it's not new in any way.
             | 
             | When it comes to concurrency I think it's good to go
             | through the curriculum:
             | 
             | - main loop + interrupt handler
             | 
             | - processes (fork)
             | 
             | - multi-core concerns
             | 
             | - threads (now is a good time to learn about mutex,
             | semaphore, critical section)
             | 
             | - lightweight threads (fiber, coroutines, green threads):
             | learn how they are different
             | 
             | - async/await statemachines written by the compiler
             | 
             | Ok, now we have the background info to make wise choices
             | and call things by their generalized names to avoid holy
             | wars.
        
           | felixgallo wrote:
           | m:n threads is the basis of every non-embedded operating
           | system, so I wouldn't say that model's been retired or is
           | even under plausible threat. Loom and such are refitting Java
           | with it, for example. The only problem with m:n is that it
           | requires a thread-aware runtime to fully implement the
           | abstraction.
        
         | fidotron wrote:
         | The message passing queues following CSP semantics regarding
         | synchronization is the big deal. If you only ever use buffered
         | queues then you're missing half of the story.
         | 
         | Occam is probably the most important totally forgotten thread
         | of programming language development.
        
           | packetlost wrote:
           | Can you elaborate or provide sources on what you mean by "CSP
           | semantics" here?
        
             | Jtsummers wrote:
             | They mean communicating sequential processes. Here's a book
             | on the topic by C.A.R. Hoare: http://www.usingcsp.com/
        
           | dekhn wrote:
           | The message passing queues in CSP aren't any different from
           | buffered queues, right? I've been asking people this for some
           | time and never have heard a convincing "no, and here's why
           | they are different". There are CSP implementations in C++ and
           | Python and their implementations seem to be buffered queues
           | that block if there are no readers.
        
             | fidotron wrote:
             | CSP doesn't have queues, only blocking channels. Some
             | implementations may provide them on top, as go does.
             | 
             | I would suggest reviewing the Hoare book on CSP, it's very
             | readable.
        
               | dekhn wrote:
               | I have read it (along with lots Russ Cox wrote about CSP
               | and Go, such as https://swtch.com/~rsc/thread/) and
               | McIlroy
               | (https://www.cs.dartmouth.edu/~doug/sieve/sieve.pdf) .
               | From what I can tell, CSP channels are semantically
               | equivalent to a particular subtype of queues and use the
               | same OS primitives.
               | 
               | I'm not an OS expert but I've worked with distributed
               | systems and threads for decades, and nobody I've asked
               | has disagreed that Go's implementation of CSP is an M:N
               | thread model with (blocking) queues.
        
         | hashmash wrote:
         | What are the expert-programmer aspects you're thinking of?
        
           | jayd16 wrote:
           | Probably that it doesn't mix well when you need to pin work
           | to a native/specific thread. This can matter when that thread
           | is used across runtimes or with single threaded concurrency
           | designs like GUI threads for qt or gtk.
        
             | arccy wrote:
             | https://pkg.go.dev/runtime#LockOSThread
        
           | dekhn wrote:
           | shared memory, https://go.dev/blog/codelab-share although of
           | course go added thread primitives. it's just that the
           | language creators explicitly wanted to steer people away from
           | the "subtle" details of multithreading.
        
       | nitwit005 wrote:
       | It's not without its downsides.
       | 
       | The fact that IO blocks, and also doesn't work with the select
       | statement, means you need to spin up a mess of extra goroutines
       | in some situations. There was some good prior discussion of this
       | here: https://news.ycombinator.com/item?id=13331284
        
       | bjoli wrote:
       | I am still waiting for people to discover concurrentML. Whenever
       | pthreads becomes a hassle I reach for CML and it hasn't failed me
       | yet. It is pretty similar to go's concurrency model, but with
       | dynamic selects that doesn't suck. And it is over 30 years old.
        
       | zokier wrote:
       | It is dubious that the article does not mention structured
       | concurrency even once, despite it being the model du jour
        
       | mighmi wrote:
       | For a few years, I found myself continuously appreciating Go's
       | design and models more and more. Right now, a bit bored of
       | working in the same paradigm for a decade, I've been trying to
       | find new approaches but find myself rather trapped - as after a
       | novel exploration, it turns out the current approach really does
       | seem to work better.
       | 
       | Go's whole concurrency model of Communicating Sequential
       | Processes is cool, although seems to be the aspect I question the
       | most. Erlang's Actor Model does seem a bit superior and Go can
       | introduce footguns at points, but I've yet to really investigate
       | the alternatives as much as I'd like.
       | 
       | Also, Beej's guide to network programming
       | https://beej.us/guide/bgnet/html/ mentioned in the beginning
       | really is quite fantastic.
        
         | vore wrote:
         | The thing I find weird about Erlang's design is that actor
         | mailboxes have no backpressure mechanism, which seems like a
         | pretty big footgun. Go channels have their own quirks, but
         | requiring them to be bounded was a good design choice.
        
           | ghayes wrote:
           | And Elixir introduced GenStage to help with this [0]. The
           | thing that I love about erlang's actor model over Go (that
           | for me a fatal flaw with Go and CSP) is the "spooky action at
           | a distance" issue. It's much easier to reason locally within
           | an erlang project, in my opinion, versus a Golang project,
           | since once a channel is created, it's often very difficult to
           | trace its usage.
           | 
           | [0] https://elixir-lang.org/blog/2016/07/14/announcing-
           | genstage/
        
           | fidotron wrote:
           | The best effort I've seen to get around this is
           | https://github.com/lpgauth/shackle
           | 
           | It was motivated by a need to eliminate OOM errors and
           | supports large scale operation at very low latency.
        
         | zer8k wrote:
         | I don't mean this in a sarcastic way but Go was explicitly
         | designed for the lowest common denominator developer. Kernighan
         | said it himself.
         | 
         | That's the problem with Go in general I think. Much like my
         | experience with Ruby on Rails it seems like once you want to go
         | venture outside the safety of the walled garden you can't. The
         | language is specifically designed to keep you as safe as
         | possible. Great for bottom-tier devs and unmotivated people
         | working on corporate TPS reports. Bad for the motivated
         | engineer. Opportunities like this (to introduce a new idea into
         | the go concurrency ecosystem) are basically impossible.
        
           | api wrote:
           | This is the central fallacy of developer culture: that
           | complexity is a sign of intelligence and that complex systems
           | are superior.
           | 
           | Simplicity is much harder than complexity. Anyone can add and
           | anyone can subtract recklessly, but it takes genius to
           | subtract _without losing too much in the process_. The genius
           | is in identifying the essential complexity in a problem and
           | finding ways to dispense with incidental complexity.
           | 
           | I've personally been using Rust more lately for various
           | reasons, but I really do like Go and think its choices are
           | excellent for its niche. In any case simplicity can be
           | practiced in any language by choosing the most
           | straightforward way to achieve things and avoiding over-
           | engineering. Rust has a lot of features and paradigms but I
           | can be productive in it by remembering that just because it's
           | there doesn't mean you always have to use it.
        
             | lelanthran wrote:
             | > Rust has a lot of features and paradigms but I can be
             | productive in it by remembering that just because it's
             | there doesn't mean you always have to use it.
             | 
             | The problem with kitchen sink languages is that _someone_
             | on the team will use a feature you don 't know about.
             | 
             | If enough people on a big enough team do that the project
             | inexorably moves towards the most complex representation of
             | any idea.
             | 
             | On solo projects, some of the most complex languages can be
             | used by effectively by a newcomer. On any other project
             | with the same stack, you need to add in significant ramp up
             | time.
        
             | throwaway894345 wrote:
             | The problem with complex languages is that while you can
             | choose to eschew complexity in the code you write, you very
             | often have to deal with the complexity in the libraries you
             | use. So in languages like Rust and C++ you tend to have a
             | high degree of complexity (at least by way of high degrees
             | of abstraction) in the standard library and the wider
             | ecosystem because abstraction (and thus complexity) are
             | valued and idiomatic in their respective ecosystems--it
             | makes it very hard to opt out of. For example, at least for
             | a good while, there wasn't a great JSON parser library
             | except serde which is immensely complex and I would spend a
             | lot of time troubleshooting errors in the macro expansions
             | for things that would Just Work in Go's encoding/json (of
             | course, there are plenty of legitimate grievances with
             | encoding/json, but these are mostly orthogonal to the
             | simplicity/complexity discussion).
        
           | Vanclief wrote:
           | I don't think that its bad for the motivated engineer, its
           | just a trade off between expressiveness and simplicity.
           | 
           | I personally find myself choosing go because I feel I am way
           | more productive with the language. By keeping me constrained,
           | I have less choices to make on how I solve the problem, and
           | focus more on solving the problem.
        
             | frakt0x90 wrote:
             | I agree about the constraints. That was my biggest
             | complaint when I learned Scala. Every single thing I built
             | there were 10 different ways I could do it and it really
             | slowed me down. I'm sure if I stuck with the language I
             | would get a feel for the idiomatic ways over time but...
             | I'd rather just make stuff.
        
           | philosopher1234 wrote:
           | This is so condescending, but I'm glad you're not pretending
           | to have some other motivation. Go was designed for new devs
           | (per pike, i dont think kernighan ever said this) and that is
           | good for all devs, even those with overinflated egos. Making
           | your software easy to understand and think about is good, and
           | effective engineers will get more done working with simple
           | code than complex code that memorializes their useless
           | brainpower.
        
             | dcow wrote:
             | There's a time and place for everything. Not everyone works
             | on a corporate team needing to KISS their software to death
             | because they probably won't even be working on it 18 months
             | later. Software is a creative expression for some people,
             | and useless brainpower for you. I'm not one to judge.
        
             | elromulous wrote:
             | Speaking of overinflated egos. While there's no question
             | pike, thompson, and k are a cut above, I think they made
             | assumptions that engineers cannot be trusted.
             | 
             | Garbage collection is another such example. Compare that to
             | e.g. rust's model which hoists the understanding of
             | lifetime to the engineer, while encumbering the compiler
             | with the dirty work. Whereas golang asks the engineer to do
             | neither (granted golang came later, so maybe they didn't
             | think it was possible).
        
               | Yasuraka wrote:
               | Even if they can be trusted (I wouldn't trust myself), GC
               | is a big time saver and I'd say well worth the trade-off.
               | There's far more performance to be had elsewhere
               | (compiler) but wherever that's or manual memory
               | management is needed, there is a language to fit that
               | need.
        
               | kaashif wrote:
               | > I think they made assumptions that engineers cannot be
               | trusted.
               | 
               | I disagree, if you don't think engineers can be trusted,
               | you wouldn't want to trust engineers to remember to close
               | files or connections, you'd have some language feature to
               | do that automatically.
               | 
               | C++ and Rust solved this with RAII and destructors.
               | 
               | In my opinion, this is on the same level as "just
               | remember to check bounds" or "just remember to free
               | memory".
        
               | cube2222 wrote:
               | The languages target different use-cases and rusts
               | ownership model does indeed slow you down vs not having
               | to think about it at all.
               | 
               | Esp. closures, which Go uses a lot all over the place,
               | are quite cumbersome in Rust.
               | 
               | (I like both languages a lot)
        
               | erik_seaberg wrote:
               | Go doesn't have immutability or enforced synchronization,
               | and something as simple as a read/write conflict in a map
               | can panic. If your language (like most) has shared
               | mutable state, you _must_ think about ownership, and it's
               | better for the compiler to check your work reliably.
        
               | throwaway894345 wrote:
               | You're making an argument in favor of Go: Go _only_ makes
               | you think about ownership in proportion to the amount of
               | shared mutable state in your program. Many Go programs
               | don't have _any_ shared mutable state and the ones that
               | do often limit this state to a small kernel that manages
               | the ownership details. In Rust, every single program has
               | to deal with ownership, and the burden is proportional to
               | that of the total amount of state in the program (not
               | just the shared mutable bit).
               | 
               | Further still, "it's better for the compiler to check
               | your work reliably" is limited to the shared mutable
               | state that is completely in the purview of a single
               | process--if you have a networked resource or a file that
               | is accessed by multiple processes, rustc won't save you.
               | In Go's niche (web services, daemons, etc), this is the
               | overwhelming majority of all state.
        
               | cube2222 wrote:
               | You must think about synchronization, really, not
               | necessarily ownership.
               | 
               | What you're saying is true, but it's still not an obvious
               | choice - there's still a tradeoff.
               | 
               | The (hypothetical, e.g. Rust) compiler can check only a
               | subset of correct programs, which means that many correct
               | programs can't be successfully checked. This is all good
               | if it only affects edge cases, it's not that good if it
               | affects common cases.
               | 
               | It additionally becomes less of an issue (in Go) if you
               | avoid shared mutable state and use channels for complex
               | cases.
               | 
               | So all in all I agree the compiler checking your
               | synchronization is nice, but it's a tradeoff (at least
               | for now), and with the current state of it I think most
               | projects will be just fine without it. On the other hand,
               | there are projects that should definitely use languages
               | that allow for static analysis like that.
        
               | erik_seaberg wrote:
               | Yeah, I'm thinking of ownership as exclusive access
               | handed off in sync, and disposal as one more duty of the
               | final owner.
               | 
               | I would feel a lot more comfortable with channels if a
               | linter would verify that every message is either mutex
               | protected or a newly created graph (maybe a deep copy)
               | never accessed again by the sender. Go already does some
               | escape analysis to minimize heap allocations, but doesn't
               | flag a goroutine receiving nested maps or array slices
               | that may be shared.
        
               | throwaway894345 wrote:
               | This seems like a silly argument. By the same token, we
               | can argue that Rust doesn't think engineers can be
               | trusted because it pushes a bunch of type and memory
               | safety onto the compiler instead of letting the engineers
               | manage it as their ancestors did. Of course, they were
               | right to do so--we shouldn't trust engineers with all of
               | these tedious, error-prone processes. And while Rust
               | solves for memory safety with borrow checking, that turns
               | out to be a pretty limited solution from a productivity
               | perspective (or at least that seems to be the majority
               | opinion among people who have extensive experience with
               | both languages) compared to borrow checkers--sometimes
               | it's appropriate to trade productivity for
               | performance/correctness, but many times it isn't.
        
               | scruple wrote:
               | > I think they made assumptions that engineers cannot be
               | trusted.
               | 
               | Having spent ~2 decades working with engineers, I'm
               | inclined to agree... Past, present, and future self
               | included!
        
             | didntcheck wrote:
             | Don't shoot the messenger, the condescension is Go and
             | Pike's. He pretty much explicitly said he wrote Go to solve
             | the problem of programmers who he considered too stupid to
             | program anything else properly, and his behaviour (both in
             | personal conduct and decision making) has continued this
             | trend of treating his users like idiot sheep who can't be
             | trusted with any language features much beyond 80s
             | imperative programming. It all feels a bit "old man
             | spiteful at kids these days"
        
           | throwaway894345 wrote:
           | I don't think this is a bad thing. Most developers think of
           | themselves as more advanced than they are and end up making a
           | mess trying to do abstraction beyond both their ability to do
           | it well and beyond what the problem requires. And even the
           | truly advanced developers very often prefer simpler tooling
           | because it allows them to concentrate their resources on the
           | complexities of the problem and not those of the tool.
           | 
           | Further, the idea that Go is somehow bad for motivated
           | engineers doesn't match my experience and it doesn't explain
           | the proliferation of interesting tools and services (much of
           | the container / devops ecosystem). I think your mistake is
           | assuming that the only kind of motivation/creativity/etc
           | involves tinkering with language runtimes and that someone
           | who is building original things like Docker, Terraform, or
           | Kubernetes must be "unmotivated".
        
           | fuzztester wrote:
           | As someone else here said, it may not be Kernighan who said
           | that, but Rob Pike.
           | 
           | And he didn't say lowest common denominator developer, he
           | said something like junior Google developer.
           | 
           | And IIRC he said it somewhere in his blog, called command-
           | central.blogspot.com or some such.
        
             | throwaway894345 wrote:
             | He was also contrasting it with C++ which has many dozens
             | of features and a developer has to understand the interplay
             | between all of them to use the language effectively.
        
               | fuzztester wrote:
               | Right, I remember, and in this case, "a developer"
               | applied to not just future junior Googlers, but also him
               | and some of his project mates, and in that same post he
               | said they hated or at least disliked those issues with
               | C++.
        
           | cube2222 wrote:
           | > once you want to go venture outside the safety of the
           | walled garden you can't. The language is specifically
           | designed to keep you as safe as possible.
           | 
           | > Bad for the motivated engineer.
           | 
           | Sounds to me like it's really just bad for the engineer who
           | wants to show off how smart they are through the
           | overcomplicated custom abstractions they build.
           | 
           | Condescending response to condescending comment aside, I
           | understand that those custom abstractions can occasionally be
           | great and have their place, but in practice their value
           | usually starts reversing (yes, reversing, not diminishing)
           | once you need to onboard many engineers frequently (junior or
           | senior alike), which is the case for a big part of tech
           | projects (think big tech, high growth startups, but also open
           | source projects where you want first-time contributors to be
           | able to dive in and solve their issue quickly).
           | 
           | And I'm saying this as a motivated engineer who loves playing
           | with various languages but uses Go for most serious things.
        
           | lelanthran wrote:
           | I think you're incorrect.
           | 
           | It's _good_ for the motivated engineer, because instead of
           | running off and creating architecture astronautical wonders
           | of over-engineering, they are getting down to business and
           | making progress to a goal.
           | 
           | Your bottom-tier engineers aren't going to be any better in
           | Go than they are going to be in Java.
        
           | Philip-J-Fry wrote:
           | >I don't mean this in a sarcastic way but Go was explicitly
           | designed for the lowest common denominator developer.
           | Kernighan said it himself.
           | 
           | What's the issue with making Go good for the lowest common
           | denominator developer? This is fantastic for businesses as
           | they can get new developers productive within a few days.
           | It's great for the ecosystem because the barrier of entry is
           | low.
           | 
           | Just because it's made simple and easy to pick up doesn't
           | mean you can't write complex software with it. It's a
           | programming language, you can write whatever you want with
           | it. If it makes writing complex software easier, then that's
           | _a good thing_. The only reason why someone would think that
           | 's not a good thing is their ego because $FAVOURITE_LANGUAGE
           | has 10x as many language features and 10x as many ways to do
           | the exact same thing.
           | 
           | What is this walled garden you think exists that prevents
           | motivated engineers from doing good work? I like to think of
           | myself as motivated and Go is the reason for that. It made
           | writing complicated concurrent software so much easier for
           | me, and it made reasoning about them so much easier. I do not
           | understand this criticism of Go.
        
             | didntcheck wrote:
             | I can't help but feel that the reason why new developers
             | reach parity with existing ones so quickly is not due to
             | the productivity of Go, but because the productivity
             | ceiling has been intentionally lowered so that everyone is
             | equally constrained. Having to keep reading reinvented
             | wheels because the language is _intentionally_ poorly
             | expressive is a burden on getting work done. There maybe is
             | only one way to do a thing, and everyone has to keep doing
             | it, rather than being able to write it once and build an
             | abstraction
             | 
             | Compare say Java 5 code written with raw servlets
             | implementations and JDBC calls versus modern Spring code.
             | The _implementation_ of the former is easier to understand
             | because everything is very explicit, but the _intent_ of
             | the latter is far easier to understand, since the
             | boilerplate is mostly gone, allowing the actual business
             | logic (what you care about 90% of the time) to be obvious.
             | And you can of course see the implementation if you need to
        
               | erik_seaberg wrote:
               | Every time I hear "our experts' code can look like our
               | beginners' code" I cringe, because experts' time is much
               | more valuable and they should be communicating concisely.
               | A beginner's job is to become an expert.
        
       | lalaithion wrote:
       | My experience with Goroutines is that eventually you run into the
       | limits of working with them and your code explodes in size. Your
       | for item := range channel {             // code goes here
       | }
       | 
       | becomes                   for {             var item Item
       | var ok bool             select {             case <-ctx.Done():
       | break             case item, ok = <-channel:                 if
       | !ok {                     break                 }             }
       | // code goes here         }
       | 
       | (If you have a solution to this pattern, please do share! I would
       | _love_ a better way.)
        
         | quadcore wrote:
         | It does become a channel mess. The trick I use is to code
         | everything using the same patterns (down to the same
         | characters, name conventions, line breaks, etc) as much as I
         | can so i can use the text editor to work (refactor) that mass.
         | It's definitely the less funny part and why I'll still use
         | clojure and it's async.io module whenever it makes sense (lots
         | of different data). That way, I macro myself out of that
         | problem.
         | 
         | Now if you cant use another language, change project. Go is not
         | perl nor python, it's a sophisticated C. Use a sophisticated C
         | for the wrong project / product and you'll want to kill
         | yourself.
        
         | chrsig wrote:
         | Generics!                   func streamingApply[T any](ch
         | channel[T], fn func(T)){               for {
         | select {                 case <-ctx.Done():
         | break                 case item, ok = <-ch:
         | if !ok {                         break                     }
         | fn(item)                 }             }         }
         | 
         | For everyone that's complained about generics being introduced
         | to go...It really does help clean up this type of repetition.
        
         | collinvandyck76 wrote:
         | If your problem fits, you can sometimes keep the former version
         | in play with the pipeline pattern, where child goroutines are
         | stopped when the parent goroutine closes the channel in a defer
         | when it's exiting.
         | 
         | That being said, my experience is that anything moderately
         | sophisticated requires the use of for/select. In my opinion,
         | that's fine, because select gives you so much power the
         | somewhat more complex form is worth it.
        
         | Philip-J-Fry wrote:
         | What's the limit here?
         | 
         | That code is written like that because it's the only way to
         | write code like that. You have a channel producing items, it
         | will produce items for as long as the channel is open. You have
         | a context to propagate cancellation.
         | 
         | You code gets bigger but it doesn't explode in size as this
         | boilerplate is static in size regardless of what computation
         | you're doing with those items.
         | 
         | You can improve that code by the way by fixing a bug and
         | reducing the lines.                   L:             for {
         | select {                 case <-ctx.Done():
         | break L                 case item, ok := <-channel:
         | if !ok {                         break L                      }
         | // code goes here                 }             }
         | 
         | If you think there's a way to write that code any more concise
         | than that, then there isn't. I don't see how this is an issue.
         | The fact that there's pretty much one way to write this code
         | makes it so much easier to parse when reading code that uses
         | this pattern. To me, that's great.
        
           | [deleted]
        
           | valczir wrote:
           | alternatively:                   var errClosed =
           | errors.New("closed")              func doAThing[T any](ctx
           | context.Context, c chan T) (T, error) {             // the
           | generics are not necessary, but they're also not awful for
           | // DRYing up this type of a select.             select {
           | case <-ctx.Done():                 var empty T
           | return empty, fmt.Errorf("could not doAThing: %w", ctx.Err())
           | case item, ok := <-c:                 if !ok {
           | var empty T                     return empty,
           | fmt.Errorf("channel of type %T is done: %w", empty,
           | errClosed)                 }                 return item, nil
           | }         }              // ... and later...         for {
           | item, err := doAThing(ctx, channel)             if err != nil
           | {                 break             }             // code
           | goes here         }
           | 
           | I dislike labeled loops, naked returns, and else statements,
           | so this is the pattern I use.
        
         | kinghajj wrote:
         | If your loop body is completely CPU-bound, then either adding a
         | select statement, or a separate goroutine that awaits
         | cancellation and closes the channel like collinvandyck76
         | suggested, will work. But commonly the body will make other IO-
         | bound calls to which the context gets passed, so you can just
         | handle the error.                   for item := range channel {
         | _, err := process(ctx, item)             if err != nil {
         | return err             }         }
        
       ___________________________________________________________________
       (page generated 2023-07-07 23:02 UTC)