[HN Gopher] Effective Go
___________________________________________________________________
Effective Go
Author : mkl95
Score : 281 points
Date : 2021-10-03 08:57 UTC (14 hours ago)
(HTM) web link (golang.org)
(TXT) w3m dump (golang.org)
| tomohawk wrote:
| What's with the partisan advocacy prominently displayed as a
| banner?
|
| EDIT: I mean, it's hardly effective. Just like the downvotes.
| gvv wrote:
| everything is political nowadays.
| hdlothia wrote:
| When were things not political?
| awb wrote:
| > Go is a new language
|
| It's been 12 years next month. Might be time to change that?
| dragonwriter wrote:
| > It's been 12 years next month.
|
| Since being announced, ~9 since 1.0.
|
| For comparison of some other currently popular languages (above
| it on the Tiobe Index), time from 1.0 (or something similar
| where typical modern versioning isn't applicable) is: Python 27
| years, JavaScript 24 years, C 43 years (from K&R, realistically
| might count 1.0 earlier), C++ 36 years, Java 25 years, C# 19
| years, Ruby 19 years, VB.Net 20 years, classic VB 30 years,
| Groovy 14 years.
|
| Go is a relatively new language. Not quite as young as Rust,
| but young for an industrially-popular language.
| [deleted]
| [deleted]
| charles_f wrote:
| As languages go, both its age and spread still justifiably
| qualify it as "new"
| [deleted]
| oxplot wrote:
| A bunch of other docs I've looked up number of times and found
| handy:
|
| - Slice tricks: https://github.com/golang/go/wiki/SliceTricks
|
| - Rest of the Golang Github Wiki:
| https://github.com/golang/go/wiki#additional-go-programming-...
|
| - Spec - very approachable: https://golang.org/ref/spec
|
| - Standard library source code - clean code with lots of idioms
| to learn from:
| https://cs.opensource.google/go/go/+/refs/tags/go1.17.1:src/
| parhamn wrote:
| Hopefully, a ton of the slice tricks will soon be replaced with
| "container/slice" (although you'll still need a few)!
|
| [1] https://github.com/golang/go/issues/45955
| louis-paul wrote:
| Also https://github.com/golang/go/wiki/CodeReviewComments
| parhamn wrote:
| Cool I had never seen this before. All seem on point, but I
| have light disagreements with this one:
|
| > The basic rule: the further from its declaration that a
| name is used, the more descriptive the name must be. For a
| method receiver, one or two letters is sufficient. Common
| variables such as loop indices and readers can be a single
| letter (i, r). More unusual things and global variables need
| more descriptive names.
|
| In the tighter parts of some algorithms I love seeing more
| descriptive variables (obvious exclusions, as noted, are
| things like i, j, x, y). It's very common to see double
| letter variable names that become hard to track (plenty in
| the go source as well).
| crecker wrote:
| > Why is there no pointer arithmetic?
|
| Safety. Without pointer arithmetic it's possible to create a
| language that can never derive an illegal address that succeeds
| incorrectly. Compiler and hardware technology have advanced to
| the point where a loop using array indices can be as efficient as
| a loop using pointer arithmetic. Also, the lack of pointer
| arithmetic can simplify the implementation of the garbage
| collector.
|
| <3
| dnautics wrote:
| > Compiler and hardware technology have advanced to the point
| where a loop using array indices can be as efficient as a loop
| using pointer arithmetic.
|
| I literally refactored some code from index arithmetic to
| pointer arithmetic and bought a 10% increase in performance for
| an (admittedly silly) numerical performance contest on Friday,
| so I'm not convinced either LLVM or the "jit compiler that
| reorders operations inside the x86 chip" are that smart yet.
|
| That said I would not doubt that most modern languages convert
| iterables that look like index arithmetic into pointer
| arithmetic, but if they do so I would suspect it's at an IR
| level above the general compiler backend.
| Matthias247 wrote:
| In which language?
|
| In Rust iterators are actually the fastest way to safely
| iterate over arrays, because they will elide bounds checks.
| If you use `array[index]` you will get a mandatory bounds
| check (branch) on each access. Using pointer arithmetic would
| avoid that, but is unsafe in Rust for obvious reasons.
|
| In C I would assume indexes and pointer arithmetic to have
| exactly the same performance, since the `array[i]` is the
| same as `*(array + i)` and there are no mandatry bounds
| checks. Might be interesting to move your code in godbolt and
| see what actually changes here.
| [deleted]
| greyman wrote:
| Yes. And on the more general sense, I noticed Go tried to
| remove everything which can obfuscate the code and make it
| harder to read, or become junior-unfriendly. At least that's my
| impression.
| masklinn wrote:
| > I noticed Go tried to remove everything which can obfuscate
| the code and make it harder to read, or become junior-
| unfriendly.
|
| That is unsurprising
|
| > They're typically, fairly young, fresh out of school,
| probably learned Java, maybe learned C or C++, probably
| learned Python. They're not capable of understanding a
| brilliant language but we want to use them to build good
| software. So, the language that we give them has to be easy
| for them to understand and easy to adopt.
| pjmlp wrote:
| The irony of that statement is that a language whose
| version 1.0 was pretty much Go in 1996, is now seen by them
| as a PhD level language.
| ori_b wrote:
| I suspect Rob would say that Java 1.0 was a better
| language than today's Java.
| pjmlp wrote:
| Limbo was being positioned against Java, so I have my
| doubts.
| ori_b wrote:
| I'm not sure how that has a bearing on how someone would
| the versions of Java.
| joconde wrote:
| Which language do you mean? Python 1.0 seems to have come
| out in 1994 (https://en.wikipedia.org/wiki/History_of_Pyt
| hon#Version_1).
| goatlover wrote:
| OCaml came out in 96. That would be PhD level in
| comparison. As for Python, it's interesting how it was
| considered a simple, easy to use language that focused on
| doing things one way back then. But now nearing version
| 4, it's turned into a complex language with many ways to
| do things. Makes one wonder how long Go will be able to
| hold out for. JS has gone the same route as Python. I
| guess Scheme managed stay simple, but it was never that
| popular. OTOH, It's cousin CL is complex.
|
| To paraphrase Stroustrup:
|
| There are two kinds of languages: the simple ones
| (Scheme, Smalltalk, early Python and JS), and the ones
| everyone uses.
| pjmlp wrote:
| I was referring to Java.
|
| OCaml is not on the languages referred by Rob Pike.
| pjmlp wrote:
| Nothing on my comment was about Python, and Rob Pike
| mentioned others, like Java.
| Adiqq wrote:
| In general, that seems like greatest improvement that can be
| made, because they optimized language itself. They made it
| efficient, but still readable, so you can just read or write
| the code without much/any Googling. I never really understood
| why simplicity was considered especially "junior-friendly",
| while actually more experienced people can benefit the most
| from it. It's much easier now to just visualize algorithms
| used in code, without introducing tricky notations. It's also
| much easier to just write your code, when you finally have
| syntax and stdlib that will cover majority of typical cases,
| instead of language, where you need to either reinvent the
| world or use over-engineered libraries to solve common
| problems.
| kaba0 wrote:
| Contrary to this mindset of simplicity over everything
| else, abstraction _is_ the very bane of existence of the
| whole field. While there is indeed accidental complexity we
| should strive to avoid, there is essential complexity which
| can only be managed through proper abstractions.
|
| So that over-engineered library that actually solves the
| problem is the only real productivity benefit we can
| achieve, since languages _don't_ beat each other in
| productivity to any significant degree (see Brooks).
|
| So while one may not want to go into macro-magic with a
| semi-junior team, no abstraction will just as much kill a
| project because LOC will explode, it will have tremendous
| amounts of buggy copy-pasted/multiple times implemented
| code, etc. And the few empirical facts we know about
| programming is that bug count correlates linearly with LOC.
| hsn915 wrote:
| Actually you _can_ do pointer arithmetics. It's just not
| straight forward, and the behavior is "unefined" in the sense
| that even if it works today, it might break in future versions.
| throwaway894345 wrote:
| Yes, but it's parked behind "unsafe" and pretty sure it will
| still work in future versions...
| [deleted]
| rawoke083600 wrote:
| Go has warts and 101 faults (depending who you are and how hard
| you squint)...
|
| But what I absolutely love about Go is how easy(relative to other
| languages) it is to 'read' most of the Go code in the wild,
| including the std libs !
|
| YMMV...
|
| Every tried reading C++ libs/header-libs ?
| msie wrote:
| Same feeling I had about Python vs Ruby. When I was working on
| a Python script I had no qualms about diving into the source of
| libraries. Although I have been scared away from Python because
| of the many different ways that libraries can be included in a
| script.
| trey-jones wrote:
| This has always been my number one praise for Go as well. A lot
| of it has to do with gofmt I think.
| closeparen wrote:
| I want to see an "Effective Go" about unit testing functions that
| call other functions, without devolving into meaningless mocking
| and error propagation boilerplate.
| morelisp wrote:
| For effective testing in Go:
|
| - Don't unit test so much (or expand your definition of "unit",
| or whatever semantic difference you prefer). Test
| functionality. (Unit tests can still be appropriate for large
| classes of complicated pure functions, e.g. parsers, but these
| don't require mocks.)
|
| - Don't mock so much; rather, stub (or mock if absolutely
| necessary) at lower levels. Use the httptest server; use the
| sqlmock or sqlite drivers; use a net.Conn with canned data;
| etc.
| closeparen wrote:
| I would appreciate any resources you could point me towards
| to help make this argument against the Staff+ engineering
| leaders at my company who are pushing standards that say
| exactly the opposite.
| morelisp wrote:
| This sounds a lot more like company politics than a
| technical issue, but I would probably start with Mitchell
| Hashimoto's talk "Advanced Testing With Go" - along with
| the just, like, reading the tests / testing tools in
| stdlib. They didn't include httptest so you could spend
| time mocking away http.Client usage behind an interface!
|
| (I should add that this is explicitly contra to e.g
| sethammons's suggestion above, which seems to be relatively
| common in the part of the Go community that come from PHP.
| I inherited a couple large projects that did this. Today
| they use sqlite instead, and both the program and test code
| is ~50% the size it used to be.)
|
| For us, stub injection points come naturally out of
| 12-factor-style application design; the program can already
| configure the address of the 2-3 other things it needs to
| talk to or files it needs to output, etc, just out of our
| need for manual testing or staging vs. production
| environments. If you have technical leadership encouraging
| Spring-but-in-Go, you'll probably hit a wall here too
| though.
|
| It's also possible you're simply writing too many functions
| that can return errors. Over-complex code makes over-
| complex tests; always think about whether you're handling
| an _error_ or a _programming mistake_ - if the latter,
| panic instead of returning.
| sagichmal wrote:
| Maintainable software projects are modeled as a dependency
| graph of components that encapsulate implementation details
| and depend on other components. func main
| foo, err := NewFoo() handle err
| bar, err := NewBar(foo) handle err
|
| Given a single component, each external dependency should
| be injected as an interface. type Fooer
| interface{ Foo() int } func NewBar(f
| Fooer) (*Bar, error) { ... }
|
| Test components in isolation by providing mock (stub, fake,
| whatever, it's all meaningless) implementations of its
| dependencies. func TestBar(t *testing.T)
| { f := &mockFoo{...} b, err :=
| NewBar(f) ...
| deanCommie wrote:
| From what I understand (from making similar complaints to
| Gophers), if you're complaining about boilerplate, you don't
| get it.
|
| The boilerplate propagation is the point in Go.
|
| Go isn't DRY. Go would rather copy and paste a bunch of code
| than introduce the "complexity" of inheritance.
| closeparen wrote:
| I am okay with the level of abstraction available in
| production code. Where it gets ridiculous is the tests. Unit
| testing the simplest, most obvious Go code is a huge chore.
| Each error-returning function call costs 15 seconds to type
| but 15 minutes to work into the tests.
| ed25519FUUU wrote:
| There is unnecessary friction, which doesn't really seem to
| fit with the Go ethos. I would have assumed from afar that
| testing gets a lot of consideration in a language like Go.
| alecthomas wrote:
| It does. It sounds like the parent has had an unfortunate
| interaction with a bad codebase, because that is not
| accurate at all.
| closeparen wrote:
| I am not being sarcastic when I say I want an "Effective
| Go" for this. If you have examples of high quality tests
| in an MVC-style service codebase, I would love to see
| them!
| sagichmal wrote:
| "MVC" is not an idiomatic pattern in Go.
| mxz3000 wrote:
| But you can obviously DRY without inheritance?
|
| I'm not sure that Go pushes you to duplicate code... Do you
| mean copy a slightly tweak code for different use cases and
| types ?
| sethammons wrote:
| I love testing in Go because I avoid meaningless mocking. Test
| structs that match interface signatures are meaningfully used
| to validate any state handling and/or error generation, and
| ensures error paths are properly exercised. In unit tests, we
| validate logs and metrics when appropriate in addition to
| return values. However, if you are mocking db, http or other
| net packages, you are likely doing it wrong. You want to know
| you handle an error from the db, you don't have to mock a db:
| you have a struct whose interface has 'GetUser(username)
| (*User, error)', and your test version returns an error when
| needed. The fact the real code will use a db is an
| implementation detail. You should be able to refactor the real
| implementation to change from a db call to an http api call and
| still have valid, useful tests. Elsewise, your unit tests are
| too coupled and hinder refactoring. Anyway, I love testing in
| Go; it is one of my favorite parts of working with it.
| eyelidlessness wrote:
| Disclaimer: I've never worked with Go, there may be some
| nuance here I'm missing.
|
| This sounds like dependency injection of test behavior as a
| substitute for the "real" IO implementations injected in
| whatever glue code accesses the unit under test. If that's a
| correct understanding, it sounds like mocking by another
| name?
|
| Not that I think there's anything necessarily wrong with
| that. And I think it that kind of inversion of control _can_
| often produce more robust designs /systems/tests. But I think
| it's a good idea to recognize that's what it is, and that it
| has similar limitations to other mocking techniques.
| closeparen wrote:
| Mocks are simply auto-generated and consistent test structs.
| And you still need to test the code which implements GetUser
| based on DB or HTTP or whatever else.
| mxz3000 wrote:
| What do you mean? If you compose functions (and objects)
| together then you have to mock things out one way or another,
| inject them in, and assert that your code interacts correctly
| with its dependencies.
|
| This doesn't seem like a go specific issue but an engineering
| one.
| closeparen wrote:
| In Java or Python you do not normally need write a test case
| for "if this dependency fails, it will be propagated to the
| caller." In Go you have to do this or your line/branch
| coverage will be abysmal.
|
| Java and Python mocking is also a lot more lightweight; you
| don't have to generate mocks ahead of using them, regenerate
| them when the interface changes or work generation into your
| build process, etc. Richer reflection APIs make it a pretty
| casual handful of characters to mock something out.
| ASinclair wrote:
| > regenerate them when the interface changes
|
| You don't need to do this in Go either. Embed the interface
| type in your mock type. You only need to implement
| functions that will be called in the function under test.
| morelisp wrote:
| If your Java code doesn't test exceptions that could be
| thrown, your branch coverage is equally abysmal _and your
| tools aren 't telling you that._
| closeparen wrote:
| The language's exception facility does what it says. We
| don't need a unit test to prove that in every particular
| case, any more than we need a unit test to establish that
| the language correctly carries out our assignments or
| function calls.
| morelisp wrote:
| You picked two languages (Java and Python) with extremely
| weak guarantees about disposal of resources when
| exceptions are thrown. Any time I acquire a non-trivial
| resource I must make sure it's disposed of properly, via
| a `close` etc. method. (And these are most cases of
| interest; acquiring trivial resources e.g. memory
| shouldn't need error handling in Go either.)
|
| This isn't theoretical. I review a lot of Python code and
| I would say in over 20% of cases I see a try block with a
| `finally` longer than two lines, it mistakenly uses a
| variable that might not be set when an exception is
| thrown earlier than the writer expected.
| closeparen wrote:
| When a controller calls a couple of gateways and a
| database access layer, it is almost never holding
| resources. I would agree that testing error handling is
| more interesting when there are resources to clean up or
| fallback logic to implement. That's just very rarely the
| case. Mostly I just need to bail out of the request.
| Macha wrote:
| Java has try-with-resources and Python has context
| managers, I wouldn't consider either of them harder to
| use correctly than defer.
| morelisp wrote:
| I'm not talking about defer at all. The contrast would be
| C++'s RAII.
| jopsen wrote:
| > In Go you have to do this or your line/branch coverage
| will be abysmal.
|
| If you're writing tests to improve coverage numbers then
| maybe you're motivation is wrong.
|
| In most languages with automatic exception propagation you
| never know what exceptions can be thrown. Is it any
| different from not testing?
| [deleted]
| closeparen wrote:
| It would be interesting to know what errors we might get,
| but unit testing an error return branch doesn't tell us
| that. It tells us that any non-nil error will be
| returned.
| AlexCoventry wrote:
| > In Go you have to do this or your line/branch coverage
| will be abysmal.
|
| If reliability is critical, having explicit error handling
| and coverage tools which expose error conditions which
| haven't been tested is very helpful.
| closeparen wrote:
| I make a sequence of calls during the service of a
| request. If any one of them fails I want to stop and
| return the error to the caller.
|
| Having to check this in the particular case of every call
| at every later underlying every handler, does not make my
| software more reliable in than when it is simply
| guaranteed in general.
| AlexCoventry wrote:
| Explicit error handling can improve reliability in some
| circumstances, because it highlights corner cases which
| are easy to ignore otherwise.
| abahlo wrote:
| Shameless plug on my opinionated [Go
| Styleguide](https://github.com/bahlo/go-styleguide) - let me know
| what you think!
| t8sr wrote:
| Since you asked for feedback: Good style guides contain
| rationale, weigh pros and cons and then make a recommendation.
| For example, you say "use an assert library", but why? You make
| an assertion (har har) about consistent error output, but I've
| seen style guides that use the exact same rationale to ban
| assert libraries.
|
| This reads more like a list of things you wrote to remind
| yourself of, than something intended for consumption by other
| programmers.
|
| It's fine to be opinionated, but a normative document must
| explain itself, so people (1) learn something and (2) know when
| a rule isn't really a rule and (3) can make an informed
| decision to change something in 5 years when you no longer work
| there.
| abahlo wrote:
| Good points, thank you!
| greyman wrote:
| I feel overall those are very reasonable rules. What I do
| differently:
|
| 1) Add context to errors - ok but sometimes the parent caller
| doesn't expect or wants that.
|
| 2) one-letter variables: I use them more, for example k,v for
| key,value loop through the map, n can be counter, or x is some
| value inside the loop.
| abahlo wrote:
| Yeah I actually agree, will make them sound less like a must.
| In early Go times people would use one-letter variables
| everywhere which was horrible.
| charles_f wrote:
| This doc goes over pages of conventions, patterns, design advice,
| tips, explanations, etc. without once using the idiom "best
| practice". It is balanced and sometimes mention that advice can
| be taken too far.
|
| I salute you for that!
| google234123 wrote:
| So why did we need go when Java already existed?
| cy_hauser wrote:
| Because Java has become an unpleasant to code in. Do you really
| think what you like is what everyone else likes?
| kaba0 wrote:
| While I agree with sibling poster that modern java is nothing
| like the old times, are we seriously comparing it to Go with
| manual error handling and the like?
| manishsharan wrote:
| If you are still coding in Java 5 then yes it is unpleasant.
| I have seen a lot of java 5 like code in Java 8 environments
| and hence the unpleasantness. However, even though I am still
| upskilling myself to use Java 11 and later, I am finding that
| my Java code is less verbose and more efficient.
| fmakunbound wrote:
| I sorely miss some kind of functional streams api in Go
| when switching between Go and Java. for-loops make the
| equivalent code absolutely awe full, plus it's not
| automatically parallelizable by replacing .stream() with
| .parallelStream(). IIRC, some Go luminary decided map,
| apply filter etc. we're no good, so I hold little hope of
| something like Java streams API in the future.
|
| Maybe someone can recommend a third party library.
| closeparen wrote:
| I question the relevance of newer Java versions.
| Established codebases don't like to upgrade and new
| codebases are likely to use trendier languages.
| vips7L wrote:
| Are we looking at the same languages? Java is far more a
| productive language with generics, lambdas, switch
| expressions, pattern matching.
| bob1029 wrote:
| > Java is far more a productive language
|
| Since when is productivity a priority for most developers?
| Seems like 2nd or 3rd fiddle based on HN sentiment these
| days.
| cy_hauser wrote:
| Java (and C# and C++) have become kitchen sink sink
| languages. Everything is just thrown in. That makes it less
| productive for me. I've decided I don't want to carry
| around that much in my head all the time in order to read
| somebody else's code. I switch languages a lot these days.
|
| Go is getting generics in (hopefully) in a few months. Some
| features are missing but most don't cause much pain and can
| be either simulated or worked around. Error handling is its
| biggest failure, but outside of that I find is at least as
| productive as other compiled languages in the same space.
| vips7L wrote:
| > Java (and C# and C++) have become kitchen sink sink
| languages. Everything is just thrown in.
|
| This is far from the truth when it comes to Java. Java
| doesn't add features willy nilly. They are thought out,
| and in general implemented better than in other
| languages. Java's WHOLE philosophy is to move slow and
| let other languages experiment with features so the
| kitchen sink doesn't get thrown in because Java has
| _actual_ backwards compatibility promises and has to
| support those features for eternity.
| dunefox wrote:
| I don't know, why did we need Java when CL/Smalltalk/etc.
| already existed?
| Redoubts wrote:
| Deployment story is a tentpole feature, of which these two have
| very different ideas.
| closeparen wrote:
| Try writing e.g. a TLS proxy in Java and then in Go, and get
| back to us.
|
| For byte arrays, sockets, encodings, cryptography, etc. the
| stdlib in unparalleled.
| kaba0 wrote:
| Not arguing that Go is a great fit to these niches, but imo
| java has a really great standard library, including these
| areas.
| matthewmacleod wrote:
| Because they're two different languages, tackling different
| problems, with entirely different sets of benefits and costs
| associated with their use.
| google234123 wrote:
| That's not enough to justify the (10s, 100?) millions of
| dollars of investment. Java is faster and safe and can do
| everything go can do. C++ can be even faster (at least there
| are no pauses from a gc) at the cost of some safety.
| kgraves wrote:
| > Java is faster and safe and can do everything go can do.
|
| Including generating static binaries without the use of
| requiring to download a runtime?
| kaba0 wrote:
| Yes, there were AOT compilers availbe for Java since
| 2000. But more recently, Graal creates very nice
| binaries.
| weavie wrote:
| Yup, with everything else being equal, given the choice I
| will always use the project written in Go.
| mindwok wrote:
| The fact that Go has gained so much adoption probably
| justifies that investment. They are completely different
| languages and much of it is personal preference.
| zerr wrote:
| Go will loose significant market share as soon as Java and .NET
| have a mature and usable AOT compiler.
| henry700 wrote:
| Soon. Still, .NET Core is already faster even without AOT on
| most cases: https://benchmarksgame-
| team.pages.debian.net/benchmarksgame/...
| tonyedgecombe wrote:
| Oracle.
| kaba0 wrote:
| Which increased the pace of development of Java and fully
| open-sourced it and is overall being a really great steward
| of the language? That Oracle?
| cy_hauser wrote:
| No, the other one. The Oracle we hate and have hated for
| more than 20 years now.
| noxer wrote:
| Still has that political banner on top. Disgusting.
| [deleted]
| ihusasmiiu wrote:
| When I look at a language I could not care less about the
| selection of keywords, the syntax used for a loop or the naming
| conventions. What I look for are the means of abstraction that
| the language provides, and Go gives nothing more than C.
| chubot wrote:
| Go has strings, slices, maps / hash tables, channels, and
| interfaces. C doesn't have any of those things. (It has quirky
| string libraries, and string literal syntax, and that's about
| it)
| qbasic_forever wrote:
| That's an explicit feature and not a bug for the design of the
| Go language. They don't want it to be a kitchen sink of a
| language with everything for everyone.
| umvi wrote:
| > Go gives nothing more than C.
|
| Huh? At the very least Go has a safe, robust string abstraction
| which C does not have. Not to mention slices, maps, range based
| for loops, a "batteries included" style standard library, and
| more...
| kgraves wrote:
| I always refer myself and junior developers to Effective Go to
| sharpen their skills when working in our codebase.
|
| Are there other hidden Go gems/guides that exist that I happen to
| not know about as well?
| val_deleplace wrote:
| Yes, definitely dozens of pretty good online resources. Here
| are 3:
|
| - Tour of Go: https://tour.golang.org/
|
| - Practical Go: Real world advice for writing maintainable Go
| programs, by Dave Cheney: https://dave.cheney.net/practical-
| go/presentations/gophercon...
|
| - Uber Go Style Guide: https://github.com/uber-
| go/guide/blob/master/style.md
| rob74 wrote:
| - Go Code Review Comments:
| https://github.com/golang/go/wiki/CodeReviewComments
| rawoke083600 wrote:
| yes a talk from one of the big wigs in GO. Think its called
| 'Rethinking Conccurency' ?
| mhoad wrote:
| Not a Go programmer but Dart also has something like this for a
| while which has been a pleasure to use, glad to see others can
| get the same benefits now.
|
| https://dart.dev/guides/language/effective-dart
| na85 wrote:
| I really liked using Go until I tried to publish a module with a
| version greater than 1.
|
| What kind of packaging system chokes on v2 of a module? What kind
| of language recommends[0] with a straight face that you just
| duplicate your entire codebase into a v2/ subdirectory?
|
| The module system is so obtuse and the documentation so poor that
| I will probably avoid go and choose other languages from now on.
| It really feels like they came to release time and someone in the
| meeting said "hey guys what if someone wants to release a new
| major version of their library?" and everyone else in the room
| had an "oh shit" moment because their amateur language design
| didn't address that corner case.
|
| [0] https://go.dev/blog/v2-go-modules
| sudhirj wrote:
| The reason I like this is because of one of the principles that
| Go espouses pretty heavily - never break backwards
| compatibility. If I install v1 of your package, I want it never
| to break. You can always improve performance, add more methods,
| etc, but the normal Go upgrade does not involve any code
| changes at all. The tools formalise that for modules as well,
| and encourage building a new module altogether if backwards
| compatibility is being broken.
|
| If it's not being broken, it's not a major version update, and
| major breaking updates are better off as new separate modules.
| grey-area wrote:
| Except _in practice_ in all real software (including the go
| language!) minor and patch versions break importers all the
| time in small ways, AND major versions are usually used to
| indicate significant new features, not just breaking changes.
|
| So your claim is reduced to 'I don't want v1 to break my
| software in ways that the dependency finds important enough
| for a major version'.
|
| Not quite as reassuring is it?
|
| Plenty of security fixes and undefined behaviour fixes break
| importers all the time, and often importers and exporters
| disagree about the severity of that. Sticking with v1 will
| not save you that pain.
|
| I preferred the situation before where there was strong
| social pressure never to break the main branch.
| miki123211 wrote:
| The problem here is that we add too much extra meaning to
| version identifiers. In semantic versioning, if you're at
| 1.38 and want to release a cool new version with many cool
| features and a much faster engine, while still retaining
| backwards compatibility, you're stuck with releasing a
| 1.39. If you call your new version 2.0, you signify that
| it's backwards incompatible.
|
| We should have two differently-looking version identifiers,
| one for compatibility-tracking purposes and one for
| signifying the scope of changes to humans. The
| compatibility version numbers could look like 2ah4, where
| the first number is the major version, the letter (or
| series of letters) is a minor, increasing alphabetically
| and going to "aa" after "z", and the last number is a
| patch.
| grey-area wrote:
| I think they are always fuzzy and that's fine, as long as
| tooling doesn't make incorrect assumptions about meaning.
|
| Different exporters and importers have different
| stability requirements and expectations - it's
| negotiated.
| hsn915 wrote:
| This is a cultural issue.
|
| It's not inevitable that by developing a module over time
| that you constantly break compability AND you want all
| people that depend on you to upgrade version.
|
| Go encourages the opposite approach.
|
| If your API is not stable, don't publish a package for
| public consumption, or, make sure to clearly mark it as
| unstable, so people know what to expect.
|
| One of the absolute best features of Go is stability. If I
| have a project written a few years ago, I can always run it
| against the latest Go compiler and _know_ that it will
| still work.
|
| Why does this matter? I'll give you a counter example.
|
| In 2016 I wanted to work on a mobile application project
| and I just picked React Native because I heard it lets you
| easily create cross platform applications.
|
| I worked on it for a few weeks, then left it for about two
| or three months. When I tried to pick it up again, I
| installed the latest versions of node and npm and upgraded
| all packages.
|
| The project stopped working. I had no idea what went wrong.
| The error messages were obtuse and several layers deep. I
| tried to debug it for weeks to no avail.
|
| As a result, I abandoned the project and built just an
| Android version using Kotlin and Android Studio.
| tialaramex wrote:
| > I installed the latest versions of node and npm and
| upgraded all packages.
|
| Laughter of recognition from this side. I am working on
| an application that has a tiny bit of Web UI. Basically
| one button, "Yes, I want this thing" and some text that
| guides the person staring at the button to decide if they
| in fact do want this or don't. I didn't write any of that
| code, and my Javascript knowledge is pretty rudimentary,
| but hey it's just a button right?
|
| On Monday I'm sat (virtually) with the guy who wrote that
| part, pair programming the dependency injection stuff so
| that this button can actually cause stuff to happen. It
| builds and runs fine on my machine, we make lots of
| progress. During the week I focus on other parts of the
| system (including a red herring where we're convinced for
| an hour we have a DST Root CA X3 expiry issue but it's
| actually a syntax error in a Puppet file on the in-house
| node classifier), but on Friday we're ready to check the
| current state. However the guy who wrote the button code
| is out. No problem though right, I understand the larger
| system, just hit build?
|
| Nope. I get a JSON parsing error, which, it turns out, is
| how npm or node communicates "I am the wrong major
| version" because of course it is. Incremental
| improvements to a single page with a button on it had
| resulted in needing a newer major version in less than
| one week.
| grey-area wrote:
| It certainly is a cultural issue which is why it
| shouldn't be enforced by tooling!
|
| I also completely agree about the value of stability,
| which is why I don't like this change which paradoxically
| encourages more churn (normalises v2,v3 etc for breaking
| changes, normalises large breaking changes and rewriting
| apis).
|
| All APIs are slightly unstable in some sense (even adding
| fields can be a breaking change), so the aim should be to
| minimise version bloat and breaking changes and provide
| stability for importers, which the go language has done
| an amazing job at for the last decade.
|
| To give you a counter-example the sengrid api in go has
| had multiple large breaking changes/rewrites and has used
| this v2/v3 etc scheme. It's still a horrible experience
| as an importer and I'd rather they remained stable
| instead of introducing massive churn with the cop-out
| that the old unsupported version didn't change.
|
| In dependencies I import I want a perpetual v1 which
| doesn't change over a decade and just slowly improves,
| not v31 - new improved rewrite this year! Which this rule
| explicitly encourages.
| throwaway34241 wrote:
| > In dependencies I import I want a perpetual v1 which
| doesn't change over a decade and just slowly improves,
| not v31 - new improved rewrite this year!
|
| > Which this rule explicitly encourages.
|
| I agree with the first part (I'd much prefer having a
| MyFunction2 in a package if necessary, than a breaking
| change to the package itself).
|
| But my takeaway from the design was the exact opposite -
| that it _discourages_ breaking changes, by making them a
| little less convenient, and that the purpose of the
| feature was to allow major versions to simultaneously
| exist to avoid build issues [1]. I think I got that
| impression by reading the long articles that the team
| lead put out about the design decisions [2], and
| observing that the standard libraries themselves rarely
| break compatibility.
|
| Unfortunately watching various discussions some people do
| feel encouraged to put out new major versions since
| upgrades are "solved" (even though that's only from a
| build system perspective, you're still causing client
| churn).
|
| [1] https://research.swtch.com/vgo-import
|
| [2] https://research.swtch.com/vgo
| tialaramex wrote:
| > All APIs are slightly unstable in some sense (even
| adding fields can be a breaking change)
|
| _Of course_ adding fields is a breaking change. It has
| never ceased to astonish me how little most programmers
| understand compatibility, I remember wrestling with
| libpng getting them to understand that e.g. no, "tidying
| up" the order of fields in a public structure you've
| published isn't safe back in the 1990s before they came
| to god and actually provided sane APIs which hide
| internal data structures.
|
| Now, it's true that Hyrum's Law means _any_ detectable
| change, even if it was never documented, will break
| somebody if there are enough people depending on your
| system. That 's a big deal if you're the Linux kernel, or
| the HTTP/1.1 protocol, as it means even trivial "this
| can't break anything" fixes may break stuff. For example,
| as I understand it Google once broke search in Asia by
| changing an internal constant so that a hashmap would re-
| allocate (thus invalidating pointers to items in the map)
| slightly earlier. C++ code relying on pointers to just
| magically stay valid across inserts was completely wrong
| before they broke it, but anybody staring at a 500 page
| on google.com doesn't care _why_ it broke they just want
| it fixed.
|
| Most of us needn't much worry about Hyrum's law. That
| would be, as an old boss repeatedly told us, "A good
| problem to have" because it means you're having enormous
| impact.
| grey-area wrote:
| Yes this was a regrettable decision which causes lots of
| unnecessary pain and confusion and I wish they'd reverse it.
|
| You can just use tags, no need to copy files, but IMO it should
| not be the package management system's job to enforce draconian
| rules about semantic versions and import paths. Otherwise go
| mod seems pretty straightforward and sensible to me though.
| throwaway894345 wrote:
| This was just to support backwards compat with GOPATH for the
| brief time that the ecosystem was transitioning to modules.
| Now that GOPATH is rarely used, it's not necessary.
| grey-area wrote:
| I was talking about the requirement for v2/v3 in paths on
| major version as a regrettable decision, not the particular
| choice of duplicating dirs (which they recommended
| initially and as I pointed out can be sidestepped with a
| tag).
| throwaway894345 wrote:
| Ah, my mistake.
| arghwhat wrote:
| > What kind of language recommends[0] with a straight face that
| you just duplicate your entire codebase into a v2/
| subdirectory?
|
| Not Go at least. You misunderstood the post completely.
|
| Here is an example of how you are supposed to change the import
| path for subsequent major versions (that is, backwards
| incompatible versions):
| https://github.com/jackc/pgx/blob/3599f646293c1b0d381214ab26...
|
| A one-line change to your modfile and the import path is
| changed.
| nemetroid wrote:
| > Not Go at least. You misunderstood the post completely.
|
| What? From the post:
|
| > The recommended strategy is to develop v2+ modules in a
| directory named after the major version suffix.
|
| > ...
|
| > We recommend that module authors follow this strategy as
| long as they have users developing in GOPATH mode.
|
| Although with a reservation, they clearly _do_ recommend it.
| Philip-J-Fry wrote:
| "As long as they have users developing in GOPATH mode."
|
| It's documentation written when GOPATH was still a widely
| used for development.
|
| I think nowadays it's less of a concern to have a module
| first code base with GOPATH unsupported. It's nicer to use
| a V2 branch. Or delegate V1 to a V1 branch and maintain V2
| in master.
| s17n wrote:
| How is a branch nicer than a subdirectory? Off the top of
| my head, it seems like backporting bugfixes might be
| easier using a branch (if you're lucky, you can just
| cherrypick the commit and it will work) - any other
| reasons?
| nemetroid wrote:
| I'm not disputing that things may have changed since the
| post was written. But the comment I responded to claimed
| that the linked post _didn't_ recommend copy-into-
| subdirectory, which it clearly does.
| cdogl wrote:
| Thanks for this informed comment. A timely reminder to me to
| read TFA, because I took parent comment at face value on
| first reading.
| nvarsj wrote:
| That's incorrect. The go blog recommends copying your entire
| project into a v2 folder. They do suggest you can use
| tags/branches instead, but it will cause problems with GOPATH
| projects.
| arghwhat wrote:
| GOPATH is luckily soon dead.
| throwaway894345 wrote:
| Which is kind of remarkable. Python still seems to try to
| support every package manager and format that ever
| existed, and that's part of why packaging in Python is so
| awful. Nice to see a language that can move forward with
| such a major change in such a short time and with
| _relatively_ little pain.
| btmcnellis wrote:
| Python is a lot older and a lot more widely used, so it's
| not as easy to just change things.
| eikenberry wrote:
| Python3 is pretty close to Go in age and was a purposely
| compatibility breaking, so it could have easily changed
| things at that point and they definitely knew about the
| problem at that point.
| dragonwriter wrote:
| Yeah, that's the thing about having so little existing
| software, especially stuff that has been essentially
| finished for a decade or longer.
|
| If Go is successful for any significant period of time,
| that will change.
| throwaway894345 wrote:
| Go is pretty established. By the time this module change
| was made, a huge swath of the cloud ecosystem had been
| written in Go, including core components like Docker,
| Kubernetes, Terraform, etc. It's also the default
| language for which Terraform and Kubernetes extensions
| are written, as well as the default language for
| container infrastructure (i.e., container runtimes, etc).
|
| I think Python's packaging story is a bummer in part
| because Python leans so hard on C which doesn't have a
| standard build system or a sane approach to building or
| packaging.
| dragonwriter wrote:
| > Go is pretty established
|
| Sure, and when it has been "pretty established" for a
| couple of decades, it'll have a chance to have the
| community weight of important legacy projects that Python
| has.
| throwaway894345 wrote:
| This is a canned argument for any criticism of older
| programming languages.
|
| First of all, it's not much of a consolation to users
| that the packaging and performance are bad "because the
| language is old", and secondly Python had already had
| half a dozen significant changes to its packaging format
| and ecosystem by the time it was Go's age (distutils,
| cheeseshop/pypi, PEP241, setuptools, eggs, easy_install,
| etc).
|
| Note that by this time, Python's package management was
| _far_ more complex than Go's and it was also far more
| painful (and indeed, still _is_ far more painful). Note
| that Go deprecated GOPATH when the language was already 9
| years old, at a point when it was quite a lot older and
| more widely used than Python's myriad package formats
| which are still supported today.
|
| We can say "No fair! Go has the benefit of hindsight!"
| and true enough, but again as a user, I don't much care--
| I just want the tool that's going to solve my problem
| with the least headache.
|
| And this isn't even touching on the Python 2-3 saga,
| which began when Python 2 was merely 8 years old and
| lasted for more than a decade. If you want to argue that
| we should use the Python 1.0 release date for our
| comparison, then fine--call me in ~4 years if the Go
| ecosystem is on the precipice of a suite of breaking
| changes which constitute an existential crisis. My money
| says that wonky happen.
| dragonwriter wrote:
| > This is a canned argument for any criticism of older
| programming languages
|
| No, its a an argument that applies to the ease of making
| disruptive changes to languages with less legacy in the
| ecosystem (age is a factor, but not the only one; its
| also a reason why Ruby, which was a similar age with the
| disruptive 1.8 to 1.9 transition had an easier time than
| Python did with 2 -> 3).
|
| It doesn't apply to _all_ criticism, or even to "older
| languages" generally (at least not equally, though
| survivorship bias means it is likely at least somewhat
| applicable to older languages that it is interesting to
| discuss in the first place.)
| [deleted]
| atombender wrote:
| It _recommends_ this, sure, but there 's no need for it.
| Doing so would be purely for backwards compatibility with
| older versions of Go tools that don't understand modules
| and rely on GOPATH. If you don't need to support older
| tools, then there's absolutely no need to copy anything.
| ithkuil wrote:
| > A one-line change to your modfile and the import path is
| changed.
|
| Another detail: since your import path change also affects
| internal imports between packages within your module, you
| also need to fixup a bunch of .go files in your repo (worst
| case all of them).
|
| I still think it's not a big deal (it's all pretty
| straightforward non-magic). Just a small nitpick that it's
| not just one go.mod change.
| arghwhat wrote:
| Yes, any consumer must pick the new import path, although
| that is true regardless of strategy.
| saturn_vk wrote:
| It looks like a subdirectory is obit recommended if you want to
| support get old versions of the toolchain. Using branches or
| tags is probably the way to go these days
| CraigJPerry wrote:
| I'm on the other end of the spectrum for this one. Breaking
| changes in a stable library (i.e. has reached v1) should be
| totally avoided.
|
| Statically typed languages, fancy features like intellij's
| structural find and replace, they just don't cut the pain and
| annoyance of fixing an unexpected change in behaviour of a
| library in a large code base.
|
| It's just horrific drudge work, utterly unenjoyable,
| thoughtless labour that needs careful attention.
|
| The go module versioning is clearly designed by someone who has
| worked on a large many developer code base before.
|
| I can have V2 and V3 linked into my binary at the same time - a
| blessing for incremental refactoring.
| _ph_ wrote:
| I think you have named the one good argument for this scheme:
| to allow a go module to exist in several versions in a
| project. Before the module system, this was just not
| possible. Which, in general, is a good idea. Programs today
| tend to have a too big dependency tree anyway, at least there
| should be only one copy of a module in the system. However,
| this is an idealistic view and depending on the number of
| dependencies, at some point it might not be possible to
| maintain this idea. npm on the other side goes totally crazy,
| every module import is local, you can have hundreds of copies
| of the same module in your system.
|
| The go module approach seems to be like a middle line. It
| puts a lot of pressure onto module maintainers not to break
| things within a major revision. They seem to be aware that
| this is not always guaranteed, that is why the module system
| not only tracks the major revision, but the exact subrevision
| of the module. But at least it can enforce, that changes
| which absolutely do break backwards compatibility are on
| differen major revisions. Of which you are allowed several
| inside your project.
| dcormier wrote:
| I really like golang, and have used it for years. But I really
| dilike gomod. It's not uncommon for using it to be a
| frustrating experience. It makes me glad that most of my work
| these days is not in golang.
| [deleted]
| avgcorrection wrote:
| > and everyone else in the room had an "oh shit" moment because
| their amateur language design didn't address that corner case.
|
| I doubt that's how it went down. They tend to be very confident
| in their design decisions. Whether that confidence is warranted
| or not.
| nvarsj wrote:
| I've been a long time golang user and while I like to give the
| golang maintainers the benefit of the doubt, I have also
| struggled heavily with go modules.
|
| I think my main issue with it is that it does things implicitly
| without you telling it to. Running go commands will modify the
| go.mod file all the time, and this is largely invisible to you.
| As a result, your build is basically a moving target and seems
| to lose all reproducibility. It also makes the entire thing
| very opaque and hard to understand at times, since you need to
| understand the implementation details of go modules to fully
| know what's happening.
|
| In comparison, take godep. While not perfect, it simply did
| what I told it to, and only modified the dependency state when
| I gave it a command. This meant I knew exactly what was
| happening in my build system, and could update dependencies
| when I chose to.
|
| If anything, go modules is too smart for its own good... And I
| think this is a big problem for a build or packaging system. It
| needs to be explicit and simple. It's odd it was designed like
| this, given the language itself strives for these ideals.
___________________________________________________________________
(page generated 2021-10-03 23:01 UTC)