[HN Gopher] How Go mitigates supply chain attacks
___________________________________________________________________
How Go mitigates supply chain attacks
Author : spacey
Score : 200 points
Date : 2022-03-31 16:38 UTC (6 hours ago)
(HTM) web link (go.dev)
(TXT) w3m dump (go.dev)
| synergy20 wrote:
| node, python, ruby...etc including go modules can lock their
| dependencies, Go really wins when it can be wrapped in a
| reasonably-sized one binary, nothing beats that in deployment,
| for all the others you have to pull in lots of packages into the
| target system
| munificent wrote:
| _> There is no way for changes in the outside world--such as a
| new version of a dependency being published--to automatically
| affect a Go build._
|
| _> Unlike most other package managers files, Go modules don't
| have a separate list of constraints and a lock file pinning
| specific versions. The version of every dependency contributing
| to any Go build is fully determined by the go.mod file of the
| main module._
|
| I don't know if this was intentional on the author's part, but
| this reads to me like it's implying that with package managers
| that _do_ use lockfiles a new version of a dependency _can_
| automatically affect a build.
|
| The purpose of a lockfile is to make that false. If you have a
| valid lockfile, then fetching dependencies is 100% deterministic
| across machines and the existence of new versions of a package
| will not affect the build.
|
| It is true that most package managers will automatically update a
| lockfile _if it 's incomplete_ instead of failing with an error.
| That's a different behavior from Go where it fails if the go.mod
| is incomplete. I suspect in practice this UX choice doesn't make
| much of a difference. If you're running CI with an incomplete
| lockfile, you've already gotten yourself into a weird state. It
| implies you have committed a dependency change without actually
| testing it, or that you tested it locally and then went out of
| your way to discard the lockfile changes.
|
| Either way, I don't see what this has to do with lockfiles as a
| concept. Unless I'm missing something, go.mod files _are_
| lockfiles.
| jchw wrote:
| It's very subtle, but there are some important differences. For
| example, lockfiles are not recursive in NPM: the NPM package
| (usually?) does not contain the lockfile and does not adhere to
| it when installed as a dependency. It will pick the newest
| version of dependencies that matches the spec in package.json.
|
| Go mod files _are_ used recursively, and rather than try to
| pick the newest possible version, it will go with the _oldest_
| version.
|
| This avoids the node-ipc issue entirely, at least until you
| update the go.mod.
| lamontcg wrote:
| How does go deal with the diamond dependency issue when
| transitively you've a dep on the same package via two
| different paths?
| jchw wrote:
| As far as I understand it... the import path that you use
| to import a package acts as its identity, and only one
| version of any given package will be installed. The way
| that it will determine this is by choosing the lowest
| version specified in any package that depends on a given
| package. Major versions of packages are required to have
| different import paths with Go modules, so when depending
| on two different major versions of the same package, they
| are treated effectively as their own package.
| typical182 wrote:
| There are some subtleties here, but go.mod files are not
| lockfiles, including go.mod files don't follow a traditional
| constraint/lockfile split, which I think is part of the point
| in that snippet you quoted.
|
| Another way they differ is when installing a top-level tool by
| default npm does not use the exact version from a library's
| lockfile to pick the selected version of another library as far
| I as understand, whereas Go does by default use the exact
| version required by a library's go.mod file in that scenario.
|
| In other words, go.mod files play a bigger role for libraries
| than a traditional lockfile does by default for libraries in
| most other ecosystems.
|
| Here's a good analysis on the contrast between go.mod and the
| default behavior of more traditional lockfiles (using the npm
| 'colors' incident as a motivating example):
|
| https://research.swtch.com/npm-colors
|
| That link also includes some comments on 'npm ci' and
| 'shrinkwrap' that I won't repeat here.
|
| All that said, go.mod files do record precise dependency
| requirements and provide reproducible builds, so it's possible
| to draw some analogies between go.mod & lockfiles if you want.
| I just wouldn't say "go.mod files are lockfiles". ;-)
| goodoldneon wrote:
| Wouldn't build size increase a lot if transitive dependencies
| were pinned to direct dependency lockfiles? Like if library A
| says "use version 1.0.0 of library X" and library B says "use
| version 1.0.1 of library X", then you'd likely end up
| bundling duplicate code in your build.
|
| Not saying the tradeoff isn't worth it, but pinning to
| dependency lockfiles isn't without downsides.
| typical182 wrote:
| FWIW, that's not what Go does. In your scenario, a Go
| binary ends up with a single copy of library X -- the 1.0.1
| version. That's because library A is stating "I require at
| least v1.0.0 of X", and library B is stating "I require at
| least v1.0.1 of X". The minimal version that satisfies both
| of those requirements is v1.0.1, and that's what ends up in
| the binary.
|
| That behavior is Go's "Minimal Version Selection" or "MVS".
| There are many longer descriptions out there, but a concise
| graphical description I saw recently and like is:
|
| https://encore.dev/guide/go.mod
|
| That's the default behavior, but a human can ask for other
| versions. For example, a consumer of A and B could do 'go
| get X@latest', or edit their own go.mod file to require X
| v1.2.3, or do 'go get -u ./...' to update all their direct
| and indirect dependencies, which would include X in this
| case, etc.
| dezren39 wrote:
| how are you not describing package.json right now what is
| the difference
| typical182 wrote:
| Continuing that example -- in Go you end up with v1.0.1
| of X _by default_ even if v1.0.2 is the latest version of
| X.
|
| That is a difference with many other package managers
| that can default to using the latest v1.0.2 of X (even if
| v1.0.2 was just published) when doing something like
| installing a command line tool. That default behavior is
| part of how people installing the 'aws-sdk' tool on a
| Saturday started immediately experiencing bad behavior
| due to the deliberate 'colors' npm package sabotage that
| happened that same Saturday.
|
| In any event, it's certainly reasonable to debate pros
| and cons of different approaches. I'm mainly trying to
| clarify the actual behavior & differences.
| amtamt wrote:
| Perhaps
|
| "A module may have a text file named go.sum in its root
| directory, alongside its go.mod file. The go.sum file contains
| cryptographic hashes of the module's direct and indirect
| dependencies."
|
| And
|
| "If the go.sum file is not present, or if it doesn't contain a
| hash for the downloaded file, the go command may verify the
| hash using the checksum database, a global source of hashes for
| publicly available modules."
|
| Should be stressed on. If I committed a dependency version
| (go.mod) and checksum (go.sum) along with the code, either I
| get a repeatable build everywhere, or build fails if dependency
| not found or found to be modified.
|
| I am not sure if all other package managers include checksum
| with dependency version.
| gmfawcett wrote:
| > the go command may verify the hash
|
| If we're talking about reproducible builds, the word "may"
| seems concerning here?
| [deleted]
| infogulch wrote:
| The author is saying that Go provides the same guarantees with
| just a package list in the go.mod file that other package
| managers need both a package list and lock file to solve.
|
| go.sum is essentially a distributed / community maintained
| transparency log of published versions of packages.
| munificent wrote:
| Maybe I'm just not familiar with it enough but I don't see
| how merging a package manifest and lockfile into a single
| file is a net win.
|
| This means it's no longer clear which dependencies are
| immediate and which are transitive. It's not clear which
| versions are user-authored constraints versus system-authored
| version selections. For dependencies that _are_ transitive,
| it 's not clear _why_ the dependency is in there and which
| versions of which other dependencies require it.
|
| Other packages separate these into two files because they are
| very different sets of information. Maybe Go's minimum
| version selection makes that not the case, but it still seems
| user-unfriendly to me to lump immediate and transitive
| dependencies together.
| typical182 wrote:
| FWIW, there are two machine-formatted sections of go.mod --
| the first for direct dependencies, the second section for
| indirect dependencies.
|
| (That's as of Go 1.17. Previously, that information was
| communicated via machine-generated comments in a single
| section).
| munificent wrote:
| That seems reasonable.
|
| I think I personally lean towards keeping them in
| separate files entirely because I like a clearer
| separation between human-authored content and machine-
| derived state.
| klodolph wrote:
| Discussing "what is a lockfile" is a bit of a headache because
| different languages have different files which do different
| things. Generally speaking, there's some file which specifies
| the dependency versions and some file with cryptographic
| checksums of the all transitive dependencies.
|
| In Go it's go.mod / go.sum. In NPM, it's package.json /
| package-lock.json. In Rust it's Cargo.toml / Cargo.lock.
|
| Diving into the exact details of what the author is saying is a
| bit outside my headspace at the moment. I think the author of
| the article may not actually understand the scenario where Go's
| package system differs. (I'm not sure I do, either.)
|
| Suppose you have your project, projectA, and its direct
| dependency, libB. Then libB has a dependency on libC.
|
| If projectA has a lockfile, you get exactly the same versions
| of libA and libB. This is true for Go, NPM, and Cargo. However,
| suppose projectA is a new project. You just created it. In Go,
| the version of libB that makes it into the lockfile will be the
| _minimum_ version that libA requires, which means that any new,
| poisoned version of libB will not transitively affect anything
| that depends on libA, such as projectA. With NPM, you get the
| latest version of libB which is compatible with libA--this
| version may be poisoned.
| munificent wrote:
| _> any new, poisoned version of libB_
|
| Conversely, you _will_ get any _old_ security-buggy version
| of libB instead.
|
| Most package managers when adding a new dependency assume
| newer versions are "better" than older versions. Go's minimum
| version system assumes older is better than newer.
|
| I don't think there's any clear argument you can make on
| first principles for which of those is actually the case.
| You'd probably have to do an empirical analysis of how often
| mailicious packages get published versus how often security
| bug fix versions get published. If the former is more common
| than the latter, then min version is likely a net positive
| for security. If the latter is more common than the former,
| then max version is probably better. You'd probably also have
| to evaluate the relative harm of malicious versions versus
| unintended security bugs.
| zeeboo wrote:
| Every change that fixes a security issue implies the
| existence of a change that introduced the security issue in
| the first place. Why is bumping a version more likely to
| remove security issues instead of introduce them?
|
| The reason why older is better than newer has more to do
| with the fact that the author has actually tested their
| software with that specific version, and so there's more of
| a chance that it actually works as they intended.
| klodolph wrote:
| > I don't think there's any clear argument you can make on
| first principles for which of those is actually the case.
|
| I don't understand why someone would try to argue from
| first principles here, it just seems like such a bizarre
| approach.
|
| Anyway, it's not just a security issue. Malicious packages
| and security fixes are only part of the picture. Other
| issues:
|
| - Despite a team's promise to use semantic versioning,
| point releases & "bugfix" releases will break downstream
| users
|
| - Other systems for determining the versions to use are
| much more unpredictable and hard to understand than
| estimated (look at Dart and Cargo)
|
| https://github.com/dart-lang/pub/blob/master/doc/solver.md
|
| https://github.com/rust-
| lang/cargo/blob/1ef1e0a12723ce9548d7...
| munificent wrote:
| _> Other systems for determining the versions to use are
| much more unpredictable and hard to understand than
| estimated (look at Dart and Cargo)_
|
| I'm one of the co-authors of Dart's package manager. :)
|
| Yes, it is complex. Code reuse is hard and there's no
| silver bullet.
| klodolph wrote:
| Nice! I hope I wasn't coming across as critical of Dart's
| package manager, or Cargo for that matter.
| munificent wrote:
| It's OK. There are always valid criticisms of all
| possible package managers. It's just a hard area with
| gnarly trade-offs.
| philosopher1234 wrote:
| Go just expects you to manually trigger the updates. Thats
| all. It still is in favor of updating to take security
| fixes, so i think your argument is wrong.
| munificent wrote:
| Let's say my_app uses package foo which uses package bar.
|
| It turns out there is a security bug in bar. The bar
| maintainers release a patch version that fixes it.
|
| In most package managers, users of my_app can and will
| get that fix with no work on the part of the author of
| foo. I'm not very familiar with Go's approach but I
| thought that unless foo's author puts out a version of
| foo that bumps its minimum version dependency on bar,
| my_app won't get the fix. Version solving will continue
| to select the old buggy version of bar because that's the
| minimum version the current version of foo permits.
| skybrian wrote:
| As I understand it, that's true in the simple case. If
| you have `my_app -> foo -> bar` then there's only one
| path to bar, and you only get a new bar when you upgrade
| foo.
|
| It's more complicated in general, with diamond
| dependencies. There needs to be a chain of module updates
| between you and foo, with the minimum case being a chain
| of length one where you specify the version of foo
| directly.
|
| So, people do need to pay attention to security patch
| announcements. But popular modules, at least, are likely
| to be get updated relatively quickly, because only one
| side of a diamond dependency needs to notice and do a
| release.
| coder543 wrote:
| > As I understand it, that's true in the simple case. If
| you have `my_app -> foo -> bar` then there's only one
| path to bar, and you only get a new bar when you upgrade
| foo.
|
| This is not correct. You can update bar independent of
| foo directly from the top-level go.mod file in your
| project.
| skybrian wrote:
| Yes, you can do that by adding a direct dependency on
| foo. I started by talking about when there _isn 't_ a
| direct dependency on foo.
|
| I explicitly talked about having a direct dependency at
| the end of the second paragraph.
| coder543 wrote:
| I'm not talking about direct dependencies. Direct and
| indirect are all listed in the go.mod file. If they
| aren't listed there, then they aren't in your final
| binary. If you delete indirect dependencies from the top-
| level go.mod, your project will fail to compile.
| coder543 wrote:
| > I thought that unless foo's author puts out a version
| of foo that bumps its minimum version dependency on bar,
| my_app won't get the fix. Version solving will continue
| to select the old buggy version of bar because that's the
| minimum version the current version of foo permits.
|
| That is incorrect. The application's go.mod defines all
| dependencies, even indirect ones. Raise the version
| there, and you raise it for all dependencies. You cannot
| have one more than one minor version of a dependency in
| the dependency graph.
| munificent wrote:
| That still implies that I need to _know_ to update the
| version constraint of what may be a very deep transitive
| dependency, doesn 't it?
| coder543 wrote:
| The version constraint is _always_ listed in your top
| level go.mod file, so you know the dependency exists, no
| digging into the dependency tree required at all, and
| it's not hidden in some lock file no one ever looks at.
| Plus, there are plenty of tools that help you with this
| problem, including the language server helping you
| directly in your editor and Dependabot on GitHub.
|
| I'm not aware of any languages that send you an email
| when your dependencies are out of date, so yes, you need
| to check them. Dependabot can do this for you and open a
| PR automatically, which will result in an email, so this
| is one way for people to stay on top of this stuff even
| for projects they deploy but don't work on every single
| week.
|
| If you're suggesting that indirect dependencies should
| automatically update themselves, then you are quite
| literally saying those code authors should have a shell
| into your production environments that you have no
| control over, compromising all your systems with a single
| package update that no one but the malicious author got
| to review. It is possible with tools like Dependabot to
| be notified proactively when updates are required so you
| can review and apply those, but it is not possible to go
| back in time and un-apply a malicious update that went
| straight to prod.
|
| Repeatedly assuming that the Go core team never thought
| through the design of Go Modules and how it relates to
| security updates is such a strange choice. Go is a very
| widely used language with tons of great tooling.
| munificent wrote:
| I'm sorry but I'm not super familiar with the workflow
| for working with dependencies in Go, I've only read about
| it. You say:
|
| > Raise the version there.
|
| Am I to understand that it's common to hand-edit the
| version constraint on a transitive dependency in your
| go.mod file?
|
| But that transitive dependency was first added there by
| the Go tool itself, right?
|
| How does a user easily keep track of what bits of data in
| the go.mod file are hand-maintained and need to be
| preserved and which things were filled in implicitly by
| the tool traversing dependency graphs?
|
| _> Repeatedly assuming that the Go core team never
| thought through the design of Go Modules_
|
| I'm certainly not assuming that. But I'm also not
| assuming they found a perfect solution to package
| management that all other package management tools failed
| to find. What's more likely is that they chose a
| different set of trade-offs, which is what this thread is
| exploring.
| geodel wrote:
| You keep harping on old buggy version where Go has been
| very clear that it is operator's explicit responsibility
| to have correct/updated/fixed versions of dependency
| running.
|
| It specially does not look good in your case considering
| you work for Google on a different programing language.
| If you have a clear point to make then compare it with
| your approach. Instead of making neutral sounding
| arguments when they are not.
| adamc wrote:
| Don't drag in ad-hominem attacks. If you want to defend
| Go's approach, explain why having it be the "operator's
| explicit responsibility" is a good policy, likely to make
| apps (in general) more secure. The obvious implication of
| the example given is that, on average, it will be a mess.
| Friday_ wrote:
| Keep it simple, just don't use dependencies.
| xxpor wrote:
| Kind of ironic for a language that was the first that I know
| about where you can straight up import libraries with a URL
| directly in your source.
| [deleted]
| throwaway894345 wrote:
| It's not a URL, it's just an identifier that sort-of
| resembles a URL similar to Kubernetes annotation conventions.
| benhoyt wrote:
| It may not be quite a URL, but it contains the URL. It
| definitely more than just "sort-of resembles a URL". See
| how the module path to URL lookup is done here:
| https://go.dev/ref/mod#vcs-find
| throwaway894345 wrote:
| I understand your meaning, but that link supports my
| claims. The package path helps the Go tool infer the
| actual URL, but it doesn't contain the URL itself (e.g.,
| the actual URL has a scheme/protocol component and
| potentially a `go-get=1` query string argument which
| don't exist as part of the path). This is what I meant
| when I said it "sort-of resembles a URL", but I
| understand from this conversation how that wording wasn't
| clear.
| benhoyt wrote:
| Fair. Strictly speaking it doesn't even "contain" a URL.
| But I think in the context of this conversation it acts
| like a URL -- it allows the Go tooling to fetch code from
| an arbitrary domain and path on the internet.
| adtac wrote:
| sure, just don't use Postgres, OAuth, payments, RPCs, metrics,
| non-standard crypto, or anything important
| nu11ptr wrote:
| One problem: unless you have an endless amount of time, money,
| patience, and...did I mention patience? that just isn't a
| viable approach for anything but the smallest of hobby projects
| Friday_ wrote:
| Build things by KISS principle.
|
| Dependencies will complicate system you are building. ( with
| things like supply chain attacks, bloat, liscensing etc...)
| throwaway894345 wrote:
| So we just don't build GUI applications unless we
| understand all of the nuances of layout, text rendering,
| graphics programming, etc sufficient to implement it
| ourselves (and without the bugs that even domain experts
| have introduced but which have been found and fixed over
| time in libraries)? Or maybe we just say "fuck users who
| speak languages that aren't easily expressed in ASCII"?
|
| There's a reason libraries exist. It's not like they were
| the default state of computing and no one has tried to
| write applications without them. On the contrary, _we
| tried_ to build applications by writing everything
| ourselves, but that doesn 't survive encounters with the
| real world. And "well if you can't do it yourself, you
| don't need it" (which may or may not be your argument, I
| genuinely can't tell) is just technologically regressive
| ideology.
| Friday_ wrote:
| Software programmer usually need to solve specific task,
| not worlds problems. If you go too generalized you will
| need 10x, 100x or 1000x more amount of time and code.
|
| Look at Big tech, this is what they are doing, they
| employ thousands workers that write millions of lines of
| code every day, only to make it work for every case in a
| world. And still can't compete with specialized solution.
| throwaway894345 wrote:
| > Software programmer usually need to solve specific
| task, not worlds problems. If you go too generalized you
| will need 10x, 100x or 1000x more amount of time and
| code.
|
| Yes, I accept this is true, but your earlier claim was
| much more specific: that using dependencies at all makes
| things worse. I gave you a specific example (GUI
| libraries) but you completely ignored it. How does your
| 0-dependencies theory survive an encounter with that
| basic example?
| Friday_ wrote:
| It's usually needed only minor part of it. You dont need
| to code whole GUI library. Only what you need to solve
| specific task.
| throwaway894345 wrote:
| No, but if your GUI involves displaying text in multiple
| languages (i.e., virtually everything), you now need to
| become an expert in text rendering (which implies a
| significant breadth of knowledge in linguistics, graphics
| programming, constraint solving systems [for things like
| word wrapping], etc).
| nu11ptr wrote:
| Except KISS is subjective, as are most things. Please don't
| take this the wrong way, but anyone who touts any one, true
| "religion" in sw development I have learned to take with 6
| grains of salt. There is no one magic way or solution.
| Everything is trade offs and I've learned this over the
| last 30 years through trying every new magic elixir that
| was going to "save us all".
|
| That said, most sw projects are determined by business
| need. I can't think of a single one I've ever written that
| hasn't needed dependencies or would have been viable had I
| not used some. Going to a theory level, there is no reason
| to think the stdlib is magically immune to the issues of a
| very popular dependency either. Shun absolutes and make the
| correct trade offs based on your business goals.
| yjftsjthsd-h wrote:
| I guess that's fine so long as you're covered by the standard
| library and/or are willing to reimplement a _lot_ of stuff
| yourself, but that 's a significant trade-off you're asking.
| 0des wrote:
| It sounds defensive but Go stdlib is all you need. I believe
| I'm qualified to say that as last year I challenged myself to
| only use stdlib, out of several languages I used over the
| course of the year on big projects, Go was painless and that
| was a unique experience. So far this year I haven't seen much
| need to add libraries to my work because everything is
| already within grasp.
| vlunkr wrote:
| The elephant in the room here is NPM, and I think the obvious
| problem there is the culture. I have a tiny app I've been playing
| with using create-react-app. There are over 800 directories in
| node_modules. That absolutely dwarfs the number of any other
| language I've used. Even in a medium sized rails app, you likely
| have some awareness of what every dependency is. It's just
| impossible with npm.
|
| One thought I've had to "reboot" the npm culture is to somehow
| curate packages that are proven to have minimal and safe
| dependencies, and projects can shift to using those. I imagine
| there has to be some sort of manual review to make that happen.
| dtgriscom wrote:
| I think the NPM organization is completely aware just how
| dangerous this all is, and is eager to hide it. For example, if
| you look up an NPM package, it'll list its direct dependencies.
| But, there's no acknowledgement whatsoever of all the stuff
| that comes along for the ride.
|
| I'd love to have a well-supported ranking of NPM packages in
| terms of their dependencies (and their dependencies'
| dependencies, etc). Knowing the breadth of immediate
| dependencies, PLUS the depth of the total dependency tree,
| would give you some inkling of just how much you're taking for
| granted when you start using a package.
| tln wrote:
| I agree that it would be nice for NPM to show the total
| footprint of a module, especially if that provides some
| social incentive to reduce the dependency count.
| kubanczyk wrote:
| I don't like the authoritative tone of the article. Especially
| given the fact that author "conveniently forgets" about
| go mod edit
|
| and go work
|
| both of which are deliberately designed counter-mitigations, i.e.
| they exist to poke small holes in the pin-everything wall. I
| agree with the spirit of the message though, the surface is much
| smaller with Go and it shows much planning went into that.
| tester756 wrote:
| How about getting safe compiler?
| Tainnor wrote:
| Most package managers have lockfiles. Yes, npm's decision to have
| both "npm install" and "npm ci", just so you can confuse and
| mislead developers, is a bit silly.
|
| But Ruby's Bundler, for example, has been refusing to run your
| code if your lockfile is inconsistent for as long as I remember.
|
| Locking dependencies is, generally, a solved problem across most
| ecosystems (despite node botching its UX). Go doesn't get to
| claim that it's superior here.
|
| But of course, supply chain attacks are still possible with lock
| files. Because _somebody_ is going to update your dependencies at
| some point (often for security reasons). And at that point you
| might be pulling in a malicious dependency which you haven 't
| carefully vetted (because nobody has time to vet all their
| dependencies thoroughly nowadays).
|
| That's still an unsolved problem, as far as I know. I don't think
| that Go has solved it.
| hobofan wrote:
| > Most package managers have lockfiles
|
| > Locking dependencies is, generally, a solved problem across
| most ecosystems
|
| As someone who is building a package manager for work, and has
| looked at pretty much every package manager out there (and
| their ecosystem adoption), I can only say that those don't
| reflect the current reality of package management (no matter
| how much I wish it were true).
|
| Bundler was the first mainstream package manager to adopt a
| lockfile (AFAIK) a mere 12 years ago. Many many language
| ecosystems predate that and are still lacking lockfiles (or
| even widespread adoption of a single compatible package
| manager).
|
| NPM only got lockfiles 5 years ago (after being pressured by
| yarn). Gradle got them less than 3.5 years ago, and Maven still
| doesn't have them (though a niche plugin for it exists). The
| Python ecosystem is still a hot mess, with ~3 competing
| solutions (Poetry, Pipenv, Conda), of which Conda just got a
| 1.0 of their decent conda-lock incubation project a month ago,
| but due to how setuputils works, the cross-platform package
| management story is broken almost beyond recovery. In Conan
| lockfiles are still an experimental feature today.
|
| I could go on and on, but I hope that I could paint a picture
| that while one could argue that with the advent of lockfiles,
| locking dependencies has become a solved problem
| _conceptually_, the current status of implementation across
| ecosystems is still horrible. I'm also constantly amazed about
| how little love is put into package managers in most language
| communities, even though they are so crucial for their
| respective ecosystems.
|
| As far as I can tell nowadays Go does have one of the better
| package managers, which given their horrible starting point is
| quite the feat. As a nice side-effect of experiments in the Go
| package ecosystem, one of the people working on go dep also
| created one of the best resources around package managers:
| https://medium.com/@sdboyer/so-you-want-to-write-a-package-m...
| hu3 wrote:
| To be fair the title of the article is "How Go Mitigates Supply
| Chain Attacks" not how it solves.
|
| And I think it does a good job at that.
|
| For example if you had any JavaScript package that depended on
| node-ipc in your project, a simple _npm install_ after cloning
| the project would download code that tries to corrupt files in
| your disk if the malicious code determined that your IP was
| from Russia. (before the malicious package was taken down
| /fixed)
|
| With Go you would have to explicitly bump dependency versions.
| Simply cloning the project and installing dependencies would
| not have downloaded the malicious version. And bumping would at
| the very least appear as a diff in a Pull Request.
| infogulch wrote:
| I'm annoyed by the false dichotomy that colors most discussions
| around package management that there are only two solutions to
| publishing software packages: 1. a carefully curated
| professionally maintained standard library, 2. the complete wild
| west where anything goes. It's not really "false" because this is
| the reality of how package managers are designed today, but it's
| false in the sense that _it doesn 't have to be this way_.
|
| You can see this tension in virtually every discussion, users
| resisting using packages that aren't published in the standard
| library for fear of attacks and poor quality, and maintainers
| that resist publishing in the standard library for fear of
| changing requirements and the appearance of better designs. Sure
| there are admissible entitlement / responsibility arguments
| against these respective positions, but that's mostly a
| distraction because _both have a valid point_.
|
| The problem is that there's no space for intermediate solutions.
| We need packaging tools to aggregate and publish groups of
| packages that relate to a particular domain, and organizational
| tools to ensure quality and continuity of these package groups
| over time. This mitigates users' fears and reduces their
| cognitive load by curating the solution space, and it mitigates
| maintainers fears of ossification and backcompat hell by enabling
| them to create new package groups.
|
| I'm saying there's an entire dimension of valid tradeoffs in this
| space, but the current design trend of package managers force us
| into one extreme or the other.
| nonameiguess wrote:
| The original solution _is_ the intermediate solution. You can
| build an entire useful userspace around nothing but libc and an
| ssl lib (take your pick between OpenSSL or GnuTLS usually).
| That is effectively what busybox is. Need to do some
| heavyweight math in Fortran? BLAS and LAPACK probably have
| everything you need. You can get really far with a C++
| application using nothing but Boost.
|
| For whatever reason, newer language ecosystems migrated away
| from that in the direction of increasingly smaller libraries
| until npm practically became a parody of it.
|
| There is no reason you can't have lots and lots of useful
| functionality packed into a few large, well-maintained, well-
| packaged, well-vetted and trusted libraries, but you need
| trustworthy organizations willing to that maintaining and
| vetting. Historically, that seemed to largely be universities
| and research labs, where the funding and incentives are a lot
| different from the weekend warriors and solo devs that dominate
| open source landscapes today. Interestingly, I think library
| projects that still have large organizations behind them keep
| with the larger old-school ethos. Look at the world of ML and
| scientific computing. NumPy and SciPy are still huge libraries.
| Same with PyTorch and Tensorflow. QuantLib is an interesting
| example because it actually doesn't have a single large
| organization behind it. A bunch of Quants just got tired of
| doing the same things from scratch over and over and decided to
| aggregate their work for their common good. But it was 22 years
| ago, so maybe it was still just different back then and the
| trend toward small libraries hadn't kicked in yet.
| throwaway894345 wrote:
| This is kind of interesting. In the Linux world, you have
| package maintainers who (among other things) vet and vouch for
| the quality of the packages they maintain. I think there are
| similar things in the Docker ecosystem these days (since Docker
| really did/does seem to be the wild west).
|
| It could be interesting if there was a similar concept for Go
| (and/or other ecosystems), except that instead of actually
| packaging the packages into artifacts (especially with the
| licensing headache that entails), it could be essentially a
| registry of verified package versions. So the "maintainers" in
| this sense are just validating the dependencies and maintaining
| a list of the approved dependencies (including their
| versions/checksums) and then automated tooling could be used by
| consumers ("consumer" here may or may not imply payment
| depending on whether this hypothetical venture is open or
| closed) to identify unverified dependencies in the consumer's
| project.
|
| I'm sure someone has thought of this already--link me to
| relevant projects if you know about any.
| lawl wrote:
| > We need packaging tools to aggregate and publish groups of
| packages that relate to a particular domain, and organizational
| tools to ensure quality and continuity of these package groups
| over time
|
| Whats stopping you? golang.org/x is kind of like that. Theres
| nobody stopping you from aggregating packages under foo.bar
| domain and build a reputation for high quality.
| infogulch wrote:
| Good question! The thing that the standard library has that I
| don't is _version aggregation_. The problem is not publishing
| under a single domain, the problem is publishing under _a
| single version_. Publishing a bunch of packages that I claim
| are high quality doesn 't help users decide which versions to
| use, they still have to make this decision on a package-by-
| package basis. Note that I may be in the middle of a big
| redesign and some individually-complete packages use the new
| design but others don't, so you can't just say use the latest
| of each because they don't all work together yet. In this
| case I still want to publish each package individually for
| people who do want fine-grained control, but I wouldn't
| publish a new version of my "package group" with these
| versions because they're not integrated yet. My point is that
| aggregations of verified-interoperable packages is a separate
| problem domain and existing tools don't suffice to solve it.
| [deleted]
| skybrian wrote:
| You could sort of do that using a Go module that points to
| all your other modules. Then anyone who depends on that
| will get the versions you specify (at a minimum).
|
| But a problem is that they would also download all the
| modules you point at, whether they use them or not.
|
| To fix that, the package system would need a "soft
| dependency" where, if a module exists, it must be at least
| the version indicated.
| infogulch wrote:
| Good idea with "soft dependencies". You'd also need to
| make the go.mod RequireSpec version argument [1] optional
| so it can be overridden by the module group.
| require example.com/my-module-group 1.2.3 require
| example.com/some/module // selects minimum version
| specified by my-module-group
|
| [1]: https://go.dev/ref/mod#go-mod-file-require
|
| I also noticed Workspaces [2] which I hadn't seen before.
| They look interesting, but appear to exist for a
| different purpose. Maybe workspaces with a bunch of
| replace directives & some cli tooling could emulate a
| system like what is described here.
|
| [2]: https://go.dev/ref/mod#workspaces
| throwaway894345 wrote:
| I suspect licensing may stop you from doing this, but I'm not
| an expert. See my sibling comment for an alternative:
| https://news.ycombinator.com/item?id=30871181
| jerf wrote:
| I'm unclear from reading your comment if you know this or
| not... but, what the Go team is describing _is_ an intermediate
| solution. Not the exact one you describe, but it is
| intermediate. There is no particular requirement to get into
| the Go package ecosystem, no gatekeepers, it 's all namespaced
| by the URLs you store your source code at, but between what the
| proxies do and the way the version requirements were specified,
| you are also not simply naked to every update someone somewhere
| pushes.
|
| Please note I'm not claiming it's perfect or that you'll like
| every aspect of it, I'm just saying it _is_ an intermediate
| solution between the two extremes.
| staticassertion wrote:
| It doesn't. Some other tools are worse. Great, saved you a read.
| glenjamin wrote:
| Something this articles glosses over is that some of these
| approaches, especially the way 'All builds are "locked"' is
| achieved with minimum version selection, and "A little copying is
| better than a little dependency" are tradeoffs against an
| alternative security model, where transitive dependencies are
| automatically updated to pick up security fixes.
|
| Part of the churn and noise in the Node.js dependency ecosystem
| actually stems from security-related issues being noted in a low-
| level module, and the ripple effects caused by that when a bunch
| of maintainers have to go around bumping lockfiles and versions.
| typical182 wrote:
| > _tradeoffs against an alternative security model, where
| transitive dependencies are automatically updated to pick up
| security fixes._
|
| One thing to keep in mind is that Go doesn't stop you from
| updating.
|
| For example, its common to do 'go get -u ./...' or 'go get
| -u=patch ./...' from your project root to update all of your
| direct and indirect dependencies.
|
| The built-in tooling & language server give you nudges, and if
| desired it can be automated via things like dependabot or
| otherwise.
|
| In practice, it means it is often a slightly slower cadence for
| typical projects in the Go ecosystem compared to say the
| Node.js ecosystem, but the upgrades still happen. That slightly
| slower pace I think has worked out so far, and was a conscious
| choice[1]:
|
| > _Many developers recoil at the idea that adding the latest B
| would not automatically also add the latest C, but if C was
| just released, there 's no guarantee it works in this build.
| The more conservative position is to avoid using it until the
| user asks. For comparison, the Go 1.9 go command does not
| automatically start using Go 1.10 the day Go 1.10 is released.
| Instead, users are expected to update on their own schedule, so
| that they can control when they take on the risk of things
| breaking._
|
| [1]
| https://go.googlesource.com/proposal/+/master/design/24301-v...
| bigdubs wrote:
| There is a deeper strategy here with go vs. node; having a
| standard library maintained by professionals.
|
| I would rather build on a common set of libraries secured by
| people who are paid full-time to maintain them, and maybe have
| slightly worse ergonomics, than have a community of libraries
| that come and go and have inconsistent quality.
|
| This standard library approach yields fewer dependencies, fewer
| changes over time, and better consistency between projects.
| hrns wrote:
| Isn't Node API an equivalent of go standard library?
| jgod wrote:
| Yes but it's super barebones. Its successor, Ryan Dahl's
| second attempt at JS runtime, Deno, has a much fuller
| standard library (inspired by Go).
| tormeh wrote:
| I wish we'd stop trying to make broken languages work.
| This feels like hill-climbing into the strangest local
| optimum possible. JS is not the best example of an
| interpreted language. Wouldn't it be better to put Python
| in the browser than to put JS on the server? Can't wait
| for WASM to be a first-rate citizen on the web so we
| don't have to deal with this anymore.
| gmfawcett wrote:
| > Wouldn't it be better to put Python in the browser than
| to put JS on the server?
|
| I think that's a categorical "no", because Python isn't
| an objectively better language than JavaScript. I'm
| saying this as a Python developer since v1.5 (>20 years).
|
| Subjective opinions are a different matter.
| bigdubs wrote:
| Yes Node.js ships with what is effectively a very thin
| standard library for some low level things like interacting
| with the file system, the process model, some security
| features like TLS.
| kelnos wrote:
| The downside of the standard library approach is that things
| tend to ossify. While I agree that slower change can be a
| good thing sometimes, putting things like a HTTP server in
| the standard library means less experimentation around
| different ways of doing things, and more difficulty getting
| performance and other improvements into the hands of language
| users.
|
| Sure, people can make a third-party module that implements a
| HTTP server, but the incumbent default that's shipped with
| the language has an inherent (and often unfair) advantage and
| a lot of inertia behind it.
|
| I don't really care about the whole "professionals" bit.
| Sure, I don't want to be relying on something mission-
| critical to me that's maintained by one person doing it in
| their spare time. But there is a world of possibilities
| between that and having a dedicated paid team. Consider,
| also, that the Go team is only funded so long as Go is
| important to Google's corporate strategy. Once it isn't,
| funding will start to dry up, and Go will have to look for a
| new funding and governance model. That's not necessarily a
| bad thing, and I'm sure Go would still succeed despite that.
| But that's kinda my point: this whole "maintained by funded
| professionals" thing doesn't really matter all that much.
| svnpenn wrote:
| > transitive dependencies are automatically updated to pick up
| security fixes
|
| Does Node do this? That seems like an awful idea. People should
| be manually updating dependencies, never automatically. Stuff
| like dependabot need to die.
| coder543 wrote:
| Dependabot gives you an easy way to review every single
| commit that went into a dependency update before you merge
| it.
|
| Dependabot is by far the most convenient way that I've seen
| to _actually_ check that your dependency updates are not
| overtly malicious.
|
| It's not some tool that just removes your lockfiles behind
| your back, as you seem to be implying.
| verdverm wrote:
| There are many gems in its design and implementation in addition
| to those mentioned in OP.
|
| https://verdverm.com/go-mods/
| unixbane wrote:
| Dependencies being immutable and identified by hash was such an
| obvious thing 20 years ago. The problem was fitting that into the
| flow of these crappy UN*X based build systems where to do any
| mundane task you need to fiddle with files and encodings and semi
| documented folder heirarchies and use a CLI tool to change other
| stuff that is too cumbersome to encode directly into files /
| text. The most obvious concrete instance of this being that it's
| cumbersome to import a dependency by hash (in Java for example,
| you really want hashes everywhere instead of reverse TLDs) but
| obviously better tools (such as a structural code editor) solves
| this perfectly. It's also annoying every time a company takes a
| nano step asymptotically towards these proper solutions and spams
| their crap and everyone buys it, but that's to be expected when
| your field is broken industrially and academically and even in
| hobbiest communities.
| doliveira wrote:
| I feel like much of the current software complexity is mostly
| caused by the pain that is compiling C dependencies and the
| outdated Unix build and configuration tools.
| armchairhacker wrote:
| Ah, the old "we've already solved this 40 years ago". Lambdas,
| immutable-by-default, async/await, etc.
| unixbane wrote:
| Having dependencies not change is literally trivial. Just
| link to them by hash. Async/await is a pragmatist garbage
| hack, and does not fit in the archetype you are trying to
| name.
| 37ef_ced3 wrote:
| Go is a distillation of many decades of software engineering
| experience. The people behind Go (e.g., Russ Cox) have learned
| from history.
|
| The peanut gallery loves to complain about superficial aspects of
| Go. Typically these are people with little or no actual
| experience using the language and tools. They fixate on imagined
| problems that don't matter in practice.
|
| But anyone who has used Go full-time for a few years is likely to
| deeply respect and appreciate it.
| kelnos wrote:
| I haven't used Go full-time, but I have used it on and off for
| more than a few years. There are certainly things I respect and
| appreciate about it, but there are also a lot of things that
| annoy me about it. Some that you might consider "superficial",
| I consider important. If I'm going to be spending all day in a
| language, I want the ergonomics of the language itself to work
| with me, not against me, and Go often does not fit the bill
| there (for me -- others' opinions are free to differ).
|
| I don't find most of this article to be all that persuasive.
| Rust, for example, has a separate lock file, which the article
| derides. That doesn't really make sense, as lock files are also
| checked into source control, so you get the same benefit that
| Go touts of go.mod. My threat model doesn't consider having a
| separate module/package repository to be much of a risk, so I
| don't care about that point all that much. Admittedly, having
| source control be the source of truth is just simpler, which is
| good, but it also means that module publishers can pull
| versions (or the entire module) for arbitrary, selfish reasons,
| and then the community is left with a lot of difficulty
| (there's also a big problem if someone wants to move their code
| from GitHub to GitLab or something like that). Centralized
| module repositories can remove this problem if they choose to.
| The Go Module Mirror appears to be a hack that tacitly admits
| this problem.
|
| I did find the "a little copying..." bit to be interesting, and
| I agree with it. With Rust, pulling in a single dependency
| tends to pull in many tens of transitive dependencies, which I
| don't like.
| merely-unlikely wrote:
| "but it also means that module publishers can pull versions
| (or the entire module) for arbitrary, selfish reasons, and
| then the community is left with a lot of difficulty"
|
| By default go get will download the source code into the
| pgk/mod folder. So if a module is pulled by the author, you
| can just use your copy of the source to fork it.
| kelnos wrote:
| That's fine if you've already downloaded it, but doesn't
| help for people trying to pull it for the first time,
| either as a direct or transitive dependency.
| s-video wrote:
| I love Go as well, but I don't think being the first to kick up
| a flame war will do anything for us.
| mperham wrote:
| I couldn't have said it better.
|
| Other ecosystems decided to use lots of little dependencies and
| I can't imagine a worse engineering decision. This way lies
| madness.
| geodel wrote:
| Agreed. There are some folks very closely associated with other
| language ecosystems but they spend more time in endlessly
| critiquing every little thing about Go.
| laurent92 wrote:
| I wonder how there are not more NPM vulnerabilities. We
| control nothing.
|
| Especially during war.
| ______-_-______ wrote:
| For a decade+, people complained that go lacks generics. Would
| you say those were all people with no experience fixated on an
| imaginary problem that doesn't matter, or were the complaints
| valid?
| philosopher1234 wrote:
| I would say that it is legitimately annoying to have to copy
| and paste data structures, but mostly doesn't matter, and the
| intensity of the complaining does not match the intensity of
| the problem.
|
| People mostly just dont like the expertise of the go devs.
| laerus wrote:
| if err != nil { go to(); fmt.Println("ok matey") }
| icholy wrote:
| let foo; try { foo = await
| is_this_better(); } catch (err) {
| console.log("if err != nil doesn't seem so bad all of a
| sudden"); }
| throw_m239339 wrote:
| Well you can actually put many exception yielding
| statements in the try, unlike with go where your code is
| full of "if err:=nil{}" after every potential error.
|
| Most of the time when an exception occurs in sequential
| operations that can fail, you don't really care where it
| failed exactly, only that the failure was caught.
|
| The irony is that Go does have half baked exceptions
| (panic,recover) on top of that error as value convention.
| philosopher1234 wrote:
| Far easier to work with and understand when you dont need
| to perform a massive disruptive ceremony to handle
| exceptions. I've been working full time in java for the
| past 4 years and basically no one handles exceptions
| because its so cumbersome and bad to read. Not so in Go.
| Go does it better.
| throw_m239339 wrote:
| > Far easier to work with and understand when you dont
| need to perform a massive disruptive ceremony to handle
| exceptions. I've been working full time in java for the
| past 4 years and basically no one handles exceptions
| because its so cumbersome and bad to read. Not so in Go.
| Go does it better.
|
| No Go doesn't do it better, Go just doesn't give you the
| choice, from a convention perspective. You have a
| discipline issue with Java, it is not an issue in the
| language itself, it's with you and your team.
|
| Ironically I've seen a few Go libs and project trying to
| re-invent optional types due to the verbosity of Go
| errors, just like people were trying to roll their own
| generics with interface {} or code generation before they
| were added to the core. It's a demonstration that some
| developers don't like the status quo.
| philosopher1234 wrote:
| > No Go doesn't do it better, Go just doesn't give you
| the choice, from a convention perspective. You have a
| discipline issue with Java, it is not an issue in the
| language itself, it's with you and your team.
|
| Its my entire company, and its with the open source
| ecosystem. Its naive to think that language constructs
| dont influence dev's decisions. You need to make it easy
| to do the right thing, not hard, and go makes it easy,
| which java fails at.
| azth wrote:
| The right thing is to automatically maintain error
| context (e.g. stack traces) which golang fails at.
| Furthermore, the vast majority of the time you will have
| a top level handler that will log the error anyway.
| golang is just introducing boilerplate for the sake of
| it, with none of the upsides. Not to mention it also
| makes it easy to accidentally ignore or overwrite errors,
| I've seen several instance of that happening over several
| years of working in golang. Something that would never
| have happened in an exception based language.
| morelisp wrote:
| > Something that would never have happened in an
| exception based language.
|
| Please, over-swallowed exceptions happen all the damn
| time. Same shit different day.
|
| A really common Java problem: T x =
| foo(); return x.bar();
|
| Now foo() can throw, OK try {
| T x = foo(); return x.bar(); } catch
| (Something exc) { return quux(); }
|
| You just ate anything from bar(). Instead you gotta do
| T x; try { x = foo(); } catch
| (Something exc) { return quux(); }
| return x.bar();
|
| Which is longer and now your happy-path logic is noised
| up even worse than most Go handling.
| throw_m239339 wrote:
| > Its my entire company, and its with the open source
| ecosystem. Its naive to think that language constructs
| dont influence dev's decisions. You need to make it easy
| to do the right thing, not hard, and go makes it easy,
| which java fails at.
|
| Go makes it pretty easy to ignore errors, it's even worse
| than Java. An uncaught managed exception will not
| compile, and an uncaught unmanaged exception might
| terminate your program. _ :=
| YieldsAnError()
|
| will will never terminate your program. Go makes it
| pretty easier to do the wrong thing.
| philosopher1234 wrote:
| You're arguing a technicality, which is a waste of time.
| We should focus on reality, not imaginary theoretical
| issues. In reality, that never happens. And people even
| develop linters (frankly unnecessary) to guarantee it, if
| you're really paranoid.
|
| This is a social problem, influenced by language design.
| You need to be thinking about the way humans actually
| behave in practice, not how they theoretically might
| behave.
|
| In practice people do not accidentally skip errors in Go.
| In practice people do not handle errors at all in Java.
| This is the responsibility of the respective cultures and
| language designs.
| throw_m239339 wrote:
| > You're arguing a technicality, which is a waste of
| time. We should focus on reality, not imaginary
| theoretical issues. In reality, that never happens. And
| people even develop linters (frankly unnecessary) to
| guarantee it, if you're really paranoid.
|
| Because you aren't? you're projecting.
|
| > This is a social problem, influenced by language
| design. You need to be thinking about the way humans
| actually behave in practice, not how they theoretically
| might behave.
|
| Sure, your problem with Java is also a social issue that
| has nothing to do with the language.
|
| > In practice people do not accidentally skip errors in
| Go. In practice people do not handle errors at all in
| Java. This is the responsibility of the respective
| cultures and language designs.
|
| This isn't Java's "culture" to not handle errors at all.
| Go's conventions imposed in its std libs are certainly
| not a superior model by any measure as I previously
| showed.
|
| Edit: Answer to your follow up since HN ratelimits
| flamewars
|
| > This is java's culture, and I assert it from
| experience. It is a result of the language design (and
| communiques from language authorities).
|
| Your appeal to authority is a logical fallacy, you have
| never demonstrated that it is Java culture. I say it
| isn't Java culture the same way you establish an
| unfounded claim.
|
| > You only showed that it was possible to misbehave in
| Go, which is totally uninteresting. You have failed to
| show that your theoretical problem is a practical
| problem. This is a perfectionist argument.
|
| You don't get to say what argument is deemed interest and
| which isn't in a discussion.
|
| Ignoring Go errors is as much as a practical problem that
| not handling Java exceptions. The difference being that
| Java does force the developer to catch managed
| exceptions, Go doesn't care, it's purely a convention. A
| language construct to deal with errors baked into a
| language is in theory and practically superior to random
| conventions established by a vendor. PERIOD.
|
| Whatever experience you had with Java isn't
| representative of Java's culture at all.
| philosopher1234 wrote:
| >Sure, your problem with Java is also a social issue that
| has nothing to do with the language.
|
| Quoting myself: "This is a social problem, influenced by
| language design."
|
| >This isn't Java's "culture" to not handle errors at all.
| Go's conventions imposed in its std libs are certainly
| not a superior model by any measure as I previously
| showed.
|
| This is java's culture, and I assert it from experience.
| It is a result of the language design (and communiques
| from language authorities).
|
| You only showed that it was possible to misbehave in Go,
| which is totally uninteresting. You have failed to show
| that your theoretical problem is a practical problem.
| This is a perfectionist argument.
| azth wrote:
| Having worked on golang code bases for several years now,
| I assure you it's not a theoretical problem. It's not fun
| to have ignored or overwritten errors that keep the
| program going as if nothing is happening.
| randomdata wrote:
| If you overwrite or ignore your variables you're going to
| have a bad time. Period. That's not unique to errors. I'm
| not sure you've said anything to convince us that making
| mistakes around errors alone is anything more than
| theoretical.
|
| Beyond defining an interface named error (early versions
| didn't even offer that), Go does not even have a concept
| of errors, so I'm not sure it is even possible for there
| to be a problem specific to errors. Certainly Go could do
| more to help you with correctness in general.
| [deleted]
| randomdata wrote:
| _> Go just doesn 't give you the choice_
|
| The encoding/json package in the Go standard library uses
| exceptions for error handling. You absolutely have a
| choice, but it's a choice you're not likely to want to
| make because exceptions are named exceptions for good
| reason: It's for exceptions, not errors. Those are very
| different things.
|
| But sometimes the trade-offs of overloading something
| beyond its intent is worth it. In those cases do it.
| Balancing trade-offs is what engineering is all about.
| ryeguy wrote:
| This comparison misses the point. Go requires you to bubble
| up errors manually, that's the difference.
| randomdata wrote:
| At least one can respect Go for being consistent, even if
| requires a little extra manual labour. The languages that
| require you to manually bubble up some types, but not
| others, are just bizarre. Like, they're onto something
| neat, but why stop halfway? I don't want to put in the
| manual work for _any_ type. If I 've decided the manual
| effort is worthwhile, I don't want weird exceptions to
| worry about. Pick a lane, programming languages.
| lostcolony wrote:
| receive
| {lack_of_erlang_error_handling_is_a_missed_opportunity,
| true} -> ok after 0 -> throw let_it_crash
| end.
| mFixman wrote:
| func ThisCouldUseExceptions() error { err :=
| Exceptions() if err != nil {
| return terrors.Propagate(err) }
| err = GeneraliseBetter() if err != nil {
| return terrors.Propagate(err) }
| value, err = InFunctionsWithMultipleErrorPoints()
| if err != nil { return
| terrors.Propagate(err) }
| value, err := AlsoTheyHelp() if err != nil {
| return terrors.Propagate(err) }
| chained_val, err := WithChaining(value) err =
| AndPreventAccidentallyIgnoringErrors(chained_val)
| if err != nil { return
| terrors.Propagate(err) } }
| nmilo wrote:
| Generally I find if you're writing code like that, your
| code is too high level. Okay, you've propagated errors,
| but what is the calling code going to do with those
| errors? I find large functions like that usually have
| enough context to handle the errors themselves. Errors,
| after all, are just conditions with a special name. It's
| not common to propagate conditions up several layers of
| code, so why do the same with errors?
| the_duke wrote:
| Go doesn't have (automatic) backtraces. If you don't wrap
| errors with a trace or with a custom message you often
| have no idea where the error came from.
| merely-unlikely wrote:
| if err != nil { return mherr.WrapErr(err,
| "optional context") }
|
| I just wrote a custom function that adds the stack trace
| to the error. I also have it setup as a code snippet in
| VS Code so all I have to do is type "if err" and hit tab.
| Yes it looks a bit verbose but it adds approximately zero
| extra work and makes error handling extremely easy by
| default.
|
| Also, turn on log flags to add line numbers.
| log.SetFlags(log.Llongfile | log.LstdFlags)
| philosopher1234 wrote:
| func ThisCouldUseExceptions() error { err :=
| Exceptions(arg) if err != nil {
| return fmt.Errorf("if a '%v' dev: %w", err) }
| err = GeneraliseBetter() if err != nil {
| return fmt.Errorf("has to actually: %w", err)
| } value, err =
| InFunctionsWithMultipleErrorPoints() if err
| != nil { return fmt.Errorf("read these
| errors: %w", err) }
| value, err := AlsoTheyHelp() if err != nil {
| return fmt.Errorf("they will actually be: %w", err)
| } chained_val, err :=
| WithChaining(value) if err != nil {
| return fmt.Errorf("useful: %w", err) }
| err = AndPreventAccidentallyIgnoringErrors(chained_val)
| if err != nil { return fmt.Errorf("this
| isnt a real problem: %w", err) } }
| bjackman wrote:
| I've been using Go for a couple of years now and it's funny: I
| do not love Go, I just work effectively in it. I am not
| passionate about it, but I recommend it for absolutely all
| appropriate use-cases.
|
| It doesnt go for intellectual satisfaction, it goes for getting
| shit done. You have to respect it for being so radically bland.
| 0des wrote:
| Go is boring and that's appreciated by myself and others in
| my peer group.
| jatone wrote:
| I get intellectual satisfaction from solving the problems not
| from the language I use to do it. go hits a fantastic blend
| in that regard.
| zdw wrote:
| Some of those "superficial aspects" are really annoying.
|
| For example, Go forced version tags to have a `v` prefix in git
| repos for their dependency system, which broke a whole host of
| CI tools that expected plain numeric values for release
| versions. There's a outsized amount of Go-specific special
| casing for this one seemingly arbitrary decision in multi-
| language CI systems.
| morelisp wrote:
| > Go forced version tags to have a `v` prefix in git repos
| for their dependency system
|
| "Forced" is a bit strong - you can pin any ref, the vX tags
| just also have some default semver-ish treatment.
|
| > a whole host of CI tools that expected plain numeric values
| for release versions
|
| Like what? Pure numeric tags are also ambiguous with git
| refs; this can be worked around by careful arguments, but it
| means most tooling was already broken when dealing with such
| things.
|
| If you were one of the people using "release-X.Y.Z" or "rXYZ"
| I feel for you though.
|
| > Go-specific special casing
|
| 'v' tags have been idiomatic for ages. Semver 1.0 went so far
| as to _mandate_ it in 2010, though that was taken out in 2.0.
| https://semver.org/spec/v1.0.0.html#tagging-specification-
| se...
| nu11ptr wrote:
| Couldn't disagree more (I spent years writing Go, and now Rust
| - will NEVER go back)
| azth wrote:
| I've used golang at my employer for several years now, and I
| came out respecting and appreciating the design decision that
| have and are going into Java/C#/Kotlin much more given the
| atrocities I've seen written in golang.
| sbmthakur wrote:
| Are those some common Golang anti-patterns?
| alasdair_ wrote:
| Off topic but: Since I assumed this was about _physical_ supply
| chain attacks (where someone nefarious will either intercept your
| package to install custom firmware etc. or even change the
| physical device in some way) - does anyone know where I could
| find a good guide on mitigating such attacks?
| codeflo wrote:
| The article, and the comments praising this approach, don't do a
| great job of explaining how any of this is substantively
| different from running the likes of yarn install --frozen-
| lockfile, or cargo build --frozen.
|
| Here's the thing: You can argue about being secure by default and
| encouraging better CI practices. I'd fully agree it isn't great
| that one has to know a somewhat obscure flag to get a secure CI
| build in those environments.
|
| But claiming in what I perceive to be in parts a somewhat
| grandiose tone to have reinvented the wheel, when you're just
| describing a standard approach, can make you sound uninformed.
| skybrian wrote:
| I think at most there's pride in their own solution, which is
| not something anyone should object to - it's pretty good. It's
| better than _some_ other systems, but no point in being
| specific.
|
| Not doing specific comparisons is likely a deliberate strategy,
| since it means the blog post is less likely to go out of date,
| and it avoids controversy if they get something wrong.
|
| Comparisons will need to be written by people familiar with
| both systems, and they're likely to go out of date quickly.
| nu11ptr wrote:
| > Unlike most other package managers files, Go modules don't have
| a separate list of constraints and a lock file pinning specific
| versions.
|
| The weird thing about the Go devs is there is always that little
| bit of elitism under the surface that I detect in their writing
| (whether it be colors in the playground, the GC, etc). I spent
| years writing Go and have now moved to Rust. What I find odd is
| the Rust team has done (IMO) one of the greater achievements in
| PL history and yet they seem to not have this elitism thing going
| on (or maybe I just haven't noticed). Go on the other hand, IMO,
| made some "interesting" language choices (like keeping null) and
| they seem to want to be celebrated for it and claim their
| achievements as new and novel.
|
| EDIT: To clarify, I'm talking about the core Go devs - those that
| work on stdlib and the compiler
| xiaq wrote:
| The elitism stems not from solving cutting-edge PL problems,
| but from making technical decisions informed by engineering
| experience.
|
| How much of that is justified is up to you, of course.
| vlunkr wrote:
| I don't see that elitism in this article. Supply chain attacks
| are a hot topic right now, so it makes sense for them to make a
| statement about where the language stands with them. They make
| compelling points, and they're not calling out specific
| language or package manager as a comparison.
| jchw wrote:
| This is a pretty genuinely confounding response, and I mean
| that with absolutely no offense intended. There is a tremendous
| amount of fighting between devs who prefer Go and Rust, and a
| tremendous amount of elitism as well, truly from both
| perspectives. Rust gained a reputation for elitism long before
| Go did; "Rust Evangelism Strike Force" was never meant to be
| pejorative, and "Rewrite it in Rust" was never meant to be a
| joke, but it became one anyways. It's not hard to see why; Rust
| is genuinely novel in a way that few other programming
| languages are. It feels the most like the "future."
|
| But I still like Go a lot. I like Go because of how easy and
| simple it feels. There is definitely elitism over simplicity,
| but the elitism I've seen and even received from Rust and C++
| programmers (...despite that I have been coding C++ forever and
| do have a few Rust projects as well...) has been pretty much
| the opposite: Go is too stupid and simple; _real_ programmers
| need absurdly complex metaprogramming to make basic CLI tools
| or what have you. Now for what it's worth, that has cooled down
| in many regards, and also, Rust is amazing and there's nothing
| wrong with advanced metaprogramming. (It's just another set of
| tradeoffs, after all. Unquestionably has its benefits.)
|
| However, whereas people who have hated on Rust have often come
| to see it for what it is (an immensely cool, novel programming
| language,) Go has received the opposite treatment. People
| soured on it. Now everyone seems sure the GC latency (which of
| course is just about state of the art) is simply too much for
| most use cases. It's seen adoption in all sorts of places and
| even been competitive with Rust software in performance, but it
| is commonly discussed as if Go is inherently obsolete because
| Rust is a better option in every way that matters. Bringing up
| Go in certain places often subjects you to ridicule, and I'm
| not joking. The memory ballast is a favorite among detractors
| to prove that the language is stupid and bad for production
| environments.
|
| So when people do try to tout the benefits of Go, it's
| routinely discredited and downplayed for some reason. It's a
| nice language to use with a stellar standard library, nice
| tooling, and pretty good runtime performance.
|
| This article doesn't mention Rust (that I noticed) and Go is
| _still_ being measured up to Rust in the comments. They both
| trade blows in different categories, but I truly believe that
| the fact that Go lacks the novelty of Rust with its borrow
| checker and language design has caused a lot of people to view
| it very negatively, and I think that is sad. People loved C for
| a lot of what it didn't have. Go is a lot different than C, but
| for me, the sentiment is very much the same.
|
| I think people see what they want to see. I like Go and Rust,
| but I find myself going back to Go for various reasons and it
| feels like every year it leads more and more people to ask for
| justification that they wouldn't for other languages. It's a
| little tiring.
| nu11ptr wrote:
| To be clear, I have no issue with anyone who prefers Go. I
| was speaking about the core devs in particular. I would agree
| that the users of the language definitely go back and forth
| "trading blows".
| jchw wrote:
| Interesting and apologies for misunderstanding. I didn't
| read the article as being elitist, though I can see how it
| reads as self-congratulatory to a degree. Maybe the matter-
| of-fact way that Go's developers state its advantages comes
| off poorly compared to, for example, coming from the
| standpoint of trying to explain how they got to their
| current design based on the challenges. Personally, I find
| articles like this easier to read because they tend to be
| more terse when written this way versus some other
| approaches that are perhaps more humble.
| howinteresting wrote:
| The problem with Go is that it is simply impossible to write
| correct code in it in many domains:
| https://fasterthanli.me/articles/i-want-off-mr-golangs-
| wild-...
|
| > Over and over, Go is a victim of its own mantra -
| "simplicity".
|
| > It constantly takes power away from its users, reserving it
| for itself.
|
| > It constantly lies about how complicated real-world systems
| are, and optimize for the 90% case, ignoring correctness.
|
| > It is a minefield of subtle gotchas that have very real
| implications - everything looks simple on the surface, but
| nothing is.
| hu3 wrote:
| That's fine, Go isn't meant to be used in many domains.
|
| It thrives in infrastructure and web servers.
| jchw wrote:
| I've read this article and debated it to the end of the
| planet. It's not that the article is factually wrong, but
| it seems to be under the impression that there is an
| objective definition for "correct code."
|
| Of course, there's degrees. To take an example, filenames.
| What is a filename? Is it a bag of bytes, a set of UNICODE
| codepoints, something more elaborate? No operating system
| agrees 100%. Go handles this by just simply not. It
| converts stuff to and from UTF-8 and if it doesn't convert,
| you can't use it. This is a limitation of Go programs that
| use Go's built-in filesystem support (roughly all of them.)
|
| That decision is a clear-cut simplification of reality. It
| can matter. However, the fact that it comes up so seldom is
| a reflection of reality: normally, you are totally able to
| make the concession that you only handle paths that are
| valid UTF-8 and nothing else. Go makes this concession _on
| your behalf_. It makes many such concessions, and it is
| documented, though not very well-known, because most
| developers don't care. What most developers want is a
| language that makes sensible tradeoffs so that their
| programs can remain simple but still reasonable. In TYOOL
| 2022, I absolutely think it is reasonable that a program
| may impose valid UNICODE filenames as a requirement.
|
| Rust handles it with OsStr, of course. Now OsStr is a good
| idea, but it pushes the decision for how to handle the
| problem further down. On one hand, this is great if you
| absolutely must handle ridiculous filenames that have
| essentially arbitrary bytes or old encodings in them,
| which, to be sure, is a real thing.
|
| The file permissions stuff is similar. If Go is going to
| present a file permissions interface and then just fudge
| it, what's the point of including it at all? Well, in my
| estimation, the point is that on POSIX systems, you can't
| really avoid dealing with file permission bits. In most
| cases, when a Go program specifies mode bits, its doing so
| so that on POSIX, the program doesn't write a file with
| permissions that don't make sense, are unusable, or
| potentially even open up security issues. (You would not
| want user-uploaded data to go into a file with +x!) On
| Windows, it can usually be ignored, at least from the scope
| of the Go software. That's, again, an opinionated choice.
| If I needed more granular control over permission, I would
| probably need more OS-specific code and interfaces anyways,
| something that is totally doable in Go.
|
| So far Go is oversimplifying things in a very opinionated
| manner. And in fact, this means that some programs are
| difficult to write correctly in Go.
|
| But, and here's the kicker, I don't usually want to write
| code that breaks the assumptions that Go makes and requires
| in its standard library. Even if I'm in Rust, if I want to
| write some utility code that deals with filenames, often
| times the string transformations I want to perform on the
| filename I will want to do in the space of valid UNICODE,
| because it's simple and predictable. Even if all I want to
| do is append a suffix before the file extension, I still
| would prefer to work in valid UNICODE only. If I'm dealing
| with say, a broken Shift-JIS filename and want to append a
| suffix, even if I try to do the right thing and treat it as
| an ASCII compatible bag of bytes, I could wind up with an
| even more broken filename as a result, because the state of
| the filename before the suffix could corrupt the suffix.
|
| The key difference is in perspective. "100% correct" is
| demonstrably unattainable. You can make more exact
| abstractions, but for sanity we make assumptions left and
| right that are "good enough." You really shouldn't use the
| user home directory or user profile directory of a given OS
| for much directly because it's honestly a different
| directory with very different semantics per OS, yet plenty
| of Rust software (and Go software!) still does this
| _anyways_.
|
| Meanwhile, while writing perfectly watertight code in Go is
| basically impossible, it's also not a fair benchmark.
| Nothing does that. Rust is _especially_ prone to stack
| overflows due to its lack of placement new, and it's
| absolutely _trivial_ to do this. Go doesn't really have
| stack overflows at all, because it has a dynamically
| expanding stack. Cargo and its ecosystem has a problem with
| dependency hell; 200 nested dependencies to install a tool
| that counts lines of source code is a real thing that
| actually exists. Go's ecosystem is definitely in better
| shape here, for reasons mentioned in the article that I
| wholeheartedly agree with. There are more things I could
| say that I _don 't_ like about Rust. I could possibly even
| fill an article about it, but I don't think that it's very
| productive. I think it's better to just acknowledge that
| most of these pain points are a direct result of the fact
| that any decision you make about language design and
| tooling will have knock-on effects down the ecosystem.
|
| This article is also unfair in that it basically pins Go, a
| language which intentionally limits scope by taking these
| concessions, with Rust, a language that happily expands
| scope to try to cover more edge cases. Both choices are
| valid and offer different tradeoffs. However, there is a
| _world_ of different programming languages, and Go is not
| the first one that implies limitations on filenames, for
| example. Hell, what happens if you write plain C code that
| tries to access broken Windows filenames? How much Win32 C
| code exists that doesn't handle UNICODE or broken
| filenames? How many times have you tried to compile some
| code and had to move it to a directory without spaces in
| the filename because the program didn't handle that?
|
| Go is an opinionated language. If you don't like its
| opinions, you won't be happy with it. Orienting this as a
| correctness issue makes it seem like the Go developers made
| these tradeoffs haphazardly and without thought. I just
| flatly disagree.
| philosopher1234 wrote:
| Lovely comment. I share your sentiment. Go is truly hated on
| hacker news these days (and, if you're brave enough to
| venture there, reviled on r/programming).
___________________________________________________________________
(page generated 2022-03-31 23:00 UTC)