[HN Gopher] Writing secure Go code
___________________________________________________________________
Writing secure Go code
Author : gus_leonel
Score : 353 points
Date : 2024-11-04 17:34 UTC (1 days ago)
(HTM) web link (jarosz.dev)
(TXT) w3m dump (jarosz.dev)
| gus_leonel wrote:
| TIL about `gosec`.
| johnisgood wrote:
| I use VSCodium when I am programming in Go, using the
| extension, because it has everything I need, and that includes
| gosec.
| goodlinks wrote:
| Does go have a bad security reputation?
|
| I get that anything can be insecure and its a constant battle as
| this article suggests, but i thought it was quite secure and
| stable generally (say on a par with .net or any other tool you
| may use to make a web app at least?)
| wbl wrote:
| You can write SQL injection in any language.
| jiehong wrote:
| You can use outdated dependencies in any language.
| valbaca wrote:
| > i thought it was quite secure and stable generally
|
| It is, but security isn't a "given" anywhere. XSS, SQL
| Injection, Dependency etc can be done by any language,
| regardless of how "secure" it claims to be.
|
| The headings are all pretty general (versioning, tooling,
| scanning, testing) but the contents are Go-specific.
|
| It's a pretty good article IMO and could/should be replicated
| for other languages as well.
| jerf wrote:
| No.
|
| Ironically, a flip side of the complaints about how Go lacks
| power is that a lot of the "standard" security vulnerabilities
| actually become harder to write. The most obvious one is
| lacking the "eval" that a dynamic language has; more subtle
| ones include things like, there is _no way_ to take a string
| and look up a type or a method in the runtime, so things like
| the Ruby YAML vuln are not assisted by the language level. To
| write something like that into Go, you 'd have to actually
| _write it in_. Though you can, if you try hard enoough.
|
| But, as sibling comments point out, nothing stops you from
| writing an SQL injection. Command injections are inhibited by
| the command only taking the "array of strings" form of a
| command, with no "just pass me a string and we'll do shell
| things to it" provided by the language, but I've dispatched
| multiple questions about how to run commands correctly in Go by
| programmers who managed to find []string{"bash", "-c", "my
| command with user input from the web here"}, so the evidence
| suggests this is still plenty easy enough to write. Putting the
| wrong perms or no perms on your resources is as easy as
| anything else; no special support for internal security
| (compare with E lang and capabilities languages). And the file
| access is still based on file names rather than inodes, so
| file-based TOCTOUs are the default in Go (just like pretty much
| everywhere else) if you aren't careful. It comes with no
| special DOS protection or integrated WAF or anything else. You
| can still store passwords directly in databases, or as their
| MD5 sums. The default HTML templating system is fairly safe but
| you can still concatenate strings outside of the template
| system and ship them out over an HTTP connection in bad ways.
| Not every race condition is automatically a security
| vulnerability, but you can certainly write race conditions in
| Go that could be security vulnerabilities.
|
| I'd say Go largely lacks the footguns some other languages
| have, but it still provides you plenty of knives you can stab
| yourself with and it won't stop you.
|
| I've been running govulncheck against my repos for a while, and
| I have seen some real vulnerabilities go by that could have
| affected my code, but rather than "get arbitrary execution"
| they tend to be "didn't correctly escape output in some
| particular edge case", which in the right circumstances can
| still be serious, but is still at least _less_ concerning than
| "gets arbitrary execution".
| Smaug123 wrote:
| > I'd say Go largely lacks the footguns some other languages
| have
|
| With the glaring exception of "I forgot to check the error
| code", which you need a linter (e.g. as provided by golangci-
| lint) for. It's critically important for security that you
| know whether the function you just called gave you a
| meaningful result! Most other languages either have sum types
| or exceptions.
| tptacek wrote:
| No it's not. This is what I meant, cross-thread, when I
| suggested being wary of arguments trying to draw
| _significant_ distinctions between memory-safe-language X
| and memory-safe-language Y. Error checking idioms and
| affordances have profound implications for correctness and
| for how you build and test code. Programmers have _strong_
| preferences. But those implications have only incidental
| connections to security, if any. Nevertheless "security"
| is a good claim to throw into a "my language is better"
| argument.
| Smaug123 wrote:
| I don't even use Golang, I maybe read two Golang repos a
| year, I find these errors in almost every repo I look at
| (probably because of the selection effect: I only look at
| the code for tools I find bugs in). One of them I
| remember was a critical vulnerability of exactly this
| form, so :shrug: Perhaps I'm just grotesquely unlucky in
| the Golang projects I see, but that makes maybe 10% of
| the Golang error-handling bugs I've found to be security
| bugs.
| tptacek wrote:
| Sounds memorable. Say more about this critical
| vulnerability?
| Smaug123 wrote:
| I'll gesture at it. It's not an open source tool, so I
| can't point at the code (and in fact I just checked and I
| don't have perms to see the Jira ticket I caused to be
| raised!), and I am wary of describing security bugs in
| company-internal code. But in general terms it was a
| service that attempted to check whether a request was
| allowed, and it ignored errors from that check. (I just
| searched history for a bit to find the error in the
| absence of any actual details about it, but it was a
| while ago and I failed.) Sorry this is not a very
| satisfying answer.
| foldr wrote:
| Any language where errors are returned as values will
| allow you to ignore errors (if you don't have proper
| linting set up, and unless it has something fancy like
| linear types). I've even seen a similar error in Haskell
| code, where someone called an isLoggedIn function inside
| a monad with the expectation that it would short-circuit
| evaluation, whereas in fact it just retuned a Bool.
| thinkharderdev wrote:
| Very true, but I do think there is an issue in the margin
| about how easy it is to ignore errors. For example, in
| Java you might have something like
|
| ``` void checkPermissions() throws AuthException ```
|
| so you have to actively ignore errors by catching the
| exception. Likewise in Rust you can do
|
| ``` fn check_permissions() -> Result<(),AuthError> ```
|
| In that case you can just use the `?` operator to short-
| circuit (and clippy will warn you if your forget to do
| that).
|
| In other words, while language design can't fully prevent
| you from ignoring precondition checks, it can make it
| harder to forget or even force you to actively ignore
| precondition failures
| randomdata wrote:
| But isn't the idiomatic Go solution something like this?
| func checkPermissions(success func())
|
| Like anything, you can still screw it up if you try hard
| enough, but it should nudge most in the right direction.
| The talk of error handling seems like a distraction or a
| case of someone confusingly trying to write code in
| another language using Go syntax.
|
| Obviously you are not forced to think of the user when
| designing an API, but you don't have to be mindful of the
| user in any language. Not even Haskell can save a
| developer who doesn't care, as noted in an earlier
| comment.
| foldr wrote:
| Go linters do a pretty good job of spotting where error
| return values have been ignored, so I'd suggest that the
| kind of bug the OP is referring to is pretty unlikely to
| happen in a Go project that's properly configured.
| Smaug123 wrote:
| Sure - my question is "why do you need to set up the
| third party linters when they're so critical to the
| correctness of your program" really. It's the general
| "yeah we'll give you these footguns which are the first
| thing every developer will learn about during their first
| incident; good luck, and I hope you know you need to do
| things _this way_! " attitude I object to.
| goodlinks wrote:
| Isnt this the same as any language though.. check if have
| permission then ignore the result seems like something
| that the language cannot protect you from?
| Smaug123 wrote:
| I mean, Golang has an unused variables _compile error_ ,
| which presumably is trying to do precisely this. It's
| like they got so close to forcing the user to acknowledge
| the possibility of errors, and then stopped just before
| the end!
| arccy wrote:
| as if DoS by exception is any better...
| Smaug123 wrote:
| Depends on the application! There's a reason we have the
| concept of "failing closed" vs "failing open": sometimes
| (very often, in fact) it's correct to shut down under
| attack, rather than to open up under attack.
| tptacek wrote:
| The subtext of that comment cuts against the argument
| you're trying to make here: a panic following a missed
| error check is always fail-closed, but exception recovery
| is not.
| jerf wrote:
| Mmm, that's fair. I tend to forget about it because it's
| not something I personally struggle with but that doesn't
| mean it's not a problem.
|
| I'd still rate it well below a string eval or a default
| shell interface that takes strings and treats them like
| shell does. You assert down below that you've seen this
| lead to a critical vulnerability and I believe you, but in
| general what happens if you forget to check errors is that
| sooner or later you get a panic or something else that goes
| so far off the rails that your program crashes, not that
| you get privs you shouldn't. As I say in another comment,
| any sort of confusing bit of code in any language _could_
| be the linchpin of some specific security vulnerability,
| but there are still capabilities that lead to more security
| issues than some other capabilities. Compared to what I 've
| seen in languages like Perl this is still only "medium-
| grade" at best.
|
| And I'm not trying to "defend" Go, which is part of why I
| gave the laundry list of issues it still has. It's just a
| matter of perspective; even missing the odd error check
| here or there is just not the same caliber problem as an
| environment where people casually blast user-sourced input
| out to shell because the language makes it easier than
| doing it right.
|
| (Independent of language I consider code that looks like
| operation = determineOperation() if
| !canIDoOperation(operation) { // handle
| failures } doOperation(operation)
|
| architecturally broken anyhow. It seems natural code to
| write, but this is a form of default allow. If you forget
| to check the operation in one place, or even perhaps forget
| to write a _return_ in the if clause, the operation
| proceeds anyhow. You need to write some structure where
| operations can 't be reached without a positive affirmation
| that it is allowed. I'd bet the code that was broken due to
| failing to check an error amounted to this in the end.
| (Edit: Oh, I see you did say that.) And, like I said, this
| is independent of Go; other than the capabilities-based
| languages this code can be written in pretty much
| anything.)
| tptacek wrote:
| I think it's a reasonable observation but it isn't a fair
| comparative security criteria. The subtext behind error
| checking critiques is that languages with idiomatic sum
| type returns avoid authz vulnerabilities, in the same way
| that memory-safety in Go eliminates UAF vulnerabilities.
| But authz vulnerabilities are endemic to the mainstream
| sum type languages, too; they're much more complicated as
| a bug class than just "am I forced to check return codes
| before using return values".
|
| Sum types are one of the few things I miss when switching
| from other languages back to Go. I like them a lot. But I
| think they're wildly overstated as a security feature.
| Sum type languages have external tooling projects to spot
| authz vulnerabilities!
| randomdata wrote:
| _> "I forgot to check the error code"_
|
| How is it that people "forget to check errors" but not
| other types, even though they are all just 1s and 0s? Or,
| to put it another way, why do programmers forget how to
| program as soon as they see the word "error"?
|
| It seems to be a real phenomenon, but I can't make sense of
| how it can happen. It is not some subtle thing like
| misspelling a word in a string constant. You are leaving
| out entire functionality from your application. It is
| almost on the order of forgetting to add the main function.
| vacuity wrote:
| I would think it's a mix of not being sure exactly what
| to do on error and not wanting to undergo the effort of
| writing error logic. You have to switch from "basic
| skeletal structure of the program" to "cover all bases",
| which isn't simple. So it's easy to have no or
| rudimentary error handling, and by the time you want to
| change it, it's hard to change. Like, "malloc can fail,
| but it would be a lot easier right now if I assume it
| won't".
| fweimer wrote:
| One thing to note about data races in Go is that the safe Go
| subset is only memory-safe if you do not have data races. The
| original post alludes to that because it mentions the race
| detector. This situation is different from Java where the
| expected effect of data races on memory safety is bounded
| (originally due to the sandbox, now bounded effects are more
| of QoI aspect). Data races in Java are still bad, and your
| code may go into infinite loops if you have them (among other
| things), but they won't turn a long into an object reference.
|
| The good news is that the Go implementation can be changed to
| handle data races more gracefully, with some additional run-
| time overhead and some increase in compiler complexity and
| run-time library complexity, but without language changes. I
| expect this to happen eventually, once someone manages to get
| code execution through a data race in a high-profile Go
| application and publishes the results.
| tptacek wrote:
| These arguments would be more compelling if they came with
| actual exploitable vulnerabilities --- in shipped code,
| with real threat models --- demonstrating them, but of
| course the lived experience of professional programmers is
| that non-contrived Go memory safety vulnerabilities are so
| rare as to be practically nonexistent.
| cyberax wrote:
| > I'd say Go largely lacks the footguns some other languages
| have
|
| It does have a couple of its own. Like
| ((*SomeStruct)(nil)).(SomeInterface) != nil.
|
| And yeah, the error handling is fucked up.
| jerf wrote:
| I was referring specifically to security footguns like
| having a string eval. While one can construct code in which
| that is the critical error that led to a security
| vulnerability, that can be said about any confusing bit of
| code in any language, and I would not judge that to
| _especially_ lead to security issues.
| cyberax wrote:
| This actually is a security footgun. In Java or C# you
| can't get security issues by trying to update a reference
| from multiple threads, because it's always atomic. In Go
| you can create type confusion because interface pointer
| updates are not atomic.
| jerf wrote:
| This sets the bar ludicrously low for "security footgun".
| If this is a "security footgun" then what is string
| evaluation in a dynamic scripting language, a "security
| foot-nuke"?
|
| Granted, there is no sharp line that can be drawn, but
| given my personal career I'd say _I 've encountered it
| personally at least once_ is a reasonable bar, if not
| quite excessively low. (tptacek would have to set the bar
| somewhere else, given his career.) Concurrency issues
| causing a security issue because of type confusion on an
| interface in a Go program is not a "every time I crack
| open a program, oi, this security vulnerability again"
| like bad HTML escaping or passing things straight to a
| shell. I mean, "concurrency issues causing type confusion
| on an interface" is _already_ not something I 've ever
| personally witnessed, _let alone_ it actually being a
| security issue rather than a difficult-to-trace panic
| issue.
|
| And I will reiterate, I _already_ say that _any_ bug can
| become a security issue in the right context. That doesn
| 't make them all "security footguns".
| cyberax wrote:
| > This sets the bar ludicrously low for "security
| footgun". If this is a "security footgun" then what is
| string evaluation in a dynamic scripting language, a
| "security foot-nuke"?
|
| Not really. Apart from dangerous serialization formats
| (e.g. Python's "pickle") it's not at all easy to eval a
| string in modern scripting languages.
|
| String evals are also not widely used anymore.
| tptacek wrote:
| Point to a real, exploitable, public vulnerability that
| exploits this behavior, and then we'll all be talking
| about the same thing.
| tgv wrote:
| About footguns, I'd like to mention an important one: in Go,
| it's hard to deserialize data wrongly. It's not like python
| and typescript where you declare your input data to be one
| thing, and then receive something else. It's a feature that
| makes server code, which after all is Go's niche,
| considerably more reliable.
|
| Safety isn't 0% or 100%, and the more a language offers, the
| better the result. Go is performant, safe, and fairly easy to
| read and write. What else do you need (in 99.9% of the
| cases)?
| js2 wrote:
| > It's not like python and typescript where you declare
| your input data to be one thing, and then receive something
| else
|
| In Python that's likely to lead to a runtime TypeError, not
| so much in TS since at runtime it's JS and JS is weakly
| typed.
|
| Besides, Python has Pydantic which everyone should really
| should be using. :-)
| tgv wrote:
| Only if you use a deserializer that's tied to your
| classes, and not put everything in a dict. And then only
| if the data encounters an operation that doesn't accept
| it. But many operations accept e.g. strings, arrays, ints
| and floats. Is there even an operation that throws a
| TypeError when using a float instead of int?
|
| Pydantic only helps (AFAIK) when you're letting it help,
| and you actually use the correct type information. It's
| not difficult to use, but it's optional, and can be
| faulty.
| perryh2 wrote:
| > Does go have a bad security reputation?
|
| Depends on who's behind the keyboard.
| Cthulhu_ wrote:
| Reminds me of when SQL injection was the hot security
| problem, which was mainly caused by PHP, but not the language
| itself but reams and reams on low quality online tutorials
| trying to keep things simple by just concatenating GET
| parameters straight into an SQL query.
| tptacek wrote:
| It has essentially the same security properties of all the
| modern non-C-languages (ie, C, C++, ObjC), with the added bonus
| of largely being designed after the deserialization pandemic
| that especially hit Java, Python, and Ruby. ~All these modern
| languages are fine for security (though: be careful with
| serialization formats in anything but Go and Rust).
|
| Arguably, Rust and Go are the two "most secure" mainstream
| languages, but in reality I don't think it much matters and
| that you're likely to have approximately the same issues
| shipping in Python as in Rust (ie: logic and systems
| programming issues, not language-level issues).
|
| Be wary of anyone trying to claim that there are _significant_
| security differences between any of the "modern" or "high-
| level" languages. These threads inexorably trend towards
| language-warring.
| quietbritishjim wrote:
| What is the "deserialisation pandemic"? It doesn't have
| obvious web search results, and I'm struggling to imagine
| what about deserialisation what be common between Java and
| Python (except that, in both cases, I'd surely just use
| protobuf if I wanted binary serialisation).
| kibwen wrote:
| See https://en.wikipedia.org/wiki/Log4Shell , but also
| historically the mess that is pickling/unpickling in Python
| (see the big scary warning at the top of
| https://docs.python.org/3/library/pickle.html#pickle-
| python-... ), and more broadly any dynamic language that
| exposes `eval` in any capacity.
| tptacek wrote:
| For many years, these were the most widespread serverside
| RCE vulnerabilities; Rails YAML might be the best-known,
| but there were a bunch of different variants in Java
| serialization, and a whole cottage subfield of
| vulnerability research deriving different sequences of
| objects/methods to bounce deserializations through. It
| was a huge problem, and my perception is that it sort of
| bled into SSRF (now the scariest vulnerability you're
| likely to have serverside) via XML deserialization.
| plorkyeran wrote:
| In the early 2000/2010s there was a popular idea that it'd
| be neat to have (de)serialization functionality that could
| perfectly roundtrip your language's native objects, without
| requiring that the objects be whatever the language uses as
| plain old data storage. In the happy case it worked super
| well and basically every language sufficiently dynamic to
| support it got a library which let you take some in memory
| objects, write them to disk, then restore them exactly as
| they were at some later time.
|
| This had the obvious-in-retrospect major problem that it
| meant that your deserialization was functionally equivalent
| to eval(), and if an attacker could ever control what you
| deserialized they could execute arbitrary code. Many
| programmers did not realize this and just plain called
| deserialization functions on untrusted data, and even when
| people did become aware that was bad it still turned lots
| of minor bugs into RCE bugs. It was often a long and
| painful migration away from insecure deserialization
| methods because of how darn convenient they were, so it
| continued to be a problem long after it was well understood
| that things like pickle were a bad idea.
| kibwen wrote:
| _> Be wary of anyone trying to claim that there are
| significant security differences between any of the "modern"
| or "high-level" languages. These threads inexorably trend
| towards language-warring._
|
| Hm, I think this is a reasonable take but taken too far.
| Presumably this out of a desire to avoid people arguing about
| this-language-feature vs. that-language-feature, but in
| practice "the language" also gets conflated with the tooling
| and the ecosystem for that language, and having good tooling
| and a good ecosystem actually does matter when it comes to
| security vulns in practice. Indeed, anyone can write SQL
| injection in any language, but having a culture of finding,
| reporting, and disseminating those vulnerabilities when they
| happen, and then having mature tooling to detect where those
| vulnerable packages are being used, and then having a
| responsive ecosystem where vulnerable packages get swiftly
| updated, those are all things that make for more secure
| languages in practice, even among languages with near-
| identical feature sets.
| pants2 wrote:
| I'd point out that one advantage Go has over Rust in terms of
| security are the coverage of standard libraries. Go has great
| support for HTTP clients/servers, cryptography primitives,
| SSH, SQL, JSON, secure RNG, etc. all in officially maintained
| standard libraries. The Rust ecosystem has some standards
| here but the most widely used HTTP client, just as an
| example, is mostly maintained by one guy[1]. I think that
| adds considerable security risk vs Go's net/http.
|
| 1. https://github.com/hyperium/hyper/graphs/contributors
| tptacek wrote:
| For what it's worth, I don't believe there's any meaningful
| security difference between Rust and Go.
| silverliver wrote:
| Good point. If you consider the size of your dependency
| graph as a risk, especially for languages that encourage
| large dependency graphs like JS and Rust, then Go has a
| very clear advantage.
| TheDong wrote:
| My own experience is that the Go stdlib has resulted in
| worse security than, for example, rust.
|
| The reason for that is that both the Rust and Go stdlib
| have a stability promise, so anything built into them can't
| change if it's insecure.
|
| For example, the 'tar' package in go by default returns
| unsanitized paths, and has led to a bunch of CVEs:
| https://github.com/golang/go/issues/55356
|
| The go stdlib can't change the tar package to make it
| secure by default because it would be a breaking change to
| do so.
|
| Rust, on the other hand, has a tar package outside of the
| stdlib, and so it can evolve to be more secure and over
| time find a better interface.
|
| We've seen that with various other packages, where the Go
| stdlib HTTP implementation defaults to no timeouts, and
| thus makes it easy to DoS yourself. Ditto for tcp. The tls
| package has similar backwards compatibility warts that make
| it less secure by default.
|
| Forcing backwards compatibility with network protocols by
| baking them into the stdlib has largely not been a security
| win in my experience.
|
| You can argue that people can build packages outside of the
| Go stdlib too, like if the stdlib "image/draw" package is
| so bad it can't be used, they can make
| "golang.org/x/image/draw", or if the stdlib crypto package
| is bad, they can make "golang.org/x/crypto"... and they
| did, but people still reach for the stdlib because it's
| easier to, which makes it an active security trap.
| tptacek wrote:
| No, I'm not going to give Rust security credit for
| vulnerabilities it avoided in library functionality that
| it simply didn't provide.
| innocentoldguy wrote:
| Elixir is "more secure" than Go due to its isolated
| processes, functional processing, and immutable data.
| tptacek wrote:
| Given the enormity of Elixir's runtime, that seems
| extremely unlikely. The kinds of bugs you expect to see in
| interpreted/VM code are different than those in compiled
| languages like Rust; someone is going to find memory
| corruption, for instance, when you index exactly the right
| weird offset off a binary, or do something weird with an
| auto-promoted bignum. We still find those kinds of bugs in
| mainstream interpreted languages built on memory-unsafe
| virtual machines and interpreters.
|
| I'm not saying Elixir is insecure; far from it. It's a
| memory-safe language. Just, it would be a weird language
| slapfight to pick with a compiled language.
| pram wrote:
| I've been maintaining a Go app for about 9 years now and I can
| just upgrade the Go version + mod for vulnerabilities (GitHub
| tells me about them automatically idk) and it works with no
| changes 99% of the time. I can't overstate how this makes
| maintaining it very stress-free.
|
| My JS apps on the other hand...
| valbaca wrote:
| I shudder to think the amount of thousands of engineering hours
| are spent in my FAANG to keep our Java services just running
| as-is with updates.
|
| And now we're moving more to Typescript on Node...UGH.
| adhamsalama wrote:
| I thought Java was robust. What's the hassle?
| coredog64 wrote:
| Not OP, but typically Spring and transitive dependencies.
| Some package that you don't even use is pulled in and has a
| CVE. Or you upgrade major Spring versions and the API
| changes underneath you.
| koito17 wrote:
| Have people considered frameworks implementing JAX-RS
| instead? Or does the breakage happen specifically in
| extensions to Spring?
|
| The only inconvenience I have experienced upgrading a
| Quarkus backend is renaming javax.* package imports to
| jakarta.*. Hopefully the next major version requires just
| as little effort (if not less).
|
| I am sure there would have been a lot more work if the
| project used extensions like the Kubernetes client. But
| overall, I have had the best experience with tools like
| Maven (for Java) and Leiningen (for Clojure). It helps to
| avoid libraries that hack or access JDK internals (e.g.
| Lombok, or ancient libraries using sun.* internal
| packages for reflection)
| vbezhenar wrote:
| The main problem is Spring Boot and some other Spring
| projects like Security.
|
| If you would use Spring MVC directly, it is very possible
| that one could upgrade Spring versions for many years
| with minimal or no changes at all.
|
| However Spring Boot regularly breaks code. And given the
| fact that it's very popular, it means that any Java
| upgrade is pain. You need to rewrite code, sometimes a
| lot of code.
|
| If you just use JAX-RS, probably simple Spring setup
| would suffice, but people usually want to slap database,
| security stuff and other things and everything is
| provided by Spring, so it's not apples-to-apples
| comparison.
| okeuro49 wrote:
| I recommend Spring Boot, it provides a "blessed" set of
| dependencies that work with each other. When you want to
| upgrade, you just need to increase one version (the
| Spring Boot version).
| blibble wrote:
| if you don't pull in 50 jars to replace the "new" keyword
| (aka spring DI), then this ceases to be a problem
| oftenwrong wrote:
| This Spring hater (me) thinks that's a fair summary. It
| also eliminates much of the safety the compiler can give
| you for free. You will only find out during Spring init,
| or even later, that your program is broken.
| cyberax wrote:
| Java is fairly robust and plenty of libraries are very low-
| velocity.
|
| JVM itself, however, has had several breaking changes
| recently. So a lot of organizations are stuck on an ancient
| version of the language.
| akdev1l wrote:
| Not really true imo.
|
| I speak from the experience of supervised the upgrade of
| thousands of services from JDK8 to JDK17
|
| There's few quirks added but:
|
| 1. JDK17 will happily run JDK8 code without any changes
| 2. Most of the issues I observed were due to project
| jigsaw (and were resolved by adding --add-opens as
| needed)
|
| I would expect 17 > 21 upgrade to have basically no
| issues as an upgrade in place
|
| I hate Java but backwards compatibility isn't one of the
| reasons why I hate it
| cyberax wrote:
| This unfortunately is not true for large codebases. The
| language and the basic library are extremely stable, but
| the overall runtime is not. So the 8->17 switch resulted
| in lots and lots of regressions.
|
| So companies either pay Oracle to maintain the old JDK8,
| or use something like Amazon Corretto. It's so bad that
| there are companies promising JDK8 support until 2031 at
| least.
|
| And yeah, upgrades past 17 are easy.
| kaba0 wrote:
| > It's so bad that there are companies promising JDK8
|
| Come on, that's absolutely not the reason behind. That
| just means that there are banks and such that still run
| goddamn windows XP completely firewalled off from the
| internet just because. Similarly, for some companies not
| touching that ancient codebase and just having it safely
| run worth the hassle and the money.
|
| Java is the most backwards compatible language and it is
| not even a close competition.
| cyberax wrote:
| > Similarly, for some companies not touching that ancient
| codebase and just having it safely run worth the hassle
| and the money. > Java is the most backwards compatible
| language and it is not even a close competition.
|
| It's amazing to have two contradicting sentences right
| next to each other.
| Yasuraka wrote:
| > Java is the most backwards compatible language and it
| is not even a close competition.
|
| In competitions consisting of Java, PHP and Python, I
| presume?
| kaba0 wrote:
| In competitions of real world code in a language that has
| been in use for close to 30 years, and I can find a java
| 1.1 program that will both compile as source on the
| latest version, AND the original compiled version itself
| will run on a modern JDK as is.
| alex-nt wrote:
| I've been working with Java for the last decade and for the
| past 5Y used the latest LTS versions in a very regulated
| environment (we have very strict patch deadlines for most
| CVEs). Rarely we hit issues with migrating to different
| versions of our dependencies. The most painful one was a
| small API change in Spring that revealed that we were doing
| something _very bad_ so it took me 1-2D in between meetings
| to investigate. It is true though that every few weeks we
| are hit by a new CVE and we have to patch a lib version,
| but TBH this is what I expect from a language that has so
| many eyes on it 's ecosystem.
| andreimackenzie wrote:
| A lot of BigCo people's (myself included) perception of
| Java is tainted by the challenges of old, inherited code
| bases. Java has been ubiquitous for a long time, and it's
| not surprising to accumulate code bases that have been
| underserved maintenance-wise over the years. Updating
| dependencies on a Java 8 codebase isn't much fun,
| especially because semvar wasn't widely followed back in
| those days.
| candiddevmike wrote:
| You can run govulncheck as part of your CI pipeline too
| stouset wrote:
| > GitHub tells me about them automatically idk
|
| GitHub tells you about _published CVEs_ which represent a small
| fraction of actual patched security vulnerabilities in the
| wild, which typically never get a CVE.
| dakiol wrote:
| Don't get it. Is it because your Go app relies in fewer
| dependencies? If so, it's just a matter of numbers I believe.
| JS apps tend to rely on more dependencies on average... but
| that doesn't need to be that way. I have plain JS apps that
| still work like the first day (even better than Go apps, since
| there's no compilation step involved).
|
| TypeScript apps on the other hand, yeah, they tend to be more
| fragile (at least from my perspective: the tsc package has
| dozen of dependencies, so anything can go wrong)
| hnlmorg wrote:
| You can do that in practically any language however that
| doesn't mean it's easy nor the norm.
|
| JavaScript has a culture of move fast and break things.
| Whereas Go has a culture of moving slow and backwards
| compatibility.
|
| It also helps that Go has a pretty extensive stdlibs whereas
| JavaScript is really more like several distinct language
| ecosystems wrapped around a common specification. So what
| works on one JavaScript runtime might not even work on
| another.
| danenania wrote:
| > but that doesn't need to be that way
|
| It kind of does though. If you need to do something with
| security implications, reinventing the wheel is usually
| higher risk than using a popular dependency. So it's not like
| you can realistically avoid this issue. At least not without
| causing bigger problems.
|
| It's also not just a coincidence that Go apps have far fewer
| dependencies. The comprehensiveness of the std lib (along
| with officially maintained /x/ packages) means that you need
| fewer direct dependencies. And just as importantly for the
| overall size of the tree, all the dependencies that you do
| need themselves have fewer dependencies. This can easily mean
| an order of magnitude difference in total transitive
| dependencies for a significant project.
| dilap wrote:
| The very first Go code I ever wrote, way back in 2011, still
| compiles and runs perfectly. It's glorious.
| listeria wrote:
| package main import "fmt" func
| main() { fmt.Println("Hello world") }
| rollulus wrote:
| My few tiny steps in JS world were alienating in that sense:
| having a brand new install of all tools, doing a "npx create-
| react-app" and got greeted with "congrats, your app is
| initialised, it has 13 vulnerable dependencies".
| hombre_fatal wrote:
| Tbf those are development deps rather than production server
| deps, and the vuln will be something like "DOS possible if
| you let users craft their own regex string as input to
| lib.foo(re) in a server ctx" rather than "by using this in
| development to build your static js app, people get remote
| access to your dev machine."
| robertlagrant wrote:
| It is a bit silly then that it reports them as
| vulnerabilities by default.
| vdvsvwvwvwvwv wrote:
| Worse CRA goes from saviour to deprecated, "use nextjs or
| vite instead" in a blink. Meta should maintain it. Nextjs
| will probably morph again in the future so you hope investing
| in learning vite is the answer. JS has this way.
|
| Meanwhile Rails is so old it is thinking it needs to find a
| partner, settle down and buy a picket fenced house.
| mplewis wrote:
| Unfortunately, in Rails, your major breaking changes
| without a migration path come from the core team.
| veidelis wrote:
| create-react-app is not maintained AFAIK
| Quekid5 wrote:
| If JS apps are the standard you measure against you'll be happy
| with most things.
| Cthulhu_ wrote:
| Except 'enterprise' Java, the indirection there is insane if
| you're using e.g. Spring. I last worked with it five or so
| years ago and I had no idea what I was doing.
| spmurrayzzz wrote:
| I've had similar experiences, but I've noticed my Node.js
| applications which have few-to-no dependencies behave in the
| same way as my Go apps in that regard. I might get some
| deprecation logs from Node letting me know about future
| changes, but generally they do just work. The apps with a heavy
| dependency graph are a different story however.
|
| This is still a feather in Go's cap given the fact that the
| standard library offers so much out of the box. I don't find
| myself reaching for dependencies that often.
| Cthulhu_ wrote:
| I'm still stuck in JS world - it's difficult to get a Go job if
| it's not already your day job - and I hate it.
|
| Currently I'm adding a React Native component library to an NX
| monorepo where I want it to work with Storybook for which I
| need to add Expo but I can't just run the generator, I need to
| extract the relevant bits from a template project and cross my
| fingers it works.
|
| I long to go back to the simplicity of my Go project where I'd
| start my day by running `make watch` and it would just work.
| (mind you, it took me a while to find a file watcher that
| worked properly)
| tapirl wrote:
| Please note, currently, there are no tools to detect the new
| footguns created by the new semantics of 3-clause "for;;" loops:
| https://github.com/golang/go/issues/66156
|
| > The second step is to keep the Go versions in our projects
| current. Even though we don't use the latest and greatest
| language features, bumping the Go version gives us all security
| patches for discovered vulnerabilities.
|
| It is not always a good strategy to use the latest toolchain
| version. There are often some fresh bugs in it. From the security
| perspective, it is better to use the previous version, which is
| also still being maintained.
| arccy wrote:
| you're literally the only person who's making a mountain out of
| a molehill coming up with ever more convoluted code to "prove"
| the change was a bad thing.
| tapirl wrote:
| It indeed is. Please read
| https://go101.org/blog/2024-03-01-for-loop-semantic-
| changes-... and https://github.com/golang/go/issues/66156
| arccy wrote:
| a lot of words and yet no real world issues identified
| tapirl wrote:
| The problems are right there. But some people choose to
| turn a blind eye.
| candiddevmike wrote:
| .0 is for local development, .1+ is for production
| tapirl wrote:
| There might be some bugs in .2:
| https://github.com/golang/go/issues/70035
| superb_dev wrote:
| The examples in that ticket are convoluted, who would write
| code like that? Has this issue been spotted in the wild?
|
| I agree that there is some issue and a lint should probably
| warn you about these, but I doubt a lot of people will run into
| it.
| tapirl wrote:
| You should read that article carefully.
|
| Some Go core team members don't agree with you:
| https://github.com/golang/go/issues/66156
| 01119288523 wrote:
| https://docs.google.com/document/d/1Cb282k7rrSi4awr0HizuRciX...
| arp242 wrote:
| Good grief, you're still banging on about this. It's been well
| over a year. Literally no one agrees with you. That happens
| sometimes. But please, accept that and give it a rest already.
| tapirl wrote:
| Copernicus was in the same position.
|
| :)
| rollulus wrote:
| As the article also mentions: instead of checking if your program
| has a dependency on something that contains vulnerabilities,
| govulncheck checks if vulnerable code is actually reached. I find
| that so awesome. (And I know, someone is going to point out that
| hipster language foo does this too and better -- it's not the
| norm).
| lynx23 wrote:
| > hipster language
|
| Funny, I always considered Go a hipster language for Google
| fanboys.
| paulddraper wrote:
| You either die a hipster or live long enough to become
| mainstream.
| Cthulhu_ wrote:
| Go is a retro nostalgia language, taking programming
| languages back to basics, removing syntax highlighting and
| advanced concepts like exceptions and function shorthands
| because that's what it was like in the 70's.
| euroderf wrote:
| I'll give up my Go when you pry this PDP-11 emulation
| from my cold, dead hand.
| rob74 wrote:
| Pray tell, what is it about Go that stops editors from
| using syntax highlighting when displaying Go code?
|
| Seriously, IMHO Go is less "retro nostalgia" and more
| trying to stick to proven concepts (e.g. there was no
| test driven development in the 70s, and Go has
| testing/documentation/examples built into the language)
| while leaving out things like exceptions and inheritance
| that, while widespread, have significant disadvantages.
| silverliver wrote:
| Perhaps, but all I really care about is having a complied,
| strongly-typed language with a fully-featured modern stdlib
| and good cross-compilation support that includes wasm. If
| that comes with an automatic admission to the Google Fanboy
| Club, then sign me up.
|
| What other well-established languages do we have that meet
| this criteria? I know .net is a strong contender but do we
| have other options?
| vlovich123 wrote:
| Rust & Java also come to mind (yes, Java can be AOT
| compiled). Erlang too if you want more fearless concurrency
| if you're OK with JIT languages. There's lots of
| alternatives to Go in its space but it does have mindshare
| and there's nothing wrong with staying on the well trodden
| path even if it's full of if err != nil instead of sane
| error chaining built into the language.
| pjmlp wrote:
| Regarding Java, since early 2000 to be more precisely,
| although it required paying for commercial JDKs like
| Excelsior JET.
|
| Nowadays besides the more well known GraalVM, there is
| OpenJ9 and its cousin Android since version 5.
|
| PTC and Aicas remain as two well known commercial Java
| vendors, with AOT toolchains, alongside bare metal and
| real time GC support, although their focus is embedded
| deployments.
| mjevans wrote:
| Go / golang added https://pkg.go.dev/errors
|
| Which includes nested / stacked errors and helper
| functions for checking them.
|
| It doesn't implement error classes, but you can create a
| stacked chain of errors which achieves the same sort of
| 'Handle a classification of error' (anything which
| includes that class).
|
| Older libraries don't use these features, as far as I
| know. So it's sort of like the half-baked enumerate
| everything sort of generic functions that older stable
| versions (like on hacker rank) ship.
| vlovich123 wrote:
| I think you missed my complaint was that unlike more
| modern languages like Rust, Go has way too much
| boilerplate for error handling and not only does it not
| have error chaining via a `?` operator, it doesn't even
| force you to check the error meaning I'm sure there's
| plenty of missed error checks in production code leaving
| all sorts of vulnerabilities lying around. The package
| you linked in no way addresses what I wrote.
| mjevans wrote:
| I've been interested in learning more about Rust, but so
| far haven't had a project that seemed like it'd be worth
| learning a whole new language structure.
|
| So, I was responding to my _understanding_ of what you
| had written, which apparently didn't adequately explain
| what you sought to those who haven't seen the thing you
| were trying to reference.
|
| I do occasionally use a helper function in golang like
| 'nilOrPanic()' which if it's given an Error type that
| isn't nil causes a panic(); which isn't so useful outside
| of development or toy exercises.
| vlovich123 wrote:
| A language like Rust makes the Error and Option types
| first-class. It'll be a compiler warning (or error? Don't
| remember right now) if you ignore the return from a
| function that returns one of these. Go requires a
| separate linter and relies on uncaught variables. Minor
| distinction but important one because defaults matter.
|
| If you want to panic on error/option (i.e. you don't
| think it's going to happen), you add an exclamation mark
| after the error. If you want to unwrap but propagate the
| error effortlessly, add a question mark. This syntactic
| sugar is a pretty common ideas at this point not unique
| to Rust. What is a bit more unique is that Error and
| Option are sum types. This means that you can't just
| access the value without unwrapping them somehow and
| unwrapping requires you to either handle the error by a
| match or conditional statement, propagate (?), or panic
| (calling .unwrap() function). But you have to make that
| decision and can't ignore it so while you have to think
| about what you want the error handling to look like, you
| can't ever accidentally forget (& since mostly you
| forward, ? Makes things easy even if you need to bridge
| different error types).
| gwd wrote:
| > have error chaining via a `?` operator
|
| Although I do frequently find typing in the boiler plate
| of _every_ _single_ _error_ a bit of a faff, it does
| prompt me each time to really think "what if an error
| really happened here". I'm inclined to think that
| something like the ? operator makes it much easier to
| just toss in the ? and not consider the implications of
| an error.
|
| > even force you to check the error[,] meaning I'm sure
| there's plenty of missed error checks in production code
|
| Something the equivalent of "#[must_use]" would certainly
| be an additional aid, (as would const pointers).
|
| EDIT but one of the tools mentioned in the blog post,
| golangci-lint, will warn you of unchecked errors.
| vlovich123 wrote:
| You'd really like the C community. They like to say
| things like "although I do find the setfaults annoying,
| it really makes me think carefully about memory ownership
| and layout". The problem is that if you don't have a
| consistent way to accomplish a task correctly, something
| like errors that could happen a nearly every function
| call, then you're very likely to make a mistake. Coupled
| with that, most people ignore testing for error
| conditions or simulating errors, so the error handling
| has a very high likelihood of having bugs.
| nitely wrote:
| They are going to add boilerplate free error handling
| sooner or later. There are many proposals for "Go 2"
| already.
| euroderf wrote:
| The %w printf verb. It yields much more than a stack
| dump. Get meaningful error annotations from every step
| back up the callstack.
| neonsunset wrote:
| Erlang is interpreted. It is in the same class of
| performance as Python and Ruby. If you want a relatively
| high-level and performant alternative with great
| concurrency support your options are C#/F# (you are
| likely find the tooling pleasant) and perhaps JVM
| languages once they adopt structured concurrency (but you
| are likely to find the tooling less pleasant).
|
| Graal Native Image support is very niche and does not
| provide the same level of experience as .NET's NativeAOT
| nor has tricks up its sleeve like static linking with
| native libraries.
| shakna wrote:
| Really...? In my experience, whilst Erlang is slower than
| most AOT languages, its an order of magnitude faster than
| Python or Ruby. Most benchmarks I've seen also back that
| up.
| neonsunset wrote:
| Unlike Python it scales with cores perfectly, which makes
| sense given that's what BEAM is designed for, but the
| baseline cost of operations is in the same group.
|
| https://benchmarksgame-
| team.pages.debian.net/benchmarksgame/...
|
| It's a bytecode-interpreted language. If it were JIT and
| statically typed we would have seen drastically different
| results. Also JIT output being slower than static
| compilation is a myth. When compilation happens does not
| dictate the kind machine code the compiler can produce
| (mostly, compiler throughput and JIT-time optimizations
| do influence this, but are not a strict limitation).
| jhogberg wrote:
| Erlang is JIT compiled since 2021.
|
| Grandparent is also correct in that it tends to be faster
| than Python et al. If we have a deeper look at the
| benchmarks [1][2], as long as there is no significant
| amount of bignum arithmetic (where both call C code) or
| standard IO involved [3] it's consistently faster than
| Python, and often by a large margin.
|
| [1]: https://benchmarksgame-
| team.pages.debian.net/benchmarksgame/...
|
| [2]: https://benchmarksgame-
| team.pages.debian.net/benchmarksgame/...
|
| [3]: Standard IO goes through several indirections to
| make it work with remote REPLs; other forms of IO do not
| suffer from this.
| igouy wrote:
| For your convenience:
|
| https://benchmarksgame-
| team.pages.debian.net/benchmarksgame/...
|
| > no significant amount of bignum arithmetic
|
| There is none shown in the charts. There is none shown
| elsewhere apart from where aribitrary precision
| arithmetic is shown explicitly: pi-digits.
| mike_hearn wrote:
| You can statically link with native libraries if you have
| static versions of them in GraalVM:
|
| https://www.graalvm.org/latest/reference-manual/native-
| image...
| neonsunset wrote:
| Does it let you bring your own .a/.lib and statically
| link it into the final product? In .NET you can pass
| DirectPInvoke + NativeLibrary build properties and link
| with anything provided the imports don't conflict
| (because at the final step it's the system-provided
| linker that statically links together the components
| nativeaot binaries comprise of, so you are effectively
| just adding another compilation object).
|
| For example, I can take a mimalloc.lib/.a, link it into
| the binary and the pinvokes into its mi_malloc,
| mi_realloc, etc. will all be just direct calls + branch
| checks for GC poll if applicable (well, you need to
| suppress gc frame transition but it's still about three
| steps in total). It will be just a single static bundle
| at the end.
|
| I know that conceptually GraalVM Native Image and
| NativeAOT are similar tools, but they mostly seem that
| way at a distance and at closer inspection they only
| partially overlap, much like C# and Java themselves do.
| mike_hearn wrote:
| You can yes although the docs don't make that obvious.
|
| https://www.blog.akhil.cc/static-jni
|
| You can also use the FFI to avoid JNI.
|
| I tend to feel that static linking is overrated. The
| moment you want easy upgrades of your app you need extra
| infrastructure anyway, and every tool for distributing
| such programs can handle directories as well as files.
| neonsunset wrote:
| > I tend to feel that static linking is overrated.
|
| I agree. My response was just meant to indicate that
| NativeAOT has comparatively more effort and focus in .NET
| ecosystem than GraalVM's Native Image in JVM's, and as a
| result is an option that is easier to opt into where it
| makes sense. There's an intention to make it play as
| nicely as possible with most common scenarios and ideally
| not require user input when enabling it even with
| difficult to analyze reflection scenarios.
| igouy wrote:
| > Erlang is interpreted.
|
| https://www.erlang.org/blog/the-road-to-the-jit/
| neonsunset wrote:
| Thanks! I was briefly aware that BEAM has JIT capability
| but performance numbers usually put it next to other
| languages with interpreted bytecode so I assumed it was
| either not enabled or used in some specific scenarios. I
| should update my knowledge.
| fmbb wrote:
| Erlang is not strongly typed and cross compiling apps
| with native dependencies is not straightforward. Running
| it on WASM is not common (is it possible?). It does not
| have a comprehensive standard library like Go.
|
| It is compiled though.
| vlovich123 wrote:
| Erlang is most definitely strongly typed [1]. Perhaps you
| confused static & dynamic typing? Easy mistake to make.
| Similarly, from what I could find Erlang is typically run
| on BEAM [2] which is an interpreter virtual machine that
| executes BEAM byte code (with a JIT option).
|
| Since I'm not an expert on either language, here's my
| take of how ChatGPT summarizes Erlang vs Go on various
| options.
|
| Go's standard library is primitives driven for general
| purpose programming while Erlang's is purpose driven for
| distributed programming. So it depends on what you mean
| by "comprehensive". For example, out of the box Erlang
| provides an environment for writing correct, robust
| distributed programs. If comprehensive means having a
| bunch of knives & start juggling that's a different use
| case.
|
| [1] https://learnyousomeerlang.com/types-or-lack-
| thereof#:~:text....
|
| [2] https://www.erlang.org/blog/beam-compiler-history/
| johnisgood wrote:
| I have nothing against "if err != nil", in fact, I like
| it.
|
| As for a replacement of Go, I would have to say Erlang or
| Elixir. I use Go code for non-serious distributed
| software, and Erlang and Elixir for more serious ones.
| That is not to say that Go cannot be used for serious
| stuff though.
| ozgrakkurt wrote:
| Zig is pretty good
| Cthulhu_ wrote:
| It kinda is if you're thinking about the manual-coffee-
| grinder-french-press hipster who eschews automatic coffee
| makers. Rob Pike doesn't believe in syntax highlighting and
| to date the Go website / interactive editor doesn't have any.
| "When I was a child, I used to speak like a child, think like
| a child, reason like a child; when I became a man, I did away
| with childish things."
|
| Anyway, that's fine, I like Go and I like grinding coffee
| manually on occasion.
| lynx23 wrote:
| Funny, I have a similar analogy when it comes to mice:
| Small children lacking verbal communication skills can only
| point at things, which is the equivalnet of using
| a"pointing device". When they grow up, they learn to speak
| meaningful sentences to express themselves. Which is
| equvalent to learning to use the command line...
| timeon wrote:
| > french-press
|
| How is that hipster? Did you mean aero-press?
| zelphirkalt wrote:
| If the code cannot be reached, what is the point of having it
| as a dependency?
|
| Does it know which part of a dependency has a vulnerability and
| check, if the execution reaches _that_ part? Then it would make
| sense.
| chucky_z wrote:
| My understanding is that the primary goal is to determine
| that if a program is pulling in a dependency, and only using
| a small part of it, to determine if that part is vulnerable
| or not. This allows a program owner to know if they need to
| do an emergency version bump in the face of a CVE or
| something like that. For some businesses doing emergency
| deployments is a massive deal.
| FiloSottile wrote:
| > Does it know which part of a dependency has a vulnerability
| and check, if the execution reaches _that_ part?
|
| Yes, govulncheck does symbol-level reachability static
| analysis, and the vulndb is manually annotated with affected
| symbols for each vulnerability.
|
| (So glad to see a comment about this at the top, I have
| sometimes feared we made a mistake in designing a low-noise
| vulnerability scanner, because I've often seen complaints
| that "it doesn't work" because it doesn't show as many
| vulnerabilities as its more popular, less accurate
| alternatives.)
| dakiol wrote:
| Go is nice, but the recent trend of using generics for many stuff
| is making harder and harder to keep Go code readable imho. See an
| example here https://eli.thegreenplace.net/2024/ranging-over-
| functions-in...
|
| I'm not saying it's hard to read, but it's harder than previous
| Go code that used little or no generics at all.
| mervz wrote:
| Meanwhile, error handling still can't get any sort of syntactic
| sugar
| divan wrote:
| As with real sugar, we humans don't have sensors that would
| tell us when there's "too much sugar".
| randomdata wrote:
| That's because nobody has yet solved the side effect problem
| of the sugar.
|
| All the proposals that have ever been given have ultimately
| boiled down to essentially `return err`, which, while
| suitable for meme comments on an internet forum, cannot be
| used in a real production application for many obvious (and
| some not immediately obvious) reasons.
|
| At least under the direction of rsc (the new leadership is
| still settling into the role so that is less clear), the will
| to add such sugar was there if a good solution was found. But
| the solution has yet to be found.
| jimbokun wrote:
| I don't know what the syntax should look like.
|
| But the most common pattern is a sequence of calls to
| functions that return an optional error plus the happy path
| value, followed by a short circuiting check of the error,
| followed by a call to another function with the happy path
| value as an argument. It's very common to have a chain of
| these kinds of calls making up the body of a function.
|
| It seems like "return err" is very useful for this pattern,
| if I understand you correctly. A function returning the
| error from the first call it makes that fails, or the happy
| path value if all the calls succeed. Seems like it should
| be possible to bake that pattern into the language, but its
| tricky doing it a way that doesn't obfuscate the underlying
| semantics, which is very important to many Go developers.
| randomdata wrote:
| _> I don 't know what the syntax should look like._
|
| I'm not sure the syntax is all that significant. There
| have been numerous proposals, but the syntax was never
| the reason for rejection. It is that the entire concept
| is unusable in the state that it is understood.
|
| That's not to say the problems can't be solved, but
| nobody has yet.
|
| _> It 's very common to have a chain of these kinds of
| calls making up the body of a function._
|
| Yes, like in Rust, for example. But it also has defined
| traits and other features on top of the chaining to deal
| with the same problems Go would suffer from it had such
| syntax. Theoretically Go could introduce the same, but it
| remains unclear how to do that in a way that makes sense
| in the Go language.
|
| Again, there is probably a solution out there, but nobody
| has come up with it yet. Surprisingly, these kind of
| things aren't sent down from the heavens by a magical
| deity. It takes human effort, which isn't there because
| they are busy ranting on HN.
|
| _> It seems like "return err" is very useful for this
| pattern_
|
| Where would you find it useful (memes aside)?
| Cthulhu_ wrote:
| There were many proposals but none of them were an actual
| improvement over the simplicity and straightforwardness of
| the existing. `if (err != nil) {` is simple, short and to the
| point, and adding language features for only this use case
| wasn't deemed worth the cost in the end.
| consteval wrote:
| The problem with this syntax is that it's not required
| anywhere, any time. It also makes the logic extraordinarily
| complex for what it is. You can very quickly get into
| branch hell. I hate to say this, but often the control flow
| is much simpler and easier to understand with exceptions.
| The "if" works fine for one level, but any deeper than that
| and it's no fun.
| timmytokyo wrote:
| Your example of go code that's harder to read is iterators, and
| I agree with you. There's no denying that code like this places
| a high cognitive load on the reader: func (al
| *AssocList[K, V]) All() iter.Seq2[K, V] { return
| func(yield func(K, V) bool) { for _, p := range
| al.lst { if !yield(p.key, p.value) {
| return } } } }
|
| But the code that actually _uses_ iterators is in my opinion
| more readable than its non-generic counterpart. So it 's really
| a question of how often you're expected to write (or read)
| iterators. And I don't expect that most programmers will be
| writing (or reading) iterators that often.
| chamomeal wrote:
| I often feel this way about heavy use of typescript generics.
| The more you lean into the crazy (and awesome) world of
| generics, the more inscrutable the code becomes to anybody
| who isn't a generics wiz. It's really like an extra language
| stacked on top of JS. I'll come back to code I wrote a year
| ago, and it'll take me a full day to figure out the types.
|
| But the simplicity of using a library or set of functions
| that have really nice generics? So awesome. The intellisense
| and type errors alone can almost be a decent form of
| documentation.
|
| The source becomes hard and weird to change, but the end
| result is a very nice DX
| timmytokyo wrote:
| On further reflection, I think what makes this example
| particularly difficult to understand is not so much its use
| of generics, but the way it uses functions. It's a function
| that returns a function that takes another function as an
| argument. The generic [K,V] type arguments are actually
| pretty straightforward.
| Cthulhu_ wrote:
| I'll admit I've only ever done one serious Go project but
| I've thankfully never felt a need to use generics, before
| generics there were the builtin list and map types that were
| themselves generics.
| Arainach wrote:
| I'm curious about your objection to the proposal. Sure,
| generics mean that libraries need a bit more syntax - that's
| true in all languages - but the actual consumption of the
| AssociationList type here is clean and readable.
|
| Most types don't need to be generics. Containers do, and I
| prefer a bit of generics syntax to copy/pasting the container
| ten times for ten types.
| JyB wrote:
| You spend more time reading code that writing it. Optimising
| for the later is a mistake. I guess the noticeable pushback
| against including generics was not unwarranted, people are
| just now starting to see the ripple effects we were warned
| about.
| consteval wrote:
| Generics are, IMO, necessary for even a semi-modern
| language. Okay, you don't need a turing complete templating
| sublanguage like C++, but you do need at least a way to
| take generic functions and create generic containers.
|
| In application code you will almost never write generics.
| To me, it's always been a non-issue.
| Arainach wrote:
| It's optimized so that it's easy to read the code that you
| read all the time: code iterating through the containers.
|
| It is dramatically less common to read through the
| implementation of containers.
| zapnuk wrote:
| Writing custom iterators always looked bad and overly
| complicated.
|
| If it's not essentials I'd rather not allow code like this in
| my codebase and use some other solution that is more readable.
| imiric wrote:
| Indeed. I still try to avoid generics whenever possible, and
| prefer a solution that doesn't use them. Thankfully, there
| aren't many scenarios where they're absolutely indispensable.
| kubb wrote:
| Write a generic instantiator that scans your codebase for
| generic usage, and makes one copy of a generic for every type
| it's used with. Then you can remove all the generics and go
| back to copy and paste.
| wepple wrote:
| Don't forget about capslock: https://github.com/google/capslock
|
| Assess your 3P modules for dangerous capabilities
| rollulus wrote:
| Nice, thanks a lot for that!
| K0nserv wrote:
| Somewhat related, I learned a surprising fact recently: Go is not
| actually memory safe. In particular because atomicity is only
| guaranteed for word size values, double word values(interface
| pointers, slices) can introduce memory unsafety in the presence
| of concurrency[0].
|
| It's one of those things that feels obvious when you see it.
|
| 0: https://blog.stalkr.net/2015/04/golang-data-races-to-
| break-m...
| tptacek wrote:
| It's easy to demonstrate contrived abuses of Go concurrency
| that break memory safety, but despite the enormous popularity
| of the language, actual shipping vulnerabilities --- _mistakes_
| in concurrency, not deliberately-engineered pathological cases,
| that yield attacker-controlled control over memory --- are
| basically nonexistent (I can 't think of a single one; there
| must be one somewhere!).
|
| Basically this is about as credible an argument as claiming
| that Rust isn't memory safe because its libraries have so much
| `unsafe` code. And that claim: not super credible.
|
| Basically, the takeaway in both cases is that it's not safe to
| allow an attacker to _write code for you_ in the language. But
| everybody assumes that 's the case anyways, because it's the
| case with virtually every other language (with one very
| notable, fraught, and well-understood exception), too.
| shitter wrote:
| What is that exception?
| tptacek wrote:
| Browser Javascript.
| pansa2 wrote:
| Also embeddable scripting languages like Lua
| ViewTrick1002 wrote:
| Instead there's a whole host of subtle footguns which while
| not leading to true memory unsafety will lead to complete
| junk data.
|
| https://www.uber.com/en-SE/blog/data-race-patterns-in-go/
| tptacek wrote:
| I don't care to litigate program correctness and
| ergonomics. Those are extremely subjective, and I don't
| feel like I ever get anywhere useful in those kinds of
| conversations. The most popular backend programming
| language in the industry is almost certainly Python, and it
| barely even has types. I still wouldn't dunk on it.
|
| This thread is about a much narrower question, which is
| code security. There, I feel like I'm on much firmer ground
| drawing and defending conclusions, and my conclusion is
| that there isn't a mainstream general-purpose modern
| language that is meaningfully more secure than Go (or than
| Rust, or than Python, etc).
| atomic128 wrote:
| Here is code to circumvent Go's memory safety without importing
| unsafe.
|
| get() reads a byte at an arbitrary address and set() writes a
| byte at an arbitrary address.
|
| This is excerpted from BUGFIX 66 ("Hack This Site"):
| func racer() { var ( ptr1 *uintptr
| ptr2 *byte race any done =
| make(chan struct{}) ) put := func(x any) {
| for { select { case <-done:
| return default: race =
| x } } } go
| put(ptr1) go put(&ptr2) for {
| var ok bool ptr1, ok = race.(*uintptr)
| if ok && ptr1 != nil { close(done)
| break } } get := func(addr
| uintptr) byte { *ptr1 = addr return
| *ptr2 } set := func(addr uintptr, to byte)
| { *ptr1 = addr *ptr2 = to
| } if get(0xdeadbeef) == 111 {
| set(0xbaaaaaad, 222) } }
| tptacek wrote:
| "Without importing unsafe" is doing a lot of work for
| examples like this.
| atomic128 wrote:
| This comes from a webpage where the challenge is to
| compromise the site, despite the fact that Go imports are
| disallowed (including unsafe). It's a puzzle game.
|
| To clarify, I think Go is magnificent and I use it for
| everything. The racer() code is just a curiosity.
| tptacek wrote:
| Right, it's a cool trick. It's just not material to real
| threat models, which is what people imply when they say
| "Go isn't memory safe".
| atomic128 wrote:
| I agree.
| tialaramex wrote:
| The fact Go has UB under data races has practical
| implications for sufficiently complex concurrent
| software. If you can induce a race on a non-trivial
| object, that's UB instantly - you can probably blow up
| the Go runtime and all bets are off.
|
| I would not characterise this fact, which is a design
| choice in Go, as similar to say a Rust soundness bug,
| which will sooner or later just get fixed. They aren't
| going to somehow magically fix this problem in Go, it's
| part of the design.
| tptacek wrote:
| My point has nothing to do with whether the language will
| achieve "soundness". It's that this is behavior that has
| not over the last 15 years produced exploitable
| vulnerabilities, despite extremely high incentives for
| those vulnerabilities to be unearthed.
| pkolaczk wrote:
| You don't need to blow up the runtime to cause a
| vulnerability due to a data race in Go:
|
| https://security.snyk.io/vuln/SNYK-
| DEBIAN13-GOLANGGITHUBGORE...
| arp242 wrote:
| That's a completely different type of vulnerability than
| the UB that's being talked about.
|
| > The call to sync.Pool.Get will then return a
| bytes.Buffer that hasn't had bytes.Buffer.Reset called on
| it. This dirty buffer will contain the HTTP request body
| from an unrelated request.
|
| This is just a good ol' logic error, that just so happens
| to also be a race.
| unscaled wrote:
| These type of UB bugs in Go are a bit of a red herring,
| since most race conditions arise from improper use of
| shared mutability, and would still be a problem even in
| the presence of full memory safety, for instance:
|
| https://github.com/golang/go/issues/37669
|
| https://github.com/golang/go/issues/48340
|
| These types of race conditions cannot happen in Rust. Not
| because Rust does not have UB, but because Rust does not
| allow multiple writable pointers ("mutable borrows") to
| the same memory region. If you want shared AND mutable
| access to memory, you must use a thread-safe construct
| such as Mutex or Cell -- or drop into unsafe code.
|
| Rust does not prevent all types of errors of course.
| Dirty buffer reuse (as in the GP example) is still
| possible in Rust. You could still have situations where a
| buffer is returned to a pool without resetting it. But
| this could only be a pure logic error where you've forgot
| to reset the buffer and it would occur _consistently_ and
| thus would be easy to reproduce and debug. In addition,
| with idiomatic Rust, you could enforce proper buffer
| cleanup in Rust by wrapping the Buffer with a type that
| implements Drop.
|
| More specifically, the vulnerability mentioned in GP is
| not possible in Rust. The description is a bit
| misleading, but the issue was not that the buffer was
| returned to the pool without being reset, but rather that
| the same buffer was returned to the pool TWICE under
| certain conditions, due to a data race. This is not
| possible in Rust. You cannot put the same owned buffer
| twice in a pool, due to Rust's move semantics (affine
| types). And if we want to be completely honest, you'd
| probably won't need to pool buffers in Rust to begin
| with, since you don't need to avoid garbage collection
| (there is none). In most cases, malloc is going to work
| good enough as your "pool".
|
| We have a serious problem as an industry, where there is
| a popular conception of memory safety and type safety as
| a binary property: a language is either safe or unsafe,
| either sound or unsound. But it's more of a spectrum, and
| not even a contiguous one at that. This comments thread
| is split between people who say that large size atomicity
| UB is not a major issue in practice and people willing to
| completely rule off Go's memory safety based on that. But
| we could just say Go sits near the safe end of the
| spectrum of memory safety -- it certainly does far better
| than C. My security concerns with Go, after nearly 9
| years of using are mostly about race conditions, memory
| leaks and lack of better mechanisms to enforce type
| safety (such as sum types and affine types).
| tptacek wrote:
| Given the total lack of empirical evidence (that is:
| language-specific vulnerabilities in the myriad large
| high-profile Go projects run all over the Internet;
| instances of bug classes not found in other mainstream
| memory-safe languages --- "memory-safe" here just to
| factor C/C++ out of that set) for those security
| concerns, why do you prioritize them?
|
| We of course continue to find SQLI, authz, SSRF,
| metacharacter parsing and cryptography vulnerabilities in
| Go codebases, the same way we do in literally every
| general-purpose programming language; what the the
| vulnerabilities we actually see, over 15 years of
| explosive growth in use, that are distinctive to Go? It's
| been 4 years since I was a full-time software security
| person, but I keep up, and did a lot of work in Go before
| then, and I'm not aware of anything beyond "if you skip
| the ,ok on a type conversion you might panic something".
| ngrilly wrote:
| I must say that I really appreciate your patience in
| addressing these comments. If the possibility of race
| conditions leading to UB and the lack of sum types in Go
| are so bad for security, then it shouldn't be difficult
| to observe exploitable vulnerabilities in real-world Go
| code bases.
| dfawcus wrote:
| In a specific scenario, I have made use of interface
| values, and type switches as a form of "tagged union" /
| "sum type".
|
| All it requires is that the module defining the
| interface, and structs implementing it, are in the same
| package, and the interface requires a private method.
|
| I used that to have a message type, passing instances of
| it over a channel of that interface, and demuxing based
| on a type switch of the message.
|
| One could use a similar scheme for return values from
| functions, just that for the simple error / no-error case
| it would not be idiomatic, however that should not
| prevent one from doing so if desired.
| ngrilly wrote:
| Yes, that's a possible pattern to emulate a sum type in
| Go.
| arp242 wrote:
| I think these are "if a tree falls in a forest and no one
| is around to hear it, does it make a sound?"-type
| musings.
|
| Whatever the case, it doesn't really affect anyone and it
| doesn't really matter.
| pkolaczk wrote:
| It's a matter of time. Spectre / meltdown were also
| considered extremely hard to use in practice. Yet they
| are considered vulnerabilities.
|
| In Golang case the data race window to corrupt memory is
| extremely narrow, so it makes it very hard to trigger it.
| That together with Go being still quite a niche language
| results in the fact we see no exploits... yet.
| arp242 wrote:
| Even if some sort of security bug is discovered tomorrow,
| then we're talking about one issue every 15 years or so.
| Whoop die doo. That barely even registers in the noise.
|
| That it "may" lead to a problem and that it's not "sound"
| is basically just meaningless.
| pkolaczk wrote:
| How many spectre / meltdown related vulnerabilities were
| detected between 1990 and 2010? Zero. So those chip
| vendors must be paranoid they patch them - were talking
| about one issue per 20 years xD Similarly how many
| hashmap collision attacks existed prior to 2010? Zero,
| but once people learned they are not just a theoretical
| problem, suddenly plenty of vulnerabilities were found.
|
| Seriously, it doesn't work like that. It's not linear.
| During the first half of those 15 years almost no one
| heard about Go, and forget about using it in critical
| systems where vulnerabilities would matter. Even at
| Google it was (still is?) very niche compared to Java,
| Python and C++ and is used mostly for userspace clis and
| orchestration, not the core stuff. There is simply very
| little incentive to attack systems written Go, when there
| exist 100x more less secure networked systems written in
| C or C++.
|
| Considering this memory unsafety thing in Go is
| fortunately very hard to exploit, there is no doubt why
| attackers don't target this weakness and it has been so
| far only a technical curiosity. Also data races in Go are
| easy to make and can lead to vulnerabilities in a much
| more direct way, without corrupting the heap. I bet those
| are exploited first (and there exist CVEs caused by races
| in Go).
| vacuity wrote:
| I note that of the Spectre/Meltdown and similar hardware
| vulns, even the hard-to-swallow kinds of mitigations for
| Spectre primarily prevent user-to-kernel hijacking only,
| which is the most important single property but doesn't
| cover inter-process hijacking. We can more or less patch
| these vulns completely, but there is a (huge) performance
| penalty to be weighed as a drawback. I do not know enough
| to say whether the Go data race bugs are an acceptable
| risk. Although, not everyone may accept it, namely if it
| strikes them just once.
| adonovan wrote:
| > They aren't going to somehow magically fix this problem
| in Go, it's part of the design.
|
| I wouldn't be entirely pessimistic.
|
| Russ's post https://research.swtch.com/gorace mentions a
| conservative representation for Go's data structures
| (essentially: more indirection) that would make it
| possible to implement them in a way that was robust to
| races, at an obvious large performance cost.
|
| More recently others have been investigating the
| possibility of using 128-bit atomic writes (on ARM and
| x86) to reduce the cost. Go's strings and interfaces are
| both 2-word structures. Slices are three words but by
| changing the field order atomicity can be achieved with
| 2-word writes. Of course it would break a lot of code
| that assumes the representation or the ABI.
| wbl wrote:
| That code is usually internal.
| kaba0 wrote:
| I mean, C is also memory safe when run within valgrind..
| at an obvious large performance cost.
| tialaramex wrote:
| To make this work you need to re-define "memory safety"
| to the point where it loses any value in a discussion
| about programming
|
| Valgrind has no way to detect trivial global or local
| array bounds misses so long as they don't stray out of
| the defined memory. It can't spot this because the
| resulting executable (the only thing Valgrind sees) is
| not doing anything that's prohibited - it's _nonsense_
| because you used a non-MSL, but the actual executable has
| some defined behaviour.
| Yoric wrote:
| Well, time will tell. As Go usage increases, it becomes a
| more tempting target, which means that more malicious
| third-parties will start poring over the code of the std
| library and the frameworks looking exactly for this kind
| of vulnerability.
|
| The same goes for Rust, Swift or Zig, of course.
| kaba0 wrote:
| How is it not material? You only need to accidentally
| write and read a map at the same time in language that is
| supposedly for concurrency (which is why not the same as
| parallelism, in its case it does largely correlate).
|
| This is a ridiculous design issue with big ramifications.
| ronsor wrote:
| You shouldn't be modifying any variable concurrently without a
| mutex. The only exception to this is if the variable is (1)
| less than or equal to the CPU word size; (2) is at a CPU word
| size aligned address; and (3) atomic memory access functions
| are used to read and write the variable.
| saghm wrote:
| Memory safety as long as you don't violate certain rules is
| what C and C++ also have. The problem is that programmers
| make mistakes because we're all human.
| tptacek wrote:
| No, the "mistakes" we talk about with C/C++ are so common
| that it's hard to think of a major C/C++ project not to
| have them, and the "mistakes" we're talking about with Go
| or "unsafe" Rust are contrivances built to demonstrate
| things an actively malicious programmer could do. Equating
| the two is, obviously, a sleight of hand.
| klabb3 wrote:
| To add to this: the built in go race detector is very
| good at catching data races. It's a runtime, but I've
| never had a race that couldn't be reproduced in the race
| detector trivially.
|
| But yes, in theory Go has a memory safety problem because
| of it. In practice though, it's that people don't use the
| race detector, which is ridiculously easy to do.
| tptacek wrote:
| Ordinary non-race-checked Go code is memory-safe in the
| sense that we mean it in the field of software security.
| oasisaimlessly wrote:
| "We"? Do you have a mouse in your pocket?
| kaba0 wrote:
| It's only on Go, leave Rust out of it. Rust's safe part
| is entirely memory safe. Unsafe is the escape hatch,
| which pretty much every language has in the form of FFI.
| tptacek wrote:
| That's not true: idiomatic Rust projects use `unsafe`
| much more liberally than other languages use FFI, because
| of shared xor mutable. That's not a knock on Rust. I
| couldn't be less interested in Rust vs. Go; I use both
| and would use them both in different situations.
| vacuity wrote:
| I doubt that "idiomatic Rust projects use unsafe
| liberally". It is a more liberal construct, perhaps, but
| IMO actual usage is usually reasonable. Unless you mean
| the standard libary's use of unsafe?
| tptacek wrote:
| I'm not saying it isn't reasonable, just that it serves a
| different role in Rust than unsafe/JNI would in Java:
| there are things you naturally want to express in Rust,
| not having anything directly to do with interfacing with
| external code, that want `unsafe` in order to (carefully)
| bypass shared xor mutable.
| throwaway894345 wrote:
| > Memory safety as long as you don't violate certain rules
| is what C and C++ also have
|
| There are numbers between 0% and 100%, thus it's possible
| that Go can be less than 100% memory safe _and_ still far
| safer than C or C++.
| tptacek wrote:
| "100% memory safe" is mostly not a thing; it's not a
| concept that gets quantified this way. The closest thing
| I think you get to a compromised notion of safety that's
| actually noteworthy is Zig's allocator behavior (which
| can in ordinary code theoretically still produce UAF
| bugs, and UAF bugs --- lifecycle bugs more generally ---
| are the most pernicious kind of memory safety
| vulnerability). Most practitioners would still call Zig
| "memory safe". You can see how much bigger a deal that
| behavior is than the one we're talking about.
|
| I think the basic takeaway here is not to tie yourself up
| in nots trying to quantify memory safety. There's a
| reason Prossimo calls Go memory safe (not "mostly memory
| safe"), along with Rust, C#, Java, Swift, Python, and
| JavaScript. Ordinary code written in any of these
| languages is just not going to have exploitable memory
| corruption vulnerabilities. Other vulnerabilities, yes!
| ngrilly wrote:
| I'm curious to hear why most practitioners would call
| Zig's allocators memory safe? Do you mean the
| std.heap.GeneralPurposeAllocator which protects against
| use-after-free when building in debug and release_safe
| mode (not release_fast)?
| kiitos wrote:
| Even when a value satisfies these architecture-dependent
| requirements, the language still does not guarantee atomicity
| of concurrent reads/writes, and programs which rely on that
| assumption are buggy.
| Yoric wrote:
| Isn't this part of the Go memory model
| (https://go.dev/ref/mem)?
| kaba0 wrote:
| Logic bugs != memory safety bugs.
|
| E.g. in java you can mess up your logic with data races,
| but the racing itself is safe and can never cause the VM to
| enter an invalid state.
| Thaxll wrote:
| Go is memory safe by modern standard.
|
| If I show you a UB in Rust without the use of unsafe does it
| means Rust is unsafe?
| Yoric wrote:
| What does that mean?
|
| If I follow correctly, assuming that there are no bugs in the
| compilers/interpreters, Go is less memory-safe than Java, C#,
| Python (with GIL), JavaScript or Rust. The only languages
| that are less memory safe would be C, C++ or Zig.
| K0nserv wrote:
| I believe UB without unsafe is considered a bug by the Rust
| language team.
|
| I should've said in my original comment, but I don't mean to
| dunk on Go. In practice the issues illustrated in the blog
| post I linked seem unlikely to cause problems in practice,
| they are interesting nevertheless.
| kaba0 wrote:
| That would mean it, yes. And yeah there is a bug in rust's
| borrow checker which can trigger something like that for some
| very special, "no human will ever write code like that" case.
| But this is an implementation detail for a semantically
| memory safe language, while in go's case having UB is a
| language primitive here.
| Thaxll wrote:
| The trigger for Go is exactly "no human will ever write
| code like that".
| pclmulqdq wrote:
| This is also true of most other "safe" languages.
| JyB wrote:
| That's like... the first thing you learn about the language and
| the primitives the language was built upon. Yes mutex are a
| thing.
| K0nserv wrote:
| Sure race conditions in general, but the subtlety that it can
| cause memory unsafety, not something I recall being
| mentioned.
| jjcm wrote:
| Great tips in here - I was not aware of `go vet` nor `go test
| -race`.
|
| FWIW, while go is not memory safe, I do find that it's much
| easier to be safe in go than it is in other languages. Its
| verboseness lends to a very clear understanding of what's
| happening in any given function. I absolutely hated this at the
| start, but now ~3 years into maintaining a go codebase, I find it
| quite nice both for debugging as well as editing old code. I know
| exactly what each function does, and what the structure of data
| is in any given context.
|
| Another interesting side effect is that AI tools seem to work
| amazingly well with golang, given how context is often local to
| the function.
| tptacek wrote:
| Go is memory-safe. It's not the definition of "memory-safe
| language" that it's impossible to write memory-unsafe code,
| only that ordinary code is memory-safe by default.
| thinkharderdev wrote:
| > ordinary code is memory-safe by default
|
| What does that mean? What constitutes "ordinary"? I'm not
| sure there is any official definition of memory safety, but I
| would consider it to mean that aside from code that is
| explicitly marked as unsafe it is impossible to write code
| that has undefined behavior.
| tptacek wrote:
| An example of extraordinary code would be code that
| interfaces with and/or pulls in non-memory-safe legacy C
| code.
|
| Another example would be code specifically contrived to
| highlight a soundness problem in the language.
|
| I used the term "extraordinary" to avoid exactly this kind
| of bickering over corner cases that aren't relevant to day-
| to-day software development (or at least, not in ways that
| aren't immediately evident when they come up.)
| thinkharderdev wrote:
| > An example of extraordinary code would be code that
| interfaces with and/or pulls in non-memory-safe legacy C
| code.
|
| That's my point though. Of course calling non-memory safe
| native code over FFI can lead to memory-safety problems
| in any language. Likewise using the "unsafe" subset that
| basically every language has. But none of that is
| required in Go. It is only required that you mutate
| shared state from different threads, which is something
| that I would imagine happens in a lot of Go code
| codebases since it is an extremely easy mistake to make.
|
| To be clear I think:
|
| 1. Go is mostly a memory safe language because it does in
| fact prevent the most common memory safety issues in
| C/C++ (UAF, buffer overflows, etc)
|
| 2. It is LESS memory safe than other modern memory-sage
| languages (Rust, Java, C#, Python, etc....)
|
| 3. The memory safety issues in Go are very difficult to
| exploit in code that is not specifically crafted to
| surface them
| kccqzy wrote:
| Good definition. I've seen Go beginners trying to append to
| a slice from multiple goroutines. It works as well as
| calling push_back on the same vector from multiple threads
| in C++. It can easily corrupt GC state and lead to
| segfaults. The beginner didn't use any advanced trickery or
| the unsafe package. Therefore Go is not a memory safe
| language.
| sshine wrote:
| > _Therefore Go is not a memory safe language._
|
| Interesting.
|
| To quote the NSA [1], " _Some examples of memory safe
| languages are Python, Java, C#, Go, Delphi /Object
| Pascal, Swift, Ruby, Rust, and Ada. Memory safe languages
| provide differing degrees of memory usage protections, so
| available code hardening defenses, such as compiler
| options, tool analysis, and operating system
| configurations, should be used for their protections as
| well._"
|
| The narrow definition of memory safety here is:
|
| Go has garbage collection, so you won't have memory leaks
| or use-after-free.
|
| Go is powerful enough that beginners can cause segfaults
| by accidentally abusing internals, okay.
|
| I'm not sure this is a very redeeming property of Go:
| Being able to crash the GC, without the flexibility of
| manual memory management.
|
| But I'm not sure I'd categorize it as "not memory safe"
| for the same reason C/C++ aren't (a trade-off).
|
| Because I don't believe that you can generally leverage
| this for the kinds of memory exploits made in C/C++.
|
| I recall that some ML dialects (Standard ML and OCaml)
| have a library function Obj.magic : 'a -> 'b which
| escapes the type system. Using this can easily cause
| segfaults. Does that mean Standard ML and OCaml are not
| memory safe? Generally, no, they're extremely safe if you
| avoid that feature, which is most likely. This is
| arguably less safe than Go, since you most likely won't
| accidentally run that function.
|
| [1]: https://media.defense.gov/2022/Nov/10/2003112742/-1/
| -1/0/CSI...
| kccqzy wrote:
| I'm trying to provide some commentary to OP's original
| term of "ordinary code" three comments above. While this
| term is inherently ambiguous and subjective, my personal
| opinion is that appending to slices simultaneously from
| multiple goroutines count as "ordinary code" but
| Obj.magic does not.
| tptacek wrote:
| Yes. And: you will run into correctness bugs quickly if
| you mutate shared references in Go code. It's only my
| contention that you won't create a security
| vulnerability, in the colloquial understanding of the
| term (ie: a panic doesn't count).
| lll-o-lll wrote:
| If I can induce a race that corrupts a data structure so
| that it leaks data back to me that I shouldn't have
| access to, does that count?
| melodyogonna wrote:
| How can you do that?
| tsimionescu wrote:
| You can, though it's much harder than in C or C++ or
| unsafe Rust for this to be exploitable. A data race on an
| interface value can give you a corrupted interface value,
| overwriting the vtable with struct contents. This can
| happen to lead to arbitrary code execution if you're
| unlucky enough, though in most cases it would be a
| SIGSEGV. It's also very hard for an attacker to craft a
| payload that can be guaranteed to reach this, though with
| a microservixe architecture with automatic restarts of
| failed services, they might get a lot of tries.
| kaba0 wrote:
| I mean, a very serious security vulnerability is/was row
| hammering, where an attacker was waiting on flipping a
| bit they have no access to by continuously flipping
| neighboring ones. Compared to that a race condition is
| "trivial" to exploit.
| consteval wrote:
| I don't think that's ordinary code. In any language, if
| you don't use a thread safe container and mutate it from
| multiple threads you'll get problems. This isn't an issue
| of memory safety but rather thread safety. You have to
| check the documentation for thread safe operations or
| otherwise use a lock. This goes for C#, Java, Go, you
| name it - the one singular exception being Rust. But,
| even Rust does not fully get around it.
| kccqzy wrote:
| You missed the point.
|
| > In any language, if you don't use a thread safe
| container and mutate it from multiple threads you'll get
| problems.
|
| Yes I agree there will be problems but what kind of
| problems do you get? Can you potentially get a memory
| safety problem, or are you guaranteed that the problem is
| not a memory safety problem?
|
| The point is that thread safety problems in Go lead to
| memory safety problems. That's not the case in Java. You
| can crash the whole Go program by doing that, but you
| cannot crash the JVM by doing the same thing.
| capitol_ wrote:
| Could you share more of your thoughts on why that kind of
| memory corruption wouldn't be exploitable? Do go have
| something in place that prevents it?
| Cthulhu_ wrote:
| To add to Go being memory safe, it automatically
| blanks/zeroes memory, unlike C.
| everybodyknows wrote:
| > corrupt GC state
|
| I understand this to mean the runtime's internal state,
| not visible to user code. If so, in general we should
| expect almost any sort of crash mode to be possible.
| Seems fair enough to call this "memory-unsafe".
| tptacek wrote:
| You'll be using an idiosyncratic definition the rest of
| the industry does not use, but you do you.
|
| What I think is happening here is another instance of a
| pattern that recurs all the time in communities like
| this: a term of art was created, "memory safety", to
| address the concept of languages that don't have buffer
| overflows, integer overflows, use-after-frees, double
| frees, controllable uninitialized pointers, and all the
| other memory lifecycle vulnerabilities. People unfamiliar
| with the state of the art heard the term, liked it, and
| have axiomatically derived their own definition for it.
| They like their definition better, and are not open to
| the idea that the term exists to serve a purpose
| orthogonal to their arguments.
|
| Another recent instance of the same phenomenon: "zero
| trust".
|
| Just as happened in the Zero Trust Wars of 2022, people,
| hearing the industry definition and intent of the term,
| scramble to reconcile their axiomatic definition with the
| state of the art, convincing themselves they were right
| all along.
|
| The problem they have in this particular argument is:
| _where are the vulnerabilities?_ Go is not a niche
| language. It is a high-profile target and has been for
| over a decade. I saw Go security talks at OWASP
| Chicago(!) in 2012(!). People have all sorts of
| hypotheses about how a memory corruption vulnerability
| --- not "memory corruption", but a _vulnerability_
| stemming from it, implying valuable attacker control over
| the result of whatever bad thing happened --- might sneak
| into a Go program. Practitioners hear those axiomatic
| arguments, try to reconcile them with empirical reality,
| and: it just doesn 't hold up.
|
| Just for whatever it's worth to hear this, if at Black
| Hat 2025 someone does to Go what James Kettle does to web
| frameworks ever year and introduces a widespread
| repeatable pattern of memory exploitability in Go race
| conditions, about half of my message board psyche will be
| really irritated (I'll have been wrong!), but the other
| half of my message board psyche will be fucking thrilled
| (there will be so much to talk about!) and _all_ of my
| vulnerability researcher psyche will be doing somersaults
| (there will be so many new targets to hit!). On net, I 'm
| rooting for myself being wrong. But if I had to bet:
| we're not going to see that talk, not at BH 2025, or
| 2026, or 2027. I'm probably not wrong about this.
| thinkharderdev wrote:
| > You'll be using an idiosyncratic definition the rest of
| the industry does not use, but you do you.
|
| What definition are you using that you seem to think is
| the one definition of memory safety that is canonical?
|
| > don't have buffer overflows, integer overflows, use-
| after-frees, double frees, controllable uninitialized
| pointers, and all the other memory lifecycle
| vulnerabilities
|
| Any guarantees about this are dependent on the language
| not having undefined behavior in its safe subset. Once
| you have undefined behavior any other guarantees made
| about memory safety are significantly weakened.
|
| > where are the vulnerabilities?
|
| I don't know of any other than code written to
| demonstrate the concept. But I imagine if you look at any
| large Golang codebase you will find race condition bugs.
| So the fact that you have potential undefined behavior
| resulting from an extremely common coding error seems
| like it might be something to be concerned about (to me
| at least). Especially given how little Golang helps you
| write safe concurrent code.
|
| That's not to say that Go is therefore totally useless
| and everyone should stop using it now because it's
| "insecure". But it also seems ... unwise ... to me to
| just pretend it's nothing because it is hard to exploit
| or that we don't have any (known) examples of it being
| exploited.
| tptacek wrote:
| You will find race condition bugs. You will not find
| memory corruption vulnerabilities. Go look.
|
| The argument is not about whether languages admit
| vulnerabilities --- all of them do. The argument is about
| whether they admit the vulnerabilities that motivate the
| term of art "memory safety". Go does not, at least not in
| any non-contrived scenario not involving "unsafe" or FFI.
|
| As for definitions, I like what Alex wrote about this;
| or, you can look at ISRG's writing about it.
|
| https://alexgaynor.net/2023/oct/02/defining-the-memory-
| safet...
| fragmede wrote:
| The problem with naming the language "go" is that my
| initial interpretation of your comment was to think that
| "go look" was a valgrind-esque tool for finding such
| things.
|
| Too bad it's not.
| thinkharderdev wrote:
| You linked me to an article that defines an unsafe
| language as:
|
| "a programming language which, by default, allows code to
| introduce memory-related vulnerabilities (use after free,
| buffer over/under flow, use of uninitialized memory, type
| confusion) and undefined behavior,"
|
| Which was my whole point. The "and undefined behavior"
| part is absolutely essential to the definition because
| you can't guarantee the first part without it.
|
| > Go does not, at least not in any non-contrived scenario
| not involving "unsafe" or FFI.
|
| It absolutely does. You can find examples of code that
| triggers undefined behavior in this thread. You can hand-
| wave them away as being "contrived" (which is literally
| correct) but that doesn't just make the problem go away.
|
| We had a whole collective meltdown (pun intended) over
| spectre and meltdown a few years back even though AFAICT
| there have never been any known exploits in the wild. But
| most people who are knowledgeable in these things still
| took it seriously because they realized that:
|
| 1. Just because you haven't found a specific instance of
| an in-the-wild exploit doesn't mean there isn't one 2.
| You don't want to necessarily wait because if something
| it exploitable in theory is will almost certainly be
| exploited sooner or later. And it takes time to sort this
| stuff out
|
| Is this relevant to your average working dev slinging
| micro-services in go? No, probably not and it's probably
| not something you should even think about for the most
| part. But if you're writing a complex, mission critical
| system with very hard security requirements? Yeah maybe
| you should worry about little bit about the fact that Go
| is, by the definition you yourself provided me, not a
| memory safe language.
| Thaxll wrote:
| Appending from multiple goroutine to an in un-
| synchronized slice is "memory safe", it's completely
| different from c/c++.
|
| It behave exactly like Java or C# which are also memory
| safe.
| kaba0 wrote:
| Not at all. Java or C# can end up in a logical bug from
| that, but they will never corrupt their runtime. So in
| java you can just try-catch whatever bad stuff happens
| there, and go on afterwards.
|
| Go programs can literally segfault from a data race.
| That's no memory safety.
| tsimionescu wrote:
| I'm not sure of C#, but Java has stronger memory
| guarantees than Go, even in the presence of a data race.
|
| In Java, all primitive types (including Object pointers)
| are atomically modified. And since all Java writes are
| primitives (Java doesn't have structs), you can never
| corrupt a data structure at the Java level. Of course,
| you can still corrupt it at a logical level (break an
| invariant established in the constructor), but not at the
| language level.
|
| Go has a guarantee that word-sized reads/writes are
| atomic, but Go has plenty of larger objects than that. In
| particular, interface values are "fat pointers" and
| exceed the word-size on all platforms, so interface
| writes are not atomic. Which means another thread can
| observe an interface value having a vtable from one
| object but data from another, and can then execute a
| method from one object on data from another object,
| potentially re-interpreting fields as values of other
| types.
| tucnak wrote:
| > Which means another thread can observe an interface
| value having a vtable from one object but data from
| another, and can then execute a method from one object on
| data from another object, potentially re-interpreting
| fields as values of other types.
|
| If this were the case, then surely someone could
| construct a program with goroutines, loops and a handful
| of interface variables--that would predictably fail,
| right? I wouldn't know how to make one. Could you, or
| ChatGPT for that matter, make one for demo's sake?
| kokada wrote:
| I am also curious, I keep reading from this thread folks
| talking that this is possible, but I can't see to find
| anything searching in Google/DDG.
|
| There is this document from Golang devs itself[1], that
| says:
|
| > Reads of memory locations larger than a single machine
| word are encouraged but not required to meet the same
| semantics as word-sized memory locations, observing a
| single allowed write w. For performance reasons,
| implementations may instead treat larger operations as a
| set of individual machine-word-sized operations in an
| unspecified order. This means that races on multiword
| data structures can lead to inconsistent values not
| corresponding to a single write. When the values depend
| on the consistency of internal (pointer, length) or
| (pointer, type) pairs, as can be the case for interface
| values, maps, slices, and strings in most Go
| implementations, such races can in turn lead to arbitrary
| memory corruption.
|
| Fair, this matches what everyone is saying in this
| thread. But I am still curious to see this in practice.
|
| [1]: https://go.dev/ref/mem
|
| Edit: I found this example from Dave Cheney:
| https://dave.cheney.net/2014/06/27/ice-cream-makers-and-
| data.... I am curious if I can replicate this in e.g.:
| Java.
|
| Edit 2: I can definitely replicate the same bug in Scala,
| so it is not like Go is unique for the example in that
| blog post.
| tsimionescu wrote:
| > Edit 2: I can definitely replicate the same bug in
| Scala, so it is not like Go is unique for the example in
| that blog post.
|
| Could you share some details on the program and the
| execution environment? Per my understanding of the Java
| memory model, a JVM should not experience this problem.
| Reads and writes to references (and to all 32 bit values)
| are explicitly guaranteed to be atomic, even if they are
| not declared volatile.
| tsimionescu wrote:
| Sure, here is an example:
|
| https://go.dev/play/p/_EJ4EvYntr2
|
| When you run this you will see that occasionally it
| prints something other than 11 or 100. If it doesn't
| happen in one run, run it again a few times.
|
| An equivalent Java program will never print anything
| else.
| kccqzy wrote:
| No. Go is worse than Java in this regard. I'll just give
| you a random example from my five seconds of search:
| https://github.com/golang/go/issues/37484
| danielheath wrote:
| Go lets you use `unsafe.Pointer` (or indeed, assembly
| intrinsics) if you really want to, but those are certainly
| not used "ordinarily".
| tsimionescu wrote:
| It's not just about that. Data races can expose an object
| in a state that was never written from any thread in Go,
| potentially corrupting even internal details not exposed.
| Simply writing a struct value from two different threads
| can expose this.
| kaba0 wrote:
| But ordinary go code is not memory safe. Data racing can
| trivially happen just by using the language's primitives. It
| requires no special keyword like unsafe, or native FFI like
| in other, actually memory safe languages (rust, or of the GCd
| kind, java, c#, Js)
| remram wrote:
| How safe is it? It has pointers and they are widely used
| (more than Rust where pointers are unsafe, but there are
| other reference types). Are those safe?
| tptacek wrote:
| Yes.
| dfawcus wrote:
| Generally.
|
| That is as it does not have pointer arithmetic, unlike C,
| and arrays / slices are bounds checked. So one will get a
| crash from a null pointer deref.
|
| The other risk with null pointer access is struct member
| access via such a pointer, but again due to lack of pointer
| arithmetic, that can't be easily triggered. The one way
| would be to have a massive struct, say much greater than
| the page size, and deref through that - fairly unlikely.
|
| The other reference types (slices, maps, interface values,
| channels) are safe unless subject to data race issues
| (multi goroutine update). However channels are safe there,
| as their role is to be used from multiple goroutines.
|
| So the path to lack of memory safety would be a data race,
| leading to type misinterpretation, hence type unsafety,
| then incorrect access and/or spatial and temporal unsafety
| as a consequence.
|
| Apart from poor design / implementation of explicit multi
| threaded apps, the most likely data race strikes me as
| accidental lexical capture by a goroutine, hence movement
| to the heap, and a resultant race. The sort of thing which
| was mentioned in a paper (by Uber?). Those should be
| amiable to detection by linters.
|
| The other case of races from poor threading design would be
| harder to automatically detect, but also harder to trigger.
| Probably avoidable by correct use of mutexes around access
| to the shared types (slices and maps), or simply by
| following an Actor or CSP design model.
| remram wrote:
| Thanks for the summary, that is very helpful.
|
| At a language level though, it is either safe or unsafe.
| If it is "generally safe" provided you use it correctly,
| I would say it is not safe, in the strict sense.
|
| I don't think data races on pointers are allowed (looking
| at the memory model: https://go.dev/ref/mem) but I am not
| sure I have understood your scenario fully. Maybe I
| should read that paper you mention.
|
| Thanks again for the detailed response!
| dfawcus wrote:
| That is an absolutist position, which some of us don't
| agree with. Taking the view that in practice there are
| degrees of "memory safety". That is generally my
| position, and that the largest benefits come from spatial
| safety, then temporal safety, in that order.
|
| On that absolute position, there possibly are no "memory
| safe" languages, not even Rust as until it's borrow
| checker "bug" is fixed, it fails the absolutist position.
| If such a bug is left unfixed for long enough, one can
| deem it as de-facto "won't fix".
|
| The Go example code provided elsewhere in the thread
| included a memory race on an "interface value", that
| being a form of "fat pointer". It was that I was
| referring to, updating only half of value, so making it
| internally inconsistent.
| uluyol wrote:
| Go very much is memory safe in the absence of data races.
|
| Data races cause issues in all languages, though it's fair to
| say that Go is affected slightly more than languages like Java.
| Rust is a bit special by making data races hard to trigger
| (impossible in safe code IIUC), but this is not typical.
| pjmlp wrote:
| Kind of, regarding Rust.
|
| It is impossible in the context of having all threads
| accessing in-process memory.
|
| If the data can be accessed externally, regardless of the
| guarantees being uphold on the Rust side, there are no
| guarantees from third parties accessing the same data.
|
| It also doesn't prevent other race issues with external data.
| galangalalgol wrote:
| Memory like that needs to be wrapped with unsafe for
| access, there is the volotile crate to mark stuff like that
| so the compuler won't optimize it away.
|
| Other than rust haskell seems like the other primary
| candidate for memory safety even across threads.
| pjmlp wrote:
| Yes, but it doesn't guarantee changes occurring from
| third parties, even if everything is done correctly on
| Rust side, and all invariants are correct, so corrupted
| data can be still be seen as valid.
| galangalalgol wrote:
| Is there any defense at all against what you are talking
| about? I mean, I could use a firewire controller to
| modify memory without the processor or OS being aware. I
| suppose you could sign every block of memory using the
| tpm, but you'd have to the signatures in the tpm, and the
| code to check the signatures, and so on.
| pjmlp wrote:
| The point is that Fearless Concurrency comes with some
| footnotes when doing the full spectrum of systems
| programming.
|
| Which tend to be ignored when talking about how Rust is
| so much better than anything else.
|
| Ye it has improved some concurrency/parallelism
| scenarios, not all of them.
| kaba0 wrote:
| Yeah, and C is memory safe in absence of memory safety bugs..
| marcus_holmes wrote:
| > I absolutely hated this at the start, but now ~3 years into
| maintaining a go codebase, I find it quite nice
|
| I've heard this so often. Thanks for sharing :)
|
| I find going back to other languages and trying to read other
| people's code is a trial. There's always the temptation to
| write "smart" code that is terse but takes a lot of parsing to
| understand.
|
| I love that I can pick up anyone's Go code and it's going to
| make sense almost immediately because everything is explicit,
| and laid out the same way (including that rhythm of "do the
| thing, test the error, do the thing, test the error")
| lynx23 wrote:
| > I know exactly what each function does
|
| Isn't this basically the same argument that C people have been
| using since, what, 40 years?
| tbiehn wrote:
| Semgrep is another great option to get value out of static
| analysis checks against both the language and a few common
| frameworks. It remains a popular choice for security folks
| writing static detection rules (and contributing them to the
| commons).
|
| You can check the open rules here;
| https://github.com/semgrep/semgrep-rules/tree/develop/go
| Xaki8271 wrote:
| Wow!
| hasnain99 wrote:
| does it work
___________________________________________________________________
(page generated 2024-11-05 23:01 UTC)