[HN Gopher] Curl audit: How a joke led to significant findings
___________________________________________________________________
Curl audit: How a joke led to significant findings
Author : campuscodi
Score : 132 points
Date : 2023-02-14 13:24 UTC (9 hours ago)
(HTM) web link (blog.trailofbits.com)
(TXT) w3m dump (blog.trailofbits.com)
| nayuki wrote:
| [2021-02-12] <Daniel Stenberg> Will I ever rewrite curl in rust?
| I don't believe in rewrites, no matter which language. I believe
| in replacing code and fixing components gradually over time. That
| _could_ mean that we have a curl written mostly in rust in 10
| years. Or in 20 years. Or not. --
| https://twitter.com/bagder/status/1360131939794042884
|
| [2023-02-14] <The article> the fuzzer quickly uncovered memory
| corruption bugs, specifically use-after-free issues, double-free
| issues, and memory leaks
| j-krieger wrote:
| People arguing against the usage of Rust just because they
| don't ,,believe" in its advantages puzzle me. They are usually
| proven wrong with multiple CVE's shortly after.
|
| Rust was never expected to be a magic fix-all, but after the
| C++ committee's stubborn refusal of introducing some sense of
| memory safety into the language in favor of keeping ABI
| compatibility, it might turn out to be a self fulfilling
| prophecy.
| regularjack wrote:
| He says "I don't believe in rewrites" , he doesn't say "I
| don't believe in Rust".
| CharlesW wrote:
| TLDR: We fuzzed something that wasn't previously fuzzed --
| specifically, CLI arguments -- and found some good bugs. Lesson:
| Don't forget to fuzz all the methods people might use to
| interface with your software.
| [deleted]
| mannykannot wrote:
| ...including environment variables, I would think (though this
| investigation apparently did not extend to them.)
| [deleted]
| andrewmcwatters wrote:
| Boy if they only used Rust, then these curl commands that no one
| would ever enter would have been memory safe.
| bolanyo wrote:
| your car has libcurl on it, but not Rust.
| leafario2 wrote:
| There are quite some cars that do, actually :)
| [deleted]
| tomjakubowski wrote:
| Not yet https://crates.io/keywords/canbus
| dralley wrote:
| VW is using Rust in the entertainment system. There is both
| interest and a bit of actual adoption.
| varjag wrote:
| As an ID.4 owner I can only remark it does not appear to
| help them.
| dralley wrote:
| I take it back, it was Volvo, not VW
|
| https://news.ycombinator.com/item?id=32960768
| justinsaccount wrote:
| Most people actively avoid having rust anywhere near their
| cars.
| woodruffw wrote:
| `curl` is widely integrated via its CLI: you'll find all kinds
| of programs (from simple shell scripts to actual compiled
| binaries) that interact with `curl` through command-line
| arguments. So the attacker model here isn't "nobody would type
| this" it's "does your SoHo router or other software shell out
| to cURL in an attacker-controllable way."
| verall wrote:
| If an attacker could control the curl CLI on your SoHo
| router, couldn't they just direct it to download their evil
| payload, and then use one of a million more common problems
| to run the evil? Or directly exfiltrate data through HTTP
| requests made by curl? I don't think anyone runs curl as root
| so there is no privilege escalation.
|
| Seriously I can't understand the threat model here.
| woodruffw wrote:
| There are different scopes: it's not the case that every
| controllable CLI invocation is also one where an attacker
| could change the URL entirely. There are cases where they
| might only be able to change the behavior of a single flag
| or option, or a subrange of characters in a specific
| protocol's URLs, etc.
|
| I think the realistic attack model here is:
|
| 1. SoHo router that has a public accessible endpoint that
| takes inputs boiling down to "safe" cURL flags;
|
| 2. Those flags can actually induce memory corruption;
|
| 3. An attacker is able to execute arbitrary remote code in
| the context of the cURL subprocess through a payload packed
| into the "safe" flag.
| [deleted]
| gigaflops wrote:
| Memory, memory, memory, and memory again. All 4 of them.
|
| Yet everyone's idea of safety still seems to be "just write bug-
| free C code, bro".
| intelVISA wrote:
| To be fair cURL is pretty aged now. I wrote a cURL-like
| recently and was surprised my ver benched 2x as fast. (though I
| cheated by not using crappy 3rd party libs)
| afrcnc wrote:
| to be fair, cURL was written when defacing websites was
| considered top-tier hacking
| pjmlp wrote:
| Morris worm happened in the 1980's.
| woodruffw wrote:
| Where did you get that from? cURL was started in 1996, which
| was around the nadir of the first wave of major memory
| exploit techniques. Lots of people in the security industry
| made their name with stack and heap exploit techniques that
| are now considered staid.
| cmeacham98 wrote:
| What memory safe languages/tools existed in 1996 and still
| exist today with some meaningful marketshare? Java (barely,
| and I think it was proprietary at the time)?
|
| It seems like "write memory safe C" was the best technique
| in 1996 if you wanted any semblance of
| portability/performance.
| eesmith wrote:
| By 1996 people were building commercial web stores in
| Python. For example, see "The Real World Case Study:
| Python in a Commercial Environment" at https://legacy.pyt
| hon.org/workshops/1997-10/proceedings/stei... .
| icedchai wrote:
| Don't forget about Perl!
| eesmith wrote:
| I didn't want to get involved in what "meaningful
| marketshare" means with respect to Perl. In the same
| vein, AOLServer (ca. 1995) used multithreaded Tcl, and
| Tcl is also still around.
| Groxx wrote:
| I believe their (and this thread's) original point was
| that even _absurdly_ widely used, studied, _and
| continuously fuzzed_ C tools written and maintained by
| many serious experts for decades....
|
| ... still have _multiple_ easily-uncovered memory issues.
|
| Nobody writes safe C. Believing otherwise is delusional.
| This has been known for decades. But many believers still
| exist.
| an-unknown wrote:
| > Nobody writes safe C. Believing otherwise is
| delusional. This has been known for decades. But many
| believers still exist.
|
| This is factually incorrect. You can write safe C code
| which is even correct according to some specification, as
| in "guaranteed bug free". This is not what most people do
| (or even know how to do) though.
| woodruffw wrote:
| I'm not aware of a _generalized_ safe subset of C. There
| are lots of compiler extensions and model annotations
| that offer security improvements, but very few will claim
| to offer spatial and temporary memory safety in the way
| that safe programming languages can.
|
| There are lots of specification driven C applications,
| which rely on (1) the specification being bug free, (2)
| the application of the specification being bug free, and
| (3) the model or proof assistant for the specification
| being sound and bug free. Getting all three of those is
| historically very difficult.
| Groxx wrote:
| No realistic code that does that exists AFAIK. It may
| have _sections_ that do it (which is great!), but not a
| total useful binary.
|
| It's a straw-man that's trotted out every time, and it's
| utterly meaningless beyond academic arguments. _Even if_
| it is true, it 's not relevant for any practical
| purposes.
| rainingmonkey wrote:
| Lua originated in 1993 and ran anywhere C would.
|
| Unfortunately it still lacks meaningful marketshare
| though.
| woodruffw wrote:
| I'm not sure what this has to do with my comment: my
| observation was only that memory corruption was a well
| understood vulnerability domain in 1996, not that there
| was a particularly wide variety of alternatives to C or
| C++.
| pjmlp wrote:
| Modula-2, Eiffel, Ada, Object Pascal,...
| [deleted]
| [deleted]
| Karellen wrote:
| > Yet everyone's idea of safety still seems to be "just write
| bug-free C code, bro".
|
| Does it though? I think "the industry" has pretty much come
| around on C being unsafe, and generally not a great choice of
| language for new projects.
|
| But the fact is that there's a lot of C code out there, and we
| can't just ditch or rewrite it all overnight. In an ideal
| world, we wouldn't be starting from here. But we are, and it's
| going to take time to get to where we need to be.
|
| Heck, take a look at the recent story about "Considering C99
| for curl": https://news.ycombinator.com/item?id=33704054
|
| From the linked article:
|
| > The slowest of the "big compilers" to adopt C99 was the
| Microsoft Visual C++ compiler, which did not adopt it properly
| until 2015 and added more compliance in 2019. A large number of
| our users/developers are still stuck on older MSVC versions so
| not even all users of this compiler suite can build C99
| programs even today, in late 2022.
|
| What are the chances that the kinds of people who are still
| doing development on systems where the C compiler is from 2015
| or earlier, or on a more niche system that Windows that lots of
| new safe languages haven't been ported to yet (and such ports
| aren't even on the radar), are going to able to start moving to
| developing in rust or whatever anytime soon?
| returningfory2 wrote:
| > Does it though? I think "the industry" has pretty much come
| around on C being unsafe, and generally not a great choice of
| language for new projects.
|
| If you look at some of the Rust threads on HN right now [0],
| you find many commenters claiming that there is no reason to
| replace C with Rust. It could be a vocal minority, but
| there's still a sense that "not everyone is on board".
|
| [0] https://news.ycombinator.com/item?id=34788714
| marcosdumay wrote:
| Oh, if you take a large overall sample of the Reddit, you
| will find the overall opinion that Rust is gay (really,
| it's way more irritating than here).
|
| Those are just people annoyed that their knowledge is
| getting outdated, and grasping for any medium-approved
| excuse to not learn the new thing. Those believe that Rust
| will replace C, they just don't like it.
|
| (Anyway, C, like COBOL, won't just die. There's no reason
| to be afraid.)
| bigbillheck wrote:
| > Oh, if you take a large overall sample of the Reddit,
| you will find the overall opinion that Rust is gay
|
| Yeah but do they like the language tho.
| arp242 wrote:
| > you find many commenters claiming that there is no reason
| to replace C with Rust
|
| I actually don't see many claim that in that thread. There
| are 196 comments and I didn't read all of them, but only
| one greyed-out top-level comment says so, and what I can
| skim from the other comments I don't really see a whole lot
| of people say "C is fine!"
|
| There are some concerns about performance, and some other
| comments about (alleged) "Rust spam", but that's not the
| same as "C is fine!"
|
| In my experience "C is unsafe" was already generally agreed
| on about 20 years ago, which is one reason many choose
| Java, Python, PHP, etc. at the time. It wasn't a universal
| consensus, but if you were to take a poll in 2003 I bet it
| would be well over 50%.
| woodruffw wrote:
| HN samples the C (and Rust) communities, but I don't think
| we should take HN's opinions as representative of either.
| j-krieger wrote:
| This. My indicator on the success of a technology has
| been the volume of disagreement among the HN crowd. Just
| look at React / Javascript frameworks. HN will hate on
| those with a fiery passion, while literal trillions of
| dollars are generated with them.
| randomdata wrote:
| _> there is no reason for replace C with Rust._
|
| If you have some reason to choose C today, I'm not sure
| Rust is a contender to stand in as a replacement. They are
| vastly different languages. That doesn't mean there is
| rejection that C's memory model is unsafe, just that there
| isn't a "better C" recognized to choose from.
| quectophoton wrote:
| IMHO Rust could be a good replacement for C when (if) it
| gets an actual spec, or at least when (if) there are
| multiple real implementations of the compiler (meaning,
| with borrow checker, compiler errors, and all).
|
| Because otherwise, you could write Rust 2018 edition (for
| example), and there's nothing preventing a situation
| where your currently-valid Rust-2018-edition program
| stops compiling (or behaves differently) on a future
| version of rustc even when compiling to that specific
| edition, just because some behavior is (in the future)
| considered a compiler bug or something. And it would be
| perfectly valid to cause such breakage, because there's
| no definition of "what is Rust", other than "what this
| specific compiler in this specific version accepts".
|
| I'm not talking about whether such possible backwards-
| incompatible change on a Rust edition would be good or
| bad. Just that with the current monoculture around a
| single implementation, there's not many guarantees.
|
| A specification, or multiple implementations, would at
| least add some (good) friction.
|
| I'm not denying its current value as a safe language;
| just that, as it is now, I don't consider it as a good
| choice for programs that must stay alive for decades,
| like `curl` or an operating system.
| com2kid wrote:
| Rust is not a C replacement, it is a C++ replacement.
|
| Many of the people writing code in C do not want to use
| C++.
|
| And many of the embedded use cases for C already involve
| very careful memory usage. If you've banned malloc and
| don't have threads, you've removed a huge swath of common
| bug sources right off the bat.
|
| For someone writing code to setup registers on a DMA
| controller, Rust has nearly 0 benefit.
|
| (That said, no one should be writing new command line
| utilities in C, parsing arbitrary data in C is
| dangerous!)
| steveklabnik wrote:
| Not everyone agrees with this characterization, including
| some well known (now former) C developers:
| http://dtrace.org/blogs/bmc/2018/09/18/falling-in-love-
| with-...
| com2kid wrote:
| Excellent post, thanks for linking. The arguments made
| for replacing C with Rust seem sound.
| dralley wrote:
| >just because some behavior is (in the future) considered
| a compiler bug or something
|
| Unsound behavior _is_ considered a compiler bug, and is
| liable to change if you accidentally rely on it. I would
| hope that having multiple compilers won 't change that.
| The motivation to actually fix soundness holes is one of
| the primary differentiators of Rust.
|
| That said, these are very rare.
| quectophoton wrote:
| I would even expect that having multiple implementations
| would help catch small things that might have been missed
| in the reference implementation.
|
| I'm looking forward to Ferrocene's specification and the
| gccrs implementation.
| nindalf wrote:
| > need multiple implementations
|
| This thinking is understandable but it's basically
| pattern matching. There were two successful languages
| that followed this model and so a language that seeks to
| replace them should adopt the same model of design by
| committee followed by multiple independent
| implementations.
|
| But there's no reason to think that a language needs
| multiple implementations to succeed. Python has had one
| implementation used by all its user and that hasn't
| prevented from becoming a top 3 language by usage. Ditto
| with other very popular languages like Go, Ruby,
| TypeScript, C#.
|
| I don't understand the concern that your code will stop
| compiling with a future version of the compiler. Firstly,
| Rust hasn't broken backwards compatibility in 8 years and
| doesn't plan to in future. Secondly, why do you need to
| upgrade the compiler? If it works on the current version
| you can continue to use that in perpetuity.
|
| Honestly, having multiple implementations would be a
| curse. I see nothing but negatives. Even the smallest
| proposals (like embedding data in a binary) take a decade
| to get buy in and 3-4 years to be implemented. Convincing
| everyone of the benefit of a feature is just too
| difficult so a lot of good ideas are never put forward,
| let alone implemented.
|
| And the difficulty of getting it working across compilers
| and operating systems. How hard is it to write C++20 code
| that will compile with all major compilers on all major
| OSs? On Rust it's trivial. To put it simply - it's
| trivial to answer the question "is this Rust code valid?"
| If rustc compiles it, it's valid today and will always be
| valid (modulo bugs).
|
| You want alternate implementations? There's a WIP - gcc-
| rs. But thankfully they haven't forked the semantics of
| the language. It should and shouldn't compile all code
| that rustc does and doesn't. Otherwise it's a bug in gcc-
| rs. Anything else would make the life of Rust users
| painful.
|
| In summary, multiple implementations is a bad idea for
| many reasons. If you can express the benefit beyond
| "that's the way C and C++ do it", I'm eager to listen.
| But please explain how Python and Go managed to succeed
| with one impl.
| Karellen wrote:
| The trouble with a single, unstandardised implementation
| is that it's only available on the OSs and CPUs that the
| lone implementation developers choose to target.
|
| By creating a standard, and allowing multiple
| implementations, someone creating a new OS or a new CPU
| architecture can create their own implementation of the
| language without having to wait for anyone else to deign
| to create the port.
|
| Maybe a new implementation won't compile very fast, or
| produce great code, or it might not even catch all the
| incorrect constructs that it ought to. But if it can make
| all the existing code written in that language suddenly
| available for the new platform, that platform instantly
| becomes orders of magnitude more useful and powerful.
|
| > How hard is it to write C++20 code that will compile
| with all major compilers on all major OSs?
|
| That depends on the code. You want to compile Firefox?
| You might be out of luck.
|
| You want to compile curl? Based on how portable it
| already is, chances are it might work out of the box once
| you've got a libc implementation working.
|
| You want to compile Gnu bash and coreutils, to get a
| basic Posix shell environment up and running? It might be
| a bit of work, but easier than having to reimplement a
| whole userland from scratch.
|
| > But please explain how Python and Go managed to succeed
| with one impl.
|
| It's not that the language can't succeed - at least, not
| on popular platforms that the devs are interested in
| targetting. It's that those devs get to dictate which
| platforms are even capable of making use of the software
| written in those languages.
|
| And that just rubs me the wrong way.
| pitterpatter wrote:
| >By creating a standard, and allowing multiple
| implementations, someone creating a new OS or a new CPU
| architecture can create their own implementation of the
| language without having to wait for anyone else to deign
| to create the port.
|
| Why does that require multiple implementations? There's
| nothing stopping said party from adding support for their
| OS/arch to rustc directly today. There are multiple
| examples of this already: fuchsia, Sony PSP, Nintendo
| Switch, etc [1]. Now if you want support in rustc itself
| out-of-the-box this does require LLVM support your target
| as well. But even then, again nothing stopping you from
| adding LLVM support too. The avr-rust project for
| instance maintained an LLVM/rustc fork for a while before
| those patches were upstreamed.
|
| [1] See the wide variety of targets with varying support
| today: https://doc.rust-lang.org/nightly/rustc/platform-
| support.htm...
| Karellen wrote:
| Off the top of my head? Maybe you want to target a small
| system with a native compiler, so users of those systems
| don't have to cross-compile, but the standard
| implementation is just too big to run on those systems?
| (e.g. tcc)
|
| Or maybe you don't want your work to be redistributed
| under the original project's license, and would prefer to
| have a copyleft/permissive/proprietary licensed version
| instead, which can still call itself a "real"
| implementation. (e.g. mono or gcj/openjdk. Or gcc,
| originally.)
| nindalf wrote:
| > It's that those devs get to dictate which platforms are
| even capable of making use of the software written in
| those languages.
|
| Adding other tiers is welcomed. Take a look at the
| supported platforms (https://doc.rust-
| lang.org/nightly/rustc/platform-support.htm...) and see
| if there's a platform you'd like to target that isn't a
| tier 3 target at least. There's HaikuOS, PlayStation1
| etc. I'm on a Tier 2 platform myself and I'm really
| happy.
|
| If you'd like even more platforms, there's an ongoing
| project to add a GCC backend to rustc
| (https://blog.antoyo.xyz/rustc_codegen_gcc-progress-
| report-20).
|
| > if it can make all the existing code written in that
| language suddenly available for the new platform, that
| platform instantly becomes orders of magnitude more
| useful and powerful.
|
| I agree with this, but creating a front end that can
| actually compile all existing code correctly is a massive
| task. If the compiler is modular enough, you should be
| able to contribute just a backend, rather than
| reimplementing a frontend. I'm strongly against only the
| frontend reimplementation, because it's no longer clear
| what "valid" Rust code is.
|
| In summary - I completely agree with the importance of
| targeting niche platforms. I think supporting multiple
| backends (LLVM now, libgccjit and Cranelift in future)
| gets us there without fracturing the ecosystem.
|
| > but easier than having to reimplement a whole userland
| from scratch.
|
| Probably. But once it's reimplemented
| (https://github.com/uutils/coreutils) I think it's really
| cool you can run get binaries for Linux, macOS and
| Windows with one build command.
| pjmlp wrote:
| C# has a ECMA standard and several implementations.
___________________________________________________________________
(page generated 2023-02-14 23:02 UTC)