[HN Gopher] Acme.sh runs arbitrary commands from a remote server
___________________________________________________________________
Acme.sh runs arbitrary commands from a remote server
Author : Kesseki
Score : 236 points
Date : 2023-06-09 02:21 UTC (20 hours ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| ronsor wrote:
| These sorts of hacks remind me of when AOL's AIM server was
| exploiting bugs in their own client to run verification code[0].
| Hardly an acceptable practice in 2023, though.
|
| [0] https://www.geoffchappell.com/notes/security/aim/index.htm
| 0x0 wrote:
| Looks like they are issuing under a sub-CA of "ssl.com" according
| to https://github.com/acmesh-
| official/acme.sh/issues/4659#issue...
|
| Interestingly, the mozilla dev-security-policy group seems to
| contain a recent discussion about including "ssl.com" in the root
| store here https://groups.google.com/a/mozilla.org/g/dev-
| security-polic...
|
| Curious to know if this could, maybe it should, have ripple
| effects to the various SSL Root CA programs. Having someone run a
| subCA that actually exploits an RCE against ACME clients doesn't
| seem very trustworthy, and any CA enabling this behaviour should
| probably be kicked out of the trust stores?
| agwa wrote:
| The sub CA is operated by ssl.com, not HiCA (which is not a
| trusted certificate authority). HiCA is relaying the
| certificate requests to ssl.com, which is properly validating
| the requests in accordance with all the requirements. ssl.com
| isn't doing anything wrong. That's why HiCA needs to exploit an
| RCE in acme.sh - ACME doesn't support relaying certificate
| requests to other CAs like this.
| 0x0 wrote:
| Someone posted a comment on github claiming they are the
| founder of Quantum (the sub CA of ssl.com - see
| https://crt.sh/?caid=200960 ) and that they are the provider
| of the HiCA service. So it does sound like there is a closer
| link here than your comment would indicate:
|
| https://github.com/acmesh-
| official/acme.sh/issues/4659#issue...
| agwa wrote:
| Quantum is not a trusted CA. ssl.com has a white-labeled
| intermediate CA with the name "Quantum" in it, but this
| intermediate CA is operated by ssl.com under all the same
| controls as ssl.com's other intermediate CAs. Quantum has
| no ability to issue trusted certificates themselves.
| 0x0 wrote:
| So the person claiming to be the founder of "QuantumCA"
| does not possess the private key corresponding to
| https://crt.sh/?caid=200960 - can we be sure the private
| key is only accessible by ssl.com's CA system? So the
| certificates listed here aren't issued by this person,
| but by the ssl.com's system? https://crt.sh/?Identity=%25
| &iCAID=200960&exclude=expired&de...
|
| Also, why would ssl.com even create a subCA named
| "QuantumCA"? Are they in business with this person
| claiming to be the founder of "QuantumCA" who appears to
| be responsible for exploiting this acme.sh 0day? What
| does this say about ssl.com's trustworthiness? Or is the
| person in the github comments lying?
| agwa wrote:
| > _So the person claiming to be the founder of
| "QuantumCA" does not possess the private key
| corresponding to https://crt.sh/?caid=200960 - can we be
| sure the private key is only accessible by ssl.com's CA
| system? So the certificates listed here aren't issued by
| this person, but by the ssl.com's system? https://crt.sh/
| ?Identity=%25&iCAID=200960&exclude=expired&de..._
|
| Correct. You can see the Quantum intermediates listed in
| ssl.com's most recent audit statement, meaning an auditor
| has verified that ssl.com has controls to protect the
| private key: https://www.cpacanada.ca/generichandlers/CPA
| CHandler.ashx?at...
|
| (The audit could be flawed, but it's the same amount of
| assurance we have for any intermediate CA's private key -
| the fact that "QuantumCA" is in the name does not change
| the risk calculus)
|
| > _Also, why would ssl.com even create a subCA named
| "QuantumCA"? Are they in business with this person
| claiming to be the founder of "QuantumCA" who appears to
| be responsible for exploiting this acme.sh 0day? What
| does this say about ssl.com's trustworthiness? Or is the
| person in the github comments lying?_
|
| There is a business relationship between QuantumCA and
| ssl.com. QuantumCA is a reseller of ssl.com, and they've
| paid extra to ssl.com so that the certificates they
| purchase get issued from an intermediate CA named
| "QuantumCA" rather than one of ssl.com's usual
| intermediate CAs which have "ssl.com" in the name. This
| lets QuantumCA pretend to be a real CA. This is a common
| practice in the industry, and I don't think it says
| anything about the trustworthiness of ssl.com, because
| the business relationship with QuantumCA doesn't in any
| way subvert the integrity of the WebPKI since ssl.com
| retains control of the issuance. Still, I wish
| intermediate CA white-labeling were banned because it
| causes terrible confusion about who is and isn't a CA.
| 0x0 wrote:
| I find it troubling that a root CA (ssl.com) is
| apparently OK with lending their name in a business
| relationship with an actor that is actively exploiting an
| acme.sh 0day.
| mholt wrote:
| I honestly don't think anyone else knew about the RCE
| until yesterday.
| tptacek wrote:
| This feels a little bit like doubling down to find ways
| to implicate the actual CA instead of the reseller. It's
| clear how mismanagement by a real CA would make a more
| interesting story than by this random no-longer-existing
| pseudo-reseller, but I don't think there's evidence to
| support that story yet.
| 0x0 wrote:
| But it's not a random pseudo-reseller? The one github
| comment from "the founder of Quantum CA" seems to say
| they are also the creator of HiCA, which is the entity
| that was exploiting the 0day in acme.sh. And the crt.sh
| link shows an intermediate CA cert named "QuantumCA",
| signed by ssl.com.
|
| So QuantumCA == HiCA == exploiters of the acme.sh 0day,
| it's all the same entity? The intermediate CA could just
| as well be named "0dayexploitersCA"? Why is it not a huge
| concern that ssl.com is fine with operating such a
| "0dayexploitersCA" intermediate?
|
| Am I missing something here?
| predictabl3 wrote:
| Sounds about par for the course for folks that think shell is a
| productively sustainable way of writing secure or reliable
| software. Not even remotely sorry about that opinion. The gall to
| claim ACME compat, then force require a single client, all so you
| can remote execute arbitrary commands. Should be enough to ruin
| the CA, but we know how people handle things like this "oh, won't
| affect _me_ " (until it does). Seemingly just so they can avoid
| some reverse proxy rules to host the challenge endpoints at the
| right place? Holy _wow_. Oh there we go baby, eval 'ing with
| arbitrary input in the shell script, complete with lazy inproper
| string quoting. I should probably just stop before I say more
| unkind things.
|
| Was it even run through shellcheck?!
|
| Stop writing this stuff in shell people. 95% of the time I review
| _any_ (posix, nushell lacks most of these issues) shell scripts,
| it 's obvious they would fall apart the second any string
| unexpectedly had a space in it. Even scripts written by darling
| companies of HN.
| electroly wrote:
| This is a Chinese CA, I don't think there was a danger of any
| of us using it, and none of them are here to receive this
| criticism. Most of your complaints are about the shell script
| that the Chinese CA is injecting and that's... not really the
| problem at all. It's amazing that it's not literally a rootkit;
| that's what I expected when they said a Chinese CA was
| injecting a shell script that acme.sh was running. You're
| missing the forest for the trees here.
|
| This doesn't get better if they had injected a better-written
| script or a Rust program or whatever else. They could still
| have injected _anything_ , and _that 's_ the problem. The
| people who wrote acme.sh (different people than the Chinese
| CA), which has the security vulnerability that the Chinese CA
| exploited, might have some soul-searching to do.
| yjftsjthsd-h wrote:
| Shell is a bad fit for many uses (an ACME client _is_ more
| ambitious than I would build in shell, personally), but the
| things it 's good at, it's _really_ good at. I have yet to find
| anything else that 's even close to as good for glue code when
| I have a handful of tools and/or a bunch of files that I need
| to string together. Unless you're writing in Ada, I promise
| whatever language you think is better has its own sharp edges
| (if we're allowing eval, then not many languages are going to
| be safe, really).
| Bystroushaak wrote:
| I usually replace shell scripts with python (using sh module:
| https://amoffat.github.io/sh/ for calling other
| scripts/programs).
| pdimitar wrote:
| Golang is better suited for this, for my needs at least.
| ianlevesque wrote:
| Yeah, lego-acme is a solid alternative.
|
| https://github.com/go-acme/lego
| yjftsjthsd-h wrote:
| Er, are we comparing ACME clients or shell alternatives?
| cozzyd wrote:
| yeah until your scripts stop running someday because
| python...
| Bystroushaak wrote:
| Can you elaborate?
| cozzyd wrote:
| Python scripts will often break with system upgrades,
| most acutely when Python2 went away, but under many other
| circumstances as well.
| yjftsjthsd-h wrote:
| I would expect python2 to only have gone away across
| major versions of an OS, which is about as non-disruptive
| as it could have been, considering.
|
| Of course, as I write this ansible is broken on one of my
| machines for reasons that appear to stem from a python
| 3.x->3.y update, so...
| l0b0 wrote:
| How did we get here?
|
| - The barrier to entry for shell scripting is tiny. You
| basically start with `ls` in an interactive shell and end up
| writing a frickin' 500-line monstrosity within a month.
|
| - POSIX has fossilised scripting languages. I hoped we'd have
| something like PowerShell by now, but although we have some
| fine alternatives, none of them seem to be good enough to
| actually overcome the inertia of POSIX.
|
| - Doing something that looks correct is easy, but doing the
| _right_ thing is super effing hard. See looping through complex
| sets of files.
|
| - People use statements like "It's just a script", as if
| scripts are somehow easy to write. Bullshit.
| superkuh wrote:
| People write in bash because you can expect bash to be forward
| compatible with how people write bash. A bash script written by
| the average person today will run on a bash interpreter from
| 2006 (or 2023). Very few languages have the developer culture
| to achieve this in practice even if it might be possible in
| theory. And it's not like Bash doesn't get new features. It
| does constantly. But unlike, say, Rust or JS, the developers
| don't instantly start using forwards incompatible features.
| It's a different mindset. If you're writing in shell you're
| writing for longevity and compatibility.
|
| Unfortunately requiring approval and authentication from a
| random incorporated entity running a CA every 90 days is
| inherently complex. And all the ACME protocol implementations
| do is hide that inherent complexity by adding even more. CA TLS
| HTTP is very fragile but it could be made less so if corporate
| browsers didn't demand short lifetime certs-- a demand
| reasonable for businesses but not for human people or things
| expected to last longer than a few years without being touched.
| predictabl3 wrote:
| This doesn't make any sense to me. Stable Rust is forward
| compatible just as much as shell scripts are. Bash, for
| example, _does_ introduce things in newer versions and they
| 're near universally ignored.
|
| All of this also excludes that the fact that your shell
| script will be just as janky in 10 years as it is now. Again,
| I regularly, regularly see shell scripts from companies that
| HN _fawns_ over that don 't pass shellcheck and have inherent
| issues that don't occur in better scripting languages, or
| full programming languages.
|
| "Forward-compat" is not unique, and not a value proposition
| to me if it inherently means less reliable function.
| superkuh wrote:
| Here's an anecdote to provide sense. I recall when I was
| using a 3 month old (1.48.0 2020-11-19) rustc on Debian 11
| (right before it was officially released) and I kept
| running into rust software I couldn't compile because rust
| devs used bleeding edge 1.50 (2021-02-11) features. This
| wasn't a debian is old thing. The rustc was literally 3
| months old at this point. At least one of the devs was a
| person I knew on IRC. He was kind enough to re-write it
| ("plotsweep" a rust software defined radio program) using
| only 1.48 and older features. But that's the exception.
| Most rust devs assume your toolchain is from 3rd party out-
| of-repository and latest and think forwards compatibility
| is useless like you do.
|
| Shell scripts can be and often are jank. But at least it's
| stable jank. Shell devs tend to write portable, stable code
| because that's the entire point of dealing with the shell
| jank.
| predictabl3 wrote:
| Yeah, I think we're not going to agree on this. I very
| much like living in a world where I get to use stable,
| released software, provided by my distro, to build
| reliable software that is forward compatible.
|
| > This wasn't a debian is old thing.
|
| From my reading and understanding, yeah, it was.
| dundarious wrote:
| There is almost never a good reason to use constructs like eval
| in any language (and it exists in many languages), just like
| there are barely any good reasons to use constructs like
| system() in C. It appears acme.sh was running eval. I haven't
| looked into it, but the most common reason I see eval in use is
| because people don't know that you do it much simpler/more
| directly by just geting the shell to run commands that are
| variables with no issue, e.g., `doit() { printf "running:
| %s\\\n" "$*" ; "$@" ; } ; doit ls /` which works in POSIX sh,
| no bashisms. (Obviously you should only call it with at least
| the first argument being trusted)
|
| But shell is really not hard if people stick to a few easy
| guidelines:
|
| - quote all var expansion, e.g., "$var" not $var or ${var}
| (very rare to need otherwise, and never for untrusted data)
|
| - use "$@" to perfectly forward arguments (not $@, not $*, not
| "$*" except when you want to turn many arguments into one)
|
| - don't use eval
|
| - use `set -eu` and explicitly handle functions/commands that
| may benignly return/exit non-zero, e.g., `diff -U10 ./a ./b ||
| true`
|
| - use printf on untrusted strings instead of echo (just use it
| generally, I say), e.g., printf %s\\\n "$var" instead of echo
| "$var". One of the few times you want to use "$*" is with
| printf though, e.g., printf %s\\\n "$*" instead of echo "$@".
| Try them out, easy to see why, as with one thing to format and
| multiple arguments, `printf %s\\\n "$@"` is equivalent to `for
| i in "$@" ; do printf %s\\\n "$i" ; done`
|
| - when using xargs, use -0 if available, or at least -d\\\n if
| available (busybox doesn't have it for example). also usually
| want to use -r
| predictabl3 wrote:
| And then run shellcheck because even those of us that painful
| have this seared into our brains make mistakes.
|
| Or just use a scripting language that eliminates many of
| these headaches (nushell) or a real programming language.
|
| I get it, I really do. At least now I have nushell for some
| sanity, but I still find myself constantly writing a shell
| script and then realizing after a few iterations that _I
| should 've just written in nushell/Rust_.
|
| As a thought experiment, how many places rolled out acme.sh
| to prod and didn't bother code reviewing it or running it
| through shellcheck?
| dundarious wrote:
| Every language exists on this spectrum, and pretty much
| every language is clustered near shell. All languages
| require you to be "principled" in a few key areas, and many
| projects (open source _and_ closed source) don 't do a
| great job at that.
|
| At least shell is good at a specific and very useful thing
| (sequences and pipelines of other commands), and is already
| installed on most machines -- even windows nowadays since
| almost everyone has git installed. My "build system" for C
| projects is just doing the following from pwsh: `&
| "$env:GIT_INSTALL_ROOT\bin\bash.exe" .\build.sh`
|
| Most software is not great. That is the issue.
| WirelessGigabit wrote:
| This means that acme.sh has an RCE. Once the patch is in I'm
| rotating all my certs, even though I use ZeroSSL.
|
| I do wonder if what HiCA did gave possibilities to post the
| private key somewhere else?
| mholt wrote:
| If the executed script transmits the key, then yes. (But the
| script we observed does not.)
| MrStonedOne wrote:
| [dead]
| Vecr wrote:
| Even if this is just RCE in the script somehow (I doubt it, it
| can probably do anything the user running it can), it's
| horrifying. It means the certificate authority could just take
| your newly generated certificates and upload them anywhere they
| want. That's a catastrophic compromise in the TLS security model.
| tialaramex wrote:
| Somebody who cares about the acme.sh client specifically would
| be able to say, but in general it's not necessary that your
| ACME client has these keys.
|
| What the ACME protocol wants to do is hand over a CSR
| (Certificate Signing Request), and get back a certificate, and
| to achieve that it has to explain how you'll prove you're
| entitled to such a certificate.
|
| Most ACME clients will also make a suitable proof (in at least
| some cases), and also generate a suitable CSR from first
| principles, for which they will need to generate a new private
| key - but that's not a necessary part of the system, and it's
| certainly not rare to generate your own CSR, either because you
| must technically, or because your own security processes say
| _strange women, lying in ponds, distributing swords is no basis
| for a system of government_ sorry, I mean, that this key is
| private and shouldn 't be on the host running ACME services.
| paulgb wrote:
| If the CA is compromised, can't they just bypass ACME
| verification and generate a cert for your domain anyway?
| dspillett wrote:
| The CA isn't directly compromised so a third party couldn't
| generate any arbitrary certificate this way. Essentially
| though, assuming my understanding is correct, it would allow
| them to be a man-in-the-middle and take copies of the keys &
| certificates used by this tool, allowing them to use keys and
| certificates generated by that tool. Also, if such a tool is
| run by root (bad practise, but not uncommon practise) or
| other significantly privileged user, they potentially have
| access to far more.
| mholt wrote:
| Yes, but with serious consequences (it would go into public
| transparency logs, at least for CAs in most public root
| stores).
|
| If the CA can access your private key, then it can reuse (or
| worse, redistribute) it without anyone knowing.
| Vecr wrote:
| Yeah, but that's more likely to be noticed in cert
| transparency and by the website operator, as there's either a
| duplicate cert in the log or the website server does not
| work.
| diarrhea wrote:
| Why is distributing the certificate dangerous? It's public
| knowledge anyway.
| Vecr wrote:
| The private key I mean, it's really supposed to be secret on
| your computer and not even the certificate authority is
| supposed to have access to it.
| AmenBreak wrote:
| Is this part of Plan 9?
| com wrote:
| Unfortunately this is related to acme.sh, a shell script tool
| to request new and replace free certificates. So far, this
| GitHub issue is quite disturbing.
| tinus_hn wrote:
| IMO it's just a demonstration why you don't write complicated
| or security sensitive code in shell script: it's basically
| impossible to get right, there's pitfalls around every corner
| and it's extremely difficult to check for mistakes.
| pazderski wrote:
| I wouldn't say there are too many pitfalls. Quote all
| variables (acme.sh seem to do well on that part) and don't
| use eval (probably the reason for this problem).
| Eeems wrote:
| I just read the entire issue thread, yes eval was the
| reason.
| MisterTea wrote:
| No. This is not the acme editor. This is what ACME they are
| referring to:
| https://en.wikipedia.org/wiki/Automatic_Certificate_Manageme...
| j16sdiz wrote:
| <quote> ... HiCA is injecting arbitrary code/commands into the
| certificate obtaining process and acme.sh is running them on the
| client machine. </quote>
| regecks wrote:
| I think the title buries the most horrifying part of this. The
| HiCA certificate authority is relying on an RCE to do an end-run
| around the semantics of the ACME HTTP-01 validation method.
|
| Fucked up and they should be booted from every root program for
| this.
| agwa wrote:
| They aren't in any root programs. They're just taking
| certificate requests and relaying them to real CAs, which is
| why they need to exploit an RCE in the ACME client, since the
| ACME client wouldn't otherwise be able to complete the
| validations required by the actual CA.
| chunk_waffle wrote:
| When confronted they just flat out shut down the service. They
| also donated $1000 to the project, and they've redirected
| requests to their payment site to the US White House's website,
| and they're from China.
|
| They were also suggesting that user's ran the utility as
| root...
|
| All really shady...
| stavros wrote:
| Wow, that's... bold.
| [deleted]
| [deleted]
| freedomben wrote:
| Major props and thanks to the HiCA person for engaging on the
| thread even though they were getting hammered. Yes they made some
| really (damn clever but) bad implementation decisions to use an
| RCE in the client to basically hack around the entire system, and
| then compounded it with other bad decisions to redirect to the US
| White House website to stop a DDoS, but the fact that they
| engaged, admitted, and explained what was going on I thought was
| really big of them.
|
| Everybody makes mistakes people. If this person was trying to be
| malicious they would have disappeared, not come straight clean
| about it on the thread.
| clysm wrote:
| But... HiCA just disappeared. Their site is gone now.
___________________________________________________________________
(page generated 2023-06-09 23:01 UTC)