[HN Gopher] The Story of the SolarWinds Hack
___________________________________________________________________
The Story of the SolarWinds Hack
Author : bokchoi
Score : 289 points
Date : 2021-04-17 09:42 UTC (1 days ago)
(HTM) web link (www.npr.org)
(TXT) w3m dump (www.npr.org)
| netfortius wrote:
| This article is a case of [a lot of] Monday morning
| quarterback[s]. Except for Mandia, I wouldn't allow any other
| exec(s) to speak about this, publicly, as to the why and how.
| Side note: I bet Bejtlich wishes he was still in that team ;-)
| wglb wrote:
| The best set of articles in my opinion are these:
| https://ciexinc.com/blog/solarwinds-articles/zetter.html
| vzaliva wrote:
| I am curious about "compiler" attack they are mentioning. Looks
| like they compromized compiler used to build the code. Any more
| technical info on this aspect?
| georgiecasey wrote:
| I think they compromised Teamcity, or at least I've heard a lot
| of mentions of Teamcity related to this hack, I could be
| completely wrong. But it's possible they just got admin access
| to the Teamcity build server and added their code just before
| the build was deployed. So not really compromising the compiler
| per se
| 1cvmask wrote:
| It's nice how they equivocate over the ease of entry and their
| security policies:
|
| There was another unsettling report about passwords. A security
| researcher in Bangalore, India, named Vinoth Kumar told NPR that
| he had found the password to a server with SolarWinds apps and
| tools on a public message board and the password was:
| "solarwinds123." Kumar said he sent a message to SolarWinds in
| November and got an automated response back thanking him for his
| help and saying the problem had been fixed.
|
| When NPR asked SolarWinds' vice president of security, Brown,
| about this, he said that the password "had nothing to do with
| this event at all, it was a password to a FTP site." An FTP site
| is what you use to transfer files over the Internet. He said the
| password was shared by an intern and it was "not an account that
| was linked to our active directory."
| freeflight wrote:
| _> An FTP site is what you use to transfer files over the
| Internet._
|
| Afaik that was the updatesever and the password originally came
| from a public Github repo where is was stored in plaintext
| since at least June 2018 [0]
|
| [0]
| https://www.theregister.com/2020/12/16/solarwinds_github_pas...
| frombody wrote:
| The password was in a file that was committed to github IIRC.
|
| I think both the intern that posted the file and the person
| making that statement both did not realize that someone had
| given the user write access.. which in my opinion was the
| actual mistake, not the ftp or the password.
| ta9999 wrote:
| Something to remember is that these sorts of things happen in
| many organizations.
|
| You should always verify the data you get, be careful with
| complex supply chains, and avoid binaries you didn't build
| yourself. Don't skip these things just because "that's the
| way it's always been done." 10 years ago very little internet
| traffic was encrypted, security still means progress not the
| status quo.
| dogman144 wrote:
| How a Vp security can ignore a privesc risk like that is pretty
| inexcusable. Ever vuln falls on a risk mgmt spectrum but that's
| a really nonsense answer to give. Weak PW mgmt on a FTP server
| that you let interns set should raise some areas of interest.
| CyberRage wrote:
| You don't get this kind of attack because you had an exposed
| FTP server.
|
| The attack implanted malicious code into their code, learning
| the tooling, process and responsibilities of the personal.
|
| They then reversed engineered the protocol and used it in
| their backdoor to look basically the same as regular
| communications.
|
| The issue is that we blindly trust 3rd party software that is
| used by hundreds of companies. this makes SolarWinds a prime
| target, one that is worth the efforts taken in this case.
| freeflight wrote:
| > You don't get this kind of attack because you had an
| exposed FTP server.
|
| Leaking the extremely weak login credentials to your
| updateserver, trough a public Github repo, is not exactly a
| glowing endorsement of how serious security seems to have
| been taken at Solwarwinds.
|
| With stuff like that being a thing, who knows where else
| they cut corners/got lazy.
|
| _> this makes SolarWinds a prime target, one that is worth
| the efforts taken in this case._
|
| A prime target, yet apparently could still not be bothered
| to put in some minimum effort to protect themselves.
| boomboomsubban wrote:
| >You don't get this kind of attack because you had an
| exposed FTP server
|
| This kind of attack needs an entry point, and an exposed
| FTP server provides the potential for one. Whether it
| actually was the entry point is a separate matter,
| willfully ignoring one unlocked door means there's likely
| to be others.
| CyberRage wrote:
| Initial access is part of the day-to-day these days.
|
| you can't cover all entry points, it's a matter of time
| for someone to make a mistake. the fact that the
| adversary showed these extreme levels of proficiency and
| dedication tells me that the vast majority of companies
| would have fallen for that. In fact, the backdoor was
| running for months on targets like Microsoft, gov
| agencies, security companies like Malwarebytes.
|
| These companies know a thing or two about security.
|
| Today we work with "assume breach" mentality that assumes
| you are already compromised.
| dogman144 wrote:
| Yeah I mean I've read the technical FEYE writeups, and yeah
| once they had a foothold it got fairly f'ing complex (hope
| those guys don't end up in my CI/CD ever, yikes).
|
| That said, foothold for this stuff more often than note
| comes from keys on public GitHub repos (or things like
| that: simple misses that are enough to throw the door
| open).
|
| I hear things like "intern" and "totally unrelated" and
| it's the dog whistles of policy/policy enforcement failures
| of these sorts of initial ways in.
|
| So many sexy hacks start from admin123 passwords
| unfortunately
| ridaj wrote:
| I don't think they ignored it? Says it was addressed. I think
| it was OK to dispel the implication that the elaborate supply
| chain attack was allowed in the first place by sloppy pw
| practices. You don't want that to be the takeaway and then
| other companies thinking, well our pw management practice is
| really good so we don't have to worry about being a victim so
| much
| rjzzleep wrote:
| They did ignore it. I think it was over a year actually. I
| don't feel like looking it up right now but it was also
| discussed in another submission. Whatever the actual number
| was it was VERY long.
| de6u99er wrote:
| Anyone know how the software update was actually compromised in
| the first place?
| someguydave wrote:
| Tim Brown VP Security of Solarwinds said: "We check code out of
| source code control, have a TeamCity environment to kick off
| the build, and here the attacker looked for Orion to be built
| and swapped a file. It was a transient virtual machine and
| that's hard to detect," he said.
|
| https://itwire.com/security/solarwinds-speaks-out,-and-softw...
|
| So it sounds like the VM host they were running builds upon was
| compromised - or maybe JetBrains was compromised?
|
| JetBrains claims there is no evidence they were compromised
| https://blog.jetbrains.com/blog/2021/01/07/an-update-on-sola...
| lambdasquirrel wrote:
| It's kinda-sorta in the article. The hackers managed to put
| some hack into the Solarwinds build scripts.
|
| Nice writing I guess, but, what allowed them to get into the
| build scripts?
| matwood wrote:
| The wiki page on the attack speculates an Office360 account was
| hacked. Presumably it was an account from an admin, and from
| there I could see them probing until finding credentials for
| the build system.
| trashcan wrote:
| No idea if it is related, but a SAML implementation security
| issue was disclosed the same (or very close) day that the
| SolarWinds attack became public knowledge. Maybe that gave
| them access to the admin account?
| trulyme wrote:
| So they hacked MS through SW which was hacked through MS?
| Ironic if true.
| wglb wrote:
| This has been a puzzle for a while, and I am not yet satisfied
| with the answer so far. The closest to an explanation is found
| here: https://ciexinc.com/blog/solarwinds-
| articles/initial.html.
|
| I feel like this says "well, we are not really sure".
| tptacek wrote:
| _Network monitoring software is a key part of the backroom
| operations we never see. [...] By its very nature, it touches
| everything -- which is why hacking it was genius._
|
| This is frustrating to read, since plenty of people did in fact
| warn that these kinds of systems were easy targets.
| bostonsre wrote:
| Yea.. not sure I'd call it genius. Think if you ask anyone that
| is a little knowledgeable about what would be the juiciest
| target for a nation state to hack, a large portion of people
| would have said something like SolarWinds.
|
| It seems like SolarWinds should have known better themselves as
| well. There is no way that their upper management didn't know
| that they would be an amazing target for a hack. Supply chain
| attacks are not that new. Their lax security seems extremely
| negligent.
| tptacek wrote:
| Ninety-nine times out of a hundred, defenders call attacks
| "genius" as a way of subverting accountability. What makes
| this particular incident pernicious is that it already had a
| built-in deflection of accountability --- the responsibility
| for ensuring that SolarWinds was fit for purpose was diffuse;
| hundreds of giant companies with large security teams all
| believed it was someone else's job to verify that SolarWinds
| could safely deliver its functionality.
|
| I've worked with people who don't operate this way, and who
| take continuous flack from CIOs for spending resources on
| verification for COTS IT management tools. But those teams
| are, in my experience, very rare --- and the SolarWinds hack
| provides further evidence of that view.
|
| It's not a perfect predictor, but a reasonable rule of thumb:
| if you've never heard of a vendor's security team, chances
| are they barely have one. That's obviously true of... most
| vendors! So you should be careful when you select one for a
| role as sensitive as fleetwide agent-based monitoring, where
| a vulnerability or a software supply chain fuckup is going to
| create mass compromise. This seems so clear to me that it
| barely counts as insight.
| tedunangst wrote:
| Compare and contrast SolarWinds and Determina, I guess.
| comboy wrote:
| Also their security team can just be a subgroup of coders
| who have some idea how their software executes.
|
| IMHO most sane vendors who want you to install something on
| your machine make it open source and use existing tools as
| much as possible. Doing it this way also decreases chances
| of some "temporary fix" changes on even otherwise secure
| software. Companies optimize for money, management tries to
| align with company values and engineers often just have to
| follow it. It's inevitable what trade-offs will be made
| unless there's some direct negative impact. For everybody
| selling their time and not being heavily invested, ignoring
| black swans and basically "eating tons of sugar" is the
| natural move.
| wglb wrote:
| > There is no way that their upper management didn't know
| that they would be an amazing target for a hack
|
| Presumes facts not in evidence.
|
| From the nyt article
| https://www.nytimes.com/2021/02/23/opinion/solarwinds-
| hack.h... "The market loves to reward corporations for risk-
| taking when those risks are largely borne by other parties,
| like taxpayers."
|
| See articles https://ciexinc.com/blog/solarwinds-
| articles/culture.html
|
| Notice the report to SolarWinds:
| https://arcticsecurity.com/guides/2021/02/12/solarwinds-
| goin...
|
| What does that tell you about their culture?
| bostonsre wrote:
| > Presumes facts not in evidence.
|
| Yep, agreed. Its hard for me to believe that they didn't
| know they were rolling the dice on security, but just my
| intuition.
| wglb wrote:
| It appears to be stronger than that: In the article https
| ://www.bloomberg.com/news/articles/2020-12-21/solarwind..
| . (pro tip: open in firefox and clear cookies)
|
| "A former security adviser at the IT monitoring and
| network management company SolarWinds Corp. said he
| warned management of cybersecurity risks and laid out a
| plan to improve it that was ultimately ignored.
|
| In a 23-page PowerPoint presentation reviewed by
| Bloomberg News, Ian Thornton-Trump recommended to company
| executives in 2017 that SolarWinds appoint a senior
| director of cybersecurity, and said he told them that
| "the survival of the company depends on an internal
| commitment to security."
|
| The following month, he terminated his relationship with
| the company, saying he believed its leadership wasn't
| interested in making changes that would have "meaningful
| impact.""
| freeflight wrote:
| _> the juiciest target for a nation state to hack_
|
| Not even just for a nation state, it's the perfect target for
| anybody looking to gain reputation/large scale access to very
| interesting systems.
|
| Hacking the keys to the kingdom from the people who are
| supposed to protect the kingdom from hacks.
|
| In terms of "pwnage level" it can't get much "better" than
| that.
| comboy wrote:
| I was just playing with Grafana a few days ago, their cloud
| version. When you install agent, it opens up ports with
| unprotected metrics on a public IP. Opens up ports on your
| production without info about it whatsoever, because it is in
| theory a push agent. Why would you do that?
|
| Support response:
|
| "Regarding your second message about port 12345 on the Grafana
| Agent -- the HTTP server only exposes the Agent's internal
| metrics and provides an API for the Agent's status. The gRPC
| server is used for agents to communicate with one another, if
| the scraping service is used. It does not allow anyone to get
| access to their metrics.
|
| That said there's we understand the concern and upon review
| will look at documenting that agents listen on 0.0.0.0 by
| default, and that you can change it by setting
| http_listen_address and grpc_listen_address in the Agent's
| server config to 127.0.0.1: (...)"
|
| Also, sometimes I feel like I'm the only person in the world
| who is not comfortable in running tons of untrusted docker
| containers. If I put a link to a binary here and tell you to
| run it, I doubt any HN reader would. But 400MB of binaries? No
| worries, I have just the right tools to run them as root.
| kevinsundar wrote:
| Not sure what your comment about docker containers is trying
| to say. In docker you need to specify port mappings before
| ports are exposed outside the container, so this wouldn't
| happen without you explicitly knowing. Containers are by
| default untrusted and access to network and filesystem is
| mediated. In fact, if you linked to a docker container I'd
| have less qualms running than a binary.
| takeda wrote:
| The filesystem is restricted, but network generally not.
| You can have a malicious container connect to rest of your
| network. The ports don't need to be open either, it can
| phone home (like how most bots do).
|
| Certain data centers restrict access to outside network
| (i.e. Internet), with now popular public cloud. It's
| actually hard to place such restriction, because majority
| tooling, including official images are expecting to be
| connected outside.
| comboy wrote:
| Yes, you have and that is reasonable. It is assumed that it
| is safely contained environment that cannot maliciously
| affect your host system. Yet every virtualization and
| container solution we've seen so far had been exploited.
|
| "But is there a vulnerability at the moment that can be
| exploited?" "None that we know of"
|
| Containers also often get generous access rights and
| internet access, even assuming container solution being
| intact.
|
| Security of the system inside is a whole big surface of
| attack that you have little control of (unless you
| meticulously analyze each docker full fucking system image
| every time you run them), and necessarily affects your
| systems because otherwise you would have no use for this
| container.
|
| https://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=docker
| [deleted]
| wglb wrote:
| I've collected many articles about this--currently 2099 articles
| --covering lots of areas. The highest quality articles are at
| https://ciexinc.com/blog/solarwinds-articles/zetter.html.
| adolph wrote:
| How fortuitous is it that a months long investigation can be
| published right when the US announces sanctions? Great job
| National Radio!
|
| _Like razor blades in peanut butter cups_ , says CrowdStrike.
| aleister_777 wrote:
| I worked for Crowdstrike. We did this hack as our primary demo.
| Dmitri is on the Atlantic Council along with the rest of the
| Obama team.
|
| There is no more evidence to suggest Russia was involved than
| there was that Dmitri himself was for political reasons. And
| Shawn Henry will fire you for saying this out loud.
| wglb wrote:
| There have been deeper dives and better explanations than this
| NPR article: https://ciexinc.com/blog/solarwinds-
| articles/zetter.html
|
| Major investigations were completed months ago.
| covidthrow wrote:
| I fear that you're being downvoted for pointing out the even
| bigger threat of our national media's exceedingly more
| dangerous acts of propaganda, presumably because readers either
| don't recognize it or are wilfully blind to it because it
| reflects their own biases.
|
| The SolarWinds hack was a devastating attack on our
| sovereignty.
|
| And so is the other.
| encryptluks2 wrote:
| The SolarWinds hack in addition to election interference
| should easily be seen as an attack worthy of taking out Putin
| IMO.
| askl56 wrote:
| Yes, let's assassinate a world leader of the country with
| the most nukes in the world based on unfounded claims of
| election interference and a hack.
|
| Do you have any idea what America did in Russia in the
| immediate aftermath of the fall of the Soviet Union? The
| 1996 election in Russia which was majorly "interfered" with
| by Clinton: https://archive.is/R7i5u, not to mention the
| fact that most Russian hacking activities are done using
| NSA backdoors which were leaked?
|
| I'm surprised and disappointed to see such flagrant ivory
| tower imperialism on HN.
| thyrsus wrote:
| I agree completely that assasination, beyond moral
| abhorrence, would work out as well as most U.S. foreign
| policy - horrifically. The last foreign policy success
| I'm aware of was the Marshall plan.
|
| I think it's a mistake to minimize Russian influence,
| when it suffices to compare the unmitigated disaster of
| U.S. actions - both government and private sector, and at
| enormous scale - in post-Soviet Russia.
| grosswait wrote:
| Your cited article is interesting but does not support
| your assertion of major interference IMO. Policy
| maneuvers for political purposes are the norm for all
| countries right or wrong.
| sudosysgen wrote:
| Really? A government saying that they would ensure no
| "negative stories would come out", literally puppetting
| international institutions to pay out money - billions -
| for an election campaign, sent US Government agents to be
| embedded into the Yeltsin campaign right as he was
| violating every law on the books and calling in favours
| from the mafia and oligarchs, and very likely also used
| intelligence agencies to help, all of that isn't major
| interference?
|
| But targeted ads are?
| takeda wrote:
| I have a family member affected by those "targeted ads"
| and can tell you this is hardly comparable to news
| agencies writing favorably or not about someone. Some
| people are seriously fucked up because of this. Just look
| what happened on January 6th.
| sudosysgen wrote:
| For the record, I think that both are elections
| interference. But there's a far cry behind publishing
| fake news and advertising it to some demographics and
| fuelling IMF money into campaigns by the billions and
| embedding foreign advisors into a criminal election
| campaign.
| thu2111 wrote:
| Because the Russia link is so strong? Dude, everything you
| read about Russia in the American press is garbage. From
| the article:
|
| "They cleaned the crime scene so thoroughly investigators
| can't prove definitively who was behind it."
|
| That's literally in the article but it doesn't stop them
| writing the whole thing as if it's utterly proven beyond
| doubt who was behind it.
| aleister_777 wrote:
| I was speaking to some Russians at a party yesterday.
| They lamented at how Russian and American media were
| mirrors of each other.
|
| Russia uses America as the core of their propaganda just
| the same as the DNC has been using Russia as their
| boogoieman since they needed a narrative to apply to
| Trump.
|
| As someone who formerly worked for Crowdstrike and has
| performed the APT-29 demo countless times, I will say
| attribution is bullshit and people frankly have no real
| data to point fingers.
| SV_BubbleTime wrote:
| > By design, the hack appeared to work only under very specific
| circumstances. Its victims had to download the tainted update
| and then actually deploy it. That was the first condition. The
| second was that their compromised networks needed to be
| connected to the Internet, so the hackers could communicate
| with their servers.
|
| Yea, wow, thanks NPR. Hard hitting stuff right there. Those are
| "very specific circumstances" that just happen to generally
| apply to a huge percentage of hacks.
|
| I normally appreciate some stories on NPR, but you're right.
| This is a narrative piece.
| meowface wrote:
| That's a perfectly valid paragraph. In a decent environment,
| outbound internet access should be restricted to only the
| hosts / networks / ports that require it. Especially for
| server environments. Many servers running the backdoored
| Orion probably tried to beacon but failed for that reason.
| (And I'd assume the backdoor would probably first verify
| outbound internet access so that the failed beacon doesn't
| generate a firewall/ACL deny event that a security team might
| detect.)
|
| Plus, the article is written to condense technical
| information into something that's as layman-friendly as
| possible. The specific malicious update has to be downloaded,
| and also installed, and also running on a server which can
| reach out to anything on the internet. Their point is that
| there are only going to be so many servers that both use this
| software and meet those conditions, and that's in part why
| the backdoor took so long to identify. This is maybe a little
| obvious to people with infosec knowledge, but definitely not
| obvious to their target audience.
|
| The article timing is interesting, but I don't think a
| coincidence is that unlikely. If you read the whole article,
| it covers enough that I could see it taking months to make.
|
| I don't think coordination with the government is that
| unlikely, either, or perhaps just a pragmatic editorial
| decision ("everyone knows sanctions are likely going to be
| placed sometime in the next few months, and maybe we should
| wait until then so we can include those details in the
| story"). Both of those scenarios are more likely than a
| coincidence, probably - but, either way, I think your post
| seems overly cynical in general.
| lanstin wrote:
| I whitelist my networks (by port and host name, At least
| until TLS 1.3 removes the visible SNI) and my top denies
| list is very interesting. They have access to bits and
| pieces of the internet (especially PyPi damn you runtime
| downloads) and all connections to an allowed port will
| succeed (they will just be closed after the SNI or IP check
| fails).
|
| Nonetheless the article was correct the hack did not need
| bizarre or rare circumstances to take effect.
| trulyme wrote:
| Wait... What do you mean by PyPi runtime downloads???
| meowface wrote:
| Not exactly sure what they meant, but maybe just that
| it's often hard to install a single Python package if the
| machine can't reach PyPI. If you download a tarball or
| wheel of some Python package on a different computer and
| transfer it to the isolated computer, there's a pretty
| good chance the package is going to have at least one
| third-party sub-dependency, in which case trying to
| install it with pip or setup.py will cause it to try to
| install more packages from PyPI.
|
| Docker's a decent way to address this, since you install
| the dependencies when building the image rather than when
| running the container, so you can build the image
| elsewhere or through CI and run the container on any
| server without having to allow access to package
| management repo servers.
| Godel_unicode wrote:
| python -m venv temp && temp/scripts/activate && python -m
| pip install -r requirements.txt && tar -czf
| temp/Lib/site-packages
|
| You're ready for offline install!
| cmclaughlin wrote:
| The article vaguely describes the build system being compromised.
| Have any details been published to indicate what build systems
| they were running and what the exploits were there?
| wglb wrote:
| There is a lot of discussion about that:
| https://ciexinc.com/blog/solarwinds-articles/drilldown.html
| ridaj wrote:
| I agree with the parallels with aviation regulation, there needs
| to be something forcing a supplier's hand to solve this. The way
| to protect against supply chain attacks is to invest in a
| security-hardened build system (eg don't build releases on dev
| workstations, do them on build farms by build software that is
| the only thing able to access the release signing keys). This
| costs too much for most companies, so if they don't have the
| obligation to build it, they'll do features instead.
| Godel_unicode wrote:
| > do them on build farms by build software that is the only
| thing able to access the release signing keys
|
| You're aware that this is exactly what solarwinds did, right?
| ridaj wrote:
| Well and then harden _that_ obv
| Godel_unicode wrote:
| Which they did. The problem of hardening your build
| infrastructure against someone who has admin access for
| months is... non-trivial.
|
| This boils down to the question of should average companies
| be including the Russian intelligence services in their
| threat model? To paraphrase James Mickens great USENIX
| paper, if your threat model includes the SVR, you're going
| to be SVR'd upon.
| Veserv wrote:
| First, you do not need to be the Russian intelligence
| services to pull off this attack. Given prevailing trends
| in the vulnerabilities market this sort of attack would
| cost at most $1M to pull of which puts it within the
| capabilities of maybe ~50,000,000 _individuals_ worldwide
| let alone organizations. If the SVR is anything like the
| CIA they are probably running at least 1,000 programs of
| similar scale simultaneously, so it is not as if the
| attack was supported by the full weight of the Russian
| intelligence services.
|
| Second, a company's threat model should include entities
| that want to attack them. Given that they are claiming
| the SVR wanted to and did attack them, it would be
| ridiculous to not include them since that would be
| empirical evidence that they are an actual threat actor.
| Even if we were to ignore empirical evidence any company
| like SolarWinds that sells to wide swaths of government
| agencies in critical capacities should absolutely be
| including foreign intelligence services in their threat
| models and should probably be required to demonstrate
| effectiveness against attacks funded to at least the
| $100M level since only at that level does it start to
| actually get problematic for state actors to run
| operations.
| wglb wrote:
| I'm not convinced about the arguments of cost. There are
| a whole lot of presumptions in that chain of reasoning.
| The initial vector seems to not require any high-prices
| vulnerabilities, but simple authorization by pass, e.g.,
| bypassing 2FA. Which could well have been a root account.
| From there, get the keys that Duo depends on, then you
| own the whole thing.
|
| I have always argued that doing what I call "defense by
| presumed motive". The logic would have been "ok, UNC2452
| wants to access DHS hacker's email. I'll go after
| SolarWinds". Better spend your energy on basic security
| principles.
| Godel_unicode wrote:
| If you want to see the difference between just throwing
| money at the problem and an actual tier 1 threat, compare
| this with the Chinese iPhone 0day from fall of 2019.
| Probably a multimillion dollar exploit, with terrible
| quality and no opsec on the c2 side . Just spending money
| doesn't get you the kind of expertise that's needed to
| pull off something like this.
|
| It's a bit like arguing Bill Gates is a serious threat to
| any naval power because he can afford to buy a nuclear
| attack sub; there's more to it than that.
| kerenua wrote:
| how are you smart
| SuchAnonMuchWow wrote:
| pro tip: don't get hacked in the first place, it will avoid
| trouble down the line ! /s
| wglb wrote:
| Wise defensive teams operate under the dictum "Presume
| Breach".
| amaccuish wrote:
| > But as CrowdStrike's decryption program chewed its way through
| the zeroes and ones, Meyers' heart sank. The crime scene was a
| bust. It had been wiped down
|
| That's a lot of words to say, we don't know who did it. I had a
| quick look but couldn't find anything, why are the fingers being
| pointed at Russia?
| ipsin wrote:
| Tool use, essentially. Of course that could be spoofed, but I
| think that's the origin of the claim.
|
| https://www.reuters.com/article/us-global-cyber-solarwinds/s...
| joe_the_user wrote:
| Tools can be stolen. In fact, if we're claiming whoever did
| this was a super-genius, they would have stolen or spoofed
| the tools they used to point at someone else. Unless they
| were Russia being so clever they were pretending to be
| someone pretending to be Russia!
|
| Edit: Your link show Kaspersky labs making the claim that
| this was the FSB. Yet the West also claims Kaspersky is
| controlled _by_ FSB! Well, you could say "they should know".
| Or maybe they want to humor the West so their ban will be
| lifted. Or maybe they aren't controlled by the FSB at all.
| But if the West can't figure that out, how do they expect to
| figure out the true origin of the hack.
|
| "A riddle wrapped in mystery, inside an enigma"
| aleister_777 wrote:
| I worked for Crowdstrike. We did this hack as our primary demo.
| Dmitri is on the Atlantic Council along with the rest of the
| Obama team. There is no more evidence to suggest Russia was
| involved than there was that Dmitri himself was for political
| reasons. And Shawn Henry will fire you for saying this out
| loud.
| wglb wrote:
| There has been considerable coverage of this attribution:
| https://ciexinc.com/blog/solarwinds-articles/attribution.htm...
| amaccuish wrote:
| The reply from ipsin was much more helpful. This list is
| mostly full of articles in general about the Solarwinds
| attack and others including "Supernova malware clues link
| Chinese threat group Spiral to SolarWinds server hacks
| (ZDNet)".
| sorokod wrote:
| _" The tradecraft was phenomenal"_
|
| Indeed, consider Figure 5 here [1]. A truly diabolical
| mastermind.
|
| But seriously, the article looks like window dressing for common
| incompetence.
|
| [1]
| https://www.microsoft.com/security/blog/2020/12/18/analyzing...
| wglb wrote:
| A common way to dodge accountability is to exaggerate the size
| of the enemy and his cleverness and goodness he had 1000
| developers working on it.
|
| Nonetheless, there are some things that are kind of impressive.
| Inserting their own code into the build process without
| touching any file.
|
| But the real level of skill, I think, is the operational
| discipline exercised by the attackers. For example, waiting two
| weeks before doing anything, erasing traces of what they did,
| and targeting very specific sites.
| Agingcoder wrote:
| Security is not my line of work, but back in the days I spent
| a while exploring the field (I especially enjoyed reverse
| engineering / subverting software ).
|
| Why is this 'operational discipline' remarkable? I'd expect
| this to be common sense (including waiting for two weeks) ,
| especially if I was a state actor and had spent a whole bunch
| of money on it? Or do you mean that the vast majority of
| hacking operations don't even bother to do this?
| wglb wrote:
| Contrast that with the smash and grab of the exchange
| vulnerability that followed.
|
| Yes, many operations make some kind of error that gives the
| whole thing away
| dmcgee wrote:
| This is the line that got me:
|
| >And so we are fairly broadly deployed software and where we
| enjoy administrative privileges in customer environments.
|
| There is a lot of talk about shoring up security practices by
| many of the people quoted here. But something that would be hard
| to admit is that maybe they should not have administrative
| privileges in customer environments. Maybe they should not
| install agents on your machine. They would never recommend you to
| do so with anyone else, except them of course, because you can
| trust them.
| wglb wrote:
| And you would think that a bit of analysis would be done on
| software and the company that built it for something that you
| install and give it full administrative control.
| Craighead wrote:
| But... then that would reveal how the software functions
| leading to possible copying, which cant be permitted
| thyrsus wrote:
| I don't like the Solarwinds Linux agent. When I last looked,
| there was still an sh syntax error in the cron job it installed
| (look for a file named 1 in your root directory) and I couldn't
| reach anyone who could understand my bug report. It also
| frequently exhausted the space in its log partition. I replaced
| the agent with SNMP 3 read only access. I don't believe I'm
| authorized to describe my employer's current monitoring
| posture.
| andix wrote:
| I'm quite sure there are a lot of attacks like that. Most of them
| just never get noticed.
|
| The best backdoors are those, which are never found.
| boomboomsubban wrote:
| Here's an excellent article on the topic, this is a huge uproar
| on an everyday occurrence.
|
| https://blog.thinkst.com/p/if-nsa-has-been-hacking-everythin...
| makomk wrote:
| Like, say, that backdoor someone wrote an article about
| recently which ran from RAM and had a sophisticated self-
| destruct mechanism that erased all traces if anyone tried to
| dump its memory? I wonder how many companies had exploits like
| that which they either didn't notice or didn't have the
| sophistication to actually catch and dump.
| tkinom wrote:
| There are defences for this: If one controls/monitors for
| every app in system for network access, as soon as any
| unusually network access are triggered, it is investigated
| and block.
|
| In my home windows setup, only windows defender, firefox and
| chrome are allowed out going internet access in regular base.
| Everything else are blocked.
|
| Windows update are only allowed when I in the mood for it
| (~once a year). Anyone can do this easily by control
| srvhost.exe 's internet access with windows firewall app.
| sundarurfriend wrote:
| > If one controls/monitors for every app in system
|
| Quis custodiet ipsos custodes? Who monitors the monitor?
|
| What happens when it is compromised, loopholed through,
| gets its inputs tampered with, etc.? For a home setup and
| its threat model, this sounds a simple, workable plan. When
| you're dealing with attacks of the level of sophistication
| described in the OP, trusting trust [1] becomes complicated
| and difficult.
|
| [1] http://users.ece.cmu.edu/~ganger/712.fall02/papers/p761
| -thom...
| pitaj wrote:
| Only pulling security updates once a year probably puts you
| at higher risk than you're protecting against by
| firewalling outbound traffic.
| thyrsus wrote:
| I think you mean "blocked, then investigated", which it
| sounds like you're doing. A company running a large variety
| of software - particularly third party - needs to have
| staffing sufficient to such investigations.
| CyberRage wrote:
| Well, usually highly targeted attacks are orchestrated with a
| reason.
|
| You want to leverage your access to perform actions because you
| got get revealed\blocked even not intentionally.
|
| Once you act, let's deploy some ransom, wipe some data, shut
| down power plants, you will be shown.
|
| Keeping a perfectly stealth backdoor for years as environments,
| software and personal change is extremely difficult.
| cyberlab wrote:
| > The routine update, it turns out, is no longer so routine
|
| Is there the rare case that we shouldn't update because the
| update could contain a malicious payload? If the update gets
| served over plaintext HTTP I would treat it as suspicious and may
| even block it from connecting at all. I run the risk of having
| outdated software, but that can be addressed by storing the
| software in a machine that's not connected to The Internet in any
| way, so it can't really do anything/talk to a C2 server (if
| someone does decide to execute an 0day with the software or
| inject malicious code via a rogue update).
| [deleted]
| covidthrow wrote:
| Plaintext transport doesn't matter if at least _one_ part of
| the payload chain is cryptographically protected /verified.
|
| If you have a machine that's air-gapped and its only IO is
| strictly humans (read: keyboard/screen, _not_ USB or other
| electronic means) then your weak point is the human, so center
| your security around that. You can look at security of lottery
| machines to get a good idea how that 's handled.
|
| But if you're updating the machine with updates, then it
| doesn't really fit that criteria, soooo....
| trynton wrote:
| @cyberlab: "Is there the rare case that we shouldn't update
| because the update could contain a malicious payload?"
|
| You don't ever update your security "computers" from some
| third-party outsourcer. What you do is have your own people
| constantly probing their own systems for potential
| vulnerability and patching it themselves.
|
| * jeez ..SolarWinds run their stuff on FTP and "active
| directory". It's got to be a joke.
|
| * "computers" .. not allowed to use the 'W' word ;]
| trynton wrote:
| Don't put remote control software* on your security
| infrastructure such that when a third party gets hacked, your
| whole network is exposed.
|
| * The presence of such software making it that much more easy to
| hijack your "computers"
|
| * Please "Hacker News", no more anti-Russian neocon propaganda.
| Who's really to blame is the idiot that put that configuration
| in, in the first place.
| arminiusreturns wrote:
| Let me introduce you to PTECH and see if you still think
| Solarwinds was worse. Warning, a conspiracy rabbit hole lies this
| way, proceed with caution, lest your view of the world be
| challenged.
|
| A start: https://www.youtube.com/watch?v=UuZRMpt_Tas&t=195
| matkoniecz wrote:
| Is there something about that available in a serious form? (not
| a movie, especially Youtube movie)
| EMM_386 wrote:
| > Is there something about that available in a serious form?
| (not a movie, especially Youtube movie)
|
| Wall Street Journal:
|
| https://www.wsj.com/articles/SB1039184086357188113
|
| Wikipedia:
|
| https://en.wikipedia.org/wiki/Ptech
| arminiusreturns wrote:
| edit: The topic level link in the youtube description sends
| you straight to the list of documentation...
|
| 1. http://en.wikipedia.org/wiki/Ptech
|
| 2. https://archive.org/details/GunsNButterIndiraSinghPtechAnd
| Th...
|
| 3. http://masshightech.bizjournals.com/masshightech/stories/2
| 00...
|
| 4. http://archive.is/RuleH
|
| 5. http://web.archive.org/web/20080905224929/http://www.theam
| er...
|
| 6. http://web.archive.org/web/20080820045652/http://www.total
| 91...
|
| 7. http://web.archive.org/web/20080515202659/http://www.judic
| ia...
|
| 8. http://web.archive.org/web/20081015113454/http://911citize
| ns...
|
| 9. http://web.archive.org/web/20080915185702/http://counterte
| rr...
|
| 10. http://rememberingmichael.wordpress.com/2008/03/20/in-
| memory...
|
| 11. http://www.abovetopsecret.com/forum/thread183761/pg1
| matkoniecz wrote:
| > hand-hold on an important topic
|
| I assume that things available only as youtube video,
| unavailable in a text form are unimportant.
|
| (not all things in text form are important, but it is a
| nice filter that basically always works)
| arminiusreturns wrote:
| I think that is a very limited and closed mindset that
| inherently precludes a lot of first hand source
| information.
|
| something about leading a horse to water I suppose
| lanstin wrote:
| The first and most important skill needed for the
| internet to be a net gain for a persons understanding is
| the ability to quickly develop and modify heuristics to
| filter out bad data. Filtering good data is fine, the
| truth repeats itself in many formulations, but believing
| bad data or even spending too long noticing it is bad and
| you are worse off than just using a library and books.
| thanks4linx wrote:
| thx for the linx haters gonna hate
| tomComb wrote:
| Given that audio and video are much less likely than written
| works to include references and are harder to reference
| themselves, I think they are poor evidence of anything
| contentions such as a conspiracy theory.
| miohtama wrote:
| How much value SolarWinds shareholders have lost because of this?
| If it is not number one incentive for investors to fix, then
| there won't be change in business practices. This is why GDPR in
| the EU has (some) teeth.
| wglb wrote:
| There is a lot of legal action about this. It isn't clear that
| the stock is long term affected:
| https://ciexinc.com/blog/solarwinds-articles/stock.html
| bane wrote:
| I think the most important thing the SolarWinds hack has revealed
| is that the massive pile of paperwork that has to be filled out,
| full of security controls, to accredit system for government
| systems, is fairly useless. It's the digital equivalent of the
| Great Wall of China. Designed by bureaucrats, impressive in size,
| a massive effort, and ultimately not going to stop the Mongols
| anyways. Security paperwork is not security.
|
| More important I think is that the months and months it takes to
| usher things through the process forces things to be out of date
| which in itself creates security problems.
|
| An actual audit of the source code + running it in an
| instrumented live test environment to capture behavior is far
| better.
| asperous wrote:
| They didn't prevent this specific hack so they are completely
| useless?
| DecoPerson wrote:
| They didn't say "completely useless," they said "fairly
| useless."
|
| If you look at this case even briefly, you should come to the
| conclusion that the "security paperwork" is fairly useless.
|
| An FTP server compromised because of a terrible password
| policy? No suspicious activity alerts of any kind? Executives
| who (based on their comments) are clearly ignorant of what
| makes software actually secure?
|
| What is the paperwork able to prevent, if it can't prevent
| such fundamental problems?
| bane wrote:
| Exactly. It's wonderful that there's a few thousand
| controls for things like password length. But things that
| cause massive security nightmares like "does the CTO care
| about security?" or "has a pentest team reviewed and
| audited the source?" are so fundamental and should be
| triggered as any new system touches more and more systems.
|
| "Does this software touch literally every other system on
| the network?" should be a question that triggers a much
| more rigorous and deeply technical evaluation and review.
|
| But the current processes don't work that way, they purely
| paperwork drills that often demonstrably make systems less
| safe.
| seereadhack wrote:
| How might you compartmentalize admin access the whole way down
| the stack at the enterprise level? What if you were to start from
| scratch?
| williesleg wrote:
| Yay china and russia!
| airstrike wrote:
| To me the "worst nightmare" was a story in the NYT about a
| hypothetical concerted attack against healthcare infrastructure,
| transit and more. Sadly I can't seem to find the link but it was
| a few years ago...
| vmh1928 wrote:
| Not so hypothetical.
|
| https://www.nytimes.com/2020/12/03/us/politics/vaccine-cyber...
|
| That's from December. Still going on.
|
| https://www.reuters.com/article/us-health-coronavirus-vaccin...
| germinalphrase wrote:
| "A 'Worst Nightmare' cyberattack" that we all... just take in
| stride? Either the consequences are themselves clandestine, or
| cyberattacks aren't as meaningful as our headlines would
| indicate.
| exporectomy wrote:
| The worst nightmare of the vice president of security at
| SolarWinds, not of the average person.
| wglb wrote:
| Unclear to me how worried he is. See https://www.turn-
| keytechnologies.com/blog/article/solarwinds... and other
| articles at https://ciexinc.com/blog/solarwinds-
| articles/culture.html.
| aleister_777 wrote:
| At least 18k+ networks were compromised. I bet _average_
| people got caught in that net.
| zaphar wrote:
| An attack directed at your own govt is potentially a
| nightmare for the average individual. If the wrong
| information is stolen it could be used much farther down the
| road. Your govt may find itself at a disadvantage at a
| critical moment.
|
| I'm a sense it's only not a nightmare if you aren't paying
| attention.
| exporectomy wrote:
| A worst nightmare for normal people would be something like
| a foreign country hacking all the US nuclear missiles and
| launching them against themselves. You know, an actual war
| caused by it, not just some minor intelligence advantage
| with no particular direct effect on normal people.
| dogman144 wrote:
| Clandestine I think. Immediate reaction steps to this from CISA
| were pretty unprecedented; a govt-wide unpluggening on a Sunday
| night of a specific vendor doesn't happen a lot.
| PeterisP wrote:
| We can compare and contrast with the effects of NotPetya,
| which caused widespread obvious economic damage (e.g. Maersk
| shipping and Merck losses) - due to the number of affected
| companies, Solarwinds had the potential to be worse, but I'm
| not sure if you can be more destructive than that without it
| being obviously visible.
| germinalphrase wrote:
| Perhaps the damage is just not visible yet. The sand has
| not been tossed in the gear box. The blue prints have not
| been built. Maybe it's a precursor event to a longer
| decline.
| dogman144 wrote:
| Two different attacks though. One was ransomware run amok,
| one was targeted supply chain attack to conduct espionage.
| PeterisP wrote:
| The attacks have more similarities than differences.
|
| First, to correct a common misconception, NotPetya
| definitely wasn't ransomware run amok - it was designed
| to look like the previously popular Petya ransomware, but
| the actual ransom and decryption key processing mechanism
| was removed as that wasn't its purpose. It was
| masquerading as ransomware, but it wasn't ransomware, it
| just destroys data by encrypting it with a non-
| recoverable key.
|
| Just as Solarwinds, NotPetya also was a targeted supply
| chain attack - it was deployed through updates from a
| previously hacked accounting/tax software company
| "Intellect Service" to all their customers in Ukraine,
| which also included many multinational companies which
| had their finance depts file tax reports in Ukraine; and
| just as Solarwinds, NotPetya is attributed to Russian
| government.
|
| The main difference is that, as you say, it seems that
| Solarwinds was (at least at the stage it was detected)
| used only for espionage, while NotPetya was designed for
| pure destruction.
| dogman144 wrote:
| Definitely correct in that NP originated a supply chain
| attack on that vendor in Kiev, I had forgotten and good
| catch.
|
| NP, as Maersk and co experienced was definitely rware (a
| variant, sure) run amok however. It's industry consensus
| that the attacker either a) didn't think of the possible
| Global blast radius or b) thought of the blast radius but
| didn't plan for how bad it would get.
|
| In a sense, SW might reflect a more mature approach:
| consider the network spread, use a different exploit and
| intent - spyware for espionage vs rware variant for
| destruction.
|
| That said, very different exploits and intents were used.
| chevill wrote:
| >I'm not sure if you can be more destructive than that
| without it being obviously visible.
|
| I don't know if it the damage was greater than NotPetya but
| you definitely can have something more destructive without
| it being immediately apparent. If you lose credit card
| numbers and PII from your customers you HAVE to report it
| to the public but there are different rules for the loss of
| incredibly valuable intellectual property.
| rst wrote:
| The Biden administration just announced sanctions against the
| Russian government for their (presumed) responsibility for the
| SolarWinds attack. They're not shrugging this one off.
|
| https://www.theverge.com/2021/4/15/22385371/russia-sanctions...
|
| (Which is itself a bit odd. The US has argued in other contexts
| for "cyber norms" which would allow pure espionage operations,
| but put more restrictions on attacks. And so far as anyone can
| tell so far, SolarWinds was a pure espionage operation -- using
| tools that could be repurposed to do something else, but you
| could say that about a lot of operations in this sphere,
| including US operations that our government wants everyone
| _else_ to shrug off. Yet here are the sanctions. I expect the
| "norms" push, to the extent that the current administration
| still wants to pursue it, will take some kind of a hit...)
___________________________________________________________________
(page generated 2021-04-18 23:03 UTC)