[HN Gopher] Cybercriminals pose as "helpful" Stack Overflow user...
___________________________________________________________________
Cybercriminals pose as "helpful" Stack Overflow users to push
malware
Author : entuno
Score : 126 points
Date : 2024-05-30 07:53 UTC (15 hours ago)
(HTM) web link (www.bleepingcomputer.com)
(TXT) w3m dump (www.bleepingcomputer.com)
| entuno wrote:
| Certainly not a new thing; but a good reminder to be very careful
| when copy and pasting code from Stackoverflow. Surprised that
| they'd not using a load of bots to upvote the malware-laden
| answers to make them seem more legitimate.
| dgellow wrote:
| I guess that would increase scrutiny too?
| entuno wrote:
| Any unusual voting pattern would certainly make it more
| likely that Stackoverflow noticed and investigated. But I
| wonder how many people are likely to try downloading a
| library they've never heard of from a post with zero upvotes?
|
| Of course, once an LLM scrapes that answer and regurgitates
| it, then you'd never know.
| ChrisMarshallNY wrote:
| I've found that LLMs develop hallucinations from
| questionable answers.
|
| Since Swift allows extensions of fundamental types, these
| hallucinations can be quite convincing.
| vertis wrote:
| A truly patient malware author wouldn't even necessarily
| need to make it obvious. You could upvote something over
| the course of six months and still have a very negative
| impact.
|
| Hell, you could create a library, not have malware in it,
| answer the questions, wait six months, add the malware. Oh
| wait, I'm not a criminal. Forget I said that.
|
| Patience is unlikely for individuals, but it becomes a real
| concern when you come to the point of state-based actors.
|
| I regularly use Python libraries in my work that emulate
| Excel. and they're so often out of date, they're so
| abandoned/unloved for the most part because the people that
| write them eventually move on and they don't have a big
| enough user base.
|
| That means that they're ripe for somebody inserting stuff.
| Worse Excel is usually used by big companies, so it's a
| target as well.
| ChrisMarshallNY wrote:
| Ugh, but not particularly new.
|
| Users have been pushing their work in answers for years. Not a
| surprise that some are malicious.
|
| In my case, I never bother looking at answers that prescribe a
| dependency. I find that it's _very_ important that I _completely
| understand_ any code that I use from SO (or any other source),
| and I usually change the code to better fit my specific use case
| (and it's usually a pretty small code sample).
| ChrisRR wrote:
| I suppose this is the one time where stackoverflow being a
| cesspool is actually useful. Any attempt to post malware would
| just be marked as duplicate and closed instantly
| entuno wrote:
| I was refraining from making a comment that these would be easy
| to spot because they were actually being helpful and
| (apparently) answering the question..
| tux3 wrote:
| Only questions are closed as duplicates. Stackoverflow is
| strict and pedantic to the point of being very unfriendly to
| newbies, but it doesn't close things at random.
|
| If you want to criticize Stackoverflow, there's plenty of solid
| ground to do that. No need to say things that aren't.
| chx wrote:
| > Stackoverflow is strict and pedantic to the point of being
| very unfriendly to newbies
|
| at what point do you stop being a newbie? I have been on SO
| for 14 years and have found "lately" or the last five or so
| years to be extraordinarily unfriendly. I am in the middle of
| solving a complex problem so I posit a question but it's
| stripped to bare bones to make it easier to answer -- and the
| answer is almost always is not what I seek but instead "why
| are you doing this". Aaaaargh. I would need to post War &
| Peace to make you understand why so I just go and delete the
| question when this happens which is too often. Example:
| https://stackoverflow.com/q/77202800/308851 lots of comments
| asking why I am doing this were deleted but downvote remains.
| I kept this one up for whatever reason although downvotes
| usually are enough to make me delete a question.
|
| My advice: if you can't help then stay away from the
| question. Alas, this likely won't reach the hopped-on-SO-
| power idiots but it's worth a try.
| croemer wrote:
| My strategy is to wait and see. Initially downvoted
| questions can become quite popular. The people doing the
| downvoting on new questions are a minority. Majority of
| traffic and votes comes over months from search engines.
| throwaway_serp wrote:
| That becomes really fun when gpt takes knowledge from
| stackoverflow
| belinder wrote:
| Yes, I would be a lot more concerned about that than people
| finding stuff on stackoverflow. I can see copilot or some other
| gpt "suggest" stuff like this soon
| bheadmaster wrote:
| The root of the problem is allowing pip to execute arbitrary code
| when installing a package from PyPI (default package index),
| combined with the complete lack of vetting of PyPI packages. Even
| something as simple as mistyping a package name can cause
| malicious code to be executed on your computer.
|
| Installing packages (i.e. source code) for programming languages
| should not execute arbitrary code.
| throwaway290 wrote:
| Same is true about NPM and post-install scripts.
|
| Actually it is even worse because JS is more popular and
| average JS dependency tree is so much more massive. Any tiny
| forgotten transitive dependency of a dependency (or dev
| dependency) 15 layers deep can pull this off. Total
| leftpadization is not a thing on pypi due to different culture
| mrspuratic wrote:
| "curl ... | bash" makes me run away from a project.
|
| There is the reductive argument that basically everything you
| download is arbitrary code, but throwing away the code that is
| run seems uniquely silly.
|
| https://news.ycombinator.com/item?id=21490151
| avz wrote:
| Agreed. Perhaps the most prominent example of this shameful
| behavior is this one:
|
| curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs |
| sh
|
| Source: https://www.rust-lang.org/tools/install
| hulitu wrote:
| Haha. This is the most secure part. When you start
| compiling, rust will happily download random "crates" and
| include them in your program, because "dependencies".
| chowells wrote:
| "curl | bash" is the exact same threat vector as "download
| this compiled installer". I wonder why that doesn't result in
| the same hyperbolic responses.
| blacksmith_tb wrote:
| Pipe to bash install approaches at least can just be snagged
| with wget to inspect the script without running it, lots of
| package managers seem harder than that to inspect/predict?
| Not that there couldn't be sneaky things you'd miss, but at
| least obvious ones might be detectable...
| fragmede wrote:
| alright, do curl | tee saved.sh | sh
|
| then
| lxgr wrote:
| Good idea, unless the attacker serves you just more code
| that fetches and executes a one-off script from memory (and
| the download link for that is dynamic and only valid once).
| fragmede wrote:
| we could go down a rabbit hole of various exploits and
| defenses, but at end of the day it all comes down to
| trust. If you don't trust the source of the code, it
| doesn't matter if it comes in via curl or a .deb signed
| with GPG, you're still trying to run untrusted code. If
| your threat model is such that you don't want to do that,
| don't do that. No one's forcing anybody to run curl |
| sudo bash at gunpoint.
| yeputons wrote:
| Does installation matter at all? Once you've installed a
| package, you're very likely to immediately include it in the
| project, build, and run. Any malicious code can easily run
| during the package's initialization in your app.
|
| Seems like prohibiting arbitrary code in installation scripts
| would only help with issues like Bumblebee's `rm -rf`:
| https://github.com/MrMEEE/bumblebee-Old-and-abbandoned/issue...
| bheadmaster wrote:
| > Does installation matter at all? Once you've installed a
| package, you're very likely to immediately include it in the
| project, build, and run. Any malicious code can easily run
| during the package's initialization in your app.
|
| True, but in the case I've mentioned, if you've mistyped the
| name of the package, you can safely uninstall it without any
| issues.
|
| Furthermore, code is often ran in (sort of) sandboxed
| environments like Docker during development, in which case
| arbitrary code on runtime is less dangerous than arbitrary
| code on install time.
| lxgr wrote:
| Is that not the case here? I'm not sure if pip has a
| mechanism similar to deb's pre/post-install hooks.
|
| > Furthermore, code is often ran in (sort of) sandboxed
| environments like Docker during development, in which case
| arbitrary code on runtime is less dangerous than arbitrary
| code on install time.
|
| Wouldn't the package install then also happen in a Docker
| container anyway, negating the problem? Or how would you
| install a package to your host environment and then use it
| from within a container?
| plonk wrote:
| Sounds like a good reason to generalize dev containers.
| At least both dev and deployment environment will be
| somewhat isolated. And the worst you can do from the dev
| container is commit bad code, which can be seen in code
| review.
| kwertzzz wrote:
| This is true, but you can inspect the package (and its
| dependencies) once installed and before importing it. Now it
| is an all manual process (with default tools, as far as I
| know), which contributes to the current state that only very
| few people inspect their packages.
|
| It can give people also a second change to notice, e.g. the
| typo in the package name.
| Zhyl wrote:
| You need an analogue of Linux distributions (Python
| distribution? Making choosing a repo a separate and deliberate
| step?).
|
| That way you could pick a repo based on your risk appetite
| rather than needing to trust PyPI. Debian style python for the
| cautious and AUR style python for the bleeding edge and
| reckless.
| bheadmaster wrote:
| Not really - just downloading files and storing them on the
| filesystem would be good enough. No need to run an arbitrary
| setup.py script.
|
| But setup.py is part of the Python's legacy, from before PyPI
| even existed.
| Snild wrote:
| Isn't that just already my linux dist, if I want to?
|
| I can install python-matplotlib from my package manager, and
| it appears in my python path. I never bother with venvs.
|
| Maybe my needs aren't advanced (or exotic) enough.
| mdhb wrote:
| This.. AND... dev machines should be isolated environments via
| something like GitHub Codespaces, local docker containers or
| Google's IDX.
|
| You also shouldn't be able to push code directly to anywhere
| other than a centralized repository. All the build stuff should
| happen in a dedicated and independent process.
|
| From a security point of view you need to accept that the idea
| of running into a malicious package as a direct or indirect
| dependency is not only not zero but fairly realistic and you
| should try to limit the blast radius as much as possible for
| when that does happen.
| nottorp wrote:
| Just air gap the dev machines.
|
| But an attacker could infiltrate them by manipulating
| magnetic fields to generate keystrokes / mouse moves so also
| faraday cage them.
| mdhb wrote:
| I don't know why you're trying to make this some kind of
| gotcha. What I said was absolutely 100% standard security
| advice. You're just making yourself look silly here.
| appplication wrote:
| This is a valid and significant issue, though for our internal
| packages running arbitrary code on execution has been a
| lifesaver. Think: complex distributed systems where all nodes
| need to be synchronized with the version on the driver.
| elric wrote:
| Other languages with dependency management frameworks (Java
| with Maven Central) go about this a little different: you can
| download as many evil dependencies as you like, but unless you
| actively execute them, nothing will happen. There are some
| caveats, especially with nasty frameworks that scan the entire
| classpath, but that's a different issue.
| hulitu wrote:
| > Installing packages (i.e. source code) for programming
| languages should not execute arbitrary code.
|
| Tell that to JS, Python and Rust people.
|
| But hey, we are secure. /s
| braiamp wrote:
| I along with others, years ago, have said that answers that don't
| address the question asked should be removed. In this case,
| malicious actors are using packages that are not related to the
| problem presented in the question. This will become more
| prevalent than package squatting.
| xattt wrote:
| Particularly if LLMs are being trained on this material.
| entuno wrote:
| I wonder if there's any weighting done based on things like
| upvotes in their training sets, or if the consider all
| answers equally?
| entuno wrote:
| You could probably create a more convincing version of this with
| a bit of collusion between multiple accounts on StackOverflow.
|
| If you have one person asking a deliberate question, someone else
| answering with a backdoored package that does actually does solve
| the problem, then a few accounts upvoting and adding comments it
| would lot a more more convincing than just a random answer with
| zero upvotes that doesn't actually solve the problem.
| colejohnson66 wrote:
| StackExchange is really good at detecting voting rings, so this
| won't work.
| Sebb767 wrote:
| But you would also need someone to search for and find that
| answer in order to actually deliver you payload. Answering
| random questions has the advantage that the original poster is
| incentivized to execute your code.
| mbil wrote:
| How about scrape SO to find new, popular questions; have an LLM
| write a convincing solution; tack on "you could also try pip
| install notmalware, which is much faster"; automatically post
| that poisoned answer.
|
| Extra credit: have the infected user's computer also start
| posting these answers to SO
| arp242 wrote:
| This would very likely set up one of the voting ring detection
| mechanisms. It's not so easy to manipulate voting on Stack
| Overflow - quite a lot of work goes on behind the scenes to
| stop this. I'm not saying it's impossible, but it's not easy
| either and requires significant effort. Combined with that your
| malicious package is also quite likely to be detected fairly
| quickly(-ish), I'm going to say that it's probably not that
| effective of a mechanism.
| skilled wrote:
| _Slightly unrelated..._
|
| This reads like yet another major attack vector through LLMs. A
| threat actor only needs to provide a few mentions of the package
| on reputable sites and it is almost guaranteed that your
| backdoored package will be mentioned by the LLM once it is
| retrained with new data, or even in cases like RAG.
|
| I am guessing I am not the first to think of this though.
| Wouldn't be surprised if this kind of attack vector is already
| being set in motion for all kinds of other purposes; product
| reviews, etc.
| vertis wrote:
| It is an important consideration for live language models. The
| example recently of a bad Reddit answer popping up in Gemini --
| putting glue on the pizza demonstrates how easy it is.
|
| While there are definitely ethical issues with the data that
| they are using. Trainers need to get a handle on this kind of
| thing because for LLMs to be truly useful they have to consume
| large parts of the web.
| nottorp wrote:
| Perhaps a ... human curated index ... would be more useful
| then? :)
|
| I mean, if you're manually curating what you feed to the LLM
| you end up doing one of the web directories of old and might
| as well skip the LLM part... or use it just to gain
| funding...
| venusenvy47 wrote:
| Do virus scanners know how to interpret base64 encoding when they
| scan a file? I assumed they would be able to catch something like
| this. Or wouldn't the OS catch this .exe file when it is being
| downloaded?
| nextworddev wrote:
| This will be unfortunately a canonical problem that LLMs can
| amplify
| RollAHardSix wrote:
| So much wasted effort. Just create an account on Upwork, play by
| Upworks rules & get complete unferreted access to any number of
| servers and websites run by small businesses and marketing
| consultancies. Sure you might have to do a bit of Wordpress dev
| to keep the illusion going but...there you go.
| apantel wrote:
| This is much bigger than this particular issue. This is the same
| as the Xz backdoor: supply chain attack. It's sad, but the fact
| is you cannot actually trust anyone else's code unless you audit
| it yourself. Even something seemingly-well-vetted like Xz can be
| a vehicle for malware.
| humanlity wrote:
| So, the real question is: where is the real helpful person?
| ashishbijlani wrote:
| Plug: I've been building Packj [1] to detect malicious
| PyPI/NPM/Ruby/PHP/etc. dependencies using behavioral analysis. It
| uses static+dynamic code analysis to scan for indicators of
| compromise (e.g., spawning of shell, use of SSH keys, network
| communication, use of decode+eval, etc). It also checks for
| several metadata attributes to detect bad actors (e.g., typo
| squatting).
|
| 1. https://github.com/ossillate-inc/packj
| 6c696e7578 wrote:
| Wasn't there an LLM that was slurping SO? Maybe that can be
| trained to flag abuse at the same time?
| positr0n wrote:
| Haha, my initial thought was: Wasn't there an LLM that was
| slurping SO? Maybe that will start suggesting malware to users.
___________________________________________________________________
(page generated 2024-05-30 23:02 UTC)