[HN Gopher] Ultralytics AI model hijacked to infect thousands wi...
       ___________________________________________________________________
        
       Ultralytics AI model hijacked to infect thousands with cryptominer
        
       Author : sandwichsphinx
       Score  : 46 points
       Date   : 2024-12-07 18:31 UTC (4 hours ago)
        
 (HTM) web link (www.bleepingcomputer.com)
 (TXT) w3m dump (www.bleepingcomputer.com)
        
       | aucisson_masque wrote:
       | > Ultralytics tools are open-source and are used by numerous
       | projects spanning a wide range of industries and applications.
       | 
       | Open source and popular doesn't necessarily mean safe.
       | 
       | Technically you can read the code source but no one does that and
       | especially for each update.
        
         | smarx007 wrote:
         | The vulnerability was not in the source code of the repo. See
         | https://blog.yossarian.net/2024/12/06/zizmor-ultralytics-inj...
         | for a deeper analysis - a great read!
        
       | anakaine wrote:
       | Google Collab banned affected users.
       | 
       | I wonder how that's going to be resolved, of if Google will just
       | do their usual and make it close to impossible to appeal and get
       | unbanned.
        
         | CatWChainsaw wrote:
         | I think they consult some arcane equation when they need to
         | make a decision. Two of the most important variables are
         | virality of the incident and whether or not anyone affected has
         | an in at Google. Mercury in retrograde messes this equation up.
        
       | Imnimo wrote:
       | Somewhat tangentially, I really dislike that Ultralytics (and
       | others) started slapping higher version numbers of their YOLO
       | variants. Redmon used the numbering scheme v2 and v3 for his
       | improvements on his original model. But Ultralytics' 11 is it's
       | own thing with no connection to Redmon. I just think it gives a
       | misleading impression of what the history is.
        
         | kookamamie wrote:
         | Agreed. YOLO, the midel name, was essentially hijacked.
        
         | daemonologist wrote:
         | Ultralytics also had, for at least ~a year, a language model
         | replying to GitHub issues using their CEO's account (without
         | any kind of disclosure). It was frequently confidently
         | incorrect and probably wasted thousands of developer hours
         | (because when the CEO replies to your issue with advice why
         | wouldn't you take it at face value?!)
         | 
         | Looks like they've since given the bot its own account but that
         | experience definitely soured me on the company.
         | 
         | (Also, there's an MIT licensed implementation of "yolov9" here:
         | https://github.com/WongKinYiu/YOLO . Affiliated with neither
         | Redmond nor Ultralytics as far as I know.)
        
           | Loughla wrote:
           | >a language model replying to GitHub issues using their CEO's
           | account (without any kind of disclosure)
           | 
           | What is the possible justification for this? And did they
           | just not do any oversight at all? Did no one notice the CEO
           | was suddenly full of shit?
        
           | tensorturtle wrote:
           | I've made several contributions to their main repo and the
           | LLM generated mush replies from various core team accounts
           | have been a horror, derailing Issues threads and such. An
           | excellent case study in how not to use LLMs.
        
       | zb3 wrote:
       | This appears to be a code injection in a PR branch name, not an
       | AI model compromise..
        
         | geraldcombs wrote:
         | Yeah, I think the story that's being missed here is that GitHub
         | allows branches named "$(curl...|bash)".
        
           | justinclift wrote:
           | Sounds like something GitHub should be easily able to detect
           | too.
           | 
           | Hopefully that's an exploit path they'll close soon, if
           | they've not done so already.
        
           | 3eb7988a1663 wrote:
           | What/where does code get evaluated from a branch name?
        
         | jerpint wrote:
         | The yolo models are "dumb" black box object detectors, it's a
         | supply chain attack, the model itself was very likely never
         | touched
        
       | quuxplusone wrote:
       | Serendipitously comes one day after this story[1] was on the
       | front page: at least one Debian maintainer failing to realize the
       | risks of non-alphanumeric usernames. "What could go wrong?" Well,
       | here's Git allowing branch names to contain dollar signs,
       | backticks, etc., because "what could go wrong?"... and... well,
       | this could.
       | 
       |  _Names_ are _identifiers_. Allowing identifiers to contain
       | anything besides identifier characters merely opens new and weird
       | attack vectors.
       | 
       | [1] https://news.ycombinator.com/item?id=42338134
        
       ___________________________________________________________________
       (page generated 2024-12-07 23:00 UTC)