[HN Gopher] Zizmor would have caught the Ultralytics workflow vu...
       ___________________________________________________________________
        
       Zizmor would have caught the Ultralytics workflow vulnerability
        
       Author : campuscodi
       Score  : 60 points
       Date   : 2024-12-08 10:34 UTC (12 hours ago)
        
 (HTM) web link (blog.yossarian.net)
 (TXT) w3m dump (blog.yossarian.net)
        
       | HeWhoLurksLate wrote:
       | Wow, I had no clue about how many ways it was possible to get
       | burned with Actions - as an ME nerd, I've set up a few CI/CD
       | workflows, and if I recall correctly, while I was reading through
       | the documentation for GitHub Actions (circa 2022) there wasn't
       | any mention of cybersecurity best practices in the general docs.
       | Is that generally considered best practice, or at least
       | acceptable?
       | 
       | I'm not a programmer by trade- I generally write one-off or two-
       | off code, but that's changing as I get deeper into simulation
       | land. For me, reading the entirety of the docs is something that
       | generally happens only when I'm troubleshooting something or an
       | LLM dragged me significantly further than my understanding and I
       | have to go learn how a library or API works.
        
       | woodruffw wrote:
       | (Author of this post.)
       | 
       | If you're interested in how this went down, the timeline
       | section[1] in particular is worth jumping to: my key takeaway is
       | that this vulnerability was _reintroduced_ , and that there's
       | only limited evidence that the Ultralytics team have done a full
       | revocation and rotation of all accounts and credentials that the
       | attacker may have had access to.
       | 
       | Given that, it's not inconceivable that a third round of
       | backdoored packages will occur. I would recommend that people
       | exercise extreme caution when installing the current versions;
       | most users would probably be best served by pinning to an older
       | version from before any indicators of compromise.
       | 
       | [1]: https://blog.yossarian.net/2024/12/06/zizmor-ultralytics-
       | inj...
        
         | the_mitsuhiko wrote:
         | One quite annoying element is that as a third party you cannot
         | access the attestations of the deleted releases any more. I
         | really wanted to see if the attestations would help here to
         | figure out what happened. But maybe I'm just not informed
         | enough about where to look.
         | 
         | Another element here is that the releases seemingly were
         | deleted and re-created? I thought that was prevented by PyPI?
        
           | woodruffw wrote:
           | The attestations are checked into the public transparency
           | log, so they're still accessible -- that's how I did a decent
           | amount of the triage in the write up. You can find them in
           | the write up by searching for "Sigstore" (I would direct link
           | them, but I'm on mobile).
           | 
           | > Another element here is that the releases seemingly were
           | deleted and re-created? I thought that was prevented by PyPI?
           | 
           | Hmm, where do you see this? The release history on PyPI
           | doesn't show any recreations[1].
           | 
           | [1]: https://pypi.org/project/ultralytics/
        
             | the_mitsuhiko wrote:
             | > You can find them in the write up by searching for
             | "Sigstore" (I would direct link them, but I'm on mobile).
             | 
             | Yeah, I know they are in sigstore, I just did not know how
             | to find them. Is there an interface for this I missed?
             | 
             | > Hmm, where do you see this? The release history on PyPI
             | doesn't show any recreations[1].
             | 
             | Then I completely misunderstood what happened. Was this in
             | fact completely made up releases that were not even
             | intended to be triggered? Eg: a bot released .41 without
             | there being an intent of being an actual .41 release? I
             | thought that UltralyticsAssistant was the developer, not
             | the attacker. Do they also control that thing?
        
               | woodruffw wrote:
               | > Is there an interface for this I missed?
               | 
               | That would be search.sigstore.dev, unless I'm
               | misunderstanding what you mean.
               | 
               | > Was this in fact completely made up releases that were
               | not even intended to be triggered? Eg: a bot released .41
               | without there being an intent of being an actual .41
               | release? I thought that UltralyticsAssistant was the
               | developer, not the attacker. Do they also control that
               | thing?
               | 
               | .41 and .42 were triggered directly from the repository.
               | One was triggered by the UltralyticsAssistant account and
               | included a human bypass, which strongly suggests that the
               | attacker controlled (and maybe still controls) that bot
               | account.
               | 
               | The last two compromised releases were published directly
               | via API token, not via the source repo, which strongly
               | suggests that the attacker either exfil'd an old API
               | token from CI/CD _or_ that they're in control of the
               | developer's account on PyPI. Those ones don't have
               | attestations, while the first two releases do (two each,
               | one per dist per release).
        
               | the_mitsuhiko wrote:
               | > .41 and .42 were triggered directly from the
               | repository. One was triggered by the UltralyticsAssistant
               | account and included a human bypass, which strongly
               | suggests that the attacker controlled (and maybe still
               | controls) that bot account.
               | 
               | Ah, but if they controlled the bot then didn't they have
               | other problems too? If that is the case, then disregard
               | my comment. I was under the impression that this was not
               | the attacker.
               | 
               | > That would be search.sigstore.dev, unless I'm
               | misunderstanding what you mean.
               | 
               | No, that's it in theory I suppose. I did try this but
               | when I used the commit I thought triggered the release
               | (cb260c243ffa3e0cc84820095cd88be2f5db86ca) I did not see
               | it show up.
        
               | woodruffw wrote:
               | > Ah, but if they controlled the bot then didn't they
               | have other problems too?
               | 
               | Yep -- my theory is that this all starts with the
               | insecure trigger + template injection, and that the
               | attacker exfil'd the bot's PAT and stale PyPI API token
               | at that point. The first round of attacks used the bot
               | PAT and cache poisoning, and then the attacker pivoted to
               | the PyPI token once the first vector was closed off.
               | 
               | > I used the commit I thought triggered the release
               | (cb260c243ffa3e0cc84820095cd88be2f5db86ca) I did not see
               | it show up
               | 
               | I think I know what's happening there: that search UI
               | only indexes by commit for attestations produced by
               | gitsign, not every attestation containing a commit. I
               | used it by finding the entry IDs from the release action
               | logs on GitHub Actions, but if/when those are flushed
               | someone who doesn't already know them will need to seek
               | the log in order to find the attestations.
               | 
               | That's not ideal; the search.sigstore.dev service would
               | ideally have more indices like crt.sh does.
        
       | IshKebab wrote:
       | Not using Bash would have prevented the vulnerability in the
       | first place.
       | 
       | STOP USING BASH.
        
         | woodruffw wrote:
         | The vulnerability has nothing to do with bash. It's a template
         | injection in GitHub Actions, which bypasses the interpolation
         | of _any_ shell or interpreter.
        
       | tigereyeTO wrote:
       | This post has left me wondering: what is zizmor? What is
       | ultralytics? Are these words actually real or is someone having a
       | stroke?
       | 
       | Not all nerds know all projects so I decided to educate myself
       | and followed OP's links to learn about Ultralytics:
       | 
       | > Ultralytics YOLO11 is a cutting-edge, state-of-the-art (SOTA)
       | model that builds upon the success of previous YOLO versions and
       | introduces new features and improvements to further boost
       | performance and flexibility.
       | 
       | Ultralytics' readme doesn't explain what ultralytics is or does.
       | Thankfully Zizmor's readme describes itself clearly:
       | 
       | > zizmor is a static analysis tool for GitHub Actions. It can
       | find many common security issues in typical GitHub Actions CI/CD
       | setups.
       | 
       | This isn't a critique on OP: I enjoyed reading about the
       | vulnerability(ies!) you found and I learned a lot. I'm just
       | generally frustrated that so many readme files on GitHub fail to
       | describe what the project actually does, Ultralytics being just
       | one example.
       | 
       | Have fun and keep hacking
        
         | bobthepanda wrote:
         | I wonder if Zizmor has anything to do with this NYC local
         | notable: https://en.wikipedia.org/wiki/Jonathan_Zizmor
        
           | hardwaregeek wrote:
           | I was legitimately wondering if Dr Zizmor made a pivot into
           | cybersecurity
        
           | woodruffw wrote:
           | I named it explicitly for Dr. Zizmor :-)
           | 
           | https://github.com/woodruffw/zizmor#the-name
        
       | throw4847285 wrote:
       | Thank you Doctor Zizmor!
        
         | IncreasePosts wrote:
         | Spotted the straphanger
        
       ___________________________________________________________________
       (page generated 2024-12-08 23:00 UTC)