https://github.com/protectai/ai-exploits Skip to content Toggle navigation Sign up * Product + Actions Automate any workflow + Packages Host and manage packages + Security Find and fix vulnerabilities + Codespaces Instant dev environments + Copilot Write better code with AI + Code review Manage code changes + Issues Plan and track work + Discussions Collaborate outside of code Explore + All features + Documentation + GitHub Skills + Blog * Solutions For + Enterprise + Teams + Startups + Education By Solution + CI/CD & Automation + DevOps + DevSecOps Resources + Learning Pathways + White papers, Ebooks, Webinars + Customer Stories + Partners * Open Source + GitHub Sponsors Fund open source developers + The ReadME Project GitHub community articles Repositories + Topics + Trending + Collections * Pricing Search or jump to... Search code, repositories, users, issues, pull requests... Search [ ] Clear Search syntax tips Provide feedback We read every piece of feedback, and take your input very seriously. [ ] [ ] Include my email address so I can be contacted Cancel Submit feedback Saved searches Use saved searches to filter your results more quickly Name [ ] Query [ ] To see all available qualifiers, see our documentation. Cancel Create saved search Sign in Sign up You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert {{ message }} protectai / ai-exploits Public * Notifications * Fork 0 * Star 107 A collection of real world AI/ML exploits for responsibly disclosed vulnerabilities License View license 107 stars 0 forks Activity Star Notifications * Code * Issues 0 * Pull requests 0 * Actions * Projects 0 * Security * Insights More * Code * Issues * Pull requests * Actions * Projects * Security * Insights protectai/ai-exploits This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository. main Switch branches/tags [ ] Branches Tags Could not load branches Nothing to show {{ refName }} default View all branches Could not load tags Nothing to show {{ refName }} default View all tags Name already in use A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch? Cancel Create 1 branch 0 tags Code * Local * Codespaces * Clone HTTPS GitHub CLI [https://github.com/p] Use Git or checkout with SVN using the web URL. [gh repo clone protec] Work fast with our official CLI. Learn more about the CLI. * Open with GitHub Desktop * Download ZIP Sign In Required Please sign in to use Codespaces. Launching GitHub Desktop If nothing happens, download GitHub Desktop and try again. Launching GitHub Desktop If nothing happens, download GitHub Desktop and try again. Launching Xcode If nothing happens, download Xcode and try again. Launching Visual Studio Code Your codespace will open once ready. There was a problem preparing your codespace, please try again. Latest commit @DanMcInerney DanMcInerney link ... d153298 Nov 16, 2023 link d153298 Git stats * 4 commits Files Permalink Failed to load latest commit information. Type Name Latest commit message Commit time h2o Initial commit for release [?] November 16, 2023 08:25 mlflow Initial commit for release [?] November 16, 2023 08:25 nmap-nse Initial commit for release [?] November 16, 2023 08:25 ray Initial commit for release [?] November 16, 2023 08:25 .gitignore Initial commit for release [?] November 16, 2023 08:25 Dockerfile Initial commit for release [?] November 16, 2023 08:25 LICENSE Initial commit for release [?] November 16, 2023 08:25 README.md link November 16, 2023 14:47 View code [ ] AI Exploits Overview Setup & Usage Docker Using the Metasploit Modules With Docker With Metasploit Installed Locally Using Nuclei Templates Using CSRF Templates Contribution Guidelines License README.md AI Exploits AI Exploits Logo The AI world has a security problem and it's not just in the inputs given to LLMs such as ChatGPT. Based on research done by Protect AI and independent security experts on the Huntr Bug Bounty Platform, there are far more impactful and practical attacks against the tools, libraries and frameworks used to build, train, and deploy machine learning models. Many of these attacks lead to complete system takeovers and/or loss of sensitive data, models, or credentials most often without the need for authentication. With the release of this repository, Protect AI hopes to demystify to the Information Security community what pratical attacks against AI/ Machine Learning infrastructure look like in the real world and raise awareness to the amount of vulnerable components that currently exist in the AI/ML ecosystem. More vulnerabilities can be found here: November Vulnerability Report Overview This repository, ai-exploits, is a collection of exploits and scanning templates for responsibly disclosed vulnerabilities affecting machine learning tools. Each vulnerable tool has a number of subfolders containing three types of utilities: Metasploit modules, Nuclei templates and CSRF templates. Metasploit modules are for security professionals looking to exploit the vulnerabilies and Nuclei templates are for scanning a large number of remote servers to determine if they're vulnerable. Setup & Usage The easiest way to use the modules and scanning templates is to build and run the Docker image provided by the Dockerfile in this repository. The Docker image will have Metasploit and Nuclei already installed along with all the necessary configuration. Docker 1. Build the image: docker build -t protectai/ai-exploits https://github.com/protectai/AI-exploits 2. Run the docker image: docker run -it --rm protectai/ai-exploits /bin/bash The latter command will drop you into a bash session in the container with msfconsole and nuclei ready to go. Using the Metasploit Modules With Docker Start the Metasploit console (the new modules will be available under the exploits/protectai category), load a module, set the options, and run the exploit. msfconsole msf6 > use exploit/protectai/ray_job_rce msf6 exploit(protectai/ray_job_rce) > set RHOSTS msf6 exploit(protectai/ray_job_rce) > run With Metasploit Installed Locally Create a folder ~/.msf4/modules/exploits/protectai and copy the exploit modules into it. mkdir -p ~/.msf4/modules/exploits/protectai cp ai-exploits/ray/msfmodules/* ~/.msf4/modules/exploits/protectai msfconsole msf6 > use exploit/protectai/ Using Nuclei Templates Nuclei is a vulnerability scanning engine which can be used to scan large numbers of servers for known vulnerabilities in web applications and networks. Navigate to nuclei templates folder such as ai-exploits/mlflow/ nuclei-templates. In the Docker container these are stored in the / root/nuclei-templates folder. Then simply point to the template file and the target server. cd ai-exploits/mlflow/nuclei-templates nuclei -t mlflow-lfi.yaml -u http://:` Using CSRF Templates Cross-Site Request Forgery (CSRF) vulnerabilities enable attackers to stand up a web server hosting a malicious HTML page that will execute a request to the target server on behalf of the victim. This is a common attack vector for exploiting vulnerabilities in web applications, including web applications which are only exposed on the localhost interface and not to the broader network. Below is a simple demo example of how to use a CSRF template to exploit a vulnerability in a web application. Start a web server in the csrf-templates folder. Python allows one to stand up a simple web server in any directory. Navigate to the template folder and start the server. cd ai-exploits/ray/csrf-templates python3 -m http.server 9999 Now visit the web server address you just stood up (http:// 127.0.0.1:9999) and hit F12 to open the developer tools, then click the Network tab. Click the link to ray-cmd-injection-csrf.html. You should see that the browser sent a request to the vulnerable server on your behalf. Contribution Guidelines We welcome contributions to this repository. Please read our Contribution Guidelines for more information on how to contribute. License This project is licensed under the Apache 2.0 License. About A collection of real world AI/ML exploits for responsibly disclosed vulnerabilities Resources Readme License View license Activity Stars 107 stars Watchers 6 watching Forks 0 forks Report repository Releases No releases published Packages 0 No packages published Contributors 2 * @DanMcInerney DanMcInerney Dan McInerney * @byt3bl33d3r byt3bl33d3r Marcello Languages * Python 80.2% * Lua 9.8% * HTML 8.3% * Dockerfile 1.7% Footer (c) 2023 GitHub, Inc. Footer navigation * Terms * Privacy * Security * Status * Docs * Contact GitHub * Pricing * API * Training * Blog * About You can't perform that action at this time.