https://github.com/lwthiker/curl-impersonate Skip to content Sign up * Why GitHub? + Features + Mobile + Actions + Codespaces + Packages + Security + Code review + Issues + Integrations + GitHub Sponsors + Customer stories * Team * Enterprise * Explore + Explore GitHub + Learn and contribute + Topics + Collections + Trending + Learning Lab + Open source guides + Connect with others + The ReadME Project + Events + Community forum + GitHub Education + GitHub Stars program * Marketplace * Pricing + Plans + Compare plans + Contact Sales + Education [ ] * # In this repository All GitHub | Jump to | * No suggested jump to results * # In this repository All GitHub | Jump to | * # In this user All GitHub | Jump to | * # In this repository All GitHub | Jump to | Sign in Sign up {{ message }} lwthiker / curl-impersonate Public * Notifications * Fork 0 * Star 0 * A special compilation of curl that makes it impersonate Firefox MIT License 0 stars 0 forks Star Notifications * Code * Issues 0 * Pull requests 0 * Actions * Projects 0 * Wiki * Security * Insights More * Code * Issues * Pull requests * Actions * Projects * Wiki * Security * Insights This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository. main Switch branches/tags [ ] Branches Tags Could not load branches Nothing to show {{ refName }} default View all branches Could not load tags Nothing to show {{ refName }} default View all tags 1 branch 0 tags Code Latest commit @lwthiker lwthiker Update README.md ... 4fe2fd3 Feb 17, 2022 Update README.md 4fe2fd3 Git stats * 14 commits Files Permalink Failed to load latest commit information. Type Name Latest commit message Commit time Dockerfile Update comments in the Dockerfile Feb 17, 2022 LICENSE Initial commit Feb 17, 2022 README.md Update README.md Feb 17, 2022 curl-configure.patch Commit the Dockerfile and all required patches Feb 17, 2022 curl-lib-nss.patch Commit the Dockerfile and all required patches Feb 17, 2022 curl-static-libnss.patch Commit the Dockerfile and all required patches Feb 17, 2022 curl_ff95 Fix the headers in curl_ff95 wrapper script Feb 17, 2022 libnghttp2-pc.patch Commit the Dockerfile and all required patches Feb 17, 2022 View code Why? How? Installation Usage Contents What's next? README.md A special compilation of curl that makes it impersonate Firefox. This curl binary is able to perform a TLS handshake that is identical to Firefox. Why? When you use an HTTP client with a TLS website, it first performs a TLS handshake. The first message of that handshake is called Client Hello. The Client Hello message that curl produces differs drastically from that of a real browser. Compare the following Wireshark capture. Left is a regular curl, right is Firefox. curl-ff-before Some web services therefore use the TLS handshake to fingerprint which HTTP client is accessing them. Notably, some bot protection platforms use this to identify curl and block it. With the modified curl in this repository, the Client Hello message looks exactly like Firefox's. This tricks TLS fingerprinters to think that it is Firefox that is accessing them, and is able to bypass some well-known bot protections. How? The modifications that were needed to make this work: * Compiling curl with nss, the TLS library that Firefox uses, instead of OpenSSL. * Modifying the way curl configures various TLS extensions and SSL options. * Running curl with some non-default flags, specifically --http2, --ciphers, and some -H headers. The resulting curl looks, from a network perspective, identical to Firefox (Firefox 95, Windows, non-incognito mode). Compare: (left is curl-impersonate, right is Firefox): curl-ff-after Read the full description in the blog post. Installation This repository contains a Dockerfile that will build curl with all the necessary modifications. Build it like any Docker image: docker build -t curl-impersonate . The resulting image contains: * /build/out/curl-impersonate - The curl binary that can impersonate Firefox. It is compiled statically against libcurl, nss, and libnghttp2 so that it won't conflict with any existing libraries on your system. You can use it from the container or copy it out. Tested to work on Ubuntu 20.04. * /build/out/curl_ff95 - A wrapper script that launches curl-impersonate with the needed headers and ciphers to impersonate Firefox 95. Copy them from the docker image using docker cp or use them in a multi-stage docker build. In addition install libnss3: sudo apt install libnss3. Even though nss is statically compiled into curl-impersonate, it is still necessary to install libnss3 because curl dynamically loads libnssckbi.so, a file containing Mozilla's list of trusted root certificates. Alternatively, use curl -k to disable certificate verification. Usage It is recommended to use the wrapper script curl_ff95 that adds all the correct headers and flags. For example: curl_ff95 https://www.google.com You can add command line flags and they will be passed on to curl. However, some flags change curl's TLS signature which may cause it to be detected. Contents This repository contains the following files: * Dockerfile - Used to build curl-impersonate with all dependencies. * curl_ff95 - Wrapper script that launches curl-impersonate with the correct flags. * curl-lib-nss.patch - The main patch that makes curl use the same TLS extensions as Firefox. * libnghttp2-pc.patch - Patch to make libnghttp2 compile statically. * curl-configure.patch - Patch to make curl compile with a static libnghttp2. * curl-static-libnss.patch - Patch to make curl compile with a static libnss. What's next? This was done in a very hacky way, but I hope it could be turned into a real project. Imagine that you could run: curl --impersonate ff95 and it would behave exactly like Firefox 95. It could then be wrappped with a nice Python library. About A special compilation of curl that makes it impersonate Firefox Resources Readme License MIT License Stars 0 stars Watchers 1 watching Forks 0 forks Releases No releases published Packages 0 No packages published Languages * Dockerfile 61.0% * Shell 39.0% * (c) 2022 GitHub, Inc. * Terms * Privacy * Security * Status * Docs * Contact GitHub * Pricing * API * Training * Blog * About You can't perform that action at this time. You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.