https://www.nginx.com/blog/http-2-rapid-reset-attack-impacting-f5-nginx-products/ Skip to content NGINX.COM < Your Cookie Settings Analytics cookies are off for visitors from the UK or EEA unless they click Accept or submit a form on nginx.com. They're on by default for everybody else. Follow the instructions here to deactivate analytics cookies. This deactivation will work even if you later click Accept or submit a form. [ ] Check this box so we and our advertising and social media partners can use cookies on nginx.com to better tailor ads to your interests. Uncheck it to withdraw consent. Cookie preferences Accept cookies for analytics, social media, and advertising, or learn more and adjust your preferences. These cookies are on by default for visitors outside the UK and EEA. Privacy Notice. Accept Web Server Load Balancing with NGINX Plus * EN + EN + Zhong Wen + Ri Ben Yu * F5 Sites + F5.com + DevCentral + MyF5 + Partner Central + LearnF5 Search [ ] [Search] * Contact F5 NGINX + F5 NGINX Sales + F5 Support + F5 Professional Services Web Server Load Balancing with NGINX Plus * Solutions & Products + F5 NGINX Solutions o # @ F5 NGINX Solutions Explore the areas where NGINX can help your organization overcome specific technical challenges. Learn More # @ Solution Quick Links - Customer Stories - Product Datasheets - Read the Blog o # @ Application Delivery @ Kubernetes @ Security @ Zero Trust # @ Web Performance @ Cloud # @ [Rocket-bla] Start Your Free Trial Get Started + F5 NGINX Products View All + NGINX Plus o # @ F5 NGINX Plus NGINX Plus is a software load balancer, API gateway, and reverse proxy built on top of NGINX. Learn More @ Compare Models @ Load Balancer @ API Gateway @ Real-Time Dashboard @ NGINXaaS for Azure @ Dynamic Modules # @ Product Quick Links - NGINX Plus - Product Datasheets - Read the Blog @ [Rocket-bla] Start Your Free Trial Get Started + NGINX App Protect o # @ F5 NGINX App Protect Modern app security solution that works seamlessly in DevOps environments. Learn More @ DoS @ WAF # @ Product Quick Links - Product Documentation - Datasheets - Blog @ [Rocket-bla] Start Your Free Trial Get Started + NGINX Amplify + NGINX Management Suite + NGINX Ingress Controller + NGINX Unit * Resources + Resources View All + Datasheets + Documentation o # @ Product Documentation Learn how to deliver, manage, and protect your applications using NGINX products. Find developer guides, API references, and more. Learn More # @ Resources Quick Links - Documentation - Webinars - Read the Blog o # @ NGINX Plus @ NGINX App Protect DoS @ NGINX App Protect WAF @ NGINX Amplify # @ NGINX Management Suite @ NGINX Ingress Controller @ NGINX Unit # @ [Rocket-bla] Start Your Free Trial Get Started + Education o # @ NGINX University Learn about NGINX products, industry trends, and connect with the experts. Learn More # @ Resources Quick Links - Documentation - Webinars - Read the Blog o # @ Ebooks & Reports @ Events @ FAQ @ Glossary # @ Training @ Webinars @ Youtube # @ [Rocket-bla] Start Your Free Trial Get Started + GitHub + Success Stories * Blog + Blog View All + Solution Blogs o # @ Solution Blogs Get technical and business-oriented blogs that help you address key technology challenges. Learn More @ Application Delivery @ Security @ Kubernetes # @ Blog Quick Links - Tech Blogs - News Blogs - Opinion Blogs @ [Rocket-bla] Start Your Free Trial Get Started + Product Blogs o # @ Product Blogs Learn how to use NGINX products to solve your technical challenges. Learn More # @ Blog Quick Links - Tech Blogs - News Blogs - Opinion Blogs o # @ NGINX Amplify @ NGINX App Protect @ NGINX Ingress Controller @ NGINX Management Suite # @ NGINX Open Source @ NGINX Plus @ NGINX Unit # @ [Rocket-bla] Start Your Free Trial Get Started * Support + Support o # @ Support Options and Resources Get the help you need from the experts, authors, maintainers, and community. Learn More @ NGINX Support Options @ DevCentral (an F5 Community) @ MyF5 (Support Portal) @ Professional Services @ Open Source Slack Community # @ NGINX Quick Links - Documentation - Webinars - Read the Blog @ [Rocket-bla] Start Your Free Trial Get Started * Partners + Explore Partners o # @ NGINX Partners Combine the power and performance of NGINX with a rich ecosystem of product integrations, custom solutions, services, and deployment options. Learn More @ All Partners @ Become a Partner @ Find a Reseller @ Explore Tech Partners # @ Partners Quick Links - AWS - Azure - Red Hat - Google @ [Rocket-bla] Start Your Free Trial Get Started * Get F5 NGINX + Trials and Pricing + GitHub + Get F5 NGINX o # @ Get F5 NGINX The NGINX Application Platform is a suite of products that together form the core of what organizations need to deliver applications with performance, reliability, security, and scale. Learn More @ Trials and Pricing @ GitHub # @ NGINX Quick Links - Documentation - Webinars - Read the Blog @ [Rocket-bla] Start Your Free Trial Get Started * Free Trial 1-800-915-9122 Search [ ] [Search] Resources > Blog > News HTTP/2 Rapid Reset Attack Impacting NGINX Products * * * * Michael Vernik Michael Vernik of F5 Sr Product Manager [Nina-Forsyth-150x150] Nina Forsyth of F5 Director of Product Management October 10, 2023 This blog post centers on a vulnerability that was recently discovered related to the HTTP/2 protocol. Under certain conditions, this vulnerability can be exploited to execute a denial-of-service attack on NGINX Open Source, NGINX Plus, and related products that implement the server-side portion of the HTTP/2 specification. To protect your systems from this attack, we're recommending an immediate update to your NGINX configuration. The Problem with HTTP/2 Stream Resets After establishing a connection with a server, the HTTP/2 protocol allows clients to initiate concurrent streams for data exchange. Unlike previous iterations of the protocol, if an end user decides to navigate away from the page or halt data exchange for any other reason, HTTP/2 provides a method for canceling the stream. It does this by issuing an RST_STREAM frame to the server, saving it from executing work needlessly. The vulnerability is exploited by initiating and rapidly canceling a large number of HTTP/2 streams over an established connection, thereby circumventing the server's concurrent stream maximum. This happens because incoming streams are reset faster than subsequent streams arrive, allowing the client to overload the server without ever reaching its configured threshold. Impact on NGINX For performance and resource consumption reasons, NGINX limits the number of concurrent streams to a default of 128 (see http2_max_concurrent_streams). In addition, to optimally balance network and server performance, NGINX allows the client to persist HTTP connections for up to 1000 requests by default using an HTTP keepalive (see keepalive_requests). By relying on the default keepalive limit, NGINX prevents this type of attack. Creating additional connections to circumvent this limit exposes bad actors via standard layer 4 monitoring and alerting tools. However, if NGINX is configured with a keepalive that is substantially higher than the default and recommended setting, the attack may deplete system resources. When a stream reset occurs, the HTTP/2 protocol requires that no subsequent data is returned to the client on that stream. Typically, the reset results in negligible server overhead in the form of tasks that gracefully handle the cancellation. However, circumventing NGINX's stream threshold enables a client to take advantage of this overhead and amplify it by rapidly initiating thousands of streams. This forces the server CPU to spike, denying service to legitimate clients. DoS Attack via HTTP2 Streams Denial-of-service by establishing HTTP/2 streams, followed by stream cancellations under abnormally high keepalive limits. Steps for Mitigating Attack Exposure As a fully featured server and proxy, NGINX provides administrators with powerful tools for mitigating denial-of-service attacks. To take advantage of these features, it is essential that the following updates are made to NGINX configuration files, minimizing the server's attack surface: * keepalive_requests should be kept at the default setting of 1000 requests * http2_max_concurrent_streams should be kept at the default setting of 128 streams We also recommend that these safety measures are added as a best practice: * limit_conn enforces a limit on the number of connections allowed from a single client. This directive should be added with a reasonable setting balancing application performance and security. * limit_req enforces a limit on the number of requests that will be processed within a given amount of time from a single client. This directive should be added with a reasonable setting balancing application performance and security. How We're Responding We experimented with multiple mitigation strategies that helped us gain an understanding into how this attack could impact our wide range of customers and users. While this research confirmed that NGINX is already equipped with all the necessary tools to avoid the attack, we wanted to take additional steps to ensure that users who do need to configure NGINX beyond recommended specifications are able to do so. Our investigation yielded a method for improving server resiliency under various forms of flood attacks that are theoretically possible over the HTTP/2 protocol. As a result, we've issued a patch that increases system stability under these conditions. To protect against such threats, we recommend that NGINX Open Source users rebuild binaries from the latest codebase and NGINX Plus customers update to the latest packages (R29p1 or R30p1) immediately. How the Patch Works To ensure the early detection of flood attacks on NGINX, the patch imposes a limit on the number of new streams that can be introduced within one event loop. This limit is set to twice the value configured using the http2_max_concurrent_streams directive. The limit will be applied even if the maximum threshold is never reached, like when streams are reset right after sending the request (as in the case of this attack). Affected Products This vulnerability impacts the NGINX HTTP/2 module ( ngx_http_v2_module). For information about your specific NGINX or F5 product that might be affected, please visit: https://my.f5.com/ manage/s/article/K000137106. For more information on CVE-2023-44487 - HTTP/2 Rapid Reset Attack, please see: https://www.cve.org/CVERecord?id=CVE-2023-44487 Acknowledgements We would like to recognize Cloudflare, Amazon, and Google for their part in the discovery and collaboration in identifying and mitigating this vulnerability. Please enable JavaScript to view the comments powered by Disqus. Hero image Learn how to deploy, configure, manage, secure, and monitor your Kubernetes Ingress controller with NGINX to deliver apps and APIs on-premises and in the cloud. Download Now About The Author Michael Vernik Michael Vernik Sr Product Manager More Blogs By Michael Vernik About The Author [Nina-Forsyth] Nina Forsyth Director of Product Management More Blogs By Nina Forsyth About F5 NGINX F5, Inc. is the company behind NGINX, the popular open source project. We offer a suite of technologies for developing and delivering modern applications. Together with F5, our combined solution bridges the gap between NetOps and DevOps, with multi-cloud application services that span from code to customer. Learn more at nginx.com or join the conversation by following @nginx on Twitter. Get Started [ICON-Rocke] Try Out NGINX Plus on Your Turf Get Started [Tech-Suppo] Ask Us a Question Contact Sales Secure And Deliver Extraordinary Digital Experiences F5 NGINX's portfolio of automation, security, performance, and insight capabilities empowers our customers to create, secure, and operate adaptive applications that reduce costs, improve operations, and better protect users. * What We Offer + Free Trial + Pricing + Products + F5 NGINX Solutions + NGINX Open Source + NGINX on Github * Resources + Documentation + Ebooks + Webinars + Datasheets + Success Stories + Blog + FAQ + Learn + Glossary * Support + Professional Services + Training + Customer Portal Login + Open Source Slack Community * Partners + NGINX on Amazon Web Services + NGINX on Google Cloud + IBM + NGINX on Microsoft Azure + NGINX and Red Hat + Find a Partner + Certified Module Program * Company + About F5 NGINX + F5 NGINX Careers + Press + Events + F5 + Get NGINX Updates Connect With Us * twitter * linkedin * facebook * youtube * rss [Now-arrivi] Copyright (c) F5, Inc. All rights reserved. | Trademarks | Policies | Privacy | California Privacy | Do Not Sell My Personal Information