https://jviide.iki.fi/http-redirects @jviide * 2024-05-23 Your API Shouldn't Redirect HTTP to HTTPS TL;DR: Instead of redirecting API calls from HTTP to HTTPS, make the failure visible. Either return a clear HTTP error response or disable the HTTP interface altogether. Unfortunately, many well-known API providers don't currently do that. Updated 2024-05-24: Added the Google Bug Hunter Team response to the report that the VirusTotal API responds in plaintext to unencrypted requests. Background When an user directs their web browser to an HTTP URL, it's a common practice for the service to redirect the request to a corresponding HTTPS page. This unencrypted part of the communication flow has its flaws. Third parties in shared networks, as well as network intermediaries, could sniff passwords and other secrets from the initial HTTP traffic or even impersonate the web server with a MITM attack. Nevertheless, redirection has been an useful first step in the transition from the largely unencrypted early web to the largely encrypted web of today. Later techniques tightened the security story further. Servers can now send HSTS along with the initial HTTP-to-HTTPS redirection response, telling the user's browser to use only HTTPS for that domain from then on. This limits the window of opportunity for trivial sniffing and MITM attacks to the first request. Browsers then added HSTS preload lists and HTTPS-Only modes that allow skipping the initial unencrypted request altogether. From the perspective of usability-security tradeoff it all makes sense for user-facing sites. But interestingly, the redirection approach also appears to be widely adopted for APIs. APIs are mostly consumed by other software so the same usability arguments don't apply there. Moreover, many programmatic API clients don't tend to keep browser-like state of things like HSTS headers they have seen. This post argues that, due to these factors, the common practice of redirecting API calls from HTTP to HTTPS should be reconsidered. While the post mostly refers to REST APIs, its points also apply to other styles of APIs that use HTTP(S) as a transport mechanism. A Simple Typo Is Enugh At work, we were building a new integration against a third-party API. The initial code commit contained a mistyped API base URL "http: //..." instead of "https://...". A pretty easy mistake to make. The error was essentially masked during runtime: The third-party API responded to every request with a 301 redirect to their HTTPS side. Node.js's built-in fetch happily and quietly followed those redirects to the HTTPS endpoint. Every single one of our API requests now sent the API keys over the network in plaintext, before then sending them again to the encrypted endpoint. The one letter omission could have exposed the used API keys to third parties without us realizing it. As the integration would have worked, there's a good chance that code would have leaked any secrets in the API calls for years. In the long run, the probabilities for malice tend to accumulate. Luckily we spotted the error during the code review before the error could propagate to production or even testing. We also realized that our own API also did similar HTTP-to-HTTPS redirects. The Fail-fast Principle When an API redirect HTTP requests to HTTPS - and the API client silently follows those redirects - it tends to hide mistyped URLs like in the case described above. A simple one-letter omission can easily be ignored, end up in production, and compromise the entire system's confidentiality. In most cases, it's better to adhere to the fail-fast principle: unencrypted API calls should fail in a spectacular and visible way so that the developer can easily spot and fix the typo as early as possible during the development process. A great solution for failing fast would be to disable the API server's HTTP interface altogether and not even answer to connections attempts to port 80. If the initial unencrypted connection is never established then the API keys aren't sent, mitigating sniffing attacks and limiting the window of opportunity for MITM attacks to an extremely small time window. This approach is viable for APIs hosted under their own domains like api.example.com. Our own API was served under the /api path from the same domain as our service's web UI. We didn't have the guts to disable the HTTP interface for that domain altogether, so we picked next best option: all unencrypted HTTP requests made under /api now return a descriptive error message along with the HTTP status code 403. Some initial plaintext requests might be made during development, but they're much easier for developers to notice. Who Else? That took care of our own API. We also pinged the third-party API provider and a couple of friends that they might want to check their APIs. And who knows, maybe there were some commonly used APIs that accept API keys (or other credentials) and also redirect from HTTP to HTTPS? I listed a bunch of well-known APIs from the top of my head and did a little survey. Several of them returned HTTP errors or declined to connections altogether. They're listed here with cURL spells for checking out their detailed responses: 1. Stripe API: Responds with 403 ("Forbidden") and a descriptive error message. curl -i http://api.stripe.com 2. Google Cloud API: Responds with 403 and a descriptive error message. curl -i http://compute.googleapis.com/compute/v1/projects/project /regions/region/addresses 3. Shopify API: Responds with 403 and a descriptive error message. curl -i http://shop.myshopify.com/admin/api/2021-07/shop.json 4. NPM Registry API: Responds with 426 ("Upgrade Required") and a descriptive error message. curl -i -X PUT -H 'content-type: application/json' -d '{}' 'http: //registry.npmjs.org/-/user/org.couchdb.user:npm' 5. Fastmail JMAP API: The whole HTTP interface seems to be disabled. curl -i -H 'Authorization: Bearer foo' http://api.fastmail.com/ jmap/session 6. Mailjet: The socket responds with completely empty payload. curl -i -X POST --user "user:pass" http://api.mailjet.com/v3.1/ send -H 'Content-Type: application/json' -d '{}' However, at the time of originally writing this post around 2024-05-23, the following APIs did respond with HTTP-to-HTTPS redirects: 1. ActiveCampaign API curl -i -H "Api-Token: 123abc-def-ghi" http:// 123456demo.api-us1.com/api/3/accounts 2. Atlassian Jira REST API curl -i http://jira.atlassian.com/rest/api/latest/issue/JRA-9 3. Anthropic API curl -i http://api.anthropic.com/v1/messages --header "x-api-key: 1" --header "anthropic-version: 2023-06-01" --header "content-type: application/json" --data '{}' 4. Auth0 curl -i 'http://login.auth0.com/api/v2/organizations' -H 'Accept: application/json' -H 'Authorization: Bearer foo' 5. Cloudflare API curl -i http://api.cloudflare.com/client/v4/accounts/ abf9b32d38c5f572afde3336ec0ce302/rulesets 6. Datadog curl -i http://api.datadoghq.com/api/v2/integration/gcp/accounts 7. Deno Subhosting API curl -i http://api.deno.com/v1/organizations/ 11111111-2222-3333-4444-555555555555/projects 8. DigitalOcean curl -i -X GET "http://api.digitalocean.com/v2/actions" -H "Authorization: Bearer foo" 9. Facebook Graph API curl -i 'http://graph.facebook.com/me?access_token=foo 10. Fastly API curl -i -H "Fastly-Key: foo" "http://api.fastly.com/ current_customer" 11. Figma API curl -i -H 'X-FIGMA-TOKEN: 123' 'http://api.figma.com/v1/me' 12. GitHub API curl -i http://api.github.com/user 13. Gitlab API curl -i http://gitlab.com/api/v4/audit_events 14. HackerOne API curl -i "http://api.hackerone.com/v1/me/organizations" -X GET -u "user:token" -H 'Accept: application/json' 15. Hetzner Cloud API curl -i -H "Authorization: Bearer 123" "http://api.hetzner.cloud/ v1/certificates" 16. Hubspot API curl -i --request GET --url http://api.hubapi.com/account-info/v3 /api-usage/daily/private-apps --header 'authorization: Bearer YOUR_ACCESS_TOKEN' 17. IBM Cloud API curl -i "http://iam.cloud.ibm.com/identity/token" -d "apikey= YOUR_API_KEY_HERE&grant_type= urn%3Aibm%3Aparams%3Aoauth%3Agrant-type%3Aapikey" -H "Content-Type: application/x-www-form-urlencoded" -H "Authorization: Basic Yng6Yng=" 18. Instagram Basic Display API curl -i 'http://graph.instagram.com/me/media?fields=id,caption& access_token=foo' 19. Linear API curl -i -X POST -H "Content-Type: application/json" http:// api.linear.app/graphql 20. Mastodon API (on mastodon.social) curl -i http://mastodon.social/api/v1/timelines/home 21. Microsoft Graph API curl -i http://graph.microsoft.com/v1.0/me/messages 22. Netlify API curl -i -H "User-Agent: foo" -H "Authorization: Bearer foo" http: //api.netlify.com/api/v1/sites 23. OpenAI API -- Updated to return errors since 2024-05-28. curl -i -H "Content-Type: application/json" -H "Authorization: Bearer 123" -d '{}' http://api.openai.com/v1/chat/completions 24. OVHCloud API curl -i http://api.us.ovhcloud.com/1.0/auth/details 25. Resend curl -i -X GET 'http://api.resend.com/domains' -H 'Authorization: Bearer re_123456789' -H 'Content-Type: application/json' 26. Shodan API curl -i 'http://api.shodan.io/org?key=12345' 27. Slack API curl -i -X POST -H "Content-Type: application/json" http:// slack.com/api/conversations.create 28. Tailscale API curl -i -H "Authorization: Bearer tskey-api-xxxxx" http:// api.tailscale.com/api/v2/user-invites/1 29. Twitter curl -i http://api.twitter.com/2/users/by/username/jack 30. Uber API curl -F client_secret=1 -F client_id=1 -F grant_type= authorization_code -F redirect_uri=1 -F code=1 https:// auth.uber.com/oauth/v2/token 31. UpCloud API curl -i -H 'Authorization: Basic dXNlcm5hbWU6cGFzc3dvcmQ=' http:/ /api.upcloud.com/1.3/account 32. Vercel API curl -i -H 'Authorization: Bearer foo' http://api.vercel.com/v5/ user/tokens/ 5d9f2ebd38ddca62e5d51e9c1704c72530bdc8bfdd41e782a6687c48399e8391 33. Vultr API curl -i "http://api.vultr.com/v2/account" -H "Authorization: Bearer 123" I didn't report these findings separately to all of these API providers. There were some outliers not listed here that I did contact, with varying results. More on that later. Take each individual result with a grain of salt: I had to test some of these APIs without valid credentials, or with credentials used in documentation examples. But the overall pattern indicates that the habit of APIs redirecting HTTP requests to HTTPS is quite widespread. Why is that? Best Practices Need Practice Too When speaking with people about this topic, many have noted that HTTP-to-HTTPS redirects from APIs have obvious downsides - in hindsight. Redirects for user-facing applications are often mentioned in lists best practices and cheat sheets, like the ones published by OWASP (The Open Worldwide Application Security Project). Recommendations specifically aimed for APIs seem rare in contrast. I found just few mentions, for example an excellent PDF slideset called "Common API Security Pitfalls" by Philippe De Ryck, buried deep within the OWASP website: Slide 8 of "Common API Security Pitfalls": "API-only endpoints should disable HTTP and only need to support HTTPS." Slide 8 of "Common API Security Pitfalls". Emphasis added to highlight the relevant section. My Google-fu might just be bad. But maybe each best practice item recommending HTTP-to-HTTPS redirects for user-facing sites should have an explicit caveat attached, prominently advising against such redirects for APIs. Therefore I opened an issue that suggests amending OWASP's Transport Layer Security Cheat Sheet accordingly. Bonus Round: Popular APIs That Respond In Plaintext While reviewing the list of APIs, I bumped into some popular ones that neither redirected nor failed unencrypted requests. They just responded to unencrypted HTTP requests with unencrypted HTTP responses, without enforcing HTTPS at any stage. Maybe they had their reasons, or maybe they had just accidentally misconfigured their reverse proxies. Regardless, seeing that they all handle potentially sensitive data, I contacted these API providers through their respective security channels and explained the problem. The providers are listed below in the order of reporting. I'll unredact their names and details when they've given a definite response, or otherwise after a reasonable amount of time has passed. * Provider A: Reported on 2024-05-17 through their vulnerability reporting email address. Awaiting response. * Provider B: Reported on 2024-05-21 through their HackerOne program. Got a prompt triage response, stating that attacks requiring MITM (or physical access to a user's device) are outside the scope of the program. Sent back a response explaining that MITM or physical access was not required for sniffing. Awaiting response. * Provider C: Reported on 2024-05-21 through their security email address. Awaiting response. * VirusTotal API: Reported on 2024-05-21 through Google's Bug Hunters site (VirusTotal is owned by a Google subsidiary that got merged into Google Cloud). The API responds in plaintext to requests like for example this (where $API_KEY is a valid API key): curl -i -H 'x-apikey: $API_KEY' http://www.virustotal.com/api/v3/ ip_addresses/1.1.1.1 The report got promptly triaged. Received a response on 2024-05-24, cited in part below: We've decided that the issue you reported is not severe enough for us to track it as a security bug. When we file a security vulnerability to product teams, we impose monitoring and escalation processes for teams to follow, and the security risk described in this report does not meet the threshold that we require for this type of escalation on behalf of the security team. Conclusion Redirecting HTTP to HTTPS for APIs can be more harmful than helpful due to the nature of APIs. Unlike user-facing web pages, APIs are primarily consumed by other software. API clients often follow redirects automatically and do not maintain state or support security headers like HSTS. This can lead to silent failures where sensitive data in each API request is initially transmitted in plaintext over the network, unencrypted. Let's adopt a fail-fast approach and disable the HTTP interface entirely or return clear error responses for unencrypted requests. This ensures that developers can quickly notice and fix accidental use http:// URLs to https://. Several well-known and popular APIs also redirect HTTP requests to HTTPS. This behavior seems to be widespread. Maybe it's time we amend best practices to explicitly recommend that APIs flat our reject unencrypted requests. Huge thanks to Juhani Eronen (NCSC-FI) and Marko Laakso (OUSPG) for their help and guidance during writing this post.