HEAD request bandwidth impact lazarus / 38-Networking and Web Programming ---------------------------------------------------------------------- karloromic · 2019-09-25 23:36 What is the bandwidth impact per HTTPMethod('HEAD'... request? Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i uses httpsend, laz_synapse, fpopenssl, openssl; var HTTP: THTTPSend; HTTP := THTTPSend.Create;if not IsSslLoaded then InitSslInterface;Assert(IsSslLoaded = True);  HTTP.Timeout := 2000;HTTP.Sock.ConnectionTimeout := 3000;  HTTP.HTTPMethod('HEAD', 'https://www.google.com'); My Task Manager (Windows 10 x64) measures up to 12 KB (sent/received combined) per request and up to 150 KB/s (uploading/downloading combined) per request. Is that normal for HEAD request? Am i reading it right? ---------------------------------------------------------------------- zamronypj · 2019-09-26 01:10 I do not understand what you mean by normal. Response of HEAD request is similar with response of GET request except that response has no body part, only headers. If server send long headers then response get bigger. ---------------------------------------------------------------------- Thaddy · 2019-09-26 04:43 Indeed, it depends on the website. The request itself has almost no overhead, but a response can be rather big in the sense that HEAD response can be even bigger than its page content. https://en.wikipedia.org/wiki/List_of_HTTP_header_fields ---------------------------------------------------------------------- karloromic · 2019-09-26 05:38 Found it: QuoteThe standard imposes no limits to the size of each header field name or value, or to the number of fields. However, most servers, clients, and proxy software impose some limits for practical and security reasons. For example, the Apache 2.3 server by default limits the size of each field to 8,190 bytes, and there can be at most 100 header fields in a single request. Okay so no limit by definition and header can be any size..  didnt know that. Thank you. ---------------------------------------------------------------------- PascalDragon · 2019-09-26 07:05 Quote from: karloromic on September 26, 2019, 01:36:34 amMy Task Manager (Windows 10 x64) measures up to 12 KB (sent/received combined) per request and up to 150 KB/s (uploading/downloading combined) per request. Is that normal for HEAD request? Am i reading it right? Don't forget that you're also doing a TLS handshake due to the HTTPS which accounts for some as well. And the speed is probably an estimate due to the transfer taking less than a second. ---------------------------------------------------------------------- karloromic · 2019-09-26 07:17 Quote from: PascalDragon on September 26, 2019, 09:05:04 amQuote from: karloromic on September 26, 2019, 01:36:34 amMy Task Manager (Windows 10 x64) measures up to 12 KB (sent/received combined) per request and up to 150 KB/s (uploading/downloading combined) per request. Is that normal for HEAD request? Am i reading it right? Don't forget that you're also doing a TLS handshake due to the HTTPS which accounts for some as well. And the speed is probably an estimate due to the transfer taking less than a second. So then not using https would be faster? QuoteThis handshake will typically take between 250 milliseconds to half a second, but it can take longer. At first, a half second might not sound like a lot of time. The primary performance problem with the TLS handshake is not how long it takes, it is when the handshake happens. ---------------------------------------------------------------------- Thaddy · 2019-09-26 07:37 Not with google.com as in your example, since it redirects to https anyway. In your example it will therefor be even slower. On your own webserver(s) you can limit the overhead though, even to nothing at all ( e.g. my http://ip.thaddy.com has no headers at all and is accepted by all browsers ) Another issue is that sites like google use compression which causes less traffic but needs decompression on the client side. That means that on older or simpler hardware the decompression phase my be significant. E.g. the internet line speed in the Netherlands averages currently 60.70 mbps (mobile) and 99.81 (fixed) which makes header overhead frankly not noticeable. (source: https://www.speedtest.net/global-index/netherlands ) What exactly are your requirements? ---------------------------------------------------------------------- karloromic · 2019-09-26 08:05 Quote from: Thaddy on September 26, 2019, 09:37:34 amNot with google.com as in your example, since it redirects to https anyway. In your example it will therefor be even slower. On your own webserver(s) you can limit the overhead though, even to nothing at all ( e.g. my http://ip.thaddy.com has no headers at all and is accepted by all browsers ) Another issue is that sites like google use compression which causes less traffic but needs decompression on the client side. That means that on older or simpler hardware the decompression phase my be significant. E.g. the internet line speed in the Netherlands averages currently 60.70 mbps which makes header overhead frankly not noticable. What exactly are your requirements? Attachment screenshot shows headers returned by your url/site. Well since i am developing Internet Connection Monitor Application, i must assume that some of the users wont have high speed internet, so i have to take into the account potential bandwidth and speed impact on the network to assure highest possible performance while saving bandwidth and not hogging the network (some people still in 2019 have very slow internet conenction due to remote area or bad internet plan, or using limited monthly bandwidth plan - no flat). In croatia average speeds are as high as in your country, atleast mine is. ---------------------------------------------------------------------- Thaddy · 2019-09-26 08:27 That response header is the *server itself* and minimized, so that is as expected. Other  webservers have the same minimal behavior. The minimum - or close - for a handshake. ( in this particular case, you will notice "upgrade" which I won't do yet, because it is needed w/o downtime right now and this is a free project ) To obtain the real headers, examine the page in your browser (right click - source). That should give you nothing except an ip string. I wonder if an internet connection monitor has any value nowadays... Actually - apart from giving you your remote address - my website is geared towards just that: testing connections. You have to make a lightweight connection to be sure there is a connection.... ( since 2003 ) BTW: I think websniffer is filling in defaults for apache itself.... Apache gives a little less detail Anyway: now you know how to limit header overhead on your own servers.... And that headers can be huge... No way to get around that.... ---------------------------------------------------------------------- karloromic · 2019-09-26 08:58 @Thaddy Here is the result from chrome using inspector: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i /////GENERAL////////////Request URL: http://ip.thaddy.com/Request Method: GETStatus Code: 200 OKRemote Address: 185.89.4.45:80Referrer Policy: no-referrer-when-downgrade /////RESPONSE HEADERS///////////Connection: Upgrade, Keep-AliveContent-Type: text/html; charset=UTF-8Date: Thu, 26 Sep 2019 08:51:31 GMTKeep-Alive: timeout=2, max=100Server: Apache/2Transfer-Encoding: chunkedUpgrade: h2,h2cVary: Accept-Encoding,User-AgentX-Content-Encoding-Over-Network: gzip /////REQUEST HEADERS//////////////Accept: text/html,application/x html+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,applica tion/signed-exchange;v=b3Accept-Encoding: gzip, deflateAccept- Language: hr-HR,hr;q=0.9,en-US;q=0.8,en;q=0.7,bs;q=0.6Cache-Control: no-cacheConnection: keep-aliveHost: ip.thaddy.comPragma: no- cacheUpgrade-Insecure-Requests: 1User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.90 Safari/537.36 ---------------------------------------------------------------------- Thaddy · 2019-09-26 09:00 Yes. That is minimal. ---------------------------------------------------------------------- karloromic · 2019-09-26 09:01 How did you disabled any additional headers in your apache server? ---------------------------------------------------------------------- Thaddy · 2019-09-26 09:20 It is the default when all - most - security options are enabled. ( I still serve http, not https on that site, precisely to have minimum response time ) ---------------------------------------------------------------------- karloromic · 2019-09-26 09:56 Quote from: Thaddy on September 26, 2019, 11:20:22 amIt is the default when all - most - security options are enabled. ( I still serve http, not https on that site, precisely to have minimum response time ) I will try to do that on my server too