Slow request
lazarus / 38-Networking and Web Programming

----------------------------------------------------------------------
M[a]nny · 2021-11-14 18:41

Hey guys,
I'm using TFPHttpClient from fphttpclient uses to create this request:

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i function
GetRequest: string;var  Client: TFPHttpClient;        Response :
TStringStream; begin        Client := TFPHttpClient.Create(nil);     
 Client.AddHeader('User-Agent', 'Mozilla/5.0 (compatible; fpweb)');  
    Client.AddHeader('Content-Type', 'application/json;
charset=UTF-8');        Client.AddHeader('Accept',
'application/json');  Client.KeepConnection := True;       
//Client.AllowRedirect := true;        Response :=
TStringStream.Create('');        try    try        Client.Post('... my
url ...', Response);          except on E:Exception do               
ShowMessage('Something bad happened : ' + E.Message);          end;  
    finally    Client.RequestBody.Free;    Client.Free;         Result
:= Response.DataString;    Response.Free;                       
end;end;
It works well but problem is that this request takes about 700ms to
response. If i run it in Chrome browser, it takes about 67ms only!
Interesting thing is when i run it again, it takes much less time, but
it still isn't fast as Chrome. What's wrong?

----------------------------------------------------------------------
trev · 2021-11-14 20:59

I'm guessing the delay is caused by DNS lookups which would explain
why the second time is quicker.

----------------------------------------------------------------------
M[a]nny · 2021-11-14 21:39

What can i do to make it faster?

----------------------------------------------------------------------
trev · 2021-11-14 21:52

Run a caching nameserver... when such a nameserver retrieves the
answer to a client query, it not only returns the answer to the client
but also stores the answer in its cache for the period of time allowed
by the TTL value of the record. The cache can then be used as a source
for subsequent requests to speed up the total round-trip time.

----------------------------------------------------------------------
M[a]nny · 2021-11-14 22:05

If I understand correctly, this can only be used if the request and
the answer are the same. But what if the request is different?
For example:

http://mypage.com/index.php?name=Willie
http://mypage.com/index.php?name=John
http://mypage.com/index.php?name=Barnie

According to this cache logic, it shouldn't affect speed, right?
But it's not clear to me that when I run any request in Google Chrome,
it always comes back extremely quickly. Why is that?

----------------------------------------------------------------------
trev · 2021-11-14 22:47

Quote from: M[a]nny on November 14, 2021, 11:05:09 pmIf I understand
correctly, this can only be used if the request and the answer are the
same. But what if the request is different?
For example:

http://mypage.com/index.php?name=Willie
http://mypage.com/index.php?name=John
http://mypage.com/index.php?name=Barnie
You misunderstand - the lookup is for mypage.com only. None of the
rest of those urls can be found in DNS records.

QuoteBut it's not clear to me that when I run any request in Google
Chrome, it always comes back extremely quickly. Why is that?
My experience with the SeaMonkey browser is that it does its own DNS
caching. I've never used Chrome, but I'd speculate that it's also
doing its own DNS caching for a period which may well exceed a
record's TTL.

Here's an example of DNS caching on my FreeBSD server - note the 1ms
query response time:

Code: Text  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i $ dig
sentry.org ; <<>> DiG 9.16.22 <<>> sentry.org;; global options: +cmd;;
Got answer:;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 37769;;
flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL:
1 ;; OPT PSEUDOSECTION:; EDNS: version: 0, flags:; udp: 1232; COOKIE:
1aad2be30b46cf47010000006191924ce3540b5e84fdcfb0 (good);; QUESTION
SECTION:;sentry.org.               IN   A ;; ANSWER
SECTION:sentry.org.     86400   IN A    45.124.52.214 ;; Query time: 1
msec;; SERVER: 192.168.1.4#53(192.168.1.4);; WHEN: Mon Nov 15 09:48:44
AEDT 2021;; MSG SIZE  rcvd: 83

----------------------------------------------------------------------
M[a]nny · 2021-11-15 22:37

Thank you for your response, I get it now. So the problem is that my
application in Lazarus takes longer time because it needs to translate
"mypage.com" to IP address (i think it's called DNS translate or so).

It's still strange because I have a application created with Java that
sends request to the same address and it works blazingly fast (at
least as fast as Google Chrome).

So, what am I supposed to do now?

----------------------------------------------------------------------
trev · 2021-11-15 23:30

You didn't mention what OS you are using, but you could try using an
OS native API which may resolve sites faster.

* macOS: NSURLSession
* Windows: WinInet

----------------------------------------------------------------------
M[a]nny · 2021-11-16 14:28

Application runs on Windows 10.