how to speed up fphttpclient?
lazarus / 38-Networking and Web Programming

----------------------------------------------------------------------
Hobbit1972 · 2021-06-04 05:54

Hi,

I need to retrieve several https documents. Using fphttpclient this is
very slow:
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i  
InitSSLInterface;  HTTPClient := tfphttpclient.create(nil);  with
HTTPClient do begin    AllowRedirect := true;   
AddHeader('accept','application/json');    AddHeader('Cache-
Control','no-cache');  end;    ...  response :=
HTTPClient.get(requeststr);   
In perl I sped up things by using caching:
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i my $ua =
LWP::UserAgent->new();$ua->protocols_allowed( [ 'http','https']
);$ua->ssl_opts( verify_hostnames => 0 );    my $conn_cache =
LWP::ConnCache->new;$conn_cache->total_capacity([1])
;$ua->conn_cache($conn_cache) ; 
Is there caching in fphttpclient too? Or do I need to switch to
another package?

Thanks!

----------------------------------------------------------------------
engkin · 2021-06-04 14:43

Probably using KeepConnection should be enough:
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i  
c.KeepConnection:=True;
But if you want to have more control, provide your own ssl handler. To
disable the verification of the certificate for instance:
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i uses ...
openssl, fphttpclient, opensslsockets, sslsockets; type  
TExOpenSSLSocketHandler=class(TOpenSSLSocketHandler)  public   
constructor Create;override;  end; constructor
TExOpenSSLSocketHandler.Create;begin  inherited Create; 
VerifyPeerCert:=False;end; 
And set it as the default handler. Assuming you understand what that
means:
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i var 
c:TFPHTTPClient; begin  InitSSLInterface; 
TSSLSocketHandler.SetDefaultHandlerClass(TExOpenSSLSocketHandler);  
c:=TFPHTTPClient.Create(nil);  c.KeepConnection:=True;  ...
I did not test the above code. It is just to give you the idea.

----------------------------------------------------------------------
Hobbit1972 · 2021-06-04 15:31

Quote from: engkin on June 04, 2021, 04:43:12 pmProbably using
KeepConnection should be enough:
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i  
c.KeepConnection:=True;Ah - thank you. That alone made execution about
three times faster.


Quote from: engkin on June 04, 2021, 04:43:12 pmBut if you want to
have more control, provide your own ssl handler. ...
This did not give any more benefits - but might do once the overall
number of http-gets increases.

Thx!l

----------------------------------------------------------------------
engkin · 2021-06-04 16:35

Quote from: Hobbit1972 on June 04, 2021, 05:31:19 pmQuote from: engkin
on June 04, 2021, 04:43:12 pmBut if you want to have more control,
provide your own ssl handler. ...
This did not give any more benefits - but might do once the overall
number of http-gets increases.

Unless you deal with many servers, no benefit is expected from it when
KeepConnection works because it runs *once* in this case.

----------------------------------------------------------------------
engkin · 2021-06-04 17:27

If you still need more speed, your next step would be using threads.