Multi-threaded web server example - performance tuning?
lazarus / 38-Networking and Web Programming

----------------------------------------------------------------------
botster · 2019-03-25 00:49

I found the following simple webserver example on Rosetta Code and
adapted it to be multi-threaded by adding the 'cthreads' unit and
changing "Serv.Threaded" to True. I have no idea if that was the
preferred way to do that since I cannot find any documentation for
'fphttpserver'. I compiled it with just the "-O3" option.

Using the Apache HTTP benchmarking tool to test it for concurrent
performance, I found the results disappointing:

> ab -c 500 -n 500 http://localhost:8080/
QuoteThis is ApacheBench, Version 2.3 <$Revision: 1748469 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd,
http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Finished 500 requests


Server Software:       
Server Hostname:        localhost
Server Port:            8080

Document Path:          /
Document Length:        12 bytes

Concurrency Level:      500
Time taken for tests:   0.302 seconds
Complete requests:      500
Failed requests:        0
Total transferred:      46000 bytes
HTML transferred:       6000 bytes
Requests per second:    1657.92 [#/sec] (mean)
Time per request:       301.583 [ms] (mean)
Time per request:       0.603 [ms] (mean, across all concurrent
requests)
Transfer rate:          148.95 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0   13  17.4      0      41
Processing:     6   31  25.1     20     223
Waiting:        6   19  17.0     18     217
Total:          6   44  39.0     20     241

Percentage of the requests served within a certain time (ms)
  50%     20
  66%     76
  75%     84
  80%     86
  90%     94
  95%    103
  98%    117
  99%    121
 100%    241 (longest request)

Here is the code:
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i program
HelloWorldServer;{$mode objfpc}{$H+} uses    cthreads, Classes,
fphttpserver; Type    TTestHTTPServer = Class(TFPHTTPServer)   
public        procedure HandleRequest(            Var ARequest:
TFPHTTPConnectionRequest;            Var AResponse :
TFPHTTPConnectionResponse); override;    end; Var    Serv :
TTestHTTPServer; procedure TTestHTTPServer.HandleRequest(    var
ARequest: TFPHTTPConnectionRequest;    var AResponse:
TFPHTTPConnectionResponse);     Var        F : TStringStream;    
begin    F:=TStringStream.Create('Hello,World!');    try       
AResponse.ContentLength:=F.Size;        AResponse.ContentStream:=F;  
    AResponse.SendContent;        AResponse.ContentStream:=Nil;   
finally        F.Free;    end;end; begin   
Serv:=TTestHTTPServer.Create(Nil);    try        Serv.Threaded:=True; 
      Serv.Port:=8080;        Serv.AcceptIdleTimeout:=1000;       
Serv.Active:=True;    finally        Serv.Free;    end;end. 
I have searched the forum (& elsewhere) and read several posts, but
found only two that seem even somewhat relevant.

The first is:
http://forum.lazarus.freepascal.org/index.php/topic,21524.msg126020.ht
ml#msg126020
But, I don't understand why one would give a threaded server its own
thread. Why not just let the server handle threads?

The second is:
http://forum.lazarus.freepascal.org/index.php/topic,31340.msg200664.ht
ml#msg200664
The last post in that thread (by Leledumbo) says:
QuotePaste server uses thread pool, fphttpserver doesn't. I think
that's the biggest factor if you compare multithreaded feature of a
web server. Michael as the author of fpWeb and the underlying fcl-web
package once said he actually never meant the embedded http server to
be used in production
Okay then.

Is there a package that is better suited to this simple example?

Or, is there a way to optimize this example for better performance?

----------------------------------------------------------------------
Leledumbo · 2019-03-25 05:11

Quote from: botster on March 25, 2019, 01:49:03 amIs there a package
that is better suited to this simple example?
No ATM.
Quote from: botster on March 25, 2019, 01:49:03 amOr, is there a way
to optimize this example for better performance?
Use what's meant to be used for production level performance:
apache/nginx as reverse proxy listening to fpweb's fastcgi service.
That's what I use at work.

----------------------------------------------------------------------
vangli · 2019-03-25 06:50

For my own interrest, did you try also using 'cmem' unit, as this in
documetation is said to be much faster on some systems?

uses
    cthreads, cmem, Classes, fphttpserver;

It could be intereseting to see if it changes something in your test.

Bent

----------------------------------------------------------------------
botster · 2019-03-26 05:33

Bent, I just tried it according to your suggestion, ie. with the
'cmem' unit.

It does seem to be just slightly better performance with more
consistent numbers over several test runs. But, it is still relatively
slow. Here are the numbers for the best run (what gets me is that
228ms outlier):

QuoteConcurrency Level:      500
Time taken for tests:   0.230 seconds
Complete requests:      500
Failed requests:        0
Total transferred:      46000 bytes
HTML transferred:       6000 bytes
Requests per second:    2174.92 [#/sec] (mean)
Time per request:       229.894 [ms] (mean)
Time per request:       0.460 [ms] (mean, across all concurrent
requests)
Transfer rate:          195.40 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    6  10.5      0      29
Processing:     0    7  14.0      3     204
Waiting:        0    6  13.7      2     204
Total:          0   13  20.4      4     228

Percentage of the requests served within a certain time (ms)
  50%      4
  66%     11
  75%     22
  80%     37
  90%     38
  95%     39
  98%     41
  99%     41
 100%    228 (longest request)

Thank you for the suggestion.

----------------------------------------------------------------------
vangli · 2019-03-26 07:17

Lee, thanks for doing the alternative test with cmem. I have never
done it myself, just taking for granted that cmem is faster and at
least do not hurt. It seems that the claim for faster memory
management in multithreaded applications are realy true. Good to know.

Hope you find your true bottleneck, whatever it is.

Regards Bent, Oslo, Norway

----------------------------------------------------------------------
asdf121 · 2019-03-26 11:48

You could try some other memory managers like FastMM4, ScaleMM or
SynScaleMM. Maybe they perform much better in multithreading
Other idea might be to optimize your code at all. Instead of creating
a new TStringStream each time, use one to serve all as it does the
same thing for all clients.
Using compression like gzip (if supported) would also help, at least
for bigger data.

----------------------------------------------------------------------
mr-highball · 2019-03-26 13:37

Have you seen this library?
https://github.com/fundamentalslib/fundamentals5

specifically : https://github.com/fundamentalslib/fundamentals5/blob/m
aster/Source/HTTP/flcHTTPServer.pas#L280

I've been meaning play around with the http components to see how they
stack up to some other packages I've used, but I just haven't gotten
around to it...

----------------------------------------------------------------------
botster · 2019-03-26 18:43

Quote from: asdf121 on March 26, 2019, 12:48:14 pmYou could try some
other memory managers like FastMM4, ScaleMM or SynScaleMM. Maybe they
perform much better in multithreading
Other idea might be to optimize your code at all. Instead of creating
a new TStringStream each time, use one to serve all as it does the
same thing for all clients.

SynScaleMM requires FastMM4, and I can get neither FastMM4 nor ScaleMM
to compile even after switching the mode to 'delphi'. I tried using
only one instance of TStringStream by moving its declaration to the
global scope and creating & initializing it once in the main
procedure. Performance is about the same, but now I get a significant
number of failed requests.


Quote from: mr-highball on March 26, 2019, 02:37:53 pmHave you seen
this library?
https://github.com/fundamentalslib/fundamentals5
I took a cursory look at that. It does not appear to implement multi-
threading. I may have missed it.


Thank you all for the ideas.

It seems that simply plugging in a few units and coding up a
comparably performing webserver, even as a trivial example, is not
easily doable. To me, that is sadly unfortunate. The first
professional work I ever did was with Turbo Pascal.

----------------------------------------------------------------------
mr-highball · 2019-03-26 19:26

You may want to consider taking another look at fundamentals
There's use of a critical section in the http server class for
handling events and the underlying server is a TF5TcpServer which even
mentions in the comments at the top of the unit "making use of worker
thread"

Like I said, I haven't had time to go through everything but I believe
it is a good option. Additionally you could install indy with
fpcupdeluxe which may also do what you want.

----------------------------------------------------------------------
botster · 2019-03-26 22:33

Quote from: mr-highball on March 26, 2019, 08:26:07 pmYou may want to
consider taking another look at fundamentals

... Additionally you could install indy with fpcupdeluxe which may
also do what you want.

You are right. The TCP Server, a dependency of the HTTP Server, does
indeed use threads. However, I can find no documentation whatsoever
for Fundamentals5; not even any examples for guidance. Maybe it's just
me, but I don't think I should have to analyze the code just to
attempt to figure out how to use it.

I downloaded and launched fpcupdeluxe; selected my preferred install
directory, went to Modules, and attempted to install 'indy':
Quotefpcupdeluxe: info: UniversalInstaller (BuildModule: indy):
Building module indy...

fpcupdeluxe: info: UniversalInstaller (BuildModule: indy): Building
module indy running all InstallExecute commands in:
Name=indy
Description="Indy 10 networking library for FPC / Lazarus"
Installdir=$(basedir)/ccr/$(name)
Enabled=0
SVNURL=https://svn.atozed.com:444/svn/Indy10/trunk/
UserName=Indy-Public-RO
Password=
ArchiveURL=https://indy.fulgan.com/ZIP/Indy10_5495.zip
AddPackage=$(Installdir)/Lib/indylaz.lpk
UnInstall=rm -Rf $(Installdir)
Category=miscellaneous

fpcupdeluxe: info: UniversalInstaller (InstallPackage): Installing
indylaz version 10.6.2
Executing: /home/user/Coding/FPC-Lazarus/lazarus/lazbuild --version
(working dir: )
fpcupdeluxe: info: UniversalInstaller (InstallPackage): Error trying
to add package indylaz
fpcupdeluxe: WARNING: UniversalInstaller (InstallPackage): Error while
installing package /home/user/Coding/FPC-
Lazarus/ccr/indy/Lib/indylaz.lpk.


SUCCESS: Fpcupdeluxe ended without errors.

Hmmm, wonder what the error was exactly.


Anyway, at this point, I have had enough frustration. This is what
invariably happens every time I have taken yet another hopeful look at
Free Pascal. Every time, no, inadequate, and/or incongruent
documentation or not being able to simply "get there from here" has
cost me many lost hours of searching and trying without accomplishing
anything productive.


Thank you all for your help.

----------------------------------------------------------------------
mr-highball · 2019-03-26 23:19

Hmm, well that's unfortunate. Better luck next time I suppose? fpc
takes some getting used to, but once you do it's a wonderful tool.

If you have room for just a little bit more frustration...
--
Out of curiosity I looked back at your original benchmarks and was
wondering what you would expect to be "good results"? Your longest
request was 241ms, amd unless I'm missing something, that doesn't seem
too bad when thinking back to response times I've seen in .net or
java.
For example do you have comparison metrics to other web
frameworks/languages for similar test runs? If you do I could try and
put together a test with fundamentals or another implementation to see
how well they stack up.

----------------------------------------------------------------------
asdf121 · 2019-03-27 12:03

Just download Indy from https://indy.fulgan.com/ and add its units to
your search path.
When compiling via command line add the needed subdir via Code: Pascal
 [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i
-FuIndy10dir/*Documentation can be found online:
http://ww2.indyproject.org/docsite/html/frames.html?frmname=topic&frmf
ile=index.html or on Indy's website for download.
I think it's the TIdHTTPServer but you'll find a lot of examples for
Indy via Google

fundamentals5 code looks indeed very good and clean but there is
absolutely no documentation or examples    So might be a good idea to
not use it or at least it's nothing you can use without spending much
time and having good coding skills...

----------------------------------------------------------------------
Akira1364 · 2019-03-27 20:48

Apart from not using a thread pool, FCL-Web kind of suffers from way-
too-much-try-finally syndrome internally.

I'm not aware of any libraries for Object Pascal other than the new
version of Brook Framework:

https://github.com/risoflora/brookframework

that would (or physically could) get close to the kind of performance
you seem to want.

----------------------------------------------------------------------
botster · 2019-03-27 22:32

Quote from: mr-highball on March 27, 2019, 12:19:07 amIf you have room
for just a little bit more frustration...

Lol. Okay, I'll bite. 

Quote from: mr-highballOut of curiosity I looked back at your original
benchmarks and was wondering what you would expect to be "good
results"?

That is a discerning question since the meaning of "good results" is
bound to be subjective without defining some criteria.

To me, that term means showing performance, productivity (RAD), or
both. In regards to productivity, I was thinking primarily of doing
more with less (ie. fewer lines of code); to a lesser extent, ease of
library/extension management; and, minorly, a general feeling of
compilation time.

Here's a little of my backstory for context. I've recently become
interested in ActivityPub (World Wide Web Consortium protocol for
decentralized social networking). While looking at the languages used
for various implementations, I saw that most were written in
scripting, or some form of JIT compiled scripting, languages. Only a
few are written in truly compiled languages, including Golang and
Rust.

I thought that creating an open-source project for ActivityPub in Free
Pascal might be a great way to showcase FP, get some eyes on it, and
perhaps garner some interest. The question I needed to answer first,
though, is would FP produce comparatively "good (enough) results" to
counter the undoubtedly forthcoming question, "Why should Free Pascal
be used when we can have A and/or B with X language?"

So, I did a 'net search for, "simple multi-threaded http server
example in X" where X = (Free Pascal | Golang | Rust). Simple examples
were easily found on page one of the results for Golang and Rust. A
simple FP example (from Rosetta Code) wasn't found until page two;
which, I suppose, is understandable since the know-it-alls say that
Pascal is a "dead" language.

My tests show Golang to be almost three times faster than FP. And, the
example I found on Stack Overflow, which I adapted by removing just
two lines related to timing, is only fifteen lines of code including
whitespace.

It took just a bit more to adapt the multi-threaded (MT) server
example in the Rust community's, "The Book". I ended up with 27 lines
and performance, surprisingly, slightly worse than that of Golang.

Neither Golang nor Rust required me to exercise package installation
contortions to implement a simple MT HTTP server. To be fair, neither
did FP when using 'fphttpserver', but it seems that is not an
acceptable solution.

Using the Apache HTTP server benchmarking tool to send a total of 500
concurrent requests, FP gives consistent total times in the low 200's
of milliseconds. Golang is consistent at around 70-80ms, and Rust,
80-100ms.

So, it appeared to me that, comparatively speaking, both Golang and
Rust have both A and B; Free Pascal, not so much. Hence, my
disappointment.

I'd be glad to post the relative code and benchmark results if you're
interested.


And, just BTW...
Quote from: mr-highballfpc takes some getting used to, but once you do
it's a wonderful tool.

I don't doubt that at all which is why I keep taking yet another
hopeful look. It's not getting used to the tool itself with which I
have so much difficulty. It's trying to navigate its ecosystem.

----------------------------------------------------------------------
Akira1364 · 2019-03-27 23:07

Quote from: botster on March 27, 2019, 11:32:58 pmUsing the Apache
HTTP server benchmarking tool to send a total of 500 concurrent
requests, FP gives consistent total times in the low 200's of
milliseconds. Golang is consistent at around 70-80ms, and Rust,
80-100ms.
Those don't actually seem as far off as it sounded like from what you
were saying before. It does however seem like you must be running
pretty high-end hardware, now that I think about the results
relatively.

How does this version of the program perform for you? (Note how I've
increased the QueueSize parameter of the server... the default is 5.
You can increase it further from there and see how that works, also.)

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i program
HelloWorldServer; {$mode ObjFPC}{$H+}{$ImplicitExceptions Off} uses 
CMem,  {$ifdef Unix}  CThreads,  {$endif}  FPHTTPServer; type 
TTestHTTPServer = class(TFPHTTPServer)  public    procedure
HandleRequest(var ARequest: TFPHTTPConnectionRequest;                
          var AResponse: TFPHTTPConnectionResponse); override;  end;  
procedure TTestHTTPServer.HandleRequest(var ARequest:
TFPHTTPConnectionRequest;                                          var
AResponse: TFPHTTPConnectionResponse);  begin    with AResponse do
begin      Content := 'Hello,World!';      Code := 200;     
SendContent();    end;  end; begin  with TTestHTTPServer.Create(nil)
do try    Threaded := True;    Port := 8080;    QueueSize := 25;   
AcceptIdleTimeout := 1000;    Active := True;  finally    Active :=
False;    Free();  end;end.
Regarding your Indy "package" download/install attempt from before, do
you actually have the Lazarus IDE installed? It might be easier in
general for you to just get the source off github, too:

https://github.com/graemeg/indy.git

----------------------------------------------------------------------
Akira1364 · 2019-03-28 00:15

I just did another little test of my own, with a modified version of
an example from Brook 5.0 (which I linked to in an earlier comment.)
The program looks like this:

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i program
HelloHTTPServer; {$mode ObjFPC}{$ImplicitExceptions Off} uses  CMem, 
{$ifdef Unix}  CThreads,  {$endif}  SysUtils,  BrookLibraryLoader, 
BrookHTTPRequest,  BrookHTTPResponse,  BrookHTTPServer; type 
THTTPServer = class(TBrookHTTPServer)  protected    procedure
DoRequest(ASender: TObject;                         ARequest:
TBrookHTTPRequest;                        AResponse:
TBrookHTTPResponse); override;  end;   procedure
THTTPServer.DoRequest(ASender: TObject;                              
  ARequest: TBrookHTTPRequest;                                 
AResponse: TBrookHTTPResponse);  begin   
AResponse.Send('Hello,world!', 'text/plain', 200);  end; begin  if not
TBrookLibraryLoader.Load(TBrookLibraryLoader.LIB_NAME) then begin   
WriteLn(ErrOutput, 'Library not loaded.');    Halt(1);  end;  with
THTTPServer.Create(nil) do begin    Port := 8080;    ConnectionLimit
:= 100;    Threaded := True;    NoFavicon := True;    Open();    if
not Active then Exit();    ReadLn;    Free();  end;end.
The AB output was, as I expected, quite good IMO:

QuoteD:\fpcupdeluxe\lazarus\components\brookframework\Examples\Console
\FPC>ab -c 500 -n 500 http://localhost:8080/
This is ApacheBench, Version 2.3 <$Revision: 1843412 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd,
http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Finished 500 requests


Server Software:
Server Hostname:        localhost
Server Port:            8080

Document Path:          /
Document Length:        12 bytes

Concurrency Level:      500
Time taken for tests:   0.062 seconds
Complete requests:      500
Failed requests:        0
Total transferred:      57000 bytes
HTML transferred:       6000 bytes
Requests per second:    8001.79 [#/sec] (mean)
Time per request:       62.486 [ms] (mean)
Time per request:       0.125 [ms] (mean, across all concurrent
requests)
Transfer rate:          890.82 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   1.0      0      16
Processing:    16   32   9.3     31      47
Waiting:        0   31  10.6     31      47
Total:         16   33   9.3     31      47

Percentage of the requests served within a certain time (ms)
  50%     31
  66%     31
  75%     31
  80%     47
  90%     47
  95%     47
  98%     47
  99%     47
 100%     47 (longest request)
Come to think of it, I do also think Indy has a thread-pool-based
scheduler component that can be used with the HTTP server, so that
might be another library test worth looking into also.

----------------------------------------------------------------------
botster · 2019-03-30 22:01

Quote from: Akira1364Time taken for tests:   0.062 seconds
That is a darn good number depending, of course, on your hardware. I'm
not running anything high-end as you suspected; just a Core 2 Duo
E7600 @ 3GHz with 4GB RAM.

I looked at the Brook framework. I'd have to download and compile a C
library to use it. So, I can write in Pascal, but I'd have to use a C
library to get performance? IMO, that would not be a very good
showcase scenario for Free Pascal and could possibly lead to being
mocked even of only lightheartedly.

I finally waded through some of the Indy10 documentation and found
only a couple of GUI-based examples online. After much trial and
error, I was able to get a minimal server operational.

It's performance is slightly worse than fphttpserver (@ low 300's of
ms) turning in test times of roughly 300ms-500ms. Granted, that is
without thread-pooling. But none of the other minimalistic servers I
coded up and tested used pooling either, unless they did so internally
by default.

----------------------------------------------------------------------
Leledumbo · 2019-03-31 16:05

Quote from: botster on March 30, 2019, 11:01:32 pmI looked at the
Brook framework. I'd have to download and compile a C library to use
it. So, I can write in Pascal, but I'd have to use a C library to get
performance? IMO, that would not be a very good showcase scenario for
Free Pascal and could possibly lead to being mocked even of only
lightheartedly.
The author wants to:
- Make Brook available not only for modern Pascal languages (v4 was
also available for Python if I'm not mistaken)
- Utilize the already proven libmicrohttpd, PCRE and some other
existing libraries (i.e. no wheel reinventing, indeed libmicrohttpd is
nicely implemented and quite oriented towards performance)

However, he knew that many of us are die hard Pascal fans that's
alergy to non-pure Pascal code, so he created another one:
https://github.com/risoflora/brookfreepascal, which is closer to the
original Brook up to v3 (I still use this at work). The unfortunate
part is that the two are not interchangeable, brookfreepascal is not
simply brookframework with libsagui replaced by the same library
implemented in pure Pascal.

----------------------------------------------------------------------
botster · 2019-04-03 20:20

Quote from: Leledumbo on March 31, 2019, 06:05:04 pmHowever, he knew
that many of us are die hard Pascal fans that's alergy to non-pure
Pascal code, so he created another one:
https://github.com/risoflora/brookfreepascal, which is closer to the
original Brook up to v3 (I still use this at work). The unfortunate
part is that the two are not interchangeable, brookfreepascal is not
simply brookframework with libsagui replaced by the same library
implemented in pure Pascal.

Indeed, they appear to be significantly different. BrookFreePascal
(BFP) appears to be mainly a CGI framework. (The README for the Brook
framework doesn't say exactly what it is or for what use cases it
would be applicable  ) A screenshot on the BFP home page
(https://risoflora.github.io/brookfreepascal/) suggests that it could
be used to create an embedded server. So, I looked through the code a
bit, and the only possible HTTP server related unit I found is
BrookFCLHttpAppBroker which ... relies on FPHttpServer.

At one point in my research, I must have gotten some search terms
right and discovered a program in an article that, I believe, you
wrote at https://pascalgeek.blogspot.com/2012/06/encryption-
decryption-and-asynchronous.html ("fcl-net, the undocumented
treasure"). It's a relatively simple multi-threaded TCP server! I
struggled and over time managed to adapt that program to correctly
read and respond to a request from a web browser. Aside from a just a
few minor changes including adding the 'cmem' unit and defining the
"Response" buffer globally, here is what I ended up with for the
TClientHandlerThread.Execute procedure:
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i procedure
TClientHandlerThread.Execute;var  Request: array[0..1023] of char; 
Done: Boolean;begin  Done := false;  repeat    try     
FClientStream.Read(Request, Length(Request))     
FClientStream.Write(Response, Length(Response))     
FClientStream.Free;      Exit;    except      on e: EStreamError do
begin        Done := true;      end;    end;  until Done; 
WriteLn(AddrToString(FClientStream.PeerAddress) + '
disconnected');end; 
That server program yields results similar to the FPHttpServer-based
server example, from Rosetta Code, that I started with:
QuoteConcurrency Level:      500
Time taken for tests:   0.232 seconds
Complete requests:      500
Failed requests:        0
Total transferred:      32000 bytes
HTML transferred:       12500 bytes
Requests per second:    2157.69 [#/sec] (mean)
Time per request:       231.729 [ms] (mean)
Time per request:       0.463 [ms] (mean, across all concurrent
requests)
Transfer rate:          134.86 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0   12  12.0     14      28
Processing:     0    8  14.0     10     198
Waiting:        0    6  13.7      5     198
Total:          0   20  21.8     28     218

Percentage of the requests served within a certain time (ms)
  50%     28
  66%     36
  75%     36
  80%     37
  90%     38
  95%     38
  98%     39
  99%     59
 100%    218 (longest request)

The numbers look good except for that last 1% of requests served
outlier of 218ms. Occasionally, though, that same server will yield an
anomalous result:
QuoteConcurrency Level:      500
Time taken for tests:   0.094 seconds
Complete requests:      500
Failed requests:        0
Total transferred:      32000 bytes
HTML transferred:       12500 bytes
Requests per second:    5317.68 [#/sec] (mean)
Time per request:       94.026 [ms] (mean)
Time per request:       0.188 [ms] (mean, across all concurrent
requests)
Transfer rate:          332.35 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        5   18  10.6     15      34
Processing:     5   14   3.7     14      32
Waiting:        3    9   4.6      9      29
Total:         18   32  11.0     24      65

Percentage of the requests served within a certain time (ms)
  50%     24
  66%     43
  75%     44
  80%     45
  90%     46
  95%     46
  98%     47
  99%     47
 100%     65 (longest request)

ALL good numbers and definitely comparable with the Golang and Rust
server implementations. But unfortunately, this type of result is the
exception, and I have no clue how to reproduce it consistently.

And, confusingly to me, the FPHttpServer-based server under "Siege"
(`siege -b -c500 -r1 http://localhost:8080/`) will be relatively
consistent in performance (1.49-3.81 secs) while the TCP server's
performance will get progressively worse on each successive run
eventually slowing to a crawl. Golang and Rust server implementations
are both rather consistent under siege at around 0.22-1.28 secs.

Anyway, I've spent more time than I should have exploring this
potential project. I sincerely appreciate everyone's comments and
suggestions.

----------------------------------------------------------------------
zamronypj · 2019-08-20 23:22

Running apache benchmark to test http server on same machine may not
yield accurate result.  Higher CPU usage by apache benchmark may
result in poor performance of http server due to both competing for
CPU usage.