HTTP NG at PARC Testing
=======================

==================================================
Comparisons and associated tests:

W3NG/W3MUX vs. multi-tcp-connection HTTP
    Compare Base NG to Multi-connection HTTP

HTTP 1.1 without Pipelining vs. HTTP with Pipelining
    Compare Multi-connection HTTP to Pipelined HTTP
    
W3NG/W3MUX vs. HTTP 1.1 with Pipelining
    Compare Base NG to Pipelined HTTP
    
W3NG/W3MUX without pipelining W3NG/W3MUX with pipelining  
    Compare Base NG to Pipelined NG

W3MUX vs.simple record-marking (sunrpcrm)
    Compare Base NG to Sunrpcrm 
    
W3NG/W3MUX with batching vs. W3NG/W3MUX without batching 
    Compare Base NG to Batchless NG 

Native Apache vs. ILU/Apache
    Compare Native Apache to ILU Apache

libwww Webbot vs. nggetbot
    Compare libwww Webbot to Native Apache 
    
NG API vs. ILU/Apache
    Compare Base NG to ILU Apache
    
NG API vs. Native Apache
    Compare Base NG to Native Apache

W3NG vs. IIOP
    Compare Base NG to IIOP

100 mbit vs. 10 mbit
    Compare Base NG to Slowed NG 
    Compare libwww Webbot to Slowed libwww Webbot

Synchronous NG vs. Asynchronous NG
    Compare Base NG to Async NG 
    
OS independence - NT and Solaris
    Compare Base NG to Solaris NT to NT Solaris to NT NT

Data Set Independence - Microscape vs. AOL vs. MS
    Compare Base NG to AOL NG to MS NG
    
    
==================================================
NOTES:

See the file /project/http-ng/docs/ng-testlab-notes.txt
for a complete description of the directory structures on the
test machines.


See the file examples/ngtest/README in the ILU sources for
a description of the nggetbot and ngwebserver programs.


Example command lines shown below (for the Sun to Sun tests)
assume the server is running on ngsun2 and the client on ngsun1.


Tests involving the nggetbot use 10 threads [good choice?].


Tests using batching use a timeout of 50 msec [good choice?].


For ngwebserver on Win32, issue a control C to make it stop and
print out tcp stats.  For Unix, send the process the INT signal
(e.g. kill -INT <ngwebserver_pid> )


Ten runs of the client are usually done in succession using a for loop,
e.g. on Unix
     foreach f (1 2 3 4 5 6 7 8 9 10)
     nggetbot ....
     end
     
e.g. on Win32
     for %f in (1 2 3 4 5 6 7 8 9 10) do nggetbot ...


==================================================
Tests:


(Note the following can be seen in Excel spreadsheet form
in /project.http-ng/docs/tests.xls)


--------------------------------------------------
Title:           Base NG
Server:          NGWebserver - Solaris
Client:          NGGetbot - Solaris
Server Batching: Yes
Client Batching: Yes
Serialization:   No
Protocol:        W3NG
Transport:       W3MUX
Interface:       TCWA synchronous
Documents:       Microscape
Bandwidth:       100

Note:            Get TCP-dump 
[e.g. 
  su root
  enable tcpdump-3.0.4
  tcpdump -s 2000 -N host ngsun1 and host ngsun2 and tcp > tcplog.txt
]


Server (ngsun2) command line:
/opt/local/ilu/examples/ngtest/ngwebserver -httptinfo tcp_0_8080 -filebase /opt/local/http-ng/serverdocs -batch 50000 -verbose 1

Client (ngsun1) command line:
/opt/local/ilu/examples/ngtest/nggetbot -urls /opt/local/http-ng/ngtesturls/mscape_ngsun2_w3ng_w3ng_w3mux_2718.urls -batch 50000 -logfile stdout -v 1 -mt 10


Comments: 
This is the base NG - that is, it has all 3 NG layers in place.

--------------------------------------------------
Title:           Multi-connection HTTP
Server:          NGWebserver - Solaris
Client:          NGGetbot - Solaris
Server Batching: Yes
Client Batching: Yes
Serialization:   No
Protocol:        HTTP 1.0
Transport:       TCP
Interface:       TCWA synchronous
Documents:       Microscape
Bandwidth:       100

Server (ngsun2) command line:
/opt/local/ilu/examples/ngtest/ngwebserver -httptinfo tcp_0_8080 -filebase /opt/local/http-ng/serverdocs -verbose 1 -fdbudget 128

Client (ngsun1) command line:
/opt/local/ilu/examples/ngtest/nggetbot -urls /opt/local/http-ng/ngtesturls/mscape_ngsun2_http_http_8080.urls -httppinfo http_1_0 -logfile stdout -v 1 -mt 10


--------------------------------------------------
Title:           Pipelined HTTP
Server:          NGWebserver - Solaris
Client:          NGGetbot - Solaris
Server Batching: Yes
Client Batching: Yes
Serialization:   Yes
Protocol:        HTTP 1.1
Transport:       TCP
Interface:       TCWA synchronous
Documents:       Microscape
Bandwidth:       100

Server (ngsun2) command line:
/opt/local/ilu/examples/ngtest/ngwebserver -httptinfo tcp_0_8080 -ngpinfo http_1_1 -ngtinfo tcp_0_2718 -filebase /opt/local/http-ng/serverdocs -batch 50000 -verbose 1

Client (ngsun1) command line:
/opt/local/ilu/examples/ngtest/nggetbot -urls /opt/local/http-ng/ngtesturls/mscape_ngsun2_w3ng_http11_2718.urls -batch 50000 -logfile stdout -v 1 -mt 10 -pipeline -serial

--------------------------------------------------
Title:           Pipelined NG
Server:          NGWebserver - Solaris
Client:          NGGetbot - Solaris
Server Batching: Yes
Client Batching: Yes
Serialization:   Yes
Protocol:        W3NG
Transport:       W3MUX
Interface:       TCWA synchronous
Documents:       Microscape
Bandwidth:       100


Server (ngsun2) command line:
/opt/local/ilu/examples/ngtest/ngwebserver -httptinfo tcp_0_8080 -filebase /opt/local/http-ng/serverdocs -batch 50000 -verbose 1

Client (ngsun1) command line:
/opt/local/ilu/examples/ngtest/nggetbot -urls /opt/local/http-ng/ngtesturls/mscape_ngsun2_w3ng_w3ng_w3mux_2718.urls -batch 50000 -logfile stdout -v 1 -mt 10 -pipeline -serial



--------------------------------------------------
Title:           Sunrpcrm
Server:          NGWebserver - Solaris
Client:          NGGetbot - Solaris
Server Batching: Yes
Client Batching: Yes
Serialization:   Yes
Protocol:        W3NG
Transport:       Sunrpcrm
Interface:       TCWA synchronous
Documents:       Microscape
Bandwidth:       100

Server (ngsun2) command line:
/opt/local/ilu/examples/ngtest/ngwebserver -httptinfo tcp_0_8080 -filebase /opt/local/http-ng/serverdocs -batch 50000 -verbose 1 -ngtinfo sunrpcrm

Client (ngsun1) command line:
/opt/local/ilu/examples/ngtest/nggetbot -urls /opt/local/http-ng/ngtesturls/mscape_ngsun2_w3ng_w3ng_sunrpcrm_2718.urls -batch 50000 -logfile stdout -v 1 -mt 10 -pipeline -serial



-------------------------------------------------
Title:           Batchless NG
Server:          NGWebserver - Solaris
Client:          NGGetbot - Solaris
Server Batching: No
Client Batching: No
Serialization:   No
Protocol:        W3NG
Transport:       W3MUX
Interface:       TCWA synchronous
Documents:       Microscape
Bandwidth:       100

Server (ngsun2) command line:
/opt/local/ilu/examples/ngtest/ngwebserver -httptinfo tcp_0_8080 -filebase /opt/local/http-ng/serverdocs -verbose 1

Client (ngsun1) command line:
/opt/local/ilu/examples/ngtest/nggetbot -urls /opt/local/http-ng/ngtesturls/mscape_ngsun2_w3ng_w3ng_w3mux_2718.urls -logfile stdout -v 1 -mt 10


-------------------------------------------------
Title:           Native Apache
Server:          Apache 1.2 - Solaris
Client:          NGGetbot - Solaris
Server Batching: (built in?)
Client Batching: Yes
Serialization:   Yes
Protocol:        HTTP 1.1
Transport:       TCP
Interface:       Native
Documents:       Microscape
Bandwidth:       100

Server (ngsun2) command line:
/opt/local/http-ng/apache/src/httpd -f /opt/local/http-ng/apache/conf/httpd.conf  

Client (ngsun1) command line:
/opt/local/ilu/examples/ngtest/nggetbot -urls /opt/local/http-ng/ngtesturls/mscape_ngsun2_http_http_8080.urls -logfile stdout -v 1 -mt 10 -pipeline -serial -batch 50000

-------------------------------------------------
Title:           libwww Webbot
Server:          Apache 1.2 - Solaris
Client:          libwww Webbot 5.1m  - Solaris
Server Batching: (built in?)
Client Batching: (built in)
Serialization:   Yes
Protocol:        HTTP 1.1
Transport:       TCP
Interface:       Native
Documents:       Microscape
Bandwidth:       100

Server (ngsun2) command line:
/opt/local/http-ng/apache/src/httpd -f /opt/local/http-ng/apache/conf/httpd.conf  

Client (ngsun1) command line:
/opt/local/http-ng/w3c-libwww-5.1m/bin/webbot -saveimg -n http://ngsun2.parc.xerox.com:8080/microscape/microscape.html

-------------------------------------------------
Title:           Slowed libwww Webbot
Server:          Apache 1.2 - Solaris
Client:          libwww Webbot 5.1m  - Solaris
Server Batching: (built in?)
Client Batching: (built in)
Serialization:   Yes
Protocol:        HTTP 1.1
Transport:       TCP
Interface:       Native
Documents:       Microscape
Bandwidth:       10

Server (ngsun2) command line:
/opt/local/http-ng/apache/src/httpd -f /opt/local/http-ng/apache/conf/httpd.conf  

Client (ngsun1) command line:
/opt/local/http-ng/w3c-libwww-5.1m/bin/webbot -saveimg -n http://ngsun2.parc.xerox.com:8080/microscape/microscape.html


-------------------------------------------------
Title:           ILU Apache
Server:          ILU/Apache 1.2 - Solaris
Client:          NGGetbot - Solaris
Server Batching: (built in ?)
Client Batching: Yes
Serialization:   Yes
Protocol:        HTTP
Transport:       TCP
Interface:       iluhttp.Resource
Documents:       Microscape
Bandwidth:       100

Note:            Get TCP-dump

Server (ngsun2) command line:
/opt/local/http-ng/apache/src/httpngd -f /opt/local/http-ng/apache/conf/httpd.conf  

Client (ngsun1) command line:
/opt/local/ilu/examples/ngtest/nggetbot -urls /opt/local/http-ng/ngtesturls/mscape_ngsun2_http_http_2719.urls -logfile stdout -v 1 -mt 10 -pipeline -serial -batch 50000

--------------------------------------------------
Title:           IIOP
Server:          NGWebserver - Solaris
Client:          NGGetbot - Solaris
Server Batching: Yes
Client Batching: Yes
Serialization:   No
Protocol:        IIOP
Transport:       W3MUX
Interface:       TCWA synchronous
Documents:       Microscape
Bandwidth:       100

Server (ngsun2) command line:
/opt/local/ilu/examples/ngtest/ngwebserver -httptinfo tcp_0_8080 -ngpinfo iiop -ngtinfo -filebase /opt/local/http-ng/serverdocs -batch 50000 -fdbudget 128 -verbose 1

Client (ngsun1) command line:
/opt/local/ilu/examples/ngtest/nggetbot -urls /opt/local/http-ng/ngtesturls/mscape_ngsun2_w3ng_iiop_2718.urls -batch 50000 -logfile stdout -v 1 -mt 10


--------------------------------------------------
Title:           Slowed NG
Server:          NGWebserver - Solaris
Client:          NGGetbot - Solaris
Server Batching: Yes
Client Batching: Yes
Serialization:   No
Protocol:        W3NG
Transport:       W3MUX
Interface:       TCWA synchronous
Documents:       Microscape
Bandwidth:       10

Slow ports on switch to 10 mbps - see Omniswitch documentation in
/project/http-ng/docs/ng-testlab-notes.txt

Server (ngsun2) command line:
/opt/local/ilu/examples/ngtest/ngwebserver -httptinfo tcp_0_8080 -filebase /opt/local/http-ng/serverdocs -batch 50000 -verbose 1

Client (ngsun1) command line:
/opt/local/ilu/examples/ngtest/nggetbot -urls /opt/local/http-ng/ngtesturls/mscape_ngsun2_w3ng_w3ng_w3mux_2718.urls -batch 50000 -logfile stdout -v 1 -mt 10



--------------------------------------------------
Title:           Async NG
Server:          NGWebserver - Solaris
Client:          NGGetbot - Solaris
Server Batching: Yes
Client Batching: Yes
Serialization:   No
Protocol:        W3NG
Transport:       W3MUX
Interface:       TCWA asynchronous
Documents:       Microscape
Bandwidth:       100

Server (ngsun2) command line:
/opt/local/ilu/examples/ngtest/ngwebserver -httptinfo tcp_0_8080 -filebase /opt/local/http-ng/serverdocs -batch 50000 -verbose 1

Client (ngsun1) command line:
/opt/local/ilu/examples/ngtest/nggetbot -urls /opt/local/http-ng/ngtesturls/mscape_ngsun2_w3ng_w3ng_w3mux_2718.urls -batch 50000 -logfile stdout -v 1 -mt 10 -sink


--------------------------------------------------
Title:           Solaris NT
Server:          NGWebserver - Solaris
Client:          NGGetbot - NT
Server Batching: Yes
Client Batching: Yes
Serialization:   No
Protocol:        W3NG
Transport:       W3MUX
Interface:       TCWA synchronous
Documents:       Microscape
Bandwidth:       100

Server (ngsun2) command line:
/opt/local/ilu/examples/ngtest/ngwebserver -httptinfo tcp_0_8080 -filebase /opt/local/http-ng/serverdocs -batch 50000 -verbose 1

Client (ngpc1) command line:
\iluwin\examples\ngtest\winrel\nggetbot -urls \http-ng\ngtesturls\mscape_ngsun2_w3ng_w3ng_w3mux_2718.urls -batch 50000 -logfile stdout -v 1 -mt 10


--------------------------------------------------
Title:           NT Solaris
Server:          NGWebserver - NT
Client:          NGGetbot - Solaris
Server Batching: Yes
Client Batching: Yes
Serialization:   No
Protocol:        W3NG
Transport:       W3MUX
Interface:       TCWA synchronous
Documents:       Microscape
Bandwidth:       100

Server (ngpc2) command line:
\iluwin\examples\ngtest\winrel\ngwebserver -filebase \http-ng\serverdocs -batch 50000 -verbose 1

Client (ngsun1) command line:
/opt/local/ilu/examples/ngtest/nggetbot -urls /opt/local/http-ng/ngtesturls/mscape_ngpc2_w3ng_w3ng_w3mux_2718.urls -batch 50000 -logfile stdout -v 1 -mt 10


--------------------------------------------------
Title:           NT NT
Server:          NGWebserver - NT
Client:          NGGetbot - NT
Server Batching: Yes
Client Batching: Yes
Serialization:   No
Protocol:        W3NG
Transport:       W3MUX
Interface:       TCWA synchronous
Documents:       Microscape
Bandwidth:       100

Server (ngpc2) command line:
\iluwin\examples\ngtest\winrel\ngwebserver -filebase \http-ng\serverdocs -batch 50000 -verbose 1

Client (ngpc1) command line:
\iluwin\examples\ngtest\winrel\nggetbot -urls \http-ng\ngtesturls\mscape_ngpc2_w3ng_w3ng_w3mux_2718.urls -batch 50000 -logfile stdout -v 1 -mt 10


--------------------------------------------------
Title:           AOL NG
Server:          NGWebserver - Solaris
Client:          NGGetbot - Solaris
Server Batching: Yes
Client Batching: Yes
Serialization:   No
Protocol:        W3NG
Transport:       W3MUX
Interface:       TCWA synchronous
Documents:       AOL
Bandwidth:       100

Server (ngsun2) command line:
/opt/local/ilu/examples/ngtest/ngwebserver -httptinfo tcp_0_8080 -filebase /opt/local/http-ng/serverdocs/aol -batch 50000 -verbose 1

Client (ngsun1) command line:
/opt/local/ilu/examples/ngtest/nggetbot -urls /opt/local/http-ng/ngtesturls/aol_ngsun2_w3ng_w3ng_w3mux_2718.urls -batch 50000 -logfile stdout -v 1 -mt 10

--------------------------------------------------
Title:           MS NG
Server:          NGWebserver - Solaris
Client:          NGGetbot - Solaris
Server Batching: Yes
Client Batching: Yes
Serialization:   No
Protocol:        W3NG
Transport:       W3MUX
Interface:       TCWA synchronous
Documents:       MS
Bandwidth:       100

Server (ngsun2) command line:
/opt/local/ilu/examples/ngtest/ngwebserver -httptinfo tcp_0_8080 -filebase /opt/local/http-ng/serverdocs/ms -batch 50000 -verbose 1

Client (ngsun1) command line:
/opt/local/ilu/examples/ngtest/nggetbot -urls /opt/local/http-ng/ngtesturls/ms_ngsun2_w3ng_w3ng_w3mux_2718.urls -batch 50000 -logfile stdout -v 1 -mt 10





//////////////////////////////////////////////////
Old notes:

Possible test factors:

1)  Generic variable factors:

  a)  OS  (NT vs. Solaris)
  b)  batching/pipelining/serialization parameters
  c)  bandwidth of connection
  d)  w3mux vs. sunrpcrm
  e)  w3ng vs. HTTP vs. IIOP
  f)  TCWA vs. iluhttp.Resource
  g)  message-level compression vs. app-level compression vs. none

2)  Fetch documents tests:

  a)  Microscape profile vs. AOL profile vs. BU profile vs. Microsoft profile
  b)  as fast as possible vs. real traffic
  c)  synch vs. asynch retrieval APIs
  d)  varying chunk size with asynch API
  e)  graph file size of object retrieved against other characteristics

3)  Forms processing tests:

  a)  forms processing

4)  Proxy & cache maintenance tests:

  a)  small entity-body job (caching)

5)  Demonstrate upgrade path

