 
  Copyright 1997-1998
  acaldero@laurel.datsi.fi.upm.es  
  (Alejandro Calderon Mateos)
   

  How to use -------------------------------------------------------------

   *  SpiderBot is a normal robot for download a web site 
      without problems.
      First use is :
    
          ./SpiderBot.exe  -url www.cool.com
    
      That make a mirror in the current directory, of web pages at 
      'www.cool.com' :

             www.cool.com/.
             www.cool.com/..
             www.cool.com/index.html
             ...

   *  If you press Ctrl-C or reboot you computer or ... *see morphy laws* 
      NO PROBLEM, you can continue downloading without loss what 
      you got. Use for recover :

          ./SpiderBot.exe -resume resume.www.cool.com

      in 'resume.<site>', SpiderBot save all necesary to recover
      gracelly. You can edit this file, but be very, very carefully.

   *  It is easy limit amount of web to download :
      you can limit amount of byte you want to download with :

          ./SpiderBot.exe -url www.mysite.web -maxbytes 1000000

      Limit is set at one Megabyte (1000000). 
      Also you can limit the number of urls :

          ./SpiderBot.exe -url www.mysite.web  -maxurls 1000

   *  Need speed ?, SpiderBot support proxy :

          ./SpiderBot.exe -url www.mysite.web -proxy myproxy.web:8080

      In this version (060), when proxy report 'error 400 Cache error...'
      SpiderBot DO NOT try connect directly with "www.mysite.web", 
      simply skip url.
      (or try again download url, if you use '-rentry')

   *  By default, HTTP/1.0 is used. With :

          ./SpiderBot.exe -url www.mysite.web -detecthttp

      SpiderBot try autodetect server version, and work with it

   *  why are you still reading ??!!

          ./SpiderBot.exe -help

      See other options, learn by yourself using it...


  ------------------------------------------------------------------------


  
