HOW TO USE :
------------

The downloaded files are stored in a directory named after the host name
and created in the current directory.

To use the robot you only have to modify the file "config.py" ; here are the
informations to give :

--- BASE CONFIG (mandatory) ---

 * the initial page :
     example -> INIT_URL = "http://www.dummy.org/area1/home.html"
	 In this example the downloaded files will be saved in the
	 directory "www.dummy.org".

--- RESTRICTIONS (optionnal) ---

 * restriction to a directory (only for "href" and "longdesc" links) :
     no restriction -> RES_DIR = ""
     example        -> RES_DIR = "area1" (will donwload only links situated
                       in the "area1" directory or in its subdirectories)
 
 * the file size limit (will not download larger files) :
    no limit -> MAX_FILE_SIZE = 0
    example  -> MAX_FILE_SIZE = 400000 (ie about 400 kB)

--- MISC (optionnal) ---

 * the log file :
    no log  -> LOG_FILE = ""
    example -> LOG_FILE = "log" 

 * the verbosity :
    normal verbosity  -> VERBOSITY = 0
    minimal verbosity -> VERBOSITY = -1
    high verbosity    -> VERBOSITY = 1

--- OFF-LINE WORK (optionnal) ---
NB: note that CHECK_MODIFIED and DOWNLOAD_MISSING need to be simultaneously
    disabled to allow completely off-line work.

 * check/do not check server for modifications of already downloaded files: 
   can be useful when a download has been interrupted and you do not want to
   recheck all files.
    default      -> CHECK_MODIFIED = 1
    do not check -> CHECK_MODIFIED = 0

 * download/do not download missing links from server: this option is not very
   useful at the moment. 
	default         -> DOWNLOAD_MISSING = 1
 	do not download -> DOWNLOAD_MISSING = 0

--- LOCALIZATION (optionnal) ---
NB  : this task can be done completely offline.
NB2 : this option WILL modify the HTML code of the downloaded pages.
NB3 : it is better to only download the files and then to localize them
      offline as this later operation can be long.

  * get rid of the BASE field and modify URLs to allow local browsing
	unhindered by absolute URLs.
	 default  -> LOCALIZE_URLS = 0
	 localize -> LOCALIZE_URLS = 1
