#-------------------------------------------------------------------
# This file is part of the wwwoffle-sql package
#
#  Version : 0.02  Released : 03/05/99
#
# Author : Peter Marbaise (pema@hippo.fido.de,2:2452/110.20)
#
#              GNU GENERAL PUBLIC LICENSE Version 2
#              (see LICENSE for the complete text)
#
#-------------------------------------------------------------------

# ---------------------------------------

The wwwoffle-sql scripts (c) by Peter Marbaise (pema@hippo.fido.de)
-------------------------------------------------------------------
This set scripts is designed for wwwoffle , the web offline reader
(http://www.gedanken.demon.co.uk/wwwoffle) and the mySQL database
(http://www.tcx.se)

I'm working with a SuSe 5.3 distribution and a 2.0.35 kernel and apache
1.3.0, but i guess there should be no problem to run the scripts on other
distributions or adapt the scripts to postgress SQL.

Working with wwwoffle, i always missed the ability to search for a
certain keyword, so that is, what the scripts are doing. Ok, with the
new wwwoffle-2.4 there comes ht://Dig with it, but to me, thats a bit
oversized for a local system, if you dont have the requirement to
search for all words in an html-document.

Take a look at http://www.fido.de/~pema/wwwoffle-sql.html for more
informations.


INSTALL
-------

SETUP-mysql generates the mysql database wwwoffle and the tabels.Call
the script as a user who is allowed to create a database. Maybe you
have to edit the password in the script. If you are using the scripts
from version 0.01 there is a UPDATE-mysql, which only adds an
additional field to the database. I take the keywords entered by the
user in that field in order not to detroy them, when the wwwoffle
directory is read again.

SETUP-apache copies the files into
/usr/local/apache/share/htdocs/wwwoffle-sql

if you have your docs in another site edit the script.

Edit your access.conf by adding the following lines

<Directory /usr/local/apache/share/htdocs/wwwoffle-sql>
AllowOverride None
Options +ExecCGI 
order allow,deny	    --> change it to allow access only from
				localhost (see apache docs)
allow from all
</Directory>


and restart the server.

Ensure that the wwwoffle-spool dir is readable by owner of http-server.

To enable keyword search in pages you are browsing, enable
AddInfoRefresh in wwwoffle.conf

If you're working with wwwoffle 2.4 the name of the AddInfoRefresh Page
has changed to AddCacheInfo.html, so you have to alter the script.

Getting started
---------------
If everything is setup, you can start it all with wwwofflesql.html main
page. From there you can start <reading> wwwoffle spool dirs, parsing the
pages and writing the entries to the database.

The script for reading the wwwoffle spool dirs is readwwwoffle.pl,
which also can be startet from a shell or called in a cron job.

readwwwoffle.pl fills up the following fields in the database
- the host (i.e. http://www.host.com)
- the page (i.e. /menue.html )
- the date
- the webserver (i.e. Apache/1.3.0)
- the size in bytes
- the version
- the keywords

The keywords are taken from the <meta name="keywords" contents="..">
directive or the <title> of a page, since not all pages contain a
keyword directive.

If you downloaded a file via ftp-protocol than the filename is taken as
a keyword and as a page reference. If the version follows the (as far
as i know) GNU standard i.e. filename-2.2.12.tar.gz, then 2.2.12 is
taken as aversion of the file.

readwwoffle.pl only looks for html documents, and archives (ending with
.exe, tar.gz,gz,tgz pp). Gif's and other files are not written to the
database

If you search for a keyword a page is generated with links to all
hosts/pages containing the keyword. Also links are generated for
editing/deleting the page or the host.

An additonal script searchkey does the same as searchkey.pl accept
output is done to STDOUT (without links), which can be piped to other
programms.

So far the scripts work for me, so maybe it will work for you too.

If you find a bug (I'm sure you find one) drop a mail to
pema@hippo.fido.de with wwwoffle-sql as subject.

Release : 0.02 03/05/99
