Subj : ANNOUNCE:  mywebget-2001.0105.pl Perl wget(1) batch download
To   : All
From : Jari Aalto+mail.perl
Date : Wed Feb 07 2001 03:00 pm

From: jari.aalto@poboxes.com (Jari Aalto+mail.perl)


    http://www.cpan.org/modules/by-authors/id/J/JA/JARIAALTO/

Manual page and example configuration file on-line at:

    http://tiny-tools.sourceforge.net/ --> mywebget.html

IMPORTANT: This is file is your companion to track frequently released files, 
because it can track "the newest" versions with heuristics. Something that Wget 
cannot.


README
    Automate periodic downloads of released files and packages.

  Wget and this program

    At this point you may wonder, where would you need this perl program
    when wget(1) C-program has been the standard for ages. Well, 1) Perl is
    cross platform and more easily extendable 2) You can put file download
    criterias to configuration file and use perl regular epxressions 3) the
    program can anlyze web-pages and "search" for the download link as you
    instruct 4) it contains heuristics to track more newer version of the
    file.

    But it does not replace webget(1) because this program does not offer as
    many options as web get, not even recursive downloads. The best advice
    is to dedicate this program to "batch" download the files that you
    monitor most of the time and use wget(1) for everything else.

  Short introduction

    This small utility makes it possible to keep a list of URLs in a
    configuration file and periodically retrieve those pages or files with
    simple commands. This utility is best suited for small batch jobs to
    download e.g. most recent versions of software files. If you use an URL
    that is already on disk, be sure to supply option --overwrite to allow
    overwriting existing files.

    While you can run this program from command line to retrieve individual
    files, program has been designed to use separate configuration file via
    --config option. In the configuration file you can control the
    downloading with separate directives like `save:' which tells to save
    the file under different name.

    The simplest way to retreive the latest version of a kit from FTP site
    is:

        mywebget.pl --new --overwite --verbose \
           http://www.example.com/kit-1.00.tar.gz

    Don't worry about the filename "kit-1.00.tar.gz". The latest version,
    say, kit-3.08.tar.gz will be retrieved. The option --new instructs to
    find newer version than the provided URL.

    If the URL ends to slash, then directory list at the remote machine is
    stored to file:

        !path!000root-file

    The content of this file can be either index.html or the directory
    listing depending on the used http or ftp protocol.

EXAMPLES
    Get file(s) from site:

        mywebget.pl http://www.example.com/dir/package.tar.gz ..

    Read a directory and store it to filename YYYY-MM-DD::!dir!000root-file.

        mywebget.pl --prefix-date --overwrite --verbose
http://www.example.com/dir/

    To update newest version of the kit, but only if there is none in the
    disk already. The --new option instructs to find nwer packages and the
    filename is used only for guidance how the file looks like:

        mywebget.pl --overwrite --skip-version --new --verbose \
            ftp://ftp.example.com/dir/packet-1.23.tar.gz

    To overwrite file and add a date prefix to the file name:

        mywebget.pl --prefix-date --overwrite --verbose \
           http://www.example.com/file.pl

        --> YYYY-MM-DD::file.pl

    To add date and WWW site prefix to the filenames:

        mywebget.pl --prefix-date --prefix-www --overwrite --verbose \
           http://www.example.com/file.pl

        --> YYYY-MM-DD::www.example.com::file.pl

    Get all updated files under KITS and use default configuration file:

        mywebget.pl --verbose --overwrite --skip-version --new --Tag kits
        mywebget.pl -v -o -s -n -T kits

    Get files as they read in the configuration file to the current
    directory, ignoring any `lcd:' and `save:' directives:

        mywebget.pl --config $HOME/config/mywebget.conf /
            --no-lcd --no-save --overwrite --verbose \
            http://www.example.com/file.pl

--- BBBS/LiI v4.01 Flag-2
 * Origin: Prism bbs (1:2320/38)

.