Subj : Re: Scripting-language for interacting with World Wide Web?
To   : comp.os.linux,comp.os.linux.misc,comp.os.linux.questions
From : Thomas D. Shepard
Date : Sat Jun 26 2004 12:40 pm

This sounds a lot like "push" technology. It really isn't too hard to
write something in any reasonably-chosen programming language to do
something like that. Perl is good, so is Python ...

On Tue, 22 Jun 2004 21:36:07 +1000, Fritz wrote:

> While not really what you want have you looked at httrack (www.httrack.com)
> 
> It's a web gathering tool that supports most of what you want (cookies, 
> sids, logins)
> 
> You can create a script to script it.
> 
> William Park wrote:
> 
>> Baard Ove Kopperud <baard.kopperud@bigfoot.com> wrote:
>> 
>>>I'm in need of a scripting-language/library/extension that enables 
>>>me to make scripts for non-interactive interaction with World Wide Web
>>>(web-pages).  What I need is something like a cross between 'wget'
>>>and 'expect'.
>>>
>>>I know there was something for 'perl' called 'elza', but as far as
>>>I can understand, work on 'elza' stopped ages ago.  The primary
>>>download-site is also closed (gone commercial)... all I found was
>>>an old(?) version at Debian.
>>>
>>>Does anyone know if 'elza' is still being worked on?  Are there
>>>any similar tools out there (for 'perl' or other languages)?
>>>Are there any specilized lanuages (e.g. based on LISP, Forth,...)?
>>>An ability to handle other protocols (FTP, NNTP, POP, IMAP, SMTP)
>>>would be a pluss.
>>>
>>>To help you suggest something suitable, here is a range of intended usage:
>>>*Automatic log-in to accounts for web-based services to prevent
>>> deletion.  Would require correct handling of cookies and secure
>>> connections.  I intend to make a list of URL and username/password
>>> pairs for the script to parse.  The script could be run by cron.
>>> (That I don't *use* what I have signed-up for once long, long ago,
>>> don't mean I want to loose my account :-).
>>>*Downloading multi-chaptered stories from various sites (e.g.
>>> fanfiction.net) automatically after specifying a story-id.  Some
>>> of the site requires log-in.
>>>*Automatically downloading a range of messages from various Yahoo-groups
>>> (and similar services).  Requires log-in.
>>>*Automatically download web-based mail (e.g. Yahoo).
>> 
>> 
>> As soon as you write a script, they'll change the interface which breaks
>> your script.  The very reason why they make it so labour-intensive, is
>> to prevent what you're trying to do with automated script. :-)
>>

-- 
Thomas D. Shepard
Sorry, you can't email me.
(Email address is fake.)

.