Subj : Re: Scripting-language for interacting with World Wide Web? To : comp.os.linux,comp.os.linux.misc,comp.os.linux.questions From : Fritz Date : Tue Jun 22 2004 10:36 pm While not really what you want have you looked at httrack (www.httrack.com) It's a web gathering tool that supports most of what you want (cookies, sids, logins) You can create a script to script it. William Park wrote: > Baard Ove Kopperud wrote: > >>I'm in need of a scripting-language/library/extension that enables >>me to make scripts for non-interactive interaction with World Wide Web >>(web-pages). What I need is something like a cross between 'wget' >>and 'expect'. >> >>I know there was something for 'perl' called 'elza', but as far as >>I can understand, work on 'elza' stopped ages ago. The primary >>download-site is also closed (gone commercial)... all I found was >>an old(?) version at Debian. >> >>Does anyone know if 'elza' is still being worked on? Are there >>any similar tools out there (for 'perl' or other languages)? >>Are there any specilized lanuages (e.g. based on LISP, Forth,...)? >>An ability to handle other protocols (FTP, NNTP, POP, IMAP, SMTP) >>would be a pluss. >> >>To help you suggest something suitable, here is a range of intended usage: >>*Automatic log-in to accounts for web-based services to prevent >> deletion. Would require correct handling of cookies and secure >> connections. I intend to make a list of URL and username/password >> pairs for the script to parse. The script could be run by cron. >> (That I don't *use* what I have signed-up for once long, long ago, >> don't mean I want to loose my account :-). >>*Downloading multi-chaptered stories from various sites (e.g. >> fanfiction.net) automatically after specifying a story-id. Some >> of the site requires log-in. >>*Automatically downloading a range of messages from various Yahoo-groups >> (and similar services). Requires log-in. >>*Automatically download web-based mail (e.g. Yahoo). > > > As soon as you write a script, they'll change the interface which breaks > your script. The very reason why they make it so labour-intensive, is > to prevent what you're trying to do with automated script. :-) > .