simple text web browser help
lazarus / 38-Networking and Web Programming

----------------------------------------------------------------------
wpflum · 2011-08-14 13:48

I'm working on a isbn scraper to pull book data off of a web site. 
What I need is a simple way to call a site something like
'www.books.com/isbn/isbnnumber' and pull the response into a
variable/list/whatever that I can then parse for the info I need.  No
other interaction is required.  So its a call, read and close
scenario.  It looks like lnet might work but it seems like a lot more
that I need, is there anything else I should look at or can some one
show me a small example program that does this??   I was going to use
perl and Lynx but since I want to roll this into a complete ebook
library program I figured I'd be better off doing the entire thing in
Lazarus. 

Any ideas/comments/rants???


P.S.  I know I could use Calibre but I have issues with how it handles
the Libraries it creates, specifically needing to move all the books
to it's own directory instead of leaving them where it finds them.

----------------------------------------------------------------------
mica · 2011-08-14 15:51

you can use this
http://www.benibela.de/sources_en.html#internettools
or this
http://wiki.lazarus.freepascal.org/fcl-xml with synapse

----------------------------------------------------------------------
wpflum · 2011-08-14 16:47

The first one

http://www.benibela.de/sources_en.html#internettools

looks interesting and if I can figure it out it might save me some
time.

What I'm really looking for right now is some straight forward code to
call the web site and receive the page.  I'm going to play with lnet
right now but would appreciate any direction.

----------------------------------------------------------------------
Leledumbo · 2011-08-14 21:38

lNet and Synapse are both capable of doing this. Just use the correct
protocol class in lNet, and Synapse' HTTPGetText (+SSL for https)
would do the job.

----------------------------------------------------------------------
felipemdc · 2011-08-16 09:19

Check my answer here:

http://www.lazarus.freepascal.org/index.php/topic,14031.msg73998.html#
msg73998

----------------------------------------------------------------------
mica · 2011-08-16 10:13

http://www.benibela.de/documentation/internettools/
or
http://www.ararat.cz/synapse/doku.php/public:howto

http://www.ararat.cz/synapse/doku.php/public:howto:httpgetpage

----------------------------------------------------------------------
wpflum · 2011-08-16 12:22

Quote from: mica on August 16, 2011, 12:13:11
pmhttp://www.benibela.de/documentation/internettools/
or
http://www.ararat.cz/synapse/doku.php/public:howto

http://www.ararat.cz/synapse/doku.php/public:howto:httpgetpage

Thank you!!  The last one is exactly what I was looking for.  Right
now I'm making a system call to run Lynx and grab its output but now I
should be able to use this code and keep more of the program self
contained.