(Solved)How to make a simple Web Scraper??? lazarus / 38-Networking and Web Programming ---------------------------------------------------------------------- lestroso · 2021-03-24 20:38 Hi to everybody... How to make a simple Web Scraper??? I need to find in certain web pages some data in a deep manner accurate ... I would like to search in a complete website a name or other data and alert if found visually or with  sound...no problem... Do you have an example to show me?? to learn it and how to use it?? I thank you so much! Lestroso ---------------------------------------------------------------------- Trenatos · 2021-03-24 22:18 https://medium.com/@marcusfernstrm/create-a-web-scraper-with- freepascal-lazarus-ffe644f8c34 ---------------------------------------------------------------------- Jurassic Pork · 2021-03-24 23:22 hello, you can try to use selenium with Webdriver4L (see here) Friendly, J.P ---------------------------------------------------------------------- Warfley · 2021-03-25 16:55 Depends on how the content of the website is generated. If it is static content, i.e. content that is transmitted with the HTML file on load of the website (usually done by blogs, news sites, etc.) it is really simple: 1. Load html content via HTTP (httpclient from the fcl should do) 2. parse HTML DOM 3. Search HTML DOM using xqueries and/or regex or other methods If the content of the website is generated dynamically, i.e. the javascript of the website downloads the content on the fly (as commonly used by social media sites) you need to replicate the behavior of the javascript. This an be as easy as calling a REST API with a simple request, but can also be arbitrarily complex. In this case it is the easiest to just load the website within a browser/javascript engine like Webkit, gecko or similars and extract the dom from them. Personally for the dynamic case I would use headless chrome to do this and communicate with it over the pipes provided by chrome (I think stdin/stdout works) and maybe inject javascript to extract the information and read that information then into my lazarus program. Another easy alternative is to write a small javascript extension that loads the page and transmits all the required information to your local program. This can then be run in an invisible browser window ---------------------------------------------------------------------- lestroso · 2021-03-25 17:08 THANKs to everybody... Could you give me an example .lpi  already made? please? I'm a  Lazarus beginner...I don't know how to start with...Best regards, lestroso. ---------------------------------------------------------------------- Trenatos · 2021-03-25 17:40 I did.. Look up. I posted a link to an article I wrote where I show how to write a web scraper. ---------------------------------------------------------------------- alaa123456789 · 2021-03-25 19:00 Quote from: lestroso on March 24, 2021, 09:38:41 pmHi to everybody... How to make a simple Web Scraper??? I need to find in certain web pages some data in a deep manner accurate ... I would like to search in a complete website a name or other data and alert if found visually or with  sound...no problem... Do you have an example to show me?? to learn it and how to use it?? I thank you so much! Lestroso https://www.youtube.com/watch?v=Nw6N8_7ZI_o ---------------------------------------------------------------------- lestroso · 2021-03-26 10:08 Thousands thanks to everybody... Now I can study from your links...Thanks again.....Lestroso