Download HTML
lazarus / 38-Networking and Web Programming

----------------------------------------------------------------------
pcurtis · 2020-10-08 09:11

Dear All,

I can download a web pages HTML WITH THIS

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i function
TForm1.SynapseDownload(URL: string): string;var  HTTPGetResult:
Boolean;  HTTPSender: THTTPSend;  S: string;begin  Result := ''; 
HTTPSender := THTTPSend.Create;   HTTPSender.Timeout:=10000;  
HTTPGetResult := HTTPSender.HTTPMethod('GET', URL);   SetLength(S,
HTTPSender.Document.Size);  HTTPSender.Document.Read(S[1],
Length(S));  Result:=S;   HTTPSender.Free;end; 
I have two questions

1. How can I tell the web server what browser I am (user agent)? For
example "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:81.0)
Gecko/20100101 Firefox/81.0"

2. Can I fool the web server into thinking I followed a link on one of
it's pages?

Thanks in advance.

----------------------------------------------------------------------
Handoko · 2020-10-08 09:22

Quote from: pcurtis on October 08, 2020, 11:11:29 am1. How can I tell
the web server what broswer I am?

Set the user agent property:
http://synapse.ararat.cz/doc/help/httpsend.THTTPSend.html#UserAgent

These are the common user agent strings:
https://www.networkinghowtos.com/howto/common-user-agent-list/

More about user agent:
https://en.wikipedia.org/wiki/User_agent

For example:
HTTPGetResult.UserAgent := 'Mozilla/5.0 (compatible; fpweb)';

----------------------------------------------------------------------
pcurtis · 2020-10-08 09:29

Cheers. Again

----------------------------------------------------------------------
Handoko · 2020-10-08 09:42

Quote from: pcurtis on October 08, 2020, 11:11:29 am2. Can I fool the
web server into thinking I followed a link on one of it's pages?

Not very sure but I believe it has something to do with HTTP referer:
https://en.wikipedia.org/wiki/HTTP_referer

HTTP referer is a field in the Headers property:
http://synapse.ararat.cz/doc/help/httpsend.THTTPSend.html#Headers

----------------------------------------------------------------------
pcurtis · 2020-10-08 15:55

Thanks again.