[SOLVED] Error 302 when attempting to fetch Wikipedia page
lazarus / 38-Networking and Web Programming

----------------------------------------------------------------------
maurobio · 2023-06-26 12:17

Dear ALL,

I have a simple test program which fetches a Wikipedia article for a
given scientific name and presents the article summary:

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i program
wiki_test; {$APPTYPE CONSOLE}{$mode objfpc}{$H+} uses  Classes,
TypInfo,SysUtils, fphttpclient, fpjson, jsonparser,
opensslsockets; const    url =
'https://en.wikipedia.org/api/rest_v1/page/summary/';    searchName =
'Vicia faba'; // <--- This works        searchName = 'Loxodonta
africana'; // <--- This does NOT work var  jData: TJSONData;  jItem,
jItems: TJSONData;  i: integer; begin  WriteLn('Searching Wikipedia
for: ' + searchName);  // Get the page summary  jData :=
GetJSON(TFPHTTPClient.SimpleGet(url + StringReplace(searchName, ' ',
'_', [rfReplaceAll])));  WriteLn(jData.FindPath('extract').AsString); 
WriteLn;  jData.Free;end.  
For some species (eg., Vicia faba) it works fine, but for others (eg.,
Loxodonta africana), the program throws an exception: "EHTTPClient:
Unexpected response status code: 302".

According to here (https://blog.airbrake.io/blog/http-
errors/302-found), this error arises when "the requested resource has
been temporarily moved to a different URI", which suggests that
HTTPClient is not handling the (somewhat strange?) redirection of
pages performed by Wikipedia.

Could someone  help me to sort out - and hopefully solve - this
problem?

Thanks in advance!

With warmest regards,

----------------------------------------------------------------------
TRon · 2023-06-26 12:34

Quote from: maurobio on June 26, 2023, 02:17:37 pmAccording to here
(https://blog.airbrake.io/blog/http-errors/302-found), this error
arises when "the requested resource has been temporarily moved to a
different URI", which suggests that HTTPClient is not handling the
(somewhat strange?) redirection of pages performed by Wikipedia.

Could someone  help me to sort out - and hopefully solve - this
problem?

That can not be realized with simpleget. You'd need to create the
httpclient object and set the allowredirect option, see also https://w
iki.freepascal.org/fphttpclient#Download_a_file_via_HTTP_protocol (4th
example)

----------------------------------------------------------------------
maurobio · 2023-06-26 13:06

Hi, @TRon!

Great! Based on the example you have pointed out, I amended my code
and it now works fine!

Here it is:

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i program
wiki_test; {$APPTYPE CONSOLE}{$mode objfpc}{$H+} uses  Classes,
TypInfo,SysUtils, fphttpclient, fpjson, jsonparser, openssl,
opensslsockets; const    url =
'https://en.wikipedia.org/api/rest_v1/page/summary/';    searchName =
'Vicia faba'; // <--- This works        searchName = 'Loxodonta
africana'; // <--- This ALSO works var  Client: TFPHttpClient;  jData:
TJSONData;  begin  InitSSLInterface;       WriteLn('Searching
Wikipedia for: ' + searchName);  Client := TFPHttpClient.Create(nil); 
try    { Allow redirections }    Client.AllowRedirect := True;   
jData := GetJSON(Client.Get(url + StringReplace(searchName, ' ', '_',
[rfReplaceAll])));  finally    Client.Free;  end;  { Get the page
summary }  WriteLn(jData.FindPath('extract').AsString);  WriteLn; 
jData.Free;end.   
Thank you very much!

With warmest regards,