Error 403 downloading with Synapse and SSL lazarus / 21-General ---------------------------------------------------------------------- tintinux · 2025-01-20 11:10 Hi I'm trying to download pages from a particular website. For example, this page. I'm using Synapse, Lazarus 3.6, FPC 3.2.2 on Ubuntu 24.04 with SSL 3.x My code is : Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i unit Unit1;{$mode objfpc}{$H+}interfaceuses  Classes,  SysUtils,  Forms,  Controls,  Graphics,  Dialogs,  StdCtrls,  httpsend,  ssl_openssl3; [...]procedure TForm1.Button1Click(Sender: TObject);var  HTTPSender: THTTPSend;begin  HTTPSender := THTTPSend.Create;  try    HTTPSender.UserAgent := 'Mozilla/5.0 (X11; Linux i686; rv:5.0) Gecko/20100101 Firefox/5.0';    HTTPSender.HTTPMethod('GET', 'https:// gw.geneanet.org/sstrebler?n=letscher+lescher&p=marie+elisabeth&oc=0&la ng=fr&type=fiche');    ShowMessage ('HTTPSender.ResultCode='+inttostr(HTTPSender.ResultCode));  finally  HTTPSender.Free;  end;end;  With the this URL and this program, I always get a 403 error. I understand that there is something specific in the web site, and they might try to avoid such download. However, with Firefox I can get the content, without error, even when I am not identified on the site, or when it is the very first request and cookies are disabled. You can try easily... I wonder whether I'm missing something or I'm doing wrong ? Is there anything I can do to download pages on this site ? Thanks for your help. ---------------------------------------------------------------------- rvk · 2025-01-20 11:35 Quote from: tintinux on January 20, 2025, 12:10:21 pmI wonder whether I'm missing something or I'm doing wrong ? Is there anything I can do to download pages on this site ? If you get the page with curl you also get a 403 error. BUT... you do get some content for the page. This page has javascript... so probably in that javascript you get redirected to the correct page. If querying this page via a browser it does not seem to give you the 403 with javascript. So there might be something the browser passes, which you are not passing. Code: Bash  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i $ curl -i " https://gw.geneanet.org/sstrebler?n=letscher+lescher&p=marie+elisabeth &oc=0&lang=fr&type=fiche"HTTP/2 403date: Mon, 20 Jan 2025 11:15:54 GMTcontent-type: text/html; charset=UTF-8accept-ch: Sec-CH-UA-Bitness, Sec-CH-UA-Arch, Sec-CH-UA-Full-Version, Sec-CH-UA-Mobile, Sec-CH-UA- Model, Sec-CH-UA-Platform-Version, Sec-CH-UA-Full-Version-List, Sec- CH-UA-Platform, Sec-CH-UA, UA-Bitness, UA-Arch, UA-Full-Version, UA- Mobile, UA-Model, UA-Platform-Version, UA-Platform, UAcritical-ch: Sec-CH-UA-Bitness, Sec-CH-UA-Arch, Sec-CH-UA-Full-Version, Sec-CH-UA- Mobile, Sec-CH-UA-Model, Sec-CH-UA-Platform-Version, Sec-CH-UA-Full- Version-List, Sec-CH-UA-Platform, Sec-CH-UA, UA-Bitness, UA-Arch, UA- Full-Version, UA-Mobile, UA-Model, UA-Platform-Version, UA-Platform, UAcross-origin-embedder-policy: require-corpcross-origin-opener- policy: same-origincross-origin-resource-policy: same-originorigin- agent-cluster: ?1permissions-policy: accelerometer=(),autoplay=(),browsing-topics=(),camera=(),clipboard- read=(),clipboard- write=(),geolocation=(),gyroscope=(),hid=(),interest- cohort=(),magnetometer=(),microphone=(),payment=(),publickey- credentials-get=(),screen-wake-lock=(),serial=(),sync- xhr=(),usb=()referrer-policy: same-originx-content-options: nosniffx- frame-options: SAMEORIGINcf-mitigated: challengecf-chl-out: gj+A6m65Nd LZYw7z5WOsQRENLN44DhQB5llm05YdHOv8kiH/pLrBTqC8zJ4DTDM74+n4OPZdMy0pSYYy UHxRPnEqSHiq3Z2+xYtU/l9KNJuEoVvfY5NcvSsJ1QS2VNsBjQrR/GfzotwEYL7haSpd/A ==$AaeCfN6KgroiTuNV2tbfpQ==cache-control: private, max-age=0, no- store, no-cache, must-revalidate, post-check=0, pre-check=0expires: Thu, 01 Jan 1970 00:00:01 GMTset-cookie: __cf_bm=OKVe3GbSbG.sphCl9rXal lHGEg6q5cU8g_bmgK1Pa_U-1737371754-1.0.1.1- NPUuPFZji97MwgsEUXwsRPL9bQRSWaqeipty_VjjYba0AzKBX8FEfukMwt27lkJQ11gBrC Iy7y5s21vFRU70WA; path=/; expires=Mon, 20-Jan-25 11:45:54 GMT; domain=.geneanet.org; HttpOnly; Secure; SameSite=Nonestrict-transport- security: max-age=31536000; includeSubDomains; preloadserver: cloudflarecf-ray: 904ea63ace796643-AMS Just a moment...