Simple https get, openssl problems
lazarus / 38-Networking and Web Programming

----------------------------------------------------------------------
lazarusjulian · 2025-05-30 02:26

Hey, I need some help please, just want to get the source text of a
simple website which contains json data, but Im really struggling and
trying to get the text now for hours. I've tried using TFPHttpClient
which leads to the message "could not initialize openssl library" and
synapse, which works but the server gives me a  "301 moved
permanently" response.

I see there are a lot of issues about openssl versions..

Maybe someone would be so nice to give me a working sample solution? a
sample project would be awesome. I know this may sound like i did not
try to solve this by myself, but I have not enough experience to get
this all sorted with openssl versions and so on.

Greetings, Julian

----------------------------------------------------------------------
Jurassic Pork · 2025-05-30 03:38

hello,
what is your O.S and your fpc lazarus versions ?
Friendly, J.P

----------------------------------------------------------------------
egsuh · 2025-05-30 04:35

In the following thread,

https://forum.lazarus.freepascal.org/index.php/topic,68908.msg533823.h
tml#msg533823

paweld says following:

QuoteOpenSSL libraries must be in the program directory (the best
solution) or in the Windows\System32 directory (Windows\SysWOW64 - if
64-bit system and the library is 32-bit).
You also have to remember that Lazarus 3.6 is being released with FPC
version 3.2.2, which supports OpenSSL versions 1.0.2 and 1.1 - it does
not support version 3

----------------------------------------------------------------------
paweld · 2025-05-30 05:18

Synapse does not have automatic redirect support. But you can add the
following unit to your project:
https://github.com/paweld/especies/blob/master/shared/LPDNetU.pas
and use it in your code, e.g.
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i uses 
LPDNetU; procedure TForm1.Button1Click(Sender: TObject);var  hc:
THTTPClient;  outstring: String;begin  hc := THTTPClient.Create;  if
hc.Get('https://forum.lazarus.freepascal.org', outstring) then   
ShowMessage('HTTP OK')  else    ShowMessage('HTTP error: ' +
hc.ResultMessage;  hc.Free;end;

----------------------------------------------------------------------
Thaddy · 2025-05-30 05:44

fcl-net/web has redirect support too, no need to go beyond the
distribution. THTTPClient is the correct option. Will add example a
bit later.
And indeed, if you compile for 32 bit, 32 bit OpenSSL, 64 bit, 64 bit
OpenSSL. And make sure they are recent, either 3.x or 1.1.1.x. The
latter is LTS version of the 1.x series.
So basically few things wrong with you code, just the OpenSSL.

----------------------------------------------------------------------
paweld · 2025-05-30 06:06

@Thaddy, it has been said more than once - the stable FPC (3.2.2) does
not support OpenSSL 3, which is the reason TS started the thread.

----------------------------------------------------------------------
Thaddy · 2025-05-30 11:26

Then copy over the trunk version because it has no dependencies on
newer features.
I am on trunk though. I guess it should be backported into 3.2.3? Have
to test the RC.

----------------------------------------------------------------------
lazarusjulian · 2025-05-30 11:29

Thank you all very much for your responses!

@paweld, thanks for the unit. Unfortunately it works not for me, the
web server 301 does not contain the redirect-link, because the link is
already correct but something is going wrong in the http request?


Try with this link:
www.kleintraktorforum.de/jsontest

in my opera browser, i get the pure json string data. But synapses
gets: Quote<html>
<head><title>301 Moved Permanently</title></head>
<body>
<center><h1>301 Moved Permanently</h1></center>
<hr><center>nginx/1.22.1</center>
</body>
</html>
I am using Windows 10, Lazarus 4.0 FPC 3.2.2.

----------------------------------------------------------------------
paweld · 2025-05-30 11:44

Sample in attachment

----------------------------------------------------------------------
olatov · 2025-05-30 11:53

Try with https:// scheme in the URL. Your link has http://, most
likely what you're seeing is a https redirection.

----------------------------------------------------------------------
paweld · 2025-05-30 12:14

In my example, it works correctly, i.e. first is:
Code: [Select]HTTP/1.1 301 Moved Permanently
Server: nginx/1.22.1
Date: Fri, 30 May 2025 12:08:13 GMT
Content-Type: text/html
Content-Length: 169
Connection: keep-alive
Location: https://www.kleintraktorforum.de/jsontestand then:
Code: [Select]HTTP/1.1 200 OK
Server: nginx/1.22.1
Date: Fri, 30 May 2025 12:08:14 GMT
Content-Type: text/html
Content-Length: 410
Connection: keep-alive
Content-Location: jsontest.html
Vary: negotiate,Accept-Encoding
TCN: choice
Last-Modified: Fri, 30 May 2025 11:20:32 GMT
ETag: "19a-636589a2ff72e;6365878cc47ac
Accept-Ranges: bytes

----------------------------------------------------------------------
lazarusjulian · 2025-05-30 22:09

paweld, thanks a lot!!
Still not getting it working (got 500 internal server response), so
guess my fault is using a wrong ssl_openssl unit, what is the best
source/way to get it? Sorry for the dumb question I am not so
experienced here.

Greetings,Julian

----------------------------------------------------------------------
paweld · 2025-05-31 09:02

you need to add “ssl_openssl3*” files to the synapse package
requirements.
Having the project from the previous post attachment open:
1. Menu > Project > Project Inspector
2. double click on synapse package
3. Add > Add Files from File System
4. select the files “ssl_openssl3.pas” and “ssl_openssl3_lib.pas” and
click “Open”
5. save the changes
6. build application and run

----------------------------------------------------------------------
lazarusjulian · 2025-06-04 23:24

Very strange...
Just to  make sure, I did a clean reinstall of Lazarus 4.0, downloaded
from https://www.lazarus-ide.org
I downloaded your sample project.
I downloaded Synapse from https://github.com/geby/synapse and
extracted all files into the sample project folder.

Tried to add the files in object inspector, but they were already
linked.
Run the project but still - 500 error.
Already deactivated windows defender firewall, if that could cause the
problem.

----------------------------------------------------------------------
Leledumbo · 2025-06-05 06:37

Quote from: lazarusjulian on June 05, 2025, 01:24:52 amRun the project
but still - 500 error.
Already deactivated windows defender firewall, if that could cause the
problem.
5xx is a server side error that can be caused by anything at the
server side, even when your request is correct. If your request is
wrong, it will respond with 4xx instead.
Quote from: lazarusjulian on May 30, 2025, 01:29:38 pmTry with this
link:
www.kleintraktorforum.de/jsontest
The server side is indeed wrong in the first place, what you see in
the browser is actually a HTML formatted content, despite JSON in the
body. Here's the unrendered response (use view source functionality of
your browser):
Code: Text  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i  <html>   
   <head>          <meta name="color-scheme" content="light dark">   
     <meta charset="utf-8">          <style type="text/css"
id="operaUserStyle"></style>        </head>         <body>         
<pre>            [{name: "John", age: 31, city: "New York"}, {name:
"Peter", age: 49, city: "Los Angeles"}, {name: "Tina", age: 23, city:
"Baltimore"}]          </pre>                  <div class="json-
formatter-container"></div>        </body></html> Content-type header
is also text/html, which is again, wrong. It should be
application/json. The issue is no longer about HTTPS / OpenSSL, it's a
broken server side implementation.

----------------------------------------------------------------------
Warfley · 2025-06-05 19:44

You should take a look at how HTTP works, specifically the HTTP
repsone codes: https://developer.mozilla.org/en-
US/docs/Web/HTTP/Reference/Status

When you get a response code starting with a 1 or a 2 (like 100, 200,
etc.) everything was alright.

If you get a code starting with a 3 as your original 301 it is a
redirect, this is not an error, simply the information that the server
requests you to try again at a different address. You can either
follow redirects manually or just set the AllowRedirect property in
your TFPHttpClient instance.

Response codes starting with 4 means that your request is wrong. Most
likely you don't have permissions (403) or you request things that are
not available (404)

Lastly there are the response codes starting with 5, those are server
side errors, which means it's not your fault that things have gone
wrong, nor that you can do anything about it. That usually is a bug or
a misconfiguration on the end of the server side hosting. E.g. a
typical error when the server software has crashed and your response
can't be served (if you ever manually hosted Gitlab, you know that all
to well).

In any case, before you start doing web stuff, maybe take a look into
how HTTP works. It's a very simple protocol and just reading the
Wikipedia Page will already get you a long way