[Solved] How to create a secure server socket using FPC units lazarus / 38-Networking and Web Programming ---------------------------------------------------------------------- jollytall · 2019-07-05 07:14 I have a fully working server. It uses the Sockets unit, and the "standard" set-up. Sort of pseudo-code: MySocket:=fpSocket(...) fpBind(MySocket...) fpListen(MySocket...) repeat   NewSocket:=fpAccept(MySocket...)   HandleSocketInNewThread(NewSocket);   until Terminated; Now I would like to make the socket secure (HTTP->HTTPS). I can do it with stunnel, but then I loose the remote IP and also probably decrease performance. So I would like to "upgrade" my sockets to TLS/SSL compatible. I would need a bit of initial help, how to start. (Remark: I do not use - for this project - Indy, but try to solve it on the lowest level possible). ---------------------------------------------------------------------- marcov · 2019-07-05 08:35 Looking at the higher level systems can then still help, even if you don't use them. And that goes for FPC's internal socket suite too. (lnet, used in various http units in fcl-web) ---------------------------------------------------------------------- SymbolicFrank · 2019-07-05 11:28 Synapse is complicated, but pretty low-level. And relatively easy to use. ---------------------------------------------------------------------- jollytall · 2019-07-06 08:48 Thanks for the answers. I spent a whole day trying to understand what is possible and how. Actually I am more confused among the various packages than I was before. What I found is that there are many approaches: There are units that are called "part of the Free Pascal run time library". These are the ones easiest to use, and I can also understand and use them: Sockets, some fpXXX, e.g. fphttpclient. If I understand well, these are the simplest, non OO, etc. On the next level are the ones called "part of the Free Component Library (FCL)". Some examples are ssockets, sslsockets, but also fpopenssl, what is based on the fpXXX name could be in the above category too, but not; it is OO as well. Then there is "Ararat Synapse", what I would expect to be one level higher, or at least parallel to the OO FCL. Still fpopnessl is based on / wrapping around thel Synapse openssl unit. I got stuck understanding the openssl unit. Then even one level higher (or maybe better to say parallel to Synapse) there is Indy10. It would work, but there are annoying issues with it: e.g. when calling fpSend it adds the Flags (actually 0, i.e. no flag) somewhere deep in the calling hierarchy, so the user has no chance to include an MSG_NOSIGNAL flag, what is needed to avoid SIGABRT while using. Would need to start changing Indy10, what I find too dangerous. My hope is that I found the following link: http://free-pascal- lazarus.989080.n3.nabble.com/Lazarus-SSL-socket-support-rework-amp- HTTPS-support-in-FPC-HTTP-Server-td4055761.html. According to this Michael rewrote the "native FPC socket support" (I assume then non-OO) to include ssl support. The issue is that I do not find anywhere the new source. Removed and reinstalled the latest Lazarus/FPC (2.0.2 and 3.0.4) but the support is not included. Any idea where that can be downloaded from? Thanks, ---------------------------------------------------------------------- k1ng · 2019-07-06 08:58 Quote from: jollytall on July 06, 2019, 10:48:57 amThen there is "Ararat Synapse", what I would expect to be one level higher, or at least parallel to the OO FCL. Still fpopnessl is based on / wrapping around thel Synapse openssl unit. I got stuck understanding the openssl unit. Synapse is more or less dead, not sure if you should start writing a new app based on it. Quote from: jollytall on July 06, 2019, 10:48:57 amThen even one level higher (or maybe better to say parallel to Synapse) there is Indy10. It would work, but there are annoying issues with it: e.g. when calling fpSend it adds the Flags (actually 0, i.e. no flag) somewhere deep in the calling hierarchy, so the user has no chance to include an MSG_NOSIGNAL flag, what is needed to avoid SIGABRT while using. Would need to start changing Indy10, what I find too dangerous. If you have questions about Indy10, just ask here. I'm sure Remy Lebeau will help you and might also commit your changes made to Indy Additionally there is Fundamentals 5 Code Library for FreePascal and Delphi which seems very lightweight compared to Indy OO code. ---------------------------------------------------------------------- jollytall · 2019-07-06 11:04 Thanks. Quote from: k1ng on July 06, 2019, 10:58:51 amSynapse is more or less dead, not sure if you should start writing a new app based on it. I really do not want to build much on it, but apparently Synapse openssl is the only open ssl implementation under FPC everybody is using, so it is unavoidable. Quote from: k1ng on July 06, 2019, 10:58:51 amIf you have questions about Indy10, just ask here. I'm sure Remy Lebeau will help you and might also commit your changes made to Indy Surely it is the case, but to be even able to ask educated questions, I will need to spend more days on it. This is what I wanted to avoid. Quote from: k1ng on July 06, 2019, 10:58:51 amAdditionally there is Fundamentals 5 Code Library for FreePascal and Delphi which seems very lightweight compared to Indy OO code. Thanks for the suggestion. Lots of good stuff, but I did not see SSL support among them. Could you direct me to the right file, please? ---------------------------------------------------------------------- Thaddy · 2019-07-06 11:13 Then you didn't look too well.... SSL support is standard, with the minor flaw that the bottom line is ssl2 instead of tls 1.1 It actually should be tls1.2. I am working on it. ---------------------------------------------------------------------- k1ng · 2019-07-06 12:04 Quote from: jollytall on July 06, 2019, 01:04:33 pmQuote from: k1ng on July 06, 2019, 10:58:51 amAdditionally there is Fundamentals 5 Code Library for FreePascal and Delphi which seems very lightweight compared to Indy OO code. Thanks for the suggestion. Lots of good stuff, but I did not see SSL support among them. Could you direct me to the right file, please? https://github.com/fundamentalslib/fundamentals5/tree/master/Source/TL S ---------------------------------------------------------------------- jollytall · 2019-07-06 15:15 Thanks. I did look as well as I could, but was mislead by "Update for Windows7". Anyway now I found. One small bug to start with: In flcSysUtils under uses for Unix/Linux SysUtils need to be included as well (line 74). So now it compiles, but still could not figure out how to make a working socket server with TLS. Have I missed an example file somewhere again? Can you show the logic, how it is supposed to work? Is somewhere a description available? Thanks, ---------------------------------------------------------------------- k1ng · 2019-07-06 16:05 Quote from: jollytall on July 06, 2019, 05:15:53 pmAnyway now I found. One small bug to start with: In flcSysUtils under uses for Unix/Linux SysUtils need to be included as well (line 74). It's there, no? Quote from: jollytall on July 06, 2019, 05:15:53 pmSo now it compiles, but still could not figure out how to make a working socket server with TLS. Have I missed an example file somewhere again? Can you show the logic, how it is supposed to work? Is somewhere a description available? I'm not the author nor have I ever used it. Unfortunately it seems to not provide any usage examples, at least I haven't found some Guess you have to use TF5TCPClient or TfclTCPClient which hooks up the TLS classes ---------------------------------------------------------------------- jollytall · 2019-07-06 18:28 Quote from: k1ng on July 06, 2019, 06:05:21 pmIt's there, no? I am not experiences on GitHub, but it seems there are two branches, and they differ in this. Now I could download both versions. I got fundamentals5-master.zip, what is the old version and fundamentals5-03181926da5009ed74b7c9b78b06d362561a5abf.zip, what is the corrected version. In the version history now I even see, that this bug was noticed and corrected before, but somehow it did not get into the -master. Quote from: k1ng on July 06, 2019, 06:05:21 pmGuess you have to use TF5TCPClient or TfclTCPClient which hooks up the TLS classes Actually I want to make a server using its own certificate, what is probably more difficult than a Client accepting the server's. I keep on trying. Nonetheless, it is very sad that people (including FCL, Synapse, Indy as well) work a lot, create something very powerful, but still it is almost impossible to use. I did a search today for some of the random units available as part of the default distribution of FCL and google found nothing (other than the source of course). ---------------------------------------------------------------------- k1ng · 2019-07-06 19:07 Quote from: jollytall on July 06, 2019, 08:28:11 pmQuote from: k1ng on July 06, 2019, 06:05:21 pmIt's there, no? I am not experiences on GitHub, but it seems there are two branches, and they differ in this. Now I could download both versions. I got fundamentals5-master.zip, what is the old version and fundamentals5-03181926da5009ed74b7c9b78b06d362561a5abf.zip, what is the corrected version. In the version history now I even see, that this bug was noticed and corrected before, but somehow it did not get into the -master. Indeed, if I download the master.zip (archive shows hash 51fd288225df36e47e89b90a8705b6898278d035 which is the newest commit) it looks like this: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i {$IFDEF FREEPASCAL}{$IFDEF UNIX}uses  BaseUnix,  Unix;{$ENDIF}{$ENDIF}  Never seen something like this before Quote from: jollytall on July 06, 2019, 08:28:11 pmQuote from: k1ng on July 06, 2019, 06:05:21 pmGuess you have to use TF5TCPClient or TfclTCPClient which hooks up the TLS classes Actually I want to make a server using its own certificate, what is probably more difficult than a Client accepting the server's. I keep on trying. Nonetheless, it is very sad that people (including FCL, Synapse, Indy as well) work a lot, create something very powerful, but still it is almost impossible to use. I did a search today for some of the random units available as part of the default distribution of FCL and google found nothing (other than the source of course). For Indy many code examples/questions are available. For fundamentalslib I found these via Google: https://code.google.com/archive/p/fundamentals/issueshttps://sourcefor ge.net/p/fundementals/discussion/Maybe someone has shared his code there as an example EDIT: found those examples: http://fndsocks.narod.ru/https://sourceforge.net/p/fundementals/discus sion/2117/thread/ee2bd3af/http://fundementals.sourceforge.net/XMLRichE ditDemo.zip ---------------------------------------------------------------------- jollytall · 2019-07-09 15:28 Thanks for all the suggestion. I went through a lot of material, but could not find a Pascal solution that is not ObjectOriented, that is easy to understand. Finally I found a C example at https://wiki.openssl.org/index.php/Simple_TLS_Server, what I could convert to Pascal. It works!!! It only uses openssl unit (what is a FP wrapper around openssl - actually a Synapse unit). ---------------------------------------------------------------------- lucamar · 2019-07-09 21:12 Would you mind sharing that example (even if just attaching it to a post), with a link to the C code? ---------------------------------------------------------------------- jollytall · 2019-07-10 07:46 The link to the original C code is in my previous Post :-). The working (but be aware very simple!) code in Pascal is: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i unit Unit1; {$mode objfpc}{$H+} interface uses cthreads, Classes, SysUtils, Forms, Controls, Graphics, Dialogs, StdCtrls,     openssl,sockets; type  TForm1 = class(TForm)    Button1: TButton;    Memo1: TMemo;    procedure Button1Click(Sender: TObject);  private    function CreateContext:PSSL_CTX;    procedure ConfigureContext(AContext:PSSL_CTX);  end; var  Form1: TForm1; implementation {$R *.lfm}function TForm1.CreateContext:PSSL_CTX;  var    ctx:PSSL_CTX;  begin  ctx:=SSLCTXnew(SslMethodV23);  if ctx=nil then    memo1.lines.add('Error in Create_Context '+IntToStr(ErrGetError));  result:=ctx;  end; procedure TForm1.ConfigureContext(AContext:PSSL_CTX);  begin  SSLCTXsetmode(AContext, 1);  // load the two PEM files  if (SSLCTXusecertificatefile(AContext, 'cert.pem', SSL_FILETYPE_PEM) <= 0) then    memo1.lines.add('Error in Create_Context Certificate '+IntToStr(ErrGetError));  if (SSLCTXusePrivateKeyfile(AContext, 'key.pem', SSL_FILETYPE_PEM) <= 0 ) then    memo1.lines.add('Error in Create_Context Private Key '+IntToStr(ErrGetError));  end; procedure TForm1.Button1Click(Sender: TObject);  var    FListeningSocket:               integer;    FActualSocket:                    integer;    FSocketAddress:                   tSockAddr;    FSocketAddressSize:             integer;    FContext:                         PSSL_CTX;    InAddress:                        in_addr;    FSSL:                           PSSL;    FBuffer:                          packed array[0..16635] of char;  const    reply='HTTP/1.1 200 OK'+#10#13#10#13+'Success'+#10#13;  begin  // call the opensll basic set-up function  InitSSLInterface;  // create and configure the context  FContext:=CreateContext;  ConfigureContext(FContext);  // make, bind a listen on a listening socket  FListeningSocket:=fpsocket(AF_INET,SOCK_STREAM,0);  if FListeningSocket=-1 then exit;  InAddress.s_addr:=htonl(0); // all addresses  with FSocketAddress do    begin    sin_family:=AF_INET;    sin_port:=htons(4443);  // to be above 1024: in Linux to avoid using root    sin_addr:=InAddress;    end;  FSocketAddressSize:=sizeOf(FSocketAddress);  if fpBind(FListeningSocket,@FSocketAddress,FSocketAddressSize)=-1 then  exit;  if fpListen(FListeningSocket,100)<>0 then // max 100 parallel connections    exit;  // the loop receiving the requests. Be aware it is  // - not in a thread making it a non-responding window for the OS  // - the actual serving socket is in the same thread (one connection at a time)  // - never exits unless there is an error  repeat    // accept the socket    FActualSocket:=fpaccept(FListeningSocket, @FSocketAddress, @FSocketAddressSize);    if FActualSocket>0 then    begin      // make and SSL handler for it      FSSL:=SSLnew(FContext);      SSLsetfd(FSSL, FActualSocket);      // handshake      if SSLaccept(FSSL) <= 0 then        memo1.lines.add('Error in Accept '+IntToStr(ErrGetError))      else      begin        // ead from the socket        FillByte(FBuffer,16636,0);        SSLRead(FSSL,@FBuffer[0],16636);      memo1.lines.add('READ from client:');        memo1.lines.add(FBuffer);        // write to the socket        SSLwrite(FSSL,@Reply[1],Length(Reply));        end;      // finsih the connection      SSLfree(FSSL);      fpShutDown(FActualSocket,2);      Application.ProcessMessages;      end;    until (FActualSocket<=0); // error in listen  // house cleaning after the loop (if gets here by mistake :-) )  fpShutDown(FListeningSocket,2);  SSLCTXfree(FContext);  DestroySSLInterface;  end; end. You can test it from a browser with https://127.0.0.1:4443 ---------------------------------------------------------------------- lucamar · 2019-07-10 11:58 Thanks Ah .. for future reference, please read this: https://wiki.freepascal.org/Forum ---------------------------------------------------------------------- jollytall · 2019-07-10 15:13 Thanks. Fixed both the code and the Subject.