How to solve this synapse ftp(s) error? [SOLVED] lazarus / 38-Networking and Web Programming ---------------------------------------------------------------------- Hartmut · 2022-08-10 16:03 EDIT: I changed sftp to ftps because sftp was wrong. Sorry. I want to use synapse version 40 for ftps. During Login I get the error:  - "140A90C4:SSL routines:func(169):reason(196)" with library "libssl- dev" ver 1.1.1-1ubuntu2.1~1 or  - "140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol" with library "libssl1.0-dev" ver 1.0.2n-1ubuntu5.10 I'm on Linux Ubuntu 18.04 LTS 64-bit with FPC 3.2.0. Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i var FTX: TFTPSend    ok: boolean; FTX:=TFTPSend.Create;FTX.TargetHost:=...FTX.T argetPort:=...FTX.Username:=...FTX.Password:=...FTX.FullSSL:=true;ok:= FTX.Login;writeln('SSLImplementation=', SSLImplementation = TSSLOpenSSL); // => TRUE Unit 'ssl_openssl' is in the uses clause of the main program. In Unit 'ssl_openssl' is written: "You need OpenSSL libraries version 0.9.7." Do I have the wrong version? How to install the correct one? My packet manager Synaptic shows only version "1.1.1-1ubuntu2.1~1". I don't know how to install another one. Here are informations about the libraries I'm aware of: Code: Text  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i hg6@i3300:/usr/lib/x86_64-linux-gnu/ruby/2.5.0$ ls -l openssl*-rw-r-- r-- 1 root root 350120 Mai 24 16:47 openssl.so hg6@i3300:/usr/lib/x86_64-linux-gnu$ ls -l libssl*-rw-r-- r-- 1 root root 328128 Jul  6 13:25 libssl3.so-rw-r--r-- 1 root root 995650 Jul  4 13:25 libssl.alrwxrwxrwx 1 root root     13 Jul  4 13:25 libssl.so -> libssl.so.1.1-rw-r--r-- 1 root root 424664 Jun 20 19:34 libssl.so.1.0.0-rw-r--r-- 1 root root 577312 Jul  4 13:25 libssl.so.1.1 hg6@i3300:/usr/lib/x86_64-linux-gnu$ ls -l libcrypto*-rw-r--r-- 1 root root 5413392 Jul  4 13:25 libcrypto.alrwxrwxrwx 1 root root      16 Jun 27  2017 libcryptopp.so.6 -> libcrypto++.so.6lrwxrwxrwx 1 root root      16 Jul  4 13:25 libcrypto.so -> libcrypto.so.1.1-rw-r--r-- 1 root root 2361984 Jun 20 19:34 libcrypto.so.1.0.0-rw-r--r-- 1 root root 2917216 Jul  4 13:25 libcrypto.so.1.1lrwxrwxrwx 1 root root      20 Jun 27  2017 libcrypto++.so.6 -> libcrypto++.so.6.0.0-rw-r--r-- 1 root root 3634520 Jun 27  2017 libcrypto++.so.6.0.0 Thanks a lot for your help in advance. ---------------------------------------------------------------------- rvk · 2022-08-10 16:43 See my post here: https://forum.lazarus.freepascal.org/index.php/topic,60230.msg449953.h tml#msg449953 Do you want SFTP or FTPS (which are very different from each other). What you are trying to do is FTPS. Are you really trying to connect to a FTPS server (or are you trying to connect to a SFTP server)? ---------------------------------------------------------------------- Hartmut · 2022-08-10 17:22 I was not aware of this difference. The server I want to use was set up by a friend who is not a great expert (like me). I must ask him, whether he implemented sftp or ftps, but currently I cannot get him. I will write the answer when I have them. ---------------------------------------------------------------------- Hartmut · 2022-08-11 08:16 Now I got the information that the FTP-Server offers:  - ftp and ftps on port 45032  - and sftp on port 45034. What I try is ftps on port 45032. With the code I showed in the 1. Topic I get the error messages showed there too. Sorry for confusion. ---------------------------------------------------------------------- rvk · 2022-08-11 08:49 I'm not sure what version 40 is but it might not support openssl 1.1.1 Synapse trunk however has a ssl_openssl11 which should support it. So if your synapse 40 doesn't contain ssl_openssl11 you may want to try trunk. https://sourceforge.net/p/synalist/code/HEAD/tree/trunk/ If it does contain ssl_openssl11, use that instead of ssl_openssl in your uses. Edit: and for openssl 3 you'll need ssl_openssl3. ---------------------------------------------------------------------- Hartmut · 2022-08-11 10:38 Thank you rvk for that informations. I downloaded and tested trunk (r266). But this still does not work: 1) With Unit 'ssl_openssl11' I get Error "1408F10B:SSL routines:ssl3_get_record:wrong version number" at Login. Command "writeln('SSLImplementation=', SSLImplementation = TSSLOpenSSL)" still shows TRUE. 2) With Unit 'ssl_openssl3' I get again Error "SSL/TLS support is not compiled!" at Login. Command "writeln('SSLImplementation=', SSLImplementation = TSSLOpenSSL3)" now shows FALSE. Does this mean, one of my libraries is bad or missing? Which one? The only openssl library which I found is: Code: Text  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i hg6@i3300:/usr/lib/x86_64-linux-gnu/ruby/2.5.0$ ls -l openssl*-rw-r-- r-- 1 root root 350120 May 24 2022 16:47 openssl.so Please pay attention to it's directory ".../ruby/2.5.0" which looks strange to me. Packet Manager Synaptic says that I have openssl ver 1.1.1-1ubuntu2.1~1 installed and does NOT offer another version to install (nothing which starts with "openssl"). ---------------------------------------------------------------------- rvk · 2022-08-11 12:37 ssl_openssl3 is only for OpenSSL 3. I only mentioned it so for future reference when upgrading you'll might need to change it again. openssl.so isn't used. Only libssl.so.1.1 and libcrypto.so.1.1 are used in ssl_openssl11 (which uses ssl_openssl11_lib where they are loaed). Both these files are in /usr/lib/x86_64-linux-gnu so that should be correct. You could try implicit TLS (but when failing that might connect you to plain FTP) Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i //FTX.FullSSL:=true;       // ExplicitFTX.AutoTLS := true;     // implicit Quote from: Hartmut on August 11, 2022, 10:16:28 am - ftp and ftps on port 45032 FTP and FTPS are both on the same port. So you could put in some OnStatus event to read out what the communication with the server is exactly. I take it you already tried plain FTP and it worked? ---------------------------------------------------------------------- sstvmaster · 2022-08-11 12:54 Here is the actual synapse trunk version: https://sourceforge.net/p/synalist/code/HEAD/tree/trunk/ It should support openssl: 1.0, 1.1 and 3.0 also tls 1, 1.1, 1.2, 1.3. ---------------------------------------------------------------------- rvk · 2022-08-11 13:01 Quote from: sstvmaster on August 11, 2022, 02:54:17 pmHere is the actual synapse trunk version: https://sourceforge.net/p/synalist/code/HEAD/tree/trunk/ It should support openssl: 1.0, 1.1 and 3.0 also tls 1, 1.1, 1.2, 1.3. Already mentioned and tried. Quote from: Hartmut on August 11, 2022, 12:38:51 pmThank you rvk for that informations. I downloaded and tested trunk (r266). But this still does not work: ---------------------------------------------------------------------- Hartmut · 2022-08-11 13:53 Thank you very much again rvk (and sstvmaster) for your help. When setting 'FTX.AutoTLS:=true' (instead of 'FTX.FullSSL') I can login without any problems, but I can't upload or download files, because this always results to error "110=Connection timed out" after 130 seconds. This is exactly the same behaviour, as when I don't use 'FTX.FullSSL' and 'FTX.AutoTLS' which means normal FTP. The only reason, why I'm trying FTPS is to see, if (if it would work) I get with FTPS the same 130 seconds timeout as with FTP or not. So my real goal is FTP, not FTPS. FTPS is only a test which I wanted to try. If you want, we can pause the FTPS problem and switch to the FTP problem. To check if the FTP/FTPS server is working correctly I tested this with TotalCommander (a wellknown filemanger which supports FTP and FTPS) and FileZilla. Both can login and upload and download without any problems both for FTP and FTPS on port 45032. EDIT: I tested plain FTP with my code on a Windows computer and it works. No 130 seconds timeout. So it must be a Linux problem or a problem on my Linux computer. ---------------------------------------------------------------------- Zvoni · 2022-08-11 14:08 I remember setting up a FTPS-Server on a FreeBSD-Box, and everytime i tried to connect to it, i got a connect, and then the server aborted the connection. The Solution/Problem was: On the Server, the TLS-Module wasn't loaded! Have you checked if this FTPS-Server has the TLS-module loaded? Are the Keys/Certs correctly created and pointed to in the FTPS-config? At the time i was playing around with FreeBSD, i used ProFTPd, and i missed in its config, that "" was MISSING! btw: Have you tried to connect with FileZilla to this FTPS-Server? ---------------------------------------------------------------------- rvk · 2022-08-11 14:59 Quote from: Hartmut on August 11, 2022, 03:53:08 pmSo my real goal is FTP, not FTPS. FTPS is only a test which I wanted to try. If you want, we can pause the FTPS problem and switch to the FTP problem.Ok. If FTP doesn't even work we can forget about FTPS. Did you try flipping FTX.PassiveMode. (I think default is true so see if setting it to false helps) ---------------------------------------------------------------------- Hartmut · 2022-08-11 15:18 Quote from: Zvoni on August 11, 2022, 04:08:24 pmHave you checked if this FTPS-Server has the TLS-module loaded? Are the Keys/Certs correctly created and pointed to in the FTPS-config? Thank you Zvoni for trying to help me. The friend, who has configured the server, says yes (but he is not an expert for this - like me). Quotebtw: Have you tried to connect with FileZilla to this FTPS- Server? Yes, as I wrote in reply #9. I added a screenshot of the certificate shown by FileZilla (there is a chain of 4 certificates and the screenshot shows the 1st). ---------------------------------------------------------------------- Zvoni · 2022-08-11 15:41 And you did see, that the server uses TLS 1.2 ? And i do hope, those Valid from/to dates in the screenshot is american notation ---------------------------------------------------------------------- Hartmut · 2022-08-11 15:58 Quote from: rvk on August 11, 2022, 04:59:20 pmOk. If FTP doesn't even work we can forget about FTPS. Please keep in mind that FTP *can* connect to the server without problems. "Only" upload and download hangs in a 130 sec timeout. FTPS cannot even connect and results an error message. QuoteDid you try flipping FTX.PassiveMode. The default is True. After setting it to False the Login succeeds, but upload and download fail after 40 millisec. But in this case both 'FTX.Sock.LastError' and 'FTX.DSock.LastError' are zero :-(( Quote from: Zvoni on August 11, 2022, 05:41:51 pmAnd you did see, that the server uses TLS 1.2 ? I saw this but I'm a beginner to that stuff so this does not tell me something. Is this a / the problem? QuoteAnd i do hope, those Valid from/to dates in the screenshot is american notation Yes, I checked this from another certificate in the chain which is valid until "09/30/2024". ---------------------------------------------------------------------- rvk · 2022-08-11 22:56 Ok, passivemode probably needs to be true. QuoteFTP sessions work in active or passive modes: Active mode. After a client initiates a session via a command channel request, the server creates a data connection back to the client and begins transferring data. Passive mode. The server uses the command channel to send the client the information it needs to open a data channel. Because passive mode has the client initiating all connections, it works well across firewalls and network address translation gateways. You might want to implement the OnStatus event to see the communication on the command port. There you can see what ports are too be used for the actual data. Maybe something goes wrong there. With PASV mode the server will respond with a port and ip for the client to connect to (other than the default command port). Maybe the ip the client gets from the server is the wrong (internal) one. You should be able to see that in the OnStatus (although it's strange that this does work in Windows). (Btw. That second data port which the server transmits with passivemode also needs to be open in the firewall on the server side.) Could you show more complete code which works under Windows and not under Linux for plain FTP? And show what the OnStatus gives you. ---------------------------------------------------------------------- Hartmut · 2022-08-12 10:15 You found the problem rvk! It works now! Thank you so much. I will report the details later, must go outside now. But I wanted to announce this good news early so that nobody else continues working on that problem. ---------------------------------------------------------------------- Hartmut · 2022-08-12 12:01 Here comes the details how the information from rvk leaded to the solution: Quote from: rvk on August 12, 2022, 12:56:52 amWith PASV mode the server will respond with a port and ip for the client to connect to (other than the default command port). Maybe the ip the client gets from the server is the wrong (internal) one. With Linux console command pftp (passive ftp client) I got the following output: Code: Text  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i hg6@i3300:~$ pftp hostname 45032Connected to xxxxx.220 HelmiNAS FTP server ready.Name (xxxxx:hg6): xxxxx331 Password required for xxxxx.Password:230 User xxxxx logged in.Remote system type is UNIX.Using binary mode to transfer files.ftp> dir227 Entering Passive Mode (192,168,178,21,217,88)   ftp: connect: Connection timed out Now I understood the meaning of line    227 Entering Passive Mode (192,168,178,21,217,88) and saw that this was an internal IP adress! I contacted the friend who had configured the server and with this info he could find and enable something like "Report external adress in Passive Mode". After this it worked :-)) Interesting is that TotalCommander (a wellknown filemanger for Windows which supports FTP and FTPS and most of it runs on Linux via WINE) is able to automatically redirect this internal IP to the correct external IP. Extract from the TotalCommander FTP logfile: Code: Text  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i ...Connect ok!PWD257 "/" is current directory.Verzeichnis einlesenTYPE A200 Type set to A.PASV227 Entering Passive Mode (192,168,178,21,217,88)Server reports local IP -> Redirect to: 95.xxx.xxx.xxxConnected to PASV port That means, it is technically possible to detect the internal IP and redirect it to the correct external IP! Seems that synapse can do this on Windows (because my code worked on Windows), but not on Linux (because the same code got a 130 sec timeout there). Can anyone explain this difference in synapse? Again thank you very much to rvk that you pointed me to the problem. ---------------------------------------------------------------------- rvk · 2022-08-12 12:12 Quote from: Hartmut on August 12, 2022, 02:01:44 pmI contacted the friend who had configured the server and with this info he could find and enable something like "Report external adress in Passive Mode". After this it worked :-))Yes, that was what I expected. Quote from: Hartmut on August 12, 2022, 02:01:44 pmThat means, it is technically possible to detect the internal IP and redirect it to the correct external IP! Seems that synapse can do this on Windows (because my code worked on Windows), but not on Linux (because the same code got a 130 sec timeout there).An 192.168.x.x is always a local IP. The FTP client could just ignore that and try the external IP on which the connection was initiated. That's what TotalCommander seems to do. I'm not aware Synapse does this. And if it does, it should also work under Linux. If you implement the OnStatus for your program you'll see those messages there too (on Windows and Linux) and you'll know which commands are sent. One possibility... When TotalCommander detects a local IP (like 192.168.x.x) this could also be a local adres within your network. If Windows is on 10.0.0.x for example and the FTP server reports 192.168.x.x, you know it's a remote location. If Linux is on 192.168.x.x and the FTP server reports 192.168.x.x, this could also be a local network adres. How would TotalCommander see the difference? You ran TotalCommander on Windows. But your program also worked under Windows. So maybe your Windows is on a different subnet from your Linux machine? ---------------------------------------------------------------------- Hartmut · 2022-08-12 15:44 Quote from: rvk on August 12, 2022, 02:12:38 pmYou ran TotalCommander on Windows. No, I ran TotalCommander on Linux via WINE (some kind of "emulator"). The logfile extract which I showed in reply #17 was created on Linux, not on Windows. QuoteBut your program also worked under Windows. So maybe your Windows is on a different subnet from your Linux machine? I don't have a Windows which is able to access the Internet. So I sent a Windows version of my program to a friend an he ran it on a pure Windows 10 (no Virtual Machine), where it worked. This computer has local IP adress 192.168.178.20. My Linux computer has local IP 10.13.x.x. So it should be easy for synapse to detect, that 192.168.x.x must be a remote location. I don't know which local IP adress TotalCommander sees on Linux via WINE. QuoteIf you implement the OnStatus for your program you'll see those messages there too (on Windows and Linux) and you'll know which commands are sent. This sounds interesting. I have found this Event and saw that it needs a class to work. It will take some work for me to implement this, because I'm not too familiar with that stuff. I want to do this just for curiosity after my project is finished. ---------------------------------------------------------------------- rvk · 2022-08-12 16:04 Quote from: Hartmut on August 12, 2022, 05:44:19 pmI don't have a Windows which is able to access the Internet. So I sent a Windows version of my program to a friend an he ran it on a pure Windows 10 (no Virtual Machine), where it worked. This computer has local IP adress 192.168.178.20.That's not the same friend which has the FTP server by any chance? Because then it's obvious that it worked. It just connected internally  Quote from: Hartmut on August 12, 2022, 05:44:19 pmMy Linux computer has local IP 10.13.x.x. So it should be easy for synapse to detect, that 192.168.x.x must be a remote location. I don't know which local IP adress TotalCommander sees on Linux via WINE.Yes. But I'm not aware Synapse does this (like TC). You might be able to force the IP of the server yourself (haven't look yet). But you'll need to implement the OnStatus to detect the PASV command. Quote from: Hartmut on August 12, 2022, 05:44:19 pmQuoteIf you implement the OnStatus for your program you'll see those messages there too (on Windows and Linux) and you'll know which commands are sent. This sounds interesting. I have found this Event and saw that it needs a class to work. It will take some work for me to implement this, because I'm not too familiar with that stuff. I want to do this just for curiosity after my project is finished. You can create a dummy class with an event and assign that event to your OnStatus. If you make it a class function you don't even need to create an insurance for that class. When you are at that point I can show an example. It would just be something like (I'm on mobile so forgive any typos) Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i Type  TMyDummyClass = class(TObject)  public    class procedure OnStatus(etc. Etc cant look up the def now);  end;//...class procedure TMyDummyClass.OnStatus(etc. Etc cant look up the def now);begin  Writeln(logline);end;//...  fTX.OnStatus := @TMyDummyClass.OnStatus; ---------------------------------------------------------------------- Hartmut · 2022-08-13 08:55 Quote from: rvk on August 12, 2022, 06:04:17 pmQuote from: Hartmut on August 12, 2022, 05:44:19 pmI don't have a Windows which is able to access the Internet. So I sent a Windows version of my program to a friend an he ran it on a pure Windows 10 (no Virtual Machine), where it worked. This computer has local IP adress 192.168.178.20.That's not the same friend which has the FTP server by any chance? Because then it's obvious that it worked. It just connected internallyIt was the same friend, so this test was useless. QuoteCode: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i Type  TMyDummyClass = class(TObject)  public    class procedure OnStatus(etc. Etc cant look up the def now);  end;//...class procedure TMyDummyClass.OnStatus(etc. Etc cant look up the def now);begin  Writeln(logline);end;//...  fTX.OnStatus := @TMyDummyClass.OnStatus;Thank you for that draft, it will help me. ---------------------------------------------------------------------- rvk · 2022-08-15 10:27 For completeness sake, I made a little example for detecting wrong remote IP with PASV for Synapse. You will need to override the ParseRemote() of TFTPSend to get to the point where you can change the FDataIP (the IP address over which the data is send). This is reported by PASV and is parsed in ParseRemote(). In that same class you can put your own OnStatus te assign the your FTX. This will detect (in ParseRemote) if the reported dataIP (in PASV) is the same as the hostIP (which is Self.Sock.GetRemoteSinIP). If not, then we probably have an internal IP and can override the FDataIP with Self.Sock.GetRemoteSinIP. (BTW. My FTP server reports an internal IP when using PASV for internal connections but an external IP for external connections. So this wrong reporting of IP only happens with wrongly configured FTP servers. But apparently it happens often enough for TotalCommander to build in the check   ) Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i program Project1; uses  FTPSend, blcksock; type  TMyFTPSend = class(TFTPSend)  public    procedure ParseRemote(Value: string); override;    class procedure OnMyStatus(Sender: TObject; Response: boolean; const Value: string);  end;   class procedure TMyFTPSend.OnMyStatus(Sender: TObject; Response: boolean; const Value: string);  begin    Writeln(Value);  end;   procedure TMyFTPSend.ParseRemote(Value: string);  begin    inherited;    Writeln('--> PASV reports data channel: ' + FDataIP + ' on ' + FDataPort);    Writeln('--> Server is on ' + Self.Sock.GetRemoteSinIP);     // Check here if FDataIP the same as IP of control channel    // if not, we probably have a internal ip     if (FDataIP <> Self.Sock.GetRemoteSinIP) then    begin      Writeln('--> So PASV is probably reporting an internal IP');      Writeln('--> Redirecting to server IP: ' + Self.Sock.GetRemoteSinIP);      FDataIP := Self.Sock.GetRemoteSinIP; // we keep FDataPort the same    end;   end; var  FTX: TFTPSend;  Ok: boolean;begin  FTX := TMyFTPSend.Create; // Note TMyFTPSend, not the default TFTPSend  FTX.TargetHost := 'ftp.mydomain.com';  FTX.TargetPort := '21';  FTX.Username := 'user';  FTX.Password := 'password';  FTX.OnStatus := @TMyFTPSend.OnMyStatus;  Ok := FTX.Login;  if Ok then Writeln('Login ok')  else    Writeln('Error loging in ' + FTX.Sock.GetErrorDescEx);  FTX.List('/', True); // reading of data channel not implemented here   Readln; // wait for enter to end console app end.