HTTPS with Synapse on Mac lazarus / 33-macOS - Mac OS X ---------------------------------------------------------------------- Frantic · 2011-03-06 08:58 Hi all! Have anyone used Synapse under Mac/FPC to perform HTTPS queries? I used it only under Win platform by adding ssl_openssl to my "uses" section (as their wiki says). What plugin is suitable for Mac? I've tried including every ssl_*.pas unit with no luck: with some of them it doesn't compile, others just don't make any difference (I get error message about missing SSL implementation when I try to access web-page via HTTPS) Any help appreciated. Thanks! ---------------------------------------------------------------------- jixian.yang · 2011-03-06 13:07 Did you download Synapse from its SVN (https://synalist.svn.sourceforge.net/svnroot/synalist/trunk/)? The stable release .39 is too old. ---------------------------------------------------------------------- Frantic · 2011-03-06 18:54 Yes, latest sources are taken from SVN ---------------------------------------------------------------------- jixian.yang · 2011-03-07 03:47 Is OpenSSL installed on you MAC? ---------------------------------------------------------------------- cpalx · 2011-03-08 04:57 and how do you do that, i have a similar problem to send email by a google account. ---------------------------------------------------------------------- Frantic · 2011-03-09 19:00 Sorry, my bad. Really, the code in latest SVN trunk (r137) works. 2 cplax: I just add ssl_openssl to 'uses' section and use HTTPSend.HTTPMethod('POST', 'https://someurl'); ---------------------------------------------------------------------- cpalx · 2011-03-10 03:14 ok, thanks, last svn works ---------------------------------------------------------------------- mdalacu · 2011-10-05 08:54 Hi, i am using latest revision 146. Can some one test if https is working through proxy (no authentication) under linux. On windows it is working just fine but i can't get it to work under Ubuntu. In  HTTPSend.ResultCode i always get 403. I've installed every ssl lib that i can think of... Please test it and provide feedback ... meaby it is an error in my code or in synapse https implementation. Let's find out! I am posting here because i don't want to start another thread only for this. Thank you very much. ---------------------------------------------------------------------- mdalacu · 2012-02-01 12:15 Bunm, any help? Please try to download anything using HTTPSend.HTTPMethod('GET', 'https://someurl'); under linux (ubuntu) and see if it works. I really need this to work but i don't know how to make it. Please post the smallest example. ---------------------------------------------------------------------- ludob · 2012-02-01 14:49 QuoteI am posting here because i don't want to start another thread only for this.Mistake. There are few mac users on this forum and even less using Synapse on Mac. Anyways, this very simple program works for me. Code: [Select]program project1; uses   Classes,httpsend,ssl_openssl,synautil; var HTTP:THTTPSend;   Result:boolean; begin   HTTP := THTTPSend.Create;   try     Result := HTTP.HTTPMethod('GET', 'https://www.google.com/');     if Result then       writeln(ReadStrFromStream(HTTP.Document,HTTP.Document.Size))     else       begin         writeln('HTTP.Sock.LastError :',HTTP.Sock.LastError, ' ; ' ,HTTP.Sock.LastErrorDesc);         writeln('HTTP.Sock.SSL.LastError :',HTTP.Sock.SSL.LastError, ' ; ' ,HTTP.Sock.SSL.LastErrorDesc);       end;   finally     HTTP.Free;   end; end. If it doesn't dump the google main page on the console, post the error messages. QuoteIn  HTTPSend.ResultCode i always get 403. I've installed every ssl libHave you also installed the -dev packages? If not you have to create yourself the symbolic links to the libraries. Fe. for openssl you have to install the libssl-dev package or create the links /usr/lib/libssl.so and /usr/lib/libcrypto.so. When installing the binary package you'll have only something like /usr/lib/libssl.so.0.9.8 and  /usr/lib/libcrypto.so.0.9.8. Synapse looks for the .so files. ---------------------------------------------------------------------- mdalacu · 2012-02-02 13:50 Thank you for your reply.After solving this problem with your help I have encountered another problem, it does not work if i use a proxy server. Under windows is working fine with or without proxy but under linux it works  only if i connect directly. The error that i am getting is "McAfee Web Gateway was not allowed to CONNECT to port 443". I think that is trying to connect to the proxy on port 443 and not 8080 as defined in connection.  Over http it works just fine. Again, on windows it is working What can i do? Thank you. ---------------------------------------------------------------------- ludob · 2012-02-02 18:47 This looks very much like a message from the proxy server meaning that you are connect all right on the correct port. Are you using the same machine (ip address) and credentials for your connection? If it works from one machine and not another, this could be an acl issue.  Check your proxy configuration. ---------------------------------------------------------------------- mdalacu · 2012-02-02 20:30 Indeed, the message is from the proxy server but the machine in question in my worksation at work and has dualboot windows and ubuntu. Both computers account members in the same OU in our domain (likewise on ubuntu). In windows the programs compiles and runs perfectly then when i boot in linux the program compiles perfectly and works on every https addres but not through that damn proxy. A very imprtant thing: if i run the exe (buid in windows) through wine it works as expected !  It clearly has something to do with proxy connect under linux, but what?! Any ideeas , what can i try next? Thank you very much ludob. ---------------------------------------------------------------------- mdalacu · 2012-02-03 07:32 I have found a partial workaround! If i change line 68 in synsock.pas from {$I ssfpc.inc} into {$I sslinux.inc} everything works perfectly, namely https connection over proxy in linux! :-) The problem is that this solution only works on x86 and not x64 because sslinux.inc uses libc unit. So the problem is in ssfpc.inc but i am over my head (~not a chance~) to fix this.  Meabe someone experienced could look into this and fix it! What can i do to help? Thank you. ---------------------------------------------------------------------- ludob · 2012-02-03 08:07 Hmm. That means that there is different behavior depending on the socket level implementation which is somewhat surprising. I don't have access to a https transparent proxy so there isn't a lot I can test. Do you have wireshark installed or can you install it? Having a trace of the connection with {$I ssfpc.inc} and with {$I sslinux.inc} would be helpful in diagnosing the problem. ---------------------------------------------------------------------- mdalacu · 2012-02-03 09:06 I have put wireshak and i have found the problem. When using sslinux the request is made to destination server on 443 port and that is corect, if i use ssfpc the request is made on port 47873, why?! Our proxy is configured to allow https request only on port 443. I have attached the wireshark dump from both cases. Our proxy has this ip address : 10.241.155.1, sort by destination and look for "CONNECT www.raiffeisenonline.ro:XXXXX" Can you help? Thank you very much. ---------------------------------------------------------------------- ludob · 2012-02-03 10:39 The difference is in the implementation of function ResolvePort. ssfpc.inc calls always htons which is swapping the low and high byte on a little endian system (intel machines are little endian). The function is called from TTCPBlockSocket.HTTPTunnelDoConnect. To be honest, I can't imagine what configuration needs ResolvePort in this function to figure out the real port number to use.  So what I suggest is to comment in blcksock.pas line 3859 (your line number could be slightly different, it is the first line of TTCPBlockSocket.HTTPTunnelDoConnect): Code: [Select]//  Port := IntToStr(ResolvePort(Port)); I'll look further into how to fix ResolvePort but above workaround should be getting you connected. ---------------------------------------------------------------------- mdalacu · 2012-02-03 11:38 It's working, thank you very much! Such a simple solution, i was tring to fix ResolvePort without success.  I have tested it under Linux x86, x64 and Windows x64 and it works perfectly! Anyway, it is something wrong in ssfpc.inc because ResolvePort function does not work, in sslinux.inc it does. I don't know if synapse is still in development, how to fix this permanently? So, ludob you are my hero ! ---------------------------------------------------------------------- ludob · 2012-02-03 12:05 The difference between the 2 ResolvePort implementations is that one returns host byte order and the other network byte order. Repairing is simple once the correct order is defined. From the HTTPTunnelDoConnect perspective ssfpc is using the wrong order. Comparing to the windows implementation, ssfpc is also wrong. I'll submit the following patch for ssfpc ResolvePort to the Synapse mailing list: Code: [Select]function ResolvePort(Port: string; Family, SockProtocol, SockType: integer): Word; var   ProtoEnt: TProtocolEntry;   ServEnt: TServiceEntry; begin   Result := StrToIntDef(Port, 0);   if Result = 0 then   begin     ProtoEnt.Name := '';     if GetProtocolByNumber(SockProtocol, ProtoEnt) then       begin       ServEnt.port := 0;       if GetServiceByName(Port, ProtoEnt.Name, ServEnt) then         Result := synsock.htons(ServEnt.port);       end;   end; end; ---------------------------------------------------------------------- mdalacu · 2012-02-03 13:06 Hi, if i make this modification in ssfpc.inc and also uncomment the line in blcksock.pas then it will not work, same result as before. Do you need some output form me? ---------------------------------------------------------------------- ludob · 2012-02-03 13:18 Thank you for testing. Could you put a breakpoint on the line uncommented in blcksock.pas and look at the value of port before it enters ResolvePort and after. It should be 443 before and after. You can step through the function if you want. It should do nothing more than the Result := StrToIntDef(Port, 0); ---------------------------------------------------------------------- mdalacu · 2012-02-03 13:47 Yes they are with the function corrected,  but if the program does not work. If i comment the line in blcksock.pas and leave the modification in ResolvePort then the program does not work. If i comment the line in blcksock.pas and revert the changes in ResolvePort then the program does work! My conclusion is that ResolvePort is called form something else and expect the strange output ResolvePort(8080) -> 36895 {i've put an writeln(port,result) inside original ResolvePort}. And when it get's called from blcksock.pas it must provide the normal values ResolvePort(443) -> 443. So we need to functions or something like this. Sorry for the long delay. What can i do? ---------------------------------------------------------------------- ludob · 2012-02-03 14:27 ResolvePort is called from SetVarSin in ssfpc where it indeed expects the port in network order. Can you change line 742 to Code: [Select]  Sin.sin_port := synsock.htons(Resolveport(port, family, SockProtocol, SockType)); and use it with the patched version of ResolvePort? ---------------------------------------------------------------------- mdalacu · 2012-02-03 14:54 Yes now it is working just fine!  Thank you very much and please send that patch to synapse for anyone benefit! ---------------------------------------------------------------------- ludob · 2012-02-03 15:37 Thanks for testing. I've sent the patch to the synapse mailing list. ---------------------------------------------------------------------- mdalacu · 2013-04-26 12:20 I see that the problem is not resolved in the latest svn snapshot (from february 2013) so i will attach the modified ssfpc.inc file for others to use. ---------------------------------------------------------------------- anna · 2015-06-30 23:47 I had error HTTP.Sock.LastError :10091 ; SSL/TLS support is not compiled! I use synapse from http://sourceforge.net/p/synalist/code/HEAD/tree/trunk/ And ssfpc.zip does not help. How to bring synapse to work? ---------------------------------------------------------------------- skalogryz · 2015-07-01 01:13 synapse trunk just works for me (no adjustments necessary). OSX 10.9.5 fpc 2.6.4 Are you using ssl_openssl unit in your project? the sample is attached. Btw, are you still interested in static linking openssl to a project? ---------------------------------------------------------------------- anna · 2015-07-01 04:00 Quote from: skalogryz on July 01, 2015, 03:13:34 amsynapse trunk just works for me (no adjustments necessary). OSX 10.9.5 fpc 2.6.4 Are you using ssl_openssl unit in your project? the sample is attached. Btw, are you still interested in static linking openssl to a project? Sorry, I missed that thread is fom Mac. I use Windows. But Lazarus must work everywhere, must not it? First I have error 103 When I edit code next , I get empty file: Code: [Select]program project1; {$mode delphi}{$H+} uses   {$IFDEF UNIX}cthreads,{$ENDIF} SysUtils,   Classes,  ssl_openssl , httpsend; var   st  : TStringList;   url : string;   res : string; begin   try     if ParamCount=0 then url:='https://www.google.com' else url:=ParamStr(1);     if ParamCount=1 then res:='' else res:=ParamStr(2);     st:=TStringList.Create;     try       HttpGetText(url, st);       {if res='' then write(st.Text)       else st.SaveToFile(res); }       st.SaveToFile('test.txt');     finally       st.Free;     end;   except     on e: exception do writeln('exception: ',e.message);   end; end. ---------------------------------------------------------------------- anna · 2015-07-01 04:23 httpsend.pas Code: [Select]function THTTPSend.InternalDoConnect(needssl: Boolean): Boolean; begin   Result := False;   FSock.CloseSocket;   FSock.Bind(FIPInterface, cAnyPort);   if FSock.LastError <> 0 then     Exit;   FSock.Connect(FTargetHost, FTargetPort);   if FSock.LastError <> 0 then     Exit;   if needssl then   begin     if (FSock.SSL.SNIHost='') then       FSock.SSL.SNIHost:=FTargetHost;     FSock.SSLDoConnect; // after that error 10091 appears     FSock.SSL.SNIHost:=''; //don't need it anymore and don't wan't to reuse it in next connection     if FSock.LastError <> 0 then       Exit;   end;   FAliveHost := FTargetHost;   FAlivePort := FTargetPort;   Result := True; end; ---------------------------------------------------------------------- Leledumbo · 2015-07-01 04:27 QuoteWhen I edit code next , I get empty file:I don't know how compatible ssl_openssl unit on Windows, but one thing for sure: https://google.com returns 302 Found (redirect response), not 200 OK with content, so you have to handle that redirect instead. ---------------------------------------------------------------------- anna · 2015-07-01 05:21 Quote from: Leledumbo on July 01, 2015, 06:27:30 amQuoteWhen I edit code next , I get empty file:I don't know how compatible ssl_openssl unit on Windows, but one thing for sure: https://google.com returns 302 Found (redirect response), not 200 OK with content, so you have to handle that redirect instead. I have traced code. No sending is made. I see in OllyDBG  WS2_32.connect call which return 0 (success). Then lazarus call FSock.SSLDoConnect; which return error.  . Application stops on this step. It contain hard-coded error 'SSL/TLS support is not compiled!' . I have no idea, how synapse works on you PC. It's miracle. ---------------------------------------------------------------------- anna · 2015-07-01 05:27 When I change function like next, I see non-encrypted http GET request on 443 port by TCP protocol. I think it is not correct, as browser uses TLS protocol over TCP protocol. Browser makes Client Hello. Synapse does not. I think that synapse has absolutely wrong  interpretation of standard. Code: [Select]function THTTPSend.InternalDoConnect(needssl: Boolean): Boolean; begin   Result := False;   FSock.CloseSocket;   FSock.Bind(FIPInterface, cAnyPort);   if FSock.LastError <> 0 then     Exit;   FSock.Connect(FTargetHost, FTargetPort);   if FSock.LastError <> 0 then     Exit;   if needssl then   begin     if (FSock.SSL.SNIHost='') then       FSock.SSL.SNIHost:=FTargetHost;     sleep(1);// marker for debugging in OllyDBG     FSock.SSLDoConnect;     sleep(2);     FSock.SSL.SNIHost:=''; //don't need it anymore and don't wan't to reuse it in next connection     {if FSock.LastError <> 0 then       Exit;    }   end;   FAliveHost := FTargetHost;   FAlivePort := FTargetPort;   Result := True; end; ---------------------------------------------------------------------- anna · 2015-07-01 05:48 Yep. It's my fault. SSLEAY32.dll and LIBEAY32.dll must be put in project folder. ---------------------------------------------------------------------- skalogryz · 2015-07-01 11:44 Well, last year you didn't like the fact that you've to have .dlls out there. That's why I asked if you are still interested in static linking it. ---------------------------------------------------------------------- skalogryz · 2015-07-01 11:46 Quote from: Leledumbo on July 01, 2015, 06:27:30 amI don't know how compatible ssl_openssl unit on Windows, but one thing for sure: https://google.com returns 302 Found (redirect response), not 200 OK with content, so you have to handle that redirect instead. It's compatible. The only trick is to have external openssl libraries available. They're available by default in OSX, but has to be installed for Windows. Btw, for google testing https://www.google.com should be used. ---------------------------------------------------------------------- anna · 2015-07-01 16:08 Quote from: skalogryz on July 01, 2015, 01:44:42 pmWell, last year you didn't like the fact that you've to have .dlls out there. That's why I asked if you are still interested in static linking it. Yes . Single file is 1000 % better then myriad of DLLs . Do you have a simple PAS-file which can be included to project? Give it to me, please. ---------------------------------------------------------------------- skalogryz · 2015-07-01 16:59 I don't have the file, but I certainly know it's possible, as shown in this thead ---------------------------------------------------------------------- Daniel Simoes · 2017-01-06 23:25 Quote from: mdalacu on April 26, 2013, 02:20:00 pmI see that the problem is not resolved in the latest svn snapshot (from february 2013) so i will attach the modified ssfpc.inc file for others to use. ludob and mdalacu, Thanks for your fix in ssfpc.inc, it works like a charm...