Force TLS1.3 with fphttpclient lazarus / 38-Networking and Web Programming ---------------------------------------------------------------------- PizzaProgram · 2022-04-17 13:02 Installed binaries downloaded  from here: https://slproweb.com/products/Win32OpenSSL.html I've wrote a little test exe using fphttpclient and it works great from both Win7 + Win10!  Found a free test-server, that shows detailed infos about my JSON I'm sending, and support TSLv1.3: https://webhook.site By capturing the trafic with Wireshark I can see: the connection was made with TLS1.2 only! One more thing to worried about:  fphttpclient is basically using sslbase.pp file, where I've found this at Line 11: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i   TSSLType = (stAny,stSSLv2,stSSLv3,stTLSv1,stTLSv1_1,stTLSv1_2); used by: TSSLSocketHandler.SSLType So TLS 1.3is not even listed there! How can I check / force TSLv1.3 ? ---------------------------------------------------------------------- Bi0T1N · 2022-04-17 13:41 Quote from: PizzaProgram on April 17, 2022, 03:02:28 pmThe only thing I can not see / test:  whether the connection was made with TLS1.3 or less ? I think the simplest solution - thus Python (you can also use e.g. fphttpserver but I haven't found any code example in a hurry) - is to setup a local https server that supports TLS 1.3 and you connect to it with your Pascal program. Quote from: PizzaProgram on April 17, 2022, 03:02:28 pmOne more thing to worried about:  fphttpclient is basically using sslbase.pp file, where I've found this at Line 11: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i   TSSLType = (stAny,stSSLv2,stSSLv3,stTLSv1,stTLSv1_1,stTLSv1_2); used by: TSSLSocketHandler.SSLType So TLS 1.3is not even listed there! How can I check / force TSLv1.3 ? OpenSSL no longer supports specifying a specific protocol version, introduced with TLSv1.3. Thus this functionality is not provided anymore and the old functions are deprecated. Nowadays it negotiates the highest version mutually supported by the client and the server during the initial handshake and hence there is only the universal TLS_method(). ---------------------------------------------------------------------- PizzaProgram · 2022-04-17 18:48 I've dived into the code more, and realized:  - Eventually this component is based on (calling) openssl.pas of Ararat Synapse, last changed in 2010-08-24. It is searching for: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i     DLLSSLName: string = 'ssleay32.dll';  DLLSSLName2: string = 'libssl32.dll';   files based on version 1.1, so there is no way it will work with libssl-3.dll ---------------------------------------------------------------------- paweld · 2022-04-17 19:05 Synapse in trunk version support openssl 1.1 and 3: https://sourceforg e.net/p/synalist/code/HEAD/tree/trunk/ssl_openssl_lib.pas ---------------------------------------------------------------------- PizzaProgram · 2022-04-18 12:08 You are right! Thanks for the link.  Those trunc .pas files are really looking for files like: 'libcrypto-3.dll', so it seems to be up to date. I'm pretty much shocked about:  - if this code is the "built-in-base" for all https communication, why did not Lazarus developers upgraded this code since the last 12 years?  There is a laz_synapse.lpk too, so no "special conversion" of the package is needed. I'll try to download and test it now... _______________________________________ Anyway, I've made some more test yesterday and realized: 1. Even if I install the latest 3.0.2 binaries,   default dir: C:\Program Files (x86)\OpenSSL-Win32\bin\ 2. Allow to install DLL into "system", 3. my test-APP is still using some old 1.0.2.10 SSL DLLs found somewhere in the: C:\Windows\SysWOW64\ Maybe this is a problem of the binary installer, not overwriting those old files from 2016, but still very strange. ---------------------------------------------------------------------- paweld · 2022-04-18 12:26 This is because searching libraries in Windows follows a specific order: https://docs.microsoft.com/en-us/windows/win32/dlls/dynamic- link-library-search-order If you want to specify where you want the libraries to be taken from you have to set the path yourself. I usually put the dll libraries in the program directory and so far I haven't had any problems with that. ---------------------------------------------------------------------- Bi0T1N · 2022-04-18 12:54 Quote from: PizzaProgram on April 17, 2022, 08:48:17 pmI've dived into the code more, and realized:  - Eventually this component is based on (calling) openssl.pas of Ararat Synapse, last changed in 2010-08-24. I doubt that - just check the history of the file. There is no need to track changes in a file. Quote from: PizzaProgram on April 17, 2022, 08:48:17 pmIt is searching for: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i     DLLSSLName: string = 'ssleay32.dll';  DLLSSLName2: string = 'libssl32.dll';   files based on version 1.1, so there is no way it will work with libssl-3.dll I already created an issue for adding support for OpenSSL 3.0. For now it should work if you just rename the DLL files as if they were from OpenSSL 1.1.1. ---------------------------------------------------------------------- PizzaProgram · 2022-04-26 09:20 Quote from: paweld on April 18, 2022, 02:26:09 pmThis is because searching libraries in Windows follows a specific order: https://docs.microsoft.com/en-us/windows/win32/dlls/dynamic-link- library-search-order If you want to specify where you want the libraries to be taken from you have to set the path yourself. I usually put the dll libraries in the program directory and so far I haven't had any problems with that. That is exactly what we, programmers should always AVOID! It should be "a default thing" that during initialization we can set our own program, where it should load those DLLs from by default. Something like: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i InitOpenSSL(default_path: string = '');If the parameter is empty, it should search for it the default (Microsoft) way. Why?  Because normally there is not just 1 EXE (+ini) file in the main/root directory of a program. And each of those EXE files may use different versions. All those EXE versions can download their own version of DLLs, or auto-update until a point, when it becomes incompatible. (This can be controlled with a simple server-side ini/conf, that a program can auto-download too.) And of course your own EXE can create it's own sub-directory to place that DLL to the right place, where it finds it later. Like: E:\MyAPP\OpenSSL\v202001\...DLL E:\MyAPP\OpenSSL\v202204\...-3.DLL This is the only way you can use different versions of your program at the same PC + same main-directory. OFF: Think about it for a moment: If ANYBODY (usually an other IT guy,) is copying/installing an other version of the "same" DLL into the system dir, (or the main-dir of your program,) they will interfere, and some of the programs will crash. Than the buyer will say: your program is a piece of *****, and you have to find out remotely what happened where and why, to prove your innocence. I've learned in the last 30 years of programming + iT stuff + user handling, that very high-level programmers can not really think "out of the box" how that module will work in "big scale", when it will be integrated into thousands of environments. I don't blame them, but it would be great if everyone would start with a few hundred "user side experiences" + "PC installations and maintaining" for the first 1-2 years, and than go deeper and deeper with coding later. But that's impossible, because the best hardcore programmers can not really "feel for" other user's problems well. ---------------------------------------------------------------------- Warfley · 2022-04-26 12:03 Quote from: PizzaProgram on April 26, 2022, 11:20:45 amAll those EXE versions can download their own version of DLLs, or auto-update until a point, when it becomes incompatible. (This can be controlled with a simple server-side ini/conf, that a program can auto-download too.) And of course your own EXE can create it's own sub-directory to place that DLL to the right place, where it finds it later. Like: E:\MyAPP\OpenSSL\v202001\...DLL E:\MyAPP\OpenSSL\v202204\...-3.DLL This might be useful for some unimportant libraries, but not for security critical libraries like OpenSSL. You should always use the system version of this library, because it is always on the most up to date version. If you provide the library through your own servers, you always introduce delay, as when the new version is published, you need to get it onto your server, which probably takes longer than microsoft takes to put it into their update system (especially as modern windows Versions will force security relevant updates so you will never be on an unsecure version). But more importantly, you should never ship an old version of OpenSSL conciously. So your example where you have an old OpenSSL library, this should never happen. If your software doesn't run with an up to date version of OpenSSL, then you should update your software, not downgrade the OpenSSL. It should be noted that this is EU law as per directive 2019/770, where article 8 states: QuoteArticle 8 Objective requirements for conformity 1.   In addition to complying with any subjective requirement for conformity, the digital content or digital service shall: [...] (b) be of the quantity and possess the qualities and performance features, including in relation to functionality, compatibility, accessibility, continuity and security, normal for digital content or digital services of the same type and which the consumer may reasonably expect, given the nature of the digital content or digital service and taking into account any public statement made by or on behalf of the trader, or other persons in previous links of the chain of transactions, particularly in advertising or on labelling unless the trader shows that: (i) the trader was not, and could not reasonably have been, aware of the public statement in question; (ii) by the time of conclusion of the contract, the public statement had been corrected in the same way as, or in a way comparable to how, it had been made; or (iii) the decision to acquire the digital content or digital service could not have been influenced by the public statement; [...] 2.   The trader shall ensure that the consumer is informed of and supplied with updates, including security updates, that are necessary to keep the digital content or digital service in conformity, for the period of time: (a) during which the digital content or digital service is to be supplied under the contract, where the contract provides for a continuous supply over a period of time; or (b) that the consumer may reasonably expect, given the type and purpose of the digital content or digital service and taking into account the circumstances and nature of the contract, where the contract provides for a single act of supply or a series of individual acts of supply.So you must keep your software up to date with respect to security, which of course means you cannot rely on outdated SSL libraries ---------------------------------------------------------------------- BeniBela · 2022-04-26 12:45 OpenSSL 1.1.0 has the  SSL_CTX_set_min_proto_version function to prevent it from using old versions (edit: the documentation says it has that function. I cannot find it on my system  ) Older OpenSSL has SSL_CTX_set_options that could be called with options like SSL_OP_NO_TLSv1_1, SSL_OP_NO_TLSv1_2, ... Now they do not seem to be exposed by FreePascal, so you have to call them directly from the ssl library. ---------------------------------------------------------------------- PascalDragon · 2022-04-26 12:57 Quote from: PizzaProgram on April 26, 2022, 11:20:45 amIt should be "a default thing" that during initialization we can set our own program, where it should load those DLLs from by default. Something like: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i InitOpenSSL(default_path: string = '');If the parameter is empty, it should search for it the default (Microsoft) way. Nothing stops your application from doing just that by calling InitSSLInterface with suitable parameters. And if you supply a path instead of only a library name Windows' LoadLibrary won't do its search. Quote from: Warfley on April 26, 2022, 02:03:22 pmQuote from: PizzaProgram on April 26, 2022, 11:20:45 amAll those EXE versions can download their own version of DLLs, or auto-update until a point, when it becomes incompatible. (This can be controlled with a simple server-side ini/conf, that a program can auto-download too.) And of course your own EXE can create it's own sub-directory to place that DLL to the right place, where it finds it later. Like: E:\MyAPP\OpenSSL\v202001\...DLL E:\MyAPP\OpenSSL\v202204\...-3.DLL This might be useful for some unimportant libraries, but not for security critical libraries like OpenSSL. You should always use the system version of this library, because it is always on the most up to date version. If you provide the library through your own servers, you always introduce delay, as when the new version is published, you need to get it onto your server, which probably takes longer than microsoft takes to put it into their update system (especially as modern windows Versions will force security relevant updates so you will never be on an unsecure version). While Microsoft nowadays indirectly provides LibreSSL by providing OpenSSH this is not updated automatically. For example my Windows 10 system is up to date with updates, but the installed version of LibreSSL is 3.0.2 while the current one is 3.4.3. One needs to update this manually, see also here. Though this is not relevant anyway, because Microsoft only provides libcrypto, but not libssl (which is probably statically linked to the OpenSSH executables) thus it isn't useable with FPC's OpenSSL unit. ---------------------------------------------------------------------- Warfley · 2022-04-26 13:10 Quote from: PascalDragon on April 26, 2022, 02:57:35 pmWhile Microsoft nowadays indirectly provides LibreSSL by providing OpenSSH this is not updated automatically. For example my Windows 10 system is up to date with updates, but the installed version of LibreSSL is 3.0.2 while the current one is 3.4.3. One needs to update this manually, see also here. Though this is not relevant anyway, because Microsoft only provides libcrypto, but not libssl (which is probably statically linked to the OpenSSH executables) thus it isn't useable with FPC's OpenSSL unit. Interesting, I haven't done any network development on windows for quite a while (at least the kind requireing openssl), and I thought I've read that windows now provides it through the system update. One more reason to not use windows (even though recently I have to use it more than I like to) But alas, my main point was about the idea of having multiple versions for different applications, which should defenetly not be done ---------------------------------------------------------------------- PizzaProgram · 2022-04-26 13:51 Yes, I agree with ALL of you. In theory. But in reality when I tried to put newer Indy-openSSL DLLs next to an older exe >> it crashed (under Delphi7). And even if we install latest OpenSSL3, if the EXE is searching for an old DLL >> we won't benefit from that. Right? That's why I've opened this topic, because I've realized Lazarus was loading the old DLLs. (And if I'm right, even with the new DLLs, it's still calling some deprecated functions, not the new ones available...  ) That's why I'm trying to make a "Special DLL" with Lazarus, and load it into Delphi7 runtime. Whenever I can and needed, I will update this "Spec-Laz-DLL" only, put it in the right directory, so I don't have to change anything in the "big program", but still keep the SSL communication secure. We never know what the future brings. Maybe they discover a new possibility to hack even SSL3.x and they deprecate everything in 2 month and introduce OpenSSL4.0, and the OpenSSL programmers change the function names and DLL names again. ---------------------------------------------------------------------- Warfley · 2022-04-26 15:52 D7 is not really the newest piece of technology (the release date of D7 is closer to the year 1989 when the WWW was created than it is to today), so that libraries for it might not be comaptible with modern OpenSSL versions is not so surprising, even though OpenSSL is a really stable library. The question then is if it wouldn't be more helpful for you to try and find another D7 library that is compatible with newer OpenSSL versions. But thats a different topic. About your problem, as long as FPCs OpenSSL unit is not OpenSSL 3 compatible, you need to stick to version 1.x. The good news, it is still supported and updated, so it provides the most up to date fixes. Even the deprecated functions are completely fine to use. It also supports TLS1.3 (even though I don't know if the FPC implementation supports this). So my recommendation would be, until OpenSSL 3 is supported, just stick to the newest build of 1.x, it works and it is secure. ---------------------------------------------------------------------- 440bx · 2022-04-26 16:42 Quote from: Warfley on April 26, 2022, 05:52:08 pmD7 is not really the newest piece of technology (the release date of D7 is closer to the year 1989 Delphi 7's release date is August 2002, a "short" 13 years after 1989. I purchased Delphi 5 in 1999-2000 (not sure which one of the two)  1989 was the year of Turbo Pascal 5.5 All that said, in 2022, it is fair to say that D7 is not the "newest piece of technology" but, it's a little more advanced than Turbo Pascal 5.5. ---------------------------------------------------------------------- trev · 2022-04-26 23:36 Quote from: Warfley on April 26, 2022, 02:03:22 pmThis might be useful for some unimportant libraries, but not for security critical libraries like OpenSSL. You should always use the system version of this library, because it is always on the most up to date version. This is not the case on macOS where the system provided versions are, in some cases, ancient (eg OpenSSL 0.98.7). Apple specifically says developers should not use the system SSL libraries but should provide their own versions in the application bundle. I believe the reasons for this are currency (ie security) and incompatible ABI changes between versions. As at March 2022, the latest OpenSSL stable version is the 3.0 series which is supported until 7th September 2026. This is also a Long Term Support (LTS) version. The previous LTS version OpenSSL 1.1.1 is on life support until 11th September 2023 (at which point all support ceases, so no bug fixes for security problems) as OpenSSL moves to version 3 (now at 3.02) which has even more significant ABI changes. All older OpenSSL versions (including 1.1.0, 1.0.2, 1.0.0, 0.9.8 and 0.9.7) are now out of support, contain multiple security vulnerabilities and should not be used if you value security and your reputation. ---------------------------------------------------------------------- PascalDragon · 2022-04-27 06:57 Quote from: 440bx on April 26, 2022, 06:42:35 pmQuote from: Warfley on April 26, 2022, 05:52:08 pmD7 is not really the newest piece of technology (the release date of D7 is closer to the year 1989 Delphi 7's release date is August 2002, a "short" 13 years after 1989. Which still makes Delphi 7 closer to 1989 than to 2022 ---------------------------------------------------------------------- 440bx · 2022-04-27 07:18 Quote from: PascalDragon on April 27, 2022, 08:57:45 amWhich still makes Delphi 7 closer to 1989 than to 2022 That is definitely true but, as I have no doubt you know, if he's going to mention 1989 then he should be comparing to Turbo Pascal 5.5 not D7 ---------------------------------------------------------------------- Warfley · 2022-04-27 08:16 Quote from: 440bx on April 27, 2022, 09:18:04 amThat is definitely true but, as I have no doubt you know, if he's going to mention 1989 then he should be comparing to Turbo Pascal 5.5 not D7 I mentioned D7 and 1989 because the thread creator is trying to access the web (created in 1989) with D7. This is not about old technology ---------------------------------------------------------------------- PizzaProgram · 2022-04-27 15:24 Quote from: Warfley on April 26, 2022, 05:52:08 pmAbout your problem, as long as FPCs OpenSSL unit is not OpenSSL 3 compatible, you need to stick to version 1.x. The good news, it is still supported and updated, so it provides the most up to date fixes. Even the deprecated functions are completely fine to use. It also supports TLS1.3 (even though I don't know if the FPC implementation supports this). I've tested, and it did NOT support TLS1.3 . That's why I've opened this topic. Quote from: Warfley on April 26, 2022, 05:52:08 pmSo my recommendation would be, until OpenSSL 3 is supported, just stick to the newest build of 1.x, it works and it is secure. Not for long! As @trev wrote it already. I'm trying to think 5-10 years ahead, so if a customer is buying a program from me, he won't be forced to upgrade or drop it in 1-2 years. That would not be fair. ------- @All:  Yes, Delphi7 has other possibilities: using ICS or maybe Ararat Synapse. http://wiki.overbyte.eu/wiki/index.php/ICS_Download Also it takes only 3.6 sec to Full-Build 1M rows of code with it. But I would like to do new enhancements with Lazarus, because I like the IDE more and this code can be upgraded easier.