Any hints to speed up Internet Connection on Windows 10 or Up? lazarus / 34-Windows ---------------------------------------------------------------------- incendio · 2023-11-15 01:17 Hi guys, I have a database application using Firebird 3 that runs over internet. I am using IBX for handling database connection. Ran it on the same computer, when loading records, Windows XP (via Virtual Box),  loaded faster, at least 2x times than Windows 10 (real OS). Both OS for testing only, so don't have anti virus/firewall/defender installed. Is there any hints how to tuning Windows 10 to run faster? ---------------------------------------------------------------------- loaded · 2023-11-15 05:09 Hi Quote from: incendio on November 15, 2023, 02:17:54 amRan it on the same computer, when loading records, Windows XP (via Virtual Box),  loaded faster, at least 2x times than Windows 10 (real OS). There may be a speed difference between virtual systems and real systems. Also, can you try it on an XP running on a real system? ---------------------------------------------------------------------- incendio · 2023-11-15 05:45 Quote from: loaded on November 15, 2023, 06:09:02 amHi Quote from: incendio on November 15, 2023, 02:17:54 amRan it on the same computer, when loading records, Windows XP (via Virtual Box),  loaded faster, at least 2x times than Windows 10 (real OS). There may be a speed difference between virtual systems and real systems. Also, can you try it on an XP running on a real system? I have XP, but from different machine, so network interface & others won't be the same. I did try it on VM, but this time with Tiny Windows 10, XP still faster. ---------------------------------------------------------------------- rvk · 2023-11-15 08:16 Quote from: incendio on November 15, 2023, 02:17:54 amI have a database application using Firebird 3 that runs over internet. WAIT... What do you mean by internet? (Or did you just mean network?) Don't expose Firebird to the internet (via port forwarding). Unless you want the whole world to have access to your server. Firebird (and lots of other database engines) is slow over networks with high latency (like internet). ---------------------------------------------------------------------- incendio · 2023-11-15 08:31 Quote from: rvk on November 15, 2023, 09:16:54 amQuote from: incendio on November 15, 2023, 02:17:54 amI have a database application using Firebird 3 that runs over internet. WAIT... What do you mean by internet? (Or did you just mean network?) Don't expose Firebird to the internet (via port forwarding). Unless you want the whole world to have access to your server. Firebird (and lots of other database engines) is slow over networks with high latency (like internet). Didn't exposed Firebird directly to the internet, access it via tunnel, is it still danger? For speed, from my test, if server has 100Mb bandwidth and clients, at least 20Mb, speed is acceptable. ---------------------------------------------------------------------- marcov · 2023-11-15 08:46 VPN, tunnel and filter drivers can vary between XP and Vista+, as only later NDIS revisions support it. XP works iirc with kernel level drivers. ---------------------------------------------------------------------- rvk · 2023-11-15 08:49 Quote from: incendio on November 15, 2023, 09:31:49 amDidn't exposed Firebird directly to the internet, access it via tunnel, is it still danger?No, access over a (ssh)tunnel or VPN is no problem and is safe. But Firebird is slow over networks with high latency. Even if your bandwidth is high. You could have a 1Gb down/1Gb up connection and still experience slowness. You need to look at the ping/latency. On a local network this is less than 1ms. On the internet it's often above 10ms. And that's the problem. There are things to speed that up in the config of Firebird but you will always have some slowness. (Firebird 3 actually improved speed over low latency networks a lot over 2.5) There are some papers on how to set things (but I need to look for them). Just out of curiosity you say it works better on Windows XP. What is the ping speed to the server on Windows XP and what is it on Windows 10? Abd what kind of tunnel are you using? ---------------------------------------------------------------------- incendio · 2023-11-15 09:59 Quote from: rvk on November 15, 2023, 09:49:34 amQuote from: incendio on November 15, 2023, 09:31:49 amDidn't exposed Firebird directly to the internet, access it via tunnel, is it still danger?No, access over a (ssh)tunnel or VPN is no problem and is safe. But Firebird is slow over networks with high latency. Even if your bandwidth is high. You could have a 1Gb down/1Gb up connection and still experience slowness. You need to look at the ping/latency. On a local network this is less than 1ms. On the internet it's often above 10ms. And that's the problem. There are things to speed that up in the config of Firebird but you will always have some slowness. (Firebird 3 actually improved speed over low latency networks a lot over 2.5) There are some papers on how to set things (but I need to look for them). Just out of curiosity you say it works better on Windows XP. What is the ping speed to the server on Windows XP and what is it on Windows 10? Abd what kind of tunnel are you using? Ping on Windows XP and Windows 10 is the same (because it is on the same machine), about 10ms. On Windows XP, to open a form with about 35 of IBDataset + IBQuery, took about 3-4 second, On Windows 10, could took about 7-8 second. Input is fast, no lag at all. Loading a single table with about 500 records and 20 columns is instantly. By the way, I am using Firebird 3. Is this slow or fast for you? For me it is acceptable. ---------------------------------------------------------------------- marcov · 2023-11-15 10:10 The ping would be through the vpn/tunnel in this case, not outside of it. ---------------------------------------------------------------------- wp · 2023-11-15 10:12 You said above that you don't have any antivirus/firewall installed. But IIRC, the Microsoft Defender is active on Win 10 by default, without you doing anything. Please have a look at the system settings and check whether Defender is on or off. ---------------------------------------------------------------------- Dzandaa · 2023-11-15 10:12 Hi, You could try this: https://www.oo-software.com/en/shutup10 It's free and remove a lot of Microsoft Spywares. That slow down Win 10. ---------------------------------------------------------------------- incendio · 2023-11-15 10:29 Quote from: wp on November 15, 2023, 11:12:05 amYou said above that you don't have any antivirus/firewall installed. But IIRC, the Microsoft Defender is active on Win 10 by default, without you doing anything. Please have a look at the system settings and check whether Defender is on or off. Quote from: Dzandaa on November 15, 2023, 11:12:33 amHi, You could try this: https://www.oo-software.com/en/shutup10 It's free and remove a lot of Microsoft Spywares. That slow down Win 10. I have disable Windows Defender and others thing with that free software & others. ---------------------------------------------------------------------- incendio · 2023-11-15 10:32 Quote from: marcov on November 15, 2023, 11:10:17 amThe ping would be through the vpn/tunnel in this case, not outside of it. Don't know how to ping via tunnel, may be slower than outside it, but still took the same time because it is run on the same machine? ---------------------------------------------------------------------- rvk · 2023-11-15 10:45 Quote from: incendio on November 15, 2023, 11:32:35 amQuote from: marcov on November 15, 2023, 11:10:17 amThe ping would be through the vpn/tunnel in this case, not outside of it. Don't know how to ping via tunnel, may be slower than outside it, but still took the same time because it is run on the same machine? If you ping the server IP on the client side it should go through the tunnel. You shouldn't be able to ping that IP directly through the internet/. What tunnel are you using? SSH/Putty? Or something else. Is the server then reachable through 127.0.0.1 on the tunneled port or do you have another method? ---------------------------------------------------------------------- Josh · 2023-11-15 10:46 another thing that comes to mind is win 10+, ipv6 has higher priority than ipv4. I do not recommend disablibg ipv6 though, but you can re-prioritize so that ipv4 has highest priority, if you google you should find various articles how to do this using elevated cmd pompt and various netsh interface ipv6 set prefixpolicy xxxxxxx  commands you can easily reset with netsh interface ipv6 reset ---------------------------------------------------------------------- Dzandaa · 2023-11-15 10:50 Hi, You could also try to disable metered Connections     Settings->Network & Internet->Properties->Metered connection. or in a console Type: gpedit.msc Computer Configuration > Administrative Templates > Network > QoS Packet Scheduler double click: "Limit reservable bandwidth" and choose "Not Configured" B-> B-> ---------------------------------------------------------------------- incendio · 2023-11-15 10:57 Quote from: rvk on November 15, 2023, 11:45:18 amQuote from: incendio on November 15, 2023, 11:32:35 am Quote from: marcov on November 15, 2023, 11:10:17 amThe ping would be through the vpn/tunnel in this case, not outside of it. Don't know how to ping via tunnel, may be slower than outside it, but still took the same time because it is run on the same machine? If you ping the server IP on the client side it should go through the tunnel. You shouldn't be able to ping that IP directly through the internet/. What tunnel are you using? SSH/Putty? Or something else. Is the server then reachable through 127.0.0.1 on the tunneled port or do you have another method? I use zebedee, don't remember if localhost reachable, must see its config file. What's the importance of that ip? ---------------------------------------------------------------------- incendio · 2023-11-15 10:59 Quote from: Dzandaa on November 15, 2023, 11:50:34 amHi, You could also try to disable metered Connections     Settings->Network & Internet->Properties->Metered connection. or in a console Type: gpedit.msc Computer Configuration > Administrative Templates > Network > QoS Packet Scheduler double click: "Limit reservable bandwidth" and choose "Not Configured" B-> B-> Already did it long time ago. My Windows 10 has been tuned, on idle, cpu utilization it about 1% and process is about 120. ---------------------------------------------------------------------- rvk · 2023-11-15 11:22 Quote from: incendio on November 15, 2023, 11:57:12 amI use zebedee, don't remember if localhost reachable, must see its config file. What's the importance of that ip? I also have used zebedee in the past. You create a local ip in the config (127.0.0.3 for example) and connect to that IP as if you connect to the server. If you don't use such local IP for connection then there is something wrong in your setup. You can't ping over zebedee so there is no way to know the ping. But if you ping outside of zebedee to a random server (google.com) it gives you a rough estemate about latency over zebedee. Make sure your zebedee config is optimized. (setting compression and maxbufsize etc. to optimum) Next... don't use TIBDataset. Or at least make sure you know how much data it is pulling in. It could be that the entire table is transfered on opening. With TIBQuery this is much more efficient. Also... don't do a SELECT * FROM TABLE. Only retrieve the columns you really need. EVERY extra data will slow things down. With accessing a database over the internet you need to really think about these things. Especially when having 35 queries on your form. Why do you have that many? Can the number be reduced if you make use of one or less queries with LEFT JOINs etc.? Also... https://www.ibphoenix.com/files/articles/net_latency_and_firebird.html and: QuoteFirebird has a rather heavy network protocol (a lot of chit- chat), so it isn't really comfortable to work accross the Internet. You can use some tunneling software like ZeBeDee, SSH or SSL to encrypt and compress the data, but latency is the main problem as a lot of messages go back and forth (and transfering a lot of small messages over Internet is much slower than one big message). For this reason it's better to use some middleware or serve the content as web pages (if applicable to your requirements) or use some kind of SOAP. Please note that Firebird 2.1 has significant improvements to the network protocol (less communication for the same queries), so make sure you use at least 2.1 if you want work over Internet or other slow link.https://www.firebirdfaq.org/faq53/ ---------------------------------------------------------------------- marcov · 2023-11-15 11:36 Quote from: rvk on November 15, 2023, 11:45:18 amIf you ping the server IP on the client side it should go through the tunnel. Depends, some tunnel types are only for certain ports. Most notably putty ---------------------------------------------------------------------- rvk · 2023-11-15 11:43 Quote from: marcov on November 15, 2023, 12:36:41 pmQuote from: rvk on November 15, 2023, 11:45:18 amIf you ping the server IP on the client side it should go through the tunnel. Depends, some tunnel types are only for certain ports. Most notably putty Zebedee also just uses a single port for communications to which you can define 'targets' (and only those targets are forwarded). That's why I asked what software was used for the "tunnel" (tunnel is just a generic term). ---------------------------------------------------------------------- incendio · 2023-11-15 11:55 Quote from: rvk on November 15, 2023, 12:22:19 pmQuote from: incendio on November 15, 2023, 11:57:12 amI use zebedee, don't remember if localhost reachable, must see its config file. What's the importance of that ip? I also have used zebedee in the past. You create a local ip in the config (127.0.0.3 for example) and connect to that IP as if you connect to the server. If you don't use such local IP for connection then there is something wrong in your setup. You can't ping over zebedee so there is no way to know the ping. But if you ping outside of zebedee to a random server (google.com) it gives you a rough estemate about latency over zebedee. Make sure your zebedee config is optimized. (setting compression and maxbufsize etc. to optimum) Next... don't use TIBDataset. Or at least make sure you know how much data it is pulling in. It could be that the entire table is transfered on opening. With TIBQuery this is much more efficient. Also... don't do a SELECT * FROM TABLE. Only retrieve the columns you really need. EVERY extra data will slow things down. With accessing a database over the internet you need to really think about these things. Especially when having 35 queries on your form. Why do you have that many? Can the number be reduced if you make use of one or less queries with LEFT JOINs etc.? Also... https://www.ibphoenix.com/files/articles/net_latency_and_firebird.html and: QuoteFirebird has a rather heavy network protocol (a lot of chit- chat), so it isn't really comfortable to work accross the Internet. You can use some tunneling software like ZeBeDee, SSH or SSL to encrypt and compress the data, but latency is the main problem as a lot of messages go back and forth (and transfering a lot of small messages over Internet is much slower than one big message). For this reason it's better to use some middleware or serve the content as web pages (if applicable to your requirements) or use some kind of SOAP. Please note that Firebird 2.1 has significant improvements to the network protocol (less communication for the same queries), so make sure you use at least 2.1 if you want work over Internet or other slow link.https://www.firebirdfaq.org/faq53/ I use local ip in zebedee config file to connect to Firebird. TIBdataset use for insert / update /delete operation,  other than that, use TIBquery. Those 35 TIBdatasets & TIBqueries, points to different tables, so can't use join query. ---------------------------------------------------------------------- rvk · 2023-11-15 11:59 Quote from: incendio on November 15, 2023, 12:55:31 pmTIBdataset use for insert / update /delete operation,  other than that, use TIBquery.TIBQuery also has INSERT/UPDATE/DELETE via the UpdateObject property (to which you can add a TIBUpdateSQL). Quote from: incendio on November 15, 2023, 12:55:31 pmThose 35 TIBdatasets & TIBqueries, points to different tables, so can't use join query. Just make sure you don't pull in the entire table with that TIBDataset. But with 35 queries on one form you can certainly expect some slowness. (I'm not sure why your Windows XP would do it faster) ---------------------------------------------------------------------- incendio · 2023-11-16 02:50 Just tried to install my app in ReviOS on VM. ReviOS is a stripped Windows 10 with no bloatware, firewall, etc. The speed to load most heavy form is almost the same with Windows 10, with is still slower than Windows XP. So, I guest the caused to this slowness are buried deep down inside Windows 10. From this small test, in my opinion, for database application over internet and if it is possible, better to use Windows XP, the speed different with Windows 10 is noticeable. ---------------------------------------------------------------------- avra · 2023-11-16 20:53 Quote from: incendio on November 16, 2023, 03:50:23 amReviOS is a stripped Windows 10 with no bloatware, firewall, etc.You can debloat Win10 to your liking with https://github.com/ChrisTitusTech/winutil ---------------------------------------------------------------------- delphius · 2023-11-17 05:50 Quote from: incendio on November 16, 2023, 03:50:23 amFrom this small test, in my opinion, for database application over internet and if it is possible, better to use Windows XP, the speed different with Windows 10 is noticeable. 1. Is the speed problem only about transferring information from a remote database? Is it possible to test the data transfer rate over different protocols from the server? This will eliminate the role of the operating system 2. Are the version and type of database drivers used identical? 3. Are both systems using the same version of the client? ---------------------------------------------------------------------- incendio · 2023-11-17 06:20 Quote from: avra on November 16, 2023, 09:53:48 pmQuote from: incendio on November 16, 2023, 03:50:23 amReviOS is a stripped Windows 10 with no bloatware, firewall, etc.You can debloat Win10 to your liking with https://github.com/ChrisTitusTech/winutil Already done that, thank anyway for the tip. ---------------------------------------------------------------------- incendio · 2023-11-17 06:25 Quote from: delphius on November 17, 2023, 06:50:27 amQuote from: incendio on November 16, 2023, 03:50:23 amFrom this small test, in my opinion, for database application over internet and if it is possible, better to use Windows XP, the speed different with Windows 10 is noticeable. 1. Is the speed problem only about transferring information from a remote database? Is it possible to test the data transfer rate over different protocols from the server? This will eliminate the role of the operating system 2. Are the version and type of database drivers used identical? 3. Are both systems using the same version of the client? 1. No, it's not possible to test it with different protocols. 2. Yes 3. Yes Just tested the app on the fresh install latest Linux Mint Cinamon on a VM, speed is a little bit slower than Windows Xp, but still faster than Windows 10. ---------------------------------------------------------------------- avra · 2023-11-17 08:37 Do both Win10 and XP machines have the same DNS server? Are results the same when you replace remote server name with IP address? Is the same result on Win10 with 32bit and 64bit version of your application? ---------------------------------------------------------------------- incendio · 2023-11-17 08:56 Quote from: avra on November 17, 2023, 09:37:54 amDo both Win10 and XP machines have the same DNS server? Are results the same when you replace remote server name with IP address? Is the same result on Win10 with 32bit and 64bit version of your application? Yes, they have the same dns server, and I can't connect via ip address, because the connection handles via the tunnel. Is there Win 10 32? I don't have it. ---------------------------------------------------------------------- rvk · 2023-11-17 09:22 Quote from: incendio on November 17, 2023, 09:56:29 amIs there Win 10 32? I don't have it. Yes, there is. But I don't think it would matter much. Can you do a https://www.waveform.com/tools/bufferbloat on both platforms (XP and W10)? What are the results. (you can copy the links at "Share your results" below the results) At least then we have an idea of the latency and speed on both platforms. But I also still think (besides the fact XP and W10 react differently) that you are on the wrong path for solving this speed issue. Those 35 datasets... do you have SELECT * in all of them? And do you buffer all the records at one? ---------------------------------------------------------------------- incendio · 2023-11-17 11:59 Quote from: rvk on November 17, 2023, 10:22:31 amQuote from: incendio on November 17, 2023, 09:56:29 amIs there Win 10 32? I don't have it. Yes, there is. But I don't think it would matter much. Can you do a https://www.waveform.com/tools/bufferbloat on both platforms (XP and W10)? What are the results. (you can copy the links at "Share your results" below the results) At least then we have an idea of the latency and speed on both platforms. But I also still think (besides the fact XP and W10 react differently) that you are on the wrong path for solving this speed issue. Those 35 datasets... do you have SELECT * in all of them? And do you buffer all the records at one? Sorry for the incomplete information. Those datasets are not opened all at once. On Form.Create, 7 TIBDataset & 7 TIBquery opened. During the process, about 15 TIBquery, opened. The rest are TIBquery that are only opened when users press Find button. All datasets opened with filter, such as Select Colum1, Column2, ColumnN where id = ... If there are some inefficiency to load the data, this will impact both XP & 10. Main problem is XP is much faster than Windows 10 to load data , can Windows 10 be optimized to at least reach near performance as XP. After testing with ReviOS, I am not sure it can. ---------------------------------------------------------------------- avra · 2023-11-17 16:15 Quote from: incendio on November 17, 2023, 09:56:29 amIs there Win 10 32? I don't have it.No, I meant to test 32bit and 64bit version of the same application on Win10x64