My server lazarus / 38-Networking and Web Programming ---------------------------------------------------------------------- xinyiman · 2015-06-19 14:46 Hello guys. I created a server that handles connections based on a database firebird (Zeoslib). I also created a client to test that worked. It seems to work, but it's slow because the security that everything reaches its destination and that you get it encrypted is my priority. I would advice you to improve performance without compromising safety. I await advice. thanks ps: Leaning on synapse to send strings and files ---------------------------------------------------------------------- xinyiman · 2015-06-20 06:47 Mainly I would like to speed up the transmission of the file . Ideas ? ---------------------------------------------------------------------- jc99 · 2015-06-20 16:57 Quote from: xinyiman on June 20, 2015, 08:47:46 amUbuntu 12.04 Lazarus: 1.0.10 FPC: 2.6.2 Is there a reason why you use such an old lazarus, or did you just forget to update your signature ? ---------------------------------------------------------------------- xinyiman · 2015-06-21 14:10 I forgot to update my signature. Sorry. Now use the 1.2.6 ---------------------------------------------------------------------- xinyiman · 2015-06-22 20:42 Any suggestions? ---------------------------------------------------------------------- fpcoder · 2015-06-22 21:10 It is more likely that someone can help if you point out some parts of your code that could be a problem. ---------------------------------------------------------------------- xinyiman · 2015-06-23 07:54 if you analyze the code you'll see that when I send a string aspect that I return the same string to make sure that the transmission is synchronous. I would like to figure out how to speed up this part if possible. Plus I want to understand how to send a file more quickly. ---------------------------------------------------------------------- JD · 2015-06-23 11:37 I haven't looked at your project, I must confess (I have time constraints  ) BUT if encryption is the priority and you only want to speed up the transmission, a) you can compress the data before sending it & then decompress it after it is received. Smaller packets are transferred but the compression/decompression has its own time factor. b) send data in smaller chunks (a few small queries perform faster than one BIG query) c) use threads to send the data so that your GUI doesn't freeze while waiting for data ---------------------------------------------------------------------- xinyiman · 2015-06-23 13:10 Thanks, I think I will consider the possibility of sending files in multithreading (stupidly had not thought of). Now I do the test. And if someone else had any suggestions to avoid the redundancy of my communications between servers and clients would make me a favor. ---------------------------------------------------------------------- fpcoder · 2015-06-23 14:57 No need to send the same data twice. Let the server calculate a hash sum for every file. Send the hash to the client together with the file. Now the client can verify all files using the hash sum. http://delphi.about.com/od/objectpascalide/a/delphi-md5-hash.htm ---------------------------------------------------------------------- xinyiman · 2015-06-23 16:35 Then I isolated the problem so as to make life easier in the unwelcome The problem I think I know is that when the server to receive the file still waiting 20 seconds even if the file has already arrived in full. There is a way to speed up this transition by retaining relatively high timeout? As I Splitto individual files into blocks of 10 Mb. Suggestions? Client code Code: [Select]                                              app:=TFileStream.Create(file2, fmopenRead);                                               Sock.SendStream(app);                                               //libero l'area di memoria che ho occupato fino ad adesso                                               app.Free; writeln('Step 2 : ' + DateTimeToStr(Now));                                               //attendo la risposta da parte del server per sapere se il file che voglio inviare è arrivato correttamente                                               DatiLetti:=Self.GetString(); writeln('Step 3 : ' + DateTimeToStr(Now)); Server code: Code: [Select]                    app:=TFileStream.Create(CartellaDestinazione + NomeFileCompressoECifrato,fmCreate);                     repeat                           //ricevo il file nello stream da decomprimere                           sock.RecvStream(app,20000);                     until Sock.LastError <> 0;                     //libero l'area di memoria che ho occupato fino ad adesso                     app.Free;                     //dico al client che ho correttamente ricevuto il file                     Self.MySendString(AperturaComandi + 'INVIO FILE: OK' + ChiusuraComandi); ---------------------------------------------------------------------- rvk · 2015-06-23 16:53 Ah, now I see what you mean by slow. The server is done sending and your client is still waiting on input, and does so for 20 seconds. Not to long ago I answered a question here that the SendStream- function send 4 extra bytes in front of the stream. But my sugestion was to use the SendStreamRaw-function because that doesn't send the filesize in front of the stream. Maybe if you send the file-size before the stream, then you only read that exact number of bytes in the client and you don't have to wait after the file transfer. Or just use the SendStream function and see if that works better. O, wait, you are using SendStream/RecvStream. In that case it shouldn't wait the 20 seconds. ---------------------------------------------------------------------- xinyiman · 2015-06-23 17:04 I did not understand, make me an example? thanks ---------------------------------------------------------------------- rvk · 2015-06-23 17:05 Code: [Select]                    repeat                           //ricevo il file nello stream da decomprimere                           sock.RecvStream(app,20000);                     until Sock.LastError <> 0; Uh, until Sock.LastError <> 0. Shouldn't that be until Sock.LastError = 0 The RecvStream receives the full stream. You only want to stay in that loop if there is an error. Actually. You can remove the whole loop because you don't have an option on the server side to resend the file on error. So this is sufficient: Code: [Select]                          sock.RecvStream(app,20000); It will receive the complete stream and there is no need for the loop. ---------------------------------------------------------------------- xinyiman · 2015-06-24 06:33 Quote from: rvk on June 23, 2015, 06:53:12 pm Maybe if you send the file-size before the stream, then you only read that exact number of bytes in the client and you don't have to wait after the file transfer. How? ---------------------------------------------------------------------- rvk · 2015-06-24 06:38 Quote from: xinyiman on June 24, 2015, 08:33:02 amHow? See my last post. Sendstream already does this. If you just remove the loop and only leave the recvstream it will work. You don't need to put the recvstream in a loop because it already receives the complete stream. Putting it in a loop will only slow it down (for that 20 seconds). So this is all you need: Code: [Select]sock.RecvStream(app,20000); ---------------------------------------------------------------------- JD · 2015-06-24 08:34 Quote from: xinyiman on June 24, 2015, 08:33:02 amQuote from: rvk on June 23, 2015, 06:53:12 pm Maybe if you send the file-size before the stream, then you only read that exact number of bytes in the client and you don't have to wait after the file transfer. How? @rvk suggestion is spot on. Reading data stops when a pre-defined number of bytes have been received. I have an example in Indy 10 but you are using Synapse; however the principle is the same. ---------------------------------------------------------------------- xinyiman · 2015-06-24 09:47 Quote from: rvk on June 24, 2015, 08:38:51 amQuote from: xinyiman on June 24, 2015, 08:33:02 amHow? See my last post. Sendstream already does this. If you just remove the loop and only leave the recvstream it will work. You don't need to put the recvstream in a loop because it already receives the complete stream. Putting it in a loop will only slow it down (for that 20 seconds). So this is all you need: Code: [Select]sock.RecvStream(app,20000);Forgive me, I knew that you were to do with the RecvStreamRAW ---------------------------------------------------------------------- rvk · 2015-06-24 10:00 Quote from: xinyiman on June 24, 2015, 11:47:03 amForgive me, I knew that you were to do with the RecvStreamRAW The suggestion I gave on stackoverflow for SendStreamRAW was for somebody who used the command "nc" on the client side to receive a file. In that case the length should not be send in advance. When using RecvStream it expects the length of the stream before the actual data and SendStream does just that, it sends the length before the data (and that's what you want). So all you need is SendStream at the server and RecvStream at the client (wihout the loop) and they first communicate the length of the stream and afterwards all the data is send (with that single function). When using SendStreamRaw the length is not send and you could use RecvStreamRaw to receive the stream. But in that case you would have to settle for the timeout because the receiving end doesn't know how much data is to be received. There is also RecvStreamSize is you do know the size in advance but in your case SendStream/RecvStream makes more sense.