Synapse bidirectional UDP lazarus / 38-Networking and Web Programming ---------------------------------------------------------------------- SymbolicFrank · 2017-08-10 14:44 I have a machine that communicates through UDP. So, I send it an UDP message that says: "Start connection", and from that moment on it will start spamming that port with lots of data. But AFAIK, I can not send data over a socket while I'm listening to it. Should I listen for just a few milliseconds, and then peek if there is a message to send in the queue, etc? That seems very inefficient. Or do I have to do everything asynchronously with callbacks, all in the same thread? That's a lot of extra code. What is the easiest way? I use Synapse ATM, but if there is something better for this, I'll use it. ---------------------------------------------------------------------- Blestan · 2017-08-10 16:24 you cannont send and receive in the same program... need to build a.cient and a server ---------------------------------------------------------------------- bytebites · 2017-08-10 16:52 You can send and receive in the same program, but you need listener- thread and non-blocking socket. ---------------------------------------------------------------------- Blestan · 2017-08-10 17:00 ok but its easy with client/server... if you want do send and receive iteasy to use pipes or ipc ---------------------------------------------------------------------- SymbolicFrank · 2017-08-10 20:24 I tried client/server, but I cannot bind them both to the same socket. ---------------------------------------------------------------------- Remy Lebeau · 2017-08-10 20:34 A UDP socket is bidirectional, just like a TCP socket is.  A single UDP socket can both send and read data, even at the same time, with multiple peers.  You don't "listen" in UDP like you do in TCP.  Simply create a UDP socket and bind it to the desired local IP/port, then send out the start message and enter a reading loop using the same UDP socket.  Each successful read will tell you which peer the data came from.  You don't have to use a non-blocking socket, either.  You could run the loop on a blocking socket using a separate thread/process, for instance, if you don't want to block the thread that sends out the start message. ---------------------------------------------------------------------- SymbolicFrank · 2017-08-10 20:49 Yes, that's what I did. But that prevents me from sending messages, like "Close connection", while doing that. Ok, in this case it's probably not a showstopper, as it spams everything that is happening on a CAN bus, so I get lots of messages each second. But if that traffic would stop for whatever reason (UDP = best effort, no guarantees), I would be unable to send a message until the listening times out. Listen for 10 ms, peek messages to send for 10 ms, repeat, etc. Just when I grew accustomed to simple and straightforward blocking threads. And now I have to change the fancy thread-safe queue to support peeking as well as dequeuing. But, can do. ---------------------------------------------------------------------- Remy Lebeau · 2017-08-10 21:12 Quote from: SymbolicFrank on August 10, 2017, 10:49:44 pmYes, that's what I did. But that prevents me from sending messages, like "Close connection", while doing that. Ok, in this case it's probably not a showstopper, as it spams everything that is happening on a CAN bus, so I get lots of messages each second. But if that traffic would stop for whatever reason (UDP = best effort, no guarantees), I would be unable to send a message until the listening times out. The only way that can happen if you are using a blocking socket and are trying to send out from the same thread that is waiting for data to come in.   So, either use a non-blocking socket, or do the reading and sending in separate threads.  It works fine either way. ---------------------------------------------------------------------- SymbolicFrank · 2017-08-10 21:16 Yes, that was my question. ---------------------------------------------------------------------- SymbolicFrank · 2017-08-14 11:31 I cannot get it to work. Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i procedure TSixTLNode.Execute;var  msg: TMessage;begin  MyUDPSocket := TUDPBlockSocket.Create;   try    MyUDPSocket.CreateSocket;    MyUDPSocket.Bind('127.0.0.1', '0');    if MyUDPSocket.LastError <> 0 then Exit;    MyUDPPort := MyUDPSocket.GetLocalSinPort;    MyUDPSocket.Connect(SendToHost, SendToPort);    if MyUDPSocket.LastError <> 0 then Exit;     while not Terminated do    begin      if MyUDPMessages.Dequeue(msg) then      begin        MyUDPSocket.SendString(msg.Data);         // TUDPBlockSocket.SendBufferTo reports 20 bytes send, which is correct      if MyUDPSocket.LastError <> 0 then        begin          MyMessages.Enqueue(TMessage.Create(0, '6TL', TCommand.Error,          IntToStr(MyUDPSocket.LastError)));        end;      end;       if MyUDPSocket.CanRead(10) then      begin        MyMessages.Enqueue(TMessage.Create(0, '6TL', TCommand.Data,          MyUDPSocket.RecvPacket(100)));      end;       Sleep(10);    end;  finally    MyUDPSocket.Free;  end;end; But, nothing gets send. When I look with Wireshark, it's all silent. And yes, I did turn off the firewall. Strange, I make lots of these kinds of things for TCP (often for Linux), which work well. But UDP is elusive. ---------------------------------------------------------------------- rvk · 2017-08-14 11:54 Two small remarks on your code. First... I've never needed to use .CreateSocket. I thought this was done automtically. Second... Shouldn't you bind the socket to 0.0.0.0 instead of 127.0.0.1? So... this will work for me: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i     // MyUDPSocket.CreateSocket;    MyUDPSocket.Bind('0.0.0.0', '0'); ---------------------------------------------------------------------- Thaddy · 2017-08-14 12:05 Quote from: rvk on August 14, 2017, 01:54:25 pm I thought this was done automtically. Makes my day ---------------------------------------------------------------------- SymbolicFrank · 2017-08-14 12:09 Thanks rvk, that does work, or at least, the first message is sent. I did it like that, because otherwise with TCP it will not bind to a socket. That is also in the comment for the Bind function. And yes, I added the CreateSocket line to show it is done. I left it out at first (because the comments for Bind and Connect say that is done automatically. And now GetLocalSinPort returns 0 as well, so I don't know which port is used. ---------------------------------------------------------------------- serbod · 2017-08-14 14:16 You don't need do Connect() after Bind() for UDP, but need to set remote address before sending. Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i   FBlockSocket.SetRemoteSin(APacket.PeerIP, IntToStr(APacket.PeerPort));  FBlockSocket.SendString(APacket.Data);  FLastError := FBlockSocket.LastError; ---------------------------------------------------------------------- Remy Lebeau · 2017-08-14 16:46 Quote from: serbod on August 14, 2017, 04:16:01 pmYou don't need do Connect() after Bind() for UDP Though, you CAN, if you know you are communicating with only one specific peer.  Connect() allows all outgoing packets to go to that peer without having to specify it each time, and it allows the socket to ignore all inbound packets from other peers.  Connect() establishes a 1-to-1 relationship between peers, in both TCP and UDP. Quote from: serbod on August 14, 2017, 04:16:01 pmbut need to set remote address before sending. Yes, if you want to be able to send to, and/or receive from, multiple peers, then don't use Connect(). ---------------------------------------------------------------------- rvk · 2017-08-14 20:25 Quote from: SymbolicFrank on August 14, 2017, 02:09:36 pmAnd now GetLocalSinPort returns 0 as well, so I don't know which port is used. I'm not sure why you would need the local port. You could listen on a different uniform port. But the reason GetLocalSinPort is 0 is because with binding to 0.0.0.0, you bind to all ip-addresses the computer has. Normally this is localhost (127.0.0.1) AND your local ip address on your LAN (which might be more if you have multiple network cards and/or WiFi). So two or more IP-addresses. But each IP-address can have a different local- port so GetLocalSinPort isn't really useful because it can be multiple ports. If you really want GetLocalSinPort to have a valid number you need to bind to your local LAN ip-address (so not 0.0.0.0 and not 127.0.0.1 but 192.168.1.something). ---------------------------------------------------------------------- Remy Lebeau · 2017-08-14 22:01 Quote from: rvk on August 14, 2017, 10:25:45 pmBut the reason GetLocalSinPort is 0 is because with binding to 0.0.0.0, you bind to all ip-addresses the computer has. More accurately, to all local IPv4 addresses. Quote from: rvk on August 14, 2017, 10:25:45 pmBut each IP-address can have a different local-port Only if you explicitly bind different sockets to each IP address individually.  When binding a single socket, regardless of whether you bind it to "0.0.0.0" (or "::0" in IPv6), or "127.0.0.1", or any other specific IP, all IP addresses bound to the socket will be bound to a single port number that they all share (if multiple IPs are bound). Quote from: rvk on August 14, 2017, 10:25:45 pmIf you really want GetLocalSinPort to have a valid number you need to bind to your local LAN ip-address (so not 0.0.0.0 and not 127.0.0.1 but 192.168.1.something). That is not true.  You can bind to "0.0.0.0" or "127.0.0.1" and still get back a valid port number.  For example, when a socket user wants to bind a socket to a random port chosen by the OS, it first binds to port 0 (whether on "0.0.0.0" or a specific IP, it doesn't matter at all) and then queries the socket for the local port that it was actually bound to.  The IP address(es) being bound to does not affect the port number that is being bound to, and vice versa. ---------------------------------------------------------------------- rvk · 2017-08-14 22:24 Quote from: Remy Lebeau on August 15, 2017, 12:01:25 amQuote from: rvk on August 14, 2017, 10:25:45 pmIf you really want GetLocalSinPort to have a valid number you need to bind to your local LAN ip-address (so not 0.0.0.0 and not 127.0.0.1 but 192.168.1.something). That is not true.  You can bind to "0.0.0.0" or "127.0.0.1" and still get back a valid port number.  For example, when a socket user wants to bind a socket to a random port chosen by the OS, it first binds to port 0 (whether on "0.0.0.0" or a specific IP, it doesn't matter at all) and then queries the socket for the local port that it was actually bound to.  The IP address(es) being bound to does not affect the port number that is being bound to, and vice versa. Then why does GetLocalSinPort return 0 if bound to 0.0.0.0 and a valid local-port number if bound to a local IP-address? Even after sending an UDP GetLocalSinPort is still 0. At what point and how can you query what port is locally used for sending UDP when bound to 0.0.0.0? ---------------------------------------------------------------------- Remy Lebeau · 2017-08-15 00:27 Quote from: rvk on August 15, 2017, 12:24:22 amThen why does GetLocalSinPort return 0 if bound to 0.0.0.0 and a valid local-port number if bound to a local IP-address? I don't know.  Maybe a bug in Synapse?  The underlying BSD socket API should behave the way I described.  Try it for yourself.  Call the API socket(), bind(), and getsockname() functions directly, and it should return a valid port number.  It does for me when I try it.  Something like this: Code: [Select]var   s: TSocket;   addr, bound: sockaddr_in;   len: integer;   port: Integer; s := socket(AF_INET, SOCK_DGRAM, IPPROTO_UDP); if s = -1 then ... FillByte(addr, sizeof(addr), 0); addr.sin_family := AF_INET; addr.sin_addr.s_addr := INADDR_ANY; // "0.0.0.0" addr.sin_port := 0; if bind(s, addr, sizeof(addr)) <> 0 then ... FillByte(bound, sizeof(bound), 0); len := sizeof(bound); if getsockname(s, bound, len) <> 0 then ... port := ntohs(bound.sin_port); I checked the latest Synapse source code.  It is basically doing the same as above.  TBlockSocket.Bind() calls synsock.Bind() (which calls the socket API bind()) followed by synsock.GetSockName() (which calls the socket API getsockname()).  The result is stored in the TBlockSocket.FLocalSin data member (which is a TVarSin, Synapse's equivalent of the standard SOCKADDR_STORAGE record to wrap the sockaddr_in and sockaddr_in6 records).  TBlockSocket.GetLocalSinPort() calls synsock.GetSinPort(), passing it FLocalSin as input.  GetSinPort() extracts the TVarSin's port number from the appropriate field (sockaddr_in.sin_port for IPv4, sockaddr_in6.sin6_port for IPv6) and converts the value from network byte order to host byte order with ntohs(). Quote from: rvk on August 15, 2017, 12:24:22 amAt what point ... can you query what port is locally used for sending UDP when bound to 0.0.0.0? For UDP, immediately after it is successfully bound with bind(). For TCP, immediately after it starts listening (via listen()) or is connected to a peer (via connect() or accept()). Quote from: rvk on August 15, 2017, 12:24:22 amand how Via getsockname() (or higher wrapper). ---------------------------------------------------------------------- rvk · 2017-08-15 08:46 Quote from: Remy Lebeau on August 15, 2017, 02:27:19 amQuote from: rvk on August 15, 2017, 12:24:22 amAt what point ... can you query what port is locally used for sending UDP when bound to 0.0.0.0? For UDP, immediately after it is successfully bound with bind(). Well, I also had a quick look in the source code and it says this: QuoteWarning: when you call : Bind('0.0.0.0','0'); then is nothing done! In this case is used implicit system bind instead.}And if I look in the TBlockSocket.Bind() I see this: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i   if (FSocket <> INVALID_SOCKET)    or not((FFamily = SF_ANY) and (IP = cAnyHost) and (Port = cAnyPort)) thenAnd 0.0.0.0 is the same as cAnyHost so Bind() really does NOTHING when called with 0.0.0.0. I checked this and the code really jumps over this "if" when called with 0.0.0.0. So Bind(0.0.0.0) does NOTHING. The Bind() is implicitly done with the first real UDP send. So, the GetSinLocal needs to be called AFTER the first SendString call. Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i       MyUDPSocket.SendString('aaaaaaaaaaaa');      MyUDPSocket.GetSinLocal; // needs to be called manually after the first UDP for Bind(0.0.0.0)    ShowMessage(MyUDPSocket.GetLocalSinIP + ' ' + IntToStr(MyUDPSocket.GetLocalSinPort)); // works This worked for me and gave me "0.0.0.0 random local port". ---------------------------------------------------------------------- Thaddy · 2017-08-15 09:34 Correct. ---------------------------------------------------------------------- SymbolicFrank · 2017-08-15 12:54 Thanks! It turns out that the device always sends to a fixed port number (it's not in the manual), but I have another project in the works where I have to do it as you guys described. ---------------------------------------------------------------------- Remy Lebeau · 2017-08-15 18:52 Quote from: rvk on August 15, 2017, 10:46:47 amWell, I also had a quick look in the source code and it says this: QuoteWarning: when you call : Bind('0.0.0.0','0'); then is nothing done! In this case is used implicit system bind instead.}And if I look in the TBlockSocket.Bind() I see this: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i   if (FSocket <> INVALID_SOCKET)    or not((FFamily = SF_ANY) and (IP = cAnyHost) and (Port = cAnyPort)) thenAnd 0.0.0.0 is the same as cAnyHost so Bind() really does NOTHING when called with 0.0.0.0. I checked this and the code really jumps over this "if" when called with 0.0.0.0. So Bind(0.0.0.0) does NOTHING. When binding to '0.0.0.0:0', that 'if' statement evaluates to false only if FSocket has not been allocated yet and FFamily is SF_ANY.  Calling TBlockSocket.CreateSocket() explicitly will allocate the socket, but only if FFamily is not SF_ANY: Code: [Select]procedure TBlockSocket.CreateSocket; var   sin: TVarSin; begin   //dummy for SF_Any Family mode   ResetLastError;   if (FFamily <> SF_Any) and (FSocket = INVALID_SOCKET) then // <-- here   begin     ...   end; end; Unfortunately, TBlockSocket.Family is SF_ANY by default: Code: [Select]constructor TBlockSocket.Create; begin   CreateAlternate(''); end; constructor TBlockSocket.CreateAlternate(Stub: string); {$IFNDEF ONCEWINSOCK} var   e: ESynapseError; {$ENDIF} begin   inherited Create;   ...   FFamily := SF_Any;   FFamilySave := SF_Any;   ... end; So, setting TBlockSocket.Family to SF_IP4 explicitly (since the code is trying to bind to an IPv4 address) should allow binding to '0'0.0.0:0' and not skip it: Code: [Select]procedure TSixTLNode.Execute; var   msg: TMessage; begin   MyUDPSocket := TUDPBlockSocket.Create;   try     MyUDPSocket.Family := SF_IP4; // <-- add this!     MyUDPSocket.CreateSocket;     MyUDPSocket.Bind('0.0.0.0', '0');     if MyUDPSocket.LastError <> 0 then Exit;     MyUDPPort := MyUDPSocket.GetLocalSinPort; // <-- should work now!     ...   finally     MyUDPSocket.Free;   end; end; Quote from: rvk on August 15, 2017, 10:46:47 amThe Bind() is implicitly done with the first real UDP send. But the OP's code is *explicitly* calling Bind(), which disables any implicit binding. Quote from: rvk on August 15, 2017, 10:46:47 amSo, the GetSinLocal needs to be called AFTER the first SendString call. Or, after an explicit call to TBlockSocket.Bind() that actually calls the underlying socket API bind() instead of skip it. ---------------------------------------------------------------------- rvk · 2017-08-16 08:12 Quote from: Remy Lebeau on August 15, 2017, 08:52:03 pmSo, setting TBlockSocket.Family to SF_IP4 explicitly (since the code is trying to bind to an IPv4 address) should allow binding to '0'0.0.0:0' and not skip it: Ah, Ok, your example is very clear. So, calling CreateSocket and Bind(0.0.0.0) with family set to SF_IP4 will actually do a real Bind and local-port is known. If family is set to SF_ANY (default), CreateSocket and Bind(0,0.0.0) will have no effect in synapse and an implicit Bind will be done with the first UDP and in that case local-port is only known after that. Setting family to SF_IP4 is a good idea anyway, so you know what you're dealing with. Thnx for the explanation. ---------------------------------------------------------------------- Thaddy · 2017-08-16 08:14 Tnx Remy. I completely missed that too. ---------------------------------------------------------------------- SymbolicFrank · 2017-08-16 09:51 Thanks Remy, that was very clear.