cannot receive messages; udp server with lnet lazarus / 38-Networking and Web Programming ---------------------------------------------------------------------- prodingus · 2022-06-30 23:04 Hi! the client works just fine: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i program test1;{$IFDEF FPC}{$MODE DELPHI}{$ENDIF}{$APPTYPE CONSOLE}uses        sysutils, crt, lnet;var        Sock:TLUDP;begin            Sock:= TLUdp.Create(nil);    Sock.connect('1.1.1.2', 28);     repeat     Sock.sendmessage('ghj');    until false;  end. on the "server" I use the hercules utility to see the data (and I see them just fine). However, when using this code to make my version of server: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i program test1;{$IFDEF FPC}{$MODE DELPHI}{$ENDIF}{$APPTYPE CONSOLE}uses        sysutils, crt, lnet;var        Sock:TLUDP;        tmp: string;begin              Sock:= TLUdp.Create(nil);    //Sock.connect('1.1.1.2', 28);        Sock.listen(word('28'));     repeat begin    Sock.getmessage(tmp);    writeln(Sock.getmessage(tmp));     sleep(100);    end; until false;  end. I get nothing. Can anyone provide a barebone "get udp message loop" example without classes, methods  and stuff? ---------------------------------------------------------------------- dje · 2022-06-30 23:38 What is up wth this line? Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i Sock.listen(word('28')); Check the examples folder for lnet. There is an example called: packages/lnet/examples/console/ludp/ludp.pp ---------------------------------------------------------------------- prodingus · 2022-06-30 23:46 I did. The "word('28')" works only that way as it throws "constant string", expected "word". ---------------------------------------------------------------------- dje · 2022-06-30 23:56 Why not just put 28? ---------------------------------------------------------------------- prodingus · 2022-07-01 00:01 Stupid me, just 28 works! Still no received data from "server". ---------------------------------------------------------------------- dje · 2022-07-01 01:22 The ludp example works fine. Your code fails here for a few reasons. The port number 28 returns Permission denied. I used 1234 with ludp and it worked. Also, the example uses the callback OnReceive to handle listening events. As a tip. Always start with a working example and then hack it to suit your requirements. ---------------------------------------------------------------------- MarkMLl · 2022-07-01 07:19 Quote from: prodingus on July 01, 2022, 02:01:32 amStupid me, just 28 works! Still no received data from "server". Look, if you'd told us in the first place what OS, version of the compiler etc. you'd got you'd have made it a damn sight easier for everybody. You can't use ports <1024 (or maybe <=, can't remember) on a unix-like OS including Linux without having root privilege. Not sure whether there's a corresponding restriction on Windows etc. MarkMLl ---------------------------------------------------------------------- prodingus · 2022-07-01 14:53 Sorry for the missing info: the testing is done on 2 win7 sp1 x64 lite vms, lazarus 1.6, FPC 3.0, SVN 51630. QuoteYour code fails here for a few reasons. The port number 28 returns Permission denied. I used 1234 with ludp and it worked. Before switching to lnet, I was testing with BlckSock (synapse?). I was still using port 28 on both vms and it I was sending and receiving just fine. QuoteAlso, the example uses the callback OnReceive to handle listening events.I don't work with classes and methods... I don't know how to use it. ---------------------------------------------------------------------- prodingus · 2022-07-01 14:56 Ps, changing port number, still no receive. ---------------------------------------------------------------------- Remy Lebeau · 2022-07-01 17:22 Quote from: prodingus on July 01, 2022, 01:04:42 amCode: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i repeat begin  Sock.getmessage(tmp);  writeln(Sock.getmessage(tmp));   sleep(100);end; until false; Why are you calling getmessage() twice per loop iteration?  Get rid of the 1st one.  Or, use the OnReceive event, like Derek suggested. Quote from: MarkMLl on July 01, 2022, 09:19:49 amYou can't use ports <1024 (or maybe <=, can't remember) on a unix-like OS including Linux without having root privilege. Not sure whether there's a corresponding restriction on Windows etc. There is no such restriction on Windows. ---------------------------------------------------------------------- prodingus · 2022-07-02 13:00 QuoteWhy are you calling getmessage() twice per loop iteration?  Get rid of the 1st one. I did try with one,  same result as before. QuoteOr, use the OnReceive event, like Derek suggested. Can someone help me on this? I need a straigthforward way (if possible) to receive udp data. ---------------------------------------------------------------------- dje · 2022-07-02 14:15 Quote from: prodingus on July 02, 2022, 03:00:45 pmI don't work with classes and methods... I don't know how to use it. Can someone help me on this? I need a straigthforward way (if possible) to receive udp data. We can go around and around in circles for days. You cant blindly brute force an API to work the way you want. There are standard processes required to work though these problems. The first... Is to verify that the example is working. 1) Have you successfully compiled and tested the example? If yes... 2) Do you understand the example? If no, then play with the code and read the LNet classes until you do, then goto (2) 3) If yes, split the example into two separate projects for client/server. 4) If your hacking fails, return to step 2 It looks like you want the API to match your desired "style of coding" rather than accept the API must be used in a certain way. Personally, I use Indy since I've been using that for almost 20 years. But, I can tell you, LNet is not a low Unix networking API. Follow the example! ---------------------------------------------------------------------- dje · 2022-07-02 14:26 One last thought. RE: I don't work with classes and methods... I don't know how to use it. You don't have to use a OOP class libraries. I know lots of coders who refuse to use them for a number of reasons. There is nothing wrong with going "old school". Here is C code for UDP client/servers. https://www.cs.cmu.edu/afs/cs/academic/class/15213-f99/www/class26/ You can convert that to Pascal, or go with C. It doesn't matter. But, really, if you don't want learn Object Pascal, then why are you asking for help? ---------------------------------------------------------------------- prodingus · 2022-07-02 14:36 Quote if you don't want learn Object Pascal, then why are you asking for help? Is it wrong to use plain Pascal instead of Object Pascal? As I created a simple loop that sends udp packets (and I receive them on the "server" side using the hercules utility), I need a simple loop that "gets" any pressent packets. ---------------------------------------------------------------------- PascalDragon · 2022-07-02 14:38 Quote from: prodingus on July 02, 2022, 04:36:06 pmQuote if you don't want learn Object Pascal, then why are you asking for help? Is it wrong to use plain Pascal instead of Object Pascal? Considering that you're using lnet you are using Object Pascal. So if you want to use the components correctly you must use the mechanisms and concepts provided by Object Pascal (which includes event handlers). ---------------------------------------------------------------------- prodingus · 2022-07-02 14:40 QuoteConsidering that you're using lnet you are using Object Pascal. So if you want to use the components correctly you must use the mechanisms and concepts provided by Object Pascal (which includes event handlers). Can these handlers be used in my example without creating classes? ---------------------------------------------------------------------- PascalDragon · 2022-07-02 14:42 Quote from: prodingus on July 02, 2022, 04:40:22 pmQuoteConsidering that you're using lnet you are using Object Pascal. So if you want to use the components correctly you must use the mechanisms and concepts provided by Object Pascal (which includes event handlers). Can these handlers be used in my example without creating classes? With the development version of FPC (3.3.1) you could use anonymous functions for that, but otherwise and without opening a rabbit hole that might lead to suffering and pain in the future: no. ---------------------------------------------------------------------- rvk · 2022-07-02 15:11 Quote from: prodingus on July 01, 2022, 04:53:00 pmBefore switching to lnet, I was testing with BlckSock (synapse?). So why did you want to switch to LNet? Quote from: prodingus on July 02, 2022, 04:40:22 pmCan these handlers be used in my example without creating classes? You can try to use a simple class with class function which handles your data. Then you wouldn't need to create an instance but can use the @TDummy.MyReceive() function directly. Not sure if that would work. I prefer Synapse ---------------------------------------------------------------------- prodingus · 2022-07-02 15:49 QuoteSo why did you want to switch to LNet?I read that lnet was asynchronus, thats why. After reading the comments on this thread, it seems lnet is no-go for me. Shall I start a new thread about sockets? ---------------------------------------------------------------------- rvk · 2022-07-02 16:05 Quote from: prodingus on July 02, 2022, 05:49:47 pmQuoteSo why did you want to switch to LNet?I read that lnet was asynchronus, thats why. If you want asynchronous reading of sockets you will always end up with classes and/or threads. That would also be the case for Synapse because it by itself is 'blocking' (the class is even called TBlockSocket in Synapse). You can make your it asynchronous by using it in threads. But if you want everything in your main procedure and don't want to work with classes and/or threads, you WANT to have blocking reading. So you need to think about what you want first. ---------------------------------------------------------------------- Warfley · 2022-07-02 17:05 The easiest way to receive data via UDP is simply using the base socket unit which is provided by the fpc without any additional packages: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i uses  sockets; const  UDPPackLen = 512;var  sock: Tsocket;  buff: Array[0..UDPPackLen-1] of Byte;  ServerAddr, FromAddr: sockaddr_in;  FromAddrLen: Integer;  ClientAddr: String;  ClientPort: Word;  MessageSize: SizeInt;begin  // Start UDP Server  sock := fpsocket(AF_INET, SOCK_DGRAM, 0);  ServerAddr.sin_family:= AF_INET;  ServerAddr.sin_addr := StrToNetAddr('1.1.1.2'); // server IP  ServerAddr.sin_port := 531; // server port  fpbind(sock, @ServerAddr, SizeOf(ServerAddr));  // Receive Data  MessageSize := fprecvfrom(sock, @buff, SizeOf(buff), 0, @FromAddr, @FromAddrLen);  ClientAddr := NetAddrToStr(FromAddr.sin_addr);  ClientPort := NToHs(FromAddr.sin_port);end. Quote from: prodingus on July 02, 2022, 05:49:47 pmQuoteSo why did you want to switch to LNet?I read that lnet was asynchronus, thats why. After reading the comments on this thread, it seems lnet is no-go for me. Shall I start a new thread about sockets? Use select on the socket to check if there is any data. But true asynchronous execution requires your application to be built around that. It's not something you can just "toogle on" using a library. Either you have threads that fire events or you use something like AsyncNet which builds on STAX ---------------------------------------------------------------------- MarkMLl · 2022-07-02 17:27 Quote from: Warfley on July 02, 2022, 07:05:49 pmThe easiest way to receive data via UDP is simply using the base socket unit which is provided by the fpc without any additional packages: I agree. And I'm afraid that my opinion of LNet is that it's a can of worms: I've come across various pathological behaviour particularly when one end or the other of a (TCP) connection dropps unexpectedly and the convoluted nature of its code- which attempts to simulate asynchronous operation without doing it "properly" using threads- makes maintenance "tricky". The variant of LNet bundled with FPC has a bit of extra Telnet handling that isn't in the one on Github, which makes it useful for writing a debugging backdoor for daemon programs (I stress that this should only be used locally because of Telnet's lack of encryption) or a console handler for e.g. a mainframe emulator. But if I needed that functionality for something going into the field I'd redo it from scratch. MarkMLl ---------------------------------------------------------------------- prodingus · 2022-07-02 18:12 QuoteThe easiest way to receive data via UDP is simply using the base socket unit which is provided by the fpc without any additional packages That's why I started a new thread. Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i uses  sockets; const  UDPPackLen = 512;var  sock: Tsocket;  buff: Array[0..UDPPackLen-1] of Byte;  ServerAddr, FromAddr: sockaddr_in;  FromAddrLen: Integer;  ClientAddr: String;  ClientPort: Word;  MessageSize: SizeInt;begin  // Start UDP Server  sock := fpsocket(AF_INET, SOCK_DGRAM, 0);  ServerAddr.sin_family:= AF_INET;  ServerAddr.sin_addr := StrToNetAddr('1.1.1.2'); // server IP  ServerAddr.sin_port := 531; // server port  fpbind(sock, @ServerAddr, SizeOf(ServerAddr));  // Receive Data  MessageSize := fprecvfrom(sock, @buff, SizeOf(buff), 0, @FromAddr, @FromAddrLen);  ClientAddr := NetAddrToStr(FromAddr.sin_addr);  ClientPort := NToHs(FromAddr.sin_port);end. How to writeln the buff?? ---------------------------------------------------------------------- MarkMLl · 2022-07-02 18:22 Put it into a string, set the length, and then proceed in the usual fashion. MarkMLl ---------------------------------------------------------------------- prodingus · 2022-07-02 18:43 Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i program sockets_1; uses        sysutils, crt, sockets; const  UDPPackLen = 512;var  sock: Tsocket;  buff: Array[0..UDPPackLen-1] of Byte;   outp:string;   ServerAddr, FromAddr: sockaddr_in;  FromAddrLen: Integer;  ClientAddr: String;  ClientPort: Word;  MessageSize: SizeInt;begin  // Start UDP Server  sock := fpsocket(AF_INET, SOCK_DGRAM, 0);  ServerAddr.sin_family:= AF_INET;  ServerAddr.sin_addr := StrToNetAddr('1.1.1.2'); // server IP  ServerAddr.sin_port := 1234; // server port  fpbind(sock, @ServerAddr, SizeOf(ServerAddr));  // Receive Data  repeat begin  MessageSize := fprecvfrom(sock, @buff, SizeOf(buff), 0, @FromAddr, @FromAddrLen);  ClientAddr := NetAddrToStr(FromAddr.sin_addr);  ClientPort := NToHs(FromAddr.sin_port);   SetString(outp, PAnsiChar(@buff[0]), 3);  writeln(outp);  sleep(100);  end; until falseend.  Unless I am wrong, this is not working. Hercules is receiveing, but this is not. ---------------------------------------------------------------------- rvk · 2022-07-02 19:38 You're binding your listener to 1.1.1.2? Is that the real addres?? Do you want broadcasts UDP or directed UDP? (also show some code of the server) You have very little error checking in your code. For example. After Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i sock := fpsocket(AF_INET, SOCK_DGRAM, 0); you could set an Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i assert(sock <> -1); to check if the sock is really assigned. (you won't get an error otherwise and nothing will work) For the loop you should do: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i if fpbind(.........) = 0 thenbegin  repeat  untilendelsebegin  writeln(socketError);end;The socketError will give you more information when your fpbind() fails. Also... change Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i MessageSize := fprecvfrom(.......)to Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i MessageSize := fprecvfrom(.......)if MessageSize = -1 then writeln(socketError); That will also give you a lot more information. You have port 1234. I do think you need to use htons(1234) but I'm not sure (and you would need to do that also on the server side). (I'm not that familiar with fpsock) You also haven't shown anything on the server side and specified anything about the network IP's. ---------------------------------------------------------------------- MarkMLl · 2022-07-02 19:43 Not working /how/? What's going wrong? the fpBind() or the receive? What's in SocketError? The rather hackish way I'd do it would be something like Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i     uses    sockets;         const      UDPPackLen = 512;     var      sock: Tsocket;      buff: AnsiString;      ServerAddr, FromAddr: sockaddr_in;      FromAddrLen: Integer;      ClientAddr: String;      ClientPort: Word;      MessageSize: SizeInt;    begin      // Start UDP Server      sock := fpsocket(AF_INET, SOCK_DGRAM, 0);      ServerAddr.sin_family:= AF_INET;      ServerAddr.sin_addr := StrToNetAddr('1.1.1.2'); // server IP      ServerAddr.sin_port := 531; // server port      fpbind(sock, @ServerAddr, SizeOf(ServerAddr));    // Receive Data      SetLength(buff, UDPPackLen);      MessageSize := fprecvfrom(sock, @buff[1], Length(buff), 0, @FromAddr, @FromAddrLen);      SetLength(buff, MessageSize);      ClientAddr := NetAddrToStr(FromAddr.sin_addr);      ClientPort := NToHs(FromAddr.sin_port);      WriteLn(buff)    end.  Note that that's untested, I'm in the middle of doing my year-end books /and/ my usual development system has blown up. But it has the advantage that you can see exactly what's in the string using e.g. the Lazarus debugger, noting of course that element doesn't exist. (Modified to emphasise that the buffer should be an AnsiString, i.e. one byte pre character) MarkMLl ---------------------------------------------------------------------- Warfley · 2022-07-02 20:20 I got really bored and wrote a small wrapper around the sockets API that makes it easier to use and less C-ish: https://github.com/Warfley/PasSimpleSockets/tree/master Just this one Unit: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i unit SimpleSockets; {$mode ObjFPC}{$H+} interface uses  SysUtils, Sockets, {$IfDef WINDOWS}WinSock2{$Else}BaseUnix{$EndIf}; type   { Basic Socket Types }   TSocketFD = Sockets.Tsocket;  TSocketType = (stIPv4, stIPv6, stDualStack);  TSocket = record    FD: TSocketFD;    SocketType: TSocketType;  end;   TAddressType = (atIN4, atIN6);  TNetworkAddress = record    Address: String;    AddressType: TAddressType;  end;   { UDP Return Types }   TUDPResult= record    FromAddr: TNetworkAddress;    FromPort: Word;    DataSize: SizeInt;  end;   generic TUDPDataMessage = record    FromAddr: TNetworkAddress;    FromPort: Word;    Data: T;  end;   TUDPStringMessage = specialize TUDPDataMessage;   { Exceptions }   EDualStackNotSupported = class(Exception);  EUnsupportedAddress = class(Exception);   { ESocketError }   ESocketError = class(Exception)  private    FCode: Integer;  public    constructor Create(ACode: Integer; const FunName: String);     property Code: Integer read FCode;  end;   EConnectionClosedException = class(Exception);  EUDPFragmentationException = class(Exception); const  MaxUDPPackageSize = 512;   { Address Management } function IN4Address(const Address: String): TNetworkAddress; inline;function IN6Address(const Address: String): TNetworkAddress; inline;function IN4MappedIN6Address(const In4Address: String): TNetworkAddress; inline;function INAddr(const Address: String): TNetworkAddress; inline; function IsIPv4Mapped(const IPv6Addr: TNetworkAddress): Boolean; inline;function ExtractIPv4Address(const IPv6Addr: TNetworkAddress): TNetworkAddress; inline; function IN6Equal(const A, B: String): Boolean;operator =(const A, B: TNetworkAddress): Boolean; inline;operator :=(const AStr: String): TNetworkAddress; inline;   { Socket Functions } function TCPSocket(AType: TSocketType): TSocket; inline;function UDPSocket(AType: TSocketType): TSocket; inline; procedure CloseSocket(const ASocket: TSocket); inline; procedure Bind(const ASocket: TSocket; const AAddress: TNetworkAddress; APort: Word);procedure TCPServerListen(const ASocket: TSocket; Backlog: Integer); inline; function TCPReceive(const ASocket: TSocket; ABuffer: Pointer; MaxSize: SizeInt; AFlags: Integer = 0): SizeInt; inline;function UDPReceive(const ASocket: TSocket; ABuffer: Pointer; MaxSize: SizeInt; AFlags: Integer = 0): TUDPResult;function TCPSend(const ASocket: TSocket; ABuffer: Pointer; ASize: SizeInt; AFlags: Integer = 0): SizeInt; inline;function UDPSend(const ASocket: TSocket; const ReceiverAddr: TNetworkAddress;                  ReceiverPort: Word; ABuffer: Pointer; ASize: SizeInt; AFlags: Integer = 0): SizeInt; inline; function TCPReceiveStr(const ASocket: TSocket; MaxLength: SizeInt; AFlags: Integer = 0): String; inline;function UDPReceiveStr(const ASocket: TSocket; MaxLength: SizeInt = MaxUDPPackageSize; AFlags: Integer = 0): TUDPStringMessage; inline;function TCPSendStr(const ASocket: TSocket; const AData: String; AFlags: Integer = 0): SizeInt; inline;function UDPSendStr(const ASocket: TSocket; const ReceiverAddr: TNetworkAddress; ReceiverPort: Word; const AData: String; AFlags: Integer = 0): SizeInt; inline; generic function TCPReceive(const ASocket: TSocket; AFlags: Integer = 0): T; inline;generic function UDPReceive(const ASocket: TSocket; AFlags: Integer = 0): specialize TUDPDataMessage; inline;generic function TCPSend(const ASocket: TSocket; constref AData: T; AFlags: Integer = 0): SizeInt; inline;generic function UDPSend(const ASocket: TSocket; constref ReceiverAddr: TNetworkAddress; ReceiverPort: Word; const AData: T; AFlags: Integer = 0): SizeInt; inline; generic function TCPReceiveArray(const ASocket: TSocket; MaxCount: SizeInt; AFlags: Integer = 0): specialize TArray; inline;generic function UDPReceiveArray(const ASocket: TSocket; MaxCount: SizeInt; AFlags: Integer = 0): specialize TUDPDataMessage>; inline;generic function TCPSendArray(const ASocket: TSocket; const AData: specialize TArray; AFlags: Integer = 0): SizeInt; inline;generic function UDPSendArray(const ASocket: TSocket; const ReceiverAddr: TNetworkAddress; ReceiverPort: Word; const AData: specialize TArray; AFlags: Integer = 0): SizeInt; inline; // Timeout in MSfunction DataPending(const ASocket: TSocket; TimeOut: Integer = 0): Boolean; { Helper }// Must be in interface because of generic functions type  _PAddressUnion = ^_TAddressUnion;  _TAddressUnion = record  case Boolean of  True: (In4Addr: Sockets.sockaddr_in);  False: (In6Addr: Sockets.sockaddr_in6);  end; procedure FillAddr(AAddress: TNetworkAddress; APort: Word; Addr: _PAddressUnion; DualStack: Boolean); inline;procedure ReadAddr(Addr: _PAddressUnion; DualStack: Boolean; out AAddress: TNetworkAddress; out APort: Word);implementation const  IPPROTO_IPV6 = {$IfDef WINDOWS}41{$Else}41{$EndIf};  IPV6_V6ONLY = {$IfDef WINDOWS}27{$Else}26{$EndIf}; procedure FillAddr(AAddress: TNetworkAddress; APort: Word;  Addr: _PAddressUnion; DualStack: Boolean);begin  if (AAddress.AddressType = atIN4) and DualStack then  AAddress := IN4MappedIN6Address(AAddress.Address);  if AAddress.AddressType = atIN4 then  begin    Addr^.In4Addr.sin_family := AF_INET;    Addr^.In4Addr.sin_port := HToNS(APort);    Addr^.In4Addr.sin_addr.s_addr := LongWord(StrToNetAddr(AAddress.Address));  end  else if AAddress.AddressType = atIN6 then  begin    Addr^.In6Addr.sin6_family := AF_INET6;    Addr^.In6Addr.sin6_port := HToNS(APort);    Addr^.In6Addr.sin6_addr := StrToHostAddr6(AAddress.Address);    Addr^.In6Addr.sin6_flowinfo := 0;    Addr^.In6Addr.sin6_scope_id := 0;  end  else    raise EUnsupportedAddress.Create('Address type ' + ord(AAddress.AddressType).ToString + ' not supported');end; procedure ReadAddr(Addr: _PAddressUnion; DualStack: Boolean; out  AAddress: TNetworkAddress; out APort: Word);begin  if Addr^.In4Addr.sin_family = AF_INET then  begin    AAddress := IN4Address(NetAddrToStr(Addr^.In4Addr.sin_addr));    APort := NToHs(Addr^.In4Addr.sin_port);  end  else if Addr^.In6Addr.sin6_family = AF_INET6 then  begin    AAddress := IN6Address(HostAddrToStr6(Addr^.In6Addr.sin6_addr));    if DualStack and IsIPv4Mapped(AAddress.Address) then      AAddress := ExtractIPv4Address(AAddress);    APort := NToHs(Addr^.In6Addr.sin6_port);  end  else    raise EUnsupportedAddress.Create('Address Family ' + Addr^.In4Addr.sin_family.ToString + ' not supported');end; function SocketInvalid(ASocket: TSocketFD): Boolean; inline;begin  {$IfDef Windows}  Result := ASocket = TSocketFD(INVALID_SOCKET);  {$Else}  Result := ASocket < 0;  {$EndIf}end; function CreateSocketFD(ADomain: TSocketType; AType: Integer; AProto: Integer): TSocketFD;var  AFam, v6Only: Integer;begin  if ADomain = stIPv4 then    AFam := AF_INET  else    AFam := AF_INET6;  Result := fpsocket(AFam, AType, AProto);  if SocketInvalid(Result) then    raise ESocketError.Create(socketerror, 'socket');   if ADomain = stDualStack then  begin    v6Only := 0;    if fpsetsockopt(Result, IPPROTO_IPV6, IPV6_V6ONLY, @v6Only, SizeOf(v6Only)) <> 0 then      raise EDualStackNotSupported.Create('Dualstack option not supported on this system: ' + socketerror.ToString);  end;end; function IN4Address(const Address: String): TNetworkAddress;begin Result := Default(TNetworkAddress); Result.Address := Address; Result.AddressType := atIN4;end; function IN6Address(const Address: String): TNetworkAddress;begin Result := Default(TNetworkAddress); Result.Address := Address; Result.AddressType := atIN6;end; function IN4MappedIN6Address(const In4Address: String): TNetworkAddress;var  InAddr: TIn_addr;begin  InAddr := StrToNetAddr(In4Address);  Result := IN6Address('::FFFF:%x:%x'.Format([(InAddr.s_bytes[1] shl 8) or InAddr.s_bytes[2],                                              (InAddr.s_bytes[3] shl 8) or InAddr.s_bytes[4]]));end; function INAddr(const Address: String): TNetworkAddress;begin Result := Default(TNetworkAddress);  if Pos(':', Address) = 0 then    Result.AddressType := atIN4  else    Result.AddressType := atIN6;  Result.Address := Address;end; function IsIPv4Mapped(const IPv6Addr: TNetworkAddress): Boolean;var  In6Addr: Sockets.TIn6Addr;begin  if IPv6Addr.AddressType = atIN4 then    Exit(True);  if IPv6Addr.AddressType  <> atIN6 then    raise EUnsupportedAddress.Create('Can only check IPv4 mapping for IPv6 addresses');  IN6Addr := StrToHostAddr6(IPv6Addr.Address);  Result := (IN6Addr.u6_addr16[0] = 0) and            (IN6Addr.u6_addr16[1] = 0) and            (IN6Addr.u6_addr16[2] = 0) and            (IN6Addr.u6_addr16[3] = 0) and            (IN6Addr.u6_addr16[4] = 0) and            (IN6Addr.u6_addr16[5] = $FFFF);end; function ExtractIPv4Address(const IPv6Addr: TNetworkAddress): TNetworkAddress;var  In6Addr: Sockets.TIn6Addr;begin  if IPv6Addr.AddressType = atIN4 then    Exit(IPv6Addr);  if IPv6Addr.AddressType  <> atIN6 then    raise EUnsupportedAddress.Create('Can only extract IPv4 mapping from IPv6 addresses');  IN6Addr := StrToHostAddr6(IPv6Addr.Address);  Result := IN4Address('%d.%d.%d.%d'.Format([IN6Addr.s6_addr8[12],                                           IN6Addr.s6_addr8[13],                                           IN6Addr.s6_addr8[14],                                           IN6Addr.s6_addr8[15]]));end; function IN6Equal(const A, B: String): Boolean;var  AAddr, BAddr: Sockets.Tin6_addr;begin  AAddr := StrToHostAddr6(A);  BAddr := StrToHostAddr6(B);  Result := (AAddr.s6_addr32[0] = BAddr.s6_addr32[0]) and            (AAddr.s6_addr32[1] = BAddr.s6_addr32[1]) and            (AAddr.s6_addr32[2] = BAddr.s6_addr32[2]) and            (AAddr.s6_addr32[3] = BAddr.s6_addr32[3]);end; operator=(const A, B: TNetworkAddress): Boolean;begin  Result := (A.AddressType = B.AddressType) and (              ((A.AddressType = atIN4) and (A.Address = B.Address)) or // IPv4: simple string equality            ((A.AddressType = atIN6) and IN6Equal(A.Address, B.Address)) // IPv6 check binary equality            );end; operator:=(const AStr: String): TNetworkAddress;begin  Result := INAddr(AStr);end; function TCPSocket(AType: TSocketType): TSocket;begin  Result.SocketType := AType;  Result.FD := CreateSocketFD(AType, SOCK_STREAM, 0);end; function UDPSocket(AType: TSocketType): TSocket;begin  Result.SocketType := AType;  Result.FD := CreateSocketFD(AType, SOCK_DGRAM, 0);end; procedure CloseSocket(const ASocket: TSocket);begin  Sockets.CloseSocket(ASocket.FD);end; procedure Bind(const ASocket: TSocket; const AAddress: TNetworkAddress;  APort: Word);var  addr: _TAddressUnion;begin  FillAddr(AAddress, APort, @addr, ASocket.SocketType = stDualStack);  if fpbind(ASocket.FD, @addr, SizeOf(addr)) <> 0 then raise    ESocketError.Create(socketerror, 'bind (%s:%d)'.Format([AAddress.Address, APort]));end; procedure TCPServerListen(const ASocket: TSocket; Backlog: Integer);begin  if fplisten(ASocket.FD, Backlog) <> 0 then raise    ESocketError.Create(socketerror, 'listen');end; function TCPReceive(const ASocket: TSocket; ABuffer: Pointer; MaxSize: SizeInt;  AFlags: Integer): SizeInt;begin  Result := fprecv(ASocket.FD, ABuffer, MaxSize, AFlags);  if Result = 0 then    raise EConnectionClosedException.Create('The connection closed')  else if Result < 0 then    raise ESocketError.Create(socketerror, 'recv');end; function UDPReceive(const ASocket: TSocket; ABuffer: Pointer; MaxSize: SizeInt;  AFlags: Integer): TUDPResult;var  {$IfNDef WINDOWS}  _addr: _TAddressUnion;  _addrLen: SizeInt;  {$EndIf}  addr: _PAddressUnion;  addrLen: PSizeInt;begin  {$IfDef WINDOWS}  // WinSock doesn't like the addr located on the stack, therefore we create a heap instance for it  New(addr);  New(addrLen);  try  {$Else}  addr := @_addr;  addrLen := @_addrLen;  {$EndIf}  addrLen^ := SizeOf(_TAddressUnion);  Result.DataSize := fprecvfrom(ASocket.FD, ABuffer, MaxSize, AFlags, Sockets.PSockAddr(addr), addrLen);  if Result.DataSize < 0 then    raise ESocketError.Create(socketerror, 'recvfrom');  ReadAddr(addr, ASocket.SocketType = stDualStack, Result.FromAddr, Result.FromPort);  {$IfDef WINDOWS}  finally    Dispose(addr);    Dispose(addrLen);  end;  {$EndIf}end; function TCPSend(const ASocket: TSocket; ABuffer: Pointer; ASize: SizeInt;  AFlags: Integer): SizeInt;begin  Result := fpsend(ASocket.FD, ABuffer, ASize, AFlags);  if Result < 0 then    raise ESocketError.Create(socketerror, 'send');end; function UDPSend(const ASocket: TSocket; const ReceiverAddr: TNetworkAddress;  ReceiverPort: Word; ABuffer: Pointer; ASize: SizeInt; AFlags: Integer  ): SizeInt;var  addr: _TAddressUnion;begin  FillAddr(ReceiverAddr, ReceiverPort, @addr, ASocket.SocketType = stDualStack);  Result := fpsendto(ASocket.FD, ABuffer, ASize, AFlags, @addr, SizeOf(addr));  if Result < 0 then    raise ESocketError.Create(socketerror, 'sendto');end; function TCPReceiveStr(const ASocket: TSocket; MaxLength: SizeInt;  AFlags: Integer): String;var  Len: SizeInt;begin  Result := '';  SetLength(Result, MaxLength);  Len := TCPReceive(ASocket, @Result[1], MaxLength, AFlags);  SetLength(Result, Len);end; function UDPReceiveStr(const ASocket: TSocket; MaxLength: SizeInt;  AFlags: Integer): TUDPStringMessage;var  UdpMessage: TUDPResult;begin  Result := Default(specialize TUDPDataMessage);  SetLength(Result.Data, MaxLength);  UdpMessage := UDPReceive(ASocket, @Result.Data[1], MaxLength, AFlags);  SetLength(Result.Data, UdpMessage.DataSize);  Result.FromAddr := UdpMessage.FromAddr;  Result.FromPort := UdpMessage.FromPort;end; function TCPSendStr(const ASocket: TSocket; const AData: String; AFlags: Integer  ): SizeInt;begin  if Length(AData) = 0 then Exit(0);  Result := TCPSend(ASocket, @AData[1], Length(AData), AFlags);end; function UDPSendStr(const ASocket: TSocket;  const ReceiverAddr: TNetworkAddress; ReceiverPort: Word; const AData: String; AFlags: Integer  ): SizeInt;begin  if Length(AData) = 0 then Exit(0);  Result := UDPSend(ASocket, ReceiverAddr, ReceiverPort, @AData[1], Length(AData), AFlags);end; generic function TCPReceive(const ASocket: TSocket; AFlags: Integer = 0): T;var  Len: SizeInt;begin  Result := Default(T);  Len := 0;  while Len < SizeOf(Result) do    Len += TCPReceive(ASocket, @PByte(@Result)[Len], SizeOf(Result) - Len, AFlags);end; generic function UDPReceive(const ASocket: TSocket; AFlags: Integer = 0): specialize TUDPDataMessage;var  UdpMessage: TUDPResult;begin  Result := Default(T);  UdpMessage := UDPReceive(ASocket, @Result, SizeOf(Result), AFlags);  if UdpMessage.DataSize < SizeOf(T) then    raise EUDPFragmentationException.Create('Receiving of fragmented data is not supported by typed receive');  Result.FromAddr := UdpMessage.FromAddr;  Result.FromPort := UdpMessage.FromPort;end; generic function TCPSend(const ASocket: TSocket; constref AData: T; AFlags: Integer = 0): SizeInt;begin  Result := TCPSend(ASocket, @AData, SizeOf(T), AFlags);end; generic function UDPSend(const ASocket: TSocket; const ReceiverAddr: TNetworkAddress; ReceiverPort: Word; constref AData: T; AFlags: Integer = 0): SizeInt;begin  Result := UDPSend(ASocket, ReceiverAddr, ReceiverPort, @AData, SizeOf(T), AFlags);end; generic function TCPReceiveArray(const ASocket: TSocket; MaxCount: SizeInt; AFlags: Integer = 0): specialize TArray;var  Len: SizeInt;begin       Result := nil;  SetLength(Result, MaxCount);  Len := 0;  repeat    Len += TCPReceive(ASocket, @PByte(@Result)[Len], MaxCount * SizeOf(T) - Len, AFlags);  until (Len mod SizeOf(T)) = 0;  SetLength(Result, Len div SizeOf(T));end; generic function UDPReceiveArray(const ASocket: TSocket; MaxCount: SizeInt; AFlags: Integer = 0): specialize TUDPDataMessage>;var  UdpMessage: TUDPResult;begin  Result.Data := nil;  SetLength(Result.Data, MaxCount);  UdpMessage := UDPReceive(ASocket, @Result.Data[0], MaxCount * SizeOf(T), AFlags);  if UdpMessage.DataSize mod SizeOf(T) > 0 then    raise EUDPFragmentationException.Create('Receiving of fragmented data is not supported by typed receive');  SetLength(Result.Data, UdpMessage.DataSize div SizeOf(T));   Result.FromAddr := UdpMessage.FromAddr;  Result.FromPort := UdpMessage.FromPort;end; generic function TCPSendArray(const ASocket: TSocket; const AData: specialize TArray; AFlags: Integer = 0): SizeInt;begin  if Length(AData) = 0 then Exit(0);  Result := TCPSend(ASocket, @AData[0], Length(AData) * SizeOf(T), AFlags);end; generic function UDPSendArray(const ASocket: TSocket; const ReceiverAddr: TNetworkAddress; ReceiverPort: Word; const AData: specialize TArray; AFlags: Integer = 0): SizeInt;begin  if Length(AData) = 0 then Exit(0);  Result := UDPSend(ASocket, ReceiverAddr, ReceiverPort, @AData[0], Length(AData) * SizeOf(T), AFlags);end; function DataPending(const ASocket: TSocket; TimeOut: Integer): Boolean;var  FDSet: TFDSet;  timeval: TTimeVal;  Ret: LongInt;begin  {$IfDef UNIX}fp{$endif}FD_ZERO(FDSet);  {$IfDef UNIX}fp{$endif}FD_SET(ASocket.FD, FDSet);  timeval.tv_sec := TimeOut div 1000;  timeval.tv_usec := (TimeOut mod 1000) * 1000;  Ret := {$IfDef UNIX}fp{$endif}select(ASocket.FD, @FDSet, nil, nil, @timeval);  if Ret < 0 then    raise ESocketError.Create(socketerror, 'select');  Result := Ret > 0;end; { ESocketError } constructor ESocketError.Create(ACode: Integer; const FunName: String);begin  inherited CreateFmt('[Socket Error: %d] %s call failed',  [ACode, FunName]);  FCode := ACode;end; end. Example: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i program example; {$mode objfpc}{$H+} uses  SimpleSockets; var  Sock: TSocket;  Msg: TUDPStringMessage;begin  Sock := UDPSocket(stDualStack);  try    Bind(Sock, '127.0.0.1', 1337);    // Simple echo server: Receive message and answer with same message    Msg := UDPReceiveStr(Sock);  WriteLn('Received from Client at ', Msg.FromAddr.Address, ':', Msg.FromPort,' message: ', Msg.Data);    UDPSendStr(Sock, Msg.FromAddr, Msg.FromPort, Msg.Data);  finally    CloseSocket(Sock);  end;  ReadLn;end.To use just copy the unit into your project and get started PS: this is something I wanted to do for quite some time, it is basically the AsyncNet version just not asynchronous. Because I think using sockets for small things (like a small UDP client or server) is way to complicated right now. But it defenitely needs completion, as it just contains the basic socket functions Also this was a test about DualStack implementations I wanted to improve in asyncnet, and this seems to work quite well and transparently @prodingus Feel free to use it, did some simple tests and at least for this it works quite well ---------------------------------------------------------------------- rvk · 2022-07-02 20:27 I also made a very very very crude sample. Sender and Receiver. At least this will print out the errors TS might get. Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i program Sender;uses  SysUtils, crt, sockets;const  UDPPackLen = 512;var  sock: Tsocket;  buff: array[0..UDPPackLen - 1] of char;  ServerAddr: sockaddr_in;  MessageSize: SizeInt;begin  sock := fpsocket(AF_INET, SOCK_DGRAM, 0);  assert(sock <> -1);  ServerAddr.sin_family := AF_INET;  ServerAddr.sin_addr := StrToNetAddr('192.168.2.11'); // server IP  ServerAddr.sin_port := htons(1234); // server port  // if fpbind(sock, @ServerAddr, SizeOf(ServerAddr)) = 0 then  if fpconnect(sock, @ServerAddr, SizeOf(ServerAddr)) = 0 then  begin    repeat      buff := timetostr(now) + #0;      MessageSize := fpsend(Sock, @Buff[0], 9, 0);      if MessageSize = -1 then      begin        writeln(socketError);      end      else      begin      writeln('Sending: ' + timetostr(now) + ' ' + MessageSize.ToString);    end;      sleep(100);    until False;  end  else  begin    writeln(socketError);    readln;  end;end. Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i program Receiver;uses  SysUtils, crt, sockets;const  UDPPackLen = 512;var  sock: Tsocket;  buff: array[0..UDPPackLen - 1] of char;  ServerAddr: sockaddr_in;  MessageSize: SizeInt;  outp: string;begin  sock := fpsocket(AF_INET, SOCK_DGRAM, 0);  assert(sock <> -1);  ServerAddr.sin_family := AF_INET;  ServerAddr.sin_addr := StrToNetAddr('192.168.2.11'); // server IP  ServerAddr.sin_port := htons(1234); // server port  if fpbind(sock, @ServerAddr, SizeOf(ServerAddr)) = 0 then  begin    repeat      MessageSize := fprecv(sock, @buff, SizeOf(buff), 0);      if MessageSize = -1 then    begin        writeln(socketError);      end      else      begin      SetString(outp, pansichar(@buff[0]), 8);        writeln('Received: ' + outp);      end;      sleep(100);    until False;  end  else  begin    writeln(socketError);    readln;  end;end. Works fine here (after allowing communication through the firewall). ---------------------------------------------------------------------- MarkMLl · 2022-07-02 20:57 Quote from: Warfley on July 02, 2022, 10:20:15 pmBecause I think using sockets for small things (like a small UDP client or server) is way to complicated right now. But it defenitely needs completion, as it just contains the basic socket functions UDP (and the equivalents via unix-domain sockets and whatever Windows offers these days) is vastly underused. But what would be really useful is a subset SCTP (message-oriented but reliable) which was also small enough to put on e.g. an Arduino endpoint. MarkMLl ---------------------------------------------------------------------- prodingus · 2022-07-02 21:18 QuoteThe socketError will give you more information when your fpbind() fails. this is the problem! it returns 10022 error. The code is running ON THE "SERVER" and the ip of that "server" is 1.1.1.2.  . Also, no firewall. ---------------------------------------------------------------------- Warfley · 2022-07-02 21:36 This error is the EINVAL error, from the WinSock2 Documentation for bind: QuoteAn invalid argument was supplied. This error is returned of the socket s is already bound to an address. Is the code you posted previously the exact code that results in this error? ---------------------------------------------------------------------- prodingus · 2022-07-02 21:39 QuoteThe rather hackish way I'd do it would be something like Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i     uses    sockets;         const      UDPPackLen = 512;     var      sock: Tsocket;      buff: String;      ServerAddr, FromAddr: sockaddr_in;    FromAddrLen: Integer;      ClientAddr: String;      ClientPort: Word;      MessageSize: SizeInt;    begin      // Start UDP Server    sock := fpsocket(AF_INET, SOCK_DGRAM, 0);      ServerAddr.sin_family:= AF_INET;      ServerAddr.sin_addr := StrToNetAddr('1.1.1.2'); // server IP      ServerAddr.sin_port := 531; // server port      fpbind(sock, @ServerAddr, SizeOf(ServerAddr));    // Receive Data      SetLength(buff, UDPPackLen);      MessageSize := fprecvfrom(sock, @buff[1], Length(buff), 0, @FromAddr, @FromAddrLen);      SetLength(buff, MessageSize);      ClientAddr := NetAddrToStr(FromAddr.sin_addr);      ClientPort := NToHs(FromAddr.sin_port);      WriteLn(buff)    end. even this returns 10022 ---------------------------------------------------------------------- rvk · 2022-07-02 21:41 Quote from: prodingus on July 02, 2022, 11:18:45 pmQuoteThe socketError will give you more information when your fpbind() fails. this is the problem! it returns 10022 error. The code is running ON THE "SERVER" and the ip of that "server" is 1.1.1.2.  . Also, no firewall. Did you also try my same Sender code I gave you? Or are you using your own (which we haven't seen yet)? Just try my Sender and Reciever code. ---------------------------------------------------------------------- prodingus · 2022-07-02 21:43 QuoteIs the code you posted previously the exact code that results in this error? QuoteDid you also try my same Sender code I gave you? Yes its the same code (before the sender receiver, I haven't tried them yet). And I did a system reboot. Nothing else networking is running. ---------------------------------------------------------------------- Warfley · 2022-07-02 21:48 The error is not the bind but the recvfrom, and the error here is because thanks to some weird quirk of WinSock, recvfrom does not like it if the address is located on the stack (or unaligned memory in general). The quick solution to this is to put the address and length on the heap: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i         uses          sockets;                 const          UDPPackLen = 512;             var          sock: Tsocket;          buff: String;        ServerAddr: sockaddr_in;          FromAddr: Psockaddr_in;        FromAddrLen: PInteger;          ClientAddr: String;          ClientPort: Word;          MessageSize: SizeInt;        begin        // Start UDP Server          sock := fpsocket(AF_INET, SOCK_DGRAM, 0);          ServerAddr.sin_family:= AF_INET;          ServerAddr.sin_addr := StrToNetAddr('1.1.1.2'); // server IP          ServerAddr.sin_port := 531; // server port          fpbind(sock, @ServerAddr, SizeOf(ServerAddr));          // Receive Data          SetLength(buff, UDPPackLen);          new(FromAddr);          new(FromAddrLen);          try            FromAddrLen^ := SizeOf(FromAddr^);            MessageSize := fprecvfrom(sock, @buff[1], Length(buff), 0, @FromAddr, @FromAddrLen);            SetLength(buff, MessageSize);            ClientAddr := NetAddrToStr(FromAddr.sin_addr);            ClientPort := NToHs(FromAddr.sin_port);          finally            Dispose(FromAddr);            Dispose(FromAddrLen);          end;        WriteLn(buff)        end. It's just a very weird windows thing, best to not think to much about Or, as stated above, youse the Unit I posted above: https://github.com /Warfley/PasSimpleSockets/blob/master/simplesockets.pas (example: http s://github.com/Warfley/PasSimpleSockets/blob/master/examples/udpechose rver.pas) which also does handle this (as I have encountered this already quite often) ---------------------------------------------------------------------- rvk · 2022-07-02 21:54 Quote from: Warfley on July 02, 2022, 11:48:22 pmThe error is not the bind but the recvfrom, and the error here is because thanks to some weird quirk of WinSock, recvfrom does not like it if the address is located on the stack (or unaligned memory in general).The code I gave didn't use fprecvfrom but fprecv. So I wonder if TS used my code or again some own changed version? ---------------------------------------------------------------------- prodingus · 2022-07-02 22:19 QuoteThe error is not the bind but the recvfrom, and the error here is because thanks to some weird quirk of WinSock, recvfrom does not like it if the address is located on the stack (or unaligned memory in general). The quick solution to this is to put the address and length on the heap: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i new(FromAddr);new(FromAddrLen); after this I get again error 10022. Also after your modifications the lines: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i ClientAddr := NetAddrToStr(FromAddr.sin_addr);ClientPort := NToHs(FromAddr.sin_port);produce error sockets_1.lpr(41,42) Fatal: Syntax error, ")" expected but "identifier SIN_ADDR" found sockets_1.lpr(41,42) Fatal: Syntax error, ")" expected but "identifier SIN_PORT" found ---------------------------------------------------------------------- Warfley · 2022-07-02 22:28 Yeah my bad it must be: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i MessageSize := fprecvfrom(sock, @buff[1], Length(buff), 0, FromAddr, FromAddrLen);            SetLength(buff, MessageSize);            ClientAddr := NetAddrToStr(FromAddr^.sin_addr);            ClientPort := NToHs(FromAddr^.sin_port);As they are now pointer ---------------------------------------------------------------------- rvk · 2022-07-02 22:31 Why are you even using fprecvfrom while you don't do anything with the from data? You can just use fprecv without the fromaddr like I showed in my example. Did you try my exact code yet? ---------------------------------------------------------------------- Warfley · 2022-07-02 22:47 I think that it is quite important to get recvfrom working, because a UDP server that is not capable of answering is probably quite pointless. What would be the point if he could get this running using recv when at some point he will need recvfrom probably anyway? ---------------------------------------------------------------------- rvk · 2022-07-02 22:54 Quote from: Warfley on July 03, 2022, 12:47:59 amWhat would be the point if he could get this running using recv when at some point he will need recvfrom probably anyway? Because the fromadress was already known. It's always the server. TS hasn't really mentioned what this is used for but from the code it is clear the server is 1.1.1.2. So the address to anwer to is known. Quote from: prodingus on July 02, 2022, 06:18:30 pma client loop that  sends continiously a message (to an ip,port), and a server loop that listens (receives) for any messages Ps, I am noob at networking. Side note: I am using 2 windows 7 vms,  ips are 1.1.1.1 and 1.1.1.2, no firewall. Besides that... If my code also generates a 10022 error there is something else wrong and it has nothing to do with recvfrom. If it works we can expand it with recvfrom if needed ---------------------------------------------------------------------- MarkMLl · 2022-07-03 07:33 Quote from: rvk on July 03, 2022, 12:31:19 amYou can just use fprecv without the fromaddr like I showed in my example. Did you try my exact code yet? I'd suggest apart from anything else that it's a matter of /courtesy/ to do so, before complaining of ongoing problems. I agree that RecvFrom() might turn out to be needed as part of the bigger picture, but the first step is to verify that addresses etc. are OK. MarkMLl ---------------------------------------------------------------------- Warfley · 2022-07-03 09:21 Quote from: rvk on July 03, 2022, 12:54:01 amBecause the fromadress was already known. It's always the server.At least one of us has a misunderstanding here, because I understood it as if he is currently writing the server, and the server probably needs to use recvfrom to identify the client Quote from: rvk on July 03, 2022, 12:54:01 amBesides that... If my code also generates a 10022 error there is something else wrong and it has nothing to do with recvfrom. If it works we can expand it with recvfrom if needed I could reproduce this error on my machine, and as I already stated, the problem with stack pointers for the memory is something that is known for the winapi for quite some time (and took me a many hours to find out when I first encountered it). But with this code it works on my machine: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i uses  sockets; const  UDPPackLen = 512; var  sock: Tsocket;  buff: String;  ServerAddr: sockaddr_in;  FromAddr: psockaddr_in;  FromAddrLen: PInteger;  ClientAddr: String;  ClientPort: Word;  MessageSize: SizeInt;begin  // Start UDP Server  sock := fpsocket(AF_INET, SOCK_DGRAM, 0);  ServerAddr.sin_family:= AF_INET;  ServerAddr.sin_addr := StrToNetAddr('127.0.0.1'); // server IP  ServerAddr.sin_port := 531; // server port  fpbind(sock, @ServerAddr, SizeOf(ServerAddr));  // Receive Data  SetLength(buff, UDPPackLen);  new(FromAddr);  new(FromAddrLen);  try    FromAddrLen^ := SizeOf(FromAddr^);    MessageSize := fprecvfrom(sock, @buff[1], Length(buff), 0, psockaddr(FromAddr), FromAddrLen);    WriteLn('err: ', socketerror);  SetLength(buff, MessageSize);    ClientAddr := NetAddrToStr(FromAddr^.sin_addr);    ClientPort := NToHs(FromAddr^.sin_port);  finally    Dispose(FromAddr);    Dispose(FromAddrLen);  end;  WriteLn(buff);  ReadLn;end.      Of course the IP needs to be changed (I don't have IP 1.1.1.2) PS: @TE are you shure you have IP 1.1.1.2, because as far as I know the 1.1.1.x IP range is owned by Cloudflare, who host their infrastructure their (as low ip numbers are a few nanoseconds faster than high ones in switching, to give them more performance for their CDN services) ---------------------------------------------------------------------- rvk · 2022-07-03 09:42 @Warfley. I didn't get a 10022 but a 10014 error with the fprecvfrom. To solve this I didn't need to create FromAddr but could just leave it on the stack. But you DO need to provide the correct FromAddrLen for it. (And that was missing from the original code) So here my code with fprecvfrom (with error checking). Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i program Receiver;uses  SysUtils, crt, sockets;const  UDPPackLen = 512;var  sock: Tsocket;  buff: array[0..UDPPackLen - 1] of char;  ServerAddr: sockaddr_in;  FromAddr: sockaddr_in;  FromAddrLen: Integer;  MessageSize: SizeInt;  outp: string;begin  sock := fpsocket(AF_INET, SOCK_DGRAM, 0);  assert(sock <> -1);  ServerAddr.sin_family := AF_INET;  ServerAddr.sin_addr := StrToNetAddr('192.168.2.11'); // server IP  ServerAddr.sin_port := htons(1234); // server port  if fpbind(sock, @ServerAddr, SizeOf(ServerAddr)) = 0 then  begin    repeat      FromAddrLen := SizeOf(FromAddr);      MessageSize := fprecvfrom(sock, @buff, Length(buff), 0, @FromAddr, @FromAddrLen);    if MessageSize = -1 then      begin        writeln(socketError);    end      else      begin        SetString(outp, pansichar(@buff[0]), 8);        writeln('Received: ' + outp + ' ' + NetAddrToStr(FromAddr.sin_addr));      end;      sleep(100);    until False;  end  else  begin    writeln(socketError);    readln;  end;end. QuoteReceived: 11:46:49 192.168.2.11 Received: 11:46:49 192.168.2.11 Received: 11:46:49 192.168.2.11 Received: 11:46:49 192.168.2.11 Received: 11:46:50 192.168.2.11 Received: 11:46:50 192.168.2.11 Received: 11:46:50 192.168.2.11 Received: 11:46:50 192.168.2.11 Received: 11:46:50 192.168.2.11 Received: 11:46:50 192.168.2.11 Received: 11:46:50 192.168.2.11 Received: 11:46:51 192.168.2.11 Received: 11:46:51 192.168.2.11 Received: 11:46:51 192.168.2.11 Received: 11:46:51 192.168.2.11 Received: 11:46:51 192.168.2.11 Received: 11:46:51 192.168.2.11 Received: 11:46:51 192.168.2.11 Received: 11:46:51 192.168.2.11 Edit: Changing your code back to local variables it also works fine as long as you set FromAddrLen. The moment you remove line 23 you'll get a 10014. I think in this code the local variables are already always correctly aligned. You do have a point that they could be misaligned but that's not the case here. Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i uses  sockets;const  UDPPackLen = 512;var  sock: Tsocket;  buff: String;  ServerAddr: sockaddr_in;  FromAddr: sockaddr_in;  FromAddrLen: Integer;  ClientAddr: String;  ClientPort: Word;  MessageSize: SizeInt;begin  // Start UDP Server  sock := fpsocket(AF_INET, SOCK_DGRAM, 0);  ServerAddr.sin_family:= AF_INET;  ServerAddr.sin_addr := StrToNetAddr('192.168.2.11'); // server IP  ServerAddr.sin_port := htons(1234); // server port  fpbind(sock, @ServerAddr, SizeOf(ServerAddr));  // Receive Data  SetLength(buff, UDPPackLen);  FromAddrLen := SizeOf(FromAddr);  MessageSize := fprecvfrom(sock, @buff[1], Length(buff), 0, @FromAddr, @FromAddrLen);  WriteLn('err: ', socketerror);  SetLength(buff, MessageSize);  ClientAddr := NetAddrToStr(FromAddr.sin_addr);  ClientPort := NToHs(FromAddr.sin_port);  WriteLn(buff);  ReadLn;end. Quoteerr: 0 11:56:52 ---------------------------------------------------------------------- prodingus · 2022-07-03 16:22 Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i program send_data_test;{$APPTYPE CONSOLE}uses        sysutils, crt, lnet;var      Sock:TLUDP;        i,j,k:integer;  begin        i:=0;          Sock:= TLUdp.Create(nil);        Sock.connect('1.1.1.2', 1234);        repeat begin                 sock.sendmessage(inttostr(i+5));              i:=i+1;                writeln(i,' ', inttostr(i+5));               sleep(10);         end; until keypressed=true;         readln; end. the client, Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i program sockets_1; uses        sysutils, crt, sockets; const  UDPPackLen = 512;var        i:integer;  sock: Tsocket;  buff: string;   outp:string;   ServerAddr: sockaddr_in;  FromAddr:  sockaddr_in;  FromAddrLen: Integer;  ClientAddr: String;  ClientPort: Word;  MessageSize: SizeInt;begin     i:=0;        // Start UDP Server      sock := fpsocket(AF_INET, SOCK_DGRAM, 0);        ServerAddr.sin_family:= AF_INET;        ServerAddr.sin_addr := StrToNetAddr('1.1.1.2'); // server IP        ServerAddr.sin_port := htons(1234); // server port        fpbind(sock, @ServerAddr, SizeOf(ServerAddr));         writeln(sock <> -1);        writeln(socketError);         // Receive Data        SetLength(buff, UDPPackLen);        FromAddrLen := SizeOf(FromAddr);        writeln(socketError);                 MessageSize := fprecvfrom(sock, @buff[1], Length(buff), 0, @FromAddr, @FromAddrLen);                //WriteLn('err: ', socketerror);                SetLength(buff, MessageSize);                ClientAddr := NetAddrToStr(FromAddr.sin_addr);                ClientPort := NToHs(FromAddr.sin_port);        repeat begin                SetLength(buff, MessageSize);                MessageSize := fprecvfrom(sock, @buff[1], Length(buff), 0, @FromAddr, @FromAddrLen);                 i:=i+1;                WriteLn(i,' ',buff);                sleep(100);        end; until false;        ReadLn;  end.  the server. The client sends with sleep(10) and the server receives (YES, its working!!!) with sleep(100). The problem 1 is that the server keeps receiving even after I stop client, and 2 the received messages stop at 10 (displays only 1 and for 11+ displays nothing). The +5 was just for check, no other reason. ---------------------------------------------------------------------- MarkMLl · 2022-07-03 16:24 Quote from: prodingus on July 03, 2022, 06:22:27 pmThe problem 1 is that the server keeps receiving even after I stop client What do you mean? MarkMLl ---------------------------------------------------------------------- rvk · 2022-07-03 16:32 Quote from: prodingus on July 03, 2022, 06:22:27 pmThe client sends with sleep(10) and the server receives (YES, its working!!!) with sleep(100). The problem 1 is that the server keeps receiving even after I stop client, and 2 the received messages stop at 10 (displays only 1 and for 11+ displays nothing). The +5 was just for check, no other reason. 1) UDP also has a buffer so it's usual you would end up with additional message if your sender is faster than the receiver. 2) You have a  SetLength(buff, MessageSize); on line 45. But MessageSize is the result of the previous fprecvfrom !!! So the MessageSize could be 0. You need to do a SetLength(buff, UDPPackLen); before the fprecvfrom to make room for the string AND a SetLength(buff, MessageSize); to cut the buffer/string to the correct received size after the fprecvfrom. So somthing like this: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i i:=0;// Start UDP Serversock := fpsocket(AF_INET, SOCK_DGRAM, 0);ServerAddr.sin_family:= AF_INET;ServerAddr.sin_addr := StrToNetAddr('1.1.1.2'); // server IPServerAddr.sin_port := htons(1234); // server portfpbind(sock, @ServerAddr, SizeOf(ServerAddr)); repeat  SetLength(buff, UDPPackLen);  MessageSize := fprecvfrom(sock, @buff[1], Length(buff), 0, @FromAddr, @FromAddrLen);  SetLength(buff, MessageSize);   ClientAddr := NetAddrToStr(FromAddr.sin_addr);  ClientPort := NToHs(FromAddr.sin_port);   i:=i+1;  WriteLn(i,' ',buff);  sleep(100);until false; No need to do the first fprecvfrom you have above the repeat. And no need to wrap the code inside repeat/until in begin/end. ---------------------------------------------------------------------- MarkMLl · 2022-07-03 16:40 Quote from: rvk on July 03, 2022, 06:32:53 pmYou need to do a SetLength(buff, UDPPackLen); before the fprecvfrom to make room for the string AND a SetLength(buff, MessageSize); to cut the buffer/string to the correct received size after the fprecvfrom. As in my original example. Also I'd note that I've edited that to use AnsiString, lest any "extended ASCII" characters appear and cause confusion. MarkMLl ---------------------------------------------------------------------- prodingus · 2022-07-03 16:46 QuoteWhat do you mean?The client sends fast and the server receives slower (seems to receive slower because of sleep(100)); as rvk said, its a buffer. Quote2) You have a  SetLength(buff, MessageSize); on line 45. But MessageSize is the result of the previous fprecvfrom !!! So the MessageSize could be 0. You need to do a SetLength(buff, UDPPackLen); before the fprecvfrom to make room for the string AND a SetLength(buff, MessageSize); to cut the buffer/string to the correct received size after the fprecvfrom. yes, that did the trick. The thing with the buffer, does it have anything to do with synchronous/asynchronous? I thought that udp packets, if you don't receive them in time, you loose them. ---------------------------------------------------------------------- rvk · 2022-07-03 16:55 Quote from: prodingus on July 03, 2022, 06:46:43 pmThe thing with the buffer, does it have anything to do with synchronous/asynchronous? I thought that udp packets, if you don't receive them in time, you loose them. No. Buffering is normal. I'm not sure how large the buffer for udp socket is on Windows but there are also some other buffers at play. Buffer in the network adapter for example. So, although UDP reception is not guaranteed, the reliabilty of receiving them can be quite high. Just make sure your reception is fast enough (or at least faster than your sending). ---------------------------------------------------------------------- prodingus · 2022-07-03 17:13 It seems that the communication is reliable, with one quirk: the last message is not received, althout the client seems to have send it judging from the client's output (must be the repeat loop?). The problem with the buffer is this: I wanted to make the client send message 1 to the server, then when the server received the message 1, send a message 2 to client, and again when the client received the message 2 send a new message 3 to server and so on. A ping-pong as fast as the connection (hops, latency) can be (2 vms using their dedicated virtual adapter should be quite fast). ---------------------------------------------------------------------- rvk · 2022-07-03 17:17 Quote from: prodingus on July 03, 2022, 07:13:17 pmThe problem with the buffer is this: I wanted to make the client send message 1 to the server, then when the server received the message 1, send a message 2 to client, and again when the client received the message 2 send a new message 3 to server and so on. A ping-pong as fast as the connection (hops, latency) can be (2 vms using their dedicated virtual adpater should be quite fast). Then why are you using UDP and not TCP connection? You can simulate it by sending an UDP message back and only send the next message when you get the ACK (acknowledgement) back but that will slow down things. A direct TCP connection seems much more appropriate for your description. BTW. You increase the I after sending but before printing to screen so you actually didn't send that last printed number. ---------------------------------------------------------------------- Warfley · 2022-07-03 17:17 Quote from: prodingus on July 03, 2022, 06:22:27 pmThe problem 1 is that the server keeps receiving even after I stop client, and 2 the received messages stop at 10 (displays only 1 and for 11+ displays nothing). The +5 was just for check, no other reason. That is the nature of UDP. UDP is connectionless, meaning you just receive datagrams, no matter from which client and what the client state is, and also it is unreliable, meaning that data can get lost. If you want to know when the client disconnects, and you require reliable transmission, you shouldn't use UDP ---------------------------------------------------------------------- prodingus · 2022-07-03 17:24 QuoteThen why are you using UDP and not TCP connection? I am going to try the ping-pong. I want to use udp instead of tcp because I want to have as low lag as possible. What I am going to do if a packet is lost is up to me to make or not a custom ack/resend. Quoteou can simulate it by sending an UDP message back and only send the next message when you get the ACK (acknowledgement) back but that will slow down things.Does udp return ack by itself? QuoteBTW. You increase the I after sending but before printing to screen so you actually didn't send that last printed number. AGAIN, correct!!! QuoteIf you want to know when the client disconnects, and you require reliable transmission, you shouldn't use UDP I don't mind that (at least yet). ---------------------------------------------------------------------- rvk · 2022-07-03 17:30 Quote from: prodingus on July 03, 2022, 07:24:32 pmDoes udp return ack by itself? No, that's the whole point of UDP. No return message and no guarantee someone receives the message. The upside is that you can also send an UDP message to the whole network (broadcast) without knowing any of their IPs. Receivers can answer that they received it (via return UDP message or TCP connection request). ---------------------------------------------------------------------- prodingus · 2022-07-03 17:35 QuoteNo, that's the whole point of UDP. No return message and no guarantee someone receives the message.That's what I thought, I missinterpreted what you wrote before. QuoteThe upside is that you can also send an UDP message to the whole network (broadcast) without knowing any of their IPs. Receivers can answer that they received it (via return UDP message or TCP connection request).I am aware of that. Back to the buffer thing, let's say I want the packets to be droped if the server does not receive them in time. How to do that? ---------------------------------------------------------------------- rvk · 2022-07-03 17:39 Quote from: prodingus on July 03, 2022, 07:35:52 pmBack to the buffer thing, let's say I want the packets to be droped if the server does not receive them in time. How to do that? Send a timestamp with the UDP message from the sender and ignore it on the receiving end if it's too old. That would mean that both computers need to be exact in timesettings. So, what us your definition of "receive them in time" ? ---------------------------------------------------------------------- prodingus · 2022-07-03 17:47 Timestamp will add small latency to the program. Receive them in time = receive when the datagram has arrived at the destination and the program reads it. I might have wrong assumptions on how things work. I thought that a udp datagram arrives, if you read it fast, then good, else the packet is gone/replaced by the next one. I set the server to sleep(1000), I still received all the datagrams. ---------------------------------------------------------------------- Warfley · 2022-07-03 17:52 There is more than just UDP and TCP, there are things in between TCP is a reliable ordered stream of data. UDP is for transmitting unreliable unordered packages. There is also RDM (Reliable Delivered Messages), which sends acks and has therefore delivery guarantees, but does not guarantee order, or SEQPACKET (Sequenced Packet), which also ensures ordering and is like TCP also connection based, but unlike TCP it does not provide a stream of data but only single packets, and therefore is more lightweight. Depending on how much guarantees you need and also how much overhead they imply, they can be useful choices as in-between options. ---------------------------------------------------------------------- Warfley · 2022-07-03 17:55 Quote from: prodingus on July 03, 2022, 07:47:37 pmTimestamp will add small latency to the program. Receive them in time = receive when the datagram has arrived at the destination and the program reads it. I might have wrong assumptions on how things work. I thought that a udp datagram arrives, if you read it fast, then good, else the packet is gone/replaced by the next one. I set the server to sleep(1000), I still received all the datagrams. If you are requiring these timing guarantees that the delay of checking a timestamp is crucial, then forget about working with the socket api all together. You either need to use user-space networking, or write your code as a kernel module. Also you will probably run into scheduling problems as you don't use a real-time operating system ---------------------------------------------------------------------- MarkMLl · 2022-07-03 17:57 Quote from: rvk on July 03, 2022, 07:30:19 pmThe upside is that you can also send an UDP message to the whole network (broadcast) without knowing any of their IPs. Receivers can answer that they received it (via return UDP message or TCP connection request). Careful there. Some of this might be OS-specific, but generally speaking UDP broadcasts don't go through routers unless there's a special proxy or similar. The overwhelming reason for UDP- apart from efficiency- is that (small) messages are received without artificial fragmentation or concatenation. MarkMLl ---------------------------------------------------------------------- prodingus · 2022-07-03 18:04 QuoteThere is more than just UDP and TCP, there are things in between TCP is a reliable ordered stream of data. UDP is for transmitting unreliable unordered packages. There is also RDM (Reliable Delivered Messages), which sends acks and has therefore delivery guarantees, but does not guarantee order, or SEQPACKET (Sequenced Packet), which also ensures ordering and is like TCP also connection based, but unlike TCP it does not provide a stream of data but only single packets, and therefore is more lightweight. Depending on how much guarantees you need and also how much overhead they imply, they can be useful choices as in-between options. Nice to know all these info. Of course these can be done manually in order to fit the needs of the program. QuoteIf you are requiring these timing guarantees that the delay of checking a timestamp is crucial, then forget about working with the socket api all together. You either need to use user-space networking, or write your code as a kernel module. Also you will probably run into scheduling problems as you don't use a real-time operating systemThis goes deep into the rabbithole... ---------------------------------------------------------------------- rvk · 2022-07-03 18:19 Quote from: MarkMLl on July 03, 2022, 07:57:27 pmQuote from: rvk on July 03, 2022, 07:30:19 pmThe upside is that you can also send an UDP message to the whole network (broadcast) without knowing any of their IPs. Receivers can answer that they received it (via return UDP message or TCP connection request). Careful there. Some of this might be OS-specific, but generally speaking UDP broadcasts don't go through routers unless there's a special proxy or similar.With whole network I meant if course internal network. For example an internal messaging system. Quote from: prodingus on July 03, 2022, 07:47:37 pmTimestamp will add small latency to the program.This could be as little as 4 bytes. So not really any problem considering the UDP is at least package size large. Quote from: prodingus on July 03, 2022, 07:47:37 pmI thought that a udp datagram arrives, if you read it fast, then good, else the packet is gone/replaced by the next one. I set the server to sleep(1000), I still received all the datagrams. That's just how it works. An UDP package can even arive seconds later depending on network circumstances, buffering and OS priority. If that's unacceptable you can send a few bytes extra (it really doesn't take any more space in a package) or leave the concept of UDP behind. Quote from: prodingus on July 03, 2022, 07:24:32 pmQuoteThen why are you using UDP and not TCP connection? I am going to try the ping-pong. I want to use udp instead of tcp because I want to have as low lag as possible. What I am going to do if a packet is lost is up to me to make or not a custom ack/resend.True but in that case you do need to handle receiving messages faster than the sender is sending them. You are not doing that. You can just flush the buffer after receiving a message. In that case you (might) loose messages but you are more certain the message you do get are more recent. ---------------------------------------------------------------------- prodingus · 2022-07-03 18:30 QuoteThat's just how it works. An UDP package can even arive seconds later depending on network circumstances, buffering and OS priority. If that's unacceptable you can send a few bytes extra (it really doesn't take any more space in a package) or leave the concept of UDP by behind. QuoteTrue but in that case you do need to handle receiving messages faster than the sender is sending them. You are not doing that. You can just flush the buffer after receiving a message. In that case you (might) loose messages but you are more certain the message you do get are more recent. The whole point of the sleep(100) and sleep(1000) in the server was to simulate the "packet had arrived and time passed since then so you lost it". It doesn't happen (!). Of course the messages are very small, but unless there is something wrong with the code examples, it seems that udp is not behaving as udp ---------------------------------------------------------------------- prodingus · 2022-07-03 18:38 I just come across this nice post and I am going to drop it here: https://stackoverflow.com/questions/7843644/how-long-does-a-udp- packet-stay-at-a-socket So to reply to my last post, udp is acting as udp ---------------------------------------------------------------------- rvk · 2022-07-03 18:38 Quote from: prodingus on July 03, 2022, 08:30:10 pmOf course the messages are very small, but unless there is something wrong with the code examples, it seems that udp is not behaving as udp  No, it's not behaving like your notion of UDP. But your notion is wrong  The buffers are there to make UDP more reliable. There is nothing in UDP protocol that says messages can't be buffered ir received out of order or after a delay. Some reading https://jvns.ca/blog/2016/08/24/find-out-where-youre- dropping-packets/ You could just read the buffer until it's empty and ONLY use the last message. In that case you get what you want (or expected). The discarded messages are 'lost'. ---------------------------------------------------------------------- prodingus · 2022-07-03 18:40 My bad, should have searched before my post #64. ---------------------------------------------------------------------- prodingus · 2022-07-03 19:03 I would like to thank everyone in this thread for their contribution and help! I am sure that other dinguses, I mean noobs, I mean newbies would find all that info usefull. Let it be no doubt that I did search the references and examples, but they are not always easy to understand (but that's just me). For the sake of the original post regarding the lnet and the handles, If someone can make an example without using classes, it would be welcome, for I know that you can write things with many different ways. Again, thank you guys! Ps I am going to test all the examples in this thread, I just need time! ---------------------------------------------------------------------- rvk · 2022-07-03 19:54 Quote from: prodingus on July 03, 2022, 09:03:22 pmFor the sake of the original post regarding the lnet and the handles, If someone can make an example without using classes, it would be welcome, for I know that you can write things with many different ways.Like it was already said... you can't use lnet without classes. But I'm curious. You say you want asynchronous communication... Do you know what that means? Now you have a program with just one flow. Line by Line. You can't achieve asynchrony that way. QuoteAsynchronous programming is a form of parallel programming that allows a unit of work to run separately from the primary application thread. When the work is complete, it notifies the main thread (as well as whether the work was completed or failed). There are numerous benefits to using it, such as improved application performance and enhanced responsiveness.https://stackify.com/when-to-use-asynchronous- programming So if you want to do asynchronous communication, you need to do some form of multiple program flow. Coming back to your original lnet server (receiver). Because lnet is event driven you DO need to signal it to do it's work in the background (well not actually background but you need to let it do it's work). You can do that with just one line. Sock.CallAction. So if you add line 13 to your original program, it will work. (Note that this isn't real asynchronous communication though. You're just misusing the library to force synchronous communications.) Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i program test1;{$IFDEF FPC}{$MODE DELPHI}{$ENDIF}{$APPTYPE CONSOLE}uses  SysUtils, crt, lnet, lnetbase;var  Sock: TLUDP;  tmp: string;begin  Sock := TLUdp.Create(nil);  Sock.listen(1234);  repeat    Sock.CallAction;    if Sock.GetMessage(tmp) > 0 then      writeln(tmp);    sleep(100);  until False;end. Bare in mind... this is not the actual correct way to do things. You are still trying to do a GetMessage while there might not even be a message on the line. This could block the CallAction again and I'm not sure the code will work correctly. The correct way would be to use the sock.OnReceive because then you know there is a message to receive. Did you look at the example ludp.pp in \config\onlinepackagemanager\packages\lnet\examples\console\ludp ?? It has a complete server/client udp example with a TLUDPTest class. So the correct way to do it would be something like this: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i program test1;{$IFDEF FPC}{$MODE DELPHI}{$ENDIF}{$APPTYPE CONSOLE}uses  SysUtils, crt, lnet, lnetbase;var  Sock: TLUDP;  tmp: string; type  TMyDummy = class    class procedure OnReceive(aSocket: TLSocket);  end; class procedure TMyDummy.OnReceive(aSocket: TLSocket);begin  if Sock.GetMessage(tmp) > 0 then    writeln(tmp);end; begin  Sock := TLUdp.Create(nil);  Sock.OnReceive := TMyDummy.OnReceive;  Sock.listen(1234);  repeat    Sock.CallAction;    sleep(100);  until False;end. You see, now when there is no message, CallAction keeps getting called. When there is a message the OnReceive from TMyDummy is called and the message is read and printed. In the example ludp.pp you see you can also send a reply back. (The TMyDummy can be called something else of course, this is just to show you can use classes, and class procedures, without creating an instance. But that might be something too advanced for now  ) ---------------------------------------------------------------------- prodingus · 2022-07-04 20:18 QuoteBut I'm curious. You say you want asynchronous communication... Do you know what that means? Now you have a program with just one flow. Line by Line. You can't achieve asynchrony that way. I read here about it and thougt it will be a good feature. Quote(Note that this isn't real asynchronous communication though. You're just misusing the library to force synchronous communications.)I hope you mean ..misusing the library to force asynchronous communications.. Let's just leave the sync/async for now, as the Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i program test1;{$IFDEF FPC}{$MODE DELPHI}{$ENDIF}{$APPTYPE CONSOLE}uses  SysUtils, crt, lnet, lnetbase;var  Sock: TLUDP;  tmp: string;begin  Sock := TLUdp.Create(nil);  Sock.listen(1234);  repeat    Sock.CallAction;    if Sock.GetMessage(tmp) > 0 then      writeln(tmp);    sleep(100);  until False;end.works fine even if there is no messages. I just tried it and it works (at least in the testing; if any problem arrives I'll let you know). QuoteDid you look at the example ludp.pp in \config\onlinepackagemanager\packages\lnet\examples\console\ludp ?? It has a complete server/client udp example with a TLUDPTest class.Yes, but I find it "difficult" because of the classes. Your latest example seems quite easier to understand and use (and more compact) Side question: as this works (even if it might have problems that I don't see in this stage of the testing): Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i rogram pointer_test; var        p: pointer; procedure a; var        x:uint64;begin        x:=5;        p:=@x;end; begin        a;         writeln( uint64(p^));        readln; end.   this doesn't: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i program multi_1;uses        SysUtils, crt, lnet;var         tmp: string;      psock:^TLUDP; procedure connect(port:integer);var        Sock: TLUDP;begin         Sock := TLUdp.Create(nil);        psock:=@sock;      Sock.listen(port);end; begin     connect(1234);     repeat        pSock^.CallAction;        pSock^.GetMessage(tmp);        writeln(tmp + ' ' + pSock^.host);        sleep(500);    until False;end. Also, I tried various ways to get the host but no success (it can be implemented inside the meagges) ---------------------------------------------------------------------- rvk · 2022-07-04 20:32 Quote from: prodingus on July 04, 2022, 10:18:02 pmQuoteDid you look at the example ludp.pp in \config\onlinepackagemanager\packages\lnet\examples\console\ludp ?? It has a complete server/client udp example with a TLUDPTest class.Yes, but I find it "difficult" because of the classes. Your latest example seems quite easier to understand and use (and more compact) As your programs get more advanced you eventually will need to go to procedures and classes. Quote from: prodingus on July 04, 2022, 10:18:02 pmas this works (even if it might have problems that I don't see in this stage of the testing): Your first example works but is very dangerous. You have a pointer to some variable that doesn't exist anymore and can get overwritten at any time. You second example has that same problem that the sock variable is local and is overwritten once you exit the procedure. So you shouldn't use pointer p anymore. It's better to declare sock as global variable in this case. (or in case of a class you can define it as local class variable/property) Quote from: prodingus on July 04, 2022, 10:18:02 pmAlso, I tried various ways to get the host but no success (it can be implemented inside the meagges) Isn't the ip address of the sender in aSocket.PeerAddress in the OnReceive event? Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i program project1;{$IFDEF FPC}{$MODE DELPHI}{$ENDIF}{$APPTYPE CONSOLE}uses  SysUtils, crt, lnet, lnetbase;var  Sock: TLUDP;  tmp: string; type  TMyDummy = class    class procedure OnReceive(aSocket: TLSocket);  end; class procedure TMyDummy.OnReceive(aSocket: TLSocket);begin  if Sock.GetMessage(tmp) > 0 then    writeln(tmp + ' from ' + aSocket.PeerAddress);end; begin  Sock := TLUdp.Create(nil);  Sock.OnReceive := TMyDummy.OnReceive;  Sock.listen(1234);  repeat    Sock.CallAction;    sleep(100);  until False;end. ---------------------------------------------------------------------- prodingus · 2022-07-04 20:56 QuoteIsn't the ip address of the sender in aSocket.PeerAddress in the OnReceive event?I was trying to add it in your first of the examples. I stumbled uppon it but couldn't find any reference of the lifetime of local variables for pascal (missed it?) but for other languages (they gone as the function ends?). In the pointer test this *seems* to not be the case; even adding sleep it gives output. The solution is obvious as you said, but less compact/neat (thats what I was trying to avoid). Is there any "control" over lifetime? ---------------------------------------------------------------------- rvk · 2022-07-04 21:08 Quote from: prodingus on July 04, 2022, 10:56:43 pmIn the pointer test this *seems* to not be the case; even adding sleep it gives output.I can guarantee that this will go wrong at some point. And so it did in your second example with sock  With just one integer as variable it might go ok for one procedure but if your program gets more complex it WILL definitely go wrong. Quote from: prodingus on July 04, 2022, 10:56:43 pmThe solution is obvious as you said, but less compact/neat (thats what I was trying to avoid). Is there any "control" over lifetime? No. If you define a local variable... that's just what it is... local. You shouldn't access it outside of that procedure. The "control" you have over the lifetime is if you make the variable global Why do you think it is less neat? You also already have a pointer p which you defined global. Why not just define Sock: TLUDP global instead of the pointer p??? BTW. TLUDP is already a class. So any variable Sock: TLUDP is already a pointer. What you are doing with psock: ^TLUDP is creating a pointer to a pointer (class variable). Really not needed and inefficient. Sock doesn't take any more memory than psock does. So this would work just fine: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i program multi_1; uses  SysUtils, crt, lnet; var  tmp: string;  Sock: TLUDP;   procedure connect(port: integer);  begin    Sock := TLUdp.Create(nil);    Sock.listen(port);  end; begin  connect(1234);  repeat    Sock.CallAction;    Sock.GetMessage(tmp);    writeln(tmp + ' ' + Sock.host);    sleep(500);  until False;end. You can also do something like this: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i program multi_1; uses  SysUtils, crt, lnet;   function CreateSockAndConnect(port: integer): TLUDP;  var    localSock: TLUDP;  begin    localSock := TLUdp.Create(nil);    localSock.listen(port);  Result := localSock;  end; var  Sock: TLUDP;  tmp: string; begin  Sock := CreateSockAndConnect(1234);  repeat    Sock.CallAction;    Sock.GetMessage(tmp);    writeln(tmp + ' ' + Sock.host);    sleep(500);  until False;end. ---------------------------------------------------------------------- prodingus · 2022-07-04 21:17 You are (again) correct. The local variable lives for one repeat loop, for loop, even if you copy-paste the lines multiple times, then it zeroes in my pointer test.  I just try to avoid the main "var" hell that I am getting every time that I start a project. ---------------------------------------------------------------------- rvk · 2022-07-04 21:20 Quote from: prodingus on July 04, 2022, 11:17:21 pmI just try to avoid the main "var" hell that I am getting every time that I start a project. That's why you can work with procedures and functions. Look at my last example (added it in an edit) passing sock as function result. You can put everything in the main loop in a separate procedure main and call it right after the main-begin. Then you have zero global variables. ---------------------------------------------------------------------- MarkMLl · 2022-07-04 21:29 Quote from: prodingus on July 04, 2022, 11:17:21 pmYou are (again) correct. The local variable lives for one repeat loop, for loop, even if you copy-paste the lines multiple times, then it zeroes in my pointer test.  I just try to avoid the main "var" hell that I am getting every time that I start a project. I think you need to explain what you mean by 'main "var" hell' in this context. I think it's worth throwing in here that Pascal's idea of local variables is shared by most languages, so please don't feel that the Pascal language (or the FPC implementation) is being obstructive. There might possibly be a way to make a local variable externally visible by making it static and returning a pointer, but that would be excruciatingly bad practice so please don't ask me to explain further :-) Global variables have an undeservedly poor reputation, but that is mainly because novice programmers tend to overuse them. In practical terms, even the Delphi and Lazarus IDEs use them heavily despite there being preferable workarounds. MarkMLl ---------------------------------------------------------------------- prodingus · 2022-07-04 22:08 QuoteThat's why you can work with procedures and functions. Look at my last example (added it in an edit) passing sock as function result. You can put everything in the main loop in a separate procedure main and call it right after the main-begin. Then you have zero global variables.Very neat example! . And just for the sake of the question: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i program pointer_test;uses sysutils, cmem;var        p: pointer; procedure a; var        x:pointer;begin        x:=getmem(1);    integer(x^):=5;    p:=x;end; begin         a;    repeat        sleep(100);              writeln( uint64(p^));    until false;        readln; end. what could possibly go wrong?? QuoteI think you need to explain what you mean by 'main "var" hell' in this contextmany vars that are common for procedures/functions. QuoteI think it's worth throwing in here that Pascal's idea of local variables is shared by most languages, so please don't feel that the Pascal language (or the FPC implementation) is being obstructive.absolutelly not! QuoteThere might possibly be a way to make a local variable externally visible by making it static and returning a pointer, but that would be excruciatingly bad practice so please don't ask me to explain further :-) Global variables have an undeservedly poor reputation, but that is mainly because novice programmers tend to overuse them. In practical terms, even the Delphi and Lazarus IDEs use them heavily despite there being preferable workarounds. I am just exploring possible options. ---------------------------------------------------------------------- rvk · 2022-07-04 22:38 Quote from: prodingus on July 05, 2022, 12:08:49 amwhat could possibly go wrong?? That example should work fine. That's because you create something in memory and copy the pointer to another pointer. The first example you had, you had a variable which disapears after the procedure and your pointer is hanging. In this example the pointer still points to some memory thats created and not released. So perfectly fine. BTW. Your original second example with psock only went wrong because you had a pointer to a pointer of which the first pointer went away after the procedure. If you did a psock := sock it would have worked (that's what i did in my example but as a function result). You really need to realize that TLUDPL is a class and variables of classes are just pointers to an instance of that class. So sock := TLUDPL.Create will create a pointer to a TLUDPL instance. It's redundant to create a pointer to Sock because Sock is already a pointer. So doing pSock := Sock would have worked because although Sock, as pointer itself, is destroyed, the memory where it points to wasn't released and pSock would have still been valid. But doing pSock := @Sock (which you did) was wrong because that points to a pointer (which is a pointer itself) which will be destroyed after the procedure. I hope I explained this a bit clearly Eventually you will continue to work more and more with procedures and functions and finally you will also grasp the notion of classes There was a website which goes over the basics of object pascal. Maybe it's some interesting reading for you. Modern Object Pascal Introduction for Programmers ---------------------------------------------------------------------- prodingus · 2022-07-06 18:37 QuoteSo sock := TLUDPL.Create will create a pointer to a TLUDPL instance.Interesting! I never thougt of it as pointer. QuoteI hope I explained this a bit clearly Yes, very clearly, but one small issue.. psock:=sock; does not work because sock is acting as a normal variable. Even casting it as pointer, psock:=pointer(sock); is not working.. Do I miss something here? ---------------------------------------------------------------------- rvk · 2022-07-06 18:39 Quote from: prodingus on July 06, 2022, 08:37:58 pmpsock:=sock; does not work because sock is acting as a normal variable. Even casting it as pointer, psock:=pointer(sock); is not working.. Do I miss something here? What type is psock? If you make psock of type TLUDP (because TLUDP is a pointer by itself) it will work fine. You don't want a pointer to a TLUDP because that's just silly. It has no advantages. ---------------------------------------------------------------------- prodingus · 2022-07-06 18:47 Yes! psock was ^tludp (because I though of it is as variable.) Changing from the previous multi_1 just for the sake of the discussion: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i program multi_1;uses        SysUtils, crt, lnet; //, lnetbasevar         tmp: string;        psock:TLUDP; procedure connect(port:integer);var      Sock: TLUDP;begin         Sock := TLUdp.Create(nil);    psock:=sock;  Sock.listen(port); end;  begin     connect(1234);         repeat      pSock.CallAction;        pSock.GetMessage(tmp);        writeln(tmp + ' ' + pSock.host);        sleep(500);        until False;end. It works! Needless to say thanks again for the insights! ---------------------------------------------------------------------- prodingus · 2022-07-07 21:18 @Warfley, I just tried to use your unit. It does not compile on my setup (lazarus 1.6, FPC 3.0, SVN 51630), I get various errors, unless there is something wrong in my configurations. I have no time to "upgrade" my lazarus build, so I won't be able to share any experience. @rvk, Before trying your other examples, is it possible to get the host (sender) address on the last example I posted? ---------------------------------------------------------------------- rvk · 2022-07-07 21:46 Quote from: prodingus on July 07, 2022, 11:18:47 pm@rvk, Before trying your other examples, is it possible to get the host (sender) address on the last example I posted? Yes, it's the same as in the example I posted here https://forum.lazarus.freepascal.org/index.php/topic,59777.msg446588.h tml#msg446588 You already have it there in your writeln as pSock.host. ---------------------------------------------------------------------- prodingus · 2022-07-07 21:59 it's not working. ---------------------------------------------------------------------- rvk · 2022-07-07 22:36 Quote from: prodingus on July 07, 2022, 11:59:54 pmit's not working. Ok. I'm not sure if it works with just the getmessage. It was working with the onreceive event with the aSocket parameter. Like this Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i program project1;{$IFDEF FPC}{$MODE DELPHI}{$ENDIF}{$APPTYPE CONSOLE}uses  SysUtils, crt, lnet, lnetbase;var  Sock: TLUDP;  tmp: string; type  TMyDummy = class    class procedure OnReceive(aSocket: TLSocket);  end; class procedure TMyDummy.OnReceive(aSocket: TLSocket);begin  if Sock.GetMessage(tmp) > 0 then    writeln(tmp + ' from ' + aSocket.PeerAddress);end; begin  Sock := TLUdp.Create(nil);  Sock.OnReceive := TMyDummy.OnReceive;  Sock.listen(1234);  repeat    Sock.CallAction;    sleep(100);  until False;end. I'm not sure if it's possible without the event. I can't check that at the moment. Maybe tomorrow. ---------------------------------------------------------------------- rvk · 2022-07-08 10:35 You need to access the TLSocket to get to the host-name of the sender. TLUDPL van handle multiple connections. Normally the correct TLSocket is passed to the OnReceive function. But because you are just calling GetMessage outright... you don't know which connection it is. If you only have one connection you can use the following line (just taking the first TLSocket in TUDPL): Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i     writeln(tmp + ' ' + Sock.Socks[0].PeerAddress); But if you have multiple connection this gets problematic. In that case you definitely need to go with my other example implementing the OnReceive event. ---------------------------------------------------------------------- prodingus · 2022-07-08 17:39 Amen man, it works!!! However I got into two problems: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i program multi_3;uses        winsock, SysUtils, lnet, crt, Classes, StreamIO, Windows, strutils;var        buff: string;    psock:TLUDP;    psock2:TLUDP;       //tludp is pointer type by itself    i: integer= 0;    j: integer= 0;    k: integer= 0;    myip: string;    msg: string; type        TConsoleOutputStream = class(TStream)    function Write(const Buffer; Count: longint): longint; override;end; function TConsoleOutputStream.Write(const Buffer; Count: longint): longint;var    charcount: longword=0;begin    WriteConsole(GetStdHandle(STD_OUTPUT_HANDLE), @Buffer, Count, charcount, nil);    Result := charcount;end; function getlocalip:string;type  pu_long = ^u_long;var  varTWSAData : TWSAData;  varPHostEnt : PHostEnt;  varTInAddr : TInAddr;  namebuf : Array[0..255] of char;begin  If WSAStartup($101,varTWSAData) <> 0 Then  Result := 'No. IP Address'  Else Begin    gethostname(namebuf,sizeof(namebuf));    varPHostEnt := gethostbyname(namebuf);    varTInAddr.S_addr := u_long(pu_long(varPHostEnt^.h_addr_list^)^);    Result := inet_ntoa(varTInAddr);  End;  WSACleanup;end;  procedure connectr(port:integer);var        Sock: TLUDP;begin        Sock := TLUdp.Create(nil);    psock:= sock;    Sock.listen(port);end; procedure connects(addr:string; port:integer);var        Sock2: TLUDP;begin        Sock2 := TLUdp.Create(nil);    psock2:= sock2;    Sock2.connect(addr, port);end;  begin        AssignStream(Output, TConsoleOutputStream.Create);        Rewrite(Output);    crt.WindMaxY := 9999;    myip:= getlocalip;    i:=0;    writeln('ip: ',myip);      write('send or listen?');        readln(k);     if k=1 then begin      connects('1.1.1.10', 1234);        msg:='ping';        psock2.sendmessage(msg);    end;     begin        connectr(1234);              repeat                pSock.CallAction;                pSock.GetMessage(buff);            if buff <> '' then begin              if {(i mod 100 = 0)} true then writeln(buff,' ',pSock.Socks[0].PeerAddress);                         connects(pSock.Socks[0].PeerAddress, 1234);                    inc(i);                                msg:=inttostr(i);                              psock2.sendmessage(msg);                    buff:='';            end;                delay(0);                until keypressed=true;    end;     readln;end. The first is the program hangs when it sends 16381 messages (I'm sure that number is not "random"). The second and most ridiculous: sleep(1) cause the program to run extremelly slow, almost no CPU utilization, whereas no sleep at all uses 100% the cpu (even with less screen updates). ---------------------------------------------------------------------- rvk · 2022-07-08 18:12 Your first problem.... You do a connects(pSock.Socks[0].PeerAddress, 1234); EACH time you get a message. But you don't close that connection. So you end up with A LOT of connections. You need to free psock after you send a message otherwise you end up with lots of TLUDPs in memory which you can't even use because pointer psock keeps getting overwritten. Try to put a psock.free at the end of the if buf <> '' block. (Best would be to use a try/finally structure but maybe that's for later.) Your second problem... Of course you have 100% cpu if you do sleep(1) or no sleep. What did you expect. Your program doesn't pause so it takes all the cpu. That's logical. ---------------------------------------------------------------------- prodingus · 2022-07-08 18:24 With sleep(1) the program acts like having sleep(500) for example ---------------------------------------------------------------------- rvk · 2022-07-08 18:29 Quote from: prodingus on July 08, 2022, 08:24:01 pmWith sleep(1) the program acts like having sleep(500) for example How fast is the sender sending? And if there (in the sender) is a same loop of receiving and sending it could be that they are waiting on each other In that case you need to show both sender and receiver code. (And at some point it becomes better to switch to event handling or threads.) ---------------------------------------------------------------------- prodingus · 2022-07-08 18:43 With sleep(0) or no sleep at all, it is very very very fast. Insert even the lowest possible, sleep(1), the program runs very very very slow. ---------------------------------------------------------------------- prodingus · 2022-07-08 18:51 Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i begin      connectr(1234);        repeat                pSock.CallAction;              pSock.GetMessage(buff);            if buff <> '' then begin              if {(i mod 100 = 0)} true then writeln(buff,' ',pSock.Socks[0].PeerAddress);                        connects(pSock.Socks[0].PeerAddress, 1234);                    psock2.sendmessage(msg+' pong'); //needed here too, else the sender is not geting reply            end;                delay(1);              until buff <> '';                 repeat                pSock.CallAction;                pSock.GetMessage(buff);            if buff <> '' then begin                if {(i mod 100 = 0)} true then writeln(buff,' ',pSock.Socks[0].PeerAddress);                        //connects(pSock.Socks[0].PeerAddress, 1234);                    inc(i);                                msg:=inttostr(i);                              psock2.sendmessage(msg+' pong');            end;              delay(1);                until keypressed=true;    end;As you said, connects once fixed the 16282 problem! ---------------------------------------------------------------------- prodingus · 2022-07-08 18:59 QuoteIn that case you need to show both sender and receiver code.the program is both sender/receiver. ---------------------------------------------------------------------- rvk · 2022-07-08 21:00 For me there is a difference between no delay, delay(1) and delay(500). But the whole thing is really iffy (unstable). If one package is missed, the whole thing stops. (you only send a first ping from the sender but wait until you get a response pong to send the next pong to the receiver). That's the downside of synchronous programming (what you are doing now). If you would use asynchonous, things could be handled differently. (I don't know if they would become faster though). Here is a version with events OnReceive. BTW, you can just use the same aSocket in OnReceive to send a message back to that same host. So there is no need to open a second socket for listening and sending. (I used GetHostIP from lcommon to get the IP. For me it's giving the wrong IP because I have multiple virtual adapters. You code for getlocalip also gave the wrong IP) Ultimately the sender would send out a broadcast over the entire internal network and any listener could latch on and begin playing  ) But that's for later. Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i program pingpong; uses  SysUtils,  lnet, lnetbase, lcommon,  crt, Classes, StreamIO, Windows; var  Sock: TLUDP;  i: integer = 0;  k: integer; type  TConsoleOutputStream = class(TStream)    function Write(const Buffer; Count: longint): longint; override;  end;   function TConsoleOutputStream.Write(const Buffer; Count: longint): longint;  var    charcount: longword = 0;  begin    WriteConsole(GetStdHandle(STD_OUTPUT_HANDLE), @Buffer, Count, charcount, nil);    Result := charcount;  end; type  TMyDummy = class    class procedure OnReceive(aSocket: TLSocket);  end;   class procedure TMyDummy.OnReceive(aSocket: TLSocket);  var    tmp: string;  begin    if aSocket.GetMessage(tmp) > 0 then    begin      writeln(tmp + ' from ' + aSocket.PeerAddress);      Inc(i);      tmp := IntToStr(i);      if k = 1 then        aSocket.sendmessage(tmp + ' ping')      else        aSocket.sendmessage(tmp + ' pong');    end;  end; begin  AssignStream(Output, TConsoleOutputStream.Create);  Rewrite(Output);  crt.WindMaxY := 9999;  i := 0;   writeln('ip: ', GetHostIP(''));  Write('1 = send or 2 = listen? ');  readln(k);   if k = 1 then  begin    Sock := TLUdp.Create(nil);    Sock.Connect('192.168.2.11', 1234); // <-- change the receiver IP here once    Sock.OnReceive := @TMyDummy.OnReceive;    Sock.sendmessage('ping');  end  else  begin    Sock := TLUdp.Create(nil);    Sock.Listen(1234);    Sock.OnReceive := @TMyDummy.OnReceive;  end;   repeat    Sock.CallAction;    delay(0);  until keypressed = True;   writeln('press enter to quit');  readln;end. No delay, delay(0), delay(1) and delay(500) all give me increasing slower flow (which is correct). ---------------------------------------------------------------------- prodingus · 2022-07-08 21:08 After searching and testing, the "slowness" when using sleep(1) seems to be another problem unrelated to this thread, as it is common across other programs too. Even using epik timer .SystemSleep(1) the cpu usage is almost zero (note that the testing is done on vms having one cpu core, so no division per cpu cores/treads) @rvk, you posted at the time I was writing this. Give me some time to process your info. But again, using my example (with all the flaws) without delay is pretty damn fast! Inserting the lowest possible delay it doesn't give me, for example, 90% cpu usage; it gets directly to 1% (cannot/don't-need-to meassure). ---------------------------------------------------------------------- Warfley · 2022-07-09 09:47 There were so many threads about this that I feel confident in saying that Sleep is probably the most misunderstood function in the whole RTL. Sleep(n) does not mean "sleep n milliseconds" it means "sleep at least n milliseconds". It basically tells the scheduler, take me of the CPU and don't reschedule me for at least this amount of time. The thing is, the scheduler is slow, really slow, and takes at least like 30 millis to reschedule. So if you write Slee(1) there is practically no difference to writing Sleep(30) (more like Sleep(50) or even more). So if your program has a loop that takes only a couple of millis to execute, and then there is a Sleep(1), the amount of time it goes to sleep will be orders of magnitude greater than your execution time, resulting in an extremely low CPU usage. The special case here is Sleep(0) because this means "Sleep at least 0 milliseconds", which is interpreted by the OS as a yield, i.e. "if there are other processes waiting to get CPU time, give it to them, otherwise let me continue". So what happens here is that if there are no other processes waiting, you won't get of the CPU and therefore stay at 100% usage, it just won't block the CPU if there are other processes that need it more. So you will fully utilize your CPU but not monopolize it, if that makes sense. Also note that if you need to be reactive within less that 30-50 milliseconds, you cannot sleep, simple as that. You can however just use blocking socket calls, as they are build to exactly do that: if nothing has arrived, don't use the CPU and as soon as something arrives wake up the process and handle it ---------------------------------------------------------------------- 440bx · 2022-07-09 13:07 Quote from: Warfley on July 09, 2022, 11:47:11 amSleep(n) does not mean "sleep n milliseconds" it means "sleep at least n milliseconds". It basically tells the scheduler, take me of the CPU and don't reschedule me for at least this amount of time. Correct. Quote from: Warfley on July 09, 2022, 11:47:11 amThe thing is, the scheduler is slow, really slow, and takes at least like 30 millis to reschedule. It's not about the scheduler being slow (it isn't), it's about what else is happening in the system.  Quote from: Warfley on July 09, 2022, 11:47:11 amSo if you write Slee(1) there is practically no difference to writing Sleep(30) (more like Sleep(50) or even more). That is not the case at all.  There is a _big_ difference between Sleep(1), Sleep(30) and Sleep(50).  This is really easy to test. Quote from: Warfley on July 09, 2022, 11:47:11 am So if your program has a loop that takes only a couple of millis to execute, and then there is a Sleep(1), the amount of time it goes to sleep will be orders of magnitude greater than your execution time, resulting in an extremely low CPU usage. It will be greater, usually about 30% but, definitely _not_ "orders of magnitue greater". Quote from: Warfley on July 09, 2022, 11:47:11 amThe special case here is Sleep(0) because this means "Sleep at least 0 milliseconds", which is interpreted by the OS as a yield, i.e. "if there are other processes waiting to get CPU time, give it to them, otherwise let me continue". That's what the documentation says but, the reality is very different.  Easiest way for a process to hog the CPU is to be in a loop of Sleep(0) and, suddenly there no other processes in the system that have any work to do.  Sleep(0) is something that should _never_ be used.  The documentation has been wrong since the days of NT4 and it's quite likely still wrong for Win11 (It's factually testable to be wrong up to and including Win7 SP1). Quote from: Warfley on July 09, 2022, 11:47:11 amSo what happens here is that if there are no other processes waiting, you won't get of the CPU and therefore stay at 100% usage, and that happens _even_ if there are other processes waiting.  Easiest way to mess Windows up is to have as many threads as there are cores running high priority sleep(0) loops.  Don't take my word for it, _try it_!!. ---------------------------------------------------------------------- Warfley · 2022-07-09 14:21 Quote from: 440bx on July 09, 2022, 03:07:38 pmIt will be greater, usually about 30% but, definitely _not_ "orders of magnitue greater". Wrong! Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i program sleeptimetest; {$Mode ObjFPC}{$H+} uses  sysUtils; procedure Spin(time: QWord);var  startTime: QWord;begin  startTime := GetTickCount64;  while GetTickCount64 - startTime < time do;end; var  startTime: QWord;begin  startTime := GetTickCount64;  Spin(2);  sleep(1);  Writeln('Time: ', GetTickCount64 - startTime, ' ms');end.Workload of 2 milliseconds, total time measured on my machine 32 milliseconds. It's a factor of 16, not 1.3. And any factor > 10 are by definition orders of magnitude. Quote from: 440bx on July 09, 2022, 03:07:38 pmThat's what the documentation says but, the reality is very different.  Easiest way for a process to hog the CPU is to be in a loop of Sleep(0) and, suddenly there no other processes in the system that have any work to do.  Sleep(0) is something that should _never_ be used.  The documentation has been wrong since the days of NT4 and it's quite likely still wrong for Win11 (It's factually testable to be wrong up to and including Win7 SP1). Wrong! Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i program sleeptest; {$mode objfpc}{$H+} uses  SysUtils;begin  While True do Sleep(0);end.Running this (and pinning it to CPU nr. 3 on my machine results in 100% load on that CPU as expected. Running it again and also pinning this to the same CPU, now both take 50% of the CPU, so sharing it equally. Introducing a new process that does not use sleep(0): Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i program fullcpuutilization; {$mode objfpc}{$H+} uses  SysUtils;begin  While True do;end.And pinning it to the same CPU results in this process taking 100% of theCPU time and the sleep processes above getting near 0% of that CPU time. It behaves exactly as I described above, when there is no competition, it will utilize the full cpu, if there is competition it will yield time to it with two processes yielding all the time results in 50% for each, and with one process that doesn't yield and always takes the full timeslots, it will completely drown the sleeping processes. Quote from: 440bx on July 09, 2022, 03:07:38 pmEasiest way to mess Windows up is to have as many threads as there are cores running high priority sleep(0) loops. No one was talking about high priority processes, but just for fun, i set the priority of one of the sleep processes to high, and guess what still 0% cpu utlilization. You are just plainly wrong. Quote from: 440bx on July 09, 2022, 03:07:38 pmDon't take my word for it, _try it_!! I don't take your word for it, I know that you are wrong, I did try it, and it shows exactly what I said. We had this same discussion last time, you were wrong then as you are wrong now. The simple 10 line programs show this without any doubt ---------------------------------------------------------------------- 440bx · 2022-07-09 14:35 Quote from: Warfley on July 09, 2022, 04:21:57 pm  startTime := GetTickCount64; That's how Aristotle concluded that the heavier an object is the faster it falls, dropping a rock and a feather. here is a hint:  you really think that the tick count gets updated every CPU clock tick ?  (just in case, the answer is _no_) Here is a piece of advice for anyone who's reading: don't _ever_ use Sleep(0). Sleep(n) will usually be reasonably close to the specified n, _usually_ within 30% (and that's only for small "n"s for larger "n"s it will be much more accurate) on a system which is running smoothly but, could be significantly more than that on a system which is not running smoothly. ---------------------------------------------------------------------- Warfley · 2022-07-09 14:51 Quote from: 440bx on July 09, 2022, 04:35:52 pmyou really think that the tick count gets updated every CPU clock tick ?  (just in case, the answer is _no_) No but it is accurate enough in the range of several milliseconds. But sure, let's use a more accurate clock: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i program sleeptimetest; {$Mode ObjFPC}{$H+} uses  sysUtils, windows; var  freq: Int64; procedure Spin(time: QWord);var  startTime, EndTime: Int64;begin  QueryPerformanceCounter(startTime);  endTime := startTime;  while (endTime - startTime) / freq * 1000 < time do    QueryPerformanceCounter(endTime);end; var  startTime, endTime: Int64;begin  QueryPerformanceFrequency(freq);  QueryPerformanceCounter(startTime);  Spin(2);  sleep(1);  QueryPerformanceCounter(endTime);  Writeln('Time: ', round((endTime - startTime) / freq * 1000), ' ms');end.18 milliseconds, factor of 9, still far away from 30% and around an order of magnitude. Look I'm still right and you are still wrong... Quote from: 440bx on July 09, 2022, 04:35:52 pmHere is a piece of advice for anyone who's readingWritten by the person who was provably wrong about everything so far... ---------------------------------------------------------------------- 440bx · 2022-07-09 14:56 Quote from: Warfley on July 09, 2022, 04:51:21 pm18 milliseconds, factor of 9, still far away from 30% and around an order of magnitude. I see you like to measure things using a rubber-band,  got to admit, good method to get whatever number you want. You should read this: https://forum.lazarus.freepascal.org/index.php/t opic,49585.msg374611.html#msg374611 ---------------------------------------------------------------------- Warfley · 2022-07-09 15:48 Ok here are some more statistics: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i program sleeptimetest; {$Mode ObjFPC}{$H+} uses  sysUtils, windows, math; var  freq: Int64; procedure Spin(time: QWord);var  startTime, EndTime: Int64;begin  QueryPerformanceCounter(startTime);  endTime := startTime;  while (endTime - startTime) / freq * 1000 < time do    QueryPerformanceCounter(endTime);end; function Sleep1Test: Integer;var  startTime, endTime: Int64;begin  QueryPerformanceCounter(startTime);  Spin(2);  sleep(1);  QueryPerformanceCounter(endTime);  Result := round((endTime - startTime) / freq * 1000);end; var  i: Integer;  sum: Integer;  startTime: Int64;  endTime: Int64;  time: integer;  mi: Integer = 9999;  ma: integer = 0;begin  QueryPerformanceFrequency(freq);  QueryPerformanceCounter(startTime);  for i := 0 to 999 do  begin    time := Sleep1Test;    mi := min(mi, time);    ma := max(ma, time);  sum += time;  end;  QueryPerformanceCounter(endTime);  WriteLn('Min: ', mi, ' Max: ', ma, ' Average: ', Round(sum / 1000), 'ms');  WriteLn('Total: ', Round((endTime - startTime) / freq * 1000));end.Results: Code: Text  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i Min: 12 Max: 18 Average: 16msTotal: 15776So my result above of 18 was on the upper end, with an average of 16 it's just a factor of 8 not 9, but still heaps away from 30%. As in your example you linked, I also computed the total time, which is also around 16k ms, which, considering only 2k milliseconds of this is actual workload, also a factor 8. Also I tried to reproduce your results from your link, by replacing the call to "Sleep1Test" with "Sleep(1)", "Sleep(8 )" and "Sleep(16)", the results for the total time are 15758, 15725 and 28692 So I can't reproduce your results. So what can be easiely seen here that the scheduler is working in time units that are somewhere around 18-20ish ms, where you yield at least the rest of your timeslice, which is depending on how much of that slice you already consumed somewhere between 0 and 18 ms, and therefore you could say that the average resolution is 16ms. That does not mean that a sleep will always take at least 16 ms (see minimum above). Also this was on my machine with a 24 (virtual) cpus and nearly no utilization. Therefore in this example the scheduler always gave the next free timeslice to the sleeping process. But that is not necessarily the case, on a system with less cpus and moderate utilization, it could be that you need to wait more than 1 timeslice, 2-3 or so, so when calling sleep, you should always assume that in the worst case it might take 50 ms or even more. The 16 ms average is the absolute best case scenario. So the TL;DR: You where wrong, the results you showed in the link you posted are not reproducible, and what I stated earlier still holds up using your own methodology. PS: tested on windows 10 64 bit ---------------------------------------------------------------------- 440bx · 2022-07-09 16:41 Quote from: Warfley on July 09, 2022, 05:48:37 pmSo I can't reproduce your results. Use the little program I posted and you'll be able to reproduce the results.  It's that simple. BTW, in a multitasking non-RTOS O/S, the best you can do is measure averages which is why the little program I posted repeats the call 1,000 times. ---------------------------------------------------------------------- Warfley · 2022-07-09 16:54 Quote from: 440bx on July 09, 2022, 06:41:27 pmUse the little program I posted and you'll be able to reproduce the results.  It's that simple. Using your exact code, just wrapping some QueryPerformanceCounter around to measure time: Code: Text  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i ms : 1 repeat count 1000Total time: 15.761s ms : 8 repeat count 1000Total time: 15.773s ms : 16 repeat count 1000Total time: 28.928sOh look the same results as before... It's just like, you know, I know what I'm talking about And again, we are talking about an completely underutilized system where the scheduler can assign every timeslot to the same program, this is the absolute best case result ---------------------------------------------------------------------- 440bx · 2022-07-09 17:19 Quote from: Warfley on July 09, 2022, 06:54:00 pmjust wrapping some QueryPerformanceCounter around to measure time: Obviously, you're messing things up somewhere because you should get the same results I get. ---------------------------------------------------------------------- Thaddy · 2022-07-09 17:27 Note sleep(0) is actually called yield() in other languages. A.K.A. give up time slice if others need it. It is not forced, though. That can - depending on the complexity of other pieces in an application - have a big hit on performance if after the yield slower code will be executed. So sleep(0) is a bit nonsense in single threaded applications, but useful in muti-threaded applications. Mind sleep(0) may yield to the OS too, so not complete nonsense, actually. But then again, most OS's do scheduling anyway. The above is applicable for at least WINAPI. See MSDN. Ergo: one can not time Sleep(0)..... although I wonder if it is possible to time just the context switch. Note that Sleep(1) WILL force a context switch. ---------------------------------------------------------------------- Warfley · 2022-07-09 18:09 Quote from: 440bx on July 09, 2022, 07:19:27 pmObviously, you're messing things up somewhere because you should get the same results I get. Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i program TestSleep;   uses    Windows    ;   procedure SleepLoop(ms : DWORD);  const    COUNTER_LIMIT = 1000;   var    counter : DWORD = 0;   begin  writeln;    writeln('ms : ', ms, ' repeat count ', COUNTER_LIMIT);  for counter := 1 to COUNTER_LIMIT do    begin      Sleep(ms);    end;  end; var  f, s, e: Int64;begin  QueryPerformanceFrequency(f);  QueryPerformanceCounter(s);  SleepLoop(1);  { use values 1, 8 and 16 }  QueryPerformanceCounter(e);  WriteLn('Total time: ', (e-s) / f:3:3, 's');  QueryPerformanceCounter(s);  SleepLoop(8);  { use values 1, 8 and 16 }  QueryPerformanceCounter(e);  WriteLn('Total time: ', (e-s) / f:3:3, 's');  QueryPerformanceCounter(s);  SleepLoop(16);  { use values 1, 8 and 16 }  QueryPerformanceCounter(e);  WriteLn('Total time: ', (e-s) / f:3:3, 's');end.    I kept your code char by char. I'm sorry if reality doesn't conform to your expectations, maybe you are working on some ancient windows version, but these results are consistent with my experience for at least the past 10 years as well as my theoretical knowledge of the topic ---------------------------------------------------------------------- Warfley · 2022-07-09 19:32 Just fired up a Windows XP and a Windows 2000 VM. Windows XP: Sleep 1: 15.711s Sleep 8: 18.734s Sleep 16: 31.398s Windows 2000: Sleep 1: 16.535s Sleep 8: 16.796s Sleep 16: 32.265s I granted both VMs 2 cores so the process could monopolize one core. The results are very similar to the modern Windows 10 Results. So I'm quite confident in saying that this behavior is how the windows sheduler works since the introduction of the NT kernel ---------------------------------------------------------------------- rvk · 2022-07-09 20:51 Quote from: prodingus on July 08, 2022, 08:43:48 pmWith sleep(0) or no sleep at all, it is very very very fast. Insert even the lowest possible, sleep(1), the program runs very very very slow.  I didn't really find it very very slow  Can you give an estimate in seconds how long 1000 or 10.000 ping pongs take with my last example? ---------------------------------------------------------------------- prodingus · 2022-07-09 21:29 This thread is on fire!! Am I the firestarter..? Anyway, QuoteI didn't really find it very very slow  Can you give an estimate in seconds how long 1000 or 10.000 ping pongs take with my last example? Using my last example (I currently work on this "itteration" as I get the grip of it), without sleep I get 10000 ping-pongs every ~4 seconds. Using epiktimer systemsleep(1) I get  10000 every ~210 seconds. Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i program multi_iv;uses        winsock, SysUtils, lnet, crt, Classes, StreamIO, Windows, strutils, lcommon, epiktimer;var        buff: string;    psock:TLUDP;    psock2:TLUDP;       //tludp is pointer type by itself    i: integer= 0;    j: integer= 0;    k: integer= 0;    myip: string;    msg: string;    ET: TEpikTimer;    now: longint;type        TConsoleOutputStream = class(TStream)    function Write(const Buffer; Count: longint): longint; override;end; function TConsoleOutputStream.Write(const Buffer; Count: longint): longint;var    charcount: longword=0;begin    WriteConsole(GetStdHandle(STD_OUTPUT_HANDLE), @Buffer, Count, charcount, nil);    Result := charcount;end; procedure connectr(port:integer);var        Sock: TLUDP;begin        Sock := TLUdp.Create(nil);    psock:= sock;    Sock.listen(port);end; procedure connects(addr:string; port:integer);var        Sock2: TLUDP;begin        Sock2 := TLUdp.Create(nil);    psock2:= sock2;    Sock2.connect(addr, port);end;  begin        AssignStream(Output, TConsoleOutputStream.Create);        Rewrite(Output);    //crt.WindMaxY := 9999;    ET := TEpikTimer.Create(nil);     myip:= GetHostIP('');    i:=0;    writeln('ip: ',myip);        write('send or listen?');        readln(k);     if k=1 then begin        connects('1.1.1.10', 1234);        msg:='1 ping';        psock2.sendmessage(msg);    end;                                       // sock for receive, sock2 for send    begin        connectr(1234);      repeat                pSock.CallAction;                pSock.GetMessage(buff);            if buff <> '' then begin              writeln(buff,' ',pSock.Socks[0].PeerAddress);                connects(pSock.Socks[0].PeerAddress, 1234);                psock2.sendmessage(msg+' pong'); //needed here too, else the sender is not geting reply            end;                ET.SystemSleep(1);              until buff <> '';         now:= gettickcount64;              repeat                pSock.CallAction;                pSock.GetMessage(buff);            if buff <> '' then begin              if (i mod 10000 = 0) then begin                        writeln(buff,' ',pSock.Socks[0].PeerAddress,' *',uint64(gettickcount64-now));                        now:= gettickcount64;                end;                inc(i);                      msg:=inttostr(i);                        psock2.sendmessage(msg+' pong');            end;                //ET.SystemSleep(1);                until keypressed=true;    end;   readln;end. ---------------------------------------------------------------------- rvk · 2022-07-09 22:17 Quote from: prodingus on July 09, 2022, 11:29:00 pmUsing my last example (I currently work on this "itteration" as I get the grip of it), without sleep I get 10000 ping-pongs every ~4 seconds. Using epiktimer systemsleep(1) I get  10000 every ~210 seconds.Ok, I see what you mean. But with your code, the 4 seconds for 10.000 ping pongs is also a lot of time without a sleep-line. If I use my code I get this: Without any sleep(). Quoteip: 172.18.144.1 1 = send or 2 = listen? 2 ping from 192.168.2.11 0 1000 ping from 192.168.2.11 78 2000 ping from 192.168.2.11 78 3000 ping from 192.168.2.11 78 4000 ping from 192.168.2.11 62 5000 ping from 192.168.2.11 63 6000 ping from 192.168.2.11 62 7000 ping from 192.168.2.11 79 8000 ping from 192.168.2.11 93 9000 ping from 192.168.2.11 94 10000 ping from 192.168.2.11 78 11000 ping from 192.168.2.11 78 press enter to quit So 765ms (0,7 seconds) for 10.000 ping pongs. That's because my version uses the events and is much more efficient with the code flow. Your code flow synchronous and also takes a lot of time with that flow itself (including waiting of the other side before sending another package). Doing a sleep(0) didn't make much difference Quoteip: 172.18.144.1 1 = send or 2 = listen? 2 ping from 192.168.2.11 0 1000 ping from 192.168.2.11 78 2000 ping from 192.168.2.11 62 3000 ping from 192.168.2.11 63 4000 ping from 192.168.2.11 62 5000 ping from 192.168.2.11 78 6000 ping from 192.168.2.11 63 7000 ping from 192.168.2.11 62 8000 ping from 192.168.2.11 78 9000 ping from 192.168.2.11 63 10000 ping from 192.168.2.11 78 press enter to quit Only when doing sleep(1) it got a lot slower. Quoteip: 172.18.144.1 1 = send or 2 = listen? 2 ping from 192.168.2.11 0 1000 ping from 192.168.2.11 29157 2000 ping from 192.168.2.11 29375 3000 ping from 192.168.2.11 29890 4000 ping from 192.168.2.11 30094 5000 ping from 192.168.2.11 30844 6000 ping from 192.168.2.11 30750 7000 ping from 192.168.2.11 31000 8000 ping from 192.168.2.11 30984 9000 ping from 192.168.2.11 30797 10000 ping from 192.168.2.11 30734 11000 ping from 192.168.2.11 30813 press enter to quit BTW. With my event version the CPU doesn't go over the 25% (sometimes even far less) even without any sleep-line. I think if you really want more control over performance you would need to go with Socket or Synapse sockets in threads. But it all depends on what you want, where you use it for and what is practical. ---------------------------------------------------------------------- 440bx · 2022-07-09 22:53 Quote from: Warfley on July 09, 2022, 08:09:11 pmI'm sorry if reality doesn't conform to your expectations, maybe you are working on some ancient windows version, but these results are consistent with my experience for at least the past 10 years as well as my theoretical knowledge of the topic I don't think Win7 SPI qualifies as "ancient" and, it's not about my expectations, it's about how things work.  Sleep(n) is obviously not exact but, it is much more precise than what you're getting.  Neither my computer nor my version of Windows is "special" and, it works quite nicely over here.  Sleep(n) is within 30% of n. ---------------------------------------------------------------------- prodingus · 2022-07-09 23:11 @rvk, trying to use your example to verify the speed difference, I cannot compile: Sock.OnReceive := @TMyDummy.OnReceive; gives Error: Incompatible types: got "" expected "" ---------------------------------------------------------------------- prodingus · 2022-07-09 23:13 Side question (@everyone): Is it possible to do something like a custom sleep that waits x number of cpu cycles, or do something like the idle proccess of windows, or wait for something without polling (something else consuming cpu cycles)?? (Forgive the vagueness..) ---------------------------------------------------------------------- 440bx · 2022-07-09 23:32 Quote from: prodingus on July 10, 2022, 01:13:30 amIs it possible to do something like a custom sleep that waits x number of cpu cycles, In user mode, Sleep(n) is the closest thing to that. Quote from: prodingus on July 10, 2022, 01:13:30 am or do something like the idle proccess of windows, The system idle process is not a real process. Quote from: prodingus on July 10, 2022, 01:13:30 am or wait for something without polling (something else consuming cpu cycles)?? there sure is.  WaitForSingle/MultipleObject, allows a process to wait until an object is signaled and, no cpu is consumed during the wait. Quote from: prodingus on July 10, 2022, 01:13:30 am(Forgive the vagueness..) No problem, you're forgiven.  Your penance is two hello-worlds and 3 binary searches. ---------------------------------------------------------------------- prodingus · 2022-07-10 00:00 QuoteQuote from: prodingus on Today at 01:13:30 am Is it possible to do something like a custom sleep that waits x number of cpu cycles, In user mode, Sleep(n) is the closest thing to that.sleep (tries to) measure time, regardless of cpu frequency, unless I am wrong. I am not asking to pause for x amount of time, but x amount of cpu cycles (don't know if its possible though). QuoteQuote from: prodingus on Today at 01:13:30 am or do something like the idle proccess of windows, The system idle process is not a real process.I am aware that its not a real process. QuoteQuote from: prodingus on Today at 01:13:30 am or wait for something without polling (something else consuming cpu cycles)?? there sure is.  WaitForSingle/MultipleObject, allows a process to wait until an object is signaled and, no cpu is consumed during the wait.Any (easy to understand) reference on this? Is this, as Warfley said, similar to blocking sockets? ---------------------------------------------------------------------- rvk · 2022-07-10 00:01 Quote from: prodingus on July 10, 2022, 01:11:02 amSock.OnReceive := @TMyDummy.OnReceive; gives Error: Incompatible types: got "" expected "" If you use DELPHI mode you don't need to put @ before an eventname to assign it to a event-variable. In FPCObj mode yoy DO need to put @ in front of the event-proc during assignment. You first need to examine... do you want or need DELPHI mode? (You did put it in one of your examples, so I used it in mine and didn't put @ in front) If you still ave mode DELPHI at the top, remove it or remove the @. ---------------------------------------------------------------------- 440bx · 2022-07-10 00:16 Quote from: prodingus on July 10, 2022, 02:00:08 amsleep (tries to) measure time, regardless of cpu frequency, unless I am wrong. I am not asking to pause for x amount of time, but x amount of cpu cycles (don't know if its possible though).You're not wrong.  There is no way to pause/wait for a specifiable "x" number of CPU cycles. Quote from: prodingus on July 10, 2022, 02:00:08 amAny (easy to understand) reference on this? The MSDN documentation on it is reasonably easy to understand and there is even an example (In C of course.) ---------------------------------------------------------------------- prodingus · 2022-07-10 00:31 Quotef you use DELPHI mode you don't need to put @ before an eventname to assign it to a event-variable. In FPCObj mode yoy DO need to put @ in front of the event-proc during assignment. You first need to examine... do you want or need DELPHI mode? (You did put it in one of your examples, so I used it in mine and didn't put @ in front) If you still ave mode DELPHI at the top, remove it or remove the @. I had object pascal by default for the compiler option, switched to delphi and worked without edit! I modified your program: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i program pingpong; uses  SysUtils,  lnet, lnetbase, lcommon,  crt, Classes, StreamIO, Windows; var  Sock: TLUDP;  i: integer = 0;  [b]j: integer = 0;[/b]  k: integer;  [b]now:longint;[/b] type  TConsoleOutputStream = class(TStream)    function Write(const Buffer; Count: longint): longint; override;  end;   function TConsoleOutputStream.Write(const Buffer; Count: longint): longint;  var    charcount: longword = 0;  begin    WriteConsole(GetStdHandle(STD_OUTPUT_HANDLE), @Buffer, Count, charcount, nil);    Result := charcount;  end; type  TMyDummy = class    class procedure OnReceive(aSocket: TLSocket);end;class procedure TMyDummy.OnReceive(aSocket: TLSocket);var        tmp: string;begin              if aSocket.GetMessage(tmp) > 0 then begin           [b]inc(j);           if (j mod 10000 = 0) then begin                writeln(tmp + ' from ' + aSocket.PeerAddress,' *',uint64(gettickcount64-now));                now:= gettickcount64;                 end;[/b]          Inc(i);          tmp := IntToStr(i);          if k = 1 then            aSocket.sendmessage(tmp + ' ping')          else            aSocket.sendmessage(tmp + ' pong');                end; end; begin  AssignStream(Output, TConsoleOutputStream.Create);  Rewrite(Output);  crt.WindMaxY := 9999;  i := 0;  [b]j:=0;[/b]   writeln('ip: ', GetHostIP(''));  Write('1 = send or 2 = listen? ');  readln(k);   if k = 1 then  begin    Sock := TLUdp.Create(nil);    Sock.Connect('1.1.1.10', 1234); // <-- change the receiver IP here once    Sock.OnReceive := TMyDummy.OnReceive;    Sock.sendmessage('ping');  end  else  begin    Sock := TLUdp.Create(nil);    Sock.Listen(1234);    Sock.OnReceive := TMyDummy.OnReceive;  end;   [b]now:= gettickcount64;[/b]  repeat      Sock.CallAction;        [b]//delay(1);[/b]  until keypressed = True;   writeln('press enter to quit');  readln;end. it runs about the same as mine  EDIT: tried to bold the changes, but didn't knew they dont work inside "code". ---------------------------------------------------------------------- rvk · 2022-07-10 00:38 Quote from: prodingus on July 10, 2022, 02:31:32 amit runs about the same as mine  I tried it from a VM to host on the same machine. Did you use two separate machines? Was your result more than 100ms per 1000? I'll try it with two machines tomorrow. If you used it on the same machine I'm not sure why your performance is so poor. Ps. I did mod 1000 to show results per 1000. Not 10.000. ---------------------------------------------------------------------- prodingus · 2022-07-10 00:42 I test them VM to VM in their own vnet. Have you compiled my code and run it? ---------------------------------------------------------------------- rvk · 2022-07-10 00:43 Quote from: prodingus on July 10, 2022, 02:42:08 amI test them VM to VM in their own vnet. Have you compiled my code and run it? I'll try tomorrow (sleeping now  ) Ps. I did mod 1000 to show results per 1000. Not 10.000 ---------------------------------------------------------------------- Warfley · 2022-07-10 10:08 Quote from: 440bx on July 10, 2022, 12:53:13 amNeither my computer nor my version of Windows is "special" and, it works quite nicely over here.  Sleep(n) is within 30% of n. I don't know whats wrong with your windows 7 version, I tested it on my Windows 10 pc, and an windows xp as well as a windows 2000 vm (I don't have Win 7 lying around so I can't test that) so unless there was something very special in Win7, this seems to be the default behavior of the NT kernel since it's inception. Maybe there is something special on your version. Quote from: prodingus on July 10, 2022, 01:13:30 amor wait for something without polling (something else consuming cpu cycles)?? This is what I mentioned earlier, to use blocking calls. A blocking networking call will pause the process/thread until. I don't know much about lnet so I don't know what exactly callaction does, but when you just call a recv or recvfrom on a socket, the OS will put the process to sleep until some message arrived and then wake it up and return. If you want to wait for a message on multiple sockets, or you want a timout on the one socket, use select poll (windows WSAPoll) or, if available epoll. You can also use the WaitForMultipleObjects, which is what WSAPoll is internally calling, but WSAPoll has compatibility with the berkley socket API Poll and therefore easier to write portable code. So the flow is, use poll to wait for data, read that data, repeat. With epoll this can easiely scale to tens of thousands of sockets being handled in microseconds. Select and poll are less efficient (O(n)) and only are capable of handling a few hundred to thousand simultanious connections without massive delays ---------------------------------------------------------------------- MarkMLl · 2022-07-10 10:22 Quote from: Warfley on July 10, 2022, 12:08:49 pmQuote from: 440bx on July 10, 2022, 12:53:13 amNeither my computer nor my version of Windows is "special" and, it works quite nicely over here.  Sleep(n) is within 30% of n. I don't know whats wrong with your windows 7 version, I tested it on my Windows 10 pc, and an windows xp as well as a windows 2000 vm (I don't have Win 7 lying around so I can't test that) so unless there was something very special in Win7, this seems to be the default behavior of the NT kernel since it's inception. Maybe there is something special on your version. It will depend heavily on the CPU load of the overall system, the tick quantum, the number of cores, the number of CPUs, the way the cache is disposed between CPUs and cores, and any hints the OS can glean from the chipset relating to the technology used to access physical memory (i.e. DIMM type and so on). All of those will be taken into account when the OS decides how to schedule ready and almost-ready threads. In addition, it is not only unsafe to assume that the scheduler will behave consistently across Windows versions, but it's also unsafe to assume that two copies of Windows of the same apparent version but with a different installation history (e.g. one started off with a slightly older DVD but was then updated) will behave the same. The fundamental rule is that unless you have a realtime OS, Sleep(n) means Sleep(n + z) where z is >= 0 but otherwise indeterminate. MarkMLl ---------------------------------------------------------------------- rvk · 2022-07-10 10:35 Quote from: prodingus on July 10, 2022, 02:42:08 amI test them VM to VM in their own vnet. Have you compiled my code and run it? Your code from same machine/host to same machine/host (running side by side). Quoteip: 172.18.144.1 1 = send or 2 = listen? 2 9999 ping from 192.168.2.11 *7297 19999 ping from 192.168.2.11 *735 29999 ping from 192.168.2.11 *765 39999 ping from 192.168.2.11 *735 49999 ping from 192.168.2.11 *734 59999 ping from 192.168.2.11 *719 69999 ping from 192.168.2.11 *703 79999 ping from 192.168.2.11 *734 press enter to quit From VM to VM Quoteip: 172.18.144.1 1 = send or 2 = listen? 2 9999 ping from 192.168.2.80 *8766 19999 ping from 192.168.2.80 *2156 29999 ping from 192.168.2.80 *2094 39999 ping from 192.168.2.80 *1906 49999 ping from 192.168.2.80 *2141 59999 ping from 192.168.2.80 *2156 69999 ping from 192.168.2.80 *2375 79999 ping from 192.168.2.80 *2203 press enter to quit So, infrastructure also has something to do with it. But did you notice the CPU doesn't go beyond 25% without any delay or sleep in this code. (Anyway... the sleep-thing is being discussed between these posts  ) ---------------------------------------------------------------------- Warfley · 2022-07-10 10:48 Quote from: MarkMLl on July 10, 2022, 12:22:27 pmIt will depend heavily on the CPU load of the overall system, the tick quantum, the number of cores, the number of CPUs, the way the cache is disposed between CPUs and cores, and any hints the OS can glean from the chipset relating to the technology used to access physical memory (i.e. DIMM type and so on). All of those will be taken into account when the OS decides how to schedule ready and almost-ready threads. In addition, it is not only unsafe to assume that the scheduler will behave consistently across Windows versions, but it's also unsafe to assume that two copies of Windows of the same apparent version but with a different installation history (e.g. one started off with a slightly older DVD but was then updated) will behave the same. The scheduler is usually something that is quite stable, for example if you look at the linux kernel git, you can see that the main functionalities of the scheduler wasn't really touched in 5-15 years. And modern day schedulers all work quite similar, they divide the time in time slices of fixed length, where only the selection of the process is different. If you call sleep(n) with n>0 then you forfit the rest of your time slice, meaning that if you sleep at completely random positions in your program your sleep will on average be at least half the timeslice. In the examples above the sleep was not at random points in the program but the program was basically nothing but sleeps, so as soon as it got resheduled after just a few operations it will have gone sleeping again, so which is why the minimum sleeping time is around the size of one timeslice. So while you are right that there are a lot of variables in the scheduling process, especially with respect for other processes, it can easiely be mitigated on a system with enough cpus such that there is at least one underutilized, as I did for both the VMs and on my machine. Also the time slice is the absolute lower bound, the scheduler will not reschedule in the same timeslice, it is not capable of doing so, therefore you can't sleep less than the remainder of your time slice, which in the examples above is pretty much guaranteed to be nearly the full amount. You can't say how a single sleep will behave, as the minimum and maximum measures in my example above have shown, but you can make a prediction on how the scheduler will on average work when controlling for the environment as much as possible. And the fact that the scheduler works exactly the same on both a fresh windows 2000, a windows xp which I regularly use for running old games, and my windows 10, which I use daily, is a very clear indication that the predictions seem to be correct and that the core of the scheduler have not been changed since the NT kernel was introduced ---------------------------------------------------------------------- Warfley · 2022-07-10 10:51 Quote from: prodingus on July 10, 2022, 02:42:08 amI test them VM to VM in their own vnet. Have you compiled my code and run it? As a recommendation, if you want to test how networking applications behave on a real network, I can recoomend Containernet for Linux with Docker. There you can setup a number of docker containers (which will then run your application) and define networking properties between them, e.g. which bandwith, delay, jitter and packet loss should happen between them. But sadly not available for testing windows software ---------------------------------------------------------------------- 440bx · 2022-07-10 11:24 Quote from: Warfley on July 10, 2022, 12:08:49 pmI don't know whats wrong with your windows 7 version, I tested it on my Windows 10 pc, and an windows xp as well as a windows 2000 vm (I don't have Win 7 lying around so I can't test that) so unless there was something very special in Win7, this seems to be the default behavior of the NT kernel since it's inception. Maybe there is something special on your version. There is nothing wrong with my Windows version but, since you insisted so much, I re-ran the tests (multiple times), here are my results:Code: Text  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i timer&TestSleep.exe&timer         (1 ms)Timer 1 on:  6:57:58 ms : 1 repeat count 1000Timer 1 off:  6:57:59  Elapsed: 0:00:01.02 timer&TestSleep.exe&timer         (8 ms)Timer 1 on:  6:58:12 ms : 8 repeat count 1000Timer 1 off:  6:58:20  Elapsed: 0:00:08.01 timer&TestSleep.exe&timer         (16 ms)Timer 1 on:  6:58:34 ms : 16 repeat count 1000Timer 1 off:  6:58:50  Elapsed: 0:00:16.02 timer&TestSleep.exe&timer         (1 ms)Timer 1 on:  7:00:21 ms : 1 repeat count 1000Timer 1 off:  7:00:22  Elapsed: 0:00:01.01 timer&TestSleep.exe&timer         (8 ms)Timer 1 on:  7:01:14 ms : 8 repeat count 1000Timer 1 off:  7:01:22  Elapsed: 0:00:08.01 timer&TestSleep.exe&timer         (1 ms)Timer 1 on:  7:03:22 ms : 1 repeat count 1000Timer 1 off:  7:03:23  Elapsed: 0:00:01.02 As you can see in the above, the times are within 2% of the sleep time.  I also ran the tests in a Win7 VM and the results were erratic and closer to what you've mentioned, as a result, I rebooted that VM and re-ran the tests and got results that were almost identical to the above but a bit slower, likely due to the fact that the test was done in a VM but, still well within the 20% variation I mentioned.  Looks like some things get "strange" sometimes in a VM. I've noticed that in other things too.  Every few weeks I have to reboot the Win7 VM I use for development because I do notice it has slowed down (I don't know why and since the fix is easy, reboot, I've never bothered to investigate the reason.) As you can see, I repeated the 1 ms test (twice in the above) multiple times to see if there would be any variation and, there wasn't.  All the tests finished within 2 to 5% of 1 ms. If you are not getting the results I am getting then, the conclusion is, there is something not quite right in your Windows configuration. I assure you, there is absolutely nothing unusual in my Win7 SP1 installation.  It's as plain vanilla as it gets. ---------------------------------------------------------------------- Warfley · 2022-07-10 12:02 Quote from: 440bx on July 10, 2022, 01:24:56 pmIf you are not getting the results I am getting then, the conclusion is, there is something not quite right in your Windows configuration. I assure you, there is absolutely nothing unusual in my Win7 SP1 installation.  It's as plain vanilla as it gets. Just tested it on my laptop, which is the most freshly installed windows I have (as I usually only use linux on it), I never changed any os level configuration on this. Same results. I also reran the vm tests multiple times after restarting the vms So I've now conducted tests on 4 machines, 2 virtual, two physical, accross 4 different windows versions, all report the same results. It cannot be my configuration that is wrong, I always get the same consistent results accross all configurations, and they are also consistent with theory. Maybe it is just that windows 7 is different with scheduling, but this cannot be a wrong configuration or vm weirdness, I get consistent results across 4 different configurations, 2 virtual 2 physical on different hardware, all using different operating systems and configurations. I measured the time both using queryperformancecounter, and also Measure-Command (as we are talking about measurements in the tens of seconds), I get the same results everywhere. If I have ever seen a consistent experiment than this is it. There is no doubt that this is not a configuration issue or VM issue on my end. ---------------------------------------------------------------------- 440bx · 2022-07-10 12:39 Quote from: Warfley on July 10, 2022, 02:02:17 pmSo I've now conducted tests on 4 machines, 2 virtual, two physical, accross 4 different windows versions, all report the same results. It cannot be my configuration that is wrong, I always get the same consistent results accross all configurations, and they are also consistent with theory. I wouldn't be so sure of that.  Sleep's behavior is something I've been testing from Windows version to Windows version since, at least, Win2000 and, it's been very consistent.  The only times I get inconsistencies is when Windows is running in a VM and, after rebooting, Sleep works as expected (and as it should, I might add.) I don't know why you're getting weird numbers but, the results should be very close to those I posted and, if they aren't then, something is wrong with the Windows configuration (unless it's in a VM.  As I've previously stated, I've seen strange results in VMs but, a reboot brings them back to what they should be.) On the other hand, I just tried the test in a Win 10 VM (which has been up for weeks) and, I got "weird" results like yours but, I'm not rebooting that thing.  Every time I reboot it, the Windows defender spends half a day scanning for viruses, which is rather annoying and, the overhead and disk access slows down the other VMs I actually use.  (I only use that Win10 VM for testing, not for development, IOW, I don't really care if it's screwed up, as long as I can use it for testing it's good enough and, I only care that the program loads, runs and the results are correct, fast or slow, I couldn't care less about that on Win10) ---------------------------------------------------------------------- Warfley · 2022-07-10 12:47 I also spun up a Windows 7 VM on both my Desktop and my Laptop, same results. So to recap, I got on 5 setups, two native on different hardware, 3 virtual, repeating multiple times (with reboots) the exact same results, not a single time the results you got. This must be a problem on your device. I can't explain your results, in fact I have never seen such results anywhere to this day. My results are so consistent there is no way this is some configuration issue, so it must be a you problem. So I will leave it at that. I don't know whats the issue at your end, but with so many different configurations giving the same result, there is no way that this isn't the "correct"/intendet behavior ---------------------------------------------------------------------- prodingus · 2022-07-10 17:42 Quote from: rvk on July 10, 2022, 12:35:35 pmQuote from: prodingus on July 10, 2022, 02:42:08 amI test them VM to VM in their own vnet. Have you compiled my code and run it? Your code from same machine/host to same machine/host (running side by side). Quoteip: 172.18.144.1 1 = send or 2 = listen? 2 9999 ping from 192.168.2.11 *7297 19999 ping from 192.168.2.11 *735 29999 ping from 192.168.2.11 *765 39999 ping from 192.168.2.11 *735 49999 ping from 192.168.2.11 *734 59999 ping from 192.168.2.11 *719 69999 ping from 192.168.2.11 *703 79999 ping from 192.168.2.11 *734 press enter to quit From VM to VM Quoteip: 172.18.144.1 1 = send or 2 = listen? 2 9999 ping from 192.168.2.80 *8766 19999 ping from 192.168.2.80 *2156 29999 ping from 192.168.2.80 *2094 39999 ping from 192.168.2.80 *1906 49999 ping from 192.168.2.80 *2141 59999 ping from 192.168.2.80 *2156 69999 ping from 192.168.2.80 *2375 79999 ping from 192.168.2.80 *2203 press enter to quit So, infrastructure also has something to do with it. But did you notice the CPU doesn't go beyond 25% without any delay or sleep in this code. (Anyway... the sleep-thing is being discussed between these posts  ) I rerun mine and your version in a laptop too. Same VM setups, exact copies (only one thread/cpu). That 25% of yours, are you on a quad core/thread? Yours vs mine results the same speed both in laptop and pc (PC ryzen, laptop 2nd gen i5). The interesting part is (without using sleep at all) I get full cpu utilization on both systems and same speed results (10.000 ping-pongs per ~ 4secs). Which means that something funny is happening, backed by your results; if the vms are causing lattency/delay/"slowness", and the getmessage "halts" the cpu untill a message has arrived in the buffer, and cpu/thread is 100% even in a high-end machine, then does- not-compute!!! Also, swith sleep(1) on yours and mine I get the same sluglish results, 100 ping-pongs per ~ 2secs, and almost no cpu utilization. ---------------------------------------------------------------------- Warfley · 2022-07-10 18:01 Quote from: rvk on July 10, 2022, 12:35:35 pmSo, infrastructure also has something to do with it. Don't assume that connections on the same machine work the same as connections accross machines (or to be more precise on the same network interface vs different network interfaces). Basically when you send data from one network interface to itself, there is no reason for the OS to do all the networking, all the wrapping in IP packages and sending it across the NIC is not necessary. So it can easiely be that the OS will simply put the messages directly into the buffer of the target application, circumventing the whole network interface. This is a common problem when trying to use wireshark or similar programs to sniff data over the loopback address without workarounds (on windows, on linux loopback goes through the network interface and can be sniffed easiely). So different behavior is expected. And about the CPU utlization with sleep(1), I know the sleep discussion went a bit of the rails, but the core message what I wanted to get across is, sleep(1) sleeps several MS, so if your your workload between the sleeps is very short (and handling a message is pretty much no effort at all), then the ratio between workload and sleeptime, which is the CPU load, will be extremely low. Try rather than sleeping every iteration, only sleep after like 1000 iterations. This way you can balance the sleeping against the worktime ---------------------------------------------------------------------- prodingus · 2022-07-10 18:06 On my previous message (perhaps I was faster writing than thinking), does the full cpu/tread utilization beed caused by the repeating of pSock.CallAction ? ---------------------------------------------------------------------- rvk · 2022-07-10 18:06 Quote from: Warfley on July 10, 2022, 08:01:33 pmQuote from: rvk on July 10, 2022, 12:35:35 pmSo, infrastructure also has something to do with it. Don't assume that connections on the same machine work the same as connections accross machines (or to be more precise on the same network interface vs different network interfaces). OS is also part of the infrastructure. I just meant to say that the slowness isn't only caused by the sleep. There is much more in play here. Also... You have one side sending a message and the other side waiting for it. This isn't done asynchronous so the waiting element on both sides could be in play too. Sleep could double or triple because of that (sender sleeps before receiving an answer and answer back (and that times two). ---------------------------------------------------------------------- prodingus · 2022-07-10 18:09 QuoteDon't assume that connections on the same machine work the same as connections accross machines (or to be more precise on the same network interface vs different network interfaces). Basically when you send data from one network interface to itself, there is no reason for the OS to do all the networking, all the wrapping in IP packages and sending it across the NIC is not necessary. So it can easiely be that the OS will simply put the messages directly into the buffer of the target application, circumventing the whole network interface. I was under the assumption that vms with their own vnet were going to have the lowest latency. ---------------------------------------------------------------------- rvk · 2022-07-10 18:09 Quote from: prodingus on July 10, 2022, 08:06:17 pmOn my previous message (perhaps I was faster writing than thinking), does the full cpu/tread utilization beed caused by the repeating of pSock.CallAction ? It could. I'm not sure how that function looks at the moment but if it just checks for communication and doesn't yield processing power, then your program just takes all the cpu (it would effectively be a Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i Repeat  // Do nothingUntil false; Which is really OS unfriendly. ---------------------------------------------------------------------- Warfley · 2022-07-10 18:26 Quote from: prodingus on July 10, 2022, 08:09:33 pmI was under the assuption that vms with their own vnet were going to have the lowest latency. How a VM works is basically it provides a virtual network interface on the vm, when the VM writes to that virtual interface, the packets will be read out by the virtualisation software, and, in this configuration, be transmitted to the virtual network interface that the virtualisation software creates on the host. From there the messages are handled by the OS, which sees the target being another virtual network interface, the one of the second VM, and send them to it, which will trigger the virtualization software to take these packages and put them into the virtual network interface on the second virtual machine, where you can read them out. This has a low latency because there is no real wire, but there still is a lot of moving memory around involved, this cannot be optimised further because the OS does not know whats behind the virtual interface provided by the VM. When you send a message from a network interface to itself, i.e. two processes communicate with each other, the OS can see that, and directly let those communicate without actually involving the network interface. It saves a few steps in the process. Also at each of these stages package loss can happen if the buffers overflow because the forwarding is not fast enough. So long story short, VMs add a lot of overhead to the system. Not as much as a real wire, but more than just two processes communicating with each other. About the CPU utilization issues, I edited my last post with some more stuff about it, but you where so much faster in posting that you might not have seen it: QuoteAnd about the CPU utlization with sleep(1), I know the sleep discussion went a bit of the rails, but the core message what I wanted to get across is, sleep(1) sleeps several MS, so if your your workload between the sleeps is very short (and handling a message is pretty much no effort at all), then the ratio between workload and sleeptime, which is the CPU load, will be extremely low. Try rather than sleeping every iteration, only sleep after like 1000 iterations. This way you can balance the sleeping against the worktime I don't know much about lnet, but I guess that internally it will use blocking calls. So the reason why the CPU load is at 100% is, that every time it is called there is data in the buffer. Therefore every time it calls recv, it will read out that data from the buffer without having to wait, so you get 100% cpu load. The question is, if you slow down the sender, does the CPU load get down? If so, the reason you have 100% CPU load is simply because you get so much data that this is the fastest you can read the data. If you then slow down the receiver through sleep, you will simply loose most of your message sent. So you better controll this on sender side not at receiver side. But if you want to slow down on receiver side, simply don't sleep every turn, but have a few turns and sleep then. You could use time measurements to make sure that for every 50 milliseconds of receiving you sleep another 20 ms, this way your CPU utilization will not be above 80% ---------------------------------------------------------------------- prodingus · 2022-07-10 18:52 QuoteThe question is, if you slow down the sender, does the CPU load get down?Running the "sender" on the first vm with sleep(1) and the "receiver" at the other vm, the receiver is still @ 100% cpu. QuoteAbout the CPU utilization issues, I edited my last post with some more stuff about it, but you where so much faster in posting that you might not have seen it: Quote And about the CPU utlization with sleep(1), I know the sleep discussion went a bit of the rails, but the core message what I wanted to get across is, sleep(1) sleeps several MS, so if your your workload between the sleeps is very short (and handling a message is pretty much no effort at all), then the ratio between workload and sleeptime, which is the CPU load, will be extremely low. Try rather than sleeping every iteration, only sleep after like 1000 iterations. This way you can balance the sleeping against the worktime Yeap I missed it, my bad (TMI on this thread, so little time). I never thought of running a counter to get sleep at steps, as it "adds" more program logic thus eating cpu cycles (even if negligible). Also on the blocking sockets thing, I get the feel that this is a "wrong" way to reduce cpu usage; it's just me of cource and my personal taste when wrighting a program. EDIT: running counter to trigger sleep also feels wrong, because its more like get-some-messages then stop, repeat. ---------------------------------------------------------------------- prodingus · 2022-07-10 18:53 QuoteI don't know much about lnet, but I guess that internally it will use blocking calls. So the reason why the CPU load is at 100% is, that every time it is called there is data in the buffer.It seems that this is not the case. ---------------------------------------------------------------------- rvk · 2022-07-10 19:16 Quote from: prodingus on July 10, 2022, 07:42:58 pmI rerun mine and your version in a laptop too. Same VM setups, exact copies (only one thread/cpu). That 25% of yours, are you on a quad core/thread? Yours vs mine results the same speed both in laptop and pc (PC ryzen, laptop 2nd gen i5). I'm on Windows 10 Pro 64 bit, an old Dell XPS8500 with Intel Core I7 3770 - 3.4 Ghz - 4 Cores - 8 Threads. When the sender is in a VM (Hyper V, Windows 10, 1 CPU) it does reach 100%. You can try to play with SetProcessAffinityMask and SetThreadAffinityMask but it wouldn't change the CPU immediately. It would make the rest of the OS more responsive because your program would run in lower priority. Maybe threads could help too but I haven't tried that. It kinda depends on what you are actually after. If you want real-time communications.... that just takes time and cpu power. BTW. It's not only the project.exe which takes a lot of CPU. conhost.exe takes it share too (probably the network handling). When my VM is 100% busy... project.exe takes about 50% and conhost 30%. Interrupts takes 15%. ---------------------------------------------------------------------- prodingus · 2022-07-10 19:25 I added a counter in order to do sleep(1) @ 1000 repeats. The first run I thougt I broke the lnet (just kidding). The results are all over the place: the first attempt had writeln @ 10.000 repeats (just as the last code I posted) only with the new counter & sleep. I got the first 10.000 ping-pongs, then I didn't see any output for about 10+ minutes; the cpu usage was constanly @ ~ 50-65% (and yes after some time I set the affinity of the vms and the fluctuations were gone). Then I changed the writeln to 100 ping-pongs and the results: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i ip: 1.1.1.10send or listen?31 ping 1.1.1.11 ping pong 1.1.1.1 *78100 pong 1.1.1.1 *1685200 pong 1.1.1.1 *46300 pong 1.1.1.1 *94400 pong 1.1.1.1 *16500 pong 1.1.1.1 *156600 pong 1.1.1.1 *15700 pong 1.1.1.1 *63800 pong 1.1.1.1 *78900 pong 1.1.1.1 *621000 pong 1.1.1.1 *781100 pong 1.1.1.1 *161200 pong 1.1.1.1 *1091300 pong 1.1.1.1 *471400 pong 1.1.1.1 *931500 pong 1.1.1.1 *311600 pong 1.1.1.1 *781700 pong 1.1.1.1 *781800 pong 1.1.1.1 *1101900 pong 1.1.1.1 *622000 pong 1.1.1.1 *782100 pong 1.1.1.1 *782200 pong 1.1.1.1 *472300 pong 1.1.1.1 *622400 pong 1.1.1.1 *1412500 pong 1.1.1.1 *462600 pong 1.1.1.1 *32762700 pong 1.1.1.1 *942800 pong 1.1.1.1 *472900 pong 1.1.1.1 *463000 pong 1.1.1.1 *943100 pong 1.1.1.1 *473200 pong 1.1.1.1 *783300 pong 1.1.1.1 *20593400 pong 1.1.1.1 *31833500 pong 1.1.1.1 *25893600 pong 1.1.1.1 *1873700 pong 1.1.1.1 *163800 pong 1.1.1.1 *783900 pong 1.1.1.1 *784000 pong 1.1.1.1 *944100 pong 1.1.1.1 *464200 pong 1.1.1.1 *784300 pong 1.1.1.1 *634400 pong 1.1.1.1 *474500 pong 1.1.1.1 *1404600 pong 1.1.1.1 *40254700 pong 1.1.1.1 *17004800 pong 1.1.1.1 *12014900 pong 1.1.1.1 *32145000 pong 1.1.1.1 *14515100 pong 1.1.1.1 *475200 pong 1.1.1.1 *935300 pong 1.1.1.1 *315400 pong 1.1.1.1 *785500 pong 1.1.1.1 *945600 pong 1.1.1.1 *625700 pong 1.1.1.1 *635800 pong 1.1.1.1 *625900 pong 1.1.1.1 *786000 pong 1.1.1.1 *636100 pong 1.1.1.1 *626200 pong 1.1.1.1 *626300 pong 1.1.1.1 *476400 pong 1.1.1.1 *1256500 pong 1.1.1.1 *11236600 pong 1.1.1.1 *476700 pong 1.1.1.1 *786800 pong 1.1.1.1 *4996900 pong 1.1.1.1 *1727000 pong 1.1.1.1 *477100 pong 1.1.1.1 *477200 pong 1.1.1.1 *627300 pong 1.1.1.1 *947400 pong 1.1.1.1 *317500 pong 1.1.1.1 *787600 pong 1.1.1.1 *477700 pong 1.1.1.1 *937800 pong 1.1.1.1 *637900 pong 1.1.1.1 *788000 pong 1.1.1.1 *1248100 pong 1.1.1.1 *168200 pong 1.1.1.1 *5778300 pong 1.1.1.1 *34958400 pong 1.1.1.1 *12638500 pong 1.1.1.1 *9528600 pong 1.1.1.1 *318700 pong 1.1.1.1 *788800 pong 1.1.1.1 *948900 pong 1.1.1.1 *469000 pong 1.1.1.1 *1109100 pong 1.1.1.1 *319200 pong 1.1.1.1 *789300 pong 1.1.1.1 *629400 pong 1.1.1.1 *639500 pong 1.1.1.1 *3749600 pong 1.1.1.1 *479700 pong 1.1.1.1 *10299800 pong 1.1.1.1 *1109900 pong 1.1.1.1 *9310000 pong 1.1.1.1 *6310100 pong 1.1.1.1 *1510200 pong 1.1.1.1 *14110300 pong 1.1.1.1 *160710400 pong 1.1.1.1 *156010500 pong 1.1.1.1 *6210600 pong 1.1.1.1 *4710700 pong 1.1.1.1 *6210800 pong 1.1.1.1 *9410900 pong 1.1.1.1 *6211000 pong 1.1.1.1 *9411100 pong 1.1.1.1 *4711200 pong 1.1.1.1 *6211300 pong 1.1.1.1 *7811400 pong 1.1.1.1 *42111500 pong 1.1.1.1 *3111600 pong 1.1.1.1 *6311700 pong 1.1.1.1 *12511800 pong 1.1.1.1 *7811900 pong 1.1.1.1 *1512000 pong 1.1.1.1 *9412100 pong 1.1.1.1 *4712200 pong 1.1.1.1 *7812300 pong 1.1.1.1 *6212400 pong 1.1.1.1 *6212500 pong 1.1.1.1 *1612600 pong 1.1.1.1 *6212700 pong 1.1.1.1 *4712800 pong 1.1.1.1 *1092 It is random when and if the program woud "hang"; actually I thing the "hang" is accumulation of sleep(1) from both sides. ---------------------------------------------------------------------- Warfley · 2022-07-10 19:27 Quote from: prodingus on July 10, 2022, 08:52:23 pmAlso on the blocking sockets thing, I get the feel that this is a "wrong" way to reduce cpu usage; it's just me of cource and my personal taste when wrighting a program. It is the only correct way if you don't want to be at 100% CPU load. There are two ways to use sockets, either blocking or nonblocking. Blocking sockets will reduce CPU usage as much as possible, as every time there is no data, it will sleep until data is there. When you are doing a sleep loop where you check if data is there and if not go to sleep, you basically build the same thing just with extra steps. Non blocking sockets are only required if you want to do other things while waiting for the socket on the same thread. This is a highly specialized use-case for when you need to build single threaded applications but still have asynchronous I/O. Usually whenever someone talks about non blocking sockets, in most cases what they really want to use is select/poll/epoll. True asynchronous programming is used rather infrequently, and most people just want a simple timeout or handle multiple sockets, or want to check if a message they are awaiting is already there. In all of these cases select/poll/epoll will do the trick, while still reducing cpu load by blocking. PS: as a little rant on the side, I must say LNet is absolutely terrible, there is no documentation, the method names give absolutely no indication what they are doing (e.g. CallAction, this doesn't call any actions, it handles events, why isn't it named HandleEventQueue or something meaningful), and from reading the code I had absolutely no idea that it is either non blocking, or threaded. You look at this code and have no idea what it does. Also the same code you wrote using Lnet, could be writting in probably half that code using the raw socket API. So you are using a library that has throusands of lines of code, so you can write more code... I thought the L stands for lightweight... This is something I have observed with many networking libraries that try to make low level protocols (i.e. TCP and UDP) easier, in the end they are just way more complicated than the raw socket API. This sort of design works well for high level protocols like SMTP or HTTP, but by trying to make low level protocols easier, they just made using them harder /rant over ---------------------------------------------------------------------- prodingus · 2022-07-10 19:33 QuotePS: as a little rant on the side, I must say LNet is absolutely terrible, there is no documentation, the method names give absolutely no indication what they are doing (e.g. CallAction, this doesn't call any actions, it handles events, why isn't it named HandleEventQueue or something meaningful), and from reading the code I had absolutely no idea that it is either non blocking, or threaded. You look at this code and have no idea what it does. Also the same code you wrote using Lnet, could be writting in probably half that code using the raw socket API. So you are using a library that has throusands of lines of code, so you can write more code... I thought the L stands for lightweight... This is something I have observed with many networking libraries that try to make low level protocols (i.e. TCP and UDP) easier, in the end they are just way more complicated than the raw socket API. This sort of design works well for high level protocols like SMTP or HTTP, but by trying to make low level protocols easier, they just made using them harder /rant over AMEN! https://c.tenor.com/eOXG7de_rCoAAAAd/steve-carell-thankyou.gif ---------------------------------------------------------------------- prodingus · 2022-07-10 19:44 QuoteIt is random when and if the program woud "hang"; actually I thing the "hang" is accumulation of sleep(1) from both sides. Scratch that, I wrote again before giving a second thought: If sleep accumulation was happening, then the cpu usage would drop, but it doesn't. ---------------------------------------------------------------------- rvk · 2022-07-10 20:04 That's why I said right from the start you're better off using Socket of Synapse sockets in threads (but I haven't tested that). And it all still depends on what you want. I think you said you just wanted to learn and all this was theoretical. Then you should not dwell too long on sleep and lnet but start exploring socket and threads for your example. ---------------------------------------------------------------------- alpine · 2022-07-10 20:05 Quote from: prodingus on July 10, 2022, 09:44:22 pmQuoteIt is random when and if the program woud "hang"; actually I thing the "hang" is accumulation of sleep(1) from both sides. Scratch that, I wrote again before giving a second thought: If sleep accumulation was happening, then the cpu usage would drop, but it doesn't. In your test code, assign a value > 0 to the Sock.Timeout and remove sleeps. Then try again. Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i  ... if k = 1 then  begin    Sock := TLUdp.Create(nil);    Sock.Connect('1.1.1.10', 1234); // <-- change the receiver IP here once    Sock.OnReceive := @TMyDummy.OnReceive;    Sock.Timeout := 50;    Sock.sendmessage('ping');  end...  Edit: Your test code from reply #119 ---------------------------------------------------------------------- Warfley · 2022-07-10 20:58 I might again recommend you trying my PasSimpleSockets out: https://github.com/Warfley/PasSimpleSockets It is basically just a thin layer around the default sockets API. A simple UDP server that sends ping-pongs would be: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i program pingpongserver; {$mode objfpc}{$H+} uses  SimpleSockets; var  Sock: TSocket;  Msg: TReceiveFromStringMessage;begin  Sock := UDPSocket(stDualStack);  try    // Binding to 0-IP (here in ipv6 format) binds to any address, with stDualStack both IPv4 and IPv6    Bind(Sock, '::0', 1337);    repeat      Msg := ReceiveStrFrom(Sock);    WriteLn('Ping received from ', Msg.FromAddr.Address, ':', Msg.FromPort,' message: ', Msg.Data);      SendStrTo(Sock, Msg.FromAddr, Msg.FromPort, Msg.Data + 'Pong');    until False;  finally    CloseSocket(Sock);  end;  ReadLn;end. If you want to work non blockingly, you can use DataAvailable (internally using select): Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i program pingpongserver; {$mode objfpc}{$H+} uses  SimpleSockets; var  Sock: TSocket;  Msg: TReceiveFromStringMessage;begin  Sock := UDPSocket(stDualStack);  try    // Binding to 0-IP (here in ipv6 format) binds to any address, with stDualStack both IPv4 and IPv6    Bind(Sock, '::0', 1337);    repeat      while not DataAvailable(Sock) do         WriteLn('Waiting for Data'); // do whatever while waiting for your data      Msg := ReceiveStrFrom(Sock);      WriteLn('Ping received from ', Msg.FromAddr.Address, ':', Msg.FromPort,' message: ', Msg.Data);      SendStrTo(Sock, Msg.FromAddr, Msg.FromPort, Msg.Data + 'Pong');    until False;  finally    CloseSocket(Sock);  end;  ReadLn;end.If you don't want 100% CPU usage, you can add a timeout to DataAvailable, which will basically say: wait until data is available or until the timeout is reached (See DataPendingTest.pas in the examples folder). Personally I find this much easier than LNet. And if you want to make this threaded, just put that code in threads yourself. PS: Note how this uses DualStack, meaning it is a server socket that support both IPv4 and IPv6 completely transparently, this is something that LNet can't even do ---------------------------------------------------------------------- alpine · 2022-07-10 21:08 @Warfley The LNet isn't so bad, actually I've checked it - the Sock.CallAction will call fpSelect() and as long there is a Sock.Timeout > 0 it will behave as expected. Default is Sock.Timeout = 0 and because of that the CPU utilization will be 100% unless a sleep(x) added (but then everything will slow down). ---------------------------------------------------------------------- 440bx · 2022-07-10 22:31 Quote from: Warfley on July 10, 2022, 08:01:33 pmAnd about the CPU utlization with sleep(1), I know the sleep discussion went a bit of the rails, but the core message what I wanted to get across is, sleep(1) sleeps several MS, Not in a properly running Windows installation.  That statement is simply incorrect. On a properly running Windows installation Sleep(n) will usually be very close to "n".  The exceptions are, when something isn't quite right in the Windows installation or the system is very busy.  The Windows scheduler isn't a run of the mill thread that only gets to execute every 15ms (or whatever the thread quantum may be.) Scheduler code runs a lot more often than that.  In addition to that, when a thread is sleeping, the quantum is irrelevant because the quantum applies to threads that are running not sleeping. In a Windows installation that is functioning as it should, the Sleep(n) measurements should be very similar to the ones I posted. In VMs the divergence is larger but, it should still be within about 20%. ---------------------------------------------------------------------- Warfley · 2022-07-10 22:35 That CallAction checks if data available isn't any better, there is still no documentation, and the name CallAction does not imply that in anyway that it does any of that. Like how are you as someone who has never used LNet supposed to know any of that? Even if I ctrl click on the Identifier to look into CallAction, all I see is: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i procedure TLUdp.CallAction;begin  if Assigned(FEventer) then    FEventer.CallAction;end;Where FEventer.CallAction is an empty virtual method. Also structurally LNet does not make sense, TLUDP is a TLConnection and therefore has things like  Disconnect, which does not make sense in any UDP context, and if you look into it it has a bunch of TCP only code and basically just closes the socket for UDP... Why not have a function CloseSocket for UDP instead of inheriting all the TCP code that does not apply to UDP. I don't even see how anyone can have the audacity to call any of this lightweight, because to me carrying a bunch of TCP only code around in your UDP class is pretty much the opposite of leightweight. I consider myself quite knowledgable about writing networking code, I've written a few networking libraries myself. But when I look at that LNet code, I don't understand what is happening. Take indy for comparison, indy is not the gold standard for ease of understanding at all, but in indy everything at least makes sense, if you create an UDP server, you add the bindings to the IP addresses, then you set active to true and it will start serving requests until you set active to false. Indy is not perfect, but thats at least easy to understand and no ambigous "CallAction" or "Disconnect" or other things whose names make 0 sense in this context. LNet is really terrible, probably the worst networking library I have seen until now. And the worst thing is, it is one of these libraries that tries to be "intelligent" and tries to anticipate what the programmer wants, e.g. by using non blocking calls by default, where you simply ask yourself why? Why do something that can be considered rather special (because as explained above, you usually don't need non blocking calls) by default? But I think I ranted enough about this library. I can just repeat, for simple things like this, I don't see the need for any library, just to use the berkley sockets api, which is already a very simple API. Maybe with a slim wrapper as I posted above, maybe not if one is comfortable in writing the helper functions directly, thats up to everyone how they please. And for more complex stuff like when threading or other more advanced stuff is required, there are much better libraries available, as much as I like to make fun of Indy for beeing bloated, it works, is well structured and easiely understandable, and will have everything you could ever wish for, if it's thread support, SSL support, Non blocking sockets, you name it. (Even though I would also recommend to implement threading by yourself, threading is hard, so it's best to exactly know what happens when) ---------------------------------------------------------------------- Warfley · 2022-07-10 22:37 Quote from: 440bx on July 11, 2022, 12:31:53 amIn a Windows installation that is functioning as it should, the Sleep(n) measurements should be very similar to the ones I posted. Remind me again, on how many different machines and windows versions have you tested this? Because as far as I remember I tested 5 different configurations on different hardware, some VMs, some not and all showed the opposite of what you where claiming... Maybe you should get some evidence before you continue talking... And if you have a VM that shows your behavio when freshly booted (as you claim) please make a snapshot and upload it so I can run it on my machine and verify it. This should be extremly easy, because from all my tests I never came close to your results. Also ive noticed that you never used the same code as I (which i posted here) for doing the measurements... Is it because you cant get your (wrong) results with a comparable test or do you think it is methodologically wrong? If you think the latter please let me know, otherwise, please use the same code or share your measurement code (including timing code) so we have comparable results And if you continue talking stuff without any evidence behind it, I will start calling you what you are, a liar. At some point my good faith is over ---------------------------------------------------------------------- alpine · 2022-07-10 23:08 @Warfley I'm just saying that LNet is not so bad considering the OP issues in that thread, that's all. Already at post #153, I was just curious what so difficult in the UDP one may have and I found most of the time two people fight about the peculiarities of the scheduler and the purpose of the sleep function. But the OP trouble is how to do an I/O properly and the fpSelect() is the right way for that, isn't it? It will give up on the slice and so on. There is nothing wrong to work with non-blocking sockets as long as you do it properly. Most of the servers do it. Yes, that requires more deep knowledge, but what else doesn't requires it? ---------------------------------------------------------------------- Warfley · 2022-07-10 23:21 QuoteThere is nothing wrong to work with non-blocking sockets as long as you do it properly. Most of the servers do it. Yes, that requires more deep knowledge, but what else doesn't requires it?Usually non blocking socjets are used to serve many sockets on one thread to have more scalable server. The problem here is that non blocking polling still requires going over every socket to see if they have any data I.e. O(n). For this, at least on Linux you should rather use epoll, which is event based and therefore functions in O(1), and then use blocking calls afterwards, that said if you really need every bit performance you use epoll with a timeout of 0 (I.e. non blocking) in a loop to avoid context switchens, and also der the sockets non blocking because this optimises the usage of Pool on large buffers, but this is just necessary for extremely utilized servers. For everything else non nlocking should be enough Only exception for this is non blocking accept, this is something um you need to do if you have a lot of incomming connections, because the TCP handshake takes ages But aside from that, if you use epoll consistently blocking calls are no problem Here is an article describing when to use both in combination: https://eklitzke.org/blocking-io-nonblocking-io-and-epoll But again this is something you only need for highly parallelizes high availability servers. I think for most people this is not so relevant ---------------------------------------------------------------------- 440bx · 2022-07-10 23:55 Quote from: Warfley on July 11, 2022, 12:37:55 amAnd if you continue talking stuff without any evidence behind it, I will start calling you what you are, a liar. At some point my good faith is over I posted the numbers I get.  That's beyond evidence, it's _fact_. As I said before, there is nothing special about my Windows installation with the possible exception that I keep it very clean.  Other than that, The ISO is from MS (the same one millions of other people have downloaded.) and there are very _few_ programs installed on it, VMware Workstation and TCCLE, which are the only two are use on a daily basis. To recap, there is nothing special about my installation and you, and everyone else, should get results very close to those I posted.  If you don't get those results (outside of a VM) then you really should be looking at your Windows installation. As far as testing Sleep, I've been testing its behavior for decades, I have long lost count of the number of machines in which I have tested it. And attached is a screenshot for you. ---------------------------------------------------------------------- rvk · 2022-07-11 00:01 Quote from: Warfley on July 11, 2022, 12:35:33 amLike how are you as someone who has never used LNet supposed to know any of that? Even if I ctrl click on the Identifier to look into CallAction, all I see is: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i procedure TLUdp.CallAction;begin  if Assigned(FEventer) then    FEventer.CallAction;end;Where FEventer.CallAction is an empty virtual method. That's because you didn't look any further. I also took a look and saw FEventer.CallAction pointed to an abstract method. But that's because TLUDP doesn't implement that abstract method but another one (not behind my computer now so I can't check the name). If you follow the correct one you end up in an event which uses fpSelect (like y.ivanov already mentioned). Synapse and Lazarus sockets etc work the same way (with the option of implementing several different method of FEventer.CallAction. I didn't look very hard for documentation but shooting down code just by looking at the code and not understanding it, is wrong. In that case you can say the same thing about Lazarus and FPC on low level (with their multiple use of inc files which makes stepping through the source code that much harder than in Delphi). There is also very little documentation about the way it is fine there. BTW The description in the docs for CallAction which explains the name. QuoteILComponent.CallAction - Method to eventize the component This method is used to "eventize" the activity in given component. It ensures that all network events are noticed and acted upon. I also prefer another library (synapse) but that's more because I'm more familiar with that one and more bare boned (events and threads would need to be implemented but the user). QuoteIndy is not perfect, but thats at least easy to understand and no ambigous "CallAction" or "Disconnect" or other things whose names make 0 sense in this context.The need for calling CallAction is because TS insisted on implementing this in a program without TApplication. I'm sure if you wanted to do this with Indy components you would have the same problem and would need to call some kind of event-handler to make sure it could do its thing. Edit: And the more you dive into the source, the more you understand. QuoteAlso note that you need to only call CallAction on one of the protocols (or directly on the eventer) once per iteration (if you’re on non-visual lnet of course), otherwise you’re just calling the same thing multiple times (Protocol.CallAction calls Eventer.CallAction)https://lnet.wordpress.com/usage/handles-eventers- and-events/ (That's why you ended up in an empty abstract event, which was the wrong place) And if you use the components you don't need CallAction (same with Indy). ---------------------------------------------------------------------- Warfley · 2022-07-11 07:32 Quote from: 440bx on July 11, 2022, 01:55:51 amI posted the numbers I get.  That's beyond evidence, it's _fact_. I don't doubt that you are getting these results, I doubt that this is the default behavior. Because from what you have written tell you only got these results consistently on one machine, and got similar results, but not consistently on a vm on this machine, and did not get these results on another (win10) vm on this machine. I tested 3 VMs as well as native on 2 different machines, and I always got the exact same results everywhere. So it is quite clear that your behavior cannot be the normal windows behavior, as you only get the results consistently on one specific setup, while I get the other results consistently on all setups. And this is why I call you a liar, calling your observed behavior the default behavior, when you know that the other behavior can be observed consistently accross many machines both virtual and physical, means that you are saying something that you know cannot be true, by definition a lie. ---------------------------------------------------------------------- alpine · 2022-07-11 07:38 Quote from: Warfley on July 11, 2022, 01:21:22 amQuoteThere is nothing wrong to work with non-blocking sockets as long as you do it properly. Most of the servers do it. Yes, that requires more deep knowledge, but what else doesn't requires it?Usually non blocking socjets are used to serve many sockets on one thread to have more scalable server. *snip* Yes, I know. I also wrote an app or two. Apart from the Berkeley sockets (incl. Winsock) I've used also other things, such as Win I/O completion ports, embedded libs like uIP, etc. And I prefer to use raw sockets just like you. @prodingus Does my suggestion solved your issue? BTW You should set the timeout for both sides, i.e. for the listener too, otherwise it will bump CPU to 100% ---------------------------------------------------------------------- Warfley · 2022-07-11 08:13 Quote from: rvk on July 11, 2022, 02:01:37 amThat's because you didn't look any further. I also took a look and saw FEventer.CallAction pointed to an abstract method. But that's because TLUDP doesn't implement that abstract method but another one (not behind my computer now so I can't check the name). If you follow the correct one you end up in an event which uses fpSelect (like y.ivanov already mentioned). Synapse and Lazarus sockets etc work the same way (with the option of implementing several different method of FEventer.CallAction. I'm not saying it's impossible to find out, but if you have to dive deep into the source code of a library, it's a bad library. Quote from: rvk on July 11, 2022, 02:01:37 amI didn't look very hard for documentation but shooting down code just by looking at the code and not understanding it, is wrong. In that case you can say the same thing about Lazarus and FPC on low level (with their multiple use of inc files which makes stepping through the source code that much harder than in Delphi). There is also very little documentation about the way it is fine there. Yes I also find that the fpc and lazarus documentation are quite lacking, in fact I think this is the biggest problem with it. Every time you have to look into the source is annying and a failure of documentation, and every time it is more than ctrl click away (because of virtual methods), it basically doesn't exist for some users. The gold standard of course is if the code speaks for itself (which of course is not always possible), and if it doesn't, there should be enough documentation to resolve this. Quote from: rvk on July 11, 2022, 02:01:37 amBTW The description in the docs for CallAction which explains the name. QuoteILComponent.CallAction - Method to eventize the component This method is used to "eventize" the activity in given component. It ensures that all network events are noticed and acted upon.Then why isn't it named "HandleNetworkEvents" or "ReactToNetworkEvents", or anyhting that conveys this message? Sure handling events involves calling some actions to handle those events, but this would be like saying "moving my eyes" instead of "reading". This reminds me of some very old projects of mine, where I was really lazy and called functions just "DoTheThing", it was good for a joke, and that I knew the code base in and out so this wouldn't be a problem, but when coming back after a few years, I had no idea what was happening in the code. Naming is probably the most important part of Programming, especially for libraries, and I can't find anything positive about the name. It's not even short, it could easiely just be called "act" to have the same amount of useful information in the name but at least be short. Quote from: rvk on July 11, 2022, 02:01:37 amThe need for calling CallAction is because TS insisted on implementing this in a program without TApplication. I'm sure if you wanted to do this with Indy components you would have the same problem and would need to call some kind of event-handler to make sure it could do its thing.In the general flow Indy is not directly comparable (I just brought it up because of the naming and structure, not necessarily that it does the same thing in the same workflow), because in Indy handles non blocking sockets differently (basically it does not try to be smart as LNet but behaves exactly the sockets API would, i.e. the actual calls are non blocking, not that there is some magic CallAction function). If you use the event based Server component, it will execute the listening in a thread and then, if you want to handle the incomming events on the mainthread (default behavior) you must check the main threads eventQueue with CheckSynchronize. This is in workflow similar to CallAction, but there are two main differences, first the event handling is completely done with the general RTL functionality, that is likely to be known already, rather than introducing something specific to that library. But more importantly, it's not named CallAction (I can't stress enough how bad this name is). Quote from: rvk on July 11, 2022, 02:01:37 amEdit: And the more you dive into the source, the more you understand. And the more you dive into the source, the more you do something that should not be required to use a library. Having to consort the manual/documentation is bad, and usually a sign of bad structure, having to consult the source is worse. I shit a lot about Indy, but just by trial and error it is usally quite easy to find out how to do things. It is not perfect, for example to find out that the TUDPServer uses threads you also must look into the source, but at least the naming and structures make sense (e.g. UDP is not a connection). The best libraries are those that can be discovered with nothing but autocomplete. Sure most libraries fall short of that benchmark, and I've written a few libraries myself that are too complicated to be self documenting, but then I usually try to either have extensively commented examples and some form of documentation (which in my case is usally a multi page readme, but TBH I am not so good with documentation myself). LNet has an UDP example, see what they write for CallAction: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i FCon.CallAction;     // "eventize" the event loop in lNetGreat... words, but why not write anything someone can understand, e.g. "React to network events or wait up to Timeout if no events occured", you know the thing that is actually happening here. ---------------------------------------------------------------------- rvk · 2022-07-11 08:37 Quote from: y.ivanov on July 11, 2022, 09:38:18 am@prodingus Does my suggestion solved your issue? BTW You should set the timeout for both sides, i.e. for the listener too, otherwise it will bump CPU to 100% I'm not TS but yes, it does work a lot better. Putting in Sock.Timeout := 10000; makes it go down to 50% cpu in my VM (1CPU) without any sleep-lines. (BTW, The reason it reaches 100% in a 1CPU VM is that there are multiple processes at work. project.exe is one of them (50%), conhost is another (30%) and Interrupts (15%). In a multi CPU machine, the processes are divided over the CPUs and the stress on the machine is a lot less.) Quote from: Warfley on July 11, 2022, 10:13:37 amQuote from: rvk on July 11, 2022, 02:01:37 amThat's because you didn't look any further. I also took a look and saw FEventer.CallAction pointed to an abstract method. But that's because TLUDP doesn't implement that abstract method but another one (not behind my computer now so I can't check the name). If you follow the correct one you end up in an event which uses fpSelect (like y.ivanov already mentioned). Synapse and Lazarus sockets etc work the same way (with the option of implementing several different method of FEventer.CallAction. I'm not saying it's impossible to find out, but if you have to dive deep into the source code of a library, it's a bad library. That's why Indy would be considered a "bad library" too, according to the same standards (although you are correct in that it does things differently). Same goes for everything in Lazarus and FPC. The only thing is that the maker of a library could name some things better or differently. But maybe English isn't their first language or they had some other reason in mind (which was the case here). But CallAction isn't normally called. If you work with the components in a Lazarus program, there is no need to call CallAction. It is done for you. QuoteThe CallAction method of all “Connections” is a method which “eventizes” the whole connection. It is valid only in non-visual lNet. Whenever you call this method, all sockets of given connection are checked for status updates (eg. if something can be received, if I can send…) and appropriate callbacks are fired. You need to call this method periodicly to assure functionality. You need to CallAction periodicly if you want to know if you can receive. (get)https://lnet.wordpress.com/ Because lnet tries to be a small simple library and there aren't any threads, there must be something called to handle events. Creator decided it should be CallAction. I have no problems with that. There a lot of things in Indy that annoy me a LOT more. (And like I already said, I like bare boned sockets or Synapse more, just because I need to do everything myself and have all in hand.) ---------------------------------------------------------------------- alpine · 2022-07-11 08:38 Quote from: Warfley on July 11, 2022, 10:13:37 am *snip* If you use the event based Server component, it will execute the listening in a thread and then, if you want to handle the incomming events on the mainthread (default behavior) you must check the main threads eventQueue with CheckSynchronize. This is in workflow similar to CallAction, but there are two main differences, first the event handling is completely done with the general RTL functionality, that is likely to be known already, rather than introducing something specific to that library. *snip* I've got an issue making this recently - using such a server component in Linux works pretty well, but when I tried to port that daemon in Windows as a TServiceApplication found that CheckSynchronize doesn't work as expected. And to make it work I had to dig deep into the RTL sources and to read what was written in the forum, considering it may be obsolete, not relevant, etc. Having such an explicit method could save me a lot of time. Just as a counterpoint. Not saying anything about the name. ---------------------------------------------------------------------- 440bx · 2022-07-11 08:39 Quote from: Warfley on July 11, 2022, 09:32:39 amI don't doubt that you are getting these results, I doubt that this is the default behavior. Because from what you have written tell you only got these results consistently on one machine, and got similar results, but not consistently on a vm on this machine, and did not get these results on another (win10) vm on this machine. I tested 3 VMs as well as native on 2 different machines, and I always got the exact same results everywhere. So it is quite clear that your behavior cannot be the normal windows behavior, as you only get the results consistently on one specific setup, while I get the other results consistently on all setups. And this is why I call you a liar, calling your observed behavior the default behavior, when you know that the other behavior can be observed consistently accross many machines both virtual and physical, means that you are saying something that you know cannot be true, by definition a lie. You should measure your language a little more carefully.  It also seems  that you should learn a little about Windows and how an O/S scheduler works. As far as VMs, as I hope you've been able to figure out, they _emulate_ hardware and no matter how good the emulation is, it's not perfect.  I don't expect VMs to operate exactly the same as real hardware but, apparently you do. I'm not going to call you a liar but, the word that comes to mind starts with "i" and, that wouldn't be a lie, it would be an accurate description of your behavior. That you don't seem to understand and that your non-VM machines are not working properly, is _your_ problem and, your problems don't make anyone a liar. You got the screenshot you requested and, that is the picture of the reality you are unable to accept. ---------------------------------------------------------------------- alpine · 2022-07-11 08:52 Quote from: rvk on July 11, 2022, 10:37:59 amQuote from: y.ivanov on July 11, 2022, 09:38:18 am@prodingus Does my suggestion solved your issue? BTW You should set the timeout for both sides, i.e. for the listener too, otherwise it will bump CPU to 100% I'm not TS but yes, it does work a lot better. Putting in Sock.Timeout := 10000; makes it go down to 50% cpu in my VM (1CPU) without any sleep-lines. As I expected. It is not to use a big timeout (I don't think you should go over, e.g. 100 ms) but the main point is to give up the slice for the reason of I/O. Otherwise it is just a waste of time, a delay. ---------------------------------------------------------------------- Warfley · 2022-07-11 09:30 Quote from: 440bx on July 11, 2022, 10:39:20 amAs far as VMs, as I hope you've been able to figure out, they _emulate_ hardware and no matter how good the emulation is, it's not perfect.  I don't expect VMs to operate exactly the same as real hardware but, apparently you do. [...] That you don't seem to understand and that your non-VM machines are not working properly, is _your_ problem and, your problems don't make anyone a liar. Please stop talking it is embarrassing how little you know about the subjects you talk about. Virtualization is not emulation: https://www.dell.com/en-us/blog/emulation-or-virtualization-what-s- the-difference/ A virtual machine runs on your actual hardware, the processes run on your real CPU and use your real memory, it does not emulate the hardware as you claim. The fact that you get this simple difference wrong shows that you have absolutely no idea about computing at all. I'm sorry but everything you said in this thread so far is factually and provably wrong. You try to lecture me and don't even know the very basics of computing. This is literal first semester CS knowledge, things you learn in the very first few month of getting any education in that area The reason I call you a liar is not because you are wrong. You don't know much about virtualization and operating systems and this is fine, many people don't know this. But because you are claiming things that are wrong after being proven that they are wrong, and telling something that is not true while knowing it is not true is a lie. ---------------------------------------------------------------------- MarkMLl · 2022-07-11 09:39 Quote from: Warfley on July 11, 2022, 11:30:25 amPlease stop talking it is embarrassing how little you know about the subjects you talk about. Virtualization is not emulation: https://www.dell.com/en- us/blog/emulation-or-virtualization-what-s-the-difference/ A virtual machine runs on your actual hardware, the processes run on your real CPU and use your real memory, it does not emulate the hardware as you claim. The fact that you get this simple difference wrong shows that you have absolutely no idea about computing at all. I'm sorry but everything you said in this thread so far is factually and provably wrong. You try to lecture me and don't even know the very basics of computing. That specific use of "virtual machine" is comparatively recent. From the time that IBM introduced VM up until some of the later 68Ks etc., it was common to refer to "virtualised hardware", and on the '386 there was "Virtual 8086 mode". At the same time, "virtual machine" was still synonymous with "interpreter", possibly augmented with JIT translation, as used by the P-System, the PARC implementation of Smalltalk and so on. It's only recently that people who are convinced that computing begins and ends with PC have redefined "Virtual Machine" to be the same as "virtualised hardware", and even there there are grey areas depending on precisely what happens when e.g. direct access of a network device is attampted. So please, both of you: cool it. And there's no need for anybody to use the L-word. MarkMLl ---------------------------------------------------------------------- Warfley · 2022-07-11 09:51 Quote from: rvk on July 11, 2022, 10:37:59 amThat's why Indy would be considered a "bad library" too, according to the same standards (although you are correct in that it does things differently). Same goes for everything in Lazarus and FPC. Yes and no. Indy get's away with a lot more because it is much better structured. With Indy I usually got most things I wanted by must creating a component, and then pressing ctrl+space to get code completion and just look at the names of the functions and properties. For example to create an UDP server, I never created a UDP server with indy, it took me like 10 minutes to learn this from scratch by just looking at the code completion, no need to look in the documentation nor the source. So yes the documentation is lacking, but at least I get around without spending hours on googling or reading the source. And again, I agree that the documentation in Lazarus and FPC is very lacking, and there are a lot of things that I think are bad because of that. I often search for "delphi xxx" rather than "lazarus xxx" because I know the delphi documentation is going to be better. Just compare the Documentation of Synchronize in Delphi: QuoteDescription Executes a method call within the main thread. Synchronize causes the call specified by AMethod to be executed using the main thread, thereby avoiding multithread conflicts. The AThread parameter associates the caller thread. For static methods, you can associate AMethod with any thread using the AThread parameter. Also, you can use nil/NULL as AThread parameter if you do not need to know the information for the caller thread in the main thread. In the current implementation, the Synchronize method can use associated thread information to wake-up the main thread on Windows platforms. If you are unsure whether a method call is thread-safe, call it from within the Synchronize method to ensure that it executes in the main thread. Execution of the current thread is suspended while the method executes in the main thread. Warning: Do not call Synchronize from within the main thread. This can cause an infinite loop. Note: You can also protect unsafe methods using critical sections or the multiread exclusive-write synchronizer. An example of when you would want to use Synchronize is when you want to interact with either a VCL or a FireMonkey component. Use an in- place anonymous method to solve the problem of passing variables to the method you want to sychronize: And here is the same description for FPC: Quote(yes there is no description, the documentation is literally just the function signature) This is really bad, and I think the biggest problem with FPC/Lazarus. Again the problem with LNet is not the missing documentation, nor the naming, it's all of the small issues combined that makes it exceptionally bad. Especially the general structure in combination with the naming and lack of documentation. It's not just CallAction, as mentioned earlier, why is a UDP socket a connection and has methods like disconnect? If I see something like that in a code, I would either assume that this socket is a conncetion based socket, like TCP, or if I know it's UDP, I would consider this to be a mistake. It's just bad from front to back. I haven't seen anything good about this library so far, except maybe that it works ---------------------------------------------------------------------- Warfley · 2022-07-11 09:58 Quote from: MarkMLl on July 11, 2022, 11:39:43 amThat specific use of "virtual machine" is comparatively recent. From the time that IBM introduced VM up until some of the later 68Ks etc., it was common to refer to "virtualised hardware", and on the '386 there was "Virtual 8086 mode". At the same time, "virtual machine" was still synonymous with "interpreter", possibly augmented with JIT translation, as used by the P-System, the PARC implementation of Smalltalk and so on. It's only recently that people who are convinced that computing begins and ends with PC have redefined "Virtual Machine" to be the same as "virtualised hardware", and even there there are grey areas depending on precisely what happens when e.g. direct access of a network device is attampted. "recently" in this context means just 30-40 years old. From the 70 years of computing history, it's about half of that period. The current understanding of VM is closer to the introduction of the Personal Computer than it is to today. Arguing something like this is a very cheap opt out, and I don't let something like this count. I don't want to be lectured by someone about how modern computers work who 1. either doesn't know anything about computers, or  2. whose knowledge of computers is stuck 30 years ago. Maybe you are right and I shouldn't call him a liar, but at the very least he ist trying to lecture me on things he clearly has no idea about ---------------------------------------------------------------------- rvk · 2022-07-11 09:59 Quote from: Warfley on July 11, 2022, 11:51:42 am... why is a UDP socket a connection and has methods like disconnect?I was under the impression Indy also has Connect and Disconnect for TIdUDPClient  Yep. Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i   TIdUDPClient = class(TIdUDPBase)//...  protected    procedure DoOnConnected; virtual;    procedure DoOnDisconnected; virtual;  public    procedure Connect; virtual;    procedure Disconnect; virtual;    function Connected: Boolean;//...  end; But Indy is a much more structured and comprehensive library. It works with threads etc. But it isn't written in 1000 lines of code (which lnet is, well, it was, it's a bit larger now ). ---------------------------------------------------------------------- MarkMLl · 2022-07-11 10:14 Quote from: Warfley on July 11, 2022, 11:58:46 amArguing something like this is a very cheap opt out, and I don't let something like this count. I don't want to be lectured by someone about how modern computers work who 1. either doesn't know anything about computers, or  2. whose knowledge of computers is stuck 30 years ago. Maybe you are right and I shouldn't call him a liar, but at the very least he ist trying to lecture me on things he clearly has no idea about I for one would prefer that you didn't use insults like that in open discussion. It's not necessary, it lowers the tone of the forum, and it definitely doesn't help resolve any of OP's issues. As far as VMs are concerned: people still refer to the Java VM, the .Net VM and so on, and in fact the entire field of virtualisation, simulation and emulation is generally poorly-delimited. Whole-system virtualisation is one particular subset of that field, and punding your shoe on the desk and demanding that your definition is the only acceptable one doesn't help anybody. And while I have absolutely no intention of investing my limited time in this, I'd add that whenever I've seen anybody demonstrate absolutely consistent results from a non-realtime scheduler it's turned out that his methodology is flawed. Although sometimes it's taken a 'scope to track down what was really going on. MarkMLl ---------------------------------------------------------------------- 440bx · 2022-07-11 10:16 Quote from: Warfley on July 11, 2022, 11:58:46 amMaybe you are right and I shouldn't call him a liar, You had an epiphany, it might be worth marking the date on the calendar. Quote from: Warfley on July 11, 2022, 11:58:46 amtrying to lecture me on things he clearly has no idea about Trying to educate you obviously failed and, lecturing you apparently did too. ---------------------------------------------------------------------- MarkMLl · 2022-07-11 10:21 Quote from: 440bx on July 11, 2022, 12:16:50 pmQuote from: Warfley on July 11, 2022, 11:58:46 amMaybe you are right and I shouldn't call him a liar, You had an epiphany, it might be worth marking the date on the calendar. Quote from: Warfley on July 11, 2022, 11:58:46 amtrying to lecture me on things he clearly has no idea about Trying to educate you obviously failed and, lecturing you apparently did too. (Clears throat) Please don't make things worse. I think this subthread could usefully wind down. MarkMLl ---------------------------------------------------------------------- Warfley · 2022-07-11 10:36 Quote from: 440bx on July 11, 2022, 12:16:50 pmTrying to educate you obviously failed and, lecturing you apparently did too. Sorry, I don't need education from someone who clearly doesn't know what a VM is. You know the truely sad thing is, before writing a post, I usually try to double check if what I am writing is correct, I get my old university scripts, some books I have around, google for documentation or other information and always try the things out in code. Meanwhile you blast out things like a VM is an emulator, which is wrong after just two seconds of googling. I have a question, you where wrong with all of your statements in this thread, but let's focus on the last one about vms and emulation, because there is no way you can argue against that you where completely wrong ther. Doesn't it bug you that you are so easiely wrong on such things? Why can't you accept that someone else might know more about a topic than you? You claimed it like it is some form of gotcha moment, to just say that I don't know what a VM is, yet you where completely and utterly wrong about that. Doesn't that make you feel you should maybe look up stuff before claiming it? Something like the claim that a vm is an emulator should have never happend if you had at least some rigorous approach to verifying your claims before making them. Yet you are wrong so many times and refuse to learn. I'm just curious if you don't have any drive to at least try to be right, at least once, or why are you always blasting falshoods with the confident like you where the omniscient tech god who knows everything ---------------------------------------------------------------------- 440bx · 2022-07-11 10:51 Quote from: MarkMLl on July 11, 2022, 12:21:20 pm(Clears throat) Please don't make things worse. I think this subthread could usefully wind down. MarkMLl You're right about that.  I'm not making things better but, he really asked for it and, continues asking for it. Quote from: Warfley on July 11, 2022, 12:36:06 pmSorry, I don't need education from someone who clearly doesn't know what a VM is. Neither you nor anyone else for that matter. Quote from: Warfley on July 11, 2022, 12:36:06 pmYou know the truely sad thing is, before writing a post, I usually try to double check if what I am writing is correct, In that case, I suggest you start triple checking because double checking isn't doing the trick. Quote from: Warfley on July 11, 2022, 12:36:06 pmMeanwhile you blast out things like a VM is an emulator, which is wrong after just two seconds of googling. I have a question, you where wrong with all of your statements in this thread, but let's focus on the last one about vms and emulation, because there is no way you can argue against that you where completely wrong ther. Doesn't it bug you that you are so easiely wrong on such things? Why can't you accept that someone else might know more about a topic than you? There is a little problem with that belief of yours.  The problem is this: have you tried running a VM inside a VM ?... just in case you haven't, current VM software can't do it.  Any ideas why by any chance ?  Can you explain why most VM software can't do that ? Quote from: Warfley on July 11, 2022, 12:36:06 pmSomething like the claim that a vm is an emulator should have never happend if you had at least some rigorous approach to verifying your claims before making them. A VM emulates a machine, some of that emulation is done with hardware assistance from the CPU and some of it is done in software. Quote from: Warfley on July 11, 2022, 12:36:06 pmI'm just curious if you don't have any drive to at least try to be right, at least once I've been right more than once and the screen shot I posted, which you requested, proves it.  Look it up again if you need to. ---------------------------------------------------------------------- marcov · 2022-07-11 10:58 IIRC lnet's author mainly wanted to use epoll/kqueue or select as last resort. I objected that lnet also should work on lnet (I was and still am an Indy user), so he also dove into Windows. Ales was quite clever, but the result, like Synapse, is a bit too Unix centric for my tastes and Indy is simply more complete and well supported by Remy. (nowadays even on this forum) As it happens in the last few years I picked up indy again (both udp and tcp clients and servers for internal communication of machine parts and to the outside world, so no protocol usage), and put all of them in an own  thread to minimally disturb other threads, and also avoid polling as much as possible. I'm quite happy how it all turned out, and it turns this can be done without modifying Indy. (but the result I think is Windows specific) ---------------------------------------------------------------------- Warfley · 2022-07-11 11:05 Quote from: 440bx on July 11, 2022, 12:51:31 pmThere is a little problem with that belief of yours.  The problem is this: have you tried running a VM inside a VM ?... just in case you haven't, current VM software can't do it.  Any ideas why by any chance ?  Can you explain why most VM software can't do that ?You must be kidding right? This is possible, vmwares ESXi virtualizer can do this for ages, virtualbox is also easiely capable of doing that and even Microsofts HyperV is capable of doing that since 2016: https://docs.microsoft.com/en-us/virtualization/hyper-v-on- windows/user-guide/nested-virtualization Please at least use google before trying to set up a gotcha question. This is just embarrasing how confident you are in being wrong. Also if these VMs would actually emulate the hardware this would be no problem, for example I can run an ARM emulator within an x86_64 VM, and can easiely run a Playstation emulator within that emulator. Quote from: 440bx on July 11, 2022, 12:51:31 pmA VM emulates a machine, some of that emulation is done with hardware assistance from the CPU and some of it is done in software.No it executes code on the real hardware in an emulated environment. And as the program measuring sleep time does not interface with the environment (i.e. network, external devices, etc.) it doesn't matter that they are emulated. It is the real NT kernel running on a real CPU using real memory. Quote from: Warfley on July 11, 2022, 12:36:06 pmI've been right more than once and the screen shot I posted, which you requested, proves it.  Look it up again if you need to. Well said in a post where you where again completely and utterly wrong, and could have found that out with just 1 second of googling ---------------------------------------------------------------------- 440bx · 2022-07-11 11:55 Quote from: Warfley on July 11, 2022, 01:05:12 pmNo it executes code on the real hardware in an emulated environment. And what environment is it that it is emulating ?... it isn't emulating Windows because that has to be installed. The saying goes, you should quit while you're ahead but, in your case, you should quit before you get further behind. It would be good too because this thread isn't about your religious beliefs about the scheduler and/or VMs.  That would be convenient, no more litter. ---------------------------------------------------------------------- Warfley · 2022-07-11 12:05 Quote from: 440bx on July 11, 2022, 01:55:00 pmAnd what environment is it that it is emulating ?... it isn't emulating Windows because that has to be installed. Peripheral hardware such as network adapter, disk, display, etc. You know all the stuff that isn't part of a core computer. Why would I be talking about windows? You must truely know nothing about computers if you would consider this being a vritual environmnet. Quote from: 440bx on July 11, 2022, 01:55:00 pmIt would be good too because this thread isn't about your religious beliefs about the scheduler and/or VMs.  That would be convenient, no more litter. All I'm doing is pointing out how wrong you are. I don't care that you know so little about computing, but the worst thing that could happen is that someone would read your posts and assume you are anything but a fool who doesn't know anything you are talking about, so every time you write something stupid, like that you cant create a vm inside a vm, which is factually wrong, I need to point out how wrong you are. Note that every time you say something stupid, I don't "religously" just say you are wrong, I always provide counter examples or links with information how wrong you are. You are the one who never put anything forward Also it's funny that you are completely wrong with your gotcha questions so you just silently drop these points. Why can't you just accept that you were wrong, like you were with everything else in this thread? ---------------------------------------------------------------------- 440bx · 2022-07-11 12:11 Quote from: Warfley on July 11, 2022, 02:05:51 pmAlso it's funny that you are completely wrong with your gotcha questions so you just silently drop these points. Why can't you just accept that you were wrong, like you were with everything else in this thread? Of course... and the Democrats should admit Biden stole the U.S election and Ukraine attacked Russia. ---------------------------------------------------------------------- Warfley · 2022-07-11 12:14 So you say you weren't wrong with nested VMs? I'm curious how you deny reality for this case. ---------------------------------------------------------------------- 440bx · 2022-07-11 12:31 Quote from: Warfley on July 11, 2022, 02:14:13 pmSo you say you weren't wrong with nested VMs? I'm curious how you deny reality for this case. Well... a little googling brought this up: QuoteNested virtualization is not automatically offered as a feature and this is also true for various third party virtualizers. For example while the VirtualBox virtualizer has existed for years, the ability to run VirtualBox inside VirtualBox using Intel CPUs was only offered as a feature in v6.1 released in 2020. [2] This demonstrates that extra code is required for this functionality and that also implies a greater attack surface.Apparently, you are right on that one, as of 2020 (kind of recent) it is possible to run a VM in another VM (quite likely because CPU manufacturers have been adding circuitry in order to enable that.) I seriously doubt that ability is available on CPUs 5 years or older. You got one right... good for you but, that doesn't mean the VM isn't emulating functionality in software. Far from it. ---------------------------------------------------------------------- marcov · 2022-07-11 12:43 Afaik VMWare already could do this already longer, but the nested VM would not be accelerated. There are several virtualization related extensions, and through the years more and more have become nestable. ---------------------------------------------------------------------- 440bx · 2022-07-11 12:46 Quote from: marcov on July 11, 2022, 02:43:16 pmAfaik VMWare already could do this already longer, but the nested VM would not be accelerated. There are several virtualization related extensions, and through the years more and more have become nestable. My copy of VMware is from 2015 and it does not allow nesting but, it may also be because I'm running an i860. ---------------------------------------------------------------------- Warfley · 2022-07-11 13:11 Quote from: 440bx on July 11, 2022, 02:31:45 pmApparently, you are right on that one, as of 2020 (kind of recent) it is possible to run a VM in another VM (quite likely because CPU manufacturers have been adding circuitry in order to enable that.) I seriously doubt that ability is available on CPUs 5 years or older. As I have written, Windows supported this feature in HyperV since 2016, so for 6 years, which is more than 5 years. So you are wrong... again Also again this point is actually kindof moot anyway, because it actually supports the argument that this is because virtualization is not emulation. In my Emulated QEMU ARM Linux session I can easiely start another emulator, because emulation completely abstracts from the underlying hardware, while virtualization requires hardware and host virtualizer software support to share the hardware with between the different devices, because it is exactly *not emulation* Quote from: 440bx on July 11, 2022, 02:31:45 pmYou got one right... good for you but, that doesn't mean the VM isn't emulating functionality in software. Far from it. So you say that because some things are emulated, this means a VM is an emulator. Guess what, on you normal windows system you also have emulated environments. E.g. virtual network interfaces, like the ones created by your VM host software. Or guess what a virtual filesystem like a network drive is: According to your same logic this means that also a native windows installation is just an emulator. The Core of a VM is exactly to not do emulation but virtualization of hardware. It provides additonally emulated behavior for peripherals, but this is an add-on and is not required for a vm Please stop shifting the discussion. You stated QuoteAs far as VMs, as I hope you've been able to figure out, they _emulate_ hardware and no matter how good the emulation is, it's not perfect.  I don't expect VMs to operate exactly the same as real hardware but, apparently you do.In the context of a program that is only running on the CPU, and does not interact with any peripheral hardware. And this is not emulated, simple as that. Therefore you are just dead wrong here. Also you don't need any emulation with a VM. If you have a dedicated graphics card and networkcard for the VM, you can fully use a VM without any emulated peripherals, just by using bare metal hardware access through PCI pass-through provided by the virtualizer. I actually did something like this once, where I wanted to run my windows partition without having to reboot from linux, so I mounted it into a vm, passed my graphics card through it, let linux only run on the integrated graphics card, an basically had a fully fledged windows on one monitor and a linux on the other one, running on the same machine, both running on fully native hardware Quote from: 440bx on July 11, 2022, 02:46:37 pmMy copy of VMware is from 2015 and it does not allow nesting but, it may also be because I'm running an i860. I think you need EXSi virtual machines, I don't know if it supports HyperV nesting ---------------------------------------------------------------------- 440bx · 2022-07-11 13:36 Quote from: Warfley on July 11, 2022, 03:11:29 pmAs I have written, Windows supported this feature in HyperV since 2016, so for 6 years, which is more than 5 years. So you are wrong... again For someone who pretends to know so much, you sure make statements that are rather flimsy.  It's _not_ just a matter of the O/S, Windows or other, it's much more a matter of having virtualization supported in hardware, that is by the CPU itself. Try to virtualize a machine on an 8086 running MS-DOS ... good luck. ---------------------------------------------------------------------- Warfley · 2022-07-11 13:49 Quote from: 440bx on July 11, 2022, 03:36:23 pmFor someone who pretends to know so much, you sure make statements that are rather flimsy.  It's _not_ just a matter of the O/S, Windows or other, it's much more a matter of having virtualization supported in hardware, that is by the CPU itself. Have you read the link that I posted above? There are no hardware requirements. Any Intel CPU that is VT-x capable can do it. So you could even use older Intel CPUs, even the Intel Atom form 14 years ago. Thats more than 5. Isn't it amazing how much you can learn if you would just read the information given to you. ---------------------------------------------------------------------- prodingus · 2022-07-11 13:52 @y.ivanov Timeout did the trick! Now the cpu is @ ~15-20%. Quotethe Sock.CallAction will call fpSelect() and as long there is a Sock.Timeout > 0 it will behave as expected. Default is Sock.Timeout = 0 and because of that the CPU utilization will be 100% unless a sleep(x) added (but then everything will slow down).Correct. QuoteDoes my suggestion solved your issue? BTW You should set the timeout for both sides, i.e. for the listener too, otherwise it will bump CPU to 100% yes, but as posted here too, https://forum.lazarus.freepascal.org/index.php/topic,13053.0.html, every other value >1 sould add more delay,but it doesn't! It's like I getting alwasys the minimum delay. QuoteYes I also find that the fpc and lazarus documentation are quite lacking, in fact I think this is the biggest problem with it. Every time you have to look into the source is annying and a failure of documentation, and every time it is more than ctrl click away (because of virtual methods), it basically doesn't exist for some users. The gold standard of course is if the code speaks for itself (which of course is not always possible), and if it doesn't, there should be enough documentation to resolve this.AMEN to that too! more replies later, TMI !!! ---------------------------------------------------------------------- 440bx · 2022-07-11 13:56 Quote from: Warfley on July 11, 2022, 03:49:34 pmThere are no hardware requirements. Any Intel CPU that is VT-x capable can do it. No hardware requirements, except VT-x.  nice! ---------------------------------------------------------------------- Warfley · 2022-07-11 14:05 Ah sorry I forgott how ignorant you are. Of course I meant "No *additional* hardware requirements". I assumed that it would be pretty clear that nested Hyper-V would inherit the hardware requirements from Hyper-V, but because I am so nice, for someone like you who doesn't understand this, I of course spell it out directly, you need at least a Hyper-V capable intel-CPU (i.e. one with VT-x support) to use Hyper-V. Again sorry for that, I sometimes forgett that you don't have the brain capacity to make such simple inferrences Or do you just pretend to be stupid so I miss that you completely dropped the point after I again showed you how wrong you are, with information that you already had, you didn't even need to google for that? Nice try, thats now another point you where provably wrong about in this thread I must say this is actually quite fun, it's like arguing with a child, you clearly have no idea what you are talking about, and disproving you literally just takes 2 seconds of reading the stuff. But you are to stubborn to admit that you are out of your debt so you keep bringing up new points that can be proven false within a few seconds of looking it up. And every time you bring something new up, I can show it to be wrong and you are just making a fool out of yourself ---------------------------------------------------------------------- SymbolicFrank · 2022-07-11 14:10 About timing: the original IBM PC had a real-time clock, that generated a timer tick interrupt 18.2 times a second, which was available through the BIOS. Yes, an IMB PC also had a second timer, but it was needed for I/O timing. And the AT had yet another one, but that needed to share an interrupt. If you wanted to time stuff, for example for making sound, you could write a new frequency to the speaker chip 18.2 times a second. That was it. Or, continuously, of course. So, for multitasking, the idea was that you would do a bit and yield to the next process with sleep(0). And with Windows, up to 95, 18.2 times a second the scheduler took control and suspended the task if it didn't and there were more. That first PC ran at 4.77 Mhz and could execute less than 1 million instructions on average each second. Context switching took a fairly long time (~180 instruction cycles) and software floating point calculations were slow. So, if just handling an interrupt or waiting for something, a few thousand cycles is plenty. But reading from memory, performing some floating point ops and writing it back was a fairly lengthy task. And the OS had to handle interruptions for I/O, or doing calculations in between each scan line. So, with, say, 1000 context switches a second, there was little time left for the processes to actually do something useful. And it isn't all that easy to make a good timing source anyway. Like, the IBM PC had a dedicated 16-bit timer as well, which was used for I/O timings and such. And with a single CPU, everything was handled through interrupts. So, it wasn't going to be very accurate anyway. Like: write a byte to the UART, read a few bytes from the disk drive, move the disk head to the next track, read a byte from the other UART, oh, and run a bit of the user's program in between. Although the IBM AT also had a high resolution real-time timer tick (1024 ticks a second). At some point (one of the Pentiums, IIRC) Intel started to add other timings to the CPU. At first it was just the amount of CPU cycles, but that has been extended quite a bit That cycle counter was the first high-speed performance counter. Still not very accurate, but with a high resolution. So, all in all, it took quite a while before good timing was generally available, and all through new API's. The basics are still the same. And therefore Microsoft still expects that sleep(1) results in a delay of ~55 milliseconds. Ok, the resolution is a bit higher, it depends on when in your allotted time slice you execute that call and how many other processes are waiting to be scheduled. And half the reason why that has never been changed is, because many software used that to do their internal timing. It was a choice between that and a spinlock that uses 100% CPU time. ---------------------------------------------------------------------- prodingus · 2022-07-11 14:16 QuoteI for one would prefer that you didn't use insults like that in open discussion. It's not necessary, it lowers the tone of the forum, and it definitely doesn't help resolve any of OP's issues. I would also like just a liitle bit more lower tone, but agreed-to- disagree disagreements are usefull for pointing things out. As far for the network thing, I will try to use something else, like Warfley's wrapper or raw sockets, but my time is limited. There are many things to learn, I was't expecting networking to be so demanding, and to my surprise sleep/delay seems to be only good for delaying, well, I don't know, just message outputs (?), but surelly not main execution loop. ---------------------------------------------------------------------- alpine · 2022-07-11 14:22 Quote from: prodingus on July 11, 2022, 03:52:25 pm@y.ivanov Timeout did the trick! Now the cpu is @ ~15-20%. Quotethe Sock.CallAction will call fpSelect() and as long there is a Sock.Timeout > 0 it will behave as expected. Default is Sock.Timeout = 0 and because of that the CPU utilization will be 100% unless a sleep(x) added (but then everything will slow down).Correct. QuoteDoes my suggestion solved your issue? BTW You should set the timeout for both sides, i.e. for the listener too, otherwise it will bump CPU to 100% yes, but as posted here too, https://forum.lazarus.freepascal.org/index.php/topic,13053.0.html, every other value >1 sould add more delay,but it doesn't! It's like I getting alwasys the minimum delay. *snip* It doesn't work that way. It is a timeout. After that time the OS (fpSelect) will just give up waiting and the control will be transferred back to the calling (your) process again. That is, the CallAction will return with no data processed. In the case data appears earlier - the control will be transferred back without further waiting. ---------------------------------------------------------------------- Warfley · 2022-07-11 14:26 Quote from: prodingus on July 11, 2022, 04:16:58 pmI would also like just a liitle bit more lower tone, but agreed-to-disagree disagreements are usefull for pointing things out. Agree to disagree gives undue credit to another oppinion that it is something that is disagreeable. So far 440bx made one factually incorrect statement after another. This is no point of contention, this is something that needs to be pointed out. And when he tries to lecture me, even though he is grossly incorrect (and I have given counter examples or links with additional information to every single claim he made) than I am going to ridicule him for that. Again if anyone reads this, they shouldn't get away thinking "oh this is a contentious issue", because it isn't. One is right, the other one is making a fool out of themselves bringing up point after point which is just completely and factually wrong. Also it's way to much fun, disproving the stuff that 440bx posts here is just way too easy. The only thing I give him is that the results he gets for the sleep duration on his machine are indeed interesting, and if he wasn't so stubborn, and demanding that this is the only correct behavior and all my other machines are just broken if I don't get the same results, maybe one could find out why this is the case and why it behaves so differently than on other machines. Personally I think the attitude "my oppinion is correct and everything that doesn't conform to it must be wrong" is just nothing that brings anyone forward ---------------------------------------------------------------------- alpine · 2022-07-11 14:32 Quote from: prodingus on July 11, 2022, 04:16:58 pmQuoteI for one would prefer that you didn't use insults like that in open discussion. It's not necessary, it lowers the tone of the forum, and it definitely doesn't help resolve any of OP's issues. I would also like just a liitle bit more lower tone, but agreed-to- disagree disagreements are usefull for pointing things out. As far for the network thing, I will try to use something else, like Warfley's wrapper or raw sockets, but my time is limited. There are many things to learn, I was't expecting networking to be so demanding, and to my surprise sleep/delay seems to be only good for delaying, well, I don't know, just message outputs (?), but surelly not main execution loop. The communications are not so simple as one might expect and that is because they inherently involve multiprocessing issues, there is at least two participants and that is in fact multiprocessing. Even when it seems like from one side it isn't. ---------------------------------------------------------------------- prodingus · 2022-07-11 14:33 QuoteThe only thing I give him is that the results he gets for the sleep duration on his machine are indeed interesting, and if he wasn't so stubborn, and demanding that this is the only correct behavior and all my other machines are just broken if I don't get the same results, maybe one could find out why this is the case and why it behaves so differently than on other machines. Personally I think the attitude "my oppinion is correct and everything that doesn't conform to it must be wrong" is just nothing that brings anyone forwardYou couldn't have said it better!! ---------------------------------------------------------------------- prodingus · 2022-07-11 14:36 QuoteIt doesn't work that way. It is a timeout. After that time the OS (fpSelect) will just give up waiting and the control will be transferred back to the calling (your) process again. That is, the CallAction will return with no data processed. In the case data appears earlier - the control will be transferred back without further waiting.Then why the amount to be specified in the timout()? ---------------------------------------------------------------------- rvk · 2022-07-11 14:42 Quote from: prodingus on July 11, 2022, 04:36:21 pmQuoteIt doesn't work that way. It is a timeout. After that time the OS (fpSelect) will just give up waiting and the control will be transferred back to the calling (your) process again. That is, the CallAction will return with no data processed. In the case data appears earlier - the control will be transferred back without further waiting.Then why the amount to be specified in the timout()? Because normally you would have more in your program flow than only the reading of communication. Screen handling, countdown label, other buttons and other tasks (sending messages for example), etc. etc. You set the timeout to some value to which you expect a package to arrive. If it hasn't arrived by than you will want to return to your program so it can do other stuff. If you set your timeout to 20 seconds, the program will wait until you receive something or 20 seconds have passed. But 20 seconds might be too long for your user to wait. So you will lower it. But setting it too low isn't helpful either because for example if you set it at 10ms and you expect normal reaction at 100ms, then every time returning to your own program flow will also take extra time (you don't need to update the screen every 10ms). If that's not needed a timeout of 100ms might seem more appropriate. So it all depends on your program design (which you need to think about before doing any programming  ) ---------------------------------------------------------------------- 440bx · 2022-07-11 14:46 Quote from: Warfley on July 11, 2022, 04:05:11 pmAh sorry I forgott how ignorant you are. Of course I meant "No *additional* hardware requirements". but, of course... You're getting really pathetic.  It would be a good idea for you to step away from the keyboard. ---------------------------------------------------------------------- marcov · 2022-07-11 14:50 Quote from: 440bx on July 11, 2022, 04:46:53 pmYou're getting really pathetic.  ENOUGH, both of you. ---------------------------------------------------------------------- 440bx · 2022-07-11 15:36 Quote from: marcov on July 11, 2022, 04:50:15 pmENOUGH, both of you. I agree with you... more than enough. ---------------------------------------------------------------------- paweld · 2022-07-12 14:28 I was very curious about the differences you get when calling sleep and did tests on several systems I have access to: - notebook with Windows 10 - 2x servers with Windows Server OS - vm on HyperV with Windows Server OS - vm on VirtualBox with Windows XP - vm on VirtualBox with Windows 10 On all configurations for sleep(X) where X is less than about 100ms the results are as @Warfley wrote. But (!!) if I run on computer at least one VM (checked on a Windows 10 notebook) - the results are as @440bx wrote. After shutting down the VMs the results back to the values given by @Warfley. Results on attachements. Have a nice day. ---------------------------------------------------------------------- SymbolicFrank · 2022-07-13 09:27 What Microsoft says about it. Not all schedulers are created equal. In the past, we had Bulldozer CPU's, where two logical cores shared one unit for complex opcodes, like division, multiplication and complex addressing. So, it makes the most sense to first fill only the even or uneven cores and only use the other half of the pairs when they get swamped. And with the P-cores and E-cores of the current Intel's CPU's, it makes sense to use the P-cores for applications and the E-cores for lightweight threads, like drivers and stuff. And, of course, if possible, it is best to schedule tasks on the same core every time (L1 cache). With AMD, the same chiplet (L2 cache). If that isn't practical, the same CPU (L2-3 cache). Failing that, the same system / processor group (memory). Context switches take a variable amount of time that way. Also, memory access. Access of a mapped page is fast. Mapping a table takes time. Loading it from disk is even slower. Reloading part of the page table from disk is slowest. You need to load the process somewhere. And garbage collectors can periodically use quite some processing time. Especially in a VM / docker container there's a lot of layers for all that stuff. Internally, the scheduler keeps a list with all processes and a queue with running processes, their priorities and affinities. Blocked threads that are waiting for I/O have a high priority, but are best scheduled on an E-core. Etc. And this queue is updated periodically. So, if you are going to look at all the individual timings, they can vary wildly. And if you graph them all, you might be able to determine what happened in the background and in the scheduler. ---------------------------------------------------------------------- Warfley · 2022-07-13 16:25 Quote from: paweld on July 12, 2022, 04:28:29 pmBut (!!) if I run on computer at least one VM (checked on a Windows 10 notebook) - the results are as @440bx wrote. After shutting down the VMs the results back to the values given by @Warfley. Results on attachements. Have a nice day. This is actually really interesting, I can't reproduce this locally with my VMs (I'm using VMWare Player without Hyper-V), if you are using microsofts Hyper-V it could be that Windows is using a different scheduling strategy when hyper-v is used, while when I am using VMWare with the vSphere hypervisor, windows doesn't know about that (directly) and proceeds as usual. If I have some time I will try it with Hyper-V (I usually don't use Hyper-V because I am using some VM softwares that do not support it, and they don't work if Hyper-V is enabled) but as it requires multiple reboots to turn on and off, I can't do it right away ---------------------------------------------------------------------- rvk · 2022-07-13 22:34 @Warfley and @440bx I have a question. I haven't entirely followed the subdiscussion here (due to some of the language) but I gather that Warfley has results ranging in 15, 15 and 30 seconds for 1000x 1, 8 and 16 ms sleep, and 440bx has much faster results even with 1 and 8 ms tests. Is that correct? I get the same 15, 15 and 30 seconds result. In both FPC and Delphi I found a simular problem with .NET sleep. https://stackoverflow.com/questions/23215970/system-threading-timer- vs-system-threading-thread-sleep-resolution-net-timer Although the default current time resolution is 1.000 ms for me in Windows 10 Pro 64 bit... Is it possible, somehow, that the timer resolution which sleep uses, is switched to 15ms (so every sleep is minimum of 15ms). (At least the System.Threading.Timer seems to be limited to a minimum of 15ms sleep) QuoteC:\Users\Rik>Clockres.exe Clockres v2.1 - Clock resolution display utility Copyright (C) 2016 Mark Russinovich Sysinternals Maximum timer interval: 15.625 ms Minimum timer interval: 0.500 ms Current timer interval: 1.000 ms (Please tell me if I don't know what I'm talking about because I'm only investigating this for the last 10 minutes  ) O, wow. Yes. I might be on to something. @Warfley, could you add the NtSetTimerResolution to your example. Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i type NTSTATUS = ULONG;function  NtSetTimerResolution(RequestedResolution : ULONG; Set_ : BOOLEAN; ActualResolution : PULONG ): NTSTATUS; stdcall; external 'ntdll';var actualResolution: ULONG;//...  NtSetTimerResolution(1, true, @actualResolution);  writeln('was or is ?? ', actualResolution); With these lines I get these results: (MUCH better then the 15, 15 and 30 seconds you were getting) Quotems : 1 repeat count 1000 Total time: 1.465s ms : 8 repeat count 1000 Total time: 8.356s ms : 16 repeat count 1000 Total time: 16.315s ---------------------------------------------------------------------- 440bx · 2022-07-14 04:31 Quote from: rvk on July 14, 2022, 12:34:55 amI gather that Warfley has results ranging in 15, 15 and 30 seconds for 1000x 1, 8 and 16 ms sleep, and 440bx has much faster results even with 1 and 8 ms tests. Is that correct? That sounds right. Quote from: rvk on July 14, 2022, 12:34:55 amIs it possible, somehow, that the timer resolution which sleep uses, is switched to 15ms (so every sleep is minimum of 15ms). As you have found out, the timer resolution can be changed but, I don't know under what circumstances Windows decides to change it. QuoteC:\Users\Rik>Clockres.exe (MUCH better then the 15, 15 and 30 seconds you were getting) Quotems : 1 repeat count 1000 Total time: 1.465s ms : 8 repeat count 1000 Total time: 8.356s ms : 16 repeat count 1000 Total time: 16.315sThose results look as they should be.  That said, it looks like there are common situations where Windows uses a different resolution resulting in values that vary significantly from those. ---------------------------------------------------------------------- Warfley · 2022-07-14 08:57 When using a VM with HyperV I also get the smaller tick counts: Code: Text  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i ms : 1 repeat count 1000Total time: 1.949s ms : 8 repeat count 1000Total time: 8.788s ms : 16 repeat count 1000Total time: 16.600sSo windows changes the resolution for hyper-v. I would guess that it is also dependent on the CPU model, I tested on my Threadripper and my i7 laptop, so two very common CPU models (well threadripper not really but the underlying archtiecture is a Ryzen 1st gen, which is rather common), so I get the "common" result of 16ms. Maybe 440bx has some "exotic" cpu. This would also explain VMs get the same result (both on his side as on mine), as VMs are running on the same CPU, it would be expected that the OS would choose the same timer interval there. ---------------------------------------------------------------------- rvk · 2022-07-14 08:59 Quote from: Warfley on July 14, 2022, 10:57:29 amSo windows changes the resolution for hyper-v. I would guess that it is also dependent on the CPU model, I tested on my Threadripper and my i7 laptop, so two very common CPU models (well threadripper not really but the underlying archtiecture is a Ryzen 1st gen, which is rather common), so I get the "common" result of 16ms. Maybe 440bx has some "exotic" cpu. This would also explain VMs get the same result (both on his side as on mine), as VMs are running on the same CPU, it would be expected that the OS would choose the same timer interval there.Did you try my NtSetTimerResolution suggestion at the place where you got the long delays (outside of a VM)? PS. In my Hyper-v Windows 10 I also got 16, 16 and 31 seconds. Using NtSetTimerResolution there it also went down to 1.6, 8.6 and 16.2 Maybe 440bx has some permanent setting in his Windows for this resolution. ---------------------------------------------------------------------- rvk · 2022-07-14 09:07 O, and I see it also matters which version of Windows 10 you are using. See this article: https://randomascii.wordpress.com/2020/10/04/windows-timer-resolution- the-great-rule-change/ Putting in this in the example also works to make it faster. Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i  TimeBeginPeriod(1); Quotems : 1 repeat count 1000 Total time: 1.890s ms : 8 repeat count 1000 Total time: 8.581s ms : 16 repeat count 1000 Total time: 16.752s Now to find the permanent setting for this  O, and QuotePrior to Windows 10, version 2004, this function affects a global Windows setting. For all processes Windows uses the lowest value (that is, highest resolution) requested by any process. Starting with Windows 10, version 2004, this function no longer affects global timer resolution. For processes which call this function, Windows uses the lowest value (that is, highest resolution) requested by any process. For processes which have not called this function, Windows does not guarantee a higher resolution than the default system resolution.So maybe 440bx is running an version prior to Win10-2004 and some other program already has set TimeBeginPeriod to 1. And because it used to be a global setting the fpc test program benefits from that ---------------------------------------------------------------------- Warfley · 2022-07-14 09:22 Quote from: rvk on July 14, 2022, 11:07:34 amPutting in this in the example also works to make it faster. Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i  TimeBeginPeriod(1); Quotems : 1 repeat count 1000 Total time: 1.890s ms : 8 repeat count 1000 Total time: 8.581s ms : 16 repeat count 1000 Total time: 16.752s Now to find the permanent setting for this  Yes, actually the NtXXX functions should not be called at all, these are the internal kernel calls in windows and are neither documented on MSDN nor guaranteed to be API stable. Either ExSetTimerResolution or TimeBeginPeriod (unit MMSystem if one is searching for it) should be used, which then do the actual kernel call in the background. It's actually one of microsofts greatest problems with backwards compatebility that some applications use the non documented kernel APIs, even though they shouldn't, and when MS changes them in a new version, suddenly a few thousand systems of paying customers break who blame MS for this. But this is another story. Just for anyone who wishes to tinker with these timers in production, don't use NtSetTimerResolution Quote from: rvk on July 14, 2022, 11:07:34 amO, and QuotePrior to Windows 10, version 2004, this function affects a global Windows setting. For all processes Windows uses the lowest value (that is, highest resolution) requested by any process. Starting with Windows 10, version 2004, this function no longer affects global timer resolution. For processes which call this function, Windows uses the lowest value (that is, highest resolution) requested by any process. For processes which have not called this function, Windows does not guarantee a higher resolution than the default system resolution.So maybe 440bx is running an version prior to Win10-2004 and some other program already has set TimeBeginPeriod to 1. And because it used to be a global setting the fpc test program benefits from that  I was already wondering why this setting does not affect other processes ---------------------------------------------------------------------- 440bx · 2022-07-14 10:00 Maybe I'm going to disappoint someone but, I don't have an "exotic" CPU, just a simple I7 860. I don't have an "exotic" version of Windows either, just Win 7 SP1, obtained from MS. There is nothing exotic about the motherboard either, it's a run of the mill ASUS. My copy of VMware is also from VMware, a bit old by now, 2015. The one thing that is possible, is that VMware sets the timer resolution upon starting and (this is speculation on my part but, based on what @paweld mentitoned, it seems to be a possibility), since I'm always running VMs (my development "machine" is a VM), I wouldn't notice it. Whatever the case may be, MS has never been clear about how the Windows scheduler carries out its functions.  This is a quote from the description of Sleep (emphasis mine):QuoteThe system clock "ticks" at a constant rate. If dwMilliseconds is less than the resolution of the system clock, the thread may sleep for less than the specified length of time. Whenever I've paid some attention to the interval, so far, I am yet to see an instance where the thread would sleep for _less_ than the specified time. As a result, I take that claim from MS with a grain of salt (and just about everything else it says about Sleep.) ---------------------------------------------------------------------- Warfley · 2022-07-14 10:55 You are right, that is defenetly not an exotic CPU. If you are using Hyper-V VMs than this could be the issue, because this is where I have the exact same behavior when starting a Hyper-V VM. ---------------------------------------------------------------------- rvk · 2022-07-14 11:01 Quote from: Warfley on July 14, 2022, 12:55:23 pmIf you are using Hyper-V VMs than this could be the issue, because this is where I have the exact same behavior when starting a Hyper-V VM. In my Hyper-V Win10, (checking version... 21H2, so later than 2004) I still have long sleep(1) of 15ms. Not sure why you have faster results in your Hyper-V. Is there a method to query the current setting/value (the one which TimeBeginPeriod changes)?? Maybe some default setting in registry or otherwise is set. ---------------------------------------------------------------------- Warfley · 2022-07-14 11:17 Maybe with ExSetTimerResolution (thin wrapper around the NT call): https://docs.microsoft.com/en-us/windows-hardware/drivers/ddi/wdm/nf- wdm-exsettimerresolution if SetResolution is false it should return the current resolution ---------------------------------------------------------------------- PascalDragon · 2022-07-14 11:24 Quote from: Warfley on July 14, 2022, 01:17:40 pmMaybe with ExSetTimerResolution (thin wrapper around the NT call): https://docs.microsoft.com/en-us/windows-hardware/drivers/ddi/wdm/nf- wdm-exsettimerresolution if SetResolution is false it should return the current resolution Ex* functions are only available from kernel mode. Either use what is provided from the Windows API or use the Nt* function (despite not documented they're very stable as well). ---------------------------------------------------------------------- Warfley · 2022-07-14 11:58 Ah you are right, I confused them with the FunctioNameEx functions. The API for this is https://docs.microsoft.com/en- us/windows/win32/api/timeapi/nf-timeapi-timegetdevcaps ---------------------------------------------------------------------- rvk · 2022-07-14 12:22 Quote from: Warfley on July 14, 2022, 01:58:50 pmAh you are right, I confused them with the FunctioNameEx functions. The API for this is https://docs.microsoft.com/en- us/windows/win32/api/timeapi/nf-timeapi-timegetdevcaps The TimeGetDevCaps only shows the min and max value. Not the current value. Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i _timecaps = packed Record  wPeriodMin: UINT;  wPeriodMax: UINT;end; This doesn't give me a different value after doing TimeBeginPeriod(1), which I would expect because it does fix the sleep(1). Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i var  TCaps: TTimeCaps; // WinAPI.MMSystem;  rc: MMResult; begin   rc := timeGetDevCaps(@TCaps, SizeOf(TCaps)); // WinAPI.MMSystem;  if rc = 0   then Writeln(Format('Min %d / Max %d', [TCaps.wPeriodMin, TCaps.wPeriodMax]))    else Writeln(Format('timeGetDevCaps returned %d', [rc]));   TimeBeginPeriod(1);   rc := timeGetDevCaps(@TCaps, SizeOf(TCaps)); // WinAPI.MMSystem;  if rc = 0   then Writeln(Format('Min %d / Max %d', [TCaps.wPeriodMin, TCaps.wPeriodMax]))    else Writeln(Format('timeGetDevCaps returned %d', [rc])); QuoteMin 1 / Max 1000000 Min 1 / Max 1000000 ---------------------------------------------------------------------- SymbolicFrank · 2022-07-18 08:44 Yes, that clockres program and the powercfg report are very interesting. And it nicely explains why it is so inconsistent: - It depends on which application has set the lowest timer. - It is rounded up or down from the last timer tick. - Some Windows versions try to emulate a slower timer by waiting approximately 15.6 ms by skipping interrupts. That also means that they're not running in sync. - Some Linuxes use a free running scheduler, which impacts VM's. In short: it depends. Don't count on it taking a specific time. Btw. The only good way to measure time in such cases is by using the functions that return the current CPU cycle count. Or an external RTC. On second thought: the CPU cycle count depends on the power saving, so an external clock source is the only reliable way.