Default and speed effect lazarus / 64-FPC development ---------------------------------------------------------------------- Okoba · 2023-05-30 12:44 Hello, I faced an interesting issue, and I can not understand why this happens. Hopefully someone with a better understanding of the compiler and assembly language could clarify. In this code, there is assing routing that copies a record to another. The test procedure performs that function, and that is simple too. The interesting part is that if I use A:= Default(TTest); beforehand, it speeds up the could by a good margin, near 2x. Using A.V := nil;  does not change anything, so I don't know what magic Default does to the code that results in such a difference. I know this is a micro benchmark, but I faced such sudden changes in speed, and I can not understand why? Lazarus: Lazarus 2.3.0 (rev main-2_3-2865-g5895004fde) FPC 3.3.1 x86_64-win64-win32/win64 Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i program Project1; uses  SysUtils; type  TTest = record    V: array of QWord;  end;   procedure Assign(var A: TTest; const B: TTest); inline;  begin    if Length(A.V) <> Length(B.V) then      SetLength(A.V, Length(B.V));    if Length(B.V) <> 0 then      Move(B.V[0], A.V[0], Length(B.V) * SizeOf(QWord));  end;   procedure Test;  var    T: QWord;    I: Int64;    A, B: TTest;  begin    //Commenting this increases the time from 180 to 330    //A := Default(TTest);    //A.V := nil;  //This once does not change the result     SetLength(B.V, 1);     T := GetTickCount64;    for I := 1 to 100 * 1000 * 1000 do    Assign(A, B);    WriteLn(GetTickCount64 - T);  end; begin  Test;  ReadLn;end. ---------------------------------------------------------------------- Nitorami · 2023-05-30 13:34 As you say, it is a microbenchmark, and I cannot reproduce it here. But I had similar effects, factor 2 simply by defining an empty string somewhere in the program. I guess not even the processor manufacturers know in detail how all their optimisation mechanisms operate together and which effect they have in the individual case. BTW your assign function is wrong, move expects number of bytes while your record length is in qwords. ---------------------------------------------------------------------- Okoba · 2023-05-30 13:37 Thank you I fixed the mistake. As you said, it happens in your case too. And in my real case, it is not a micro benchmark, I just isolated the code to have a sample to share here. ---------------------------------------------------------------------- Blaazen · 2023-05-30 15:35 This may be coincidence caused by code aligning (generated asm is aligned to some nice value like 32B or so). ---------------------------------------------------------------------- Okoba · 2023-05-30 15:37 This happens a lot for me, so I like to have more info about it and maybe have way to nudge FPC to the better direction. ---------------------------------------------------------------------- Martin_fr · 2023-05-30 16:07 Quote from: Blaazen on May 30, 2023, 05:35:59 pmThis may be coincidence caused by code aligning (generated asm is aligned to some nice value like 32B or so). My thought too. Especially because it moves the for loop. And that is a For loop that is rather short. On Intel there is a cache for translated micro ops, which use 32 bit bounds, and if the loop ends up fitting -> wooooom. However that cache isn't everything. Modern CPU do a lot of different optimizations. If you try to go all out for one of them, you may loose on another one. https://www.youtube.com/watch?v=r-TLSBdHe1A ---------------------------------------------------------------------- Okoba · 2023-05-30 16:13 Thank you Martin. And thanks for the link. So the Default is doing nothing of benefit? Only the result value allocates a memory that makes the alignment better? Is there a low level way to find out about this and use it or know more about it? As I saw this alignment issue  (adding a var or a function result) makes a difference. It is usable for very delicate functions (like filling a memory or finding something). ---------------------------------------------------------------------- Martin_fr · 2023-05-30 16:22 Quote from: Okoba on May 30, 2023, 06:13:23 pmThank you Martin. And thanks for the link. So the Default is doing nothing of benefit? Only the result value allocates a memory that makes the alignment better? Is there a low level way to find out about this and use it or know more about it? As I saw this alignment issue  (adding a var or a function result) makes a difference. It is usable for very delicate functions (like filling a memory or finding something). You can check the alignment in the assembler win of the debugger. But, it wont work magically for every loop. It requires very specific conditions (and may even vary depending on the CPU generation on which it runs). There is a $Align directive, which afaik can also be used for loops. I have once (and once only) seen this on a longer loop. Probably because a specific part in the loop did get on the sweet spot. Otherwise very short loops, with very high iteration counts are probably likely to benefit. Like your test loop - did likely fit into 32 bytes (max 64), - and then if also started at a 32 byte bounds, - and doing massive iterations.... - **And** additionally working repeatedly on the same data A and B => so the data was in cache already. ==> That is quality food for the CPUs internal optimizer. But that does not really happen exactly like this in real life. (yet the effect can happen in real life too) But I am not an expert, so I may be wrong with my above assumptions. ---------------------------------------------------------------------- Okoba · 2023-05-31 07:15 Thank you for the explanation. I know nothing about assembly, but maybe I should, if I like to use these optimization. My first thought was, maybe there is something in my code, that I can change. ---------------------------------------------------------------------- Okoba · 2023-05-31 07:32 Just to be clear, what does default() do that makes the code change? And is removing it, changes any behaviour in the sample code? Is it redundant and I just seeing a side effect of it in the outputted assemenly? ---------------------------------------------------------------------- Martin_fr · 2023-05-31 09:20 Quote from: Okoba on May 31, 2023, 09:32:31 amJust to be clear, what does default() do that makes the code change? And is removing it, changes any behaviour in the sample code? Is it redundant and I just seeing a side effect of it in the outputted assemenly? Default assigns a value to the (entire) record. - In this case the value is zero / nil. - In this case the record and the field in it are the same thing (so the record could have padding) So in this case it does the same change as the ":= nil" => but it may use different assembler code. And that may mean the code occupies a different count of bytes. And subsequent code gets shifted. => changing alignment (in this case of the loop) => having effects on how fast it runs. Your CPU does a lot more than you expect. "mov rax, [address]" (move data between mem and register) The CPU produces a whole program from that. So called micro-ops. And those get executed. That translation can be cached. And that can have an effect on the speed. But caching at bounds is not all. Sometimes a list of steps can be changed in order, still meaning the same (e.g. "A:=1; b:=2;" can be swapped). That also happens sometimes with conditions and other specific ops. Depending on the order, the speed may differ (And no, don't try to reorder your Pascal code => that wont do no good. This is about the assembler level. You can't affect that) Because, if in the correct order, your CPU can do them in parallel. But only if in the correct order. And there are way more things affecting speed. What you can affect, is in which order to access data in memory. And this can have an impact on cache usage (data/memory cache). And looking at algorithms used: Keyword "Big O" Then again => Is that code actually worth the optimization? How often is it called? How much percentage time of your app does it take? If you speed it up by 5% (in real life, not in an artificial benchmark), how much does your app gain? Because lets pretend that code makes it to 10% of the runtime of your app. And you speed it up by 5% => Your app will be only 0.5% faster. ---------------------------------------------------------------------- 440bx · 2023-05-31 09:43 Quote from: Okoba on May 31, 2023, 09:15:35 amI know nothing about assembly, but maybe I should, if I like to use these optimization. My first thought was, maybe there is something in my code, that I can change. I just want to add to, and reinforce, what @Martin_fr has mentioned. The first thing to understand is that program optimization is a tricky little "critter".  For instance, optimizing a parallel algorithm is usually very different from optimizing a single threaded algorithm for the same purpose.  After that, the algorithm's Big O is a crucial characteristic.  After that, come CPU-friendly optimizations, e.g, avoiding cache misses, keeping execution units running and a very long list of "arcane" details that are CPU architecture dependent.  Knowing assembler _and_ the CPU architecture is required for making these optimizations. A programmer should decide first, if the algorithm is going to be parallel or single threaded, then choose an algorithm based on it's Big O (usually affects ease of implementation), then the assembly code generated by the compiler.    More often than not, the first two are where a programmer's efforts are concentrated.  The last one is rarely justified in applications programming but common when writing an O/S. I always recommend a programmer learn, at least, some assembler and, it wouldn't hurt any for a programmer to read either intel's or AMD's CPU's documentation (particularly the system software manual) HTH. ---------------------------------------------------------------------- Okoba · 2023-05-31 11:34 Thank you very much Martin and 440bx. Those were a good read and code alignment is an interesting point. ---------------------------------------------------------------------- Okoba · 2023-05-31 11:37 I was working on the sample and found another problem. I watched the assembly code and it does not seem a alignment problem. The outputted assembly has Initialize and FPC_COPY on every loop step, and it is near 10X slower. Why adding the first Default out of the loop, effects the loop in such manner? Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i program Project1; uses  SysUtils; type  TTest = record    V: array of QWord;  end;   procedure Assign(var A: TTest; const B: TTest); inline;  begin    if Length(A.V) <> Length(B.V) then      SetLength(A.V, Length(B.V));    if Length(B.V) <> 0 then      Move(B.V[0], A.V[0], Length(B.V) * SizeOf(QWord));  end;   procedure Assign(var A: TTest; const B: QWord); inline;  begin    if Length(A.V) <> 1 then      SetLength(A.V, 1);    A.V[0] := B;  end;   operator := (const AValue: QWord): TTest; //inline;  begin    Assign(Result, AValue);  end;   procedure Test;  var    T: QWord;    I: Int64;    A: TTest;  begin    //A := Default(TTest);     T := GetTickCount64;    for I := 1 to 1000 * 1000 * 1000 do      //Assign(A, I); //1140ms with and without Default      A := I; //1900ms with Default and 15500 without (near 10X slower)    WriteLn(GetTickCount64 - T);  end; begin  Test;  ReadLn;end. ---------------------------------------------------------------------- ASerge · 2023-05-31 14:29 Quote from: Okoba on May 31, 2023, 01:37:44 pmWhy adding the first Default out of the loop, effects the loop in such manner? The implementation of the default value, as well as auto-assignment when declaring (for management type), is implemented through the implicit declaration of another variable with a given value and then its assignment. In this case, the compiler, seeing that the variable is already bound somewhere, makes a safe assignment. To do this, a temporary variable is created, an assignment goes into it, and then it is copied via fpc_copy_proc to our variable. If there are no references to the variable, the compiler optimizes and does not create a temporary variable, avoiding calling the fpc_copy_proc. ---------------------------------------------------------------------- Martin_fr · 2023-05-31 14:43 At this point you should tell us - your fpc version - exact settings to compile with Not sure what you mean with "Initialize", but I can see the "FPC_COPY" In both cases, fpc creates a temporary variable. This is needed, because for managed types (dyn array), the variable "result" is a "var param". That is "result" is passed as parameter. Because the actual variable MUST NOT be changed until the function returns (otherwise, in many other case you will get wrong data), a temp variable is passed to the function. When the function returns the temp var is assigned to the real var. (In this case using FPC_COPY ). Now I don't know fpc internals, but I can see this. WITHOUT "A := Default(TTest);" the variable "A" is ONLY accessed in the loop => the value is NOT used anywhere else. It *appears* fpc is actually noting that. Because FPC knows that the value in "A" is never used, it does not copy the result to A. Smart decision.  Screws your benchmark though ---------------------------------------------------------------------- Martin_fr · 2023-05-31 14:45 Quote from: ASerge on May 31, 2023, 04:29:06 pmIf there are no references to the variable, the compiler optimizes and does not create a temporary variable, avoiding calling the fpc_copy_proc. Actually here it does create a temp in the case of "not accessed elsewhere". Maybe different depending on fpc version and settings. ---------------------------------------------------------------------- Okoba · 2023-05-31 16:33 I noted the exact version on the first post. And it is ran on the Release mode. By "Initialize" I meant the assembly code preparing a clean value (It was similar to the assembly of A := Default(TTest) line). Oh so FPC knows the value is not used so it reuses the value. Smart. I wish this feature had a name so I can comment or made a feature request about it. As the more smart way was for it too look at "outside of loop". I reported such issue before at https://gitlab.com/freepascal.org/fpc/source/-/issues/39725. Maybe they are the same thing. What do you think? What is your version? ---------------------------------------------------------------------- Martin_fr · 2023-05-31 16:51 Quote from: Okoba on May 31, 2023, 06:33:48 pmI noted the exact version on the first post. And it is ran on the Release mode. My bad, sorry. QuoteOh so FPC knows the value is not used so it reuses the value. Smart. I wish this feature had a name so I can comment or made a feature request about it. As the more smart way was for it too look at "outside of loop". I reported such issue before at https://gitlab.com/freepascal.org/fpc/source/-/issues/39725. Maybe they are the same thing. What do you think? What is your version? There probably is a name... "variable lifetime analysis", but there is some other term, I just can't recall. Some term describing the way code (and the data belonging to it) is analysed to be able to "rewrite" the order of operation for optimization.... Something that afaik is "work in progress" in FPC. (and has been for a long time).  But I may be wrong. I am not part of the fpc team. QuoteMaybe they are the same thing. very hard to answer question. May even depend on how broad you make "the same". E.g. "Yes they are both optimizations", but obviously not what you meant. But I don't think they are a total overlap. Both need information about the "usage" of a variable. But apparat from that, the inlining seems different. I don't know, but I think I heard that FPC generates some form of pre- compiled code for "inlining".  (You can inline, even if you have only a ppu, and no source code) => That would mean this code wants a pointer for a "var param", and then it can't be used in a register. To actually use a register (the whole time, including in the inlined code), the compiler would have to rewrite the code for the function at the time when it is inlining it. And that is totally different from just determining, how a variable is used during its lifetime. ---------------------------------------------------------------------- Okoba · 2023-06-01 09:13 Thank you very much for the notes. I added an issue and a comment to ask for FPC team opinion: https://gitlab.com/freepascal.org/fpc/source/-/issues/40303 ---------------------------------------------------------------------- Martin_fr · 2023-06-01 10:37 Well, your real issue is obviously that you are using that code, and that you believe it slows down your app. Question is still, how much it really affects your final app. (see last part of this post) In any case, in your app, you will need the result to be used, therefore you will not have an otherwise unused local var as target (and non local vars are afaik always treated like being used / but there is a whole lot more to it). So in your real live, that final assignment (temp to your var) has to happen. And there is no easy way round it. And since it is a managed var, it is taking extra time. (managed vars need ref count checking, and that is done (to an extend) thread save). Even if code is inlined, those temp var effects may still happen (probably will). And if you are using a non-local var (class field, global) , then they may not even be optimizable / because that would change the behaviour of the app => not yours maybe, but in general, and outside the scope of a compiler being able to know). If you must have a managed var, then the Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i //a,b: array of integer;  b := a;  setlength(b, length(b));may be more efficient. But you can't move it into a subroutine => then you will have temp vars again Test case: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i program Project1;var  a,b: array of integer;begin  a := nil;  SetLength(a, 1000);  b := a;  writeln(ptruint(pointer(a)));  writeln(ptruint(pointer(b))); // pointing same address   setlength(b, length(b));  writeln(ptruint(pointer(a)));  writeln(ptruint(pointer(b))); // pointing diff address  readln;end. Well, maybe you can use a function, if you pass the arguments as "var param" Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i procedure CopyData(const Source: TFoo; var Dest: TFoo);Because then you don't get any ref count interference when calling the procedure. You do get a pointer, but that should have less impact. And you loose having the data in a register (if ever it was), which again can have impact. Though not sure if "SetLength" is register friendly. Or for that matter, any of the internally called function when managed types are involved. The only faster bit, then would likely be to use pointers instead of managed types. (I assume you don't use threads) Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i var TData: ^Integer.And use GetMem or AllocMem. Copy the date yourself. Use pointermath {$T+} to access the data "AData[n]" (like an array) But consider first, if it is really worth it. How much time are you going to save in your entire app? Is your app currently doing a loop with 1,000,000 or more such assignments? Is it stack like half a minute on that loop? Even then, you still need to copy the memory, so it still will take time. If your app is not stuck for such long on that loop, then likely the saving will not make a major difference to the overall speed of your app. If you are about to optimize your app, then I recommend "valgrind --tool=callgrind" and kcachegrind. Though that requires you to be able to run the app on Linux (a VM will do). You get a nice chart where time was spent, and how much of the time. ---------------------------------------------------------------------- Okoba · 2023-06-01 11:03 Very helpful, as always. The Assign procedure is like what you said for CopyData. I wanted to use the operator to make life easier. I should check using pointers too, and I sometimes do. My main concern was pointing out a potential issue and learning "why this weird thing happens", and you made it much more clear for me. About optimization, this code is just a part of my app, and sure, if I optimize it, it will not even speed up my program by 10%, but it will speed up that task by an order of magnitude and take less progress time to show the user. The problem is that I need to benchmark each part of my project to find any mistakes, like cleaning up every part of an engine before assembling it. And that code was only a sample, I wanted to know why Default() effects my code, why an operator is making things slower, and whether I should prevent using them. But to answer your question, in my program, it is a loop going through records in the database. Very interesting point about valgrind. So you use Linux or a VM, and configure valgrind with Lazarus to have a better look? I like this idea. ---------------------------------------------------------------------- Martin_fr · 2023-06-01 11:18 As you may have learned from the video: An unrelated change => such as changing the order in which 2 procedures are declared, may add a 2 digit percentage speed change. (up or down). In your case, it wasn't some side effect on the CPU, but just that some simplified samples can be optimized better. I do recommend valgrind. Really. It is worth it. (You get the accumulated time of all calls to a method). One other maybe interesting thing. Though I have last tested that years ago. Because smaller code snippets can sometimes be better optimized (including register allocation, at least back then), I have seen that splitting a method into 2 procedures can (or "was once able to") gain speed. This was in 2008 / So a very old compiler now. / May be different today https://gitlab.com/freepascal.org/fpc/source/- /issues/10275#note_644215372 ---------------------------------------------------------------------- Okoba · 2023-06-01 11:39 You are right about changing the order. In such low level codes, any minor change may end up opening a new possibility, as these functions would be called billions of times. On your note, yes, of course, splitting different bodies of code is a very good habit. It gives the compiler more registers to work with, in addition to having more maintainable code. In that note, I think it is an old sample, in my current working habits, I would split those functions, but not as subfunctions (FPC has a problem optimizing those), and I would inline them, to let FPC know that they are using the same variables. Generally, it led to more speed, as I will prevent the cost of a function call. But if those functions were totally different with different variables, they should not be inlined, as it will limit the compiler's available register count. All these thoughts are my feelings about how the compiler works, not actual facts, as I know nothing about its internals. I will make a VM and test the valgrind way.