Single / Double / Float speed on x86_64 lazarus / 64-FPC development ---------------------------------------------------------------------- Schmitty2005 · 2025-12-11 18:54 Based of off this conversation : https://forum.lazarus.freepascal.org/index.php/topic,72887.0.html Single precision  takes the same amount of time as double precision when doing Sine, Cos, Tanh, and likley other trig functions.  Using the math lib from C, user tetrastes  sped it  up tremendously ( about 5 times ) .  I was looking at the Math unit code in the repository, and found that the x87 calls are used for the x86_64 CPU.    .  https: //github.com/fpc/FPCSource/blob/49c9a1f3230f095cdbe3d88687367bf9273897 11/rtl/x86_64/math.inc#L233  Is there a better way to handle the x86_64 functions ? Is there a way to make a separate call function for singles to gain that speed for those who want it? I would think that if someone is using 'singles' they don't care about precision and want speed.  'Singles' are taking just as long to calculate as 'double' precision. At that point, why bother with singles ? I am just a hobbyist.  Are there technical reasons that this is so slow and has to remain that way?  I understand a change could break old code.  What about a compiler switch ?    Can this be changed to make the FPC compiler better and faster when using single precision  ? Maybe this is too complex to fix  and would require too much of a rewrite ? What is the best fast math unit that is maintained well enough and works on linux and windows  ? BTW : Delphi is slow  with single precision too. ---------------------------------------------------------------------- marcov · 2025-12-11 20:24 You do have {$excessprecision off} near the top of your code? https://www.freepascal.org/daily/doc/prog/progsu19.html ---------------------------------------------------------------------- Schmitty2005 · 2025-12-11 21:39 With this code Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i {$MODE DELPHI}{$EXCESSPRECISION OFF}Program fpccpuload;     Uses Math;     Const      MAX = 99999999;     Var      x : Integer;      c : single;    Begin      x := 0;      While x < MAX Do        Begin          c := sin(cos(tanh(x)));          inc(x);        End;    End.  Regardless of single or  double with  {$excessprecision on} or {$excessprecision off} On my system: Code: [Select]real 0m8.245s user 0m8.241s sys 0m0.006s I have also tried various optimizations, and others have as well in the link to the other discussion.  The speed increase from optimizations are negligible. ---------------------------------------------------------------------- srvaldez · 2025-12-11 23:09 hello Schmitty2005 do you have specific task where you need to compute trig and exp functions very fast? perhaps with limited range? have a look at this pdf https://shibata.naist.jp/~n-sibata/pdfs/isc10simd.pdf it has some good info, I may post some code later ---------------------------------------------------------------------- srvaldez · 2025-12-12 00:50 here's the C code for the SinCos function with a test Code: C  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i #include #include #include #include #include  double my_timer() {        LARGE_INTEGER la={ {0, 0} };      double t;        QueryPerformanceCounter(&la);        t = (double)la.QuadPart/10000000.0;        return t;} typedef struct sincos_t { double s, c; } sincos_t ; #define N 3 sincos_t xsincos0(double s) { // 0 ≤ s < π/4        int i ;        // Argument reduction        s = s * pow(2, -N);        s = s*s; // Evaluating Taylor series        s = ((((s/1814400 - 1.0/20160)*s + 1.0/360)*s - 1.0/12)*s + 1)*s;        for ( i=0;i 0) then  begin    temp := sc.c;    sc.c := sc.s;    sc.s := temp;  end;  if (((q and 4) <> 0) <> (d < 0)) then    sc.s := -sc.s;  if (((q + 2) and 4) <> 0) then    sc.c := -sc.c;  Result := sc;end; function xsin(d: Double): Double;var  sc: sincos_t;begin  sc := xsincos(d);  Result := sc.s;end; function xcos(d: Double): Double;var  sc: sincos_t;begin  sc := xsincos(d);  Result := sc.c;end; function xtan(d: Double): Double;var  sc: sincos_t;begin  sc := xsincos(d);  Result := sc.s / sc.c;end; end.  TestSinCos.pas Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i program TestSinCos;{$mode objfpc}{$H+} uses sysutils, windows, math, SinCosCalc;var  y, s: Double;  time1,time2:Double;  i, j : int32;begin    time1:=now;    s:=0;    for j:=1 to 1000 do    begin      for i:=-10000 to 10000 do        begin            y:=sin(i);          s:=s+y;        end;    end;    time2:=now;    WriteLn('Sum of sin in the range of -10000 and 10000 = ', s);    Writeln('time taken ',((time2-time1)*86400):12:10,' second.');    Writeln;    time1:=now;    s:=0;    for j:=1 to 1000 do    begin        for i:=-10000 to 10000 do        begin            y:=xsin(i);            s:=s+y;        end;    end;    time2:=now;    WriteLn('Sum of xsin in the range of -10000 and 10000 = ', s);    Writeln('time taken ',((time2-time1)*86400):12:10,' second.');end.  output Sum of sin in the range of -10000 and 10000 = -7.1054828687522331E-012 time taken 0.5750000244 second. Sum of xsin in the range of -10000 and 10000 =  8.8806739739766272E-013 time taken 0.7619998651 second. The C xsin is over 4 times faster than the Crt lib ---------------------------------------------------------------------- H₂SO₄ · 2025-12-12 06:12 Quote from: Schmitty2005 on December 11, 2025, 07:54:10 pmAt that point, why bother with singles ? Memory usage! When doing 3D graphics, you want to minimize the amount of data being sent to the GPU, and it's not like the extra precision would make a visual difference anyways. Machine learning researchers take this to yet another level by using 16-bit floats, though at that point the format really does become unusable for general-purpose calculations. For example, here are some consecutive (!)  bfloat16  values: Value        - Exponent Mantissa 1.0          0 01111111 0000000 1.0078125    0 01111111 0000001 1.015625     0 01111111 0000010 1.0234375    0 01111111 0000011 1.03125      0 01111111 0000100 ---------------------------------------------------------------------- tetrastes · 2025-12-12 10:01 Quote from: srvaldez on December 12, 2025, 01:50:56 amThe C xsin is over 4 times faster than the Crt lib What C compiler are you using? gcc version 14.2.0 (x86_64-win32-seh-rev2, Built by MinGW-Builds project) using ucrt: without optimization sum of sin from -10000 to 10000 = -3.55265816764927e-12 elapsed time 0.302194199999576 seconds sum of xsin from -10000 to 10000 = -1.33232314070142e-12 elapsed time 1.13876749999963 seconds with -O3 sum of sin from -10000 to 10000 = -3.55265816764927e-12 elapsed time 0.249794700000166 seconds sum of xsin from -10000 to 10000 = -1.33232314070142e-12 elapsed time 0.278383999999278 seconds Microsoft (R) C/C++ Optimizing Compiler Version 19.41.34120 for x64 (VS 2022 v.17.11.3): without optimization sum of sin from -10000 to 10000 = -3.55265816764927e-12 elapsed time 0.49480290000065 seconds sum of xsin from -10000 to 10000 = -1.33232314070142e-12 elapsed time 2.84119210000063 seconds with /O2 sum of sin from -10000 to 10000 = -3.55265816764927e-12 elapsed time 0.253984400000263 seconds sum of xsin from -10000 to 10000 = -1.33232314070142e-12 elapsed time 1.47731719999865 seconds ---------------------------------------------------------------------- srvaldez · 2025-12-12 11:50 Quote from: tetrastes on December 12, 2025, 11:01:15 amQuote from: srvaldez on December 12, 2025, 01:50:56 amThe C xsin is over 4 times faster than the Crt lib What C compiler are you using? gcc version 14.2.0 (MinGW-W64 x86_64-msvcrt-posix-seh, built by Brecht Sanders, r2) gcc -static -Wall -O3 -march=native TestSinCos.c -o TestSinCos Processor   Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz, 3600 Mhz, 8 Core(s), 16 Logical Processor(s) ---------------------------------------------------------------------- marcov · 2025-12-12 11:54 There are multiple different aspects. Probably $excessprecision is only for generated code, not for library code, and means that singles are not always promoted to doubles. However the sin/cos stuff touches libraries not having overloads for single. That might be fixed in trunk, I don't know, it's been a while that I played with such options. Besides that there is also the normal vs fastmath way of doing things. Fastmath is an optimization common in C compilers to have results less precise but faster.  Afaik precise still means copro use, as per standard, and that is what FPC and Delphi do. FPC currently has no solution for that. Fastmath  is sometimes enabled with higher -O. But how and with which optimization level it is enabled exactly in C is afaik compiler and architecture dependent. Fastmath is common, but not standarized afaik. So this whole topic is multi facetted. General promotion of single to double {$excessprecision off}Generally only having double/extended functions in math for non elemental operations (maybe (partially) fixed in trunk)Having fastmath or not (using SSE approx for operations, FPC has an option -Ofastmath, but that is only for generated code, not for library code in assembler) ---------------------------------------------------------------------- tetrastes · 2025-12-12 12:56 Quote from: srvaldez on December 12, 2025, 12:50:42 pmQuote from: tetrastes on December 12, 2025, 11:01:15 amQuote from: srvaldez on December 12, 2025, 01:50:56 amThe C xsin is over 4 times faster than the Crt lib What C compiler are you using? gcc version 14.2.0 (MinGW-W64 x86_64-msvcrt-posix-seh, built by Brecht Sanders, r2) gcc -static -Wall -O3 -march=native TestSinCos.c -o TestSinCos Processor   Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz, 3600 Mhz, 8 Core(s), 16 Logical Processor(s) gcc for win64 using MSVCRT as runtime library generates very inefficient x87 code for floating point, instead of using functions from CRT, which are much faster for x64. ---------------------------------------------------------------------- LV · 2025-12-12 14:04 I was passing by 🤔 12th Gen Intel(R) Core(TM) i7-12700H (2.30 GHz) Code: Text  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i ------------------------gcc 14.2.0 -o3 -fopenmp ------------------------omp_set_num_threads = 1                           Time = 0.919000 secSum = -0.000000000017208 omp_set_num_threads = 20                             Time = 0.699000 secSum = -0.000000000038277 ------------------------fpc 3.2.2 -o3 ------------------------program ParallelSin;........................THREAD_COUNT = 1                           Time = 4.832000 secSum = -0.000000000071054 THREAD_COUNT = 20                             Time = 0.440000 secSum = 0.000000033305696 program ParallelSinFast;........................THREAD_COUNT = 1                           Time = 1.813000 secSum = -0.000000000106582 THREAD_COUNT = 20                             Time = 0.127000 secSum = -0.000000140812062  P.S. Code: Text  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i   N1 = -10000;  N2 = 10000;  LOOPS = 10000; ---------------------------------------------------------------------- LV · 2025-12-12 15:38 In multithreaded Pascal code, you can observe a decrease in precision. To address this, I implemented Kahan's algorithm, which adds minor overhead. Code: Text  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i omp_set_num_threads = 20Time = 0.662000 secSum = -3.830e-11 THREAD_COUNT = 20Time = 0.149000 sec Sum = 4.547E-12        Time_C_omp————————————————————————————— = 4.442Time_Pascal_Parallel_KahanSum ---------------------------------------------------------------------- srvaldez · 2025-12-12 16:13 thanks LV  using threads combined with open-mp is tricky  not for the faint of heart I also compiled it using the Intel C compiler, without threads and without omp the time was about .7 seconds, with /Qiopenmp .1 seconds but I am not comfortable using threads or omp ---------------------------------------------------------------------- BrunoK · 2025-12-12 16:48 Quote from: srvaldez on December 12, 2025, 01:50:56 amHow can one compare an algorithm written in C giving checking values of Quotesum of sin from -10000 to 10000 = -1.33221211839896e-12 sum of xsin from -10000 to 10000 = -1.77641235055148e-12and an algorithm in FPC giving a checking values of QuoteSum of sin in the range of -10000 and 10000 = -7.1054828687522331E-012 Sum of xsin in the range of -10000 and 10000 =  8.8806739739766272E-013 Obviously, the FPC algorithm does not do the same thing hence timing comparison is irrelevant. ---------------------------------------------------------------------- srvaldez · 2025-12-12 16:59 in the case of the sin function, the sum from -N to N should be zero, so the closer to zero the more accurate ---------------------------------------------------------------------- BrunoK · 2025-12-12 17:12 Quote from: srvaldez on December 12, 2025, 05:59:47 pmin the case of the sin function, the sum from -N to N should be zero, so the closer to zero the more accurate What does represent the sum of xsin, what should it be ? If both should be 0 then sum of sin is better in C but xsin is better in FPC 3.2.2 and above. In FPC, if you use Linux 64 bit, you get the floating point unit x87 FPU Extended precision (80 bit extended) calculation but if using Win 64, you get Double floating ) point (64 bit double) calculation. So what is it you want ? ---------------------------------------------------------------------- srvaldez · 2025-12-12 17:20 this was only a silly benchmark, the main reason for doing a sum is so that the compiler hopefully won't eliminate the loop and as aside it gives a hint on accuracy ---------------------------------------------------------------------- BrunoK · 2025-12-12 17:35 Quote from: srvaldez on December 12, 2025, 06:20:23 pmthis was only a silly benchmark, the main reason for doing a sum is so that the compiler hopefully won't eliminate the loop and as aside it gives a hint on accuracy What was Sum of xsin in the range supposed to yield ? The benchmark is not so silly because it illustrate the uncertain results of simulations, like parallel processing of COVID deaths models not matching single processor / multi threaded code results or non replicable simulations of some climate models (and others ...) not matching or not converging to a reasonable state. ---------------------------------------------------------------------- srvaldez · 2025-12-12 17:41 Quote from: BrunoK on December 12, 2025, 06:35:07 pmWhat was Sum of xsin in the range supposed to yield ? xsin is just an approximation of the sin function so in this case the sum should be zero or close ---------------------------------------------------------------------- tetrastes · 2025-12-12 20:10 Just in case, fpc cannot optimize Power(2, -3) to 0.125, as gcc does.