Precision of COMP
lazarus / 65-FV-Textmode IDE

----------------------------------------------------------------------
Elap · 2016-02-22 10:59

When I run the program below in Turbo Pascal, I get the fifth power as
30973432405238707, but in FP 2.6.4 I get 30973432405238700.

In general I don't seem to get the number of significant digits that I
would expect for large numbers like these.

Have you any idea why this discrepancy occurs?  I am using Windows 7
Professional and use the console IDE.

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i
{$R+}PROGRAM BUG3; USES    CRT; VAR    I_comp:      COMP;    I_comp5:
   COMP; BEGIN    CLRSCR;     I_comp:=1987;   
I_comp5:=I_comp*I_comp*I_comp*I_comp*I_comp;     WRITELN(I_comp:4:0,'
to the fifth power is ',I_comp5:17:0);    READLN;END.

----------------------------------------------------------------------
marcov · 2016-02-22 11:14

I use FPC 3.0 and 3.1.1 both show  *707, the value according to
calculator.

----------------------------------------------------------------------
BeniBela · 2016-02-22 11:51

Floating point conversions were completely broken in 2.6.x

Still not working optimally in 3.x, but now it is too precise

----------------------------------------------------------------------
Elap · 2016-02-22 12:06

Thank you for that.

I'm using Qword which seems to be fine.

----------------------------------------------------------------------
BeniBela · 2016-02-22 12:07

That becomes buggy if you do range checked calculations over
high(int64) ...