Subj : Re: Weird Math
To   : Andy Madigan
From : Brendan Eich
Date : Wed Sep 15 2004 04:02 pm

Andy Madigan wrote:
> I've been running some scripts through rhino, and one script kept on
> getting an equation wrong, I finally found the problem, I ran a simple
> test through the Rhino Shell, here's the output:
> 
> js> 100 - 69.1
> 30.900000000000006


That's correct, per the IEEE-754 floating point standard, which is used 
by JS, Java, and other "virtual machine" languages on all platforms, and 
by C, C++, etc. ("real machine" languages) on almost all modern 
architectures.


> Am I going nuts? Is this something about how javascript does math that
> I don't know? (Note: 100 - 59.1 is correct)


Any finite-precision binary number format will have rounding errors when 
dealing with certain base-10 numbers.  Notice that:

69.1 = 64 + 4 + 1 + 1/16 + 1/32 + 1/256 + 1/512 + 1/4096 + 1/8192 + ...

Use Number.prototype.toFixed to round appropriately:

js> (100 - 69.1).toFixed(1)
30.9
js> (100 - 69.1).toFixed(2)
30.90
js> (100 - 69.1).toFixed(10)
30.9000000000

/be

.