Subj : Re: js_NumberToString asserting
To   : netscape.public.mozilla.jseng
From : gappodi@hotmail.com (Gappodi)
Date : Fri Mar 19 2004 06:20 am

Brendan,

Sorry, there should have been an "e" in the number

js> var i=7.551073205401564445536e-06
js> print (i)
Assertion failure: numEnd < buffer + bufferSize, at jsdtoa.c:2578
Aborted

It went away after changing ( in jsdtoa.h )
#define DTOSTR_STANDARD_BUFFER_SIZE 26

It was 25 earlier.

I am not sure if this is the right thing to do.

Tks & Regds
GP


Brendan Eich <brendan@meer.net> wrote in message news:<405A8B54.5060403@meer.net>...
> Gappodi wrote:
> > With a numeric value like this the engine asserts while trying to js_NumberToString.
> > 
> > var i = 7.1073205401563615-06;
> > 
> > print ( i );
> 
> I get no such assertion:
> 
> js> var i = 7.1073205401563615-06;
> js>
> js> print ( i );
> 1.107320540156362
> js> print ( i.toFixed(15) );
> 1.107320540156362
> js> print ( i.toFixed(16) );
> 1.1073205401563619
> js> print ( i.toFixed(17) );
> 1.10732054015636194
> js> print ( i.toFixed(18) );
> 1.107320540156361943
> js> print ( i.toFixed(19) );
> 1.1073205401563619432
> js> print ( i.toFixed(20) );
> 1.10732054015636194322
> 
> > Anything that can be done to increase the precision?
> 
> No, JS per ECMA-262 uses IEEE 754 double precision.  Note that toFixed 
> can be used to show more or fewer digits, but the rounding error caused 
> by the number format's precision limit is still there.
> 
> /be

.