Newsgroups: comp.lang.fortran
Path: utzoo!utgpu!news-server.csri.toronto.edu!rpi!zaphod.mps.ohio-state.edu!wuarchive!m.cs.uiuc.edu!roundup.crhc.uiuc.edu!ux1.cso.uiuc.edu!hirchert
From: hirchert@ncsa.uiuc.edu (Kurt Hirchert)
Subject: Re: What is zero, REALly??
Message-ID: <1991Jun12.152217.2889@ncsa.uiuc.edu>
Originator: hirchert@harriett.ncsa.uiuc.edu
Keywords: real numbers, old Fortran, zero comparison
Sender: usenet@ux1.cso.uiuc.edu (News)
Organization: Nat'l Ctr for Supercomp App (NCSA) @ University of Illinois
References: <1991Jun11.143915.11722@milton.u.washington.edu>
Distribution: usa
Date: Wed, 12 Jun 1991 15:22:17 GMT
Lines: 46

In article <1991Jun11.143915.11722@milton.u.washington.edu> djo7613@milton.u.washington.edu (Dick O'Connor) writes:
>I'm trying to port an old Fortran program written to run on an IBM main-
>frame to something a bit smaller.  Part of the "truth test" of the new code
>(which is Pascal, but that's neither HNT) is that I have a printout of an
>old run of input and output data that I have to be able to regenerate.
>Fine except for one set of readings, which end up being a little bit off
>in a way that's not DIRECTLY related to rounding.  I believe it's indirectly
>related to rounding via The Law Of Small Numbers (a handy catch-all excuse
>we use around here!), and I'd like some feedback from people with experience in
>how Fortran handles very small numbers.
>
>The routine in question takes the arithmetic mean of N positive real numbers
>in the range, say 40.0 to 90.0, and uses subtraction to find the number that
>is the furthest away from that mean.  It computes a "difference" from the mean
>for each number and determines the greatest difference using the DABS
>(double-precision absolute value) function.  The problem case is a
>specific case where N equals 2, so both real numbers have the same difference.
>The original code acts differently depending on which of the N numbers is
>the furthest distant from the mean, and does it in the case N=2 with a
>  IF ( DABS(A(1)) - DABS(A(2)) ) 800,900,900
>where A is a double precision real array.  In almost every case, the middle
>branch is taken, but there's one case in particular where the first branch
>is taken, meaning there is a measurable difference between two numbers which
>should be identical.
>
>I believe it's a case of reaching the limits to which a binary machine can
>express a decimal-based number, with the result that two "identical" numbers
>are in fact only "very close", and their subtraction yields a number detectably
>different from zero.  Anyone have any experience to corroborate or refute this?

The problem is not specific to Fortran, nor is it specific to binary machines.
The root of the problem is that machine precision is limited.  To illustrate,
imagine that Dick's problem were being run on a decimal machine with three
digits of accuracy.  Suppose the numbers were 41.0 and 41.5.  Summing them,
we get 82.5.  Dividing by 2 to get the average, we get 41.25, but the machine
can only represent 3 digits, so it must choose either 41.2 or 41.3.  That
roundoff means that when we compute the differences, they will be different
(.2 and .3) rather than identical (.25 and .25).  If we suppose the bottom
digit of the representation to be essentially random, then this kind of
roundoff should occur half the time.  The fact that you are not seeing this
suggests a different distribution.  My guess would be that the bottom digit
is turning out even more than half the time, but that is not the only
possible explanation of what you are seeing.
-- 
Kurt W. Hirchert     hirchert@ncsa.uiuc.edu
National Center for Supercomputing Applications
