From dan@obluda.cz  Thu Jul  4 20:01:05 2002
Return-Path: <dan@obluda.cz>
Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125])
	by hub.freebsd.org (Postfix) with ESMTP id 1CDCD37B400
	for <FreeBSD-gnats-submit@freebsd.org>; Thu,  4 Jul 2002 20:01:05 -0700 (PDT)
Received: from xkulesh.vol.cz (xkulesh.vol.cz [195.250.154.106])
	by mx1.FreeBSD.org (Postfix) with ESMTP id 5C56A43E09
	for <FreeBSD-gnats-submit@freebsd.org>; Thu,  4 Jul 2002 20:01:02 -0700 (PDT)
	(envelope-from dan@obluda.cz)
Received: from xkulesh.vol.cz (localhost [127.0.0.1])
	by xkulesh.vol.cz (8.12.5/8.12.5) with ESMTP id g6530vR4000266
	for <FreeBSD-gnats-submit@freebsd.org>; Fri, 5 Jul 2002 05:00:57 +0200 (CEST)
	(envelope-from dan@obluda.cz)
Received: (from dan@localhost)
	by xkulesh.vol.cz (8.12.5/8.12.5/Submit) id g652QpV3013194;
	Fri, 5 Jul 2002 04:26:51 +0200 (CEST)
Message-Id: <200207050226.g652QpV3013194@xkulesh.vol.cz>
Date: Fri, 5 Jul 2002 04:26:51 +0200 (CEST)
From: Dan Lukes <dan@obluda.cz>
Reply-To: Dan Lukes <dan@obluda.cz>
To: FreeBSD-gnats-submit@freebsd.org
Cc:
Subject: __dtoa broken with -O2 or -O3 optimisation
X-Send-Pr-Version: 3.113
X-GNATS-Notify:

>Number:         40209
>Category:       bin
>Synopsis:       __dtoa broken with -O2 or -O3 optimisation
>Confidential:   no
>Severity:       serious
>Priority:       high
>Responsible:    freebsd-bugs
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Thu Jul 04 20:10:02 PDT 2002
>Closed-Date:    Mon Mar 31 04:17:08 PST 2003
>Last-Modified:  Mon Mar 31 04:17:08 PST 2003
>Originator:     Dan Lukes
>Release:        FreeBSD 5.0-CURRENT i386
>Organization:
Obludarium
>Environment:
src/lib/libc/stdlib/strtod.c,v 1.21 2002/05/28 16:59:40
gcc version 3.1 [FreeBSD] 20020509 (prerelease)

>Description:
        when you compile libc (or stdlib/strtod.c at least) with -O3 or -O2
then __dtoa is broken. It cause printf malfunction for [eEfFgG] conversions 
and has huge impact on several system utilities

worse scenario - as it damage awk used during buildworld, other unexpected
and not so easy trackable problems may occur because some utilities are
build from damaged sources

	It is very dangerous as it doesn't render system unusable 
(awk's damaged sources cause some warnings only during buildworld). 


>How-To-Repeat:
compile libc with -O2 or -O3
compile and execute
  printf("'%e' '%f' '%g'\n",(double)10,(double)10,(double)10);
you will get
  ':.000000' ':.000000e+00' ':'

It seems to be gcc's optimization problem only.

NOTE:
  __dtoa's spec_case variable is used uninitialised when mode={8,9}
  spec_case is initialised on lines 2252 and 2254 for modes 0,1,4,5
  then it is referenced on 2316 when leftright!=0 e.g.
  for mode 0,1,4,5,8,9 => 
       => for mode 8,9 spec_case referenced unitinialised

NOTE 2:
  actual 4_RELENG code has no problem even with optimised libc

>Fix:
I can't present real fix.
workaround: Don't put -O2 or -O3 into /etc/make.conf's CFLAGS
>Release-Note:
>Audit-Trail:

From: Bruce Evans <bde@zeta.org.au>
To: Dan Lukes <dan@obluda.cz>
Cc: FreeBSD-gnats-submit@FreeBSD.ORG
Subject: Re: bin/40209: __dtoa broken with -O2 or -O3 optimisation
Date: Fri, 5 Jul 2002 18:32:52 +1000 (EST)

 On Fri, 5 Jul 2002, Dan Lukes wrote:
 
 > >Description:
 >         when you compile libc (or stdlib/strtod.c at least) with -O3 or -O2
 > then __dtoa is broken. It cause printf malfunction for [eEfFgG] conversions
 > and has huge impact on several system utilities
 > ...
 > >How-To-Repeat:
 > compile libc with -O2 or -O3
 > compile and execute
 >   printf("'%e' '%f' '%g'\n",(double)10,(double)10,(double)10);
 > you will get
 >   ':.000000' ':.000000e+00' ':'
 >
 > It seems to be gcc's optimization problem only.
 
 It could also be a bug in strtod.c.
 
 Can you do some more work to isolate the bug?  I haven't had time.
 Single stepping through __dtoa() compiled by the new and old gcc (or
 even the new gcc with -O and -O2) until different behaviour is reached
 might work.  You could also try a current version of __dtoa (it's in
 netlib in dtoa.tar.gz and gdtoa.tar.gz; I don't have a URL handy).
 
 See also PR 38884.
 
 Bruce
 

From: Dan Lukes <dan@obluda.cz>
To: FreeBSD-gnats-submit@FreeBSD.ORG
Cc: Bruce Evans <bde@zeta.org.au>
Subject: Re: bin/40209: __dtoa broken with -O2 or -O3 optimisation
Date: Mon, 15 Jul 2002 03:31:55 +0200

 Bruce Evans wrote:
 
 > Can you do some more work to isolate the bug?  I haven't had time.
 
 	Well, I did the very time-expensive depth-in analysis of the code. It 
 still seems to be bug in optimisation code of GCC.
 
 	I used the copy of strtod.c source to create small test program, compiled 
 it with gcc then disassembled by IDA (because I discovered that the gdb 
 has sometimes incorrect assumptions about variable locations, and - we 
 can't trust the gdb if we are searching gcc bugs) and checked it for 
 functional equivalence with source C code.
 
 	The machine code didn't what it should do. Relevant parts of C source 
 code (with line numbers):
 
   226: #define word0(x) ((ULong *)&x)[1]
   227: #define word1(x) ((ULong *)&x)[0]
 --- skipped ---
 1933:                d2 = d;
 1934:                word0(d2) &= Frac_mask1;
 1935:                word0(d2) |= Exp_11;
 --- text on lines 1936-1968 isn't executable code ---
 1969:                denorm = 0;
 
 Related machine code (with disassembler and my comments; 
 disassembly-flavor set to 'intel', focus your eyes for comments starting 
 with '#### !!!?'):
 
 ; +-------------------------------------------------------------------+
 ; |    This file is generated by The Interactive Disassembler (IDA)   |
 ; |  Copyright (c) 2001 by DataRescue sa/nv, <ida@datarescue.com>     |
 ; |           Licensed to: Dan Lukes, 1 user upg, 03/2002             |
 ; +-------------------------------------------------------------------+
 ; File Name   :	a.out
 ; Format      :	ELF (Executable)
 ; Source File :	'crtstuff.c'
 ; Source File :	'x.c'
 ; Source File :	'MYstrtod.c'
 ; NOTE: MYstrtod.c is exact copy od strtod,
 ; the rename of __dtoa to MY__dtoa is the only modification
 
 		model flat
 
 ; ========== [removed] ===========
 
 ; Segment type:	Pure code
 ; Segment permissions: Read/Execute
 _text 
 	segment	dword public 'CODE' use32
 		assume cs:_text, ds:_data
 		;org 80480B8h
 
 ; ========== [removed] ===========
 
 ; ------------- S U B R O U T I N E --------------------------------
 ; Attributes: bp-based frame
 ; char *MY__dtoa(double	d,int mode,int ndigits,int *decpt,int *sign,
 ;                 char **rve,char **resultp)
 		public MY__dtoa
 MY__dtoa 
 proc near		; CODE XREF: main+52 p
 
 ; local variables
 i 
 	= dword	ptr -9Ch
 var_94 
 	= dword	ptr -94h
 j 
 	= dword	ptr -90h
 s0 
 	= dword	ptr -8Ch
 s 
 	= dword	ptr -88h
 S 
 	= dword	ptr -84h
 mhi 
 	= dword	ptr -80h
 mlo 
 	= dword	ptr -7Ch
 b 
 	= dword	ptr -78h
 denorm 
 	= dword	ptr -74h
 spec_case 
 = dword	ptr -70h
 s5 
 	= dword	ptr -6Ch
 m5 
 	= dword	ptr -68h
 m2 
 	= dword	ptr -64h
 leftright 
 = dword	ptr -60h
 k_check 
 	= dword	ptr -5Ch
 k0 
 	= dword	ptr -58h
 k 
 	= dword	ptr -54h
 ilim1 
 	= dword	ptr -50h
 ilim0 
 	= dword	ptr -4Ch
 ilim 
 	= dword	ptr -48h
 dig 
 	= dword	ptr -44h
 b5 
 	= dword	ptr -40h
 b2 
 	= dword	ptr -3Ch
 L 
 	= dword	ptr -38h
 ds_ 
 	= qword	ptr -34h
 eps 
 	= dword	ptr -2Ch
 d2 
 	= qword	ptr -28h
 be 
 	= dword	ptr -20h
 bbits 
 	= dword	ptr -1Ch
 local_d 
 	= qword	ptr -18h
 var_C 
 	= byte ptr -0Ch
 ; subroutine parameters
 d 
 	= qword	ptr  8
 mode 
 	= dword	ptr  10h
 ndigits 
 	= dword	ptr  14h
 decpt 
 	= dword	ptr  18h
 sign 
 	= dword	ptr  1Ch
 rve 
 	= dword	ptr  20h
 resultp 
 	= dword	ptr  24h
 
 ; ========== [removed] ===========
 
 ;C:1929: if ( (i = (int)((word0(d) >> Exp_shift1) & 
 (Exp_mask>>Exp_shift1))) ) {
 		mov	edx, dword ptr [ebp+local_d+4]
 		mov	ecx, edx
 		shr	ecx, 14h	; Exp_shift1
 		mov	esi, ecx
 		add	esp, 10h	; Add
 		and	esi, 7FFh	; Exp_mask>>Exp_shift1
 		mov	[ebp+b], eax
 		jz	loc_804A31C	; Jump if Zero (ZF=1)
 ;C:1933: d2 = d;
 ; #### !!!? following two instructions move only bottom	half of	d to d2
 		mov	eax, dword ptr [ebp+local_d]
 		mov	dword ptr [ebp+d2], eax
 ;C:1934:  word0(d2) &= Frac_mask1;
 ; #### !!!? upper half of d2 is	still not copied from d, so it is used 
 uninitialised
 		mov	eax, dword ptr [ebp+d2+4]
 		and	eax, 0FFFFFh	; Frac_mask1
 		mov	edi, dword ptr [ebp+local_d+4]
 ;C:1935: word0(d2) |= Exp_11;
 		or	eax, 3FF00000h	; Exp_11
 ;C:1963:  i -=Bias;
 		sub	esi, 1023	; Bias
 		mov	ebx, [ebp+bbits]
 ; #### !!!? there is completed the copy	of upper half of d to d2
 ; #### !!!? (via edi register),
 ; #### !!!? but	it is overwritten by eax computed
 ; #### !!!? from uninitialized upper half of d2
 		mov	dword ptr [ebp+d2+4], edi
 		mov	dword ptr [ebp+d2+4], eax
 ;C:1969: denorm	= 0;
 		mov	[ebp+denorm], 0
 
 ; ========== [removed] ===========
 
 ===================================================================
 
 The code on line 1933 is improperly compiled because the compiler 
 ignored dependencies on lines 1934 and 1935.
 
 If you start the bad optimized code under gcc with breakpoint on line 
 1933 and you manually set d2 ('set d2=d') whenever the code stops, the 
 program start to work O.K.
 
 If somebody is interested in complete disassembly of compiled __dtoa 
 with comments containing original source C code, lets me know.
 
 The minimal testing environment can be created using following patch source:
 
 *** /dev/null	Mon Jul 15 01:54:32 2002
 --- Makefile	Mon Jul  8 18:14:01 2002
 ***************
 *** 0 ****
 --- 1,23 ----
 + #OPTFLAGS=-fno-schedule-insns2
 + #OPTFLAGS=-fno-strict-aliasing
 + CFLAGS=-g -Wall -pipe -O3 $(OPTFLAGS)
 +
 + ALL: a.out
 +
 + MYstrtod.c: /usr/src/lib/libc/stdlib/strtod.c
 + 	sed -e 's/__dtoa/MY__dtoa/g' /usr/src/lib/libc/stdlib/strtod.c > MYstrtod.c
 +
 + MYstrtod.o: MYstrtod.c Makefile
 + 	$(CC) $(CFLAGS) -I/usr/src/include -c MYstrtod.c -o MYstrtod.o
 +
 + x.o:	x.c Makefile
 + 	$(CC) $(CFLAGS) -c x.c -o x.o
 +
 + a.out: x.o MYstrtod.o Makefile
 + 	$(CC) $(CFLAGS) -static x.o MYstrtod.o
 +
 + clean:
 + 	-rm x.o a.out MYstrtod.o MYstrtod.c
 +
 + run: a.out
 + 	-./a.out
 *** /dev/null	Mon Jul 15 01:54:32 2002
 --- x.c	Tue Jul  9 01:00:23 2002
 ***************
 *** 0 ****
 --- 1,21 ----
 + #include <stdio.h>
 +
 + #define mode 3
 + #define ndigits 6
 +
 + char *MY__dtoa(double, int, int, int *, int *, char **, char **);
 +
 + int main(argc, argv) {
 + int i, dsgn;
 + int xdecpt=1;
 + char *xdtoaresult, *xresult, *rve;
 +
 +  for (i=11;i<14;i++) {
 +   xdecpt=1; xdtoaresult = xresult = rve = NULL;
 +   xresult = MY__dtoa((double)i, mode, ndigits, &xdecpt, &dsgn, &rve, 
 &xdtoaresult);
 +   xdtoaresult[rve - xresult]='\0';
 +   puts(xdtoaresult);
 +  }
 +
 +  return(0);
 + }
 
 	Please note the only first round (i=11) give improper results as the 
 memory place where d2 lives is already initialised from previous run 
 (e.g. i=12 and i=13 return expected values).
 
 	Disabling schedule-insns2 OR strict-aliasing during compilation create 
 code that seems to work (but it msy contain other undiscovered bugs).
 
 ==================================
 
 	During depth-in analysis of the strtod.c code I discovered it's dirty 
 code, somewhat. It never test's if malloc has been succesfull - it just 
 reference it without checking. Look into Balloc function and __dtoa line 
 2042. IMHO, good code should never abends unless programmer decide to 
 abend it, so current beta-quality strtod.c code is acceptable in 
 current, but it's not suitable for stable.
 	Should I submit separate PR about it ? (I see no simple fix without 
 rewriting large amount of code as affected functions has no concept of 
 "internal error/resource unavaiable" reporting and/or handling).
 
 							Dan
 
 
 
 
 Note: english isn't my native language
 
 -- 
 Dan Lukes     tel: +420 2 21914205, fax: +420 2 21914206
 root of  FIONet, KolejNET,  webmaster  of www.freebsd.cz
 AKA: dan@obluda.cz, dan@freebsd.cz,dan@kolej.mff.cuni.cz
 

From: Matt Watson <mwatson@apple.com>
To: freebsd-gnats-submit@FreeBSD.org, dan@obluda.cz
Cc:  
Subject: Re: bin/40209: __dtoa broken with -O2 or -O3 optimisation
Date: Sun, 25 Aug 2002 13:28:43 -0700

     226: #define word0(x) ((ULong *)&x)[1]
     227: #define word1(x) ((ULong *)&x)[0]
 
 This has always been non-conforming C, and is now causing problems due 
 to the default behavior of -fstrict-aliasing in recent GCC 
 distributions. C99 section 6.5 says:
 
         [#7] An object shall have its stored value accessed only  by
         an lvalue expression that has one of the following types:
 
         <list not including the above cast>
 
 You can use type-punning with a union to fix this, or use 
 -fno-strict-aliasing.
 
 matt.
 
State-Changed-From-To: open->feedback 
State-Changed-By: dwmalone 
State-Changed-When: Sat Aug 31 15:26:49 PDT 2002 
State-Changed-Why:  
Dan - are there issues still to be addressed in this PR, or does 
the union patch fix them all? 

http://www.freebsd.org/cgi/query-pr.cgi?pr=40209 

From: Dan Lukes <dan@obluda.cz>
To: freebsd-gnats-submit@FreeBSD.org, dan@obluda.cz
Cc:  
Subject: Re: bin/40209: __dtoa broken with -O2 or -O3 optimisation
Date: Tue, 03 Sep 2002 01:07:31 +0200

 Oh, I'm sorry - I already sent the "yes it works" message, but i must mate a mistake in address ...
 
 	Yes, the main problem addressed by this PR is succesfully resolved by "union patch" that has been already commited to current.
 
 	But we still should not close this PR as there are still unresolved "spec_case used uninitialised" problem (see "NOTE" in original PR). I discovered (by stepping throught 
 code) that that specc_case is sometimes used uninitialized for modes 4,5,8,9 (the original note speak only about modes 8,9). I hesitate my understanding of __dtoa code is 
 not sufficient to offer a patch as I don't know what the spec_case is used for.
 
 								Dan
 
 
 -- 
 Dan Lukes     tel: +420 2 21914205, fax: +420 2 21914206
 root of  FIONet, KolejNET,  webmaster  of www.freebsd.cz
 AKA: dan@obluda.cz, dan@freebsd.cz,dan@kolej.mff.cuni.cz
 

From: Dan Lukes <dan@obluda.cz>
To: freebsd-gnats-submit@FreeBSD.org, dan@obluda.cz
Cc:  
Subject: Re: bin/40209: __dtoa broken with -O2 or -O3 optimisation
Date: Tue, 03 Sep 2002 02:46:39 +0200

 ... I forgot ...
 
 	There are also unresolved mallocations without test for NULL ...
 
 						Dan
 
State-Changed-From-To: feedback->open 
State-Changed-By: keramida 
State-Changed-When: Tue Mar 25 08:05:10 PST 2003 
State-Changed-Why:  
Dan Lukes has replied that there are still problems, some time ago. 

Dan, have you worked on this since September? 
Any new developments? 
Tests against later -CURRENT sources? 

http://www.freebsd.org/cgi/query-pr.cgi?pr=40209 
State-Changed-From-To: open->closed 
State-Changed-By: das 
State-Changed-When: Mon Mar 31 04:11:31 PST 2003 
State-Changed-Why:  
This problem is due to the fact that gcc3 with -fstrict-aliasing 
does not produce the intended output for code that violates C's 
aliasing rules.  Seems to happen only for quantities > 32-bits. 

The new version of dtoa() imported a few weeks ago does not violate 
C's aliasing rules and seems to work properly.  I do not intend to 
MFC this code, but then again, the old code works fine with gcc2. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=40209 
>Unformatted:
