From nobody@FreeBSD.org  Thu May 20 06:38:14 2004
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125])
	by hub.freebsd.org (Postfix) with ESMTP id 8068416A4CE
	for <freebsd-gnats-submit@FreeBSD.org>; Thu, 20 May 2004 06:38:14 -0700 (PDT)
Received: from www.freebsd.org (www.freebsd.org [216.136.204.117])
	by mx1.FreeBSD.org (Postfix) with ESMTP id 6224A43D55
	for <freebsd-gnats-submit@FreeBSD.org>; Thu, 20 May 2004 06:38:14 -0700 (PDT)
	(envelope-from nobody@FreeBSD.org)
Received: from www.freebsd.org (localhost [127.0.0.1])
	by www.freebsd.org (8.12.11/8.12.11) with ESMTP id i4KDcEnB086634
	for <freebsd-gnats-submit@FreeBSD.org>; Thu, 20 May 2004 06:38:14 -0700 (PDT)
	(envelope-from nobody@www.freebsd.org)
Received: (from nobody@localhost)
	by www.freebsd.org (8.12.11/8.12.11/Submit) id i4KDcE1J086633;
	Thu, 20 May 2004 06:38:14 -0700 (PDT)
	(envelope-from nobody)
Message-Id: <200405201338.i4KDcE1J086633@www.freebsd.org>
Date: Thu, 20 May 2004 06:38:14 -0700 (PDT)
From: Jonathan Wakely <jwakely@mintel.com>
To: freebsd-gnats-submit@FreeBSD.org
Subject: Unacceptable stringstream performance
X-Send-Pr-Version: www-2.3

>Number:         66941
>Category:       bin
>Synopsis:       [patch] gcc2.95 (FreeBSD-specific): fix unacceptable stringstream performance
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    freebsd-bugs
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Thu May 20 06:40:19 PDT 2004
>Closed-Date:    Fri Jun 15 11:28:01 GMT 2007
>Last-Modified:  Fri Jun 15 11:28:01 GMT 2007
>Originator:     Jonathan Wakely
>Release:        4.9
>Organization:
Mintel International
>Environment:
FreeBSD cartman.mintel.co.uk 4.9-STABLE-20040212-SESNAP FreeBSD 4.9-STABLE-20040212-SESNAP #1: Mon Feb 16 10:48:30 GMT 2004     jason@cartman.mintel.co.uk:/usr/obj/usr/src/sys/CARTMAN  i386

>Description:
      The /usr/include/g++/sstream header provided with FreeBSD's GCC 2.95.4 performs very badly. Every single character written to the stream causes the buffer to allocate one extra character, copy the existing buffer contents, and append the new character. Once the buffer gets large the overhead of reallocating and copying for every single character becomes enormous.

The C++ standard requires that the appends happen in amortised constant time. This implies the buffer should grow exponentially so that the overhead of reallocating+copying happens less frequently as the buffer grows.

The performance makes it impractical to use <sstream> for any sizable chunk of data, forcing you to use the unsafe <strstream> instead.

>How-To-Repeat:
      Testcase:

#include <iostream>
#include <iomanip>
#include <sstream>
#include <strstream>
#include <time.h>

template <typename SStreamT>
    clock_t
    test(unsigned count)
    {
        SStreamT s;
        const clock_t start = ::clock();
        for (unsigned i = 0; i < count; ++i)
        {
            s << ' ';
        }
        return ::clock() - start;
    }

int main()
{
    using namespace std;

    const unsigned count[] = {10000, 100000, 1000000};
    cout << setw(18) << "iterations"
        << setw(18) << count[0]
        << setw(18) << count[1]
        << setw(18) << count[2] << endl
        << setw(18) << "strstream"
        << setw(18) << test<strstream>(count[0])
        << setw(18) << test<strstream>(count[1])
        << setw(18) << test<strstream>(count[2]) << endl
        << setw(18) << "stringstream"
        << setw(18) << test<stringstream>(count[0])
        << setw(18) << test<stringstream>(count[1])
        << setw(18) << test<stringstream>(count[2]) << endl;
}

Running this on an unloaded 4-way Xeon gives:

        iterations             10000            100000           1000000
         strstream                 0                 1                 3
      stringstream                 2               503            129648

i.e. it takes roughly 1000s to write 1000000 characters to a buffer!

>Fix:
            I've been patching the file on all our development servers for months, without problems (except when OS upgrades overwrite the file with broken versions again). The patched version grows the buffer exponentially, separately tracking the unused capacity and only reallocating when that spare capacity is exhausted.

With my patch the above testcase produces:
Patched on same system:
        iterations             10000            100000           1000000
         strstream                 0                 0                 4
      stringstream                 0                 2                15

I'll attach the patch to this PR
>Release-Note:
>Audit-Trail:

From: "Jonathan Wakely" <jwakely@mintel.com>
To: FreeBSD-gnats-submit@FreeBSD.org, freebsd-bugs@FreeBSD.org
Cc:  
Subject: Re: misc/66941: Unacceptable stringstream performance
Date: Thu, 20 May 2004 14:47:31 +0100

 --0__=CgwBdOWltXmKD7rL5svL8tczv9OKuLWgjrqnxupQdt2vh8iahcnHaO6V
 Content-type: text/plain; charset=us-ascii
 Content-Disposition: inline
 
 
 This is the patch we've been using:
 
 (See attached file: sstream.patch)
 
 --0__=CgwBdOWltXmKD7rL5svL8tczv9OKuLWgjrqnxupQdt2vh8iahcnHaO6V
 Content-type: application/octet-stream; 
 	name="sstream.patch"
 Content-Disposition: attachment; filename="sstream.patch"
 Content-transfer-encoding: base64
 
 LS0tIC91c3IvaW5jbHVkZS9nKysvc3N0cmVhbS5vcmlnCVRodSBGZWIgMTIgMDU6NTI6MjEgMjAw
 NAorKysgL3Vzci9pbmNsdWRlL2crKy9zc3RyZWFtCVRodSBNYXkgMjAgMTA6Mjg6MDMgMjAwNApA
 QCAtNDUsNyArNDUsNyBAQAogICAgIGV4cGxpY2l0CiAgICAgc3RyaW5nYnVmKGludCB3aGljaD1p
 b3M6OmlufGlvczo6b3V0KQogICAgICAgOiBzdHJlYW1idWYoKSwgbW9kZShzdGF0aWNfY2FzdDxp
 b3M6Om9wZW5fbW9kZT4od2hpY2gpKSwKLQlzdHJlYW0oTlVMTCksIHN0cmVhbV9sZW4oMCkKKwlz
 dHJlYW0oTlVMTCksIHN0cmVhbV9sZW4oMCksIHN0cmVhbV9jYXBhY2l0eSgwKQogICAgIHsKICAg
 ICAgIHN0cmluZ2J1Zl9pbml0KCk7CiAgICAgfQpAQCAtNTMsNyArNTMsNyBAQAogICAgIGV4cGxp
 Y2l0CiAgICAgc3RyaW5nYnVmKGNvbnN0IHN0cmluZyAmc3RyLCBpbnQgd2hpY2g9aW9zOjppbnxp
 b3M6Om91dCkKICAgICAgIDogc3RyZWFtYnVmKCksIG1vZGUoc3RhdGljX2Nhc3Q8aW9zOjpvcGVu
 X21vZGU+KHdoaWNoKSksCi0Jc3RyZWFtKE5VTEwpLCBzdHJlYW1fbGVuKDApCisJc3RyZWFtKE5V
 TEwpLCBzdHJlYW1fbGVuKDApLCBzdHJlYW1fY2FwYWNpdHkoMCkKICAgICB7CiAgICAgICBpZiAo
 bW9kZSAmIChpb3M6OmlufGlvczo6b3V0KSkKIAl7CkBAIC04Myw4ICs4Myw4IEBACiAgICAgc3Ry
 KGNvbnN0IHN0cmluZyYgc3RyKQogICAgIHsKICAgICAgIGRlbGV0ZVtdIHN0cmVhbTsKLSAgICAg
 IHN0cmVhbV9sZW4gPSBzdHIuc2l6ZSgpOwotICAgICAgc3RyZWFtID0gbmV3IGNoYXJfdHlwZVtz
 dHJlYW1fbGVuXTsKKyAgICAgIHN0cmVhbV9jYXBhY2l0eSA9IHN0cmVhbV9sZW4gPSBzdHIuc2l6
 ZSgpOworICAgICAgc3RyZWFtID0gbmV3IGNoYXJfdHlwZVtzdHJlYW1fY2FwYWNpdHldOwogICAg
 ICAgc3RyLmNvcHkoc3RyZWFtLCBzdHJlYW1fbGVuKTsKICAgICAgIHN0cmluZ2J1Zl9pbml0KCk7
 CiAgICAgfQpAQCAtMTA1LDEyICsxMDUsMTUgQEAKIAl7CiAJICBpZiAoYyAhPSBFT0YpCiAJICAg
 IHsKLQkgICAgICBzdHJlYW1zaXplIG9sZF9zdHJlYW1fbGVuID0gc3RyZWFtX2xlbjsKKwkgICAg
 ICBpZiAoc3RyZWFtX2xlbiA+PSBzdHJlYW1fY2FwYWNpdHkpCisJCXsKKwkJICBzdHJlYW1fY2Fw
 YWNpdHkgPSAyICogc3RyZWFtX2xlbiArIDEwMDsKKwkJICBjaGFyX3R5cGUqIG5ld19zdHJlYW0g
 PSBuZXcgY2hhcl90eXBlW3N0cmVhbV9jYXBhY2l0eV07CisJCSAgbWVtY3B5KG5ld19zdHJlYW0s
 IHN0cmVhbSwgc3RyZWFtX2xlbik7CisJCSAgZGVsZXRlW10gc3RyZWFtOworCQkgIHN0cmVhbSA9
 IG5ld19zdHJlYW07CisJCX0KIAkgICAgICBzdHJlYW1fbGVuICs9IDE7Ci0JICAgICAgY2hhcl90
 eXBlKiBuZXdfc3RyZWFtID0gbmV3IGNoYXJfdHlwZVtzdHJlYW1fbGVuXTsKLQkgICAgICBtZW1j
 cHkobmV3X3N0cmVhbSwgc3RyZWFtLCBvbGRfc3RyZWFtX2xlbik7Ci0JICAgICAgZGVsZXRlW10g
 c3RyZWFtOwotCSAgICAgIHN0cmVhbSA9IG5ld19zdHJlYW07CiAJICAgICAgc3RyaW5nYnVmX3N5
 bmMoZ3B0cigpLWViYWNrKCksIHBwdHIoKS1wYmFzZSgpKTsKIAkgICAgICBzcHV0YyhjKTsKIAkg
 ICAgICByZXMgPSBjOwpAQCAtMjI1LDYgKzIyOCw3IEBACiAgICAgaW9zOjpvcGVuX21vZGUJbW9k
 ZTsKICAgICBjaGFyX3R5cGUqCQlzdHJlYW07CiAgICAgc3RyZWFtc2l6ZQkJc3RyZWFtX2xlbjsK
 KyAgICBzdHJlYW1zaXplCQlzdHJlYW1fY2FwYWNpdHk7CiAgIH07CiAMCiAgIGNsYXNzIGlzdHJp
 bmdzdHJlYW0gOiBwdWJsaWMgaXN0cmVhbSB7Cg==
 
 --0__=CgwBdOWltXmKD7rL5svL8tczv9OKuLWgjrqnxupQdt2vh8iahcnHaO6V--
 

From: "Jonathan Wakely" <jwakely@mintel.com>
To: FreeBSD-gnats-submit@FreeBSD.org, freebsd-bugs@FreeBSD.org
Cc:  
Subject: Re: misc/66941: Unacceptable stringstream performance
Date: Tue, 1 Jun 2004 09:19:00 +0100

 I should have mentioned that the sstream file does not exist in any
 official release of GCC 2.95, it was added to FreeBSD's unofficial GCC
 2.95.4 release, which is why I reported it here.
 
 The file did exist in the GCC repository briefly between April 2000 and
 Feb 2001 but was not included in any release AFAICT.
 
 It's very unlikely that the patch will be applied to the official GCC
 tree, since the file has been removed and is only in the Attic:
 http://gcc.gnu.org//cgi-bin/cvsweb.cgi/gcc/libstdc++/Attic/sstream
 
 However, if FreeBSD are going to ship the file I believe it should be
 patched so it performs reasonably.
 
 jon
 
 
 

From: David Schultz <das@FreeBSD.ORG>
To: Jonathan Wakely <jwakely@mintel.com>
Cc: FreeBSD-gnats-submit@FreeBSD.ORG, freebsd-bugs@FreeBSD.ORG
Subject: Re: misc/66941: Unacceptable stringstream performance
Date: Wed, 2 Jun 2004 01:36:58 -0700

 On Tue, Jun 01, 2004, Jonathan Wakely wrote:
 > I should have mentioned that the sstream file does not exist in any
 > official release of GCC 2.95, it was added to FreeBSD's unofficial GCC
 > 2.95.4 release, which is why I reported it here.
 > 
 > The file did exist in the GCC repository briefly between April 2000 and
 > Feb 2001 but was not included in any release AFAICT.
 > 
 > It's very unlikely that the patch will be applied to the official GCC
 > tree, since the file has been removed and is only in the Attic:
 > http://gcc.gnu.org//cgi-bin/cvsweb.cgi/gcc/libstdc++/Attic/sstream
 
 I believe it lives in
 	src/contrib/libstdc++/include/std/std_sstream.h
 in the FreeBSD repository, and in
 	 gcc/libstdc++-v3/include/std/std_sstream.h
 in the gcc repository.  It is subsequently installed as
 /usr/include/g++/sstream when libstdc++ is installed.
 
 I glanced at the gcc repository[1] and FreeBSD 5.2-CURRENT, and it
 appears that the file has changed significantly since gcc 2.95.
 However, AFAICT, it still has the same bug.  Unless I'm missing
 something, you probably want to report this problem to the gcc
 folks.  I'm not the right person to say whether the fix will find
 its way back into FreeBSD 4.X, though.
 
 
 [1] http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/libstdc%2b%2b-v3/include/std/std_sstream.h

From: "Jonathan Wakely" <jwakely@mintel.com>
To: David Schultz <das@FreeBSD.ORG>
Cc: FreeBSD-gnats-submit@FreeBSD.ORG, freebsd-bugs@FreeBSD.ORG
Subject: Re: misc/66941: Unacceptable stringstream performance
Date: Wed, 2 Jun 2004 10:17:22 +0100

 Forgot to CC the bug trackers:
 
 
 
 
 > I believe it lives in
 >     src/contrib/libstdc++/include/std/std_sstream.h
 > in the FreeBSD repository, and in
 >      gcc/libstdc++-v3/include/std/std_sstream.h
 > in the gcc repository.  It is subsequently installed as
 > /usr/include/g++/sstream when libstdc++ is installed.
 
 No, this is wrong. These are completely different files, both files you
 list above are the libstdc++-v3 version for GCC 3.x
 
 I'm talking about the sstream header for GCC 2.95.4 which does not exist
 in the official GCC 2.95 branch; see the URL I posted previously, which
 shows the file is in the Attic of the libstdc++ directory, which is for
 v2 of the library.
 In the FreeBSD 4.x tree this file is /usr/src/contrib/libstdc++/sstream.
 Where that file lives in FreeBSD 5.x is irrelevant.
 
 GCC 2.95 used libstdc++-v2, and the sstream header used by FreeBSD
 2.95.4 was written for use with v2.
 
 For GCC 3.x there was a complete rewrite of the C++ library,
 libstdc++-v3. FreeBSD 5.x uses v3.
 
 As a maintainer of libstdc++-v3 I can assure you that any bug reported
 against v2 will be ignored.
 See http://gcc.gnu.org/onlinedocs/libstdc++/faq/index.html#4_4_interface
 
 So if FreeBSD 4.x is going to continue to ship this sstream (which
 remember, was never officially sanctioned by any GCC release) then it is
 up to the maintainer of the FreeBSD GCC 2.95.4 tree to fix this header.
 
 
 > I glanced at the gcc repository[1] and FreeBSD 5.2-CURRENT, and it
 > appears that the file has changed significantly since gcc 2.95.
 
 It's a complete rewrite. In both cases you're looking at GCC 3.x
 sources, I'm talking about a bug in the 2.95 sources.
 
 > However, AFAICT, it still has the same bug.  Unless I'm missing
 > something, you probably want to report this problem to the gcc
 > folks.  I'm not the right person to say whether the fix will find
 > its way back into FreeBSD 4.X, though.
 
 You're missing something.
 
 The header I'm talking about comes with GCC 2.95.4 which is not an
 official GCC release, it was made by the FreeBSD team. The header in
 question was removed from the official GCC 2.95 branch but shipped
 anyway by FreeBSD team.
 
 Comparing the header to it's libstdc++-v3 replacement in the FreeBSD 5.x
 tree is completely pointless, they're totally unrelated.
 
 Since 4.x is still the recommended stable release and our company, and
 numerous others, will not be moving to FreeBSD 5 (and therefore GCC 3)
 for some time this bug causes a serious performance problem in the
 stable release's default C++ compiler.
 
 I know the bug is not present in GCC 3, as I help maintain it, but
 FreeBSD 4 still uses GCC 2.95 where this bug is very present. Someone in
 the FreeBSD team added that file to the sources, so there must be
 someone who can fix it as well - I've supplied the patch!
 
 jon
 
 
 
 
 
 

From: "Jonathan Wakely" <jwakely@mintel.com>
To: David Schultz <das@FreeBSD.ORG>
Cc: FreeBSD-gnats-submit@FreeBSD.ORG, freebsd-bugs@FreeBSD.ORG
Subject: Re: misc/66941: Unacceptable stringstream performance
Date: Wed, 2 Jun 2004 10:50:26 +0100

 This is the file in the FreeBSD repository:
 
 http://www.freebsd.org/cgi/cvsweb.cgi/src/contrib/libstdc%2b%2b/Attic/sstream?hideattic=0
 
 It has been removed from the main trunk, but still exists on the
 RELENG_4 branch, and IMHO should be fixed on that branch.
 
 
 

From: "Jonathan Wakely" <jwakely@mintel.com>
To: David Schultz <das@FreeBSD.ORG>
Cc: FreeBSD-gnats-submit@FreeBSD.ORG, freebsd-bugs@FreeBSD.ORG
Subject: Re: misc/66941: Unacceptable stringstream performance
Date: Wed, 2 Jun 2004 11:09:38 +0100

 This is the Defect Report that makes it clear my improvement is
 standard-conforming, and indeed required for reasonable performance:
 
 http://anubis.dkuug.dk/jtc1/sc22/wg21/docs/lwg-defects.html#169
 
 
 
 

From: "Jonathan Wakely" <jwakely@mintel.com>
To: David Schultz <das@FreeBSD.ORG>
Cc: FreeBSD-gnats-submit@FreeBSD.ORG, freebsd-bugs@FreeBSD.ORG
Subject: Re: misc/66941: Unacceptable stringstream performance
Date: Wed, 2 Jun 2004 11:21:11 +0100

 My apologies, it seems that sstream header *was* included in the
 official 2.95.3 release (I had been assured otherwise) but the fact
 remains that the upstream sources will not be fixed since the GCC 2.95
 branch is inactive
 
 http://savannah.gnu.org/cgi-bin/viewcvs/gcc/gcc/libstdc%2B%2B/Attic/sstream?only_with_tag=gcc-2_95_3&hideattic=1&search=None
 
 
 
State-Changed-From-To: open->closed 
State-Changed-By: linimon 
State-Changed-When: Fri Jun 15 11:27:06 UTC 2007 
State-Changed-Why:  
Thanks for the submission, but since this PR was filed, FreeBSD 4 has 
been EOLed.  Thus, this PR is obsoleted by subsequent events. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=66941 
>Unformatted:
