From greid@ukug.uk.freebsd.org  Tue Oct 31 06:21:58 2000
Return-Path: <greid@ukug.uk.freebsd.org>
Received: from mta03-svc.ntlworld.com (mta03-svc.ntlworld.com [62.253.162.43])
	by hub.freebsd.org (Postfix) with ESMTP id 9CE2737B4CF
	for <FreeBSD-gnats-submit@freebsd.org>; Tue, 31 Oct 2000 06:21:57 -0800 (PST)
Received: from m521-mp1-cvx1b.gui.ntl.com ([62.252.10.9])
          by mta03-svc.ntlworld.com
          (InterMail vM.4.01.02.27 201-229-119-110) with ESMTP
          id <20001031142152.GKTZ283.mta03-svc.ntlworld.com@m521-mp1-cvx1b.gui.ntl.com>
          for <FreeBSD-gnats-submit@freebsd.org>;
          Tue, 31 Oct 2000 14:21:52 +0000
Message-Id: <Pine.BSF.4.21.0010311419040.4159-100000@sobek.nevernet.net>
Date: Tue, 31 Oct 2000 14:24:15 +0000 (GMT)
From: George Reid <greid@ukug.uk.freebsd.org>
To: FreeBSD-gnats-submit@freebsd.org
Subject: [PATCH] Increase speed of split(1)

>Number:         22442
>Category:       bin
>Synopsis:       [PATCH] Increase speed of split(1)
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    freebsd-bugs
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          change-request
>Submitter-Id:   current-users
>Arrival-Date:   Tue Oct 31 06:30:01 PST 2000
>Closed-Date:    Sun Jan 13 03:32:46 UTC 2008
>Last-Modified:  Sun Jan 13 03:32:46 UTC 2008
>Originator:     George Reid
>Release:        FreeBSD 5.0-CURRENT i386
>Organization:
FreeBSD UK User Group
>Environment:
>Description:
The speed of split(1) can be increased by using dynamically-allocated
buffers.

The following results are from a large file created with
dd if=/dev/zero of=bigfile bs=1024k count=300

$ /usr/bin/time split -b 1024k bigfile
       90.19 real         0.02 user        22.42 sys

$ /usr/bin/time ./mysplit -b 1024k bigfile
       81.81 real         0.04 user        25.60 sys

This could represent problems on systems with low memory (perhaps further
modification is needed), but given that the average amount of memory being
distributed with PCs nowadays, this might be useful.

>How-To-Repeat:

>Fix:

--- split.c.orig	Tue Oct 31 14:11:46 2000
+++ split.c	Tue Oct 31 14:16:30 2000
@@ -62,7 +62,7 @@
 long	 numlines;			/* Line count to split on. */
 int	 file_open;			/* If a file open. */
 int	 ifd = -1, ofd = -1;		/* Input/output file descriptors. */
-char	 bfr[MAXBSIZE];			/* I/O buffer. */
+char	 *bfr;				/* I/O buffer. */
 char	 fname[MAXPATHLEN];		/* File name prefix. */
 regex_t	 rgx;
 int	 pflag;
@@ -176,11 +176,16 @@
 	int dist, len;
 	char *C;
 
+	if((bfr = (char *)malloc(bytecnt)) == NULL)
+		err(EX_OSERR, "malloc");
+
 	for (bcnt = 0;;)
-		switch ((len = read(ifd, bfr, MAXBSIZE))) {
+		switch ((len = read(ifd, bfr, bytecnt))) {
 		case 0:
+			free(bfr);
 			exit(0);
 		case -1:
+			free(bfr);
 			err(EX_IOERR, "read");
 			/* NOTREACHED */
 		default:
@@ -195,22 +200,29 @@
 				    len -= bytecnt, C += bytecnt) {
 					newfile();
 					if (write(ofd,
-					    C, (int)bytecnt) != bytecnt)
+					    C, (int)bytecnt) != bytecnt) {
+						free(bfr);
 						err(EX_IOERR, "write");
+					}
 				}
 				if (len != 0) {
 					newfile();
-					if (write(ofd, C, len) != len)
+					if (write(ofd, C, len) != len) {
+						free(bfr);
 						err(EX_IOERR, "write");
+					}
 				} else
 					file_open = 0;
 				bcnt = len;
 			} else {
 				bcnt += len;
-				if (write(ofd, bfr, len) != len)
+				if (write(ofd, bfr, len) != len) {
+					free(bfr);
 					err(EX_IOERR, "write");
+				}
 			}
 		}
+	free(bfr);
 }
 
 /*
@@ -227,6 +239,9 @@
 	if ((infp = fdopen(ifd, "r")) == NULL)
 		err(EX_NOINPUT, "fdopen");
 
+	if((bfr = (char *)malloc(MAXBSIZE)) == NULL)
+		err(EX_OSERR, "malloc");
+
 	/* Process input one line at a time */
 	while (fgets(bfr, sizeof(bfr), infp) != NULL) {
 		const int len = strlen(bfr);
@@ -254,9 +269,13 @@
 			newfile();
 
 		/* Write out line */
-		if (write(ofd, bfr, len) != len)
+		if (write(ofd, bfr, len) != len) {
+			free(bfr);
 			err(EX_IOERR, "write");
+		}
 	}
+
+	free(bfr);
 
 	/* EOF or error? */
 	if (ferror(infp))


>Release-Note:
>Audit-Trail:
Responsible-Changed-From-To: freebsd-bugs->greid 
Responsible-Changed-By: dd 
Responsible-Changed-When: Sun Jul 22 08:50:16 PDT 2001 
Responsible-Changed-Why:  
Over to originator. 

http://www.FreeBSD.org/cgi/query-pr.cgi?pr=22442 
State-Changed-From-To: open->suspended 
State-Changed-By: linimon 
State-Changed-When: Wed Apr 5 04:31:26 UTC 2006 
State-Changed-Why:  
Did this patch ever get committed? 

http://www.freebsd.org/cgi/query-pr.cgi?pr=22442 
Responsible-Changed-From-To: greid->linimon 
Responsible-Changed-By: linimon 
Responsible-Changed-When: Thu Apr 6 04:50:44 UTC 2006 
Responsible-Changed-Why:  
Take over from inactive committer to track feedback. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=22442 
Responsible-Changed-From-To: linimon->freebsd-bugs 
Responsible-Changed-By: linimon 
Responsible-Changed-When: Sun May 6 00:41:13 UTC 2007 
Responsible-Changed-Why:  
Turn this back over to the pool. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=22442 
State-Changed-From-To: suspended->closed 
State-Changed-By: das 
State-Changed-When: Sun Jan 13 03:32:21 UTC 2008 
State-Changed-Why:  
Mail to submitter bounces. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=22442 
>Unformatted:
