From nobody@FreeBSD.org  Mon May 29 22:16:08 2006
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125])
	by hub.freebsd.org (Postfix) with ESMTP id 714BA16B4DA
	for <freebsd-gnats-submit@FreeBSD.org>; Mon, 29 May 2006 22:16:08 +0000 (UTC)
	(envelope-from nobody@FreeBSD.org)
Received: from www.freebsd.org (www.freebsd.org [216.136.204.117])
	by mx1.FreeBSD.org (Postfix) with ESMTP id 238D443D6E
	for <freebsd-gnats-submit@FreeBSD.org>; Mon, 29 May 2006 22:16:08 +0000 (GMT)
	(envelope-from nobody@FreeBSD.org)
Received: from www.freebsd.org (localhost [127.0.0.1])
	by www.freebsd.org (8.13.1/8.13.1) with ESMTP id k4TMG8XO054982
	for <freebsd-gnats-submit@FreeBSD.org>; Mon, 29 May 2006 22:16:08 GMT
	(envelope-from nobody@www.freebsd.org)
Received: (from nobody@localhost)
	by www.freebsd.org (8.13.1/8.13.1/Submit) id k4TMG77F054981;
	Mon, 29 May 2006 22:16:07 GMT
	(envelope-from nobody)
Message-Id: <200605292216.k4TMG77F054981@www.freebsd.org>
Date: Mon, 29 May 2006 22:16:07 GMT
From: "Philip M. Gollucci" <pgollucci@p6m7g8.com>
To: freebsd-gnats-submit@FreeBSD.org
Subject: /etc/groups the parser dies on large groups
X-Send-Pr-Version: www-2.3

>Number:         98111
>Category:       kern
>Synopsis:       [libc] [patch] /etc/groups: the parser dies on large groups
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    maxim
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Mon May 29 22:20:16 GMT 2006
>Closed-Date:    Sat Jul 01 07:27:16 GMT 2006
>Last-Modified:  Tue Aug 29 05:10:22 GMT 2006
>Originator:     Philip M. Gollucci
>Release:        6.1-RELEASE
>Organization:
P6M7G8 Consulting
>Environment:
FreeBSD minotaur.apache.org 6.1-RELEASE FreeBSD 6.1-RELEASE #0: Thu May 11 11:50:25 PDT 2006     root@minotaur.apache.org:/usr/obj/usr/src/sys/SMP-turbo  i38
>Description:
in /etc/groups ... the Apache Software Foundation(ASF) has some large
groups where the parser just craps out.  They been successfully using the
below patch for at least 1.5 years in 5.3-R, 5.4-R, and 6.1-R.

This patch was written by Justin Erenkrantz <justin@erenkrantz.com>.


>How-To-Repeat:
cat /etc/groups 
g:*:1010:a,b,c,d,e,f,g,h,i,j,k,l.m,n,o,p,q,r,s,t,u,v,w,x,y,z,aa,bb,cc,dd,ee,ff,g
g,hh,ii,jj,kk,ll,mm,nn,oo,pp,qq,rr,ss,tt,uu,vv,ww,xx,yy,zz,aaa,bbb,ccc,ddd,eee,f
ff,ggg,hhh,iii,jjj,kkk,lll,mmm,nnn,ooo,ppp,qqq,rrr,sss,ttt,uuu,vvv,www,xxx,yyy,z
zz,aaaa,bbbb,cccc,dddd,eeee,ffff,gggg,hhhh,iiii,jjjj,kkkk,llll,mmmm,nnnn,oooo,pp
pp,qqqq,rrrr,ssss,tttt,uuuu,vvvv,wwww,xxxx,yyyy,zzzz,aaaaa,bbbbb,ccccc,ddddd,eee
ee,fffff,ggggg,hhhhh,iiiii,jjjjj,kkkkk,iiiii,lllll,mmmmm,nnnnn,ooooo,ppppp,qqqqq
,rrrrr,sssss,ttttt,uuuuu,vvvvv,wwwww,xxxxx,yyyyy,zzzzz,pgollucci

id pgollucci
uid=1001(pgollucci) gid=0(wheel) groups=0(wheel)

should have said
uid=1001(pgollucci) gid=0(wheel) groups=0(wheel), 1010(g)

pgollucci:*:1001:0::0:0:Philip M. Gollucci:/home/pgollucci:/bin/tcsh
>Fix:
--- lib/libc/gen/getgrent.c.bak  Tue Jun 28 09:34:10 2005
+++ lib/libc/gen/getgrent.c  Tue Jun 28 10:21:08 2005
@@ -446,6 +446,7 @@
  char      *buffer;
  size_t       bufsize, linesize;
  int      rv, stayopen, *errnop;
+    fpos_t           pos;

  name = NULL;
  gid = (gid_t)-1;
@@ -481,6 +482,7 @@
    stayopen = st->stayopen;
  }
  rv = NS_NOTFOUND;
+    fgetpos(st->fp, &pos);
  while ((line = fgetln(st->fp, &linesize)) != NULL) {
    if (line[linesize-1] == '\n')
      linesize--;
@@ -502,11 +504,15 @@
        &buffer[linesize + 1], bufsize - linesize - 1, errnop);
    if (rv & NS_TERMINATE)
      break;
+        fgetpos(st->fp, &pos);
  }
  if (!stayopen && st->fp != NULL) {
    fclose(st->fp);
    st->fp = NULL;
  }
+    if (st->fp != NULL && rv == NS_RETURN && *errnop == ERANGE) {
+        fsetpos(st->fp, &pos);
+    }
  if (rv == NS_SUCCESS && retval != NULL)
    *(struct group **)retval = grp;
  return (rv);

>Release-Note:
>Audit-Trail:

From: Maxim Konovalov <maxim@macomnet.ru>
To: "Philip M. Gollucci" <pgollucci@p6m7g8.com>
Cc: bug-followup@freebsd.org
Subject: Re: misc/98111: /etc/groups the parser dies on large groups
Date: Tue, 30 May 2006 12:35:44 +0400 (MSD)

 Hi Philip,
 
 yes, I have something like that sitting in my src tree.  Hope I'll
 work it out soon.
 
 -- 
 Maxim Konovalov
State-Changed-From-To: open->patched 
State-Changed-By: maxim 
State-Changed-When: Thu Jun 1 16:04:32 UTC 2006 
State-Changed-Why:  
Fixed in HEAD.  Thanks! 


Responsible-Changed-From-To: freebsd-bugs->maxim 
Responsible-Changed-By: maxim 
Responsible-Changed-When: Thu Jun 1 16:04:32 UTC 2006 
Responsible-Changed-Why:  
Feedbacks trap. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=98111 
State-Changed-From-To: patched->closed 
State-Changed-By: maxim 
State-Changed-When: Sat Jul 1 07:26:53 UTC 2006 
State-Changed-Why:  
Merged to RELENG_6. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=98111 

From: Kirk R Webb <kwebb@flux.utah.edu>
To: bug-followup@FreeBSD.org, pgollucci@p6m7g8.com
Cc:  
Subject: Re: kern/98111 : [libc] [patch] /etc/groups: the parser dies on large groups
Date: Mon, 10 Jul 2006 18:29:41 -0600

 There appears to still be an unhandled case here.  The case where the 
 group entry line does not fit in the provided buffer is handled fine, but 
 the other situation that ISN'T handled is when the parsed group structure 
 doesn't fit into the latter portion of the buffer (the remainder after the 
 current group entry line).
 
 It's easy to reproduce this problem with version 1.34 of getgrent.c; just 
 use the following group line:
 
 badgroup:*:666:foo1,foo2,foo3,foo4,foo5,foo6,foo7,foo8,foo9,foo10,foo11,foo12,foo13,foo14,foo15,foo16,foo17,foo18,foo19,foo20,foo21,foo22,foo23,foo24,foo25,foo26,foo27,foo28,foo29,foo30,foo31,foo32,foo33,foo34,foo35,foo36,foo37,foo38,foo39,foo40,foo41,foo42,foo43,foo44,foo45,foo46,foo47,foo48,foo49,foo50,foo51,foo52,foo53,foo54,foo55,foo56,foo57,foo58,foo59,foo60,foo61,foo62,foo63,foo64,foo65,foo66,foo67,foo68,foo69,foo70,foo71,foo72,foo73,foo74,foo75,foo76,foo77,foo78,foo79,foo80,foo81,foo82,foo83,foo84,f oo85,foo86,foo87,foo88,foo89,foo90,foo91,foo92,foo93,foo94,foo95,foo96,foo97,foo98,foo99,foo100,foo101,foo102
 
 If you remove the last entry (foo102), the group reappears.  Add about 75 
 more foo*** entries, and it also reappears (once it blows over the size of 
 the original buffer).
 
 Here is a patch to getgrent.c (HEAD) that fixes the fix introduced in 
 1.34 to work in both overrun cases:
 
 --- getgrent.c.bad  Mon Jul 10 17:47:36 2006
 +++ getgrent.c	Mon Jul 10 17:21:55 2006
 @@ -770,18 +770,17 @@
  		 * pointer for the member list terminator.
  		 */
  		if (bufsize <= linesize + _ALIGNBYTES + sizeof(char *)) {
 -			fseeko(st->fp, pos, SEEK_SET);
  			*errnop = ERANGE;
  			rv = NS_RETURN;
  			break;
  		}
 -		pos = ftello(st->fp);
  		memcpy(buffer, line, linesize);
  		buffer[linesize] = '\0';
  		rv = __gr_parse_entry(buffer, linesize, grp, 
  		    &buffer[linesize + 1], bufsize - linesize - 1, errnop);
  		if (rv & NS_TERMINATE)
  			break;
 +		pos = ftello(st->fp);
  	}
  	if (!stayopen && st->fp != NULL) {
  		fclose(st->fp);
 @@ -789,6 +788,8 @@
  	}
  	if (rv == NS_SUCCESS && retval != NULL)
  		*(struct group **)retval = grp;
 +        else if (*errnop == ERANGE)
 +                fseeko(st->fp, pos, SEEK_SET);
  	return (rv);
  }
  

From: Kirk R Webb <kwebb@flux.utah.edu>
To: bug-followup@FreeBSD.org, pgollucci@p6m7g8.com
Cc:  
Subject: Re: kern/98111 : [libc] [patch] /etc/groups: the parser dies on large groups
Date: Mon, 10 Jul 2006 19:16:46 -0600

 RE my last post:  A similar fix also needs to be applied to the 
 compat_group function in getgrent.c
 
 -Kirk

From: Maxim Konovalov <maxim@macomnet.ru>
To: Kirk R Webb <kwebb@flux.utah.edu>
Cc: bug-followup@freebsd.org
Subject: Re: kern/98111 : [libc] [patch] /etc/groups: the parser dies on
 large groups
Date: Fri, 14 Jul 2006 21:30:44 +0400 (MSD)

 Hi Kirk,
 
 On Tue, 11 Jul 2006, 00:30-0000, Kirk R Webb wrote:
 
 >  There appears to still be an unhandled case here.  The case where the
 >  group entry line does not fit in the provided buffer is handled fine, but
 >  the other situation that ISN'T handled is when the parsed group structure
 >  doesn't fit into the latter portion of the buffer (the remainder after the
 >  current group entry line).
 >
 >  It's easy to reproduce this problem with version 1.34 of getgrent.c; just
 >  use the following group line:
 >
 >  badgroup:*:666:foo1,foo2,foo3,foo4,foo5,foo6,foo7,foo8,foo9,foo10,foo11,foo12,foo13,foo14,foo15,foo16,foo17,foo18,foo19,foo20,foo21,foo22,foo23,foo24,foo25,foo26,foo27,foo28,foo29,foo30,foo31,foo32,foo33,foo34,foo35,foo36,foo37,foo38,foo39,foo40,foo41,foo42,foo43,foo44,foo45,foo46,foo47,foo48,foo49,foo50,foo51,foo52,foo53,foo54,foo55,foo56,foo57,foo58,foo59,foo60,foo61,foo62,foo63,foo64,foo65,foo66,foo67,foo68,foo69,foo70,foo71,foo72,foo73,foo74,foo75,foo76,foo77,foo78,foo79,foo80,foo81,foo82,foo83,foo8 4,f oo85,foo86,foo87,foo88,foo89,foo90,foo91,foo92,foo93,foo94,foo95,foo96,foo97,foo98,foo99,foo100,foo101,foo102
 >
 >  If you remove the last entry (foo102), the group reappears.  Add about 75
 >  more foo*** entries, and it also reappears (once it blows over the size of
 >  the original buffer).
 >
 >  Here is a patch to getgrent.c (HEAD) that fixes the fix introduced in
 >  1.34 to work in both overrun cases:
 [...]
 
 Yes, you are right.  I missed the fact that buffer is used for two
 purposes:
 
 1) storing a group line from the group file;
 
 2) __gr_parse_entry()  parses the buffer and tries to put the group
    members to the remaining part of the jbuffer and can fail if there
    is no enough room for them.
 
 I'll commit your patch shortly.  Thanks!
 
 -- 
 Maxim Konovalov

From: Mark Costlow <cheeks@swcp.com>
To: bug-followup@FreeBSD.org, pgollucci@p6m7g8.com, maxim@macomnet.ru
Cc:  
Subject: Re: kern/98111: [libc] [patch] /etc/groups: the parser dies on large groups
Date: Mon, 28 Aug 2006 23:07:57 GMT

 On Fri, 14 Jul 2006, 21:30:44 +0400 (MSD), Maxim Konavalov wrote:
 
 > On Tue, 11 Jul 2006, 00:30-0000, Kirk R Webb wrote:
 > 
 > >  There appears to still be an unhandled case here.  The case where the
 > >  group entry line does not fit in the provided buffer is handled fine, but
 > >  the other situation that ISN'T handled is when the parsed group structure
 > >  doesn't fit into the latter portion of the buffer (the remainder after the
 > >  current group entry line).
 > >
 > >  It's easy to reproduce this problem with version 1.34 of getgrent.c; just
 > >  use the following group line:
 > >
 > >  badgroup:*:666:foo1,foo2,foo3,foo4,foo5,foo6,foo7,foo8,foo9,foo10,foo11,foo12,foo13,foo14,foo15,foo16,foo17,foo18,foo19,foo20,foo21,foo22,foo23,foo24,foo25,foo26,foo27,foo28,foo29,foo30,foo31,foo32,foo33,foo34,foo35,foo36,foo37,foo38,foo39,foo40,foo41,foo42,foo43,foo44,foo45,foo46,foo47,foo48,foo49,foo50,foo51,foo52,foo53,foo54,foo55,foo56,foo57,foo58,foo59,foo60,foo61,foo62,foo63,foo64,foo65,foo66,foo67,foo68,foo69,foo70,foo71,foo72,foo73,foo74,foo75,foo76,foo77,foo78,foo79,foo80,foo81,foo82,foo83,fo o8 4,f oo85,foo86,foo87,foo88,foo89,foo90,foo91,foo92,foo93,foo94,foo95,foo96,foo97,foo98,foo99,foo100,foo101,foo102
 > >
 > >  If you remove the last entry (foo102), the group reappears.  Add about 75
 > >  more foo*** entries, and it also reappears (once it blows over the size of
 > >  the original buffer).
 > >
 > >  Here is a patch to getgrent.c (HEAD) that fixes the fix introduced in
 > >  1.34 to work in both overrun cases:
 > [...]
 > 
 > Yes, you are right.  I missed the fact that buffer is used for two
 > purposes:
 > 
 > 1) storing a group line from the group file;
 > 
 > 2) __gr_parse_entry()  parses the buffer and tries to put the group
 >    members to the remaining part of the jbuffer and can fail if there
 >    is no enough room for them.
 > 
 > I'll commit your patch shortly.  Thanks!
 
 Hi there. On a 6.1-STABLE system with this getgrent.c:
 
 "$FreeBSD: src/lib/libc/gen/getgrent.c,v 1.32.8.2 2006/07/28 16:17:49 maxim
 Exp $");
 
 We had problems with seg faults.  Perl would segfault when getgrnam() was used
 on a group with > 200 members, as would "pw groupmod".
 
 This patch against 1.32.8.2 works for me (based partly on something posted by
 <pgollucci@p6m7g8.com>.  I think the basic problem is that in some cases
 st->fp was closed and then fseeko() was called on the NULL file pointer.
 
 
 
 *** getgrent.c.orig     Mon Aug 28 12:34:59 2006
 --- getgrent.c  Mon Aug 28 16:50:40 2006
 ***************
 *** 510,519 ****
                 fclose(st->fp);
                 st->fp = NULL;
         }
         if (rv == NS_SUCCESS && retval != NULL)
                 *(struct group **)retval = grp;
 -       else if (*errnop == ERANGE)
 -               fseeko(st->fp, pos, SEEK_SET);
         return (rv);
   }
   
 --- 510,520 ----
                 fclose(st->fp);
                 st->fp = NULL;
         }
 +       if (st->fp != NULL && rv == NS_RETURN && *errnop == ERANGE) {
 +               fseeko(st->fp, pos, SEEK_SET);
 +       }
         if (rv == NS_SUCCESS && retval != NULL)
                 *(struct group **)retval = grp;
         return (rv);
   }
   
 ***************
 *** 1066,1072 ****
         }
         if (rv == NS_SUCCESS && retval != NULL)
                 *(struct group **)retval = grp;
 !       else if (*errnop == ERANGE)
                 fseeko(st->fp, pos, SEEK_SET);
         return (rv);
   #undef set_lookup_type
 --- 1067,1073 ----
         }
         if (rv == NS_SUCCESS && retval != NULL)
                 *(struct group **)retval = grp;
 !       else if (*errnop == ERANGE && st->fp != NULL)
                 fseeko(st->fp, pos, SEEK_SET);
         return (rv);
   #undef set_lookup_type
 
 -- 
 Mark Costlow    | Southwest Cyberport | Fax:   +1-505-232-7975
 cheeks@swcp.com | Web:   www.swcp.com | Voice: +1-505-232-7992
 
 abq-strange.com -- Interesting photos taken in Albuquerque, NM
                    Last post: Big Red Ride @ 2006-05-13 16:12:17

From: Maxim Konovalov <maxim@macomnet.ru>
To: Mark Costlow <cheeks@swcp.com>
Cc: bug-followup@FreeBSD.org, pgollucci@p6m7g8.com
Subject: Re: kern/98111: [libc] [patch] /etc/groups: the parser dies on large
 groups
Date: Tue, 29 Aug 2006 09:03:24 +0400 (MSD)

 Hi Mark,
 
 > Hi there. On a 6.1-STABLE system with this getgrent.c:
 >
 > "$FreeBSD: src/lib/libc/gen/getgrent.c,v 1.32.8.2 2006/07/28 16:17:49 maxim
 > Exp $");
 >
 > We had problems with seg faults.  Perl would segfault when
 > getgrnam() was used on a group with > 200 members, as would "pw
 > groupmod".
 >
 > This patch against 1.32.8.2 works for me (based partly on something
 > posted by <pgollucci@p6m7g8.com>.  I think the basic problem is that
 > in some cases st->fp was closed and then fseeko() was called on the
 > NULL file pointer.
 >
 >
 >
 > *** getgrent.c.orig     Mon Aug 28 12:34:59 2006
 > --- getgrent.c  Mon Aug 28 16:50:40 2006
 > ***************
 > *** 510,519 ****
 >                 fclose(st->fp);
 >                 st->fp = NULL;
 >         }
 >         if (rv == NS_SUCCESS && retval != NULL)
 >                 *(struct group **)retval = grp;
 > -       else if (*errnop == ERANGE)
 > -               fseeko(st->fp, pos, SEEK_SET);
 >         return (rv);
 >   }
 >
 > --- 510,520 ----
 >                 fclose(st->fp);
 >                 st->fp = NULL;
 >         }
 > +       if (st->fp != NULL && rv == NS_RETURN && *errnop == ERANGE) {
 > +               fseeko(st->fp, pos, SEEK_SET);
 > +       }
 >         if (rv == NS_SUCCESS && retval != NULL)
 >                 *(struct group **)retval = grp;
 >         return (rv);
 >   }
 >
 > ***************
 > *** 1066,1072 ****
 >         }
 >         if (rv == NS_SUCCESS && retval != NULL)
 >                 *(struct group **)retval = grp;
 > !       else if (*errnop == ERANGE)
 >                 fseeko(st->fp, pos, SEEK_SET);
 >         return (rv);
 >   #undef set_lookup_type
 > --- 1067,1073 ----
 >         }
 >         if (rv == NS_SUCCESS && retval != NULL)
 >                 *(struct group **)retval = grp;
 > !       else if (*errnop == ERANGE && st->fp != NULL)
 >                 fseeko(st->fp, pos, SEEK_SET);
 >         return (rv);
 >   #undef set_lookup_type
 
 Yes, it seems to be right.  It's not clear for me why the code for
 files_group() and compat_group differs.  I tweak the patch above
 sligthly.  Could you please test it in your environment?  TIA!
 
 Index: getgrent.c
 ===================================================================
 RCS file: /home/ncvs/src/lib/libc/gen/getgrent.c,v
 retrieving revision 1.35
 diff -u -p -r1.35 getgrent.c
 --- getgrent.c	14 Jul 2006 17:45:33 -0000	1.35
 +++ getgrent.c	29 Aug 2006 05:02:23 -0000
 @@ -788,7 +788,7 @@ files_group(void *retval, void *mdata, v
  	}
  	if (rv == NS_SUCCESS && retval != NULL)
  		*(struct group **)retval = grp;
 -	else if (*errnop == ERANGE)
 +	else if (rv == NS_RETURN && *errnop == ERANGE && st->fp != NULL)
  		fseeko(st->fp, pos, SEEK_SET);
  	return (rv);
  }
 @@ -1342,7 +1342,7 @@ fin:
  	}
  	if (rv == NS_SUCCESS && retval != NULL)
  		*(struct group **)retval = grp;
 -	else if (*errnop == ERANGE)
 +	else if (rv == NS_RETURN && *errnop == ERANGE && st->fp != NULL)
  		fseeko(st->fp, pos, SEEK_SET);
  	return (rv);
  #undef set_lookup_type
 %%%
 
 -- 
 Maxim Konovalov
>Unformatted:
