From pmc@phase23.dinoex.sub.de  Sat Apr 11 14:13:38 2009
Return-Path: <pmc@phase23.dinoex.sub.de>
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id E134E1065675
	for <FreeBSD-gnats-submit@freebsd.org>; Sat, 11 Apr 2009 14:13:38 +0000 (UTC)
	(envelope-from pmc@phase23.dinoex.sub.de)
Received: from uucp.dinoex.sub.de (uucp.dinoex.sub.de [194.45.71.2])
	by mx1.freebsd.org (Postfix) with ESMTP id 6182C8FC17
	for <FreeBSD-gnats-submit@freebsd.org>; Sat, 11 Apr 2009 14:13:37 +0000 (UTC)
	(envelope-from pmc@phase23.dinoex.sub.de)
Received: from uucp.dinoex.sub.de (uucp@uucp.dinoex.sub.de [194.45.71.2] (may be forged))
	by uucp.dinoex.sub.de (8.14.2/8.14.2) with ESMTP id n3BED7O0041393
	for <FreeBSD-gnats-submit@freebsd.org>; Sat, 11 Apr 2009 16:13:07 +0200 (CEST)
	(envelope-from pmc@phase23.dinoex.sub.de)
Received: from phase23.dinoex.sub.de (uucp@localhost)
	by uucp.dinoex.sub.de (8.14.2/8.14.2/Submit) with UUCP id n3BED7iM041392
	for FreeBSD-gnats-submit@freebsd.org; Sat, 11 Apr 2009 16:13:07 +0200 (CEST)
	(envelope-from pmc@phase23.dinoex.sub.de)
Received: from gate.oper.dinoex.org (gate-e [192.168.98.2])
	by citylink.dinoex.sub.de (8.14.2/8.14.2) with ESMTP id n3BDPhPh095224
	for <FreeBSD-gnats-submit@freebsd.org>; Sat, 11 Apr 2009 15:25:44 +0200 (CEST)
	(envelope-from pmc@disp.oper.dinoex.org)
Received: from disp.oper.dinoex.org (disp-fe.oper.dinoex.org [192.168.96.5])
	by gate.oper.dinoex.org (8.14.2/8.14.2) with ESMTP id n3BDO0rX095013
	for <FreeBSD-gnats-submit@freebsd.org>; Sat, 11 Apr 2009 15:24:01 +0200 (CEST)
	(envelope-from pmc@disp.oper.dinoex.org)
Received: from disp.oper.dinoex.org (localhost [127.0.0.1])
	by disp.oper.dinoex.org (8.14.3/8.14.3) with ESMTP id n3BDNNM7062960
	for <FreeBSD-gnats-submit@freebsd.org>; Sat, 11 Apr 2009 15:23:23 +0200 (CEST)
	(envelope-from pmc@disp.oper.dinoex.org)
Received: (from pmc@localhost)
	by disp.oper.dinoex.org (8.14.3/8.14.3/Submit) id n3BDNMs3062959;
	Sat, 11 Apr 2009 15:23:23 +0200 (CEST)
	(envelope-from pmc)
Message-Id: <200904111323.n3BDNMs3062959@disp.oper.dinoex.org>
Date: Sat, 11 Apr 2009 15:23:23 +0200 (CEST)
From: Peter Much <pmc@citylink.dinoex.sub.org>
Reply-To: Peter Much <pmc@citylink.dinoex.sub.org>
To: FreeBSD-gnats-submit@freebsd.org
Cc:
Subject: [panic] writing to gjournal on a gvinum volume
X-Send-Pr-Version: 3.113
X-GNATS-Notify:

>Number:         133604
>Category:       kern
>Synopsis:       [gvinum] [panic] writing to gjournal on a gvinum volume
>Confidential:   no
>Severity:       non-critical
>Priority:       medium
>Responsible:    freebsd-geom
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          change-request
>Submitter-Id:   current-users
>Arrival-Date:   Sat Apr 11 14:20:03 UTC 2009
>Closed-Date:    Wed May 06 19:46:50 UTC 2009
>Last-Modified:  Wed May 06 19:46:50 UTC 2009
>Originator:     Peter Much
>Release:        FreeBSD 7.2-PRERELEASE i386
>Organization:
n/a
>Environment:
System: FreeBSD disp.oper.dinoex.org 7.2-PRERELEASE FreeBSD 7.2-PRERELEASE #1: Thu Apr 9 22:23:20 CEST 2009 root@disp.oper.dinoex.org:/usr/src/sys/i386/compile/D1R72V1 i386

>Description:

A gjournal can be created on a vinum volume. But when writing the 
filesystem onto it, 2-3 seconds later the system always does crash. 
The panic comes from the gv_v worker process, interrupt-called.

I am managing all my filesystems with gvinum (it seems the best 
approach to a volume manager that we currently have), so I need
the other GEOM goodies (like crypt, compress, whatever) *above*
a vinum volume.
When I tried last (on RELEASE 5), there seemed to be no way at all
to achieve such.
Now it looks quite good - gbde and geli seem to work fine, their 
products can even be brought back into vinum as a "new" drive, so
to slice various filesystems from them.

Sadly, journaling is the only thing I really need from RELEASE 7,
and I only need it for about one filesystem, for the database of
my tapebackup-server (to overcome the "full_page_writes" need in 
postgresql and get smaller redo-logs).

But I am not sure if coupling gvinum with other GEOM tools is 
indeed an intended functionality ; therefore I pronounce this as 
a change-request ;)

>How-To-Repeat:

 - create some simple concat volume 'myjfs' in gvinum 
 - gjournal load
 - gjournal label -chs 100m gvinum/myjfs
 - newfs -J /dev/gvinum/myjfs.journal

>Fix:

When exporting the gvinum volume with ggated to localhost, then bringing 
it in again with ggatec, and creating the journal on ggate0, the
panic does not happen. But this performs rather bad and occasionally
hangs completely - maybe needs careful buffer tuning.
I'm trying ZFS now - which seems to run fine above gvinum - and
would even be a nice replacement for gvinum in toto - but I dowbt this 
will run on that tapebackup-server without installing a lot more ram
(there is only 256MB).
>Release-Note:
>Audit-Trail:
Responsible-Changed-From-To: freebsd-bugs->freebsd-geom 
Responsible-Changed-By: linimon 
Responsible-Changed-When: Sat Apr 11 22:10:10 UTC 2009 
Responsible-Changed-Why:  
Over to maintainer(s). 

http://www.freebsd.org/cgi/query-pr.cgi?pr=133604 

From: Ulf Lilleengen <ulf.lilleengen@gmail.com>
To: bug-followup@FreeBSD.org, pmc@citylink.dinoex.sub.org
Cc:  
Subject: Re: kern/133604: [gvinum] [panic] writing to gjournal on a gvinum 
	volume
Date: Tue, 14 Apr 2009 10:21:54 +0200

 --000e0cd25b640a4f6004677f8657
 Content-Type: text/plain; charset=UTF-8
 Content-Transfer-Encoding: 7bit
 
 On Sun, Apr 12, 2009 at 12:10 AM, <linimon@freebsd.org> wrote:
 
 > Old Synopsis: [panic] writing to gjournal on a gvinum volume
 > New Synopsis: [gvinum] [panic] writing to gjournal on a gvinum volume
 >
 > Responsible-Changed-From-To: freebsd-bugs->freebsd-geom
 > Responsible-Changed-By: linimon
 > Responsible-Changed-When: Sat Apr 11 22:10:10 UTC 2009
 > Responsible-Changed-Why:
 > Over to maintainer(s).
 >
 > http://www.freebsd.org/cgi/query-pr.cgi?pr=133604
 > _______________________________________________
 > freebsd-geom@freebsd.org mailing list
 > http://lists.freebsd.org/mailman/listinfo/freebsd-geom
 > To unsubscribe, send any mail to "freebsd-geom-unsubscribe@freebsd.org"
 
 
 Are you able to try gvinum from HEAD to see if it works there? I'll take a
 look into gvinum on 7.2-PRE as well, but just in case you would like to try
 it.
 
 -- 
 Ulf Lilleengen
 
 --000e0cd25b640a4f6004677f8657
 Content-Type: text/html; charset=UTF-8
 Content-Transfer-Encoding: quoted-printable
 
 <br><br><div class=3D"gmail_quote">On Sun, Apr 12, 2009 at 12:10 AM,  <span=
  dir=3D"ltr">&lt;<a href=3D"mailto:linimon@freebsd.org">linimon@freebsd.org=
 </a>&gt;</span> wrote:<br><blockquote class=3D"gmail_quote" style=3D"border=
 -left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-lef=
 t: 1ex;">
 Old Synopsis: [panic] writing to gjournal on a gvinum volume<br>
 New Synopsis: [gvinum] [panic] writing to gjournal on a gvinum volume<br>
 <br>
 Responsible-Changed-From-To: freebsd-bugs-&gt;freebsd-geom<br>
 Responsible-Changed-By: linimon<br>
 Responsible-Changed-When: Sat Apr 11 22:10:10 UTC 2009<br>
 Responsible-Changed-Why:<br>
 Over to maintainer(s).<br>
 <br>
 <a href=3D"http://www.freebsd.org/cgi/query-pr.cgi?pr=3D133604" target=3D"_=
 blank">http://www.freebsd.org/cgi/query-pr.cgi?pr=3D133604</a><br>
 _______________________________________________<br>
 <a href=3D"mailto:freebsd-geom@freebsd.org">freebsd-geom@freebsd.org</a> ma=
 iling list<br>
 <a href=3D"http://lists.freebsd.org/mailman/listinfo/freebsd-geom" target=
 =3D"_blank">http://lists.freebsd.org/mailman/listinfo/freebsd-geom</a><br>
 To unsubscribe, send any mail to &quot;<a href=3D"mailto:freebsd-geom-unsub=
 scribe@freebsd.org">freebsd-geom-unsubscribe@freebsd.org</a>&quot;</blockqu=
 ote><div>=C2=A0</div></div>Are you able to try gvinum from HEAD to see if i=
 t works there? I&#39;ll take a look into gvinum on 7.2-PRE as well, but jus=
 t in case you would like to try it.<br clear=3D"all">
 <br>-- <br>Ulf Lilleengen<br>
 
 --000e0cd25b640a4f6004677f8657--

From: Ulf Lilleengen <lulf@FreeBSD.org>
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/133604: [gvinum] [panic] writing to gjournal on a gvinum
	volume
Date: Sun, 19 Apr 2009 13:02:54 +0200

 On Tue, Apr 14, 2009 at 09:00:07AM +0000, Ulf Lilleengen wrote:
 > The following reply was made to PR kern/133604; it has been noted by GNATS.
 > 
 > From: Ulf Lilleengen <ulf.lilleengen@gmail.com>
 > To: bug-followup@FreeBSD.org, pmc@citylink.dinoex.sub.org
 > Cc:  
 > Subject: Re: kern/133604: [gvinum] [panic] writing to gjournal on a gvinum 
 > 	volume
 > Date: Tue, 14 Apr 2009 10:21:54 +0200
 > 
 >  --000e0cd25b640a4f6004677f8657
 >  Content-Type: text/plain; charset=UTF-8
 >  Content-Transfer-Encoding: 7bit
 >  
 >  On Sun, Apr 12, 2009 at 12:10 AM, <linimon@freebsd.org> wrote:
 >  
 >  > Old Synopsis: [panic] writing to gjournal on a gvinum volume
 >  > New Synopsis: [gvinum] [panic] writing to gjournal on a gvinum volume
 >  >
 >  > Responsible-Changed-From-To: freebsd-bugs->freebsd-geom
 >  > Responsible-Changed-By: linimon
 >  > Responsible-Changed-When: Sat Apr 11 22:10:10 UTC 2009
 >  > Responsible-Changed-Why:
 >  > Over to maintainer(s).
 >  >
 >  > http://www.freebsd.org/cgi/query-pr.cgi?pr=133604
 >  > _______________________________________________
 >  > freebsd-geom@freebsd.org mailing list
 >  > http://lists.freebsd.org/mailman/listinfo/freebsd-geom
 >  > To unsubscribe, send any mail to "freebsd-geom-unsubscribe@freebsd.org"
 >  
 >  
 >  Are you able to try gvinum from HEAD to see if it works there? I'll take a
 >  look into gvinum on 7.2-PRE as well, but just in case you would like to try
 >  it.
 >  
 
 Ok, I tracked down the issue. The problem is in both HEAD and 7.x, so I will
 fix it in HEAD first. The problem is the bio_cflags part of the bio that
 gjournal sends down to gvinum. When gjournal does a read, it sets GJ_BIO_READ
 in bio_cflags. When the request arrives to gvinum it check cflags for
 GV_BIO_DONE to check if the request is a done request, in which case it will
 be treated like a bio of gvinum. The crash comes from GJ_BIO_READ having the
 same value as GV_BIO_DONE. I'll generate a fix that makes gvinum not depend
 on the bio_cflags value of external BIOs.
 
 -- 
 Ulf Lilleengen

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/133604: commit references a PR
Date: Wed,  6 May 2009 19:34:42 +0000 (UTC)

 Author: lulf
 Date: Wed May  6 19:34:32 2009
 New Revision: 191856
 URL: http://svn.freebsd.org/changeset/base/191856
 
 Log:
   - Split up the BIO queue into a queue for new and one for completed requests.
     This is necessary for two reasons:
     1) In order to avoid collisions with the use of a BIOs flags set by a consumer
        or a provider
     2) Because GV_BIO_DONE was used to mark a BIO as done, not enough flags was
        available, so the consumer flags of a BIO had to be misused in order to
        support enough flags. The new queue makes it possible to recycle the
        GV_BIO_DONE flag into GV_BIO_GROW.
     As a consequence, gvinum will now work with any other GEOM class under it or
     on top of it.
   
   - Use bio_pflags for storing internal flags on downgoing BIOs, as the requests
     appear to come from a consumer of a gvinum volume. Use bio_cflags only for
     cloned BIOs.
   - Move gv_post_bio to be used internally for maintenance requests.
   - Remove some cases where flags where set without need.
   
   PR:		kern/133604
 
 Modified:
   head/sys/geom/vinum/geom_vinum.c
   head/sys/geom/vinum/geom_vinum.h
   head/sys/geom/vinum/geom_vinum_plex.c
   head/sys/geom/vinum/geom_vinum_raid5.c
   head/sys/geom/vinum/geom_vinum_var.h
 
 Modified: head/sys/geom/vinum/geom_vinum.c
 ==============================================================================
 --- head/sys/geom/vinum/geom_vinum.c	Wed May  6 19:18:19 2009	(r191855)
 +++ head/sys/geom/vinum/geom_vinum.c	Wed May  6 19:34:32 2009	(r191856)
 @@ -81,18 +81,6 @@ gv_orphan(struct g_consumer *cp)
  }
  
  void
 -gv_post_bio(struct gv_softc *sc, struct bio *bp)
 -{
 -
 -	KASSERT(sc != NULL, ("NULL sc"));
 -	KASSERT(bp != NULL, ("NULL bp"));
 -	mtx_lock(&sc->bqueue_mtx);
 -	bioq_disksort(sc->bqueue, bp);
 -	wakeup(sc);
 -	mtx_unlock(&sc->bqueue_mtx);
 -}
 -
 -void
  gv_start(struct bio *bp)
  {
  	struct g_geom *gp;
 @@ -111,8 +99,10 @@ gv_start(struct bio *bp)
  		g_io_deliver(bp, EOPNOTSUPP);
  		return;
  	}
 -
 -	gv_post_bio(sc, bp);
 +	mtx_lock(&sc->bqueue_mtx);
 +	bioq_disksort(sc->bqueue_down, bp);
 +	wakeup(sc);
 +	mtx_unlock(&sc->bqueue_mtx);
  }
  
  void
 @@ -125,9 +115,11 @@ gv_done(struct bio *bp)
  
  	gp = bp->bio_from->geom;
  	sc = gp->softc;
 -	bp->bio_cflags |= GV_BIO_DONE;
  
 -	gv_post_bio(sc, bp);
 +	mtx_lock(&sc->bqueue_mtx);
 +	bioq_disksort(sc->bqueue_up, bp);
 +	wakeup(sc);
 +	mtx_unlock(&sc->bqueue_mtx);
  }
  
  int
 @@ -179,8 +171,12 @@ gv_init(struct g_class *mp)
  	gp->softc = g_malloc(sizeof(struct gv_softc), M_WAITOK | M_ZERO);
  	sc = gp->softc;
  	sc->geom = gp;
 -	sc->bqueue = g_malloc(sizeof(struct bio_queue_head), M_WAITOK | M_ZERO);
 -	bioq_init(sc->bqueue);
 +	sc->bqueue_down = g_malloc(sizeof(struct bio_queue_head),
 +	    M_WAITOK | M_ZERO);
 +	sc->bqueue_up = g_malloc(sizeof(struct bio_queue_head),
 +	    M_WAITOK | M_ZERO);
 +	bioq_init(sc->bqueue_down);
 +	bioq_init(sc->bqueue_up);
  	LIST_INIT(&sc->drives);
  	LIST_INIT(&sc->subdisks);
  	LIST_INIT(&sc->plexes);
 @@ -969,7 +965,8 @@ gv_worker(void *arg)
  				gv_cleanup(sc);
  				mtx_destroy(&sc->bqueue_mtx);
  				mtx_destroy(&sc->equeue_mtx);
 -				g_free(sc->bqueue);
 +				g_free(sc->bqueue_down);
 +				g_free(sc->bqueue_up);
  				g_free(sc);
  				kproc_exit(ENXIO);
  				break;			/* not reached */
 @@ -984,38 +981,40 @@ gv_worker(void *arg)
  
  		/* ... then do I/O processing. */
  		mtx_lock(&sc->bqueue_mtx);
 -		bp = bioq_takefirst(sc->bqueue);
 +		/* First do new requests. */
 +		bp = bioq_takefirst(sc->bqueue_down);
 +		if (bp != NULL) {
 +			mtx_unlock(&sc->bqueue_mtx);
 +			/* A bio that interfered with another bio. */
 +			if (bp->bio_pflags & GV_BIO_ONHOLD) {
 +				s = bp->bio_caller1;
 +				p = s->plex_sc;
 +				/* Is it still locked out? */
 +				if (gv_stripe_active(p, bp)) {
 +					/* Park the bio on the waiting queue. */
 +					bioq_disksort(p->wqueue, bp);
 +				} else {
 +					bp->bio_pflags &= ~GV_BIO_ONHOLD;
 +					g_io_request(bp, s->drive_sc->consumer);
 +				}
 +			/* A special request requireing special handling. */
 +			} else if (bp->bio_pflags & GV_BIO_INTERNAL) {
 +				p = bp->bio_caller1;
 +				gv_plex_start(p, bp);
 +			} else {
 +				gv_volume_start(sc, bp);
 +			}
 +			mtx_lock(&sc->bqueue_mtx);
 +		}
 +		/* Then do completed requests. */
 +		bp = bioq_takefirst(sc->bqueue_up);
  		if (bp == NULL) {
  			msleep(sc, &sc->bqueue_mtx, PRIBIO, "-", hz/10);
  			mtx_unlock(&sc->bqueue_mtx);
  			continue;
  		}
  		mtx_unlock(&sc->bqueue_mtx);
 -
 -		/* A bio that is coming up from an underlying device. */
 -		if (bp->bio_cflags & GV_BIO_DONE) {
 -			gv_bio_done(sc, bp);
 -		/* A bio that interfered with another bio. */
 -		} else if (bp->bio_cflags & GV_BIO_ONHOLD) {
 -			s = bp->bio_caller1;
 -			p = s->plex_sc;
 -			/* Is it still locked out? */
 -			if (gv_stripe_active(p, bp)) {
 -				/* Park the bio on the waiting queue. */
 -				bioq_disksort(p->wqueue, bp);
 -			} else {
 -				bp->bio_cflags &= ~GV_BIO_ONHOLD;
 -				g_io_request(bp, s->drive_sc->consumer);
 -			}
 -		/* A special request requireing special handling. */
 -		} else if (bp->bio_cflags & GV_BIO_INTERNAL ||
 -		    bp->bio_pflags & GV_BIO_INTERNAL) {
 -			p = bp->bio_caller1;
 -			gv_plex_start(p, bp);
 -		/* A fresh bio, scheduled it down. */
 -		} else {
 -			gv_volume_start(sc, bp);
 -		}
 +		gv_bio_done(sc, bp);
  	}
  }
  
 
 Modified: head/sys/geom/vinum/geom_vinum.h
 ==============================================================================
 --- head/sys/geom/vinum/geom_vinum.h	Wed May  6 19:18:19 2009	(r191855)
 +++ head/sys/geom/vinum/geom_vinum.h	Wed May  6 19:34:32 2009	(r191856)
 @@ -127,7 +127,6 @@ void	gv_remove_event(struct gv_softc *, 
  void	gv_drive_tasted(struct gv_softc *, struct g_provider *);
  void	gv_drive_lost(struct gv_softc *, struct gv_drive *);
  void	gv_setup_objects(struct gv_softc *);
 -void	gv_post_bio(struct gv_softc *, struct bio *);
  void	gv_start(struct bio *);
  int	gv_access(struct g_provider *, int, int, int);
  void	gv_cleanup(struct gv_softc *);
 
 Modified: head/sys/geom/vinum/geom_vinum_plex.c
 ==============================================================================
 --- head/sys/geom/vinum/geom_vinum_plex.c	Wed May  6 19:18:19 2009	(r191855)
 +++ head/sys/geom/vinum/geom_vinum_plex.c	Wed May  6 19:34:32 2009	(r191856)
 @@ -48,6 +48,8 @@ static int	gv_plex_offset(struct gv_plex
  		    int *, int);
  static int 	gv_plex_normal_request(struct gv_plex *, struct bio *, off_t,
  		    off_t,  caddr_t);
 +static void	gv_post_bio(struct gv_softc *, struct bio *);
 +
  void
  gv_plex_start(struct gv_plex *p, struct bio *bp)
  {
 @@ -111,7 +113,7 @@ gv_plex_start(struct gv_plex *p, struct 
  		 */
  		if (cbp->bio_caller2 != NULL && gv_stripe_active(p, cbp)) {
  			/* Park the bio on the waiting queue. */
 -			cbp->bio_cflags |= GV_BIO_ONHOLD;
 +			cbp->bio_pflags |= GV_BIO_ONHOLD;
  			bioq_disksort(p->wqueue, cbp);
  		} else {
  			s = cbp->bio_caller1;
 @@ -209,7 +211,7 @@ gv_plex_normal_request(struct gv_plex *p
  		goto bad;
  
  	err = gv_plex_offset(p, boff, bcount, &real_off,
 -	    &real_len, &sdno, (bp->bio_pflags & GV_BIO_SYNCREQ));
 +	    &real_len, &sdno, (bp->bio_pflags & GV_BIO_GROW));
  	/* If the request was blocked, put it into wait. */
  	if (err == GV_ERR_ISBUSY) {
  		bioq_disksort(p->rqueue, bp);
 @@ -239,12 +241,12 @@ gv_plex_normal_request(struct gv_plex *p
  		/* If the subdisk is up, just continue. */
  		break;
  	case GV_SD_DOWN:
 -		if (bp->bio_cflags & GV_BIO_INTERNAL)
 +		if (bp->bio_pflags & GV_BIO_INTERNAL)
  			G_VINUM_DEBUG(0, "subdisk must be in the stale state in"
  			    " order to perform administrative requests");
  		goto bad;
  	case GV_SD_STALE:
 -		if (!(bp->bio_cflags & GV_BIO_SYNCREQ)) {
 +		if (!(bp->bio_pflags & GV_BIO_SYNCREQ)) {
  			G_VINUM_DEBUG(0, "subdisk stale, unable to perform "
  			    "regular requests");
  			goto bad;
 @@ -273,8 +275,6 @@ gv_plex_normal_request(struct gv_plex *p
  	cbp->bio_data = addr;
  	cbp->bio_done = gv_done;
  	cbp->bio_caller1 = s;
 -	if ((bp->bio_cflags & GV_BIO_SYNCREQ))
 -		cbp->bio_cflags |= GV_BIO_SYNCREQ;
  
  	/* Store the sub-requests now and let others issue them. */
  	bioq_insert_tail(p->bqueue, cbp); 
 @@ -282,8 +282,8 @@ gv_plex_normal_request(struct gv_plex *p
  bad:
  	G_VINUM_LOGREQ(0, bp, "plex request failed.");
  	/* Building the sub-request failed. If internal BIO, do not deliver. */
 -	if (bp->bio_cflags & GV_BIO_INTERNAL) {
 -		if (bp->bio_cflags & GV_BIO_MALLOC)
 +	if (bp->bio_pflags & GV_BIO_INTERNAL) {
 +		if (bp->bio_pflags & GV_BIO_MALLOC)
  			g_free(bp->bio_data);
  		g_destroy_bio(bp);
  		p->flags &= ~(GV_PLEX_SYNCING | GV_PLEX_REBUILDING |
 @@ -311,9 +311,9 @@ gv_plex_normal_done(struct gv_plex *p, s
  		/* Just set it to length since multiple plexes will
  		 * screw things up. */
  		pbp->bio_completed = pbp->bio_length;
 -		if (pbp->bio_cflags & GV_BIO_SYNCREQ)
 +		if (pbp->bio_pflags & GV_BIO_SYNCREQ)
  			gv_sync_complete(p, pbp);
 -		else if (pbp->bio_pflags & GV_BIO_SYNCREQ)
 +		else if (pbp->bio_pflags & GV_BIO_GROW)
  			gv_grow_complete(p, pbp);
  		else
  			g_io_deliver(pbp, pbp->bio_error);
 @@ -392,7 +392,7 @@ gv_plex_raid5_done(struct gv_plex *p, st
  
  		/* Handle parity data. */
  		if (TAILQ_EMPTY(&wp->bits)) {
 -			if (bp->bio_parent->bio_cflags & GV_BIO_CHECK)
 +			if (bp->bio_parent->bio_pflags & GV_BIO_CHECK)
  				i = gv_check_parity(p, bp, wp);
  			else
  				i = gv_normal_parity(p, bp, wp);
 @@ -424,16 +424,16 @@ gv_plex_raid5_done(struct gv_plex *p, st
  	if (pbp->bio_inbed == pbp->bio_children) {
  		/* Hand it over for checking or delivery. */
  		if (pbp->bio_cmd == BIO_WRITE &&
 -		    (pbp->bio_cflags & GV_BIO_CHECK)) {
 +		    (pbp->bio_pflags & GV_BIO_CHECK)) {
  			gv_parity_complete(p, pbp);
  		} else if (pbp->bio_cmd == BIO_WRITE &&
 -		    (pbp->bio_cflags & GV_BIO_REBUILD)) {
 +		    (pbp->bio_pflags & GV_BIO_REBUILD)) {
  			gv_rebuild_complete(p, pbp);
 -		} else if (pbp->bio_cflags & GV_BIO_INIT) {
 +		} else if (pbp->bio_pflags & GV_BIO_INIT) {
  			gv_init_complete(p, pbp);
 -		} else if (pbp->bio_cflags & GV_BIO_SYNCREQ) {
 -			gv_sync_complete(p, pbp);
  		} else if (pbp->bio_pflags & GV_BIO_SYNCREQ) {
 +			gv_sync_complete(p, pbp);
 +		} else if (pbp->bio_pflags & GV_BIO_GROW) {
  			gv_grow_complete(p, pbp);
  		} else {
  			g_io_deliver(pbp, pbp->bio_error);
 @@ -480,7 +480,7 @@ gv_check_parity(struct gv_plex *p, struc
  			bp->bio_parent->bio_error = EAGAIN;
  
  			/* ... but we rebuild it. */
 -			if (bp->bio_parent->bio_cflags & GV_BIO_PARITY) {
 +			if (bp->bio_parent->bio_pflags & GV_BIO_PARITY) {
  				s = pbp->bio_caller1;
  				g_io_request(pbp, s->drive_sc->consumer);
  				finished = 0;
 @@ -546,6 +546,18 @@ gv_plex_flush(struct gv_plex *p)
  	}
  }
  
 +static void
 +gv_post_bio(struct gv_softc *sc, struct bio *bp)
 +{
 +
 +	KASSERT(sc != NULL, ("NULL sc"));
 +	KASSERT(bp != NULL, ("NULL bp"));
 +	mtx_lock(&sc->bqueue_mtx);
 +	bioq_disksort(sc->bqueue_down, bp);
 +	wakeup(sc);
 +	mtx_unlock(&sc->bqueue_mtx);
 +}
 +
  int
  gv_sync_request(struct gv_plex *from, struct gv_plex *to, off_t offset,
      off_t length, int type, caddr_t data)
 @@ -566,14 +578,14 @@ gv_sync_request(struct gv_plex *from, st
  	}
  	bp->bio_length = length;
  	bp->bio_done = gv_done;
 -	bp->bio_cflags |= GV_BIO_SYNCREQ;
 +	bp->bio_pflags |= GV_BIO_SYNCREQ;
  	bp->bio_offset = offset;
  	bp->bio_caller1 = from;		
  	bp->bio_caller2 = to;
  	bp->bio_cmd = type;
  	if (data == NULL)
  		data = g_malloc(length, M_WAITOK);
 -	bp->bio_cflags |= GV_BIO_MALLOC; /* Free on the next run. */
 +	bp->bio_pflags |= GV_BIO_MALLOC; /* Free on the next run. */
  	bp->bio_data = data;
  
  	/* Send down next. */
 @@ -613,7 +625,7 @@ gv_sync_complete(struct gv_plex *to, str
  	    	    BIO_WRITE, bp->bio_data);
  	/* If it was a write, read the next one. */
  	} else if (bp->bio_cmd == BIO_WRITE) {
 -		if (bp->bio_cflags & GV_BIO_MALLOC)
 +		if (bp->bio_pflags & GV_BIO_MALLOC)
  			g_free(bp->bio_data);
  		to->synced += bp->bio_length;
  		/* If we're finished, clean up. */
 @@ -684,10 +696,10 @@ gv_grow_request(struct gv_plex *p, off_t
  	bp->bio_caller1 = p;
  	bp->bio_offset = offset;
  	bp->bio_length = length;
 -	bp->bio_pflags |= GV_BIO_SYNCREQ; /* XXX: misuse of pflags AND syncreq.*/
 +	bp->bio_pflags |= GV_BIO_GROW;
  	if (data == NULL)
  		data = g_malloc(length, M_WAITOK);
 -	bp->bio_cflags |= GV_BIO_MALLOC;
 +	bp->bio_pflags |= GV_BIO_MALLOC;
  	bp->bio_data = data;
  
  	gv_post_bio(sc, bp);
 @@ -720,7 +732,7 @@ gv_grow_complete(struct gv_plex *p, stru
  		    BIO_WRITE, bp->bio_data);
  	/* If it was a write, read next. */
  	} else if (bp->bio_cmd == BIO_WRITE) {
 -		if (bp->bio_cflags & GV_BIO_MALLOC)
 +		if (bp->bio_pflags & GV_BIO_MALLOC)
  			g_free(bp->bio_data);
  
  		/* Find the real size of the plex. */
 @@ -790,7 +802,7 @@ gv_init_request(struct gv_sd *s, off_t s
  	bp->bio_done = gv_done;
  	bp->bio_error = 0;
  	bp->bio_length = length;
 -	bp->bio_cflags |= GV_BIO_INIT;
 +	bp->bio_pflags |= GV_BIO_INIT;
  	bp->bio_offset = start;
  	bp->bio_caller1 = s;
  
 @@ -908,8 +920,8 @@ gv_parity_request(struct gv_plex *p, int
  		return;
  	}
  
 -	bp->bio_cflags = flags;
 -	bp->bio_cflags |= GV_BIO_MALLOC;
 +	bp->bio_pflags = flags;
 +	bp->bio_pflags |= GV_BIO_MALLOC;
  
  	/* We still have more parity to build. */
  	bp->bio_offset = offset;
 @@ -927,14 +939,14 @@ gv_parity_complete(struct gv_plex *p, st
  	int error, flags;
  
  	error = bp->bio_error;
 -	flags = bp->bio_cflags;
 +	flags = bp->bio_pflags;
  	flags &= ~GV_BIO_MALLOC;
  
  	sc = p->vinumconf;
  	KASSERT(sc != NULL, ("gv_parity_complete: NULL sc"));
  
  	/* Clean up what we allocated. */
 -	if (bp->bio_cflags & GV_BIO_MALLOC)
 +	if (bp->bio_pflags & GV_BIO_MALLOC)
  		g_free(bp->bio_data);
  	g_destroy_bio(bp);
  
 @@ -986,14 +998,14 @@ gv_rebuild_complete(struct gv_plex *p, s
  	off_t offset;
  
  	error = bp->bio_error;
 -	flags = bp->bio_cflags;
 +	flags = bp->bio_pflags;
  	offset = bp->bio_offset;
  	flags &= ~GV_BIO_MALLOC;
  	sc = p->vinumconf;
  	KASSERT(sc != NULL, ("gv_rebuild_complete: NULL sc"));
  
  	/* Clean up what we allocated. */
 -	if (bp->bio_cflags & GV_BIO_MALLOC)
 +	if (bp->bio_pflags & GV_BIO_MALLOC)
  		g_free(bp->bio_data);
  	g_destroy_bio(bp);
  
 
 Modified: head/sys/geom/vinum/geom_vinum_raid5.c
 ==============================================================================
 --- head/sys/geom/vinum/geom_vinum_raid5.c	Wed May  6 19:18:19 2009	(r191855)
 +++ head/sys/geom/vinum/geom_vinum_raid5.c	Wed May  6 19:34:32 2009	(r191856)
 @@ -65,9 +65,9 @@ gv_raid5_start(struct gv_plex *p, struct
  	wp->parity = NULL;
  	TAILQ_INIT(&wp->bits);
  
 -	if (bp->bio_cflags & GV_BIO_REBUILD)
 +	if (bp->bio_pflags & GV_BIO_REBUILD)
  		err = gv_raid5_rebuild(p, wp, bp, addr, boff, bcount);
 -	else if (bp->bio_cflags & GV_BIO_CHECK)
 +	else if (bp->bio_pflags & GV_BIO_CHECK)
  		err = gv_raid5_check(p, wp, bp, addr, boff, bcount);
  	else
  		err = gv_raid5_request(p, wp, bp, addr, boff, bcount, &delay);
 @@ -120,8 +120,8 @@ gv_raid5_start(struct gv_plex *p, struct
  		}
  
  		/* If internal, stop and reset state. */
 -		if (bp->bio_cflags & GV_BIO_INTERNAL) {
 -			if (bp->bio_cflags & GV_BIO_MALLOC)
 +		if (bp->bio_pflags & GV_BIO_INTERNAL) {
 +			if (bp->bio_pflags & GV_BIO_MALLOC)
  				g_free(bp->bio_data);
  			g_destroy_bio(bp);
  			/* Reset flags. */
 @@ -277,7 +277,7 @@ gv_raid5_rebuild(struct gv_plex *p, stru
  		return (EINVAL);
  
  	case GV_SD_STALE:
 -		if (!(bp->bio_cflags & GV_BIO_REBUILD))
 +		if (!(bp->bio_pflags & GV_BIO_REBUILD))
  			return (ENXIO);
  
  		G_VINUM_DEBUG(1, "sd %s is reviving", broken->name);
 @@ -326,7 +326,6 @@ gv_raid5_rebuild(struct gv_plex *p, stru
  	cbp = gv_raid5_clone_bio(bp, broken, wp, NULL, 1);
  	if (cbp == NULL)
  		return (ENOMEM);
 -	cbp->bio_cflags |= GV_BIO_REBUILD;
  	wp->parity = cbp;
  
  	p->synced = boff;
 @@ -400,7 +399,7 @@ gv_raid5_request(struct gv_plex *p, stru
  
  	/* If synchronizing request, just write it if disks are stale. */
  	if (original->state == GV_SD_STALE && parity->state == GV_SD_STALE &&
 -	    bp->bio_cflags & GV_BIO_SYNCREQ && bp->bio_cmd == BIO_WRITE) {
 +	    bp->bio_pflags & GV_BIO_SYNCREQ && bp->bio_cmd == BIO_WRITE) {
  		type = REQ_TYPE_NORMAL;
  	/* Our parity stripe is missing. */
  	} else if (parity->state != GV_SD_UP) {
 
 Modified: head/sys/geom/vinum/geom_vinum_var.h
 ==============================================================================
 --- head/sys/geom/vinum/geom_vinum_var.h	Wed May  6 19:18:19 2009	(r191855)
 +++ head/sys/geom/vinum/geom_vinum_var.h	Wed May  6 19:34:32 2009	(r191856)
 @@ -108,7 +108,7 @@
  #define	GV_DFLT_SYNCSIZE	65536
  
  /* Flags for BIOs, as they are processed within vinum. */
 -#define	GV_BIO_DONE	0x01
 +#define	GV_BIO_GROW	0x01
  #define	GV_BIO_MALLOC	0x02
  #define	GV_BIO_ONHOLD	0x04
  #define	GV_BIO_SYNCREQ	0x08
 @@ -117,7 +117,7 @@
  #define	GV_BIO_CHECK	0x40
  #define	GV_BIO_PARITY	0x80
  #define GV_BIO_INTERNAL \
 -    (GV_BIO_SYNCREQ | GV_BIO_INIT | GV_BIO_REBUILD |GV_BIO_CHECK)
 +    (GV_BIO_SYNCREQ | GV_BIO_INIT | GV_BIO_REBUILD | GV_BIO_CHECK | GV_BIO_GROW)
  
  /* Error codes to be used within gvinum. */
  #define	GV_ERR_SETSTATE		(-1)	/* Error setting state. */
 @@ -233,7 +233,10 @@ struct gv_softc {
  	struct mtx		equeue_mtx;	/* Event queue lock. */
  	struct mtx		bqueue_mtx;	/* BIO queue lock. */
  	struct mtx		config_mtx;	/* Configuration lock. */
 -	struct bio_queue_head	*bqueue;	/* BIO queue. */
 +	struct bio_queue_head	*bqueue_down;	/* BIO queue incoming
 +						   requests. */
 +	struct bio_queue_head	*bqueue_up;	/* BIO queue for completed
 +						   requests. */
  	struct g_geom		*geom;		/* Pointer to our VINUM geom. */
  };
  #endif
 _______________________________________________
 svn-src-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/svn-src-all
 To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
 
State-Changed-From-To: open->closed 
State-Changed-By: lulf 
State-Changed-When: Wed May 6 19:45:37 UTC 2009 
State-Changed-Why:  
- A fix was committed. A standalone fix will probably not be committed to -STABLE until a possible MFC of gvinum from HEAD. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=133604 
>Unformatted:
