From nobody@FreeBSD.org  Thu Feb  2 04:19:21 2006
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125])
	by hub.freebsd.org (Postfix) with ESMTP id F191E16A420
	for <freebsd-gnats-submit@FreeBSD.org>; Thu,  2 Feb 2006 04:19:21 +0000 (GMT)
	(envelope-from nobody@FreeBSD.org)
Received: from www.freebsd.org (www.freebsd.org [216.136.204.117])
	by mx1.FreeBSD.org (Postfix) with ESMTP id 7607843D53
	for <freebsd-gnats-submit@FreeBSD.org>; Thu,  2 Feb 2006 04:19:21 +0000 (GMT)
	(envelope-from nobody@FreeBSD.org)
Received: from www.freebsd.org (localhost [127.0.0.1])
	by www.freebsd.org (8.13.1/8.13.1) with ESMTP id k124JLGd069217
	for <freebsd-gnats-submit@FreeBSD.org>; Thu, 2 Feb 2006 04:19:21 GMT
	(envelope-from nobody@www.freebsd.org)
Received: (from nobody@localhost)
	by www.freebsd.org (8.13.1/8.13.1/Submit) id k124JLoF069216;
	Thu, 2 Feb 2006 04:19:21 GMT
	(envelope-from nobody)
Message-Id: <200602020419.k124JLoF069216@www.freebsd.org>
Date: Thu, 2 Feb 2006 04:19:21 GMT
From: Erik Nygren <bsd@nygrens.info>
To: freebsd-gnats-submit@FreeBSD.org
Subject: hifn driver hangs after a short while when using GEOM_ELI
X-Send-Pr-Version: www-2.3

>Number:         92716
>Category:       kern
>Synopsis:       [hifn] [hang] hifn driver hangs after a short while when using GEOM_ELI
>Confidential:   no
>Severity:       serious
>Priority:       high
>Responsible:    freebsd-bugs
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Thu Feb 02 04:20:00 GMT 2006
>Closed-Date:    
>Last-Modified:  Wed Feb 25 13:30:02 UTC 2009
>Originator:     Erik Nygren
>Release:        6.0-RELEASE-p4
>Organization:
Uni2
>Environment:
FreeBSD farmor.nygrens.info 6.0-RELEASE-p4 FreeBSD 6.0-RELEASE-p4 #278: Fri Jan 27 08:53:01 CET 2006     root@farmor.nygrens.info:/usr/obj/usr/src/sys/FARMOR  i386
>Description:
When using geli together with a Soekris VPN1401 (hifn 7955), disk access
freezes  after a while. 

I have tried this in a few different computers, and they all behave the same.
Everything works fine for a while, and then the diskwrites just come to a halt.
Only way out is reboot, which will then fail to write out a few buffers to disk.
>How-To-Repeat:
geli init -s 4096 ad2s1a
geli attach ad2s1a
newfs -n -U -m 6 ad2s1a.eli
mount /dev/ad2s1a.eli /mnt
while true 
do
  time dd if=/dev/random of=/mnt/testfile bs=1M count=50
done
# Now just wait for a few minutes, possibly doing "iostat 1" in another window.
>Fix:
This issue (I think) have been seen in the hifn driver, and possibly even
been solved in the OpenBSD driver. However, trying to apply the OpenBSD
fixes to the FreeBSD driver does not work for me.

Google turns up a few hits like this one:
http://news.vejas.lt/article.php?id=15833&group=gmane.os.netbsd.bugs
that I think are related. But unfortunately the suggested patch does not
solve the problem for me.
>Release-Note:
>Audit-Trail:

From: Patrick Lamaiziere <patfbsd@davenulle.org>
To: bug-followup@FreeBSD.org, bsd@nygrens.info
Cc:  
Subject: Re: kern/92716: [hifn] [hang] hifn driver hangs after a short while
 when using GEOM_ELI
Date: Fri, 9 Jan 2009 19:05:44 +0100

 I'm not able to reproduce this problem on FreeBSD 8-Current/i386
 
 Tested on a Soekris Net5501 box with a vpn1411 (hifn 7955) mini-pci
 card. That runs two days without any problem here.
 
 $ uname -a
 FreeBSD malpractice.lamaiziere.net 8.0-CURRENT FreeBSD 8.0-CURRENT #5:
 Mon Dec 29 21:20:22 CET 2008 root@:/usr/obj/usr/src/sys/NET5501 i386
 
 May be we should close this PR?
 Regards.

From: Oliver Peter <lists@peter.de.com>
To: bug-followup@FreeBSD.org, bsd@nygrens.info
Cc:  
Subject: Re: kern/92716: [hifn] [hang] hifn driver hangs after a short while
 when using GEOM_ELI
Date: Tue, 24 Feb 2009 14:15:21 +0000

 I'm having exactly the same problem under a
 relatively fresh 8.0/amd64:
 
 # uname -a
 FreeBSD labs.foonic.org 8.0-CURRENT FreeBSD 8.0-CURRENT #0: Fri Feb 20 17:53:44 CET 2009     root@labs.foonic.org:/usr/obj/usr/src/sys/LABS  amd64
 
 # dmesg | grep hifn
 hifn0 mem 0xfebff000-0xfebfffff,0xfebfc000-0xfebfdfff,0xfebf0000-0xfebf7fff irq 17 at device 9.0 on pci0
 hifn0: [ITHREAD]
 hifn0: Hifn 7955, rev 0, 32KB dram, pll=0x801<ext clk, 4x mult>
 
 # kldstat
 Id Refs Address            Size     Name
  1   22 0xffffffff80100000 779f28   kernel
  2    1 0xffffffff8087a000 18d540   zfs.ko
  3    2 0xffffffff80a08000 35b8     opensolaris.ko
  4    1 0xffffffff80a0c000 18bb8    geom_eli.ko
  5    4 0xffffffff80a25000 2a348    crypto.ko
  6    3 0xffffffff80a50000 d908     zlib.ko
  7    1 0xffffffff80a5e000 beb8     hifn.ko
  8    1 0xffffffff80a6a000 43f8     cryptodev.ko
  9    1 0xffffffff80c22000 1c9e     nullfs.ko
 10    1 0xffffffff80c24000 16f4     fdescfs.ko
 
 dd hangs up within 5 minutes, I have to reboot
 the machine after that:
 
 # dd if=/dev/zero of=/dev/ad1.eli bs=512k
 
 

From: Patrick Lamaiziere <patfbsd@davenulle.org>
To: Oliver Peter <lists@peter.de.com>
Cc: bug-followup@FreeBSD.org
Subject: Re: kern/92716: [hifn] [hang] hifn driver hangs after a short while
 when using GEOM_ELI
Date: Tue, 24 Feb 2009 19:46:09 +0100

 Le Tue, 24 Feb 2009 14:40:02 GMT,
 Oliver Peter <lists@peter.de.com>:
 
 > Subject: Re: kern/92716: [hifn] [hang] hifn driver hangs after a
 > short while when using GEOM_ELI
 > Date: Tue, 24 Feb 2009 14:15:21 +0000
 > 
 >  I'm having exactly the same problem under a
 >  relatively fresh 8.0/amd64:
 
 Oliver, please could you try the patch in kern/130286 ?
 http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/130286
 
 There is a problem in the locking of the sessions in hifn(4), that
 could be the problem (not sure). But I'm not able to reproduce this
 problem with geli on my hardware (a Soekris net5501/i386 - VPN1411)
 
 You can build hifn with the HIFN_DEBUG option, it may help.

From: Oliver Peter <lists@peter.de.com>
To: Patrick Lamaiziere <patfbsd@davenulle.org>
Cc: bug-followup@FreeBSD.org
Subject: Re: kern/92716: [hifn] [hang] hifn driver hangs after a short
	while when using GEOM_ELI
Date: Wed, 25 Feb 2009 10:53:48 +0000

 On Tue, Feb 24, 2009 at 07:46:09PM +0100, Patrick Lamaiziere wrote:
 > Le Tue, 24 Feb 2009 14:40:02 GMT,
 > Oliver Peter <lists@peter.de.com>:
 > 
 > > Subject: Re: kern/92716: [hifn] [hang] hifn driver hangs after a
 > > short while when using GEOM_ELI
 > > Date: Tue, 24 Feb 2009 14:15:21 +0000
 > > 
 > >  I'm having exactly the same problem under a
 > >  relatively fresh 8.0/amd64:
 > 
 > Oliver, please could you try the patch in kern/130286 ?
 > http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/130286
 >
 > There is a problem in the locking of the sessions in hifn(4), that
 > could be the problem (not sure). But I'm not able to reproduce this
 > problem with geli on my hardware (a Soekris net5501/i386 - VPN1411)
 > 
 > You can build hifn with the HIFN_DEBUG option, it may help.
 
 What I did:
      - applied your patch to my tree
      - switched back to GENERIC (including WITNESS option)
      - enabled HIFN_DEBUG in my config
 
 But no change at all.  The dd command still hangs up after
 a couple of minutes and the only way out is to reboot.
 
 Also I don't see any debug output from the hifn driver. 
 
 Now I'm on
 FreeBSD labs.foonic.org 8.0-CURRENT FreeBSD 8.0-CURRENT #0: Wed Feb 25 11:15:04 CET 2009     root@labs.foonic.org:/usr/obj/usr/src/sys/GENERIC  amd64
 
 The sourcetree is from 2009-02-20.
 
 -- 
 Oliver PETER     email: oliver@peter.de.com     ICQ# 113969174
 "I'm just a simple man trying to make my way in the universe."
                                                 -- Jango Fett

From: "Patrick Lamaiziere" <patfbsd@davenulle.org>
To: "Oliver Peter" <lists@peter.de.com>
Cc: bug-followup@freebsd.org
Subject: Re: kern/92716: [hifn] [hang] hifn driver hangs after a short 
     while when using GEOM_ELI
Date: Wed, 25 Feb 2009 14:27:28 +0100 (CET)

 >> Oliver, please could you try the patch in kern/130286 ?
 >> http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/130286
 >>
 >> There is a problem in the locking of the sessions in hifn(4), that
 >> could be the problem (not sure). But I'm not able to reproduce this
 >> problem with geli on my hardware (a Soekris net5501/i386 - VPN1411)
 >>
 >> You can build hifn with the HIFN_DEBUG option, it may help.
 >
 > What I did:
 >      - applied your patch to my tree
 >      - switched back to GENERIC (including WITNESS option)
 >      - enabled HIFN_DEBUG in my config
 >
 > But no change at all.  The dd command still hangs up after
 > a couple of minutes and the only way out is to reboot.
 
 So I don't know, sorry
 
 > Also I don't see any debug output from the hifn driver.
 
 You have to set the sysctl hw.hifn.debug to 1
 
 There is an OpenBSD PR that could be related, NetBSD also has this change.
 
 http://cvs.openbsd.org/cgi-bin/query-pr-wrapper?full=yes&numbers=3739
 
 If hifn misses an interrupt, geli will hang.
 
 You can test it
 In hifn_crypto() at ligne 2053
 http://fxr.watson.org/fxr/source/dev/hifn/hifn7751.c#L2053
 
 -	if (dma->cmdu > 1) {
 -		sc->sc_dmaier |= HIFN_DMAIER_C_WAIT;
 -		WRITE_REG_1(sc, HIFN_1_DMA_IER, sc->sc_dmaier);
 -	}
 +	sc->sc_dmaier |= HIFN_DMAIER_C_WAIT;
 +	WRITE_REG_1(sc, HIFN_1_DMA_IER, sc->sc_dmaier);
 
 Regards.
 
>Unformatted:
