From leres@ee.lbl.gov  Thu May  4 02:04:16 2006
Return-Path: <leres@ee.lbl.gov>
Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125])
	by hub.freebsd.org (Postfix) with ESMTP id 5BCEF16A400
	for <freebsd-gnats-submit@freebsd.org>; Thu,  4 May 2006 02:04:16 +0000 (UTC)
	(envelope-from leres@ee.lbl.gov)
Received: from fun.ee.lbl.gov (fun.ee.lbl.gov [131.243.1.81])
	by mx1.FreeBSD.org (Postfix) with ESMTP id 2206643D46
	for <freebsd-gnats-submit@freebsd.org>; Thu,  4 May 2006 02:04:16 +0000 (GMT)
	(envelope-from leres@ee.lbl.gov)
Received: from fun.ee.lbl.gov (localhost [127.0.0.1])
	by fun.ee.lbl.gov (8.13.6/8.13.6) with ESMTP id k4424Fkk016556;
	Wed, 3 May 2006 19:04:15 -0700 (PDT)
Received: from fun.ee.lbl.gov (leres@localhost)
	by fun.ee.lbl.gov (8.13.6/8.13.6/Submit) with ESMTP id k4424FP9016553;
	Wed, 3 May 2006 19:04:15 -0700 (PDT)
Message-Id: <200605040204.k4424FP9016553@fun.ee.lbl.gov>
Date: Wed, 03 May 2006 19:04:15 -0700
From: Craig Leres <leres@ee.lbl.gov>
To: freebsd-gnats-submit@freebsd.org
Cc: brad@openbsd.org
Subject: [sk] [patch] broken 32-bit register operations
X-Send-Pr-Version: 3.113
X-GNATS-Notify:

>Number:         96743
>Category:       kern
>Synopsis:       [sk] [patch] broken 32-bit register operations
>Confidential:   no
>Severity:       non-critical
>Priority:       medium
>Responsible:    yongari
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Thu May 04 02:10:19 GMT 2006
>Closed-Date:    Mon Oct 25 21:51:25 UTC 2010
>Last-Modified:  Mon Oct 25 21:51:25 UTC 2010
>Originator:     Craig Leres
>Release:        FreeBSD 6.1-BETA4 i386
>Organization:
Lawrence Berkeley National Laboratory
>Environment:
	fox 15 % uname -a
	FreeBSD fox.ee.lbl.gov 6.1-BETA4 FreeBSD 6.1-BETA4 #29: Wed May  3 17:45:44 PDT 2006     leres@fox.ee.lbl.gov:/usr/src/6.1-BETA4/sys/i386/compile/LBLSMP  i386

	SysKonnect SK-9843 gige adapter

>Description:
	While attempting to apply a local patch that implements
	receive hardware timestamps (nice for research and intrusion
	detection applications) with the syskonnect SK-9843, I
	noticed that some 32-bit register operations are broken.
	Most of the registers, including XM_MODE and XM_TSTAMP_READ,
	must be accessed 16-bits at a time. One step in enabling
	timestamps is to set the XM_MODE_TIMESTAMP bit in the XM_MODE
	register but it was not possible to access the high bytes
	of the register.

	I developed my timestamp patch with 4.8-RELEASE using version
	1.8.2.1 of if_skreg.h which defines SK_XM_READ_4() and
	SK_XM_WRITE_4() macros that use sk_win_read_2() and
	sk_win_read_2() to read and write 16 bits at a time, as
	required by the hardware.

	Version 1.8.2.2 of the include file change the macros to
	use sk_win_read_4() and sk_win_write_4() which only return
	the low order 16-bits.

	This problem probably wasn't detected before now because
	the current driver only seems to ever access bits in the
	low bytes of the registers (e.g. XM_MODE_RX_PROMISC in
	XM_MODE).

>How-To-Repeat:
	This patch helps demonstrate the problem:

==============================================================================

*** if_sk.c.virgin	Wed May  3 17:56:30 2006
--- if_sk.c	Wed May  3 17:56:37 2006
***************
*** 2532,2537 ****
--- 2532,2540 ----
  
  	/* Save the XMAC II revision */
  	sc_if->sk_xmac_rev = XM_XMAC_REV(SK_XM_READ_4(sc_if, XM_DEVID));
+ printf("sk%d: XM_DEVID 0x%x (0x%x/0x%x)\n",
+     sc_if->sk_unit, SK_XM_READ_4(sc_if, XM_DEVID),
+     SK_XM_READ_2(sc_if, XM_DEVID_HI), SK_XM_READ_2(sc_if, XM_DEVID_LO));
  
  	/*
  	 * Perform additional initialization for external PHYs,

==============================================================================

	After applying the patch and building and booting the kernel,
	the driver reports:

	    skc0: <SysKonnect Gigabit Ethernet (V1.0)> port 0x3000-0x30ff mem 0xdd300000-0xd
	    d303fff irq 48 at device 1.0 on pci3
	    skc0: SysKonnect SK-NET Gigabit Ethernet Adapter SK-9843 SX rev. (0x0)
	    sk0: <XaQti Corp. XMAC II> on skc0
	    sk0: Ethernet address: 00:00:5a:9a:80:18
	    sk0: XM_DEVID 0xae20 (0xe0/0xae20)

	Notice that SK_XM_READ_4() returned the same value as the
	low bytes SK_XM_READ_2() and that the high bytes have 0xe0,
	not zero.

>Fix:
	The appended patch (vs. 1.29.2.1 of if_skreg.h) re-enables
	the 2-byte versions if the SK_XM_READ_4() and SK_XM_WRITE_4()
	macros. It also protects the SK_XM_WRITE_4() macro.

	After booting the new kernel, the debugging printout
	shows the complete device id value:

	    sk0: XM_DEVID 0xe0ae20 (0xe0/0xae20)

	It looks like this patch would also apply to the -current
	version (1.35).

==============================================================================

*** if_skreg.h.virgin	Wed May  3 18:28:19 2006
--- if_skreg.h	Wed May  3 18:29:56 2006
***************
*** 1150,1156 ****
  #define SK_XMAC_REG(sc, reg)	(((reg) * 2) + SK_XMAC1_BASE +		\
  	(((sc)->sk_port) * (SK_XMAC2_BASE - SK_XMAC1_BASE)))
  
! #if 0
  #define SK_XM_READ_4(sc, reg)						\
  	((sk_win_read_2(sc->sk_softc,					\
  	SK_XMAC_REG(sc, reg)) & 0xFFFF) |				\
--- 1150,1160 ----
  #define SK_XMAC_REG(sc, reg)	(((reg) * 2) + SK_XMAC1_BASE +		\
  	(((sc)->sk_port) * (SK_XMAC2_BASE - SK_XMAC1_BASE)))
  
! /* 
!  * For at least some registers (e.g. XM_MODE) with at least some
!  * boards (e.g. SK-9843) you can only read/write 16 bits at a time.
!  */
! #if 1
  #define SK_XM_READ_4(sc, reg)						\
  	((sk_win_read_2(sc->sk_softc,					\
  	SK_XMAC_REG(sc, reg)) & 0xFFFF) |				\
***************
*** 1158,1167 ****
  	SK_XMAC_REG(sc, reg + 2)) & 0xFFFF) << 16))
  
  #define SK_XM_WRITE_4(sc, reg, val)					\
! 	sk_win_write_2(sc->sk_softc, SK_XMAC_REG(sc, reg),		\
! 	((val) & 0xFFFF));						\
! 	sk_win_write_2(sc->sk_softc, SK_XMAC_REG(sc, reg + 2),		\
! 	((val) >> 16) & 0xFFFF)
  #else
  #define SK_XM_READ_4(sc, reg)		\
  	sk_win_read_4(sc->sk_softc, SK_XMAC_REG(sc, reg))
--- 1162,1173 ----
  	SK_XMAC_REG(sc, reg + 2)) & 0xFFFF) << 16))
  
  #define SK_XM_WRITE_4(sc, reg, val)					\
! 	do {								\
! 		sk_win_write_2(sc->sk_softc, SK_XMAC_REG(sc, reg),	\
! 		((val) & 0xFFFF));					\
! 		sk_win_write_2(sc->sk_softc, SK_XMAC_REG(sc, reg + 2),	\
! 		((val) >> 16) & 0xFFFF);				\
! 	} while (0)
  #else
  #define SK_XM_READ_4(sc, reg)		\
  	sk_win_read_4(sc->sk_softc, SK_XMAC_REG(sc, reg))
>Release-Note:
>Audit-Trail:
Responsible-Changed-From-To: freebsd-bugs->yongari 
Responsible-Changed-By: yongari 
Responsible-Changed-When: Thu Jul 27 11:03:49 UTC 2006 
Responsible-Changed-Why:  
I'll handle this. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=96743 
State-Changed-From-To: open->feedback 
State-Changed-By: yongari 
State-Changed-When: Thu Jul 27 11:15:26 UTC 2006 
State-Changed-Why:  
XMAC documentation says that NP(node processor) interface supports 
both 16bit/32bit modes. Your results shows it uses 16bit mode only. 
I don't have SysKonnect SK-NET hardware so it's hard to verify this. 
As multicast filter setup uses SK_XM_WRITE_4 macros, I think 
multicasting filtering didn't work on your hardware without using 
two 16bit accesses. 
Would you verify that the hardware really works for multicasting 
packet after applying your patch? 

http://www.freebsd.org/cgi/query-pr.cgi?pr=96743 
State-Changed-From-To: feedback->closed 
State-Changed-By: yongari 
State-Changed-When: Mon Oct 25 21:50:57 UTC 2010 
State-Changed-Why:  
Close(feedback timed out). 
I double checked XaQti XQ11800FP 1000Mbps Gigabit Ethernet 
Controller data sheet and it clearly says NP supports both 16bits 
and 32bits operation. I also checked Linux and Vendor's driver and 
they also used 32bits register access for Genesis controllers. The 
only guess I have at this moment is you may have wrong access 
pattern to the register. To access 32bits quantity with two 16bits 
operation, you may have to access low address first and then high 
address. Just accessing high 16bits of a 32bit register would yield 
incorrect result. 
Please check data sheet to see which register supports both 16bits 
and 32bits and which register supports 16bits only(pp. 40). 

http://www.freebsd.org/cgi/query-pr.cgi?pr=96743 
>Unformatted:
