From inyaoo@gmail.com  Thu Apr 14 02:02:22 2011
Return-Path: <inyaoo@gmail.com>
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id B3594106564A
	for <FreeBSD-gnats-submit@freebsd.org>; Thu, 14 Apr 2011 02:02:22 +0000 (UTC)
	(envelope-from inyaoo@gmail.com)
Received: from mail-fx0-f54.google.com (mail-fx0-f54.google.com [209.85.161.54])
	by mx1.freebsd.org (Postfix) with ESMTP id 458358FC13
	for <FreeBSD-gnats-submit@freebsd.org>; Thu, 14 Apr 2011 02:02:21 +0000 (UTC)
Received: by fxm11 with SMTP id 11so1214607fxm.13
        for <FreeBSD-gnats-submit@freebsd.org>; Wed, 13 Apr 2011 19:02:21 -0700 (PDT)
Received: by 10.223.1.76 with SMTP id 12mr175366fae.118.1302746541095;
        Wed, 13 Apr 2011 19:02:21 -0700 (PDT)
Received: from localhost (server106963.santrex.net [212.117.176.90])
        by mx.google.com with ESMTPS id n2sm344978fam.4.2011.04.13.19.02.17
        (version=SSLv3 cipher=OTHER);
        Wed, 13 Apr 2011 19:02:19 -0700 (PDT)
Message-Id: <86oc49bnff.fsf@gmail.com>
Date: Thu, 14 Apr 2011 06:02:12 +0400
From: Pan Tsu <inyaoo@gmail.com>
To: FreeBSD-gnats-submit@freebsd.org
Subject: [patch] x11/nvidia-driver: EnableMSI doesn't work

>Number:         156386
>Category:       ports
>Synopsis:       [patch] x11/nvidia-driver: EnableMSI doesn't work
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    danfe
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Thu Apr 14 02:10:07 UTC 2011
>Closed-Date:    Fri Nov 11 15:49:58 UTC 2011
>Last-Modified:  Fri Nov 11 15:49:58 UTC 2011
>Originator:     Pan Tsu
>Release:        FreeBSD 9.0-CURRENT amd64
>Organization:
>Environment:
$ sysctl -n hw.nvidia.registry.EnableMSI
1

$ sudo pciconf -lvcb | sed -n '/vgapci/,/@pci/p' | sed '$d'
vgapci0@pci0:1:0:0:     class=0x030000 card=0x34811458 chip=0x062210de rev=0xa1 hdr=0x00
    vendor     = 'NVIDIA Corporation'
    device     = 'NVIDIA GeForce 9600 GT (G94)'
    class      = display
    subclass   = VGA
    bar   [10] = type Memory, range 32, base 0xf6000000, size 16777216, enabled
    bar   [14] = type Prefetchable Memory, range 64, base 0xe0000000, size 268435456, enabled
    bar   [1c] = type Memory, range 64, base 0xf4000000, size 33554432, enabled
    bar   [24] = type I/O Port, range 32, base 0xb000, size 128, enabled
    cap 01[60] = powerspec 3  supports D0 D3  current D0
    cap 05[68] = MSI supports 1 message, 64 bit
    cap 10[78] = PCI-Express 2 endpoint max data 128(128) link x16(x16)
    cap 09[b4] = vendor (length 20)
ecap 0002[100] = VC 1 max VC0
ecap 0004[128] = unknown 1
ecap 000b[600] = unknown 1
>Description:
Since 256.29 nvidia driver allows setting EnableMSI via sysctl which was
long available on linux before. However, sysctls are initialized at the
last stage of module loading when the device is already attached.

Linux uses NVreg_EnableMSI as a module parameter via modprobe(8) but
it doesn't seem to work on FreeBSD when set via kenv(1) or loader(8).
>How-To-Repeat:
1. load nvidia.ko
2. set hw.nvidia.registry.EnableMSI via sysctl
3. start X11 # ???
4. check pciconf -lc and/or vmstat -i

1. apply the patch
2. kenv hint.nvidia.0.msi=1
3. kldload nvidia
4. check pciconf -lc and/or vmstat -i

`pciconf -lc' diff
  -    cap 05[68] = MSI supports 1 message, 64 bit 
  +    cap 05[68] = MSI supports 1 message, 64 bit enabled with 1 message

`vmstat -i' diff
   interrupt                          total       rate
  -irq16: vgapci0++                    3639         13
   irq18: uhci2 ehci0*                 XXXX          X
   irq20: fxp0                         XXXX          X
   irq21: uhci1                        XXXX          X
   irq22: hpet0                        XXXX          X
   irq23: uhci3 ehci1                  XXXX          X
  +irq256: vgapci0                     5167         20
   irq257: hdac0                       XXXX          X
   irq258: ahci1:ch0                   XXXX          X
   irq259: ahci1:ch1                   XXXX          X
>Fix:
Below workaround allows one to enable MSI via device.hints(5)

  hint.nvidia.%d.msi=1

not sure if it'll work with different settings on each card with
a multicard setup.

--- a.diff begins here ---
Index: x11/nvidia-driver/files/patch-src-nvidia_subr.c
===================================================================
RCS file: x11/nvidia-driver/files/patch-src-nvidia_subr.c
diff -N x11/nvidia-driver/files/patch-src-nvidia_subr.c
--- /dev/null	1 Jan 1970 00:00:00 -0000
+++ x11/nvidia-driver/files/patch-src-nvidia_subr.c	14 Apr 2011 00:47:22 -0000
@@ -0,0 +1,13 @@
+--- src/nvidia_subr.c~
++++ src/nvidia_subr.c
+@@ -252,8 +252,8 @@ int nvidia_alloc_hardware(device_t dev)
+         sc->BAR_recs[i] = res;
+     }
+ 
+-    if ((rm_read_registry_dword(sp, NULL, "NVreg",
+-            "EnableMSI", &enable_msi) == RM_OK) && (enable_msi != 0)) {
++    if ((resource_int_value(device_get_name(dev), device_get_unit(dev),
++            "msi", &enable_msi) == 0) && (enable_msi != 0)) {
+         count = pci_msi_count(dev);
+         if ((count == 1) && (pci_alloc_msi(dev, &count) == 0))
+             sc->irq_rid = 1;
--- a.diff ends here ---
>Release-Note:
>Audit-Trail:
Responsible-Changed-From-To: freebsd-ports-bugs->danfe 
Responsible-Changed-By: edwin 
Responsible-Changed-When: Thu Apr 14 02:10:19 UTC 2011 
Responsible-Changed-Why:  
Over to maintainer (via the GNATS Auto Assign Tool) 

http://www.freebsd.org/cgi/query-pr.cgi?pr=156386 

From: Pan Tsu <inyaoo@gmail.com>
To: FreeBSD-gnats-submit@FreeBSD.org
Cc:  
Subject: Re: ports/156386: [patch] x11/nvidia-driver: EnableMSI doesn't work
Date: Fri, 01 Jul 2011 20:24:45 +0400

 Better to match sysctls which is more intuitive for a FreeBSD user, e.g.
 
   hw.nvidia.registry.EnableAGPFW
   hw.nvidia.registry.EnableAGPSBA
   hw.nvidia.registry.EnableMSI
 
 --- a.diff begins here ---
 Index: x11/nvidia-driver/files/patch-src-nvidia_os_registry.c
 ===================================================================
 RCS file: x11/nvidia-driver/files/patch-src-nvidia_os_registry.c
 diff -N x11/nvidia-driver/files/patch-src-nvidia_os_registry.c
 --- /dev/null	1 Jan 1970 00:00:00 -0000
 +++ x11/nvidia-driver/files/patch-src-nvidia_os_registry.c	1 Jul 2011 16:18:18 -0000
 @@ -0,0 +1,21 @@
 +--- src/nvidia_os_registry.c~
 ++++ src/nvidia_os_registry.c
 +@@ -71,13 +71,17 @@ RM_STATUS os_registry_init(void)
 +     nv_parm_t *entry;
 +     NvU32 i;
 +      nv_stack_t *sp;
 ++    char tn[64];
 + 
 +     NV_UMA_ZONE_ALLOC_STACK(sp);
 +     if (sp == NULL)
 +         return RM_ERR_NO_FREE_MEM;
 + 
 +-    for (i = 0; (entry = &nv_parms[i])->name != NULL; i++)
 ++    for (i = 0; (entry = &nv_parms[i])->name != NULL; i++) {
 ++        snprintf(tn, sizeof(tn), "hw.nvidia.registry.%s", entry->name);
 ++        TUNABLE_INT_FETCH(tn, entry->data);
 +         rm_write_registry_dword(sp, NULL, entry->node, entry->name, *entry->data);
 ++    }
 + 
 +     NV_UMA_ZONE_FREE_STACK(sp);
 + 
 --- a.diff ends here ---

From: Nali Toja <nalitoja@gmail.com>
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: ports/156386: [patch] x11/nvidia-driver: EnableMSI doesn&#39;t work
Date: Tue, 11 Oct 2011 15:22:10 +0000

 The PR is obsolete since 285.05.09 while legacy versions do not provide
 MSI support. From NVIDIA_Changelog:
 
   ======== Release 285 Entries ========
 
       * Implemented support for FreeBSD tunables.
State-Changed-From-To: open->closed 
State-Changed-By: danfe 
State-Changed-When: Fri Nov 11 15:48:34 UTC 2011 
State-Changed-Why:  
Not an issue (obsolete) since NVidia driver version 285.05.09. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=156386 
>Unformatted:
