FreeBSD 13.1 ZFS w/NFS
reddit / freebsd

----------------------------------------------------------------------
MyJewishFuzzyNutz · 2022-07-08 18:20

Hello,

I just cannot get NFSv4 working on FreeBSD 13.1, despite having the
correct entries in /etc/exports and /etc/rc.conf.  Regardless, I have
a 300GB zpool for home running on a 4 drive RAID10 LUN.  The VM is
configured on ESXi 7.0.3U3d with plenty of CPU/RAM.  When I try to do
an NFS4 mount it hangs and says permission denied (-vvv) as root user.

I have done some tweaking by editing
/usr/src/sys/fs/nfsserver/nfs\_dport.c and only leaving the
IO\_NODELOCKED option (line 702 I think...) and have the option
vfs.zfs.cache\_flush\_disable=1 in /boot/loader.conf.

My fio tests on local ZFS give me 1.5GB/sec read and 136MB/sec write,
but on the NFSv3 client mounting the zpool, I get 109MB/sec read and
41.4MB/sec write (sequential with bs=4M and iodepth=256).  When I do
random readwrite fio tests with bs=4k and iodepth=64, I get 10MB/sec
reads and 3.3MB/sec writes.

What am I missing here?  A quick scp from another FreeBSD VM on the
same box/LUN gives me about 60-70MB/sec.

Thanks for the help!

/etc/rc.conf:

    local_unbound_enable="YES"
    sshd_enable="YES"
    ntpd_enable="YES"
    # Set dumpdev to "AUTO" to enable crash dumps, "NO" to disable
    dumpdev="AUTO"
    zfs_enable="YES"
    linux_enable="yes"
    gmond_enable="no"
    zabbix_agentd_enable="no"
    nfs_client_enable="yes"
    winbindd_enable="yes"
    mysql_enable="yes"
    mysql_dbdir="/mysqldb1"
    samba_server_enable="yes"
    nfs_server_enable="yes"
    rpcbind_enable="yes"
    mountd_flags="-r"
    mountd_enable="yes"
    nfs_flags="-t -u -n 32"
    nfsv4_server_enable=yes
    openssh_enable="yes"
    nfsuserd_enable="yes"

change to /usr/src/sys/fs/nfsserver/nfs\_nfsdport.c (the C commented
lines):

    //      if (*stable == NFSWRITE_UNSTABLE)
                    ioflags = IO_NODELOCKED;
    //      else
    //              ioflags = (IO_SYNC | IO_NODELOCKED);

----------------------------------------------------------------------
AimForTheAce · 2022-07-08 18:43

Please post the content of /etc/exports.

----------------------------------------------------------------------
Ikinoki · 2022-07-08 20:41

I'm sorry I didn't quite grasp what runs your NFSv4 server and what is
the client? Second problem is what is in your /etc/exports?

This is barely documented and very horribly programmed but works if
you do exactly as in manual

----------------------------------------------------------------------
jhx0 · 2022-07-09 09:48

The ordering of the services in rc.conf is important for NFS.
Try this in your rc.conf:


rpcbind\_enable="yes"
mountd\_flags="r"
mountd\_enable="yes"
nfsv4\_server\_enable=yes
nfs\_server\_enable="yes"
nfs\_flags="-t -u -n 32"


Otherwise: you need to post the contents of /etc/exports

----------------------------------------------------------------------
okapi-a · 2022-07-10 15:25

With ZFS, you shouldn't need to use `/etc/exports`. Instead, we'd need
to see the output of `zfs get sharenfs dataset`

----------------------------------------------------------------------
jcigar · 2022-07-11 14:01

IMHO it's best to use `/etc/exports`... or is there a way now to share
a dataset readonly to subnet1 and rw to subnet2?

----------------------------------------------------------------------
jcigar · 2022-07-11 14:02

order doesn't matter and rcpbind is not need for v4