[SOLVED] rpc.mountd: Unable to watch /proc/fs/nfsd/clients: No such file or directory

mfoley · 01-21-2024, 04:53 PM

I've just upgraded one of my machines to Slackware 15.0. I export NFS files from this host. The new NFS package is nfs-utils-2.5.4-x86_64-1. The previous Slackware 14.2 version was nfs-utils-1.3.3-x86_64-2.

When I try to run nfsd I get the message in syslog shown in this subject:

Code:

rpc.mountd: Unable to watch /proc/fs/nfsd/clients: No such file or directory

showmount on the server gives:

Code:

# showmount -a
rpc mount dump: RPC: Unable to receive; errno = Connection refused

nfsd, rpc.statd and rpcbind are running, rpc.mountd is not. From another host I cannot nfsmount the exported directories.

In syslog of the client trying to do the mount I get:

Code:

Jan 21 22:21:45 webserver kernel: [    0.374148] Installing knfsd (copyright (C) 1996 okir@monad.swb.de).

How do I fix this? It worked fine with Slackware 14.2.

henca · 01-22-2024, 12:50 AM

What is the permissions of the files /etc/rc.d/rc.nfsd and /etc/rc.d/rc.rpc ? You will need both executed at boot on an NFS server.

regards Henrik

mfoley · 01-22-2024, 01:12 AM

Quote:

Originally Posted by henca

What is the permissions of the files /etc/rc.d/rc.nfsd and /etc/rc.d/rc.rpc ? You will need both executed at boot on an NFS server.

regards Henrik

Code:

-rwxr-xr-x 1 root root 2920 Jun 15  2021 /etc/rc.d/rc.nfsd
-rwxr-xr-- 1 root root 2295 Jan 20 12:37 /etc/rc.d/rc.rpc

As mentioned, nfsd, rpc.statd and rpcbind are running according to ps.

mfoley · 01-22-2024, 11:51 AM

Any thoughts on this? Surely I can't be the only one upgrading to Slackware 15.0 whose had this issue. I've reverted back to nfs-utils-1.3.3-x86_64-2 and that is working. The fstab entry on my client is:

Code:

ohprsstorage:/mnt/RAID/public /mnt/public  nfs  nfsvers=3,rw,_netdev    0 0

The client is still running Slackware 14.2

I'm going to leave this open in the hopes that some Slackware guru can tell me why the nfs-utils-2.5.4-x86_64-1 package doesn't work.

Petri Kaukasoina · 01-22-2024, 12:08 PM

What kernel version are you running on 15.0?

Code:

uname -r

jayjwa · 01-22-2024, 12:16 PM

It's been awhile since I served NFS, and I'm on Current, but I seem to have this in my /etc/fstab:

Code:

nfsd            /proc/fs/nfsd           nfsd            noauto          0      0

Looks like you might not have the filesystems mount. You should see stuff in /proc/fs/nfsd and /proc/fs/nfs. Do you have an /etc/exports filled out? Did you check rpcinfo to see that nfs and friends are gathered 'round the table?

Code:

 rpcinfo
   program version netid     address                service    owner
    100000    4    tcp6      ::.0.111               portmapper superuser
    100000    3    tcp6      ::.0.111               portmapper superuser
    100000    4    udp6      ::.0.111               portmapper superuser
    100000    3    udp6      ::.0.111               portmapper superuser
    100000    4    tcp       0.0.0.0.0.111          portmapper superuser
    100000    3    tcp       0.0.0.0.0.111          portmapper superuser
    100000    2    tcp       0.0.0.0.0.111          portmapper superuser
    100000    4    udp       0.0.0.0.0.111          portmapper superuser
    100000    3    udp       0.0.0.0.0.111          portmapper superuser
    100000    2    udp       0.0.0.0.0.111          portmapper superuser
    100000    4    local     /var/run/rpcbind.sock  portmapper superuser
    100000    3    local     /var/run/rpcbind.sock  portmapper superuser
    100024    1    udp       0.0.0.0.137.216        status     32
    100024    1    tcp       0.0.0.0.193.243        status     32
    100024    1    udp6      ::.158.23              status     32
    100024    1    tcp6      ::.137.215             status     32
    100002    1    udp       0.0.0.0.162.61         rusersd    superuser
    100008    1    udp       0.0.0.0.222.158        walld      superuser
    100011    1    udp       0.0.0.0.229.196        rquotad    superuser
    100011    2    udp       0.0.0.0.229.196        rquotad    superuser
    100011    1    tcp       0.0.0.0.239.35         rquotad    superuser
    100011    2    tcp       0.0.0.0.239.35         rquotad    superuser
    100011    1    udp6      ::.204.173             rquotad    superuser
    100011    2    udp6      ::.204.173             rquotad    superuser
    100011    1    tcp6      ::.231.12              rquotad    superuser
    100011    2    tcp6      ::.231.12              rquotad    superuser
    100003    3    tcp       0.0.0.0.8.1            nfs        superuser
    100003    4    tcp       0.0.0.0.8.1            nfs        superuser
    100227    3    tcp       0.0.0.0.8.1            nfs_acl    superuser
    100003    3    tcp6      ::.8.1                 nfs        superuser
    100003    4    tcp6      ::.8.1                 nfs        superuser
    100227    3    tcp6      ::.8.1                 nfs_acl    superuser
    100021    1    udp       0.0.0.0.128.17         nlockmgr   superuser
    100021    3    udp       0.0.0.0.128.17         nlockmgr   superuser
    100021    4    udp       0.0.0.0.128.17         nlockmgr   superuser
    100021    1    tcp       0.0.0.0.182.185        nlockmgr   superuser
    100021    3    tcp       0.0.0.0.182.185        nlockmgr   superuser
    100021    4    tcp       0.0.0.0.182.185        nlockmgr   superuser
    100021    1    udp6      ::.162.62              nlockmgr   superuser
    100021    3    udp6      ::.162.62              nlockmgr   superuser
    100021    4    udp6      ::.162.62              nlockmgr   superuser
    100021    1    tcp6      ::.162.237             nlockmgr   superuser
    100021    3    tcp6      ::.162.237             nlockmgr   superuser
    100021    4    tcp6      ::.162.237             nlockmgr   superuser
    100005    1    udp       0.0.0.0.157.209        mountd     superuser
    100005    1    tcp       0.0.0.0.193.131        mountd     superuser
    100005    1    udp6      ::.142.158             mountd     superuser
    100005    1    tcp6      ::.231.47              mountd     superuser
    100005    2    udp       0.0.0.0.139.227        mountd     superuser
    100005    2    tcp       0.0.0.0.160.243        mountd     superuser
    100005    2    udp6      ::.160.119             mountd     superuser
    100005    2    tcp6      ::.165.219             mountd     superuser
    100005    3    udp       0.0.0.0.207.22         mountd     superuser
    100005    3    tcp       0.0.0.0.199.217        mountd     superuser
    100005    3    udp6      ::.192.81              mountd     superuser
    100005    3    tcp6      ::.164.195             mountd     superuser

showmount -e
Export list for server.lan:
/home 192.168.20.0/24

showmount -a
All mount points on server.lan:
192.168.20.5:/home

cat /etc/exports
# See exports(5) for a description.
# This file contains a list of all directories exported to other computers.
# It is used by rpc.nfsd and rpc.mountd.
/home   192.168.20.0/24(ro)

My guess is you don't have the filesystem there. I vaguely recall having to (maybe?) manually mount it but it was mounted automatically today.

mfoley · 01-22-2024, 12:29 PM

Quote:

Originally Posted by Petri Kaukasoina

What kernel version are you running on 15.0?

Code:

uname -r

4.14.67

Quote:

Originally Posted by jayjwa

It's been awhile since I served NFS, and I'm on Current, but I seem to have this in my /etc/fstab:

Code:

nfsd            /proc/fs/nfsd           nfsd            noauto          0      0

Looks like you might not have the filesystems mount. You should see stuff in /proc/fs/nfsd and /proc/fs/nfs. Do you have an /etc/exports filled out? Did you check rpcinfo to see that nfs and friends are gathered 'round the table?

I don't know what that fstab entry is supposed to do. I've never seen an fstab mount going to the /proc filesystem. Maybe that's something new? I'll have to look into that.

I did check /proc/1/net/rpc/nfsd and it was there. I'll check the /proc/fs entries after hours so users can work. Yes, I have /etc/exports, It was there from before when this was running before upgrading to Slackware 15.0 (two days ago). I'll also try the rpcinfo command after hours.

Quote:

showmount -e
Export list for server.lan:
/home 192.168.20.0/24

showmount -a
All mount points on server.lan:
192.168.20.5:/home

My showmount gave:

Code:

# showmount -a
rpc mount dump: RPC: Unable to receive; errno = Connection refused

Quote:

My guess is you don't have the filesystem there. I vaguely recall having to (maybe?) manually mount it but it was mounted automatically today.

Well, this all worked fine with nfs-utils-1.3.3-x86_64-2, so there's something different happening.

Petri Kaukasoina · 01-22-2024, 12:33 PM

Quote:

Originally Posted by mfoley

4.14.67

/proc/fs/nfsd/clients is only in kernel 5.3 and up. Are you running the old kernel from 14.2?

henca · 01-22-2024, 12:51 PM

I have configured Slackware 15.0 systems as NFS servers and have not seen these problems. I have no line in my /etc/fstab for /proc/fs/nfsd, but nevertheless it is mounted.

Some attempts to find what I might have but might be missing on the failing system:

Code:

henca@munin:/tmp> fgrep nfs /etc/fstab
henca@munin:/tmp> /usr/sbin/showmount 
Hosts on munin.lkp.se:
192.168.43.40
henca@munin:/tmp> /usr/sbin/showmount -e
Export list for munin.lkp.se:
/myth/recordings *
henca@munin:/tmp> ls /proc/fs/nfsd/
clients/         max_block_size    pool_stats               threads
export_features  max_connections   pool_threads             unlock_filesystem
export_stats     nfsv4gracetime    portlist                 unlock_ip
exports          nfsv4leasetime    reply_cache_stats        v4_end_grace
filehandle       nfsv4recoverydir  supported_krb5_enctypes  versions
henca@munin:/tmp> df /proc/fs/nfsd/
Filesystem     1K-blocks  Used Available Use% Mounted on
nfsd                   0     0         0    - /proc/fs/nfsd
henca@munin:/tmp> lsmod | grep nfs
henca@munin:/tmp> cat /proc/config.gz | gzip -d - | grep NFS
CONFIG_KERNFS=y
CONFIG_NFS_FS=y
CONFIG_NFS_V2=y
CONFIG_NFS_V3=y
CONFIG_NFS_V3_ACL=y
CONFIG_NFS_V4=y
CONFIG_NFS_SWAP=y
CONFIG_NFS_V4_1=y
CONFIG_NFS_V4_2=y
CONFIG_PNFS_FILE_LAYOUT=y
CONFIG_PNFS_BLOCK=y
CONFIG_PNFS_FLEXFILE_LAYOUT=y
CONFIG_NFS_V4_1_IMPLEMENTATION_ID_DOMAIN="kernel.org"
# CONFIG_NFS_V4_1_MIGRATION is not set
CONFIG_NFS_V4_SECURITY_LABEL=y
# CONFIG_NFS_USE_LEGACY_DNS is not set
CONFIG_NFS_USE_KERNEL_DNS=y
CONFIG_NFS_DEBUG=y
# CONFIG_NFS_DISABLE_UDP_SUPPORT is not set
# CONFIG_NFS_V4_2_READ_PLUS is not set
CONFIG_NFSD=y
CONFIG_NFSD_V2_ACL=y
CONFIG_NFSD_V3=y
CONFIG_NFSD_V3_ACL=y
CONFIG_NFSD_V4=y
CONFIG_NFSD_PNFS=y
CONFIG_NFSD_BLOCKLAYOUT=y
CONFIG_NFSD_SCSILAYOUT=y
CONFIG_NFSD_FLEXFILELAYOUT=y
CONFIG_NFSD_V4_2_INTER_SSC=y
# CONFIG_NFSD_V4_SECURITY_LABEL is not set
CONFIG_NFS_ACL_SUPPORT=y
CONFIG_NFS_COMMON=y
CONFIG_NFS_V4_2_SSC_HELPER=y
henca@munin:/tmp> cat /etc/slackware-version 
Slackware 15.0
henca@munin:/tmp>

For me, nfsd has worked fine on Slackware 15.0 with kernels 5.15.19, 5.15.117 and 5.15.145 (and maybe some more kernels), I have only been running NFS servers on machines with huge kernels. All my installations has been full Slackware installations with some custom additions, but to my knowledge, none of those additions should affect the NFS server capability.

I don't know for sure what mounted /proc/fs/nfsd on my system, but in /etc/rc.d/rc.nfsd I can see that /proc/fs/nfs is mounted.

regards Henrik

lostintime · 01-22-2024, 01:33 PM

Quote:

4.14.67

Well, looks like the system was not updated fully to 15.0.

With the Slackware 15.0 5.15.x kernel, there should be a kernel module that gets loaded:

lsmod | grep nfsd

Without that module the error messages seem plausible.

Petri Kaukasoina · 01-22-2024, 01:52 PM

nfs-utils-2.5.4 needs /proc/fs/nfsd/clients which is not offered before kernel 5.3.

henca · 01-23-2024, 12:55 AM

Quote:

Originally Posted by lostintime

Well, looks like the system was not updated fully to 15.0.

With the Slackware 15.0 5.15.x kernel, there should be a kernel module that gets loaded:

lsmod | grep nfsd

Without that module the error messages seem plausible.

I don't have any such module loaded on my Slackware 15.0 NFS servers, but again, I am using a slightly customized huge kernel with CONFIG_NFSD=y.

regards Henrik

mfoley · 01-25-2024, 03:57 PM

My current kernel version is 4.16.67, so not 5.3. Inerestingly slackpkg search shows 5.15.145 installed:

Code:

# slackpkg search kernel

Looking for kernel in package list. Please wait... DONE

The list below shows all packages with name matching "kernel".

[ installed ] - kernel-firmware-20231222_a7dee43-noarch-1
[ installed ] - kernel-generic-5.15.145-x86_64-1
[ installed ] - kernel-headers-5.15.145-x86-1
[ installed ] - kernel-huge-5.15.145-x86_64-1
[ installed ] - kernel-modules-5.15.145-x86_64-1
[ installed ] - kernel-source-5.15.145-noarch-1

I did run lilo after the upgrade. /boot shows 4.5.15:

Code:

lrwxrwxrwx 1 root root       21 2024-01-20 21:06 vmlinuz -> vmlinuz-huge-5.15.145
lrwxrwxrwx 1 root root       24 2024-01-20 21:06 vmlinuz-generic -> vmlinuz-generic-5.15.145
-rw-r--r-- 1 root root  7812416 2023-12-24 01:38 vmlinuz-generic-5.15.145
lrwxrwxrwx 1 root root       21 2024-01-20 21:06 vmlinuz-huge -> vmlinuz-huge-5.15.145
-rw-r--r-- 1 root root  8671136 2018-09-03 03:09 vmlinuz-huge-4.14.67
-rw-r--r-- 1 root root 11622048 2023-12-24 01:35 vmlinuz-huge-5.15.145

uname:

Code:

# uname -a
Linux OHPRSstorage.hprs.local 4.14.67 #1 SMP Mon Sep 3 03:00:40 EDT 2018 x86_64 AMD Ryzen 5 1600 Six-Core Processor AuthenticAMD GNU/Linux

Note that the date on the uname is Sep 3 2018. Which is the timestamp on the /boot/vmlinuz-huge-4.14.67. In looking at my lilo.conf I have:

Code:

# Linux bootable partition config begins
image = /boot/vmlinuz-huge-4.14.67
  root = /dev/md1
  label= Linux-4.14.67
  append="rcu_nocbs=0-11"
  read-only

image = /boot/vmlinuz
  root = /dev/md1
  label = Linux
  read-only
# Linux bootable partition config ends

So that might explain things. After business hours I will remove the "image /boot/vmlinuz-huge-4.14.67" partition and re-lilo, The 'append="rcu_nocbs=0-11"' was put in there many moons ago to fix a crashing problem with the Ryzen processor. One of many references: https://forum.level1techs.com/t/cont...ryzen/130004/5. Since this machine is using this same processor I will put this setting on the "image = /boot/vmlinuz" partition config.

mfoley · 01-25-2024, 05:41 PM

Success! I reinstalled nfs-utils 2.5.4, removed the vmlinuz-huge-4.14.67 from /etc/lilo, re-lilo'ed and rebooted. All is running well now. Thanks LQ experts!

glorsplitz · 01-28-2024, 12:10 PM

out of curiosity, you had a phantom/ghost kernel in your lilo.conf, as "slackpkg search kernel" did not show anything 4.14.67

which kernel is attached to /boot/vmlinuz as this is a link?

you have no fallback?

I boot generic kernel and huge as fallback