Dmesg full of nfs: server ... not responding, timed out

Support
1

I had a qnap mounted on my server using the following command, mount -t nfs4 server:/POOL1 /mnt/qnap2 ; and after I unmounted the qnap these messages started appearing in the logs. [24192439.075181] nfs: server server… not responding, timed out
I mounted this manually as it is a transient device used for moving data around the globe, so there are no entries in /etc/fstab.
/proc/mounts has no reference to the old mount point.
lsof has no references..
restarting nfs.service has no effect.
Netstat has no references.
df -k has no references
Server is an active data repository that can’t be restarted at this time…

I’m missing something simple but can’t find it.

2

Hello

First, could you please run the following command?

cat /proc/self/mounts | grep -i nfs || echo “no nfs”

Even if it doesn’t show up with mount or df, there’s a possibility it might still be present only in /proc/self/mounts.

Thanks

3

Nothing that wasn’t expected
cat /proc/self/mounts | grep -i nfs || echo “no nfs”

sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0
nfsd /proc/fs/nfsd nfsd rw,relatime 0 0

server1:/space/user13_export_home_oracle /export/home/oracle nfs rw,relatime,vers=3,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=10.10.7.111,mountvers=3,mountport=20048,mountproto=udp,local_lock=none,addr=10.10.7.111 0 0
server2:/POOL1/data2 /data2 nfs rw,relatime,vers=3,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=10.10.8.84,mountvers=3,mountport=20048,mountproto=udp,local_lock=none,addr=10.10.8.84 0 0
server2:/POOL2 /data2 nfs rw,relatime,vers=3,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=10.10.8.84,mountvers=3,mountport=20048,mountproto=udp,local_lock=none,addr=10.10.8.84 0 0
server1:/space/costman/DBASE /costman/DBASE nfs rw,relatime,vers=3,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=10.10.7.111,mountvers=3,mountport=20048,mountproto=udp,local_lock=none,addr=10.10.7.111 0 0
server2:/POOL3/data23b /data23b nfs rw,relatime,vers=3,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=10.10.8.84,mountvers=3,mountport=20048,mountproto=udp,local_lock=none,addr=10.10.8.84 0 0
server2:/POOL3/data23c /data23c nfs rw,relatime,vers=3,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=10.10.8.84,mountvers=3,mountport=20048,mountproto=udp,local_lock=none,addr=10.10.8.84 0 0
server2:/POOL3/data23a /data23a nfs rw,relatime,vers=3,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=10.10.8.84,mountvers=3,mountport=20048,mountproto=udp,local_lock=none,addr=10.10.8.84 0 0
server2:/POOL1 /data1 nfs rw,relatime,vers=3,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=10.10.8.84,mountvers=3,mountport=20048,mountproto=udp,local_lock=none,addr=10.10.8.84 0 0
server3:/POOL4/data24a /data24a nfs rw,relatime,vers=3,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=10.10.7.44,mountvers=3,mountport=20048,mountproto=udp,local_lock=none,addr=10.10.7.44 0 0
server1:/space/user13_user13 /usr/user13 nfs ro,relatime,vers=3,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=10.10.7.111,mountvers=3,mountport=20048,mountproto=udp,local_lock=none,addr=10.10.7.111 0 0
server2:/POOL3/data23d /data23d nfs rw,relatime,vers=3,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=10.10.8.84,mountvers=3,mountport=20048,mountproto=udp,local_lock=none,addr=10.10.8.84 0 0
server1:/space/work /work nfs ro,relatime,vers=3,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=10.10.7.111,mountvers=3,mountport=20048,mountproto=udp,local_lock=none,addr=10.10.7.111 0 0
server1:/space/u01 /u01 nfs rw,relatime,vers=3,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=10.10.7.111,mountvers=3,mountport=20048,mountproto=udp,local_lock=none,addr=10.10.7.111 0 0
server2:/POOL3/data2-hou-3b /data2-hou-3b nfs rw,relatime,vers=3,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=10.10.8.84,mountvers=3,mountport=20048,mountproto=udp,local_lock=none,addr=10.10.8.84 0 0
server1:/space/user13_export_linux /export/linux nfs rw,relatime,vers=3,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=10.10.7.111,mountvers=3,mountport=20048,mountproto=udp,local_lock=none,addr=10.10.7.111 0 0
server2:/POOL3 /data3 nfs rw,relatime,vers=3,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=10.10.8.84,mountvers=3,mountport=20048,mountproto=udp,local_lock=none,addr=10.10.8.84 0 0
server1:/space/user13_export_home_user13 /export/home/user13 nfs rw,relatime,vers=3,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=10.10.7.111,mountvers=3,mountport=20048,mountproto=udp,local_lock=none,addr=10.10.7.111 0 0
server3:/POOL4/data4_staging /data4/staging nfs rw,relatime,vers=3,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=10.10.7.44,mountvers=3,mountport=20048,mountproto=udp,local_lock=none,addr=10.10.7.44 0 0
server1:/space/user13_export_user13 /export/user13 nfs rw,relatime,vers=3,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=10.10.7.111,mountvers=3,mountport=20048,mountproto=udp,local_lock=none,addr=10.10.7.111 0 0
server1:/space/working /working nfs rw,relatime,vers=3,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=10.10.7.111,mountvers=3,mountport=20048,mountproto=udp,local_lock=none,addr=10.10.7.111 0 0
server2:/POOL3/data2-hou-3a /data2-hou-3a nfs rw,relatime,vers=3,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=10.10.8.84,mountvers=3,mountport=20048,mountproto=udp,local_lock=none,addr=10.10.8.84 0 0

4

Thanks for the details.

From /proc/self/mounts, all of the NFS mounts are hard and use timeo=600,retrans=2.
Note that for NFS, timeo is in tenths of a second, so timeo=600 means a 60-second timeout.

Also, it looks like the mount point /data2 is mounted twice
server2:/POOL1/data2 /data2
server2:/POOL2 /data2

Reference
https://linux.die.net/man/5/nfs

To identify which server/export is actually timing out, could you please share the output of the following commands?

dmesg -T | egrep -i ‘nfs: server|not responding|timed out’ | tail -n 80
nfsstat -m

5

only showing last 18 lines as it is the same message in dmesg.

[Wed Dec 17 10:29:38 2025] nfs: server qnap2-10g not responding, timed out
[Wed Dec 17 10:29:43 2025] nfs: server qnap2-10g not responding, timed out
[Wed Dec 17 10:30:38 2025] nfs: server qnap2-10g not responding, timed out
[Wed Dec 17 10:30:49 2025] nfs: server qnap2-10g not responding, timed out
[Wed Dec 17 10:31:41 2025] nfs: server qnap2-10g not responding, timed out
[Wed Dec 17 10:31:54 2025] nfs: server qnap2-10g not responding, timed out
[Wed Dec 17 10:32:41 2025] nfs: server qnap2-10g not responding, timed out
[Wed Dec 17 10:32:59 2025] nfs: server qnap2-10g not responding, timed out
[Wed Dec 17 10:33:44 2025] nfs: server qnap2-10g not responding, timed out
[Wed Dec 17 10:34:05 2025] nfs: server qnap2-10g not responding, timed out
[Wed Dec 17 10:34:44 2025] nfs: server qnap2-10g not responding, timed out
[Wed Dec 17 10:35:10 2025] nfs: server qnap2-10g not responding, timed out
[Wed Dec 17 10:35:47 2025] nfs: server qnap2-10g not responding, timed out
[Wed Dec 17 10:36:17 2025] nfs: server qnap2-10g not responding, timed out
[Wed Dec 17 10:36:48 2025] nfs: server qnap2-10g not responding, timed out
[Wed Dec 17 10:37:23 2025] nfs: server qnap2-10g not responding, timed out
[Wed Dec 17 10:37:49 2025] nfs: server qnap2-10g not responding, timed out
[Wed Dec 17 10:38:29 2025] nfs: server qnap2-10g not responding, timed out

and there is no output from nfsstat -m

[root@cal-pool5 ~]# nfsstat -m
[root@cal-pool5 ~]#

6

Hello,

The kernel is timing out while waiting for an NFS RPC response from qnap2-10g.
This typically points to a server-side NFS issue, network instability , or storage overload on the NAS.

One odd thing is that nfsstat -m produces no output on this host, so I’d like to confirm the current NFS mounts and basic reachability.

Could you please share the output of the following commands?

findmnt -t nfs,nfs4
ls -l /proc/self/mountstats; head -n 40 /proc/self/mountstats
getent hosts qnap2-10g
nc -vz qnap2-10g 2049

Thanks

7

Note .. Since this is a production server I’ve mounted a second qnap1 not the one showing up in dmesg. The second qnap was mounted using same command.
mount -t nfs4 qnap1-10g:/POOL1 /mnt/whitecap And i’ts working fine.

#findmnt -t nfs,nfs4

TARGET SOURCE FSTYPE OPTIONS
/mnt/whitecap qnap1-10g:/POOL1 nfs rw,relatime,vers=3,rsize=1048576,wsize=1048576,namlen=255,

#ls -l /proc/self/mountstats; head -n 40 /proc/self/mountstats

-r-------- 1 root root 0 Dec 17 16:38 /proc/self/mountstats
device sysfs mounted on /sys with fstype sysfs
device proc mounted on /proc with fstype proc
device devtmpfs mounted on /dev with fstype devtmpfs
device securityfs mounted on /sys/kernel/security with fstype securityfs
device tmpfs mounted on /dev/shm with fstype tmpfs
device devpts mounted on /dev/pts with fstype devpts
device tmpfs mounted on /run with fstype tmpfs
device tmpfs mounted on /sys/fs/cgroup with fstype tmpfs
device cgroup mounted on /sys/fs/cgroup/systemd with fstype cgroup
device pstore mounted on /sys/fs/pstore with fstype pstore
device bpf mounted on /sys/fs/bpf with fstype bpf
device cgroup mounted on /sys/fs/cgroup/cpu,cpuacct with fstype cgroup
device cgroup mounted on /sys/fs/cgroup/blkio with fstype cgroup
device cgroup mounted on /sys/fs/cgroup/hugetlb with fstype cgroup
device cgroup mounted on /sys/fs/cgroup/pids with fstype cgroup
device cgroup mounted on /sys/fs/cgroup/net_cls,net_prio with fstype cgroup
device cgroup mounted on /sys/fs/cgroup/memory with fstype cgroup
device cgroup mounted on /sys/fs/cgroup/cpuset with fstype cgroup
device cgroup mounted on /sys/fs/cgroup/freezer with fstype cgroup
device cgroup mounted on /sys/fs/cgroup/perf_event with fstype cgroup
device cgroup mounted on /sys/fs/cgroup/rdma with fstype cgroup
device cgroup mounted on /sys/fs/cgroup/devices with fstype cgroup
device configfs mounted on /sys/kernel/config with fstype configfs
device /dev/md2 mounted on / with fstype ext3
device /dev/md1 mounted on /usr with fstype ext4
device systemd-1 mounted on /proc/sys/fs/binfmt_misc with fstype autofs
device hugetlbfs mounted on /dev/hugepages with fstype hugetlbfs
device debugfs mounted on /sys/kernel/debug with fstype debugfs
device mqueue mounted on /dev/mqueue with fstype mqueue
device /dev/md3 mounted on /var with fstype ext4
device /dev/md0 mounted on /boot with fstype ext4
device sunrpc mounted on /var/lib/nfs/rpc_pipefs with fstype rpc_pipefs
device auto.direct.Linux mounted on /redacted-5e with fstype autofs
device auto.direct.Linux mounted on /redacted5d with fstype autofs
device auto.direct.Linux mounted on /redacted5c with fstype autofs
device auto.direct.Linux mounted on /redacted5a with fstype autofs
device tracefs mounted on /sys/kernel/debug/tracing with fstype tracefs
device datapool/dbload_staging mounted on /dbload_staging with fstype zfs
device datapool/POOL5 mounted on /POOL5 with fstype zfs
device nfsd mounted on /proc/fs/nfsd with fstype nfsd

#getent hosts qnap2-10g

#nc -vz qnap2-10g 2049

Ncat: Version 7.92 ( Ncat - Netcat for the 21st Century )
Ncat: Could not resolve hostname “qnap2-10g”: Name or service not known. QUITTING.

8

qnap2-10g is not resolvable on this client right now (getent hosts qnap2-10g returns nothing, and nc -vz qnap2-10g 2049 fails with “Name or service not known”).
So the immediate issue is DNS/hosts (name resolution) for qnap2-10g.
qnap1-10g mounts fine, so the NFS client itself looks OK.