NFS mount cannot access on SES 7 with arm

Archive SUSE Enterprise Storage
1

Environment

SUSE Enterprise Server 7

Situation

Deploy NFS ganesha service on arm successfully after apply new patch: PTF-23231, the ganesha service show online:

# ceph -s
  cluster:
    id:     a9f4e940-3190-11ec-9221-98039bbde460
    health: HEALTH_OK

  services:
    mon: 3 daemons, quorum oss-hr350a01,oss-hr350a03,oss-hr350a02 (age 2d)
    mgr: oss-hr350a01.remjwr(active, since 4M), standbys: oss-hr350a03.xdjamm, oss-hr350a02.xdofyi
    mds: 4 fs {cephfs:0=cephfs.oss-hr350a10.hopawd=up:active,k3s-demo:0=oss-nwcs.oss-hr350a03.htuznq=up:active,oss-llt:0=oss-llt.oss-hr350a05.qclcvp=up:active,oss-nwcs:0=oss-nwcs.oss-ses-admin.guagms=up:active} 3 up:standby
    osd: 48 osds: 48 up (since 4M), 48 in (since 4M)
    rgw: 3 daemons active (oss.rgw.oss-hr350a07.ytvgym, oss.rgw.oss-hr350a08.lpimvk, oss.rgw.oss-hr350a09.iyushr)

  task status:

  data:
    pools:   17 pools, 521 pgs
    objects: 527.38k objects, 181 GiB
    usage:   592 GiB used, 167 TiB / 168 TiB avail
    pgs:     521 active+clean

  io:
    client:   170 B/s rd, 4.0 KiB/s wr, 0 op/s rd, 0 op/s wr

 # ceph orch ps --daemon_type nfs
NAME                      HOST          STATUS        REFRESHED  AGE  VERSION  IMAGE NAME                                             IMAGE ID      CONTAINER ID
nfs.ganesha.oss-hr350a06  oss-hr350a06  running (2h)  3m ago     3M   3.0.3    registry.suse.com/ptf/23231/ses/7/ceph/ceph:PTF-23231  f24eba565f08  6182ad3f31bf

Create new NFS mount on SES GUI like this:

image
image749×769 32 KB 

# rados --pool ganesha_config --namespace ganesha ls
debugconf-nfs.ganesha
rec-0000000000000008:nfs.ganesha.oss-hr350a06
rec-000000000000000a:nfs.ganesha.oss-hr350a06
grace
rec-0000000000000004:nfs.ganesha.oss-hr350a06
rec-0000000000000002:nfs.ganesha.oss-hr350a06
export-1
rec-0000000000000010:nfs.ganesha.oss-hr350a06
export-4
rec-000000000000000e:nfs.ganesha.oss-hr350a06
export-3
conf-nfs.ganesha
rec-0000000000000006:nfs.ganesha.oss-hr350a06
export-2
rec-000000000000000c:nfs.ganesha.oss-hr350a06

# rados --pool ganesha_config --namespace ganesha get export-4 /tmp/export-3
# cat /tmp/export-3
EXPORT {
    export_id = 3;
    path = "/new-test";
    pseudo = "/new-test-alias1";
    access_type = "RW";
    squash = "no_root_squash";
    protocols = 4;
    transports = "UDP", "TCP";
    FSAL {
        name = "CEPH";
        user_id = "k3sfs";
        filesystem = "k3s-demo";
        secret_access_key = "***********************";
    }
}

# rados --pool ganesha_config --namespace ganesha get conf-nfs.ganesha /tmp/conf-nfs.ganesha
# cat /tmp/conf-nfs.ganesha
%url "rados://ganesha_config/ganesha/export-1"

%url "rados://ganesha_config/ganesha/export-3"

%url "rados://ganesha_config/ganesha/export-2"

%url "rados://ganesha_config/ganesha/export-4"

We can see the NFS mount is created successfuly.

While mount it on client, it report error:

# mount -t nfs4 oss-hr350a06:/new-test-alias1 /tmp/oss-nwcs -v
mount.nfs4: timeout set for Wed Mar 16 06:15:52 2022
mount.nfs4: trying text-based options 'vers=4.2,addr=xx.xx.xx.xx,clientaddr=xx.xx.xx.xx'
mount.nfs4: mount(2): No such file or directory
mount.nfs4: trying text-based options 'addr=xx.xx.xx.xx'
mount.nfs4: prog 100003, trying vers=3, prot=6
mount.nfs4: portmap query retrying: RPC: Unable to receive
mount.nfs4: prog 100003, trying vers=3, prot=17
mount.nfs4: portmap query failed: RPC: Unable to receive - Connection refused
mount.nfs4: trying text-based options 'vers=4.2,addr=xx.xx.xx.xx,clientaddr=xx.xx.xx.xx'
mount.nfs4: mount(2): No such file or directory
mount.nfs4: trying text-based options 'addr=xx.xx.xx.xx'
mount.nfs4: prog 100003, trying vers=3, prot=6
mount.nfs4: portmap query retrying: RPC: Unable to receive
mount.nfs4: prog 100003, trying vers=3, prot=17
mount.nfs4: portmap query failed: RPC: Unable to receive - Connection refused
mount.nfs4: trying text-based options 'vers=4.2,addr=xx.xx.xx.xx,clientaddr=xx.xx.xx.xx'
mount.nfs4: mount(2): No such file or directory
mount.nfs4: trying text-based options 'addr=xx.xx.xx.xx'
mount.nfs4: prog 100003, trying vers=3, prot=6
mount.nfs4: portmap query retrying: RPC: Unable to receive
mount.nfs4: prog 100003, trying vers=3, prot=17
mount.nfs4: portmap query failed: RPC: Unable to receive - Connection refused

It show “No such file or directory” error here.

The only workaround is to restart NFS ganecha service on node or using ceph orch:

oss-hr350a06:~ # systemctl restart ceph-a9f4e940-3190-11ec-9221-98039bbde460@nfs.ganesha.oss-hr350a06.service

Then the NFS mount can work now:

# mount -t nfs4 oss-hr350a06:/new-test-alias1 /tmp/oss-nwcs -v
mount.nfs4: timeout set for Wed Mar 16 06:20:37 2022
mount.nfs4: trying text-based options 'vers=4.2,addr=xx.xx.xx.xx,clientaddr=xx.xx.xx.xx'

Please help to look this issue.