How to export CephFS through NFS-Ganesha

NFS-Ganesha can publish a CephFS directory as an NFSv4 export for application hosts that cannot mount CephFS directly. The export keeps CephFS as the backing filesystem while presenting a standard NFS endpoint to Linux clients.

The nfs manager module owns the NFS cluster and stores export definitions in Ceph instead of local files on the gateway host. In this setup, cephadm manages the NFS-Ganesha daemon, cephfs is the existing filesystem name, and /cephfs is the NFSv4 pseudo path presented to clients.

Run the cluster and export commands from a Ceph admin shell that can reach the monitors. Choose a client network that should be allowed to mount the export, keep root squashing enabled unless the workload needs root identity passthrough, and prove the handoff from a separate NFS client before giving the path to applications.

1. Check that the CephFS filesystem is active before exporting it.

   $ ceph fs status cephfs
   cephfs - 3 clients
   ======
   RANK  STATE           MDS             ACTIVITY     DNS    INOS
    0    active  cephfs.ceph-node1.a1b2  Reqs: 0 /s    12     15
         POOL         TYPE     USED  AVAIL
   cephfs_metadata  metadata  96.0M  2.0T
   cephfs_data      data      1.3G   2.0T
   STANDBY MDS
   cephfs.ceph-node2.c3d4

2. Enable the nfs manager module.

   $ ceph mgr module enable nfs

3. Create an NFS-Ganesha cluster on the selected gateway host.

   $ ceph nfs cluster create cephfs-nfs "1 ceph-node1"

   Replace ceph-node1 with the gateway host or placement string that should run the NFS daemon.

4. Check that the NFS daemon is running.

   $ ceph orch ps --service_name nfs.cephfs-nfs --refresh
   NAME                           HOST        PORTS   STATUS         REFRESHED  AGE
   nfs.cephfs-nfs.0.0.ceph-node1  ceph-node1  *:2049  running (35s)  5s ago     35s

5. Show the client endpoint for the NFS cluster.

   $ ceph nfs cluster info cephfs-nfs --format json-pretty
   {
       "cephfs-nfs": {
           "backend": [
               {
                   "hostname": "ceph-node1",
                   "ip": "192.0.2.21",
                   "port": 2049
               }
           ],
           "port": 2049
       }
   }

6. Create the CephFS export.

   $ ceph nfs export create cephfs \
     --cluster-id cephfs-nfs \
     --pseudo-path /cephfs \
     --fsname cephfs \
     --path=/ \
     --client_addr 192.0.2.0/24 \
     --squash root_squash \
     --sectype sys

   Exporting / publishes the whole CephFS namespace through this NFS pseudo path. Use a subdirectory path for application-scoped exports.

7. List the saved export.

   $ ceph nfs export ls cephfs-nfs --format json-pretty
   [
       "/cephfs"
   ]

8. Inspect the export definition before testing from a client.

   $ ceph nfs export info cephfs-nfs /cephfs --format json-pretty
   {
       "export_id": 1,
       "path": "/",
       "cluster_id": "cephfs-nfs",
       "pseudo": "/cephfs",
       "access_type": "RW",
       "squash": "root_squash",
       "protocols": [
           4
       ],
       "transports": [
           "TCP"
       ],
       "fsal": {
           "name": "CEPH",
           "fs_name": "cephfs"
       },
       "clients": [
           {
               "addresses": [
                   "192.0.2.0/24"
               ],
               "access_type": "RW",
               "squash": "root_squash"
           }
       ]
   }

9. Create a mount point on an allowed NFS client.

   $ sudo mkdir -p /mnt/cephfs-nfs

10. Mount the export from the client.

    $ sudo mount -t nfs -o vers=4.1 ceph-node1:/cephfs /mnt/cephfs-nfs

    Use the hostname, backend IP address, or ingress virtual IP returned by ceph nfs cluster info.

11. Check that the client sees the NFS mount.

    $ findmnt /mnt/cephfs-nfs
    TARGET           SOURCE              FSTYPE OPTIONS
    /mnt/cephfs-nfs ceph-node1:/cephfs nfs4   rw,relatime,vers=4.1,proto=tcp

12. Write a test file through the NFS mount.

    $ printf 'nfs ganesha test\n' > /mnt/cephfs-nfs/ganesha-test.txt

    Run the write as a user that has permission on the exported CephFS path. With root_squash, client-side root is mapped before CephFS permissions are checked.

13. Read the test file through the NFS mount.

    $ cat /mnt/cephfs-nfs/ganesha-test.txt
    nfs ganesha test

14. Remove the test file from the export.

    $ rm /mnt/cephfs-nfs/ganesha-test.txt

15. Unmount the client test mount.

    $ sudo umount /mnt/cephfs-nfs