How to map a Ceph RBD image on Linux

A Ceph RBD image becomes usable on a Linux host only after the kernel RBD client maps it to a local block device. That mapping is the handoff between cluster storage and the tools that create filesystems, mount volumes, or attach a disk to a workload.

The client host needs ceph-common, a minimal /etc/ceph/ceph.conf, and a scoped CephX keyring for a user allowed to use the target pool. Current Ceph documentation uses rbd device map for kernel-client mappings; the command loads the RBD kernel module when needed and returns the local device path.

Format only a new or intentionally empty image. mkfs.ext4 overwrites existing filesystem metadata, and a normal ext4 filesystem should not be mounted read-write from more than one host at the same time.

1. Install the Ceph client tools on the Linux host.

   $ sudo apt install ceph-common
   Reading package lists... Done
   Building dependency tree... Done
   The following NEW packages will be installed:
     ceph-common
   ##### snipped #####
   Setting up ceph-common ...

   Debian and Ubuntu provide rbd through ceph-common. On other Linux distributions, install the package that provides the rbd client and kernel mapping helper.

2. Create the local Ceph configuration directory.

   $ sudo install -d -m 755 /etc/ceph

3. Add the client cluster configuration.

   $ sudoedit /etc/ceph/ceph.conf

   /etc/ceph/ceph.conf

   [global]
       fsid = 11111111-2222-3333-4444-555555555555
       mon_host = 192.0.2.11,192.0.2.12,192.0.2.13

   Use the real cluster FSID and monitor addresses from the Ceph admin host. Keep production hostnames and internal addresses out of saved transcripts and screenshots.

4. Install the scoped RBD client keyring.

   $ sudo install -m 600 ceph.client.rbd-client.keyring /etc/ceph/ceph.client.rbd-client.keyring

   The keyring should contain the client.rbd-client identity with mon 'profile rbd', osd 'profile rbd pool=rbd', and mgr 'profile rbd pool=rbd' caps for this example pool.

5. Confirm the client files are present with safe permissions.

   $ sudo ls -l /etc/ceph/ceph.conf /etc/ceph/ceph.client.rbd-client.keyring
   -rw-r--r-- 1 root root 130 Jun 29 09:18 /etc/ceph/ceph.conf
   -rw------- 1 root root  74 Jun 29 09:18 /etc/ceph/ceph.client.rbd-client.keyring

   A world-readable keyring exposes the CephX secret to local users. Keep client keyrings at mode 600 unless a stricter host policy is in place.

6. Check cluster health with the client identity.

   $ ceph -s --id rbd-client
     cluster:
       id:     11111111-2222-3333-4444-555555555555
       health: HEALTH_OK
   
     services:
       mon: 3 daemons, quorum ceph-node1,ceph-node2,ceph-node3
       mgr: ceph-node1(active), standbys: ceph-node2
       osd: 9 osds: 9 up, 9 in
   
     data:
       pools:   4 pools, 96 pgs
       objects: 262.15k objects, 1.1 TiB
       usage:   3.4 TiB used, 56 TiB / 60 TiB avail
       pgs:     96 active+clean

   Delay the client mount when the cluster reports degraded, backfilling, remapped, or stuck placement groups.
   Related: How to check Ceph cluster health

7. Inspect the target RBD image before mapping it.

   $ rbd info rbd/vm-100-disk-0 --id rbd-client
   rbd image 'vm-100-disk-0':
           size 64 GiB in 16384 objects
           order 22 (4 MiB objects)
           snapshot_count: 0
           id: 2d0f4c6b8c91
           block_name_prefix: rbd_data.2d0f4c6b8c91
           format: 2
           features: layering, exclusive-lock, object-map, fast-diff, deep-flatten
           op_features:
           flags:

   Confirm the pool, image name, and size before exposing the image as a local disk. If the kernel client rejects an image feature during mapping, create the image with kernel-compatible features or use a librbd consumer such as a hypervisor that supports that feature set.

8. Map the RBD image through the kernel client.

   $ sudo rbd device map rbd/vm-100-disk-0 --id rbd-client
   /dev/rbd0

9. List mapped RBD devices.

   $ rbd device list
   id pool namespace image         snap device
   0  rbd            vm-100-disk-0 -    /dev/rbd0

   The stable symlink is normally under /dev/rbd/pool/image, such as /dev/rbd/rbd/vm-100-disk-0. Use the returned /dev/rbd0 path if the symlink has not appeared yet.

10. Check the local block device size.

    $ lsblk /dev/rbd0
    NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
    rbd0 252:0    0  64G  0 disk

11. Create an ext4 filesystem on the new mapped image.

    $ sudo mkfs.ext4 -m 0 -L rbd-data /dev/rbd/rbd/vm-100-disk-0
    mke2fs 1.47.2 (1-Jan-2025)
    Creating filesystem with 16777216 4k blocks and 4194304 inodes
    Filesystem UUID: 4f3e7b28-6d2f-4d11-a922-66a9cf88e301
    Superblock backups stored on blocks:
        32768, 98304, 163840, 229376, 294912
    ##### snipped #####
    Writing superblocks and filesystem accounting information: done

    This command destroys any filesystem already stored in the image. Skip it when mapping an image that already contains data.

12. Create the mount point.

    $ sudo mkdir -p /mnt/rbd-data

13. Mount the mapped RBD filesystem.

    $ sudo mount /dev/rbd/rbd/vm-100-disk-0 /mnt/rbd-data

    Do not mount the same non-clustered filesystem read-write on multiple hosts. Use a clustered filesystem or a single active writer when several clients need access to the same block image.

14. Confirm the mounted filesystem.

    $ findmnt /mnt/rbd-data
    TARGET        SOURCE                         FSTYPE OPTIONS
    /mnt/rbd-data /dev/rbd/rbd/vm-100-disk-0    ext4   rw,relatime

15. Write a small smoke-test file through the mounted RBD image.

    $ sudo sh -c 'printf "rbd client map test\n" > /mnt/rbd-data/client-map-test.txt'

16. Read the file back from the mounted filesystem.

    $ sudo cat /mnt/rbd-data/client-map-test.txt
    rbd client map test

17. Remove the temporary smoke-test file.

    $ sudo rm /mnt/rbd-data/client-map-test.txt