How to resize a Ceph RBD image

Ceph RBD image size controls the maximum virtual disk capacity that block clients can use. Storage administrators resize an image when a Linux mount, virtual machine disk, or application data volume outgrows its original allocation, or when a deliberately planned reduction is needed.

The cluster-side capacity change is made with rbd resize, but the client that consumes the image still has its own block-device, partition, and filesystem state. A mapped Linux data volume usually needs the kernel RBD device refreshed before the partition table or filesystem can grow into the new image size.

Growing is the normal online path when the filesystem supports it. Shrinking is the dangerous direction because RBD will reduce the image once --allow-shrink is supplied, so the filesystem and any partition must already be smaller than the target image size before the image itself is reduced.

1. Check cluster health before changing image capacity.

   $ ceph -s
     cluster:
       id:     11111111-2222-3333-4444-555555555555
       health: HEALTH_OK
   
     services:
       mon: 3 daemons, quorum ceph-node1,ceph-node2,ceph-node3
       mgr: ceph-node1(active), standbys: ceph-node2
       osd: 9 osds: 9 up, 9 in
   
     data:
       pools:   4 pools, 96 pgs
       objects: 262.15k objects, 1.1 TiB
       usage:   3.4 TiB used, 56 TiB / 60 TiB avail
       pgs:     96 active+clean

   Delay the resize when the cluster reports degraded, backfilling, remapped, or stuck PGs. A size change adds metadata work to the same cluster that must protect the existing image contents.

   Related: How to check Ceph cluster health

2. Inspect the current image size.

   $ rbd info rbd/vm-100-disk-0
   rbd image 'vm-100-disk-0':
           size 64 GiB in 16384 objects
           order 22 (4 MiB objects)
           snapshot_count: 0
           id: 36e5f1f6a2b7
           block_name_prefix: rbd_data.36e5f1f6a2b7
           format: 2
           features: layering, exclusive-lock, object-map, fast-diff, deep-flatten

   The sample grows image rbd/vm-100-disk-0 from 64 GiB to 80 GiB. Replace the pool, image name, and target size with values from the real workload.

3. Check which clients have the image open.

   $ rbd status rbd/vm-100-disk-0
   Watchers:
       watcher=192.0.2.21:0/4213 client.314159 cookie=18446462598732840961

   The watcher identifies an active librbd or kernel RBD client. Coordinate the filesystem grow, guest rescan, or workload stop with the host or hypervisor that owns the mounted disk.

   Take a snapshot or backup before any shrink. For a smaller target, stop writes, reduce the filesystem and partition first, and use --allow-shrink only after the consumer is already below the target image size. XFS cannot be shrunk in place.

   Related: How to create a Ceph RBD snapshot

4. Resize the RBD image to the new capacity.

   $ rbd resize --size 80G rbd/vm-100-disk-0
   Resizing image: 100% complete...done.

   The --size value accepts M, G, or T units. Omit --allow-shrink for growth so an accidental smaller value is rejected.

5. Confirm the image reports the new size.

   $ rbd info rbd/vm-100-disk-0
   rbd image 'vm-100-disk-0':
           size 80 GiB in 20480 objects
           order 22 (4 MiB objects)
           snapshot_count: 0
           id: 36e5f1f6a2b7
           block_name_prefix: rbd_data.36e5f1f6a2b7
           format: 2
           features: layering, exclusive-lock, object-map, fast-diff, deep-flatten

6. List the mapped RBD device on the Linux client.

   $ rbd device list
   id  pool  namespace  image          snap  device
   0   rbd              vm-100-disk-0  -     /dev/rbd0

   Run client-side steps on the host that maps the image. If the image is a virtual machine disk, rescan and grow the disk from the guest or hypervisor path instead of treating the host as the filesystem owner.

7. Refresh the kernel RBD mapping after the image resize.

   $ echo 1 | sudo tee /sys/bus/rbd/devices/0/refresh
   1

   Use the id from rbd device list in the /sys/bus/rbd/devices/0/refresh path. Replace 0 with the mapped device ID. Writing 1 makes the kernel reread the image header for that mapped device.

8. Grow the partition when the mapped image contains a partition table.

   $ sudo growpart /dev/rbd0 1
   CHANGED: partition=1 start=2048 old: size=134215647 end=134217694 new: size=167770079 end=167772126

   Skip this step when the filesystem is created directly on /dev/rbd0. For a partitioned data image, grow the partition before growing the filesystem inside it.

9. Grow the filesystem into the larger partition.

   $ sudo resize2fs /dev/rbd0p1
   resize2fs 1.47.2 (1-Jan-2025)
   Filesystem at /dev/rbd0p1 is mounted on /mnt/rbd-data; on-line resizing required
   old_desc_blocks = 8, new_desc_blocks = 10
   The filesystem on /dev/rbd0p1 is now 20971259 (4k) blocks long.

   Use sudo xfs_growfs /mnt/rbd-data for a mounted XFS filesystem. Use the filesystem tool that matches the data volume, not the RBD image name.

10. Verify the mounted filesystem sees the intended capacity.

    $ df -h /mnt/rbd-data
    Filesystem      Size  Used Avail Use% Mounted on
    /dev/rbd0p1      79G   28G   52G  36% /mnt/rbd-data