How to remove a brick from a GlusterFS volume

Removing a GlusterFS brick without the staged migration can make files disappear from client mounts even when the old brick path still contains data. A safe shrink starts by confirming the exact brick entry, the volume layout, and enough free space on the remaining bricks before the brick is marked for removal.

The gluster volume remove-brick workflow marks one or more bricks for removal, starts a migration task, and keeps the volume online while data is moved away from the retiring brick. Distributed replicated and distributed dispersed volumes must remove a complete replica or disperse set from the same sub-volume, while a plain distributed volume depends on the migration to keep files visible after the brick leaves the layout.

Commit the removal only after the status output reports completed with no failures. The force path skips migration and leaves data on the removed brick inaccessible through GlusterFS clients, so reserve it for a deliberate data-loss or already-recovered scenario and check the old brick path before reusing the disk.

Related: How to add a brick to a GlusterFS volume
Related: How to rebalance a GlusterFS volume
Related: How to replace a GlusterFS brick

Steps to remove a brick from a GlusterFS volume:

  1. Display the current brick layout and identify the exact brick entry to remove. 
    $ sudo gluster volume info volume1

Volume Name: volume1
Type: Distribute
Status: Started
Number of Bricks: 3
Transport-type: tcp
Bricks:
Brick1: node1:/srv/gluster/brick1
Brick2: node2:/srv/gluster/brick1
Brick3: node3:/srv/gluster/brick2

    For distributed replicated and distributed dispersed volumes, remove the full replica or disperse set from the same sub-volume. Removing only part of a protected set changes the intended data-protection layout.

  2. Confirm the volume is started, the bricks are online, and no other volume task is already running. 
    $ sudo gluster volume status volume1
Status of volume: volume1
Gluster process                             TCP Port  RDMA Port  Online  Pid
------------------------------------------------------------------------------
Brick node1:/srv/gluster/brick1             49152     0          Y       1443
Brick node2:/srv/gluster/brick1             49153     0          Y       1398
Brick node3:/srv/gluster/brick2             49154     0          Y       1522

Task Status of Volume volume1
------------------------------------------------------------------------------
There are no active volume tasks

    Do not start a remove-brick migration while another rebalance, heal, replace-brick, or remove-brick task is still active for the same volume.

    Related: How to check GlusterFS volume status

  3. Check free space on the remaining brick filesystems before starting the migration. 
    $ sudo df -h /srv/gluster/brick1
Filesystem      Size  Used Avail Use% Mounted on
/dev/sdb1       2.0T  720G  1.3T  36% /srv/gluster

    Run the free-space check on each server that will keep bricks for the volume. The remaining bricks must have room for data migrated away from the removed brick.

  4. Start the remove-brick migration for the retiring brick. 
    $ sudo gluster volume remove-brick volume1 node3:/srv/gluster/brick2 start
volume remove-brick start: success
ID: 5fd02a44-f315-4c78-b8cb-8dc8f59d7b4d

    The start phase automatically launches the migration task that moves data from the removed brick to the remaining layout.

  5. Check the remove-brick status until the migration reports completed. 
    $ sudo gluster volume remove-brick volume1 node3:/srv/gluster/brick2 status
                                    Node  Rebalanced-files    size  scanned  failures  skipped     status  run time in sec
                               ---------  ----------------  ------  -------  --------  -------  ---------  ---------------
                                  node3              1187    10GB     1.1M         0        0  completed              233

    Do not commit while the status is in progress or while the failures column is non-zero. Investigate the failed files first and rerun the status check before continuing.

  6. Commit the brick removal after the migration completes. 
    $ sudo gluster volume remove-brick volume1 node3:/srv/gluster/brick2 commit
Removing brick(s) can result in data loss. Do you want to Continue? (y/n) y
volume remove-brick commit: success

    Using force removes the brick entry immediately and makes files that still reside only on that brick unavailable from GlusterFS clients.

  7. Confirm the volume definition no longer lists the removed brick. 
    $ sudo gluster volume info volume1

Volume Name: volume1
Type: Distribute
Status: Started
Number of Bricks: 2
Transport-type: tcp
Bricks:
Brick1: node1:/srv/gluster/brick1
Brick2: node2:/srv/gluster/brick1
  8. Confirm the remaining bricks are online and the remove-brick task is gone. 
    $ sudo gluster volume status volume1
Status of volume: volume1
Gluster process                             TCP Port  RDMA Port  Online  Pid
------------------------------------------------------------------------------
Brick node1:/srv/gluster/brick1             49152     0          Y       1443
Brick node2:/srv/gluster/brick1             49153     0          Y       1398

Task Status of Volume volume1
------------------------------------------------------------------------------
There are no active volume tasks
  9. Inspect the removed brick path before wiping or reusing the disk. 
    $ sudo find /srv/gluster/brick2 -mindepth 1 -maxdepth 2 -print
/srv/gluster/brick2/.glusterfs

    If user data files remain on the removed brick path, copy them back through a mounted GlusterFS client path before deleting the old brick contents.