How to add a brick to a GlusterFS volume

A GlusterFS volume can run out of brick capacity while clients still need the same mount point and volume name. Adding a brick expands the volume layout in place, but the new storage should be checked before it is attached so the cluster does not accept a wrong path or an undersized filesystem.

The gluster volume add-brick command records one or more HOST:PATH brick entries in the volume configuration. For a distributed volume, one new brick can extend the distribute set; for distributed replicated or distributed dispersed volumes, the added bricks must match the replica or disperse grouping so GlusterFS can keep the intended data-protection layout.

After the add operation, the volume still needs a rebalance to update directory layout and move existing files across the expanded set. Run the change during a maintenance window for busy volumes, keep the new brick directory empty before adding it, and use a dedicated filesystem such as XFS instead of a directory on the root filesystem.

Steps to add a brick to a GlusterFS volume:

Display the current brick layout for the volume.

$ sudo gluster volume info volume1

Volume Name: volume1
Type: Distribute
Status: Started
Number of Bricks: 2
Transport-type: tcp
Bricks:
Brick1: node1:/srv/gluster/brick1
Brick2: node2:/srv/gluster/brick1

Confirm the target server is connected to the trusted pool.

$ sudo gluster peer status
Number of Peers: 2

Hostname: node2
Uuid: 9b947671-67d1-4a46-8e8a-42b2d2b2b3f0
State: Peer in Cluster (Connected)

Hostname: node3
Uuid: 3e61c12d-0463-42be-a9e7-5d5875b75d6c
State: Peer in Cluster (Connected)

If the target server is missing, add it to the trusted pool before using its brick path.
Related: How to create a GlusterFS trusted storage pool

Create the brick directory on the target server.
```
$ sudo mkdir -p /srv/gluster/brick2
```
Do not reuse an application directory or an old brick path that still contains files. GlusterFS expects a new brick directory to start empty.
Confirm the brick directory is empty.
```
$ sudo find /srv/gluster/brick2 -mindepth 1 -maxdepth 1 -print
```
No output means the directory has no visible entries. If the command prints files or directories, stop and choose an empty brick path.
Confirm the brick path is on the intended filesystem.
```
$ sudo df -Th /srv/gluster/brick2
Filesystem     Type  Size  Used Avail Use% Mounted on
/dev/sdc1      xfs   500G   35G  465G   8% /srv/gluster
```
A brick on / or another shared system filesystem can fill the operating system volume and disrupt glusterd plus other services.
Add the new brick path to the volume.
```
$ sudo gluster volume add-brick volume1 node3:/srv/gluster/brick2
volume add-brick: success
```
For distributed replicated and distributed dispersed volumes, add bricks in a count that matches the replica or disperse layout. Add the full group in one command so each new sub-volume is formed correctly.

Confirm the volume lists the new brick.

$ sudo gluster volume info volume1

Volume Name: volume1
Type: Distribute
Status: Started
Number of Bricks: 3
Transport-type: tcp
Bricks:
Brick1: node1:/srv/gluster/brick1
Brick2: node2:/srv/gluster/brick1
Brick3: node3:/srv/gluster/brick2

Start a rebalance to move existing data onto the expanded layout.
```
$ sudo gluster volume rebalance volume1 start
volume rebalance: volume1: success: Rebalance on volume volume1 has been started successfully
```
Rebalance can compete with client I/O while files and directory layouts are processed. Schedule it during a lower-traffic window when possible.

Related: How to rebalance a GlusterFS volume

Check rebalance status until each node reports completed state.

$ sudo gluster volume rebalance volume1 status

                                    Node  Rebalanced-files    size  scanned  failures  skipped     status  run time in sec
                               ---------  ----------------  ------  -------  --------  -------  ---------  ---------------
                                   node1              1187    10GB     1.1M         0        0  completed              233
                                   node2              1234    10GB     1.2M         0        0  completed              245
                                   node3              1199    10GB     1.1M         0        0  completed              240

Large file counts can keep the status at in progress for an extended time. Re-run the status command and watch for increasing scanned or rebalanced counts.

Verify the new brick is online.

$ sudo gluster volume status volume1
Status of volume: volume1
Gluster process                             TCP Port  RDMA Port  Online  Pid
------------------------------------------------------------------------------
Brick node1:/srv/gluster/brick1             49152     0          Y       1443
Brick node2:/srv/gluster/brick1             49153     0          Y       1398
Brick node3:/srv/gluster/brick2             49154     0          Y       1522

Create a small smoke-test file through a mounted GlusterFS client path.
```
$ sudo touch /mnt/volume1/.add-brick-check
```
Replace /mnt/volume1 with a real client mount for volume1. If the admin server is not a client, run this check from a mounted client host.

Confirm the smoke-test file is visible through the mount.

$ ls -l /mnt/volume1/.add-brick-check
-rw-r--r-- 1 root root 0 Jun 16 09:20 /mnt/volume1/.add-brick-check

Remove the smoke-test file.
```
$ sudo rm /mnt/volume1/.add-brick-check
```