How to create a GlusterFS arbiter volume

Choosing an arbiter layout helps a small GlusterFS cluster reduce split-brain risk without storing a third full copy of every file. An arbiter volume keeps two data bricks and one metadata-only arbiter brick in each replica set, so the create command must list bricks in the order that matches the intended failure domains.

Current GlusterFS syntax uses replica 2 arbiter 1 for this (2 + 1) layout. Older releases also accept replica 3 arbiter 1, but replica 2 arbiter 1 makes the two data bricks and one arbiter brick explicit. The last brick in each (2 + 1) group becomes the arbiter.

The arbiter brick stores directory entries, file names, extended attributes, and GlusterFS metadata, not file data. Place it on a separate node or failure domain, keep the brick directory empty before creation, and leave the automatically selected client quorum behavior in place after the volume is created.

Steps to create a GlusterFS arbiter volume:

  1. Check peer connectivity from a trusted-pool node.
    $ sudo gluster peer status
    Number of Peers: 2
    
    Hostname: node2
    Uuid: 0f3e0b7c-2e59-4c73-9c1c-3e65b6c1e8c9
    State: Peer in Cluster (Connected)
    
    Hostname: node3
    Uuid: 8b7e3e2a-1a6e-4f2b-b4a6-2a0b9a4f7a6d
    State: Peer in Cluster (Connected)
  2. Create the data brick directory on each data-brick node.
    $ sudo mkdir -p /bricks/volume1/brick

    Use an empty directory on a dedicated brick filesystem. Creating a brick on the system root filesystem can fill the operating system volume and may force unsafe force usage.

  3. Create the arbiter brick directory on the arbiter node.
    $ sudo mkdir -p /bricks/volume1/arbiter

    Size the arbiter filesystem by expected file count and metadata overhead, not by the data capacity of the two full bricks.

  4. Create the volume with the two data bricks first and the arbiter brick last.
    $ sudo gluster volume create volume1 replica 2 arbiter 1 transport tcp node1:/bricks/volume1/brick node2:/bricks/volume1/brick node3:/bricks/volume1/arbiter
    volume create: volume1: success: please start the volume to access data

    The last brick in each (2 + 1) group becomes the arbiter brick. For a distributed arbiter volume, list complete groups of three bricks in the same order.

  5. Start the new arbiter volume.
    $ sudo gluster volume start volume1
    volume start: volume1: success
  6. Verify the arbiter layout from the volume information.
    $ sudo gluster volume info volume1
    Volume Name: volume1
    Type: Replicate
    Status: Started
    Number of Bricks: 1 x (2 + 1) = 3
    Transport-type: tcp
    Bricks:
    Brick1: node1:/bricks/volume1/brick
    Brick2: node2:/bricks/volume1/brick
    Brick3: node3:/bricks/volume1/arbiter (arbiter)

    Some releases also show Snapshot Count, Volume ID, or extra volume options in this output.

  7. Confirm client quorum remains automatic for the arbiter volume.
    $ sudo gluster volume get volume1 cluster.quorum-type
    Option                                  Value
    ------                                  -----
    cluster.quorum-type                     auto

    GlusterFS sets client quorum to auto for arbiter volumes. Do not change it unless a recovery procedure from trusted cluster documentation requires that exception.

  8. Check that all brick processes and self-heal daemons are online.
    $ sudo gluster volume status volume1
    Status of volume: volume1
    Gluster process                             TCP Port  RDMA Port  Online  Pid
    ------------------------------------------------------------------------------
    Brick node1:/bricks/volume1/brick           49152     0          Y       1873
    Brick node2:/bricks/volume1/brick           49153     0          Y       1934
    Brick node3:/bricks/volume1/arbiter         49154     0          Y       2011
    Self-heal Daemon on node1                   N/A       N/A        Y       2098
    Self-heal Daemon on node2                   N/A       N/A        Y       2147
    Self-heal Daemon on node3                   N/A       N/A        Y       2210
    
    Task Status of Volume volume1
    ------------------------------------------------------------------------------
    There are no active volume tasks