How to create a dispersed GlusterFS volume

Creating a dispersed GlusterFS volume is the point where the brick count and redundancy policy become part of the storage layout. The command must match the number of bricks in the disperse set, or the volume can be created with weaker protection than intended or stop at an interactive redundancy prompt.

A dispersed volume uses erasure coding. Each file is split into data fragments and redundancy fragments across one disperse set, and redundancy defines how many bricks in that set can be unavailable while data remains recoverable. A six-brick set with redundancy 2 stores four data fragments and two parity fragments per stripe.

Use one brick per peer in a disperse set when the goal is node-level fault tolerance. GlusterFS warns when multiple bricks in the same disperse set are placed on one peer, and the official syntax requires redundancy greater than 0 with a total brick count greater than 2 * redundancy. Keep brick sizes aligned inside the set because usable space is limited by the smallest brick.

Steps to create a dispersed GlusterFS volume:

  1. Confirm all storage peers are connected to the trusted pool.
    $ sudo gluster peer status
    Number of Peers: 5
    
    Hostname: node2
    Uuid: 0b6c2f2d-7b7e-4f3f-9d4a-2f3e5b7d1a9c
    State: Peer in Cluster (Connected)
    
    Hostname: node3
    Uuid: 6a4f5c3d-1a2b-4c5d-9e0f-1a2b3c4d5e6f
    State: Peer in Cluster (Connected)
    ##### snipped #####
  2. Choose the disperse and redundancy values for the set.

    For a single dispersed volume, disperse is the brick count in the set. The example below uses disperse 6 and redundancy 2, so Disperse Data should later report 4.

  3. Create an empty brick directory below the brick filesystem on every participating peer.
    $ sudo mkdir -p /srv/gluster/brick1/brick

    Use dedicated storage for production bricks, and place the brick directory below the filesystem mount point instead of using the mount point itself.

  4. Create the dispersed volume from one peer in the trusted pool.
    $ sudo gluster volume create volume1 disperse 6 redundancy 2 transport tcp \
    node1:/srv/gluster/brick1/brick node2:/srv/gluster/brick1/brick node3:/srv/gluster/brick1/brick \
    node4:/srv/gluster/brick1/brick node5:/srv/gluster/brick1/brick node6:/srv/gluster/brick1/brick
    volume create: volume1: success: please start the volume to access data

    transport tcp is the default transport in current GlusterFS documentation, but keeping it explicit makes the volume definition easier to review.

    Do not add force unless the brick placement warning is understood and accepted. Multiple bricks from the same disperse set on one peer reduce node-failure protection.

  5. Start the new volume.
    $ sudo gluster volume start volume1
    volume start: volume1: success

    Start the volume before mounting it, because client operations can hang when a stopped volume is mounted.

  6. Inspect the single-set erasure-coding layout values.
    $ sudo gluster volume info volume1
    Volume Name: volume1
    Type: Disperse
    Status: Started
    Number of Bricks: 1 x 6 = 6
    Transport-type: tcp
    Disperse Data: 4
    Redundancy: 2
    Bricks:
    Brick1: node1:/srv/gluster/brick1/brick
    Brick2: node2:/srv/gluster/brick1/brick
    Brick3: node3:/srv/gluster/brick1/brick
    Brick4: node4:/srv/gluster/brick1/brick
    Brick5: node5:/srv/gluster/brick1/brick
    Brick6: node6:/srv/gluster/brick1/brick
  7. Review every brick process in the started dispersed volume.
    $ sudo gluster volume status volume1
    Status of volume: volume1
    Gluster process                              TCP Port  RDMA Port  Online  Pid
    -------------------------------------------------------------------------------
    Brick node1:/srv/gluster/brick1/brick        49152     0          Y       1712
    Brick node2:/srv/gluster/brick1/brick        49153     0          Y       1730
    Brick node3:/srv/gluster/brick1/brick        49154     0          Y       1748
    Brick node4:/srv/gluster/brick1/brick        49155     0          Y       1764
    Brick node5:/srv/gluster/brick1/brick        49156     0          Y       1781
    Brick node6:/srv/gluster/brick1/brick        49157     0          Y       1799
    
    Task Status of Volume volume1
    -------------------------------------------------------------------------------
    There are no active volume tasks
  8. Mount the started volume from a client when the brick status is online.