How to create a distributed dispersed GlusterFS volume
Creating a distributed dispersed GlusterFS volume is a layout decision as much as a volume creation command. The brick order decides which bricks form each erasure-coded set, and a wrong order can leave a volume running with weaker node-failure protection than the operator intended.
A distributed dispersed volume combines multiple disperse sets into one namespace. Each set stores data fragments plus parity fragments, while the distribute layer places new files across the available sets. In a disperse 3 redundancy 1 layout, each set has two data bricks and one parity brick, so the example below uses two sets for a total of six bricks.
Plan the set boundaries before running gluster volume create. Total bricks must be a multiple of the disperse count, redundancy must be greater than 0, and 2 * redundancy must stay smaller than the disperse count. Keep bricks in each set on different failure domains when node-level protection matters, and keep brick sizes aligned because usable capacity in a set follows the smallest brick.
Steps to create a distributed dispersed GlusterFS volume:
- Review peer connectivity and hostname resolution for every brick host.
$ sudo gluster peer status Number of Peers: 2 Hostname: node2 Uuid: 0b6c2f2d-7b7e-4f3f-9d4a-2f3e5b7d1a9c State: Peer in Cluster (Connected) Hostname: node3 Uuid: 6a4f5c3d-1a2b-4c5d-9e0f-1a2b3c4d5e6f State: Peer in Cluster (Connected)
Use hostnames that resolve consistently from every peer before creating brick paths in the volume definition.
- Choose the disperse and redundancy values for each set.
The example below uses disperse 3 and redundancy 1. Six bricks therefore become two disperse sets, and gluster volume info should later report 2 x (2 + 1) = 6.
- Create empty brick directories below the brick filesystems on each participating peer.
$ sudo mkdir -p /srv/gluster/brick1/brick /srv/gluster/brick2/brick
Use dedicated storage for production bricks, and place each brick directory below the filesystem mount point instead of using the mount point itself.
- Create the distributed dispersed volume from one peer in the trusted pool.
$ sudo gluster volume create volume1 disperse 3 redundancy 1 transport tcp \ node1:/srv/gluster/brick1/brick node2:/srv/gluster/brick1/brick node3:/srv/gluster/brick1/brick \ node1:/srv/gluster/brick2/brick node2:/srv/gluster/brick2/brick node3:/srv/gluster/brick2/brick volume create: volume1: success: please start the volume to access data
The first three bricks form the first disperse set, and the next three form the second set. Current GlusterFS documentation treats tcp as the default transport, but keeping it explicit makes the volume definition easier to review.
Do not add force unless the brick placement warning is understood and accepted. A forced layout can bypass safety checks that protect fault-domain separation.
- Start the new volume.
$ sudo gluster volume start volume1 volume start: volume1: success
Start the volume before mounting it, because client operations can hang when a stopped volume is mounted.
- Check the distribute layer and erasure-coded set count.
$ sudo gluster volume info volume1 Volume Name: volume1 Type: Distributed-Disperse Status: Started Snapshot Count: 0 Number of Bricks: 2 x (2 + 1) = 6 Transport-type: tcp Bricks: Brick1: node1:/srv/gluster/brick1/brick Brick2: node2:/srv/gluster/brick1/brick Brick3: node3:/srv/gluster/brick1/brick Brick4: node1:/srv/gluster/brick2/brick Brick5: node2:/srv/gluster/brick2/brick Brick6: node3:/srv/gluster/brick2/brick
- Review both disperse sets in the started volume.
$ sudo gluster volume status volume1 Status of volume: volume1 Gluster process TCP Port RDMA Port Online Pid ------------------------------------------------------------------------------- Brick node1:/srv/gluster/brick1/brick 49152 0 Y 2214 Brick node2:/srv/gluster/brick1/brick 49153 0 Y 2279 Brick node3:/srv/gluster/brick1/brick 49154 0 Y 2306 Brick node1:/srv/gluster/brick2/brick 49155 0 Y 2351 Brick node2:/srv/gluster/brick2/brick 49156 0 Y 2398 Brick node3:/srv/gluster/brick2/brick 49157 0 Y 2440 Task Status of Volume volume1 ------------------------------------------------------------------------------- There are no active volume tasks
Related: How to check GlusterFS volume status
- Mount the started volume from a client when the brick status is online.