How to improve GlusterFS performance

A GlusterFS volume can remain online while one brick, volume layout, or translator setting slows directory listings, sync-heavy writes, or client metadata lookups. Performance work should begin with a short baseline and per-brick profile so cache or threading changes are tied to the workload that is actually slow.

GlusterFS serves files through a translator stack that includes replication, distribution, caching, write-behind, read-ahead, metadata cache, and directory-read behavior. Volume options can be changed while the cluster stays online, but the same option can help one workload and hurt another when it increases memory pressure, weakens ordering expectations, or hides the real bottleneck behind client cache.

Capacity and layout changes affect performance as much as individual options. Adding bricks can increase parallelism, while heal and rebalance tasks add background disk and network I/O, so tune during controlled windows and verify that peers stay connected and bricks remain online after each change.

Steps to improve GlusterFS volume performance:

Open a terminal on a node that belongs to the trusted storage pool.
Confirm the GlusterFS management service is running.
```
$ systemctl is-active glusterd
active
```
Some distributions use glusterfs-server.service instead of glusterd.service. Use the installed unit name when checking service state.
Related: How to manage the GlusterFS service with systemctl

Check peer connectivity before measuring performance.

$ sudo gluster peer status
Number of Peers: 2

Hostname: node2
Uuid: 6770f88c-9ec5-4cf8-b9f5-658fa17b6bdc
State: Peer in Cluster (Connected)

Hostname: node3
Uuid: 5a3c65f3-1b4d-4d6e-93d4-4c24f0b6b5bf
State: Peer in Cluster (Connected)

Disconnected peers, down bricks, or active heal backlogs can look like a tuning problem because clients retry or wait for replica recovery.

Check the volume layout and brick count.

$ sudo gluster volume info volume1

Volume Name: volume1
Type: Distributed-Replicate
Volume ID: 9b9f2d4f-3c3a-4c3e-8c10-5b2f8b1f1e32
Status: Started
Snapshot Count: 0
Number of Bricks: 2 x 2 = 4
Transport-type: tcp
Bricks:
Brick1: node1:/srv/gluster/brick1
Brick2: node2:/srv/gluster/brick1
Brick3: node3:/srv/gluster/brick1
Brick4: node4:/srv/gluster/brick1
##### snipped #####

The layout controls where parallelism and redundancy come from. A replicated pair protects data, while distributed-replicate spreads files across replica sets.

Check live brick state for the target volume.

$ sudo gluster volume status volume1
Status of volume: volume1
Gluster process                             TCP Port  RDMA Port  Online  Pid
------------------------------------------------------------------------------
Brick node1:/srv/gluster/brick1             49152     0          Y       2143
Brick node2:/srv/gluster/brick1             49153     0          Y       2311
Brick node3:/srv/gluster/brick1             49154     0          Y       2449
Brick node4:/srv/gluster/brick1             49155     0          Y       2520

Task Status of Volume volume1
------------------------------------------------------------------------------
There are no active volume tasks

Online should show Y for every expected brick before tuning starts.
Related: How to check GlusterFS volume status

Start profiling for a short sampling window.
```
$ sudo gluster volume profile volume1 start
Starting volume profile on volume1 has been successful
```
Profiling adds brick CPU and I/O work, so keep the sampling window short and stop profiling when the measurement is complete.
Related: How to profile GlusterFS volume performance
Run the workload that reproduces the slow path.

Use the same application action, benchmark, directory listing, or file copy for the baseline and every later test run. Changing the workload between tests makes option changes impossible to compare.

Display the profiling report for the volume.

$ sudo gluster volume profile volume1 info
Brick: node1:/srv/gluster/brick1
----------------------------------
Cumulative Stats:
   Block Size:             131072b+
 No. of Reads:                    0
No. of Writes:                   64
 %-latency   Avg-latency   Min-Latency   Max-Latency   No. of calls         Fop
 ---------   -----------   -----------   -----------   ------------        ----
     18.13  142101.89 ns   19542.00 ns 1001084.00 ns              9      LOOKUP
     71.81   79132.11 ns   42458.00 ns  626542.00 ns             64       WRITE

    Duration: 1 seconds
   Data Read: 0 bytes
Data Written: 8388608 bytes
##### snipped #####

Compare the busiest FOP rows across bricks. A single brick with higher latency or a workload dominated by LOOKUP, READDIRP, WRITE, or FSYNC points to a more specific tuning target.

Stop profiling after the baseline sample.

$ sudo gluster volume profile volume1 stop
Stopping volume profile on volume1 has been successful

Review current performance options before changing anything.

$ sudo gluster volume get volume1 all
Option                                   Value
------                                   -----
performance.write-behind                 on
performance.read-ahead                   off
performance.readdir-ahead                off
performance.io-cache                     off
performance.quick-read                   on
performance.nl-cache                     off
performance.client-io-threads            on
performance.parallel-readdir             off
##### snipped #####

Choose one option that matches the measured bottleneck.

performance.readdir-ahead and performance.parallel-readdir target directory scans, performance.md-cache and metadata cache groups target repeated stat/xattr lookups, and write-behind settings affect write acknowledgement behavior. Avoid changing unrelated options in the same test window.
Apply one option change to the volume.
```
$ sudo gluster volume set volume1 performance.readdir-ahead on
volume set: success
```
Apply one change at a time and record the baseline numbers before moving to the next option. Multiple simultaneous changes make regressions hard to isolate.

Related: How to set GlusterFS volume options

Verify the stored option value.

$ sudo gluster volume get volume1 performance.readdir-ahead
Option                                   Value
------                                   -----
performance.readdir-ahead                on

Repeat the same workload and profile check after the change.

Keep the same client count, file set, mount path, and runtime window used for the baseline. Keep the option only when the measured latency or throughput improves without increasing errors, heal backlog, or client-side stalls.

Re-check brick state after the tuning pass.

$ sudo gluster volume status volume1
Status of volume: volume1
Gluster process                             TCP Port  RDMA Port  Online  Pid
------------------------------------------------------------------------------
Brick node1:/srv/gluster/brick1             49152     0          Y       2143
Brick node2:/srv/gluster/brick1             49153     0          Y       2311
Brick node3:/srv/gluster/brick1             49154     0          Y       2449
Brick node4:/srv/gluster/brick1             49155     0          Y       2520

Task Status of Volume volume1
------------------------------------------------------------------------------
There are no active volume tasks

Schedule layout changes only after option tuning and health checks are clear.

Adding or removing bricks and running rebalance can reduce client performance while files are redistributed, so perform the work during a low-traffic window.
Related: How to add a brick to a GlusterFS volume
Related: How to rebalance a GlusterFS volume
Related: How to remove a brick from a GlusterFS volume