How to create a GlusterFS geo-replication session

Creating a geo-replication session keeps a secondary copy of a GlusterFS volume updated asynchronously, providing a practical recovery path when the primary cluster becomes unavailable. Replication across sites reduces recovery time for file services that cannot tolerate long outages.

Geo-replication is driven by gsyncd on the primary cluster, which tracks file changes and synchronizes them to the secondary cluster over SSH using an rsync-based pipeline. The session is configured from the primary cluster by pairing a primary volume with a secondary target in the form user@host::volume, then distributing keys so each primary node can reach the secondary nodes.

Reliable session control depends on synchronized clocks and passwordless SSH for the chosen replication account. Matching UID/GID mappings between clusters avoids unexpected ownership changes, and the secondary volume should be empty before the first start to avoid unexpected reconciliation. Session creation only registers the relationship until the session is started.

1. Confirm the primary volume is started on a primary cluster node.

   $ sudo gluster volume status gvol-primary
   Status of volume: gvol-primary
   Gluster process                               TCP Port  RDMA Port  Online  Pid
   ------------------------------------------------------------------------------
   Brick pnode1.example.com:/bricks/gvol-primary  49152     0         Y       2149
   Brick pnode2.example.com:/bricks/gvol-primary  49152     0         Y       2198
   Self-heal Daemon on pnode1.example.com        N/A       N/A       Y       2287
   Self-heal Daemon on pnode2.example.com        N/A       N/A       Y       2331

   Related: How to create a GlusterFS volume
   Related: How to start and stop a GlusterFS volume

2. Confirm the secondary volume is started on a secondary cluster node.

   $ sudo gluster volume status gvol-secondary
   Status of volume: gvol-secondary
   Gluster process                                TCP Port  RDMA Port  Online  Pid
   ------------------------------------------------------------------------------
   Brick snode1.example.com:/bricks/gvol-secondary 49152     0         Y       1764
   Brick snode2.example.com:/bricks/gvol-secondary 49152     0         Y       1812
   Self-heal Daemon on snode1.example.com         N/A       N/A       Y       1901
   Self-heal Daemon on snode2.example.com         N/A       N/A       Y       1948

   Keep the secondary volume empty and idle until recovery workflows require access.

3. Create the geo-replication account on each secondary node.

   $ sudo useradd --create-home --shell /bin/bash geoaccount

   Account geoaccount must exist on all secondary nodes addressed by the session target, preferably with consistent UID/GID.

4. Confirm system time is synchronized on each node involved in the session.

   $ timedatectl show --property=NTPSynchronized --value
   yes

   Clock drift can prevent geo-replication state changes and trigger recurring sync failures.

5. Generate geo-replication SSH keys on a primary node.

   $ sudo gluster-georep-sshkey generate
   Generating SSH keys for geo-replication.

   Key material is managed under the glusterd state directory and is used by push-pem session creation.

6. Create the geo-replication session from the primary cluster using push-pem authentication.

   $ sudo gluster volume geo-replication gvol-primary geoaccount@snode1.example.com::gvol-secondary create push-pem
   geo-replication: session created

   Hostname snode1.example.com must resolve and accept SSH from every primary node participating in the session.

   Swapping primary and secondary volumes creates replication in the wrong direction.

7. Install the geo-replication .pem keys for the session on each secondary node.

   $ sudo /usr/libexec/glusterfs/set_geo_rep_pem_keys.sh geoaccount gvol-primary gvol-secondary
   Setting up geo-replication PEM keys for geoaccount... done.

   Some distributions install the helper under /usr/lib/glusterfs with the same script name.

8. Verify the new geo-replication session appears in the status output.

   $ sudo gluster volume geo-replication gvol-primary geoaccount@snode1.example.com::gvol-secondary status
   MASTER NODE          MASTER VOL     SLAVE                                   STATUS
   pnode1.example.com   gvol-primary    geoaccount@snode1.example.com::gvol-secondary  Stopped
   pnode2.example.com   gvol-primary    geoaccount@snode1.example.com::gvol-secondary  Stopped

   A newly created session commonly reports Stopped until started.