How to create a Pacemaker cluster

After installing cluster components such as Pacemaker, Corosync, and pcs, the first thing to do is to create a cluster. It then enables us to create and assign resources such as shared IP addresses or server services such as MySQL to our cluster.

A typical cluster consists of at least 3 nodes, though creating a cluster from just two nodes is still possible. It avoids a condition called split-brain where each node in a cluster thinks that it is the only active node and causes unexpected behavior in a cluster system. The condition is a lot less likely the more nodes there are in a cluster.

A Pacemaker cluster could be created from the terminal using the pcs (Pacemaker/Corosync Configuration System) tool.

1. Launch terminal application on all nodes for the cluster.
2. Install pacemaker, corosync and pcs on all of your cluster nodes.

   Related: How to install Pacemaker, Corosync and pcs on CentOS, Red Hat or Fedora
   Related: How to install Pacemaker, Corosync and pcs on Ubuntu or Debian

3. Open hosts file using your favorite txt editor on all the cluster nodes.

   $ sudo vi /etc/hosts
   [sudo] password for user:

4. Add node's IP and host name information for all cluster nodes to setup a local DNS system.

   192.168.111.11 node-01
   192.168.111.12 node-02

   Cluster nodes are to communicate using assigned name, and it is crucial for each nodes to use static IP address to prevent communication error between the nodes.

5. Test local DNS setup by pinging each nodes from each other using the configured host name.

   $ ping -c3 node-01 && ping -c3 node-01
   PING node-01 (192.168.111.11) 56(84) bytes of data.
   64 bytes from node-01 (192.168.111.11): icmp_seq=1 ttl=64 time=0.061 ms
   64 bytes from node-01 (192.168.111.11): icmp_seq=2 ttl=64 time=0.062 ms
   64 bytes from node-01 (192.168.111.11): icmp_seq=3 ttl=64 time=0.133 ms
   
   --- node-01 ping statistics ---
   3 packets transmitted, 3 received, 0% packet loss, time 2001ms
   rtt min/avg/max/mdev = 0.061/0.085/0.133/0.034 ms
   PING node-01 (192.168.111.11) 56(84) bytes of data.
   64 bytes from node-01 (192.168.111.11): icmp_seq=1 ttl=64 time=0.050 ms
   64 bytes from node-01 (192.168.111.11): icmp_seq=2 ttl=64 time=0.073 ms
   64 bytes from node-01 (192.168.111.11): icmp_seq=3 ttl=64 time=0.070 ms
   
   --- node-01 ping statistics ---
   3 packets transmitted, 3 received, 0% packet loss, time 2002ms
   rtt min/avg/max/mdev = 0.050/0.064/0.073/0.012 ms

6. Auth node.

   $ sudo pcs host auth node-01 node-02 node-03
   Username: hacluster
   Password:
   node-01: Authorized
   node-03: Authorized
   node-02: Authorized

7. Authorise all nodes from one of the cluster nodes.

   $ sudo pcs cluster auth node-01 node-02
   Username: hacluster
   Password:
   node-02: Authorized
   node-01: Authorized

8. Create a cluster specifying all the nodes as member from one of the cluster nodes.

   $ sudo pcs cluster setup --name clustername node-01 node-02
   Destroying cluster on nodes: node-01, node-02...
   node-02: Stopping Cluster (pacemaker)...
   node-01: Stopping Cluster (pacemaker)...
   node-01: Successfully destroyed cluster
   node-02: Successfully destroyed cluster
   
   Sending 'pacemaker_remote authkey' to 'node-01', 'node-02'
   node-01: successful distribution of the file 'pacemaker_remote authkey'
   node-02: successful distribution of the file 'pacemaker_remote authkey'
   Sending cluster config files to the nodes...
   node-01: Succeeded
   node-02: Succeeded
   
   Synchronizing pcsd certificates on nodes node-01, node-02...
   node-02: Success
   node-01: Success
   Restarting pcsd on the nodes in order to reload the certificates...
   node-02: Success
   node-01: Success

9. Start cluster for all the cluster nodes from any of the cluster nodes.

   $ sudo pcs cluster start --name clustername --all
   node-01: Starting Cluster (corosync)...
   node-02: Starting Cluster (corosync)...
   node-01: Starting Cluster (pacemaker)...
   node-02: Starting Cluster (pacemaker)...

10. Configure cluster options from any of the cluster nodes.

    $ sudo pcs property set stonith-enabled=false 
    $ sudo pcs property set no-quorum-policy=ignore 

11. Show cluster info to confirm a successful setup from any of the cluster nodes.

    $ sudo pcs cluster status
    Cluster Status:
     Stack: corosync
     Current DC: node-02 (version 1.1.20-5.el7_7.1-3c4c782f70) - partition with quorum
     Last updated: Mon Sep 30 04:40:20 2019
     Last change: Mon Sep 30 04:39:46 2019 by root via cibadmin on node-01
     2 nodes configured
     0 resources configured
    
    PCSD Status:
      node-02: Online
      node-01: Online