How to troubleshoot Pacemaker quorum loss with PCS

Quorum loss in a Pacemaker cluster blocks safe resource management and commonly stops services to prevent split-brain. Restoring quorum returns predictable scheduling and failover behavior across the partition.

In a Pacemaker + Corosync stack, node membership and the votequorum subsystem determine whether a partition is allowed to start or move resources. The pcs CLI exposes the designated controller (DC), node membership, and quorum state, while corosync-quorumtool and corosync-cfgtool help validate vote counts and ring connectivity.

Temporary quorum loss typically points to node failure, network partition, or blocked/unstable Corosync traffic on the ring interface. Relaxing quorum protections (for example, no-quorum-policy set to ignore) can cause data divergence unless fencing is proven functional. Collect status and logs first, then restore node communication before changing quorum policy.

Steps to troubleshoot Pacemaker quorum loss with PCS:

Check the cluster status for quorum state.

$ sudo pcs status
Cluster name: clustername
Cluster Summary:
  * Stack: corosync (Pacemaker is running)
  * Current DC: node-01 (version 2.1.6-6fdc9deea29) - partition WITHOUT quorum
  * Last updated: Thu Jan  1 12:04:22 2026 on node-01
  * Last change:  Thu Jan  1 09:01:14 2026 by root via cibadmin on node-01
  * 4 nodes configured
  * 3 resource instances configured

Node List:
  * Online: [ node-01 ]
  * OFFLINE: [ node-02 node-03 node-04 ]

Full List of Resources:
  * fence-dummy-node-01	(stonith:fence_dummy):	 Stopped
  * fence-dummy-node-02	(stonith:fence_dummy):	 Stopped
  * fence-dummy-node-03	(stonith:fence_dummy):	 Stopped

Daemon Status:
  corosync: active/enabled
  pacemaker: active/enabled
  pcsd: active/enabled

partition WITHOUT quorum indicates insufficient votes to safely manage resources.

Inspect quorum and vote calculations from Corosync.

$ sudo corosync-quorumtool -s
Quorum information
------------------
Date:             Thu Jan  1 12:04:22 2026
Quorum provider:  corosync_votequorum
Nodes:            1
Node ID:          1
Ring ID:          1.4e
Quorate:          No

Votequorum information
----------------------
Expected votes:   3
Highest expected: 3
Total votes:      1
Quorum:           2 Activity blocked
Flags:            

Membership information
----------------------
    Nodeid      Votes Name
         1          1 node-01 (local)

Total votes below Quorum confirms a non-quorate partition.

List the nodes currently contributing votes to quorum.

$ sudo corosync-quorumtool -l

Membership information
----------------------
    Nodeid      Votes Name
         1          1 node-01 (local)

Review Corosync ring status on an online node.

$ sudo corosync-cfgtool -s
Local node ID 1, transport knet
LINK ID 0 udp
	addr	= 192.0.2.11
	status:
		nodeid:          1:	localhost
		nodeid:          2:	disconnected
		nodeid:          3:	disconnected

A ring that is not active or shows faults typically indicates broken Corosync traffic on the configured interface.

Review recent Corosync logs for membership changes or token timeouts.
```
$ sudo journalctl -u corosync --since "15 minutes ago" --no-pager | tail -n 20
-- No entries --
```
Frequent token timeout messages often correlate with packet loss, MTU mismatch, or blocked UDP traffic on the ring.

Verify node name resolution for the configured cluster nodes.

$ getent hosts node-02 node-03
192.0.2.12      node-02.example.net node-02 node-02 node-02
192.0.2.12      node-02.example.net node-02 node-02 node-02
192.0.2.12      node-02.example.net node-02 node-02 node-02
192.0.2.13      node-03.example.net node-03 node-03 node-03
192.0.2.13      node-03.example.net node-03 node-03 node-03
192.0.2.13      node-03.example.net node-03 node-03 node-03

Incorrect name resolution can split membership by sending Corosync traffic to an unexpected address or interface.

Verify quorum policy settings.
```
$ sudo pcs property show no-quorum-policy
no-quorum-policy: stop
```
Changing no-quorum-policy to ignore can permit split-brain unless fencing is validated and reliable.
Restore connectivity for the missing nodes.

Common causes include node power loss, corosync stopped on the missing node, firewall rules blocking UDP 5404/5405, packet loss, MTU mismatch, or a divergent /etc/corosync/corosync.conf.

Recheck the cluster status to confirm quorum returns.

$ sudo pcs status
Cluster name: clustername
Cluster Summary:
  * Stack: corosync (Pacemaker is running)
  * Current DC: node-01 (version 2.1.6-6fdc9deea29) - partition with quorum
  * Last updated: Thu Jan  1 12:04:36 2026 on node-01
  * Last change:  Thu Jan  1 09:01:14 2026 by root via cibadmin on node-01
  * 4 nodes configured
  * 3 resource instances configured

Node List:
  * Online: [ node-01 node-02 node-03 ]
  * OFFLINE: [ node-04 ]

Full List of Resources:
  * fence-dummy-node-01	(stonith:fence_dummy):	 Started node-01
  * fence-dummy-node-02	(stonith:fence_dummy):	 Started node-02
  * fence-dummy-node-03	(stonith:fence_dummy):	 Started node-03

Daemon Status:
  corosync: active/enabled
  pacemaker: active/enabled
  pcsd: active/enabled

Confirm Corosync reports a quorate partition and expected votes.

$ sudo corosync-quorumtool -s
Quorum information
------------------
Date:             Thu Jan  1 12:04:36 2026
Quorum provider:  corosync_votequorum
Nodes:            3
Node ID:          1
Ring ID:          1.56
Quorate:          Yes

Votequorum information
----------------------
Expected votes:   3
Highest expected: 3
Total votes:      3
Quorum:           2  
Flags:            Quorate 

Membership information
----------------------
    Nodeid      Votes Name
         1          1 node-01 (local)
         2          1 node-02
         3          1 node-03