Message availability depends on both reachable brokers and surviving queue replicas. In a RabbitMQ active-active deployment, PCS can keep a broker service running on each cluster node while RabbitMQ handles message replication and queue-leader elections.

The systemd:rabbitmq-server resource controls the installed service, and a Pacemaker clone gives each node its own instance. It adopts an already formed RabbitMQ cluster with matching Erlang cookies and separate local data directories; cloning three unrelated brokers does not join them or copy their messages. Its service monitor also does not establish that a queue can accept or deliver messages.

A three-member quorum queue can remain available when one broker stops because the other two replicas retain a majority. Client applications still need multiple broker addresses or a health-checked load balancer, reconnect handling, publisher confirms, and consumer acknowledgements. A controlled node-standby test checks service withdrawal and message survival; power-loss fencing and application reconnection need their own tests.

Steps to set up RabbitMQ active-active with PCS:

  1. Confirm that all three Pacemaker nodes are online with quorum.
    $ sudo pcs status
    Cluster name: clustername
    Cluster Summary:
      * Stack: corosync (Pacemaker is running)
      * Current DC: node-01 (version 3.0.1-3.0.1) - partition with quorum
      * Last updated: Mon Nov  9 15:10:11 2026 on node-01
    ##### snipped #####
    Node List:
      * Online: [ node-01 node-02 node-03 ]
    
    Full List of Resources:
      * fence-ipmi-node-01	(stonith:fence_ipmilan):	 Started node-02
      * fence-ipmi-node-02	(stonith:fence_ipmilan):	 Started node-03
      * fence-ipmi-node-03	(stonith:fence_ipmilan):	 Started node-01
    ##### snipped #####

    The nodes here are node-01, node-02, and node-03, with RabbitMQ already installed as rabbitmq-server.service. Cluster commands run on node-01 unless another node is named. A listed fence resource is not proof that power control works; tested fencing is a prerequisite.
    Related: How to check Pacemaker cluster status
    Related: How to create a STONITH resource in Pacemaker

  2. Confirm that RabbitMQ reports the same three running cluster members.
    $ sudo rabbitmqctl cluster_status
    Cluster status of node rabbit@node-01 ...
    ##### snipped #####
    Disk Nodes
    
    rabbit@node-01
    rabbit@node-02
    rabbit@node-03
    
    Running Nodes
    
    rabbit@node-01
    rabbit@node-02
    rabbit@node-03
    ##### snipped #####

    The remaining commands use PCS 0.12 clone syntax and a RabbitMQ 4.x quorum queue. The messaging test needs the management plugin reachable locally at 127.0.0.1:15672 on each broker, curl, Python 3, and a test account named ha-check with the management tag. Its permissions belong only to a dedicated ha-check virtual host, with configure/read access to pcs-check and write access to the default exchange. Existing application queues and credentials stay separate.

    The handover changes who starts RabbitMQ after boot. A maintenance window allows recovery if cluster management interrupts broker connections. If the handover must be reversed, removing rabbitmq_service-clone through PCS stops its brokers; sudo systemctl enable --now rabbitmq-server.service on each node then restores systemd startup. RabbitMQ data directories must remain intact.
    Related: How to delete a Pacemaker resource
    Related: How to manage Pacemaker cluster services with systemctl in Linux

  3. Disable independent RabbitMQ boot startup on every broker node.
    $ sudo systemctl disable rabbitmq-server.service

    The same command applies on all three nodes. Without --now it leaves the running brokers available for Pacemaker to discover. The service remains unmasked so Pacemaker can start it.

  4. Create the RabbitMQ service and its clone in one PCS operation.
    $ sudo pcs resource create rabbitmq_service systemd:rabbitmq-server \
      op start timeout=180s op stop timeout=120s \
      op monitor interval=30s clone --future

    The default clone placement allows one instance per cluster node. Creating the clone together with its service avoids an intermediate singleton resource that could stop the other brokers. The --future flag selects the PCS 0.12 syntax.
    Related: How to create a Pacemaker resource
    Related: How to create a cloned resource in Pacemaker
    Related: How to set Pacemaker clone resource options

  5. Check that the RabbitMQ clone is started on all three nodes.
    $ sudo pcs status resources
      * Clone Set: rabbitmq_service-clone [rabbitmq_service]:
        * Started: [ node-01 node-02 node-03 ]

    A Starting, Stopped, or failed instance means the handover is incomplete. If startup is still in progress, the status can be checked again after it settles.

  6. Declare a temporary quorum queue through the local API on node-01.
    $ curl --fail --silent --show-error --user ha-check \
      -H 'content-type: application/json' -X PUT \
      --data '{"durable":true,"arguments":{"x-queue-type":"quorum","x-expires":600000}}' \
      http://127.0.0.1:15672/api/queues/ha-check/pcs-check
    Enter host password for user 'ha-check':

    Curl prompts for the account password rather than placing it in shell history. The request creates a three-replica queue while all three brokers are running and prints no response body on success. The unused queue expires after ten minutes. HTTP is confined to loopback here; remote management access needs HTTPS.

  7. Publish a persistent test message through node-01.
    $ curl --fail --silent --show-error --user ha-check \
      -H 'content-type: application/json' -X POST \
      --data '{"properties":{"delivery_mode":2},"routing_key":"pcs-check","payload":"pcs-ha-message","payload_encoding":"string"}' \
      http://127.0.0.1:15672/api/exchanges/ha-check/amq.default/publish
    Enter host password for user 'ha-check':
    {"routed":true}

    The routed value must be true. The HTTP API provides a small messaging check; it does not replace an AMQP application test.

    Node standby evacuates every Pacemaker resource from node-01 and disconnects clients using that broker. The remaining two RabbitMQ nodes must stay online to preserve queue quorum. If the test fails, sudo pcs node unstandby node-01 restores the node's eligibility to run resources.

  8. Put node-01 into standby from node-02.
    $ sudo pcs node standby node-01

    This withdraws one broker under cluster control without disabling fencing or stopping the other brokers.
    Related: How to run a Pacemaker failover test with PCS

  9. Check that only node-01's RabbitMQ instance has stopped.
    $ sudo pcs status resources
      * Clone Set: rabbitmq_service-clone [rabbitmq_service]:
        * Started: [ node-02 node-03 ]
        * Stopped: [ node-01 ]

    The messaging check belongs after this state appears; otherwise the first broker might still be serving the queue.

  10. Retrieve the retained message through node-02.
    $ curl --fail --silent --show-error --user ha-check \
      -H 'content-type: application/json' -X POST \
      --data '{"count":1,"ackmode":"ack_requeue_true","encoding":"auto"}' \
      http://127.0.0.1:15672/api/queues/ha-check/pcs-check/get | python3 -m json.tool
    Enter host password for user 'ha-check':
    [
        {
            "payload_bytes": 14,
            "redelivered": false,
    ##### snipped #####
            "payload": "pcs-ha-message",
            "payload_encoding": "string"
        }
    ]

    The returned pcs-ha-message shows that the queue remains usable after the first broker withdraws. A brief leader election can delay the response; [] or an HTTP error does not prove message survival. The ack_requeue_true mode puts this test message back for the final check.

  11. Return node-01 to service from node-02.
    $ sudo pcs node unstandby node-01
  12. Confirm that Pacemaker has restarted the RabbitMQ clone on node-01.
    $ sudo pcs status resources
      * Clone Set: rabbitmq_service-clone [rabbitmq_service]:
        * Started: [ node-01 node-02 node-03 ]

    The expected state is one started instance on each node, with no failed actions. If node-01 is still starting, it has not yet completed recovery.

  13. Consume the test message through node-03 after all brokers return.
    $ curl --fail --silent --show-error --user ha-check \
      -H 'content-type: application/json' -X POST \
      --data '{"count":1,"ackmode":"ack_requeue_false","encoding":"auto"}' \
      http://127.0.0.1:15672/api/queues/ha-check/pcs-check/get | python3 -m json.tool
    Enter host password for user 'ha-check':
    [
        {
            "payload_bytes": 14,
            "redelivered": true,
    ##### snipped #####
            "payload": "pcs-ha-message",
            "payload_encoding": "string"
        }
    ]

    The returned pcs-ha-message is readable through the third broker after recovery. This request acknowledges and removes only the test message; the clone remains running on all three nodes.

Share this guide