How to enable the Ceph PG autoscaler

Placement groups are the units Ceph uses to distribute a pool across OSDs. When pool sizes change, manually chosen PG counts can become too small or too large, which leaves data placement less balanced than the cluster can support.

The PG autoscaler runs in the manager layer and compares pool usage, replication or erasure-coding rate, raw capacity, and target-size hints. Pool mode on lets Ceph adjust the PG count after its recommendation crosses the autoscaler threshold, while warn only raises recommendations and off ignores the pool.

Enable autoscaling on a pool whose application tag and expected size are known, and keep the cluster-wide noautoscale flag clear when automatic changes should run. Increasing PGs can trigger peering and data movement, so begin from HEALTH_OK and watch the affected PGs return to active+clean.

Related: How to create a replicated pool in Ceph
Related: How to troubleshoot stuck placement groups in Ceph

Steps to enable the Ceph PG autoscaler:

  1. Check the cluster health before changing pool PG behavior. 
    $ ceph -s
  cluster:
    id:     11111111-2222-3333-4444-555555555555
    health: HEALTH_OK

  services:
    mon: 3 daemons, quorum ceph-node1,ceph-node2,ceph-node3
    mgr: ceph-node1(active), standbys: ceph-node2
    osd: 9 osds: 9 up, 9 in

  data:
    pools:   3 pools, 73 pgs
    objects: 262.15k objects, 1.1 TiB
    usage:   3.4 TiB used, 56 TiB / 60 TiB avail
    pgs:     73 active+clean

    Delay autoscaler changes when the cluster already reports degraded, remapped, backfilling, or stuck PGs. A PG count change adds more placement work to the same OSD set.

    Related: How to check Ceph cluster health

  2. Review autoscaler recommendations for all pools. 
    $ ceph osd pool autoscale-status
POOL                   SIZE  TARGET SIZE  RATE  RAW CAPACITY  RATIO   FINAL RATIO  TARGET RATIO  EFFECTIVE RATIO  BIAS  PG_NUM  NEW PG_NUM  AUTOSCALE  BULK
device_health_metrics     0                3.0         60 TiB  0.0000        0.0312                                  1.0       1              on         False
rbd                    1 TiB                3.0         60 TiB  0.0500        0.2500                                  1.0      64         256  warn       False
metadata              16 GiB                3.0         60 TiB  0.0008        0.0312                                  1.0       8              on         False

    NEW PG_NUM shows the PG count Ceph recommends when it differs enough from the current PG_NUM. Blank values usually mean the current PG count is inside the autoscaler's tolerance for that pool.

  3. Check whether the cluster-wide autoscaler block is set. 
    $ ceph osd pool get noautoscale
noautoscale: false

    The noautoscale flag disables PG autoscaling across pools even when an individual pool mode is on.

  4. Clear the cluster-wide autoscaler block when the previous command returns true. 
    $ ceph osd pool unset noautoscale
  5. Set a target ratio for the pool when its expected share of cluster data is known. 
    $ ceph osd pool set rbd target_size_ratio 0.70
set pool 4 target_size_ratio to 0.7

    Use ratios across pools that should share the same raw capacity budget. Omit this setting when the pool already contains enough representative data for the autoscaler to judge from current usage.

    Do not set both target_size_ratio and target_size_bytes on the same pool. Ceph uses the ratio when both exist and raises a health warning for the conflicting hints.

  6. Enable automatic PG scaling for the pool. 
    $ ceph osd pool set rbd pg_autoscale_mode on
set pool 4 pg_autoscale_mode to on

    Use warn instead of on when change approval must stay manual. Use off only for pools whose PG count is intentionally managed outside the autoscaler.

  7. Confirm the pool mode changed to on. 
    $ ceph osd pool get rbd pg_autoscale_mode
pg_autoscale_mode: on
  8. Review the autoscaler status after enabling the pool. 
    $ ceph osd pool autoscale-status
POOL                   SIZE  TARGET SIZE  RATE  RAW CAPACITY  RATIO   FINAL RATIO  TARGET RATIO  EFFECTIVE RATIO  BIAS  PG_NUM  NEW PG_NUM  AUTOSCALE  BULK
device_health_metrics     0                3.0         60 TiB  0.0000        0.0312                                  1.0       1              on         False
rbd                    1 TiB                3.0         60 TiB  0.0500        0.7000         0.7000            0.7000   1.0      64         256  on         False
metadata              16 GiB                3.0         60 TiB  0.0008        0.0312                                  1.0       8              on         False

    The pool is enabled when AUTOSCALE shows on. If NEW PG_NUM remains populated, Ceph still has a pending or in-progress PG count change for that pool.

  9. Watch cluster health until any PG adjustment finishes. 
    $ ceph -s
  cluster:
    id:     11111111-2222-3333-4444-555555555555
    health: HEALTH_OK

  services:
    mon: 3 daemons, quorum ceph-node1,ceph-node2,ceph-node3
    mgr: ceph-node1(active), standbys: ceph-node2
    osd: 9 osds: 9 up, 9 in

  data:
    pools:   3 pools, 265 pgs
    objects: 262.15k objects, 1.1 TiB
    usage:   3.4 TiB used, 56 TiB / 60 TiB avail
    pgs:     265 active+clean

    When the PG count settles, the target pool should show AUTOSCALE on and the cluster should return to HEALTH_OK with PGs in active+clean state.