How to set a quota on CephFS

CephFS directory quotas limit how much data or how many files a workload can place below one point in a shared filesystem. They are useful for tenant, project, or application trees that should not consume the capacity of the whole CephFS namespace.

CephFS stores quotas as virtual extended attributes on the directory. ceph.quota.max_bytes controls byte usage, ceph.quota.max_files controls file count, and the client that changes those attributes needs the MDS p capability in addition to read/write access.

Quota enforcement happens in the mounted client, so keep cluster capacity monitoring in place and use quotas as guardrails rather than a security boundary for hostile clients. With path-restricted client keys, place the quota on the restricted directory or a child directory that the mounted client can see.

1. Confirm the target path is on a CephFS mount.

   $ findmnt -T /mnt/cephfs/projects/app
   TARGET      SOURCE        FSTYPE       OPTIONS
   /mnt/cephfs ceph-fuse[/]  fuse.ceph-fuse rw,relatime,user_id=0,group_id=0,allow_other

   Replace /mnt/cephfs/projects/app with the directory that should receive the quota.

2. Inspect the quota-setting client's capabilities.

   $ ceph auth get client.app-quota
   [client.app-quota]
           key = ##### snipped #####
           caps mds = "allow rwp fsname=cephfs path=/projects/app"
           caps mon = "allow r fsname=cephfs"
           caps osd = "allow rw tag cephfs data=cephfs"

   The p flag allows the client to set CephFS layout and quota attributes. Add it from an admin shell before setting quotas if the mds cap shows only rw.
   Related: How to create a path-limited CephFS client key

3. Set the byte quota on the directory.

   $ setfattr -n ceph.quota.max_bytes -v 104857600 /mnt/cephfs/projects/app

   The value above sets a 100 MiB recursive limit for the directory tree. Use the byte value that matches the workload policy.

4. Set a file-count quota when the directory also needs a file limit.

   $ setfattr -n ceph.quota.max_files -v 10000 /mnt/cephfs/projects/app

   Skip this step when only byte usage should be limited.

5. Verify the byte quota attribute by name.

   $ getfattr -n ceph.quota.max_bytes /mnt/cephfs/projects/app
   # file: mnt/cephfs/projects/app
   ceph.quota.max_bytes="104857600"

   CephFS clients hide ceph.* virtual attributes from generic extended-attribute listings, so read each quota attribute by its exact name.

6. Verify the file-count quota attribute by name.

   $ getfattr -n ceph.quota.max_files /mnt/cephfs/projects/app
   # file: mnt/cephfs/projects/app
   ceph.quota.max_files="10000"

7. Write a bounded file below the byte quota.

   $ dd if=/dev/zero of=/mnt/cephfs/projects/app/quota-test-under.bin bs=1M count=10 status=none

   Use a test size safely below the configured quota so the check does not disrupt application data.

8. Confirm the under-limit test file exists.

   $ ls -lh /mnt/cephfs/projects/app/quota-test-under.bin
   -rw-r--r-- 1 root root 10M Jun 29 10:12 /mnt/cephfs/projects/app/quota-test-under.bin

9. Attempt a bounded write that exceeds the byte quota.

   $ dd if=/dev/zero of=/mnt/cephfs/projects/app/quota-test-over.bin bs=1M count=120
   dd: error writing '/mnt/cephfs/projects/app/quota-test-over.bin': Disk quota exceeded
   ##### snipped #####

   CephFS quota enforcement is cooperative and can stop a writer shortly after the configured limit is crossed, so a partial over-limit file can remain until it is removed.

10. Remove the quota test files.

    $ rm -f /mnt/cephfs/projects/app/quota-test-under.bin /mnt/cephfs/projects/app/quota-test-over.bin