How to benchmark MariaDB and MySQL performance

Benchmarking MariaDB or MySQL before a hardware change, parameter tune, or version upgrade shows whether throughput improved, latency regressed, or the bottleneck simply moved to storage, CPU, or connection handling.

Sysbench uses bundled oltp_*.lua workloads to create a repeatable schema, run a controlled query mix over the MySQL protocol, and report transactions per second, query rate, reconnects, and latency. That makes it practical to compare one server version, storage layout, or parameter set against another with the same dataset, thread count, and runtime.

Connection path and workload shape matter as much as raw speed. Local localhost logins on Unix-like systems often use a socket, while explicit TCP tests use 127.0.0.1 or another host plus a port, and some MySQL deployments with skip_name_resolve need an IP-based account for predictable authentication. Keep the connection method, table size, thread count, and runtime identical across every comparison, and remove the sbtest objects when measurements are complete.

1. Run the benchmark on a staging clone or a dedicated test instance.

   An OLTP benchmark can saturate CPU, memory, and disk I/O, so do not run aggressive tests on a busy production primary without an approved window and rollback plan.

2. Create a dedicated sbtest database and TCP benchmark user.

   $ mysql --user=root --password --execute="CREATE DATABASE sbtest; CREATE USER 'sbtest'@'127.0.0.1' IDENTIFIED BY '<password>'; GRANT ALL PRIVILEGES ON sbtest.* TO 'sbtest'@'127.0.0.1';"
   Enter password:

   Many MariaDB packages authenticate local root over the Unix socket, so sudo mariadb or sudo mysql can be the correct administrative entry point.

   These examples use explicit TCP to 127.0.0.1. If the benchmark should use a local Unix socket instead, create the matching 'sbtest'@'localhost' account and replace host or port options with --mysql-socket=/path/to/mysql.sock. On MySQL, a connection to 127.0.0.1 usually resolves to the localhost account unless skip_name_resolve is enabled, but an explicit IP-based account keeps the TCP path predictable across both engines.

3. Prepare a repeatable read-only dataset sized for the comparison.

   $ sysbench --db-driver=mysql --mysql-host=127.0.0.1 --mysql-port=3306 --mysql-user=sbtest --mysql-password='<password>' --mysql-db=sbtest --tables=2 --table-size=10000 oltp_read_only prepare
   Creating table 'sbtest1'...
   Inserting 10000 records into 'sbtest1'
   Creating a secondary index on 'sbtest1'...
   Creating table 'sbtest2'...
   Inserting 10000 records into 'sbtest2'
   Creating a secondary index on 'sbtest2'...

   Increase --tables and --table-size until the working set and index size resemble the system being compared, otherwise the run mostly measures a tiny hot cache.

   Large datasets consume disk quickly and can take much longer to build, especially on slower storage tiers.

4. Run a fixed-duration baseline with explicit concurrency and interval reporting.

   $ sysbench --db-driver=mysql --mysql-host=127.0.0.1 --mysql-port=3306 --mysql-user=sbtest --mysql-password='<password>' --mysql-db=sbtest --tables=2 --table-size=10000 --threads=4 --time=10 --report-interval=5 oltp_read_only run
   Running the test with following options:
   Number of threads: 4
   Report intermediate results every 5 second(s)
   ##### snipped #####
   
   [ 5s ] thds: 4 tps: 430.76 qps: 6900.33 (r/w/o: 6038.02/0.00/862.32) err/s: 0.00 reconn/s: 0.00
   [ 10s ] thds: 4 tps: 568.31 qps: 9087.10 (r/w/o: 7950.49/0.00/1136.61) err/s: 0.00 reconn/s: 0.00
   SQL statistics:
       queries performed:
           read:                            70000
           write:                           0
           other:                           10000
           total:                           80000
       transactions:                        5000   (499.43 per sec.)
       queries:                             80000  (7990.81 per sec.)
       ignored errors:                      0      (0.00 per sec.)
       reconnects:                          0      (0.00 per sec.)
   General statistics:
       total time:                          10.0113s
       total number of events:              5000
   Latency (ms):
            avg:                                    8.01
            max:                                  149.12

   Use transactions per second, queries per second, zero ignored errors, and zero reconnects as the first comparison points before tuning around latency.

5. Save each result to a file before changing server settings or rerunning the test.

   $ sysbench --db-driver=mysql --mysql-host=127.0.0.1 --mysql-port=3306 --mysql-user=sbtest --mysql-password='<password>' --mysql-db=sbtest --tables=2 --table-size=10000 --threads=4 --time=10 --report-interval=5 oltp_read_only run > mariadb-read-only-4t-10s.txt
   $ grep -E 'transactions:|queries:|ignored errors:|reconnects:' mariadb-read-only-4t-10s.txt
       transactions:                        5000   (499.43 per sec.)
       queries:                             80000  (7990.81 per sec.)
       ignored errors:                      0      (0.00 per sec.)
       reconnects:                          0      (0.00 per sec.)

   Keep the workload name, thread count, and runtime in the filename so later comparisons remain readable.

6. Repeat the same scenario several times and compare medians rather than trusting one pass.

   Warm-cache tests and cold-cache tests are both valid, but the numbers only stay comparable when every run starts from the same cache state, connection path, and server settings.

7. Switch to a mixed read/write workload when the target system handles inserts, updates, or deletes under load.

   $ sysbench --db-driver=mysql --mysql-host=127.0.0.1 --mysql-port=3306 --mysql-user=sbtest --mysql-password='<password>' --mysql-db=sbtest --tables=2 --table-size=10000 --threads=4 --time=10 --report-interval=5 oltp_read_write run

   Small tables or unrealistically hot datasets can produce lock conflicts and ignored errors in oltp_read_write, so grow the dataset until the workload resembles the real working set before comparing one server against another.

8. Remove the sysbench tables when the measurements are complete.

   $ sysbench --db-driver=mysql --mysql-host=127.0.0.1 --mysql-port=3306 --mysql-user=sbtest --mysql-password='<password>' --mysql-db=sbtest --tables=2 oltp_read_only cleanup
   Dropping table 'sbtest1'...
   Dropping table 'sbtest2'...

9. Drop the benchmark database and account after cleanup.

   $ mysql --user=root --password --execute="DROP DATABASE sbtest; DROP USER 'sbtest'@'127.0.0.1';"
   Enter password:

   Drop the actual host-scoped account used for the run, such as 'sbtest'@'localhost' for socket tests or another client IP for a remote benchmark host.

10. Confirm that the benchmark database is gone.

    $ mysql -vvv --user=root --password --execute="SHOW DATABASES LIKE 'sbtest';"
    Enter password:
    --------------
    SHOW DATABASES LIKE 'sbtest'
    --------------
    
    Empty set (0.00 sec)
    
    Bye

1. Run the benchmark client from a separate host when the goal is end-to-end server capacity rather than single-host loopback performance.
2. Record the server version, storage class, CPU count, buffer pool size, and any changed variables next to each result file.
3. Change one variable at a time so a faster or slower run can be explained afterward.
4. Keep backup jobs, replica catch-up bursts, schema migrations, and other noisy background work out of the benchmark window when possible.
5. Use the same sysbench script and the same data size on MySQL and MariaDB so the comparison reflects the server rather than the workload generator.