Using mostly the same test methodology as described in an earlier note I compared MySQL 5.1 and 5.5 for an IO-bound workload. The biggest difference is that I tested 8 tables with 64M rows each and about 128G data total. I still used a 16G InnoDB buffer pool and the mysqld servers were IO bound. Two test servers were used -- one that can do ~1600 random disk reads per second via 8 SAS disks and the other that can do 35,000 16kb page reads per second via 1 PCI-based flash card.
The same tests were run. The read-only test does one query per transaction that fetches one row by primary key. The read-write test does one fetch and one update by primary key.
Results are reported as described in the previous note. Eight instances of sysbench are run concurrently and each uses a different table. Each instance uses 1, 2, 4, 8, 16, 32 and then 64 threads so that the mysqld server handles 8, 16, 32, 64, 128, 256, 512 and then 1024 concurrent connections. In most cases below I only report results up to 256 or 512 connections to avoid long lines. The results are the number of transactions per second. The read-only test does one query per transaction. The read-write test does one query and one update per transaction.
It is also interesting to consider the peak QPS/IOPs ratio for the servers where IOPs capacity is ~1600 for the SAS server and ~35,000 for the flash server. There is one query per transaction for the read-only test and two for the read-write test. The ratios for the read-only test, MySQL 5.1.52 + Facebook patch and innodb_thread_concurrency=0 at 128 concurrent users are 2.4 for SAS and 1.1 for flash. The ratios for the read-write test with innodb_thread_concurrency=32 and the doublewrite buffer enabled are 1.0 for SAS and 0.22 for flash. With the doublewrite buffer disabled the ratio for the read-write test is 0.28 for flash.
The binlog was disabled for all tests as I don't want to encounter bottlenecks from not having group commit.
Release innodb_io_concurrency tickets on disk reads
Domas was right (again). For the read-only test, throughput is higher with innodb_thread_concurrency=0. This test is IO bound and with innodb_thread_concurrency=32 there are at most 32 pending disk reads because the concurrency tickets are not released by threads that block for a read (they are released on row lock waits).
These are results for the SAS server. The server can do ~1600 random IOPs and was able to do more QPS than IOPs because some queries in the the db buffer pool and because (NCQ + request scheduling) means that the disk can do more than 1600 IOPs with a highly concurrent workload at the cost of additional latency.
Results for the read-only test with innodb_thread_concurrency=0.
8 16 32 64 128 256 512 threads
1069 1691 2483 3148 3745 3770 3776 mysql5147
1116 1759 2531 3214 3822 3851 3849 mysql5152
1103 1734 2426 3070 3654 3686 3687 mysql558
Results for the read-only test with innodb_thread_concurrency=32.
1076 1693 2483 2517 2512 2515 2514 mysql5147
1118 1754 2527 2556 2558 2558 2557 mysql5152
1111 1730 2423 2455 2456 2451 2455 mysql558
Results for the flash server that can do ~35,000 IOPs. Results are still better with innodb_thread_concurrency=0 but the difference is less than for SAS.
Results for the read-only test with innodb_thread_concurrency=0.
16121 20102 26877 30743 30182 28883 25077 mysql5147
16310 20961 29217 35984 37213 33721 28927 mysql5152
16352 19890 26409 29854 29289 27796 24464 mysql558
Results for the read-only test with innodb_thread_concurrency=32.
16907 21624 26764 26676 26009 24874 24378 mysql5147
19109 23596 28597 28243 29199 28178 27292 mysql5152
18774 21131 26397 26143 25245 23824 23551 mysql558
innodb_io_concurrency=0 is still a problem for read-write tests
For IO bound workloads and the read-write test, performance degrades at high concurrency with innodb_io_concurrency=0. It did not degrade for read-only tests. Since it is still not always safe to use innodb_thread_concurrency=0, this is another reason to release concurrency tickets on disk reads.
Results for the SAS server.
Results with innodb_io_concurrency=0
8 16 32 64 128 256 512 1024 threads
493 613 782 936 974 1000 178 66 mysql5147
515 628 798 961 1022 1023 157 59 mysql5152
495 608 767 922 999 996 150 1055 mysql558
Results with innodb_io_concurrency=32
8 16 32 64 128 256 512 1024 threads
483 615 781 781 784 784 784 785 mysql5147
508 629 799 802 801 802 800 802 mysql5152
487 607 768 772 770 769 770 769 mysql558
Results for the flash server.
Flash results with innodb_thread_concurrency=0
2919 2641 2619 2576 2606 989 194 78 mysql5147
3742 3721 3796 3829 3835 608 128 119 mysql5152
2890 2670 2667 2642 2612 425 99 70 mysql558
Flash results with innodb_thread_concurrency=32
3218 2875 2793 2715 2680 2655 2602 2558 mysql5147
5119 4327 4113 3958 3850 3744 3672 3646 mysql5152
2922 2761 2788 2738 2699 2655 2635 2610 mysql558
The doublewrite buffer can reduce performance
Peak TPS is better with the doublewrite buffer disabled for flash. This is not an issue for SAS although it might be if I had more IOPs capacity. But don't run without it unless you can recreate your data on a crash.
SAS results with innodb_thread_concurrency=32:
8 16 32 64 128 256 512 1024 threads
508 629 799 802 801 802 800 802 doublewrite enabled
516 630 801 803 803 804 804 805 doublewrite disabled
Flash results with innodb_thread_concurrency=32:
8 16 32 64 128 256 512 threads
5119 4327 4113 3958 3850 3744 3672 doublewrite enabled
4805 5096 5194 5117 4985 4813 4736 doublewrite disabled
PlanetMySQL Voting: Vote UP / Vote DOWN