Recently we did a small POC for SSD caching and we got some interesting results. One would believe that if you cache on local SSD inside the server you would get significant improvement in IO. Generally you will see some performance increase but not always like we experienced.
There are several things affecting the performance of local SSD. In this post I will cover SSD disks and disk controllers.
Most common SSD disks are 2.5″ SATA factor. SATA interface performance greatly limits throughput of a disk. Below in the table I’ve listed throughput speeds of different interfaces.
As you can see SATA1-SATA3 interfaces, SAS and SAS 6G interfaces are slower than widely adopted dual port 4Gbit FC, 8Gbit FC or 10G iSCSI (in the table above FC and iSCSI speeds are for single link). Only high end PCIe/NVMe devices can match or outperform the throughput performance of FC and modern iSCSI. If a workload generates large amount of throughput it could potentially slow down if caching device maximum throughput is slower then your original storage.
One way to increase local disk performance is to increase amount of disks. Example instead of one 800 GB disk use two 400 GB disks. Below in the table I calculated maximum read speed of all disks that you would need to create 800 GB cache. I took different sizes of Intel SSD DC S3710 Series disks.
When selecting a caching software make sure to check if it distributes cache to all SSD disks to maximize the performance. If the software does not distribute cache to all SSDs consider using striped (raid 0) disk as a cache device.
Most caching software vendors have recommendations about different SSD disks – check them out before purchasing the disks.
Disk controllers can have a huge affects to performance of SSD disks. Some entry level controllers may not perform as good as high end controllers. My advice here is to check your vendor recommendations for your disk controller or if you have not bought a controller yet check out the vendor recommendations and also caching software vendor recommendations.
Also disk controller settings can affect performance – check vendor recommendations. Some keywords – read/write cache, drive write cache, SSD pass-trough. In my experience enable drive write cache, change read/write ratio to 25% read / 75% write and if possible enable SSD pass-trough which enables OS to bypass disk controller.