SSD caching could decrease performance – part 2

In the second part of the “SSD caching could decrease performance” I will cover IO read and write ratio and IO size affects to SSD. Part 1 is accessible here.

Read IO and write IO ratio

Most real workloads are mixed IO workloads – both disk reads and writes. Read and Write ratio is split between disk reads and disk writes. Many cases enterprise SSD disks have equally good read and write performance. But lately MLC and especially TLC drives have made their way into enterprise market and with some of them read and write performance is not equal. In addition SSD disks may become slower over time due to small amount of free blocks. To mitigate the free block issue SSD vendors are installing extra capacity inside the disks – example Intel DC S3700 SSD has about 32% extra capacity.

SSD disks usually handle reads better than writes. Before selecting your SSD disk vendor and model I recommend to do some research. If possible purchase some different models that would suite your needs and test them in real life scenarios.

IO size

When it comes to performance IO size matters. Large IO could potentially overload a small number of SSD disks and with it affect the performance. I would avoid caching workloads with IO size above 128KB. In my personal experience I have seen a situation with a database where SSD caching was hindering performance due to database multi-block reads.

My recommendations for a more successful SSD caching project

  • Determine VMs that would benefit from SSD caching – VMs doing certain amount or more IO.
  • Analyze the IO workloads – no point of doing read caching when server is only writing. IO size.
  • Check your hardware – controller speeds and capabilities. No point to connect fast SSD to a crappy controller.
  • Find a suitable SSD disks for caching. Price vs performance.
  • Talk with storage admins – might be that SSD in array would make more sense than SSD in server.
Advertisements

SSD caching could decrease performance – part 1

Recently we did a small POC for SSD caching and we got some interesting results. One would believe that if you cache on local SSD inside the server you would get significant improvement in IO. Generally you will see some performance increase but not always like we experienced.

There are several things affecting the performance of local SSD. In this post I will cover SSD disks and disk controllers.

SSD disks

Most common SSD disks are 2.5″ SATA factor. SATA interface performance greatly limits throughput of a disk. Below in the table I’ve listed throughput speeds of different interfaces.

Storage interface speeds

As you can see SATA1-SATA3 interfaces, SAS and SAS 6G interfaces are slower than widely adopted dual port 4Gbit FC, 8Gbit FC or 10G iSCSI (in the table above FC and iSCSI speeds are for single link). Only high end PCIe/NVMe devices can match or outperform the throughput performance of  FC and modern iSCSI. If a workload generates large amount of throughput it could potentially slow down if caching device maximum throughput is slower then your original storage.

One way to increase local disk performance is to increase amount of disks. Example instead of one 800 GB disk use two 400 GB disks. Below in the table I calculated maximum read speed of all disks that you would need to create 800 GB cache. I took different sizes of Intel SSD DC S3710 Series disks.

Intel SSD read speed As you can see single 800 GB disk is rated at 550 MB per seconds, but same amount of cache (800 GB) with four 200 GB disks will give us about 2200 MB per second of read speed.

When selecting a caching software make sure to check if it distributes cache to all SSD disks to maximize the performance. If the software does not distribute cache to all SSDs consider using striped (raid 0) disk as a cache device.

Most caching software vendors have recommendations about different SSD disks – check them out before purchasing the disks.

Disk controllers

Disk controllers can have a huge affects to performance of SSD disks. Some entry level controllers may not perform as good as high end controllers. My advice here is to check your vendor recommendations for your disk controller or if you have not bought a controller yet check out the vendor recommendations and also caching software vendor recommendations.

Also disk controller settings can affect performance – check vendor recommendations. Some keywords – read/write cache, drive write cache, SSD pass-trough. In my experience enable drive write cache, change read/write ratio to 25% read / 75% write and if possible enable SSD pass-trough which enables OS to bypass disk controller.

Related posts

SSD caching could decrease performance – part 2

PrimoCache – Disk caching software for Windows

PrimoCache is caching software for Windows servers and workstations that can use DRAM and/or SSD to speed up both read and write opeartions. PrimoCache has two levels of cache – level-1 is DRAM and level-2 is SSD. A 90-day trial version can be downloaded from Romex Software website.

Main Features (source Romex Software website)

  • Supports physical memory, solid-state drives and flash drives as cache storage.
  • Implements a two-level caching architecture.
  • Supports persistent level-2 caching.
  • Supports Write-Through and Write-Deferred caching modes.
  • Supports TRIM command.
  • Supports OS Invisible Memory.
  • Implements an intelligent and self-tuning cache replacement algorithm.
  • Supports caching strategies: Read/Write Caching, Read-Only Caching and Write-Only Caching.
  • Supports performance statistics and monitor.
  • Supports caching for multiple volumes
  • Supports caching for volumes with proprietary file system.
  • Supports caching for volumes on basic and dynamic disks.
  • Supports plug and play.
  • Supports command-line interface.

More info: http://www.romexsoftware.com/en-us/primo-cache/index.html

Testing and results

For testing I selected a virtual machine which is running some management software with integrated mysql db server. For some reason mysql was causing heavy read IO – 2000 to 3000 iops. I created a 4GB read-only cache. After 30 minutes cache was warmed and mysql was no longer stressing the storage array. Cache hit rate is almost 100% with 1,76GB of cache still free.

PrimoCache application

PrimoCache application

From VMware performance charts it is visible that read IO dropped almost to zero.

Read IO before and after enabling cache

Read IO before and after enabling cache

Read IOPS also disappeared.

Read IOPS before and after enabling cache

Read IOPS before and after enabling cache

Other use case

Write-deferred cache will turn small random IOs into large sequential IOs. This caching option could potentially speed up non-ssd disks which are not that great for small random writes.

Licensing

PrimoCache is licensed per PC/server. Non-commercial license starts at $29.95, business license starts at $119. License includes lifetime free updates and technical support.

Other similar solutions

SuperSpeed SuperCache Express – http://www.superspeed.com/servers/supercache.php
HGST ServerCache – http://www.hgst.com/software/HGST-server-cache

Conclusion

Typically in virtualized environment one would look for a solution in hypervisor layer. But this typically means you need to license several hosts and probably equip them with SSD disks. All of this adds up to cost. This cost might not make sense if you only have one or two machines that would benefit from the solution. Here’s where PrimoCache or other similar solutions might help.