Deduplicate VMWare virtual machines with Windows Server 2012 R2

I previously played with VMWare ESXi using NFS from Windows Server 2012. When Windows Server 2012 R2 became available I decided to add deduplication into the mix. For those who does not know Microsoft included a new feature in Windows Server 2012 called Data Deduplication. Data Deduplication enables you to dedupe files stored on NTFS volumes. Previously I was not able to run deduplication on Windows Server 2012 because it does not support deduplication for open files. Starting from Windows Server 2012 R2 Microsoft supports deduplication for open files.

For NFS share I decided to reuse already existing Windows Server 2012 R2 running on Hyper-V. This server is also used to provide deduplicated storage to Hyper-V hosts via SMB3. For NFS I created 500GB LUN and enabled deduplication (VDI mode). Since I was not sure about the performance of that deduplicated NFS datastore I decided to select VMs that did not have high I/O demand. I ended up moving 19 Linux based development servers to this NFS datastore.

Deduplication result is quite good. I achieved about 8x data reduction rate. It helped that all those 19 Linux servers are quite similar.

NFS statsPerformance numbers on the datastore are following:
IOPS: between 20-300 iops
Read Latency: < 5ms
Write Latency: <15ms

Write latency is not that good. But since those servers are only running some script jobs during the night no one has complained.

One thing to remember when using Windows Server based NFS with VMware is that when you move the VM to Windows Server based NFS datastore all the disks get inflated. So after storage vMotion VM will consume all the space it’s allocated. In my case I can not move any VMs which have been allocated more then 500 GBs to this datastore .

In future I’m planning to expand the usage of deduplicated storage for VMWare because it can deliver significant disk space and cost savings.

More information:

Introduction to Data Deduplication in Windows Server 2012

What’s New in Data Deduplication in Windows Server 2012 R2

WS2012 Storage – NFS Server – Configure NFS for VMware vSphere 5.1 Home Lab

Advertisements

5 thoughts on “Deduplicate VMWare virtual machines with Windows Server 2012 R2

  1. Hi

    Finally i found one who is trying the same I want to accomplish. Do you run into any problems? the same with your previous post (Deduplication on not VDI Hyper-V)
    I’m building now a Storage Server with Storage Spaces, cheap 1TB COnsumer SATA SSD. Presenting Deduplicated SMB3 and NFS Storage for a Hyper-V and ESX

  2. I have not had any issues. I have actually extended testing of Storage Spaces + NFS + Dedupe + VMWare. I will post an updated blog post regarding that any day now. One thing I thought about yesterday was that if I could get my hands on some TLC based large SSD-s I would try to used them as HDD tier in Storage Spaces instead of regular spinning media.

  3. Pingback: Deduplicate VMWare virtual machines with Windows Server 2012 R2 – part 2 | Kalle's playground
  4. Hello Kalle, I’ve been testing a similar configuration and I’m having a problem: 2012r2 exporting NFS and mounted on ESXi 5.5u1. Seems to work, share is there but when I try to make vmotion (Or copy) a machine to the NFS datastore it fails on the first vmdk (80Gb), other little files are copied. I’ve read on other posts that this configuration does not work but obviously you have it running. Do you have any hint to help me?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s