Step by step how to install Nutanix CE nested on VMware ESXi

I know that it is not ideal to run Nutanix CE in nested configuration as performance will be affected, but if you don’t have separate compatible hardware available you are still able to install it and play with it. Since version ce-2015.06.08-beta Nutanix Community Edition can run nested on different VMware products. This post will describe how to install Nutanix CE nested on ESXi.

ESXi host configuration

2 x 8 core Intel Xeon E5-2680
384GB RAM
500GB SSD
1024GB HDD
VMWare ESXi 5,5 U2

Nutanix CE VM configuration

12 vCPUs ( 2 sockets x 6 cores)
128GB RAM
400GB vmdk on SSD
900GB vmdk on HDD
8GB Nutanix CE image as vmdk
1 x Intel e1000 virtual nic

Preparations

I’m assuming that registration and downloading Nutanix CE is already done.

Step 1 – prepare the boot disk

  1. extract the ce-2015.06.08-beta.img from the archive
  2. rename the ce-2015.06.08-beta.img to ce-flat.vmdk
  3. create disk descriptor file or download it from Joep Piscaer (http://www.virtuallifestyle.nl) blog here -> https://www.virtuallifestyle.nl/2015/06/nextconf-running-nutanix-community-edition-nested-on-fusion/.
  4. Rename ce.txt to ce.vmdk.

Step 2 – prepare the VM

  1. create a VM
    1. create new Centos 4/5/6/7 (64bit) VM
    2. 12vCPUs (minimum of 4 vCPU)
    3. 128GB RAM (minimum of 16GB RAM)
    4. 900GB vmdk for HDD – map as SCSI0:0 (HDD has to be atleast 500+ GB)
    5. 350GB vmdk for SSD – map as SCSI0:1 (SSD has to be atleast 200+ GB)
    6. Intel e1000 NIC
  2. browse the datastore where you stored the VM and upload ce-flat.vmdk and ce.vmdk
  3. edit VM
    1. Enable virtualization for VM – select CPU -> Hardware virtualization -> Expose hardware assisted virtualization to the guest OS.Nutanix CE VM CPU settings
    2. add new SATA controller
    3. attach existing disk ce.vmdk as SATA disk

Step 3 – installation

  1. Start the VM -> press F2 to enter BIOS -> setup SATA disk as default boot disk
  2. boot the server from SATA disk
  3. Optional: Modify SSD IOps requirements
    1. login as root (password nutanix/4u)
    2. edit /home/install/phx_iso/phoenix/sysUtil.py lines “SSD_rdIOPS_thresh = 5000” and “SSD_wrIOPS_thresh = 5000”
  4. Optional: Modify Controller VM (CVM) memory
    1. login as root (password nutanix/4u)
    2. edit /home/install/phx_iso/phoenix/sysUtil.py line “SVM_GB_RAM = 16”
  5. to start installation login as user “install”

Step 4 – enable “Promiscuous mode”

  1. enable “Promiscuous mode” on the vSwitch or virtual machine network where you have nutanix-ce VM connected. Otherwise controller VM is not accessible.
  2. Update 03.08.2016 – you may also need to enable “Forged Transmits”. Thanks to Thiago to pointing this out.

Conclusion

I used these steps to successfully install Nutanix CE on top of VMware ESXi 5.5 U2 running on HP server which is not in the Nutanix HCL.

More information

Joep Piscaer describes how to install Nutanix CE to VMware Fusion which also works for VMware Workstation – https://www.virtuallifestyle.nl/2015/06/nextconf-running-nutanix-community-edition-nested-on-fusion/

Running Nutanix Community Edition nested on VMware Workstation

It is now possible to install and run Nutanix Community Edition (Version 2015.06.06) nested on VMware Fusion and VMware Workstation. 

Joep Piscaer has detailed install guide for Fusion that also works for Workstation – http://www.virtuallifestyle.nl/2015/06/nextconf-running-nutanix-community-edition-nested-on-fusion/

UPDATE – This issue occurred with version ce-2015.06.06-beta. This issue no longer exist with version ce-2015.06.08-beta.

Only problem I ran into was that I was getting following error from the installer:

INFO: Nutanix Community Edition cannot be installed on this hardware. Reason:
INFO: A set of disks matched the minimum requirements was not found.

After some digging I found that both of the added SCSI drives were identified as spinning disks although they both were stored on SSD. To fix this issue:

  • Login as root. Password – “nutanix/4u”
  • Identify your “SSD” and “HDD” -> “fdisk -l”. In my setup “SSD” is “/dev/sda”
  • Run “cat /sys/block/sda/queue/rotational”. 0 means SSD and 1 means HDD.
  • Run “echo 0 > /sys/block/sda/queue/rotational” to change the value from 1 to 0.
  • Rerun the installer

Now the installer was happy with my setup and after few minutes I had Nutanix Community Edition up and running on VMware Workstation.

Nutanix – No rebuild capacity available

Nutanix resilency status

“No rebuild capacity available” warning appears to Nutanix clusters when amount of free disk space in clusters is lower than capacity of a single node. This means when you loose one node it is not possible to fully rebuild data resiliency. There are options to fix this – free up space or add more nodes to your cluster. You can free up space by removing obsolete VMs or reclaim dead space within running VMs. I covered here how to reclaim space within VMs.

To check how much storage capacity single node provides go to Hardware -> Table -> select one host -> check “Storage Capacity” value.

64GB DDR4 memory module now available for HP ProLiant Gen9 servers

HP has made available 64GB memory modules for their servers – HP 64GB (1x64GB) Quad Rank x4 DDR4-2133 CAS-15-15-15 Load Reduced Memory Kit – 726724-B21. List price on HP Simple Configurator is USD4999. For comparison – 32GB module list price is USD899. With 64GB module available it is now possible to configure ProLiant BL460c Gen9 with 1TB of RAM and ProLiant DL380 Gen9 with 1,5TB of RAM.