Nested ESXi 6.7 fails to boot

Recently I moved some of my nested ESXi 6.7 VMs from standalone host to a 3 node vSAN cluster. Standalone host is running ESXi 6.7 on HPE DL380 G7 with Xeon 5600 series CPU. vSAN cluster is running similar hardware. I moved the ESXi 6.7 VMs while they were offline. When I powered on the VMs they failed to boot. I saw following error message:

VMB: 553:
Unsupported CPU: Intel family 0x06, model 0x25, stepping 0x1
Intel(R) Xeon(R) CPU X5670 @ 2.93GHz
See http://www.vmware.com/resources/compatibility

CPU unsupported

Only difference between standalone host and cluster hosts was that I had EVC mode enabled on the cluster – Intel® “Westmere” Generation. After I disabled EVC mode the ESXi 6.7 VMs booted without any issues.

 

Home lab update – 2018

Past couple of months I have been working on to update and upgrade my home lab.

My LAB now includes:

3 node VSAN cluster with HPE DL360 G7 SFF, HPE DL380 G7 SFF and HPE DL380 G7 LFF.
A standalone ESXi running on HPE DL380 G7 for running vCenter 6.7 U1 and other supporting services.
A standalone Windows Server HPE DL380 Gen8 to run VMs in WMware Workstation and file server service.

Currently network is 1G. Planning to upgrade to 10G in the future.

Some things I discovered during building the lab.
HPE DL380 G7 LFF with HP P410i also accepts 8TB disks. HPE quick specs only include disks as large as 4TB.
HPE DL380 Gen8 also works with DDR3 16GB 1067Mhz Quad Rank RDIMM memory modules. I was able to install 128GB per CPU. The operating frequency was reduced to 800Mhz.

Extended my home lab

I’ve recently extended my home lab with additional capacity. In addition to my Windows Server + VMware Workstation (info here) I’ve added refurbished HPE DL380 G7 server with following configuration:

1 x Intel Xeon Processor X5650 2.66Ghz
96GB RAM
1TB HDD
will add SSD in the future

The added server is running VMWare ESXi 6.7. It hosts vCenter 6.7 appliance and also few virtual ESXi 6.7 instances. HPE G7 series servers are not officially supported by VMWare to run ESXi 6.7 but it seems to be working for now.

I found my refurbished HPE G7 server from Ebay.

My home lab configuration

Beginning of the year I replaced my old home lab (HP ML110 G5) with a new custom configuration.

Configuration

  • Intel i7-4820K
  • Gigabyte X79-UP4
  • 8 x 8GB Kingston DDR3
  • 250GB Samsung EVO – OS drive
  • 500GB Samsung EVO – virtual machines
  • 6 x 4TB WD Red – Storage space pool
  • Windows Server 2012 R2
  • VMware Workstation 10

Hardware

CPU and motherboard was selected by following criterias – 64GB RAM, decent amount of SATA ports and reasonable price. SSD for OS and VMs was obvious choice.  6x4TB was included because I use the same machine to store my media library and backups.

Storage spaces

I created one storage space pool from six 4TB disks. From pool I carved out 16TB parity disk for media library, 2TB parity disk for backups (dedupe enabled) and 4TB striped disk for virtual machines which needed more space then I was able to provide from SSD.

VMware Workstation

I chosed VMware Workstation because of following reasons

  • Easy to run nested hypervisors – ESXi, KVM, HyperV and XenServer
  • Linked clones – feature that allows me to clone from single template a number of similar virtual machines. Saves a lot of space on SSD disk.

What I do with my home lab?

A lot of different things.

  • Hypervisors – ESXi, HyperV, XenServer
  • Windows infrastructure – AD, DNS, File server, etc
  • Linux servers – glusterfs test servers, nagios, cacti, etc
  • Storage appliances – FreeNAS, QuadStor, etc
  • Testing new things
  • and much more ….