Invalid CPU reservation for the latency-sensitive VM

Recently some VMs went down during regular patching. When I checked they were not powered on and when I tried to power them on I got an error – “Invalid CPU reservation for the latency-sensitive VM, (sched.cpu.min) should be at least 6990 MHz.”.

What had happened is that someone had changed this VM latency sensitivity to “High” without doing proper CPU and RAM reservations. It would not have been a problem during a restart of a VM but I had set advanced setting called “vmx.reboot.PowerCycle” to TRUE since I needed VM to get some new CPU features. This setting causes VM to power cycle during normal OS reboot. And since the reservations were not properly done VM failed to power on. After fixing the reservations VM successfully powered on. The error message about RAM reservation looks like this – “Invalid memory setting: memory reservation (sched.mem.min) should be equal to memsize(4096)”.

To check the VM latency sensitivity using PowerCLI I use Virten.net.VimAutomation module written by Florian Grehl (his blog) – https://www.powershellgallery.com/packages/Virten.net.VimAutomation/1.3.0. This module has 3 useful commands for viewing and changing VM latency sensitivity – Get-VMLatencySensitivity, Get-VMLatencySensitivityBulk and Set-VMLatencySensitivity.

Change LUN queue depth in ESXi 6.7

Some time ago I had to change default queue depth for all LUNs in cluster.

First I needed to determine which driver (module) my HBA is using. I used following script for that.

### Script start 
$esx_hosts = get-vmhost esxi1*

foreach ($esx_host in $esx_hosts) {
Write-Host $esx_host
$esxcli = Get-EsxCli -VMhost $esx_host -V2
$esxcli.storage.core.adapter.list.invoke() |select HBAName, Driver, Description
}
### Script end

Output looks like this

To change the LUN queue depth parameter I used following script

### Script start
$esx_hosts = get-vmhost esx1*

foreach($esx_host in $esx_hosts){
Write-Host $esx_host
$esxcli=get-esxcli -VMHost $esx_host -V2
$args1 = $esxcli.system.module.parameters.set.createArgs()
$args1.parameterstring = “lpfc_lun_queue_depth=128”
$args1.module = “brcmfcoe”
$esxcli.system.module.parameters.set.invoke($args1)
}
### Script end

After running this you need to restart ESXi host.

After that I used following script to set “Maximum Outstanding Disk Requests for virtual machines”

### Script start
$esx_hosts = Get-VMHost esx1*

foreach ($esx_host in $esx_hosts)
{
$esxcli=get-esxcli -VMHost $esx_host -V2
$devices = $esxcli.storage.core.device.list.invoke()
foreach ($device in $devices)
{
 if ($device.device -imatch “naa.6”)
 {
  $arguments3 = $esxcli.storage.core.device.set.CreateArgs()
  $arguments3.device = $device.device
  $arguments3.schednumreqoutstanding = 128
  Write-Host $device.Device
  $esxcli.storage.core.device.set.invoke($arguments3)
  }
 }
}
### Script end

To check the LUN queue depth I use following script

### Script start
$esx_hosts = Get-VMHost esx1*
foreach($esx_host in $esx_hosts){
$esxcli=get-esxcli -VMHost $esx_host -V2
$ds_list = $esxcli.storage.core.device.list.invoke()

foreach ($ds1 in $ds_list) {
 $arguments3 = $esxcli.storage.core.device.list.CreateArgs()
 $arguments3.device = $ds1.device
 $esxcli.storage.core.device.list.Invoke($arguments3) | select Device,DeviceMaxQueueDepth,NoofoutstandingIOswithcompetingworlds
 }
}
### Script end