I got a question about using Windows Defender Credential Guard in VMware virtual machines. I did some digging and found following things:
Info from https://docs.microsoft.com/en-us/windows/security/identity-protection/credential-guard/credential-guard-requirements
Windows Defender Credential Guard requires:
- Support for Virtualization-based security (required)
- Secure boot (required)
- TPM 2.0 either discrete or firmware (preferred – provides binding to hardware)
- UEFI lock (preferred – prevents attacker from disabling with a simple registry key change)
The Virtualization-based security requires:
- 64-bit CPU
- CPU virtualization extensions plus extended page tables
- Windows hypervisor
As of today only VMware Workstation 14 has option to enable Virtualization-based security. I created a case to VMWare where I inquired about support for Virtualization-based security in vSphere and answer I got that it will be available in the future version.
Recently we had a case where we tried to restore a VM and it failed. Although all backups finished successfully. We also noticed that single file recovery from that VM was not available. After taking another look at the backup jobs we noticed that the affected VM had only backed up 5 files instead of thousands of files which is normal when “Enable file recovery from VM backup” is enabled.
After some investigation together with Veritas we discovered that Changed Block Tracking (CBT) file was corrupted. We deleted the cbt files from VM directory when VM was powered off. After VM was powered on again new cbt files were created. After that everything started to work correctly.
VMWare KB article about enabling/disabling Changed Block Tracking (CBT) – https://kb.vmware.com/s/article/1031873
Recently I saw couple of VMs which were giving me a warning – “Number of cores per socket cannot be greater than number of virtual CPUs”
This happens when number of vcpu-s is set to smaller number than cores per socket. In my case developer used API to set number of vcpu-s to 2 and number of cores per socket to 4. He made a mistake of thinking that number of vcpu-s was actually number of sockets. After correcting the value on number of vcpu-s to 8 the warning disappeared.
I discovered an issue with my vSphere 6.5 (build 5973321) when trying to delegate permissions via new HTML5 UI – when I try to search for a Active Directory group nothing is found. Same operation in old Flash based UI successfully found the group. I also tried with latest vSphere build 7119157 – the issue exists in that version as well. Authentication source Active Directory is configured as “Active Directory (Windows Integrated Authentication).
As the old UI works I’ll be opening a support case sometimes in the new year to confirm the issue with VMWare.
05.01.2018 Update: According to VMWare support HTML5 GUI is not fully supported and this type of issues may occur. It will be fixed when HTML5 GUI will be fully supported.
I recently worked with Graylog and Grafana to collect, analyze and visualize VMWare ESXi logs. I’ve started a new page where I will collect all the search strings I have found useful.
Some examples of Grafana charts
I will be updating that page with new strings as I’m upgrading from vSphere 5.5 to vSphere 6.5.
I was doing some Shared Nothing Live Migrations between two VMware clusters (version 5.5) and I was getting following error at 25% of the migration – “Incompatible device backing specified for device ’13′”. Searching from internet indicated issues with network adapter but in this case network adapter was not the case.
Issue in this case was a raw device mapping (RDM) that had a different LUN ID in destination cluster.
vMotion between the clusters worked for VM when the datastore was made visible for all the hosts. Storage vMotion did not work in destination cluster – got same error.
Solution for me was to present destination datastore to original hosts and perform Storage vMotion in original location and then perform a vMotion to destination cluster.
Another solution I tested
- Shutdown the VM
- Remove the RDM
- Perform migration to destination cluster
- Reattach the RDM
Recently I noticed some Linux VM backups were failing and sometimes even crashing with following errors:
An error occurred while taking a snapshot: msg.snapshot.error-QUIESCINGERROR.
An error occurred while saving the snapshot: msg.snapshot.error-QUIESCINGERROR.
On closer look another error was visible in hostd.log file – Error when enabling the sync provider.
All of these VMs had one thing in common – they were running Docker containers.
I was not able to figure out why it happened but I was able to find a workaround – disable the VMWare Sync driver.
Copy-paste from Veritas KB article – https://www.veritas.com/support/en_US/article.000021419
Steps to Disable VMware vmsync driver
To prevent the vmsync driver from being called during the quiesce phase of a VMware snapshot, edit the VMware Tools configuration file as follows:
1) Open a console session to the Redhat Linux virtual machine.
2) Navigate to the /etc/vmware-tools directory
3) Using a text editor, modify the tools.conf file with the following entry
enableSyncDriver = false
Note: If the tools.conf file does not exist, create a new empty file and add the above parameters.