UPDATE as of 9/9/14: For HP customers using Emulex 10GbE adapters, HP has made publicly available the latest code that addresses the VM disconnect issue when VMQ is enabled among other enhancements. The download portal is currently located here: http://ow.ly/Bi7Yt. Please read, understand and follow the update documentation provided by HP and contact HP tech support for further information. Thank you for your continued patience.
UPDATE as of 8/4/14: We are pleased to inform you that the July 2014 Special Release for Windows Server 2012 and Windows Server 2012 R2 CNA Ethernet Driver is now available for Emulex branded (non OEM) OCe111xx model adapters. Please refer to this link to download the driver kit and firmware. Please read and follow the special instructions within the Release Notes. For non-Emulex branded adapters, please contact Emulex Tech Support here.
UPDATE AS OF 7/23/14: Emulex is in the process of rolling out updated Microsoft Windows 2012 and 2012 R2 VMQ solutions for our customers. Testing of a Windows WHCK certified NIC driver update will be completed in 1-2 weeks. This initial “hotfix” will be for Emulex branded OCe11102 and OCe11101 products and will include a required firmware update. As testing completes on hotfix solutions for additional product configurations, notices and links will be posted on this blog. Thanks for your continued patience.
It has been reported that some customers using Microsoft Windows 2012 or Windows 2012 R2 and Hyper-V may experience a loss of network connectivity with one or more virtual machines (VMs). Emulex has identified some VMQ specific cases involving our network adapters and the root causes for the issues. We are working closely with our OEM partners to prepare a field upgradeable solution. As we track the releases of the update by Emulex and our server OEMs, we will use this blog to inform you of the current status. In the interim, we will advise on operational alternatives to ensure VM connectivity and stability.
The problem occurs when the VMQ feature is enabled. The operating system continuously shifts CPU affinity as needed for each of the queues. During the time a queue is disabled then re-enabled, a Hyper-V VM may experience an apparent loss of network connectivity. This condition might be either momentary or permanent.
- Tech Support: Whether you have Emulex branded adapters or OEM branded adapters, we highly recommend contacting Emulex Tech Support as a first step if you think you are experiencing this problem. They can help confirm if your problem is VMQ related and attempt to offer immediate solutions beyond those made available on this blog. Contact Emulex tech support at +1 800-854-7112 or 714-885-3402 or submit a case online at http://www.emulex.com/support/.
- Driver/Firmware update: A driver/firmware fix is the best solution to this problem as it will maintain performance and flexibility.
Status on 6/19/2014: Updated driver and firmware code that addresses the VMQ network connectivity issues is currently going through comprehensive validation and test and is estimated to be available for Emulex branded products by early August. Driver releases of specific OEM configurations are being evaluated now. We will update this blog as soon as we confirm the timing and location of each OEM release.
- Configuration Workarounds:
Until a driver/firmware fix is available, the best known workaround to ensure reliable VM network connectivity is to disable the VMQ feature. We may add other workarounds to this blog as procedures are developed.
Disable VMQ on Adapter – Disabling the VMQ feature of the adapter ports will prevent network connectivity disconnect on a VM altogether. It is estimated that disabling VMQ will significantly reduce throughput going through the network adapter. Here are the steps to disable VMQ:
Go to the properties of the adapter and click on Configure.
Click on Transmit and Change the Value from Enabled to Disabled.
Also click on Virtual Machine Queues and change the Value from Enabled to Disabled.
Verify by running powershell command Get-NetAdapterVmQueue and no VMQs are listed
We appreciate your patience and will provide updates here, as necessary.