Emulex Blogs

Microsoft Windows 2012/2012 R2 Hyper-V VMs losing network connectivity: a workaround

Posted June 19th, 2014 by Mark Jones

UPDATE AS OF 7/23/14: Emulex is in the process of rolling out updated Microsoft Windows 2012 and 2012 R2 VMQ solutions for our customers.  Testing of a Windows WHCK certified NIC driver update will be completed in 1-2 weeks. This initial “hotfix” will be for Emulex branded OCe11102 and OCe11101 products and will include a required firmware update.  As testing completes on hotfix solutions for additional product configurations, notices and links will be posted on this blog.  Thanks for your continued patience.

~~

It has been reported that some customers using Microsoft Windows 2012 or Windows 2012 R2 and Hyper-V may experience a loss of network connectivity with one or more virtual machines (VMs).  Emulex has identified some VMQ specific cases involving our network adapters and the root causes for the issues.  We are working closely with our OEM partners to prepare a field upgradeable solution.  As we track the releases of the update by Emulex and our server OEMs, we will use this blog to inform you of the current status.  In the interim, we will advise on operational alternatives to ensure VM connectivity and stability.

Symptoms

The problem occurs when the VMQ feature is enabled.  The operating system continuously shifts CPU affinity as needed for each of the queues. During the time a queue is disabled then re-enabled, a Hyper-V VM may experience an apparent loss of network connectivity. This condition might be either momentary or permanent.

Solution hierarchy

  1.  Tech Support: Whether you have Emulex branded adapters or OEM branded adapters, we highly recommend contacting Emulex Tech Support as a first step if you think you are experiencing this problem.  They can help confirm if your problem is VMQ related and attempt to offer immediate solutions beyond those made available on this blog.  Contact Emulex tech support at +1 800-854-7112 or 714-885-3402 or submit a case online at http://www.emulex.com/support/.
  1.  Driver/Firmware update:  A driver/firmware fix is the best solution to this problem as it will maintain performance and flexibility.

Status on 6/19/2014:  Updated driver and firmware code that addresses the VMQ network connectivity issues is currently going through comprehensive validation and test and is estimated to be available for Emulex branded products by early August.  Driver releases of specific OEM configurations are being evaluated now. We will update this blog as soon as we confirm the timing and location of each OEM release.

  1.  Configuration Workarounds:

Until a driver/firmware fix is available, the best known workaround to ensure reliable VM network connectivity is to disable the VMQ feature.  We may add other workarounds to this blog as procedures are developed.

Disable VMQ on Adapter – Disabling the VMQ feature of the adapter ports will prevent network connectivity disconnect on a VM altogether.  It is estimated that disabling VMQ will significantly reduce throughput going through the network adapter. Here are the steps to disable VMQ:

Go to the properties of the adapter and click on Configure.


vmq groupGo to the advanced page and scroll down to Virtual Machine Queue

Click on Transmit and Change the Value from Enabled to Disabled.

vmq2

Also click on Virtual Machine Queues and change the Value from Enabled to Disabled.
Verify by running powershell command Get-NetAdapterVmQueue and no VMQs are listed

vmq3

We appreciate your patience and will provide updates here, as necessary.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

The purpose of our blog is to facilitate an ongoing conversation on what's going on in our industry, with our partners and customers. We encourage your comments. Your ideas and feedback are what makes our blogs interesting, timely and useful for our readers.

We want to publish your comments, however, all comments are moderated. Offensive, off-topic or fraudulent comments won't be approved. We also expect a basic level of civility; disagreements are expected, but mutual respect is a must. We will not post comments that contain vulgar or abusive language; personal attacks of any kind; or offensive terms that target specific ethnic or racial groups. Comments that make accusations will also not be posted.

By submitting a comment, you agree to these terms; having your name displayed with your comment and that you are 18 years old or older. Your name and personal information will not be used for any other purpose, and your e-mail address will not be published.

  1. Jeremy Strachan says:

    We also have seen the VMQ issue rear its ugly head in New Zealand today with a loss of most VM’s connectivity at a client site this afternoon – HyperV 2012 R2 brand new install on HP DL380 G8 Hardware.

    Look forward to a resolution.

    Reply

  2. Yusuf Assenjee says:

    Any update on when the corrected driver will be available?

    Reply

  3. Ron says:

    What is the current status of VMQ related issues, fixes and workarounds?

    Thanks.

    Reply

  4. Maxim Gladovskiy says:

    Hello,
    is there any updated information about new drivers/firmware?

    Reply