Today, we are happy to have a guest blog from Kevin Bossman, product marketing manager at Lenovo as he talks about the Emulex Virtual Fabric Adapter 5 for Flex System servers and how, with RDMA and network offloading, Emulex and Lenovo are accelerating workloads for today’s high-performance data centers.
By Kevin Bossman
Flex System offers high-performance Ethernet and converged networking switches and adapters that can fit into your existing network and future IT environment. These highly flexible products coupled with on-demand scalability offer an easy way to scale as your IT requirements grow. Lenovo offers an array of networking communication technologies ranging from Ethernet to Fibre Channel to InfiniBand to iSCSI.
Today, we are witnessing explosive growth of data communication facilitated by expansion and proliferation of Internet access across the globe. Considering this gigantic requirement, the natural question that comes to mind is, “What is the common technology feature that supports such unprecedented growth of data?“ The most common answer is Ethernet. Emulex is helping to show why Ethernet is quickly becoming the faster-growing data communication device. In this blog, I will discuss how two new features from Emulex’s new Virtual Fabric Adapter 5 (VFA5) chipset are facilitating this trend.
Read more here on the Lenovo blog!
I’m Tom Boucher and I’m part of the Emulex engineering team that covers the Emulex and Lenovo relationship. My role is a systems engineer, so that makes me the one nerd on the team that works with the Lenovo sales engineers and also with any Lenovo partner systems engineer types in understanding how Emulex technology works with Lenovo technology to make cool stuff.
I am more of a fan of “explain cool stuff” than “ramble on about features,” so I decided to tackle a complex topic: the Unified Fabric Port (UFP) feature of Lenovo networking switches.
First, if you’d like the technical details, you should read the product guide (previously known as IBM Redbooks) (http://www.redbooks.ibm.com/abstracts/sg248223.html?Open). It’s a great document that will tell you all the things we can do with the Flex System switches (like the EN4093R or the CN4093) or the 8264 rack switch for System x servers.
Now, before I give you a quick primer on UFP, I should tell you how we got to UFP and why we like it so much.
Back in 2009, the BladeCenter team and what is now Lenovo Networking, created a technology called virtual NIC (vNIC). It was designed to allow for multiple independent networks on top of a single 10Gb Ethernet (10GbE) link.
It was wildly popular, but due to the way it was designed, it did vNIC very well by creating a virtual L2 switch using 802.1QinQ (or Q-in-Q) virtual LANs (VLANs) tagging that used to be used for metropolitan area networks or inside hosting centers. It was initially put on the BladeCenter Virtual Fabric switch and each virtual switch needed an uplink to the outside world if you wanted to do anything more than moving Ethernet packets within the BladeCenter chassis. Now, all Lenovo networking switches support this feature (see above switches mentioned).
vNIC (or now “Virtual Fabric vNIC” as it’s called) has a few drawbacks. One concern was that you could not create a failover path from a single switch for a vNIC group. If you lost the uplink, all the vNICs in that group would ‘fail’ and you would need redundant links on the second switch to pass that traffic until the single uplink was restored.
Since 802.1QinQ created virtual Layer 2 switches, we could only tag with VLANs on the host using virtual switch tagging or tag on the upstream switch by using native VLANs on the switch port uplink, or a few other methods, which was a drawback. This wasn’t always optimal as four vNICs required four 10GbE uplinks from each switch, and that can get rather expensive. By design, those uplinks weren’t fully utilized as you could be using less than that by creating slower vNICs than 10GbE.
So Lenovo Networking created a new version of the technology called UFP. This added a bunch of new capabilities and attempted to address some drawbacks of the initial vNIC design and customer feature requests. What I’m going to talk about is how that changed the vNIC behavior.
Now, instead of a virtual Layer 2 switch, we use the 802.1Qbg standard to create a virtual Layer 3 network. Utilizing virtual Ethernet bridging and virtual Ethernet port aggregation, we are able to be VLAN-aware. This allows for higher-level networking (Layer 3) to be performed in the switch instead of being handed off to another switch for Layer 3 VLAN routing, similar to the original vNIC. This allows more complex networks to be in switch-on-switches like the 8264 or in multiple stacked CN4093/EN4093Rs in a Flex System server.
Each vNIC can now have different modes:
- Tunnel mode
- Provides Q-in-Q mode, where the vPort is customer VLAN-independent (which is similar to vNIC Virtual Fabric Dedicated Uplink Mode).
- Trunk mode
- Provides a traditional 802.1Q trunk mode (multi-VLAN trunk link) to the vNIC (vPort) interface; permits host side tagging.
- Access mode
- Provides a traditional access mode (single untagged VLAN) to the vNIC (vPort) interface, which is similar to a physical port in access mode.
- Fibre Channel over Ethernet (FCoE) mode
- Provides FCoE functionality to the vPort.
- Auto-VLAN mode
- Auto-VLAN creation for Qbg and Lenovo VMready environments.
Only one vPort (vPort 2) per physical port can be bound to FCoE. If you don’t want FCoE, vPort 2 can be configured for one of the other modes.
If you download the product guide, there is a nice table on page 6 that does a great job of breaking down the feature/capability by vNIC type, which will help with the details on each mode above and why each one is needed.
UFP uses vPorts to create isolation between vNICs within the compute node and maintains that isolation within the switch. vNICs within the compute node are created (up to four per 10GbE NIC port currently) and can be assigned to separate vSwitches or be seen as a virtual HBA within the hypervisor or bare bones operating system.
If you are upgrading an environment that was Virtual Fabric vNIC and would like to use UFP in a similar manner, you would want to use UFP Tunnel mode, which is a Q-in-Q mode. The vPort is customer VLAN-independent and behaves most similarly to the original Virtual Fabric mode. This is also the ‘default’ if you just turn on UFP without configuring it.
How do you know which mode might work in your network? Again, I’ll reference the product guide, which has a nice table on page 61 that breaks down what features typically used on a network are available in each type of vPort.
So, this should give you a brief idea of the UFP feature, so you can see if this might be something you would be interested in deploying in your environment.
I’d love to hear from you. Please send me feedback or questions about what I’ve written or topics that you’d like me to cover in the future.
Last week, I was watching (again) the new Star Trek movie on TV. And you know every Trek movie has the part where Captain Kirk is demanding more power from Scotty so the U.S.S. Enterprise can escape whatever predicament they’re in.
Just a few weeks ago, Lenovo announced their new Yoga 3 Pro laptop. You can use it in any of four different use modes: laptop, stand, tent, and tablet. It has a new innovative watchband-inspired hinge and it’s as thin as a pencil.
But why should the Lenovo PC portfolio have all the fun?
Today, Lenovo is announcing the Flex System CN4058S 8-port and CN4052 2-port Virtual Fabric Adapters (VFAs)! And, just like their PC cousins, these adapters also have four different use modes, all in one adapter:
- Network Interface Card (NIC)
- Overlay networking
- RDMA over Converged Ethernet (RoCE)
So what does all of this mean for you? The Flex System CN4058S and CN4052 are based on Emulex Virtual Fabric Adapter 5 technology and offer:
Reduced Cost and Complexity
- With the Flex System CN4058S and CN4052 adapters, you’ll get converged data, storage and low-latency RDMA networking on a common Ethernet fabric, giving you flexible storage protocol options for running heterogeneous workloads on your converged infrastructure.
- The CN4058S and CN4052 adapters deliver server power savings due to lower CPU usage, ranging from 50 watts to almost 100 watts per server vs. competing adapters1.
- The CN4058S and CN4052 VFA5 adapters not only support Virtual Fabric Mode (Virtual NIC [vNIC] vNIC1) and Switch Independent Mode (vNIC2), but also Unified Fabric Port (UFP). Emulex is the exclusive supplier of UFP, which allows you to maximize your server adapter bandwidth allocations dynamically as traffic needs change.
- The CN4058S offers up to 32 vNICs, perfect for high-performance servers running server virtualization.
Accelerating Data and Improving Performance
- The Flex System CN4058S and CN4052 adapters support for RoCE, which accelerates applications such as Microsoft SQL Server when using Windows Server SMB Direct, delivering application and storage acceleration through faster I/O operations.
- Emulex Virtual Network Exceleration™ (VNeX) technology which offloads the header encapsulation process of next generation overlay network protocols, simplifying virtual machine (VM) mobility and network scalability, while optimizing server performance when compared to adapters without offload capabilities. Emulex VNeX technology allows customers to maintain CPU utilization thresholds and reduce CPU usage fluctuations, while adding more workloads to each server in virtual networking environments.
Workload and Storage Connectivity Flexibility
- The Flex System CN4058S and CN4052 adapters deliver up to a 4x boost in small packet network performance1, vs. previous generation adapters, which is critical to scale transaction-heavy and clustered applications and workloads such as telco, Web-scale and messaging environments. These adapters support FCoE offload, accelerating storage protocol processing and delivers up to 1.5 million I/O operations per second (IOPS), which is 2x better than previous generation adapters, enabling the server’s processing resources to focus on applications and improves the server’s performance1. They also support iSCSI offload, providing performance that is superior to iSCSI solutions based on software initiators with standard NICs, and support NIC traffic, iSCSI or Fibre Channel over Ethernet (FCoE) offloads on the same port (i.e. concurrent storage).
The CN4058S and CN4052 are members of the VFA 5 family, which is also supported on System x racks.
To learn more about the Flex System CN4058S and CN4052, view the Redbook and video.
- Based on IT Brand Pulse OCe14000 Performance Test Report, July 2014
Anyone who has been around the IBM System x, BladeCenter and IBM Flex System product portfolio over the last few years will know Emulex and the Virtual Fabric Architecture (VFA) that we support. We have enabled competitive differentiation for many years and provide the very best Ethernet and Fibre Channel connectivity products in the business.
New workloads and the way our customers are using infrastructure has shifted, again. Even the way network traffic is moving around the data center is changing, and because of that, the way we architect solutions must change in order to fully realize and leverage the return on investment (ROI) for the latest IBM System x3850/x3950 X6 servers, storage and network data center building blocks…
Emulex did something very special this quarter, as we announced our new OneConnect® OCe14000 family of 10Gb and 40Gb Ethernet (10/40GbE) Network Adapters and Converged Network Adapters (CNAs), based on the next generation Emulex Engine™ (XE) 100 series of I/O controllers. We have been working hard on the industry’s latest trends and directions and working out which ones will stick with us over the next few years and how that will affect the way our technology will be used by IBM and our end customers. Emulex wanted the truth…and could handle the truth (sorry, I had to get that in!) and have a product that is so good, it will change the way our customers architect their solutions for enterprise workloads, web-scale applications, virtualized environments and software-defined networking (SDN) deployments. The new Emulex Virtual Fabric Adapter (VFA) 5 for IBM System x has the ability to scale performance in three dimensions; higher bandwidth, reduced latency and increased I/O operations per second (IOPS), to address the explosive growth of IP traffic and global storage.
Emulex VFA technology has been reducing cost, complexity and increasing performance since 2009, and the newly announced Emulex VFA5 for IBM System x builds upon this success and adds features and functionality that gives our competition a ‘CODE RED’ (Okay, last movie pun, honest!) and will change the game all over again, just as Emulex did in 2009.
The Emulex VFA5 for IBM System x is optimized to meet the needs of enterprises, cloud providers and telcos, while delivering a powerful set of features and capabilities that include:
High-Performance Virtualization: The Emulex VFA5 uses highly efficient and scalable hardware offload technology to transfer the overhead of virtual networking, which provides up to 50 percent better CPU utilization compared to standard NICs when used for VMware VirtualWire connection, thereby increasing the number of virtual machines (VMs) supported per server1. In addition, the VFA5 delivers a fundamental 4x performance increase in small packet network performance required to scale transaction-heavy and clustered applications.
Rapid, Secure and Scalable Hybrid Cloud Connectivity: Emulex Virtual Network Exceleration™ (VNeX) offload technology provides up to 70 percent better performance vs. software-only implementations of emerging Virtual Network Fabric standards such as Network Virtualization using Generic Routing Encapsulation (NVGRE) used by Microsoft Hyper-V Network Virtualization and Virtual Extensible LAN (VXLAN) used in VMware’s NSX to overcome the limitations of legacy data center networks by enabling flexible virtual workload mobility reducing deployment of VMs or network reconfiguration from days to minutes. When combined with Microsoft Dynamic VMQ or VMware NetQueue, the Emulex VFA5 provides the industry’s best platform for increasing VM density.
Application Delivery with Advanced RoCE Architecture (Coming 2H2014): The Emulex VFA5 is based on a low-latency RDMA over Converged Ethernet (RoCE) architecture that helps enterprise IT and cloud data centers optimize applications delivery for VDI, big data, next generation NoSQL and in-memory databases, along with traditional enterprise IT workloads. These new adapters will support Windows Server SMB Direct and Linux NFS protocols.
Increased Block Protocol Performance: The Emulex VFA5 increases total block protocol I/O operations per second (IOPS) by 50 percent over previous generation VFAs and builds upon Emulex’s proven history of enterprise-class storage reliability2.
All of these new features and building upon those of previous Emulex VFAs – with IBM Virtual Fabric vNIC and Switch Agnostic vNIC, enterprise hardened Fibre Channel over Ethernet (FCoE) drivers for unsurpassed enterprise storage interoperability and a cost-optimized modular LOM (ML)2 form factor for IBM x3850/x3950 X6 – make the Emulex VFA5 the adapter that sets the golden standard for virtualized, converged and cloud environments.
At Emulex, A Few Good Men (and women too) through innovation and the IBM Virtual Fabric Architecture foundation have designed and built an adapter that will continue to deliver benefits to our customers and optimize the way smart architects and technical pre-sales specialists build our their solutions. You wanted the truth? Now you have it.
- Based on Emulex Implementers Lab blog
- Based on Emulex Labs testing
IBM recently announced their new IBM System x3650 M4 HD and IBM System x3550 M4 servers.
The IBM System x3650 M4 HD is designed to be the ultimate in high-density storage servers for your big data or business-critical workloads. The IBM System x3550 M4 is a 1U-high, rack optimized server that features increased frequency, optimized performance and improved systems management for business-critical applications and cloud deployments. Both feature the new Intel® Xeon® E5-2600 v2 series processors.
IBM will add the following enhancements provided by Emulex to the IBM System x3650 M4 HD and IBM System x3550 M4 offerings in the fourth quarter, 2013:
- Emulex Dual Port 10Gb Ethernet (10GbE) Virtual Fabric Adapter (VFA) IIIr PCI Express (PCIe) Adapter for IBM System x: This adapter can help you break the I/O bottleneck by allowing you to allocate bandwidth where it’s needed, delivering maximum application agility. It achieves line rate 10Gb per second performance with support for TCP/IP stateless offloads and TCP Chimney offload.
- Emulex VFA III Fibre Channel over Ethernet (FCoE)/iSCSI License for IBM System x Feature on Demand (FoD): this allows you to upgrade the VFA III to support storage networks using iSCSI or Fibre Channel over Ethernet (FCoE) with full protocol offload.
- Emulex Dual Port 10GbE SFP+ VFA IIIr Embedded Adapter for IBM System x: this VFA is based on the standard Emulex Dual Port 10GbE VFA IIIr PCIe Adapter, but it is a different form factor designed to fit in a special mezzanine slot on select servers, freeing up the PCIe slots for other uses.
- Emulex Embedded VFA III FCoE/iSCSI License for IBM System x FoD; this feature allows you to upgrade the Embedded VFA III adapter to support storage networks using iSCSI or FCoE with full protocol offload.
With IBM Virtual Fabric, converged protocols such as iSCSI and FCoE are supported on select configurations via IBM’s FoD process. By using a common infrastructure for Ethernet and storage area networks, and by virtualizing your network adapters, you can reduce your infrastructure capital expenditures. Both adapters support IBM’s Virtual Fabric Mode with IBM switches. This allows you to create four virtual NIC (vNIC) ports per physical port for a maximum of eight vNICs per adapter. These adapters also support Switch Independent Mode, which allows you to use the adapters with other Ethernet switches.
The official IBM announcement letter for the IBM System x3650 M4 HD can be found here.
The official IBM announcement letter for the IBM System x3550 M4 can be found here.
Emulex is headed to Las Vegas to be part of IBM Edge 2013, IBM’s premier storage and optimized solutions event!
From June 10-14, IBM will host 3000 of their valued customers and Business Partners at the Mandalay Bay Resort. There will be more than 350 technical sessions, geared for novices to experts.
Emulex will be there to show how IBM-branded convergence and storage products provided by Emulex optimize IBM Flex System, BladeCenter and System x servers.
Come visit us in the Solution Center (booth #202), where you can register to win our GRAND PRIZE giveaway: a Ducati StreetFighter motorcycle! We’ll select the winner on Tuesday June 12 at 7 p.m. at the Solutions Center. Or, attend any of our 10 presentations throughought the show and you could win an iPad mini.
Here is a rundown of what we’re speaking on and where, at next week’s IBM Edge:
Emulex Enables Cloud and Big Data with IBM Flex Systems (#1903)
Speaker: Shaun Walsh
Tuesday June 11, 2:00 – 3:00pm
South Pacific Room D
Roundtable: Software Defined Networking (#1823)
Panelist: Sanjeev Datla
Tuesday June 11, 3:30 – 5:00pm
South Seas Ballroom F
Winning with IBM Flex System and Emulex Provided I/O (#1839)
Speaker: Chris Sorensen
Thursday June 13, 10:30 – 11:30am
South Pacific Room D
IBM and Emulex are Better Together (#1929)
Speaker: Chris Sorensen
Monday June 10, 7:40 – 8:00pm
Solution Center Theater #1
Trends & Directions in Ethernet and High Performance I/O (#1720)
Speaker: Brent Mosbrook
Tuesday June 11, 7:30 – 8:15am
Virtual I/O for Virtual Servers (#1719)
Speaker: Tom Boucher
Tuesday June 11, 12:30 – 1:30pm
Thursday June 13, 3:15 – 4:15pm
Maximize your Emulex Provided I/O Solutions with IBM Flex System, BladeCenter and System x (#1925)
Speaker: Tom Boucher
Tuesday June 11, 3:15 – 4:15pm
Trends & Directions in Storage I/O (#1927)
Speaker: Ron Renwick
Wednesday June 12, 8:00 – 8:45am
Configuring FCoE Solutions for IBM Flex System and BladeCenter (#1926)
Speaker: Tom Boucher
Wednesday June 12, 12:30 – 1:30pm
Trends & Directions in Ethernet and Storage I/O (#1928)
Speakers: Brent Mosbrook and Ron Renwick
Thursday June 13, 8:00 – 8:45am
It has been a while since my last blog post, but just recently I read Steve Jobs, The Exclusive Biography by Walter Isaacson…wow, talk about inspiration!
It made me think of what Emulex and IBM are doing together and the strategy we have of making great products greater by using innovative technology and simple design.
In 1974, Steve Jobs was a technician for Atari in Los Gatos, California where he undertook a project from Atari’s Allan Alcorn and Nolan Bushnell to build an arcade machine that had great design, slick performance and most importantly, was technically simple to build. In fact, Bushnell was so insistent that the Atari ‘Breakout’ arcade game should be technically efficient, he offered Steve a bonus of $8,000 if he could do it using 50 chips or less. At which point, Steve managed to persuade his friend Steve Wozniak to help, and four days later using just 45 chips, the ‘Breakout’ prototype was ready. This was one of several small projects that led Steve and ‘Woz’ to create Apple in 1976, and we all know how that turned out!
The ‘Think Different’ campaign was first launched by Apple in 1997. I have seen it before without really thinking about the slogan too much. The thing is, technology visionaries like Steve Jobs just see technology differently. In an age where Bill Gates and Microsoft were very much pushing open systems and the licensing model that eventually ensured Microsoft Windows was to become the world’s most successful operating system (OS), Steve was pushing a closed system. Conversely, Steve wanted to have tight integration between the hardware, software and even to the extent of controlling retail outlets to make sure the customers and users had the very best experience possible when using his products. Steve thought that this philosophy was critical to the success of Apple and there is no doubt that history has proven him right.
In recent years, IBM and Emulex have worked together to bring (in some small part, at least) these philosophies into our products and design mantras. Now I am not suggesting for a second that suddenly IBM has closed its systems for tighter integration of software and hardware. Indeed IBM is one of the leading vendors in the world when it comes to open standards. What I am trying to say is that, certainly from a networking standpoint, that IBM and Emulex are driving the next generation of server platforms including, IBM System x, IBM BladeCenter, IBM Flex Systems and IBM Power Systems that have simple network design and powerful functionality, which creates massive customer value.
IBM has made Emulex their I/O partner of choice because we offer technology that ensures competitive differentiation between IBM and its competitors. The Emulex Virtual Fabric Adapter (VFA) series of Ethernet products for IBM enable IBM customers to virtualize and converge their Ethernet and storage fabrics using virtual Network Interface Card (vNIC) technology, which in turn, reduces the amount of physical network adapters required when configuring a virtualized environment. In fact, technology provided by Emulex has such leadership when it comes to VFAs that IBM has selected technology provided by Emulex to be part of their LAN on motherboard (LOM) design (Steve no doubt would be pleased to hear of a smarter and more integrated solution). Consequently, making the default LOM Ethernet adapter in many of IBM’s latest products, such as IBM System x3750, IBM BladeCenter HS23, IBM Flex System x220 and x240 servers, a solution provided by Emulex, in addition to being widely available across standard NIC, modular LOM (MLOM) and CFFh adapters. Regardless of what form factor your customers choose, IBM uses the same chip ASIC provided by Emulex for all of these option cards, which means IBM customers have only one file of firmware and drivers they need to manage through one management interface, which ensures functionality, such as NIC teaming, becomes seamless, rather than fragmented when compared to other I/O vendors.
When configuring new solutions, additional considerations must now be made. With technology provided by Emulex being selected for the IBM 10Gb Ethernet (10GbE) LOM design, it means that customers are already using the Emulex management tool, OneCommand® Manager (which manages all Ethernet and Fibre Channel adapters provided by Emulex), which can be fully integrated into IBM Systems Director and VMware vCenter™ Server, it becomes harder to see why configuring another I/O vendor would add value? By selecting IBM Ethernet and Fibre Channel products provided by Emulex together, customers will benefit from one firmware and driver file, one management system and easy NIC teaming over one ASIC covering ALL IBM System x, IBM BladeCenter and IBM Flex System formats. To add another I/O vendor increases complexity, doubles the amount of management software required and makes complex networking configuration more difficult. So when contemplating new projects and considering network design, remember the future isn’t what it used to be but Emulex and IBM have simplified network design and made some great products. Design different, think different and WIN different with IBM and Emulex.
These new IBM-branded solutions are refreshes of the current Emulex Virtual Fabric Adapter II for IBM BladeCenter (90Y3550) and Emulex Virtual Fabric Adapter Advanced for IBM BladeCenter (90Y3566).
The new adapters now support pNIC, Virtual Fabric Mode and Switch Independent Mode out-of-the-box. Existing adapters support only pNIC and Virtual Fabric Mode out-of-the-box, but Switch Independent Mode is supported via a firmware upgrade.
Switch Independent Mode allows you to use the adapter with all supported 10Gb Ethernet (10GbE) switches. Virtual Fabric Mode is only supported with Emulex Virtual Fabric Adapters on IBM servers with IBM System Networking’s Virtual Fabric Switch.
The new adapters are supported on the IBM BladeCenter HS22 and HS23E.
The Emulex Virtual Fabric Adapter II for IBM BladeCenter (00Y3266) can be upgraded to FCoE and iSCSI functionality via the Virtual Fabric Adapter Advanced Upgrade (49Y4265).
The official announcement letter from IBM can be found here and the IBM Redbook can be found here. Check out our microsite here for everything you need to know about IBM I/O connectivity provided by Emulex for IBM server, networking and storage solutions.