Virtualization gives us the benefit to better utilize hardware resources. In the past applications only used a small piece of hardware resources. Applying a hypervisor on top of the hardware enabled us to run multiple applications/servers on the same piece of hardware therefore better utilizing the hardware.
However there are applications demanding huge amounts of resources to perform well. Immediately we can start a discussion whether it’s better to keep these applications on physical hardware instead of virtualizing them. Virtualizing these types of applications still bring lots advantages like HA (high availability) in case of hardware failure, but also disaster recovery can be easier to implement in a virtual environment (if the application doesn’t provide an easy out of the box DR ability).
How SR-IOV ties in
SR-IOV or Single-Root I/O Virtualization bypasses the very well-known vSwitch. It basically allows a virtual machine to directly communicate with the network adapter instead of the vSwitch. Eliminating the vSwitch will decrease latency. The following VIDEO explains SR-IOV in more detail.
Step 1: BIOS
|1. Enable the SR-IOV feature in the advanced section of the BIOS.
2. Enable Advanced Mode Support for the Emulex network card in the BIOS (HOW TO).
Step 2: Flex Fabric
|1. Power down the virtual machine.|
2. Create a server profile.
3. NIC (Port) 1 & 2 are meant for ESXi host management only.
4. NIC (Port) 3 & 4 are meant for virtual machine production network.
5. Start the server so the new flex fabric settings are applied during boot sequence.
NOTE 1: Don’t use NIC (Port) 1 & 2 for virtual machine production, because SR-IOV in this case uses all above the first 2.
NOTE 2: Make sure that the production VLAN or the VLAN were the application resides is on a separate VLAN than the ESXi management of your ESXi host. Otherwise the SR-IOV implementation will not be successful.
|6. Enable the SR-IOV option for the NIC(s) needed in the server profile.|
7. Click the edit button.
|8. The AUTO setting will enable 24 virtual functions for SR-IOV. If more VFs are needed the option Custom can be selected.|
9. For Boot order use the default USE-BIOS setting.
10. Click OK.
Step 3: ESXi Configuration
Step 4: Virtual Machine Configuration
- You can no longer suspend a virtual machine.
- You can no longer vMotion a virtual machine.
- You can no longer take or restore snapshots of a virtual machine.
- You can no longer hot add CPU or RAM (shutdown the VM, add CPU or RAM and power on the VM).
- If a cluster is available, HA can recover the VM but is not able to start because the wrong PCI is mapped. Therefore a manual action is required:
- Delete the PCI devices of the old ESXi host.
- Add the PCI devices of the new ESXi host with preferably the same ID’s, so no reconfiguration in the VM is needed.
- Minimal HP Flex Fabric firmware: 4.11.
- Minimal HP ProLiant Blade 460c G8 firmware: I31.
- VMware Enterprise Plus License.
- Enable SR-IOV in the BIOS.
- Make sure the Advanced Mode Support is Enabled for the Emulex network card (HOW TO).
This article is based upon the following hardware configuration:
- HP Blade C Enclosure.
- HP ProLiant Blade 460c G8.
- HP Flex Fabric.
- VMware ESXi 5.5 U2 (special HP VMware build, includes all the correct drivers needed for SR-IOV, if incorrect drivers are used the settings will work only temporarily and will not be persistent).