Virtual Network Interface Controller (vNIC)

vNIC adapter with 2 vNIC backing devices and vNIC failover.

The big disadvantage of SR-IOV, as described above, is that LPARs with logical SR-IOV ports cannot be moved (LPM). After the introduction of SR-IOV on POWER systems, there were a number of suggestions for workarounds. However, all of these workarounds require, on the one hand, a special configuration and, on the other hand, a number of reconfigurations to be carried out before and after an LPM operation. In everyday practice, however, this unnecessarily complicates LPM operations.

With the introduction of vNICs, client LPARs can use SR-IOV adapters and still support LPM. As with VSCSI and VFC, a pair of adapters is used for this purpose: the so-called vNIC adapter is used in a virtual slot on the client LPAR and an associated vNIC server adapter is used on a virtual I/O server. The logical SR-IOV port is assigned to the virtual I/O server. The vNIC server adapter, also known as the vNIC backing device, serves as a proxy for the logical SR-IOV port. The interaction of the various adapters is shown in figure 7.19.

Communication path of vNIC for control information and data.
Figure 7.19: Communication path of vNIC for control information and data.

In order to achieve good performance, only control information is transmitted from the vNIC adapter of the client to the vNIC server adapter on the virtual I/O server, which is transmitted in turn from the vNIC server adapter, via the associated logical SR-IOV port (ent adapter), to the corresponding logical port (virtual function) of the SR-IOV adapter. The data itself is transferred between the vNIC client adapter and the logical port of the SR-IOV adapter via DMA (Direct Memory Access) with the help of the hypervisor. In particular, there is no copying of the data via the virtual I/O server. The vNIC adapter on the client is a purely virtual adapter, so LPM works without any problems. The client does not own the logical SR-IOV port and does not access it itself via the PCIe bus (switch).

Adding Logical SR-IOV Ports

SR-IOV Ethernet port with internal switch and 3 logical ports.

In order that an LPAR can use a virtual function of an SR-IOV adapter in PowerVM, a so-called logical port must be created for the LPAR. Which logical ports already exist can be displayed with the command “ms lssriov” and the option “-l” (logical port):

$ ms lssriov -l ms03
LOCATION_CODE  ADAPTER  PPORT  LPORT  LPAR  CAPACITY  CURR_MAC_ADDR  CLIENTS
$

Since the SR-IOV adapters have just been configured to shared mode, there are of course no logical ports yet. To add a logical SR-IOV port to an LPAR, the command “lpar addsriov” (add SR-IOV logical port) is used. In addition to the LPAR, the adapter ID and the port ID of the physical port must be specified. Alternatively, a unique suffix of the physical location code of the physical port can also be specified:

$ lpar addsriov aix22 P1-C11-T1
$

The creation can take a few seconds. A quick check shows that a logical port has actually been created:

$ ms lssriov -l ms03
LOCATION_CODE                   ADAPTER  PPORT  LPORT     LPAR   CAPACITY  CURR_MAC_ADDR  CLIENTS
U78AA.001.VYRGU0Q-P1-C11-T1-S1  1        0      27004001  aix22  2.0       a1b586737e00   -
$

Similar to a managed system for virtual Ethernet, an internal switch is implemented on the SR-IOV adapters for each physical Ethernet port, see figure above. One of the virtual functions is assigned to each logical port. The associated LPARs access the logical ports directly via the PCI Express bus (PCIe switch).

An LPAR can easily have several logical SR-IOV ports. With the command “lpar lssriov” (list SR-IOV logical ports) all logical ports of an LPAR can be displayed:

$ lpar lssriov aix22
LPORT     REQ  ADAPTER  PPORT  CONFIG_ID  CAPACITY  MAX_CAPACITY  PVID  VLANS  CURR_MAC_ADDR  CLIENTS
27004001  Yes  1        0      0          2.0       100.0         0     all    a1b586737e00   -
$

There are a number of attributes that can be specified for a logical port when it is created. Among other things, the following properties can be configured:

    • capacity – the guaranteed capacity for the logical port.
    • port_vlan_id – the VLAN ID for untagged packets or 0 to switch off VLAN tagging.
    • promisc_mode – switch promiscuous mode on or off.

The complete list of attributes and their possible values can be found in the online help (“lpar help addsriov“).

As an example we add another logical port with port VLAN-ID 55 and a capacity of 20% to the LPAR aix22:

$ lpar addsriov aix22 P1-C4-T2 port_vlan_id=55 capacity=20
$

The generated logical port thus has a guaranteed share of 20% of the bandwidth of the physical port P1-C4-T2! The LPAR now has 2 logical SR-IOV ports:

$ lpar lssriov aix22
LPORT     REQ  ADAPTER  PPORT  CONFIG_ID  CAPACITY  MAX_CAPACITY  PVID  VLANS  CURR_MAC_ADDR  CLIENTS
27004001  Yes  1        0      0          2.0       100.0         0     all    a1b586737e00   -
2700c003  Yes  3        2      1          20.0      100.0         55    all    a1b586737e01   -
$

After the logical ports have been added to the LPAR using the PowerVM Hypervisor, they appear in the Defined state. The logical ports appear under AIX as ent devices, like all other Ethernet adapters!

aix22 # lsdev -l ent\*
ent0 Available       Virtual I/O Ethernet Adapter (l-lan)
ent1 Defined   00-00 PCIe2 10GbE SFP+ SR 4-port Converged Network Adapter VF (df1028e214100f04)
ent2 Defined   01-00 PCIe2 100/1000 Base-TX 4-port Converged Network Adapter VF (df1028e214103c04)
aix22 #

After the config manager cfgmgr has run, the new ent devices are in the Available state and can be used in exactly the same way as all other Ethernet adapters.

7.6. SR-IOV

7.6.1. Activating Shared Modes

7.6.2. Configuration of Physical SR-IOV Ports

7.6.3. Adding Logical SR-IOV Ports

7.6.4. Changing a Logical SR-IOV Port

7.6.5. Removing Logical SR-IOV Ports

7.6.6. Setting an SR-IOV Adapter from Shared back to Dedicated