7.6.1. Activating Shared Mode
SR-IOV capable PCIe cards are initially in “dedicated” operating mode. This corresponds to classic cards without SR-IOV support. The card must then be assigned to an LPAR (usually a virtual I/O server) and can only be used by this LPAR. Direct access from other LPARs is not possible.
A list of the SR-IOV capable PCIe adapters can be displayed with the help of the command “ms lssriov” (list SR-IOV):
$ ms lssriov ms03
PHYS_LOC SLOT_ID CONFIG_STATE SRIOV_STATUS ADAPTER_ID PHYS_PORTS LOGICAL_PORTS
U78AA.001.VYRGU0Q-P1-C4 21010020 dedicated null null null null
U78AA.001.VYRGU0Q-P1-C6 2102001b dedicated null null null null
U78AA.001.VYRGU0Q-P1-C11 21020013 dedicated null null null null
U78AA.001.VYRGU0Q-P1-C12 21030014 dedicated null null null null
$
There are 4 PCIe adapter cards in slots P1-C4, P1-C6, P1-C11 and P1-C12. All 4 cards are still in dedicated mode (column CONFIG_STATE). In order to be able to use the virtualization with SR-IOV, the cards must first be brought into the so-called “shared mode“. This is done with the help of the command “ms chsriov” (change SR-IOV), whereby the slot ID (column SLOT_ID) or a unique suffix of the physical location code must be specified:
$ ms chsriov ms03 21010020 shared
$ ms chsriov ms03 P1-C6 shared
$ ms chsriov ms03 21020013 shared
$ ms chsriov ms03 P1-C12 shared
$
Each SR-IOV adapter is assigned a unique adapter ID when switching to shared mode. The adapters are simply numbered consecutively. If you want to use a specific adapter ID for an adapter, you can use the adapter_id attribute:
$ ms chsriov ms03 21020013 shared adapter_id=1
$ ms chsriov ms03 P1-C6 shared adapter_id=2
$ ms chsriov ms03 21010020 shared adapter_id=3
$ ms chsriov ms03 P1-C12 shared adapter_id=4
$
When switching to shared mode, the adapters are reinitialized. This can take some time. Not all information is available while an adapter is initializing, as shown in the following output:
$ ms lssriov ms03
PHYS_LOC SLOT_ID CONFIG_STATE SRIOV_STATUS ADAPTER_ID PHYS_PORTS LOGICAL_PORTS
U78AA.001.VYRGU0Q-P1-C4 21010020 sriov initializing unavailable unavailable unavailable
U78AA.001.VYRGU0Q-P1-C6 2102001b sriov running 2 4 48
U78AA.001.VYRGU0Q-P1-C11 21020013 sriov initializing unavailable unavailable unavailable
U78AA.001.VYRGU0Q-P1-C12 21030014 sriov missing unavailable unavailable unavailable
$
The adapter in slot P1-C6 is initialized and ready for operation (running), the two adapters in slot P1-C4 and P1-C11 are still initializing and the adapter in slot P1-C12 has not yet started initializing.
After a few seconds (usually less than a minute) all adapters are initialized and the output looks like this:
$ ms lssriov ms03
PHYS_LOC SLOT_ID CONFIG_STATE SRIOV_STATUS ADAPTER_ID PHYS_PORTS LOGICAL_PORTS
U78AA.001.VYRGU0Q-P1-C11 21020013 sriov running 1 4 48
U78AA.001.VYRGU0Q-P1-C6 2102001b sriov running 2 4 48
U78AA.001.VYRGU0Q-P1-C4 21010020 sriov running 3 4 48
U78AA.001.VYRGU0Q-P1-C12 21030014 sriov running 4 4 48
$
Each of the 4 adapters has 4 physical ports (column PHYS_PORTS) and support up to 48 logical ports (LOGICAL_PORTS). Logical ports can be created by the administrator, whereby these are then assigned to an LPAR at the same time. One of the available virtual functions is assigned to each logical port. The physical functions can only be used by the hypervisor. This ensures that LPARs cannot make any changes to the configuration of an SR-IOV adapter.