Monitoring virtual FC Client Traffic

With the LPAR tool, statistics for all virtual FC clients can be displayed at any time using the “vios fcstat” command. This allows you to determine at any time which client LPARs have which I/O throughput (when using NPIV).

Which NPIV-capable FC adapters are available on a virtual I/O server can easily be found out with “vios lsnports“:

$ vios lsnports ms15-vio1
NAME  PHYSLOC                     FABRIC  TPORTS  APORTS  SWWPNS  AWWPNS
fcs0  U78CB.001.XXXXXXX-P1-C5-T1  1       64      62      2032    2012
fcs1  U78CB.001.XXXXXXX-P1-C5-T2  1       64      62      2032    2012
fcs2  U78CB.001.XXXXXXX-P1-C5-T3  1       64      61      2032    1979
fcs3  U78CB.001.XXXXXXX-P1-C5-T4  1       64      61      2032    1979
fcs4  U78CB.001.XXXXXXX-P1-C3-T1  1       64      50      3088    3000
fcs5  U78CB.001.XXXXXXX-P1-C3-T2  1       64      63      3088    3077
$

We display the FC client statistics with the command “vios fcstat”. By default, the data for all virtual FC clients of the specified virtual I/O server are shown every 10 seconds:

$ vios fcstat ms15-vio1
HOSTNAME   PHYSDEV  WWPN                DEV    INREQS    INBYTES      OUTREQS    OUTBYTES     CTRLREQS
ms15-vio1  fcs1     0x210000XXXXX56EC5  fcs1   774.75/s  129.51 MB/s  1332.71/s   92.96 MB/s  20
aixtsmp1   fcs2     0xC050760XXXXX0058  fcs6   318.10/s   83.39 MB/s  481.34/s   126.18 MB/s  0
ms15-vio1  fcs2     0x210000XXXXX56EC6  fcs2   318.10/s   83.39 MB/s  480.78/s   126.03 MB/s  0
aixtsmp1   fcs5     0xC050760XXXXX003E  fcs0   583.98/s   60.35 MB/s  1835.17/s  124.86 MB/s  0
ms15-vio1  fcs5     0x10000090XXXXX12D  fcs5   583.70/s   60.27 MB/s  1836.21/s  124.92 MB/s  0
ms15-vio1  fcs0     0x21000024XXXXXEC4  fcs0   923.19/s  165.08 MB/s  1032.81/s   17.25 MB/s  46
aixtsmp3   fcs1     0xC050760XXXXX00E4  fcs0   775.12/s  129.48 MB/s  1047.32/s   17.15 MB/s  20
aixtsmp3   fcs0     0xC050760XXXXX00DE  fcs1   775.78/s  128.99 MB/s  1037.99/s   17.39 MB/s  20
aixtsmp1   fcs1     0xC050760XXXXX0056  fcs5     0.00/s    0.00 B/s   290.39/s    76.12 MB/s  0
aixtsmp1   fcs0     0xC050760XXXXX0052  fcs4   142.89/s   36.12 MB/s    0.00/s     0.00 B/s   26
ms15-vio1  fcs4     0x10000090XXXXX12C  fcs4   234.97/s    4.58 MB/s  621.78/s    11.12 MB/s  40
cus1dbp01  fcs4     0xC050760XXXXX0047  fcs0   243.55/s    5.05 MB/s  432.33/s     9.95 MB/s  0
cus1dbi01  fcs4     0xC050760XXXXX0044  fcs1     0.94/s   10.42 KB/s   87.28/s   459.26 KB/s  0
...
HOSTNAME   PHYSDEV  WWPN                DEV    INREQS     INBYTES      OUTREQS    OUTBYTES     CTRLREQS
aixtsmp1   fcs5     0xC050760XXXXX003E  fcs0   1772.84/s  162.24 MB/s  1309.30/s   70.60 MB/s  68
ms15-vio1  fcs5     0x10000090XXXXX12D  fcs5   1769.13/s  161.95 MB/s  1305.60/s   70.54 MB/s  68
ms15-vio1  fcs1     0x21000024XXXXXEC5  fcs1   883.55/s   118.97 MB/s  1551.97/s  108.78 MB/s  43
ms15-vio1  fcs2     0x21000024XXXXXEC6  fcs2   201.09/s    52.72 MB/s  497.26/s   130.35 MB/s  0
aixtsmp1   fcs2     0xC050760XXXXX0058  fcs6   201.09/s    52.72 MB/s  495.40/s   129.87 MB/s  0
ms15-vio1  fcs0     0x21000024XXXXXEC4  fcs0   923.54/s   128.89 MB/s  1234.98/s   23.31 MB/s  65
aixtsmp3   fcs0     0xC050760XXXXX00DE  fcs1   876.93/s   118.93 MB/s  1234.98/s   23.32 MB/s  44
aixtsmp3   fcs1     0xC050760XXXXX00E4  fcs0   884.17/s   119.07 MB/s  1223.50/s   23.00 MB/s  43
aixtsmp1   fcs1     0xC050760XXXXX0056  fcs5     0.00/s     0.00 B/s   325.83/s    85.41 MB/s  0
...
^C
$

The LPAR name, the physical FC port (PHYSDEV) on the virtual I/O server, the WWPN of the client adapter, the virtual FC client port (DEV), as well as the number of requests (INREQS and OUTREQS) and thereby transferred bytes (INBYTES and OUTBYTES). The transfer rates are output in KB/s, MB/s or GB/s. The output can be very long on larger systems! The output is sorted according to throughput, i.e. the most active virtual client adapters are output first. With the option ‘-t‘ (top) the output can be restricted to a desired number of data records: e.g. with ‘-t 10‘ only the top ten adapters with the highest throughput are shown. In addition, the interval length (in seconds) can be specified via a further argument, here is a short example:

$ vios fcstat -t 10 ms15-vio1 2
HOSTNAME   PHYSDEV  WWPN                DEV   INREQS     INBYTES      OUTREQS    OUTBYTES     CTRLREQS
ms15-vio1  fcs1     0x21000024XXXXXEC5  fcs1  1034.58/s   86.56 MB/s  2052.23/s  160.11 MB/s  20
ms15-vio1  fcs5     0x10000090XXXXX12D  fcs5  1532.63/s  115.60 MB/s  1235.72/s  118.32 MB/s  40
aixtsmp1   fcs5     0xC050760XXXXX003E  fcs0  1510.33/s  114.88 MB/s  1236.49/s  118.27 MB/s  40
aixtsmp3   fcs1     0xC050760XXXXX00E4  fcs0  1036.11/s   86.67 MB/s  1612.25/s   44.86 MB/s  20
aixtsmp3   fcs0     0xC050760XXXXX00DE  fcs1  1031.50/s   86.29 MB/s  1588.02/s   44.27 MB/s  20
ms15-vio1  fcs0     0x21000024XXXXXEC4  fcs0  1029.58/s   86.31 MB/s  1567.63/s   43.65 MB/s  20
aixtsmp1   fcs1     0xC050760XXXXX0056  fcs5    0.00/s     0.00 B/s   436.52/s   114.43 MB/s  0
ms15-vio1  fcs2     0x21000024XXXXXEC6  fcs2    0.00/s     0.00 B/s   435.75/s   114.23 MB/s  0
aixtsmp1   fcs2     0xC050760XXXXX0058  fcs6    0.00/s     0.00 B/s   432.68/s   113.42 MB/s  0
ms15-vio1  fcs4     0x10000090XXXXX12C  fcs4  144.99/s     0.78 MB/s  478.83/s     2.22 MB/s  46
HOSTNAME   PHYSDEV  WWPN                DEV   INREQS    INBYTES      OUTREQS    OUTBYTES     CTRLREQS
aixtsmp1   fcs5     0xC050760XXXXX003E  fcs0  758.14/s   35.55 MB/s  1822.99/s  112.60 MB/s  0
ms15-vio1  fcs5     0x10000090XXXXX12D  fcs5  757.38/s   35.52 MB/s  1821.46/s  112.59 MB/s  0
ms15-vio1  fcs0     0x21000024XXXXXEC4  fcs0  944.23/s   85.09 MB/s  1657.58/s   41.40 MB/s  2
aixtsmp3   fcs0     0xC050760XXXXX00DE  fcs1  943.47/s   85.15 MB/s  1636.90/s   40.68 MB/s  2
ms15-vio1  fcs1     0x21000024XXXXXEC5  fcs1  949.21/s   84.88 MB/s  1586.74/s   39.41 MB/s  2
aixtsmp3   fcs1     0xC050760XXXXX00E4  fcs0  946.53/s   84.64 MB/s  1584.83/s   39.40 MB/s  2
ms15-vio1  fcs4     0x10000090XXXXX12C  fcs4   39.44/s  449.92 KB/s  676.97/s     3.63 MB/s  10
cus1dbp01  fcs4     0xC050760XXXXX0047  fcs0   29.10/s  471.69 KB/s  310.92/s     1.28 MB/s  4
cus1mqp01  fcs4     0xC050760XXXXX002C  fcs0    1.91/s    4.71 KB/s  230.12/s     1.66 MB/s  0
cus2orap01 fcs4     0xC050760XXXXX000F  fcs0    0.77/s    4.31 KB/s   48.25/s   263.49 KB/s  0
^C
$

The option ‘-s‘ (select) can be used to select and show only data records from a specific client (‘-s hostname = aixtsmp1‘) or only data records from a specific physical port (‘-s physdev = fcs1‘):

$ vios fcstat -s hostname=aixtsmp1 ms15-vio1 2
HOSTNAME  PHYSDEV  WWPN                DEV   INREQS     INBYTES      OUTREQS    OUTBYTES     CTRLREQS
aixtsmp1  fcs5     0xC050760XXXXX003E  fcs0  1858.72/s   51.14 MB/s  1231.82/s  104.20 MB/s  0
aixtsmp1  fcs2     0xC050760XXXXX0058  fcs6    6.94/s     1.82 MB/s    6.94/s     1.82 MB/s  0
aixtsmp1  fcs4     0xC050760XXXXX0042  fcs2    0.39/s     1.19 KB/s    0.39/s   395.05 B/s   0
aixtsmp1  fcs1     0xC050760XXXXX0056  fcs5    0.39/s     7.72 B/s     0.00/s     0.00 B/s   1
aixtsmp1  fcs0     0xC050760XXXXX0052  fcs4    0.00/s     0.00 B/s     0.00/s     0.00 B/s   0
aixtsmp1  fcs3     0xC050760XXXXX005A  fcs7    0.00/s     0.00 B/s     0.00/s     0.00 B/s   0
HOSTNAME  PHYSDEV  WWPN                DEV   INREQS     INBYTES      OUTREQS    OUTBYTES     CTRLREQS
aixtsmp1  fcs5     0xC050760XXXXX003E  fcs0  1760.48/s  111.48 MB/s  1125.70/s   95.20 MB/s  0
aixtsmp1  fcs2     0xC050760XXXXX0058  fcs6    8.53/s     2.24 MB/s  484.61/s   127.04 MB/s  0
aixtsmp1  fcs1     0xC050760XXXXX0056  fcs5    0.00/s     0.00 B/s   469.04/s   122.96 MB/s  0
aixtsmp1  fcs4     0xC050760XXXXX0042  fcs2    0.37/s     1.14 KB/s    0.00/s     0.00 B/s   0
aixtsmp1  fcs0     0xC050760XXXXX0052  fcs4    0.00/s     0.00 B/s     0.00/s     0.00 B/s   0
aixtsmp1  fcs3     0xC050760XXXXX005A  fcs7    0.00/s     0.00 B/s     0.00/s     0.00 B/s   0
^C
$

With the “vios fcstat” command, FC throughput of any LPAR can be shown at any time in an extremely simple way, at the push of a button, so to speak.

If the intervals are smaller, the accuracy of the displayed values suffers. At 2 second intervals the inaccuracy is approx. 10%. However, the relationship between the displayed values is still correct.

The “label” Attribute for FC Adapters

As of AIX 7.2 TL4 and VIOS 3.1.1.10 there is a new attribute “label” for physical FC adapters. The administrator can set this attribute to any character string (maximum 255 characters). Even if the attribute is only informative, it can be extremely useful in PowerVM virtualization environments. If you have a large number of managed systems, it is not always clear to which FC fabric a certain FC port is connected. This can of course be looked up in the documentation of your systems, but it does involve a certain amount of effort. It is easier if you link this information directly with the FC adapters, which is exactly what the new “label” attribute allows in a simple way. On AIX:

# chdev -l fcs0 -U -a label="Fabric_1"
fcs0 changed
# lsattr -El fcs0 -a label -F value
Fabric_1
#

On virtual I/O servers, the attribute can also be set using the padmin account:

/home/padmin> chdev -dev fcs1 -attr label="Fabric_2" -perm
fcs1 changed
/home/padmin> lsdev -dev fcs1 -attr label                
value

Fabric_2
/home/padmin>

The attribute is also defined for older FC adapters.

If the “label” attribute is consistently used, it is always possible to determine online for each FC adapter to which fabric the adapter is connected to. This information only needs to be stored once for each FC adapter.

(Note: The “label” attribute is not implemented for AIX 7.1, at least not until 7.1 TL5 SP6.)

WWPN of FC ports in Open Firmware

The following article deals with WWPN of FC ports in Open Firmware.

Port and node WWNs of FC ports can be found very easily in the Open Firmware, even when the ioinfo command is no longer available, as is the case with new POWER9 firmware. The hardware structure of a POWER system is available in the Open Firmware in the form of a device tree. Hardware components such as PCI bridges, processors and PCI cards are represented as device nodes in this tree.

With the command “dev /” you can access the device nodes, starting with the root node (“/” or slash):

0 > dev /  ok
0 >

In the device tree you can navigate with the commands dev, ls and pwd similar to the Unix file system. An ls on the root node shows all available device nodes (as well as some “package nodes” which are not discussed here).

The hierarchy is visualized in the device tree by indenting the device nodes:

0 > ls 
0000020939c0: /ibm,serial
000002094ae8: /chosen
000002094d60: /packages
000002094e58:   /disassembler
...0000020af578: /cpus
0000020b5200:   /PowerPC,POWER7@0
...
0000020ba640: /memory@0
...
00000226cad0: /pci@800000020000120
00000229d750:   /pci@0
0000022a0018:     /pci@2
0000022a28e0:       /ethernet@0
0000022b4a28:       /ethernet@0,1
0000022c6b70:     /pci@4
0000022c9438:       /ethernet@0
0000022db580:       /ethernet@0,1
000002277fd8: /pci@800000020000121
0000022ed7d0:   /fibre-channel@0
0000023026e0:     /fp
000002303240:     /disk
000002304de0:     /tape
000002306270:   /fibre-channel@0,1
00000231b180:     /fp
00000231bce0:     /disk
00000231d880:     /tape
...
ok
0 >

The example output shows 2 FC ports. Both FC ports are children of the device node pci@800000020000121, which can be found directly under the root node /.

With the command “dev / pci@800000020000121” we first navigate to this node and then display the child or child nodes using “ls“:

0 > dev /pci@800000020000121  ok
0 > ls
0000022ed7d0: /fibre-channel@0
0000023026e0:   /fp
000002303240:   /disk
000002304de0:   /tape
000002306270: /fibre-channel@0,1
00000231b180:   /fp
00000231bce0:   /disk
00000231d880:   /tape
ok
0 >

We next move into the device node of the first FC port fiber-channel@0.

With the command “pwd” we check briefly the position in the device tree and then use “ls” to look at the available subnodes:

0 > dev fibre-channel@0  ok
0 > pwd /pci@800000020000121/fibre-channel@0 ok
0 > ls
0000023026e0: /fp
000002303240: /disk
000002304de0: /tape
ok
0 >

Each device node has a number of properties, which depend on the type of the underlying hardware component.

The properties of a device node can be displayed with the command “.properties” (the command name begins with a “.“):

0 > .properties
ibm,loc-code            U5802.001.008C110-P1-C2-T1
vendor-id               000010df
device-id               0000f100
...
name                    fibre-channel
...
manufacturer            456d756c 657800
copyright               436f7079 72696768 74202863 29203230 30302d32 30313220 456d756c 657800
device_type             fcp
model                   10N9824
...
port-wwn                10000000 c9b12345
node-wwn                20000000 c9b12345
...
ok
0 >

In addition to the location code, the port WWN (port-wwn) and the node WWN (node-wwn) are displayed.

If you would like to know more about the structure of WWNs, please refer to the article:  Numbers: FC World Wide Names (WWNs)

Of course, you can also find out the MAC address of an ethernet port in the same way. With “dev ..” you can move up one level in the device tree, just like in a Unix file system. But you can also abbreviate and go straight to the top, which we do here in the following. Then we display all available device nodes again:

0 > dev /  ok
0 > ls 
...
00000226cad0: /pci@800000020000120
00000229d750:   /pci@0
0000022a0018:     /pci@2
0000022a28e0:       /ethernet@0
0000022b4a28:       /ethernet@0,1
0000022c6b70:     /pci@4
0000022c9438:       /ethernet@0
0000022db580:       /ethernet@0,1
...
ok
0 >

As an example, we select the device node /pci@800000020000120/pci@0/pci@2/ethernet@0.1 and again let us display the properties:

0 > dev /pci@800000020000124/pci@0/pci@2/ethernet@0,1  ok
0 > pwd /pci@800000020000124/pci@0/pci@2/ethernet@0,1 ok
0 > .properties
ibm,loc-code            U5802.001.008C110-P1-C4-T2
vendor-id               00008086
device-id               000010bc
...
name                    ethernet
...
device_type             network
...
max-frame-size          00000800
address-bits            00000030
local-mac-address       00145eea 1234
mac-address             00145eea 1234
...
0 >

The MAC address is available here by the property mac-address.

If you want to leave the device tree, you can do this with the command “device-end“:

0 > device-end  ok
0 >

We hope this article about WWPN of FC ports in Open Firmware was both helpful and informative.

Numbers: FC World Wide Names (WWNs)

Most of us know WWNs as 64-bit WWNs, written as 16 hexadecimal digits. The knowledge that there are different formats of WWNs and that there are also 128-bit WWNs is not quite as well known. In this article, therefore, the different formats of WWNs are briefly presented.

The basic structure of 64-bit WWNs looks like this:

+---+----------------+
|NAA| NAME           |
+---+----------------+
4-bit 60-bit

The 4-bit NAA (Network Address Authority) field specifies the type of address and the format of the address.

There are a number of different possibilities for the 60-bit NAME field.

 

1. Format 1 Address (NAA = 0001)

+---+--------+------------------------+
|NAA|Reserved| 48-bit IEEE MAC Address|
+---+--------+------------------------+
4-bit 12-bit   48-bit

In the Reserved (12-bit) field, all bits must be set to 0!

Example:

1 000 00507605326d (To clarify the format, the fields are separated by spaces)

 

2. Format 2 Address (NAA = 0010)

+---+---------------+-----------------------+
|NAA|Vendor Assigned|48-bit IEEE MAC Address|
+---+---------------+-----------------------+
4-bit  12-bit         48-bit

The 12-bit “Vendor Assigned” field can be used arbitrarily by the manufacturer.

Example:

2 001 00507605326d (To clarify the format, the fields are separated by spaces)

 

3. Format 3 Address (NAA = 0011)

+---+-----------------+
|NAA|Vendor Assigned  |
+---+-----------------+
4-bit 60-bit

The field “Vendor Assigned” (60-bit) is assigned arbitrarily by the manufacturer. Thus, this type of address is not unique worldwide and therefore usually not found in practice.

Example:

3 0123456789abcde (To clarify the format, the fields are separated by spaces)

 

4. Format 4 Address (NAA = 0100)

+---+---------+--------------+
|NAA|Reserved | IPv4 Address |
+---+---------+--------------+
4-bit 28-bit     32-bit

The “IPv4 Address” (32-bit) field contains a 32-bit IPv4 address.

Example for IP 10.0.0.1:

4 0000000 0a000001 (To clarify the format, the fields are separated by spaces)

 

5. Format 5 Address (NAA = 0101)

+---+-------+-----------------+
|NAA| OUI   | Vendor Assigned |
+---+-------+-----------------+
4.bit 24-bit 36-bit

The OUI (24-bit) field contains the 24-bit IEEE-assigned ID (Organizational Unique ID).

The field “Vendor Assigned” (36-bit) can be assigned arbitrarily by the manufacturer.

Example:

5 005076 012345678 (To clarify the format, the fields are separated by spaces)

 

6. Format 6 Address (NAA = 0110)

Format 6 addresses are 128-bit addresses and are often used for LUNs on the SAN.

+---+-------+---------------+-------------------------+
|NAA|  OUI  |Vendor Assigned|Vendor Assigned Extension|
+---+-------+---------------+-------------------------+
4.bit 24-bit  36-bit          64-bit

The OUI (24-bit) field contains the 24-bit ID assigned by the IEEE.

The field “Vendor Assigned” (36-bit) can be arbitrarily assigned by the manufacturer.

The field “Vendor Assigned Extension” (64-bit) can also be assigned arbitrarily by the manufacturer.

Example:

6 005076 012345678 0123456789abcdef (To clarify the format, the fields are separated by spaces)

 

7. IEEE EUI-64 Address (NAA=11)

In the case of this address format, the NAA field is shortened to only 2 bits, where NAA is 11.

+---+-------------+---------------+
|NAA|OUI shortened|Vendor Assigned|
+---+-------------+---------------+
2-bit 22-bit       40-bit

The “OUI shortened” field (22-bit) is a 22-bit shortened version of the IEEE-assigned 24-bit ID.

(The two least significant bits of the first byte are omitted and the remaining 6 bits are shifted 2 bits to the right to make room for the two NAA bits.)

The field “Vendor Assigned” (40-bit) can be arbitrarily assigned by the manufacturer.

These types of addresses are often used in the area of virtualization, e.g. when it comes to NPIV (N_Port ID Virtualization).

Example:

c05076 0123456789 (To clarify the format, the fields are separated by spaces)