In larger environments with many managed systems and multiple SAN fabrics, it’s not always clear which SAN fabric an FC port belongs to despite good documentation. In many cases, the hardware is far from the screen, possibly even in a very different building or geographically farther away, so you can not just check the wiring on site.
This blog post will show you how to use Live Partition Mobility (LPM) to find all the FC ports that belong to a given SAN fabric.
We use the LPAR tool for the sake of simplicity, but you can also work with commands from the HMC CLI without the LPAR tool, so please continue reading even if the LPAR tool is not available!
In the following, we have named our SAN fabrics “Fabric1” and “Fabric2.” However, the procedure described below can be used with any number of SAN fabrics.
Since LPM is to be used, we first need an LPAR. We create the LPAR on one of our managed systems (ms09) with the LPAR tool:
$ lpar –m ms09 create fabric1 Creating LPAR fabric1: done Register LPAR done $
Of course you can also use the HMC GUI or the HMC CLI to create the LPAR. We named the new LPAR after our SAN Fabric “fabric1“. Every other name is just as good!
Next, our LPAR needs a virtual FC adapter mapped to an FC port of fabric “Fabric1“:
$ lpar –p standard addfc fabric1 10 ms09-vio1 fabric1 10 ms09-vio1 20 $
The LPAR tool has selected slot 20 for the VFC server adapter on VIOS ms09-vio1 and created the client adapter as well as the server adapter. Of course, client and server adapters can be created in exactly the same way via the HMC GUI or the HMC CLI. Since the LPAR is not active, the ‘-p standard‘ option specified that only the profile should be adjusted.
To map the VFC server adapter to a physical FC port, we need the vfchost adapter number on the VIOS ms09-vio1:
$ vios npiv ms09-vio1 VIOS ADAPT NAME SLOT CLIENT OS ADAPT STATUS PORTS … ms09-vio1 vfchost2 C20 (3) unknown - NOT_LOGGED_IN 0 … $
In slot 20 we have the vfchost2, so this must now be mapped to an FC port of fabric “Fabric1“. We map to the FC port fcs8, which we know to belong to fabric “Fabric1“. If we are wrong, we will see this shortly.
Let’s take a look at the WWPNs for the virtual FC Client Adapter:
$ lpar -p standard vslots fabric1 SLOT REQ TYPE DATA 0 yes serial/server remote: (any)/any hmc=1 1 yes serial/server remote: (any)/any hmc=1 10 no fc/client remote: ms09-vio1(1)/20 c050760XXXXX00b0,c050760XXXXX00b1 $
Equipped with the WWPNs, we now ask our storage colleagues to create a small LUN for these WWPNs, which should only be visible in the fabric “Fabric1“. After the storage colleagues have created the LUN and adjusted the zoning accordingly, we activate our new LPAR in OpenFirmware mode and open a console:
$ lpar activate –p standard –b of fabric1 $ lpar console fabric1 Open in progress Open Completed. IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM ... 1 = SMS Menu 5 = Default Boot List 8 = Open Firmware Prompt 6 = Stored Boot List Memory Keyboard Network SCSI Speaker ok 0 >
Of course, this is also possible without problems with GUI or HMC CLI.
In OpenFirmware mode we start ioinfo and check if the small LUN is visible. If it is not visible, then the FC port fcs8 does not belong to the right fabric!
0 > ioinfo !!! IOINFO: FOR IBM INTERNAL USE ONLY !!! This tool gives you information about SCSI,IDE,SATA,SAS,and USB devices attached to the system Select a tool from the following 1. SCSIINFO 2. IDEINFO 3. SATAINFO 4. SASINFO 5. USBINFO 6. FCINFO 7. VSCSIINFO q - quit/exit ==> 6 FCINFO Main Menu Select a FC Node from the following list: # Location Code Pathname ------------------------------------------------- 1. U9117.MMC.XXXXXXX7-V10-C10-T1 /vdevice/vfc-client@3000000a q - Quit/Exit ==> 1 FC Node Menu FC Node String: /vdevice/vfc-client@3000000a FC Node WorldWidePortName: c050760XXXXXX0016 ------------------------------------------ 1. List Attached FC Devices 2. Select a FC Device 3. Enable/Disable FC Adapter Debug flags q - Quit/Exit ==> 1 1. 500507680YYYYYYY,0 - 10240 MB Disk drive Hit a key to continue... FC Node Menu FC Node String: /vdevice/vfc-client@3000000a FC Node WorldWidePortName: c050760XXXXXX0016 ------------------------------------------ 1. List Attached FC Devices 2. Select a FC Device 3. Enable/Disable FC Adapter Debug flags q - Quit/Exit ==> q
The LUN appears, the WWPN 500507680YYYYYYY is the WWPN of the corresponding storage port, which is unique worldwide and can only be seen in the fabric “Fabric1“!
Activating the LPAR in OpenFirmware mode has served two purposes, firstly to verify that the LUN is visible and our mapping to fcs8 was correct, secondly, the system now has the information which WWPNs need to be found during an LPM operation, so that the LPAR can be moved!
We deactivate the LPAR again.
$ lpar shutdown –f fabric1 $
If we now perform an LPM validation on the inactive LPAR, then a validation can only be successful on a managed system that has a virtual I/O server with a connection to the fabric “Fabric1“. Using a for loop, let’s try that for some managed systems:
$ for ms in ms10 ms11 ms12 ms13 ms14 ms15 ms16 ms17 ms18 ms19 do echo $ms lpar validate fabric1 $ms >/dev/null 2>&1 if [ $? -eq 0 ] then echo connected else echo not connected fi done
The command to validate on the HMC CLI is, migrlpar,.
Since we are not interested in validation messages, we redirect all validation messages to /dev/null.
Here’s the output of the for loop:
ms10 connected ms11 connected ms12 connected ms13 connected ms14 connected ms15 connected ms16 connected ms17 connected ms18 connected ms19 connected
Obviously, all managed systems are connected to fabric “Fabric1“. That’s not very surprising, because they were cabled exactly like that.
It would be more interesting to know which FC port on the managed systems (Virtual I/O servers) are connected to the fabric “Fabric1“. To do this, we need a list of virtual I/O servers for each managed system and the list of NPIV-capable FC ports for each virtual I/O server.
The list of virtual I/O servers can be obtained easily with the following command:
$ vios -m ms11 list ms11-vio1 ms11-vio2 $
On the HMC CLI you can use the command: lssyscfg -r lpar -m ms11 -F “name lpar_env”.
The NPIV-capable ports can be found out with the following command:
$ vios lsnports ms11-vio1 ms11-vio1 name physloc fabric tports aports swwpns awwpns ms11-vio1 fcs0 U78AA.001.XXXXXXX-P1-C5-T1 1 64 60 2048 1926 ms11-vio1 fcs1 U78AA.001.XXXXXXX-P1-C5-T2 1 64 60 2048 2023 ... $
The command lsnports is used on the virtual I/O server. Of course you can do this without the LPAR tool.
With the LPM validation (and of course also with the migration) one can indicate which FC port on the target system is to be used, we show this here once with two examples:
$ lpar validate fabric1 ms10 virtual_fc_mappings=10/ms10-vio1///fcs0 >/dev/null 2>&1 $ echo $? 0 $ lpar validate fabric1 ms10 virtual_fc_mappings=10/ms10-vio1///fcs1 >/dev/null 2>&1 $ echo $? 1 $
The validation with target ms10-vio1 and fcs0 was successful, i.e. this FC port is attached to fabric “Fabric1“. The validation with targets ms10-vio1 and fcs1 was not successful, i.e. that port is not connected to the fabric “Fabric1“.
Here is the command that must be called on the HMC, if the LPAR tool is not used:
$ lpar -v validate fabric1 ms10 virtual_fc_mappings=10/ms10-vio1///fcs0 hmc02: migrlpar -m ms09 -o v -p fabric1 -t ms10 -v -d 5 -i 'virtual_fc_mappings=10/ms10-vio1///fcs0' $
To find out all the FC ports that are connected to the fabric “Fabric1“, we need to loop through the managed systems to be checked, for each managed system we then need a loop across all VIOS of the managed system and finally a loop over each FC ports of the VIOS performing an LPM validation.
We have put things together in the following script. To make sure that it does not get too long, we have omitted some checks:
$ cat bin/fabric_ports
#! /bin/ksh
# Copyright © 2018, 2019 by PowerCampus 01 GmbH
LPAR=fabric1
STATE=$( lpar prop -F state $LPAR | tail -1 )
print "LPAR: $LPAR"
print "STATE: $STATE"
if [ "$STATE" != "Not Activated" ]
then
print "ERROR: $LPAR must be in state 'Not Activated'"
exit 1
fi
fcsCount=0
fcsSameFabricCount=0
for ms in $@
do
print "MS: $ms"
viosList=$( vios -m $ms list )
for vios in $viosList
do
rmc_state=$( lpar -m $ms prop -F rmc_state $vios | tail -1 )
if [ "$rmc_state" = "active" ]
then
fcList=
vios -m $ms lsnports $vios 2>/dev/null | \
while read vio fcport rest
do
if [ "$fcport" != "name" ]
then
fcList="${fcList} $fcport"
fi
done
for fcport in $fcList
do
print -n "${vios}: ${fcport}: "
lpar validate $LPAR $ms virtual_fc_mappings=10/${vios}///${fcport} </dev/null >/dev/null 2>&1
case "$?" in
0)
print "yes"
fcsSameFabricCount=$( expr $fcsSameFabricCount + 1 )
;;
*) print "no" ;;
esac
fcsCount=$( expr $fcsCount + 1 )
done
else
print "${vios}: RMC not active"
fi
done
done
print "${fcsCount} FC-ports investigated"
print "${fcsSameFabricCount} FC-ports in same fabric"
$
As an illustration we briefly show a run of the script over some managed systems. We start the script with time to see how long it takes:
$ time bin/fabric_ports ms10 ms11 ms12 ms13 ms14 ms15 ms16 ms17 ms18 ms19 LPAR: fabric1 STATE: Not Activated MS: ms10 ms10-vio3: RMC not active ms10-vio1: fcs0: yes ms10-vio1: fcs2: yes ms10-vio1: fcs4: no ms10-vio1: fcs6: no ms10-vio2: fcs0: yes ms10-vio2: fcs2: yes ms10-vio2: fcs4: no ms10-vio2: fcs6: no MS: ms11 ms11-vio3: RMC not active ms11-vio1: fcs0: no ms11-vio1: fcs1: no ms11-vio1: fcs2: no ms11-vio1: fcs3: yes ms11-vio1: fcs4: no … ms19-vio2: fcs2: no ms19-vio2: fcs3: no ms19-vio2: fcs0: no ms19-vio2: fcs1: no ms19-vio2: fcs4: no ms19-vio2: fcs5: no 132 FC-ports investigated 17 FC-ports in same fabric real 2m33.978s user 0m4.597s sys 0m8.137s $
In about 150 seconds, 132 FC ports were examined (LPM validations performed). This means a validation took about 1 second on average.
We have found all the FC ports that are connected to the fabric “Fabric1“.
Of course, this can be done analogously for other fabrics.
A final note: not all ports above are cabled!