HMC: Which network interfaces have a link

The HMC allows the network interface configuration to be displayed using the “lshmc -n” command. It cannot be determined from the output whether a network interface currently has a link or not. However, this can be determined relatively easily via the sysfs file system, mounted under /sys. First, you can use the following command to determine which network interfaces the HMC has:

hscroot@hmc02:~> ls -1d /sys/class/net/eth*
/sys/class/net/eth0
/sys/class/net/eth1
/sys/class/net/eth2
/sys/class/net/eth3
/sys/class/net/eth4
/sys/class/net/eth5
hscroot@hmc02:~>

(Output is from a 7063-CR2.)

The link status is recorded in the operstate file and can be shown with a simple cat command:

hscroot@hmc02:~> cat /sys/class/net/eth*/operstate
up
up
down
down
down
down
hscroot@hmc02:~>

In the example case shown, the first two network interfaces (eth0 and eth1) have the link status up.

The current speed can be displayed by showing the content of the speed file:

hscroot@hmc02:~> cat /sys/class/net/eth*/speed
1000
1000
-1
-1
-1
-1
hscroot@hmc02:~>

There are a number of additional files for each network interface with additional information.

IBM PowerVM: Add a virtual Ethernet adapter to an LPAR

A virtual Ethernet adapter is to be added to the LPAR aix01 with IBM PowerVM. The data in detail:

    • HMC: hmc01
    • managed system: ms25
    • LPAR: aix01
    • profile: standard
    • virtual slot number: 4
    • Port-VLAN-ID: 900
    • virtual Ethernet switch: ETHERNET0(default)
    • additional VLANs: none

The command on the associated HMC hmc01 is:

hscroot@hmc01:~> chhwres -m ms25 -r virtualio --rsubtype eth -o a -p aix01 -s 2 -a 'ieee_virtual_eth=0,port_vlan_id=900'
hscroot@hmc01:~>

If the currently used profile of the LPAR is not automatically synchronized, then the additional virtual Ethernet adapter should also be added to the profile:

hscroot@hmc01:~> chsyscfg -r prof -m ms25 -i 'lpar_name=aix01,name=standard,"virtual_eth_adapters+=""4/0/900///0"""'
hscroot@hmc01:~>

With our LPAR tool, the command to use looks like this:

$ lpar addeth aix01 4 900
$

The current profile is automatically adjusted.

Detailed information on the LPAR tool and virtual Ethernet adapters can be found here: Virtual Ethernet

Error when deleting a SEA

The following SEA on a virtual I/O server is no longer required:

$ lsdev -dev ent48
name             status      description
ent48            Available   Shared Ethernet Adapter
$

Attempting to delete the SEA using rmvdev fails with the following error message:

$ rmvdev -sea ent48

Some error messages may contain invalid information
for the Virtual I/O Server environment.

Method error (/usr/lib/methods/ucfgcommo):
        0514-062 Cannot perform the requested function because the
                 specified device is busy.

$

The SEA is still in use. One possibility is the use of LLDP. This can be checked with the lsdev command:

$ lsdev -dev ent48 -attr lldpsvc
value

yes
$

In this case LLDP is active on the SEA and must first be stopped before the SEA can be deleted. Stopping LLDP on the SEA can be easily done by changing the lldpsvc attribute to the value “no“:

$ chdev -dev ent48 -attr lldpsvc=no
ent48 changed
$

Another attempt to delete the SEA ent48 is now successful:

$ rmvdev -sea ent48
ent48 deletedError deleting a SEA
$

More information on SEAs can be found here: Shared Ethernet Adapter

LPAR tool: Console

lpar console

A console can be opened for an LPAR at any time using the LPAR tool:

$ lpar console lpar01
Open in progress
Open completed.
PowerPC Firmware
SMS 1.7 (c) Copyright IBM Corp. 2000,2008 All rights reserved.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Main Menu
1. Select Language
2. Setup Remote IPL (Initial Program Load)
3. Change SCSI Settings
4. Select Console
5. Select Boot Options
…

To terminate a console session, the escape sequence “~.” used.

Some LPAR tool commands support opening a console via the “-c” (console) option:

    • Activating an LPAR with “lpar activate -c“.
    • Shutting down an LPAR with “lpar shutdown -c“.
    • Shutting down the operating system with “lpar osshutdown -c“.
    • Initiating a system dump for an LPAR with “lpar dumprestart -c“.

A presentation on the subject can be found here: Console with the LPAR tool

NTP Configuration on HMC with LPAR tool

NTP-Konfiguration HMC

The current status of NTP on an HMC can then be displayed using “hmc lsntp“:

$ hmc lsntp
NAME   XNTP     XNTPSTATUS    XNTPSERVER
hmc01  disable  -             -
hmc02  enable   synchronized  192.168.189.77,192.168.189.78
hmc03  enable   synchronized  192.168.189.77,192.168.189.78
$

Another NTP server can be added to the NTP configuration of an HMC using the “hmc addntpserver” command:

$ hmc addntpserver hmc01 192.168.189.77
$ hmc addntpserver hmc01 192.168.189.78
$

A check with “hmc lsntp” shows that two NTP servers are now configured, but NTP is still not activated:

$ hmc lsntp hmc01
NAME   XNTP     XNTPSTATUS  XNTPSERVER
hmc01  disable  -           192.168.189.77,192.168.189.78
$

NTP can now be activated with the command “hmc enablentp“:

$ hmc enablentp hmc01
$

The first sync may take a while:

$ hmc lsntp hmc01
NAME   XNTP    XNTPSTATUS      XNTPSERVER
hmc01  enable  unsynchronized  192.168.189.77,192.168.189.78
$

The time on the HMC is not yet synchronized immediately after enabling NTP (XNTPSTATUS: unsynchronized).

A detailed status for each NTP server can be obtained by using the “-a” option (all NTP servers):

$ hmc lsntp –a hmc01
NAME    SERVER         STATE          POLL_FREQ_SECONDS  SECONDS_SINCE_LAST_POLL
hmc01  192.168.189.77  not connected  64                 0
hmc01  192.168.189.78  not connected  64                 0
$

As soon as synchronization with one of the NTP servers is achieved, the overall status is synchronized:

$ hmc lsntp hmc01
NAME   XNTP    XNTPSTATUS    XNTPSERVER
hmc01  enable  synchronized  192.168.189.77,192.168.189.78
$

A more detailed description can be found here: NTP Configuration on the HMC

The LPAR tool can be downloaded for testing here: Download

LPAR tool 1.7.0.1 is now available

Version 1.7.0.1 of the LPAR tool is now available in our download area.

The new version supports the following new features, among others:

    • Installation of IFixes and updates on the HMC (hmc help updhmc)
    • System firmware updates (and more) of managed systems (ms help updatelic)
    • Display FLRT data with online query at IBM (hmc help flrt, ms help flrt, lpar help flrt)
    • Configuration of NTP on HMCs (hmc help ntp)

Versions for Linux, AIX and Macos are available.

All versions include a test license valid until September 30th, 2022.

So download, install and then try it out!

View IOS Version as normal User

On a virtual I/O server, the IOS version can be displayed as user padmin using the ioslevel command:

padmin> ioslevel
3.1.2.10
padmin>

As user root (after using oem_setup_env), the IOS version can be shown as follows:

# /usr/ios/cli/ioscli ioslevel
3.1.2.10
#

However, both commands do not work as a normal, non-privileged user:

$ ioslevel
ksh: ioslevel: not found.
$ /usr/ios/cli/ioscli ioslevel
Access to run command is not valid.

$

The IOS version is simply stored in a text file and can be easily displayed as a normal user with the cat command:

$ cat /usr/ios/cli/ios.level
3.1.2.10
$

Virtual Network Interface Controller (vNIC)

vNIC adapter with 2 vNIC backing devices and vNIC failover.

The big disadvantage of SR-IOV, as described above, is that LPARs with logical SR-IOV ports cannot be moved (LPM). After the introduction of SR-IOV on POWER systems, there were a number of suggestions for workarounds. However, all of these workarounds require, on the one hand, a special configuration and, on the other hand, a number of reconfigurations to be carried out before and after an LPM operation. In everyday practice, however, this unnecessarily complicates LPM operations.

With the introduction of vNICs, client LPARs can use SR-IOV adapters and still support LPM. As with VSCSI and VFC, a pair of adapters is used for this purpose: the so-called vNIC adapter is used in a virtual slot on the client LPAR and an associated vNIC server adapter is used on a virtual I/O server. The logical SR-IOV port is assigned to the virtual I/O server. The vNIC server adapter, also known as the vNIC backing device, serves as a proxy for the logical SR-IOV port. The interaction of the various adapters is shown in figure 7.19.

Communication path of vNIC for control information and data.
Figure 7.19: Communication path of vNIC for control information and data.

In order to achieve good performance, only control information is transmitted from the vNIC adapter of the client to the vNIC server adapter on the virtual I/O server, which is transmitted in turn from the vNIC server adapter, via the associated logical SR-IOV port (ent adapter), to the corresponding logical port (virtual function) of the SR-IOV adapter. The data itself is transferred between the vNIC client adapter and the logical port of the SR-IOV adapter via DMA (Direct Memory Access) with the help of the hypervisor. In particular, there is no copying of the data via the virtual I/O server. The vNIC adapter on the client is a purely virtual adapter, so LPM works without any problems. The client does not own the logical SR-IOV port and does not access it itself via the PCIe bus (switch).

Shared Ethernet Adapter

SEA with multiple trunking adapters and VLANs

Despite SR-IOV and vNIC, Shared Ethernet is still the most widely used virtualization solution, when it comes to virtualizing Ethernet. The POWER Hypervisor implements internal virtual IEEE802.1q compatible network switches, which, in conjunction with so-called shared Ethernet adapters or SEAs for short, take over the connection to external networks. The shared Ethernet adapters are implemented in software as a layer 2 bridge by the virtual I/O server.

As shown in figure 8.2, a shared Ethernet adapter can have several so-called trunking adapters. The SEA shown has the 3 trunking adapters ent8, ent9 and ent10, all 3 of which are connected to the virtual switch with the name ETHMGMT. In the case shown, all trunking adapters support VLAN tagging. In addition to the port VLAN IDs (PVIDs), the 3 trunking adapters also have additional VLANs via VLAN tagging. In addition to the connection to the virtual switch via the trunking adapter, the SEA also has a connection to an external network by the physical network adapter (ent0). Network packets from client LPARs to external systems are forwarded to the SEA via one of the trunking adapters and then to the external network via the associated physical network adapter. Network packets from external systems to client LPARs are forwarded by the SEA via the trunking adapter with the correct VLAN to the virtual switch, which then forwards the packets to the client LPAR.

SEA with multiple trunking adapters and VLANs
Figure 8.2: SEA with multiple trunking adapters and VLANs.

In the simplest case, a SEA consists of just one trunking adapter. A SEA can have up to 16 trunking adapters, whereby each of the trunking adapters can have up to 20 additional VLANs in addition to the port VLAN ID.

Which SEAs already exist on a virtual I/O server can be found out with the help of the command “vios lssea” (list SEAs):

$ vios lssea ms05-vio1
                                       TIMES   TIMES    TIMES    BRIDGE 
NAME   HA_MODE  PRIORITY  STATE       PRIMARY  BACKUP  FLIPFLOP  MODE
ent33  Sharing  1         PRIMARY_SH  1        1       0         Partial
ent34  Sharing  1         PRIMARY_SH  1        1       0         Partial
$

Some basic information is displayed for each SEA, such as the HA mode (see later), the priority of the SEA, as well as information on how often the SEA was already primary or backup.