View as PDF

Virtualization with PowerVM > Practical Introduction > Virtual I/O Server > Shared Ethernet Adapter

8.5.2. SEA with VLAN Tagging

If multiple VLANs are used, which should be the standard in most environments, there are several possibilities how a shared Ethernet adapter can look exactly like. A single trunking adapter supports up to 20 VLANs in addition to the port VLAN ID. So additional trunking adapters only have to be used if more than 20 VLANs are to be used. A SEA can have up to 16 trunking adapters. How many trunking adapters are ultimately used, depends on the administrator and the number of VLANs to be supported. For example, if 10 VLANs are to be used, between one and ten trunking adapters can be used. In practice, a smaller number of VLANs are typically configured on a individual trunking adapter (typically 3 to 6 VLANs), but this depends heavily on the environment and is not considered further here.

Each shared Ethernet adapter supports a maximum of one untagged VLAN. Typically, however, all required VLANs are used as tagged VLANs and unused VLANs are used for the necessary port VLAN IDs of the trunking adapters only. In many environments, single-digit and low-two-digit VLAN IDs are reserved for this purpose, and are not used for real VLANs.

In the following, another shared Ethernet adapter is created, this time with VLAN tagging and the VLANs 100, 110, 200, 205 and 210. We decided to create two trunking adapters, one with VLANs 100 and 110 and a second with the VLANs 200, 205 and 210. First, however, another virtual switch is created for the new shared Ethernet adapter:

$ ms addvswitch ms05 ETHTEST2
$

We use the two free virtual slots 61 and 62 for the two trunking adapters:

$ lpar addeth -t 1 -i -s ETHTEST2 ms05-vio1 61 1 100,110 
$ lpar addeth -t 1 -i -s ETHTEST2 ms05-vio1 62 2 200,205,210 
$

To check this, the candidates for another SEA are listed again:

$ vios lssea -c ms05-vio1
NAME   STATUS     PHYSLOC                      PARENT  DESCRIPTION
ent3   Available  U78AA.001.VYRGU0Q-P1-C7-T4   pci1    4-Port Gigabit Ethernet PCI-Express Adapter (e414571614102004)
ent2   Available  U78AA.001.VYRGU0Q-P1-C7-T3   pci1    4-Port Gigabit Ethernet PCI-Express Adapter (e414571614102004)
ent39  Available  U8205.E6C.05E4E5Q-V1-C61-T1  vio0    Virtual I/O Ethernet Adapter (l-lan)
ent41  Available  U8205.E6C.05E4E5Q-V1-C62-T1  vio0    Virtual I/O Ethernet Adapter (l-lan)
$

We use ent2 as the physical adapter and the two adapters we just created ent39 (slot 61) and ent41 (slot 62) as the trunking adapter:

$ vios mksea ms05-vio1 ent2 ent39 ent41
SEA ent42 created
$

The physical location code (or a unique suffix) can be specified instead of the device names:

$ vios mksea ms05-vio1 P1-C7-T3 C61-T1 C62-T1
SEA ent42 created
$

As before, all adapters of a SEA can be displayed using “vios lssea“, we inspect the newly created SEA ent42:

$ vios lssea -a ms05-vio1 ent42
SEA    LNAGG  NAME   TYPE     STATUS  SPEED                  VSWITCH   MODE  PHYSLOC
ent42  -      ent2   real     Up      1000 Mbps Full Duplex  -         -     U78AA.001.VYRGU0Q-P1-C7-T3
ent42  -      ent41  virtual  -       -                      ETHTEST2  VEB   U8205.E6C.05E4E5Q-V1-C62-T1
ent42  -      ent39  virtual  -       -                      ETHTEST2  VEB   U8205.E6C.05E4E5Q-V1-C61-T1
$

The distribution of the supported VLANs of the SEA can be listed with the option ‘-V‘ (VLANs):

$ vios lssea -V ms05-vio1 ent42
SEA    LNAGG  NAME   TYPE     VSWITCH   MODE  ACTIVE  PRIORITY  PVID  VLAN_TAG_IDS
ent42  -      ent2   real     -         -     -       -         -     -
ent42  -      ent41  virtual  ETHTEST2  VEB   True    1         2     200,205,210
ent42  -      ent39  virtual  ETHTEST2  VEB   True    1         1     100,110
$

The shared Ethernet adapter just created is shown in figure 8.4.

SEA with 2 trunking adapters and 5 VLANs.
Figure 8.4: SEA with 2 trunking adapters and 5 VLANs.

In the following, the path of an Ethernet frame from an LPAR with PVID 110 to an external host is shown in individual steps:

1. The LPAR sends an untagged Ethernet frame via the virtual Ethernet adapter ent0 (figure 8.5a).

LPAR with PVID 110 sends Ethernet frames via the virtual Ethernet adapter ent0.
Figure 8.5a: LPAR with PVID 110 sends Ethernet frames via the virtual Ethernet adapter ent0.

2. The Ethernet frame is forwarded to the connected virtual switch ETHTEST2 and a VLAN header with the PIVD of the virtual Ethernet adapter is added to the frame (figure 8.5b).

The Ethernet frame is passed on to the virtual switch ETHTEST2 and is tagged there with the VLAN ID 110 (PVID).
Figure 8.5b: The Ethernet frame is passed on to the virtual switch ETHTEST2 and is tagged there with the VLAN ID 110 (PVID).

3. Since the destination is not connected directly to the virtual Ethernet switch ETHTEST2, the virtual Ethernet switch ETHTEST2 uses the trunking adapter for the VLAN 110 to forward the frame. The trunking adapter for the VLAN 110 is the adapter ent39 (figure 8.5c), which belongs to the shared Ethernet adapter ent42 of the virtual I/O server shown.

The Ethernet frame is forwarded from the virtual switch ETHTEST2 via the trunking adapter ent39 for the VLAN 110 to the SEA ent42.
Figure 8.5c: The Ethernet frame is forwarded from the virtual switch ETHTEST2 via the trunking adapter ent39 for the VLAN 110 to the SEA ent42.

4. Figure 8.5d finally shows how the shared Ethernet adapter ent42 forwards the Ethernet frame using its physical adapter ent2 into the external network. The Ethernet frame is still tagged with the VLAN ID 110. The switches in the external network then forward the Ethernet frame to the target system. In the figure it is assumed that the target system itself supports VLAN tagging and receives the frame with a VLAN header, but it is also possible that the target system uses an untagged port with PVID 110 and thus receives the frame without a VLAN header.

The SEA ent42 sends the Ethernet frame to the external network via the physical adapter ent2. There, it is forwarded to the target system.
Figure 8.5d: The SEA ent42 sends the Ethernet frame to the external network via the physical adapter ent2. There, it is forwarded to the target system.

In the following, the steps involved in transporting an Ethernet frame from an external host to an LPAR are shown:

1. An external host sends an Ethernet frame with VLAN 110 to an LPAR (figure 8.6a). The Ethernet frame is either tagged with a VLAN header by the external host itself, or the VLAN header was added to the Ethernet frame by the connected network switch.

An external host sends an Ethernet frame into the connected network.
Figure 8.6a: An external host sends an Ethernet frame into the connected network.

2. The Ethernet frame is forwarded in the direction of the target LPAR to the physical adapter ent2 of the virtual I/O server, which is part of the shared Ethernet adapter ent42 on the virtual I/O server (figure 8.6b).

The Ethernet frame is forwarded from the external switches to the physical adapter ent2 of the managed system. This is part of the SEA ent42 on a virtual I/O server.
Figure 8.6b: The Ethernet frame is forwarded from the external switches to the physical adapter ent2 of the managed system. Which is part of the SEA ent42 on a virtual I/O server.

3. The shared Ethernet adapter ent42 selects the trunking adapter ent39 for forwarding the Ethernet frame to the virtual switch ETHTEST2, since the adapter ent39 supports VLAN 110 (figure 8.6c).

Since the Ethernet frame has the VLAN ID 110, the SEA ent42 forwards the frame to the virtual switch ETHTEST2 via the trunking adapter ent39.
Figure 8.6c: Since the Ethernet frame has the VLAN ID 110, the SEA ent42 forwards the frame to the virtual switch ETHTEST2 via the trunking adapter ent39.

4. The virtual Ethernet switch ETHTEST2 forwards the Ethernet frame via the destination port to the adapter ent0 of the destination LPAR, figure 8.6d. Since the virtual adapter has the VLAN 110 as the port VLAN ID, the VLAN header is removed, when the frame is transported. The adapter ent0 receives the Ethernet frame as an untagged frame.

The virtual Ethernet switch ETHTEST2 forwards the Ethernet frame to the adapter of the target LPAR, the VLAN header is removed in the process.
Figure 8.6d: The virtual Ethernet switch ETHTEST2 forwards the Ethernet frame to the adapter of the target LPAR, the VLAN header is removed in the process.

A special situation arises, when an LPAR uses one of the PVIDs of the trunking adapter as the VLAN ID. In the SEA created above, the two trunking adapters have PVIDs 1 and 2. In the following, we will look at 2 LPARs that use VLAN IDs 1 and 2 respectively. First we look again at the path from the LPAR to the external host:

1. The LPARs each send an untagged Ethernet frame via the virtual Ethernet adapter ent0 (figure 8.7a).

LPAR1 with PVID 1 and LPAR2 with PVID 2 each send an Ethernet frame via the virtual Ethernet adapter ent0.
Figure 8.7a: LPAR1 with PVID 1 and LPAR2 mit PVID 2 each send an Ethernet frame via the virtual Ethernet adapter ent0.

2. The two Ethernet frames are forwarded to the connected virtual switch ETHTEST2 and a VLAN header is added there for both frames (Figure 8.7b). The frame from LPAR1 gets the VLAN ID 1 and the frame from LPAR2 gets the VLAN ID 2 as VLAN header.

The Ethernet frames are passed on to the virtual switch ETHTEST2 and are tagged there, with VLAN ID 1 respectively 2.
Figure 8.7b: The Ethernet frames are passed on to the virtual switch ETHTEST2 and are tagged there, with VLAN ID 1 respectively 2.

3. The frame from LPAR1 with VLAN ID 1 is forwarded from the virtual switch to the SEA ent42 via the associated trunking adapter ent39. Since VLAN 1 is the PVID of the trunking adapter, the VLAN header is removed (figure 8.7c). The frame from LPAR2 with VLAN ID 2 is also forwarded to the SEA ent42. However, the associated trunking adapter is the adapter ent41 here. The VLAN header is also removed, since VLAN 2 is the PVID of the trunking adapter ent41. Both frames are now untagged! It is no longer possible to identify which VLAN the two Ethernet frames originally belonged to!

The two Ethernet frames are forwarded from the virtual switch ETHTEST2 via the trunking adapters ent39 respectively ent41 to the SEA ent42. The VLAN headers are removed.
Figure 8.7c: The two Ethernet frames are forwarded from the virtual switch ETHTEST2 via the trunking adapters ent39 respectively ent41 to the SEA ent42. The VLAN headers are removed.

4. Both untagged Ethernet frames are forwarded from the shared Ethernet adapter ent42 to the external network via the physical adapter ent2 (figure 8.7d).

The SEA ent42 sends the Ethernet frame to the external network via the physical adapter ent2. There, it is forwarded to the target system.
Figure 8.7d: The SEA ent42 forwards both untagged Ethernet frames to the external network via the physical adapter ent2.

5. Whether the target systems can actually be reached, depends on whether they can be reached from ent2 with untagged frames. A port VLAN ID should be configured for the physical adapter ent2 on the associated switch port. That means untagged frames that are sent from ent2 to the external network are assigned to this port VLAN ID.

The path of a frame from the external network to the client LPARs is interesting, in the event that the frame does not have a VLAN header:

1. An Ethernet frame without a VLAN header is forwarded from the external network to the physical adapter ent2 of the managed system (figure 8.8a). The physical adapter ent2 belongs to the shared Ethernet adapter ent42 on the virtual I/O server shown.

The external network sends an Ethernet frame without a VLAN header to the physical adapter ent2, which belongs to the SEA ent42.
Figure 8.8a: The external network sends an Ethernet frame without a VLAN header to the physical adapter ent2, which belongs to the SEA ent42.

2. The shared Ethernet adapter ent42 must forward the Ethernet frame. However, a problem arises here: it is not clear which of the two trunking adapters has to be used. The Ethernet frame does not belong to any VLAN because it is untagged. Both trunking adapters ent39 and ent41 can in principle forward untagged frames. If the frame is forwarded via ent39, the frame is tagged with the PVID 1 of ent39. If the frame is forwarded via ent41, the frame is tagged with PVID 2 by ent41. Depending on which trunking adapter is used, the frame would be assigned to a different VLAN! With a SEA with the maximum possible trunking adapters, there would be 16 different possibilities.

From the point of view of the SEA ent42, there are 2 different possibilities for forwarding the Ethernet frame: via ent39 (PVID 1) or ent41 (PVID 2).
Figure 8.8b: From the point of view of the SEA ent42, there are 2 different possibilities for forwarding the Ethernet frame: via ent39 (PVID 1) or ent41 (PVID 2).

3. The trunking adapter used to forward untagged Ethernet frames from a shared Ethernet adapter is defined in the configuration of the SEA. The corresponding attribute is called pvid_adapter. It can optionally be specified, when creating a SEA with “vios mksea“. The first specified trunking adapter is used by default. As figure 8.8c shows, the trunking adapter ent39 is stored in the pvid_adapter attribute of the shared Ethernet adapter ent42. The untagged frame is therefore forwarded to the virtual switch ETHTEST2 via ent39, with a VLAN header with PVID 1 from ent39 being added. The value of pvid_adapter can be easily displayed using the “vios lsattr” command:

$ vios lsattr ms05-vio1 ent42 pvid_adapter
value
ent39
$
The pvid_adapter attribute of the SEA ent42 determines to which trunking adapter untagged frames are forwarded, here to ent39 with PVID 1. The frame is tagged with the PVID 1 of ent39.
Figure 8.8c: The pvid_adapter attribute of the SEA ent42 determines to which trunking adapter untagged frames are forwarded, here to ent39 with PVID 1. The frame is tagged with the PVID 1 of ent39.

4. The virtual switch ETHTEST2 forwards the Ethernet frame to LPAR1 because the frame belongs to VLAN 1 and the virtual adapter ent0 of LPAR1 has configured this VLAN ID as the PVID. (Of course, the destination MAC address must also match the MAC address of ent0 of LPAR1, but we assume that.) The VLAN header is removed during the forwarding.

The Ethernet frame is delivered to the destination, LPAR1. Since VLAN ID 1 is configured as the port VLAN ID of the virtual Ethernet adapter ent0, the VLAN header is removed.
Figure 8.8d: The Ethernet frame is delivered to the destination, LPAR1. Since VLAN ID 1 is configured as the port VLAN ID of the virtual Ethernet adapter ent0, the VLAN header is removed.

Note: It is not possible to reach LPAR2 (VLAN 2) from an external system. Every Ethernet frame coming from the outside without a VLAN header is always forwarded by the shared Ethernet adapter ent42 to the trunking adapter ent39, as it is configured as the trunking adapter to be used for untagged frames by the attribute pvid_adapter. This means that external, untagged, frames are always assigned to VLAN 1, since this is the PVID of the default trunking adapter.

Best practice is to use unused VLAN IDs for the PVIDs of the trunking adapters. All VLANs used by client LPARs should be configured as additional VLAN IDs on the trunking adapters.