You are currently on IBM Systems Media’s archival website. Click here to view our new website.

Linux on Power > Tips and Techniques > Systems Management

VIOS 101: Network


Shared Ethernet Adapter

Shared Ethernet adapters (SEAs) are designed to bridge real adapters to Virtual Ethernet adapters. Rather than having a dedicated Ethernet adapter in every LPAR, SEA involves having the actual adapter (or a group of adapters aggregated together) assigned to the VIO server. The LPARs talk using Virtual Ethernet to the VIO server and any traffic that needs to go outside the box is sent out via the Shared Ethernet adapter in the VIO server. For redundancy, normally two VIO servers are acting as SEAs and it’s common to set one up as primary with the other running as the SEA failover VIO server. SEA is used to reduce the number of Ethernet cards on a system and to allow for failover and redundancy.

SEAs can be set up in failover mode or load sharing mode. With failover, the Ethernet adapters on the backup SEA sit idle until a planned or unplanned failure on the primary VIO’s SEA. With load sharing, you can set some VLANs to use the SEA on VIOS1, and other VLANs to use VIOS2, and still have both SEAs ready to take over should the other VIOS fail. Another flavor is SEA failover with NIB.

SEA Failover. SEA failover is the technology most people are familiar with and is designed to work with two VIO servers, which requires an HMC. An aggregate is created on each VIO server using the mkvdev command. That aggregate is then turned into an SEA where one VIO is set up with a trunk priority of 1 and the other with a trunk priority of 2. It’s important to get this right to avoid spanning tree loops. Prior to PowerVM 2.2.3, a VLAN ID had to be reserved for a control channel that is used by the two VIO servers to send commands as well as keep-alive messages. PowerVM 2.2.3 introduced simplified SEA failover where a control channel doesn’t need to be defined. If it’s not defined, then the SEA will default to using VLAN 4095. It’s important that this VLAN is not defined anywhere else.

With SEA failover, one VIO is primary and has the full bandwidth of all the adapter ports in the SEA on that primary VIO. If the primary VIO fails or loses its connectivity, then the secondary VIO becomes primary and network connectivity continues. If each VIO is on a separate switch, then you also get redundancy across switches. Assuming you have two adapters in the aggregate on each VIO (total of four) then you can use the bandwidth of two adapters.

Depending on the actual setup, a failover for SEA failover can take up to 30 seconds. This depends on network switch and spanning tree setup.

SEA with NIB. SEA with NIB is an SEA that’s defined with one of the adapters set up as a backup adapter. That allows the NIB adapter to be on a separate switch to the primary aggregate. This means that if the primary aggregate or switch fails, then the NIB adapter on the same VIO gets activated. Additionally, there is also failover to the secondary VIO. As already mentioned, NIB uses ping to monitor the active aggregate so it creates a lot of ping traffic. It also reduces available bandwidth. Comparing it to the previous example: Assuming you have two adapters in the aggregate on each VIO (total of four) then you can use the bandwidth of only one adapter, as the second adapter on each VIO is reserved for NIB. This is a very expensive solution when using 10Gb/s adapters.

SEA Failover with load balancing/sharing. Load sharing was introduced with PowerVM v2.2.1. In this case, we define two SEAs and the SEAs negotiate the set of VLAN IDs that they are responsible for bridging. Once negotiation is complete, each SEA bridges the assigned trunk adapters and the associated VLANs. Thus, both SEAs are in use, allowing you to take advantage of all of the adapters in both VIO servers. If a failure occurs, the remaining active SEA bridges all trunk adapters and the associated VLANs. To define this, each SEA needs to have the ha_mode=sharing attribute rather than the ha_mode=auto:

One major difference is the available bandwidth. Comparing the three options where we have two adapters on each VIO (four adapters total) we see:

Type					Bandwidth
SEA Failover				2 adapters
SEA Failover with NIB			1 adapter
SEA Failover with load sharing		4 adapters

SEA Failover Versus NIB

SEA failover has some advantages over NIB. In particular, it’s simpler as SEA failover is implemented on the VIO server and the client configuration is simple. NIB is more complex, especially at the client LPAR where a second Virtual Ethernet adapter on a different VLAN is required for the NIB adapter.

Additionally, SEA failover has the added support of IEEE 802.1Q VLAN tagging and it simplifies NIM installation because only a single virtual Ethernet device is required on the client partition. Finally, SEA failover is significantly less “pingy” than NIB. With SEA failover, only the SEAs send out periodic ping requests for checking the network availability. With NIB, every client partition will send out ping requests, resulting in more network traffic.

Summary

As you can see, you have multiple ways to virtualize the network for LPARs on your servers. Networking can be as complex or as simple as you want to make it—it’s highly recommended that the design be as simple as possible and that the network team be part of the design so you can best determine where to take advantage of larger MTU sizes to improve performance. Additionally, techniques such as flow control, large receive and large send can assist with performance. More information on these is provided in the network performance session at the movies site in the references (see box).

Jaqui Lynch is an independent consultant, focusing on enterprise architecture, performance and delivery on Power Systems with AIX and Linux.





Advertisement

Advertisement

2019 Solutions Edition

A Comprehensive Online Buyer's Guide to Solutions, Services and Education.

LINUX ON POWER > TIPS AND TECHNIQUES > SYSTEMS MANAGEMENT

Planned Active Use of Elastic CoD Into Power Systems Solutions

LINUX ON POWER > TIPS AND TECHNIQUES > SYSTEMS MANAGEMENT

Introducing IBM FlashSystem 840

LINUX ON POWER > ADMINISTRATOR > SYSTEMS MANAGEMENT

HMC Architecture Options

IBM Systems Magazine Subscribe Box Read Now Link Subscribe Now Link iPad App Google Play Store