Dual HMC Cabling on the IBM eServer 9119-595 and 9119-590 Servers
Published 02 March 2005, updated 21 February 2006
Authors: Scott Vetter
Cabling an HMC to an IBM eServer p5 and i5 595 or p5 590 requires additional planning, as the HMCs are required.
For the 9119-590 and 9119-595 servers, an HMC must be directly attached. In general, we recommend private service networks for all systems because of the simplified setup and greater security; however, a private network is required between HMC and the servers for the following reasons:
- The Bulk Power Controllers (BPCs) are dependent upon the HMC to provide them with DHCP addresses. (There is no way to set a static address.)
- The HMC must distribute information to the Service Processors (SPs) and BPCs about the topology of the network.
- The SPs acquire their TCP/IP address information from the HMC, as they are connected to the HMC through an internal hub in the BPC, and then through the network connection from the BPC to the HMC.
The use of two HMCs requires the use of different DHCP address ranges, so that the two Ethernet interfaces in each service processor and BPC are configured on different subnets.
If there are multiple systems (additional Managed Server Frames) then an external hub or switch is required for each of the private networks.
The following figure shows the connections between a primary and a secondary HMC and the redundant BPCs in a system frame. If additional powered frames are attached, hubs are used to connect the network. Notice the need for a network connection between each HMC and the corresponding BPC in a given configuration.
Problems obtaining BPC or SP IP addresses in a private switched network
After introducing an Ethernet switch to a working setup or on initial install, some SPs or BPCs do not receive an auto-assigned IP address from the DHCP server (HMC). A direct connection works.
Some Ethernet switches that are available to customers support spanning tree protocol on each port by default. This protocol prevents a device attached to a port from creating infinite loops in the network topology.
Enabling this protocol on the switched ports used by a SP or a BPC may cause some unexpected DHCP behavior. Some, but not all, SPs or BPCs will get an IP address.
The problem is caused when the BPC or SP boots up and the switch enables the port but will not forward any of the packets until the switch has checked the spanning tree for loops. This check may take between 8 seconds to 2 minutes depending on the number of switches clustered together. During this period when the switch is checking the spanning tree for the port used by the HMC, the HMC may send out all its DHCP packets and retries. These packets are discarded by the switch, and the SP or the BPC will never receive an IP address from the DHCP server.
The switch provides different settings to prevent this problem from occurring. If the spanning tree is enabled and the customer requires it to remain enabled, portfast must also be enabled. This setting allows the switch to forward all packets from the port onto the LAN immediately and not check the spanning tree first.
If the spanning tree is enabled and can be disabled for the ports used for the BPC or the SP, the problem is eliminated with this change.
This is a limitation between various network hardware and software devices. Different manufacturers may use different terminology regarding the settings. The switch must be modified using portfast or other commands to accommodate connectivity delays. After validating the exact ports on the switch the SPs or BPCs are using, a network administrator must log in to the switch to make these changes directly to those specific ports.
This material has not been submitted to any formal IBM test and is published AS IS. It has not been the subject of rigorous review. IBM assumes no responsibility for its accuracy or completeness. The use of this information or the implementation of any of these techniques is a client responsibility and depends upon the client's ability to evaluate and integrate them into the client's operational environment.
Follow IBM Redbooks
Follow IBM Redbooks