Broadcom Gigabit Ethernet Adapter Device Driver Installation README File This README file contains the latest information about installing the Linux ethernet device drivers for the Broadcom Gigabit Ethernet Adapter found in your IBM server. CONTENTS ________ 1.0 Known Problems 2.0 Change History 3.0 Installation and Configuration 3.1 Linux 3.1.1 Device Driver Installation 3.1.1.1 Patching PCI Files (Optional) 3.1.2 Network Installation 3.1.3 Module Parameters 3.2 Teaming Functions 3.2.1 BASP Installation 3.2.2 Configuring Teaming 3.2.2.1 BASPCFG 3.2.2.2 Startup scripts 3.2.2.3 Configuration scripts 3.2.2.4 Broadcom NICE patches 3.3 Uninstall - RPM package 4.0 Web Sites and Support Phone Number 5.0 Trademarks and Notices 6.0 Disclaimer 1.0 Known Problems ____________________ - The current version of the driver has only been tested on Redhat 6.2 and 7 Linux distributions for i386 (kernel version 2.2.14 and 2.2.16), and the 2.4.0 test kernel. The driver has only been tested as a loadable module. 2.0 Change History _____________________ Changes made in this diskette, version 1.41: - none, initial release For Broadcom Advanced Server Program (BASP), version 1.19: - none, initial release 3.0 Installation and Configuration ____________________________________ This file describes the Linux driver for Broadcom's BCM5700 10/100/1000 Mbps Ethernet Network Interface Chip as well as the Broadcom Advanced Server Program (BASP). The code is released in a source RPM format. The driver file name is bcm5700-1.4.1-1.src.rpm. BASP for Linux is shipped in mixed forms, where the platform and kernel specific files are in source code, and the core file is in object form. The BASP file name is basplnx-1.1.9-1.src.rpm. 3.1 Linux ----------- 3.1.1 Device Driver Installation ---------------------------------- To install the Linux driver, follow these steps: 1) Install the source RPM package: rpm -i bcm5700-.src.rpm 2) CD to the RPM path and build the binary driver for your kernel: cd /usr/src/{redhat,OpenLinux,turbo, ..} rpm -bb SPECS/bcm5700.spec Note that the RPM path is different for different Linux distributions. 3) Install the newly built package: rpm -i RPMS/i386/bcm5700-.i386.rpm The driver will be installed as /lib/modules//net/bcm5700.o. 4) Load the driver: insmod bcm5700 NOTE: A newly installed Redhat 7's Enterprise kernel needs to be recompiled to regenerate versioned symbols before any newly compiled driver can load. 3.1.1.1 Patching PCI Files (Optional) ------------------------------------- To use Redhat's kudzu hardware detection utility, a number of files containing PCI vendor and device information need to be patched with information on the BCM5700 chip. A patch file (pci.patch) is included for Rehat 7. Apply the patch by doing the following: patch -N -p1 -d /usr < pci.patch Run kudzu: kudzu 3.1.2 Network Installation --------------------------- For network installations through NFS, FTP, or HTTP (using a network boot disk or PXE), a driver diskette that contains the BCM5700 driver is needed. The driver diskette's image for Redhat 7.0 is located in network_install/rh70/dd.img. Boot drivers for other Linux versions can be compiled by modifying the Makefile and the make environment. Further information is available from Redhat's website. To create the driver diskette for Redhat 7.0, use dd if=dd.img of=/dev/fd0H1440. 3.1.3 Module Parameters ------------------------ Optional parameters for the driver can be supplied as command line arguments to the insmod command. These parameters take the form =value[,value,...] where the multiple values for the same parameter are for multiple NICs installed in the system. Note that default values will be used when invalid values are selected. All the parameters are listed below:. line_speed Selects the line speed of the link. This parameter is used together with full_duplex to select the speed and duplexity of the link. The valid values are: 0 Autonegotiate (default) 10 10 Mbps 100 100 Mbps Note that selecting 1000 Mbps manually is invalid. 1000 Mbps can only be achieved through autonegotiation. full_duplex Selects the duplexity of the link. This paramter is used together with line_speed to select the speed and duplexity of the link. Note that this parameter is ignored if line_speed is 0. The valid values are: 0 half duplex 1 full duplex (default) rx_flow_control Enables or disables receiving flow control (pause) frames. This parameter is used together with auto_flow_control. The valid values are: 0 pause receive disabled (default) 1 pause receive enabled if auto_flow_control is set to 0, or pause receive advertised if auto_flow_control is set to 1 tx_flow_control Enables or disables transmitting flow control (pause) frames. This parameter is used together with auto_flow_control. The valid values are: 0 pause transmit disabled (default) 1 pause transmit enabled if auto_flow_control is set to 0, or pause transmit advertised if auto_flow_control is set to 1 auto_flow_control Enables or disables autonegotiation of flow control. This parameter is used together with rx_flow_control and tx_flow_control to determine the advertised flow control capability. The valid values are: 0 flow control autonegotiation disabled (default) 1 flow control autonegotiation enabled with capability specified in rx_flow_control and tx_flow_control (only valid if line_speed is set to 0) mtu Enables jumbo frames up to the specified MTU size. The valid range is from 1500 to 8184. Default is 1500. 3.2 Teaming Functions ----------------------- Broadcom Advanced Server Program (BASP) is a kernel module designed for Linux 2.2 kernel that provides load-balancing, fault-tolerance, and VLAN features. These features are provided by creating teams that consist of multiple NIC interfaces. A team can consist of 1 to 8 NIC interfaces and each interface can be designated primary or standby. All primary NICs in a team will participate in Load-balancing operations by sending and receiving a portion of the total traffic. Standby interfaces will take over in the event that all primary interfaces have lost their links. VLANs can be added to a team to allow multiple VLANs with different VLAN IDs and a virtual device is created for each VLAN added. BASP supports Smart Load-balance, SLB and Generic trunking. In SLB mode, all the NIC drivers must support Broadcom NIC Extension (NICE). In this distribution, several NIC drivers patched with NICE are included. SLB mode works with all ethernet switches without configuring the switch ports to any special trunking mode. Only IP traffic will be load-balanced in both inbound and outbound directions. The Generic trunking mode does not require NICE and can work with any NIC, however, it requires the Ethernet switch to support and be properly configured. This mode is protocol-independent and all traffic should be load-balanced and fault- tolerant. 3.2.1 BASP Installation ------------------------- To install BASP, follow these steps: 1) install RPM source package, run % rpm -i basplnx-{version}.src.rpm 2) CD to the RPM path and build the binary driver for the kernel % cd /usr/src/{redhat, OpenLinux, turbo, ...} % rpm -bb SPECS/basplnx.spec Note that the RPM path is different for different Linux distributions. 3) Install the newly built package % rpm -i RPMS/i386/basplnx-{version}.i386.rpm The driver and other required files will be installed. 4) To load the driver, % insmod basp 5) Follow the Configuration section to set up the teams. Files ----- Makefile makefile baspcfg precompiled configuration utility bcmtype.h commonly use type header file blf.c BASP module entry points blf.h ioctl interface blfcore.h core interface blfcore.o pre-compiled core object blfopt.h automatic generated header file by Make blfver.h version header file nicext.h NICE header file pal.c platform abstraction implementation pal.h header for paltform abstraction release.txt this file nice-2.2.16 contains NICE enabled driver sources code scripts contains sample scripts scripts/basp init script, goes to /etc/rc.d/init.d scripts/baspteam start/stop script, goes to /etc/basp scripts/baspif start/stop network i/f, goes to /etc/basp scripts/team-sample sample script of SLB team with 3 NICs scripts/team-gec sample script of GEC team with 3 NICs 3.2.2 Configuring Teaming --------------------------- BASP distribution includes an utility program and several scripts for team configuration. The configuration process involves 1) copy a configuration script from "/etc/basp/samples" directory to "/etc/basp" directory. Note the configuration script must be prefixed with "team-". 2) modify the configuration script to (a) change the team type (b) add/delete physical network interfaces (c) add/delete virtual network interfaces The syntax of the configuration script can be found below. 3) manually start the team for the first time, % /etc/rc.d/init.d/basp start 4) run "netcfg" to configure the IP addresses of the virtual network interfaces. Note that starting the team is only required for the first time, and the teams will be started automatically in the subsequent reboot. 3.2.2.1 BASPCFG ---------------- baspcfg is a command line tool to configure the BASP teams, add/remove NICs and add/remove virtual device. Following is the usage of this tool: baspcfg commands: addteam -- create a team delteam -- delete a team addva [macaddr] -- add a virtual adapter to a team delva -- del a virtual adapter from a team bind -- bind a physical adapter to a team unbind -- unbind a physical adapter from a team show [tid] -- display team configurations where tid -- An unique ID for each team, starting from 0 type -- Team type: 0=SLB, 1=FEC/GEC tname -- ASCII string of the team vlan_id -- VLAN ID: from 1 to 4094, 0=untagged or no VLAN vname -- ASCII string of the virtual device macaddr -- MAC address (optional), e.g. 00:10:18:00:11:44 role -- Role of the physical device: 0=primary, 1=hot-standby device -- ASCII string of the physical device, e.g. eth0 3.2.2.2 Startup scripts ------------------------ - basp This script is intended to be installed in /etc/rc.d/init.d directory. After copy the script, run "chkconfig --add basp". This script will be executed at runlevel 2, 3, 4 and 5. When basp is run, it will search the /etc/basp directory to list all the file with "team-" prefix, and then it will invoke "baspteam" script to add or delete the teams. It is normal that each "team-*" file in /etc/basp representing 1 team. - baspteam This script is called by basp to add or delete a team. To install, create "/etc/basp" directory and copy this script over. To manually add a team: % baspteam team-sample add To delete a team: % baspteam team-sample del Note that "team-sample" is the config script. - team-sample This script contains a SLB team configuration with 3 NICs: eth0, eth1 and eth2. The team name is "TeamSample". All 3 NICs are primary. One virtual interface is also created for this team and the name of the virtual interface is "sw0". The "sw0" is the device where ifconfig should be run against to set up the IP address. No VLAN is enabled in the script. This script and team-gec are intended to be customized. Refer to the configuration scripts section for details. This script should be copied to /etc/basp directory and retained "team-" prefix. - team-gec This configuration script creates a GEC team with 3 network interfaces, eth0, eth1 and eth2. The team name is "TeamGEC". All 3 NICs are primary. One virtual interface is added to the team with the name "sw0" and the VLAN is not enabled. This script and team-sample are intended to be customized. Refer to the configuration scripts section for details. This script should be copied to /etc/basp directory and retained "team-" prefix. 3.2.2.3 Configuration scripts ------------------------------ Both team-sample and team-gec are configuration scripts that follow the same syntax, as following TEAM_ID: this number uniquely identifies a team. TEAM_TYPE: 0 = SLB, 1 = Generic Trunking/GEC/FEC TEAM_NAME: ascii name of the team TEAM_PAx_NAME: ascii name of the physical interface x, where x can be 0 to 7. TEAM_PAx_ROLE: role of the physical interface x 0 = Primary, 1 = Hot-standby. This field must be 0 for Generic Trunking/GEC/FEC team. TEAM_VAx_NAME: ascii name of the virtual interface x, where x can be 0 to 63 TEAM_VAx_VLAN: 802.1Q VLAN ID of the virtual interface x. For untagged virtual interface, i.e. without VLAN enable, set it to 0. The valid VLAN ID can be 0 to 4094. 3.2.2.4 Broadcom NICE patches ------------------------------ Also included in this distributions are network device drivers patched with Broadcom NICE support. These drivers are originally taken from Linux 2.2.16 kernel distribution. To installed patched drivers: 1) Copy the Broadcom NICE header file, "nicext.h", to Linux kernel include directory, % cp /usr/src/nice-2.2.16/nicext.h /usr/src/linux/include/linux 2) Rename the original network device driver under Linux kernel source tree, "/usr/src/linux/drivers/net". 3) Copy the patched drivers to Linux kernel network driver source directory, i.e. "/usr/src/linux/drivers/net". 4) Follow the kernel rebuild instructions to configure kernel support of these drivers, i.e. % cd /usr/src/linux % make config 5) If the patched drivers is configured into kernel, goto step (7). If the patched drivers is configured as module, goto step (6). 6) In the case of supporting only module version of these drivers, it is possible to simply run following to compile patched driver and to install them into the proper module directory by % make modules % make modules_install There is no need to compile the complete kernel and goto step (8). 7) rebuild the kernel to compile these patched drivers % make clean % make dep % make 8) Either reboot the system or unload/load the patched modules. Run configuration scripts to test the patch. 3.3 Uninstall - RPM package ---------------------------- To uninstall an RPM package, % rpm -e basplnx-{version}.i386.rpm and to reboot the system, % reboot 4.0 WEB Sites and Support Phone Number ________________________________________ IBM Support Web Site: http://www.pc.ibm.com/support IBM Marketing Netfinity Web Site: http://www.pc.ibm.com/netfinity If you have any questions about this update, or problems applying the update go to the following Help Center World Telephone Numbers URL: http://www.pc.ibm.com/qtechinfo/YAST-3P2QLY.html. 5.0 Trademarks and Notices ____________________________ The following terms are trademarks of the IBM Corporation in the United States or other countries or both: IBM Netfinity Broadcom, Smart Load Balancing, and BASP are trademarks or registered trademarks of Broadcom Corporation. RedHat is a registered trademark of RedHat Corporation. Other company, product, and service names may be trademarks or service marks of others. 6.0 Disclaimer _______________ THIS DOCUMENT IS PROVIDED "AS IS" WITHOUT WARRANTY OF ANY KIND. IBM DISCLAIMS ALL WARRANTIES, WHETHER EXPRESS OR IMPLIED, INCLUDING WITHOUT LIMITATION, THE IMPLIED WARRANTIES OF FITNESS FOR A PARTICULAR PURPOSE AND MERCHANTABILITY WITH RESPECT TO THE INFORMATION IN THIS DOCUMENT. BY FURNISHING THIS DOCUMENT, IBM GRANTS NO LICENSES TO ANY PATENTS OR COPYRIGHTS. Note to U.S. Government Users -- Documentation related to restricted rights -- Use, duplication or disclosure is subject to restrictions set forth in GSA ADP Schedule Contract with IBM Corp.