Does anyone have some tips for setting up Broadcom's 802.3AD support or just getting their driver compiled / installed?
First off, I noticed redhat claims the broadcom bcm5708 driver has been added to the rhel4 kernel. But I don't see /proc/net/nicinfo which broadcom claims the driver creates when it's successfully initialized.
So, I'm trying to build the driver based on the source files downloaded from Broadcom:
I downloaded their source rpm, installed it and now trying to build a binary rpm
(pwd is /usr/src/redhat)
# rpmbuild -bb SPECS/bnx2.spec
it starts off OK, and then bombs
make -C SUBDIRS=/usr/src/redhat/BUILD/bnx2-1.4.43b modules make: *** SUBDIRS=/usr/src/redhat/BUILD/bnx2-1.4.43b: No such file or directory. Stop. make: *** [default] Error 2
but the directory it's complaining about does indeed exist
Here's the output from dmesg regarding my card
Broadcom NetXtreme II Gigabit Ethernet Driver bnx2 v1.4.38 (February 10, 2006) ACPI: PCI interrupt 0000:09:00.0[A] -> GSI 16 (level, low) -> IRQ 169 divert: allocating divert_blk for eth0 eth0: Broadcom NetXtreme II BCM5708 1000Base-T (B2) PCI-X 64-bit 133MHz found at mem f4000000, IRQ 169, node addr 00188b4ea2b3 ACPI: PCI interrupt 0000:05:00.0[A] -> GSI 16 (level, low) -> IRQ 169 divert: allocating divert_blk for eth1 eth1: Broadcom NetXtreme II BCM5708 1000Base-T (B2) PCI-X 64-bit 133MHz found at mem f8000000, IRQ 169, node addr 00188b4ea2b1
Any input is welcomed!
-Gordon
Gordon McLellan wrote:
Does anyone have some tips for setting up Broadcom's 802.3AD support or just getting their driver compiled / installed?
First off, I noticed redhat claims the broadcom bcm5708 driver has been added to the rhel4 kernel. But I don't see /proc/net/nicinfo which broadcom claims the driver creates when it's successfully initialized.
The bcm5708 driver is there - as your dmesg output states.
May be '/proc/net/nicinfo' is something the native driver from Broadcom creates/uses- and RedHat chose not to port that part?
I know nothing about Broadcom's 802.3AD support - but why not just use the 802.3AD bonding driver that the kernel provides? - see:
http://wiki.centos.org/TipsAndTricks/BondingInterfaces
James Pearson
Doh! That was easy ... I had just been reading "old" documentation, I guess from the time before the bonding driver supported 802.3ad natively?
I've setup the files as per the rhel4 example, and I'm receiving an error about speed and duplex. Although both nics are connected, and the machine answers to the bond0 ip, there is no link aggregation indicated by the switch. The NICs are connected to an HP Procurve 1800. I have LACP turned on for the ports the nics are connected to. The switch indicates the protocol is enabled for those ports, but the ports are not listed as an aggregation group.
from /var/log/messages
Feb 2 13:23:12 vmhost ifup: Enslaving eth0 to bond0 Feb 2 13:23:13 vmhost kernel: bnx2: eth0: using MSI Feb 2 13:23:13 vmhost kernel: bonding: Warning: failed to get speed/duplex from eth0, speed forced to 100Mbps, duplex forced to Full. Feb 2 13:23:13 vmhost kernel: bonding: bond0: enslaving eth0 as an active interface with an up link. Feb 2 13:23:13 vmhost ifup: Cannot set new settings: Invalid argument Feb 2 13:23:13 vmhost ifup: not setting speed Feb 2 13:23:13 vmhost ifup: not setting duplex Feb 2 13:23:13 vmhost ifup: not setting autoneg Feb 2 13:23:13 vmhost ifup: Enslaving eth1 to bond0 Feb 2 13:23:13 vmhost kernel: bnx2: eth1: using MSI Feb 2 13:23:13 vmhost kernel: bonding: Warning: failed to get speed/duplex from eth1, speed forced to 100Mbps, duplex forced to Full. Feb 2 13:23:13 vmhost kernel: bonding: bond0: enslaving eth1 as an active interface with an up link. Feb 2 13:23:13 vmhost ifup: Cannot set new settings: Invalid argument Feb 2 13:23:13 vmhost ifup: not setting speed Feb 2 13:23:13 vmhost ifup: not setting duplex Feb 2 13:23:13 vmhost ifup: not setting autoneg Feb 2 13:23:13 vmhost network: Bringing up interface bond0: succeeded Feb 2 13:23:15 vmhost kernel: bnx2: eth0 NIC Link is Up, 1000 Mbps full duplex Feb 2 13:23:15 vmhost kernel: bnx2: eth1 NIC Link is Up, 1000 Mbps full duplex
I'm not sure if it matters, or if the error itself is an error?
here's the config for eth1, similar to eth0
[root@vmhost network-scripts]# cat ifcfg-eth1 DEVICE=eth1 BOOTPROTO=none HWADDR=00:18:8B:4E:A2:B1 ONBOOT=yes TYPE=Ethernet MASTER=bond0 SLAVE=yes # added this line following suggested solution for speed/duplex error ETHTOOL_OPTS="speed 1000 duplex full autoneg off"
and config for bond0 [root@vmhost network-scripts]# cat ifcfg-bond0 DEVICE=bond0 IPADDR=192.168.1.6 NETMASK=255.255.255.0 NETWORK=192.168.1.0 BROADCAST=192.168.1.255 GATEWAY=192.168.1.1 ONBOOT=yes BOOTPROTO=none USERCTL=no
and modprobe.conf [root@vmhost network-scripts]# cat /etc/modprobe.conf alias eth0 bnx2 alias eth1 bnx2 alias eth2 e1000 alias eth3 e1000 alias scsi_hostadapter megaraid_sas alias usb-controller ehci-hcd alias usb-controller1 uhci-hcd
# modes: # mode=0 (Balance Round Robin) # mode=1 (Active backup) # mode=2 (Balance XOR) # mode=3 (Broadcast) # mode=4 (802.3ad) # mode=5 (Balance TLB) # mode=6 (Balance ALB)
alias bond0 bonding options bond0 -o bond0 mode=4 miimon=500
Gordon
I changed the modprobe.conf file to look like this:
[root@vmhost network-scripts]# cat /etc/modprobe.conf alias eth0 bnx2 alias eth1 bnx2 alias eth2 e1000 alias eth3 e1000 alias scsi_hostadapter megaraid_sas alias usb-controller ehci-hcd alias usb-controller1 uhci-hcd
alias bond0 bonding options bond0 -o mode=802.3ad miimon=100
then I shut down all the interfaces, removed the bonding module, and then did a /etc/init.d/networking restart and all was well
-
Gordon McLellan -
James Pearson