[CentOS] cross link connection fall down

Thu Jun 6 06:58:43 UTC 2013
Leon Fauster <leonfauster at googlemail.com>

Hey all,

i am fighting with a strange behavior here - i have two systems
connected via a direct crosslink wire. After booting, the connection 
is up and both systems can reach the other one. After a small time window 
the interface is down. 

$ dmesg |grep eth4
igb 0000:41:00.2: added PHC on eth4
igb 0000:41:00.2: eth4: (PCIe:5.0Gb/s:Width x4)
igb 0000:41:00.2: eth4: PBA No: G13158-000
8021q: adding VLAN 0 to HW filter on device eth4
igb: eth4 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
igb: eth4 NIC Link is Down
igb: eth4 NIC Link is Up 10 Mbps Full Duplex, Flow Control: RX/TX
igb: eth4 NIC Link is Down
igb: eth4 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
igb: eth4 NIC Link is Down
igb: eth4 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
igb: eth4 NIC Link is Down
igb: eth4 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
igb: eth4 NIC Link is Down
igb: eth4 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
igb: eth4 NIC Link is Down



if i bring the connection manually up (ethtool -r eth4), i just 
have to wait and the status shows "down" again.

$ grep eth4 /var/log/messages
Jun  5 18:35:17 node2 kernel: igb 0000:41:00.2: added PHC on eth4
Jun  5 18:35:17 node2 kernel: igb 0000:41:00.2: eth4: (PCIe:5.0Gb/s:Width x4)
Jun  5 18:35:17 node2 kernel: igb 0000:41:00.2: eth4: PBA No: G13158-000
Jun  5 18:35:17 node2 kernel: 8021q: adding VLAN 0 to HW filter on device eth4
Jun  5 18:35:17 node2 kernel: igb: eth4 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
Jun  5 18:38:10 node2 kernel: igb: eth4 NIC Link is Down
Jun  5 19:23:21 node2 kernel: igb: eth4 NIC Link is Up 10 Mbps Full Duplex, Flow Control: RX/TX
Jun  5 19:23:47 node2 kernel: igb: eth4 NIC Link is Down
Jun  5 19:23:50 node2 kernel: igb: eth4 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
Jun  5 19:24:57 node2 kernel: igb: eth4 NIC Link is Down
Jun  5 19:24:60 node2 kernel: igb: eth4 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
Jun  5 19:25:45 node2 kernel: igb: eth4 NIC Link is Down
Jun  5 19:25:60 node2 kernel: igb: eth4 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
Jun  5 19:48:57 node2 kernel: igb: eth4 NIC Link is Down

 
its a quad nic

$ cat /etc/udev/rules.d/70-persistent-net.rules
...
# PCI device 0x8086:0x1521 (igb)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="...", ATTR{type}=="1", KERNEL=="eth*", NAME="eth4"


the eth4 link is not in use right now.

the other ports on the nic are currently in use (bonded).

Had anyone ever seen such a behavior (on el6)? 

--
Thanks

LF

PS: cable already substituted / el6 system is up to date