Hello, I understand that this forum related to PowerConnect switches, but anyway I'll try to ask my question here since i didn't find more suitable place.
We have set of Force10 MXL switches configured in stack mode inside Dell m1000e chassis, several of them periodically crashing with following or similar errors related to heatbeat issue:
[12/15 1:52:5] IRC-(irc):CPBLOB:0: ==================================================== [12/15 1:52:5] IRC-(irc):CPBLOB:0: WARNING: 9 heartbeats not received from peer, BCM link status 1 [12/15 1:52:6] ****** ERROR TME-(chmgr):CPBLOB:0: Error: f10_tmeGetPeerIpAddrByName failed service 56(F10STKMGR) inst 4 [12/15 1:52:10] IRC-(irc):CPBLOB:0: ==================================================== [12/15 1:52:10] IRC-(irc):CPBLOB:0: WARNING: 10 heartbeats not received from peer, BCM link status 1 [12/15 1:52:10] IPC-(irc):CPBLOB:0: IPC link with CP slot 0 is in state Yellow - Skewed Heartbeat Packet(s), total missed HB: 10. Collect system status. [12/15 1:52:11] ****** ERROR TME-(chmgr):CPBLOB:0: Error: f10_tmeGetPeerIpAddrByName failed service 56(F10STKMGR) inst 4 [12/15 1:52:15] IRC-(irc):CPBLOB:0: ==================================================== [12/15 1:52:15] IRC-(irc):CPBLOB:0: WARNING: 11 heartbeats not received from peer, BCM link status 1 [12/15 1:52:16] ****** ERROR TME-(chmgr):CPBLOB:0: Error: f10_tmeGetPeerIpAddrByName failed service 56(F10STKMGR) inst 4 [12/15 1:52:20] IRC-(irc):CPBLOB:0: ==================================================== [12/15 1:52:20] IRC-(irc):CPBLOB:0: WARNING: 12 heartbeats not received from peer, BCM link status 1 [12/15 1:52:21] ****** ERROR TME-(chmgr):CPBLOB:0: Error: f10_tmeGetPeerIpAddrByName failed service 56(F10STKMGR) inst 4 [12/15 1:52:25] IRC-(irc):CPBLOB:0: ==================================================== [12/15 1:52:25] IRC-(irc):CPBLOB:0: WARNING: 13 heartbeats not received from peer, BCM link status 1 [12/15 1:52:25] IPC-(irc):CPBLOB:0: IPC link with CP slot 0 is in state Red - Bad or Dead Connection, total missed HB: 13. Collect system status. [12/15 1:52:26] ****** ERROR TME-(chmgr):CPBLOB:0: Error: f10_tmeGetPeerIpAddrByName failed service 56(F10STKMGR) inst 4
Main problem that failed switch is not just rebooting, it's hanging in transit state and leaving the stack.
So both switches in stack are showing itself as Master, and we're getting classic split-brain.
We already tried to replace stack cable, failing switch, tried to upgrade firmware,
currently we're using 9.9
Any thoughts?