[CentOS] Centos machine sometimes unreachable

[Lamar Owen]

On Wednesday, August 22, 2012 09:20:20 AM m.roth at 5-cent.us wrote:
> Good thought! Could someone have moved non-computer hardware, like, say, a
> desk or chair, or there's been some utility work, and the cable run's
> impacted occasionally... or stepped on?

Or even vibrated loose.

As a lesson in 'vibrational torque' aka the 'whimmy diddle effect' (see https://en.wikipedia.org/wiki/Gee-haw_whammy_diddle ), I'll relate what happened to a machine in our newly rennovated Research Building Data Center.  The machine in question is an EMC Clariion CX3-80 with three 40U racks.  Each rack needs two 30A 208V single-phase circuits.  Due to the way the electrical was run for this machine, one UPS is powering one side of the three cabinets, and a second, smaller UPS is powering the other side.  The secondary feed's L6-30R's are mounted to the pedestals of the raised floor; the primary feed's L6-30R's are in cast aluminum pendants lying on the subfloor, with the receptacles pointing up.

An electrician was drilling two 1/4 inch holes for concrete anchors for a conduit run sixteen feet away from the CX3-80 a couple of days ago, and I started getting a bunch of alerts from the CX3-80.  I looked at the array, and yellow lights were lit on all three cabinets.  Hmm, what's up with that; power loss to the whole side of the array?

I lifted the panel over the L6-30R's in the pendant on the subfloor, and all of the twist-lock plugs were unplugged from their pendant receptacles!  No one was in the data center but the electrician, and the power was fine right before he started drilling the two small holes.  I personally had plugged in the plugs that morning, and had set the cords to apply the correct torque to maintain the twistlock, and had fully seated and locked the plugs in the receptacles.  The vibration of the hammer drill sixteen feet away hit the right resonance, and 'whimmy diddled' the plugs out of their receptacles.

Thermal cycling can also exert torques, and one of my preventive maintenance steps in all of our data center spaces with twist lock plugs is to reseat the twistlock once per quarter.