Robert Moskowitz wrote:
Karl R. Balsmeier wrote:
Hi,
Running a bunch of CentOS servers on 2U supermicro equipment, usually with 8 drives on Adaptec SmartRAID V or LSI Megaraid.
In designing a rack in a data center, the question arises as to how to measure the amperage the server uses, prior to investing in power drops that actually do that for you, even remotely, for about $260.00-300USD per unit. This number is the important one these days in determining rack/datacenter design.
I use the Radio Shack Amp meter. It has an inductance jaw that you put over ONE wire to the device. This means you need a place on the circuit that separates the hot and neutral wires. I have a special rig: Mail plug - separate wires - Female receptacle. Of course I have to unplug the device, put this rig in, test, then put things back.
That's one way to measure the current, then multiply by the input voltage to get watts. I think a better way is to determine the max current that the system will use. If you have a 500 watt PSU, assume 85% efficiency, then by using the input voltage you can calculate the max input power the server is going to draw. I'd design the datacenter to support the max power level the server will need, not just what it takes to run the thing. Start up will draw the most power, approaching the max power output of the PSU, then it'll lower some once all the drives are up and spinning.
It's somewhat an art, but you can get reasonably close to the entire power draw of your rack. In a pinch, I've added up all the UPSes and told the electrician 12000VA. They'll know what to do with that number.
HTH
Mark
On Tue, Aug 01, 2006 at 08:03:58PM -0700, Mark Schoonover wrote:
get watts. I think a better way is to determine the max current that the system will use. If you have a 500 watt PSU, assume 85% efficiency, then by using the input voltage you can calculate the max input power the server is going to draw. I'd design the datacenter to support the max power level the server will need, not just what it takes to run the thing. Start up will draw the most power, approaching the max power output of the PSU, then it'll lower some once all the drives are up and spinning.
Using this method, you will probably overbuild for power.
Power supplies in good servers are much higher-capacity than is necessary for the typical application.
Regarding drive spin-up, i think a reasonable number for spin-up of a 15k drive is 30 watts (i couldn't quickly find a spec for spin-up power, but ~10 watts idle is 'typical' for seagate 15k drives). Three of those per server is 90 watts. During spin-up time, the CPU(s) will be idle so will be drawing a lot less power than they would at full load, so you have a bit of savings there.
Depending on your application, overprovisioning your power might not be a bad thing - next year's servers will probably use more.
It's somewhat an art, but you can get reasonably close to the entire power draw of your rack. In a pinch, I've added up all the UPSes and told the electrician 12000VA. They'll know what to do with that number.
This is good advice.
The "kill a watt" meters referenced in another post are a great suggestion too.
danno -- dan pritts - systems administrator - internet2 734/352-4953 office 734/834-7224 mobile
On Wed, 2006-08-09 at 11:54 -0400, Dan Pritts wrote:
On Tue, Aug 01, 2006 at 08:03:58PM -0700, Mark Schoonover wrote:
<snip>
Depending on your application, overprovisioning your power might not be a bad thing - next year's servers will probably use more.
Good advice, IMO I have always overpowered the units I build. Theaoy being that the greater capacity used a much less than rating will run cooler (assuming the PS had at least cooling design adequate to the rating). Cooler = longer lived, generally.
I'm still running a 1989 tower chassis (another of my preferences) with an AMD 100MHz, 36MB ram, couple slow Champ HDs as one of my IPCop stations. That unit still has original PS or only replacement, not sure anymore.
I'm sure that rack-mounts powering multiple blades may also benefit from a little oversizing, although maybe not as much benefit, figuring they should be more robust at the start.
<snip>
danno
<snip sig stuff>
We followed the advice of whomever recommended the Kill-A-Watt tool, and learned that basically a very fast dual-core opteron running 4 BG RAM, 8 SCSI hd's at 15K spin on LSI-Logic RAID ran at 1.7 amps. Up from the 1.2 and 1.4 (conservative) estimate I had made.
You plug the kill-a-watt into the outlet, it has a plug for your server cable. It gives a read in digital or what amperage you draw. 20.00 USD. Handy.
Much better than unshielding a cable to expose it's neutral or positive wire (scary) and using a "clamp ammeter", which may not even work at all because 1 amp is so tiny and they usually tackle 300 amp situations.
Spreading power load across multiple circuits is, and will be the name of the game for a while.
On a 20 amp circuit, the most i'm willing to allow is 12, so it can spike to 16 under heavy load, and only get to 18 if the math is slightly off. When you add another circuit into the equation, you effectively get to divide by 2. Just enough to offset any bad math.
It was interesting to see it spike up to 1.7. When reading the stickers on the power supply unit, can we safely conclude that the +5V = 0 - 2 is actually saying "this power supply goes between 0 and 2 amps". There are alot of numbers and symbols, but this is the one relevant to our discussion right?
Anyone care to explain the +5V and -5V stuff?
Thanks alot for the advice so far, helped massively.
-karl
Dan Pritts danno@internet2.edu wrote: On Tue, Aug 01, 2006 at 08:03:58PM -0700, Mark Schoonover wrote:
get watts. I think a better way is to determine the max current that the system will use. If you have a 500 watt PSU, assume 85% efficiency, then by using the input voltage you can calculate the max input power the server is going to draw. I'd design the datacenter to support the max power level the server will need, not just what it takes to run the thing. Start up will draw the most power, approaching the max power output of the PSU, then it'll lower some once all the drives are up and spinning.
Using this method, you will probably overbuild for power.
Power supplies in good servers are much higher-capacity than is necessary for the typical application.
Regarding drive spin-up, i think a reasonable number for spin-up of a 15k drive is 30 watts (i couldn't quickly find a spec for spin-up power, but ~10 watts idle is 'typical' for seagate 15k drives). Three of those per server is 90 watts. During spin-up time, the CPU(s) will be idle so will be drawing a lot less power than they would at full load, so you have a bit of savings there.
Depending on your application, overprovisioning your power might not be a bad thing - next year's servers will probably use more.
It's somewhat an art, but you can get reasonably close to the entire power draw of your rack. In a pinch, I've added up all the UPSes and told the electrician 12000VA. They'll know what to do with that number.
This is good advice.
The "kill a watt" meters referenced in another post are a great suggestion too.
danno -- dan pritts - systems administrator - internet2 734/352-4953 office 734/834-7224 mobile _______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
--------------------------------- Talk is cheap. Use Yahoo! Messenger to make PC-to-Phone calls. Great rates starting at 1¢/min.
On Wed, Aug 09, 2006 at 09:16:39AM -0700, Karl Balsmeier wrote:
It was interesting to see it spike up to 1.7. When reading the stickers on the power supply unit, can we safely conclude that the +5V = 0 - 2 is actually saying "this power supply goes between 0 and 2 amps". There are alot of numbers and symbols, but this is the one relevant to our discussion right?
I can't say definitively about the labels on the PSU, but I *can* tell you that only part of your system load is 5V - it also uses 12V. This is (mostly? entirely?) for mechanical stuff, eg drive & fan motors.
Anyone care to explain the +5V and -5V stuff?
I believe +5V means 5v dc negative ground (what we generally work with when we talk about DC).
-5V means positive ground. Positive ground is used on some older cars & motorcycles, and in telco equipment.
danno
Dan Pritts wrote:
Depending on your application, overprovisioning your power might not be a bad thing - next year's servers will probably use more.
Next year's servers will probably use less power.
Of course, next year you will probably have to increase your capacity 10-fold, so that 20% power reduction will not amount to a hill of beans.
-
Dan Pritts -
Karl Balsmeier -
Mark Schoonover -
Robert Moskowitz -
William L. Maltby