I want to add up the quotas I've assigned on a particular partition and see if the total is bigger than the disk. It's possible to do this (awkwardly) using repquota or quota. Is there no more accurate/elegant way? I can't be the first person to worry that more quota has been issued than the disk can supply. mahalo and Happy New Year, Dave
On Thursday, December 30, 2010 09:53:25 pm Dave wrote:
I want to add up the quotas I've assigned on a particular partition and see if the total is bigger than the disk. It's possible to do this (awkwardly) using repquota or quota. Is there no more accurate/elegant way?
I don't think so. I haven't seen any switch on any of the usual commands (repquota etc) to get this. I guess you'll have to do some scripting to add up the "used" values in order to compare them with your partition size.
If you find/create the elegant way, please share...
Happy New Year! Jorge
2010/12/31 Jorge Fábregas jorge.fabregas@gmail.com:
On Thursday, December 30, 2010 09:53:25 pm Dave wrote:
I want to add up the quotas I've assigned on a particular partition and see if the total is bigger than the disk. It's possible to do this (awkwardly) using repquota or quota. Is there no more accurate/elegant way?
I don't think so. I haven't seen any switch on any of the usual commands (repquota etc) to get this. I guess you'll have to do some scripting to add up the "used" values in order to compare them with your partition size.
If you find/create the elegant way, please share...
Is there a best practice? People have to be doing something! Dave
Happy New Year! Jorge _______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
On 01/01/2011 05:56 PM, Dave wrote:
Is there a best practice? People have to be doing something!
I think that's unlikely. If you don't "oversubscribe" your disk space as a matter of policy, you'll force upgrades earlier than most people would consider them necessary. Most users, I'd expect, will be well under quota most of the time. You'd commit all of your disk space to quota long before the space was actually used. In your scenario, you'd be required to expand the disk array whenever it was committed to quota, even if actual use was very low. Every site that I know of which uses quotas handles disk upgrades when utilization requires it, not when quota subscription does.
On Sat, Jan 1, 2011 at 10:06 PM, Gordon Messmer yinyang@eburg.com wrote:
On 01/01/2011 05:56 PM, Dave wrote:
Is there a best practice? People have to be doing something!
I think that's unlikely. If you don't "oversubscribe" your disk space as a matter of policy, you'll force upgrades earlier than most people would consider them necessary. Most users, I'd expect, will be well under quota most of the time. You'd commit all of your disk space to quota long before the space was actually used. In your scenario, you'd be required to expand the disk array whenever it was committed to quota, even if actual use was very low. Every site that I know of which uses quotas handles disk upgrades when utilization requires it, not when quota subscription does.
So, is it fair to rephrase that as "ignore quotas, pay attention to actual usage"?
I agree that some degree of oversubscription is probably desireable, and it would be much easier to just add storage whenever it looks to be getting fullish. My situation right now makes that difficult - budget is gone, so I can't add storage, and my users sometimes start up a big simulation that could potentially fill the disk right before the weekend. If the hoggy simulation crashes itself, that's okay, but if it brings down a lot of other jobs submitted by other users, I look bad. I guess even if there was some good tool support, this task is doomed to make everyone unhappy.
mahalo, Dave
On Jan 3, 2011, at 2:39 PM, Dave tdbtdb+centos@gmail.com wrote:
On Sat, Jan 1, 2011 at 10:06 PM, Gordon Messmer yinyang@eburg.com wrote:
On 01/01/2011 05:56 PM, Dave wrote:
Is there a best practice? People have to be doing something!
I think that's unlikely. If you don't "oversubscribe" your disk space as a matter of policy, you'll force upgrades earlier than most people would consider them necessary. Most users, I'd expect, will be well under quota most of the time. You'd commit all of your disk space to quota long before the space was actually used. In your scenario, you'd be required to expand the disk array whenever it was committed to quota, even if actual use was very low. Every site that I know of which uses quotas handles disk upgrades when utilization requires it, not when quota subscription does.
So, is it fair to rephrase that as "ignore quotas, pay attention to actual usage"?
I agree that some degree of oversubscription is probably desireable, and it would be much easier to just add storage whenever it looks to be getting fullish. My situation right now makes that difficult - budget is gone, so I can't add storage, and my users sometimes start up a big simulation that could potentially fill the disk right before the weekend. If the hoggy simulation crashes itself, that's okay, but if it brings down a lot of other jobs submitted by other users, I look bad. I guess even if there was some good tool support, this task is doomed to make everyone unhappy.
Maybe you can have the users run these in containers like OpenVz that are set to clean themselves up after they finish?
-Ross
On Jan 3, 2011, at 8:10 PM, Ross Walker rswwalker@gmail.com wrote:
On Jan 3, 2011, at 2:39 PM, Dave tdbtdb+centos@gmail.com wrote:
On Sat, Jan 1, 2011 at 10:06 PM, Gordon Messmer yinyang@eburg.com wrote:
On 01/01/2011 05:56 PM, Dave wrote:
Is there a best practice? People have to be doing something!
I think that's unlikely. If you don't "oversubscribe" your disk space as a matter of policy, you'll force upgrades earlier than most people would consider them necessary. Most users, I'd expect, will be well under quota most of the time. You'd commit all of your disk space to quota long before the space was actually used. In your scenario, you'd be required to expand the disk array whenever it was committed to quota, even if actual use was very low. Every site that I know of which uses quotas handles disk upgrades when utilization requires it, not when quota subscription does.
So, is it fair to rephrase that as "ignore quotas, pay attention to actual usage"?
I agree that some degree of oversubscription is probably desireable, and it would be much easier to just add storage whenever it looks to be getting fullish. My situation right now makes that difficult - budget is gone, so I can't add storage, and my users sometimes start up a big simulation that could potentially fill the disk right before the weekend. If the hoggy simulation crashes itself, that's okay, but if it brings down a lot of other jobs submitted by other users, I look bad. I guess even if there was some good tool support, this task is doomed to make everyone unhappy.
Maybe you can have the users run these in containers like OpenVz that are set to clean themselves up after they finish?
Or use Amazon's elastic computing cloud to provision simulation VMs, run the simulations, report results, then completely disappear.
-Ross
On Jan 3, 2011, at 8:12 PM, Ross Walker rswwalker@gmail.com wrote:
On Jan 3, 2011, at 8:10 PM, Ross Walker rswwalker@gmail.com wrote:
On Jan 3, 2011, at 2:39 PM, Dave tdbtdb+centos@gmail.com wrote:
On Sat, Jan 1, 2011 at 10:06 PM, Gordon Messmer yinyang@eburg.com wrote:
On 01/01/2011 05:56 PM, Dave wrote:
Is there a best practice? People have to be doing something!
I think that's unlikely. If you don't "oversubscribe" your disk space as a matter of policy, you'll force upgrades earlier than most people would consider them necessary. Most users, I'd expect, will be well under quota most of the time. You'd commit all of your disk space to quota long before the space was actually used. In your scenario, you'd be required to expand the disk array whenever it was committed to quota, even if actual use was very low. Every site that I know of which uses quotas handles disk upgrades when utilization requires it, not when quota subscription does.
So, is it fair to rephrase that as "ignore quotas, pay attention to actual usage"?
I agree that some degree of oversubscription is probably desireable, and it would be much easier to just add storage whenever it looks to be getting fullish. My situation right now makes that difficult - budget is gone, so I can't add storage, and my users sometimes start up a big simulation that could potentially fill the disk right before the weekend. If the hoggy simulation crashes itself, that's okay, but if it brings down a lot of other jobs submitted by other users, I look bad. I guess even if there was some good tool support, this task is doomed to make everyone unhappy.
Maybe you can have the users run these in containers like OpenVz that are set to clean themselves up after they finish?
Or use Amazon's elastic computing cloud to provision simulation VMs, run the simulations, report results, then completely disappear.
I was just thinking if you had a couple of base simulation disk images created for containers on LVM you could create clones (writable snapshots) for each container at the time they start, have the simulation run and when the container stops the snapshot can be destroyed, cleaning up the used disk space.
All this can be scriptable and can kick off via cron or 'at', or whatever.
-Ross
On 4/01/11 6:39 AM, Dave wrote:
I agree that some degree of oversubscription is probably desireable, and it would be much easier to just add storage whenever it looks to be getting fullish. My situation right now makes that difficult - budget is gone, so I can't add storage, and my users sometimes start up a big simulation that could potentially fill the disk right before the weekend. If the hoggy simulation crashes itself, that's okay, but if it brings down a lot of other jobs submitted by other users, I look bad. I guess even if there was some good tool support, this task is doomed to make everyone unhappy.
I have a similar problem with ANSYS users in an HPC environment, in that un-schooled ANSYS users can generate vast quantities of (garbage) data very quickly. For some of these users that seem particularly unable to learn how to use ANSYS I've spun their home directories off to individual partitions. That way they only crash themselves and there is a very definite hard limit on what they can fill.
Perhaps not as elegant as quotas but it sure works.
I don't run quotas per say but monitor users under a kind of "fair use" policy. Just have a df based script that runs once a day. The vast majority of my users don't generate much data so I'd be really over subscribed. Its the small number of FEA and CFD users that fill my drives.
Still, its a big task to continually educate and remind users on the appropriate etiquette when using shared resources.
Cheers -pete
On Monday, January 03, 2011 11:39:38 am Dave wrote:
On Sat, Jan 1, 2011 at 10:06 PM, Gordon Messmer yinyang@eburg.com wrote:
On 01/01/2011 05:56 PM, Dave wrote:
Is there a best practice? People have to be doing something!
I think that's unlikely. If you don't "oversubscribe" your disk space as a matter of policy, you'll force upgrades earlier than most people would consider them necessary. Most users, I'd expect, will be well under quota most of the time. You'd commit all of your disk space to quota long before the space was actually used. In your scenario, you'd be required to expand the disk array whenever it was committed to quota, even if actual use was very low. Every site that I know of which uses quotas handles disk upgrades when utilization requires it, not when quota subscription does.
So, is it fair to rephrase that as "ignore quotas, pay attention to actual usage"?
I agree that some degree of oversubscription is probably desireable, and it would be much easier to just add storage whenever it looks to be getting fullish. My situation right now makes that difficult - budget is gone, so I can't add storage, and my users sometimes start up a big simulation that could potentially fill the disk right before the weekend. If the hoggy simulation crashes itself, that's okay, but if it brings down a lot of other jobs submitted by other users, I look bad. I guess even if there was some good tool support, this task is doomed to make everyone unhappy.
If you have no money for an upgrade, your hand is forced. You have several choices...
1) Do nothing, pray that your users don't exceed disk space available. If you have numerous customers and your average usage is far below quotas, this is likely to work.
2) Change your TOS to account for the rare case where you actually run out of disk space. Sorta like number 1, but more honest.
3) Reduce user quotas so you'd never overcommit.
4) Offer a premium service to high-needs customers to help cover your costs.
5) Dump your high needs customers and keep everybody else happy.
That's pretty much it. (shrug)
It's probably out of place for me to question your adminstrative decisions, but are we really talking about imposing limits to one of the cheapest things that there ARE in computer serviceland - disk space - at an average cost of about $70 per TERABYTE of commodity storage? Even if you went with SAS drives, the price only rises to about $150 per 750 GB - just how much space are your end users likely to need?
Maybe you "can't afford" to throw in another disk drive and mount as your /home directory, but more importantly, can you afford not to?
On 1/3/2011 1:39 PM, Dave wrote:
So, is it fair to rephrase that as "ignore quotas, pay attention to actual usage"?
I agree that some degree of oversubscription is probably desireable, and it would be much easier to just add storage whenever it looks to be getting fullish. My situation right now makes that difficult - budget is gone, so I can't add storage, and my users sometimes start up a big simulation that could potentially fill the disk right before the weekend. If the hoggy simulation crashes itself, that's okay, but if it brings down a lot of other jobs submitted by other users, I look bad. I guess even if there was some good tool support, this task is doomed to make everyone unhappy.
To take this in a slightly different direction, if all of your users more or less cooperate, you might slice out dedicated (real or virtual) resources to run jobs for them and add something like Hudson (http://hudson-ci.org/) to schedule/serialize the runs. It is normally used to do 'continuous integration' builds whenever code changes in a source control system, but it can really control any jobs you want with a variety of triggers across multiple cross platform nodes. With this approach it might be possible to share/reuse the same space, gathering the results at the end of the job instead of having users competing, each using up their own space.
On 01/03/2011 11:39 AM, Dave wrote:
So, is it fair to rephrase that as "ignore quotas, pay attention to actual usage"?
"ignore quotas" is maybe too broad. Quota is a useful mechanism to keep individual users from using all of a system's resources and disrupting access for other users. Most sites over-commit quota and monitor actual usage, taking manual action if/when the system is over utilized.
-
Benjamin Smith -
Dave -
Gordon Messmer -
Jorge Fábregas -
Les Mikesell -
Peter Brady -
Ross Walker