Is there a flag for the df command to get the total disk space used on all filesystems as one number? I have a server with a lot of mounted shares. I'm looking for a simple way to measure rate of data growth across all shares as one total value.
Is there a flag for the df command to get the total disk space used on all filesystems as one number? I have a server with a lot of mounted shares. I'm looking for a simple way to measure rate of data growth across all shares as one total value.
Not directly, but you can add up all the entries mounted from /dev with a simple awk statement:
df -kl | awk '/^/dev// { avail += $3/1024 } END { printf("%d Mb used\n",avail)} '
Dear Sean,
No, there isn't. You'd have to parse the df output to get that value. I suggest using the -P switch to df, so you don't have to deal with multi-line output per filesystem.
The following will return kilobytes of disk space used (third column in the df -kP output):
df -kP |grep -v ^Filesystem |awk '{sum += $3} END { print sum; } '
Best, -at
On 8/11/08, Sean Carolan scarolan@gmail.com wrote:
Is there a flag for the df command to get the total disk space used on all filesystems as one number? I have a server with a lot of mounted shares. I'm looking for a simple way to measure rate of data growth across all shares as one total value. _______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
On Mon, Aug 11, 2008 at 12:07:09PM -0700, Aleksey Tsalolikhin wrote:
value. I suggest using the -P switch to df, so you don't have to deal with multi-line output per filesystem.
Ugh, hasn't RedHat fixed that? Sun have (for a long time) automatically done this if stdout is not a terminal.
*sigh*.
So my previous mail should really be df -Pkl | awk '/^/dev// { avail += $3/1024 } END { printf("%d Mb used\n",avail)} '
Stephen Harris wrote:
On Mon, Aug 11, 2008 at 12:07:09PM -0700, Aleksey Tsalolikhin wrote:
value. I suggest using the -P switch to df, so you don't have to deal with multi-line output per filesystem.
Ugh, hasn't RedHat fixed that? Sun have (for a long time) automatically done this if stdout is not a terminal.
*sigh*.
So my previous mail should really be df -Pkl | awk '/^/dev// { avail += $3/1024 } END { printf("%d Mb used\n",avail)} '
Hi,
This is work well on my laptop that running Fedora 9 with no LVM but on CentOS 5.2 with LVM it cann't calculate the LVM volume, below is output of my system :
[root@centos-svr ~]# df -kPl Filesystem 1024-blocks Used Available Capacity Mounted on /dev/mapper/VolGroup00-LogVol00 274405432 18584656 241656808 8% / /dev/sda2 101105 19096 76788 20% /boot tmpfs 1682508 0 1682508 0% /dev/shm
and with the command of Stephen : [root@centos-svr ~]# df -Pkl | awk '/^/dev// { avail += $3/1024 } END { printf("%d Mb used\n",avail)} ' 18173 Mb used
So it seam to be bot calculated on /dev/mapper/VolGroup00-LogVol00 I am just a newbie with Linux and stupid enough to not to know how to fix the command so just post it and hope you guys can improve the script ;)
Thanks for the tip.
Have fun. regards,
On Wed, Aug 13, 2008 at 8:56 PM, Lunix1618 lunix1618@gmail.com wrote:
Hi,
This is work well on my laptop that running Fedora 9 with no LVM but on CentOS 5.2 with LVM it cann't calculate the LVM volume, below is output of my system :
[root@centos-svr ~]# df -kPl Filesystem 1024-blocks Used Available Capacity Mounted on /dev/mapper/VolGroup00-LogVol00 274405432 18584656 241656808 8% / /dev/sda2 101105 19096 76788 20% /boot tmpfs 1682508 0 1682508 0% /dev/shm
and with the command of Stephen : [root@centos-svr ~]# df -Pkl | awk '/^/dev// { avail += $3/1024 } END { printf("%d Mb used\n",avail)} ' 18173 Mb used
Well, I get 18167, but that's not too far off.
mhr
On Thu, Aug 14, 2008 at 01:12:58AM -0700, MHR wrote:
On Wed, Aug 13, 2008 at 8:56 PM, Lunix1618 lunix1618@gmail.com wrote:
[root@centos-svr ~]# df -kPl Filesystem 1024-blocks Used Available Capacity Mounted on /dev/mapper/VolGroup00-LogVol00 274405432 18584656 241656808 8% / /dev/sda2 101105 19096 76788 20% /boot tmpfs 1682508 0 1682508 0% /dev/shm
and with the command of Stephen : [root@centos-svr ~]# df -Pkl | awk '/^/dev// { avail += $3/1024 } END { printf("%d Mb used\n",avail)} ' 18173 Mb used
Well, I get 18167, but that's not too far off.
And, remember, that the output of "df" might have changed in between times you ran "df" and you ran the "awk" command; there's only 7Mbytes difference. Did someone delete a 7Mbyte file? Send email? Finish a print job? Or... could be plenty of reasons for the used amount to go down.
The awk script works correctly, as can be tested easily:
$ cat h Filesystem 1024-blocks Used Available Capacity Mounted on /dev/mapper/VolGroup00-LogVol00 274405432 18584656 241656808 8% / /dev/sda2 101105 19096 76788 20% /boot tmpfs 1682508 0 1682508 0% /dev/shm
$ cat h | awk '/^/dev// { avail += $3/1024 } END { printf("%d Mb used\n",avail)}' 18167 Mb used
Stephen Harris wrote:
And, remember, that the output of "df" might have changed in between times you ran "df" and you ran the "awk" command; there's only 7Mbytes difference. Did someone delete a 7Mbyte file? Send email? Finish a print job? Or... could be plenty of reasons for the used amount to go down.
The awk script works correctly, as can be tested easily:
$ cat h Filesystem 1024-blocks Used Available Capacity Mounted on /dev/mapper/VolGroup00-LogVol00 274405432 18584656 241656808 8% / /dev/sda2 101105 19096 76788 20% /boot tmpfs 1682508 0 1682508 0% /dev/shm
$ cat h | awk '/^/dev// { avail += $3/1024 } END { printf("%d Mb used\n",avail)}' 18167 Mb used
But the whole system used only 18 MB ? That's not true.
Stephen Harris wrote:
On Thu, Aug 14, 2008 at 01:12:58AM -0700, MHR wrote:
On Wed, Aug 13, 2008 at 8:56 PM, Lunix1618 lunix1618@gmail.com wrote:
[root@centos-svr ~]# df -kPl Filesystem 1024-blocks Used Available Capacity Mounted on /dev/mapper/VolGroup00-LogVol00 274405432 18584656 241656808 8% / /dev/sda2 101105 19096 76788 20% /boot tmpfs 1682508 0 1682508 0% /dev/shm
and with the command of Stephen : [root@centos-svr ~]# df -Pkl | awk '/^/dev// { avail += $3/1024 } END { printf("%d Mb used\n",avail)} ' 18173 Mb used
Oppps, sorry my fault... 18,173 MB equal 18 GB sorry again.
On Thu, Aug 14, 2008 at 3:50 AM, Stephen Harris lists@spuddy.org wrote:
On Thu, Aug 14, 2008 at 01:12:58AM -0700, MHR wrote:
On Wed, Aug 13, 2008 at 8:56 PM, Lunix1618 lunix1618@gmail.com wrote:
[root@centos-svr ~]# df -kPl Filesystem 1024-blocks Used Available Capacity Mounted on /dev/mapper/VolGroup00-LogVol00 274405432 18584656 241656808 8% / /dev/sda2 101105 19096 76788 20% /boot tmpfs 1682508 0 1682508 0% /dev/shm
and with the command of Stephen : [root@centos-svr ~]# df -Pkl | awk '/^/dev// { avail += $3/1024 } END { printf("%d Mb used\n",avail)} ' 18173 Mb used
Well, I get 18167, but that's not too far off.
And, remember, that the output of "df" might have changed in between times you ran "df" and you ran the "awk" command; there's only 7Mbytes difference. Did someone delete a 7Mbyte file? Send email? Finish a print job? Or... could be plenty of reasons for the used amount to go down.
Clarification - I just took the numbers above and used a calculator - my system would never produce numbers anything like that for any of my machines....
mhr
On Thu, Aug 14, 2008 at 10:09:23AM -0700, MHR wrote:
On Thu, Aug 14, 2008 at 3:50 AM, Stephen Harris lists@spuddy.org wrote:
On Thu, Aug 14, 2008 at 01:12:58AM -0700, MHR wrote:
On Wed, Aug 13, 2008 at 8:56 PM, Lunix1618 lunix1618@gmail.com wrote:
[root@centos-svr ~]# df -kPl Filesystem 1024-blocks Used Available Capacity Mounted on /dev/mapper/VolGroup00-LogVol00 274405432 18584656 241656808 8% / /dev/sda2 101105 19096 76788 20% /boot tmpfs 1682508 0 1682508 0% /dev/shm
and with the command of Stephen : [root@centos-svr ~]# df -Pkl | awk '/^/dev// { avail += $3/1024 } END { printf("%d Mb used\n",avail)} ' 18173 Mb used
Well, I get 18167, but that's not too far off.
And, remember, that the output of "df" might have changed in between times you ran "df" and you ran the "awk" command; there's only 7Mbytes difference. Did someone delete a 7Mbyte file? Send email? Finish a print job? Or... could be plenty of reasons for the used amount to go down.
Clarification - I just took the numbers above and used a calculator - my system would never produce numbers anything like that for any of my machines....
One trick is to sent the "df -Pkl" info to a file. Then cat that file into the "awk" script. Also cat that file to stdout during debugging.
Sending the df output to a file does a number of things. It removes any race risk that you might be seeing. And it lets you and the community "check yer work".
When in this discussion did the variable name avail get assigned to the Used col header in the line atributed to Stephen?
Something like:
$ cat /tmp/checkspace #!/bin/bash df -Pkl > /tmp/checkingdiskspce echo -e "\nInput is:" cat /tmp/checkingdiskspce echo -e "\nAdding up the bits" cat /tmp/checkingdiskspce | awk '/^/dev// { used += $3/1024 } END { printf("%d Mb Used\n", used)} ' echo -e "\nNow df with a human flag" df -h
On Thu, Aug 14, 2008 at 2:36 PM, Nifty Cluster Mitch niftycluster@niftyegg.com wrote:
$ cat /tmp/checkspace #!/bin/bash df -Pkl > /tmp/checkingdiskspce echo -e "\nInput is:" cat /tmp/checkingdiskspce echo -e "\nAdding up the bits" cat /tmp/checkingdiskspce | awk '/^/dev// { used += $3/1024 } END { printf("%d Mb Used\n", used)} '
This is simpler (and does not involve as many execs & forks) as:
awk '/^/dev// { used += $3/1024 } END { printf("%d Mb Used\n", used)} ' /tmp/checkingdiskspce
mhr
On Thu, Aug 14, 2008 at 02:45:43PM -0700, MHR wrote:
On Thu, Aug 14, 2008 at 2:36 PM, Nifty Cluster Mitch niftycluster@niftyegg.com wrote:
$ cat /tmp/checkspace #!/bin/bash df -Pkl > /tmp/checkingdiskspce echo -e "\nInput is:" cat /tmp/checkingdiskspce echo -e "\nAdding up the bits" cat /tmp/checkingdiskspce | awk '/^/dev// { used += $3/1024 } END { printf("%d Mb Used\n", used)} '
This is simpler (and does not involve as many execs & forks) as:
awk '/^/dev// { used += $3/1024 } END { printf("%d Mb Used\n", used)} ' /tmp/checkingdiskspce
True, yet if the goal is "df | awk" with no tmp file at all the final edit and cleanup is cleaner. If the goal is to present the result of "df" combined with a bottom line summary your line may be better.
I did notice in this discussion that no one looked at inode counts. A filesystem might be "full" for want of an inode.... I cannot recall if ext[23] will allocate additional inodes dynamically like xfs will. Since xfs will allocate them but not delete then a run-away could cause a lot to be allocated on xfs confusing space use.
Other interesting system admin topics not addressed includes sparse files. For some knowing about sparse files is important for backup tools. Also allocation block size mismatch to average file sizes. Lots of small byte count files on a large allocation block causes book keeping confusion. Some tiny files never allocate a block as the inode can contain some data on some filesystems.
Just looked at the mk2fs man page the -N, -i and -I flags answer my question about dynamic inode allocation (Answer=no).
Nifty Cluster Mitch wrote:
I did notice in this discussion that no one looked at inode counts. A filesystem might be "full" for want of an inode.... I cannot recall if ext[23] will allocate additional inodes dynamically like xfs will.
ext3 doesn't(at least not by default). I had a system fill up on inodes about a month ago, probably the first time in 3-4 years that I've seen that happen.
nate
On Thu, Aug 14, 2008 at 03:23:24PM -0700, Nifty Cluster Mitch wrote:
On Thu, Aug 14, 2008 at 02:45:43PM -0700, MHR wrote:
On Thu, Aug 14, 2008 at 2:36 PM, Nifty Cluster Mitch
$ cat /tmp/checkspace #!/bin/bash df -Pkl > /tmp/checkingdiskspce echo -e "\nInput is:" cat /tmp/checkingdiskspce echo -e "\nAdding up the bits" cat /tmp/checkingdiskspce | awk '/^/dev// { used += $3/1024 } END { printf("%d Mb Used\n", used)} '
This is simpler (and does not involve as many execs & forks) as:
awk '/^/dev// { used += $3/1024 } END { printf("%d Mb Used\n", used)} ' /tmp/checkingdiskspce
True, yet if the goal is "df | awk" with no tmp file at all the final edit and cleanup is cleaner. If the goal is to present the result of
Boggle-riffic! if you want to see the input as well, then do it in the awk side. So we have a "df" and an "awk"; one pass through the output... everything is optimal!
df -Pkl | awk 'BEGIN { print "\nInput is:" } { print $0 } /^/dev// {used += $3/1024 } END { printf("\nAdding up the bits:\n%d MB Used\n",used)}'
Input is: Filesystem 1024-blocks Used Available Capacity Mounted on /dev/sda6 4061540 3182144 669752 83% / /dev/sda5 449567400 28064608 398297708 7% /datadisk /dev/sda3 93327 11124 77384 13% /boot tmpfs 2041736 0 2041736 0% /dev/shm
Adding up the bits: 30525 MB Used
(of course I only made that multiple lines for readability; you could put it all on one line if you really wanted to :-))
I did notice in this discussion that no one looked at inode counts. A filesystem might be "full" for want of an inode.... I cannot
When you're adding up all used space over multiple disks you're not concerned with a disk filling up. You're just looking for total usage. Off the top of my head I can only think of one (bad) reason to do it; some sort of billing system.
The numbers don't even taken into account the %age of space reserved for root :-)
Other interesting system admin topics not addressed includes sparse files. For some knowing about sparse files is important for backup tools. Also allocation block size
Only for broken backup tools that don't handle sparse files :-) And, yes, I had one of those in 1990... tar on a DEC Ultrix 3.1 system doesn't grok sparse files. Bleh! Fortunately "dump" did it properly.
As long as you only want the absolute amount of data (not the percentage of total file space that is used) you could use "du -sh /" on that server.
--On 11. August 2008 14:00:09 -0500 Sean Carolan scarolan@gmail.com wrote:
Is there a flag for the df command to get the total disk space used on all filesystems as one number? I have a server with a lot of mounted shares. I'm looking for a simple way to measure rate of data growth across all shares as one total value. _______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Sean Carolan wrote:
Is there a flag for the df command to get the total disk space used on all filesystems as one number? I have a server with a lot of mounted shares. I'm looking for a simple way to measure rate of data growth across all shares as one total value.
You've had a few replies as to the actual command(s) to use to achieve this, but what about looking at it from a different perspective?
Is having one number useful for different data volumes? If one is a SQL database that remains static, and another is a shared disk used by the marketing department and its usage changes by gigs a week, then you're not really able to judge when a particular disk is going to need more capacity. One overall number has very limited use.
Of course maybe that really is what you're after, in which case all this is redundant.. :)
But another way to measure usage would be to feed the data daily (or weekly, or hourly, or whatever) into something like RRD Tool. Then it will come up with some pretty graphs of usage per disk. Then you can also calculate the total as well as another field, but I believe that separate data volumes warrant measuring separately.
RRD Tool can be a bit complex to talk to directly, but if you use something like Cacti (http://www.cacti.net/), then I think you will get more value out of your data. I've never used Cacti myself, but it looks like a very nice package. And it makes talking to RRD Tool much easier.
That and you can produce lots of pretty graphs for management that prove you need upgrades and more imortantly, *where*. :)
-
Aleksey Tsalolikhin -
Dirk H. Schulz -
Lunix1618 -
MHR -
nate -
Nifty Cluster Mitch -
Sean Carolan -
Spiro Harvey, Knossos Networks Ltd -
Stephen Harris