on a CentOS release 6.8 (Final) I was looking around and noticed the following in a directory.
[jason@server /app-vol/applications/]$ls -ld temp drwxr-xr-x 4 jason users 2899968 Feb 18 06:31 temp/ [jason@server /app-vol/applications/]$
How does the directory *itself* have a size of 2.8 megs? yes the contents in that temp directory add up to 2.8 megs, but.. umm. huh?
when I run a stat on the temp directory, i get File: `temp' Size: 2899968 Blocks: 5672 IO Block: 4096 directory Device: fd02h/64770d Inode: 1836137 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 1022/ jason) Gid: ( 501/ users) Access: 2017-02-22 06:31:51.414190071 -0500 Modify: 2017-02-18 06:31:56.135448424 -0500 Change: 2017-02-18 06:31:56.135448424 -0500
on other "normal" directories, I see something like Size: 4096 Blocks: 8 IO Block: 4096 directory
Jason
On 02/22/2017 06:34 AM, Jason Welsh wrote:
How does the directory *itself* have a size of 2.8 megs?
If you write a large number of directory entries in a directory, the directory will grow in order to provide storage for those directory entries. You can imagine a directory as a text file containing all of the file names in the directory, with references to the location of those files, if that helps you understand why the directory itself will grow.
So its normal behavior.. thanks!
Jason
On 02/22/2017 01:40 PM, Gordon Messmer wrote:
On 02/22/2017 06:34 AM, Jason Welsh wrote:
How does the directory *itself* have a size of 2.8 megs?
If you write a large number of directory entries in a directory, the directory will grow in order to provide storage for those directory entries. You can imagine a directory as a text file containing all of the file names in the directory, with references to the location of those files, if that helps you understand why the directory itself will grow.
CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
On 02/22/2017 12:45 PM, Jason Welsh wrote:
So its normal behavior.. thanks!
Jason
On 02/22/2017 01:40 PM, Gordon Messmer wrote:
On 02/22/2017 06:34 AM, Jason Welsh wrote:
How does the directory *itself* have a size of 2.8 megs?
If you write a large number of directory entries in a directory, the directory will grow in order to provide storage for those directory entries. You can imagine a directory as a text file containing all of the file names in the directory, with references to the location of those files, if that helps you understand why the directory itself will grow.
And for ext2/3/4 at least, an enlarged directory never shrinks. If the size gets really excessive (perhaps a rogue process created a huge number of files, which have since been deleted), you need to move everything you want to save to a new directory, remove the bloated directory, and then rename the new directory back to the old name.
You can also run e2fsck with the "-D" (optimize directories) option, but that acts on _all_ the directories in the filesystem and can take quite a long time.
On 2/22/2017 10:40 AM, Gordon Messmer wrote:
On 02/22/2017 06:34 AM, Jason Welsh wrote:
How does the directory *itself* have a size of 2.8 megs?
If you write a large number of directory entries in a directory, the directory will grow in order to provide storage for those directory entries. You can imagine a directory as a text file containing all of the file names in the directory, with references to the location of those files, if that helps you understand why the directory itself will grow.
on many modern file systems, larger directories are stored as some sort of B-Tree or hash tree, so there's quite a lot of indexing data in there along with the actual directory entries.
On 23/02/17 06:04, John R Pierce wrote:
on many modern file systems, larger directories are stored as some sort of B-Tree or hash tree, so there's quite a lot of indexing data in there along with the actual directory entries
So I gather this depends on the file system.
On my ext4 file system, I have a directory that has >2TB and the directory entry itself only shows:
$ ls -ld Stuff drwxrwxr-x 146 akk akk 36864 Feb 21 21:18 Stuff/
$ du -bs Stuff 2093651427987 Stuff
Not sure what to take away from that!
On Thu, 23 Feb 2017 07:27:05 +1100 Anthony K akcentos@anroet.com wrote:
On 23/02/17 06:04, John R Pierce wrote:
on many modern file systems, larger directories are stored as some sort of B-Tree or hash tree, so there's quite a lot of indexing data in there along with the actual directory entries
So I gather this depends on the file system.
On my ext4 file system, I have a directory that has >2TB and the directory entry itself only shows:
$ ls -ld Stuff drwxrwxr-x 146 akk akk 36864 Feb 21 21:18 Stuff/
$ du -bs Stuff 2093651427987 Stuff
Not sure what to take away from that!
certainly sounds like (-) bs
d
CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
On 2/22/2017 12:27 PM, Anthony K wrote:
On my ext4 file system, I have a directory that has >2TB and the directory entry itself only shows:
$ ls -ld Stuff drwxrwxr-x 146 akk akk 36864 Feb 21 21:18 Stuff/
$ du -bs Stuff 2093651427987 Stuff
ls -ld is showing the size of the actual directory, NOT the size of the files stored within and under it.
du -s adds up the size on disk of all the stuff IN the directory.
note also that size on disk even for a single file can be larger than the size of the file as files are allocated in blocks.
On 23/02/17 07:42, John R Pierce wrote:
On 2/22/2017 12:27 PM, Anthony K wrote:
On my ext4 file system, I have a directory that has >2TB and the directory entry itself only shows:
$ ls -ld Stuff drwxrwxr-x 146 akk akk 36864 Feb 21 21:18 Stuff/
$ du -bs Stuff 2093651427987 Stuff
ls -ld is showing the size of the actual directory, NOT the size of the files stored within and under it.
That I understand.
However, I was trying to compare my system with the OP's and noticed that I have a directory with >2TB of used capacity in a folder sized ~36kb - a 56793929:1. In the OP's situation, he has a directory with ~2.8MB of used capacity in a folder sized ~2MB - a 1:1 ratio. That's the part I'm confused about.
Without knowing what the OP's file system but assuming he too is using EXT4, what would the directory be storing that's so different from mine?
On 2/22/2017 5:51 PM, Anthony K wrote:
However, I was trying to compare my system with the OP's and noticed that I have a directory with >2TB of used capacity in a folder sized ~36kb - a 56793929:1. In the OP's situation, he has a directory with ~2.8MB of used capacity in a folder sized ~2MB - a 1:1 ratio. That's the part I'm confused about.
Without knowing what the OP's file system but assuming he too is using EXT4, what would the directory be storing that's so different from mine?
a bajillion small files vs a few large ones. worse, if you've created a bajillion files in a directory, the directory grows, even if you delete them, the directory doesn't shrink, at least with the ext? file systems.
On 02/22/2017 07:02 PM, John R Pierce wrote:
Without knowing what the OP's file system but assuming he too is using EXT4, what would the directory be storing that's so different from mine?
a bajillion small files vs a few large ones.
Not to be pedantic, but the size of a directory has nothing to do with the size of the files it contains. It doesn't matter if they're small files or large ones. The only thing that matters is the number of filenames in the directory, and possibly the length of those file names.
On 23/02/17 14:33, Gordon Messmer wrote:
On 02/22/2017 07:02 PM, John R Pierce wrote:
Without knowing what the OP's file system but assuming he too is using EXT4, what would the directory be storing that's so different from mine?
a bajillion small files vs a few large ones.
Not to be pedantic, but the size of a directory has nothing to do with the size of the files it contains. It doesn't matter if they're small files or large ones. The only thing that matters is the number of filenames in the directory, and possibly the length of those file names.
Thanks for setting me straight. I guess the OP must have, like you said, a *bajillion*, puny sized files:
$ mkdir junk
$ for u in {1..10000}; touch junk/junk-${u}; done
$ ls -ld junk drwxr-xr-x 2 akk users 270336 Feb 23 17:19 junk
$ du -bs junk 270336 junk
On 02/22/2017 12:27 PM, Anthony K wrote:
On 23/02/17 06:04, John R Pierce wrote:
on many modern file systems, larger directories are stored as some sort of B-Tree or hash tree, so there's quite a lot of indexing data in there along with the actual directory entries
So I gather this depends on the file system.
The structure will vary slightly, but you can reasonably expect that a directory will act like a file, expanding in size to accommodate its directory entries.
On my ext4 file system, I have a directory that has >2TB and the directory entry itself only shows:
$ ls -ld Stuff drwxrwxr-x 146 akk akk 36864 Feb 21 21:18 Stuff/
It might help to understand what a directory entry is. See "man 3 readdir".
A directory entry is generally what you see if you run "ls -i" in a directory. It is an inode number, and a name associated with that inode number. The size of the directory typically represents the largest that individual directory has had to be in order to contain the list of inodes and names immediately within it. The directory's size is not influenced by the size of files within it, nor by the contents of sub-directories.
Does that clarify why there's no relationship between the size of the "Stuff" directory reported by "ls -ld", and the size of its contents reported by "du"?
-
Anthony K -
Dave Stevens -
Gordon Messmer -
Jason Welsh -
John R Pierce -
Robert Nichols