I've read in many places that file systems on Linux do not suffer the same fragmentation problems of Windows systems. No one has provided a clear explanation as to why fragmentation is not an issue for file systems such as ext2/3, reiserfs, xfs, etc. Just curious.
Fong Vang wrote:
I've read in many places that file systems on Linux do not suffer the same fragmentation problems of Windows systems. No one has provided a clear explanation as to why fragmentation is not an issue for file systems such as ext2/3, reiserfs, xfs, etc. Just curious. _______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
From what I've read it seems to be because Linux file system code finds the next available contiguous block that the file will fit completely into on the disk. This is instead of just starting writing it to the first available free block wether its big enough to contiguously fit the file or not, like on FAT file systems.
Also I think that newer versions of NTFS are much more resistant to fragmentation than FAT filesystems - to the point where you have much less need to run defrag programs on them.
On Fri, 2005-08-26 at 00:12 -0700, Fong Vang wrote:
I've read in many places that file systems on Linux do not suffer the same fragmentation problems of Windows systems. No one has provided a clear explanation as to why fragmentation is not an issue for file systems such as ext2/3, reiserfs, xfs, etc. Just curious.
Instead of rehashing this for the 23rd time on a list, I'll create a blog entry later this evening from my past posts. I'll send you the link when I put it up.
Short version: - Strict separation of binaries, data and temporary files - Reservation of disk space (filesystems never completely fill) - Allocation approaches of inode design filesystems
Bryan J. Smith wrote:
On Fri, 2005-08-26 at 00:12 -0700, Fong Vang wrote:
I've read in many places that file systems on Linux do not suffer the same fragmentation problems of Windows systems. No one has provided a clear explanation as to why fragmentation is not an issue for file systems such as ext2/3, reiserfs, xfs, etc. Just curious.
Instead of rehashing this for the 23rd time on a list, I'll create a blog entry later this evening from my past posts. I'll send you the link when I put it up.
Short version:
- Strict separation of binaries, data and temporary files
- Reservation of disk space (filesystems never completely fill)
- Allocation approaches of inode design filesystems
Might post that blog link onlist, I'd be interested as well :-).
On Fri, 2005-08-26 at 08:19, Bryan J. Smith wrote:
Instead of rehashing this for the 23rd time on a list, I'll create a blog entry later this evening from my past posts. I'll send you the link when I put it up.
Short version:
- Strict separation of binaries, data and temporary files
- Reservation of disk space (filesystems never completely fill)
- Allocation approaches of inode design filesystems
Please cover the common and difficult cases of many continuously growing files and the situation where many small files have been created and removed and those non-contiguous blocks are now the only remaining space on the drive when you need to write a large file. And how to fix this after the fact...
speaking of many continuosly growing files... like wgetting a few (3-4) big files (even GBs) at a slow rate (100 kB/s). Is there some way to do preallocation (we know how big the files are gonna be...), does wget (or other such programs) support this? What about cp, etc. It would seem to be one of those useful things :)
Cheers, MaZe.
On Fri, 26 Aug 2005, Les Mikesell wrote:
On Fri, 2005-08-26 at 08:19, Bryan J. Smith wrote:
Instead of rehashing this for the 23rd time on a list, I'll create a blog entry later this evening from my past posts. I'll send you the link when I put it up.
Short version:
- Strict separation of binaries, data and temporary files
- Reservation of disk space (filesystems never completely fill)
- Allocation approaches of inode design filesystems
Please cover the common and difficult cases of many continuously growing files and the situation where many small files have been created and removed and those non-contiguous blocks are now the only remaining space on the drive when you need to write a large file. And how to fix this after the fact...
Les Mikesell lesmikesell@gmail.com wrote:
Please cover ... cut ...
I will. I will cover many things. I am going to take several hours to detail many things. E.g., how XFS uses extents and other features to prevent fragmentation, but also how that can be a performance issue for XFS in some cases (e.g., temporary files).
I am _purposely_ not posting on this matter on-list because I don't want the "OT" dogs to start pissing on me again. So please, if you make requests, please do it _off-list_. I will post the link on-list, and people can comment on my blog directly, and I will add any additional information requested.
This way I will not be accused of going off-topic and taking down the list because it has *0* to do with CentOS. Please be considerate of the fact that I'm purposely avoiding this thread on-list. ;->
On Fri, 2005-08-26 at 08:55 -0700, Bryan J. Smith wrote:
Les Mikesell lesmikesell@gmail.com wrote:
Please cover ... cut ...
I will. I will cover many things. I am going to take several hours to detail many things. E.g., how XFS uses extents and other features to prevent fragmentation, but also how that can be a performance issue for XFS in some cases (e.g., temporary files).
I am _purposely_ not posting on this matter on-list because I don't want the "OT" dogs to start pissing on me again. So please, if you make requests, please do it _off-list_. I will post the link on-list, and people can comment on my blog directly, and I will add any additional information requested.
This way I will not be accused of going off-topic and taking down the list because it has *0* to do with CentOS. Please be considerate of the fact that I'm purposely avoiding this thread on-list. ;->
Well, I can sure respect what you are doing here, Brian, and I, myself am looking forward to the blog as I am also interested in the de-frag thang.
John Rose
Bryan J. Smith wrote:
Les Mikesell lesmikesell@gmail.com wrote:
Please cover ... cut ...
I will. I will cover many things. I am going to take
The time has come, the walrus said, to speak of many things...
I am _purposely_ not posting on this matter on-list because I don't want the "OT" dogs to start pissing on me again. So
Oh, *that* would never happen on this list, the people are too nice *here*.
Mike
On Fri, 2005-08-26 at 13:04 -0500, Mike McCarty wrote:
The time has come, the walrus said, to speak of many things...
Okay, here it is: http://thebs413.blogspot.com/2005/08/filesystem-fundamentals-and-practices.h...
You'll quickly note that it's a discussion that touches many things, but follows a specific set of context and considerations that I use. I do not advocate that specific filesystems are better or worse than others, or how you should layout your filesystem -- I just document why I adopted some filesystems for my own reasons, and why I follow my layout strategy.
I tried it put it into words (even if almost rambling after ~6 hours on a Friday through early Saturday morning) and will try to "clean it up more" in the coming days or weeks. So please be understanding even if you disagree with my viewpoints, they are the ones I follow, and they have really helped me completely eliminate any issues with my clients.
Bryan J. Smith wrote:
On Fri, 2005-08-26 at 13:04 -0500, Mike McCarty wrote:
The time has come, the walrus said, to speak of many things...
Okay, here it is: http://thebs413.blogspot.com/2005/08/filesystem-fundamentals-and-practices.h...
Nicely done. Thanks for taking the time to write this.
Based on this, you would recommend XFS for filesystems greater than 100G that will be shared via NFS .. true?
I've been testing NFS read/write speeds to some 800G EXT3 RAID partitions I have on a CentOS 3.4 macine - specifically comparing them against NetApp and another low end RAID device. The linux NFS shares seem to be 20-30 percent slower than the RAID and even worse compared to the NetApp. The local read/write speeds are good so I'm presuming some NFS tuning is in order.
Questions: 1. Is XFS likely to help with nfs read/write performance? 2. Any tuning suggestions for a linux nfs file server that has a mix of Linux and Solaris clients?
Additonaly Info: The linux machines are a brand new Dell PowerEdge 2850 with 3x300G disks in a RAID 5 config.
-Mark
On Sat, 2005-08-27 at 05:16 -0500, Bryan J. Smith wrote:
On Fri, 2005-08-26 at 13:04 -0500, Mike McCarty wrote:
The time has come, the walrus said, to speak of many things...
Okay, here it is: http://thebs413.blogspot.com/2005/08/filesystem-fundamentals-and-practices.h...
Thank you very much for posting that. It was a very interesting read.
Marc.
-
Bryan J. Smith -
Fong Vang -
Les Mikesell -
Maciej Żenczykowski -
Marc Deslauriers -
Mark Belanger -
Mike McCarty -
rado -
Tim Edwards -
William A. Mahaffey III