Al Sparks wrote:
A couple of questions. Are there any linux tools that can de-frag an ext2/3 partition?
Are there any advantages to doing so if you're running hardware RAID5?
Are there advantages / disadvantages if you're running LVM? === Al
you're in luck cause you don't defrag an ext2/3 partition at all. defrag is for windows file systems. Ext file systems are a different animal all-together.
Why? What's different between NTFS and ext2/3 that defragging is needed in one but not the other? === Al
you're in luck cause you don't defrag an ext2/3 partition at all. defrag is for windows file systems. Ext file systems are a different animal all-together.
Why? What's different between NTFS and ext2/3 that defragging is needed in one but not the other?
IMHO, defragging is highly overrated in NTFS too. it was the old FAT/FAT32 file system that suffered from horrible performance when heavily fragmented.
that said, the best way to defrag a file system is to dump it to external media, delete then recreate the file system and restore the dump.
That or run buzzsaw (win) which is a continuous defragmenter (well, when the system is idle that is) that runs in the background and only costs $10. Pagedefrag from sysinternals doesn't hurt either (and it's free off ms' website). And no, I'm not affiliated with either I just like the products.
Geoff
Sent from my BlackBerry wireless handheld.
-----Original Message----- From: John R Pierce pierce@hogranch.com
Date: Thu, 20 Sep 2007 18:38:40 To:CentOS mailing list centos@centos.org Subject: Re: [CentOS] Disk De-Fraging in Linux
you're in luck cause you don't defrag an ext2/3 partition at all. defrag is for windows file systems. Ext file systems are a different animal all-together.
Why? What's different between NTFS and ext2/3 that defragging is needed in one but not the other?
IMHO, defragging is highly overrated in NTFS too. it was the old FAT/FAT32 file system that suffered from horrible performance when heavily fragmented.
that said, the best way to defrag a file system is to dump it to external media, delete then recreate the file system and restore the dump.
_______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
IME defragging is highly needed in busy servers that use a lot of disk. Every single one of my clients have seen server performance tank when fragmentation goes over 15% or so..including the workstations. In vista it's still necessary.
John R Pierce wrote:
you're in luck cause you don't defrag an ext2/3 partition at all. defrag is for windows file systems. Ext file systems are a different animal all-together.
Why? What's different between NTFS and ext2/3 that defragging is needed in one but not the other?
IMHO, defragging is highly overrated in NTFS too. it was the old FAT/FAT32 file system that suffered from horrible performance when heavily fragmented.
that said, the best way to defrag a file system is to dump it to external media, delete then recreate the file system and restore the dump.
CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
On 9/20/07, gjgowey@tmo.blackberry.net gjgowey@tmo.blackberry.net wrote:
That or run buzzsaw (win) which is a continuous defragmenter (well, when the system is idle that is) that runs in the background and only costs $10. Pagedefrag from sysinternals doesn't hurt either (and it's free off ms' website). And no, I'm not affiliated with either I just like the products.
Geoff
On 9/21/07, William Warren hescominsoon@emmanuelcomputerconsulting.com wrote:
IME defragging is highly needed in busy servers that use a lot of disk. Every single one of my clients have seen server performance tank when fragmentation goes over 15% or so..including the workstations. In vista it's still necessary.
Last I checked, CentOS was Linux?
Brian Mathis wrote:
On 9/20/07, gjgowey@tmo.blackberry.net gjgowey@tmo.blackberry.net wrote:
That or run buzzsaw (win) which is a continuous defragmenter (well, when the system is idle that is) that runs in the background and only costs $10. Pagedefrag from sysinternals doesn't hurt either (and it's free off ms' website). And no, I'm not affiliated with either I just like the products.
Geoff
On 9/21/07, William Warren hescominsoon@emmanuelcomputerconsulting.com wrote:
IME defragging is highly needed in busy servers that use a lot of disk. Every single one of my clients have seen server performance tank when fragmentation goes over 15% or so..including the workstations. In vista it's still necessary.
Last I checked, CentOS was Linux?
thank you. I'm glad someone said it cause I was starting to wonder what list I was reading.
In answer to "what's the difference between NTFS and ext2/3?" Friggin night and day brother! Night and DAY. But I digress. I think what is needed here is a tried and true RTFM. That is the response I was expecting would find it's way into this thread, but I've not yet seen it. Maybe further down the thread perhaps. :)
Al Sparks wrote:
Al Sparks wrote:
A couple of questions. Are there any linux tools that can de-frag an ext2/3 partition?
ext2, yes. ext3, no. And it is called a filesystem. A partition is a completely different thing. none for ext3 because it was not really necessary for ext2 and so the tool never got updated.
Are there any advantages to doing so if you're running hardware RAID5?
Are there advantages / disadvantages if you're running LVM? === Al
you're in luck cause you don't defrag an ext2/3 partition at all. defrag is for windows file systems. Ext file systems are a different animal all-together.
Why? What's different between NTFS and ext2/3 that defragging is needed in one but not the other?
Implementation and design?
On 9/20/07, Al Sparks data345@yahoo.com wrote:
Why? What's different between NTFS and ext2/3 that defragging is needed in one but not the other?
Mostly it has to do with the way information is ordered and written to the filesystem. The old FAT filesystem (and to a lesser extent NTFS) was sloppy, and needed some periodic cleanup. EXT2/3 are a bit more elegant in their design and structure the data such that a defrag isn't really necessary. The filesystem *will* check itself every 180 days or every so many reboots via fsck. This is configurable with tune2fs.
That's basically the high level explanation for the differences. Once you start getting down into the really low level stuff, that's where I fall asleep no matter how much caffiene I've had.
On Thursday 20 September 2007, Al Sparks wrote:
Why? What's different between NTFS and ext2/3 that defragging is needed in one but not the other? === Al
And this is the right question to ask...
Anyway - the answer about defragging, if you really care to understand it, is pretty length.
FAT used to be horrible. It would always simply take the first available cluster and use that to store data. This resulted in a lot of fragmentation.
NTFS is much better already. It still profits from defragging but the results don't make that much of a difference anymore as long as your partition doesn't get close to being full. It tries to allocate contiguous blocks and will even add some buffer to the end for file growth.
ext2/3 is similar to ntfs in its fragmentation resistance. It however, has 2 more advantages. First, linux uses swap devices and stuff like mmapped files are still movable. In windows, swap files and some other files are not movable. The second advantage is reserved space. By default, each ext2/3 filesystem has 5% of its space reserved for root. ext2/3 simply assume you will never get past 95% full - so the data is laid out accordingly. Since you know you have at least 5% free disk blocks, you can leave a little bit more unallocated space at the end of each file... Its not much but it adds up over time.
The worst possible scenario I've found for ext3 so far is cvs. With every checkin, cvs has to modify the whole file. It does so by writing a completely new file, then deleting the old one and moving the new file in place. This means that each time, the filesystem has to allocate new space.
For a long time, I balanced stuff between servers, removed outdated code and so on. bi-monthly fsck would show about 1-2% fragmentation at about 75% filesystem full. Then a few large projects were imported. filesystem usage went up to 98% (someone did a tune2fs -m 0) and then the problems really started. I'm just about to go home now - 2am. I spent the last few hours reorganizing the cvs filesystem. A filesystem check showed 61% fragmentation! I moved old code off to a secondary server, then coppied things off, recreated the filesystem and then copied the data back.
Results were impressive - my I/O subsystem can take about 1800 io ops per second. The result before that, was about 1.1MB/sec throughput measured in iostat with a few cvs processes running at the same time. After the reorg... again 1800 ios - but my throughput rose to a more useful 24 MB/sec...
Anyway - bullet points: * there is no good way to measure (on a filesystem level) fragmentation other than fsck * try filefrag to check for fragmentation on a per file basis. * there is no online ext2/3 defragger that works on block level * there is a offline defragger for ext2 on block level e2defrag. ext3 would have to be converted to ext2 and back to ext3 after the defrag. * there are some filelevel fragmentation tools. They basically work by copying files around. This works on filesystems that had high utilization for a while, got fragmented but are now mostly empty again. I tried some of that on my cvs server but none ended up giving me good results. * if fsck shows high fragmentation (>5% in my opinions) you should make sure the filesystem doesn't get that full and if you really want to defrag, copy the data off and back on. Its the best way to do it.
And now I'm off to bed.
Peter.
On Fri, Sep 21, 2007 at 01:48:07AM -0400, Peter Arremann wrote:
the filesystem doesn't get that full and if you really want to defrag, copy the data off and back on. Its the best way to do it.
This is, in fact, the historically correct answer as well. Backup, newfs, restore. aka "Nuke it from orbit; it's the only way to be sure".
-
Al Sparks -
Brian Mathis -
Feizhou -
gjgowey＠tmo.blackberry.net -
Jim Perrin -
John R Pierce -
Mark Weaver -
Peter Arremann -
Stephen Harris -
William Warren