I have something like 300G I routinely backup. This includes some large 12Gig images and other files.
I had been using ext3 on an external USB disk for part of the process.
Under ext3 doing "rsync -a /home /mnt/external_back/backup.jun.27.2011" it took 200 minutes.
I took the same computer, same external HD and reformatted it for ext4 (mkfs.ext4 /dev/sdd1). I then started the same rsync as above. The time was reduced to 170 minutes.
This is on Centos 5.6 x86_64.
I did notice the [ext4-dio-unwrit] process that starts up. One process for each core. I was concerned when they did not go away. But google says that is normal. :)
Jerry
On Mon, Jun 27, 2011 at 2:10 PM, Jerry Geis geisj@pagestation.com wrote:
I have something like 300G I routinely backup. This includes some large 12Gig images and other files.
I had been using ext3 on an external USB disk for part of the process.
Under ext3 doing "rsync -a /home /mnt/external_back/backup.jun.27.2011" it took 200 minutes.
I took the same computer, same external HD and reformatted it for ext4 (mkfs.ext4 /dev/sdd1). I then started the same rsync as above. The time was reduced to 170 minutes.
This is on Centos 5.6 x86_64.
I did notice the [ext4-dio-unwrit] process that starts up. One process for each core. I was concerned when they did not go away. But google says that is normal. :)
Jerry _______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Thanx for the feeback Jerry,
What filesystem does the storage on the server use, as matter of interest?
On Mon, Jun 27, 2011 at 2:36 PM, Jerry Geis geisj@pagestation.com wrote:
Thanx for the feeback Jerry,
What filesystem does the storage on the server use, as matter of interest?
Sure, the server still has ext3.
Jerry _______________________________________________
Ok, so I'm sure you'll see a larger speed improvement (but still limited to the USB port) if the server used EXT4 as well.
You could probably only do this once the data is fully backed up though, just for safety's sake.
On 6/27/2011 8:10 AM, Jerry Geis wrote:
I have something like 300G I routinely backup. This includes some large 12Gig images and other files.
I had been using ext3 on an external USB disk for part of the process.
Under ext3 doing "rsync -a /home /mnt/external_back/backup.jun.27.2011" it took 200 minutes.
I took the same computer, same external HD and reformatted it for ext4 (mkfs.ext4 /dev/sdd1). I then started the same rsync as above. The time was reduced to 170 minutes.
A time reduction for using ext4 in that scenario does not surprise me. Under ext3, deleting large mult-gigabyte files requires a lot of activity as it tracks down and marks the blocks as free. In ext4, this process is a lot faster due to the use of extents to track which blocks are used by large files.
Just the faster deletion times with ext4 might account for the time difference. Under ext3, deleting a 10GB file might take a minute or two, but it will only take a few seconds under ext4.
(It's the primary reason that I started using ext4 last year for any shares / file systems where I needed to store large files.)
-
Jerry Geis -
Rudi Ahlers -
Thomas Harold