On Tue, 12 Apr 2011, Marian Marinov wrote: > On Tuesday 12 April 2011 10:36:54 Alain Péan wrote: >> Le 12/04/2011 09:23, Matthew Feinberg a écrit : >>> Hello All >>> >>> I have a brand spanking new 40TB Hardware Raid6 array to play around >>> with. I am looking for recommendations for which filesystem to use. I am >>> trying not to break this up into multiple file systems as we are going >>> to use it for backups. Other factors is performance and reliability. >>> >>> CentOS 5.6 >>> >>> array is /dev/sdb >>> >>> So here is what I have tried so far >>> reiserfs is limited to 16TB >>> ext4 does not seem to be fully baked in 5.6 yet. parted 1.8 does not >>> support creating ext4 (strange) >>> >>> Anyone work with large filesystems like this that have any >>> suggestions/recommendations? >> >> Hi Matthew, >> >> I would go for xfs, which is now supported in CentOS. This is what I use >> for a 16 TB storage, with CentOS 5.3 (Rocks Cluster), and it woks fine. >> No problem with lengthy fsck, as with ext3 (which does not support such >> capacities). I did not try yet ext4... >> >> Alain > > I have Raid6 Arrays with 30TB. We have tested XFS and its write performance > was really dissapointing. So we looked at Ext4. It is really good for our > workloads, but it lacks the ability to grow over 16TB. So we crated two > partitions on the raid with ext4. > > The RAID rebuild time is around 2 days, max 3 if the workload is higher. So I > presume that for 40TB it will be around 4 days. > > Marian > For interest how much *memory* would you need in your raid management node to support "fsck" on a 40TB array. I imagine it would be very high. Steve