On Monday, April 04, 2011 11:09:29 PM Warren Young wrote: > I did this test with Bonnie++ on a 3ware/LSI 9750-8i controller, with > eight WD 3 TB disks attached. Both tests were done with XFS on CentOS > 5.5, 32-bit. (Yes, 32-bit. Hard requirement for this application.) [snip] > For the RAID-6 configuration, I used the 3ware card's hardware RAID, > creating a single ~16 TB volume, formatted XFS. [snip] > Dropping to 16.37 TB on the RAID configuration by switching > to RAID-6 let us put almost the entire array under a single 16 TB XFS > filesystem. You really, really, really don't want to do this. Not on 32-bit. When you roll one byte over 16TB you will lose access to your filesystem, silently, and it will not remount on a 32-bit kernel. XFS works best on a 64-bit kernel for a number of reasons; the one you're likely to hit first is the 16TB hard limit for *occupied* file space; you can mkfs an XFS filesystem on a 17TB or even larger partition or volume, but the moment the occupied data rolls over the 16TB boundary you will be in disaster recovery mode, and a 64-bit kernel will be required for rescue. The reason I know this? I had it happen. On a CentOS 32-bit backup server with a 17TB LVM logical volume on EMC storage. Worked great, until it rolled 16TB. Then it quit working. Altogether. /var/log/messages told me that the filesystem was too large to be mounted. Had to re-image the VM as a 64-bit CentOS, and then re-attached the RDM's to the LUNs holding the PV's for the LV, and it mounted instantly, and we kept on trucking. There's a reason upstream doesn't do XFS on 32-bit.