[CentOS] XFS : Taking the plunge

Steve Brooks steveb at mcs.st-and.ac.uk
Tue Jan 21 16:59:31 UTC 2014


On Tue, 21 Jan 2014, Keith Keller wrote:

> On 2014-01-21, Steve Brooks <steveb at mcs.st-and.ac.uk> wrote:
>>
>>> mkfs.xfs -d su=512k,sw=14 /dev/sda
>>
>> where "512k" is the Stripe-unit size of the single logical device built on
>> the raid controller. "14" is from the total number of drives minus two
>> (raid 6 redundancy).
>
> The usual advice on the XFS list is to use the defaults where possible.
> But you might want to ask there to see if they have any specific advice.

Thanks for the reply Keith. Yes I will ask on the list, I did read that 
when built on mdadm raid devices it is geared up to tune itself but with 
hardware raid it may take manual tuning.


>> I mounted the filesystem with the default options assuming they would be
>> sensible but I now believe I should have specified the "inode64" mount
>> option to avoid all the inodes will being stuck in the first TB.
>>
>> The filesystem however is at 87% and does not seem to have had any
>> issues/problems.
>>
>>> df -h | grep raid
>> /dev/sda               51T   45T  6.7T  87% /raidstor

> Wow, impressive!  I know of a much smaller fs which got bit by this
> issue.  What probably happened is, as a new fs, the entire first 1TB was
> able to be reserved for inodes.

Yes and the output of "df -i" shows only

Filesystem  Inodes      IUsed   IFree       IUse% 
/dev/sda    2187329088  189621  2187139467    1%

So few inodes are used because the data is from "hpc" used to run MHD 
(magneto hydro-dynamics)  simulations on the Sun many of the files are 
snapshots of the simulation at various instances "93G" in size etc.

>> Another question is could I now safely remount with the "inode64" option
>> or will this cause problems in the future? I read this below in the XFS
>> FAQ but wondered if it have been fixed (backported?) into el6.4?
>
> I have mounted a large XFS fs that previously didn't use inode64 with
> it, and it went fine.  (I did not attempt to roll back.)  You *must*
> umount and remount for this option to take effect.  I do not know when
> the inode64 option made it to CentOS, but it is there now.

Ok so I am sort of wondering for this filesystem if it is actually worth 
it given lack of inodes does not look like it will be an issue.


>> I also noted that "xfs_check" ran out of memory and so after some reading
>> noted that it is reccommended to use "xfs_repair -n -vv" instead as it
>> uses far less memory. One remark is so why is "xfs_check" there at all?
>
> The XFS team is working on deprecating it.  But on a 51TB filesystem
> xfs_repair will still use a lot of memory.  Using -P can help, but it'll
> still use quite a bit (depending on the extent of any damage and how many
> inodes, probably a bunch of other factors I don't know).

Yes this bothers me a bit, I issued a " xfs_repair -n -vv" and that told 
me I only needed "6G" I guess with only a few inodes and a clean 
filesystem it makes sense. I did read a good solution on the XFS mailing 
list which seems really neat..

"Add an SSD of sufficient size/speed for swap duty to handle xfs_repair 
requirements for filesystems with arbitrarily high inode counts.  Create a 
100GB swap partition and leave the remainder unallocated. The unallocated 
space will automatically be used for GC and wear leveling, increasing the 
life of all cells in the drive."

Steve




More information about the CentOS mailing list