Hi there,
I've discovered that most of the hard drives used in our cluster got misaligned partitions, thus crippling perfs. Is there any way to fix that without having to delete/recreate properly aligned partitions, then format it and refill disks ? I'd be glad not to have to toy with moving several 10s of TB disk by disk :D (most disks are JBOD as we're using a fault tolerant network FS, moosefs not to name it). <insert your favorite search engine> wasn't helpful, unfortunately. Drives are ext4, driven by C6 x86_64.
Thanks, Laurent.
On 12/12/2012 09:36 AM, Laurent Wandrebeck wrote:
I've discovered that most of the hard drives used in our cluster got misaligned partitions, thus crippling perfs. Is there any way to fix that without having to delete/recreate properly aligned partitions, then format it and refill disks ? I'd be glad not to have to toy with moving several 10s of TB disk by disk :D (most disks are JBOD as we're using a fault tolerant network FS, moosefs not to name it).
The data is going to have to be moved. There's just no way around it, and shifting it a few sectors one way or the other is going to be no faster, and a lot more dangerous, than copying it to a new partition. Copying to a different drive should be somewhat faster than copying to the same drive since neither drive needs to share its controller with both the read and write streams.
On Wed, Dec 12, 2012 at 9:36 AM, Laurent Wandrebeck l.wandrebeck@gmail.com wrote:
I've discovered that most of the hard drives used in our cluster got misaligned partitions, thus crippling perfs. Is there any way to fix that without having to delete/recreate properly aligned partitions, then format it and refill disks ? I'd be glad not to have to toy with moving several 10s of TB disk by disk :D (most disks are JBOD as we're using a fault tolerant network FS, moosefs not to name it). <insert your favorite search engine> wasn't helpful, unfortunately. Drives are ext4, driven by C6 x86_64.
Hmmm, might be a fun test of your fault-tolerance to remove disks one at a time and add them back empty with the partitions properly aligned. Maybe...
On Wed, Dec 12, 2012 at 4:36 PM, Laurent Wandrebeck l.wandrebeck@gmail.com wrote:
Hi there,
I've discovered that most of the hard drives used in our cluster got misaligned partitions, thus crippling perfs. Is there any way to fix that without having to delete/recreate properly aligned partitions, then format it and refill disks ? I'd be glad not to have to toy with moving several 10s of TB disk by disk :D (most disks are JBOD as we're using a fault tolerant network FS, moosefs not to name it). <insert your favorite search engine> wasn't helpful, unfortunately. Drives are ext4, driven by C6 x86_64.
boot the server with a gparted livecd / pxe ; move the partitions around with an offset of 1MB at the start of the disk/volume. I do not know if this trick will work with lvm, in this case you need to move the data around and recreate the volumes with the correct offset.
Have fun!
Thank you all for your feedback. I was afraid that I had to move data around, well, I'll do then :) It'll just be quite long and boring…
@Robert: Even if copying to another disk or box would be speedier, I'm talking about moving twice ~90TB here, so… @Les: I've had enough disk failures to be sure fault-tolerance already works like a charm ^^ @Natxo: I'll do it keeping boxes up, disk per disk (fs lags a bit when automatic replication kicks in due to 12/16/24/36 disks disappearing from the pool like a single man :D)
Thanks again !